diff --git a/README.md b/README.md
index 8e99058..145d43a 100755
--- a/README.md
+++ b/README.md
@@ -1,62 +1,8 @@
-# NASA Cognitive State Determination :thought_balloon:
-[](https://opensource.org/licenses/Apache-2.0)
-
-
-
-
-
-In the near future, Astronauts will need to carry out autonomous operations as they venture beyond Low Earth Orbit. For many of the activities that the astronauts perform, the same training is provided to all crew members months ahead of their mission. A system that provides real-time operations support that is optimized and tailored to each astronaut's psychophysiological state at the time of the activities and which can be used during the training leading up to the mission would significantly improve human spaceflight operations.
-
-In this challenge, solvers were asked to determine the present cognitive state of a trainee based on biosensor data, as well as predict what their future cognitive state will be, three seconds in the future.
-
-## Data loading
-
-The provided data has a high frequency (1 sec = thousands of rows), but the labeling was done manually. Also, final predictions are expected to have a frequency of one (one second = one row). I decided to transform the data so second equals one row both for training and testing. I rounded all timestamps to the closes second and tested several approaches:
-1. Take the first value
-2. Take the last value
-3. Calculate the mean value for numerical values and mode for the target
-4. Calculate mean and std for numerical values and mode for target
-
-The first approach showed the best results.
-
-## EDA
-
-During EDA, I noticed that the timestamps might have “holes” between neighbors. The hole is defined as: if the delta time between two neighbors' rows is above one second, there is a “hole” between them. These two neighbors belong to different “sessions.” I noticed that the sensor data between other sessions might be different. I also noticed that the actual target is always constant within these sessions. I incorporated my findings into feature generation and postprocessing of my predictions.
-
-
-## Creating target
-
-We have to predict cognitive start for time `t` and `t+3`. The target for `t` is equal to the value of the `induced_state` column. The target for `t+3` is the same as the target for `t` because the cognitive state is the same within the session. The data and target are the same, so I decided to train a single model, and use the same model for making predictions for `t` and `t+3`. Note: There is a separation between `t` and `t+3` models in the code. I decided to keep it just in case the data would be different in the future.
-
-
-## Feature generation
-
-I had several ideas on features generation, and I combined them into the following groups.
-
-1. Raw sensor data. Provide data “as is.”
-2. Rolling statistics with different time windows (5, 999999 seconds) for both separate sessions and “global” (i.e. no separate sessions). Rolling statistincs include: mean, std, z-score: [x - mean(x)] / std(x)
-3. Shift features, i.e. the value of sensor data a second ago, two secodns ago, etc.
-4. Features based on the interactions between sensor data, e.g., the value of Zephyr_HR divided by the value of Zephyr_HRV.
-5. Features based on the distances between eyes positions and gazing points.
-
-
-## Validation
-
-I used stratified group k fold for validation. Stratified = each fold has approximately the same number of samples for each class. Group = provided `test_suite` column.
-
-
-## Model
-
-I used the Lightgbm classifier. I optimized hyperparameters using Optuna. The final prediction is the average predictions of several Lightgbm classifiers with different hyperparameters.
-
-## Postprocessing
-
-As mentioned in EDA, the target is the same within a single session. Therefore, I post-processed predictions by calculating the rolling average of the model’s predictions from the beginning of the session till time `t` (including time `t`) for which we’re making predictions.
-
-## Important features
-
-To determine the most important features for making predictions for time t, I used built-in SHAP values calculation. Then, I selected top-3 sensor features from the output.
-
-## Libraries
-
-You can find the list of used libraries in `requirenments.txt`.
+# NASA Cognitive State Determination - Fault Tolerance Marathon Match
+## STEPS to Build & Run
+
+- Step 1 - cd code & build docker `docker build -t latest .`
+- Step 2 - run model training
+`docker run -v /path_to_data/:/work/data/ -v /path_to_save_model/:/work/model/ latest sh train.sh /work/data/data_training.zip`
+- Step 3 - run model prediction
+`docker run -v /path_to_data/:/data/ -v /path_to_model/:/work/model/ latest sh test.sh /data/test_data_file.zip /data/file_to_save_predictions.csv`
diff --git a/code/.idea/.gitignore b/code/.idea/.gitignore
new file mode 100644
index 0000000..26d3352
--- /dev/null
+++ b/code/.idea/.gitignore
@@ -0,0 +1,3 @@
+# Default ignored files
+/shelf/
+/workspace.xml
diff --git a/code/.idea/code.iml b/code/.idea/code.iml
new file mode 100644
index 0000000..77c3a9b
--- /dev/null
+++ b/code/.idea/code.iml
@@ -0,0 +1,14 @@
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/code/.idea/inspectionProfiles/profiles_settings.xml b/code/.idea/inspectionProfiles/profiles_settings.xml
new file mode 100644
index 0000000..105ce2d
--- /dev/null
+++ b/code/.idea/inspectionProfiles/profiles_settings.xml
@@ -0,0 +1,6 @@
+
+
+
+
+
+
\ No newline at end of file
diff --git a/code/.idea/misc.xml b/code/.idea/misc.xml
new file mode 100644
index 0000000..83964c7
--- /dev/null
+++ b/code/.idea/misc.xml
@@ -0,0 +1,4 @@
+
+
+
+
\ No newline at end of file
diff --git a/code/.idea/modules.xml b/code/.idea/modules.xml
new file mode 100644
index 0000000..23968dc
--- /dev/null
+++ b/code/.idea/modules.xml
@@ -0,0 +1,8 @@
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/code/topcoder_cognitive_state/CONSTANTS.py b/code/topcoder_cognitive_state/CONSTANTS.py
index 98e3a80..e6c855b 100644
--- a/code/topcoder_cognitive_state/CONSTANTS.py
+++ b/code/topcoder_cognitive_state/CONSTANTS.py
@@ -1,3 +1,7 @@
+"""
+This module: saving constant values
+"""
+
TARGET2LABEL = {
"low": 0,
"medium": 1,
@@ -9,3 +13,58 @@
LABEL2TARGET = dict(zip(TARGET2LABEL.values(), TARGET2LABEL.keys()))
METADATA_COLUMNS = ["test_suite"]
NAN_VALUES = [-9999.9]
+DATA_COLS = [
+ # features
+ "E4_BVP",
+ "E4_GSR",
+ "LooxidLink_EEG_A3",
+ "LooxidLink_EEG_A4",
+ "LooxidLink_EEG_FP1",
+ "LooxidLink_EEG_FP2",
+ "LooxidLink_EEG_A7",
+ "LooxidLink_EEG_A8",
+ "Muse_EEG_TP9",
+ "Muse_EEG_AF7",
+ "Muse_EEG_AF8",
+ "Muse_EEG_TP10",
+ "Muse_PPG_0",
+ "Muse_PPG_1",
+ "Muse_PPG_2",
+ "Myo_GYR_X",
+ "Myo_GYR_Y",
+ "Myo_GYR_Z",
+ "Myo_EMG_0",
+ "Myo_EMG_1",
+ "Myo_EMG_2",
+ "Myo_EMG_3",
+ "Myo_EMG_4",
+ "Myo_EMG_5",
+ "Myo_EMG_6",
+ "Myo_EMG_7",
+ "PICARD_fnirs_0",
+ "PICARD_fnirs_1",
+ "Polar_bpm",
+ "Polar_hrv",
+ "ViveEye_pupilPos_L_X",
+ "ViveEye_pupilPos_L_Y",
+ "ViveEye_pupilPos_R_X",
+ "ViveEye_pupilPos_R_Y",
+ "ViveEye_gazeOrigin_L_X",
+ "ViveEye_gazeOrigin_L_Y",
+ "ViveEye_gazeOrigin_L_Z",
+ "ViveEye_gazeOrigin_R_X",
+ "ViveEye_gazeOrigin_R_Y",
+ "ViveEye_gazeOrigin_R_Z",
+ "ViveEye_gazeDirection_L_X",
+ "ViveEye_gazeDirection_L_Y",
+ "ViveEye_gazeDirection_L_Z",
+ "ViveEye_gazeDirection_R_X",
+ "ViveEye_gazeDirection_R_Y",
+ "ViveEye_gazeDirection_R_Z",
+ "ViveEye_eyeOpenness_L",
+ "ViveEye_pupilDiameter_L",
+ "ViveEye_eyeOpenness_R",
+ "ViveEye_pupilDiameter_R",
+ "Zephyr_HR",
+ "Zephyr_HRV",
+ ]
diff --git a/code/topcoder_cognitive_state/load_data.py b/code/topcoder_cognitive_state/load_data.py
index 1c41aea..7ef2922 100644
--- a/code/topcoder_cognitive_state/load_data.py
+++ b/code/topcoder_cognitive_state/load_data.py
@@ -1,13 +1,14 @@
+"""
+This module: loading data
+"""
from typing import Tuple
import time
-
+from constants import METADATA_COLUMNS, NAN_VALUES, DATA_COLS
import pandas as pd
from tqdm import tqdm
-from topcoder_cognitive_state.CONSTANTS import METADATA_COLUMNS, NAN_VALUES
-
-def _test_missing_features(df: pd.DataFrame) -> pd.DataFrame:
+def _test_missing_features(data: pd.DataFrame) -> pd.DataFrame:
"""
This test contains three tests which are run manually:
@@ -15,65 +16,7 @@ def _test_missing_features(df: pd.DataFrame) -> pd.DataFrame:
2. Check if some columns have None values
3. Check if some columns have -9999.9 (missing) values
"""
- # cols = ['ViveEye_gazeOrigin_L_X', 'ViveEye_gazeOrigin_L_Y', 'ViveEye_gazeOrigin_L_Z']
- # cols = ['Myo_EMG_0', 'Myo_EMG_1', 'Myo_EMG_2', 'Myo_EMG_3', 'Myo_EMG_4', 'Myo_EMG_5', 'Myo_EMG_6']
- # cols = ['Polar_bpm', 'Polar_hrv', 'tlx_score']
- cols = [
- # features
- "tlx_score",
- "E4_BVP",
- "E4_GSR",
- "LooxidLink_EEG_A3",
- "LooxidLink_EEG_A4",
- "LooxidLink_EEG_FP1",
- "LooxidLink_EEG_FP2",
- "LooxidLink_EEG_A7",
- "LooxidLink_EEG_A8",
- "Muse_EEG_TP9",
- "Muse_EEG_AF7",
- "Muse_EEG_AF8",
- "Muse_EEG_TP10",
- "Muse_PPG_0",
- "Muse_PPG_1",
- "Muse_PPG_2",
- "Myo_GYR_X",
- "Myo_GYR_Y",
- "Myo_GYR_Z",
- "Myo_EMG_0",
- "Myo_EMG_1",
- "Myo_EMG_2",
- "Myo_EMG_3",
- "Myo_EMG_4",
- "Myo_EMG_5",
- "Myo_EMG_6",
- "Myo_EMG_7",
- "PICARD_fnirs_0",
- "PICARD_fnirs_1",
- "Polar_bpm",
- "Polar_hrv",
- "ViveEye_pupilPos_L_X",
- "ViveEye_pupilPos_L_Y",
- "ViveEye_pupilPos_R_X",
- "ViveEye_pupilPos_R_Y",
- "ViveEye_gazeOrigin_L_X",
- "ViveEye_gazeOrigin_L_Y",
- "ViveEye_gazeOrigin_L_Z",
- "ViveEye_gazeOrigin_R_X",
- "ViveEye_gazeOrigin_R_Y",
- "ViveEye_gazeOrigin_R_Z",
- "ViveEye_gazeDirection_L_X",
- "ViveEye_gazeDirection_L_Y",
- "ViveEye_gazeDirection_L_Z",
- "ViveEye_gazeDirection_R_X",
- "ViveEye_gazeDirection_R_Y",
- "ViveEye_gazeDirection_R_Z",
- "ViveEye_eyeOpenness_L",
- "ViveEye_pupilDiameter_L",
- "ViveEye_eyeOpenness_R",
- "ViveEye_pupilDiameter_R",
- "Zephyr_HR",
- "Zephyr_HRV",
- ]
+ cols = DATA_COLS
# case 1 - no column
# df = df.drop(cols, axis=1)
@@ -84,11 +27,11 @@ def _test_missing_features(df: pd.DataFrame) -> pd.DataFrame:
# case 3 - missing values
for col in cols:
- df[col] = -9999.9
- return df
+ data[col] = -9999.9
+ return data
-def read_and_prepare_data_chunk(df: pd.DataFrame) -> pd.DataFrame:
+def read_and_prepare_data_chunk(data: pd.DataFrame) -> pd.DataFrame:
"""
Read raw data and prepare it for processing.
I.e., create columns if they are missing,
@@ -96,67 +39,12 @@ def read_and_prepare_data_chunk(df: pd.DataFrame) -> pd.DataFrame:
etc.
Args:
- df (pd.DataFrame): input raw data
+ data (pd.DataFrame): input raw data
Returns:
pd.DataFrame: processed data
"""
- EXPECTED_COLUMNS = [
- # features
- "tlx_score",
- "E4_BVP",
- "E4_GSR",
- "LooxidLink_EEG_A3",
- "LooxidLink_EEG_A4",
- "LooxidLink_EEG_FP1",
- "LooxidLink_EEG_FP2",
- "LooxidLink_EEG_A7",
- "LooxidLink_EEG_A8",
- "Muse_EEG_TP9",
- "Muse_EEG_AF7",
- "Muse_EEG_AF8",
- "Muse_EEG_TP10",
- "Muse_PPG_0",
- "Muse_PPG_1",
- "Muse_PPG_2",
- "Myo_GYR_X",
- "Myo_GYR_Y",
- "Myo_GYR_Z",
- "Myo_EMG_0",
- "Myo_EMG_1",
- "Myo_EMG_2",
- "Myo_EMG_3",
- "Myo_EMG_4",
- "Myo_EMG_5",
- "Myo_EMG_6",
- "Myo_EMG_7",
- "PICARD_fnirs_0",
- "PICARD_fnirs_1",
- "Polar_bpm",
- "Polar_hrv",
- "ViveEye_pupilPos_L_X",
- "ViveEye_pupilPos_L_Y",
- "ViveEye_pupilPos_R_X",
- "ViveEye_pupilPos_R_Y",
- "ViveEye_gazeOrigin_L_X",
- "ViveEye_gazeOrigin_L_Y",
- "ViveEye_gazeOrigin_L_Z",
- "ViveEye_gazeOrigin_R_X",
- "ViveEye_gazeOrigin_R_Y",
- "ViveEye_gazeOrigin_R_Z",
- "ViveEye_gazeDirection_L_X",
- "ViveEye_gazeDirection_L_Y",
- "ViveEye_gazeDirection_L_Z",
- "ViveEye_gazeDirection_R_X",
- "ViveEye_gazeDirection_R_Y",
- "ViveEye_gazeDirection_R_Z",
- "ViveEye_eyeOpenness_L",
- "ViveEye_pupilDiameter_L",
- "ViveEye_eyeOpenness_R",
- "ViveEye_pupilDiameter_R",
- "Zephyr_HR",
- "Zephyr_HRV",
- # target
+ expected_columns = DATA_COLS + [
"induced_state",
]
@@ -164,49 +52,49 @@ def read_and_prepare_data_chunk(df: pd.DataFrame) -> pd.DataFrame:
# df = _test_missing_features(df)
# test_suite
- if "test_suite" not in df.columns:
- df["test_suite"] = "test"
+ if "test_suite" not in data.columns:
+ data["test_suite"] = "test"
- df["time"] = pd.to_datetime(df["time"], unit="us")
- df["timestamp"] = df["time"].dt.round("1s")
- df = df.drop("time", axis=1)
+ data["time"] = pd.to_datetime(data["time"], unit="us")
+ data["timestamp"] = data["time"].dt.round("1s")
+ data = data.drop("time", axis=1)
- ags = df.groupby(METADATA_COLUMNS + ["timestamp"]).first()
+ ags = data.groupby(METADATA_COLUMNS + ["timestamp"]).first()
- fillna_constant = 0
+ fillna_constant = -999
# replace missing features with zero
- missing_features = [col for col in EXPECTED_COLUMNS if col not in ags.columns]
+ missing_features = [col for col in expected_columns if col not in ags.columns]
for col in missing_features:
ags[col] = fillna_constant
- # fill none values with 0
+ # fill none values with -999
ags = ags.fillna(fillna_constant)
- # fill -999 values with 0
- for v in NAN_VALUES:
- ags = ags.replace(v, fillna_constant)
+ # fill -999 values with -999
+ for value in NAN_VALUES:
+ ags = ags.replace(value, fillna_constant)
# keep columns in the same order
- ags = ags[EXPECTED_COLUMNS]
+ ags = ags[expected_columns]
return ags
-def get_dummy_template(df: pd.DataFrame) -> pd.DataFrame:
+def get_dummy_template(data: pd.DataFrame) -> pd.DataFrame:
"""
The template is needed to match the expected sample submission format.
"""
- df["time"] = pd.to_datetime(df["time"], unit="us")
- df["timestamp"] = df["time"].dt.round("1s")
- df = df.drop("time", axis=1)
- dummy_template = df.drop_duplicates(
+ data["time"] = pd.to_datetime(data["time"], unit="us")
+ data["timestamp"] = data["time"].dt.round("1s")
+ data = data.drop("time", axis=1)
+ dummy_template = data.drop_duplicates(
subset=METADATA_COLUMNS + ["timestamp"], keep="first"
).reset_index(drop=True)
dummy_template = dummy_template[METADATA_COLUMNS + ["timestamp"]]
return dummy_template
-def get_needed_data(df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame]:
+def get_needed_data(data: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame]:
"""
Read data for training/testing and prepare template format for submission
@@ -214,8 +102,8 @@ def get_needed_data(df: pd.DataFrame) -> Tuple[pd.DataFrame, pd.DataFrame]:
res1 - pd.DataFrame - read data
res2 - pd.DataFrame - template for submission
"""
- res1 = read_and_prepare_data_chunk(df)
- res2 = get_dummy_template(df)
+ res1 = read_and_prepare_data_chunk(data)
+ res2 = get_dummy_template(data)
return res1, res2
diff --git a/code/topcoder_cognitive_state/model.py b/code/topcoder_cognitive_state/model.py
index fcd3527..c041175 100644
--- a/code/topcoder_cognitive_state/model.py
+++ b/code/topcoder_cognitive_state/model.py
@@ -1,71 +1,23 @@
-from typing import List, Tuple
+"""
+This module: model functions
+"""
+# pylint: disable-msg=too-many-locals
+from typing import List, Tuple
import pandas as pd
import numpy as np
from lightgbm import LGBMClassifier
+from constants import DATA_COLS
+from processing import FeaturesGenerator
+
-from topcoder_cognitive_state.processing import FeaturesGenerator
-
-
-ALLOWED_IMPORTANT_FEATURES = set(
- [
- "E4_BVP",
- "E4_GSR",
- "LooxidLink_EEG_A3",
- "LooxidLink_EEG_A4",
- "LooxidLink_EEG_FP1",
- "LooxidLink_EEG_FP2",
- "LooxidLink_EEG_A7",
- "LooxidLink_EEG_A8",
- "Muse_EEG_TP9",
- "Muse_EEG_AF7",
- "Muse_EEG_AF8",
- "Muse_EEG_TP10",
- "Muse_PPG_0",
- "Muse_PPG_1",
- "Muse_PPG_2",
- "Myo_GYR_X",
- "Myo_GYR_Y",
- "Myo_GYR_Z",
- "Myo_EMG_0",
- "Myo_EMG_1",
- "Myo_EMG_2",
- "Myo_EMG_3",
- "Myo_EMG_4",
- "Myo_EMG_5",
- "Myo_EMG_6",
- "Myo_EMG_7",
- "PICARD_fnirs_0",
- "PICARD_fnirs_1",
- "Polar_bpm",
- "Polar_hrv",
- "ViveEye_eyeOpenness_L",
- "ViveEye_pupilDiameter_L",
- "ViveEye_pupilPos_L_X",
- "ViveEye_pupilPos_L_Y",
- "ViveEye_gazeOrigin_L_X",
- "ViveEye_gazeOrigin_L_Y",
- "ViveEye_gazeOrigin_L_Z",
- "ViveEye_gazeDirection_L_X",
- "ViveEye_gazeDirection_L_Y",
- "ViveEye_gazeDirection_L_Z",
- "ViveEye_eyeOpenness_R",
- "ViveEye_pupilDiameter_R",
- "ViveEye_pupilPos_R_X",
- "ViveEye_pupilPos_R_Y",
- "ViveEye_gazeOrigin_R_X",
- "ViveEye_gazeOrigin_R_Y",
- "ViveEye_gazeOrigin_R_Z",
- "ViveEye_gazeDirection_R_X",
- "ViveEye_gazeDirection_R_Y",
- "ViveEye_gazeDirection_R_Z",
- "Zephyr_HR",
- "Zephyr_HRV",
- ]
-)
+ALLOWED_IMPORTANT_FEATURES = set(DATA_COLS)
class Model:
+ """
+ This is main model class
+ """
def __init__(
self,
features: List[str],
@@ -88,23 +40,23 @@ def __init__(
self.models_3 = models_3
self.num_classes = 6
- def ensure_features(self, x: pd.DataFrame) -> pd.DataFrame:
+ def ensure_features(self, data: pd.DataFrame) -> pd.DataFrame:
"""
Transform dataframe into the same format which was used during training.
I.e., ensure that all features are present. If a feature is missing - replace it with 0.
Ensure order of features in the dataframe.
Args:
- x (pd.DataFrame): input data
+ data (pd.DataFrame): input data
Returns:
pd.DataFrame: processed data
"""
for col in self.features:
- if col not in x.columns:
- x[col] = 0
- x = x[self.features]
- return x
+ if col not in data.columns:
+ data[col] = 0
+ data = data[self.features]
+ return data
def predict(
self, x_raw: pd.DataFrame
@@ -116,13 +68,14 @@ def predict(
x_raw (pd.DataFrame): raw input data
Returns:
- Tuple[np.array, np.array, np.array, np.array, List[List[str]]]: predictions and most important features
+ Tuple[np.array, np.array, np.array, np.array, List[List[str]]]:
+ predictions and most important features
"""
# generate features
- x, _, _, group = self.preprocessor.generate_features(x_raw, get_targers=False)
+ data, _, _, group = self.preprocessor.generate_features(x_raw, get_targers=False)
# make predictions
- y_hat_1, most_important_features = make_predictions(x, group, self.models_1)
+ y_hat_1, most_important_features = make_predictions(data, group, self.models_1)
y_hat_3 = y_hat_1
# transform predictions into labels
@@ -131,40 +84,40 @@ def predict(
return y_hat_1, y_hat_3, y_hat_1_label, y_hat_3_label, most_important_features
-def sum_arrays(arrs: List[np.array]) -> np.array:
+def sum_arrays(arrays: List[np.array]) -> np.array:
"""
Calculate sum of list of arrays
"""
- x = arrs[0]
- if len(arrs) == 1:
- return x
- for x2 in arrs[1:]:
- x += x2
- return x
+ data = arrays[0]
+ if len(arrays) == 1:
+ return data
+ for _data in arrays[1:]:
+ data += _data
+ return data
-def mean_arrays(arrs) -> np.array:
+def mean_arrays(arrays) -> np.array:
"""
Calculate the mean of the list of arrays
"""
- return sum_arrays(arrs) / len(arrs)
+ return sum_arrays(arrays) / len(arrays)
-def postprocess_preds(preds: np.array, group: pd.Series) -> pd.DataFrame:
+def postprocess_preds(predicts: np.array, group: pd.Series) -> pd.DataFrame:
"""
Smooth predictions by running rolling mean within the group
Args:
- preds (np.array): array of predicted probs
+ predicts (np.array): array of predicted probs
group (pd.Series): group id
Returns:
pd.DataFrame: smoothed predicted probs
"""
- df = pd.DataFrame(preds)
- df["group"] = group
- res = df.groupby("group").rolling(window=999_999, min_periods=1).mean().values
- return res
+ data = pd.DataFrame(predicts)
+ data["group"] = group
+ result = data.groupby("group").rolling(window=999_999, min_periods=1).mean().values
+ return result
def make_predictions(
@@ -190,7 +143,8 @@ def make_predictions(
contribs = model.predict_proba(data, pred_contrib=True)
# contribs - list of n_samples * (n_features+1)*n_classes
- # drop shap sum column: (n_samples, (n_features + 1) * n_classes) -> n_samples, n_features * n_classes
+ # drop shap sum column: (n_samples, (n_features + 1) * n_classes) ->
+ # n_samples, n_features * n_classes
y_shap_pred = np.vstack(contribs)
n_features = data.shape[1]
indexes = [
diff --git a/code/topcoder_cognitive_state/opt_params.py b/code/topcoder_cognitive_state/opt_params.py
index 019fccb..06ee0a4 100644
--- a/code/topcoder_cognitive_state/opt_params.py
+++ b/code/topcoder_cognitive_state/opt_params.py
@@ -1,28 +1,31 @@
+"""
+This module: fine tune hyper params.
+"""
+
+import os
import sys
import warnings
import copy
import logging
import optuna
-
-from topcoder_cognitive_state.load_data import read_data
-from topcoder_cognitive_state.processing import FeaturesGenerator
-from topcoder_cognitive_state.model import Model # noqa: F401
-from topcoder_cognitive_state.train import train_models
-
+from load_data import read_data
+from processing import FeaturesGenerator
+from train import train_models
+os.environ["CUDA_VISIBLE_DEVICES"] = "0"
warnings.filterwarnings("ignore")
def main():
"""
- Optimize model's hyperparams using optuna
+ hyperparams fine tune using optuna
"""
- if len(sys.argv) < 2 or len(sys.argv[1]) == 0:
- print("Training input file is missing.")
+ if len(sys.argv[1]) == 0 or len(sys.argv) < 2:
+ print("Input file of opt params is missing.")
return 1
- if len(sys.argv) < 3 or len(sys.argv[2]) == 0:
- print("Path to log is missing")
+ if len(sys.argv[2]) == 0 or len(sys.argv) < 3:
+ print("log path is missing")
return 1
path_to_log = sys.argv[2]
@@ -32,14 +35,12 @@ def main():
handlers=[logging.FileHandler(path_to_log, mode="w"), logging.StreamHandler()],
)
- print("Training started.")
-
input_file = sys.argv[1]
_ = sys.argv[2]
data, _ = read_data(input_file)
processor = FeaturesGenerator()
- X, Y1, Y3, META = processor.generate_features_train(data)
+ train_x, train_y1, train_y3, meta = processor.generate_features_train(data)
default_params = {
"num_leaves": 127,
@@ -52,45 +53,44 @@ def main():
"reg_alpha": 1.0,
"reg_lambda": 1.0,
"random_state": 42,
+ "device": "gpu",
+ "gpu_platform_id": 1,
+ "gpu_device_id": 0
}
def objective(trial):
params = copy.deepcopy(default_params)
params.update(
{
- "num_leaves": trial.suggest_int("num_leaves", 7, 255, 8),
+ "num_leaves": trial.suggest_int("num_leaves", 7, 255, 4),
"max_depth": trial.suggest_int("max_depth", 3, 14, 1),
"min_child_weight": trial.suggest_loguniform(
- "min_child_weight", 1e-18, 1
+ "min_child_weight", 1e-3, 1
),
"min_child_samples": trial.suggest_int(
"min_child_samples", 1, 100, 1, log=True
),
- "min_split_gain": trial.suggest_loguniform("min_split_gain", 1e-18, 1),
- "subsample": trial.suggest_float("subsample", 0.1, 1.0),
+ "min_split_gain": trial.suggest_loguniform("min_split_gain", 1e-3, 1),
+ "subsample": trial.suggest_float("subsample", 0.6, 1.0),
"colsample_bytree": trial.suggest_float("colsample_bytree", 0.1, 1.0),
- "reg_alpha": trial.suggest_float("reg_alpha", 0.0, 10),
- "reg_lambda": trial.suggest_float("reg_lambda", 0.0, 10),
+ "reg_alpha": trial.suggest_float("reg_alpha", 1, 10),
+ "reg_lambda": trial.suggest_float("reg_lambda", 1, 10),
}
)
- _, _, test_score = train_models(X, Y1, Y3, META, params_to_train=[params])
+ _, _, test_score = train_models(train_x, train_y1, train_y3, meta, params_to_train=[params])
- logging.info(f"Next itter score - {test_score}")
- for k, v in params.items():
- logging.info(f"'{k}': {v},")
- logging.info("")
return test_score
study = optuna.create_study(direction="maximize")
study.optimize(objective, n_trials=100)
- logging.info("Number of finished trials: {}".format(len(study.trials)))
+ logging.info("Number of finished trials: %s", len(study.trials))
logging.info("Best trial:")
best_trial = study.best_trial
- logging.info(" Value: {}".format(best_trial.value))
+ logging.info(" Value: %s", best_trial.value)
logging.info(" Params: ")
for key, value in best_trial.params.items():
- logging.info(" {}: {}".format(key, value))
+ logging.info(" %s: %s", key, value)
if __name__ == "__main__":
diff --git a/code/topcoder_cognitive_state/processing.py b/code/topcoder_cognitive_state/processing.py
index b6b7d35..cd32320 100644
--- a/code/topcoder_cognitive_state/processing.py
+++ b/code/topcoder_cognitive_state/processing.py
@@ -1,32 +1,23 @@
-from typing import List, Tuple
-import re
+"""
+This module: processing data and get features.
+"""
+# pylint: disable-msg=too-many-locals
+from typing import Tuple
+import re
+import warnings
import pandas as pd
import numpy as np
from tqdm import tqdm
+from constants import TARGET2LABEL, LABEL2TARGET
-from topcoder_cognitive_state.CONSTANTS import TARGET2LABEL, LABEL2TARGET
+warnings.filterwarnings("ignore")
-def get_distance(x1: List[pd.Series], x2: List[pd.Series]) -> pd.Series:
+class FeaturesGenerator:
"""
- Calculate l2 distance between points
-
- Args:
- x1 (List[pd.Series]): list of point x1 coordinates, e.g. X1, Y1, Z1
- x2 (List[pd.Series]): list of point x2 coordinates, e.g. X2, Y2, Z2
-
- Returns:
- pd.Series: l2 distance
+ FeaturesGenerator is used for feature generating
"""
- delta = 0
- for a1, a2 in zip(x1, x2):
- delta += (a1 - a2) ** 2
- delta = delta ** (1 / len(x1))
- return delta
-
-
-class FeaturesGenerator:
def __init__(self, target_column: str = "induced_state"):
"""
Generate features using raw data
@@ -35,22 +26,22 @@ def __init__(self, target_column: str = "induced_state"):
target_column (str, optional): target column. Defaults to "induced_state".
"""
self.target_column = target_column
-
self.target2label = TARGET2LABEL
self.label2target = LABEL2TARGET
- def rename_cols(self, x: pd.DataFrame):
+ @classmethod
+ def rename_cols(cls, data: pd.DataFrame):
"""
Lightgbm doesn't work well with all features names, so we need to rename some of them."""
- x = x.rename(columns=lambda x: re.sub("[^A-Za-z0-9_]+", "", x))
- return x
+ data = data.rename(columns=lambda x: re.sub("[^A-Za-z0-9_]+", "", x))
+ return data
- def get_targets(self, x: pd.DataFrame, future: int = 1) -> pd.Series:
+ def get_targets(self, data: pd.DataFrame, future: int = 1) -> pd.Series:
"""
Create training targets
Args:
- x (pd.DataFrame): input data
+ data (pd.DataFrame): input data
future (int, optional): steps to look into future.
Future=1 equals the current moment (t).
Future=2 equals to the next moment (t+1).
@@ -60,199 +51,176 @@ def get_targets(self, x: pd.DataFrame, future: int = 1) -> pd.Series:
pd.Series: target
"""
if future == 0:
- y = x[self.target_column]
+ target = data[self.target_column]
else:
- y = (
- x.groupby("session_id")[self.target_column]
+ target = (
+ data.groupby("session_id")[self.target_column]
.shift(-1 * future)
.fillna(method="ffill") # replace missing targets with previous value
)
- ind = y.isnull()
- y[ind] = x.loc[ind, self.target_column]
- y_label = self.apply_target2label(y)
- return y_label
+ ind = target.isnull()
+ target[ind] = data.loc[ind, self.target_column]
+ target_label = self.apply_target2label(target)
+ return target_label
- def apply_target2label(self, y: pd.Series) -> pd.Series:
+ def apply_target2label(self, target: pd.Series) -> pd.Series:
+ """
+ target to label
+ Args:
+ target:
+ Returns: label results
+ """
# 'low' -> 0, 'medium' -> 1, etc
- return pd.Series(y).map(self.target2label)
-
- def apply_label2target(self, y: pd.Series) -> pd.Series:
- # -> 'low', 1 -> 'medium', etc
- return pd.Series(y).map(self.label2target)
+ return pd.Series(target).map(self.target2label)
- def calc_eyes_distances(self, x: pd.DataFrame) -> pd.DataFrame:
+ def apply_label2target(self, target: pd.Series) -> pd.Series:
"""
- Calculate features based on raw features from the eyes tracker.
-
+ label to target
Args:
- x (pd.DataFrame): input data
-
- Returns:
- pd.DataFrame: new dataframe with new features
+ target:
+ Returns: target results
"""
- new_features = pd.DataFrame({})
- shifts = [1, 3]
- eyes = ["L", "R"]
-
- # L / R eyes
- new_features["ViveEye_pupilPos_LR_distance"] = get_distance(
- (x["ViveEye_pupilPos_L_X"], x["ViveEye_pupilPos_L_Y"]),
- (x["ViveEye_pupilPos_R_X"], x["ViveEye_pupilPos_R_Y"]),
- )
-
- # distances
- for s in shifts:
- for pos in eyes:
- new_features[f"ViveEye_pupilPos_distance_{s}_{pos}"] = get_distance(
- (x[f"ViveEye_pupilPos_{pos}_X"], x[f"ViveEye_pupilPos_{pos}_Y"]),
- (
- x.groupby("session_id")[f"ViveEye_pupilPos_{pos}_X"]
- .shift(s)
- .values,
- x.groupby("session_id")[f"ViveEye_pupilPos_{pos}_Y"]
- .shift(s)
- .values,
- ),
- )
-
- # L / R eyes
- for feature in ["ViveEye_gazeOrigin", "ViveEye_gazeDirection"]:
- new_features["{feature}_LR_distance"] = get_distance(
- (x[f"{feature}_L_X"], x[f"{feature}_L_Y"], x[f"{feature}_L_Z"]),
- (x[f"{feature}_R_X"], x[f"{feature}_R_Y"], x[f"{feature}_R_Z"]),
- )
- for s in shifts:
- for pos in eyes:
- new_features[f"distance_{feature}_{s}_{pos}"] = get_distance(
- (
- x[f"{feature}_{pos}_X"],
- x[f"{feature}_{pos}_Y"],
- x[f"{feature}_{pos}_Z"],
- ),
- (
- x.groupby("session_id")[f"{feature}_{pos}_X"]
- .shift(s)
- .values,
- x.groupby("session_id")[f"{feature}_{pos}_Y"]
- .shift(s)
- .values,
- x.groupby("session_id")[f"{feature}_{pos}_Z"]
- .shift(s)
- .values,
- ),
- )
- return new_features
-
- def get_session_id_and_time_since_break(self, data: pd.DataFrame) -> pd.DataFrame:
+ # -> 'low', 1 -> 'medium', etc
+ return pd.Series(target).map(self.label2target)
+
+ @classmethod
+ def get_session_id_and_time_since_break(cls, data: pd.DataFrame) -> pd.DataFrame:
"""
Calculate session id and time since the last break.
"""
tmp = data.copy().reset_index()
diff = tmp["timestamp"].diff().dt.total_seconds()
tmp["session_id"] = np.cumsum(diff > 1)
-
data["session_id"] = tmp["session_id"].values
return data
def generate_features(
- self, x: pd.DataFrame, get_targers: bool = False
+ self, data: pd.DataFrame, get_targers: bool = False
) -> Tuple[pd.DataFrame, pd.Series, pd.Series, pd.Series]:
"""
Process raw data and create new features for training
Args:
- x (pd.DataFrame): raw input data
+ data (pd.DataFrame): raw input data
get_targers (bool, optional): generate targets? Needed for training. Defaults to False.
Returns:
Tuple[pd.DataFrame, pd.Series, pd.Series, pd.Series]: output
"""
- x = self.get_session_id_and_time_since_break(x)
- x = x.reset_index(drop=True)
+ data = self.get_session_id_and_time_since_break(data)
+ data = data.reset_index(drop=True)
# create targets for t and t+3 models
if get_targers:
- y1 = self.get_targets(x, future=0)
- y3 = self.get_targets(x, future=3)
+ target1 = self.get_targets(data, future=0)
+ target3 = self.get_targets(data, future=3)
else:
- y1 = None
- y3 = None
- if self.target_column in x.columns:
- x = x.drop(self.target_column, axis=1)
+ target1 = None
+ target3 = None
+ if self.target_column in data.columns:
+ data = data.drop(self.target_column, axis=1)
# additional featurers
- for c1, c2 in [
+ for feature1, feature2 in [
("Zephyr_HR", "Zephyr_HRV"),
("Polar_bpm", "Polar_hrv"),
("Zephyr_HRV", "Polar_hrv"),
]:
- x[f"{c1}_div_{c2}"] = x[c1] / x[c2]
+ data[f"{feature1}_div_{feature2}"] = data[feature1] / data[feature2]
- dfs = [x]
+ datas = [data]
# create rolling stats features
- windows = [5, 999_999]
- cols = list(x.columns)
- for w in windows:
+ windows = [5, 10, 20, 50, 100, 500, 1000]
+ cols = list(data.columns)
+ for window in windows:
rolling_mean = (
- x.groupby("session_id")[cols]
- .rolling(min_periods=1, window=w)
+ data.groupby("session_id")[cols]
+ .rolling(min_periods=1, window=window)
.mean()
.reset_index(drop=True)
)
+ rolling_max = (
+ data.groupby("session_id")[cols]
+ .rolling(min_periods=1, window=window)
+ .max()
+ .reset_index(drop=True)
+ )
+ rolling_median = (
+ data.groupby("session_id")[cols]
+ .rolling(min_periods=1, window=window)
+ .median()
+ .reset_index(drop=True)
+ )
rolling_std = (
- x.groupby("session_id")[cols]
- .rolling(min_periods=1, window=w)
+ data.groupby("session_id")[cols]
+ .rolling(min_periods=1, window=window)
.std()
.reset_index(drop=True)
)
- normed = (x - rolling_mean) / (rolling_std + 1)
+ normed = (data - rolling_mean) / (rolling_std + 1)
- normed = normed.add_prefix(f"normed_by_session_{w}_")
- rolling_mean = rolling_mean.add_prefix(f"mean_by_session_{w}_")
- rolling_std = rolling_std.add_prefix(f"std_by_session_{w}_")
+ normed = normed.add_prefix(f"normed_by_session_{window}_")
+ rolling_mean = rolling_mean.add_prefix(f"mean_by_session_{window}_")
+ rolling_median = rolling_median.add_prefix(f"median_by_session_{window}_")
+ rolling_max = rolling_max.add_prefix(f"max_by_session_{window}_")
+ rolling_std = rolling_std.add_prefix(f"std_by_session_{window}_")
- dfs += [rolling_mean, rolling_std, normed]
+ datas += [rolling_max, rolling_mean, rolling_median, rolling_std, normed]
# create global stats features
windows = [999_999]
global_cols = cols
- for w in windows:
+ for window in windows:
rolling_mean = (
- x[global_cols]
- .rolling(min_periods=2, window=w)
+ data[global_cols]
+ .rolling(min_periods=2, window=window)
.mean()
.reset_index(drop=True)
)
+ rolling_max = (
+ data[global_cols]
+ .rolling(min_periods=2, window=window)
+ .max()
+ .reset_index(drop=True)
+ )
+ rolling_median = (
+ data[global_cols]
+ .rolling(min_periods=2, window=window)
+ .median()
+ .reset_index(drop=True)
+ )
rolling_std = (
- x[global_cols]
- .rolling(min_periods=2, window=w)
+ data[global_cols]
+ .rolling(min_periods=2, window=window)
.std()
.reset_index(drop=True)
)
- normed = (x - rolling_mean) / (rolling_std + 1)
+ normed = (data - rolling_mean) / (rolling_std + 1)
- normed = normed.add_prefix(f"normed_global_{w}_")
- rolling_mean = rolling_mean.add_prefix(f"mean_global_{w}_")
- rolling_std = rolling_std.add_prefix(f"std_global_{w}_")
+ normed = normed.add_prefix(f"normed_global_{window}_")
+ rolling_mean = rolling_mean.add_prefix(f"mean_global_{window}_")
+ rolling_median = rolling_median.add_prefix(f"median_global_{window}_")
+ rolling_max = rolling_max.add_prefix(f"max_global_{window}_")
+ rolling_std = rolling_std.add_prefix(f"std_global_{window}_")
- dfs += [rolling_mean, rolling_std, normed]
+ datas += [rolling_max, rolling_mean, rolling_median, rolling_std, normed]
# shift features
- for s in [1, 3]:
- gr_s = x.groupby("session_id")[cols].shift(s).reset_index(drop=True)
- tmp = x - gr_s
- tmp = tmp.add_prefix(f"shift_{s}_")
- dfs += [tmp]
+ for shift in [1, 3, 5]:
+ gr_s = data.groupby("session_id")[cols].shift(shift).reset_index(drop=True)
+ tmp = data - gr_s
+ tmp = tmp.add_prefix(f"shift_{shift}_")
+ datas += [tmp]
- df = pd.concat(dfs, axis=1)
- df = pd.concat([x, self.calc_eyes_distances(df)], axis=1)
+ datas = pd.concat(datas, axis=1)
+ # df = pd.concat([x, self.calc_eyes_distances(df)], axis=1)
- session_id = df["session_id"].copy()
+ session_id = datas["session_id"].copy()
- df = df.drop("session_id", axis=1)
- df = self.rename_cols(df)
- return df, y1, y3, session_id
+ datas = datas.drop("session_id", axis=1)
+ datas = self.rename_cols(datas)
+ return datas, target1, target3, session_id
def generate_features_train(
self, data: pd.DataFrame
@@ -268,24 +236,25 @@ def generate_features_train(
indexes = list(set(indexes))
# for each comb - generate features
- X, Y1, Y3, META = [], [], [], []
+ feature, target1, target3, meta = [], [], [], []
for index in tqdm(indexes):
index_data_to_select = index # use full index
- x = data.loc[index_data_to_select]
+ data_index = data.loc[index_data_to_select]
- r, y1, y3, session_id = self.generate_features(x, get_targers=True)
+ _feature, _target1, _target3, session_id = self.generate_features(data_index,
+ get_targers=True)
- X.append(r)
- Y1.append(y1)
- Y3.append(y3)
+ feature.append(_feature)
+ target1.append(_target1)
+ target3.append(_target3)
# group id = person + task
- task = pd.Series([index] * r.shape[0])
+ task = pd.Series([index] * _feature.shape[0])
_meta = task.astype("str") + "__" + session_id.astype("str")
- META.append(_meta)
+ meta.append(_meta)
- X = pd.concat(X, axis=0).reset_index(drop=True)
- Y1 = pd.concat(Y1, axis=0).reset_index(drop=True)
- Y3 = pd.concat(Y3, axis=0).reset_index(drop=True)
- META = pd.concat(META, axis=0).reset_index(drop=True)
- return X, Y1, Y3, META
+ feature = pd.concat(feature, axis=0).reset_index(drop=True)
+ target1 = pd.concat(target1, axis=0).reset_index(drop=True)
+ target3 = pd.concat(target3, axis=0).reset_index(drop=True)
+ meta = pd.concat(meta, axis=0).reset_index(drop=True)
+ return feature, target1, target3, meta
diff --git a/code/topcoder_cognitive_state/test.py b/code/topcoder_cognitive_state/test.py
index 025bc1d..a81e8a1 100755
--- a/code/topcoder_cognitive_state/test.py
+++ b/code/topcoder_cognitive_state/test.py
@@ -1,15 +1,17 @@
+"""
+This module: predict input data.
+"""
+
from typing import List
import sys
import warnings
import time
import pickle
-
import numpy as np
import pandas as pd
from tqdm import tqdm
-
-from topcoder_cognitive_state.load_data import read_data
-from topcoder_cognitive_state.model import Model
+from load_data import read_data
+from model import Model
warnings.filterwarnings("ignore")
@@ -19,19 +21,19 @@ def arrays_to_str_list(arr: np.array) -> List[str]:
Transform predictions arrays into a list of strings to match the submission format.
"""
result = []
- for i in range(arr.shape[0]):
- tmp = list(arr[i, :])
- tmp = "[" + " ".join([str(s) for s in tmp]) + "]"
+ for index in range(arr.shape[0]):
+ tmp = list(arr[index, :])
+ tmp = "[" + " ".join([str(_tmp) for _tmp in tmp]) + "]"
result.append(str(tmp))
return result
-def lists_to_str_list(arr: List[List[str]]) -> List[str]:
+def lists_to_str_list(array: List[List[str]]) -> List[str]:
"""
Transform most important features arrays into a list of strings to match submission format.
"""
result = []
- for tmp in arr:
+ for tmp in array:
tmp = "[" + " ".join(["'" + str(s) + "'" for s in tmp]) + "]"
result.append(str(tmp))
return result
@@ -103,11 +105,11 @@ def make_predictions(
t_start = time.time()
# get unique
- tmp = [i[0] for i in data.index]
+ temp = [index[0] for index in data.index]
test_suites = []
- for t in tmp:
- if t not in test_suites:
- test_suites.append(t)
+ for _temp in temp:
+ if _temp not in test_suites:
+ test_suites.append(_temp)
# make predictions
result = []
@@ -131,6 +133,11 @@ def make_predictions(
def main():
+ """
+ main entry
+ Returns:
+
+ """
if len(sys.argv) < 2 or len(sys.argv[1]) == 0:
print("Testing input file is missing.")
return 1
@@ -145,8 +152,8 @@ def main():
output_file = sys.argv[2]
model_file = sys.argv[3]
- with open(model_file, "rb") as f:
- model = pickle.load(f)
+ with open(model_file, "rb") as file:
+ model = pickle.load(file)
# load data
data, dummies = read_data(input_file)
diff --git a/code/topcoder_cognitive_state/train.py b/code/topcoder_cognitive_state/train.py
index 62315c5..b3a6bfe 100755
--- a/code/topcoder_cognitive_state/train.py
+++ b/code/topcoder_cognitive_state/train.py
@@ -1,65 +1,71 @@
+"""
+This module: train model.
+"""
+# pylint: disable-msg=too-many-locals
+
from typing import List, Optional, Dict, Tuple
import sys
import warnings
import pickle
-
+import pandas as pd
from sklearn.metrics import roc_auc_score
+from sklearn.model_selection import StratifiedGroupKFold
import numpy as np
from tqdm import tqdm
from lightgbm import LGBMClassifier
-import pandas as pd
-
-from topcoder_cognitive_state.load_data import read_data
-from topcoder_cognitive_state.processing import FeaturesGenerator
-from sklearn.model_selection import StratifiedGroupKFold
-from topcoder_cognitive_state.model import Model
+from load_data import read_data
+from processing import FeaturesGenerator
+from model import Model
warnings.filterwarnings("ignore")
-# lightgbm model hyperparams
+
PARAMS = [
{
- "num_leaves": 151,
- "learning_rate": 0.12,
+ "num_leaves": 203,
+ 'max_depth': 3,
+ "learning_rate": 0.1,
"n_estimators": 500,
- "min_child_samples": 4,
- "subsample": 0.96,
+ "min_child_samples": 30,
+ "subsample": 0.622584668281795,
"subsample_freq": 5,
- "colsample_bytree": 0.115,
- "reg_alpha": 0.105,
- "reg_lambda": 9.90,
- "min_child_weight": 0.005519,
- "min_split_gain": 1.94e-14,
+ "colsample_bytree": 0.17974037482084979,
+ "reg_alpha": 3.6558239011297524,
+ "reg_lambda": 5.865351102213303,
+ "min_child_weight": 0.4388354272235035,
+ "min_split_gain": 0.006193549861032794,
"random_state": 42,
},
{
- "num_leaves": 79,
- "learning_rate": 0.12,
- "n_estimators": 600,
- "min_child_samples": 14,
- "subsample": 0.75,
+ "num_leaves": 227,
+ 'max_depth': 3,
+ "learning_rate": 0.1,
+ "n_estimators": 500,
+ "min_child_samples": 84,
+ "subsample": 0.7048761531428217,
"subsample_freq": 5,
- "colsample_bytree": 0.75,
- "reg_alpha": 2.2,
- "reg_lambda": 1.5,
- "random_state": 424242,
- "min_child_weight": 6.681437316563333e-12,
- "min_split_gain": 0.00039529173804292325,
+ "colsample_bytree": 0.2806650533512902,
+ "reg_alpha": 6.062594325136659,
+ "reg_lambda": 4.427396504735556,
+ "random_state": 42,
+ "min_child_weight": 0.12139028719958726,
+ "min_split_gain": 0.0015430980179050107,
},
{
- "num_leaves": 23,
- "learning_rate": 0.12,
+ "num_leaves": 247,
+ 'max_depth': 3,
+ "learning_rate": 0.1,
"n_estimators": 500,
- "min_child_samples": 30,
- "subsample": 0.6,
+ "min_child_samples": 87,
+ "subsample": 0.8640475862646084,
"subsample_freq": 5,
- "colsample_bytree": 0.4,
- "reg_alpha": 0,
- "reg_lambda": 0,
- "random_state": 4242,
- "min_child_weight": 0.28,
- "min_split_gain": 9.793058539831146e-08,
+ "colsample_bytree": 0.22858610039131327,
+ "reg_alpha": 3.750962348348377,
+ "reg_lambda": 4.394908953876854,
+ 'random_state': 42,
+ "min_child_weight": 0.021180547250587045,
+ "min_split_gain": 0.0018611688327432564,
},
]
@@ -78,42 +84,41 @@ def get_auc(y_true: np.array, probas_pred: np.array) -> Tuple[str, float, bool]:
aucs = []
preds = np.array(probas_pred)
preds = preds.reshape(-1, 6)
- for i in range(6):
- auc = roc_auc_score(y_true == i, preds[:, i])
+ for index in range(6):
+ auc = roc_auc_score(y_true == index, preds[:, index])
aucs.append(auc)
score = np.mean(auc)
return "mean_auc", score, True
def drop_null_targets(
- X_raw: pd.DataFrame, y_raw: pd.Series
+ x_raw: pd.DataFrame, y_raw: pd.Series
) -> Tuple[pd.DataFrame, pd.Series]:
"""
Drop all rows where target is missing
Args:
- X_raw (pd.DataFrame): features
+ x_raw (pd.DataFrame): features
y_raw (pd.Series): targets
Returns:
Tuple[pd.DataFrame, pd.Series]: features and targets without rows with missing data
"""
- X = X_raw.reset_index(drop=True)
- y = y_raw.reset_index(drop=True)
- ind = y.notnull()
- return X.loc[ind], y[ind]
+ x_raw = x_raw.reset_index(drop=True)
+ y_raw = y_raw.reset_index(drop=True)
+ ind = y_raw.notnull()
+ return x_raw.loc[ind], y_raw[ind]
def train_model(
- X: pd.DataFrame, Y1: pd.Series, Y3: pd.Series
+ feature: pd.DataFrame, target1: pd.Series
) -> Tuple[List[LGBMClassifier], List[LGBMClassifier]]:
"""
Train models for t and t+3 predictions
Args:
- X (pd.DataFrame): input features
- Y1 (pd.Series): t targets
- Y3 (pd.Series): t+3 targets
+ feature (pd.DataFrame): input features
+ target1 (pd.Series): t targets
Returns:
List[LGBMClassifier]: list of trained models
@@ -121,36 +126,36 @@ def train_model(
models_1 = []
models_3 = []
for params in PARAMS:
- print(params)
model_1 = LGBMClassifier(**params)
- model_1.fit(X, Y1, eval_set=(X, Y1), verbose=100)
+ model_1.fit(feature, target1, eval_set=(feature, target1), verbose=100)
models_1 += [model_1]
models_3 += [model_1]
- print("\n")
return models_1, models_3
def train_models(
- X: pd.Dataframe,
- Y1: pd.Series,
- Y3: pd.Series,
- META: pd.Series,
+ feature: pd.DataFrame,
+ target1: pd.Series,
+ target3: pd.Series,
+ meta: pd.Series,
params_to_train: Optional[List[Dict]] = None,
) -> Tuple[List[LGBMClassifier], List[LGBMClassifier], float]:
"""
Train models
Args:
- X (pd.DataFrame): input features
- Y1 (pd.Series): t targets
- Y3 (pd.Series): t+3 targets
- META (pd.Series): metadata
- params_to_train (Optional[List[Dict]], optional): list of dicts of hyperparams to use for training. Defaults to None.
+ feature (pd.DataFrame): input features
+ target1 (pd.Series): t targets
+ target3 (pd.Series): t+3 targets
+ meta (pd.Series): metadata
+ params_to_train (Optional[List[Dict]], optional): list of dicts of hyperparams to use for
+ training. Defaults to None.
Returns:
- Tuple[List[LGBMClassifier], List[LGBMClassifier], float]: t and t+3 trained models, mean validation score
+ Tuple[List[LGBMClassifier], List[LGBMClassifier], float]: t and t+3 trained models,
+ mean validation score
"""
if params_to_train is None:
params_to_train = PARAMS
@@ -158,12 +163,12 @@ def train_models(
# use repeated statified group kfold for training
folds = list(
StratifiedGroupKFold(n_splits=5, shuffle=True, random_state=42).split(
- X, Y1, META
+ feature, target1, meta
)
)
folds += list(
StratifiedGroupKFold(n_splits=5, shuffle=True, random_state=100).split(
- X, Y1, META
+ feature, target1, meta
)
)
@@ -176,29 +181,29 @@ def train_models(
fin_scores = []
for fold_ind, (train_index, test_index) in tqdm(enumerate(folds)):
# get split
- X_train, X_test = X.loc[train_index], X.loc[test_index]
- y_1_train, y_1_test = Y1[train_index], Y1[test_index]
- y_3_train, y_3_test = Y3[train_index], Y3[test_index]
+ x_train, x_test = feature.loc[train_index], feature.loc[test_index]
+ y_1_train, y_1_test = target1[train_index], target1[test_index]
+ y_3_train, y_3_test = target3[train_index], target3[test_index]
# meta_train, meta_val = META[train_index], META[test_index]
# drop null targets
- X_train_1, y_train_1 = drop_null_targets(X_train, y_1_train)
- _, _ = drop_null_targets(X_train, y_3_train)
- X_val_1, y_val_1 = drop_null_targets(X_test, y_1_test)
- _, _ = drop_null_targets(X_test, y_3_test)
+ x_train_1, y_train_1 = drop_null_targets(x_train, y_1_train)
+ _, _ = drop_null_targets(x_train, y_3_train)
+ x_val_1, y_val_1 = drop_null_targets(x_test, y_1_test)
+ _, _ = drop_null_targets(x_test, y_3_test)
try:
# train models
model_1 = LGBMClassifier(**params)
model_1.fit(
- X_train_1,
+ x_train_1,
y_train_1,
- eval_set=(X_val_1, y_val_1),
+ eval_set=(x_val_1, y_val_1),
verbose=100,
early_stopping_rounds=50,
)
auc_1 = get_auc(
- y_true=y_val_1, probas_pred=model_1.predict_proba(X_val_1)
+ y_true=y_val_1, probas_pred=model_1.predict_proba(x_val_1)
)[1]
scores_1.append(auc_1)
models_1.append(model_1)
@@ -208,13 +213,10 @@ def train_models(
fin_score = 0.7 * auc_1 + 0.3 * auc_3
fin_scores.append(fin_score)
- except Exception as e:
- print(f"Error: {e}. Ignoring")
- print(f"Fold {fold_ind}. Score: {fin_score}")
- print("Params")
- print(params)
+ except ValueError as exception:
+ print(f"Error: {exception}. Ignoring")
+ print(f"Fold {fold_ind}. Score: {fin_scores}")
mean_score = np.mean(fin_scores)
- print(f"Mean score = {mean_score}")
return models_1, models_3, mean_score
@@ -237,17 +239,17 @@ def main():
data, _ = read_data(input_file)
processor = FeaturesGenerator()
- X, Y1, Y3, _ = processor.generate_features_train(data)
- models_1, models_3 = train_model(X, Y1, Y3)
+ feature, target1, _, _ = processor.generate_features_train(data)
+ models_1, models_3 = train_model(feature, target1)
main_model = Model(
- features=list(X.columns),
+ features=list(feature.columns),
preprocessor=processor,
models_1=models_1,
models_3=models_3,
)
- with open(output_file, "wb") as f:
- pickle.dump(main_model, f, pickle.HIGHEST_PROTOCOL)
+ with open(output_file, "wb") as file:
+ pickle.dump(main_model, file, pickle.HIGHEST_PROTOCOL)
print("Training finished.")
return 0