prediction_uncertainty breaks subkey matching

[rushk014]

prediction_uncertainty breaks subkey matching

Currently prediction_uncertainty uses dict_to_df to perform aggregation and then recreates Subkeys.
As a result the recreated Subkeys lose their prior _output_matching and _sep_dicts_id information. This can break pipelines that rely on mean_dict/std_dict for later parts of the pipeline.

Reproducible Example

X, y = sklearn.datasets.make_classification(n_samples=100, n_features=5)
X_train, X_test, y_train, y_test = init_args(train_test_split(X, y), names=['xtr', 'xte', 'ytr', 'yte'])

subsampling_funcs = [partial(sklearn.utils.resample, n_samples=80, random_state=i) for i in range(5)]
subsampling_set = Vset(name='subsample', modules=subsampling_funcs, output_matching=True)
X_trains, y_trains = subsampling_set(X_train, y_train)

subsampling_set_test = Vset(name='subsample_test', modules=subsampling_funcs, output_matching=True)
X_tests, y_tests = subsampling_set_test(X_test, y_test)

models = [LogisticRegression(max_iter=1000, tol=0.1), DecisionTreeClassifier()]
modeling_set = Vset(name='model', modules=models, module_keys=["LR", "DT"])
modeling_set.fit(X_trains, y_trains)

# clamp mean predictions over test-set subsamples
mean_dict, std_dict, pred_stats_df = modeling_set.predict(X_tests, with_uncertainty=True, group_by=['subsample_test'])
mean_dict = {k: np.round(v) if k != PREV_KEY else v for k, v in mean_dict.items()}

failed_metrics = binary_metrics_set.evaluate(mean_dict, y_tests)
failed_metrics

failed_metrics should have matched mean_dict keys of the form (subsample_test_0, ) with y_tests keys of the form (yte, subsample_test_0), but cannot because the Subkeys have different _sep_dicts_id due to recreation.

Expected Output

{(subsample_test_0, yte, Acc): 0.925,
 (subsample_test_1, yte, Acc): 0.9125,
 (subsample_test_2, yte, Acc): 0.8875,
 (subsample_test_3, yte, Acc): 0.9,
 (subsample_test_4, yte, Acc): 0.9,
 (subsample_test_0, yte, Bal_Acc): 0.9268292682926829,
 (subsample_test_1, yte, Bal_Acc): 0.9,
 (subsample_test_2, yte, Bal_Acc): 0.8902439024390244,
 (subsample_test_3, yte, Bal_Acc): 0.9024390243902439,
 (subsample_test_4, yte, Bal_Acc): 0.9069767441860466, ...}

Actual Output

{(subsample_test_0, yte, subsample_test_0, Acc): 0.9125,
 (subsample_test_0, yte, subsample_test_1, Acc): 0.5875,
 (subsample_test_0, yte, subsample_test_2, Acc): 0.4875,
 (subsample_test_0, yte, subsample_test_3, Acc): 0.475,
 (subsample_test_0, yte, subsample_test_4, Acc): 0.6,
 (subsample_test_1, yte, subsample_test_0, Acc): 0.625,
 (subsample_test_1, yte, subsample_test_1, Acc): 0.925,
 (subsample_test_1, yte, subsample_test_2, Acc): 0.525,
 (subsample_test_1, yte, subsample_test_3, Acc): 0.4375,
 (subsample_test_1, yte, subsample_test_4, Acc): 0.5375, ... }