xgboost and pandas

[nareshshah139]

Hello there. There seems to be a constant feature names mismatch error when I use xgboost and pandas along with eli5.

Potential reasons:

1. eli5 does not access the feature names from within xgboost which can cause potential fallouts if the features are not in the same order as they were when they were put into xgboost or they are re-ordered once they enter eli5.

Would really appreciate help debugging and will push in a patch if fixed.

[kmike]

Hey @nareshshah139,

Could you provide an example?

[nareshshah139]

---

ValueError Traceback (most recent call last)
in ()
1 import eli5
----> 2 eli5.show_prediction(clf,valid_xs.loc[1])

C:\Program Files\Anaconda3\lib\site-packages\eli5\ipython.py in show_prediction(estimator, doc, **kwargs)
240 """
241 format_kwargs, explain_kwargs = _split_kwargs(kwargs)
--> 242 expl = explain_prediction(estimator, doc, **explain_kwargs)
243 html = format_as_html(expl, **format_kwargs)
244 return HTML(html)

C:\Program Files\Anaconda3\lib\site-packages\singledispatch.py in wrapper(*args, **kw)
208
209 def wrapper(*args, **kw):
--> 210 return dispatch(args[0].class)(*args, **kw)
211
212 registry[object] = func

C:\Program Files\Anaconda3\lib\site-packages\singledispatch.py in wrapper(*args, **kw)
208
209 def wrapper(*args, **kw):
--> 210 return dispatch(args[0].class)(*args, **kw)
211
212 registry[object] = func

C:\Program Files\Anaconda3\lib\site-packages\eli5\xgboost.py in explain_prediction_xgboost(xgb, doc, vec, top, top_targets, target_names, targets, feature_names, feature_re, feature_filter, vectorized)
166 X = X.tocsc()
167
--> 168 proba = predict_proba(xgb, X)
169 scores_weights = _prediction_feature_weights(xgb, X, feature_names)
170

C:\Program Files\Anaconda3\lib\site-packages\eli5\sklearn\utils.py in predict_proba(clf, X)
47 if is_probabilistic_classifier(clf):
48 try:
---> 49 proba, = clf.predict_proba(X)
50 return proba
51 except NotImplementedError:

C:\Program Files\Anaconda3\lib\site-packages\xgboost-0.6-py3.5.egg\xgboost\sklearn.py in predict_proba(self, data, output_margin, ntree_limit)
475 class_probs = self.booster().predict(test_dmatrix,
476 output_margin=output_margin,
--> 477 ntree_limit=ntree_limit)
478 if self.objective == "multi:softprob":
479 return class_probs

C:\Program Files\Anaconda3\lib\site-packages\xgboost-0.6-py3.5.egg\xgboost\core.py in predict(self, data, output_margin, ntree_limit, pred_leaf)
939 option_mask |= 0x02
940
--> 941 self._validate_features(data)
942
943 length = ctypes.c_ulong()

C:\Program Files\Anaconda3\lib\site-packages\xgboost-0.6-py3.5.egg\xgboost\core.py in _validate_features(self, data)
1179
1180 raise ValueError(msg.format(self.feature_names,
-> 1181 data.feature_names))
1182
1183 def get_split_value_histogram(self, feature, fmap='', bins=None, as_pandas=True):

ValueError: feature_names mismatch: ['PassengerId', 'Pclass', 'Age', 'SibSp', 'Parch', 'Fare'] ['f0', 'f1', 'f2', 'f3', 'f4', 'f5']
expected Age, PassengerId, Pclass, Parch, SibSp, Fare in input data
training data did not have the following fields: f0, f3, f2, f5, f1, f4

This an example error.

[nareshshah139]

Just rewrote the Titanic example using pandas.

[kmike]

To make it easier for us, could you please upload the rewritten Titanic tutorial somewhere, e.g. post it as a gist?

[nareshshah139]

https://github.com/nareshshah139/titanic_rebranded/blob/master/ELI5_Example.ipynb

[kmike]

Thanks @nareshshah139! I'll take a look later today.

[kmike]

I don't think it is a problem with eli5. You can get the same exception if you call clf.predict_proba(valid_xs) in your notebook.

The problem is that valid_xs has a different number of columns than train_xs. It happened because get_dummies is not using the same encoding for training and validation data - train and validation data has a different number of unique values in some columns. In scikit-learn this is usually solved with OneHotEncoder: for train data .fit_transform is called, but on validation/test data .transform method is called, so that it uses the same columns as in training.

[nareshshah139]

Updated with new IPython notebook. Please check. Still getting an identical error despite identical columns.

[nareshshah139]

Here's the link again:

https://github.com/nareshshah139/titanic_rebranded/blob/master/ELI5_Example.ipynb

Vote to reopen the issue. Apologies for the error earlier. Was just making something in a hurry.

[unnir]

Can't start eli5 with a pandas dataframe, is there an example how to use it with pandas? Still can't get the idea, + google did not really help with that question

[lopuhin]

@unnir indeed, we lack a pandas example in the docs. Here is a small example of using pandas with explain_weights in the tests:

eli5/tests/test_xgboost.py

Lines 310 to 315 in 6809dd2

	X, y, feature_names = boston_train
	df = pd.DataFrame(X, columns=feature_names)
	reg = XGBRegressor().fit(df, y)
	# it shoud pick up feature names from DataFrame columns
	res = explain_weights(reg)

and this is an example of using explain_prediction with pandas:

eli5/tests/test_xgboost.py

Lines 328 to 335 in 6809dd2

	X, y, feature_names = boston_train
	feature_names = ["%s.%s" % (name, idx)
	for idx, name in enumerate(feature_names)]
	df = pd.DataFrame(X, columns=feature_names)
	reg = XGBRegressor()
	reg.fit(df, y)
	res = explain_prediction(reg, df.iloc[0])

[unnir]

ok, thank you! I will try to make a PR with a pandas example, if I figure out how to use eli5 and pandas 🤔

[unnir]

Yeah 😋 I did it, at least for the boston dataset, thank you for your hit.

Also, wait for my PR :) * hopefully this week

[lopuhin]

@unnir yeah, we don't have a regression example either, so if you do a pandas + boston example, that would be perfect I think 👍