Roadmap for Upcoming Features

Feature Requests

Multi-modal model

Suggested in https://github.com/microsoft/FLAML/discussions/279. It'll be good to have a concrete use case.

Multivariate time series forecasting

Requested in https://github.com/microsoft/FLAML/issues/204. Multivariate time series forecasting is enabled since v0.7 with some limitations. There is ongoing work to address these limitations.

Improve efficiency for multi-tasking

https://github.com/microsoft/FLAML/issues/277. The current solution is to fit multiple single-output models. It will be slow when the number of tasks is large. The same issue applies to time series forecasting when there are multiple time series in the data (differentiated by categorical columns). There is ongoing research for this problem.

Fix certain hyperparameter values during search

Asked in

• https://github.com/microsoft/FLAML/discussions/271.
• https://github.com/microsoft/FLAML/issues/16.
• gitter.

Right now the solution is to use a derived estimator. We can make it easier by adding an argument in AutoML.fit(). It is a good issue for newcomers: https://github.com/microsoft/FLAML/issues/307.

How to decide value of time_budget

A recurring question is how to decide value of time_budget. For example,

• https://github.com/microsoft/FLAML/issues/155.
• https://github.com/microsoft/FLAML/discussions/311.

Our current recommendation is at https://github.com/microsoft/FLAML/wiki/Time-budget. Any improvement on it will be beneficial to lots of users. It is a good research problem too.

Support Python 3.10

A good workitem for new contributors: https://github.com/microsoft/FLAML/issues/308.

Pull the number of iterations completed for each learner

A simple feature to add. Good for new contributors: https://github.com/microsoft/FLAML/issues/58.

Decide how many labeled training examples are needed

Asked in https://github.com/microsoft/FLAML/discussions/289#discussioncomment-1690433. Related research:

• ABC: Efficient Selection of Machine Learning Configuration on Large Dataset
• Efficiently Approximating Selectivity Functions using Low Overhead Regression Models

It will be a unique feature to integrate these techniques into FLAML. Help wanted.

Prediction quality

Requested in https://github.com/microsoft/FLAML/issues/214 and https://github.com/microsoft/FLAML/issues/355. Help wanted.

A related discussion in https://github.com/microsoft/FLAML/discussions/351.

R library

Requested in https://github.com/microsoft/FLAML/issues/15. Help wanted.

Imbalance

Throw a warning and let the user know about class imbalance before training. If imbalance is detected, wrap the classifiers with BalancedBaggingClassifier etc. to overcome imbalance. Source: https://github.com/microsoft/FLAML/discussions/27. Help wanted.

Feature Selection by FLAML

https://github.com/microsoft/FLAML/issues/258. Help wanted.

Support "groups" for catboost

https://github.com/microsoft/FLAML/issues/304. Help wanted.

Visualize feature importance, SHAP/LIME explanation, optimization history

• https://github.com/microsoft/FLAML/issues/246.
• https://github.com/microsoft/FLAML/issues/193.

Though we have some partial solutions, there is room for improvement. Help wanted.

Use early_stop_rounds

https://github.com/microsoft/FLAML/issues/172. We made some investigation about the effectiveness of using early_stop_rounds for lightgbm and xgboost. The results are inconclusive. Suggestions are welcome.

Search space for CatBoost

https://github.com/microsoft/FLAML/issues/144. Help wanted.

ONNX/ONNXML export

https://github.com/microsoft/FLAML/issues/20. Help wanted.

Fair AutoML

Integrate Fair AutoML.