Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BlendSearch in UDF mode #1283

Open
khosravym opened this issue Feb 25, 2024 · 6 comments
Open

BlendSearch in UDF mode #1283

khosravym opened this issue Feb 25, 2024 · 6 comments

Comments

@khosravym
Copy link

When I select the BlendSearch search algorithm, I consistently encounter the following warning, whereas it does not appear when using CFO:

You passed a space parameter to OptunaSearch that contained unresolved search space definitions. OptunaSearch should however be instantiated with fully configured search spaces only. To use Ray Tune's automatic search space conversion, pass the space definition as part of the param_space argument to tune.Tuner() instead.

@Programmer-RD-AI
Copy link
Contributor

Union[ Dict[str, "OptunaDistribution"], List[Tuple], Callable[["OptunaTrial"], Optional[Dict[str, Any]]], ]

The above is the space parameter structure that is expected could you check with what you pass to BlendSearch?
and if possible send it...?

@Nevermetyou65
Copy link

I got the same warning. I used the code in this example
https://github.com/microsoft/FLAML/blob/main/test/tune_example.py

@Programmer-RD-AI
Copy link
Contributor

I belive that Passing the space definition as part of the param_space argument to tune.Tuner() when using BlendSearch with OptunaSearch in UDF mode should resolve this issue...

@Nevermetyou65
Copy link

Sorry, But what argument are you referring to? I can not find anything about tune.Tuner() in the doc.
Is it in this doc?
https://microsoft.github.io/FLAML/docs/reference/tune/tune

Do you have any code example?
Thanks

@Programmer-RD-AI
Copy link
Contributor

Programmer-RD-AI commented Jun 23, 2024

Hi,
I was thinking of something similar to the following code segment:

from flaml.tune import tune

# Define the parameter space for tuning
param_space = {
    "n_estimators": tune.randint(lower=4, upper=1024),
    "num_leaves": tune.randint(lower=4, upper=1024),
    "min_child_samples": tune.randint(lower=2, upper=100),
}

# Define the settings for the AutoML run
settings = {
    ...,
    "config": param_space,
}

# Run the AutoML search
automl.fit(X_train=X_train, y_train=y_train, **settings)```

@Nevermetyou65
Copy link

Nevermetyou65 commented Jun 23, 2024

Hmm...

I got the warning from this example: https://github.com/microsoft/FLAML/blob/main/test/tune_example.py
It does not use automl.fit but tune.run() for a user-defined function. But looking at your code, I see it's the same thing.
The dictionary setting should work the same as this line.

analysis = tune.run(
        train_lgbm,
        metric="mse",
        mode="min",
        config=config_search_space,
        low_cost_partial_config=low_cost_partial_config,
        points_to_evaluate=points_to_evaluate,
        time_budget_s=3,
        num_samples=-1,
    )

And that is where I got warning. Set search_alg = "CFO" as make waning gone though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants