Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I force SMAC to try the default configurations? #1140

Open
Strawberry9583 opened this issue Aug 20, 2024 · 2 comments
Open

How can I force SMAC to try the default configurations? #1140

Strawberry9583 opened this issue Aug 20, 2024 · 2 comments

Comments

@Strawberry9583
Copy link

Description

I have an unexpected case when I am using SMAC2.1.0.

My tuned version of the solver is even worse than the default configured versions, even though I set the default configuration when running SMAC. We tried to get improved performance of geode and or-tools but failed. I also checked and tried the configuration during SMAC running; the default configuration hasn't been tried at all.

Are there any methods that I can ask SMAC to try the default configuration?

Steps/Code to Reproduce

SMAC2.1.0

Expected Results

SMAC can run the default configuration I have set in the configuration space at first. Then SMAC tried other configurations.

Actual Results

SMAC never tried the default configuration in the whole optimization period.

Versions

@benjamc
Copy link
Collaborator

benjamc commented Aug 27, 2024

Could you please add a minimal working example how you exactly have setup SMAC to use the default configuration?
In general, you can pass initial_design=DefaultInitialDesign() to the facade (from smac.initial_design import DefaultInitialDesign).

@benjamc
Copy link
Collaborator

benjamc commented Sep 25, 2024

You can do something like this:

...
configurations = [your_default_configuration]
smac = HyperparameterOptimizationFacade(
        scenario,
        task.evaluate,
        overwrite=True,

        # Modify the initial design to use our custom initial design
        initial_design=HyperparameterOptimizationFacade.get_initial_design(
            scenario, 
            n_configs=0,  # Do not use the default initial design
            additional_configs=configurations  # Use the configurations previously evaluated as initial design
                                               # This only passes the configurations but not the cost!
        )
    )

(from the warmstart example )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants