Interpolation issues when using the Compose API

[dagap]

This has been mentioned before but I cannot find a solution that is working in my case. I am creating a hydra config object at the top level as:

with initialize(version_base="1.2", config_path=config_dir, job_name=job_name):
    self.cfg = compose(
                config_name=config_name,
                return_hydra_config=True,
            )

Now, I am getting the following errors downstream:

omegaconf.errors.InterpolationToMissingValueError: MissingMandatoryValue while resolving interpolation: Missing mandatory value: hydra.job.num
    full_key: hydra.sweep.subdir
    reference_type=SweepDir
    object_type=SweepDir

A minimum reproducible example:

# configs/dev.yaml  i.e. top level config
hydra:
  run:
    dir: "logs/"
  sweep:
    dir: "sweeps/"
    subdir: ${hydra.job.num}

The driver code:

from hydra import compose, initialize
from omegaconf import OmegaConf


def do_main():
    with initialize(version_base="1.2", config_path="../configs", job_name="dev"):
        cfg = compose(
            config_name="dev",
            return_hydra_config=True,
        )

        resolved = OmegaConf.to_container(cfg, resolve=True)  # This line causes the error


if __name__ == "__main__":
    do_main()

[Jasha10]

It's hard to tell exactly what's going on without a full reproducible example.
A few things to check:

• Are you sure that your hydra.yaml file is being included the composed config?
• Are you sure that the sweep.subdir setting from your hydra.yaml file is being merged into the correct place in your output config? You need a setting for hydra.sweep.subdir, specifically.

[dagap]

@Jasha10 I edited the original post to create a reproducible example. I get the same error when moving the hydra settings to top level config.

[Jasha10]

The underlying issue here is that the hydra.sweep.subdir key in your returned cfg object is an interpolation to a missing value (i.e. to hydra.job.num). Calling OmegaConf.resolve attempts to dereference that missing value, resulting in the MissingMandatoryValue error.

Are you trying to run a multirun sweep? I don't think that hydra.job.num will be set unless you're running a sweep. The compose API does not support multirun mode, so I don't think you can get around hydra.job.num being missing if you're using compose.

To prevent the error, a few options include:

• pass return_hydra_configs=False to compose (instead of return_hydra_configs=True)
• use @hydra.main+--multirun, which should result in setting hydra.job.num to something non-missing
• Use something other than subdir: ${hydra.job.num} in your config
• Remove the hydra.sweep.subdir setting from your config alltogether

[dagap]

@Jasha10 Thank you for the prompt help and reply. So, of course, I tried removing the subdir all together and I still get:

omegaconf.errors.InterpolationToMissingValueError: MissingMandatoryValue while resolving interpolation: Missing mandatory value: hydra.job.num
    full_key: hydra.sweep.subdir
    reference_type=SweepDir
    object_type=SweepDir

So, somehow this value is getting passed down to the configuration engine.

I am not trying to use multirun atm but this was part of pytorch lightning template project that I am using. However, I tried removing it and it did not help.

not returning the hydra object, breaks other things in the project with HydraConfig not found errors as I am using the output_dir: ${hydra:runtime.output_dir} as another config. Maybe this is also something I would need to handle myself.

[Jasha10]

Perhaps try passing the keyword argument throw_on_missing=False to your OmegaConf.resolve call?

[dagap]

This is False by default. I wonder if this is a bug and not getting handled properly.

I managed to solve it by basically overriding things explicitly as you had suggested. So I did something like this when I am using the compose API

paths = self.cfg.get("paths")
if paths:
    self.cfg["paths"]["work_dir"] = cwd
    self.cfg["paths"]["output_dir"] = str(output_dir)

I just manage this by hand and set return_hydra_config=False and for the moment it seems to do the trick and I have both the hydra.mainstyle and config API style working.

Many thanks for your help on this.