"X appears more than once in the final defaults list" / "Multiple values for X"

[Bonnevie]

How do we in general deal with conflicting defaults? If I have two configs

bar.yaml

name: bar
bar: 1

defaults:
   foo: bar-nested

and baz.yaml

name: baz
baz: 2

defaults:
  foo: baz-nested

The names will merge gracefully, but the defaults will cause a conflict with the error message in the title. How do we merge them (or restructure them)? I can put override foo in one of them, but that constrains how they can be composed and with what.

[Jasha10]

Hi @Bonnevie,
could you please provide a reproducible example to get the error message you're seeing?

[Bonnevie]

I can't upload a zip here right? I created a Gist here, but I can't replicate the folder structure, so I just put a diagram - I don't know if there's an obvious smarter way to share this?

Tbh, this is not giving me the exact error I mentioned in the OP. Rather, I'm getting
Multiple values for content/foo. To override a value use 'override content/foo: bar-nested'

But the core issue is the same. Let me try to describe the use-case: I have a lot of different configurable units (e.g. foo in the example), and it's a bit of a pain to write them out explicitly in every base config. So I define configuration files that instantiate several of these units simultaneously, using a nested default list (e.g. baz and bar that both instantiate a foo unit). But some of these blocks can have overlapping default lists that now clash. The hope would be that when merging nested defaults lists, they merge in much the same way as the configurations themselves, i.e. last value trumps the others.

[Jasha10]

I see.
In this case, I'd recommend using the Non-config group defaults syntax.
For example, in baz.yaml, you should replace this

defaults:
  - /content/foo: baz-nested

with this:

defaults:
  - /content/foo/baz-nested

In bar.yaml, replace this

defaults:
  - /content/foo: bar-nested

with this:

defaults:
  - /content/foo/bar-nested

Using the gist that you created, I'm now getting the following output from python run.py:

{'content': {'foo': {'foo-value': 2, 'bar-value': 'r', 'baz-value': 'z'}, 'name': 'baz', 'bar': 1, 'baz': 2}}

(I also had to modify the defaults list of base_config.yaml to make this work; I think you had a typo in that file. I changed bar to content/bar-block: bar and baz to content/baz-block: baz).

Merry Christmas 🎅🏼🎄

[Bonnevie]

@Jasha10 merry Christmas!
The remaining problem with non-config groups is that they merge the dictionary keys rather than overriding, so you can get mixed-up configurations - in the above example, you might have that the foo object (it could be a machine learning model) can be instantiated with either a bar-value or a baz-value, but not both.

I don't know if the MWE got maybe a tad too abstract... Maybe a better example of how this can have practical relevance is here https://gist.github.com/Bonnevie/5a6d051dc69a410433dedce4b578c65f, where the idea is that you have some global-level "patch" configurations, and sometimes you want to be able to patch a config group, but you cannot be sure whether the configuration you are patching is using a default (with_default_node.yaml) or not (base.yaml). So one requires the set_node.yaml patch configuration to use override, while the other requires it doesn't, so it seems you cannot use the same patch generically.

[omry]

The second one loaded should be an override:

name: baz
baz: 2

defaults:
  override foo: baz-nested

Could be good idea to indicate that if you want to change a previously defined defaults list entry you should use override in the error message.

[omry]

The first definition of the entry defines the composition order. later overrides are allowing you to change the selected value.
I am not understanding what use case you have that is not satisfied by this, but regardless this is extremely unlikely to change.

[Bonnevie]

@omry hi Omry. In retrospect the initial example was quite a bit too abstract. I tried to provide a more pragmatic example in a second gist in my reply to @Jasha10 above.
I think the issue comes up when trying to combine multiple long global-level defaults lists with overlap or applying global-level patches in scenarios where you cannot be sure if the patched element was previously added or not, like in my gist.

Thanks for your great work on Hydra, it's a great tool!

[omry]

I see.
Config composition order matters and should be explicit and clear.
In such scenarios you will want to determine it explicitly and have sub configs override to their preferred choice.
Such determination can happen in explicitly somewhere.

At the end of the day, Hydra is creating a single defaults list from all the default lists mentioned in the configs sourced from your primary one, and then composes the final config.
One of the design goals is this is to ensure that the config composition order does not accidentally change based on sub configs. the override keyword is there to differentiate between changing the choice of a defaults list entry and inserting a defaults list entry (which may override config values in mentioned in default list entries before it).

[Bonnevie]

Thanks for taking the time out to reply. Sounds like we probably have to think about different ways to go about it. :)

[omry]

Yeah.
You need to decide if you are expecting to be overriding an existing default list entry defined earlier, or if you are introducing a new default list entry. In the later case, you will likely need to introduce it in multiple places where only one is actually sourced.

Take a look at the debugging facilities mentioned here for better understanding of what is happening at runtime.