Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce top-level keys #1947

Merged

Conversation

Chris-Peterson444
Copy link
Contributor

Subiquity currently ignores invalid top-level keys, but this has likely been a major source of confusion of autoinstall capabilities.

In future versions, the following autoinstall config will throw an AutoinstallValidationError:

version: 2
interactive-sections:
  - identity
 literally-anything: lmao

This patch adds warnings for version 1 and will begin to throw an AutoinstallValidationError on these instances in version 2 once it has been enabled.

@Chris-Peterson444
Copy link
Contributor Author

Supersedes #1899 and requires #1945

Copy link
Collaborator

@dbungert dbungert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally the plan is good but I'd like some restructuring.


for key in bad_keys:
warning = f"Unrecognized top-level key {key!r}"
log.warning(warning)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as mentioned elsewhere the double log / ctx feels a bit tedious, please refactor a bit in a future PR.

self,
autoinstall_config: dict[str, Any],
) -> dict[str, Any]:
"""Removes all unkown keys from autoinstall config."""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment is inverted, since we're returning autoinstall with known controllers removed, we're actually returning the unknowns

server_config: dict[str, Any] = self._strip_controller_keys(autoinstall_config)
valid_keys: set[str] = set(self.base_schema["properties"].keys())
top_level_keys: set[str] = set(server_config.keys())
bad_keys: set[str] = top_level_keys - valid_keys
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And because server_config is the unknowns, that means that this calculation of bad_keys works because it already has the unknowns stored in top_level_keys and we subtract any valid_keys (which won't be there) so we get the correct answer but in a bit of an obfuscated way.

@@ -552,8 +553,93 @@ async def apply_autoinstall_config(self, context):
await controller.apply_autoinstall_config()
await controller.configured()

def _strip_controller_keys(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of want you to change how this function works, to have it return two dicts.
One dict of the known-good keys, one of the known-bad keys.
I think it would simplify the later operations, as

  • you won't have to separately calculate bad_keys (this will tell you)
  • the calculation of the good keys will be in the same spot (right now there are a few similar-looking things and the logic is spread over this function and the caller), and the calling function can just take action


"""

server_config: dict[str, Any] = self._strip_controller_keys(autoinstall_config)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

server_config is a bit generic, following the restructure of _strip_controller_keys maybe it could be good_keys / bad_keys?

self.assertEqual(self.server.autoinstall_config, stripped_ai_data)

# Not OK in Version >= 2
log_mock.reset_mock()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I kind of don't trust reset_mock - it's tough to convince myself that everything that matters has been reset - are the later tests only passing because of retained state from earlier? What do you think about splitting this test in two parts to cover your version=1 and version=2 cases? Common setup logic could be refactored.

@Chris-Peterson444 Chris-Peterson444 force-pushed the strict-top-level-keys branch 2 times, most recently from 5445369 to 991563a Compare March 21, 2024 02:46
@Chris-Peterson444
Copy link
Contributor Author

I've made updates per your suggestions @dbungert. I really liked your recommendation for the changes to the stripping function. The filtering function is really nice and made testing pretty simple. Thanks!

Copy link
Collaborator

@dbungert dbungert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with one typo fix when underlying PRs are resolved

self,
autoinstall_config: dict[str, Any],
) -> tuple[dict[str, Any], dict[str, Any]]:
"""Separates autoinstall config by known and uknown keys"""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thanks!

@Chris-Peterson444 Chris-Peterson444 force-pushed the strict-top-level-keys branch 3 times, most recently from 4150e56 to b7ff1c1 Compare March 22, 2024 21:02
@Chris-Peterson444 Chris-Peterson444 force-pushed the strict-top-level-keys branch 2 times, most recently from e6f4c70 to 8a35182 Compare March 26, 2024 00:10
Copy link
Collaborator

@dbungert dbungert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have two minor notes, please consider then it's good for merge I think.

subiquity/server/tests/test_server.py Outdated Show resolved Hide resolved
subiquity/server/tests/test_server.py Show resolved Hide resolved
Subiquity currently ignores invalid top-level keys, but this has likely
been a major source of confusion of autoinstall capabilities.

In future versions, the following autoinstall config will throw an
AutoinstallValidationError:

    version: 2
    interactive-sections:
        - identity
    literally-anything: lmao

This patch adds warnings for version 1 and will begin to throw
an AutoinstallValidationError on these instances in version 2 once
it has been enabled.
@Chris-Peterson444 Chris-Peterson444 merged commit ce0683a into canonical:main Mar 26, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants