-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: snapshot comparison on dicts with different levels #109
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -235,10 +235,10 @@ def calculate_diff_on_dicts( | |
) -> Dict[str, dict]: | ||
"""The static method to calculate a difference between two dictionaries. | ||
|
||
By default dictionaries are compared by going down all nested levels, to the point where key-value pairs are just strings | ||
or numbers. It is possible to configure which keys from these pairs should be compared (by default we compare all | ||
available keys). This is done using the `properties` parameter. It's a list of the bottom most level keys. For example, | ||
when comparing route tables snapshots are formatted like: | ||
By default dictionaries are compared by going down all nested levels. It is possible to configure which keys on each | ||
level should be compared (by default we compare all available keys). This is done using the `properties` parameter. | ||
It's a list of keys that can be compared or skipped on each level. For example, when comparing route tables snapshots are | ||
formatted like: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Maybe it's worth adding a comment about the fact that of a key does not exist in a dict it's ignored. And that this is true in both bases:
|
||
|
||
```python showLineNumbers | ||
{ | ||
|
@@ -258,8 +258,9 @@ def calculate_diff_on_dicts( | |
} | ||
``` | ||
|
||
The bottom most level keys are: | ||
The keys to process here can be: | ||
|
||
- 'default_0.0.0.0/0_ethernet1/3', | ||
- `virtual-router`, | ||
- `destination`, | ||
- `nexthop`, | ||
|
@@ -405,11 +406,11 @@ def calculate_diff_on_dicts( | |
|
||
common_keys = left_side_to_compare.keys() & right_side_to_compare.keys() | ||
if common_keys: | ||
next_level_value = left_side_to_compare[next(iter(common_keys))] | ||
at_lowest_level = True if not isinstance(next_level_value, dict) else False | ||
keys_to_check = ( | ||
ConfigParser(valid_elements=set(common_keys), requested_config=properties).prepare_config() | ||
if at_lowest_level | ||
ConfigParser(valid_elements=set(common_keys), | ||
requested_config=properties, | ||
ignore_invalid_config=True).prepare_config() | ||
if properties | ||
else common_keys | ||
) | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -159,6 +159,7 @@ def __init__( | |
self, | ||
valid_elements: Iterable, | ||
requested_config: Optional[List[Union[str, dict]]] = None, | ||
ignore_invalid_config: Optional[bool] = False | ||
): | ||
"""ConfigParser constructor. | ||
|
||
|
@@ -168,13 +169,19 @@ def __init__( | |
* if `requested_config` is `None` we immediately treat it as if `all` was passed implicitly | ||
(see [`dialect`](/panos/docs/panos-upgrade-assurance/dialect)) - it's expanded to `valid_elements` | ||
* `_requested_config_names` is introduced as `requested_config` stripped of any element configurations. Additionally, we | ||
do verification if elements of this variable match `valid_elements`. An exception is thrown if not. | ||
do verification if all elements of this variable match `valid_elements`, if they do not, an exception is thrown by default. | ||
`request_config` is checked at top level level key in case of nested dictionaries within the list. | ||
* if `ignore_invalid_config` is set to `True`, we ignore any invalid configurations passed in the `requested_config` - | ||
(no exception thrown) and we remove these invalid configurations from `_requested_config_names` and | ||
`requested_config`. | ||
|
||
# Parameters | ||
|
||
valid_elements (iterable): Valid elements against which we check the requested config. | ||
requested_config (list, optional): (defaults to `None`) A list of requested configuration items with an optional | ||
configuration. | ||
ignore_invalid_config (bool, optional): (defaults to `False`) Whether to ignore invalid configurations in | ||
`requested_config` or throw an exception. | ||
|
||
# Raises | ||
|
||
|
@@ -188,9 +195,19 @@ def __init__( | |
self._requested_config_names = set( | ||
[ConfigParser._extract_element_name(config_keyword) for config_keyword in self.requested_config] | ||
) | ||
|
||
non_existing_keys = set() | ||
for config_name in self._requested_config_names: | ||
if not self._is_element_included(element=config_name): | ||
raise exceptions.UnknownParameterException(f"Unknown configuration parameter passed: {config_name}.") | ||
non_existing_keys.add(config_name) | ||
|
||
if not ignore_invalid_config: # if invalid config is not accepted | ||
if non_existing_keys: # and non existing keys found | ||
raise exceptions.UnknownParameterException(f"Unknown configuration parameters passed: {non_existing_keys}.") | ||
else: # invalid config is accepted | ||
self._requested_config_names.difference_update(non_existing_keys) # remove non-existing keys | ||
self._remove_invalid_requested_config(invalid_keys=non_existing_keys) | ||
|
||
else: | ||
self._requested_config_names = set(valid_elements) | ||
self.requested_config = list(valid_elements) # Meaning 'all' valid tests | ||
|
@@ -248,6 +265,18 @@ def _extract_element_name(config: Union[str, dict]) -> str: | |
else: | ||
raise exceptions.WrongDataTypeException("Config definition is neither string or dict") | ||
|
||
def _remove_invalid_requested_config(self, invalid_keys: set) -> None: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm just wondering if it make sense to build a method for 3 lines of code that won't be used anywhere else besides the constructor |
||
"""Remove invalid keys from the requested configuration (`self.requested_config`). | ||
|
||
# Parameters | ||
|
||
invalid_keys (set): A set of keys to remove from `self.requested_config`. | ||
""" | ||
|
||
for config_element in self.requested_config[:]: | ||
if ConfigParser._extract_element_name(config_element) in invalid_keys: | ||
self.requested_config.remove(config_element) | ||
|
||
def _expand_all(self) -> None: | ||
"""Expand key word `'all'` to `self.valid_elements`. | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should get rid of the commented lines
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, once I add the unit tests corresponding to these I will remove them in this PR.