Skip to content

Commit

Permalink
doc: updated docs to include explanation of json matching
Browse files Browse the repository at this point in the history
Signed-off-by: lbrady <[email protected]>
  • Loading branch information
liambrady committed Aug 3, 2023
1 parent c072095 commit f8407da
Showing 1 changed file with 83 additions and 0 deletions.
83 changes: 83 additions & 0 deletions doc/source/mutest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -119,5 +119,88 @@ used to check that the state of the interface actually changes.
step("r1", 'vtysh -c "conf t\n interface eth0\n shut"')
match_step("r1", 'vtysh -c "show interface eth0", "DOWN", "Check for interface DOWN")
An additional json variant of various functions are also available, namely
:py:func:`match_step_json`, :py:func:`wait_step_json`, and :py:func:`step_json`.
These functions act similarly to their base variants, however, instead of
performing a regular expression search, they will pass if the json output of
``cmd`` matches the provided expected json (``match``). By default, two json
successfully match if the following rules hold true:
1. All data within any object present in ``match`` must also exist and be of
equal value within a similarly located object present in the json output of
``cmd``. For example, if:
.. code-block:: python
json1 = '{"foo":"foo"}'
json2 = '{"foo":"foo", "bar":"bar"}'
# Then, the following results are observed:
_, ret = match_step_json("r1", f"echo '{json2}'", json1) # Successfully matches!
print(ret) # Prints match: `{'foo': 'foo', 'bar': 'bar'}`
_, ret = match_step_json("r1", f"echo '{json1}'", json2) # Fails to match...
print(ret) # Prints diff: `{'dictionary_item_removed': ["root['bar']"]}`
2. All objects within any array present in ``match`` must also exist and contain
equivalent data (as per rule #1) within a similarly located array present in
the json output of ``cmd``. Array order is desregarded. For example, if:
.. code-block:: python
json1 = '[{"foo":"foo"}]'
json2 = '[{"foo":"foo"}, {"bar":"bar"}]'
json3 = '[{"bar":"bar"}, {"foo":"foo"}]'
# Then, the following results are observed:
_, ret = match_step_json("r1", f"echo '{json2}'", json1) # Successfully matches!
print(ret) # Prints match: `[{'foo': 'foo'}, {'bar': 'bar'}]`
_, ret = match_step_json("r1", f"echo '{json1}'", json2) # Fails to match...
print(ret) # Prints diff: `{'iterable_item_removed': {'root[1]': {'bar': 'bar'}}}`
_, ret = match_step_json("r1", f"echo '{json2}'", json3) # Successfully matches!
print(ret) # Prints match: `[{'foo': 'foo'}, {'bar': 'bar'}]`
3. All other data within an array present in either ``match`` or the json
output of ``cmd`` must also exist in the other json and be of equal value.
Array order is disregarded. For example, if:
.. code-block:: python
json1 = '["foo"]'
json2 = '["foo", "bar"]'
json3 = '["bar", "foo"]'
# Then, the following results are observed:
_, ret = match_step_json("r1", f"echo '{json2}'", json1) # Fails to match...
print(ret) # Prints diff: `{'iterable_item_added': {'root[1]': 'bar'}}`
_, ret = match_step_json("r1", f"echo '{json1}'", json2) # Fails to match...
print(ret) # Prints diff: `{'iterable_item_removed': {'root[1]': 'bar'}}`
_, ret = match_step_json("r1", f"echo '{json2}'", json3) # Successfully matches!
print(ret) # Prints match: `['foo', 'bar']`
These rules apply no matter what ``level`` of the array (``list``) or object
(``dict``). For example, if:
.. code-block:: python
json1 = '{"level1": ["level2", {"level3": ["level4"]}]}'
json2 = '{"level1": ["level2", {"level3": ["level4"], "l3": "l4"}]}'
json3 = '{"level1": ["level2", {"level3": ["level4", {"level5": "l6"}]}]}'
json4 = '{"level1": ["level2", {"level3": ["level4", "l4"]}]}'
# Then, the following results are observed:
_, ret = match_step_json("r1", f"echo '{json2}'", json1) # Successfully matches (rule #1)!
print(ret) # Match: `{'level1': ['level2', {'level3': ['level4'], 'l3': 'l4'}]}`
_, ret = match_step_json("r1", f"echo '{json3}'", json1) # Successfully matches (rule #2)!
print(ret) # Match: `{'level1': ['level2', {'level3': ['level4', {'level5': 'l6'}]}]}`
_, ret = match_step_json("r1", f"echo '{json4}'", json1) # Fails to match (rule #3)...
print(ret) # Diff: `{'iterable_item_added': {"root['level1'][1]['level3'][1]": 'l4'}}`
This behavior can be overrided to perform an exact json match if ``exact_match``
is set to ``True``.
To see all the available functions and their specifications see
:ref:`mutest-api`.

0 comments on commit f8407da

Please sign in to comment.