add expect_one_of feature in jsonpath tolerance

Signed-off-by: saravanan palanisamy <[email protected]>

add debug log - jsonpath tolerance

Signed-off-by: saravanan palanisamy <[email protected]>

add new field for expect alternative values

Signed-off-by: saravanan palanisamy <[email protected]>

add debug log for expect_alt

Signed-off-by: saravanan palanisamy <[email protected]>

add test cases for expect_alt

Signed-off-by: saravanan palanisamy <[email protected]>

rename the second expect field & add changelog

Signed-off-by: saravanan palanisamy <[email protected]>

CHANGELOG.md

@@ -4,6 +4,19 @@
 
 [Unreleased]: https://github.com/chaostoolkit/chaostoolkit-lib/compare/1.15.0...HEAD
 
+### Added
+
+- New field `expect_one_of` in `jsonpath` tolerance. Sometimes the json payload
+  values will be dynamic, e.g. field `status` in response payload may provide
+  any values(either `ok` or `error` or `info`). Using `expect` field, we can
+  mention only one value as expected value but sometimes steady state can be
+  met with one or more values. e.g. you want to define two values either `ok` or
+  `info` as expected value. In these cases, you can use both `expect` and
+  `expect_one_of` to define both expected values.[#191][191]
+
+[191]: https://github.com/chaostoolkit/chaostoolkit-lib/pull/191
+
+
 ## [1.15.0][] - 2020-09-11
 
 [1.15.0]: https://github.com/chaostoolkit/chaostoolkit-lib/compare/1.14.1...1.15.0
@@ -136,7 +149,7 @@
   was interrupted from a control. With the strategies, you can now decide
   that they are always applied, never or only when the experiment deviated.
   This is a flag passed to the settings as follows:
-  
+
   ```
   runtime:
     rollbacks:
@@ -215,13 +228,13 @@
 ### Added
 
 - Optional default value for environment variable in configuration
-- Warn the user for an action process returning a non-zero exit code 
+- Warn the user for an action process returning a non-zero exit code
 - Support for process path relative to homedir ~
 - Indicate path in validation when path is not found nor executable [#159][159]
 
 ### Changed
 
-- Changed the method's one-step minimum requirement. 
+- Changed the method's one-step minimum requirement.
   An experiment with an empty method (without any activities) is now valid.
 
 [159]: https://github.com/chaostoolkit/chaostoolkit-lib/issues/159
@@ -333,7 +346,7 @@
 
 ### Changed
 
-- Fix to ensure a control's `configuration` parameter is populated when it the 
+- Fix to ensure a control's `configuration` parameter is populated when it the
   control is being `configured` [#114][114]
 - Load and apply global controls, those declared in the settings, from the
   `run_experiment` function rather than out of band [#116][116]
@@ -400,7 +413,7 @@
 #### Added
 
 - a new tolerance type called `range` to support scenarios such as:
-  
+
   value type is:
   ```
   {
@@ -765,7 +778,7 @@
 
 ### Changed
 
--   Log a message when loading the configuration 
+-   Log a message when loading the configuration
 -   Raise `InvalidExperiment` when a configuration or secret references a key
     in the environment and that key does not exist (it may not be set however)
     [#40][40]. This bails the experiment at validation time so before it runs.

chaoslib/hypothesis.py

@@ -334,8 +334,21 @@ def _(tolerance: dict, value: Any, configuration: Configuration = None,
             else:
                 result = values == expect
 
+        if "expect" in tolerance and result is False:
+            expect_one_of = tolerance.get("expect_one_of")
+            if "expect_one_of" in tolerance:
+                if not isinstance(expect_one_of, list):
+                    result = values == [expect_one_of]
+                else:
+                    result = values == expect_one_of
+
         if result is False:
-            if "expect" in tolerance:
+            if "expect" in tolerance and "expect_one_of" in tolerance:
+                logger.debug(
+                    "jsonpath found '{}' but expected '{}' or '{}'".format(
+                        str(values), str(tolerance["expect"]),
+                        str(tolerance["expect_one_of"])))
+            elif "expect" in tolerance:
                 logger.debug(
                     "jsonpath found '{}' but expected '{}'".format(
                         str(values), str(tolerance["expect"])))

tests/test_tolerance.py

@@ -139,6 +139,18 @@ def test_tolerance_jsonpath_must_match_expected_value():
         }
     ) is True
 
+    t = {
+        "type": "jsonpath",
+        "path": "$.foo[?(@.baz)].baz",
+        "expect": [["hello", "bonjour"]],
+        "expect_one_of": [["hello", "joe"]]
+    }
+    ensure_hypothesis_tolerance_is_valid(t)
+    assert within_tolerance(
+        t, value={
+            'foo': {"baz": ["hello", "joe"]}
+        }
+    ) is True
 
     t = {
         "type": "jsonpath",
@@ -152,6 +164,19 @@ def test_tolerance_jsonpath_must_match_expected_value():
         }
     ) is True
 
+    t = {
+        "type": "jsonpath",
+        "path": "$.foo[?(@.baz)].baz",
+        "expect": [[["hello"], ["bonjour"]]],
+        "expect_one_of": [[["hello"], ["joe"]]]
+    }
+    ensure_hypothesis_tolerance_is_valid(t)
+    assert within_tolerance(
+        t, value={
+            'foo': {"baz": [["hello"], ["joe"]]}
+        }
+    ) is True
+
     t = {
         "type": "jsonpath",
         "path": "$.foo[?(@.baz)].baz",
@@ -201,7 +226,7 @@ def test_tolerance_jsonpath_must_match_expected_values():
     assert within_tolerance(
         t, value={
             'foo': [{"baz": "hello"}, {"baz": "bonjour"}]
-        }, 
+        },
     ) is True