Documentation Clarification Needed: Limitation on Using "from" Field for Output Variables in Test Steps

[haimengzhang]

Hello! While working with Venom, we encountered an issue related to the from field in test steps. Currently, it appears that the from field does not accept non-variable values, which can be problematic when attempting to create a nested variable within a string. For example, when using a string like This is a hello message: {{ .helloMessage }} as output, we encountered difficulties due to this limitation.

Here's an example of the problem:

name: Example test suite
vars:
   helloMessage: Hello World

testcases:
- name: testA
  steps:
  - type: exec
    script: echo 'Message is none'
    vars:
      myvariable:
        from:  'Message is {{ .helloMessage}}'
        default: "somevalue"

- name: testB
  steps:
  - type: exec
    script: echo {{.testA.myvariable}}
    assertions:
    - result.code ShouldEqual 0
    - result.systemout ShouldContainSubstring Hello

The output error is:

Example test suite
        • testA
                • exec PASS
        • testB
                • exec FAIL
                  Testcase "testB", step #0-0: Assertion "result.systemout ShouldContainSubstring Hello" failed. expected 'somevalue' to contain 'Hello' but it wasn't (example-test.yaml:0)

Suggested Improvement

Update the Venom documentation to explicitly mention this limitation in the usage of the from field with non-variable values. Providing clarity on this matter will help users understand the current constraints and find suitable workarounds.

Proposed Documentation Update

To address this issue in the documentation, you could include a note or example to inform users about the limitation of the from field not accepting non-variable values. For instance:

Note on 'from' Field Limitation with Non-Variable Values

Please note that as of the current version of Venom, the from field in test steps does not accept non-variable values when creating nested variables within a string. This limitation means that if you intend to use a non-variable value within the from field, such as in the example below:

name: Example test suite
vars:
   helloMessage: Hello World

testcases:
- name: testA
  steps:
  - type: exec
    script: echo 'Message is none'
    vars:
      myvariable:
        from:  'Message is {{ .helloMessage }}'
        default: "somevalue"

- name: testB
  steps:
  - type: exec
    script: echo {{ .testA.myvariable }}
    assertions:
    - result.code ShouldEqual 0
    - result.systemout ShouldContainSubstring Hello

[catmeme]

I ran into this trying to use from: result.code