Runnable posargs/kwargs

[jpavlav]

@clebergnu Hey there! I was still plugging away at my last question and have come up with an answer for my use-case, but ran into something strange. I had been attempting something similar to the example you provided:

tests = []
for collection in collections:
   tests.append(Runnable(kind='exec-test', uri='run_kselftest.sh', args=('-c', collection))

suite = TestSuite("kselftests", tests=tests)

The executable I was pointing to was a simple bash script meant to verify that the arguments I provided to the Runnable were in fact being passed to it:

#!/bin/bash

while getopts "u:" opt; do
    case $opt in
        u)
            echo "${OPTARG}"
            ;;
    esac
done

After a whole bunch of attempted debugging, I finally took a look at the tests for the Runnable class, and subsequently the Runnable class itself and found my answer:

Runnable init method

Funnily enough after inspecting the tests in the suite I was creating, the answer had been in front of my face the whole time:

In [11]: tests = [
    ...:     Runnable(
    ...:         kind='exec-test',
    ...:         uri=f'{Path.cwd()}/test.sh',
    ...:         args=('-u', collection)
    ...:     )
    ...:     for collection
    ...:     in collections
    ...: ]
    ...:
    ...: suite2 = TestSuite('funtests', tests=tests, config=config2)

In [12]: suite2.tests
Out[12]:
[<Runnable kind="exec-test" uri="/Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh" config="{}" args="()" kwargs="{'args': ('-u', 'one')}" tags="None" requirements="None">,
 <Runnable kind="exec-test" uri="/Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh" config="{}" args="()" kwargs="{'args': ('-u', 'two')}" tags="None" requirements="None">]

Because Runnable accepts **kwargs and since the args are being passed as a keyword argument, they are being expanded. And it looks like with an exec-test kind, runnable.kwargs are used as environment variables for the target executable. In my case, this resulted in the tests being skipped, which is still interesting, and ultimately, unexpected behavior:

In [13]: with Job(config2, test_suites=[suite2]) as j:
    ...:     j.run()
    ...:
JOB ID     : 00bddd7e9b07bba76f8f09abdca5d1ef0559f642
JOB LOG    : /Users/jpalmer/avocado/job-results/job-2021-08-12T10.12-00bddd7/job.log
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 1.86 s

I say it's unexpected, because when the same list of Runnables are created without any kwargs they execute as expected:

In [14]: tests = [
    ...:     Runnable(
    ...:         kind='exec-test',
    ...:         uri=f'{Path.cwd()}/test.sh'
    ...:     )
    ...:     for collection
    ...:     in collections
    ...: ]
    ...:
    ...: suite2 = TestSuite('funtests', tests=tests, config=config2)

In [15]: with Job(config2, test_suites=[suite2]) as j:
    ...:     j.run()
    ...:
JOB ID     : 7f8e30effa34dd4aa559e8814e9de3edf4cbfdff
JOB LOG    : /Users/jpalmer/avocado/job-results/job-2021-08-12T10.13-7f8e30e/job.log
 (funtests-1/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: STARTED
 (funtests-2/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: STARTED
 (funtests-1/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: PASS (0.02 s)
 (funtests-2/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: PASS (0.02 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.80 s

Ultimately the solution for my use-case was to provide the arguments I wanted passed to the executable as positional arguments:

tests = [
    Runnable(
        'exec-test',
        f'{Path.cwd()}/test.sh'
        '-u',
        collection,
    )
    for collection
    in collections
]

☝️ This executes my test.sh script with -u collection. The resulting stdout file in the job results directory has the collection in it. Having said that, after continuing to poke around I found that if I modify the get_command_args method to pop kwargs['args'] and update self.args with that value, it works as expected:

diff --git a/avocado/core/nrunner.py b/avocado/core/nrunner.py
index a7db05dc..03be5c1a 100755
--- a/avocado/core/nrunner.py
+++ b/avocado/core/nrunner.py
@@ -141,6 +141,9 @@ class Runnable:
             args.append('-c')
             args.append(json.dumps(self.config))

+        if self.kwargs.get('args'):
+            self.args = self.args + self.kwargs.pop('args')
+
         for arg in self.args:
             args.append('-a')
             if arg.startswith('-'):

So TLDR:

• ☝️ The above change handles the case that the user creates a Runnable with an args keyword argument and ensures that those arguments are included in the subsequently run avocado-runner command. I can submit a PR for this if folks think it should be the solution to the problem I've encountered.
• There is a silent bug that causes any Runnable with keyword argument values that are not strings to be SKIPPED. Example (keyword=tuple):

In [74]: from pathlib import Path
    ...:
    ...: from avocado.core.job import Job
    ...: from avocado.core.suite import TestSuite
    ...: from avocado.core.nrunner import Runnable
    ...: from avocado.utils.network.ports import find_free_port
    ...: collections = ['one', 'two']
    ...: config2 = {'run.test_runner': 'nrunner'}
    ...:
    ...: tests = [
    ...:     Runnable(
    ...:         'exec-test',
    ...:         f'{Path.cwd()}/test.sh',
    ...:         args=('-u', collection)
    ...:     )
    ...:     for collection
    ...:     in collections
    ...: ]
    ...:
    ...: suite2 = TestSuite('funtests', tests=tests, config=config2)

In [75]: with Job(config2, test_suites=[suite2]) as j:
    ...:     j.run()
    ...:
JOB ID     : b6ae79b0cc43e7bd57257594680947edf6f0b1c5
JOB LOG    : /Users/jpalmer/avocado/job-results/job-2021-08-12T13.47-b6ae79b/job.log
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 2 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 0.96 s

Working (keyword=str):

   ...: config2 = {'run.test_runner': 'nrunner'}
   ...:
   ...: tests = [
   ...:     Runnable(
   ...:         'exec-test',
   ...:         f'{Path.cwd()}/test.sh',
   ...:         keyword_arg='something'
   ...:     )
   ...:     for collection
   ...:     in collections
   ...: ]
   ...:
   ...: suite2 = TestSuite('funtests', tests=tests, config=config2)
<ipython-input-1-9db036116e14>:6: DeprecationWarning: Network as module will be deprecated, please use utils.network.ports instead.
  from avocado.utils.network.ports import find_free_port

In [2]: with Job(config2, test_suites=[suite2]) as j:
   ...:     j.run()
   ...:
JOB ID     : d5af7d72f4a4697c445ec46e68b496f04038daf0
JOB LOG    : /Users/jpalmer/avocado/job-results/job-2021-08-12T13.51-d5af7d7/job.log
 (funtests-1/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: STARTED
 (funtests-2/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: STARTED
 (funtests-2/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: PASS (0.02 s)
 (funtests-1/2) /Users/jpalmer/Repositories/github.com/Personal/avocado/test.sh: PASS (0.02 s)
RESULTS    : PASS 2 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 1.12 s```

[willianrampazzo]

Reproducible with one of our examples:

$ examples/jobs/passjob_custom.py 
JOB ID     : 4ca719f00ffeac1e63f18b364d3812873f2025af
JOB LOG    : /home/wrampazz/avocado/job-results/job-2021-08-12T16.57-4ca719f/job.log
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 1 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /home/wrampazz/avocado/job-results/job-2021-08-12T16.57-4ca719f/results.html
JOB TIME   : 1.66 s

[willianrampazzo]

I tried the "Create issue from discussion" on the right side, but it is not working for me.

[willianrampazzo]

FYI, the example was working on release 89.0 and is broken on 90.0.

[willianrampazzo]

Bisecting showed the following as the offending commit:

ab381d0 is the first bad commit
commit ab381d0
Author: Jan Richter [email protected]
Date: Mon Jun 28 14:03:13 2021 +0200

Task.status_services, from config

After the #4688 the `Task` has access to the 'nrunner.status_server_uri'
config variable, and it can be used for creating `Task.staus_services`.

Reference: #4352
Signed-off-by: Jan Richter <[email protected]>

avocado/core/nrunner.py | 3 +++
avocado/plugins/runner_nrunner.py | 21 ++++++++-------------
2 files changed, 11 insertions(+), 13 deletions(-)

[jpavlav]

I went ahead and just hit that create issue button on the discussion:

#4846