How to use ubuntu VM for a single test run?

[abitrolly]

I need to test that image build works, but docker in default fedora VM is non-functional.

$ cat test02/main.fmf
summary: Test `docker` Dockerfile locations
require: docker
test: |
  docker build . -t test02
  docker inspect test02 | grep -C3 "level=zero"

tmt run for the test above fails.

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Shared connection to 127.0.0.1 closed.

So how to try ubuntu image instead of fedora in VM?

[happz]

I need to test that image build works, but docker in default fedora VM is non-functional.

$ cat test02/main.fmf
summary: Test `docker` Dockerfile locations
require: docker
test: |
  docker build . -t test02
  docker inspect test02 | grep -C3 "level=zero"

tmt run for the test above fails.

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Shared connection to 127.0.0.1 closed.

I'd argue that "not running" would be a more fitting term, exactly as the error message suggests. Installing the package does not enable or start the corresponding services. The test should run smoothly after adding systemctl start docker in a prepare step.

So how to try ubuntu image instead of fedora in VM?

Usually, image: ubuntu should work well enough. However, there seems to be some trouble with a URL the backend generates for ubuntu, and the image URL does not exist. Using an existing one helps, and image: https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img worked for me.

[LecrisUT]

Sorry, it also needs:

discover:
  how: fmf

I just got tripped by that as well

Also I'm available on @lecris:matrix.org if you want to discuss more synchronously.

@LecrisUT I created test02/plan.fmf, but it was not recognized.

Yeah, the name of files are arbitrary. tmt uses the content to detect if it's a plan or test, specifically if it has execute it's a plan, if it has test it's a test. I know, it's not quite intuitive.

[happz]

main.fmf serves sort of like index.html - if there are other files and main.fmf, main.fmf other files inherit from main.fmf. Which is exactly what's happening here, note tmt complaining about additional properties (require, test) spotted in the plan. They are not in plan.fmf, but are inherited from main.fmf.

This also means there is no test defined, because main.fmf provides no test, it's nothing but a template for other files to inherit from. Can you rename main.fmf to test.fmf to stop this unwanted inheritance? That should expose the test as a standalone entity, and discover should notice it.

[abitrolly]

The fact that names are arbitrary, but main.fmf is not arbitrary, and that mere presence of this file changes the behavior, is very confusing.

@happz index.html analogy in not quite correct - nothing includes index.html, nothing inherits from it, all directives inside index.html are explicit in what the browser should pull. There is no hidden logic in index.html.

I was about to understand tmt model as "use filenames to detect tests and plans, then lump everything together under corresponding trees", but now I am confused. If filenames don't matter, then the model could be "analyze .fmf file content and decide what is it", but it seems more complicated, like "analyze .fmf file content, lump everything together with main.fmf and decide what is what". I would really prefer filename inference with explicit keys for overriding things, like:

plan:
  prepare:
test:
  script:

[happz]

If filenames don't matter, then the model could be "analyze .fmf file content and decide what is it", but it seems more complicated, like "analyze .fmf file content, lump everything together with main.fmf and decide what is what".

That is exactly what happens. All fmf files under the tree are loaded and squashed into a single tree, with main.fmf providing defaults for their corresponding subtrees. Then tmt "decides" what kinds of objects are the leaves of this tree: if there's execute key, it's a plan, if there's test, it's a test, and story means the node is a story.

I would really prefer filename inference with explicit keys for overriding things, like:

I'm afraid that's not supported, at least not in the form of having a dedicated key whose content would be a test, and a key whose content would be a plan, test vs plan in your example. It's based on detecting presence of keys in those objects.

happz@multivac /tmp/foo $ l
total 4
drwxr-xr-x.  3 happz happz  80 Jul  3 12:41 .
drwxrwxrwt. 37 root  root  900 Jul  3 12:40 ..
drwxr-xr-x.  2 happz happz  60 Jul  3 12:38 .fmf
-rw-r--r--.  1 happz happz  56 Jul  3 12:38 stuff.fmf
happz@multivac /tmp/foo $ cat stuff.fmf 
/plan:
  execute:
    how: tmt
/test:
  test: /bin/true
happz@multivac /tmp/foo $ tmt
Found 1 test: /stuff/test.
Found 1 plan: /stuff/plan.
Found 0 stories.
happz@multivac /tmp/foo $ tmt plan show
/stuff/plan
                 enabled true

happz@multivac /tmp/foo $ tmt test show
/stuff/test
                    test /bin/true
                    path /
               framework shell
                  manual false
                     tty false
                duration 5m
                 enabled true
                  result respect
       restart_max_count 1
     restart_with_reboot false

happz@multivac /tmp/foo $ 

There are both plans and tests in the same file, which is which is detected by the presence of given keys, see above.

Adding main.fmf tells tmt to use this new file as a source of default values:

happz@multivac /tmp/foo $ cat main.fmf 
contact: [email protected]
happz@multivac /tmp/foo $ tmt plan show
/stuff/plan
                 contact [email protected]
                 enabled true

happz@multivac /tmp/foo $ tmt test show
/stuff/test
                 contact [email protected]
                    test /bin/true
                    path /
               framework shell
                  manual false
                     tty false
                duration 5m
                 enabled true
                  result respect
       restart_max_count 1
     restart_with_reboot false

happz@multivac /tmp/foo $ 

Applies to both plan and test. A two-edged sword, both plan and test would inherit from it, which may introduce invalid keys to one of them:

happz@multivac /tmp/foo $ cat main.fmf 
contact: [email protected]
report:
  how: html
happz@multivac /tmp/foo $ tmt plan show
/stuff/plan
                 contact [email protected]
                 enabled true

happz@multivac /tmp/foo $ tmt test show
        warn: /stuff/test: - 'report' does not match any of the regexes: '^extra-'
/stuff/test
                 contact [email protected]
                    test /bin/true
                    path /
               framework shell
                  manual false
                     tty false
                duration 5m
                 enabled true
                  result respect
       restart_max_count 1
     restart_with_reboot false

happz@multivac /tmp/foo $ 

To avoid this, it's common to separate plans and tests into different subdirectories, to give these "subtrees" their own "root" main.fmf:

happz@multivac /tmp/foo $ l
total 4
drwxr-xr-x.  5 happz happz 120 Jul  3 12:45 .
drwxrwxrwt. 37 root  root  900 Jul  3 12:46 ..
drwxr-xr-x.  2 happz happz  60 Jul  3 12:38 .fmf
-rw-r--r--.  1 happz happz  21 Jul  3 12:45 main.fmf
drwxr-xr-x.  2 happz happz  80 Jul  3 12:45 plans
drwxr-xr-x.  2 happz happz  60 Jul  3 12:45 tests
happz@multivac /tmp/foo $ cat main.fmf 
contact: [email protected]
happz@multivac /tmp/foo $ cat plans/main.fmf 
report:
  how: html
happz@multivac /tmp/foo $ cat plans/stuff.fmf 
/plan:
  execute:
    how: tmt
happz@multivac /tmp/foo $ cat tests/stuff.fmf 
/test:
  test: /bin/true
happz@multivac /tmp/foo $ tmt plan show
/plans/stuff/plan
                 contact [email protected]
                 enabled true

happz@multivac /tmp/foo $ tmt test show
/tests/stuff/test
                 contact [email protected]
                    test /bin/true
                    path /tests
               framework shell
                  manual false
                     tty false
                duration 5m
                 enabled true
                  result respect
       restart_max_count 1
     restart_with_reboot false

happz@multivac /tmp/foo $ 

[LecrisUT]

The fact that names are arbitrary, but main.fmf is not arbitrary, and that mere presence of this file changes the behavior, is very confusing.

The thing is there are 2 technologies at play here:

• fmf: A merge of yaml and file structures. Its only purpose is to read *.fmf and give you a dict.
  You can navigate the dict objects created via fmf ls and fmf show.
  For more info on its features: https://fmf.readthedocs.io/en/stable/features.html
• tmt: Takes the dict and decides what's a plan/test, etc., adding the default values and more.
  This is what controls all of the executions.

I shouldn't have said arbitrary, but basically, the name of files only define the /path/to/foo. There are 3 ways to define it. Hopefully this section can be of help.