Fork us on GitHub Follow us on Google+ Follow us on Facebook Follow us on Twitter

Changes between Version 3 and Version 4 of TestInVM


Ignore:
Timestamp:
2018-11-17T21:33:25Z (4 weeks ago)
Author:
Vojtech Horky
Comment:

Scenario files

Legend:

Unmodified
Added
Removed
Modified
  • TestInVM

    v3 v4  
    3535 `--debug`:: Prints (a lot of) debugging messages.
    3636
     37
     38== Scenario files
     39
     40The scenario file is stored in YAML format and has rather simple structure that is best described on example. We will again use the example from above where the scenario compiles a simple C program.
     41
     42{{{#!yaml
     43meta:
     44  name: "Hello from PCC"
     45  harbours:
     46    - pcc
     47    - binutils
     48
     49tasks:
     50  - boot
     51  - command:
     52      args: "cp /src/c/demos/hello/hello.c /tmp/hello.c"
     53  - command:
     54      args: "cd /tmp"
     55  - command:
     56      args: "pcc hello.c"
     57  - command:
     58      args: "./a.out"
     59      assert: "hello!"
     60}}}
     61
     62The part `meta` has one mandatory item: `name` should briefly describe the scenario. This text is also used in CI output. `harbours` key is used solely by CI script and lists packages that has to be part of the image. If you run `test-in-vm.py` manually, it is your responsibility to prepare the right image!
     63
     64The actual testing scenario is a list of tasks that are executed one after another. And unless stated otherwise, failure in one task aborts the whole run. The first task is always `boot` that simply waits for the machine to boot into userspace/GUI. The only other available task (so far) is `command` that executes single command in the terminal. `args` is the actual command and without any other arguments, the script only checks that the command has not ended with a failure (i.e. exit code 0).
     65
     66To check that a specific string appears in the output of the command, one can use the `assert` key. If this string is not found in the output and the command terminates (i.e. the prompt reappears), the scenario is aborted. To check for a text that shall not appear (i.e. its appearance is a reason for abortion), use `negassert`. If we expect that the program would abort (i.e. we intentionally dereference NULL pointer etc.), we can add `ignoreabort: True` to skip checking the program exit code.
     67
     68So far, the biggest scenario is `scenarios/base/fs.yml` that checks that it is possible to create FAT image inside HelenOS, mount it and write data to it. This scenario also uses `name` to better describe why we launch each command in the sequence. Use it as a good example for your own experimenting.