Automated testing of HelenOS in a virtual machine

Part of HelenOS continuous testing are also tests that are executed inside HelenOS. That is, we launch a VM (QEMU or MSIM at the moment) and check that command executed inside HelenOS gives the expected output. This page contains a brief overview of how to write such testing scenarios and how to run them.

Prerequisites

For this to work, you need to have Python 3, QEMU, ImageMagick (i.e. convert command) and SED. For testing on mips32, you will also need MSIM, Xvfb (or Xephyr), xterm and xdotool.

test-in-vm.py

The testing script is in the root of the ​CI repository. To try it, prepare a HelenOS image (or grab one from ​ci.helenos.org) for one of the supported architectures. Here, we will assume you have downloaded ​HelenOS with binutils and PCC for amd64 into helenos-amd64-with-binutils-pcc.iso into clone of CI repository.

For this image, we choose to run a scenario where we compile and run a Hello, World! program written in C. Scenarios are located under scenarios folder in CI repository, their format will be described later on.

To actually run the scenario, we launch the test-in-vm.py script and specify path to the boot image, architecture and path to the scenario. Advanced options will be described later.

./test-in-vm.py \
    --arch amd64 \
    --image helenos-amd64-with-binutils-pcc.iso \
    --scenario scenarios/coast/pcc/hello.yml

When you run this script, you will see that it starts QEMU with HelenOS and waits until the terminal appears. Then it automatically types commands that compiles hello.c and launches the compiled a.out binary. QEMU is then terminated and if all went okay, we will see a message about successful pass of the scenario (or error message explaining the reason for failure).