Automatic testing of a Fedora Cloud image with gherkin, qemu, pexpect and travis

Automatic testing - in all it’s glory - has been one of my problems for some time now.

The result for the automated testing of the current Fedora Cloud image is: image

A screencast can be found here.

tl;dr

Not to long ago I wrote about using pexpect to interact with virtual machines. This time I extend the last experiment and add gherkin (throughpython-behave) to describe the features to be tested.

image

I like gherkin, because it enables non-programmers to write test cases, to describe their needs, the features they want to so. Much has been written about it elsewhere and there are nice recommendations of how good features should be written.

A feature to describe the basic bootability of a cloud image could look like:

Feature: Basic bootability
  Cover the basic boot process. Ensure that we see a bootloader
  and a login prompt is displayed.

  Background: Some virtual machine with Fedora
    Given a default VM
      and the latest Fedora cloud image
      and that the VM is turned on

  Scenario: A bootloader is expected right after boot
       When we wait for 30 seconds at most
       Then we expect the bootloader prompt to appear

  Scenario: The kernel and systemd is started
       When we wait for 60 seconds at most
       Then we expect the kernel to be loaded
        and systemd to be running

  Scenario: A login prompt is displayed
       When we wait for 480 seconds at most
       Then we expect a login prompt

(The feature is suboptimal written.)

How each of this lines is achieved is up to the programmer / the implementation.

The implementation in this case is the same as I’ve previously written about. qemu and pexpect ironed together.

As the implementation is quite standalone (only python and qemu are required), it was just logical to run it on Travis CI.

image

The result is that it passes (after a bit of forth and back). Look at the history to see, that failures are also catched.

The .travis.yml file is surprisingly clean:

language: python

python:
  - "2.7"

before_install:
 - "sudo apt-get update"
 - "sudo apt-get install qemu curl"
 - "pip install -r requirements.txt"

script:
 - "curl -L -O http://download.fedoraproject.org/pub/fedora/linux/updates/20/Images/x86_64/Fedora-x86_64-20-20140407-sda.qcow2"
 - "behave"
 - "rm -vf *.qcow2"

All of this can be found in this repository. Feel free to test, fork and improve!

Things I like about this approach:

  • Separation of concern - Implementation is independent of specification
  • Works well for local setups and testing in the cloud
  • It’s simple

::: {#footer} [ May 23rd, 2014 11:02am ]{#timestamp} [fedora]{.tag} [cloud]{.tag} [automation]{.tag} [pexpect]{.tag} [python]{.tag} [qemu]{.tag} [virtexpect]{.tag} :::