isabella232 / beats-tester

E2E tests for the Beats

The Beats Tester

Vagrant + Ansible setup for testing the OS packages and basic e2e tests for all the Beats.

OS matrix

OS	Support
awslinux	✅
awslinux2	✅
centos6-32	✅
centos6-64	✅
centos7-64	✅
centos8-64	✅
debian10-64	✅
debian8-64	✅
debian9-64	✅
sles12-64	✅
ubuntu1204-32	✅
ubuntu1404-64	✅
ubuntu1604-64	✅
ubuntu1804-64	✅
ubuntu2004-64	✅
win12-64	✅

Execute

First, you need to bring the machines up:

make setup

Then you can use Ansible to run the tests. Because they involve lots of VMs and commands executed over SSH, these tests are slow (currently 15 minutes in total). However, while creating tests or checking something quickly, you can use different Ansible commands to execute only a subset.

Here are some execution examples:

• Test the most recent nightly builds (excluding OS X):

    RUN_SETTINGS=snapshot make run
  

• All tests, all platforms, a particular release:

    # Edit version contained in run-settings-released.yml.
    RUN_SETTINGS=released make run
  

• Only a particular Beat, Packetbeat in the example:

    export ANSIBLE_EXTRA_FLAGS="--tags packetbeat"
    make run
  

• Only a particular OS, Debian 8 amd64 in the example:

    # Instead using 'make setup' launch vagrant machines manually.
    machine=tester-debian8-64
    vagrant up $machine
    vagrant ssh-config $machine > ssh_config
    export ANSIBLE_LIMIT="$machine"
    make run
  

• Only a set of ansible groups in batches:

    # Run tests in batches for all hosts
    make batch
  
    # Run tests in batches for all Linux hosts
    GROUPS='centos debian sles' make batch
  
    # Run tests in batches for all Debian hosts
    GROUPS=debian make batch
  
    # Run tests in batches for Windows hosts
    GROUPS=windows make batch
  

• Enable Ansible debug.

    export ANSIBLE_VERBOSE="-vvv"
    make run
  

• Executing tests on your local macOS machine. In System Preferences -> Sharing enable Remote Login (SSH). Don't forget to disable Remote Login when you are done.

    export ANSIBLE_EXTRA_FLAGS="--ask-sudo-pass"
    export ANSIBLE_LIMIT="localhost"
    make run
  

• To run just the APM Server tests, set ANSIBLE_EXTRA_FLAGS="--tags apm-server" in the environment.

Cleanup

You need to tear down the VMs when you are finished.

make clean