README.md

Tests

There are two types of tests: "testinfra" tests, and "behave" tests.

• Testinfra tests are split into two groups, lets call them blackbox- and whitebox tests. The blackbox tests run on your laptop and test the OAS cluster from the outside. For example, the certificate check will check if the OAS will return valid certificates for the provided services. The whitebox tests run on the OAS host and check i.e. if docker is installed in the right version etc.
• Behave tests run in a browser and test if all the interfaces are up and running and correctly connected to each other.

Run testinfra tests

Test host configured in ../clusters/CLUSTERNAME/inventory.yml

export INVENTORY=../clusters/CLUSTERNAME/inventory.yml
py.test -v --ansible-inventory=${INVENTORY} --hosts='ansible://*'

Specify host manually:

py.test -v --hosts='ssh://root@example.openappstack.net'

Run only tests tagged with prometheus:

py.test -v --ansible-inventory=${INVENTORY} --hosts='ansible://*' -m prometheus

Cert tests

Run cert test manually using the ansible inventory file:

ADDRESS='example.openappstack.net' py.test -v -m 'certs' \
  --connection=ansible \
  --ansible-inventory=${INVENTORY} \
  --hosts='ansible://*'

Run cert test manually against a different cluster, not configured in any ansible inventory file, either by using pytest:

ADDRESS='example.openappstack.net' py.test -v -m 'certs'

or directly (allows better debugging since pytest won't eat stdout):

ADDRESS='example.openappstack.net' pytest/test_certs.py

Run behave tests

The behave tests are controlled by the openappstack CLI command suite. To run the behave tests, run the following command in this repository:

python -m openappstack CLUSTERNAME test 

In the future, this command will run all tests, but now only behave is implemented. To learn more about the test subcommand, run:

python -m openappstack CLUSTERNAME test --help

Known Issues

• Default ssh backend for testinfra tests is paramiko, which doesn't work oout of the box. It fails to connect to the host because the ed25519 hostkey was not verified. Therefore we need to force plain ssh:// with either connection=ssh or --hosts=ssh://…

Running bootstrap CI with local gitlab-runner docker executor

Export the following environment variables like this:

export CI_REGISTRY_IMAGE='open.greenhost.net:4567/openappstack/openappstack'
export SSH_PRIVATE_KEY="$(cat ~/.ssh/id_ed25519_oas_ci)"
export COSMOS_API_TOKEN='…'

then:

gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap