# Tests

There are two types of tests: "testinfra" tests, and "behave" tests.

- Testinfra tests are split into two groups, lets call them blackbox- and
  whitebox tests.  The blackbox tests run on your laptop and test the OAS
  cluster from the outside. For example, the certificate check will check if the
  OAS will return valid certificates for the provided services.  The whitebox
  tests run on the OAS host and check i.e. if docker is installed in the right
  version etc.
- Behave tests run in a browser and test if all the interfaces are up and
  running and correctly connected to each other.

## Run *testinfra* tests

Test host configured in `../clusters/CLUSTERNAME/inventory.yml`

    export INVENTORY=../clusters/CLUSTERNAME/inventory.yml
    py.test -v --ansible-inventory=${INVENTORY} --hosts='ansible://*'

Specify host manually:

    py.test -v --hosts='ssh://root@example.openappstack.net'

Run only tests tagged with `prometheus`:

    py.test -v --ansible-inventory=${INVENTORY} --hosts='ansible://*' -m prometheus

### Cert tests

Run cert test manually using the ansible inventory file:

    ADDRESS='example.openappstack.net' py.test -v -m 'certs' \
      --connection=ansible \
      --ansible-inventory=${INVENTORY} \
      --hosts='ansible://*'

Run cert test manually against a different cluster, not configured in any
ansible inventory file, either by using pytest:

    ADDRESS='example.openappstack.net' py.test -v -m 'certs'

or directly (allows better debugging since pytest won't eat stdout):

    ADDRESS='example.openappstack.net' pytest/test_certs.py


## Run *behave* tests

The behave tests are controlled by the `openappstack` CLI command suite. To run
the behave tests, run the following command in this repository:

    python -m openappstack CLUSTERNAME test

In the future, this command will run all tests, but now only *behave* is
implemented. To learn more about the `test` subcommand, run:

    python -m openappstack CLUSTERNAME test --help

## Known Issues

- Default ssh backend for testinfra tests is `paramiko`, which doesn't work oout
  of the box. It fails to connect to the host because the `ed25519` hostkey was
  not verified. Therefore we need to force plain ssh:// with either
  `connection=ssh` or `--hosts=ssh://…`

# Running CI with local gitlab-runner docker executor

Export the following environment variables like this:

    export CI_REGISTRY_IMAGE='open.greenhost.net:4567/openappstack/openappstack'
    export SSH_PRIVATE_KEY="$(cat ~/.ssh/id_ed25519_oas_ci)"
    export COSMOS_API_TOKEN='…'

then:

    gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap