Skip to content
Snippets Groups Projects
Verified Commit d2355890 authored by Varac's avatar Varac
Browse files

Include command line tests in documentation

parent 0c616b04
No related branches found
No related tags found
No related merge requests found
......@@ -13,6 +13,11 @@ descibe how to give feedback via our issue tracker at the [end of these instruct
First we'd like you to setup an OpenAppStack cluster by yourself, following the
[installation tutorial](https://docs.openappstack.net/en/latest/installation_instructions.html).
## Command line tests
Please run the [command line tests](troubleshooting.md) which checks the overall
functionality of your cluster and include the output in your feedback.
## Nextcloud
### Logging into Nextcloud
......
......@@ -2,6 +2,90 @@
Note: `cluster$` indicates that the commands should be run as root on your OAS cluster.
## Run the cli tests
To get an overall status of your cluster you can run the tests from the
command line.
There are two types of tests: [testinfra](https://testinfra.readthedocs.io/en/latest/)
tests, and [behave](https://behave.readthedocs.io/en/latest/) tests.
### Testinfra tests
Testinfra tests are split into two groups, lets call them `blackbox` and
`whitebox` tests. The blackbox tests run on your provisioning machine and test
the OAS cluster from the outside. For example, the certificate check will check
if the OAS will return valid certificates for the provided services.
The whitebox tests run on the OAS host and check i.e. if docker is installed
in the right version etc.
To run the test against your cluster, first export the `CLUSTER_DIR` environment
variabel with the location of your cluster config directory:
export CLUSTER_DIR="../clusters/CLUSTERNAME"
Run all tests:
py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
#### Advance usage
Specify host manually:
py.test -s --hosts='ssh://root@example.openappstack.net'
Run only tests tagged with `prometheus`:
py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m prometheus
Run cert test manually using the ansible inventory file:
py.test -s --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m certs
Run cert test manually against a different cluster, not configured in any
ansible inventory file, either by using pytest:
FQDN='example.openappstack.net' py.test -sv -m 'certs'
or directly:
FQDN='example.openappstack.net' pytest/test_certs.py
#### Known Issues
- Default ssh backend for testinfra tests is `paramiko`, which doesn't work oout
of the box. It fails to connect to the host because the `ed25519` hostkey was
not verified. Therefore we need to force plain ssh:// with either
`connection=ssh` or `--hosts=ssh://…`
#### Running tests with local gitlab-runner docker executor
Export the following environment variables like this:
export CI_REGISTRY_IMAGE='open.greenhost.net:4567/openappstack/openappstack'
export SSH_PRIVATE_KEY="$(cat ~/.ssh/id_ed25519_oas_ci)"
export COSMOS_API_TOKEN='…'
then:
gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap
## Behave tests
Behave tests run in a headless browser and test if all the interfaces are up
and running and correctly connected to each other. They are integrated in the
`openappstack` CLI command suite.
To run the behave tests, run the following command in this repository:
python -m openappstack CLUSTERNAME test
In the future, this command will run all tests, but now only *behave* is
implemented. To learn more about the `test` subcommand, run:
python -m openappstack CLUSTERNAME test --help
## Upgrading
If you encounter problems when you upgrade your cluster, please make sure first
......
# Tests
There are two types of tests: "testinfra" tests, and "behave" tests.
- Testinfra tests are split into two groups, lets call them blackbox- and
whitebox tests. The blackbox tests run on your laptop and test the OAS
cluster from the outside. For example, the certificate check will check if the
OAS will return valid certificates for the provided services. The whitebox
tests run on the OAS host and check i.e. if docker is installed in the right
version etc.
- Behave tests run in a browser and test if all the interfaces are up and
running and correctly connected to each other.
## Run *testinfra* tests
Export `CLUSTER_DIR` env var with the location of your cluster config directory:
export CLUSTER_DIR="../clusters/CLUSTERNAME"
Run all tests:
py.test -sv --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*'
Specify host manually:
py.test -sv --hosts='ssh://root@example.openappstack.net'
Run only tests tagged with `prometheus`:
py.test -sv --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m prometheus
### Cert tests
Run cert test manually using the ansible inventory file:
py.test -sv --ansible-inventory=${CLUSTER_DIR}/inventory.yml --hosts='ansible://*' -m certs
Run cert test manually against a different cluster, not configured in any
ansible inventory file, either by using pytest:
FQDN='example.openappstack.net' py.test -sv -m 'certs'
or directly:
FQDN='example.openappstack.net' pytest/test_certs.py
## Run *behave* tests
The behave tests are controlled by the `openappstack` CLI command suite. To run
the behave tests, run the following command in this repository:
python -m openappstack CLUSTERNAME test
In the future, this command will run all tests, but now only *behave* is
implemented. To learn more about the `test` subcommand, run:
python -m openappstack CLUSTERNAME test --help
## Known Issues
- Default ssh backend for testinfra tests is `paramiko`, which doesn't work oout
of the box. It fails to connect to the host because the `ed25519` hostkey was
not verified. Therefore we need to force plain ssh:// with either
`connection=ssh` or `--hosts=ssh://…`
# Running CI with local gitlab-runner docker executor
Export the following environment variables like this:
export CI_REGISTRY_IMAGE='open.greenhost.net:4567/openappstack/openappstack'
export SSH_PRIVATE_KEY="$(cat ~/.ssh/id_ed25519_oas_ci)"
export COSMOS_API_TOKEN='…'
then:
gitlab-runner exec docker --env CI_REGISTRY_IMAGE="$CI_REGISTRY_IMAGE" --env SSH_PRIVATE_KEY="$SSH_PRIVATE_KEY" --env COSMOS_API_TOKEN="$COSMOS_API_TOKEN" bootstrap
Please refer to `../docs/troubleshooting.md` for running the tests.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment