CI: Migrate to kubernetes executor
https://docs.gitlab.com/runner/executors/kubernetes.html
For the context see https://open.greenhost.net/openappstack/single-sign-on/-/merge_requests/39#note_27578:
The other thing that makes me extremely frustrated is the CI. I will disable all functional and end to end test in the pipeline for now. They work (I ran them locally) but the gitlab docker-compose integration is still so bad that it is just impossible to run the tests in CI using the gitlab service feature.
The main issue is that we need to have control over when the containers are started (there is a kratos migration container now) and we also need to be able to mount a volume into the containers.
As far as I know the gitlab service feature cannot handle this so we need to migrate to another virtualization method. Docker in docker with a docker compose image comes to mind but in my experience that also doesn't make things much better. Does anyone have an idea? @arie @maarten @varac ? Maybe we can use k8s?
Todo
-
Created an kubernetes cluster ( gitlab-k8s-executor.openappstack.net
)using OAS first installation part (shared cluster config here: https://code.greenhost.net/openappstack/cluster-configs/-/tree/master/clusters/gitlab-k8s-executor) -
Integrate the cluster into gitlab -
Create a cluster management repo using the Cluster Management Project Template -
Enable ingress, gitlab-runner and prometheus apps in the cluster management helmfile.yaml -
Enable prometheus integration for the cluster. Basic metrics can be seen in the gitlab-k8s-executor cluster health tab -
Add the new gitlab-runner as a group runner to the openappstack group -
Create test repo for trying CI on the new runner -
Set the old docker runner to only run when docker
job tag is used -
Set the new kubernetes runner to run also when no job tag is used -
Fix pipeline issues related to new kubernetes executor (!483 (merged), https://open.greenhost.net/openappstack/website/-/merge_requests/48) -
Make sure pipeline succeeds both on kubernetes and docker runner
Follow-up issues
- Unregister and delete runner.openappstack.net: #938 (closed)