Resolve "Add login panel CI/CD to create (docker) image"

Closes #103 (closed)

docs/development.md

+# Local dev with Kratos
+
+The main role for this repo is provide Single-Sign-On. The architecture to make
+this happen has a lot of moving components. A quick overview:
+
+ - Hydra: Hydra is an Identity Provider, or IdP for short. It means connected
+   applications connect to Hydra to start a session with a user. Hydra provides
+   the application with the username and other roles/claims for the application. 
+   This is done using the OIDC protocol. Hydra is developed by Ory and has
+   security as one of their top priorities. Also it is fully OpenSource.
+
+ - Login application: If hydra hits a new session/user, it has to know if this
+   user has access. To do so, the user has to login. Hydra does not support
+   this, so it will redirect to a login application. This is developed by the
+   Stackspin team (Greenhost) and part of this repository. It is a Python Flask
+   application. 
+   Because the security decisions made by kratos (see below), a lot of the 
+   interaction is done in the web-browser, rather then server-side. 
+   This means the login application has an UI component which relies heavily on 
+   JavaScript. As this is a relatively small application, it is based on
+   traditional Bootstrap + Jquery. This elements the requirement for yet an
+   other build environment.
+
+ - Kratos: This is Identity Manager and contains all the user profiles and
+   secrets (passwords). Kratos is designed to work mostly between UI (browser)
+   and kratos directly, over a public API endpoint without an extra server side
+   component/application. So authentication, form-validation etc, are all handled
+   by Kratos. Kratos only provides an API and not UI itself.
+   Kratos provides a Admin API, which is only used from the server-side flask
+   app to create/delete users.
+
+ - Postgres: All three components need to store data. This is done in a postgres
+   database server. There is once instance, with three databases. As all
+   databases are very small this will not lead to resource limitation problems.
+
+## Prerequisites
+
+The current login panel is not yet installed available in released  versions 
+of Stackspin. However, this does not prevent us from developing already on the
+login panel. Experience with `helm` and `kubernetes` is expected when you follow
+this manual.
 
+On your provisioning machine, make sure to checkout:
 
-# Introduction
+`git@open.greenhost.net:stackspin/single-sign-on.git`
 
-kratos manages the user database. It has profiles of all users and keeps track
-of lost password policies, welcome e-mails, TOTP (future), First, Last name etc.
+Be sure to check out the latest main branch. Or select a more modern branch if you
+want to test / install (optional) improvements of login panel.
 
-Kratos is a flexible identity manager where our own "schema" can be defined with
-the information we want for Stackspin. 
+Once this is all fetched, installation can be done with the following steps:
 
-Kratos has a public API, which should be accessible for the world, and an admin API
-which is ONLY accessible for our panel/board to manage users.
+1. Create an overwrite ConfigMap file:
 
-At the point of writing BOTH end-points are not public yet. We can use SSH port
-forwards during development.
+   For local development, we have to configure the endpoint of the application to
+   be pointing to our development system. In this example, we use `localhost` on
+   http.
 
-# Installation
+   Because of CORS and strict configuration, all needs to end up on the same
+   system. With modern browser, it even have to run on the same port (at least with
+   firefox). As we want to mimic the real life setup as much as possible as,
+   we will do this by running a local proxy. In production this will be handled by
+   kubernetes ingress configuration.
 
-The current kratos version is not yet installed in the released versions of 
-Stackspin. However, this does not prevent us from developing already with
-kratos. To use / add the kratos backend, the following needs to be done:
+   First we will tell kratos and hydra where to find the right endpoints. An
+   overview of all relevant end-points:
 
-On your provisioning machine, make sure to checkout:
+   The endpoints used by the browser are (public accessible)
 
-`git@open.greenhost.net:stackspin/single-sign-on.git`
+   - `localhost/api` -> kratos public API
+   - `localhost/login` -> login flask app
 
-Be sure to check out the latest main. Or select a more modern brach if you
-want to test / install (optional) improvements of the kratos system. 
+   The endpoint used by the login app/API are:
+   - `localhost:8000` -> kratos Admin API (only local accessible)
+   - `localhost/api`  -> kratos Public API
+   - `localhost:4445` -> hydra Admin API (only local accessible)
+   - `localhost:5432` -> PostgreSQL
 
-Once this is all fetched, installation can be done with the following steps:
+   To reflect those public endpoints in your cluster, we have to override the 
+   default URLs in the cluster. We do this with a ConfigMap.
+
+   It is essential SMTP/e-mail is working during development, so an example 
+   is included on how to override those if SMTP is not working on your 
+   cluster. Otherwise those lines are irrelevant.
 
-1. Suspend the automatic updating: 
-   As we are gonna use a non-release version, the flux application management system will rollback
-   changes to follow the released versions. However, during development we want
-   to prevent this. We can suspend the service with:
+   Create a file with the following content:
 
 ```
-flux suspend source chart stackspin-single-sign-on
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: stackspin-single-sign-on-override
+data:
+  values.yaml: |
+    kratos:
+      kratos:
+        config:
+          courier:
+            smtp:
+             # Kratos enforces the use of STARTTLS. Be sure your SMTP provider
+             # supports that (if not, it is time to switch providers)
+             #
+             # Uncomment and correct below lines if e-mail is not working in your
+             # cluster
+             # connection_uri: smtp://user@password@smtp.example.com:25/
+             # from_address: stackspin-admin@example.com
+
+          # For development, we forward all to our local server (or your dev server
+          # if that is remote)
+          serve:
+            public:
+              base_url: http://localhost/api/
+
+          selfservice:
+            default_browser_return_url: http://localhost/login/login
+
+            flows:
+              recovery:
+                ui_url: http://localhost/login/recovery
+
+              login:
+                ui_url: http://localhost/login/login
+
+              settings:
+                ui_url: http://localhost/login/settings
+
+              registration:
+                ui_url: http://localhost/login/registration
+
+    hydra:
+      hydra:
+        config:
+          urls:
+            # For development we redirect to localhost (or your dev server)
+            login: http://localhost/login/auth
+            consent: http://localhost/login/consent
+            logout: http://localhost/login/logout
 ```
 
-2. Make a backup of the current keys and configuration values. We needs those
-   when we install the new version of the `single-sign-on` helmchart:
+2. Apply the ConfigMap to your cluster:
+ 
+   ```
+   kubectl apply -n stackspin -f stackspin-single-sign-on-override.yaml
+   ```
+
+3. Tell flux to reconcile the configuration
+
+   Normally flux will do this on some interval. We will tell flux to apply
+   the override immediately.
+
+   ```
+   flux reconcile kustomization core
+   flux reconcile helmrelease -n stackspin single-sign-on
+   ```
+
+## Development
+
+1. Setup port redirects
+
+To be able to work on the Login panel, we have to configure our development
+system to access all the remote services and endpoints.
+
+A helper script is available in this directory to setup and redirect the
+relevant ports locally. It will open ports 8000, 8080, 4445, 5432 to get access
+to all APIs:
 
 ```
-helm get values single-sign-on -n stackspin > /to/a/path/my_cluster_values.yaml
+cd project_root/login
+./set-ssh-tunnel.sh "stackspin.example.com"
 ```
 
-3. Install all helm dependencies
+(the tunnel goes to the kubernetes node, so *not* to your provisioning machine,
+ it will uses SSH port forwarding to map ports, as a result you will also have
+ SSH session to your kubernetes node. Do not close this session, as closing the
+ session will close the forwarded ports as well)
+
+2. Configure a local proxy
+
+Because of strict CORS headers, we have to map the public kratos API and login
+app which we will run locally, with a local proxy.
+
+This can be done with any proxy server, for example with NGINX. Be sure you have
+NGINX installed and listening on port 80 locally (`sudo apt-get install nginx`)
+should be enough. 
+
+Now configure NGINX with this configuration in `/etc/nginx/sites-enabled/default`
 
 ```
-helm dep update
-```
 
-4. Configure variables
+server {
+    listen 80 default_server;
+    listen [::]:80 default_server;
+
+    root /var/www/html;
+
+    index index.html;
+
+    server_name _;
 
-Currently the is a default password configured for postgresql in `values.yaml`.
-It is advised to change this password before installation.
+    location / {
+        # First attempt to serve request as file, then
+        # as directory, then fall back to displaying a 404.
+        try_files $uri $uri/ =404;
+    }
 
-The password can be found at:
+    # Flask app
+    location /login/ {
+        proxy_pass http://127.0.0.1:5000/;
+        proxy_redirect     default;
+        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
+    }
 
+    # Kratos Public
+    location /api/  {
+        proxy_pass http://127.0.0.1:8080/;
+        proxy_redirect     default;
+        proxy_set_header   X-Forwarded-For   $proxy_add_x_forwarded_for;
+    }
+}
 ```
-postgresql:
-  postgresqlDatabase: kratos
-  postgresqlUsername: kratos
-  postgresqlPassword: ChangeThisPassword
+
+Reload your NGINX:
+
 ```
+sudo systemctl reload nginx.service
+```
+
+3. Run FLASK app
 
-Please make sure it is in sync with the password at
+Now it is time to start the flask app. Please make sure you are using python 3 in your environment. And install the required dependencies:
 
 ```
-kratos:
-  kratos:
-    config:
-      dsn: postgres://kratos:ChangeThisPassword@single-sign-on-postgresql:5432/kratos
-``` 
+cd projectroot/login
+pip3 install -r requirements.txt
+```
 
-Same is true for the SMTP service, this config entry can be found at:
+Then verify if you are happy with the settings in the `source_env` file:
 
 ```
-kratos:
-  kratos:
-    config:
-      courier:                                                                   
-        smtp:                                                                    
-          connection_uri: smtps://username:password@smtp.example.com:456/
+cat source_env
+
+export FLASK_RUN_HOST=0.0.0.0
+export FLASK_RUN_PORT=5000
+export HYDRA_ADMIN_URL=http://localhost:4445
+export KRATOS_PUBLIC_URL=http://localhost/api
+export KRATOS_ADMIN_URL=http://localhost:8000
+export PUBLIC_URL=http://localhost/login
+
+export DATABASE_URL="postgresql://stackspin:stackspin@localhost/stackspin"
+export APP_SETTINGS="config.DevelopmentConfig"
 ```
 
-5. Install the single-sign-on helmchart with kratos service
+Normally you only need to change the database password if you did not use the
+default.
+
+Assuming you did not populate the database yet, run this to populate it:
 
 ```
-cd helmchart/single-sign-on
-helm upgrade -f /to/a/path/my_cluster_values.yaml single-sign-on . -n stackspin --debug
+. source_env
+flask db upgrade
 ```
 
-This will install the latest version. 
+If that all looks fine, it is time to add you first user:
 
-# Development
+```
+flask user create myemail@example.com
+```
 
-To develop, one needs access to kratos from the development system. A helper 
-script is available in this directory to setup the redirect the ports, 
-giving access to localhost port 8000 and 8080 for the admin/public port of
-kratos.
+And now it is time to start the app:
 
 ```
-./set-ssh-tunnel.sh `stackspin.example.com`
+./run.sh
 ```
 
-(the tunnel goes to the kubernetes node, so *not* to your provisioning machine.
+If this starts smoothly, you should be ready to go.
 
-kratos API is specified on their website:
+## Test you setup
 
-https://www.ory.sh/kratos/docs/reference/api/
+Hydra and kratos are now configured to redirect to localhost when they receive a
+request. So to test the setup, you can go to one of your applications (for
+example nextcloud), what we expect when you click the login button is the
+following:
 
-Some example can be found in:
+- Nextcloud redirect to Hydra (on sso.example.com)
+- Hydra does not have a session, so ask to authorize on: http://localhost/login/auth
+- Kratos does not have a session, so the login panel will ask to login on:
+  http://localhost/login/login
+- You do not have a password setup yet, so you click "recover account", which
+  should bring you to: http://localhost/login/recovery
+- You enter your email address and request a reset token. Check you e-mail. The
+  email should contain a link to http://localhost/api/self-service/recovery/..
+- The link logs you in in kratos and ask you to setup a password. Complete this
+  step and you account is ready.
 
-```
-./api-examples.sh
-```
+We started the flow with trying to reach nextcloud. Because we
+did a password recovery in between, this information is lost. If you go again to
+nextcloud manually, you should now be logged in automatically.
 
+If you retry this, but now with a password (for example in a privacy window or
+by removing you cookies), you should be redirected automatically after login.