[BUG] Keycloak state (users) are not persisted on atlas-server stop/start

[pippinlee]

Describe the bug
Any user created in keycloak admin will not be persisted if keycloak is restarted. Currently the state is read from atlas.json but in ideal setup we would persist any change that can then be used when services is started up again.

To Reproduce
Steps to reproduce the behavior:

1. atlas-server start -p
2. open keycloak admin console
3. create user
4. atlas-server stop
5. atlas-server start -p
6. open keycloak admin console
7. previous user created no longer exists

Expected behavior
Internally the keycloak container uses an internal h2 database, but we can also persist to another container.

[amackillop]

There is an export.sh script located in the foundations_authentication that has been used for exporting the keycloak state so that new users are persisted. We can use this script to update the atlas.json configuration that keycloak imports on atlas-server start.

[kyledef]

We can also provide documentation that shows how to connect the keycloak server to an external database to allow persistence of users through restarts.
This will be similar to how we use it in our internal CI cluster. This might be the easiest path for a work around, but not necessarily a solution.

[amackillop]

@kyledef I was considering actually using a db like in CI as the solution if we are okay with having another container up on atlas server start.

[kyledef]

I'm suggesting to provide instruction for the end user to be able to add their own database container to the system. I don't think the overhead of an additional container makes sense for people who don't need the authentication.

For now it could placed in the documentation as "enabling authentication" or "adding users". Let's avoid adding a new container to the atlas installer.

[ekhl]

I second the approach from @kyledef to remove the number of moving parts.

[mohammedri]

@amackillop any thoughts on an expected timeline for this task? Do you have any huge roadblocks?

[amackillop]

No roadblocks. Just trying to decide on a path. I like the idea of reducing moving parts for non auth users but It would be nice to go all the way with this ie don't spin up keycloak at all. But with auth, we can spin up both containers if it is easy to make that delineation work. I don't particularly like the UX of having to do your own db setup.

Another solution would be to utilize the keycloak API to add users via the CLI. Then the export script could be run automatically and keep things persistent. That is building on top of a hack though, I'd like to get rid of the import thing all together.

On that note though, CLI for auth may still be a nice UX and can abstract away keycloak all together from the user if we can cover the necessary actions that a user would do on the keycloak admin page.

[amackillop]

Anyway I think I can take care of this in 2 weeks. @mohammedri

Branch: https://github.com/dessa-oss/atlas/tree/amackillop/keycloak-persist-users

[amackillop]

I landed on a simple solution that ends up reducing complexity. Don't know why I didn't think of it before. Instead of using the atlas.json file to load the initial state of keycloak, I just mounted the embedded H2 database directory to a place in foundations home.

If we ship atlas with some initial db files then it will initialize correctly and persist users for free. We also get to remove all the docker code plus the configuration and simply use the base jboss/keycloak image directly from dockerhub instead.

Keycloak explicitly advises against using the embeded db "in production" due to performance issues but I don't think that is an issue for us at this time.

[amackillop]

Work in progress PR: #135

[mohammedri]

Closed by: #161

[amackillop]

Merged. We went with mounting the H2 DB files in foundations home.

[mullerrwd]

Any info regarding the release of the next build which includes #161?