Skip to content

Commit

Permalink
Add WIP local dev instructions
Browse files Browse the repository at this point in the history
Make it easier to spin up supporting services (db and keycloak) locally
for development. The keycloak server is not currently used by the server
code.

Update README with steps for how to initialize the local environemnt.
  • Loading branch information
eest committed Dec 19, 2024
1 parent 5705049 commit 38aa282
Show file tree
Hide file tree
Showing 10 changed files with 189 additions and 0 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
sunet-cdn-manager
sunet-cdn-manager.toml
sunet-cdn-manager-dev.toml
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ This is the SUNET CDN manager server which serves an API and UI for
configuring the CDN service.

## Development
### Running tests
Running tests require a local PostgreSQL 15 or newer. The need for
at least version 15 is because the database setup expects the "Constrain
ordinary users to user-private schemas" schema usage pattern as described at
Expand All @@ -14,6 +15,32 @@ brew install postgresql@17
export PATH="/opt/homebrew/opt/postgresql@17/bin:$PATH"
```

### Setting up a local dev enviroment
Start database and keycloak:
```
docker compose -p sunet-cdn-manager -f local-dev/docker-compose.yml up
```

Initialize the sunet-cdn-manager realm in keycloak:
```
local-dev/keycloak/setup.sh
```

Create a config file for connecting insecurely to the local PostgreSQL database:
```
sed -e 's/"verify-full"/"disable"/' -e 's/"password"/"cdn"/' sunet-cdn-manager.toml.sample > sunet-cdn-manager-dev.toml
```

Initialize the sunet-cdn-manager database (this will print out a superuser username and password):
```
./sunet-cdn-manager --config sunet-cdn-manager-dev.toml init
```

Start the service:
```
./sunet-cdn-manager --config sunet-cdn-manager-dev.toml server
```

### Formatting and linting
When working with this code at least the following tools are expected to be
run at the top level directory prior to commiting:
Expand Down
26 changes: 26 additions & 0 deletions local-dev/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
services:
db:
image: "postgres:17.2-bookworm"
environment:
- POSTGRES_PASSWORD=dev
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
- ./docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
keycloak:
image: "quay.io/keycloak/keycloak:26.0.7"
environment:
- KC_DB=postgres
- KC_DB_URL=jdbc:postgresql://db:5432/keycloak
- KC_DB_USERNAME=keycloak
- KC_DB_PASSWORD=keycloak
- KC_BOOTSTRAP_ADMIN_USERNAME=admin
- KC_BOOTSTRAP_ADMIN_PASSWORD=admin
ports:
- "8080:8080"
depends_on:
- db
command: "start-dev"
volumes:
postgres_data:
27 changes: 27 additions & 0 deletions local-dev/docker-entrypoint-initdb.d/init-cdn-db.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
#!/bin/bash
set -e

# Create database named after user, then create a schema named the same as the
# user which is also owned by that user. Because search_path (SHOW
# search_path;) starts with "$user" by default this means any tables will be
# created in that user-specific SCHEMA by default instead of falling back to
# "public". This follows the "secure schema usage pattern" summarized as
# "Constrain ordinary users to user-private schemas" from
# https://www.postgresql.org/docs/current/ddl-schemas.html#DDL-SCHEMAS-PATTERNS
#
# "In PostgreSQL 15 and later, the default configuration supports this usage
# pattern. In prior versions, or when using a database that has been upgraded
# from a prior version, you will need to remove the public CREATE privilege
# from the public schema"
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" --dbname "$POSTGRES_DB" <<-EOSQL
CREATE USER cdn WITH PASSWORD 'cdn';
CREATE DATABASE cdn;
GRANT ALL PRIVILEGES ON DATABASE cdn TO cdn;
CREATE USER keycloak WITH PASSWORD 'keycloak';
CREATE DATABASE keycloak;
GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak;
\c cdn;
CREATE SCHEMA cdn AUTHORIZATION cdn;
\c keycloak;
CREATE SCHEMA keycloak AUTHORIZATION keycloak;
EOSQL
10 changes: 10 additions & 0 deletions local-dev/keycloak/keycloak-device-client.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"protocol":"openid-connect",
"clientId":"sunet-cdn-manager-devices",
"publicClient":true,
"standardFlowEnabled":false,
"directAccessGrantsEnabled":false,
"attributes":{
"oauth2.device.authorization.grant.enabled":true
}
}
3 changes: 3 additions & 0 deletions local-dev/keycloak/keycloak-realm.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"realm": "sunet-cdn-manager"
}
16 changes: 16 additions & 0 deletions local-dev/keycloak/keycloak-server-client.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"protocol":"openid-connect",
"clientId":"sunet-cdn-manager-server",
"publicClient":false,
"standardFlowEnabled":true,
"directAccessGrantsEnabled":false,
"attributes":{
"post.logout.redirect.uris":"http://localhost:8081/*"
},
"redirectUris":[
"http://localhost:8081/*"
],
"webOrigins":[
"http://localhost:8081/*"
]
}
5 changes: 5 additions & 0 deletions local-dev/keycloak/keycloak-user-password.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"temporary":false,
"type":"password",
"value":"testuser"
}
8 changes: 8 additions & 0 deletions local-dev/keycloak/keycloak-user.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"username":"testuser",
"firstName":"Test",
"lastName":"User",
"email":"[email protected]",
"emailVerified":false,
"enabled":true
}
66 changes: 66 additions & 0 deletions local-dev/keycloak/setup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/usr/bin/env bash

set -eu

# Keep in mind that these settings need to match the contents of the json
# files.
realm="sunet-cdn-manager"
user="admin"

# Make it so we can run the script from anywhere
cd $(dirname $0)

# Get access token from username/password
access_token=$(curl -s \
-d "client_id=admin-cli" \
-d "username=admin" \
-d "password=admin" \
-d "grant_type=password" \
"http://localhost:8080/realms/master/protocol/openid-connect/token" | jq -r .access_token)

# Only do anything if the realm does not exist
realm_response=$(curl -s -X GET \
-H "Authorization: bearer $access_token" \
"http://localhost:8080/admin/realms/$realm")

if ! echo $realm_response | grep -q "Realm not found."; then
echo "Realm '$realm' alredy exists, doing nothing"
exit 1
fi

echo "Creating realm '$realm'"
curl -X POST \
-H "Authorization: bearer $access_token" \
-H "Content-Type: application/json" \
-d @keycloak-realm.json \
"http://localhost:8080/admin/realms"

echo "Creating user '$user'"
# The sed is needed to strip out a \r character present in the header printed by
# curl, without this the content of user_id is not usable in the next step.
user_id=$(curl -si -X POST \
-H "Authorization: bearer $access_token" \
-H "Content-Type: application/json" \
-d @keycloak-user.json \
"http://localhost:8080/admin/realms/$realm/users" | awk -F/ '/^Location:/{print $NF}' | sed 's/\r$//')

echo "Setting password for user '$user'"
curl -X PUT \
-H "Authorization: bearer $access_token" \
-H "Content-Type: application/json" \
-d @keycloak-user-password.json \
"http://localhost:8080/admin/realms/$realm/users/$user_id/reset-password"

echo "Creating oauth2 confidential client for sunet-cdn-manager server"
curl -X POST \
-H "Authorization: bearer $access_token" \
-H "Content-Type: application/json" \
-d @keycloak-server-client.json \
"http://localhost:8080/admin/realms/$realm/clients"

echo "Creating oauth2 public client for requesting device grants"
curl -X POST \
-H "Authorization: bearer $access_token" \
-H "Content-Type: application/json" \
-d @keycloak-device-client.json \
"http://localhost:8080/admin/realms/$realm/clients"

0 comments on commit 38aa282

Please sign in to comment.