Skip to content

Commit

Permalink
Update argus-build-server from folk to simple build-steps
Browse files Browse the repository at this point in the history
  • Loading branch information
benedictmulongo committed Dec 19, 2024
1 parent 154371f commit 0f000bf
Show file tree
Hide file tree
Showing 11 changed files with 388 additions and 2 deletions.
38 changes: 38 additions & 0 deletions .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Build and Save Docker Image

on:
push:
branches: [ main ]

jobs:
build:
runs-on: ubuntu-latest

steps:
# Checkout code from repository
- name: Checkout code
uses: actions/checkout@v4

# Set up Docker Buildx (for building multi-platform images)
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

# Build Docker image
- name: Build Docker image
run: |
docker build -t argus-api:${{ github.sha }} -f docker/Dockerfile .
# Save Docker image as an artifact
- name: Save Docker image as artifact
run: |
docker save argus-api:${{ github.sha }} -o argus-api-${{ github.sha }}.tar
# Ensure the artifact is available for download
continue-on-error: true

# Upload the saved Docker image tarball as an artifact
- name: Upload Docker image artifact
uses: actions/upload-artifact@v4
with:
name: argus-api-test
path: argus-api-${{ github.sha }}.tar
retention-days: 5
22 changes: 22 additions & 0 deletions .jenkins.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
# Documentation: https://github.com/SUNET/bootstrap-docker-builds
git:
branch: main
triggers:
github_push: true
clean_workspace: true
pre_build_script:
- cp docker/Dockerfile .
script:
- echo "Build SUNET Custom ARGUS Image"
builders:
- docker
docker_name: sunet/argus-server
docker_tags:
- "v1.27.0_sunet-${GIT_COMMIT}"
docker_force_tag: true
docker_no_cache: true
docker_skip_tag_as_latest: false
docker_build_args: |
BRANCH_NAME=v1.27.0
REPO_URL=https://github.com/Uninett/Argus.git
36 changes: 34 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,34 @@
# sunet-argus-server
sunet-argus-server
# Argus Build Server

The **sunet-argus-server** is used to build a specific version of the code hosted in the [Argus Server GitHub repository](https://github.com/Uninett/Argus.git) <https://github.com/Uninett/Argus.git>.

To change the version being built, you can update the `.jenkins.yaml` file with the desired settings. For example:

```yaml
docker_tags:
- "v1.27.0_sunet-${GIT_COMMIT}"
docker_build_args: |
BRANCH_NAME=v1.27.0
REPO_URL="https://github.com/Uninett/Argus.git"
```
## Key Points
- The `REPO_URL` should generally remain unchanged as it points to the main repository.
- The `BRANCH_NAME` should specify the tag version from the Uninett repository that you want to test.
- Update `docker_tags` to match the version you wish to use.

## Better Tag Selection

Alternatively, you can use the Python script `bump-tag.py` to automatically select a specific tag and update the `.jenkins.yaml` file. To do this, follow these steps:

```bash
chmod +x bump-tag.py
./bump-tag.py
```

This script simplifies the process by determining the appropriate tag to use and applying the changes to the `.jenkins.yaml` file and commiting the code.

## Notes

Ensure that the selected tag in `BRANCH_NAME` matches a valid version from the Uninett repository. Properly updating the `docker_tags` and `BRANCH_NAME` ensures consistency between the build configurations and the repository version you intend to test.
104 changes: 104 additions & 0 deletions bump-tag.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
#!/usr/bin/env python3

import subprocess
import tempfile
import yaml
from jinja2 import Template


context = {
"docker_tag_name": "v1.27.0_sunet-${GIT_COMMIT}",
"branch_name": "v1.27.0",
"repository_url": "https://github.com/Uninett/Argus.git"
}


def get_commit():
# Capture the most recent commit hash using `git log --show-signature -1`
result = subprocess.run(
"git log --show-signature -1",
shell=True,
check=True,
capture_output=True,
text=True,
)

# Extract the commit hash from the git log output
git_commit = None
for line in result.stdout.splitlines():
if line.startswith("commit"):
git_commit = line.split()[1] # Commit hash is the second word
break
return git_commit


def get_template(filename=".jenkins.yaml.jinja"):
template_content = None
# Read the Jinja2 template YAML file
with open(filename, "r") as template_file:
template_content = template_file.read()

return template_content

# Use Python's tempfile module to create a temporary directory in memory
with tempfile.TemporaryDirectory() as temp_dir:
# Clone the repository with --bare to save bandwidth
subprocess.run(
f"git clone --bare {context['repository_url']} {temp_dir} > /dev/null 2>&1",
shell=True,
check=True,
)

# Run the 'git for-each-ref' command to get the tags sorted by date
result = subprocess.run(
f"git -C {temp_dir} for-each-ref --sort=-creatordate --format '%(refname:short)' refs/tags",
shell=True,
check=True,
capture_output=True,
text=True,
)

# Split the output into a list of tags
tags = result.stdout.splitlines()
len_tags = len(tags)

# Print the tags
print(f"Tags from {context['repository_url']} listed by creation date:")
for index, tag in enumerate(tags):
print(f"{index+1}. {tag}")

tag_number = input("Choose the tag you want to build: ")
tag_number = int(tag_number)
assert (
tag_number >= 1 and tag_number <= len_tags + 1
), "The number should be one in the list above"
print(f"You have choosed: {tag_number}, {tags[tag_number-1]}")
git_commit = get_commit()[:8]
context['branch_name'] = tags[tag_number-1]
context['docker_tag_name'] = f"{tags[tag_number-1]}_sunet-" + "${GIT_COMMIT}"

# Todo. change .jenkins.yaml with the new version
template = Template(get_template(filename="templates/.jenkins.yaml.jinja"))
rendered_yaml = template.render(context)
yaml_data = yaml.safe_load(rendered_yaml)
# Print the rendered YAML content
print("Updated .jenkins.yaml:\n")
yaml_output = yaml.dump(yaml_data, default_flow_style=False)
tabbed_yaml_output = "\n".join(f"\t{line}" for line in yaml_output.splitlines())
print(tabbed_yaml_output)

# TODOD. push changes to git
file_to_remplace = ".jenkins.yaml"
with open(file_to_remplace, "w") as output_file:
output_file.write(rendered_yaml)

# Prompt the user for a commit message
commit_message = input("Enter the commit message: ")

print("\tCommitting changes...")
# Commit the changes
subprocess.run('git add .jenkins.yaml', shell=True, check=True)
subprocess.run(f'git commit -m "{commit_message}"', shell=True, check=True)

print("pushing changes to repository...")
subprocess.run("git push -u origin main", shell=True, check=True)
41 changes: 41 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
version: '3.5'

services:
argus-api:
build:
context: .
dockerfile: ./docker/Dockerfile
args:
BRANCH_NAME: "v1.25.0"
ports:
- "8000:8000"
environment:
- TIME_ZONE=Europe/Stockholm
- DJANGO_SETTINGS_MODULE=dockersettings
- DJANGO_SETTINGS_MODULE=argus.site.settings.dockerdev
- PYTHONPATH=/var/opt/extrapython
- STATIC_ROOT=/static
- DATABASE_URL=postgresql://argus:HahF9araeKoo@postgres/argus
- ARGUS_REDIS_SERVER=redis
depends_on:
- postgres
- redis
volumes:
- ${PWD}:/argus

postgres:
image: "postgres:14"
volumes:
- postgres:/var/lib/postgresql/data:Z
environment:
- POSTGRES_USER=argus
- POSTGRES_PASSWORD=HahF9araeKoo
- POSTGRES_DB=argus

redis:
image: "redis:latest"
restart: always

volumes:
postgres:
driver: local
35 changes: 35 additions & 0 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Defines a production image for the Argus API server
# Needs the repository root directory as its context
FROM python:3.10
ENV DEBIAN_FRONTEND=noninteractive

# Define an ARG variable
ARG REPO_URL=https://github.com/Uninett/Argus.git
ARG BRANCH_NAME=v1.27.0
ARG BASE_FOLDER=/var/opt

RUN apt-get update && apt-get install -y --no-install-recommends tini build-essential

# Install some dev requirements that aren't part of the minimal dependencies:
RUN pip install psycopg2-binary django-extensions python-dotenv gunicorn 'uvicorn[standard]'

# Make an unprivileged user to run the server
RUN useradd --system argus
# Ensure this user has privileges to collect static resources in /static
RUN mkdir -p /static && chown argus /static
ENV STATIC_ROOT=/static

RUN git clone --depth 1 --branch ${BRANCH_NAME} ${REPO_URL} /src
RUN pip install /src && rm -rf /src

# Install API backend settings suitable for Docker deployment
RUN mkdir /extrapython
COPY docker/dockersettings.py /extrapython/
ENV PYTHONPATH=/extrapython
ENV DJANGO_SETTINGS_MODULE=dockersettings

ENV PORT=8000
EXPOSE 8000
COPY docker/docker-entrypoint.sh /api-entrypoint.sh
USER argus
ENTRYPOINT ["/usr/bin/tini", "-v", "--", "/api-entrypoint.sh"]
37 changes: 37 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Docker production image of Argus

Whereas the top-level Dockerfile is development-oriented, this directory
contains definitions to build a production-oriented Docker image of the Argus
API backend component.

To build this image, the build context needs to be that of the git repository
root. Something like this would work (when the current working directory is
here):

```shell
docker build -t argus -f Dockerfile ..
```

Or, from the top level directory:

```shell
docker build -t argus -f docker/Dockerfile .
```

## Configuration of the running container

This image runs with default production settings, with a few tweaks from
[dockersettings.py](dockersettings.py). This means that the most useful
settings can be overriden through the use of environment variables exported to
the container. Consult the documentation section on [site-specific
settings](http://argus-server.rtfd.io/en/latest/site-specific-settings.html).

## Limitations

This is not a complete Argus environment. This image only defines the backend
API server component. It still depends on a PostgreSQL and a Redis server to be
functional. Also, the Argus front-end component is needed to have a functional
user interface against the API server.

For a full production environment example, take a look at
https://github.com/Uninett/argus-docker
11 changes: 11 additions & 0 deletions docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/bin/bash -e

django-admin collectstatic --noinput
django-admin migrate --noinput
exec gunicorn \
argus.ws.asgi:application \
-k uvicorn.workers.UvicornWorker \
--forwarded-allow-ips="*" \
--access-logfile - \
-b 0.0.0.0:$PORT \
$@
25 changes: 25 additions & 0 deletions docker/dockersettings.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
"""Settings for production-like deployments in Docker"""

from argus.site.settings.prod import *

# Allow all hosts to reach backend, since all requests will typically come from
# outside the container:
ALLOWED_HOSTS = ["*"]

# Uncomment to enable both Email and SMS-as-email notification backends,
# leave commented out to keep just the default email backend:
# MEDIA_PLUGINS = [
# "argus.notificationprofile.media.email.EmailNotification",
# "argus.notificationprofile.media.sms_as_email.SMSNotification",
# ]

ARGUS_SEND_NOTIFICATIONS = True
DEFAULT_SMS_MEDIA="argus.notificationprofile.media.sms_as_email.SMSNotification"
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend"
CSRF_TRUSTED_ORIGINS = ['https://*.sunet.se','https://*.127.0.0.1','https://*.sunet.se:9000']
CORS_ORIGIN_WHITELIST = ['https://*sunet.se','https://*.sunet.se:9000']

MEDIA_PLUGINS = [
"argus.notificationprofile.media.email.EmailNotification",
"argus.notificationprofile.media.sms_as_email.SMSNotification",
]
22 changes: 22 additions & 0 deletions templates/.jenkins.yaml.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
# Documentation: https://github.com/SUNET/bootstrap-docker-builds
git:
branch: main
triggers:
github_push: true
clean_workspace: true
pre_build_script:
- cp docker/Dockerfile .
script:
- echo "Build SUNET Custom ARGUS Image"
builders:
- docker
docker_name: sunet/argus-server
docker_tags:
- "{{ docker_tag_name }}"
docker_force_tag: true
docker_no_cache: true
docker_skip_tag_as_latest: false
docker_build_args: |
BRANCH_NAME={{ branch_name }}
REPO_URL={{ repository_url }}
Loading

0 comments on commit 0f000bf

Please sign in to comment.