-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update argus-build-server from folk to simple build-steps
- Loading branch information
1 parent
154371f
commit 0f000bf
Showing
11 changed files
with
388 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
name: Build and Save Docker Image | ||
|
||
on: | ||
push: | ||
branches: [ main ] | ||
|
||
jobs: | ||
build: | ||
runs-on: ubuntu-latest | ||
|
||
steps: | ||
# Checkout code from repository | ||
- name: Checkout code | ||
uses: actions/checkout@v4 | ||
|
||
# Set up Docker Buildx (for building multi-platform images) | ||
- name: Set up Docker Buildx | ||
uses: docker/setup-buildx-action@v3 | ||
|
||
# Build Docker image | ||
- name: Build Docker image | ||
run: | | ||
docker build -t argus-api:${{ github.sha }} -f docker/Dockerfile . | ||
# Save Docker image as an artifact | ||
- name: Save Docker image as artifact | ||
run: | | ||
docker save argus-api:${{ github.sha }} -o argus-api-${{ github.sha }}.tar | ||
# Ensure the artifact is available for download | ||
continue-on-error: true | ||
|
||
# Upload the saved Docker image tarball as an artifact | ||
- name: Upload Docker image artifact | ||
uses: actions/upload-artifact@v4 | ||
with: | ||
name: argus-api-test | ||
path: argus-api-${{ github.sha }}.tar | ||
retention-days: 5 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
--- | ||
# Documentation: https://github.com/SUNET/bootstrap-docker-builds | ||
git: | ||
branch: main | ||
triggers: | ||
github_push: true | ||
clean_workspace: true | ||
pre_build_script: | ||
- cp docker/Dockerfile . | ||
script: | ||
- echo "Build SUNET Custom ARGUS Image" | ||
builders: | ||
- docker | ||
docker_name: sunet/argus-server | ||
docker_tags: | ||
- "v1.27.0_sunet-${GIT_COMMIT}" | ||
docker_force_tag: true | ||
docker_no_cache: true | ||
docker_skip_tag_as_latest: false | ||
docker_build_args: | | ||
BRANCH_NAME=v1.27.0 | ||
REPO_URL=https://github.com/Uninett/Argus.git |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,34 @@ | ||
# sunet-argus-server | ||
sunet-argus-server | ||
# Argus Build Server | ||
|
||
The **sunet-argus-server** is used to build a specific version of the code hosted in the [Argus Server GitHub repository](https://github.com/Uninett/Argus.git) <https://github.com/Uninett/Argus.git>. | ||
|
||
To change the version being built, you can update the `.jenkins.yaml` file with the desired settings. For example: | ||
|
||
```yaml | ||
docker_tags: | ||
- "v1.27.0_sunet-${GIT_COMMIT}" | ||
docker_build_args: | | ||
BRANCH_NAME=v1.27.0 | ||
REPO_URL="https://github.com/Uninett/Argus.git" | ||
``` | ||
## Key Points | ||
- The `REPO_URL` should generally remain unchanged as it points to the main repository. | ||
- The `BRANCH_NAME` should specify the tag version from the Uninett repository that you want to test. | ||
- Update `docker_tags` to match the version you wish to use. | ||
|
||
## Better Tag Selection | ||
|
||
Alternatively, you can use the Python script `bump-tag.py` to automatically select a specific tag and update the `.jenkins.yaml` file. To do this, follow these steps: | ||
|
||
```bash | ||
chmod +x bump-tag.py | ||
./bump-tag.py | ||
``` | ||
|
||
This script simplifies the process by determining the appropriate tag to use and applying the changes to the `.jenkins.yaml` file and commiting the code. | ||
|
||
## Notes | ||
|
||
Ensure that the selected tag in `BRANCH_NAME` matches a valid version from the Uninett repository. Properly updating the `docker_tags` and `BRANCH_NAME` ensures consistency between the build configurations and the repository version you intend to test. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,104 @@ | ||
#!/usr/bin/env python3 | ||
|
||
import subprocess | ||
import tempfile | ||
import yaml | ||
from jinja2 import Template | ||
|
||
|
||
context = { | ||
"docker_tag_name": "v1.27.0_sunet-${GIT_COMMIT}", | ||
"branch_name": "v1.27.0", | ||
"repository_url": "https://github.com/Uninett/Argus.git" | ||
} | ||
|
||
|
||
def get_commit(): | ||
# Capture the most recent commit hash using `git log --show-signature -1` | ||
result = subprocess.run( | ||
"git log --show-signature -1", | ||
shell=True, | ||
check=True, | ||
capture_output=True, | ||
text=True, | ||
) | ||
|
||
# Extract the commit hash from the git log output | ||
git_commit = None | ||
for line in result.stdout.splitlines(): | ||
if line.startswith("commit"): | ||
git_commit = line.split()[1] # Commit hash is the second word | ||
break | ||
return git_commit | ||
|
||
|
||
def get_template(filename=".jenkins.yaml.jinja"): | ||
template_content = None | ||
# Read the Jinja2 template YAML file | ||
with open(filename, "r") as template_file: | ||
template_content = template_file.read() | ||
|
||
return template_content | ||
|
||
# Use Python's tempfile module to create a temporary directory in memory | ||
with tempfile.TemporaryDirectory() as temp_dir: | ||
# Clone the repository with --bare to save bandwidth | ||
subprocess.run( | ||
f"git clone --bare {context['repository_url']} {temp_dir} > /dev/null 2>&1", | ||
shell=True, | ||
check=True, | ||
) | ||
|
||
# Run the 'git for-each-ref' command to get the tags sorted by date | ||
result = subprocess.run( | ||
f"git -C {temp_dir} for-each-ref --sort=-creatordate --format '%(refname:short)' refs/tags", | ||
shell=True, | ||
check=True, | ||
capture_output=True, | ||
text=True, | ||
) | ||
|
||
# Split the output into a list of tags | ||
tags = result.stdout.splitlines() | ||
len_tags = len(tags) | ||
|
||
# Print the tags | ||
print(f"Tags from {context['repository_url']} listed by creation date:") | ||
for index, tag in enumerate(tags): | ||
print(f"{index+1}. {tag}") | ||
|
||
tag_number = input("Choose the tag you want to build: ") | ||
tag_number = int(tag_number) | ||
assert ( | ||
tag_number >= 1 and tag_number <= len_tags + 1 | ||
), "The number should be one in the list above" | ||
print(f"You have choosed: {tag_number}, {tags[tag_number-1]}") | ||
git_commit = get_commit()[:8] | ||
context['branch_name'] = tags[tag_number-1] | ||
context['docker_tag_name'] = f"{tags[tag_number-1]}_sunet-" + "${GIT_COMMIT}" | ||
|
||
# Todo. change .jenkins.yaml with the new version | ||
template = Template(get_template(filename="templates/.jenkins.yaml.jinja")) | ||
rendered_yaml = template.render(context) | ||
yaml_data = yaml.safe_load(rendered_yaml) | ||
# Print the rendered YAML content | ||
print("Updated .jenkins.yaml:\n") | ||
yaml_output = yaml.dump(yaml_data, default_flow_style=False) | ||
tabbed_yaml_output = "\n".join(f"\t{line}" for line in yaml_output.splitlines()) | ||
print(tabbed_yaml_output) | ||
|
||
# TODOD. push changes to git | ||
file_to_remplace = ".jenkins.yaml" | ||
with open(file_to_remplace, "w") as output_file: | ||
output_file.write(rendered_yaml) | ||
|
||
# Prompt the user for a commit message | ||
commit_message = input("Enter the commit message: ") | ||
|
||
print("\tCommitting changes...") | ||
# Commit the changes | ||
subprocess.run('git add .jenkins.yaml', shell=True, check=True) | ||
subprocess.run(f'git commit -m "{commit_message}"', shell=True, check=True) | ||
|
||
print("pushing changes to repository...") | ||
subprocess.run("git push -u origin main", shell=True, check=True) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
version: '3.5' | ||
|
||
services: | ||
argus-api: | ||
build: | ||
context: . | ||
dockerfile: ./docker/Dockerfile | ||
args: | ||
BRANCH_NAME: "v1.25.0" | ||
ports: | ||
- "8000:8000" | ||
environment: | ||
- TIME_ZONE=Europe/Stockholm | ||
- DJANGO_SETTINGS_MODULE=dockersettings | ||
- DJANGO_SETTINGS_MODULE=argus.site.settings.dockerdev | ||
- PYTHONPATH=/var/opt/extrapython | ||
- STATIC_ROOT=/static | ||
- DATABASE_URL=postgresql://argus:HahF9araeKoo@postgres/argus | ||
- ARGUS_REDIS_SERVER=redis | ||
depends_on: | ||
- postgres | ||
- redis | ||
volumes: | ||
- ${PWD}:/argus | ||
|
||
postgres: | ||
image: "postgres:14" | ||
volumes: | ||
- postgres:/var/lib/postgresql/data:Z | ||
environment: | ||
- POSTGRES_USER=argus | ||
- POSTGRES_PASSWORD=HahF9araeKoo | ||
- POSTGRES_DB=argus | ||
|
||
redis: | ||
image: "redis:latest" | ||
restart: always | ||
|
||
volumes: | ||
postgres: | ||
driver: local |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Defines a production image for the Argus API server | ||
# Needs the repository root directory as its context | ||
FROM python:3.10 | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
|
||
# Define an ARG variable | ||
ARG REPO_URL=https://github.com/Uninett/Argus.git | ||
ARG BRANCH_NAME=v1.27.0 | ||
ARG BASE_FOLDER=/var/opt | ||
|
||
RUN apt-get update && apt-get install -y --no-install-recommends tini build-essential | ||
|
||
# Install some dev requirements that aren't part of the minimal dependencies: | ||
RUN pip install psycopg2-binary django-extensions python-dotenv gunicorn 'uvicorn[standard]' | ||
|
||
# Make an unprivileged user to run the server | ||
RUN useradd --system argus | ||
# Ensure this user has privileges to collect static resources in /static | ||
RUN mkdir -p /static && chown argus /static | ||
ENV STATIC_ROOT=/static | ||
|
||
RUN git clone --depth 1 --branch ${BRANCH_NAME} ${REPO_URL} /src | ||
RUN pip install /src && rm -rf /src | ||
|
||
# Install API backend settings suitable for Docker deployment | ||
RUN mkdir /extrapython | ||
COPY docker/dockersettings.py /extrapython/ | ||
ENV PYTHONPATH=/extrapython | ||
ENV DJANGO_SETTINGS_MODULE=dockersettings | ||
|
||
ENV PORT=8000 | ||
EXPOSE 8000 | ||
COPY docker/docker-entrypoint.sh /api-entrypoint.sh | ||
USER argus | ||
ENTRYPOINT ["/usr/bin/tini", "-v", "--", "/api-entrypoint.sh"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,37 @@ | ||
# Docker production image of Argus | ||
|
||
Whereas the top-level Dockerfile is development-oriented, this directory | ||
contains definitions to build a production-oriented Docker image of the Argus | ||
API backend component. | ||
|
||
To build this image, the build context needs to be that of the git repository | ||
root. Something like this would work (when the current working directory is | ||
here): | ||
|
||
```shell | ||
docker build -t argus -f Dockerfile .. | ||
``` | ||
|
||
Or, from the top level directory: | ||
|
||
```shell | ||
docker build -t argus -f docker/Dockerfile . | ||
``` | ||
|
||
## Configuration of the running container | ||
|
||
This image runs with default production settings, with a few tweaks from | ||
[dockersettings.py](dockersettings.py). This means that the most useful | ||
settings can be overriden through the use of environment variables exported to | ||
the container. Consult the documentation section on [site-specific | ||
settings](http://argus-server.rtfd.io/en/latest/site-specific-settings.html). | ||
|
||
## Limitations | ||
|
||
This is not a complete Argus environment. This image only defines the backend | ||
API server component. It still depends on a PostgreSQL and a Redis server to be | ||
functional. Also, the Argus front-end component is needed to have a functional | ||
user interface against the API server. | ||
|
||
For a full production environment example, take a look at | ||
https://github.com/Uninett/argus-docker |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#!/bin/bash -e | ||
|
||
django-admin collectstatic --noinput | ||
django-admin migrate --noinput | ||
exec gunicorn \ | ||
argus.ws.asgi:application \ | ||
-k uvicorn.workers.UvicornWorker \ | ||
--forwarded-allow-ips="*" \ | ||
--access-logfile - \ | ||
-b 0.0.0.0:$PORT \ | ||
$@ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
"""Settings for production-like deployments in Docker""" | ||
|
||
from argus.site.settings.prod import * | ||
|
||
# Allow all hosts to reach backend, since all requests will typically come from | ||
# outside the container: | ||
ALLOWED_HOSTS = ["*"] | ||
|
||
# Uncomment to enable both Email and SMS-as-email notification backends, | ||
# leave commented out to keep just the default email backend: | ||
# MEDIA_PLUGINS = [ | ||
# "argus.notificationprofile.media.email.EmailNotification", | ||
# "argus.notificationprofile.media.sms_as_email.SMSNotification", | ||
# ] | ||
|
||
ARGUS_SEND_NOTIFICATIONS = True | ||
DEFAULT_SMS_MEDIA="argus.notificationprofile.media.sms_as_email.SMSNotification" | ||
EMAIL_BACKEND = "django.core.mail.backends.smtp.EmailBackend" | ||
CSRF_TRUSTED_ORIGINS = ['https://*.sunet.se','https://*.127.0.0.1','https://*.sunet.se:9000'] | ||
CORS_ORIGIN_WHITELIST = ['https://*sunet.se','https://*.sunet.se:9000'] | ||
|
||
MEDIA_PLUGINS = [ | ||
"argus.notificationprofile.media.email.EmailNotification", | ||
"argus.notificationprofile.media.sms_as_email.SMSNotification", | ||
] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
--- | ||
# Documentation: https://github.com/SUNET/bootstrap-docker-builds | ||
git: | ||
branch: main | ||
triggers: | ||
github_push: true | ||
clean_workspace: true | ||
pre_build_script: | ||
- cp docker/Dockerfile . | ||
script: | ||
- echo "Build SUNET Custom ARGUS Image" | ||
builders: | ||
- docker | ||
docker_name: sunet/argus-server | ||
docker_tags: | ||
- "{{ docker_tag_name }}" | ||
docker_force_tag: true | ||
docker_no_cache: true | ||
docker_skip_tag_as_latest: false | ||
docker_build_args: | | ||
BRANCH_NAME={{ branch_name }} | ||
REPO_URL={{ repository_url }} |
Oops, something went wrong.