Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add imagefactory docker compose example #58

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions examples/imagefactory/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
.env
keys/*
schematics/*
99 changes: 99 additions & 0 deletions examples/imagefactory/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Siderolabs self-hosted imagefactory example

This code runs [sidero imagefactory](https://github.com/siderolabs/image-factory) in [docker compose](https://docs.docker.com/compose/).

It also deploys a few companion components:
* upstream `ghcr.io` [registry](https://distribution.github.io/distribution/) [mirror](https://distribution.github.io/distribution/recipes/mirror/) to avoid potential upstream rate limitings and speed up builds by caching previously pulled image layers
* a script that applies prepared `talos` [image schematics](https://www.talos.dev/v1.8/learn-more/image-factory/#schematics)
* a [registry](https://distribution.github.io/distribution/) used as storage and cache for the generated images

# how to use

tested with `talos` version `v1.8.1`, via [iPXE boot](https://www.talos.dev/v1.8/talos-guides/install/bare-metal-platforms/pxe/) and directly applying generated images to disk via `hcloud` (use it's [packer](https://developer.hashicorp.com/packer/integrations/hetznercloud/hcloud) integration and point [this](https://github.com/siderolabs/contrib/blob/9cd1e1c9d2469b77d2278eb07e7f61c09bb32d40/examples/terraform/hcloud/packer/hcloud_talosimage.pkr.hcl#L18) URL to your `imagefactory` instance).

## preparation

some preparation is required.

### signing keys
see [official docs](https://github.com/siderolabs/image-factory?tab=readme-ov-file#development).

```shell
mkdir -pv keys
openssl ecparam -name prime256v1 -genkey -noout -out keys/cache-signing-key.key
```

### schematics

Refer to the [official docs](https://www.talos.dev/v1.8/learn-more/image-factory/#schematics) on how to create these.
The [script](./scripts/sync-schematics.sh) will find and apply all files in `schematics/*.yaml`.

Example:
```yaml
# schematics/example.yaml
customization:
extraKernelArgs:
- gfxmode=1280x1024
- console=ttyS0,115200
- net.ifnames=0
- talos.platform=metal
systemExtensions:
officialExtensions:
- siderolabs/amd-ucode
- siderolabs/fuse3
- siderolabs/intel-ucode
- siderolabs/iscsi-tools
- siderolabs/qemu-guest-agent
- siderolabs/tailscale
- siderolabs/util-linux-tools
meta:
- key: 12
value: |
machineLabels:
env: prod
type: controlplane
```

### environment variables

copy the [docker compose env file](https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/#env-file) and adjust the example values.

```shell
cp env.example .env
vim .env
```

Adjust all domains and `EXT_IP` to where you want to expose your `imagefactory` instance.
This is relevant for payloads sent to `iPXE` clients and URLs generated in the UI.

## run

after preparation is done, run `docker compose up -d`.

# miscellaneous & troubleshooting

This is a community contribution so expect no official support.
Some trouble I ran into:

## TLS

The configuration used does *not* deploy TLS, so you should put this behind something like a reverse proxy that does.

## connection timeouts

Image generation can take some time, so clients might have to increase their connection timeout limits. If images are cached, [TTFB](https://en.wikipedia.org/wiki/Time_to_first_byte) is very short, if not `TTFB` can take up to several minutes.

## iPXE and https

If you want to [iPXE](https://ipxe.org)-boot from this via `https`, keep in mind that by default `iPXE` does *not* support `https` and you need to compile your own, enabling [this](https://ipxe.org/buildcfg/download_proto_https) flag. This is a pitfall for reverse proxies that automatically redirect plaintext `http` requests to `https`.

## URLs not working

There is a tiny problem in the `imagefactory` frontend: The URLs generated contain the external domain used and it is duplicated for some reason. This is particularly mean because the URL _visible_ in the UI looks correct, but the `HTML` `href` is not.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this should be fixed is the proper URL (with http:// is passed)

Make sure to sanitize the URL before use, the resulting URL works as expected. Not sure yet as to _why_ this happens (misconfiguration or might be a bug).
Querying the `API` seems to return the correct URL.

## registry resource consumption

* When building a large number of images, make sure to provide sufficient storage to the `registry` container and monitor `docker volumes` as it grows in size quite rapidly.
* the image generation process is compute heavy and can take some time, depending on the compute power available.
62 changes: 62 additions & 0 deletions examples/imagefactory/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
services:
# generated images are pushed to this registry
registry:
image: registry:2
ports:
- ${EXTERNAL_IP:-127.0.0.1}:5000:5000
volumes:
# hint: when generating a large number of different schemas, this volume can grow quite large
- registry:/var/lib/registry:rw
# upstream ghcr mirror, caches previously pulled image layers and prevents rate limiting
registry-ghcr:
image: registry:2
environment:
REGISTRY_PROXY_REMOTEURL: http://ghcr.io
volumes:
- registry-ghcr:/var/lib/registry:rw
# triggers image builds for all schematics defined in `./schematics/*.yaml`
schematics:
image: alpine:3
environment:
IMAGE_FACTORY_URL: http://imagefactory:6000
REGISTRY_URL: registry:5000
TALOS_VERSION: ${SCHEMATICS_TALOS_VERSION?}
ARCH: ${SCHEMATICS_TALOS_ARCH?}
VALIDATE: ${SCHEMATICS_VALIDATE?}
SLEEP_TIME: ${SCHEMATICS_SLEEP_TIME?}
command: >
/scripts/sync-schematics.sh
volumes:
- ${PWD}/scripts:/scripts:ro
- ${PWD}/schematics:/schematics:ro
# container running the actual imagefactory
imagefactory:
image: ghcr.io/siderolabs/image-factory:${IMGFAC_VERSION?}
# required for losetup
privileged: true
volumes:
- ${PWD}/keys:/keys:ro
# required for losetup
- /dev:/dev
ports:
- ${EXTERNAL_IP:-127.0.0.1}:6000:6000
command: >
-http-port=:6000
-external-url=http://${IMGFAC_EXTERNAL_URL?}
-external-pxe-url=http://${IMGFAC_EXTERNAL_URL?}
-cache-signing-key-path=/keys/cache-signing-key.key
-cache-repository=registry:5000/cache
-insecure-cache-repository=true
-image-registry=registry-ghcr:5000
-insecure-image-registry=true
-schematic-service-repository=registry:5000/image-factory/schematic
-insecure-schematic-service-repository
-installer-internal-repository=registry:5000/siderolabs
-insecure-installer-internal-repository=true
-installer-external-repository=${IMGFAC_EXT_REPO?}/siderolabs
-secureboot=${IMGFAC_SECUREBOOT?}

volumes:
registry:
registry-tls:
registry-ghcr:
11 changes: 11 additions & 0 deletions examples/imagefactory/env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
EXTERNAL_IP=127.0.0.1

SCHEMATICS_TALOS_VERSION=1.8.1
SCHEMATICS_TALOS_ARCH=amd64
SCHEMATICS_VALIDATE=false
SCHEMATICS_SLEEP_TIME=600

IMGFAC_VERSION=v0.5.0
IMGFAC_EXTERNAL_URL=imgfac.example.com
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a proper URL (when passed as an argument), this might be the reason the UI doesn't work properly, e.g. http://imgfac.example.com/

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd argue since no reverse-proxy is part of this scenario, we can assume it is always http.

See 9d937af

IMGFAC_EXT_REPO=reg.imgfac.example.com
IMGFAC_SECUREBOOT=false
70 changes: 70 additions & 0 deletions examples/imagefactory/scripts/sync-schematics.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!/bin/ash
set -eo pipefail

trap "exit 0" SIGINT SIGTERM

: ${IMAGE_FACTORY_URL:?}
: ${REGISTRY_URL:?}
: ${TALOS_VERSION:?}
: ${ARCH:?}
: ${VALIDATE:?}
: ${SLEEP_TIME:?}

apk add crane yq

RESULTS_FILE="${RESULTS_FILE:-/tmp/results}"
while true; do
echo '' > "${RESULTS_FILE}"
for SCHEMATIC in /schematics/*.yaml ; do
# this triggers image generation based on the schema provided
# docs: https://github.com/siderolabs/image-factory?tab=readme-ov-file#post-schematics
echo "apply ${SCHEMATIC}"
RESPONSE_FILE=/tmp/wget-response.json
wget \
--header 'Content-Type: application/yaml' \
-O "${RESPONSE_FILE}" \
--post-file=${SCHEMATIC} \
${IMAGE_FACTORY_URL}/schematics \

# parse the image ID from the response
SCHEMA_ID=$(yq .id < "${RESPONSE_FILE}")
if test -z "${SCHEMA_ID}" ; then
echo 'SCHEMA_ID was empty'
exit 1
fi
TMP_FILE="/tmp/${SCHEMA_ID}.tar"
rm "${RESPONSE_FILE}"

# docs: https://github.com/siderolabs/image-factory?tab=readme-ov-file#get-imageschematicversionpath
echo 'download container'
wget \
-O ${TMP_FILE} \
${IMAGE_FACTORY_URL}/image/${SCHEMA_ID}/${TALOS_VERSION}/installer-${ARCH}.tar

# optional: this calls `crane validate <image>`, validating the generated image is well formed
# docs: https://github.com/google/go-containerregistry/blob/main/cmd/crane/doc/crane_validate.md
if [ "${VALIDATE}" == 'true' ] ; then
echo 'validate container'
crane validate --tarball ${TMP_FILE}
fi

echo 'publish container'
crane push \
--insecure \
${TMP_FILE} \
${REGISTRY_URL}/installer/${SCHEMA_ID}:${TALOS_VERSION}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why this is needed, Image Factory itself can work as a container registry (to pull installer images from)?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really remember to be honest. I wrote this code some while ago and I wasn't aware the registry I added here is actually not needed, I thought imagefactory needs this as a companion, it made sense in my head I guess.

Do you want me to remove the external registry (called registry in docker-compose.yaml) and related code?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This registry should be there, but there's no need to push anything there. All I'm asking is to remove this file completely, as it's not needed


rm -v ${TMP_FILE}
echo "${SCHEMATIC} ${SCHEMA_ID}" >> "${RESULTS_FILE}"
done

# this prints the image IDs resulting from each schema,
# which can then be handed out to clients.
echo "--- results ---"
cat "${RESULTS_FILE}"
echo "---------------"

echo "all done, sleep ${SLEEP_TIME} sec."
sleep ${SLEEP_TIME} &
wait $!
done