Skip to content

Commit

Permalink
Merge 539b522 into 4f1d2ea
Browse files Browse the repository at this point in the history
  • Loading branch information
yanksyoon committed Apr 11, 2024
2 parents 4f1d2ea + 539b522 commit a70b7fd
Show file tree
Hide file tree
Showing 19 changed files with 614 additions and 132 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/integration_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
pre-run-script: scripts/pre-integration-test.sh
provider: lxd
test-tox-env: integration-juju2.9
modules: '["test_charm_fork_repo", "test_charm_no_runner", "test_charm_scheduled_events", "test_charm_one_runner", "test_charm_metrics_success", "test_charm_metrics_failure", "test_self_hosted_runner", "test_charm_with_proxy", "test_charm_with_juju_storage", "test_debug_ssh"]'
modules: '["test_charm_base_image", "test_charm_fork_repo", "test_charm_no_runner", "test_charm_scheduled_events", "test_charm_one_runner", "test_charm_metrics_success", "test_charm_metrics_failure", "test_self_hosted_runner", "test_charm_with_proxy", "test_charm_with_juju_storage", "test_debug_ssh"]'
integration-tests:
name: Integration test with juju 3.1
uses: canonical/operator-workflows/.github/workflows/integration_test.yaml@main
Expand All @@ -25,7 +25,7 @@ jobs:
pre-run-script: scripts/pre-integration-test.sh
provider: lxd
test-tox-env: integration-juju3.1
modules: '["test_charm_fork_repo", "test_charm_no_runner", "test_charm_scheduled_events", "test_charm_one_runner", "test_charm_metrics_success", "test_charm_metrics_failure", "test_self_hosted_runner", "test_charm_with_proxy", "test_charm_with_juju_storage", "test_debug_ssh"]'
modules: '["test_charm_base_image", "test_charm_fork_repo", "test_charm_no_runner", "test_charm_scheduled_events", "test_charm_one_runner", "test_charm_metrics_success", "test_charm_metrics_failure", "test_self_hosted_runner", "test_charm_with_proxy", "test_charm_with_juju_storage", "test_debug_ssh"]'
# openstack tests use microstack, whose setup is kind of special
# - due to the huge resource requirements, we use self-hosted runners for these tests
# - microstack requires juju 3.2 and microk8s 1.26
Expand All @@ -42,6 +42,6 @@ jobs:
channel: 1.26-strict/stable
microk8s-addons: "dns ingress hostpath-storage"
test-tox-env: integration-juju3.2
modules: '["test_openstack"]'
modules: '["test_openstack_base_image", "test_openstack_one_runner"]'
self-hosted-runner: true
self-hosted-runner-label: two-xlarge
7 changes: 7 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@
# See LICENSE file for licensing details.

options:
base-image:
type: string
default: "jammy"
description: >-
The base ubuntu OS image to use for the runners. Codename (e.g. "jammy") or version tag
(e.g. 22.04) is supported as input. Currently only supports LTS versions of jammy and higher,
i.e. jammy, noble.
denylist:
type: string
default: ""
Expand Down
14 changes: 14 additions & 0 deletions docs/how-to/set-base-image.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# How to set base image

This charm supports deploying the runners on different base images.

By using [`juju config`](https://juju.is/docs/juju/juju-config) to change the
[charm configuration labels](https://charmhub.io/github-runner/configure#base-image), the runner
can be deployed with a different Ubuntu base image. Latest two LTS images "jammy" and "noble" are
supported. The default base image is "jammy".

```shell
juju config <APP_NAME> base-image=<BASE_IMAGE_TAG_OR_NAME>
```

An example of a BASE_IMAGE_TAG_OR_NAME value would be "jammy", "22.04", "noble", "24.04".
14 changes: 7 additions & 7 deletions scripts/build-lxd-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ cleanup() {
HTTP_PROXY="$1"
HTTPS_PROXY="$2"
NO_PROXY="$3"
MODE="$4"
BASE_IMAGE="$4"
MODE="$5"

if [[ -n "$HTTP_PROXY" ]]; then
/snap/bin/lxc config set core.proxy_http "$HTTP_PROXY"
Expand All @@ -63,9 +64,9 @@ fi
cleanup '/snap/bin/lxc info builder &> /dev/null' '/snap/bin/lxc delete builder --force' 'Cleanup LXD VM of previous run' 10

if [[ "$MODE" == "test" ]]; then
retry '/snap/bin/lxc launch ubuntu-daily:jammy builder --device root,size=5GiB' 'Starting LXD container'
retry "/snap/bin/lxc launch ubuntu-daily:$BASE_IMAGE builder --device root,size=5GiB" 'Starting LXD container'
else
retry '/snap/bin/lxc launch ubuntu-daily:jammy builder --vm --device root,size=8GiB' 'Starting LXD VM'
retry "/snap/bin/lxc launch ubuntu-daily:$BASE_IMAGE builder --vm --device root,size=8GiB" 'Starting LXD VM'
fi
retry '/snap/bin/lxc exec builder -- /usr/bin/who' 'Wait for lxd agent to be ready' 30
if [[ -n "$HTTP_PROXY" ]]; then
Expand All @@ -86,7 +87,6 @@ retry '/snap/bin/lxc exec builder -- /usr/bin/nslookup github.com' 'Wait for net

/snap/bin/lxc exec builder -- /usr/bin/apt-get update
/snap/bin/lxc exec builder --env DEBIAN_FRONTEND=noninteractive -- /usr/bin/apt-get upgrade -yq
/snap/bin/lxc exec builder --env DEBIAN_FRONTEND=noninteractive -- /usr/bin/apt-get install linux-generic-hwe-22.04 -yq
# This will remove older version of kernel as HWE is installed now.
/snap/bin/lxc exec builder -- /usr/bin/apt-get autoremove --purge

Expand Down Expand Up @@ -146,9 +146,9 @@ fi
/snap/bin/lxc publish builder --alias builder --reuse -f

# Swap in the built image
/snap/bin/lxc image alias rename jammy old-jammy || true
/snap/bin/lxc image alias rename builder jammy
/snap/bin/lxc image delete old-jammy || true
/snap/bin/lxc image alias rename "$BASE_IMAGE" "old-$BASE_IMAGE" || true
/snap/bin/lxc image alias rename builder "$BASE_IMAGE"
/snap/bin/lxc image delete "old-$BASE_IMAGE" || true

# Clean up LXD instance
cleanup '/snap/bin/lxc info builder &> /dev/null' '/snap/bin/lxc delete builder --force' 'Cleanup LXD instance' 10
11 changes: 6 additions & 5 deletions scripts/build-openstack-image.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ HTTPS_PROXY="$3"
NO_PROXY="$4"
DOCKER_PROXY_SERVICE_CONF="$5"
DOCKER_PROXY_CONF="$6"
BASE_IMAGE="$7"

# retry function
retry() {
Expand Down Expand Up @@ -108,15 +109,15 @@ sudo modprobe nbd
# cleanup any existing mounts
cleanup

retry "sudo wget https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-$BIN_ARCH.img \
-O jammy-server-cloudimg-$BIN_ARCH.img" "Downloading cloud image" 3
retry "sudo wget https://cloud-images.ubuntu.com/$BASE_IMAGE/current/$BASE_IMAGE-server-cloudimg-$BIN_ARCH.img \
-O $BASE_IMAGE-server-cloudimg-$BIN_ARCH.img" "Downloading cloud image" 3

# resize image - installing dependencies requires more disk space
sudo qemu-img resize jammy-server-cloudimg-$BIN_ARCH.img +1.5G
sudo qemu-img resize "$BASE_IMAGE-server-cloudimg-$BIN_ARCH.img" +1.5G

# mount nbd
echo "Connecting network block device to image"
sudo qemu-nbd --connect=/dev/nbd0 jammy-server-cloudimg-$BIN_ARCH.img
sudo qemu-nbd --connect=/dev/nbd0 "$BASE_IMAGE-server-cloudimg-$BIN_ARCH.img"
sudo mkdir -p /mnt/ubuntu-image
retry "sudo mount -o rw /dev/nbd0p1 /mnt/ubuntu-image" "Mounting nbd0p1 device" 3

Expand Down Expand Up @@ -208,4 +209,4 @@ sudo sync
cleanup

# Reduce image size by removing sparse space & compressing
sudo virt-sparsify --compress jammy-server-cloudimg-$BIN_ARCH.img jammy-server-cloudimg-$BIN_ARCH-compressed.img
sudo virt-sparsify --compress "$BASE_IMAGE-server-cloudimg-$BIN_ARCH.img" "$BASE_IMAGE-server-cloudimg-$BIN_ARCH-compressed.img"
74 changes: 59 additions & 15 deletions src-docs/charm_state.py.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ State of the Charm.
---------------
- **ARCHITECTURES_ARM64**
- **ARCHITECTURES_X86**
- **BASE_IMAGE_CONFIG_NAME**
- **DENYLIST_CONFIG_NAME**
- **DOCKERHUB_MIRROR_CONFIG_NAME**
- **GROUP_CONFIG_NAME**
Expand All @@ -26,10 +27,11 @@ State of the Charm.
- **LABELS_CONFIG_NAME**
- **COS_AGENT_INTEGRATION_NAME**
- **DEBUG_SSH_INTEGRATION_NAME**
- **LTS_IMAGE_VERSION_TAG_MAP**

---

<a href="../src/charm_state.py#L100"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L101"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

## <kbd>function</kbd> `parse_github_path`

Expand Down Expand Up @@ -74,6 +76,22 @@ Supported system architectures.



---

## <kbd>class</kbd> `BaseImage`
The ubuntu OS base image to build and deploy runners on.



**Attributes:**

- <b>`JAMMY`</b>: The jammy ubuntu LTS image.
- <b>`NOBLE`</b>: The noble ubuntu LTS image.





---

## <kbd>class</kbd> `CharmConfig`
Expand All @@ -98,7 +116,7 @@ Some charm configurations are grouped into other configuration models.

---

<a href="../src/charm_state.py#L418"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L419"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `check_reconcile_interval`

Expand Down Expand Up @@ -127,7 +145,7 @@ Validate the general charm configuration.

---

<a href="../src/charm_state.py#L374"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L375"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `from_charm`

Expand Down Expand Up @@ -166,7 +184,7 @@ Raised when charm config is invalid.

- <b>`msg`</b>: Explanation of the error.

<a href="../src/charm_state.py#L212"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L213"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>function</kbd> `__init__`

Expand Down Expand Up @@ -207,7 +225,7 @@ The charm state.

---

<a href="../src/charm_state.py#L788"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L855"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `from_charm`

Expand Down Expand Up @@ -252,7 +270,7 @@ Charm configuration related to GitHub.

---

<a href="../src/charm_state.py#L136"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L137"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `from_charm`

Expand Down Expand Up @@ -297,7 +315,7 @@ Represent GitHub organization.

---

<a href="../src/charm_state.py#L88"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L89"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>function</kbd> `path`

Expand Down Expand Up @@ -330,7 +348,7 @@ Represent GitHub repository.

---

<a href="../src/charm_state.py#L67"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L68"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>function</kbd> `path`

Expand All @@ -346,6 +364,31 @@ Return a string representing the path.
Path to the GitHub entity.


---

## <kbd>class</kbd> `ImmutableConfigChangedError`
Represents an error when changing immutable charm state.

<a href="../src/charm_state.py#L787"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>function</kbd> `__init__`

```python
__init__(msg: str)
```

Initialize a new instance of the ImmutableConfigChangedError exception.



**Args:**

- <b>`msg`</b>: Explanation of the error.





---

## <kbd>class</kbd> `ProxyConfig`
Expand All @@ -372,7 +415,7 @@ Return the aproxy address.

---

<a href="../src/charm_state.py#L643"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L659"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `check_use_aproxy`

Expand Down Expand Up @@ -402,7 +445,7 @@ Validate the proxy configuration.

---

<a href="../src/charm_state.py#L605"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L621"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `from_charm`

Expand Down Expand Up @@ -433,6 +476,7 @@ Runner configurations for the charm.

**Attributes:**

- <b>`base_image`</b>: The ubuntu base image to run the runner virtual machines on.
- <b>`virtual_machines`</b>: Number of virtual machine-based runner to spawn.
- <b>`virtual_machine_resources`</b>: Hardware resource used by one virtual machine for a runner.
- <b>`runner_storage`</b>: Storage to be used as disk for the runner.
Expand All @@ -442,7 +486,7 @@ Runner configurations for the charm.

---

<a href="../src/charm_state.py#L556"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L572"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `check_virtual_machine_resources`

Expand Down Expand Up @@ -473,7 +517,7 @@ Validate the virtual_machine_resources field values.

---

<a href="../src/charm_state.py#L534"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L550"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `check_virtual_machines`

Expand Down Expand Up @@ -502,7 +546,7 @@ Validate the virtual machines configuration value.

---

<a href="../src/charm_state.py#L489"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L500"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `from_charm`

Expand Down Expand Up @@ -565,7 +609,7 @@ SSH connection information for debug workflow.

---

<a href="../src/charm_state.py#L731"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L747"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>classmethod</kbd> `from_charm`

Expand Down Expand Up @@ -598,7 +642,7 @@ Raised when given machine charm architecture is unsupported.

- <b>`arch`</b>: The current machine architecture.

<a href="../src/charm_state.py#L688"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/charm_state.py#L704"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>function</kbd> `__init__`

Expand Down
Loading

0 comments on commit a70b7fd

Please sign in to comment.