Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spawn reactive runner #8

Merged
merged 21 commits into from
Sep 19, 2024
Merged

Spawn reactive runner #8

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
03a1718
add public interface
cbartz Sep 11, 2024
b2ee88b
WIP checkin
cbartz Sep 12, 2024
62092fa
change reactive.runner_manager.reconcile signature
cbartz Sep 13, 2024
a606b48
Merge branch 'main' into feat/reactive-spawning
cbartz Sep 13, 2024
6f6a3b2
move reactive types into own module
cbartz Sep 13, 2024
fd91b02
refactor config and use it in script
cbartz Sep 13, 2024
6e0875d
WIP checkin unit tests
cbartz Sep 13, 2024
5e4218b
implement spawning a runner
cbartz Sep 16, 2024
b879f4a
lint
cbartz Sep 16, 2024
821bfb4
Add unit tests for github_client
cbartz Sep 16, 2024
cab8ee5
add cleanup
cbartz Sep 17, 2024
63077e8
write get_job_info method
cbartz Sep 17, 2024
bd0b1a3
rename run_url -> job_url
cbartz Sep 17, 2024
af162a5
Fix parsing of job_url
cbartz Sep 17, 2024
ae8a520
validate path of job_url
cbartz Sep 18, 2024
237ad25
rename reactive_config -> runner_config
cbartz Sep 18, 2024
7739291
Merge branch 'main' into feat/reactive-spawning
cbartz Sep 18, 2024
e78f285
rename manager_name -> name
cbartz Sep 18, 2024
71d2ec2
add pymongo depedency
cbartz Sep 18, 2024
132f748
bump version in pyproject.toml
cbartz Sep 18, 2024
d72fa72
Merge branch 'main' into feat/reactive-spawning
cbartz Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

[project]
name = "github-runner-manager"
version = "0.1.5"
version = "0.2.0"
authors = [
{ name = "Canonical IS DevOps", email = "[email protected]" },
]
Expand Down Expand Up @@ -46,7 +46,7 @@ omit = [
]

[tool.coverage.report]
fail_under = 79
fail_under = 80
show_missing = true


Expand Down
1 change: 1 addition & 0 deletions requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ jinja2
kombu==5.4.1
openstacksdk>=3,<4
pydantic < 2
pymongo==4.8.0
34 changes: 29 additions & 5 deletions src-docs/github_client.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,19 +81,43 @@ Delete the self-hosted runner from GitHub.

---

<a href="../src/github_runner_manager/github_client.py#L203"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/github_client.py#L244"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `get_job_info`

```python
get_job_info(
get_job_info(path: GitHubRepo, job_id: str) → JobInfo
```

Get information about a job identified by the job id.



**Args:**

- <b>`path`</b>: GitHub repository path in the format '<owner>/<repo>'.
- <b>`job_id`</b>: The job id.



**Returns:**
The JSON response from the API.

---

<a href="../src/github_runner_manager/github_client.py#L203"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `get_job_info_by_runner_name`

```python
get_job_info_by_runner_name(
path: GitHubRepo,
workflow_run_id: str,
runner_name: str
) → JobStats
) → JobInfo
```

Get information about a job for a specific workflow run.
Get information about a job for a specific workflow run identified by the runner name.



Expand All @@ -107,7 +131,7 @@ Get information about a job for a specific workflow run.

**Raises:**

- <b>`TokenError`</b>: if there was an error with the Github token crdential provided.
- <b>`TokenError`</b>: if there was an error with the Github token credential provided.
- <b>`JobNotFoundError`</b>: If no jobs were found.


Expand Down
14 changes: 5 additions & 9 deletions src-docs/manager.runner_manager.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ Configuration for the runner manager.

**Attributes:**

- <b>`name`</b>: A name to identify this manager.
- <b>`token`</b>: GitHub personal access token to query GitHub API.
- <b>`path`</b>: Path to GitHub repository or organization to registry the runners.

Expand All @@ -85,7 +86,7 @@ Configuration for the runner manager.
### <kbd>method</kbd> `__init__`

```python
__init__(token: str, path: GitHubOrg | GitHubRepo) → None
__init__(name: str, token: str, path: GitHubOrg | GitHubRepo) → None
```


Expand All @@ -98,7 +99,7 @@ __init__(token: str, path: GitHubOrg | GitHubRepo) → None

---

<a href="../src/github_runner_manager/manager/runner_manager.py#L94"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/manager/runner_manager.py#L96"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

## <kbd>class</kbd> `RunnerManager`
Manage the runners.
Expand All @@ -110,16 +111,12 @@ Manage the runners.
- <b>`manager_name`</b>: A name to identify this manager.
- <b>`name_prefix`</b>: The name prefix of the runners.

<a href="../src/github_runner_manager/manager/runner_manager.py#L102"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/manager/runner_manager.py#L104"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `__init__`

```python
__init__(
manager_name: str,
cloud_runner_manager: CloudRunnerManager,
config: RunnerManagerConfig
)
__init__(cloud_runner_manager: CloudRunnerManager, config: RunnerManagerConfig)
```

Construct the object.
Expand All @@ -128,7 +125,6 @@ Construct the object.

**Args:**

- <b>`manager_name`</b>: A name to identify this manager.
- <b>`cloud_runner_manager`</b>: For managing the cloud instance of the runner.
- <b>`config`</b>: Configuration of this class.

Expand Down
13 changes: 8 additions & 5 deletions src-docs/manager.runner_scaler.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Module for scaling the runners amount.

---

<a href="../src/github_runner_manager/manager/runner_scaler.py#L28"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/manager/runner_scaler.py#L26"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

## <kbd>class</kbd> `RunnerInfo`
Information on the runners.
Expand Down Expand Up @@ -50,17 +50,20 @@ __init__(

---

<a href="../src/github_runner_manager/manager/runner_scaler.py#L49"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/manager/runner_scaler.py#L47"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

## <kbd>class</kbd> `RunnerScaler`
Manage the reconcile of runners.

<a href="../src/github_runner_manager/manager/runner_scaler.py#L52"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/manager/runner_scaler.py#L50"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `__init__`

```python
__init__(runner_manager: RunnerManager, reactive_config: ReactiveConfig | None)
__init__(
runner_manager: RunnerManager,
reactive_runner_config: RunnerConfig | None
)
```

Construct the object.
Expand All @@ -70,7 +73,7 @@ Construct the object.
**Args:**

- <b>`runner_manager`</b>: The RunnerManager to perform runner reconcile.
- <b>`reactive_config`</b>: Reactive runner configuration.
- <b>`reactive_runner_config`</b>: Reactive runner configuration.



Expand Down
65 changes: 47 additions & 18 deletions src-docs/openstack_cloud.openstack_runner_manager.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,24 +78,29 @@ __init__(image: str, flavor: str, network: str) → None

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L114"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L101"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

## <kbd>class</kbd> `OpenStackRunnerManager`
Manage self-hosted runner on OpenStack cloud.
## <kbd>class</kbd> `OpenStackRunnerManagerConfig`
Configuration for OpenStack runner manager.



**Attributes:**

- <b>`name_prefix`</b>: The name prefix of the runners created.
- <b>`name`</b>: The name of the manager.
- <b>`prefix`</b>: The prefix of the runner names.
- <b>`cloud_config`</b>: The configuration for OpenStack cloud.
- <b>`server_config`</b>: The configuration for OpenStack server.
- <b>`runner_config`</b>: The configuration for the GitHub runner.
- <b>`service_config`</b>: The configuration for supporting services.

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L122"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../<string>"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `__init__`

```python
__init__(
manager_name: str,
name: str,
prefix: str,
cloud_config: OpenStackCloudConfig,
server_config: OpenStackServerConfig | None,
Expand All @@ -104,18 +109,42 @@ __init__(
) → None
```









---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L135"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

## <kbd>class</kbd> `OpenStackRunnerManager`
Manage self-hosted runner on OpenStack cloud.



**Attributes:**

- <b>`name_prefix`</b>: The name prefix of the runners created.

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L142"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `__init__`

```python
__init__(config: OpenStackRunnerManagerConfig) → None
```

Construct the object.



**Args:**

- <b>`manager_name`</b>: A name to identify this manager.
- <b>`prefix`</b>: The prefix to runner name.
- <b>`cloud_config`</b>: The configuration for OpenStack authorisation.
- <b>`server_config`</b>: The configuration for creating OpenStack server. Unable to create runner if None.
- <b>`runner_config`</b>: The configuration for the runner.
- <b>`service_config`</b>: The configuration of supporting services of the runners.
- <b>`config`</b>: The configuration for the openstack runner manager.


---
Expand All @@ -133,7 +162,7 @@ The prefix of runner names.

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L322"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L354"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `cleanup`

Expand All @@ -156,7 +185,7 @@ Cleanup runner and resource on the cloud.

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L172"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L181"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `create_runner`

Expand Down Expand Up @@ -186,7 +215,7 @@ Create a self-hosted runner.

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L271"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L289"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `delete_runner`

Expand All @@ -210,7 +239,7 @@ Delete self-hosted runners.

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L297"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L323"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `flush_runners`

Expand All @@ -233,7 +262,7 @@ Remove idle and/or busy runners.

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L217"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L230"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `get_runner`

Expand All @@ -256,7 +285,7 @@ Get a self-hosted runner by instance id.

---

<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L239"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
<a href="../src/github_runner_manager/openstack_cloud/openstack_runner_manager.py#L257"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>

### <kbd>method</kbd> `get_runners`

Expand Down
Loading
Loading