Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: make ref checkout for picasso scripts configurable #37

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,11 @@ on:
required: false
type: string
default: '3.12'
PICASSO_VERSION:
BryanttV marked this conversation as resolved.
Show resolved Hide resolved
description: 'Picasso version to use for the workflow scripts and utility functions. This should be a valid branch, tag or commit and it should match the version of the workflow used.'
required: false
type: string
default: 'v1'
secrets:
DOCKERHUB_USERNAME:
description: 'DockerHub username for login'
Expand Down Expand Up @@ -70,7 +75,7 @@ jobs:
uses: actions/checkout@v4
with:
repository: edunext/picasso
ref: v1
ref: ${{ inputs.PICASSO_VERSION }}
path: picasso

- name: Checkout strains repository for build configurations
Expand Down
12 changes: 12 additions & 0 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,18 @@ Before using the workflow, ensure that you have set up the following configurati
- Enables limiting parallelism with buildkit to decrease resource consumption for those setups with low-powered machines. Default is ``true``.
- boolean
- Input
* - RUNNER_WORKFLOW_LABEL (Optional)
- The label of the runner workflow to use. Default is ``ubuntu-24.04``.
- string
- Input
* - PYTHON_VERSION (Optional)
- The Python version to use in the workflow. Default is ``3.12``.
- string
- Input
* - PICASSO_VERSION (Optional)
- Picasso version to use for the workflow scripts and utility functions. This should be a valid branch, tag or commit and it should match the version of the workflow used. Default is the latest release major version, e.g., ``v1``.
- string
- Input

Usage
*****
Expand Down
8 changes: 8 additions & 0 deletions docs/source/how_tos/configure_workflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ Inputs are the parameters that you can set to customize the behavior of the Pica
* ``STRAIN_PATH (required)``: The path to the directory that contains the strain configuration file. This path should be relative to the root of the repository.
* ``SERVICE (required)``: The name of the service that will be built. This service should be supported by Tutor or by a tutor plugin previously installed.
* ``ENABLE_LIMIT_BUILDKIT_PARALLELISM (optional)``: If set to ``true``, the build process parallel steps will be limited by 3, which is the threshold found where both Open edX and MFE images are built without running out of resources in the Github Actions runner (please, see `PR #12`_ for more details) . If set to ``false``, the buildkit configuration default will be used. Default is ``true``. Set to ``false`` if you have a runner with more resources.
* ``RUNNER_WORKFLOW_LABEL (optional)``: This label should match the label of the runner that will execute the workflow. By default, the workflow will use one of the `Github Actions standard runners`_, e.g., ``ubuntu-24.04``.
* ``PYTHON_VERSION (optional)``: The Python version to use in the workflow. By default, the workflow will use Python 3.12.
* ``PICASSO_VERSION (optional)``: Picasso version to use for the workflow scripts and utility functions. This should be a valid branch, tag or commit and it should match the version of the workflow used. By default, the workflow will use the latest release major version, e.g., ``v1``.

These inputs can be set in the workflow file that calls the Picasso workflow using the ``with`` keyword, by manually setting them in the workflow file, or by using the ``workflow_dispatch`` event. For more details on how to set input values in Github Actions, please refer to the `Workflow syntax for GitHub Actions`_ documentation.

Expand All @@ -34,6 +37,9 @@ Here is an example of how to use the Picasso workflow with the inputs set:
STRAIN_PATH: redwood/base
SERVICE: mfe
ENABLE_LIMIT_BUILDKIT_PARALLELISM: false
RUNNER_WORKFLOW_LABEL: ubuntu-latest

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we include these inputs in the README?

PYTHON_VERSION: 3.10
PICASSO_VERSION: main

.. _`PR #12`: https://github.com/eduNEXT/picasso/pull/12

Expand Down Expand Up @@ -93,3 +99,5 @@ Or using an AWS registry:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
AWS_REGION: ${{ secrets.AWS_REGION }}

.. _`Github Actions standard runners`: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
5 changes: 3 additions & 2 deletions docs/source/topic_guides/reusable_build_workflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,15 @@ The Picasso Workflow is designed to build Open edX Docker images for Tutor envir

Key features of the Picasso Workflow include:

- **Runs on GitHub-hosted runners**: By default, the workflow uses ``ubuntu-latest`` runners to execute jobs. This can't be changed to self-hosted runners for the time being.
- **Runs on GitHub-hosted runners**: By default, the workflow uses GitHub hosted runners to execute jobs. This can't be changed to self-hosted runners for the time being.
- **Builds and pushes Docker images**: The workflow pushes images to Dockerhub by default. This can be customized to push images to other registries.
- **Supports multiple services**: You can specify the service to build (e.g., ``openedx``, ``mfe``, ``codejail``, etc.) using the ``SERVICE`` input.
- **Customizable repository and strain**: The workflow allows for specifying the repository, branch, and path to the strain being built. This enables building images from different configurations.
- **Configurable BuildKit parallelism**: By default, the workflow limits parallelism during the build process to optimize resource usage, although this can be changed using the ``ENABLE_LIMIT_BUILDKIT_PARALLELISM`` input. This is useful for low-powered machines, like `Github Actions standard runners`_.
- **Private repository access**: SSH keys are used to clone private repositories securely. The SSH private key should be stored as a secret in the repository, and must have access to the repository specified in ``STRAIN_REPOSITORY``.
- **Configures docker registry**: The workflow sets up the Docker registry to push images to Dockerhub or AWS ECR.
- **Extra commands**: The workflow allows running additional custom commands with ``tutor picasso run-extra-commands``. For details, refer to the `tutor-contrib-picasso`_ documentation.
- **Environment setup**: The workflow sets up and configures Tutor Virtual Environments (TVM), installs necessary plugins like ``tutor-contrib-picasso``, and prepares the environment to build and push Docker images using the `Tutor CLI`_.
- **Environment setup**: The workflow sets up installs necessary plugins like ``tutor-contrib-picasso``, and prepares the environment to build and push Docker images using the `Tutor CLI`_.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The README still mentions TVM:

- **Environment setup**: The workflow sets up and configures Tutor Virtual Environments (TVM), installs necessary plugins like ``tutor-contrib-picasso``, and prepares the environment to build and push Docker images using the `Tutor CLI`_.


.. _tutor-contrib-picasso: https://github.com/eduNEXT/tutor-contrib-picasso/
.. _Github Actions standard runners: https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
Expand Down