Thank you for your interest in the trestlebot project.
Before you start contributing, please take a moment to read through the guide below.
WARNING: This project is currently under initial development. APIs may be changed incompatibly from one commit to another.
- Contributing to trestlebot
Some initial contributions could be:
- Improving the documentation
- Adding test coverage
- Try out issues that have the label
good first issue
- Opening an issue for bugs or feature requests
When submitting a pull request, please follow these guidelines:
- Ensure that you have an issue submitted first and reference it in your pull request.
- Ensure that your code passes all CI tests.
- Please keep the pull request focused on a single issue or feature, if possible.
For a reproducible development environment, we use Dev Containers. See devcontainer.json for more information. Note that this does not include the podman
installation to avoid the requirement for containers with elevated privileges.
For workflow diagrams, see the diagrams under the docs
folder.
- CI Provider - Runs or builds and runs trestle-bot container
- Trestle Bot - Provides logic for managing workspace and containerized environment for use in workflows
- Compliance-Trestle - Upstream library that provides core logic for how OSCAL content is managed
actions
- Provides specific logic fortrestle-bot
tasks that are packaged as Actions. See README.md for more information.entrypoints
- Provides top level logic for specific user-facing tasks. These tasks are not necessarily related in any way so they are not organized into a hierarchical command structure, but they do inherit logic and flags from a base class.provider.py, github.py, and gitlab.py
- Git provider abstract class and concrete implementations for interacting with the API.tasks
- Pre-tasks can be configured before the main git logic is run. Any task that does workspace management should go here.tasks/authored
- Theauthored
package contains logic for managing authoring tasks for single instances of a top-level OSCAL model. These encapsulate logic from thecompliance-trestle
library and allows loose coupling betweentasks
andauthored
types.transformers
- This contains data transformation logic; specifically for rules.
We document decisions using Architectural Decision Records. The team will decide when an ADR will be put in place, but this is generally done to document impactful architectural decisions.
- Create an instance of the ADR template using
trestle author docs create-sample -tn decisions -tr docs/architecture
. - This can be tested locally via
trestle author docs validate --task-name decisions -hv -tr docs/architecture
Each README.md
under the actions
directory have an Actions Inputs and Action Outputs section. These sections are generated from the action.yml
file in the directory. To update the README.md
files, run the following command:
make update-action-readmes
The CI workflows for trestle-bot leverage third party actions pinned to a hash value which is updated by dependabot.yml
. The purpose of pinning actions to a full length commit SHA is to ensure that the action's code and behavior remain consistent. Actions that are pinned to full length commit SHAs act as immutable releases which allow for distinction between versions and an accurate history log. When selecting a commit SHA to include, the SHA value that is associated with the version of the action should be chosen from the associated action's repository. Dependabot checks for the action's reference against the latest version ensuring a secure and consistent approach to managing dependencies and version updating.
To generate a pin for a third party action, there should be a full length commit SHA associated with the version of the action being referenced. The reference used is the full length SHA, tag, or branch that dependabot will use when updating dependencies and bumping versions.
- The syntax for a specified action is:
OWNER/REPOSITORY@TAG-OR-SHA
. - The syntax for a specified reusable workflow is:
OWNER/REPOSITORY/PATH/FILENAME@TAG-OR-SHA
.
This approach is used for authoring CI workflows that utilize versioned actions to produce frequent updates from dependabot for python and GitHub Actions.
Please use the SPDX license identifier in all source files.
# SPDX-License-Identifier: Apache-2.0
This project uses black
and isort
for formatting and flake8
for linting. You can run these commands to format and lint your code.
Linting checks can be run as a pre-commit hook and are verified in CI.
make format
make lint
For non-Python files, we use Megalinter to lint in a CI task. See megalinter.yaml for more information.
We encourage the use of type hints in Python code to enhance readability, maintainability, and robustness of the codebase. Type hints serve as documentation and aid in catching potential errors during development. For static type analysis, we utilize mypy
. Running make lint
will run mypy
checks on the codebase.
- SonarCloud - We use SonarCloud to analyze code quality, coverage, and security. To not break GitHub security model, this will not run on a forked repository.
- Semgrep - Identify issues in the local development environment before committing code. These checks are also run in CI.
Run all tests with make test
or make test-slow
to run all tests including end-to-end.
For information on end-to-end tests, see README.md.
# Run all tests
make test
make test-slow
# Run specific tests
make test-e2e
make develop
poetry run trestlebot-autosync
poetry run trestlebot-rules-transform
poetry run trestlebot-create-cd
poetry run trestlebot-sync-upstreams
poetry run trestlebot-create-ssp
For this guide, we will be using podman
to test trestlebot in a running container.
- Build the image
podman build -f Dockerfile -t localhost:5000/trestlebot:latest
- Create an environment variables file if testing with the entrypoint script.
The entrypoint script is where the logic for GitHub specific integrations should be. The environment variables file will contain variables set by GitHub Actions.
Example file named envfile
cat envfile
...
GITHUB_OUTPUT=
INPUT_SKIP_ITEMS=
INPUT_DRY_RUN=true
INPUT_SKIP_ASSEMBLE=false
INPUT_SKIP_REGENERATE=false
INPUT_REPOSITORY=.
INPUT_BRANCH=test
INPUT_MARKDOWN_PATH=markdown/profiles
INPUT_OSCAL_MODEL=profile
INPUT_SSP_INDEX_PATH=
INPUT_COMMIT_MESSAGE=
INPUT_COMMIT_USER_NAME=testuser
[email protected]
INPUT_FILE_PATTERN=*.md,*.json
INPUT_COMMIT_AUTHOR_NAME=
INPUT_COMMIT_AUTHOR_EMAIL=
INPUT_TARGET_BRANCH=
GITHUB_ACTIONS=true
- Use
podman secret
to store sensitive information like API tokens
cat my-token.txt | podman secret create repo-secret -
- Run the container
podman run --entrypoint /entrypoint.sh --secret repo-secret,type=env,target=TRESTLEBOT_REPO_ACCESS_TOKEN --env-file=envfile -v my-trestle-space:/data -w /data localhost:5000/trestlebot:latest
Once work on a release has been completed:
- Select the new release number. Use the principles of semantic versioning to select the new release number.
- Follow the GitHub documentation on creating a release.
- Once the release is published, the
publish
workflow will be triggered. An image will be built, tested, and published toquay.io
. This process also signs and verifies the image withcosign
.
- Initial releases will have a
major
tag (if stable),major
.minor
, and the full version. - The latest release will be rebuilt every thirty days to pull in base image updates. The same tags will
be published with the addition of
full-version
.date
tag. - Images can be built adhoc for testing purposes with the
workflow_dispatch
trigger.