diff --git a/docs/README.md b/docs/README.md index ecbb142d..9242c007 100644 --- a/docs/README.md +++ b/docs/README.md @@ -6,7 +6,9 @@ Mend Renovate CE was formerly known as "Mend Renovate On-Premises". 1. [Overview](./overview.md) 1. Installation ([Helm](./installation-helm.md)) -1. Configuration ([GitHub](./configuration-github.md), [GitLab](./configuration-gitlab.md)) +1. Configuration + - Renovate Community Edition - ([GitHub](./configure-renovate-ce-github.md), [GitLab](./configure-renovate-ce-gitlab.md)) + - Renovate Enterprise Edition - ([GitHub](./configure-renovate-ee-github.md), [GitLab](./configure-renovate-ee-gitlab.md)) 1. Migration ([Renovate On-Premises to Renovate Community](./migrating-to-renovate-ce.md)) 1. API ([API Documentation](./api.md)) 1. [Advanced topics](./advanced.md) diff --git a/docs/configuration-gitlab.md b/docs/configuration-gitlab.md deleted file mode 100644 index 14afbf69..00000000 --- a/docs/configuration-gitlab.md +++ /dev/null @@ -1,116 +0,0 @@ -# Mend Renovate Configuration for GitLab - -Mend Renovate is available for teams that use GitLab for development. -It may be used for self-hosted GitLab instances as well as for repositories hosted on gitlab.com - -## Mend Renovate features - -#### Job scheduler - -The Mend Renovate's Docker container contains a built-in job scheduler that defaults to enqueing all repositories once per hour. -This saves the need for configuring and monitoring any external `cron` process. - -#### Webhook handler - -Mend Renovate also supports a webserver to listen for system webhooks received from GitLab. - -In particular, Renovate checks webhooks for: - -- Projects it has just been added to -- Commits to `main` branch for "important" files such as `package.json` and `renovate.json` -- Any commits made to Renovate's branches -- Closing or merging of Renovate PRs - -Each of the above results in a job being enqueued for the relevant repository, so that the bot will appear responsive to users. - -#### Priority job queue - -Priority-based queuing is essential for providing a responsive experience for bot users. -For example, if a user makes an update to the config in an onboarding PR, they ideally want to see the results immediately. -By assigning onboarding updates the highest priority in the queue, the bot's update to the onboarding PR can proceed as the very next job, even if many others were in the queue already. - -In general, job priority is based on the probability that a user may be "waiting" for the bot to do something. -That's why onboarding updates are highest priority, and other high priority updates include merging of Renovate PRs because that very often results in other PRs needing updates or rebasing afterwards. - -## Mend Renovate Installation and Setup - -#### Bot Account creation - -You should use a dedicated "bot account" for Renovate. -Apart from reducing the chance of conflicts, it is better for teams if the actions they see from Renovate are clearly marked as coming from a dedicated bot account and not from a teammate's account, which could be confusing at times. -e.g. did the bot automerge that PR, or did a human do it? - -If you are running your own instance of GitLab, it's suggested to name the account "Renovate Bot" with username "renovate-bot". -Create this account and then create a Personal Access Token for it with `api`, `read_user` and `write_repository` permissions. - -It's best not add this bot account to any repositories yet. - -#### Bot Server setup - -The server setup for Mend Renovate for GitLab is essentially the same as for GitHub Enterprise, so you can look at those examples. -Mend Renovate needs only a low-mid range server with Docker capabilities (e.g. 1 VCPU with 3.75GB of RAM). - -## Configuration - -#### Mend Renovate environment variables - -Mend Renovate requires configuration via environment variables in addition to Renovate OSS's regular configuration: - -**`MEND_RNV_ACCEPT_TOS`**: Set this environment variable to `y` to consent to [Mend's Terms of Service](https://www.mend.io/terms-of-service/). - -**`MEND_RNV_LICENSE_KEY`**: This should be the license key you obtained after registering at [https://www.mend.io/renovate-community/](https://www.mend.io/renovate-community/). - -**`MEND_RNV_PLATFORM`**: Set this to `gitlab`. - -**`MEND_RNV_ENDPOINT`**: This is the API endpoint for your GitLab host. e.g. like `https://gitlab.company.com/api/v4/`. Include the trailing slash. - -**`MEND_RNV_GITLAB_PAT`**: A Personal Access Token for the GitLab bot account. - -**`MEND_RNV_WEBHOOK_SECRET`**: Optional: Will default to `renovate` if not configured. - -**`MEND_RNV_ADMIN_API_ENABLED`**: Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. - -**`MEND_RNV_SERVER_API_SECRET`**: Required if Admin APIs are enabled. - -**`MEND_RNV_SQLITE_FILE_PATH`**: Optional: Provide a path to persist the database. (eg. '/db/renovate-ce.sqlite', where 'db' is defined as a volume. - -**`MEND_RNV_CRON_JOB_SCHEDULER`**: This configuration option accepts a 5-part cron schedule and is _optional_. It defaults to `0 * * * *` (i.e. once per hour exactly on the hour) if it is unset. If decreasing the interval then be careful that you do not exhaust the available hourly rate limit of the app on GitHub server or cause too much load. - -**`MEND_RNV_CRON_APP_SYNC`**: # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 \* \* \*' (every 4 hours, on the hour) - -**`GITHUB_COM_TOKEN`**: A Personal Access Token for a user account on github.com (i.e. _not_ an account on your GitHub Enterprise instance). This is used for retrieving changelogs and release notes from repositories hosted on github.com and it does not matter who it belongs to. It needs only read-only access privileges. Note: This is required if you are using a self-hosted GitHub Enterprise or GitLab instance but should not be configured if your `RENOVATE_ENDPOINT` is `https://api.github.com`. - -#### Core Renovate Configuration - -"Core" Renovate functionality (i.e. same functionality you'd find in the CLI version or the hosted app) can be configured using environment variables (e.g. `RENOVATE_XXXXXX`) or via a `config.js` file that you mount inside the Mend Renovate container to `/usr/src/app/config.js`. - -#### System Hook - -To activate Mend Renovate's webhook ability, a GitLab administrator needs to configure a System Hook that points to the Renovate installation. - -Configure it to point to Mend Renovate's server, e.g. `http://renovate.yourcompany.com:8080/webhook` or `https://1.2.3.4/webhook`. - -Remember: Renovate's webhook listener binds to port 8080 inside its container, but you can map it (using Docker) to whatever external port you require, including port 80. - -Set the "Secret Token" to the same value you configured for `MEND_RNV_WEBHOOK_SECRET` earlier, or set it to `"renovate"` if you left it as default. - -Set Hook triggers for "Push events", "Merge request events". - -Once your System Hook is added, Renovate's webhook handler will receive events from _all_ repositories. -Therefore, Renovate maintains a list of all repositories it has access to and discards events from all others. - -Note: You will need to create a webhook with "Issue events" for each repository in which you want the Dependency Dashboard issue to be interactive, because Issue events aren't included in System hooks. - -## Testing Mend Renovate - -At this point you should be ready to test out Mend Renovate. -You probably want to create a test repo before adding Mend Renovate to any "real" ones. -To simulate normal conditions, create the repository from a regular account and add a package file. - -#### Enabling Renovate - -To enable Renovate on your test repository, add the bot user you created to the project with "Developer" permissions. -Remember to add a webhook with "Issue events" to the repository if you wish to enable interactive dashboard issues. - -Adding Renovate as a Developer to a repository cause a system hook to be sent to Renovate which in turn enqueues a job for the Renovate Worker. -The repository should receive an onboarding PR immediately after. diff --git a/docs/configuration-github.md b/docs/configure-renovate-ce-github.md similarity index 57% rename from docs/configuration-github.md rename to docs/configure-renovate-ce-github.md index 3172720c..b0525632 100644 --- a/docs/configuration-github.md +++ b/docs/configure-renovate-ce-github.md @@ -1,8 +1,8 @@ -# Mend Renovate Configuration - GitHub +# Configuration - Mend Renovate Community Edition for GitHub -## Provision App +## Create and Configure the GitHub App (bot) -Before running Mend Renovate CE or EE, you need to provision it as an App on GitHub, and retrieve the ID + private key provided. +Before running Mend Renovate, you need to provision it as an App on GitHub, and retrieve the ID + private key provided. If you're running a self-hosted instance of GitHub Enterprise, it is suggested to name the app "Renovate" so that it shows up as easily recognizable as "renovate[bot]" in Pull Requests. If you're running against `github.com` then the name Renovate is already taken by the hosted Mend Renovate app, so you will need something else like "YourCompany Renovate". @@ -45,7 +45,9 @@ If you don't care about the value, then enter 'renovate' as that is the default You can use the [Renovate icon](https://docs.renovatebot.com/assets/images/logo.png) for the app/bot if you desire. -## Configure Mend Renovate +## Configure Mend Renovate CE + +### Mend Renovate environment variables Mend Renovate requires configuration via environment variables in addition to Renovate OSS's regular configuration: @@ -61,17 +63,17 @@ Mend Renovate requires configuration via environment variables in addition to Re **`MEND_RNV_GITHUB_APP_KEY`**: A string representation of the private key of the provisioned Renovate app on GitHub. To insert the value directly into a Docker Compose environment variable, open the PEM file in a text editor and replace all new lines with "\n" so that the entire key is on one line. Alternatively, you can skip setting this key as an environment variable and instead mount it as a file to `/usr/src/app/renovate.private-key.pem`, as shown in the example Docker Compose file. -**`MEND_RNV_WEBHOOK_SECRET`**: This configuration option must be set unless you configured it to 'renovate', which is default. +**`MEND_RNV_WEBHOOK_SECRET`**: Optional: Defaults to `renovate` **`MEND_RNV_ADMIN_API_ENABLED`**: Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. **`MEND_RNV_SERVER_API_SECRET`**: Required if Admin APIs are enabled. -**`MEND_RNV_SQLITE_FILE_PATH`**: Optional: Provide a path to persist the database. (eg. '/db/renovate-ce.sqlite', where 'db' is defined as a volume. +**`MEND_RNV_SQLITE_FILE_PATH`**: Optional: Provide a path to persist the database. (eg. '/db/renovate-ce.sqlite', where 'db' is defined as a volume) -**`MEND_RNV_CRON_JOB_SCHEDULER`**: This configuration option accepts a 5-part cron schedule and is _optional_. It defaults to `0 * * * *` (i.e. once per hour exactly on the hour) if it is unset. If decreasing the interval then be careful that you do not exhaust the available hourly rate limit of the app on GitHub server or cause too much load. +**`MEND_RNV_CRON_JOB_SCHEDULER`**: Optional: Accepts a 5-part cron schedule. Defaults to `0 * * * *` (i.e. once per hour exactly on the hour). If decreasing the interval then be careful that you do not exhaust the available hourly rate limit of the app on GitHub server or cause too much load. -**`MEND_RNV_CRON_APP_SYNC`**: # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 \* \* \*' (every 4 hours, on the hour) +**`MEND_RNV_CRON_APP_SYNC`**: Optional: AppSync schedule. Defaults to `0 0,4,8,12,16,20 \* \* \*` (every 4 hours, on the hour) **`GITHUB_COM_TOKEN`**: A Personal Access Token for a user account on github.com (i.e. _not_ an account on your GitHub Enterprise instance). This is used for retrieving changelogs and release notes from repositories hosted on github.com and it does not matter who it belongs to. It needs only read-only access privileges. Note: This is required if you are using a self-hosted GitHub Enterprise or GitLab instance but should not be configured if your `RENOVATE_ENDPOINT` is `https://api.github.com`. @@ -79,4 +81,44 @@ Mend Renovate requires configuration via environment variables in addition to Re The core Renovate OSS functionality can be configured using environment variables (e.g. `RENOVATE_XXXXXX`) or via a `config.js` file that you mount inside the Mend Renovate container to `/usr/src/app/config.js`. -**npm Registry** If using your own npm registry, you may find it easiest to update your `docker-compose.yml` to include a volume that maps an `.npmrc` file to `/home/ubuntu/.npmrc`. The RC file should contain `registry=...` with the registry URL your company uses internally. This will allow Renovate to find shared configs and other internally published packages. +**npm Registry** If using your own npm registry, you may find it easiest to update your Docker Compose file to include a volume that maps an `.npmrc` file to `/home/ubuntu/.npmrc`. The RC file should contain `registry=...` with the registry URL your company uses internally. This will allow Renovate to find shared configs and other internally published packages. + +## Run Mend Renovate + +You can run Mend Renovate from a Docker command line prompt, or by using a Docker Compose file. An example is provided below. + +**Docker Compose File**: Renovate CE on GitHub + +```yaml +version: "3.6" +services: + renovate: + image: ghcr.io/mend/renovate-ce:6.0.0-full + restart: on-failure + ports: + - "80:8080" # Receive webhooks on port 80 + environment: + # Optional: Define log level. Set to Debug for more verbose output + # - LOG_LEVEL=DEBUG + # Provide Mend License and accept Terms of Service + - MEND_RNV_ACCEPT_TOS # Set to 'y' to accept the Mend Renovate Terms of Service + - MEND_RNV_LICENSE_KEY # Set this to the key you received by email + # Provide connection details for the GitHub App + - MEND_RNV_PLATFORM=github + - MEND_RNV_ENDPOINT # Required for GitHub Enterprise Server; not for GitHub.com. Include the trailing slash. + - MEND_RNV_GITHUB_APP_ID # GitHub App ID + - MEND_RNV_GITHUB_APP_KEY # GitHub App Key (PEM file). Alternatively mount as a volume below + - MEND_RNV_WEBHOOK_SECRET # Optional: defaults to 'renovate' + # Optional settings for Mend Renovate + # - MEND_RNV_ADMIN_API_ENABLED # Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + # - MEND_RNV_SERVER_API_SECRET # Required if Admin APIs are enabled + # - MEND_RNV_SQLITE_FILE_PATH=/db/renovate-ce.sqlite # Optional: Provide a path to persist the database. Needs 'db' volume defined (below). + # - MEND_RNV_CRON_JOB_SCHEDULER # Optional Job enqueue schedule: defaults to '0 * * * *' (hourly, on the hour) + # - MEND_RNV_CRON_APP_SYNC # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 * * *' (every 4 hours, on the hour) + # Core Renovate settings + # - GITHUB_COM_TOKEN # Personal Access Token for github.com (used for retrieving changelogs) + # volumes: + # Optional: You can use a file mount to persist the database between sessions + # - "/tmp/db/:/db/" # Unix-style file mounting for the db + # - "C:\\tmp/db/:/db/" # Windows-style file mounting for the db +``` diff --git a/docs/configure-renovate-ce-gitlab.md b/docs/configure-renovate-ce-gitlab.md new file mode 100644 index 00000000..7d6a3cee --- /dev/null +++ b/docs/configure-renovate-ce-gitlab.md @@ -0,0 +1,137 @@ +# Configuration - Mend Renovate Community Edition for GitLab + +## Configure Renovate Bot Account on GitLab + +### Renovate Bot Account + +Create a GitLab user account to act as the "Renovate Bot". +If you are running your own instance of GitLab, it's suggested to name the account "Renovate Bot" with username "renovate-bot". + +Note: In GitLab, the "Renovate Bot" is not an App or Plugin; it's a GitLab user account that's been given the right permissions and System Hooks. +You should use a dedicated "bot account" for Renovate, instead of using someone's personal user account. +Apart from reducing the chance of conflicts, it is better for teams if the actions they see from Renovate are clearly marked as coming from a dedicated bot account and not from a teammate's account, which could be confusing at times. +e.g. Did the bot automerge that PR, or did a human do it? + +### Personal Access Token + +Once the account is created, create a Personal Access Token for it with the following permissions: + * `api` + * `read_user` + * `write_repository` + +### System Hook + +To activate Mend Renovate's webhook ability, create a System Hook that points to the Renovate installation. + +Create a System Hook (in Admin area) + +1. Set the webhook URL to point to the Renovate server url followed by `/webhook`. (e.g. `http://renovate.yourcompany.com:8080/webhook` or `https://1.2.3.4/webhook`) +2. Set the webhook secret to the same value configured for `MEND_RNV_WEBHOOK_SECRET` (defaults to `renovate`) +3. Set Hook triggers for: + * `Push events` + * `Merge request events` + +Remember: Renovate's webhook listener binds to port 8080 inside its container, but you can map it (using Docker) to whatever external port you require, including port 80. + +Set the "Secret Token" to the same value configured for `MEND_RNV_WEBHOOK_SECRET`, or set it to `"renovate"` if you left it as default. + +Once your System Hook is added, Renovate's webhook handler will receive events from _all_ repositories. +Therefore, Renovate maintains a list of all repositories it has access to and discards events from all others. + +### Repo Webhooks + +Because Issue events aren't included in System hooks, a webhook must be individually configured for each repository in which you want the Dependency Dashboard issue to be interactive. + +In the Repository settings, create a Webhook with the following settings: +1. Set the webhook URL - same as System Hook URL +2. Set the webhook secret - same as System Hook secret +3. Set Hook triggers for: + * `Issue events` + +This will need to be repeated for every repository that is onboarded to the Renovate Bot account. +It is usually easiest to create the repo webhook while adding the Renovate Bot account to a repo. + + +## Configure Mend Renovate CE + +### Mend Renovate environment variables + +Mend Renovate requires configuration via environment variables in addition to Renovate OSS's regular configuration: + +**`MEND_RNV_ACCEPT_TOS`**: Set this environment variable to `y` to consent to [Mend's Terms of Service](https://www.mend.io/terms-of-service/). + +**`MEND_RNV_LICENSE_KEY`**: This should be the license key you obtained after registering at [https://www.mend.io/renovate-community/](https://www.mend.io/renovate-community/). + +**`MEND_RNV_PLATFORM`**: Set this to `gitlab`. + +**`MEND_RNV_ENDPOINT`**: This is the API endpoint for your GitLab host. e.g. like `https://gitlab.company.com/api/v4/`. Include the trailing slash. + +**`MEND_RNV_GITLAB_PAT`**: Personal Access Token for the GitLab bot account. + +**`MEND_RNV_WEBHOOK_SECRET`**: Optional: Defaults to `renovate` + +**`MEND_RNV_ADMIN_API_ENABLED`**: Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + +**`MEND_RNV_SERVER_API_SECRET`**: Required if Admin APIs are enabled. + +**`MEND_RNV_SQLITE_FILE_PATH`**: Optional: Provide a path to persist the database. (eg. '/db/renovate-ce.sqlite', where 'db' is defined as a volume) + +**`MEND_RNV_CRON_JOB_SCHEDULER`**: Optional: Accepts a 5-part cron schedule. Defaults to `0 * * * *` (i.e. once per hour exactly on the hour). If decreasing the interval then be careful that you do not exhaust the available hourly rate limit of the app on GitHub server or cause too much load. + +**`MEND_RNV_CRON_APP_SYNC`**: Optional: AppSync schedule. Defaults to `0 0,4,8,12,16,20 \* \* \*` (every 4 hours, on the hour) + +**`GITHUB_COM_TOKEN`**: A Personal Access Token for a user account on github.com (i.e. _not_ an account on your GitHub Enterprise instance). This is used for retrieving changelogs and release notes from repositories hosted on github.com and it does not matter who it belongs to. It needs only read-only access privileges. Note: This is required if you are using a self-hosted GitHub Enterprise or GitLab instance but should not be configured if your `RENOVATE_ENDPOINT` is `https://api.github.com`. + +### Core Renovate Configuration + +The core Renovate OSS functionality can be configured using environment variables (e.g. `RENOVATE_XXXXXX`) or via a `config.js` file that you mount inside the Mend Renovate container to `/usr/src/app/config.js`. + +**npm Registry** If using your own npm registry, you may find it easiest to update your `docker-compose.yml` to include a volume that maps an `.npmrc` file to `/home/ubuntu/.npmrc`. The RC file should contain `registry=...` with the registry URL your company uses internally. This will allow Renovate to find shared configs and other internally published packages. + +## Enabling Renovate on Repositories + +To enable Renovate on a repository, in the project's _Settings -> Members_ section, add the bot user and give it `Developer` permissions. + +Remember to add a webhook with "Issue events" to the repository if you wish to enable interactive dashboard issues. (See _Create Repo Webhooks_ above) + +Adding Renovate as a Developer to a repository cause a system hook to be sent to Renovate which in turn enqueues a job for the Renovate Worker. +The repository should receive an onboarding PR immediately after. + +## Run Mend Renovate + +You can run Mend Renovate from a Docker command line prompt, or by using a Docker Compose file. An example is provided below. + +**Docker Compose File**: Renovate CE on GitLab + +```yaml +version: "3.6" +services: + renovate: + image: ghcr.io/mend/renovate-ce:6.0.0-full + restart: on-failure + environment: + # Optional: Define log level. Set to Debug for more verbose output + # - LOG_LEVEL=DEBUG + # Provide Mend License and accept Terms of Service + - MEND_RNV_ACCEPT_TOS # Set to 'y' to accept the Mend Renovate Terms of Service + - MEND_RNV_LICENSE_KEY # Set this to the key you received by email + # Provide connection details for the GitLab Bot + - MEND_RNV_PLATFORM=gitlab + - MEND_RNV_ENDPOINT # GitLab endpoint, include trailing slash + - MEND_RNV_GITLAB_PAT # Personal Access Token for bot account + - MEND_RNV_WEBHOOK_SECRET # Optional: defaults to 'renovate' + # Optional settings for Mend Renovate + # - MEND_RNV_ADMIN_API_ENABLED # Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + # - MEND_RNV_SERVER_API_SECRET # Required if Admin APIs are enabled + # - MEND_RNV_SQLITE_FILE_PATH=/db/renovate-ce.sqlite # Optional: Provide a path to persist the database. Needs 'db' volume defined (below). + # - MEND_RNV_CRON_JOB_SCHEDULER # Optional Job enqueue schedule: defaults to '0 * * * *' (hourly, on the hour) + # - MEND_RNV_CRON_APP_SYNC # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 * * *' (every 4 hours, on the hour) + # Core Renovate settings + # - GITHUB_COM_TOKEN # Personal Access Token for github.com (used for retrieving changelogs) + ports: + - "80:8080" # Receive webhooks on port 80 + # volumes: + # Optional: You can use a file mount to persist the database between sessions + # - "/tmp/db/:/db/" # Unix-style file mounting for the db + # - "C:\\tmp/db/:/db/" # Windows-style file mounting for the db +``` \ No newline at end of file diff --git a/docs/configure-renovate-ee-github.md b/docs/configure-renovate-ee-github.md new file mode 100644 index 00000000..f8a88f7b --- /dev/null +++ b/docs/configure-renovate-ee-github.md @@ -0,0 +1,159 @@ +# Configuration - Mend Renovate Enterprise Edition for GitHub + +## Create and Configure the GitHub App (bot) + +Before running Mend Renovate, you need to provision it as an App on GitHub, and retrieve the ID + private key provided. + +If you're running a self-hosted instance of GitHub Enterprise, it is suggested to name the app "Renovate" so that it shows up as easily recognizable as "renovate[bot]" in Pull Requests. +If you're running against `github.com` then the name Renovate is already taken by the hosted Mend Renovate app, so you will need something else like "YourCompany Renovate". + +The App requires the following permissions: + +- Repository permissions + - Administration: Read-only + - Checks: Read & write + - Contents: Read & write + - Issues: Read & write + - Metadata: Read-only + - Pull Requests: Read & write + - Commit statuses: Read & write + - Dependabot alerts: Read-only (optional) + - Workflows: Read & write +- Organization permissions + - Members: Read-only + +The App should also subscribe to the following webhook events: + +- Security Advisory +- Check run +- Check suite +- Issues +- Pull request +- Push +- Repository +- Status + +Description, Homepage, User authorization callback URL, and Setup URL are all unimportant so you may set them to whatever you like. + +The Mend Renovate webhook listener binds to port 8080 by default, however it will bind to `process.env.PORT` instead if that is defined. +Note: The Mend Renovate image takes care of exposing port 8080 of the container, so if you change this port then you will need to take care of any exposing/mapping of ports yourself. +In the [Docker Compose example config](https://github.com/mend/renovate-cc-ee/tree/main/examples/), the default port 8080 is used and then mapped to port 80 on the host. + +For the Webhook URL field, point it to `/webhook` on port 80 (or whatever port you mapped to) of the server that you will run Mend Renovate on, e.g. http://1.2.3.4/webhook +Be sure to enter a webhook secret too. +If you don't care about the value, then enter 'renovate' as that is the default secret that the webhook handler process uses. + +You can use the [Renovate icon](https://docs.renovatebot.com/assets/images/logo.png) for the app/bot if you desire. + +## Configure Mend Renovate EE + +Renovate Enterprise runs with one **_Server_** container and one or more **_Worker_** containers. +See below for a list of environment variables that relate to each. + +### Environment variables - Renovate Enterprise Server + +**`MEND_RNV_ACCEPT_TOS`**: Set this environment variable to `y` to consent to [Mend's Terms of Service](https://www.mend.io/terms-of-service/). + +**`MEND_RNV_LICENSE_KEY`**: For a Renovate Enterprise license key, contact Mend via the [Renovate Enterprise webpage](https://www.mend.io/renovate-enterprise/). + +**`MEND_RNV_PLATFORM`**: Set this to `github`. + +**`MEND_RNV_ENDPOINT`**: This is the API endpoint for your GitHub Enterprise installation. Required for GitHub Enterprise Server; not for GitHub.com. Include the trailing slash. + +**`MEND_RNV_GITHUB_APP_ID`**: The GitHub App ID of the provisioned Renovate app on GitHub. + +**`MEND_RNV_GITHUB_APP_KEY`**: A string representation of the private key of the provisioned Renovate app on GitHub. To insert the value directly into a Docker Compose environment variable, open the PEM file in a text editor and replace all new lines with "\n" so that the entire key is on one line. Alternatively, you can skip setting this key as an environment variable and instead mount it as a file to `/usr/src/app/renovate.private-key.pem`, as shown in the example Docker Compose file. + +**`MEND_RNV_WEBHOOK_SECRET`**: Optional: Defaults to `renovate` + +**`MEND_RNV_SERVER_API_SECRET`**: Set an API secret. Must match the Worker instances and Admin APIs for communicating with the Server. + +**`MEND_RNV_ADMIN_API_ENABLED`**: Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + +**`MEND_RNV_SQLITE_FILE_PATH`**: Optional: Provide a path to persist the database. (eg. '/db/renovate-ce.sqlite', where 'db' is defined as a volume) + +**`MEND_RNV_CRON_JOB_SCHEDULER`**: Optional: Accepts a 5-part cron schedule. Defaults to `0 * * * *` (i.e. once per hour exactly on the hour). If decreasing the interval then be careful that you do not exhaust the available hourly rate limit of the app on GitHub server or cause too much load. + +**`MEND_RNV_CRON_APP_SYNC`**: Optional: AppSync schedule. Defaults to `0 0,4,8,12,16,20 \* \* \*` (every 4 hours, on the hour) + +**`GITHUB_COM_TOKEN`**: A Personal Access Token for a user account on github.com (i.e. _not_ an account on your GitHub Enterprise instance). This is used for retrieving changelogs and release notes from repositories hosted on github.com and it does not matter who it belongs to. It needs only read-only access privileges. Note: This is required if you are using a self-hosted GitHub Enterprise instance but should not be configured if your `RENOVATE_ENDPOINT` is `https://api.github.com`. + +### Environment variables - Renovate Enterprise Worker + +The Worker container needs to define only the following variables: + +* **`MEND_RNV_SERVER_HOSTNAME`**: The hostname of the Renovate Enterprise `server` container (eg. http://renovate-ee-server:8080) +* **`MEND_RNV_SERVER_API_SECRET`**: Set to same as Server +* **`MEND_RNV_ACCEPT_TOS`**: Set to same as Server +* **`MEND_RNV_LICENSE_KEY`**: Set to same as Server + +## Configure Renovate Core + +The core Renovate OSS functionality can be configured using environment variables (e.g. `RENOVATE_XXXXXX`) or via a `config.js` file that you mount inside the Mend Renovate container to `/usr/src/app/config.js`. + +**npm Registry** If using your own npm registry, you may find it easiest to update your Docker Compose file to include a volume that maps an `.npmrc` file to `/home/ubuntu/.npmrc`. The RC file should contain `registry=...` with the registry URL your company uses internally. This will allow Renovate to find shared configs and other internally published packages. + +## Run Mend Renovate + +You can run Mend Renovate from a Docker command line prompt, or by using a Docker Compose file. An example is provided below. + +**Docker Compose File**: Renovate EE on GitHub + +```yaml +version: '3.6' + +x-controller-shared-variables: &variables-controller + MEND_RNV_SERVER_HOSTNAME: http://renovate-ee-server:8080 + MEND_RNV_SERVER_API_SECRET: # This secret will be used by the Worker and by Admin APIs + # Provide Mend License and accept Terms of Service + MEND_RNV_ACCEPT_TOS: # Set to 'y' to accept the Mend Renovate Terms of Service + MEND_RNV_LICENSE_KEY: # Set this to the Renovate Enterprise key obtained from Mend + +services: + renovate-ee-server: + restart: on-failure + image: ghcr.io/mend/renovate-ee-server:6.0.0 + ports: + - "80:8080" # Receive webhooks on port 80 + environment: + <<: *variables-controller + # Optional: Define log level. Set to Debug for more verbose output + # LOG_LEVEL: debug + # Provide connection details for the Renovate Bot/App + MEND_RNV_PLATFORM: # Set to `github` or `gitlab` + MEND_RNV_ENDPOINT: # Required for GitLab or GitHub Enterprise Server; not for GitHub.com. Include the trailing slash. + MEND_RNV_GITHUB_APP_ID: # GitHub Only! GitHub App ID + MEND_RNV_GITHUB_APP_KEY: # GitHub Only! GitHub App Key (PEM file). Alternatively mount as a volume below + MEND_RNV_WEBHOOK_SECRET: # Optional: defaults to 'renovate' + # Optional settings for Mend Renovate + # MEND_RNV_ADMIN_API_ENABLED: # Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + # MEND_RNV_SQLITE_FILE_PATH: /db/renovate-ee.sqlite # Optional: Provide a path to persist the database. Needs 'db' volume defined (below). + # MEND_RNV_CRON_JOB_SCHEDULER: # Optional Job enqueue schedule: defaults to '0 * * * *' (hourly, on the hour) + # MEND_RNV_CRON_APP_SYNC: # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 * * *' (every 4 hours, on the hour) + # Core Renovate settings + # GITHUB_COM_TOKEN: # Personal Access Token for github.com (used for retrieving changelogs) + # volumes: + # Optional: You can use a file mount to persist the database between sessions + # - "/tmp/db/:/db/" # Unix-style file mounting for the db + # - "C:\\tmp/db/:/db/" # Windows-style file mounting for the db + healthcheck: + test: curl --fail http://renovate-ee-server:8080/health || exit 1 + interval: 60s + retries: 5 + start_period: 20s + timeout: 10s + + renovate-ee-worker: + restart: on-failure + deploy: + # Post deploy: + # $ docker-compose -f docker-compose-renovate-ee.yml up --scale renovate-ee-worker=3 -d --no-recreate + replicas: 2 + image: ghcr.io/mend/renovate-ee-worker:6.0.0-full + depends_on: + - renovate-ee-server + environment: + <<: *variables-controller + # Optional: Define log level. Set to Debug for more verbose output + # LOG_LEVEL: debug +``` diff --git a/docs/configure-renovate-ee-gitlab.md b/docs/configure-renovate-ee-gitlab.md new file mode 100644 index 00000000..24869528 --- /dev/null +++ b/docs/configure-renovate-ee-gitlab.md @@ -0,0 +1,165 @@ +# Configuration - Mend Renovate Enterprise Edition for GitLab + +## Configure Renovate Bot Account on GitLab + +### Renovate Bot Account + +Create a GitLab user account to act as the "Renovate Bot". +If you are running your own instance of GitLab, it's suggested to name the account "Renovate Bot" with username "renovate-bot". + +Note: In GitLab, the "Renovate Bot" is not an App or Plugin; it's a GitLab user account that's been given the right permissions and System Hooks. +You should use a dedicated "bot account" for Renovate, instead of using someone's personal user account. +Apart from reducing the chance of conflicts, it is better for teams if the actions they see from Renovate are clearly marked as coming from a dedicated bot account and not from a teammate's account, which could be confusing at times. +e.g. Did the bot automerge that PR, or did a human do it? + +### Personal Access Token + +Once the account is created, create a Personal Access Token for it with the following permissions: + * `api` + * `read_user` + * `write_repository` + +### System Hook + +To activate Mend Renovate's webhook ability, create a System Hook that points to the Renovate installation. + +Create a System Hook (in Admin area) + +1. Set the webhook URL to point to the Renovate server url followed by `/webhook`. (e.g. `http://renovate.yourcompany.com:8080/webhook` or `https://1.2.3.4/webhook`) +2. Set the webhook secret to the same value configured for `MEND_RNV_WEBHOOK_SECRET` (defaults to `renovate`) +3. Set Hook triggers for: + * `Push events` + * `Merge request events` + +Remember: Renovate's webhook listener binds to port 8080 inside its container, but you can map it (using Docker) to whatever external port you require, including port 80. + +Set the "Secret Token" to the same value configured for `MEND_RNV_WEBHOOK_SECRET`, or set it to `"renovate"` if you left it as default. + +Once your System Hook is added, Renovate's webhook handler will receive events from _all_ repositories. +Therefore, Renovate maintains a list of all repositories it has access to and discards events from all others. + +### Repo Webhooks + +Because Issue events aren't included in System hooks, a webhook must be individually configured for each repository in which you want the Dependency Dashboard issue to be interactive. + +In the Repository settings, create a Webhook with the following settings: +1. Set the webhook URL - same as System Hook URL +2. Set the webhook secret - same as System Hook secret +3. Set Hook triggers for: + * `Issue events` + +This will need to be repeated for every repository that is onboarded to the Renovate Bot account. +It is usually easiest to create the repo webhook while adding the Renovate Bot account to a repo. + + +## Configure Mend Renovate EE + +Renovate Enterprise runs with one **_Server_** container and one or more **_Worker_** containers. +See below for a list of environment variables that relate to each. + +### Environment variables - Renovate Enterprise Server + +**`MEND_RNV_ACCEPT_TOS`**: Set this environment variable to `y` to consent to [Mend's Terms of Service](https://www.mend.io/terms-of-service/). + +**`MEND_RNV_LICENSE_KEY`**: For a Renovate Enterprise license key, contact Mend via the [Renovate Enterprise webpage](https://www.mend.io/renovate-enterprise/). + +**`MEND_RNV_PLATFORM`**: Set this to `gitlab`. + +**`MEND_RNV_ENDPOINT`**: This is the API endpoint for your GitHub Enterprise installation. Required for GitHub Enterprise Server; not for GitHub.com. Include the trailing slash. + +**`MEND_RNV_GITLAB_PAT`**: Personal Access Token for the GitLab bot account. + +**`MEND_RNV_WEBHOOK_SECRET`**: Optional: Defaults to `renovate` + +**`MEND_RNV_SERVER_API_SECRET`**: Set an API secret. Must match the Worker instances and Admin APIs for communicating with the Server. + +**`MEND_RNV_ADMIN_API_ENABLED`**: Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + +**`MEND_RNV_SQLITE_FILE_PATH`**: Optional: Provide a path to persist the database. (eg. '/db/renovate-ce.sqlite', where 'db' is defined as a volume) + +**`MEND_RNV_CRON_JOB_SCHEDULER`**: Optional: Accepts a 5-part cron schedule. Defaults to `0 * * * *` (i.e. once per hour exactly on the hour). If decreasing the interval then be careful that you do not exhaust the available hourly rate limit of the app on GitHub server or cause too much load. + +**`MEND_RNV_CRON_APP_SYNC`**: Optional: AppSync schedule. Defaults to `0 0,4,8,12,16,20 \* \* \*` (every 4 hours, on the hour) + +**`GITHUB_COM_TOKEN`**: A Personal Access Token for a user account on github.com (i.e. _not_ an account on your GitHub Enterprise instance). This is used for retrieving changelogs and release notes from repositories hosted on github.com and it does not matter who it belongs to. It needs only read-only access privileges. + +### Environment variables - Renovate Enterprise Worker + +The Worker container needs to define only the following variables: + +* **`MEND_RNV_SERVER_HOSTNAME`**: The hostname of the Renovate Enterprise `server` container (eg. http://renovate-ee-server:8080) +* **`MEND_RNV_SERVER_API_SECRET`**: Set to same as Server +* **`MEND_RNV_ACCEPT_TOS`**: Set to same as Server +* **`MEND_RNV_LICENSE_KEY`**: Set to same as Server + + +## Configure Renovate Core + +The core Renovate OSS functionality can be configured using environment variables (e.g. `RENOVATE_XXXXXX`) or via a `config.js` file that you mount inside the Mend Renovate container to `/usr/src/app/config.js`. + +**npm Registry** If using your own npm registry, you may find it easiest to update your Docker Compose file to include a volume that maps an `.npmrc` file to `/home/ubuntu/.npmrc`. The RC file should contain `registry=...` with the registry URL your company uses internally. This will allow Renovate to find shared configs and other internally published packages. + +## Run Mend Renovate + +You can run Mend Renovate from a Docker command line prompt, or by using a Docker Compose file. An example is provided below. + +**Docker Compose File**: Renovate EE on GitHub + +```yaml +version: '3.6' + +x-controller-shared-variables: &variables-controller + MEND_RNV_SERVER_HOSTNAME: http://renovate-ee-server:8080 + MEND_RNV_SERVER_API_SECRET: # This secret will be used by the Worker and by Admin APIs + # Provide Mend License and accept Terms of Service + MEND_RNV_ACCEPT_TOS: # Set to 'y' to accept the Mend Renovate Terms of Service + MEND_RNV_LICENSE_KEY: # Set this to the Renovate Enterprise key obtained from Mend + +services: + renovate-ee-server: + restart: on-failure + image: ghcr.io/mend/renovate-ee-server:6.0.0 + ports: + - "80:8080" # Receive webhooks on port 80 + environment: + <<: *variables-controller + # Optional: Define log level. Set to Debug for more verbose output + # LOG_LEVEL: debug + # Provide connection details for the Renovate Bot/App + MEND_RNV_PLATFORM: # Set to `github` or `gitlab` + MEND_RNV_ENDPOINT: # Required for GitLab or GitHub Enterprise Server; not for GitHub.com. Include the trailing slash. + MEND_RNV_GITHUB_APP_ID: # GitHub Only! GitHub App ID + MEND_RNV_GITHUB_APP_KEY: # GitHub Only! GitHub App Key (PEM file). Alternatively mount as a volume below + MEND_RNV_WEBHOOK_SECRET: # Optional: defaults to 'renovate' + # Optional settings for Mend Renovate + # MEND_RNV_ADMIN_API_ENABLED: # Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. + # MEND_RNV_SQLITE_FILE_PATH: /db/renovate-ee.sqlite # Optional: Provide a path to persist the database. Needs 'db' volume defined (below). + # MEND_RNV_CRON_JOB_SCHEDULER: # Optional Job enqueue schedule: defaults to '0 * * * *' (hourly, on the hour) + # MEND_RNV_CRON_APP_SYNC: # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 * * *' (every 4 hours, on the hour) + # Core Renovate settings + # GITHUB_COM_TOKEN: # Personal Access Token for github.com (used for retrieving changelogs) + # volumes: + # Optional: You can use a file mount to persist the database between sessions + # - "/tmp/db/:/db/" # Unix-style file mounting for the db + # - "C:\\tmp/db/:/db/" # Windows-style file mounting for the db + healthcheck: + test: curl --fail http://renovate-ee-server:8080/health || exit 1 + interval: 60s + retries: 5 + start_period: 20s + timeout: 10s + + renovate-ee-worker: + restart: on-failure + deploy: + # Post deploy: + # $ docker-compose -f docker-compose-renovate-ee.yml up --scale renovate-ee-worker=3 -d --no-recreate + replicas: 2 + image: ghcr.io/mend/renovate-ee-worker:6.0.0-full + depends_on: + - renovate-ee-server + environment: + <<: *variables-controller + # Optional: Define log level. Set to Debug for more verbose output + # LOG_LEVEL: debug +``` diff --git a/docs/overview.md b/docs/overview.md index 4e1ff699..04623f6c 100644 --- a/docs/overview.md +++ b/docs/overview.md @@ -11,6 +11,35 @@ Essentially, it is an alternative to running the `renovate` CLI tool, with the f - Webhook listener to enable dynamic reactions to repository events - Administration APIs for probing the system state or triggering jobs +## Mend Renovate features + +#### Job scheduler + +The Mend Renovate's Docker container contains a built-in job scheduler that defaults to enqueing all repositories once per hour. +This saves the need for configuring and monitoring any external `cron` process. + +#### Webhook handler + +Mend Renovate also supports a webserver to listen for system webhooks received from GitLab. + +In particular, Renovate checks webhooks for: + +- Projects it has just been added to +- Commits to `main` branch for "important" files such as `package.json` and `renovate.json` +- Any commits made to Renovate's branches +- Closing or merging of Renovate PRs + +Each of the above results in a job being enqueued for the relevant repository, so that the bot will appear responsive to users. + +#### Priority job queue + +Priority-based queuing is essential for providing a responsive experience for bot users. +For example, if a user makes an update to the config in an onboarding PR, they ideally want to see the results immediately. +By assigning onboarding updates the highest priority in the queue, the bot's update to the onboarding PR can proceed as the very next job, even if many others were in the queue already. + +In general, job priority is based on the probability that a user may be "waiting" for the bot to do something. +That's why onboarding updates are highest priority, and other high priority updates include merging of Renovate PRs because that very often results in other PRs needing updates or rebasing afterwards. + ## Architecture Logically, Mend Renovate consists of four components: @@ -65,7 +94,9 @@ Naturally, it is recommended that you use Renovate itself for detecting and upda ## Running Mend Renovate -[Examples using Docker Compose](https://github.com/mend/renovate-cc-ee/tree/main/examples) can be found in the `examples/` directory of this repository. +Examples using Docker Compose can be found at the bottom of the configuration documentation for each product. +- Configuration - Renovate Community Edition - ([GitHub](./configure-renovate-ce-github.md), [GitLab](./configure-renovate-ce-gitlab.md)) +- Configuration - Renovate Enterprise Edition - ([GitHub](./configure-renovate-ee-github.md), [GitLab](./configure-renovate-ee-gitlab.md)) Request `/status` on your IP address hosting the deployment to check if it is running correctly. It is recommended not to expose that endpoint to the internet as it could leak information about private repository names. diff --git a/examples/README.md b/examples/README.md deleted file mode 100644 index a253a398..00000000 --- a/examples/README.md +++ /dev/null @@ -1,5 +0,0 @@ -# Mend Renovate examples - -It's our goal that Mend Renovate's use be flexible, e.g. support whatever container management solution you prefer, such as plain Docker, Docker Compose, or Kubernetes. - -We've chosen Docker Compose as our main example here because (a) it's simpler than Kubernetes, (b) the number of environment variables necessary would look unwieldy to Docker CLI commands. diff --git a/examples/docker-compose-github.yml b/examples/docker-compose-github.yml deleted file mode 100644 index d6d5a1cc..00000000 --- a/examples/docker-compose-github.yml +++ /dev/null @@ -1,34 +0,0 @@ -version: "3.6" -services: - renovate: - image: ghcr.io/mend/renovate-ce:6.0.0-full - restart: on-failure - environment: - # Optional: Define log level. Set to Debug for more verbose output - # - LOG_LEVEL=DEBUG - # Provide Mend License and accept Terms of Service - - MEND_RNV_ACCEPT_TOS # Set to 'y' to accept the Mend Renovate Terms of Service - - MEND_RNV_LICENSE_KEY # Set this to the key you received by email - # Provide connection details for the GitHub App - - MEND_RNV_PLATFORM=github - - MEND_RNV_ENDPOINT # Required for GitHub Enterprise Server; not for GitHub.com. Include the trailing slash. - - MEND_RNV_GITHUB_APP_ID # GitHub App ID - - MEND_RNV_GITHUB_APP_KEY # GitHub App Key (PEM file). Alternatively mount as a volume below - - MEND_RNV_WEBHOOK_SECRET # Optional: defaults to 'renovate' - # Optional settings for Mend Renovate - # - MEND_RNV_ADMIN_API_ENABLED # Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. - # - MEND_RNV_SERVER_API_SECRET # Required if Admin APIs are enabled - # - MEND_RNV_SQLITE_FILE_PATH=/db/renovate-ce.sqlite # Optional: Provide a path to persist the database. Needs 'db' volume defined (below). - # - MEND_RNV_CRON_JOB_SCHEDULER # Optional Job enqueue schedule: defaults to '0 * * * *' (hourly, on the hour) - # - MEND_RNV_CRON_APP_SYNC # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 * * *' (every 4 hours, on the hour) - # Core Renovate settings - # - GITHUB_COM_TOKEN # Personal Access Token for github.com (used for retrieving changelogs) - ports: - - "80:8080" # Receive webhooks on port 80 - volumes: - # You can optionally mount the GitHub private key instead of exposing it in env - - "./renovate.private-key.pem:/usr/src/app/renovate.private-key.pem" - - "/tmp/renovate/:/tmp/" # Use a volume for this as the disk cache churns frequently - # Optional: You can use a file mount to persist the database between sessions - # - "/tmp/db/:/db/" # Unix-style file mounting for the db - # - "C:\\tmp/db/:/db/" # Windows-style file mounting for the db diff --git a/examples/docker-compose-gitlab.yml b/examples/docker-compose-gitlab.yml deleted file mode 100644 index bda442cc..00000000 --- a/examples/docker-compose-gitlab.yml +++ /dev/null @@ -1,31 +0,0 @@ -version: "3.6" -services: - renovate: - image: ghcr.io/mend/renovate-ce:6.0.0-full - restart: on-failure - environment: - # Optional: Define log level. Set to Debug for more verbose output - # - LOG_LEVEL=DEBUG - # Provide Mend License and accept Terms of Service - - MEND_RNV_ACCEPT_TOS # Set to 'y' to accept the Mend Renovate Terms of Service - - MEND_RNV_LICENSE_KEY # Set this to the key you received by email - # Provide connection details for the GitLab Bot - - MEND_RNV_PLATFORM=gitlab - - MEND_RNV_ENDPOINT # GitLab endpoint, include trailing slash - - MEND_RNV_GITLAB_PAT # Personal Access Token for bot account - - MEND_RNV_WEBHOOK_SECRET # Optional: defaults to 'renovate' - # Optional settings for Mend Renovate - # - MEND_RNV_ADMIN_API_ENABLED # Optional: Set to 'true' to enable Admin APIs. Defaults to 'false'. - # - MEND_RNV_SERVER_API_SECRET # Required if Admin APIs are enabled - # - MEND_RNV_SQLITE_FILE_PATH=/db/renovate-ce.sqlite # Optional: Provide a path to persist the database. Needs 'db' volume defined (below). - # - MEND_RNV_CRON_JOB_SCHEDULER # Optional Job enqueue schedule: defaults to '0 * * * *' (hourly, on the hour) - # - MEND_RNV_CRON_APP_SYNC # Optional AppSync schedule: defaults to '0 0,4,8,12,16,20 * * *' (every 4 hours, on the hour) - # Core Renovate settings - # - GITHUB_COM_TOKEN # Personal Access Token for github.com (used for retrieving changelogs) - ports: - - "80:8080" # Receive webhooks on port 80 - volumes: - - "/tmp/renovate/:/tmp/" # Use a volume for this as the cache will cause a lot of churn - # Optional: You can use a file mount to persist the database between sessions - # - "/tmp/db/:/db/" # Unix-style file mounting for the db - # - "C:\\tmp/db/:/db/" # Windows-style file mounting for the db