feat: opt-in gh app integration

[plyr4]

this PR adds opt-in GitHub App integrations to the server.

the main differences are:

• integrate with an OAuth GitHub App by providing an app id and private key
• use the GitHub App to generate the .netrc password for builds when the app is installed and it can access the repo
• adds webhook handlers for GitHub App installation and installation_repositories events, and syncs existing repos when changes are made
• adds Checks functionality, pulled from the original hackathon efforts feat: add github app #1070
  • pulled out this code, it will be a follow-up PR

New Flags

Key	Description	Default Value
VELA_SCM_APP_ID	set to the App ID for the GitHub App	N/A
VELA_SCM_APP_PRIVATE_KEY	the string value for the GitHub App private key generated through GitHub	N/A

Required GitHub App Configurations

Permissions

the GitHub App requires the following permissions at the very minimum:

• contents:read
• checks:write

builds would request write permissions through the git yaml block, see below.

Subscribed Events

• Installation target

OAuth

the same configurations and oauth scopes should be assigned to the GitHub App, including:

• oauth callback url set to /authenticate (like usual)
• Webhook URL set to the base url (like usual)

New YAML block: git

integrating with a GitHub App allows the use of the git YAML block for customizing the permissions allocated to the netrc password embedded into Vela steps.

git:
  token:
    repositories:
      - foo/bar
      - helloworld
    permissions:
      contents: write
      checks: write

this lets users customize the list of repositories that the netrc password has access to, but that list is restricted to ONLY the repos that the GitHub App org installation has been given access to.

by default, the compiler will use the following configurations unless otherwise provided:

git:
  token:
    repositories:
      - VELA_BUILD_REPO
    permissions:
      contents: read
      checks: write

⚠️ Netrc Considerations

this WILL impact builds, check out the following list of things to consider when migrating to GitHub App

Cloning Private Repositories

Vela builds might lose the ability to read/write from certain private repos that the repo owner may have had access to due to the new restrictive policies set on the netrc token.

GitHub Apps do not support providing access to repos that are outside the installation org. meaning, for a Vela build to access private repos, Go modules, etc, that are outside of the repo's org then the build author must provide override the clone step and use an alternative authentication method like a PAT

⚠️ OAuth Considerations

For this release, we recommend using a combination of both an OAuth app and a GitHub App with Authorization disabled.
The rest of the code base is not prepared to require user App installations, see the below examples for why.

Enabling Private Repositories (/source/repos)

Vela users will lose the ability to enable private repositories unless the GitHub App is installed to their personal account. this is due to changes to the default permissions when using a GitHub App as an OAuth provider.
see: https://docs.github.com/en/[email protected]/apps/using-github-apps/authorizing-github-apps#difference-between-authorization-and-installation

[codecov]

Codecov Report

Attention: Patch coverage is 45.04717% with 466 lines in your changes missing coverage. Please review.

Project coverage is 56.50%. Comparing base (9a4003b) to head (cd2deac).

Files with missing lines	Patch %	Lines
scm/github/app_transport.go	25.56%	130 Missing and 1 partial ⚠️
scm/github/app_install.go	0.00%	97 Missing ⚠️
scm/github/github.go	13.43%	57 Missing and 1 partial ⚠️
scm/github/repo.go	73.14%	22 Missing and 7 partials ⚠️
scm/github/github_client.go	74.22%	19 Missing and 6 partials ⚠️
api/repo/repair.go	0.00%	23 Missing ⚠️
api/auth/get_token.go	0.00%	22 Missing ⚠️
scm/github/opts.go	35.71%	18 Missing ⚠️
api/webhook/post.go	0.00%	12 Missing ⚠️
compiler/native/native.go	0.00%	12 Missing ⚠️
... and 8 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1217      +/-   ##
==========================================
- Coverage   56.87%   56.50%   -0.38%     
==========================================
  Files         599      605       +6     
  Lines       32869    33617     +748     
==========================================
+ Hits        18695    18995     +300     
- Misses      13538    13971     +433     
- Partials      636      651      +15     

Files with missing lines	Coverage Δ
compiler/native/environment.go	87.22% <100.00%> (+0.46%)	⬆️
compiler/registry/github/github.go	100.00% <100.00%> (ø)
compiler/registry/github/template.go	58.53% <100.00%> (ø)
compiler/types/yaml/build.go	91.04% <100.00%> (+0.27%)	⬆️
compiler/types/yaml/git.go	100.00% <100.00%> (ø)
database/repo/table.go	100.00% <ø> (ø)
database/testutils/api_resources.go	94.81% <100.00%> (+0.01%)	⬆️
database/types/repo.go	96.73% <100.00%> (+0.03%)	⬆️
internal/webhook.go	100.00% <ø> (ø)
mock/server/repo.go	0.00% <ø> (ø)
... and 26 more

[JordanSussman]

but that list is restricted to ONLY the repos that the GitHub App org installation has been given access to.

I think I know the answer, but just to confirm: if I install the GitHub app on both the foo and bar organizations and run a build within a repo in the foo org, does that mean I won’t be able to clone a repo from the bar org, even if I specify bar/<repo> in the token section?

[plyr4]

but that list is restricted to ONLY the repos that the GitHub App org installation has been given access to.

I think I know the answer, but just to confirm: if I install the GitHub app on both the foo and bar organizations and run a build within a repo in the foo org, does that mean I won’t be able to clone a repo from the bar org, even if I specify bar/<repo> in the token section?

that's correct, org installations are scoped to repos in that org, unfortunately. its extremely similar to fine-grained access tokens (the beta PATs).

the workaround is to supply a github classic PAT created by a user with access to all the things

[ecrupper]

I think that is definitely more standard