Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GitHub app auth #21

Merged
merged 3 commits into from
Jan 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -94,3 +94,5 @@ typings/

# End of https://www.gitignore.io/api/node
xunit.xml

github-app.pem
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM node:14
FROM node:16

ENV NODE_ENV production
ENV PORT 3000
Expand Down
17 changes: 10 additions & 7 deletions IncrementalsPlugin.js
Original file line number Diff line number Diff line change
Expand Up @@ -223,11 +223,7 @@ class IncrementalsPlugin {
}
this.context.log.info('Archive entries %o', entries);

const pom = entries.find(entry => entry.endsWith('.pom'));
if (!pom) {
this.context.log.error('No POM');
throw new FailRequestError('No POM');
}
const pom = entries[0].path;
this.context.log.info('Found a POM %s', pom);
const pomURL = config.INCREMENTAL_URL + pom;
const check = await this.fetch(pomURL);
Expand All @@ -241,12 +237,19 @@ class IncrementalsPlugin {
*/
const upload = await this.uploadToArtifactory(archivePath, pomURL);

const result = await this.github.createStatus(folderMetadataParsed.owner, folderMetadataParsed.repo, buildMetadataParsed.hash, pomURL.replace(/[^/]+$/, ''))
const entriesForDisplay = entries.map(entry => {
return {
artifactId: entry.artifactId,
url: config.INCREMENTAL_URL + entry.path.replace(/[^/]+$/, '')
};
})

const result = await this.github.createStatus(folderMetadataParsed.owner, folderMetadataParsed.repo, buildMetadataParsed.hash, entriesForDisplay)
// ignore any actual errors, just log it
.catch(err => err);

if (result.status >= 300) {
this.context.log.error('Failed to create github status, code: %d for repo: %s/%s, check your GitHub credentials', result.status, folderMetadataParsed.owner, folderMetadataParsed.repo);
this.context.log.error('Failed to create github status, code: %d for repo: %s/%s, check your GitHub credentials, err: %s', result.status, folderMetadataParsed.owner, folderMetadataParsed.repo, result);
} else {
this.context.log.info('Created github status for pom: %s', pom);
}
Expand Down
4 changes: 2 additions & 2 deletions Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ if (JENKINS_URL.contains('ci.jenkins.io')) {
checkout scm
}
stage('NPM Install') {
runDockerCommand('node:14', 'npm ci')
runDockerCommand('node:16', 'npm ci')
}
stage('Lint and Test') {
runDockerCommand('node:14', 'npm run test')
runDockerCommand('node:16', 'npm run test')
}
}
}
Expand Down
18 changes: 11 additions & 7 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Incremental Build Publisher


This Azure Function is responsible for bouncing incrementally built (see
This service is responsible for uploading incrementally built (see
link:https://github.com/jenkinsci/jep/tree/master/jep/305[JEP-305]) core and
plugin artifacts into the incrementals Maven repository (see
link:https://github.com/jenkins-infra/iep/tree/master/iep-009[IEP-9]).
Expand All @@ -13,8 +13,14 @@ link:https://github.com/jenkins-infra/iep/tree/master/iep-009[IEP-9]).
|===
| Variable | Description

| `GITHUB_TOKEN`
| A GitHub OAuth/Personal Access Token which can be used for GitHub API queries, ideally also has access to set the `repo.status` permission.
| `GITHUB_APP_ID`
| The App ID for a GitHub app

| `GITHUB_APP_PRIVATE_KEY`
| The private key for a GitHub app

| `GITHUB_APP_INSTALLATION_ID`
| The installation ID for the organisation / user account that the GitHub app has been installed to

| `JENKINS_HOST`
| A Jenkins instance (defaults to `https://ci.jenkins.io/`) to which URLs are
Expand Down Expand Up @@ -57,7 +63,7 @@ defaults to

== Valid Requests

This Function only supports one kind of request, and is expected to only be
This service only supports one kind of request, and is expected to only be
called by the `buildPlugin()` step or other Jenkins Pipelines which are
publishing incrementally built artifacts into Artifactory.

Expand All @@ -71,8 +77,6 @@ publishing incrementally built artifacts into Artifactory.
This URL is expected to be the value of a `BUILD_URL` environment variable from
Jenkins.



== Testing

Unit tests can simply be run by executing `make check` in this directory.
Expand All @@ -85,5 +89,5 @@ Function, for example:

[source,bash]
----
curl -i -d '{"build_url":"http://localhost:8080/job/dumb/3/"}' http://localhost:7071/api/incrementals-publisher
curl -H "Authorization: Bearer ${PRESHARED_KEY}" -H 'Content-Type: application/json' -i -d '{"build_url":"https://ci.jenkins.io/job/Plugins/job/jenkins-infra-test-plugin/job/master/52/"}' http://localhost:3000
----
3 changes: 2 additions & 1 deletion lib/config.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
module.exports = {};
Object.entries({
GITHUB_TOKEN: 'invalid-dummy-secret',
GITHUB_APP_ID: 'invalid-dummy-id',
GITHUB_APP_PRIVATE_KEY: 'invalid-dummy-secret',
PERMISSIONS_URL: 'https://ci.jenkins.io/job/Infra/job/repository-permissions-updater/job/master/lastSuccessfulBuild/artifact/json/github.index.json',
JENKINS_HOST: 'https://ci.jenkins.io/',
INCREMENTAL_URL: 'https://repo.jenkins-ci.org/incrementals/',
Expand Down
44 changes: 30 additions & 14 deletions lib/github.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,28 @@
*/

const { Octokit } = require("@octokit/rest");
const { createAppAuth } = require('@octokit/auth-app');
const { request } = require("@octokit/request");

const GITHUB_TOKEN = process.env.GITHUB_TOKEN || 'invalid-dummy-secret';
const APP_ID = process.env.GITHUB_APP_ID;
const PRIVATE_KEY = process.env.GITHUB_APP_PRIVATE_KEY;

const INSTALLATION_ID = process.env.GITHUB_APP_INSTALLATION_ID || 22187127

async function getRestClient() {
return new Octokit({
authStrategy: createAppAuth,
auth: {
appId: APP_ID,
privateKey: PRIVATE_KEY,
installationId: INSTALLATION_ID
}
})
}

module.exports = {
commitExists: async (owner, repo, ref) => {
let github = new Octokit({
auth: GITHUB_TOKEN
});
const github = await getRestClient();
/*
* Ensure that the commit is actually present in our repository! No sense
* doing any work with it if it's somehow not published.
Expand All @@ -22,18 +36,20 @@ module.exports = {
return !!commit;
},

createStatus: async (owner, repo, sha, target_url) => {
let github = new Octokit({
auth: GITHUB_TOKEN
});
return github.repos.createCommitStatus({
createStatus: async (owner, repo, head_sha, entries) => {
const github = await getRestClient();
return github.rest.checks.create({
owner,
repo,
sha,
state: 'success',
target_url,
description: 'Deployed to Incrementals.',
context: 'continuous-integration/jenkins/incrementals'
name: 'Incrementals',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this also looks like a change. Your changing from a commit status to a check?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, so I can get multi modules giving all links, was horrid UX for jenkins core before and others you had to replace the first modules artifact id with the one you wanted in the URL, it never got the hpi / war

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had filed an issue to this effect somewhere. Maybe only jenkins-infra/helpdesk#2426.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

head_sha,
status: 'completed',
conclusion: 'success',
details_url: entries[0].url,
output: {
title: 'Deployed to Incrementals',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should pick text that is more self-descriptive for people who do not already know what this means? Somehow indicating that this snapshot-ish build is available for download or consumption from Maven in draft PRs, linking to jenkins.io docs, etc.

summary: entries.map(entry => `- [${entry.artifactId}](${entry.url})`).join('\n')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And a good follow-up would be to show

<dependency>
    <groupId>org.jenkins-ci.plugins</groupId>
    <artifactId>stuff</artifactId>
    <version>1.2-rcdead1234beef</version>
</dependency>

Copy link
Member

@lemeurherve lemeurherve Feb 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in #83 & #84

image

}
});
}
};
5 changes: 4 additions & 1 deletion lib/permissions.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ module.exports = {
const zip = new StreamZip({file: archive});

zip.on('entry', async function (entry) {
entries.push(entry.name);
let ok = !!applicable.find(file => {
const isMatch = wcmatch(file, { separator: '|' })
return isMatch(entry.name) || entry.name.startsWith(file);
Expand Down Expand Up @@ -57,6 +56,10 @@ module.exports = {
const groupId = result.project.groupId[0];
const artifactId = result.project.artifactId[0];
const version = result.project.version[0];
entries.push({
artifactId,
path: entry.name
});
log.info(util.format('Parsed %s with url=%s tag=%s GAV=%s:%s:%s', entry.name, url, tag, groupId, artifactId, version));
const expectedPath = groupId.replace(/[.]/g, '/') + '/' + artifactId + '/' + version + '/' + artifactId + '-' + version + '.pom';
if (tag !== hash) {
Expand Down
Loading