Skip to content
This repository has been archived by the owner on Sep 3, 2024. It is now read-only.

Commit

Permalink
Add asset, organization, and relationships
Browse files Browse the repository at this point in the history
  • Loading branch information
RPGPH authored and eXtremeX committed Jan 27, 2023
1 parent d37c6e0 commit 8367ed4
Show file tree
Hide file tree
Showing 35 changed files with 3,217 additions and 151 deletions.
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,32 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

### Added

- The following entities are added:

| Resources | Entity `_type` | Entity `_class` |
| ------------- | ------------------------- | --------------- |
| Account | `hackerone_account` | `Account` |
| Assessment | `hackerone_assessment` | `Assessment` |
| Organization | `hackerone_organization` | `Organization` |
| Program Asset | `hackerone_program_asset` | `Entity` |

- The following relationships are added:

| Source Entity `_type` | Relationship `_class` | Target Entity `_type` |
| ------------------------- | --------------------- | ------------------------- |
| `hackerone_account` | **HAS** | `hackerone_organization` |
| `hackerone_account` | **HAS** | `hackerone_program` |
| `hackerone_account` | **HAS** | `hackerone_program_asset` |
| `hackerone_organization` | **HAS** | `hackerone_program` |
| `hackerone_program_asset` | **HAS** | `hackerone_report` |
| `hackerone_program` | **PERFORMED** | `hackerone_assessment` |
| `hackerone_program` | **IDENTIFIED** | `hackerone_report` |
| `hackerone_program` | **SCANS** | `hackerone_program_asset` |

## 1.0.0 - 2022-12-21

### Changed
Expand Down
25 changes: 18 additions & 7 deletions docs/jupiterone.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,18 +80,29 @@ https://github.com/JupiterOne/sdk/blob/main/docs/integrations/development.md

The following entities are created:

| Resources | Entity `_type` | Entity `_class` |
| --------- | ------------------- | ----------------------- |
| Finding | `hackerone_report` | `Finding` |
| Service | `hackerone_program` | `Service`, `Assessment` |
| Resources | Entity `_type` | Entity `_class` |
| ------------- | ------------------------- | ----------------------- |
| Account | `hackerone_account` | `Account` |
| Assessment | `hackerone_assessment` | `Assessment` |
| Finding | `hackerone_report` | `Finding` |
| Organization | `hackerone_organization` | `Organization` |
| Program Asset | `hackerone_program_asset` | `Entity` |
| Service | `hackerone_program` | `Service`, `Assessment` |

### Relationships

The following relationships are created:

| Source Entity `_type` | Relationship `_class` | Target Entity `_type` |
| --------------------- | --------------------- | --------------------- |
| `hackerone_program` | **HAS** | `hackerone_report` |
| Source Entity `_type` | Relationship `_class` | Target Entity `_type` |
| ------------------------- | --------------------- | ------------------------- |
| `hackerone_account` | **HAS** | `hackerone_organization` |
| `hackerone_account` | **HAS** | `hackerone_program` |
| `hackerone_account` | **HAS** | `hackerone_program_asset` |
| `hackerone_organization` | **HAS** | `hackerone_program` |
| `hackerone_program_asset` | **HAS** | `hackerone_report` |
| `hackerone_program` | **PERFORMED** | `hackerone_assessment` |
| `hackerone_program` | **IDENTIFIED** | `hackerone_report` |
| `hackerone_program` | **SCANS** | `hackerone_program_asset` |

### Mapped Relationships

Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@
},
"dependencies": {
"@lifeomic/attempt": "^3.0.3",
"hackerone-client": "^1.0.7"
"hackerone-client": "^1.0.7",
"node-fetch": "2"
},
"peerDependencies": {
"@jupiterone/integration-sdk-core": "^8.22.0"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"headers": [
{
"name": "date",
"value": "Tue, 20 Dec 2022 17:46:08 GMT"
"value": "Mon, 23 Jan 2023 10:20:14 GMT"
},
{
"name": "content-type",
Expand All @@ -53,7 +53,7 @@
},
{
"name": "connection",
"value": "close"
"value": "keep-alive"
},
{
"name": "www-authenticate",
Expand All @@ -65,7 +65,7 @@
},
{
"name": "x-request-id",
"value": "22b9c8dd-ac19-4075-8796-b09c18877610"
"value": "1c8651f7-88a1-4107-9a2f-497c1c7e53df"
},
{
"name": "cache-control",
Expand Down Expand Up @@ -105,7 +105,7 @@
},
{
"name": "content-security-policy",
"value": "default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src www.youtube-nocookie.com; connect-src 'self' errors.hackerone.net *.browser-intake-datadoghq.com; font-src 'self'; form-action 'self'; frame-ancestors 'none'; frame-src hackerone.integration-configuration.com api-60d81e65.duosecurity.com; img-src 'self' data: cover-photos.hackerone-user-content.com hackathon-photos.hackerone-user-content.com profile-photos.hackerone-user-content.com hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; media-src 'self' hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; script-src 'self'; style-src 'self' 'unsafe-inline'; report-uri https://errors.hackerone.net/api/30/security/?sentry_key=374aea95847f4040a69f9c8d49a3a59d&sentry_environment=production"
"value": "default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src www.youtube-nocookie.com; connect-src 'self' errors.hackerone.net *.browser-intake-datadoghq.com; font-src 'self'; form-action 'self'; frame-ancestors 'none'; frame-src hackerone.integration-configuration.com api-60d81e65.duosecurity.com; img-src 'self' data: cover-photos.hackerone-user-content.com hackathon-photos.hackerone-user-content.com profile-photos.hackerone-user-content.com hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; media-src 'self' marketing-assets.hackerone-user-content.com hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; script-src 'self'; style-src 'self' 'unsafe-inline'; report-uri https://errors.hackerone.net/api/30/security/?sentry_key=374aea95847f4040a69f9c8d49a3a59d&sentry_environment=production"
},
{
"name": "cf-cache-status",
Expand All @@ -117,25 +117,25 @@
},
{
"name": "cf-ray",
"value": "77ca32187d69a038-SLC"
"value": "78dfcbac3a538b75-HKG"
}
],
"headersSize": 1490,
"headersSize": 1539,
"httpVersion": "HTTP/1.1",
"redirectURL": "",
"status": 401,
"statusText": "Unauthorized"
},
"startedDateTime": "2022-12-20T17:46:07.936Z",
"time": 210,
"startedDateTime": "2023-01-23T10:20:13.809Z",
"time": 371,
"timings": {
"blocked": -1,
"connect": -1,
"dns": -1,
"receive": 0,
"send": 0,
"ssl": -1,
"wait": 210
"wait": 371
}
}
],
Expand Down
18 changes: 9 additions & 9 deletions src/__recordings__/handle-auth-error_3507045143/recording.har
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@
"headers": [
{
"name": "date",
"value": "Tue, 20 Dec 2022 17:46:08 GMT"
"value": "Mon, 23 Jan 2023 10:20:13 GMT"
},
{
"name": "content-type",
Expand All @@ -53,7 +53,7 @@
},
{
"name": "connection",
"value": "close"
"value": "keep-alive"
},
{
"name": "www-authenticate",
Expand All @@ -65,7 +65,7 @@
},
{
"name": "x-request-id",
"value": "1f732d4b-f636-4b7f-b93a-37f4927d1925"
"value": "7adf02b4-d3b3-423d-85be-79962386605d"
},
{
"name": "cache-control",
Expand Down Expand Up @@ -105,7 +105,7 @@
},
{
"name": "content-security-policy",
"value": "default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src www.youtube-nocookie.com; connect-src 'self' errors.hackerone.net *.browser-intake-datadoghq.com; font-src 'self'; form-action 'self'; frame-ancestors 'none'; frame-src hackerone.integration-configuration.com api-60d81e65.duosecurity.com; img-src 'self' data: cover-photos.hackerone-user-content.com hackathon-photos.hackerone-user-content.com profile-photos.hackerone-user-content.com hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; media-src 'self' hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; script-src 'self'; style-src 'self' 'unsafe-inline'; report-uri https://errors.hackerone.net/api/30/security/?sentry_key=374aea95847f4040a69f9c8d49a3a59d&sentry_environment=production"
"value": "default-src 'none'; base-uri 'self'; block-all-mixed-content; child-src www.youtube-nocookie.com; connect-src 'self' errors.hackerone.net *.browser-intake-datadoghq.com; font-src 'self'; form-action 'self'; frame-ancestors 'none'; frame-src hackerone.integration-configuration.com api-60d81e65.duosecurity.com; img-src 'self' data: cover-photos.hackerone-user-content.com hackathon-photos.hackerone-user-content.com profile-photos.hackerone-user-content.com hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; media-src 'self' marketing-assets.hackerone-user-content.com hackerone-us-west-2-production-attachments.s3.us-west-2.amazonaws.com; script-src 'self'; style-src 'self' 'unsafe-inline'; report-uri https://errors.hackerone.net/api/30/security/?sentry_key=374aea95847f4040a69f9c8d49a3a59d&sentry_environment=production"
},
{
"name": "cf-cache-status",
Expand All @@ -117,25 +117,25 @@
},
{
"name": "cf-ray",
"value": "77ca32145d02a03e-SLC"
"value": "78dfcba9cfe48b75-HKG"
}
],
"headersSize": 1490,
"headersSize": 1539,
"httpVersion": "HTTP/1.1",
"redirectURL": "",
"status": 401,
"statusText": "Unauthorized"
},
"startedDateTime": "2022-12-20T17:46:07.276Z",
"time": 635,
"startedDateTime": "2023-01-23T10:20:13.417Z",
"time": 377,
"timings": {
"blocked": -1,
"connect": -1,
"dns": -1,
"receive": 0,
"send": 0,
"ssl": -1,
"wait": 635
"wait": 377
}
}
],
Expand Down
22 changes: 11 additions & 11 deletions src/__recordings__/validate-invocation_3585932400/recording.har

Large diffs are not rendered by default.

110 changes: 109 additions & 1 deletion src/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,100 @@ import {

import { IntegrationConfig } from './config';
import { retry } from '@lifeomic/attempt';
import { Report } from './types';
import {
HackerOneOrganization,
HackerOneProgram,
HackerOneStructuredScope,
Report,
} from './types';
import fetch, { Response } from 'node-fetch';

export type ResourceIteratee<T> = (each: T) => Promise<void> | void;
export const HACKERONE_CLIENT_404_ERROR = 'StatusCodeError: 404'; // TODO/HACK: Underlying library not re-throwing status codes correctly so this instead does a substring match to detect if a 404 (e.g., no direct access to err.errors[])

export class APIClient {
private baseUrl = 'https://api.hackerone.com/v1/';
private hackeroneClient;
private limit = 50;

constructor(readonly config: IntegrationConfig) {
this.hackeroneClient = new HackeroneClient(
config.hackeroneApiKey,
config.hackeroneApiKeyName,
);
}

private withBaseUrl = (path: string) => `${this.baseUrl}${path}`;

// To query endpoints not supported in hackerone-client
private async request(uri: string): Promise<Response> {
try {
const result = await retry(
async () => {
const response = await fetch(uri, {
headers: {
Authorization: `Basic ${Buffer.from(
`${this.config.hackeroneApiKeyName}:${this.config.hackeroneApiKey}`,
).toString('base64')}`,
},
});
if (!response.ok) {
throw new IntegrationProviderAPIError({
endpoint: uri,
status: response.status,
statusText: response.statusText,
});
}
return response;
},
{
delay: 1000,
factor: 2,
maxAttempts: 10,
handleError: (err, context) => {
const rateLimitType = err.response.headers.get('X-RateLimit-Type');
// only retry on 429 && per second limit
if (!(err.status === 429 && rateLimitType === 'QPS')) {
context.abort();
}
},
},
);
return result;
} catch (error) {
throw new IntegrationProviderAPIError({
endpoint: uri,
status: error.status,
statusText: error.statusText,
});
}
}

private async paginatedRequest<T>(
uri: string,
iteratee: ResourceIteratee<T>,
): Promise<void> {
try {
let current = `${uri}?page[number]=1&page[size]=${this.limit}`;
let response: Response;
do {
response = await this.request(current);

const { data, links } = await response.json();
current = links?.last !== links?.self ? links?.next : '';

for (const resource of data) await iteratee(resource);
} while (current);
} catch (err) {
throw new IntegrationProviderAPIError({
cause: new Error(err.message),
endpoint: uri,
status: err.statusCode,
statusText: err.message,
});
}
}

public async verifyAuthentication(): Promise<void> {
try {
await this.hackeroneClient.getPrograms();
Expand Down Expand Up @@ -71,6 +151,34 @@ export class APIClient {
factor: 2, //exponential backoff factor. with 30 sec start and 3 attempts, longest wait is 2 min
});
}

// API key is for a single organization only. Pagination is
// unnecessary but is implemented due to endpoint design.
public async fetchOrganization(
iteratee: ResourceIteratee<HackerOneOrganization>,
): Promise<void> {
const url = this.withBaseUrl('me/organizations');
await this.paginatedRequest(url, iteratee);
}

public async iteratePrograms(
iteratee: ResourceIteratee<HackerOneProgram>,
): Promise<void> {
const programs = await this.hackeroneClient.getPrograms();
const { data } = JSON.parse(programs);

for (const program of data) {
await iteratee(program);
}
}

public async iterateProgramAsset(
programId: string,
iteratee: ResourceIteratee<HackerOneStructuredScope>,
): Promise<void> {
const url = this.withBaseUrl(`programs/${programId}/structured_scopes`);
await this.paginatedRequest(url, iteratee);
}
}

let apiClient: APIClient;
Expand Down
26 changes: 26 additions & 0 deletions src/steps/account/converter.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import {
createIntegrationEntity,
Entity,
IntegrationInstance,
} from '@jupiterone/integration-sdk-core';
import { IntegrationConfig } from '../../config';
import { Entities } from '../constants';

export function createAccountEntity(
data: IntegrationInstance<IntegrationConfig>,
): Entity {
const { config, ...rest } = data;
return createIntegrationEntity({
entityData: {
source: rest,
assign: {
_key: `hackerone_account`,
_type: Entities.ACCOUNT._type,
_class: Entities.ACCOUNT._class,
name: data.name,
id: data.id,
program: data.config.hackeroneProgramHandle,
},
},
});
}
21 changes: 21 additions & 0 deletions src/steps/account/index.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
import { executeStepWithDependencies } from '@jupiterone/integration-sdk-testing';
import { buildStepTestConfigForStep } from '../../../test/config';
import { Recording, setupProjectRecording } from '../../../test/recording';
import { Steps } from '../constants';

// See test/README.md for details
let recording: Recording;
afterEach(async () => {
await recording.stop();
});

test('fetch-account', async () => {
recording = setupProjectRecording({
directory: __dirname,
name: 'fetch-account',
});

const stepConfig = buildStepTestConfigForStep(Steps.ACCOUNT);
const stepResult = await executeStepWithDependencies(stepConfig);
expect(stepResult).toMatchStepMetadata(stepConfig);
});
Loading

0 comments on commit 8367ed4

Please sign in to comment.