Skip to content

Commit

Permalink
docs(circlrci): add doc for circleci (#702)
Browse files Browse the repository at this point in the history
  • Loading branch information
d4x1 authored Jan 19, 2024
1 parent c376c63 commit 2a360b6
Show file tree
Hide file tree
Showing 13 changed files with 362 additions and 201 deletions.
86 changes: 86 additions & 0 deletions docs/Configuration/CircleCI.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
---
title: "CircleCI"
sidebar_position: 8
description: Config UI instruction for CircleCI
---

Visit Config UI at : `http://localhost:4000` and go to `Connections` page.

## Step 1 - Add Data Connections

### Step 1.1 - Authentication

![image](/img/ConfigUI/circleci-add-data-connections.png)

#### Connection Name

Give your connection a unique name to help you identify it in the future.

#### Endpoint URL

For CircleCI, you do not need to enter the REST API endpoint URL, which is always `https://circleci.com/api/`.


#### Token

Learn about [Managing API Tokens](https://circleci.com/docs/managing-api-tokens/).

Tokens you have generated that can be used to access the CircleCI API. Apps using these tokens can act as you and have full read- and write-permissions!
There are two types of API token(Personal and Project) you can create within CircleCI.


#### Proxy URL (Optional)

If you are behind a corporate firewall or VPN you may need to utilize a proxy server. Enter a valid proxy server address on your network, e.g. `http://your-proxy-server.com:1080`


#### Fixed Rate Limit (Optional)

DevLake uses a dynamic rate limit to collect CircleCI data. You can adjust the rate limit if you want to increase or lower the speed.
Learn more about [CircleCI API rate limit](https://circleci.com/docs/api-developers-guide/#rate-limits).


#### Test and Save Connection

Click `Test Connection`, if the connection is successful, click `Save Connection` to add the connection.

### Step 1.2 - Configure Data Scope

![image](/img/ConfigUI/circleci-choose-data-scope.png)

#### Projects

Select the CircleCI projects to collect.

### Step 1.3 - Adding Scope Config (Optional)
![image](/img/ConfigUI/circleci-scope-config.png)
You can add a `transformation` to standardize the data. A `transformation` acts on the CircleCI data in the [tool layer](/docs/DataModels/ToolLayerSchema.md), transforming it to the [domain layer](/docs/DataModels/DevLakeDomainLayerSchema.md).

## Step 2 - Collect Data in a Project
### Step 2.1 - Create a Project
Collecting CircleCI data requires creating a project first. You can visit the Project page from the side menu and create a new project by following the instructions on the user interface.

![create-a-project](images/create-a-project.png)

### Step 2.2 - Add a CircleCI Connection
You can add a previously configured CircleCI connection to the project and select the boards for which you wish to collect the data for.
Please note: if you don't see the repositories you are looking for, please check if you have added them to the connection first.

![add-a-connection](images/add-a-connection-project.png)

### Step 2.3 - Set the Sync Policy
There are three settings for Sync Policy:
- Data Time Range: You can select the time range of the data you wish to collect. The default is set to the past six months.
- Sync Frequency: You can choose how often you would like to sync your data in this step by selecting a sync frequency option or enter a cron code to specify your preferred schedule.
- Skip Failed Tasks: sometime a few tasks may fail in a long pipeline; you can choose to skip them to avoid spending more time in running the pipeline all over again.

![sync-policy](images/sync-policy.png)

### Step 2.4 - Start Data Collection
Click on "Collect Data" to start collecting data for the whole project. You can check the status in the Status tab on the same page.
![collect-data](images/collect-data.png)


## Troubleshooting

If you run into any problem, please check the [Troubleshooting](/Troubleshooting/Configuration.md) or [create an issue](https://github.com/apache/incubator-devlake/issues)
8 changes: 4 additions & 4 deletions docs/DataModels/DevLakeDomainLayerSchema.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,10 +119,10 @@ This table shows the assignee(s) of issues. Multiple entries can exist per issue

This table shows the labels of issues. Multiple entries can exist per issue. This table can be used to filter issues by label name.

| **field** | **type** | **length** | **description** | **key** |
|:-----------|:---------|:-----------|:------------------------------------------------------------------|:-------------|
| `name` | varchar | 255 | Label name. Collect from GitHub issue labels or Jira issue labels | |
| `issue_id` | varchar | 255 | Issue ID | FK_issues.id |
| **field** | **type** | **length** | **description** | **key** |
|:-------------|:---------|:-----------|:------------------------------------------------------------------|:-------------|
| `label_name` | varchar | 255 | Label name. Collect from GitHub issue labels or Jira issue labels | |
| `issue_id` | varchar | 255 | Issue ID | FK_issues.id |

#### issue_comments

Expand Down
Loading

0 comments on commit 2a360b6

Please sign in to comment.