Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚨 Source Gitlab: certification improvements: epics pk, date-time format, docs, allowed hosts #31700

Merged
merged 11 commits into from
Oct 26, 2023
Merged

🚨 Source Gitlab: certification improvements: epics pk, date-time format, docs, allowed hosts #31700

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
44beb1a
changed epics PK from iid to id
darynaishchenko Oct 23, 2023
caae1d4
updated format for date-time fileds in deployments, group/projects me…
darynaishchenko Oct 23, 2023
a57e96c
updated docs and allowed hosts, encreased connector version
darynaishchenko Oct 23, 2023
0f3590f
updated changelog
darynaishchenko Oct 23, 2023
6a67e4f
revert pk changes, add pk definition for epic docs
darynaishchenko Oct 23, 2023
ef9810a
added breakingChanges to metadata
darynaishchenko Oct 23, 2023
a95b4ce
Automated Commit - Formatting Changes
darynaishchenko Oct 23, 2023
24b9eb0
added migration guide
darynaishchenko Oct 23, 2023
8eccb8d
expand epic pk descriprion
darynaishchenko Oct 24, 2023
27d8798
updated descriptions
darynaishchenko Oct 26, 2023
6679125
updated upgradeDeadline
darynaishchenko Oct 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-gitlab/acceptance-test-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ acceptance_tests:
tests:
- config_path: "secrets/config.json"
backward_compatibility_tests_config:
disable_for_version: 1.7.1
disable_for_version: 1.8.4
darynaishchenko marked this conversation as resolved.
Show resolved Hide resolved
basic_read:
tests:
- config_path: "secrets/config.json"
Expand Down
12 changes: 10 additions & 2 deletions airbyte-integrations/connectors/source-gitlab/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ data:
sl: 200
allowedHosts:
hosts:
- "*"
- ${api_url}
davydov-d marked this conversation as resolved.
Show resolved Hide resolved
connectorBuildOptions:
baseImage: docker.io/airbyte/python-connector-base:1.1.0@sha256:bd98f6505c6764b1b5f99d3aedc23dfc9e9af631a62533f60eb32b1d3dbab20c
connectorSubtype: api
connectorType: source
definitionId: 5e6175e5-68e1-4c17-bff9-56103bbb0d80
dockerImageTag: 1.8.4
dockerImageTag: 2.0.0
dockerRepository: airbyte/source-gitlab
documentationUrl: https://docs.airbyte.com/integrations/sources/gitlab
githubIssueLabel: source-gitlab
Expand All @@ -23,6 +23,14 @@ data:
oss:
enabled: true
releaseStage: generally_available
releases:
breakingChanges:
2.0.0:
message:
In this release, several streams were updated to date-time field format, as declared in the Gitlab API.
These changes impact pipeline.created_at and pipeline.updated_at fields for stream Deployments and expires_at field for stream Group Members and stream Project Members.
Users will need to refresh the source schema and reset affected streams after upgrading.
darynaishchenko marked this conversation as resolved.
Show resolved Hide resolved
upgradeDeadline: "2023-11-07"
suggestedStreams:
streams:
- merge_requests
Expand Down
6 changes: 4 additions & 2 deletions airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/deployments.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,8 @@
"type": ["null", "object"],
"properties": {
"created_at": {
"type": ["null", "string"]
"type": ["null", "string"],
"format": "date-time"
},
"id": {
"type": ["null", "integer"]
Expand All @@ -195,7 +196,8 @@
"type": ["null", "string"]
},
"updated_at": {
"type": ["null", "string"]
"type": ["null", "string"],
"format": "date-time"
},
"web_url": {
"type": ["null", "string"]
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/group_members.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
},
"expires_at": {
"type": ["null", "string"],
"format": "date"
"format": "date-time"
},
"created_by": {
"avatar_url": {
Expand Down
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/project_members.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
},
"expires_at": {
"type": ["null", "string"],
"format": "date"
"format": "date-time"
},
"created_by": {
"avatar_url": {
Expand Down
8 changes: 8 additions & 0 deletions docs/integrations/sources/gitlab-migrations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Gitlab Migration Guide

## Upgrading to 2.0.0


In the 2.0.0 config change, several streams were updated to date-time field format, as declared in the Gitlab API.
These changes impact `pipeline.created_at` and` pipeline.updated_at` fields for stream Deployments and `expires_at` field for stream Group Members and stream Project Members.
You will need to refresh the source schema and reset affected streams after upgrading.
10 changes: 4 additions & 6 deletions docs/integrations/sources/gitlab.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,6 @@ This page contains the setup guide and reference information for the Gitlab Sour
## Prerequisites

- Gitlab instance or an account at [Gitlab](https://gitlab.com)
- Start date (Optional)
- GitLab Groups (Optional)
- GitLab Projects (Optional)

<!-- env:cloud -->

Expand Down Expand Up @@ -49,7 +46,7 @@ Log into [GitLab](https://gitlab.com) and then generate a [personal access token
3. On the source setup page, select **GitLab** from the Source type dropdown and enter a name for this connector.
4. Click `Authenticate your GitLab account` by selecting Oauth or Personal Access Token for Authentication.
5. Log in and Authorize to the GitLab account.
6. **API URL** - The URL to access you self-hosted GitLab instance or `gitlab.com` (default).
6. **API URL (Optional)** - The URL to access your self-hosted GitLab instance or `gitlab.com` (default).
7. **Start date (Optional)** - The date from which you'd like to replicate data for streams.
8. **Groups (Optional)** - List of GitLab group IDs, e.g. `airbytehq` for single group, `airbytehq another-repo` for multiple groups.
9. **Projects (Optional)** - List of GitLab projects to pull data for, e.g. `airbytehq/airbyte`.
Expand Down Expand Up @@ -97,7 +94,7 @@ This connector outputs the following streams:
- [Deployments](https://docs.gitlab.com/ee/api/deployments/index.html)
- [Group Labels](https://docs.gitlab.com/ee/api/group_labels.html)
- [Project Labels](https://docs.gitlab.com/ee/api/labels.html)
- [Epics](https://docs.gitlab.com/ee/api/epics.html) \(only available for GitLab Ultimate and GitLab.com Gold accounts\)
- [Epics](https://docs.gitlab.com/ee/api/epics.html) \(only available for GitLab Ultimate and GitLab.com Gold accounts. Stream Epics uses iid field as primary key for more convenient search and matching with UI. Iid is the internal ID of the epic, number of Epic on UI.\)
- [Epic Issues](https://docs.gitlab.com/ee/api/epic_issues.html) \(only available for GitLab Ultimate and GitLab.com Gold accounts\)

## Additional information
Expand All @@ -112,7 +109,8 @@ Gitlab has the [rate limits](https://docs.gitlab.com/ee/user/gitlab_com/index.ht

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:-------------------------------------------------------------------------------------------|
| 1.8.4 | 2023-10-19 | [31599](https://github.com/airbytehq/airbyte/pull/31599) | Base image migration: remove Dockerfile and use the python-connector-base image |
| 2.0.0 | 2023-10-23 | [31700](https://github.com/airbytehq/airbyte/pull/31700) | Add correct date-time format for Deployments, Projects and Groups Members streams |
| 1.8.4 | 2023-10-19 | [31599](https://github.com/airbytehq/airbyte/pull/31599) | Base image migration: remove Dockerfile and use the python-connector-base image |
| 1.8.3 | 2023-10-18 | [31547](https://github.com/airbytehq/airbyte/pull/31547) | Add validation for invalid `groups_list` and/or `projects_list` |
| 1.8.2 | 2023-10-17 | [31492](https://github.com/airbytehq/airbyte/pull/31492) | Expand list of possible error status codes when handling expired `access_token` |
| 1.8.1 | 2023-10-12 | [31375](https://github.com/airbytehq/airbyte/pull/31375) | Mark `start_date` as optional, migrate `groups` and `projects` to array |
Expand Down
Loading