🚨 Source Gitlab: certification improvements: epics pk, date-time form…

…at, docs, allowed hosts (#31700) Co-authored-by: darynaishchenko <[email protected]>
airbytehq · Oct 26, 2023 · d7ba34d · d7ba34d
1 parent bee749b
commit d7ba34d
Show file tree

Hide file tree

Showing 7 changed files with 29 additions and 13 deletions.
diff --git a/airbyte-integrations/connectors/source-gitlab/acceptance-test-config.yml b/airbyte-integrations/connectors/source-gitlab/acceptance-test-config.yml
@@ -18,7 +18,7 @@ acceptance_tests:
     tests:
       - config_path: "secrets/config.json"
         backward_compatibility_tests_config:
-          disable_for_version: 1.7.1
+          disable_for_version: 1.8.4
   basic_read:
     tests:
       - config_path: "secrets/config.json"

diff --git a/airbyte-integrations/connectors/source-gitlab/metadata.yaml b/airbyte-integrations/connectors/source-gitlab/metadata.yaml
@@ -4,13 +4,13 @@ data:
     sl: 200
   allowedHosts:
     hosts:
-      - "*"
+      - ${api_url}
   connectorBuildOptions:
     baseImage: docker.io/airbyte/python-connector-base:1.1.0@sha256:bd98f6505c6764b1b5f99d3aedc23dfc9e9af631a62533f60eb32b1d3dbab20c
   connectorSubtype: api
   connectorType: source
   definitionId: 5e6175e5-68e1-4c17-bff9-56103bbb0d80
-  dockerImageTag: 1.8.4
+  dockerImageTag: 2.0.0
   dockerRepository: airbyte/source-gitlab
   documentationUrl: https://docs.airbyte.com/integrations/sources/gitlab
   githubIssueLabel: source-gitlab
@@ -23,6 +23,14 @@ data:
     oss:
       enabled: true
   releaseStage: generally_available
+  releases:
+    breakingChanges:
+      2.0.0:
+        message:
+          In this release, several streams were updated to date-time field format, as declared in the Gitlab API.
+          These changes impact pipeline.created_at and pipeline.updated_at fields for stream Deployments and expires_at field for stream Group Members and stream Project Members.
+          Users will need to refresh the source schema and reset affected streams after upgrading.
+        upgradeDeadline: "2023-11-09"
   suggestedStreams:
     streams:
       - merge_requests

diff --git a/airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/deployments.json b/airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/deployments.json
@@ -180,7 +180,8 @@
           "type": ["null", "object"],
           "properties": {
             "created_at": {
-              "type": ["null", "string"]
+              "type": ["null", "string"],
+              "format": "date-time"
             },
             "id": {
               "type": ["null", "integer"]
@@ -195,7 +196,8 @@
               "type": ["null", "string"]
             },
             "updated_at": {
-              "type": ["null", "string"]
+              "type": ["null", "string"],
+              "format": "date-time"
             },
             "web_url": {
               "type": ["null", "string"]

diff --git a/airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/group_members.json b/airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/group_members.json
@@ -35,7 +35,7 @@
     },
     "expires_at": {
       "type": ["null", "string"],
-      "format": "date"
+      "format": "date-time"
     },
     "created_by": {
       "avatar_url": {

diff --git a/airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/project_members.json b/airbyte-integrations/connectors/source-gitlab/source_gitlab/schemas/project_members.json
@@ -35,7 +35,7 @@
     },
     "expires_at": {
       "type": ["null", "string"],
-      "format": "date"
+      "format": "date-time"
     },
     "created_by": {
       "avatar_url": {

diff --git a/docs/integrations/sources/gitlab-migrations.md b/docs/integrations/sources/gitlab-migrations.md
@@ -0,0 +1,8 @@
+# Gitlab Migration Guide
+
+## Upgrading to 2.0.0
+
+
+In the 2.0.0 config change, several streams were updated to date-time field format, as declared in the Gitlab API.
+These changes impact `pipeline.created_at` and` pipeline.updated_at` fields for stream Deployments and `expires_at` field for stream Group Members and stream Project Members.
+You will need to refresh the source schema and reset affected streams after upgrading.
diff --git a/docs/integrations/sources/gitlab.md b/docs/integrations/sources/gitlab.md
@@ -5,9 +5,6 @@ This page contains the setup guide and reference information for the Gitlab Sour
 ## Prerequisites
 
 - Gitlab instance or an account at [Gitlab](https://gitlab.com)
-- Start date (Optional)
-- GitLab Groups (Optional)
-- GitLab Projects (Optional)
 
 <!-- env:cloud -->
 
@@ -49,7 +46,7 @@ Log into [GitLab](https://gitlab.com) and then generate a [personal access token
 3. On the source setup page, select **GitLab** from the Source type dropdown and enter a name for this connector.
 4. Click `Authenticate your GitLab account` by selecting Oauth or Personal Access Token for Authentication.
 5. Log in and Authorize to the GitLab account.
-6. **API URL** - The URL to access you self-hosted GitLab instance or `gitlab.com` (default).
+6. **API URL (Optional)** - The URL to access your self-hosted GitLab instance or `gitlab.com` (default).
 7. **Start date (Optional)** - The date from which you'd like to replicate data for streams.
 8. **Groups (Optional)** - List of GitLab group IDs, e.g. `airbytehq` for single group, `airbytehq another-repo` for multiple groups.
 9. **Projects (Optional)** - List of GitLab projects to pull data for, e.g. `airbytehq/airbyte`.
@@ -97,7 +94,7 @@ This connector outputs the following streams:
 - [Deployments](https://docs.gitlab.com/ee/api/deployments/index.html)
 - [Group Labels](https://docs.gitlab.com/ee/api/group_labels.html)
 - [Project Labels](https://docs.gitlab.com/ee/api/labels.html)
-- [Epics](https://docs.gitlab.com/ee/api/epics.html) \(only available for GitLab Ultimate and GitLab.com Gold accounts\)
+- [Epics](https://docs.gitlab.com/ee/api/epics.html) \(only available for GitLab Ultimate and GitLab.com Gold accounts. Stream Epics uses iid field as primary key for more convenient search and matching with UI. Iid is the internal ID of the epic, number of Epic on UI.\)
 - [Epic Issues](https://docs.gitlab.com/ee/api/epic_issues.html) \(only available for GitLab Ultimate and GitLab.com Gold accounts\)
 
 ## Additional information
@@ -112,7 +109,8 @@ Gitlab has the [rate limits](https://docs.gitlab.com/ee/user/gitlab_com/index.ht
 
 | Version | Date       | Pull Request                                             | Subject                                                                                    |
 |:--------|:-----------|:---------------------------------------------------------|:-------------------------------------------------------------------------------------------|
-| 1.8.4 | 2023-10-19 | [31599](https://github.com/airbytehq/airbyte/pull/31599) | Base image migration: remove Dockerfile and use the python-connector-base image |
+| 2.0.0   | 2023-10-23 | [31700](https://github.com/airbytehq/airbyte/pull/31700) | Add correct date-time format for Deployments, Projects and Groups Members streams          |
+| 1.8.4   | 2023-10-19 | [31599](https://github.com/airbytehq/airbyte/pull/31599) | Base image migration: remove Dockerfile and use the python-connector-base image            |
 | 1.8.3   | 2023-10-18 | [31547](https://github.com/airbytehq/airbyte/pull/31547) | Add validation for invalid `groups_list` and/or `projects_list`                            |
 | 1.8.2   | 2023-10-17 | [31492](https://github.com/airbytehq/airbyte/pull/31492) | Expand list of possible error status codes when handling expired `access_token`            |
 | 1.8.1   | 2023-10-12 | [31375](https://github.com/airbytehq/airbyte/pull/31375) | Mark `start_date` as optional, migrate `groups` and `projects` to array                    |