[Non-Forked] Source Pipedrive : Migrate Python CDK to Low-code CDK #30553

archangelic · 2023-09-18T20:38:46Z

What

Non-forked branch for PR #29293

How

Describe the solution

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user?

For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.

If there are breaking changes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Actions

Expand the relevant checklist and delete the others.

New Connector

Community member or Airbyter

Community member? Grant edit access to maintainers (instructions)
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Connector version is set to 0.0.1
- Dockerfile has version 0.0.1
Documentation updated
- Connector's README.md
- Connector's bootstrap.md. See description and examples
- docs/integrations/<source or destination>/<name>.md including changelog with an entry for the initial version. See changelog example
- docs/integrations/README.md

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

Create a non-forked branch based on this PR and test the below items on it
Build is successful
If new credentials are required for use in CI, add them to GSM. Instructions.

Connector Generator

Issue acceptance criteria met
PR name follows PR naming conventions
If adding a new generator, add it to the list of scaffold modules being tested
The generator test modules (all connectors with -scaffold in their name) have been updated with the latest scaffold by running ./gradlew :airbyte-integrations:connector-templates:generator:generateScaffolds then checking in your changes
Documentation which references the generator is updated as needed

github-actions · 2023-09-18T20:39:09Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

github-actions · 2023-09-18T20:51:02Z

source-pipedrive test report (commit `a033fc4080`) - ❌

⏲️ Total pipeline duration: 04mn11s

Step	Result
Connector package install	✅
Build source-pipedrive docker image for platform linux/x86_64	✅
Acceptance tests	❌
Code format checks	✅
Validate airbyte-integrations/connectors/source-pipedrive/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-pipedrive test

github-actions · 2023-09-18T21:07:35Z

source-pipedrive test report (commit `a033fc4080`) - ❌

⏲️ Total pipeline duration: 04mn21s

Step	Result
Connector package install	✅
Build source-pipedrive docker image for platform linux/x86_64	✅
Acceptance tests	❌
Code format checks	✅
Validate airbyte-integrations/connectors/source-pipedrive/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-pipedrive test

vercel · 2023-09-20T17:51:07Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Sep 21, 2023 3:34pm

github-actions · 2023-09-20T20:39:51Z

source-pipedrive test report (commit `ec08dab5fc`) - ✅

⏲️ Total pipeline duration: 03mn14s

Step	Result
Connector package install	✅
Build source-pipedrive docker image for platform linux/x86_64	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-pipedrive/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-pipedrive test

marcosmarxm

Some comments.

docs/integrations/sources/pipedrive-migrations.md

airbyte-integrations/connectors/source-pipedrive/source_pipedrive/manifest.yaml

airbyte-oss-build-runner · 2023-09-20T22:06:14Z

source-pipedrive test report (commit `d97b030851`) - ✅

⏲️ Total pipeline duration: 02mn03s

Step	Result
Connector package install	✅
Build source-pipedrive docker image for platform linux/x86_64	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-pipedrive/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-pipedrive test

marcosmarxm · 2023-09-20T22:10:40Z

airbyte-integrations/connectors/source-pipedrive/source_pipedrive/manifest.yaml

@@ -242,7 +242,7 @@ definitions:
      path_extractor: "product"
      path: "/recents"

-  products_fields_stream:
+  product_fields_stream:
    $ref: "#/definitions/base_stream"
    name: "products_fields"


You need to change here too.

airbyte-oss-build-runner · 2023-09-20T22:23:16Z

source-pipedrive test report (commit `e405312fe7`) - ✅

⏲️ Total pipeline duration: 02mn45s

Step	Result
Connector package install	✅
Build source-pipedrive docker image for platform linux/x86_64	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-pipedrive/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-pipedrive test

marcosmarxm

Mal only need to check this thread about the field id inside the product_fields schema. After solved that LGTM everything is workign great :)

marcosmarxm

LGTM

airbyte-oss-build-runner · 2023-09-21T15:39:06Z

source-pipedrive test report (commit `0858c9a7d5`) - ✅

⏲️ Total pipeline duration: 02mn51s

Step	Result
Connector package install	✅
Build source-pipedrive docker image for platform linux/x86_64	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-pipedrive/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-pipedrive test

…30553) Co-authored-by: Aviraj Gour <[email protected]> Co-authored-by: Aviraj Gour <[email protected]> Co-authored-by: marcosmarxm <[email protected]>

avirajsingh7 and others added 5 commits August 9, 2023 22:48

migrated to low code

43715e1

metadata and docs modified

5cf197f

suggested changed

9be5217

Merge branch 'master' into Hackathon/pipedrive

15e6dcd

Merge branch 'master' into Hackathon/pipedrive

880f28d

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/pipedrive labels Sep 18, 2023

fixes for pagination in streams

a033fc4

archangelic mentioned this pull request Sep 18, 2023

✨ Source Pipedrive : Migrate Python CDK to Low-code CDK #29293

Closed

11 tasks

fix stream states and parameters for filter stream

3f29136

archangelic added 2 commits September 20, 2023 10:57

Merge remote-tracking branch 'origin' into mal/pipedrive-pr-29293

14c42a6

add migrations

265b3a5

vercel bot deployed to Preview September 20, 2023 17:59 View deployment

make changes to auth, change breaking changes message

81234ff

archangelic added the checklist-action-run label Sep 20, 2023

vercel bot deployed to Preview September 20, 2023 18:49 View deployment

formatting fix

ec08dab

marcosmarxm requested changes Sep 20, 2023

View reviewed changes

changes from review

d97b030

archangelic requested a review from marcosmarxm September 20, 2023 22:02

vercel bot deployed to Preview September 20, 2023 22:09 View deployment

marcosmarxm requested changes Sep 20, 2023

View reviewed changes

change products_fields to product_fields

e405312

archangelic requested a review from marcosmarxm September 20, 2023 22:18

marcosmarxm reviewed Sep 21, 2023

View reviewed changes

make product_field schema compatible with normalization

0858c9a

marcosmarxm approved these changes Sep 21, 2023

View reviewed changes

archangelic merged commit fe09992 into master Sep 21, 2023
18 checks passed

archangelic deleted the mal/pipedrive-pr-29293 branch September 21, 2023 17:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Non-Forked] Source Pipedrive : Migrate Python CDK to Low-code CDK #30553

[Non-Forked] Source Pipedrive : Migrate Python CDK to Low-code CDK #30553

archangelic commented Sep 18, 2023

github-actions bot commented Sep 18, 2023 •

edited by archangelic

Loading

github-actions bot commented Sep 18, 2023

github-actions bot commented Sep 18, 2023

vercel bot commented Sep 20, 2023 •

edited

Loading

github-actions bot commented Sep 20, 2023

marcosmarxm left a comment

airbyte-oss-build-runner commented Sep 20, 2023

marcosmarxm Sep 20, 2023

airbyte-oss-build-runner commented Sep 20, 2023

marcosmarxm left a comment

marcosmarxm left a comment

airbyte-oss-build-runner commented Sep 21, 2023

[Non-Forked] Source Pipedrive : Migrate Python CDK to Low-code CDK #30553

[Non-Forked] Source Pipedrive : Migrate Python CDK to Low-code CDK #30553

Conversation

archangelic commented Sep 18, 2023

What

How

Recommended reading order

🚨 User Impact 🚨

Pre-merge Actions

Community member or Airbyter

Airbyter

Community member or Airbyter

Airbyter

github-actions bot commented Sep 18, 2023 • edited by archangelic Loading

Before Merging a Connector Pull Request

github-actions bot commented Sep 18, 2023

source-pipedrive test report (commit a033fc4080) - ❌

github-actions bot commented Sep 18, 2023

source-pipedrive test report (commit a033fc4080) - ❌

vercel bot commented Sep 20, 2023 • edited Loading

github-actions bot commented Sep 20, 2023

source-pipedrive test report (commit ec08dab5fc) - ✅

marcosmarxm left a comment

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 20, 2023

source-pipedrive test report (commit d97b030851) - ✅

marcosmarxm Sep 20, 2023

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 20, 2023

source-pipedrive test report (commit e405312fe7) - ✅

marcosmarxm left a comment

Choose a reason for hiding this comment

marcosmarxm left a comment

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 21, 2023

source-pipedrive test report (commit 0858c9a7d5) - ✅

github-actions bot commented Sep 18, 2023 •

edited by archangelic

Loading

source-pipedrive test report (commit `a033fc4080`) - ❌

source-pipedrive test report (commit `a033fc4080`) - ❌

vercel bot commented Sep 20, 2023 •

edited

Loading

source-pipedrive test report (commit `ec08dab5fc`) - ✅

source-pipedrive test report (commit `d97b030851`) - ✅

source-pipedrive test report (commit `e405312fe7`) - ✅

source-pipedrive test report (commit `0858c9a7d5`) - ✅