✨Source Looker - Migrate Python CDK to Low-code CDK

[topefolorunso]

What

Migrate Looker Python CDK to Low-code CDK fixes https://github.com/airbytehq/airbyte-internal-issues/issues/7033

How

Developed using UI Builder and the Low-code CDK

Recommended reading order

1. spec.yaml
2. manifest.yaml

🚨 User Impact 🚨

Are there any breaking changes? What is the end result perceived by the user?

For connector PRs, use this section to explain which type of semantic versioning bump occurs as a result of the changes. Refer to our Semantic Versioning for Connectors guidelines for more information. Breaking changes to connectors must be documented by an Airbyte engineer (PR author, or reviewer for community PRs) by using the Breaking Change Release Playbook.

If there are breaking changes, please merge this PR with the 🚨🚨 emoji so changelog authors can further highlight this if needed.

Pre-merge Actions

Expand the relevant checklist and delete the others.

Updating a connector

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Unit & integration tests added

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• If new credentials are required for use in CI, add them to GSM. Instructions.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 31, 2024 10:50pm

[topefolorunso]

@natikgadzhi see error encountered

jsonschema.exceptions.ValidationError: The manifest version 3.8.2 is greater than the airbyte-cdk package version (0.90.0). Your manifest may contain features that are not in the current CDK version.

[natikgadzhi]

@topefolorunso right now, spec command fails on a built connector (i.e. if you try and airbyte-ci connectors --name source-looker build and then docker run airbyte/source-looker spec — that crashes with an error that the manifest version is above the CDK version.

This means that version value in manifest.yaml is too high compared to the CDK version that is defined in pyproject.toml and poetry.lock (0.90 right now).

To fix this, we could either pin the version in manifest to 0.90.0, or upgrade the CDK. I would recommend upgrading the CDK and other dev dependencies:

1. Edit pyproject.toml and make sure that the CDK is pinned to ^3.8.2 to satisfy the manifest version.
2. Make sure that pytest and other dev dependencies in pyproject.toml are also set to "*" so they can upgrade to pytest 8.
3. Run poetry lock in the directory of the connector.
4. Commit resulting changes to pyproject.toml and poetry.lock.

Ping me for review again once that's done! Same principle applies for other connectors you'll work on.

I'll try to make sure that you have access to CAT reports in the coming week.

[natikgadzhi]

Thank you for this! The code itself looks reasonable, left some comments. Let me kick of regression tests on this.

[natikgadzhi]

@artem1205 does the CDK provide error classes that are intended to use in low-code connectors? I.e. instead of LookerException that is custom, we could have something like AuthenticationError that would be a config error, consistent with what error handlers raise on HTTP401?

/cc @pnilan maybe you know?

[topefolorunso]

@natikgadzhi see error after bumping to CDK 4.0.1 and python = "^3.10,<3.12". could it be the connector base image?

[natikgadzhi]

EZ, set Python in pyproject.toml to ^3.10

[topefolorunso]

EZ, set Python in pyproject.toml to ^3.10

@natikgadzhi same error

[topefolorunso]

All good here @natikgadzhi. Please review again

[topefolorunso]

@natikgadzhi Regression test passed here

[natikgadzhi]

Unfortunately, just the run that finished does not mean that the actual test is passed (it's deeper in the reports that I hope I can make public later on).

I'm slow this week, I might be able to merge this, but I'll ask you to be on the lookup for bugs and regressions.

[topefolorunso]

be on the lookup for bugs and regressions

Got it, I'll be on standby