🚨🚨🐛 Source Notion: JSON Schema Updates #30587

ChristoGrab · 2023-09-19T17:24:07Z

What

Updates to JSON schemas for all existing streams

How

Added unexpected columns for all streams
Added date-time format to last_edited_time and created_time fields for all streams
BREAKING CHANGE: Renamed text property to rich_text in objects that contain rich text. Notion changelog for this update.
BREAKING CHANGE: Updated type of rich_text property in Pages stream from object to array of rich_text objects, which should also resolve this PR.
BREAKING CHANGE: Updated schema of Rich Text object to reflect API object. Notion docs
Updated schema for callout, code and to-do objects in Blocks stream
BREAKING CHANGE: Changed type of phone_number property in Database stream from string to object

User Impact

Many of the changes are simply additions of missing fields, but every stream other than Users is affected in some way by removal/renaming of properties. Pages properties rich_text and phone_number also have a type change.

Customers will have to refresh the schemas and reset data.

github-actions · 2023-09-19T17:24:26Z

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

PR name follows PR naming conventions
Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan.
Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
You've updated the connector's metadata.yaml file any other relevant changes, including a breakingChanges entry for major version bumps. See metadata.yaml docs
Secrets in the connector's spec are annotated with airbyte_secret
All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
Migration guide updated in docs/integrations/<source or destination>/<name>-migrations.md with an entry for the new version, if the version is a breaking change. See migration guide example
If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

Check for hidden checklists in your PR description
Toggle the github label checklist-action-run on/off to re-run the checklist CI.

vercel · 2023-09-20T05:18:56Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 9, 2023 5:13pm

airbyte-oss-build-runner · 2023-09-25T17:20:21Z

source-notion test report (commit `6b2b5815e5`) - ❌

⏲️ Total pipeline duration: 03mn25s

Step	Result
Connector package install	✅
Build source-notion docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-notion/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	❌

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

maxi297

Just one small question but given this is address, we're good to go!

maxi297 · 2023-09-25T20:33:30Z

docs/integrations/sources/notion.md

@@ -107,7 +107,8 @@ The connector is restricted by Notion [request limits](https://developers.notion

 | Version | Date       | Pull Request                                             | Subject                                                                      |
 | :------ | :--------- | :------------------------------------------------------- | :--------------------------------------------------------------------------- |
-| 1.1.2   | 2023-08-30 | [29999](https://github.com/airbytehq/airbyte/pull/29999) | Update error handling during connection check
+| 1.2.0   | 2023-09-19 | [30587](https://github.com/airbytehq/airbyte/pull/30587) | Source-wide schema update                                                    |


Should this be 2.0.0?

Oh whoops, thanks for catching that! Fixed 👍

airbyte-oss-build-runner · 2023-09-25T21:33:13Z

source-notion test report (commit `974c2f2604`) - ✅

⏲️ Total pipeline duration: 04mn15s

Step	Result
Connector package install	✅
Build source-notion docker image for platform linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate airbyte-integrations/connectors/source-notion/metadata.yaml	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

airbyte-oss-build-runner · 2023-10-04T16:31:27Z

source-notion test report (commit `dae994798e`) - ✅

⏲️ Total pipeline duration: 03mn34s

Step	Result
Connector package install	✅
Build source-notion docker image for platform(s) linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate metadata for source-notion	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

airbyte-oss-build-runner · 2023-10-04T16:41:20Z

source-notion test report (commit `db33a1ac47`) - ✅

⏲️ Total pipeline duration: 01mn20s

Step	Result
Connector package install	✅
Build source-notion docker image for platform(s) linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate metadata for source-notion	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

…tehq/airbyte into christo/fix-notion-schemas

airbyte-oss-build-runner · 2023-10-09T16:10:55Z

source-notion test report (commit `4c71f01e1b`) - ❌

⏲️ Total pipeline duration: 04mn30s

Step	Result
Connector package install	✅
Build source-notion docker image for platform(s) linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌
Code format checks	✅
Validate metadata for source-notion	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

…tehq/airbyte into christo/fix-notion-schemas

airbyte-oss-build-runner · 2023-10-09T16:26:57Z

source-notion test report (commit `54d566e864`) - ❌

⏲️ Total pipeline duration: 03mn45s

Step	Result
Connector package install	✅
Build source-notion docker image for platform(s) linux/x86_64	✅
Unit tests	✅
Acceptance tests	❌
Code format checks	✅
Validate metadata for source-notion	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

airbyte-oss-build-runner · 2023-10-09T17:16:37Z

source-notion test report (commit `d41e6423e2`) - ✅

⏲️ Total pipeline duration: 03mn40s

Step	Result
Connector package install	✅
Build source-notion docker image for platform(s) linux/x86_64	✅
Unit tests	✅
Acceptance tests	✅
Code format checks	✅
Validate metadata for source-notion	✅
Connector version semver check	✅
Connector version increment check	✅
QA checks	✅

🔗 View the logs here

☁️ View runs for commit in Dagger Cloud

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=source-notion test

Co-authored-by: ChristoGrab <[email protected]>

ChristoGrab added 8 commits September 15, 2023 12:07

update schemas to pass acceptance tests

2321458

update user schema and version bump

c2bf0e6

fix heading schema

6ca28a9

update database and page schema

ce03fb6

update pages

ab790c4

blocks and databases

67d793c

alphabetize blocks properties for ease of reading

c27db10

add missing properties

8741ebf

octavia-squidington-iii added the area/connectors Connector related issues label Sep 19, 2023

octavia-squidington-iii added the connectors/source/notion label Sep 19, 2023

ChristoGrab changed the title ~~Source Notion: JSON Schema Updates~~ 🚨🚨🐛 Source Notion: JSON Schema Updates Sep 19, 2023

docs version bump

00ae5c7

octavia-squidington-iii added the area/documentation Improvements or additions to documentation label Sep 19, 2023

ChristoGrab requested review from sherifnada and lazebnyi September 19, 2023 18:06

ChristoGrab marked this pull request as ready for review September 19, 2023 18:29

add migration guide

86c0b99

airbytehq deleted a comment from github-actions bot Sep 19, 2023

ChristoGrab added checklist-action-run and removed checklist-action-run labels Sep 19, 2023

add format to date fields

13caece

ChristoGrab mentioned this pull request Sep 20, 2023

Source Notion: Avoid KeyError when no next_cursor exists #30641

Closed

ChristoGrab linked an issue Sep 21, 2023 that may be closed by this pull request

Source Notion: Add undeclared columns to spec #24560

Closed

ChristoGrab mentioned this pull request Sep 21, 2023

Source Notion: Add undeclared columns to spec #24560

Closed

ChristoGrab and others added 2 commits September 22, 2023 14:40

update breaking change message

281e117

Automated Commit - Formatting Changes

9c0e53b

octavia-approvington requested a review from a team as a code owner September 22, 2023 20:45

ChristoGrab added the breaking-change Don't merge me unless you are ready. label Sep 25, 2023

maxi297 approved these changes Sep 25, 2023

View reviewed changes

ChristoGrab and others added 2 commits September 25, 2023 15:25

fix changelog version

9e679f6

Fix changelog version

974c2f2

vercel bot deployed to Preview September 25, 2023 21:29 View deployment

ChristoGrab mentioned this pull request Sep 26, 2023

Source Notion: Add Availability Strategy #30750

Merged

Merge branch 'master' into christo/fix-notion-schemas

dae9947

vercel bot deployed to Preview October 4, 2023 16:27 View deployment

Automated Commit - Formatting Changes

db33a1a

ChristoGrab and others added 4 commits October 9, 2023 11:59

Merge branch 'christo/fix-notion-schemas' of https://github.com/airby…

38f4f5d

…tehq/airbyte into christo/fix-notion-schemas

Merge branch 'master' into christo/fix-notion-schemas

f90f492

Update acceptance-test-config.yml

4c71f01

Merge branch 'christo/fix-notion-schemas' of https://github.com/airby…

b67b086

…tehq/airbyte into christo/fix-notion-schemas

ChristoGrab and others added 2 commits October 9, 2023 12:21

Merge branch 'master' into christo/fix-notion-schemas

54d566e

Merge branch 'christo/fix-notion-schemas' of https://github.com/airby…

c407109

…tehq/airbyte into christo/fix-notion-schemas

fix duplicate rich_text property and update migration guide

d41e642

vercel bot deployed to Preview October 9, 2023 17:13 View deployment

ChristoGrab merged commit b5c3149 into master Oct 9, 2023
16 checks passed

ChristoGrab deleted the christo/fix-notion-schemas branch October 9, 2023 18:55

ariesgun pushed a commit to ariesgun/airbyte that referenced this pull request Oct 23, 2023

🚨🚨🐛 Source Notion: JSON Schema Updates (airbytehq#30587)

1c48fc4

Co-authored-by: ChristoGrab <[email protected]>

ChristoGrab mentioned this pull request Oct 27, 2023

🐛 Source Notion: fix table_row.cells property in Blocks stream #31899

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🚨🚨🐛 Source Notion: JSON Schema Updates #30587

🚨🚨🐛 Source Notion: JSON Schema Updates #30587

ChristoGrab commented Sep 19, 2023 •

edited

Loading

github-actions bot commented Sep 19, 2023 •

edited by ChristoGrab

Loading

vercel bot commented Sep 20, 2023 •

edited

Loading

airbyte-oss-build-runner commented Sep 25, 2023

maxi297 left a comment

maxi297 Sep 25, 2023

ChristoGrab Sep 25, 2023

airbyte-oss-build-runner commented Sep 25, 2023

airbyte-oss-build-runner commented Oct 4, 2023

airbyte-oss-build-runner commented Oct 4, 2023

airbyte-oss-build-runner commented Oct 9, 2023

airbyte-oss-build-runner commented Oct 9, 2023

airbyte-oss-build-runner commented Oct 9, 2023

🚨🚨🐛 Source Notion: JSON Schema Updates #30587

🚨🚨🐛 Source Notion: JSON Schema Updates #30587

Conversation

ChristoGrab commented Sep 19, 2023 • edited Loading

What

How

User Impact

github-actions bot commented Sep 19, 2023 • edited by ChristoGrab Loading

Before Merging a Connector Pull Request

vercel bot commented Sep 20, 2023 • edited Loading

airbyte-oss-build-runner commented Sep 25, 2023

source-notion test report (commit 6b2b5815e5) - ❌

maxi297 left a comment

Choose a reason for hiding this comment

maxi297 Sep 25, 2023

Choose a reason for hiding this comment

ChristoGrab Sep 25, 2023

Choose a reason for hiding this comment

airbyte-oss-build-runner commented Sep 25, 2023

source-notion test report (commit 974c2f2604) - ✅

airbyte-oss-build-runner commented Oct 4, 2023

source-notion test report (commit dae994798e) - ✅

airbyte-oss-build-runner commented Oct 4, 2023

source-notion test report (commit db33a1ac47) - ✅

airbyte-oss-build-runner commented Oct 9, 2023

source-notion test report (commit 4c71f01e1b) - ❌

airbyte-oss-build-runner commented Oct 9, 2023

source-notion test report (commit 54d566e864) - ❌

airbyte-oss-build-runner commented Oct 9, 2023

source-notion test report (commit d41e6423e2) - ✅

ChristoGrab commented Sep 19, 2023 •

edited

Loading

github-actions bot commented Sep 19, 2023 •

edited by ChristoGrab

Loading

vercel bot commented Sep 20, 2023 •

edited

Loading

source-notion test report (commit `6b2b5815e5`) - ❌

source-notion test report (commit `974c2f2604`) - ✅

source-notion test report (commit `dae994798e`) - ✅

source-notion test report (commit `db33a1ac47`) - ✅

source-notion test report (commit `4c71f01e1b`) - ❌

source-notion test report (commit `54d566e864`) - ❌

source-notion test report (commit `d41e6423e2`) - ✅