Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow creation for JSON type #44

Open
SPTKL opened this issue Nov 21, 2022 · 0 comments
Open

Allow creation for JSON type #44

SPTKL opened this issue Nov 21, 2022 · 0 comments
Assignees

Comments

@SPTKL
Copy link

SPTKL commented Nov 21, 2022

Description

BigQuery natively supports JSON data using the JSON data type.

By using the JSON data type, you can ingest semi-structured JSON into BigQuery without providing a schema for the JSON data upfront. This lets you store and query data that doesn’t always adhere to fixed schemas and data types. By ingesting JSON data as a JSON data type, BigQuery can encode and process each JSON field individually. You can then query the values of fields and array elements within the JSON data by using the field access operator, which makes JSON queries easy to use and cost efficient.

Proposal

To work within the current framework of JSON schema, I'm thinking that if we have a object type field without properties specified, then we treat it as a JSON type field e.g.

from singer_sdk import typing as th

schema = th.PropertiesList(
    th.Property(
        "jsonTypeFieldName",
        th.ObjectType,
    ),
    th.Property(
        "recordTypeFieldName",
        th.ObjectType(
            th.Property(
                ...
            ),
            ...
        )
    ),
    ...
)

Whereas if a object type field does have properties, then it's a RECORD type field

@RuslanBergenov RuslanBergenov self-assigned this Feb 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants