Skip to content

Allow creation for JSON type #44

@SPTKL

Description

@SPTKL

Description

BigQuery natively supports JSON data using the JSON data type.

By using the JSON data type, you can ingest semi-structured JSON into BigQuery without providing a schema for the JSON data upfront. This lets you store and query data that doesn’t always adhere to fixed schemas and data types. By ingesting JSON data as a JSON data type, BigQuery can encode and process each JSON field individually. You can then query the values of fields and array elements within the JSON data by using the field access operator, which makes JSON queries easy to use and cost efficient.

Proposal

To work within the current framework of JSON schema, I'm thinking that if we have a object type field without properties specified, then we treat it as a JSON type field e.g.

from singer_sdk import typing as th

schema = th.PropertiesList(
    th.Property(
        "jsonTypeFieldName",
        th.ObjectType,
    ),
    th.Property(
        "recordTypeFieldName",
        th.ObjectType(
            th.Property(
                ...
            ),
            ...
        )
    ),
    ...
)

Whereas if a object type field does have properties, then it's a RECORD type field

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions