-
Notifications
You must be signed in to change notification settings - Fork 53
Description
Description
BigQuery natively supports JSON data using the JSON data type.
By using the JSON data type, you can ingest semi-structured JSON into BigQuery without providing a schema for the JSON data upfront. This lets you store and query data that doesn’t always adhere to fixed schemas and data types. By ingesting JSON data as a JSON data type, BigQuery can encode and process each JSON field individually. You can then query the values of fields and array elements within the JSON data by using the field access operator, which makes JSON queries easy to use and cost efficient.
Proposal
To work within the current framework of JSON schema, I'm thinking that if we have a object type field without properties specified, then we treat it as a JSON type field e.g.
from singer_sdk import typing as th
schema = th.PropertiesList(
th.Property(
"jsonTypeFieldName",
th.ObjectType,
),
th.Property(
"recordTypeFieldName",
th.ObjectType(
th.Property(
...
),
...
)
),
...
)Whereas if a object type field does have properties, then it's a RECORD type field