Skip to content

[Bug]: Add SchemaFieldNumber annotations to Iceberg to prevent potential update compatibility issues #36778

@chamikaramj

Description

@chamikaramj

What happened?

Iceberg uses SchemaCoder in several locations.

coder = registry.getSchemaCoder(SnapshotInfo.class);

coder = SchemaRegistry.createDefault().getSchemaCoder(ReadTask.class);

coder = SchemaRegistry.createDefault().getSchemaCoder(ReadTaskDescriptor.class);

Using SchemaCoder (as opposed to RowCoder, for example) can result in update compatibility issues for Dataflow Runner v2. These can be addressed by adding SchemaFieldNumber annotations similar to #36295.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions