Data: Moving GenericFileWriterFactory to the new FormatModel API#15334
Data: Moving GenericFileWriterFactory to the new FormatModel API#15334pvary merged 5 commits intoapache:mainfrom
Conversation
|
|
||
| /** | ||
| * @deprecated This constructor is deprecated as of version 1.11.0 and will be removed in 1.12.0. | ||
| * Position deletes that include row data are no longer supported. Use {@link |
There was a problem hiding this comment.
Position deletes that include row data are no longer supported
is this still true? i noticed its missing in the new deprecation message
There was a problem hiding this comment.
I removed the deprecation comment here because this method was introduced in the current release cycle, before we decided to deprecate the entire class. Now that the class itself is deprecated, and the replacement class doesn’t provide a way to set the position‑delete row‑data schema, the constuctor is effectively deprecated as well. However, since there’s no new equivalent method, we don’t really have a suitable place to add a dedicated deprecation message.
data/src/main/java/org/apache/iceberg/data/GenericFileWriterFactory.java
Outdated
Show resolved
Hide resolved
data/src/main/java/org/apache/iceberg/data/BaseFileWriterFactory.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
should those methods be no-ops instead? Otherwise users will hit this exception when calling those methods. Typically when we deprecate something we keep/maintain the functionality without failing immediately
There was a problem hiding this comment.
These methods were originally defined in the abstract BaseFileWriterFactory so that subclasses, such as GenericFileWriterFactory could implement them. BaseFileWriterFactory also relied on them to configure writers. The new RegistryBasedFileWriterFactory neither needs nor uses these methods, so any external calls to them would now represent a potential source of errors.
I considered three options:
- Remove the methods – This causes compile‑time failures when subclasses use
@Override. - Keep the methods but throw an exception – This fails fast but breaks callers that still rely on the old configuration path.
- Keep the methods as no‑ops – This avoids failures but silently ignores calls, which can mask bugs.
Both options (2) and (3) behave unpredictably if someone overrides newDataWriter, newEqualityDeleteWriter, or newPositionDeleteWriter and continues to rely on the old configure... methods: option (2) throws, option (3) silently discards the call.
After thinking through the trade‑offs, the best approach seems to be keeping the methods as-is and marking them as deprecated, leaving their behavior unchanged. This provides a clear migration signal without introducing new failure modes.
data/src/main/java/org/apache/iceberg/data/GenericFileWriterFactory.java
Outdated
Show resolved
Hide resolved
data/src/main/java/org/apache/iceberg/data/GenericFileWriterFactory.java
Outdated
Show resolved
Hide resolved
data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java
Outdated
Show resolved
Hide resolved
data/src/main/java/org/apache/iceberg/data/RegistryBasedFileWriterFactory.java
Outdated
Show resolved
Hide resolved
| } else { | ||
| LOG.warn( | ||
| "Deprecated feature used. Position delete row schema is used to create the position delete writer."); | ||
| MetricsConfig metricsConfig = |
There was a problem hiding this comment.
should this part of the code actually mimic
to maintain existing behavior? Because right now I think it's a copy ofThere was a problem hiding this comment.
I originally misunderstood this comment.
Fixed now. Please check
data/src/main/java/org/apache/iceberg/data/GenericFileWriterFactory.java
Outdated
Show resolved
Hide resolved
|
Merged to main. |
Part of: #12298
Implementation of the new API: #12774
Migrating of the GenericFileWriterFactory