Commit f3eed3d
committed
[Flink Runner] Add translators for PrimitiveUnboundedRead and PrimitiveBoundedRead
This commit adds explicit translators for SplittableParDo.PrimitiveUnboundedRead
and SplittableParDo.PrimitiveBoundedRead to the Flink streaming transform
translators.
The Flink classic runner calls convertReadBasedSplittableDoFnsToPrimitiveReadsIfNecessary()
when NOT using beam_fn_api experiment, which converts SDF-wrapped reads to
PrimitiveUnboundedRead and PrimitiveBoundedRead. Without translators for these
transforms, pipelines using unbounded sources like KinesisIO.read() fail with:
'No translator known for PrimitiveUnboundedRead'
Changes:
- Add PrimitiveUnboundedReadTranslator class
- Add PrimitiveBoundedReadTranslator class
- Modify getTranslator() to handle these transforms before URN lookup
- Add unit tests for the new translators
- Update CHANGES.md with bugfix entry
Related to #205301 parent 586b7c1 commit f3eed3d
File tree
2 files changed
+6
-4
lines changed- runners/flink/src/test/java/org/apache/beam/runners/flink
2 files changed
+6
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| 87 | + | |
87 | 88 | | |
88 | 89 | | |
89 | 90 | | |
| |||
Lines changed: 5 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
196 | 196 | | |
197 | 197 | | |
198 | 198 | | |
199 | | - | |
| 199 | + | |
| 200 | + | |
200 | 201 | | |
201 | 202 | | |
202 | 203 | | |
| |||
208 | 209 | | |
209 | 210 | | |
210 | 211 | | |
211 | | - | |
| 212 | + | |
| 213 | + | |
212 | 214 | | |
213 | 215 | | |
214 | 216 | | |
| |||
288 | 290 | | |
289 | 291 | | |
290 | 292 | | |
291 | | - | |
292 | | - | |
| 293 | + | |
293 | 294 | | |
294 | 295 | | |
295 | 296 | | |
| |||
0 commit comments