[Bug]: Cross-language pipeline options are not picked up in Java DoFns

[ahmedabu98]

What happened?

I noticed this when testing with Managed BigQueryIO in Python SDK:

options = PipelineOptions([
  "--project=google.com:clouddfe",
  "--region=us-central1",
  "--temp_location=gs://ahmedabualsaud-wordcount/tmp",
])


with beam.Pipeline(options=options) as p:
  (p
   | beam.Create([beam.Row(int=999)])
   | beam.managed.Write("bigquery",
                        config={"table": "google.com:clouddfe.ahmedabualsaud.test_table"},
                        expansion_service="localhost:12345"
                        ))

I would expect these pipeline options to be picked up on Java's side. With some logging, I see the Java SDK recognizing the options at expand time in PTransforms (i.e. expand()), but not at runtime in DoFns (i.e. @ProcessElement).

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[Suvrat1629]

@ahmedabu98 I am trying to understand how this would work. So do i tweak the expand() method and the "@ProcessElement" to pass it to the DoFns or is there some other way to do this?

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.

[github-actions]

This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.

[tarun-google]

.take-issue