[Feature Request]: Add support for pyarrow >= 19.0.0

[clementmangin]

What would you like to happen?

apache-beam python SDK is presently incompatible with langchain-google-vertexai>=2.0.21 (see langchain-ai/langchain-google#912) because the latter depends on pyarrow>=19.0.1,<20.0.0 whereas apache-beam depends on pyarrow>=3.0.0,<19.0.0.

There is a fix incoming for langchain-google-vertexai to add support for pyarrow==19.0.0, but apache-beam may benefit from supporting newer versions of pyarrow nonetheless.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[clementmangin]

Seems like it will be addressed in #36845