[DCP Ingestion - Stale Reads] Enable Stale Reads in CDC Web server which runs mixer#31
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a hardcoded USE_STALE_READS environment variable to the shared configuration for Cloud Run services. Feedback indicates that applying this globally poses a data consistency risk for ingestion jobs and limits maintainability; it is recommended to move this setting to the web service configuration and make it configurable via a variable.
| { | ||
| name = "USE_STALE_READS" | ||
| value = var.use_spanner ? "true" : "false" | ||
| } |
There was a problem hiding this comment.
Enabling USE_STALE_READS in the shared environment variables list (cloud_run_shared_env_variables) is problematic for two reasons:
- Data Consistency Risk: This setting is applied to both the web service and the
dc_data_job(data management/ingestion). Stale reads can lead to data corruption or inconsistent states if the ingestion job performs read-modify-write operations or relies on reading the most recent updates. Data management tasks typically require strong consistency. - Maintainability: Hardcoding the value to
"true"prevents users from disabling this feature (e.g., for debugging or specific consistency requirements) without modifying the module code.
It is recommended to move this environment variable to the web service configuration only (in main.tf) and make it configurable via a variable in variables.tf.
|
/gemini review |
There was a problem hiding this comment.
Code Review
This pull request introduces the USE_STALE_READS environment variable to the Cloud Run service, with its value determined by the Spanner usage flag. Feedback suggests centralizing this configuration in locals.tf to maintain consistency with the module's existing pattern for environment variables.
2 participants
This is a very small change to the Terraform script to add the USE_STALE_READS environment variable to the Cloud Run Service running the CDC image. This server startups mixer, and will pass in the right feature flags based on the USE_STALE_READS env variable.
See the complementary change in datacommonsorg/website#6215
Currently deployed in https://pantheon.corp.google.com/run/detail/us-central1/gabe-test-datacommons-web-service/observability/logs?e=13803378&inv=1&invt=Ab1zPA&mods=-monitoring_api_staging&project=datcom-website-dev