[clickhouse] Add `trace_id_timestamps` table with materialized view #7723

mahadzaryab1 · 2025-12-10T05:11:20Z

Which problem is this PR solving?

Towards [clickhouse] Implement Trace Reader For ClickHouse Storage #7134

Description of the changes

Adds the trace_id_timestamps table with a materialized view to the spans table. The table contains the starting and end timestamp for each trace_id. By maintaining precomputed time bounds for every trace, the query engine can infer an appropriate time window and prune irrelevant partitions, significantly reducing the amount of data scanned and improving overall query performance.

How was this change tested?

CI

Checklist

I have read https://github.com/jaegertracing/jaeger/blob/master/CONTRIBUTING_GUIDELINES.md
I have signed all commits
I have added unit tests for the new functionality
I have run lint and test steps successfully
- for jaeger: make lint test
- for jaeger-ui: npm run lint and npm run test

Signed-off-by: Mahad Zaryab <[email protected]>

codecov · 2025-12-10T05:19:47Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.62%. Comparing base (d843097) to head (5dd0055).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #7723      +/-   ##
==========================================
+ Coverage   95.61%   95.62%   +0.01%     
==========================================
  Files         311      311              
  Lines       15511    15513       +2     
==========================================
+ Hits        14831    14835       +4     
+ Misses        532      531       -1     
+ Partials      148      147       -1

Flag	Coverage Δ
badger_v1	`9.90% <0.00%> (-0.01%)`	⬇️
badger_v2	`2.07% <0.00%> (-0.01%)`	⬇️
cassandra-4.x-v1-manual	`14.05% <0.00%> (-0.01%)`	⬇️
cassandra-4.x-v2-auto	`2.06% <0.00%> (-0.01%)`	⬇️
cassandra-4.x-v2-manual	`2.06% <0.00%> (-0.01%)`	⬇️
cassandra-5.x-v1-manual	`14.05% <0.00%> (-0.01%)`	⬇️
cassandra-5.x-v2-auto	`2.06% <0.00%> (-0.01%)`	⬇️
cassandra-5.x-v2-manual	`2.06% <0.00%> (-0.01%)`	⬇️
clickhouse	`1.98% <0.00%> (-0.01%)`	⬇️
elasticsearch-6.x-v1	`18.79% <0.00%> (-0.01%)`	⬇️
elasticsearch-7.x-v1	`18.83% <0.00%> (-0.01%)`	⬇️
elasticsearch-8.x-v1	`18.99% <0.00%> (-0.01%)`	⬇️
elasticsearch-8.x-v2	`2.07% <0.00%> (-0.01%)`	⬇️
elasticsearch-9.x-v2	`2.07% <0.00%> (-0.01%)`	⬇️
grpc_v1	`9.69% <0.00%> (-0.01%)`	⬇️
grpc_v2	`2.07% <0.00%> (-0.01%)`	⬇️
kafka-3.x-v2	`2.07% <0.00%> (-0.01%)`	⬇️
memory_v2	`2.07% <0.00%> (-0.01%)`	⬇️
opensearch-1.x-v1	`18.87% <0.00%> (-0.01%)`	⬇️
opensearch-2.x-v1	`18.87% <0.00%> (-0.01%)`	⬇️
opensearch-2.x-v2	`2.07% <0.00%> (-0.01%)`	⬇️
opensearch-3.x-v2	`2.07% <0.00%> (-0.01%)`	⬇️
query	`2.07% <0.00%> (-0.01%)`	⬇️
tailsampling-processor	`0.59% <0.00%> (-0.01%)`	⬇️
unittests	`94.19% <100.00%> (+0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

github-actions · 2025-12-10T05:21:13Z

Metrics Comparison Summary

Total changes across all snapshots: 53

Detailed changes per snapshot

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

Total Changes: 53

🆕 Added: 0 metrics
❌ Removed: 53 metrics
🔄 Modified: 0 metrics

❌ Removed Metrics

http_server_request_body_size_bytes (18 variants)

View diff sample

-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

- `http_server_request_duration_seconds` (17 variants)

View diff sample

-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.005",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.01",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.025",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.05",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.075",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_request_duration_seconds{http_request_method="GET",http_response_status_code="503",le="0.1",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

- `http_server_response_body_size_bytes` (18 variants)

View diff sample

-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="+Inf",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="0",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="100",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="1000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="10000",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
-http_server_response_body_size_bytes{http_request_method="GET",http_response_status_code="503",le="25",network_protocol_name="http",network_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_schema_url="",otel_scope_version="0.63.0",server_address="localhost",server_port="13133",url_scheme="http"}
...

➡️ View full metrics file

mahadzaryab1 added 6 commits December 9, 2025 21:06

Add Script For Creating Trace ID Timestamps Table

a68eb24

Signed-off-by: Mahad Zaryab <[email protected]>

Add Script For Creating Trace ID Timestamps Materialized View

6fb51cf

Signed-off-by: Mahad Zaryab <[email protected]>

Embed Queries

02455b3

Signed-off-by: Mahad Zaryab <[email protected]>

Execute Scripts

cceab86

Signed-off-by: Mahad Zaryab <[email protected]>

Update Tests

5ca67f6

Signed-off-by: Mahad Zaryab <[email protected]>

Drive By Change: Semicolon

5dd0055

Signed-off-by: Mahad Zaryab <[email protected]>

mahadzaryab1 requested a review from a team as a code owner December 10, 2025 05:11

mahadzaryab1 requested a review from albertteoh December 10, 2025 05:11

mahadzaryab1 changed the title ~~[clickhouse] Add trace_id_timestamps table with materialized view~~ [clickhouse] Add trace_id_timestamps table with materialized view Dec 10, 2025

dosubot bot added area/storage enhancement labels Dec 10, 2025

mahadzaryab1 added the changelog:exprimental Change to an experimental part of the code label Dec 10, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[clickhouse] Add `trace_id_timestamps` table with materialized view #7723

[clickhouse] Add `trace_id_timestamps` table with materialized view #7723

Uh oh!

mahadzaryab1 commented Dec 10, 2025

Uh oh!

codecov bot commented Dec 10, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Dec 10, 2025

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

❌ Removed Metrics

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

[clickhouse] Add trace_id_timestamps table with materialized view #7723

Are you sure you want to change the base?

[clickhouse] Add trace_id_timestamps table with materialized view #7723

Uh oh!

Conversation

mahadzaryab1 commented Dec 10, 2025

Which problem is this PR solving?

Description of the changes

How was this change tested?

Checklist

Uh oh!

codecov bot commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Dec 10, 2025

Metrics Comparison Summary

summary_metrics_snapshot_cassandra

📊 Metrics Diff Summary

❌ Removed Metrics

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

[clickhouse] Add `trace_id_timestamps` table with materialized view #7723

[clickhouse] Add `trace_id_timestamps` table with materialized view #7723

codecov bot commented Dec 10, 2025 •

edited

Loading