Add benchmark (CEL vs FIELD vs Direct) filter_state access by PetrMc · Pull Request #6797 · istio/proxy

PetrMc · 2026-01-27T21:32:51Z

What this PR does / why we need it:

we need clear understanding of the resources used by different methods.

Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes istio/istio#58868

Special notes for your reviewer:
on my local machine is acting not consistantly but i get similar memory use but FIELD is outperforming CEL by approximately 34x (run 1) and 49x (run 2).

attempt 1:

ubuntu@devbox:~/github/proxy$ bazel run //source/extensions/filters/http/peer_metadata:filter_state_benchmark
INFO: Analyzed target //source/extensions/filters/http/peer_metadata:filter_state_benchmark (414 packages loaded, 29083 targets configured).
INFO: Found 1 target...
Target //source/extensions/filters/http/peer_metadata:filter_state_benchmark up-to-date:
  bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark
INFO: Elapsed time: 8883.570s, Critical Path: 885.74s
INFO: 2657 processes: 92 internal, 2565 processwrapper-sandbox.
INFO: Build completed successfully, 2657 total actions
INFO: Running command line: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark
2026-01-27T12:55:29-08:00
Running /home/ubuntu/.cache/bazel/_bazel_ubuntu/7f61383ba1c4bb725390614e1c03ab26/execroot/io_istio_proxy/bazel-out/k8-fastbuild/bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark
Run on (8 X 2089.58 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 24576 KiB (x1)
Load Average: 4.15, 8.50, 10.60
***WARNING*** ASLR is enabled, the results may have unreproducible noise in them.
***WARNING*** Library was built as DEBUG. Timings may be affected.
--------------------------------------------------------------------------------
Benchmark                      Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------
BM_FilterState_CEL         69565 ns        69559 ns         9864 bytes_per_second=3.60581Mi/s alloc_per_iter=263B
BM_FilterState_FIELD        2037 ns         2037 ns       332308 bytes_per_second=123.156Mi/s alloc_per_iter=263B
BM_FilterState_Direct        453 ns          453 ns      1577568 bytes_per_second=16.8409Mi/s alloc_per_iter=0B (no allocation, direct access)
[external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_reloadable_features_runtime_initialized to: false
[external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_quic_always_support_server_preferred_address to: true

attempt 2:

$ bazel run //source/extensions/filters/http/peer_metadata:filter_state_benchmark
INFO: Analyzed target //source/extensions/filters/http/peer_metadata:filter_state_benchmark (252 packages loaded, 5662 targets configured).
INFO: Found 1 target...
Target //source/extensions/filters/http/peer_metadata:filter_state_benchmark up-to-date:
  bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark
INFO: Elapsed time: 59.562s, Critical Path: 2.34s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark
2026-01-27T13:01:30-08:00
Running /home/ubuntu/.cache/bazel/_bazel_ubuntu/7f61383ba1c4bb725390614e1c03ab26/execroot/io_istio_proxy/bazel-out/k8-fastbuild/bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark
Run on (8 X 1716.38 MHz CPU s)
CPU Caches:
  L1 Data 48 KiB (x4)
  L1 Instruction 32 KiB (x4)
  L2 Unified 1280 KiB (x4)
  L3 Unified 24576 KiB (x1)
Load Average: 10.59, 5.96, 8.42
***WARNING*** ASLR is enabled, the results may have unreproducible noise in them.
***WARNING*** Library was built as DEBUG. Timings may be affected.
--------------------------------------------------------------------------------
Benchmark                      Time             CPU   Iterations UserCounters...
--------------------------------------------------------------------------------
BM_FilterState_CEL        365568 ns       356229 ns         3404 bytes_per_second=720.985Ki/s alloc_per_iter=263B
BM_FilterState_FIELD        7435 ns         7286 ns       162165 bytes_per_second=34.4231Mi/s alloc_per_iter=263B
BM_FilterState_Direct       1588 ns         1586 ns       640617 bytes_per_second=4.81016Mi/s alloc_per_iter=0B (no allocation, direct access)
[external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_reloadable_features_runtime_initialized to: false
[external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_quic_always_support_server_preferred_address to: true

kyessenov · 2026-01-27T22:53:12Z

source/extensions/filters/http/peer_metadata/filter_state_benchmark.cc

+  setupCelFilterState(stream_info);
+
+  // CEL format: %CEL(filter_state.downstream_peer.workload)%
+  const std::string format = "%CEL(filter_state.downstream_peer.workload)%";


Thanks for testing it. I suspect there's some hidden allocation here that's causing most of the hit. I'd expect it to be up to 3x slower but not 30x.

thanks @kyessenov i run it on debug build on Linux VM - so the results certainly not precise. Any suggestions on how to get more accurate measurements?

You should use release config to benchmark. If you use tcmalloc, there's an alloc profiler that you can utilize to find it. Are you trying to optimize CEL? FIELD will always be faster.

thanks i'll do that. for the context we implemented FIELD Accessor here istio/istio#58348 as it seem to be the right architecture decision and now working on confirming that the change is justified in terms of performance. We had a side discussion with @zirain on that topic #6765 (comment) and need to make sure all questions are addressed.

yeah, if it's 30x faster, FIELD is a reasonable new feature.
If it's just 3x in ns level, which make no sense to store another object in memory.

done optimized build and 3 runs below all show consistent over 30x difference.

run 1

INFO: Found 1 target... Target //source/extensions/filters/http/peer_metadata:filter_state_benchmark up-to-date: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark INFO: Elapsed time: 8622.036s, Critical Path: 534.99s INFO: 2440 processes: 143 internal, 2297 processwrapper-sandbox. INFO: Build completed successfully, 2440 total actions INFO: Running command line: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark 2026-01-28T10:38:29-08:00 Running /home/ubuntu/.cache/bazel/_bazel_ubuntu/7f61383ba1c4bb725390614e1c03ab26/execroot/io_istio_proxy/bazel-out/k8-opt/bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark Run on (8 X 1567.25 MHz CPU s) CPU Caches: L1 Data 48 KiB (x4) L1 Instruction 32 KiB (x4) L2 Unified 1280 KiB (x4) L3 Unified 24576 KiB (x1) Load Average: 4.35, 7.64, 9.20 ***WARNING*** ASLR is enabled, the results may have unreproducible noise in them. -------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------- BM_FilterState_CEL 5302 ns 5277 ns 129492 bytes_per_second=47.5332Mi/s alloc_per_iter=263B BM_FilterState_FIELD 128 ns 128 ns 6817691 bytes_per_second=1.91971Gi/s alloc_per_iter=263B BM_FilterState_Direct 21.9 ns 21.9 ns 31837857 bytes_per_second=348.119Mi/s alloc_per_iter=0B (no allocation, direct access) [external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_quic_always_support_server_preferred_address to: true [external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_reloadable_features_runtime_initialized to: false

run 2

$ bazel run -c opt //source/extensions/filters/http/peer_metadata:filter_state_benchmark INFO: Analyzed target //source/extensions/filters/http/peer_metadata:filter_state_benchmark (1 packages loaded, 6 targets configured). INFO: Found 1 target... Target //source/extensions/filters/http/peer_metadata:filter_state_benchmark up-to-date: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark INFO: Elapsed time: 3.604s, Critical Path: 0.14s INFO: 1 process: 1 internal. INFO: Build completed successfully, 1 total action INFO: Running command line: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark 2026-01-28T10:40:35-08:00 Running /home/ubuntu/.cache/bazel/_bazel_ubuntu/7f61383ba1c4bb725390614e1c03ab26/execroot/io_istio_proxy/bazel-out/k8-opt/bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark Run on (8 X 1791.25 MHz CPU s) CPU Caches: L1 Data 48 KiB (x4) L1 Instruction 32 KiB (x4) L2 Unified 1280 KiB (x4) L3 Unified 24576 KiB (x1) Load Average: 1.83, 5.38, 8.15 ***WARNING*** ASLR is enabled, the results may have unreproducible noise in them. -------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------- BM_FilterState_CEL 3985 ns 3980 ns 133361 bytes_per_second=63.0244Mi/s alloc_per_iter=263B BM_FilterState_FIELD 118 ns 118 ns 7033442 bytes_per_second=2.08059Gi/s alloc_per_iter=263B BM_FilterState_Direct 16.9 ns 16.9 ns 31438278 bytes_per_second=450.314Mi/s alloc_per_iter=0B (no allocation, direct access) [external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_quic_always_support_server_preferred_address to: true [external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_reloadable_features_runtime_initialized to: false

run 3

INFO: Analyzed target //source/extensions/filters/http/peer_metadata:filter_state_benchmark (0 packages loaded, 0 targets configured). INFO: Found 1 target... Target //source/extensions/filters/http/peer_metadata:filter_state_benchmark up-to-date: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark INFO: Elapsed time: 1.721s, Critical Path: 0.45s INFO: 1 process: 1 internal. INFO: Build completed successfully, 1 total action INFO: Running command line: bazel-bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark 2026-01-28T10:40:44-08:00 Running /home/ubuntu/.cache/bazel/_bazel_ubuntu/7f61383ba1c4bb725390614e1c03ab26/execroot/io_istio_proxy/bazel-out/k8-opt/bin/source/extensions/filters/http/peer_metadata/filter_state_benchmark Run on (8 X 1400.36 MHz CPU s) CPU Caches: L1 Data 48 KiB (x4) L1 Instruction 32 KiB (x4) L2 Unified 1280 KiB (x4) L3 Unified 24576 KiB (x1) Load Average: 1.69, 5.29, 8.11 ***WARNING*** ASLR is enabled, the results may have unreproducible noise in them. -------------------------------------------------------------------------------- Benchmark Time CPU Iterations UserCounters... -------------------------------------------------------------------------------- BM_FilterState_CEL 2764 ns 2763 ns 199363 bytes_per_second=90.7722Mi/s alloc_per_iter=263B BM_FilterState_FIELD 70.3 ns 70.3 ns 9706858 bytes_per_second=3.48362Gi/s alloc_per_iter=263B BM_FilterState_Direct 21.4 ns 21.4 ns 46790167 bytes_per_second=355.875Mi/s alloc_per_iter=0B (no allocation, direct access) [external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_reloadable_features_runtime_initialized to: false [external/com_google_absl/absl/flags/internal/flag.cc : 147] RAW: Restore saved value of envoy_quic_always_support_server_preferred_address to: true

zirain

LGTM, defer to @kyessenov

Add benchmark comparing CEL vs FIELD vs Direct filter_state access

33a197c

PetrMc requested a review from a team as a code owner January 27, 2026 21:32

istio-testing added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jan 27, 2026

kyessenov reviewed Jan 27, 2026

View reviewed changes

zirain reviewed Jan 28, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add benchmark (CEL vs FIELD vs Direct) filter_state access#6797

Add benchmark (CEL vs FIELD vs Direct) filter_state access#6797
PetrMc wants to merge 1 commit intoistio:masterfrom
PetrMc:petrmc/58868-benchmark-clean

PetrMc commented Jan 27, 2026

Uh oh!

kyessenov Jan 27, 2026

Uh oh!

PetrMc Jan 27, 2026

Uh oh!

kyessenov Jan 28, 2026

Uh oh!

PetrMc Jan 28, 2026

Uh oh!

zirain Jan 28, 2026

Uh oh!

PetrMc Jan 28, 2026

Uh oh!

zirain left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

PetrMc commented Jan 27, 2026

Uh oh!

kyessenov Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

PetrMc Jan 27, 2026

Choose a reason for hiding this comment

Uh oh!

kyessenov Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

PetrMc Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

zirain Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

PetrMc Jan 28, 2026

Choose a reason for hiding this comment

Uh oh!

zirain left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants