typev: add baseline, json and limited-conn tests by praisethemoon · Pull Request #714 · MDA2AV/HttpArena

praisethemoon · 2026-05-17T11:21:46Z

Description

Adds new tests for typev: bsaeline, json and limited-conn.

Local tests successfully pass.

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command	Description
`/benchmark -f <framework>`	Run all benchmark tests
`/benchmark -f <framework> -t <test>`	Run a specific test
`/benchmark -f <framework> --save`	Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.

Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

praisethemoon · 2026-05-17T11:25:20Z

Actually I forgot benchmark-site.sh, let me run that first, will update with results.

praisethemoon · 2026-05-17T12:38:08Z

Good to go 🤞

MDA2AV · 2026-05-17T12:57:37Z

/benchmark -f typev --save

github-actions · 2026-05-17T12:57:45Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T13:01:51Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,719,130	5526.1%	2.5GiB	NEW	NEW
baseline	4096	1,704,967	5643.1%	2.5GiB	NEW	NEW
pipelined	512	4,592,102	5099.2%	3.4GiB	-90.1%	+1005.3%
pipelined	4096	2,324,368	2588.8%	1.8GiB	-94.9%	+265.0%
limited-conn	512	1,073,551	5870.3%	220MiB	NEW	NEW
limited-conn	4096	746,836	3836.1%	367MiB	NEW	NEW
json	4096	46,995	885.0%	311MiB	NEW	NEW

Full log

  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    425us     86us    654us   2.22ms   39.80ms

  2275091 requests in 5.00s, 2097712 responses
  Throughput: 419.39K req/s
  Bandwidth:  36.00MB/s
  Status codes: 2xx=2097712, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2097712 / 2097712 responses (100.0%)
  Reconnects: 385569
  Per-template: 698498,698978,700236
  Per-template-ok: 698498,698978,700236
[info] CPU 2694.3% | Mem 236MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    372us     79us   1.06ms   2.45ms   26.50ms

  3543325 requests in 5.00s, 3447400 responses
  Throughput: 689.25K req/s
  Bandwidth:  59.16MB/s
  Status codes: 2xx=3447400, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3447385 / 3447400 responses (100.0%)
  Reconnects: 440543
  Per-template: 1147821,1149509,1150055
  Per-template-ok: 1147821,1149509,1150055
[info] CPU 4129.6% | Mem 327MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    297us     80us    846us   2.36ms   17.40ms

  3819680 requests in 5.00s, 3734181 responses
  Throughput: 746.56K req/s
  Bandwidth:  64.08MB/s
  Status codes: 2xx=3734181, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3734168 / 3734181 responses (100.0%)
  Reconnects: 456812
  Per-template: 1243313,1245227,1245628
  Per-template-ok: 1243313,1245227,1245628
[info] CPU 3836.1% | Mem 367MiB

=== Best: 746836 req/s (CPU: 3836.1%, Mem: 367MiB) ===
[info] input BW: 57.69MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   192.07ms   92.90ms   95.30ms    3.91s    5.00s

  298841 requests in 5.00s, 704 responses
  Throughput: 140 req/s
  Bandwidth:  689.13KB/s
  Status codes: 2xx=704, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 704 / 704 responses (100.0%)
  Latency overflow (>5s): 1
  Reconnects: 298170
  Per-template: 59,88,162,128,154,35,78
  Per-template-ok: 59,88,162,128,154,35,78
[info] CPU 125.8% | Mem 188MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   21.34ms    169us    345us   206.60ms    3.21s

  484479 requests in 5.00s, 201269 responses
  Throughput: 40.24K req/s
  Bandwidth:  213.56MB/s
  Status codes: 2xx=201269, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 201260 / 201269 responses (100.0%)
  Reconnects: 291167
  Per-template: 28603,29258,28978,28268,28513,28600,29040
  Per-template-ok: 28603,29258,28978,28268,28513,28600,29040
[info] CPU 295.1% | Mem 291MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   23.89ms    170us    329us   199.30ms    3.51s

  515838 requests in 5.00s, 234978 responses
  Throughput: 46.98K req/s
  Bandwidth:  248.84MB/s
  Status codes: 2xx=234978, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 234970 / 234978 responses (100.0%)
  Reconnects: 289176
  Per-template: 32777,34446,34476,33117,33151,33458,33545
  Per-template-ok: 32777,34446,34476,33117,33151,33458,33545
[info] CPU 885.0% | Mem 311MiB

=== Best: 46995 req/s (CPU: 885.0%, Mem: 311MiB) ===
[info] input BW: 2.24MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

praisethemoon · 2026-05-17T13:08:41Z

That is a lot of things gone wrong. Let me investigate.

praisethemoon · 2026-05-17T14:46:37Z

Fixed truncation bug for json benchmark & pipeline's fast path.

praisethemoon · 2026-05-17T14:50:06Z

/benchmark -f typev

github-actions · 2026-05-17T14:50:14Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T15:16:00Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,902,170	6317.8%	172MiB	NEW	NEW
baseline	4096	1,823,637	6813.6%	376MiB	NEW	NEW
pipelined	512	44,248,252	6463.0%	266MiB	-4.3%	-15.6%
pipelined	4096	44,736,355	5932.2%	878MiB	-2.8%	+73.9%
limited-conn	512	1,068,116	5954.4%	215MiB	NEW	NEW
limited-conn	4096	753,528	4034.1%	349MiB	NEW	NEW
json	4096	61,012	1059.3%	303MiB	NEW	NEW

Full log

  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    427us     81us    976us   2.54ms   6.30ms

  3211253 requests in 5.00s, 3099170 responses
  Throughput: 619.61K req/s
  Bandwidth:  53.18MB/s
  Status codes: 2xx=3099170, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3099170 / 3099170 responses (100.0%)
  Reconnects: 422040
  Per-template: 1031954,1034435,1032781
  Per-template-ok: 1031954,1034435,1032781
[info] CPU 3579.8% | Mem 242MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    329us     78us    778us   2.44ms   27.70ms

  3537333 requests in 5.00s, 3444451 responses
  Throughput: 688.67K req/s
  Bandwidth:  59.11MB/s
  Status codes: 2xx=3444451, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3444656 / 3444451 responses (100.0%)
  Reconnects: 437336
  Per-template: 1147258,1149175,1148003
  Per-template-ok: 1147258,1149175,1148003
[info] CPU 4200.4% | Mem 328MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    302us     79us    808us   2.35ms   3.74ms

  3843506 requests in 5.00s, 3767643 responses
  Throughput: 753.24K req/s
  Bandwidth:  64.65MB/s
  Status codes: 2xx=3767643, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3767624 / 3767643 responses (100.0%)
  Reconnects: 452670
  Per-template: 1255202,1257461,1254961
  Per-template-ok: 1255202,1257461,1254961
[info] CPU 4034.1% | Mem 349MiB

=== Best: 753528 req/s (CPU: 4034.1%, Mem: 349MiB) ===
[info] input BW: 58.21MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   33.71ms    167us    331us    1.40s    1.43s

  337713 requests in 5.00s, 45468 responses
  Throughput: 9.09K req/s
  Bandwidth:  48.52MB/s
  Status codes: 2xx=45468, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 45468 / 45468 responses (100.0%)
  Reconnects: 292418
  Per-template: 6134,6133,6835,6855,6577,6521,6413
  Per-template-ok: 6134,6133,6835,6855,6577,6521,6413
[info] CPU 131.0% | Mem 216MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   25.12ms    170us    335us   188.00ms    2.83s

  500676 requests in 5.00s, 218028 responses
  Throughput: 43.59K req/s
  Bandwidth:  231.44MB/s
  Status codes: 2xx=218028, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 218023 / 218028 responses (100.0%)
  Reconnects: 290390
  Per-template: 29889,30838,32070,31693,31645,31640,30248
  Per-template-ok: 29889,30838,32070,31693,31645,31640,30248
[info] CPU 295.6% | Mem 287MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   19.29ms    168us    338us   452.50ms    2.90s

  575594 requests in 5.00s, 305061 responses
  Throughput: 60.99K req/s
  Bandwidth:  322.68MB/s
  Status codes: 2xx=305061, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 305054 / 305061 responses (100.0%)
  Reconnects: 282590
  Per-template: 41769,44527,44435,44310,43813,43739,42461
  Per-template-ok: 41769,44527,44435,44310,43813,43739,42461
[info] CPU 1059.3% | Mem 303MiB

=== Best: 61012 req/s (CPU: 1059.3%, Mem: 303MiB) ===
[info] input BW: 2.91MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

praisethemoon · 2026-05-17T15:50:26Z

I may have caught a VM bug (type-v) in the JSON benchmark. Investigting 👀

praisethemoon · 2026-05-17T17:09:57Z

/benchmark -f typev

github-actions · 2026-05-17T17:10:06Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T17:36:30Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,942,708	6597.2%	251MiB	NEW	NEW
baseline	4096	1,905,202	6399.9%	396MiB	NEW	NEW
pipelined	512	47,451,849	6497.7%	276MiB	+2.6%	-12.4%
pipelined	4096	47,228,358	6194.5%	356MiB	+2.6%	-29.5%
limited-conn	512	888,512	4662.8%	226MiB	NEW	NEW
limited-conn	4096	1,262,343	6600.7%	312MiB	NEW	NEW
json	4096	564,577	6385.3%	380MiB	NEW	NEW

Full log

  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.27ms     92us    218us   198.60ms   221.10ms

  6134764 requests in 5.00s, 6128471 responses
  Throughput: 1.23M req/s
  Bandwidth:  105.16MB/s
  Status codes: 2xx=6128471, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6128471 / 6128471 responses (100.0%)
  Reconnects: 618844
  Per-template: 2042945,2042739,2042787
  Per-template-ok: 2042945,2042739,2042787
[info] CPU 5986.4% | Mem 271MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.18ms     93us    217us   199.30ms   209.40ms

  6311712 requests in 5.00s, 6311717 responses
  Throughput: 1.26M req/s
  Bandwidth:  108.32MB/s
  Status codes: 2xx=6311717, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6311692 / 6311717 responses (100.0%)
  Reconnects: 631171
  Per-template: 2103899,2103843,2103950
  Per-template-ok: 2103899,2103843,2103950
[info] CPU 6600.7% | Mem 312MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.19ms     94us    220us   198.20ms   209.30ms

  6292211 requests in 5.00s, 6292216 responses
  Throughput: 1.26M req/s
  Bandwidth:  107.98MB/s
  Status codes: 2xx=6292216, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6292073 / 6292216 responses (100.0%)
  Reconnects: 629205
  Per-template: 2097532,2097318,2097223
  Per-template-ok: 2097532,2097318,2097223
[info] CPU 6369.7% | Mem 318MiB

=== Best: 1262343 req/s (CPU: 6600.7%, Mem: 312MiB) ===
[info] input BW: 97.51MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.28ms    173us    318us   33.60ms    1.18s

  2810794 requests in 5.00s, 2810796 responses
  Throughput: 561.97K req/s
  Bandwidth:  2.91GB/s
  Status codes: 2xx=2810796, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2810786 / 2810796 responses (100.0%)
  Reconnects: 112245
  Per-template: 398918,400797,401691,403342,404061,402484,399493
  Per-template-ok: 398918,400797,401691,403342,404061,402484,399493
[info] CPU 6354.4% | Mem 355MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.25ms    172us    317us   35.70ms    1.18s

  2806821 requests in 5.00s, 2806817 responses
  Throughput: 561.18K req/s
  Bandwidth:  2.91GB/s
  Status codes: 2xx=2806817, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2806807 / 2806817 responses (100.0%)
  Reconnects: 112129
  Per-template: 398609,400617,400984,402762,403188,400758,399889
  Per-template-ok: 398609,400617,400984,402762,403188,400758,399889
[info] CPU 6580.2% | Mem 341MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   6.26ms    173us    319us   37.30ms    1.22s

  2822956 requests in 5.00s, 2822889 responses
  Throughput: 564.41K req/s
  Bandwidth:  2.93GB/s
  Status codes: 2xx=2822889, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2822879 / 2822889 responses (100.0%)
  Reconnects: 112778
  Per-template: 400487,401670,403751,405644,405884,403625,401818
  Per-template-ok: 400487,401670,403751,405644,405884,403625,401818
[info] CPU 6385.3% | Mem 380MiB

=== Best: 564577 req/s (CPU: 6385.3%, Mem: 380MiB) ===
[info] input BW: 26.92MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

praisethemoon · 2026-05-17T17:42:13Z

🎊 Finally, seems to work as intended! Thanks for your patience :)

praisethemoon · 2026-05-17T17:55:49Z

I will try and do a final micro optimization for the json bench.

praisethemoon · 2026-05-17T18:17:00Z

/benchmark -f typev

github-actions · 2026-05-17T18:17:09Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T18:54:02Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,921,218	6568.2%	194MiB	NEW	NEW
baseline	4096	1,896,595	6494.5%	385MiB	NEW	NEW
pipelined	512	47,926,896	6404.5%	283MiB	+3.6%	-10.2%
pipelined	4096	48,788,460	6418.8%	453MiB	+6.0%	-10.3%
limited-conn	512	898,664	4564.1%	216MiB	NEW	NEW
limited-conn	4096	1,283,994	6399.3%	307MiB	NEW	NEW
json	4096	694,920	6405.0%	364MiB	NEW	NEW

Full log

  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.76ms     96us    247us   42.90ms   209.60ms

  6358110 requests in 5.00s, 6275708 responses
  Throughput: 1.25M req/s
  Bandwidth:  107.70MB/s
  Status codes: 2xx=6275708, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6275707 / 6275708 responses (100.0%)
  Reconnects: 709937
  Per-template: 2090455,2092194,2093058
  Per-template-ok: 2090455,2092194,2093058
[info] CPU 6181.2% | Mem 275MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.14ms     94us    217us   197.70ms   208.70ms

  6400948 requests in 5.00s, 6400954 responses
  Throughput: 1.28M req/s
  Bandwidth:  109.85MB/s
  Status codes: 2xx=6400954, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6400933 / 6400954 responses (100.0%)
  Reconnects: 640082
  Per-template: 2133560,2133848,2133525
  Per-template-ok: 2133560,2133848,2133525
[info] CPU 6628.2% | Mem 304MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.13ms     94us    215us   197.70ms   208.60ms

  6421695 requests in 5.00s, 6420235 responses
  Throughput: 1.28M req/s
  Bandwidth:  110.18MB/s
  Status codes: 2xx=6419972, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6419952 / 6420235 responses (100.0%)
  Reconnects: 643259
  Per-template: 2139955,2139722,2140275
  Per-template-ok: 2139955,2139722,2140275

  WARNING: 263/6420235 responses (0.0%) had unexpected status (expected 2xx)
[info] CPU 6399.3% | Mem 307MiB

=== Best: 1283994 req/s (CPU: 6399.3%, Mem: 307MiB) ===
[info] input BW: 99.19MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.20ms    156us    266us   10.80ms   905.80ms

  3465703 requests in 5.00s, 3465431 responses
  Throughput: 692.84K req/s
  Bandwidth:  2.33GB/s
  Status codes: 2xx=3465431, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3465419 / 3465431 responses (100.0%)
  Reconnects: 138391
  Per-template: 493178,494664,495731,496861,496314,495005,493665
  Per-template-ok: 493178,494664,495731,496861,496314,495005,493665
[info] CPU 6442.0% | Mem 361MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.26ms    155us    265us   12.20ms   904.00ms

  3438219 requests in 5.00s, 3438222 responses
  Throughput: 687.45K req/s
  Bandwidth:  2.32GB/s
  Status codes: 2xx=3438222, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3438208 / 3438222 responses (100.0%)
  Reconnects: 137339
  Per-template: 488372,489991,492762,493009,492737,491104,490233
  Per-template-ok: 488372,489991,492762,493009,492737,491104,490233
[info] CPU 6625.0% | Mem 342MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.19ms    156us    266us   11.90ms   905.90ms

  3475702 requests in 5.00s, 3474600 responses
  Throughput: 694.73K req/s
  Bandwidth:  2.34GB/s
  Status codes: 2xx=3474600, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3474591 / 3474600 responses (100.0%)
  Reconnects: 139114
  Per-template: 493695,495351,497116,498425,498688,496899,494417
  Per-template-ok: 493695,495351,497116,498425,498688,496899,494417
[info] CPU 6405.0% | Mem 364MiB

=== Best: 694920 req/s (CPU: 6405.0%, Mem: 364MiB) ===
[info] input BW: 33.14MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

MDA2AV · 2026-05-17T19:07:19Z

No problem, I just click to approve these notifications in my phone, you can submit as many you need

praisethemoon · 2026-05-17T20:19:04Z

/benchmark -f typev

github-actions · 2026-05-17T20:19:10Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T20:29:12Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,861,475	5831.6%	183MiB	NEW	NEW
baseline	4096	1,918,275	6565.5%	370MiB	NEW	NEW
pipelined	512	46,798,774	6397.9%	255MiB	+1.2%	-19.0%
pipelined	4096	47,906,275	6247.1%	362MiB	+4.1%	-28.3%
limited-conn	512	927,829	4755.1%	213MiB	NEW	NEW
limited-conn	4096	1,311,903	6431.9%	326MiB	NEW	NEW
json	4096	1,079,225	6563.0%	454MiB	NEW	NEW

Full log

  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.11ms     93us    214us   194.80ms   207.00ms

  6464293 requests in 5.00s, 6463909 responses
  Throughput: 1.29M req/s
  Bandwidth:  110.93MB/s
  Status codes: 2xx=6463909, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6463909 / 6463909 responses (100.0%)
  Reconnects: 646744
  Per-template: 2154619,2154626,2154664
  Per-template-ok: 2154619,2154626,2154664
[info] CPU 6284.6% | Mem 279MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.07ms     93us    214us   193.80ms   205.20ms

  6540139 requests in 5.00s, 6539951 responses
  Throughput: 1.31M req/s
  Bandwidth:  112.24MB/s
  Status codes: 2xx=6539951, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6539924 / 6539951 responses (100.0%)
  Reconnects: 654000
  Per-template: 2179813,2180248,2179863
  Per-template-ok: 2179813,2180248,2179863
[info] CPU 6631.4% | Mem 317MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.06ms     94us    214us   194.20ms   205.30ms

  6559903 requests in 5.00s, 6559516 responses
  Throughput: 1.31M req/s
  Bandwidth:  112.57MB/s
  Status codes: 2xx=6559516, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6560163 / 6559516 responses (100.0%)
  Reconnects: 655960
  Per-template: 2186625,2186659,2186209
  Per-template-ok: 2186625,2186659,2186209
[info] CPU 6431.9% | Mem 326MiB

=== Best: 1311903 req/s (CPU: 6431.9%, Mem: 326MiB) ===
[info] input BW: 101.34MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.36ms    106us    177us   3.05ms   560.70ms

  5332377 requests in 5.00s, 5332298 responses
  Throughput: 1.07M req/s
  Bandwidth:  3.59GB/s
  Status codes: 2xx=5332298, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 5332298 / 5332298 responses (100.0%)
  Reconnects: 212936
  Per-template: 757190,760750,762967,764584,765084,762924,758799
  Per-template-ok: 757190,760750,762967,764584,765084,762924,758799
[info] CPU 6322.6% | Mem 451MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.39ms    106us    177us   3.58ms   567.40ms

  5328833 requests in 5.00s, 5328152 responses
  Throughput: 1.07M req/s
  Bandwidth:  3.59GB/s
  Status codes: 2xx=5328152, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 5328403 / 5328152 responses (100.0%)
  Reconnects: 213445
  Per-template: 757763,758907,761361,764518,764894,762001,758691
  Per-template-ok: 757763,758907,761361,764518,764894,762001,758691
[info] CPU 6723.1% | Mem 391MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.35ms    106us    178us   2.77ms   560.10ms

  5396124 requests in 5.00s, 5396125 responses
  Throughput: 1.08M req/s
  Bandwidth:  3.63GB/s
  Status codes: 2xx=5396125, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 5396103 / 5396125 responses (100.0%)
  Reconnects: 215481
  Per-template: 767185,769434,771961,773960,774106,771719,767738
  Per-template-ok: 767185,769434,771961,773960,774106,771719,767738
[info] CPU 6563.0% | Mem 454MiB

=== Best: 1079225 req/s (CPU: 6563.0%, Mem: 454MiB) ===
[info] input BW: 51.46MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

praisethemoon · 2026-05-17T20:30:38Z

Nice numbers! Good to go from my end 🙌

praisethemoon · 2026-05-17T20:38:11Z

I will take a quick look at short lived bench, I think i want to play around with more shards 🤔

MDA2AV · 2026-05-17T20:47:20Z

/benchmark -f typev --save

github-actions · 2026-05-17T20:47:27Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T20:56:05Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,865,595	6472.4%	275MiB	NEW	NEW
baseline	4096	1,916,753	6439.9%	292MiB	NEW	NEW
pipelined	512	43,128,227	6555.6%	268MiB	-6.8%	-14.9%
pipelined	4096	47,909,920	6198.4%	555MiB	+4.1%	+9.9%
limited-conn	512	288,546	1386.9%	178MiB	NEW	NEW
limited-conn	4096	663,145	3258.4%	408MiB	NEW	NEW
json	4096	544,745	3102.4%	442MiB	NEW	NEW

Full log

  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.76ms     67us    159us   107.80ms    1.15s

  3277281 requests in 5.00s, 3277286 responses
  Throughput: 655.22K req/s
  Bandwidth:  56.24MB/s
  Status codes: 2xx=3277286, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3277276 / 3277286 responses (100.0%)
  Reconnects: 327765
  Per-template: 1092644,1092055,1092577
  Per-template-ok: 1092644,1092055,1092577
[info] CPU 3202.6% | Mem 257MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.73ms     66us    157us   114.10ms    1.13s

  3250436 requests in 5.00s, 3250248 responses
  Throughput: 649.87K req/s
  Bandwidth:  55.78MB/s
  Status codes: 2xx=3250248, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3250227 / 3250248 responses (100.0%)
  Reconnects: 325030
  Per-template: 1083057,1083421,1083749
  Per-template-ok: 1083057,1083421,1083749
[info] CPU 3075.3% | Mem 385MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.67ms     67us    160us   104.30ms    1.13s

  3316497 requests in 5.00s, 3315725 responses
  Throughput: 662.90K req/s
  Bandwidth:  56.90MB/s
  Status codes: 2xx=3315725, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 3315708 / 3315725 responses (100.0%)
  Reconnects: 331619
  Per-template: 1105007,1105279,1105422
  Per-template-ok: 1105007,1105279,1105422
[info] CPU 3258.4% | Mem 408MiB

=== Best: 663145 req/s (CPU: 3258.4%, Mem: 408MiB) ===
[info] input BW: 51.23MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.82ms     81us    150us   15.70ms    1.94s

  2700757 requests in 5.00s, 2700757 responses
  Throughput: 539.95K req/s
  Bandwidth:  1.82GB/s
  Status codes: 2xx=2700757, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2700757 / 2700757 responses (100.0%)
  Reconnects: 107903
  Per-template: 384397,385237,386251,387242,386913,385866,384851
  Per-template-ok: 384397,385237,386251,387242,386913,385866,384851
[info] CPU 3207.9% | Mem 315MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.86ms     81us    149us   11.30ms    1.98s

  2707948 requests in 5.00s, 2707818 responses
  Throughput: 541.39K req/s
  Bandwidth:  1.82GB/s
  Status codes: 2xx=2707818, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2707799 / 2707818 responses (100.0%)
  Reconnects: 108221
  Per-template: 385374,386626,385999,388170,388758,387515,385357
  Per-template-ok: 385374,386626,385999,388170,388758,387515,385357
[info] CPU 2121.8% | Mem 396MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   5.58ms     81us    150us   11.40ms    1.76s

  2723920 requests in 5.00s, 2723726 responses
  Throughput: 544.57K req/s
  Bandwidth:  1.84GB/s
  Status codes: 2xx=2723726, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 2723710 / 2723726 responses (100.0%)
  Reconnects: 108821
  Per-template: 387773,388549,388575,389874,390896,390393,387650
  Per-template-ok: 387773,388549,388575,389874,390896,390393,387650
[info] CPU 3102.4% | Mem 442MiB

=== Best: 544745 req/s (CPU: 3102.4%, Mem: 442MiB) ===
[info] input BW: 25.98MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

praisethemoon · 2026-05-17T21:01:56Z

/benchmark -f typev --save

github-actions · 2026-05-17T21:02:02Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-05-17T21:12:00Z

Benchmark Results

Framework: typev | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline	512	1,871,533	6498.3%	251MiB	NEW	NEW
baseline	4096	1,918,007	6737.7%	384MiB	NEW	NEW
pipelined	512	47,930,259	6303.1%	276MiB	+3.6%	-12.4%
pipelined	4096	48,398,726	6254.3%	355MiB	+5.2%	-29.7%
limited-conn	512	912,050	4779.8%	211MiB	NEW	NEW
limited-conn	4096	1,282,861	6597.3%	304MiB	NEW	NEW
json	4096	1,077,677	6730.8%	402MiB	NEW	NEW

Full log

  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.72ms     95us    240us   64.50ms   205.60ms

  6367531 requests in 5.00s, 6275174 responses
  Throughput: 1.25M req/s
  Bandwidth:  107.69MB/s
  Status codes: 2xx=6275174, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6275158 / 6275174 responses (100.0%)
  Reconnects: 719861
  Per-template: 2090911,2092607,2091640
  Per-template-ok: 2090911,2092607,2091640
[info] CPU 6280.4% | Mem 269MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.13ms     93us    215us   194.90ms   205.50ms

  6414717 requests in 5.00s, 6414306 responses
  Throughput: 1.28M req/s
  Bandwidth:  110.08MB/s
  Status codes: 2xx=6414306, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6414283 / 6414306 responses (100.0%)
  Reconnects: 641598
  Per-template: 2138396,2138222,2137665
  Per-template-ok: 2138396,2138222,2137665
[info] CPU 6597.3% | Mem 304MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  10
  Templates: 3
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.16ms     93us    218us   196.90ms   207.60ms

  6362896 requests in 5.00s, 6361664 responses
  Throughput: 1.27M req/s
  Bandwidth:  109.18MB/s
  Status codes: 2xx=6361664, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 6361637 / 6361664 responses (100.0%)
  Reconnects: 637393
  Per-template: 2120735,2120331,2120571
  Per-template-ok: 2120735,2120331,2120571
[info] CPU 6363.9% | Mem 304MiB

=== Best: 1282861 req/s (CPU: 6597.3%, Mem: 304MiB) ===
[info] input BW: 99.10MB/s (avg template: 81 bytes)
[info] saved results/limited-conn/4096/typev.json
httparena-bench-typev
httparena-bench-typev

==============================================
=== typev / json / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.34ms    106us    177us   3.09ms   562.60ms

  5363740 requests in 5.00s, 5363745 responses
  Throughput: 1.07M req/s
  Bandwidth:  3.61GB/s
  Status codes: 2xx=5363745, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 5363728 / 5363745 responses (100.0%)
  Reconnects: 214170
  Per-template: 762388,764875,767419,769008,769615,767256,763167
  Per-template-ok: 762388,764875,767419,769008,769615,767256,763167
[info] CPU 6393.6% | Mem 452MiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.34ms    105us    175us   1.36ms   569.90ms

  5388384 requests in 5.00s, 5388386 responses
  Throughput: 1.08M req/s
  Bandwidth:  3.63GB/s
  Status codes: 2xx=5388386, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 5388367 / 5388386 responses (100.0%)
  Reconnects: 215192
  Per-template: 766282,768778,770454,773085,772850,770671,766247
  Per-template-ok: 766282,768778,770454,773085,772850,770671,766247
[info] CPU 6730.8% | Mem 402MiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 7
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.37ms    106us    176us   3.08ms   577.50ms

  5379785 requests in 5.00s, 5379036 responses
  Throughput: 1.08M req/s
  Bandwidth:  3.63GB/s
  Status codes: 2xx=5379036, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 5379023 / 5379036 responses (100.0%)
  Reconnects: 215566
  Per-template: 764598,766000,768934,771991,771206,770129,766165
  Per-template-ok: 764598,766000,768934,771991,771206,770129,766165
[info] CPU 6445.5% | Mem 448MiB

=== Best: 1077677 req/s (CPU: 6730.8%, Mem: 402MiB) ===
[info] input BW: 51.39MB/s (avg template: 50 bytes)
[info] saved results/json/4096/typev.json
httparena-bench-typev
httparena-bench-typev
[info] skip: typev does not subscribe to json-comp
[info] skip: typev does not subscribe to json-tls
[info] skip: typev does not subscribe to upload
[info] skip: typev does not subscribe to api-4
[info] skip: typev does not subscribe to api-16
[info] skip: typev does not subscribe to static
[info] skip: typev does not subscribe to async-db
[info] skip: typev does not subscribe to crud
[info] skip: typev does not subscribe to fortunes
[info] skip: typev does not subscribe to baseline-h2
[info] skip: typev does not subscribe to static-h2
[info] skip: typev does not subscribe to baseline-h2c
[info] skip: typev does not subscribe to json-h2c
[info] skip: typev does not subscribe to baseline-h3
[info] skip: typev does not subscribe to static-h3
[info] skip: typev does not subscribe to gateway-64
[info] skip: typev does not subscribe to gateway-h3
[info] skip: typev does not subscribe to production-stack
[info] skip: typev does not subscribe to unary-grpc
[info] skip: typev does not subscribe to unary-grpc-tls
[info] skip: typev does not subscribe to stream-grpc
[info] skip: typev does not subscribe to stream-grpc-tls
[info] skip: typev does not subscribe to echo-ws
[info] skip: typev does not subscribe to echo-ws-pipeline
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

praisethemoon · 2026-05-17T21:17:54Z

Calling it a run! Good to go from my end. (For real this time)

MDA2AV · 2026-05-17T21:18:54Z

Nice improvements already

typev: add baseline, json and limited-conn tests

c789346

praisethemoon requested review from Kaliumhexacyanoferrat and MDA2AV as code owners May 17, 2026 11:21

typev: buffer-based response path reducing allocations

0cac880

Benchmark results: typev

0a6afa1

typev: restore pipeline fast path + allocation-free parsing

4310b5d

typev: update compiler & bytecode (codegen bug), removed fixed file opt

097311d

type: json micro opt: compact json

e0ec5c3

typev: inplace copy instead of manual iterate and push

6116a70

typev: 4 -> 32 shards

2e6ee6e

Benchmark results: typev

5d19891

typev: revert back to 4 shards

973a2cc

Benchmark results: typev

9980940

MDA2AV approved these changes May 17, 2026

View reviewed changes

MDA2AV merged commit cebaaaa into MDA2AV:main May 17, 2026

Conversation

praisethemoon commented May 17, 2026

Description

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

MDA2AV commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

praisethemoon commented May 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

MDA2AV commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

MDA2AV commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Uh oh!

github-actions Bot commented May 17, 2026

Benchmark Results

Uh oh!

praisethemoon commented May 17, 2026

Uh oh!

MDA2AV commented May 17, 2026

Uh oh!

Reviewers

Assignees

praisethemoon commented May 17, 2026 •

edited

Loading