Refactor the ebpf frame data to be variable length #943
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- use unique FrameMappingData to reduce the Frame size - construct FrameMappingData directly from the cached ELF info - remove the FileID LRU cache as unneeded - replace processmnanager.processInfo.mappings map with a sorted slice of the []Mappings - atomically do batch updates of the []Mappings - remove processmanager.processInfo.mappingsByFileID as redudant TODO: - eim should probably get libpf.FrameMappingFile instead of host.FileID - eventually move processmanager.Mapping to interpreters and pass it as the mapping type
- use slices.SortFunc - add mapping.Length to processmanager Mapping - move sorting and publishing of changed mappings *after* the handling of the ebpf modifications -> the sort reordered things and mpAdd pointers point to wrong data - don't hold lock during interpreter synchronize mappings call
stdout is line cached. its glibc extension that reading stdin flushes stdout, but its not standard. fix test case to explicitly flush stdout.
fabled
force-pushed
the
tt-variable-length-frame
branch
5 times, most recently
from
5017685 to
b915153
Compare
Convert the ebpf frame structure to variable length interpreter specific data. This has the benefit of using less data for many frame types, but also allow sending more data if it benefits the interpreter. Set the frame data maximum to 3kB uint64:s to allow large traces. Update the tracer to do frame caching to not pressure GC too much.
fabled
force-pushed
the
tt-variable-length-frame
branch
from
b915153 to
dd14424
Compare
dalehamel
reviewed
Nov 10, 2025
|
|
||
| // The frames of the stack trace. | ||
| Frame frames[MAX_FRAME_UNWINDS]; | ||
| u64 frame_data[3072]; |
Contributor
There was a problem hiding this comment.
I think it would be good to also add a check to the size of the overall PerCPURecord struct, which seems to be 32768, eg:
// https://github.com/torvalds/linux/blob/e9a6fb0bcdd7609be6969112f3fbfcce3b1d4a7c/include/linux/percpu.h#L24C39-L24C47
_Static_assert(sizeof(PerCPURecord) <= (32 << 10), "Per CPU record too large")
AFAICT, this is where the limit for a per cpu record is defined:
And here is where the check seems to happen:
I validated this (on main) by increasing the size of the Frame buffer. It is able to load the program with a frame buffer size of 1285, and a total size of exactly 32 << 10
time=2025-11-10T12:59:37.444-05:00 level=WARN msg="per_cpu_records, PerCPUArray(keySize=4, valueSize=32768, maxEntries=1, flags=0)"
but fails with 1286:
time=2025-11-10T13:00:30.257-05:00 level=WARN msg="per_cpu_records, PerCPUArray(keySize=4, valueSize=32792, maxEntries=1, flags=0)"
time=2025-11-10T13:00:30.258-05:00 level=ERROR msg="Failed to start agent controller: failed to load eBPF tracer: failed to load eBPF code: failed to load eBPF maps: failed to load per_cpu_records: creating map: map create: cannot allocate memory"
This buffer is the largest factor in that, taking up 24576 bytes, but we still have some room to potentially increase this (though, should leave room for additions to other struct members of PerCPURecord)
This was referenced Nov 11, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Convert the ebpf frame structure to variable length interpreter specific data. This has the benefit of using less data for many frame types, but also allow sending more data if it benefits the interpreter.
Set the frame data maximum to 3kB uint64:s to allow large traces.
Update the tracer to do frame caching to not pressure GC too much.
fixes #940
includes also PR #749 so that should be merged first (keeping this as a draft until its merged)