Performance: Add warmup cache#18473
Conversation
mh0lt
left a comment
There was a problem hiding this comment.
How is this code dealing with hash collisions ?
Given that you are using a 64 bit hash this is quite likely, if this happens the value returned won't necessarily be for the expected key - which is likely to cause hard to debug issues.
|
@mh0lt this is not likely, do the math yourself. it is very hard even with birthday paradox |
|
btw in the entire history of the chain there is 100% a collision no doubt but for it to happen, you need to use THAT pair and that probability is actually bounded in the best-case (pessimistic) by random reads. in reality, we dont do random reads in blockchain. pattern exists so even that pessimistic unlikely case is too optimistic. |
|
@antonis19 can you approve? |
| if be.cache != nil { | ||
| prev, prevStep, foundInCache = be.cache.GetAndEvictBranch(prefix) | ||
| } | ||
| if !foundInCache { | ||
| prev, prevStep, err = ctx.Branch(prefix) | ||
| if err != nil { | ||
| return 0, err | ||
| } |
There was a problem hiding this comment.
This code can be simplified into ctx.Branch(prefix) the behavior will be different based on the concrete type of ctx (i.e if it is with cache or without cache)
| if !cell.loaded.storage() { | ||
| hph.metrics.StorageLoad(cell.storageAddr[:cell.storageAddrLen]) | ||
| update, err := hph.ctx.Storage(cell.storageAddr[:cell.storageAddrLen]) | ||
| update, err := hph.storageFromCacheOrDB(cell.storageAddr[:cell.storageAddrLen]) |
There was a problem hiding this comment.
Can use hph.ctx , just have 2 concrete implementations (1 with cache, 1 without). The desired implementation can be instantiated based on the config

This PR introduces a warmup cache for the commitment trie processing that reduces redundant DB reads during block execution. Data fetched during the warmup phase is cached and reused during the actual trie computation, improving performance.
Key Changes
WarmupCache- A cache that stores pre-fetched branch, account, and storage data usingmaphash.Mapfor efficient byte slice lookups without allocationsWarmuper- The warmuper now optionally caches data it reads, making it available during trie processingBranchEncodercache support - Branch lookups check the cache first before hitting the DBArchitecture Diagram