mempool to slasher stats bridge#1067
Conversation
🧪 Network TestsTo run network tests for this PR, use: gh workflow run network-tests.yml -f pr_number=1067Available test options:
Test types: Results will be posted as workflow runs in the Actions tab. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## feature/slasher-session #1067 +/- ##
==========================================================
Coverage ? 58.06%
==========================================================
Files ? 465
Lines ? 78608
Branches ? 78608
==========================================================
Hits ? 45647
Misses ? 30838
Partials ? 2123 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
7ac1bd1 to
ceef721
Compare
c959be2 to
2b844cc
Compare
ce42645 to
efd9043
Compare
| stats: AnchorPeerStats { points_proven: 0 }, | ||
| }) | ||
| .collect::<Vec<_>>(); | ||
| anchor_stats_history.sort_unstable_by_key(|a| a.validator_idx); |
There was a problem hiding this comment.
anchor_stats_history is sorted here, but signatures_history is not sorted. But seems like it is required for them to be with the same sort order in the TLRead::read_from implementation
There was a problem hiding this comment.
That's right, because mempool emits stats already sorted and they are just passed as-is through collator.
Mempool stats don't need additional layer of validator_idx|PeerId remapping and sorting - there's a PeerSchedule inside mempool to keep all v(sub)set data
There was a problem hiding this comment.
misunderstood at first, fixed
fd6f7e2 to
1641371
Compare
0e96c8a to
e5216ab
Compare
1641371 to
ba0e4b9
Compare
e5216ab to
587cd70
Compare
allows to record anchor stats before current batch is advanced by validator
ba0e4b9 to
96b61f3
Compare
Model (pre-review): anchor stats belong to blocks in a slasher batch.
Corner cases:
received_and_collated=truefast path, when block is both collated and received from network. Then validation step is omitted and signatures are not reported to slasher. We actually have mempool stats for that block, but currently they are not reported.validate_block()returnsValidationStatus::Skipped) anchor stats are collected but not reported to slasher too.Slasher contract is left untouched intentionally until we agree on the models and code flow
The drawbacks of suggested design are:
max_consensus_lag_rounds=210 rounds of anchor stats, otherwise old vsetvalidator_idxs will be mixed with new one (mempool makes no difference between vset and subset changes), so subsets should not be changed too oftencommit_history_rounds=20 rounds lagInherited from general design: stats may not be reported to slasher near validation session end because detached validation tasks can finish after the session has already been closed on the slasher side; same for already flushed batches during the same session. Also in case a collated block is replaced by accepted by the quorum, its stats remain in slasher.
Parent branch code suggestion #1068: in
ValidatorSessionScope.remap_idsusePeerIdtoslotmap (instead ofvalidator_idx->slot) to allow cross-subset stat collection in mempool (likereferences_skipped: see commitfix(consensus): mempool stats across subset change)Updates after review
Model: anchor stats are imported during a slasher batch (disregarding block collation).
Old corner cases are removed.
New corner cases:
on_block_validated()oron_block_skipped()) - batches didn't overlap, so anchors imported for the first block were ignoredfeat(slasher): batches overlap; shouldn't we convert it into a PR to parent branch first, priorTODO: Split or grow the previous batch to not discard events?try_import_init_anchors()are ignoredanchors_cache.remove_last_imported_above(next_chain_time)removes anchors from block input, but stats are already reported - are anchors imported the same way on every node?