feat(sync): quorum/sync subsystem hardening (proof spot-check, bad-tfile cache, chain re-selection)

[adequatelimited]

Summary

Subsystem overhaul hardening the node sync flow against fabricated or corrupt peer advertisements. Builds on F-09 (#148): where that change ensured quorum members share the same chain, this change ensures the chain they advertise is actually the chain they serve, and that the node degrades gracefully when it is not.

Scope is intentionally narrow: only src/network.c, src/sync.c, src/bin/mochimo.c, and two new files (src/syncguard.{h,c}). Consensus rules, transaction validation, PoW, ledger format, and all other subsystems are unchanged.

Commits

• bc9b1cb — Phase 1: data structures and session caches (syncguard.{h,c})
• 1125f13 — Phase 2: proof spot-check in scan_quorum()
• ca39658 — Phase 3: tfile spot-check and bad-tfile caching in resync()
• 44d3549 — Phase 4: chain re-selection on quorum exhaustion
• 628b45e — fix(syncguard): spot-check proof by bnum offset, not tfile tail

(Plus the F-09 commit 0411f05 at the base, already merged to master via #149.)

Why this overhaul

The F-09 narrow fix solved the quorum assembly determinism problem but left several gaps that the audit identified during review:

Gap	State before this PR	Impact
Pre-download proof spot-check	None — peers self-report weight/hash/bnum, trusted up to tfile validation	Malicious peer can waste node's bandwidth serving invalid tfiles
Known-bad tfile caching	None — a failed tfile is discarded, the next quorum member's tfile is re-downloaded and re-validated from scratch	Repeated validation cost for the same bad tfile distributed across multiple coordinated attackers
Chain re-selection on failure	None — quorum members are removed one-by-one until empty, then restart() (exit)	Infinite sync loop if all quorum members share a corrupt chain
Fabricated-weight defense	None — scan trusts peer-reported weight until tfile download fails	Sybil attack can stall node sync indefinitely

What changed

Phase 1: src/syncguard.{h,c} — data structures and session caches

Session-local in-memory caches:

• Bad-chain exclusion list — (weight, hash) pairs that failed sync this session
• Bad-tfile hash cache — SHA-256 hashes of tfiles that failed any validation step
• Per-peer proof segment cache — last NTFTX trailers validated for each quorum candidate, keyed by IP

All caches zero-init at process start. They persist across resync() retries within the same process so the node does not fall into repeated failures against the same bad actors.

Also provides:

• sg_hash_file() — SHA-256 of a file on disk, for bad-tfile cache keys
• sg_proof_match_tfile() — byte-exact comparison of cached proof against the corresponding bnum range in a downloaded tfile
• sg_validate_proof_chain() — structural chain validation (bnum increments by 1, phash links, tip matches advertised). No PoW check; that is too expensive per-peer and is fully validated downstream in validate_tfile_pow().

Phase 2: src/network.c — proof spot-check in scan_quorum()

Adds get_tf_proof() helper to request NTFTX trailers via the existing OP_TF opcode (no new wire protocol). Receives to a per-IP temp file on disk for safety under the OMP parallel scan loop.

In scan_quorum(), each candidate peer is now:

1. Checked against the bad-chain exclusion list (skip if known bad)
2. Asked for its last NTFTX trailers
3. Structurally validated via sg_validate_proof_chain()
4. Admitted to the quorum only if proof-valid AND matching the current high-chain hash. Their validated proof is stored via sg_proof_store() for the Phase 3 tfile tail check.

Effect: a peer advertising a fabricated (weight, hash, bnum) triple that can't produce a self-consistent NTFTX proof segment is rejected immediately.

Phase 3: src/sync.c — tfile spot-check and bad-tfile caching in resync()

The tfile acquisition loop now:

1. Downloads tfile from a quorum member
2. Hashes the downloaded file (SHA-256) and consults the bad-tfile cache — skip if a previous peer already served identical bad content
3. Spot-check: seek to the proof's bnum * sizeof(BTRAILER) offset in the downloaded tfile and byte-compare against the cached proof from Phase 2. Mismatch = the peer served a different chain than it advertised. Add the tfile hash to the bad cache and drop the peer.
4. Existing validate_tfile() + validate_tfile_pow() checks. On any failure, add the tfile hash to the bad cache and drop the peer.
5. Verify advertised (bnum, weight) is actually met by the tfile. On mismatch, cache and drop.

Previously, any tfile failure immediately returned VERROR from resync(). Now the loop iterates through quorum members, giving the node a chance to find a good tfile without exiting the process.

Phase 4: src/sync.c, src/sync.h, src/bin/mochimo.c — chain re-selection

Replaces restart() calls in resync()'s gettfile and getneo phases with sg_bad_chain_add() + return VERROR. The bootstrap loop in mochimo.c then re-scans, and scan_quorum() (which already consults sg_bad_chain_check() from Phase 2) skips the excluded chain. Adds a highhash parameter to resync() so it knows which chain to mark bad.

restart() exit(1)'d the process and lost all session caches. The new behavior keeps the caches alive across chain-selection retries.

Design fix: proof matching by bnum offset

Initially Phase 3 compared the proof to the tfile's tail, but the chain may advance between scan_quorum() and resync(). Since the tfile stores trailers in bnum-continuous order (trailer at byte offset N * sizeof(BTRAILER) has bnum == N), the proof should be sought at proof[0].bnum * sizeof(BTRAILER) and matched in place. The tfile may have advanced past the proof's tip; we only care that the proof's own historical range matches byte-exactly.

Threat model

Attack	Current defense	Defense after this PR
Single malicious peer advertising fake weight	Tfile validation eventually rejects	Proof spot-check in Phase 2 rejects before tfile download
Sybil group at fabricated weight	Tfile validation eventually rejects but node may loop	Proof spot-check + bad-chain caching + chain re-selection
Coordinated bad tfile distribution	Repeated full validation of each bad copy	Bad-tfile hash cache short-circuits after first validation
Partial tfile with crafted tail	Passes validate_tfile_pow() if PoW is real	Proof segment byte-match catches tail-tampering
Bandwidth amplification via malicious catchup peer	b_update() rejects bad blocks but wastes bandwidth	Quorum membership now predicated on proof validation; malicious peers are filtered before catchup

Deferred follow-ups

This PR is intentionally conservative on the following, which can be handled separately:

• Plurality-based chain selection (design doc Stage 5): two-pass scan to pick the chain with the most peers at max weight. Strictly stronger than F-09 numerically-highest-hash tiebreaking during legitimate splits. Not implemented because (a) it adds scan latency and memory, (b) the other defenses in this PR already counter Sybil-inflated quorums, and (c) splits resolve naturally once block propagation continues. Left as future work if operators see split-related issues in practice.
• Persistent (across-restart) bad-chain / bad-tfile caches with staleness controls.
• Full PoW validation on proof segments (currently structural only to keep the scan fast).

Verification

• Clean build: make NO_CUDA=1 passes on gcc-13 / Ubuntu WSL (all warnings-as-errors).
• End-to-end smoke test on mainnet (with PoW bypass for speed; reverted before commit):
  • Multiple peers qualified with qualified (proof verified) log messages
  • Proof fetch and structural validation work
  • Tfile spot-check correctly rejected peers advertising a chain whose tfile didn't match
  • After the offset fix, a legitimate peer's tfile passed the spot-check
  • tfile.dat is valid and matches advertised bnum and weight
  • catchup() progressed through blocks applying Update-block/Pseudo-block normally

Test plan

• Clean build on Ubuntu x64, macOS arm64, Ubuntu arm64 CI
• make test passes
• Live mainnet sync from a fresh working directory reaches steady-state
• Behavior against a mock malicious peer advertising an unbuildable chain: rejected before tfile download (requires test harness)
• Behavior against a mock malicious peer serving a tfile whose tail doesn't match its proof: tfile cached as bad, peer dropped, next peer tried

[adequatelimited]

This subsystem is a must-have for v3.1, so will need to be included in that code, but is not required for the current audit remediation cycle.