feat: add zallet-rpc-diff parity harness (MVP)#404
Open
CreativesOnchain wants to merge 17 commits into
Open
Conversation
CreativesOnchain
changed the title
- Add tools/zallet-rpc-diff workspace with three crates: - zallet-parity-core: library with custom Error types - zallet-parity-cli: binary (zallet-rpc-diff) with clap, color-eyre, tracing - zallet-parity-testkit: test helpers with wiremock - Add DESIGN_NOTE.md defining MATCH/DIFF/MISSING/ERROR categories, report schema, and manifest format - Add CI workflow (.github/workflows/zallet-rpc-diff.yml) - All crates compile cleanly in the workspace This satisfies Milestone 1 deliverables: - Clearly defined report format and manifest structure - Rust crate + CLI skeleton ready for maintainer review
CreativesOnchain
force-pushed
the
feat/zallet-rpc-diff-milestone-1
branch
from
bb258ae to
84e8790
Compare
CreativesOnchain
changed the title
… tests - Add MethodNotFound error variant to lib.rs Error enum - Engine detects JSON-RPC -32601 and maps to ParityResult::Missing - report.rs: MISSING is now a first-class category with its own counter - testkit: add mock_method_not_found() and mock_rpc_error() helpers - engine tests: 4 cases covered — MATCH, DIFF, MISSING, ERROR - All 5 workspace tests pass
…context Milestone 3 implementation: normalizer.rs - sort_keys(): recursive BTreeMap-based canonical JSON key ordering - apply_ignore_paths(): RFC 6901 JSON Pointer path removal via jsonptr v0.7.1 - normalize(): combined pipeline (sort → remove paths) - 10 unit tests covering ordering, nesting, noop, and combined cases differ.rs - DiffEntry: serializable struct with RFC 6901 path + upstream/target values - diff_values(): recursive walker that returns Vec<DiffEntry> - 8 unit tests covering scalars, nested objects, arrays, missing keys, RFC escaping manifest.rs - Add ignore_paths: Vec<String> field to MethodEntry (defaults to empty) - 2 manifest parsing tests via tempfile engine.rs - Wire full normalization pipeline before comparison - ParityResult::Diff now carries Vec<DiffEntry> (structured paths) - 2 new engine tests: ordering-only diff → MATCH, ignore_path → MATCH report.rs - ParityResultReport::Diff exposes diff_count + diff_paths (no payload dump) - Markdown table shows exact changed paths per method Total: 27 tests passing
- Default: 30 seconds (DEFAULT_REQUEST_TIMEOUT const) - New with_timeout() constructor for configurable timeouts - Satisfies M2 deliverable: 'explicit per-request timeouts'
…F classification
expected_diffs.rs
- ExpectedDiffEntry: TOML-parseable struct with method, reason, diff_paths
- ExpectedDiffs::load(): reads expected_diffs.toml from disk
- ExpectedDiffs::none(): empty fallback (no file required)
- is_expected(): method-level (any diff) and field-level (path-prefix) matching
- 9 unit tests: parsing, method-level matching, field-level coverage, partial coverage
engine.rs
- ParityResult::ExpectedDiff { diff_entries, reason } — new variant
- run_all() now accepts &ExpectedDiffs parameter
- Promotes Diff → ExpectedDiff when diff paths are fully covered by an entry
- 3 new engine tests: method-level, field-level covered, partial-coverage still DIFF
report.rs
- RunSummary gains expected_diffs counter
- ParityResultReport::ExpectedDiff { diff_count, diff_paths, reason }
- Markdown output shows 📋 Expected Diff row with reason inline
main.rs (CLI)
- --expected-diffs flag (default: expected_diffs.toml, optional)
- Silently falls back to ExpectedDiffs::none() if file absent
- CLI summary line shows all 5 outcome categories
Total: 39 tests passing
manifest.toml (v1) - 24 real Zcash JSON-RPC methods across 5 categories: blockchain, network, wallet, shielded, mining, utility - Each method includes: ignore_paths for volatile fields, tags for filtering - Inline comments explain every ignore_path decision RUNBOOK.md - Prerequisites: node setup, wallet key parity, network match - Installation: build from source, or cargo run - Configuration: env vars (endpoints.env) vs CLI flags - Invocation patterns: minimal, custom output, with expected-diffs, verbose - Exit code table: 0 (clean) / 1 (diffs found) / 2 (tool failure) - Outcome interpretation: MATCH/DIFF/EXPECTED_DIFF/MISSING/ERROR table - Extension guide: how to add methods + ignore_paths vs expected_diffs guidance - Troubleshooting: common ERROR/DIFF/MISSING failure modes examples/ - endpoints.env — env var template for UPSTREAM_URL / TARGET_URL - expected_diffs.toml — 4 documented known zcashd vs Zallet divergences - sample_report.json — illustrates all 5 outcome categories - sample_report.md — Markdown version of the same sample
main.rs — CLI UX polish - ExitCode 0: all MATCH or EXPECTED_DIFF (clean) - ExitCode 1: any DIFF, MISSING, or ERROR found - ExitCode 2: tool-level failure (config/manifest/IO error) - Rich --help: long_about on binary, per-arg long_help with format examples - Actionable error messages: manifest path, expected-diffs path, endpoint URL all included in fatal error output - Stderr advisory messages for DIFF / MISSING / ERROR outcomes - Progress bar message shows method count upfront - Summary line now shows all 5 categories with emoji labels CI (.github/workflows/zallet-rpc-diff.yml) - working-directory: tools/zallet-rpc-diff (all steps scoped correctly) - --workspace flag on all cargo invocations - --locked flag for reproducible builds - Cargo registry + target cache (actions/cache@v4) - Clippy: --all-targets --all-features -D warnings - Single consolidated 'ci' job with logical step ordering Code quality - cargo fmt --all applied; format check passes clean - cargo clippy --all-targets -- -D warnings: zero warnings - 39 tests passing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This draft PR introduces the
zallet-rpc-diffparity harness — a reproducible, production-grade CLI tool that compareszcashdand Zallet wallet JSON-RPC responses, classifying each method asMATCH/DIFF/MISSING/ERROR.This addresses the upstream request in #16.
✅ MVP Deliverables
Design & Scaffolding
See
tools/zallet-rpc-diff/DESIGN_NOTE.mdwhich explicitly defines the result categories,report.jsonschema, and the versionedmanifest.tomlformat.Rust Crate + CLI Skeleton
zallet-parity-core— library with customErrortypes and result categorization.zallet-parity-cli— binaryzallet-rpc-diffwithclapv4,color-eyre, andtracing.zallet-parity-testkit— test helpers suite.CI Workflow
.github/workflows/zallet-rpc-diff.yml— scoped workspace checks (build, test, fmt, clippy).Execution Engine & Reporting
Implemented an async runner in
zallet-parity-coreusingtokio::task::JoinSet. It executes the entire manifest suite concurrently against both upstream and target endpoints.Deep Semantic Comparison
Integrated
assert-json-diffto perform recursive comparison ofserde_json::Valueresponses. It correctly identifies differences while being resilient to key ordering.Premium CLI & Reporting
indicatifprogress bars for real-time feedback during large suites.report.json(for automation) and a human-readablereport.md(for PR review).Automated Verification Suite
Implemented a robust testing suite using
wiremockin thetestkit:cargo test --workspaceQuestions for Maintainers
tools/zallet-rpc-diff/acceptable, or would you prefertesting/zallet-rpc-diff/?memberslist, or kept as a self-contained sub-workspace?manifest.tomlis a sample; I'm happy to align the v1 allowlist with your priorities.TODOs (Future Milestones)