Skip to content

test: add coverage for validator registry and service (both at 0%) #493

@tcoratger

Description

@tcoratger

Summary

src/lean_spec/subspecs/validator/registry.py and src/lean_spec/subspecs/validator/service.py both have 0% test coverage — no test files exist. These are actively used production modules (imported by __main__.py, node.py, and each other) that manage validator key loading and drive block/attestation production.

Coverage stats:

File Statements Covered Coverage
validator/registry.py 118 0 0%
validator/service.py 159 0 0%

What needs testing

Part 1: registry.py — Validator key management

ValidatorManifestEntry (Pydantic model)

  • Field validation: Verify parse_pubkey_hex converts integer pubkeys to zero-padded hex strings (YAML parsers may auto-convert 0x... to int)
  • String passthrough: Verify hex string pubkeys pass through unchanged
  • Model construction: Verify all fields (index, attestation/proposal pubkey hex, privkey files) are stored correctly

ValidatorManifest (Pydantic model)

  • YAML loading: Test from_yaml_file() with a valid manifest file (use tmp_path fixture)
  • Field validation: Verify all metadata fields (key_scheme, hash_function, encoding, lifetime, etc.)
  • Validators list: Verify nested ValidatorManifestEntry objects are parsed correctly

load_node_validator_mapping()

  • Normal loading: YAML with multiple nodes mapping to validator indices
  • Empty file: Returns empty dict (YAML parses to None)
  • Single node: One node with multiple indices

ValidatorRegistry (dataclass)

  • add() / get(): Add entries and retrieve by index
  • get() miss: Returns None for unknown index
  • __contains__: Membership check for known/unknown indices
  • indices(): Returns ValidatorIndices with all registered indices
  • primary_index(): Returns first index, or None for empty registry
  • __len__: Correct count after adds
  • from_yaml(): Integration test loading from real YAML files (use tmp_path)
    • Happy path: node with assigned validators
    • Unknown node: returns empty registry
    • Missing manifest entry: logs warning, skips validator
    • Missing key file: raises ValueError
    • Corrupt key file: raises ValueError
  • from_secret_keys(): Convenience constructor with dict of key pairs

ValidatorEntry (frozen dataclass)

  • Construction: Verify index and both secret keys are stored
  • Immutability: Frozen dataclass rejects attribute assignment

Part 2: service.py — Block and attestation production

ValidatorService — Properties and lifecycle

  • stop() / is_running: Verify running flag transitions
  • blocks_produced / attestations_produced: Counter properties start at 0

ValidatorService._sign_block()

  • Happy path: Signs block root with proposal key, returns SignedBlock with correct structure
  • Unknown validator: Raises ValueError
  • Signature structure: Verify BlockSignatures contains proposer signature and attestation signatures

ValidatorService._sign_attestation()

  • Happy path: Signs attestation data root with attestation key, returns SignedAttestation
  • Unknown validator: Raises ValueError
  • Correct fields: validator_id, data, and signature match expectations

ValidatorService._sign_with_key()

  • Key advancement: Verify key is advanced until the target slot is in the prepared interval
  • Registry update: Verify the updated entry is persisted back to the registry
  • Both key types: Test with both "attestation_secret_key" and "proposal_secret_key"

ValidatorService._maybe_produce_block() (async)

  • Proposer match: When our validator is the proposer, produces and emits a block
  • Not proposer: No block produced when another validator is scheduled
  • No head state: Returns early without producing
  • Production failure (AssertionError): Logs and skips gracefully

ValidatorService._produce_attestations() (async)

  • All validators attest: Each registered validator produces an attestation
  • Block wait logic: Test the polling loop that waits for the current slot's block
  • Local processing: Attestation is processed locally before publishing (gossipsub doesn't self-deliver)
  • Duplicate prevention: Same slot doesn't produce duplicate attestations
  • Slot pruning: _attested_slots is pruned to bound memory

ValidatorService.run() (async, main loop)

  • Interval routing: Interval 0 triggers block production, interval ≥ 1 triggers attestation
  • Empty registry: Loop continues without producing
  • Sleep behavior: Sleeps until next interval when current is already handled

Why this matters

  • Block production correctness: The service decides when and how blocks are signed. Bugs here could produce invalid blocks, double-sign, or miss slots.
  • Key management safety: The registry loads and manages XMSS secret keys. Incorrect loading (wrong file, wrong index) could sign with the wrong key.
  • OTS key advancement: _sign_with_key() advances one-time signature keys. A bug could exhaust keys prematurely or reuse them.
  • Async timing logic: The main loop and attestation wait logic involve subtle timing — easy to introduce races or missed intervals.

How to test

Running tests with coverage

# Registry only
uv run pytest tests/lean_spec/subspecs/validator/test_registry.py -v \
  --cov=src/lean_spec/subspecs/validator/registry --cov-report=term-missing

# Service only
uv run pytest tests/lean_spec/subspecs/validator/test_service.py -v \
  --cov=src/lean_spec/subspecs/validator/service --cov-report=term-missing

# Both
uv run pytest tests/lean_spec/subspecs/validator/ -v \
  --cov=src/lean_spec/subspecs/validator --cov-report=term-missing

Target: ≥90% line and branch coverage for registry, ≥80% for service (async main loop is harder to cover fully).

Verifying with the coverage gate

uvx tox -e coverage-gate

Test file locations

tests/lean_spec/subspecs/validator/test_registry.py
tests/lean_spec/subspecs/validator/test_service.py

Testing tips

  • Registry tests are straightforward — use tmp_path for YAML files and mock SSZ key files
  • Service tests require mocking SyncService, SlotClock, and the forkchoice Store. Use pytest-asyncio for async tests
  • _sign_with_key needs real or mocked XMSS keys — check existing test fixtures in tests/ for patterns
  • Registry tests are a natural good first issue entry point; service tests are more involved

Using Claude Code subagents

If you're tackling this issue with Claude Code, use these agents for an efficient workflow:

1. code-tester agent — Generate the tests

Use this agent to scaffold comprehensive tests. Prompt example:

Generate unit tests for src/lean_spec/subspecs/validator/registry.py. Cover ValidatorManifestEntry field validation, ValidatorManifest.from_yaml_file(), load_node_validator_mapping(), and all ValidatorRegistry methods including from_yaml() with tmp_path YAML fixtures. Also generate async tests for src/lean_spec/subspecs/validator/service.py covering _sign_block, _sign_attestation, _maybe_produce_block, _produce_attestations, and the main run() loop.

2. doc-writer agent — Document the test modules

After tests are written, use this agent to add educational documentation. Prompt example:

Add documentation to the validator test files explaining the testing strategy: why YAML loading edge cases matter, why key advancement must be tested, and what security properties each test group validates.

Workflow

  1. Start with test_registry.py (simpler, good first issue material)
  2. Run coverage, iterate until ≥90%
  3. Move to test_service.py (async, needs mocks)
  4. Run coverage, iterate until ≥80%
  5. Run doc-writer to polish documentation
  6. Run uvx tox -e all-checks to pass all quality checks

Metadata

Metadata

Assignees

Labels

good first issueGood for newcomerstestsScope: Changes to the spec tests

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions