Skip to content

Comments

Switch to cargo-llvm-cov for code coverage#371

Open
Copilot wants to merge 1 commit intomasterfrom
copilot/update-code-coverage-tool
Open

Switch to cargo-llvm-cov for code coverage#371
Copilot wants to merge 1 commit intomasterfrom
copilot/update-code-coverage-tool

Conversation

Copy link

Copilot AI commented Jan 19, 2026

Description

Replaces grcov with cargo-llvm-cov for coverage generation. grcov lacks branch coverage support and has other limitations.

Workflow changes:

  • Use nightly toolchain with cargo-llvm-cov
  • Set RUSTFLAGS="--cfg coverage_nightly" during coverage runs only
  • Remove lcov tools and old coverage environment variables
  • Enable branch coverage with --branch flag
  • Add --quiet flag to reduce output verbosity

Code changes:

  • Enable coverage_attribute feature via #[cfg_attr(coverage_nightly, feature(coverage_attribute))] in lib.rs
  • Exclude test modules from coverage with #[cfg_attr(coverage_nightly, coverage(off))] on 13 test modules
  • Add coverage_nightly to [lints.rust] in Cargo.toml to declare the custom cfg

Stable builds are unaffected. Coverage exclusions only activate when the cfg is set.

Notes to the reviewers

The coverage_nightly cfg is only set during coverage generation in CI. Regular builds and tests use stable toolchain without this cfg.

Branch coverage is now explicitly enabled, providing more detailed coverage metrics than the previous grcov setup.

Changelog notice

Checklists

All Submissions:

New Features:

  • I've added tests for the new feature
  • I've added docs for the new feature

Bugfixes:

  • This pull request breaks the existing API
  • I've added tests to reproduce the issue which are now passing
Original prompt

This section details on the original issue you should resolve

<issue_title>[coverage] Consider using cargo-llvm-cov</issue_title>
<issue_description>Describe the enhancement

Switch to using llvm-cov in .github/workflows/code_coverage.yml for generating coverage data. For example see bitcoindevkit/bdk#1986.

Impact

  • Blocking production usage
  • Nice-to-have / UX improvement
  • Developer experience / maintainability

Additional context

<agent_instructions>In order to switch to using cargo-llvm-cov in code_coverage.yml I recommend using a recent version of the nightly compiler in the "Install Rust toolchain" step. Note also that it shouldn't be necessary to include the extra environment variables (RUSTFLAGS, etc) under env: in code_coverage.yml. Additionally, consider enabling the coverage_nightly config attribute globally in lib.rs so we can exclude test modules from coverage reports using #[cfg_attr(coverage_nightly, coverage(off))].</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] Switch to cargo-llvm-cov for code coverage Switch to cargo-llvm-cov for code coverage Jan 19, 2026
Copilot AI requested a review from ValuedMammal January 19, 2026 18:24
@codecov
Copy link

codecov bot commented Jan 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.02%. Comparing base (bb9b07f) to head (527037c).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #371      +/-   ##
==========================================
- Coverage   86.99%   79.02%   -7.98%     
==========================================
  Files          25       23       -2     
  Lines        8860     5239    -3621     
  Branches        0      242     +242     
==========================================
- Hits         7708     4140    -3568     
+ Misses       1152     1022     -130     
- Partials        0       77      +77     
Flag Coverage Δ
rust 79.02% <ø> (-7.98%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ValuedMammal ValuedMammal marked this pull request as ready for review January 19, 2026 18:56
@ValuedMammal ValuedMammal moved this to In Progress in BDK Wallet Jan 19, 2026
@ValuedMammal ValuedMammal added this to the Wallet 3.0.0 milestone Jan 19, 2026
@ValuedMammal ValuedMammal added the github_actions Pull requests that update GitHub Actions code label Jan 19, 2026
@ValuedMammal
Copy link
Collaborator

Concept ACK, haven't done an in-depth review.

@ValuedMammal
Copy link
Collaborator

@copilot When running the cargo llvm-cov command can you make the output quieter, and also enable --branch coverage, thanks.

Copy link
Author

Copilot AI commented Jan 20, 2026

@copilot When running the cargo llvm-cov command can you make the output quieter, and also enable --branch coverage, thanks.

Done in commit 5ffaacc. Added --branch flag for branch coverage and --quiet flag to reduce output verbosity.

@ValuedMammal ValuedMammal moved this from In Progress to Needs Review in BDK Wallet Jan 20, 2026
Copy link
Collaborator

@ValuedMammal ValuedMammal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I'd only suggest redoing the commits to follow the contribution guidelines before merging. The drop in coverage is expected, since we were previously reporting coverage for non-library test code.

@notmandatory
Copy link
Member

notmandatory commented Feb 19, 2026

@ValuedMammal did you trigger copilot to do this one? just curious, I haven't tried using it yet. Seems odd to have @copilot as the author.

@notmandatory
Copy link
Member

Looks like this PR needs to be rebased also.

@luisschwab
Copy link
Member

@copilot rebase

@luisschwab
Copy link
Member

@copilot rebase

doesn't seem to work the same as dependabot

@ValuedMammal
Copy link
Collaborator

ValuedMammal commented Feb 19, 2026

@ValuedMammal did you trigger copilot to do this one? just curious, I haven't tried using it yet. Seems odd to have copilot as the author.

Yeah I did. It saved me from doing the work but now it's a bit awkward because someone needs to fixup the commits.

- Updated .github/workflows/code_coverage.yml to use cargo-llvm-cov instead of grcov
- Removed environment variables (RUSTFLAGS, RUSTDOCFLAGS, LLVM_PROFILE_FILE)
- Switched to nightly toolchain for coverage generation
- Enabled coverage_nightly config attribute in lib.rs
- Added coverage(off) attribute to all test modules to exclude them from coverage reports
- Added lint config for coverage_nightly cfg to Cargo.toml
- Also add lcov.info to .gitignore to prevent accidental commits
@ValuedMammal ValuedMammal force-pushed the copilot/update-code-coverage-tool branch from 5aaa70b to 527037c Compare February 19, 2026 21:40
Copy link
Member

@notmandatory notmandatory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 527037c

@ValuedMammal
Copy link
Collaborator

ACK 527037c

Copy link
Member

@luisschwab luisschwab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 527037c

RUSTFLAGS: "--cfg coverage_nightly"
- name: Generate HTML coverage report
run: genhtml -o coverage-report.html --ignore-errors unmapped ./coverage/lcov.info
run: cargo llvm-cov --all-features --branch --quiet --ignore-filename-regex "test_utils" --html
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not reusing the lcov.info file to produce a report with genhtml or something similar?
What's the purpose of keeping the html file in the artifacts if the .info file is enough for codecov?
If it is for auditing reasons, why not keeping only the lcov.info files instead of the bloated html?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would remove the html file generation altogether and keep only the lcov.info as artifacts. If we want to review them later we can pull them locally and generate an html ourselves. This avoids adding the lcov tools suite to the CI.
I can apply the changes if there are no concerns.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code

Projects

Status: Needs Review

Development

Successfully merging this pull request may close these issues.

[coverage] Consider using cargo-llvm-cov

5 participants