ci: collect EVE coverage per matrix job in Eden CI#1173
Open
eriknordmark wants to merge 2 commits into
Open
Conversation
6 tasks
CollectEveCoverage previously SCP'd the raw covmeta.* / covcounters.* files into a temp dir, ran `go tool covdata textfmt` on them, and let the deferred cleanup wipe the binary files on return. Only the text profile at <output-dir>/eden_e2e_coverage.txt survived. Callers that want to merge coverage across multiple runs (e.g. across the Eden test matrix) need the raw covdata so they can use `go tool covdata merge`; the text profile alone only supports concatenation, which is fine for a per-line summary but doesn't aggregate the way merge does. Copy the SCP target into <output-dir>/covdata/ instead of an auto-deleted temp dir. The text-profile output path is unchanged. Signed-off-by: eriknordmark <erik@zededa.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each suite in the Eden test matrix runs on its own runner with its own ./eden setup -> start -> stop -> make clean lifecycle, so per-job collection is the only point at which to capture covdata from a coverage-instrumented EVE build. After make clean the runner is wiped and there is no second chance. Add a Collect/Upload coverage pair to the run-eden-test composite action: invoke `eden eve collect-coverage` with `if: always()` and best-effort error handling so a failed test still surfaces whatever covdata was written, and upload the eden_coverage/ output directory (both the eden_e2e_coverage.txt text profile and the raw covdata subtree). Add a coverage-merge job to test.yml that downloads every covdata-* artifact, runs `go tool covdata merge` on the raw covmeta.* / covcounters.* files to produce a single merged covdata directory, converts it to a text profile via `covdata textfmt`, and runs `go tool cover -func` for a per-package summary. Producing useful coverage data requires the EVE artifact consumed by these jobs to be built with COVER=y so zedbox registers the SIGUSR2 handler and sets GOCOVERDIR=/persist/coverage. That is a separate change on the EVE side. Signed-off-by: eriknordmark <erik@zededa.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2744e25 to
c900dfb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Per-job coverage collection for the Eden test matrix plus a fan-in
merge job, intended to be consumed by a coverage-instrumented EVE
build on the lf-edge/eve side.
Each suite in the Eden test matrix expands into ~12 independent
runners with their own
./eden setup -> start -> stop -> make cleanlifecycle, so per-job collection is the only point at which to
capture covdata. After
make cleanthe runner is wiped — there is nolater opportunity.
Two commits:
collect-coverage: preserve raw covdata in outputCollectEveCoverageSCP'd the rawcovmeta.*/covcounters.*files into a temp dir, ran
go tool covdata textfmton them, andlet the deferred cleanup wipe the binary files on return. Only the
text profile at
<output-dir>/eden_e2e_coverage.txtsurvived.Callers that want to merge coverage across multiple runs (e.g.
across the Eden test matrix) need the raw covdata so they can use
go tool covdata merge; the text profile alone only supportsconcatenation, which is fine for a per-line summary but doesn't
aggregate the way merge does.
Copy the SCP target into
<output-dir>/covdata/instead of anauto-deleted temp dir. The text-profile output path is unchanged.
ci: collect eden e2e coverage per matrix jobrun-eden-testcomposite action gains a Collect/Upload coveragepair, gated as best-effort (
if: always()and|| true), thatinvokes
./eden eve collect-coverageand uploads the resultingeden_coverage/directory (text profile + raw covdata subtree) asartifact
covdata-<suite>-fs-<fs>-tpm-<tpm>-attempt-<n>.test.ymlgains acoverage-mergejob that downloads everycovdata-*artifact, runsgo tool covdata mergeon the rawcovmeta.* / covcounters.* files to produce a single merged covdata
directory, converts it to a text profile via
covdata textfmt,and runs
go tool cover -funcfor a per-package summary.Companion PR
This change is the prerequisite for
lf-edge/eve#5950,
which adds an opt-in
COVER=yEVE build (gated on acoveragePRlabel) and wires
eden-trusted.ymlto consume the resultingartifact. The EVE PR calls
lf-edge/eden/.github/workflows/test.yml@master, so this PR has toland first; otherwise the
tests-master-coverjob on the EVE sidewill run tests but emit no covdata-merged artifact.
How to test and validate this PR
Without a coverage-instrumented EVE artifact, the new steps run as
no-ops:
eden.yml(the eden-only CI path that consumeslfedge/eve:16.6.0) still passes after this change. TheCollect coveragestep warns ("no covcounters appeared")because zedbox isn't built with
-cover, and theUpload coverage artifactstep skips because theeden_e2e_coverage.txthashFiles check is false. No new artifactis uploaded.
coverage-mergefan-in job runs to completion andemits the "No raw covdata found across jobs" warning instead of
failing.
COVER=yand run a single suite viamake eden-cover(or by passing--coverage-dir <dir>toeden test); confirm<dir>/eden_e2e_coverage.txtAND<dir>/covdata/coverage/(containing
covmeta.*andcovcounters.*) both exist afterthe run.
master, exercise the fullpath end-to-end by labelling a coverage-instrumented EVE PR with
coverageand confirming thecovdata-mergedartifact isproduced at the end of the workflow run.
Notes
No user-facing changes. CI / dev-tooling only.