ci: COVER=y build/test path via coverage label#5950
Draft
eriknordmark wants to merge 1 commit into
Draft
Conversation
Wire end-to-end opt-in coverage collection for Eden runs against EVE. Adding the `coverage` PR label causes a parallel COVER=y EVE image build threaded through PR Gate and the Eden trusted workflow into a second Eden run, which consumes the instrumented artifact and emits the covdata-merged artifact via eden's coverage-merge job. build-eve composite action: a new `cover` input rebuilds pillar with COVER=y then builds the eve rootfs with the instrumented pillar package. The cover artifact is tagged `evebuild/pr-cover:<pr_id>` and uploaded as `eve-cover-<hv>-<arch>-<platform>` so it does not collide with the non-cover artifact on the runner or in docker. build.yml: `eve-amd64-kvm-generic-cover` runs in parallel with the existing amd64-kvm-generic build, gated on `contains(labels, 'coverage')`. Non-coverage PRs see this job as skipped at workflow start. pr-gate.yml: Check Build Result fetches all jobs from the PR build run once and holds the gate while the cover job is in a non-terminal status. The pull_request_review trigger fires on review submission regardless of PR build progress, so on a coverage-labeled PR the cover build can still be in flight at that moment; without the wait, the non-cover build's success would let the gate pass and eden-trusted would start tests-master-cover before the eve-cover-* artifact was ready. Cover-build failure is a soft signal: build_ok is decided by the non-cover conclusion, so a failed cover build does not block the PR -- tests-master-cover will surface the failure loudly by failing to download the artifact. eden-trusted.yml: detect the `coverage` label via `gh api` (JSON mode bypasses the classic-Projects GraphQL deprecation that breaks `gh pr view` on lf-edge repos), thread it through `has_coverage` on the context job's outputs, and add a `tests-master-cover` job that runs alongside `tests-master`. The cover job consumes the eve-cover-* artifact and the evebuild/pr-cover tag. The cover path is master-only for now: stable-branch eden tags (1.0.15) predate eden's `eden eve collect-coverage` command, so adding tests-<rel>-cover jobs would have no effect until that eden change is backported to a new tag. Signed-off-by: eriknordmark <erik@zededa.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Wire end-to-end opt-in coverage collection for Eden runs against EVE.
Adding the
coveragePR label causes a parallelCOVER=yEVE imagebuild threaded through PR Gate and the Eden trusted workflow into a
second Eden run, which consumes the instrumented artifact and emits
the
covdata-mergedartifact via eden'scoverage-mergejob.Four pieces:
.github/actions/build-eve/action.yml— newcoverinputrebuilds pillar with
COVER=ythen builds the eve rootfs with theinstrumented pillar package. The cover artifact is tagged
evebuild/pr-cover:<pr_id>and uploaded aseve-cover-<hv>-<arch>-<platform>so it does not collide with thenon-cover artifact on the runner or in docker.
.github/workflows/build.yml—eve-amd64-kvm-generic-coverruns in parallel with the existing
amd64-kvm-genericbuild,gated on
contains(labels, 'coverage'). Non-coverage PRs see thisjob as skipped at workflow start.
.github/workflows/pr-gate.yml— Check Build Result fetchesall jobs from the PR build run once and holds the gate while the
cover job is in a non-terminal status. The
pull_request_reviewtrigger fires on review submission regardless of PR build progress,
so on a coverage-labeled PR the cover build can still be in flight
at that moment; without the wait, the non-cover build's success
would let the gate pass and eden-trusted would start
tests-master-coverbefore theeve-cover-*artifact was ready.Cover-build failure is a soft signal:
build_okis decided by thenon-cover conclusion, so a failed cover build does not block the
PR —
tests-master-coverwill surface the failure loudly byfailing to download the artifact.
.github/workflows/eden-trusted.yml— detect thecoveragelabel via
gh api, thread it throughhas_coverageon the contextjob's outputs, and add a
tests-master-coverjob that runsalongside
tests-masterand consumes theeve-cover-*artifactplus the
evebuild/pr-covertag.The cover path is master-only for now: stable-branch eden tags
(
1.0.15) predate eden'seden eve collect-coveragecommand, soadding
tests-<rel>-coverjobs would have no effect until that edenchange is backported to a new tag.
PR dependencies
Depends on lf-edge/eden#1173 (
ci-coverage-collectionbranch oneriknordmark/eden):collect-coverage: preserve raw covdata in output— exposes theraw
covmeta.*/covcounters.*files in<output-dir>/covdata/alongside the existing text profile, so
go tool covdata mergecanbe used across runs.
ci: collect eden e2e coverage per matrix job— adds abest-effort
Collect coverage/Upload coverage artifactpairto
run-eden-test, and acoverage-mergefan-in job totest.ymlthat produces thecovdata-mergedartifact this PRexpects at the end of each Eden run.
This PR's
tests-master-coverjob calls intolf-edge/eden/.github/workflows/test.yml@master, so the eden changemust be merged first; otherwise the cover job will succeed at running
tests but emit no covdata artifact.
How to test and validate this PR
lf-edge/evewith thecoveragelabelset, against
master.eve (amd64, kvm, generic)andeve (amd64, kvm, generic, cover)jobs run and upload
eve-kvm-amd64-genericandeve-cover-kvm-amd64-genericartifacts respectively.after both build jobs have reached a terminal state, even if a
review is submitted while the cover build is still in flight.
tests-masterandtests-master-coverrun, and that the cover variant'scoverage-mergejob produces acovdata-mergedartifactcontaining
combined_coverage.txtandcoverage-summary.txt.coveragelabel andconfirm:
eve-amd64-kvm-generic-coverjob is skipped at workflowstart (no extra build cost).
cover_status == "completed"(skipped is a terminal state) and falls through to the existing
non-cover gate logic with no behavior change.
tests-master-coveris gated off byhas_coverage == 'false'and does not run.
Changelog notes
No user-facing changes. Internal CI only.
PR Backports
eden tag pinned by
eden-trusted.ymlfor 16.0-stable.Coverage on stable branches can be enabled later by backporting the
eden change to a new
1.0.xtag and then mirroring this PR withtests-<rel>-coverjobs pinned at the new tag.Checklist
for the initial cover matrix entry)