Skip to content

feat(jenkins): parse test_metadata from build description#1012

Open
fruch wants to merge 3 commits into
scylladb:masterfrom
fruch:sct-test-metadata
Open

feat(jenkins): parse test_metadata from build description#1012
fruch wants to merge 3 commits into
scylladb:masterfrom
fruch:sct-test-metadata

Conversation

@fruch
Copy link
Copy Markdown
Collaborator

@fruch fruch commented May 28, 2026

Summary

  • Adds parse_test_metadata_from_description() utility that extracts SCT pipeline metadata (tier, test_type, duration_class, supported_backends) from Jenkins job descriptions
  • Supports two formats: job description (markdown ### TestMetadata block) and build description (HTML <b>Metadata:</b> block)
  • Adds JenkinsService.get_build_test_metadata() method to fetch and parse metadata for a given job/build
  • Includes unit tests for the parser with real-world examples

Context

SCT PR scylladb/scylla-cluster-tests#14818 adds test_metadata to test-case YAML files and writes it to the Jenkins job description during job creation. The format in the job description is:

test: longevity_test.LongevityTest.test_custom_time | backend: aws | region: eu-west-1 | config: test-cases/longevity/longevity-10gb-3h.yaml

Basic longevity test running cassandra-stress write workload at QUORUM consistency
for ~4 hours on a 6-node single-DC cluster with SisyphusMonkey nemesis. Validates
cluster stability under moderate write load with continuous chaos operations.

### TestMetadata
tier: tier1
test_type: longevity
duration_class: short
supported_backends: ['aws', 'gce', 'azure']

The parser extracts the ### TestMetadata block (key: value lines) from this description.

A secondary HTML format is also supported for backward compatibility with currentBuild.description:

<b>Metadata:</b> tier=tier1 | type=longevity | duration=short | backends=['aws', 'gce']

Scope

This PR only adds reading and parsing. Storage/persistence of the metadata (on ArgusTest, SCTTestRun, or elsewhere) is intentionally left for a design discussion with the team.

Discussion Points

  • Where should this metadata live? Options:
    • New columns on ArgusTest (job-level, updated during scan-jenkins)
    • New columns on SCTTestRun (per-run, extracted from sct_config)
    • Both (job-level defaults + per-run overrides)
  • Should JenkinsMonitor.collect() call get_build_test_metadata() and store it during scans?
  • Should the frontend expose filtering/grouping by tier/test_type?

Add parse_test_metadata_from_description() that extracts SCT pipeline
metadata (tier, test_type, duration_class, supported_backends) from
the HTML block written to Jenkins build descriptions by SCT's
createArgusTestRun.groovy (scylladb/scylla-cluster-tests#14818).

Also adds JenkinsService.get_build_test_metadata() to fetch and parse
metadata for a given job/build.

NOTE: This PR only adds reading and parsing. Storage/persistence of
the metadata on ArgusTest or SCTTestRun is left for a follow-up
design discussion with the team.
@fruch fruch requested review from k0machi and soyacz May 28, 2026 09:03
@fruch
Copy link
Copy Markdown
Collaborator Author

fruch commented May 28, 2026

This is the suggested "interface" for the test metadata, it's gonna be available on the job description

this PR is just ref to see where it should handled, it's not saving or doing anything with the metadata yet

…ata parsing

Add _JOB_METADATA_RE regex to parse ### TestMetadata blocks from Jenkins
job descriptions (set during job creation by SCT). The job description
format uses key: value lines instead of the HTML pipe-separated format.
Both formats are now supported with job description taking priority.

Added real-world test using longevity-10gb-3h pipeline output.
_METADATA_RE = re.compile(
r"<b>Metadata:</b>\s*"
r"tier=(?P<tier>[^\s|]+)\s*\|\s*"
r"type=(?P<test_type>[^\s|]+)\s*\|\s*"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks very fragile.
Why not using yaml/json/xml?

Add parse_test_metadata_from_config_xml() that reads the <testMetadata>
XML element from Jenkins job config.xml. This is the primary source of
truth — set at job creation, not editable via UI. The description parser
remains as fallback for older jobs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants