Skip to content

SRE-3534 ci: adapt CI to use el9.7 for Build and Unit tests #17376

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
grom72 wants to merge 60 commits into from
+61 −55
Closed

SRE-3534 ci: adapt CI to use el9.7 for Build and Unit tests #17376

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
900350b
SRE-3522 ci: test CI with el9.7
grom72 Jan 14, 2026
8590393
Trigger FT skip NLT
grom72 Jan 15, 2026
1750cc8
Force 9.7 via Jenkins file
grom72 Jan 15, 2026
0cabf7b
Switch to EL9
grom72 Jan 15, 2026
eded0e1
Use DISTRO provided by pipeline-lib
grom72 Jan 15, 2026
afd3f01
Merge remote-tracking branch 'origin/master' into grom72/SRE-3522
grom72 Jan 16, 2026
fea5137
Trigger HW FT on el9.7
grom72 Jan 16, 2026
f02ea77
Trigger UT and FT on el9
grom72 Jan 16, 2026
a0e4556
Trigger HW FT only
grom72 Jan 16, 2026
3f8c7c2
Trigger only HW FT
grom72 Jan 16, 2026
6100e53
Trigger full build
grom72 Jan 21, 2026
a644496
SRE_3459 New explicit parameter: 'image_version'
osalyk Nov 25, 2025
c4ca941
update files after code review
osalyk Jan 29, 2026
17f8d94
fix Jenkinsfile
osalyk Jan 29, 2026
77a96eb
update Jenkinsfile
osalyk Jan 30, 2026
8855d3a
try new branch from pipeline-lib
osalyk Jan 30, 2026
3ffdc55
improve NLT test
osalyk Jan 30, 2026
d796541
add changes to Jenkinsfile
osalyk Jan 30, 2026
aea39df
Trigger Valgtind build on EL9.7
grom72 Feb 2, 2026
4afb096
Trigger Valgrinf build on EL9.7
grom72 Feb 2, 2026
2f85ada
Test requires more than 1700 s on el 9.7
grom72 Feb 3, 2026
8c6955f
Merge remote-tracking branch 'origin/master' into grom72/SRE-3522
grom72 Feb 3, 2026
50c4c0f
add some changes to Jenkinsfile
osalyk Feb 4, 2026
d4871a1
Remove OS versions from stage names
grom72 Feb 4, 2026
bb4c509
Revert "Remove OS versions from stage names"
grom72 Feb 4, 2026
b5725a0
Address review comments
grom72 Feb 4, 2026
987c12c
updating stage names
osalyk Feb 4, 2026
bd05432
Normalize distro name by removing minor version number
grom72 Feb 5, 2026
c44a30b
SRE-3466 test: use image_version in Functional Hardware Medium Verbs
osalyk Feb 5, 2026
7ec12f3
Simplify stages names
grom72 Feb 5, 2026
5e787be
Merge remote-tracking branch 'origin/master' into osalyk/SRE-3459
osalyk Feb 9, 2026
6c8e73a
remove clamav-lib because DAOS-18358 is resolved
osalyk Feb 9, 2026
2fe3068
Harmonize Jenkins file with the latest pipeline-lib
grom72 Feb 9, 2026
5071cef
Merge remote-tracking branch 'origin/master' into grom72/SRE-3522
grom72 Feb 9, 2026
9eaa0e9
Merge remote-tracking branch 'origin/master' into osalyk/SRE-3459
osalyk Feb 10, 2026
16fb638
Merge remote-tracking branch 'origin/master' into osalyk/SRE-3459
osalyk Feb 13, 2026
329073b
Merge remote-tracking branch 'origin/master' into osalyk/SRE-3459
osalyk Feb 17, 2026
4115f7b
Merge remote-tracking branch 'origin/master' into grom72/SRE-3522
grom72 Feb 19, 2026
1bc16af
All changes are available in pipeline-lib master
grom72 Feb 19, 2026
fb619ee
Limit changes to only these stages that don't fail
grom72 Feb 20, 2026
81699d4
Merge remote-tracking branch 'origin/master' into grom72/SRE-3522
grom72 Feb 20, 2026
36e8b62
Limit changes to only these stages that don't fail (2nd)
grom72 Feb 20, 2026
167b00b
Try to restore original test timeout
grom72 Feb 24, 2026
902f1f5
Fault injection testing on EL 9 should work
grom72 Feb 25, 2026
98e278d
revert changes on NLT tests
osalyk Feb 25, 2026
c87638d
Merge remote-tracking branch 'origin/master' into osalyk/SRE-3459
osalyk Feb 25, 2026
8398642
improve stashes
osalyk Feb 25, 2026
c588357
SRE-3534 ci: Upgrate NLT based tests to el9.7
grom72 Feb 25, 2026
7b8215b
Revert "Fault injection testing on EL 9 should work"
grom72 Feb 26, 2026
3230306
Merge remote-tracking branch 'origin/grom72/SRE-3522' into grom72/SRE…
grom72 Mar 2, 2026
03ad78e
Merge remote-tracking branch 'origin/grom72/SRE-3534-NLT' into grom72…
grom72 Mar 2, 2026
63c90c4
Merge remote-tracking branch 'origin/osalyk/SRE-3459' into grom72/SRE…
grom72 Mar 2, 2026
51fc906
Draft
grom72 Mar 2, 2026
3128349
SRE-3534 ci: adapt CI to use el9.7 for Build and UT
grom72 Mar 3, 2026
f3935bb
Trigger Functional Tests with new parameters values
grom72 Mar 3, 2026
cfb7f11
SRE-3534 ci: test Functional Test on EL9
grom72 Mar 3, 2026
ee14072
Fix Fault injection logs location
grom72 Mar 3, 2026
f106467
For pipeline-lib test purpose only
grom72 Mar 3, 2026
e06e3b1
Revert "For pipeline-lib test purpose only"
grom72 Mar 3, 2026
e603a7b
CI_MORE_FUNCTIONAL_PR_TESTS is obsolete
grom72 Mar 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 53 additions & 47 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Map nlt_test() {
print 'Unstash failed, results from NLT stage will not be included'
}
sh label: 'Fault injection testing using NLT',
script: './ci/docker_nlt.sh --class-name el8.fault-injection fi'
script: './ci/docker_nlt.sh --class-name el9.fault-injection fi'
List filesList = []
filesList.addAll(findFiles(glob: '*.memcheck.xml'))
int vgfail = 0
Expand Down Expand Up @@ -317,19 +317,19 @@ pipeline {
description: 'Continue testing if a previous stage is Unstable')
booleanParam(name: 'CI_UNIT_TEST',
defaultValue: true,
description: 'Run the Unit Test on EL 8 test stage')
description: 'Run the Unit Test test stage')
booleanParam(name: 'CI_NLT_TEST',
defaultValue: true,
description: 'Run the NLT test stage')
booleanParam(name: 'CI_UNIT_TEST_MEMCHECK',
defaultValue: true,
description: 'Run the Unit Test with memcheck on EL 8 test stage')
booleanParam(name: 'CI_FI_el8_TEST',
description: 'Run the Unit Test with memcheck test stage')
booleanParam(name: 'CI_FI_TEST',
defaultValue: true,
description: 'Run the Fault injection testing on EL 8 test stage')
booleanParam(name: 'CI_TEST_EL8_RPMs',
description: 'Run the Fault injection testing test stage')
booleanParam(name: 'CI_TEST_EL_RPMs',
defaultValue: true,
description: 'Run the Test RPMs on EL 8 test stage')
description: 'Run the Test RPMs on EL stage')
booleanParam(name: 'CI_TEST_LEAP15_RPMs',
defaultValue: true,
description: 'Run the Test RPMs on Leap 15 test stage')
Expand All @@ -343,19 +343,17 @@ pipeline {
defaultValue: false,
description: 'Run the Functional on EL 8 with Valgrind test stage')
booleanParam(name: 'CI_FUNCTIONAL_el8_TEST',
defaultValue: true,
defaultValue: false,
description: 'Run the Functional on EL 8 test stage')
booleanParam(name: 'CI_FUNCTIONAL_el9_TEST',
defaultValue: false,
defaultValue: true,
description: 'Run the Functional on EL 9 test stage')
booleanParam(name: 'CI_FUNCTIONAL_leap15_TEST',
defaultValue: false,
description: 'Run the Functional on Leap 15 test stage' +
' Requires CI_MORE_FUNCTIONAL_PR_TESTS')
description: 'Run the Functional on Leap 15 test stage')
booleanParam(name: 'CI_FUNCTIONAL_ubuntu20_TEST',
defaultValue: false,
description: 'Run the Functional on Ubuntu 20.04 test stage' +
' Requires CI_MORE_FUNCTIONAL_PR_TESTS')
description: 'Run the Functional on Ubuntu 20.04 test stage')
booleanParam(name: 'CI_FUNCTIONAL_HARDWARE_TEST_SKIP',
defaultValue: false,
description: 'Skip Functional Hardware (Test Hardware) stage')
Expand Down Expand Up @@ -529,7 +527,7 @@ pipeline {
expression { !skip_build_stage() }
}
parallel {
stage('Build on EL 8.8') {
stage('Build on EL 8') {
when {
beforeAgent true
expression { !skip_build_stage('el8') }
Expand All @@ -544,7 +542,8 @@ pipeline {
" -t ${sanitized_JOB_NAME()}-el8 " +
' --build-arg DAOS_PACKAGES_BUILD=no ' +
' --build-arg DAOS_KEEP_SRC=yes ' +
' --build-arg REPOS="' + prRepos() + '"'
' --build-arg REPOS="' + prRepos() + '"' +
' --build-arg POINT_RELEASE=.10 '
}
}
steps {
Expand Down Expand Up @@ -581,7 +580,7 @@ pipeline {
}
}
}
stage('Build on EL 9.6') {
stage('Build on EL 9') {
when {
beforeAgent true
expression { !skip_build_stage('el9') }
Expand All @@ -597,7 +596,7 @@ pipeline {
' --build-arg DAOS_PACKAGES_BUILD=no ' +
' --build-arg DAOS_KEEP_SRC=yes ' +
' --build-arg REPOS="' + prRepos() + '"' +
' --build-arg POINT_RELEASE=.6 '
' --build-arg POINT_RELEASE=.7 '
}
}
steps {
Expand Down Expand Up @@ -634,7 +633,7 @@ pipeline {
}
}
}
stage('Build on Leap 15.5') {
stage('Build on Leap 15') {
when {
beforeAgent true
expression { !skip_build_stage('leap15') }
Expand Down Expand Up @@ -684,7 +683,7 @@ pipeline {
}
}
}
stage('Build on Leap 15.5 with Intel-C and TARGET_PREFIX') {
stage('Build on Leap 15 with Intel-C and TARGET_PREFIX') {
when {
beforeAgent true
expression { !skip_build_stage('leap15', 'icc') }
Expand Down Expand Up @@ -730,7 +729,7 @@ pipeline {
expression { !skipStage() }
}
parallel {
stage('Unit Test on EL 8.8') {
stage('Unit Test') {
when {
beforeAgent true
expression { !skipStage() }
Expand All @@ -739,11 +738,14 @@ pipeline {
label cachedCommitPragma(pragma: 'VM1-label', def_val: params.CI_UNIT_VM1_LABEL)
}
steps {
job_step_update(
unitTest(timeout_time: 60,
unstash_opt: true,
inst_repos: daosRepos(),
inst_rpms: unitPackages()))
job_step_update(
unitTest(timeout_time: 60,
unstash_opt: true,
inst_repos: daosRepos(),
inst_rpms: unitPackages(target: 'el9'),
image_version: 'el9.7',
)
)
}
post {
always {
Expand All @@ -752,7 +754,7 @@ pipeline {
}
}
}
stage('Unit Test bdev on EL 8.8') {
stage('Unit Test bdev') {
when {
beforeAgent true
expression { !skipStage() }
Expand All @@ -765,7 +767,8 @@ pipeline {
unitTest(timeout_time: 60,
unstash_opt: true,
inst_repos: daosRepos(),
inst_rpms: unitPackages()))
inst_rpms: unitPackages(target: 'el9'),
image_version: 'el9.7'))
}
post {
always {
Expand All @@ -774,7 +777,7 @@ pipeline {
}
}
}
stage('NLT on EL 8.8') {
stage('NLT') {
when {
beforeAgent true
expression { params.CI_NLT_TEST && !skipStage() }
Expand All @@ -789,7 +792,8 @@ pipeline {
test_script: 'ci/unit/test_nlt.sh',
unstash_opt: true,
unstash_tests: false,
inst_rpms: unitPackages()))
inst_rpms: unitPackages(target: 'el9'),
image_version: 'el9.7'))
// recordCoverage(tools: [[parser: 'COBERTURA', pattern:'nltir.xml']],
// skipPublishingChecks: true,
// id: 'tlc', name: 'Fault Injection Interim Report')
Expand All @@ -800,7 +804,7 @@ pipeline {
unitTestPost artifacts: ['nlt_logs/'],
testResults: 'nlt-junit.xml',
always_script: 'ci/unit/test_nlt_post.sh',
valgrind_stash: 'el8-gcc-nlt-memcheck'
valgrind_stash: 'nlt-memcheck'
recordIssues enabledForFailure: true,
failOnError: false,
ignoreQualityGate: true,
Expand All @@ -814,7 +818,7 @@ pipeline {
}
}
}
stage('Unit Test with memcheck on EL 8.8') {
stage('Unit Test with memcheck') {
when {
beforeAgent true
expression { !skipStage() }
Expand All @@ -828,18 +832,19 @@ pipeline {
unstash_opt: true,
ignore_failure: true,
inst_repos: daosRepos(),
inst_rpms: unitPackages()))
inst_rpms: unitPackages(target: 'el9'),
image_version: 'el9.7'))
}
post {
always {
unitTestPost artifacts: ['unit_test_memcheck_logs.tar.gz',
'unit_test_memcheck_logs/**/*.log'],
valgrind_stash: 'el8-gcc-unit-memcheck'
valgrind_stash: 'unit-memcheck'
job_status_update()
}
}
} // stage('Unit Test with memcheck on EL 8.8')
stage('Unit Test bdev with memcheck on EL 8.8') {
} // stage('Unit Test with memcheck')
stage('Unit Test bdev with memcheck') {
when {
beforeAgent true
expression { !skipStage() }
Expand All @@ -853,17 +858,18 @@ pipeline {
unstash_opt: true,
ignore_failure: true,
inst_repos: daosRepos(),
inst_rpms: unitPackages()))
inst_rpms: unitPackages(target: 'el9'),
image_version: 'el9.7'))
}
post {
always {
unitTestPost artifacts: ['unit_test_memcheck_bdev_logs.tar.gz',
'unit_test_memcheck_bdev_logs/**/*.log'],
valgrind_stash: 'el8-gcc-unit-memcheck-bdev'
valgrind_stash: 'unit-bdev-memcheck'
job_status_update()
}
}
} // stage('Unit Test bdev with memcheck on EL 8')
} // stage('Unit Test bdev with memcheck')
}
}
stage('Test') {
Expand Down Expand Up @@ -990,14 +996,14 @@ pipeline {
}
} // post
} // stage('Functional on Ubuntu 20.04')
stage('Fault injection testing on EL 8.8') {
stage('Fault injection testing on EL 9.7') {
when {
beforeAgent true
expression { !skipStage() }
}
agent {
dockerfile {
filename 'utils/docker/Dockerfile.el.8'
filename 'utils/docker/Dockerfile.el.9'
label 'docker_runner'
additionalBuildArgs dockerBuildArgs(repo_type: 'stable',
parallel_build: true,
Expand Down Expand Up @@ -1040,16 +1046,16 @@ pipeline {
stash name: 'fault-inject-valgrind',
includes: '*.memcheck.xml',
allowEmpty: true
archiveArtifacts artifacts: 'nlt_logs/el8.fault-injection/',
archiveArtifacts artifacts: 'nlt_logs/el9.fault-injection/',
allowEmptyArchive: true
job_status_update()
}
}
} // stage('Fault injection testing on EL 8.8')
stage('Test RPMs on EL 8.6') {
} // stage('Fault injection testing on EL 9.7')
stage('Test RPMs on EL 9.6') {
when {
beforeAgent true
expression { params.CI_TEST_EL8_RPMs && !skipStage() }
expression { !skipStage() }
}
agent {
label params.CI_UNIT_VM1_LABEL
Expand All @@ -1066,7 +1072,7 @@ pipeline {
rpm_test_post(env.STAGE_NAME, env.NODELIST)
}
}
} // stage('Test RPMs on EL 8.6')
} // stage('Test RPMs on EL 9.6')
stage('Test RPMs on Leap 15.5') {
when {
beforeAgent true
Expand Down Expand Up @@ -1259,8 +1265,8 @@ pipeline {
} // stages
post {
always {
valgrindReportPublish valgrind_stashes: ['el8-gcc-nlt-memcheck',
'el8-gcc-unit-memcheck',
valgrindReportPublish valgrind_stashes: ['nlt-memcheck',
'unit-memcheck',
'fault-inject-valgrind']
job_status_update('final_status')
jobStatusWrite(job_status_internal)
Expand Down
4 changes: 3 additions & 1 deletion ci/provisioning/post_provision_config_common_functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ retry_dnf() {
if command -v dnf4; then
dnfx="dnf4"
fi
"$dnfx" config-manager --disable 'epel*' || true
if "$dnfx" repolist | grep -q '^epel'; then
"$dnfx" config-manager --disable 'epel*'
fi
fi
return 0
fi
Expand Down
3 changes: 0 additions & 3 deletions ci/provisioning/post_provision_config_nodes_EL.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,5 @@ group_repo_post() {
distro_custom() {
# TODO: This code is not exiting on failure.

# Use a more recent python version for unit testing, this allows us to also test installing
# pydaos into virtual environments.
dnf -y install python39 python39-devel
dnf -y install python3.11 python3.11-devel
}
9 changes: 5 additions & 4 deletions ci/unit/required_packages.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,17 @@
#
set -eu

# No longer used but provided by pipeline-lib
# Provided by pipeline-lib
# distro="$1"
# quick_build="${2:-false}"

OPENMPI_VER=""
PY_MINOR_VER=""

export DISTRO="el8" # should also work for el9
pkgs="$(utils/rpms/package_version.sh argobots lib) \
boost-python3$PY_MINOR_VER-devel \
DISTRO="${1:?ERROR: Missing distro argument. Usage: $0 <distro>}"
export DISTRO="${DISTRO%%.*}"

pkgs="boost-python3$PY_MINOR_VER-devel \
capstone \
$(utils/rpms/package_version.sh argobots lib) \
$(utils/rpms/package_version.sh argobots debug) \
Expand Down
Loading