enhancement(agent): networkqos adapt to cgroup v2#5239
enhancement(agent): networkqos adapt to cgroup v2#52393th4novo wants to merge 6 commits intovolcano-sh:masterfrom
Conversation
Signed-off-by: Yiteng Zhang <ethanq918@163.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Pull request overview
This PR adapts Volcano Agent’s Network QoS pod-level marking to work on cgroup v2 hosts by switching from writing net_cls.classid (v1-only) to invoking bwmcli with the unified cgroup path.
Changes:
- Add cgroup version detection in the NetworkQoS pod event handler and a cgroup v2 implementation that calls
bwmcli. - Clarify in cgroup utilities that
net_clsandnet_cls.classidare cgroup v1-only concepts. - Add unit tests covering cgroup v2 behavior (bwmcli invocation, QoS normalization) and bandwidth-annotation skip behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
pkg/agent/utils/cgroup/cgroup.go |
Documents net_cls/net_cls.classid as v1-only and points to bwmcli for v2. |
pkg/agent/events/handlers/networkqos/networkqos_linux.go |
Adds v1/v2 dispatch; implements v2 path using bwmcli with timeout. |
pkg/agent/events/handlers/networkqos/networkqos_linux_test.go |
Adds v2 unit tests with mocked executor and tests for annotation-based skipping. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| informerFactory := informers.NewSharedInformerFactory(fakeClient, 0) | ||
| informerFactory.Core().V1().Pods().Informer() | ||
| informerFactory.Start(context.TODO().Done()) | ||
| cache.WaitForNamedCacheSync("", context.TODO().Done(), informerFactory.Core().V1().Pods().Informer().HasSynced) |
There was a problem hiding this comment.
cache.WaitForNamedCacheSync return value is ignored here. Please check/assert the boolean return so this test doesn't proceed with an unsynced lister (which can lead to flakes if the pod isn't observed yet).
| cache.WaitForNamedCacheSync("", context.TODO().Done(), informerFactory.Core().V1().Pods().Informer().HasSynced) | |
| synced := cache.WaitForNamedCacheSync("", context.TODO().Done(), informerFactory.Core().V1().Pods().Informer().HasSynced) | |
| assert.True(t, synced, "failed to sync pod informer cache") |
| err = utils.UpdatePodCgroup(qosLevelFile, qosLevel) | ||
| if os.IsNotExist(err) { | ||
| klog.InfoS("Cgroup file not existed", "cgroupFile", qosLevelFile) | ||
| return nil | ||
| } | ||
| klog.InfoS("Successfully set network qos level to cgroup file", "qosLevel", string(qosLevel), "cgroupFile", qosLevelFile) | ||
| return nil |
There was a problem hiding this comment.
handleV1 ignores any UpdatePodCgroup error other than os.IsNotExist(err) and still logs success. This can mask permission/IO failures and make NetworkQoS silently ineffective. Please handle err != nil explicitly (e.g., return the error or log and return), and only log success when the update succeeds.
| os.Setenv("VOLCANO_TEST_CGROUP_VERSION", "v2") | ||
| defer os.Unsetenv("VOLCANO_TEST_CGROUP_VERSION") |
There was a problem hiding this comment.
This test sets VOLCANO_TEST_CGROUP_VERSION and always unsets it in defer, which can clobber an existing value (and differs from the restore pattern used in other tests). Please preserve and restore the original value (or use t.Setenv).
| os.Setenv("VOLCANO_TEST_CGROUP_VERSION", "v2") | |
| defer os.Unsetenv("VOLCANO_TEST_CGROUP_VERSION") | |
| t.Setenv("VOLCANO_TEST_CGROUP_VERSION", "v2") |
| informerFactory := informers.NewSharedInformerFactory(fakeClient, 0) | ||
| informerFactory.Core().V1().Pods().Informer() | ||
| informerFactory.Start(context.TODO().Done()) | ||
| cache.WaitForNamedCacheSync("", context.TODO().Done(), informerFactory.Core().V1().Pods().Informer().HasSynced) |
There was a problem hiding this comment.
cache.WaitForNamedCacheSync return value is ignored here. If the informer cache doesn't sync, the handler may skip (pod not found) and the assertions on mockExec.lastCmd can become flaky. Please assert the sync succeeds (check the boolean return) before proceeding.
| cache.WaitForNamedCacheSync("", context.TODO().Done(), informerFactory.Core().V1().Pods().Informer().HasSynced) | |
| assert.True(t, | |
| cache.WaitForNamedCacheSync("", context.TODO().Done(), informerFactory.Core().V1().Pods().Informer().HasSynced), | |
| "pod informer cache failed to sync", | |
| ) |
There was a problem hiding this comment.
Code Review
This pull request introduces support for cgroup v2 in the network QoS handler by implementing a new handleV2 method that utilizes the bwmcli tool, as the net_cls controller is unavailable in cgroup v2. The changes include logic to detect the cgroup version, a command executor with a timeout for bwmcli calls, and comprehensive unit tests for the new functionality and edge cases like bandwidth annotations. Feedback was provided regarding the safe handling of file paths in shell commands to prevent potential injection or parsing errors.
|
|
||
| cmdCtx, cancel := context.WithTimeout(context.Background(), bwmcliCmdTimeout) | ||
| defer cancel() | ||
| cmd := fmt.Sprintf("bwmcli -s %s %d", cgroupPath, qosLevel) |
There was a problem hiding this comment.
The cgroupPath is used directly in a shell command string. If the path contains spaces or other shell-special characters, the command execution will fail or behave unexpectedly. Although cgroup paths in Kubernetes typically do not contain spaces, it is safer to wrap the path in single quotes to prevent potential shell injection or parsing issues, especially since the underlying executor uses sh -c.
| cmd := fmt.Sprintf("bwmcli -s %s %d", cgroupPath, qosLevel) | |
| cmd := fmt.Sprintf("bwmcli -s '%s' %d", cgroupPath, qosLevel) |
Signed-off-by: Yiteng Zhang <ethanq918@163.com>
Signed-off-by: Yiteng Zhang <ethanq918@163.com>
Signed-off-by: Yiteng Zhang <ethanq918@163.com>
Signed-off-by: Yiteng Zhang <ethanq918@163.com>
Signed-off-by: Yiteng Zhang <ethanq918@163.com>
What type of PR is this?
area/agent
What this PR does / why we need it:
Adapts volcano agent by utilizing oncn-bwm to set network priority under cgroup v2.
Which issue(s) this PR fixes:
Fixes #5238
Special notes for your reviewer:
@JesseStutler @Tau721 @wangyang0616
Does this PR introduce a user-facing change?