Skip to content

fix(release-0.2): retry on Bucket update conflict during BucketAccess reconcile#302

Open
myasnikovdaniil wants to merge 1 commit into
kubernetes-sigs:release-0.2from
myasnikovdaniil:bucketaccess-conflict-retry
Open

fix(release-0.2): retry on Bucket update conflict during BucketAccess reconcile#302
myasnikovdaniil wants to merge 1 commit into
kubernetes-sigs:release-0.2from
myasnikovdaniil:bucketaccess-conflict-retry

Conversation

@myasnikovdaniil
Copy link
Copy Markdown

What this PR does

Carries the same RetryOnConflict pattern adopted in #197 (BucketClaim controller) into the BucketAccess sidecar reconciler, where the same race exists on the parent Bucket finalizer-add path.

In sidecar/pkg/bucketaccess/bucketaccess_controller.go, the Add path does:

bucket, _ := buckets().Get(...)
// ...
if controllerutil.AddFinalizer(bucket, BABucketFinalizer) {
    buckets().Update(ctx, bucket, ...)   // <-- 409 here
}

Between Get and Update, the Bucket reconciler running in the same controller process can mutate the object and bump resourceVersion, producing:

Operation cannot be fulfilled on buckets.objectstorage.k8s.io "bucket-...":
the object has been modified; please apply your changes to the latest
version and try again

surfaced to users as a FailedGrantAccess event on the BucketAccess.

This PR wraps the Bucket finalizer add in retry.RetryOnConflict(retry.DefaultRetry, ...) and re-Gets the Bucket inside the closure so each retry mutates the freshest version. Same pattern as #197.

Why release-0.2 only

The same code path does not exist on main — the v1alpha2 BucketAccess sidecar (sidecar/pkg/reconciler/bucketaccess.go) only reads Buckets and only mutates the BucketAccess itself, so the cross-object Bucket finalizer race is gone architecturally.

Related

Release note

[sidecar] Retry on Bucket update conflict during BucketAccess reconcile to eliminate spurious FailedGrantAccess events caused by the in-process race between the Bucket and BucketAccess reconcilers.

Test plan

  • go build ./... and go vet ./... pass
  • go test ./sidecar/pkg/bucketaccess/... passes
  • Manual verification: the FailedGrantAccess: ... object has been modified event reproducible against v0.2.2 stops firing once the patch is applied.

… reconcile

The BucketAccess sidecar reconciler does a Get -> AddFinalizer -> Update
on the parent Bucket to attach BABucketFinalizer. When the Bucket
reconciler in the same controller process mutates the Bucket between
our Get and Update, the Update fails with:

  Operation cannot be fulfilled on buckets.objectstorage.k8s.io "<name>":
  the object has been modified; please apply your changes to the latest
  version and try again

surfaced to users as a FailedGrantAccess event on the BucketAccess.

Wrap the Bucket finalizer add in retry.RetryOnConflict and re-Get the
Bucket inside the closure so each retry mutates the freshest version,
matching the pattern adopted in kubernetes-sigs#197 for the BucketClaim controller.

Related to kubernetes-sigs#173.

Signed-off-by: Myasnikov Daniil <myasnikovdaniil2001@gmail.com>
@linux-foundation-easycla
Copy link
Copy Markdown

CLA Not Signed

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

Welcome @myasnikovdaniil!

It looks like this is your first PR to kubernetes-sigs/container-object-storage-interface 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/container-object-storage-interface has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Apr 30, 2026
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

Hi @myasnikovdaniil. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: myasnikovdaniil
Once this PR has been reviewed and has the lgtm label, please assign blaineexe for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Apr 30, 2026
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 30, 2026
@BlaineEXE
Copy link
Copy Markdown
Member

@myasnikovdaniil please take a look at the CLA requirements added by the bot

@BlaineEXE
Copy link
Copy Markdown
Member

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants