Skip to content

I can configure an exponential backoff supplier for the Job Worker #813

@LennartKleymann

Description

@LennartKleymann

Is your feature request related to a problem? Please describe.

When the gateway returns a RESOURCE_EXHAUSTED error, the worker repeatedly retries to activate jobs without any delay. This "thundering herd" behavior overloads the gateway and worsens the resource exhaustion issue.

Describe the solution you'd like

Implement a configurable exponential backoff strategy for the job worker, specifically for RESOURCE_EXHAUSTED errors. The backoff should reset after a successful activation.

The configuration should include:

  • backoffFactor
  • minDelay
  • maxDelay
  • jitter (optional)

Describe alternatives you've considered
Keeping the current fixed retry mechanism is insufficient as it does not address the core problem of overwhelming the gateway under load.

Additional context
This feature aligns with best practices for resilient systems and is already present in other clients, such as the Java client.
https://docs.camunda.io/docs/apis-tools/java-client/job-worker/#backoff-configuration
https://github.com/camunda/camunda/blob/b9165e9759143e80e7e3bd2a884837cf141276a1/clients/java/src/main/java/io/camunda/zeebe/client/api/worker/BackoffSupplier.java

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions