getBatch adding same job to several different batches #3097
christhegrand
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
We've been noticing in our logs recently that if getBatch is called concurrently by several different workers, the same job can be added to several different batches.
I would have expected the lock that BullMQ sets in Redis to prevent this from happening. Some of the jobs eventually fail with errors such as
Missing key for job a2a66ac9057404fb2737203d0d058ad08ad7fc360f80a4762205654c2be95338bd69c5b89603768dfc90c2acbf5724606db81d9dcba01046bf524f71691f116d. completedorLock mismatch for job a2a66ac9057404fb2737203d0d058ad08ad7fc360f80a4762205654c2be95338bd69c5b89603768dfc90c2acbf5724606db81d9dcba01046bf524f71691f116d. Cmd completed from active, but we are still seeing the same jobs getting processed more than once. This is not caused by stalling, which I know the Bull documentation mentions can lead to double processing.Is this the intended behavior? If so, is there a workaround for this case to prevent several different workers from processing the same jobs?
I am seeing this in BullMQ Pro 7.17.2, which I know is a bit out of date now, but I do not see anything that looks like it might fix this issue in later releases in the CHANGELOG.
Beta Was this translation helpful? Give feedback.
All reactions