Skip to content

Likely concurrency issue pq: 'duplicate key value violates unique constraint \"UQE_pipelines_s\"' #3884

@fernandrone

Description

@fernandrone

Component

server

Describe the bug

We're running Woodpecker internally. We've noticed that a few builds per week (out of thousands) are not being triggered. Upon further investigation we saw those logs on the server:

{"level":"error","repo":"<redacted>","error":"pq: duplicate key value violates unique constraint \"UQE_pipelines_s\"","time":"2024-07-10T19:39:48Z","message":"failed to save pipeline for <redacted>"}
{"level":"error","error":"Error #01: failed to save pipeline for <redacted>\n","ip":"140.82.115.147","latency":19.702502,"method":"POST","path":"/api/hook","status":500,"time":"2024-07-10T19:39:48Z","user-agent":"GitHub-Hookshot/715fe7b","time":"2024-07-10T19:39:48Z"}

This happens randomly with multiple repositories and no particular pattern so I can't really reproduce it.

My assumption (without having really delved much into the problem or this particular section of the codebase) is that there's a concurrency issue where multiple concurrent triggers from different repositories may end up generating the same pipeline id somehow.

The volume is low enough that it has gone unnoticed for months.

Steps to reproduce

Not really reproducible, other than "generate a lot of builds and hope a conflict happens". I'm still not sure if using multiple workflows per repository (and/or triggering builds in multiple repositories vs multiple builds in the same repository) is part of the equation or not.

Expected behavior

No response

System Info

2.3.0

Kubernetes backend

Additional context

No response

Validations

  • Read the docs.
  • Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
  • Checked that the bug isn't fixed in the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingserver

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions