Merge pull request #13786 from valentijnscholten/jira-rate-limiting

JIRA: add retry/rate limit support

Author: rossops

docs/content/en/share_your_findings/troubleshooting_jira.md

@@ -101,6 +101,34 @@ To correct this issue, you can add the 'Epic Name' field to your Project's issue
 
 ![image](images/epic_name_error.png)
 
+## Configuring JIRA Connection Retries and Timeouts
+
+DefectDojo's JIRA integration includes configurable retry and timeout settings to handle rate limiting and connection issues. These settings are important for maintaining system responsiveness, especially when using Celery workers.
+
+### Available Configuration Variables
+
+The following environment variables control JIRA connection behavior:
+
+- **`DD_JIRA_MAX_RETRIES`** (default: `3`): Maximum number of retry attempts for recoverable errors. The integration will automatically retry on HTTP 429 (Too Many Requests), HTTP 503 (Service Unavailable), and connection errors. See the [JIRA rate limiting documentation](https://developer.atlassian.com/cloud/jira/platform/rate-limiting/) for more information.
+
+- **`DD_JIRA_CONNECT_TIMEOUT`** (default: `10` seconds): Connection timeout for establishing a connection to the JIRA server.
+
+- **`DD_JIRA_READ_TIMEOUT`** (default: `30` seconds): Read timeout for waiting for a response from the JIRA server after the connection is established.
+
+**Note on Rate Limiting**: The jira library has a built-in maximum wait time of 60 seconds for rate limiting retries. If JIRA's `Retry-After` header indicates a wait time longer than 60 seconds, the request will fail and not be retried. This is a limitation of the jira library version currently in use.
+
+### Why Conservative Values Matter
+
+**Important**: It is recommended to use conservative (lower) values for these settings. Here's why:
+
+1. **Celery Task Blocking**: JIRA operations in DefectDojo run as asynchronous Celery tasks. When a task is waiting for a retry delay, it blocks that Celery worker from processing other tasks.
+
+2. **Worker Pool Exhaustion**: If multiple JIRA operations are retrying with long delays, you can quickly exhaust your Celery worker pool, causing other tasks (not just JIRA-related) to queue up and wait.
+
+3. **System Responsiveness**: Long retry delays can make the system appear unresponsive, especially during JIRA outages or rate limiting events.
+
+JIRA Rate limiting is new, so please let us know on Slack or GitHub what works best for you.
+
 ## Jira and DefectDojo are out of sync
 
 Sometimes Jira is down, or DefectDojo is down, or there was bug in a webhook. In this case, Jira can become out of sync with DefectDojo. If this is the case for lots of issues, manual reconciliation might not be feasible. For this scenario there is the management command 'jira_status_reconciliation'.

dojo/jira_link/helper.py

@@ -433,14 +433,19 @@ def has_jira_configured(obj):
 
 
 def connect_to_jira(jira_server, jira_username, jira_password):
+    max_retries = getattr(settings, "JIRA_MAX_RETRIES", 3)
+    timeout = getattr(settings, "JIRA_TIMEOUT", (10, 30))
+
     return JIRA(
         server=jira_server,
         basic_auth=(jira_username, jira_password),
-        max_retries=0,
+        max_retries=max_retries,
+        timeout=timeout,
         options={
             "verify": settings.JIRA_SSL_VERIFY,
             "headers": settings.ADDITIONAL_HEADERS,
-        })
+        },
+    )
 
 
 def get_jira_connect_method():

dojo/settings/settings.dist.py

@@ -249,6 +249,16 @@
     # When interacting with jira tickets that attached finding groups, we should no be opening any findings
     # on the DefectDojo side because jira has no way of knowing if a finding really should be reopened or not
     DD_JIRA_WEBHOOK_ALLOW_FINDING_GROUP_REOPEN=(bool, False),
+    # JIRA connection retry and timeout settings: https://developer.atlassian.com/cloud/jira/platform/rate-limiting/
+    # Maximum number of retry attempts for recoverable errors (429, 503, ConnectionError)
+    # See https://jira.readthedocs.io/ for more in the jira library used by DefectDojo
+    # Note: The jira library has a built-in maximum wait time of 60s for rate limiting retries.
+    # If JIRA's Retry-After header indicates a wait time longer than 60s, the request will fail and not be retried.
+    DD_JIRA_MAX_RETRIES=(int, 3),
+    # Connection timeout (seconds) for establishing a connection to the JIRA server
+    DD_JIRA_CONNECT_TIMEOUT=(int, 10),
+    # Read timeout (seconds) for waiting for a response from the JIRA server
+    DD_JIRA_READ_TIMEOUT=(int, 30),
     # You can set extra Jira issue types via a simple env var that supports a csv format, like "Work Item,Vulnerability"
     DD_JIRA_EXTRA_ISSUE_TYPES=(str, ""),
     # if you want to keep logging to the console but in json format, change this here to 'json_console'
@@ -1714,6 +1724,12 @@ def saml2_attrib_map_format(din):
 JIRA_SSL_VERIFY = env("DD_JIRA_SSL_VERIFY")
 JIRA_DESCRIPTION_MAX_LENGTH = env("DD_JIRA_DESCRIPTION_MAX_LENGTH")
 JIRA_WEBHOOK_ALLOW_FINDING_GROUP_REOPEN = env("DD_JIRA_WEBHOOK_ALLOW_FINDING_GROUP_REOPEN")
+# JIRA connection retry and timeout settings
+JIRA_MAX_RETRIES = env("DD_JIRA_MAX_RETRIES")
+JIRA_CONNECT_TIMEOUT = env("DD_JIRA_CONNECT_TIMEOUT")
+JIRA_READ_TIMEOUT = env("DD_JIRA_READ_TIMEOUT")
+# Combine timeouts into a tuple for the JIRA library: (connect_timeout, read_timeout)
+JIRA_TIMEOUT = (JIRA_CONNECT_TIMEOUT, JIRA_READ_TIMEOUT)
 
 # ------------------------------------------------------------------------------
 # LOGGING