-
Notifications
You must be signed in to change notification settings - Fork 15
Add debug-github-ci and debug-jenkins-ci skills #76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
neubig
wants to merge
3
commits into
main
Choose a base branch
from
add-ci-debug-skills
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,165 @@ | ||
| # Debug GitHub CI | ||
|
|
||
| Automated debugging of GitHub Actions CI failures using OpenHands agents. | ||
|
|
||
| ## Overview | ||
|
|
||
| This extension provides **dual-mode functionality**: | ||
|
|
||
| 1. **Skill Mode**: A knowledge skill (`SKILL.md`) that provides guidance and context for debugging CI failures interactively via OpenHands conversations. | ||
|
|
||
| 2. **Plugin Mode**: An executable GitHub Action (`action.yml`) that can be triggered automatically when CI fails, running an autonomous debug agent. | ||
|
|
||
| Use **skill mode** when working interactively with OpenHands to debug CI issues. Use **plugin mode** to automate CI failure analysis in your GitHub workflows. | ||
|
|
||
| ## Quick Start | ||
|
|
||
| Copy the workflow file to your repository: | ||
|
|
||
| ```bash | ||
| mkdir -p .github/workflows | ||
| curl -o .github/workflows/debug-ci-failure.yml \ | ||
| https://raw.githubusercontent.com/OpenHands/extensions/main/plugins/debug-github-ci/workflows/debug-ci-failure.yml | ||
| ``` | ||
|
|
||
| Then configure the required secrets (see [Installation](#installation) below). | ||
|
|
||
| ## Features | ||
|
|
||
| - **Automatic CI Failure Analysis**: Triggered when workflow runs fail | ||
| - **Log Analysis**: Fetches and analyzes failed job logs to identify root causes | ||
| - **Actionable Suggestions**: Posts comments with specific fixes and commands | ||
| - **Error Pattern Recognition**: Identifies common CI failure patterns | ||
| - **Context-Aware**: Considers recent commits and PR changes | ||
|
|
||
| ## Plugin Contents | ||
|
|
||
| ``` | ||
| plugins/debug-github-ci/ | ||
| ├── README.md # This file | ||
| ├── action.yml # Composite GitHub Action | ||
| ├── skills/ # Symbolic links to debug skills | ||
| │ └── debug-github-ci -> ../../../skills/debug-github-ci | ||
| ├── workflows/ # Example GitHub workflow files | ||
| │ └── debug-ci-failure.yml | ||
| └── scripts/ # Python scripts for debug execution | ||
| ├── agent_script.py # Main CI debug agent script | ||
| └── prompt.py # Prompt template for debugging | ||
| ``` | ||
|
|
||
| Notes: | ||
| - The marketplace manifest uses the repo-wide `pluginRoot: "./skills"`, so `source: "./debug-github-ci"` resolves to `skills/debug-github-ci`. | ||
| - The `plugins/debug-github-ci/skills/debug-github-ci` symlink mirrors the `pr-review` plugin pattern so the plugin bundle can reference the matching skill content without duplicating `SKILL.md`. | ||
|
|
||
| ## Installation | ||
|
|
||
| ### 1. Copy the Workflow File | ||
|
|
||
| Copy the workflow file to your repository's `.github/workflows/` directory: | ||
|
|
||
| ```bash | ||
| mkdir -p .github/workflows | ||
| curl -o .github/workflows/debug-ci-failure.yml \ | ||
| https://raw.githubusercontent.com/OpenHands/extensions/main/plugins/debug-github-ci/workflows/debug-ci-failure.yml | ||
| ``` | ||
|
|
||
| ### 2. Configure Secrets | ||
|
|
||
| Add the following secrets in your repository settings (**Settings → Secrets and variables → Actions**): | ||
|
|
||
| | Secret | Required | Description | | ||
| |--------|----------|-------------| | ||
| | `LLM_API_KEY` | Yes | API key for your LLM provider | | ||
| | `GITHUB_TOKEN` | Auto | Provided automatically by GitHub Actions | | ||
|
|
||
| ### 3. Customize the Workflow (Optional) | ||
|
|
||
| Edit the workflow file to customize: | ||
|
|
||
| ```yaml | ||
| - name: Debug CI Failure | ||
| uses: OpenHands/extensions/plugins/debug-github-ci@main | ||
| with: | ||
| # LLM model to use | ||
| llm-model: anthropic/claude-sonnet-4-5-20250929 | ||
|
|
||
| # Secrets | ||
| llm-api-key: ${{ secrets.LLM_API_KEY }} | ||
| github-token: ${{ secrets.GITHUB_TOKEN }} | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Automatic Triggers | ||
|
|
||
| CI debugging is automatically triggered when: | ||
|
|
||
| 1. A workflow run fails on the repository | ||
| 2. The `debug-ci` label is added to a PR with failed checks | ||
|
|
||
| ### Manual Trigger | ||
|
|
||
| You can manually trigger debugging: | ||
|
|
||
| 1. Go to **Actions** tab | ||
| 2. Select **Debug CI Failure** workflow | ||
| 3. Click **Run workflow** | ||
| 4. Enter the failed run ID | ||
|
|
||
| ## Action Inputs | ||
|
|
||
| | Input | Required | Default | Description | | ||
| |-------|----------|---------|-------------| | ||
| | `llm-model` | No | `anthropic/claude-sonnet-4-5-20250929` | LLM model to use | | ||
| | `llm-base-url` | No | `''` | Custom LLM endpoint URL | | ||
| | `run-id` | No | Auto | Specific workflow run ID to debug | | ||
| | `extensions-repo` | No | `OpenHands/extensions` | Extensions repository | | ||
| | `extensions-version` | No | `main` | Git ref (tag, branch, or SHA) | | ||
| | `llm-api-key` | Yes | - | LLM API key | | ||
| | `github-token` | Yes | - | GitHub token for API access | | ||
|
|
||
| ## What Gets Analyzed | ||
|
|
||
| The agent analyzes: | ||
|
|
||
| 1. **Failed job logs**: Console output from failed steps | ||
| 2. **Error messages**: Specific error patterns and stack traces | ||
| 3. **Recent commits**: Changes that may have caused the failure | ||
| 4. **Workflow configuration**: Issues in the workflow YAML | ||
| 5. **Dependencies**: Version conflicts or missing packages | ||
|
|
||
| ## Output | ||
|
|
||
| The agent posts a comment with: | ||
|
|
||
| - **Root Cause Analysis**: What caused the failure | ||
| - **Suggested Fixes**: Specific commands or code changes | ||
| - **Prevention Tips**: How to avoid similar failures | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Debug Not Triggered | ||
|
|
||
| 1. Ensure the workflow file is in `.github/workflows/` | ||
| 2. Verify secrets are configured correctly | ||
| 3. Check that the workflow has `workflow_run` trigger permissions | ||
|
|
||
| ### Analysis Incomplete | ||
|
|
||
| 1. Check if logs are available (some expire after 90 days) | ||
| 2. Verify the `GITHUB_TOKEN` has read access to workflow logs | ||
| 3. Increase timeout if analysis takes too long | ||
|
|
||
| ## Security | ||
|
|
||
| - Uses `workflow_run` event for secure access to secrets | ||
| - Only analyzes public workflow logs | ||
| - Does not execute any code from the failed run | ||
|
|
||
| ## Contributing | ||
|
|
||
| See the main [extensions repository](https://github.com/OpenHands/extensions) for contribution guidelines. | ||
|
|
||
| ## License | ||
|
|
||
| This plugin is part of the OpenHands extensions repository. See [LICENSE](../../LICENSE) for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| --- | ||
| name: OpenHands Debug GitHub CI | ||
| description: Automated CI failure debugging using OpenHands agent | ||
| author: OpenHands | ||
|
|
||
| branding: | ||
| icon: alert-circle | ||
| color: red | ||
|
|
||
| inputs: | ||
| llm-model: | ||
| description: LLM model to use for debugging | ||
| required: false | ||
| default: anthropic/claude-sonnet-4-5-20250929 | ||
| llm-base-url: | ||
| description: LLM base URL (optional, for custom LLM endpoints) | ||
| required: false | ||
| default: '' | ||
| run-id: | ||
| description: Specific workflow run ID to debug (auto-detected if not provided) | ||
| required: false | ||
| default: '' | ||
| extensions-repo: | ||
| description: GitHub repository for extensions (owner/repo) | ||
| required: false | ||
| default: OpenHands/extensions | ||
| extensions-version: | ||
| description: Git ref to use for extensions (tag, branch, or commit SHA) | ||
| required: false | ||
| default: main | ||
| llm-api-key: | ||
| description: LLM API key (required) | ||
| required: true | ||
| github-token: | ||
| description: GitHub token for API access (required) | ||
| required: true | ||
| use-cache: | ||
| description: | | ||
| Enable uv cache for faster dependency installation (default: false). | ||
| Disabled by default since debug tools benefit from fresh state to | ||
| avoid masking environment-related issues. | ||
| required: false | ||
| default: 'false' | ||
|
|
||
| runs: | ||
| using: composite | ||
| steps: | ||
| - name: Checkout extensions repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: ${{ inputs.extensions-repo }} | ||
| ref: ${{ inputs.extensions-version }} | ||
| path: extensions | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Install uv | ||
| uses: astral-sh/setup-uv@v6 | ||
| with: | ||
neubig marked this conversation as resolved.
Show resolved
Hide resolved
neubig marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| enable-cache: ${{ inputs.use-cache == 'true' }} | ||
|
|
||
| - name: Install GitHub CLI | ||
| shell: bash | ||
| run: | | ||
| # The bundled workflow template runs on ubuntu-24.04, so apt is the | ||
| # simplest way to guarantee gh is available on the runner. | ||
| sudo apt-get update | ||
neubig marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| sudo apt-get install -y gh | ||
|
|
||
| - name: Determine run ID | ||
| id: get-run-id | ||
| shell: bash | ||
| env: | ||
| GITHUB_TOKEN: ${{ inputs.github-token }} | ||
| INPUT_RUN_ID: ${{ inputs.run-id }} | ||
| run: | | ||
| if [ -n "$INPUT_RUN_ID" ]; then | ||
| echo "run_id=$INPUT_RUN_ID" >> $GITHUB_OUTPUT | ||
| elif [ -n "${{ github.event.workflow_run.id }}" ]; then | ||
| echo "run_id=${{ github.event.workflow_run.id }}" >> $GITHUB_OUTPUT | ||
| else | ||
| # Get the most recent failed run | ||
| RUN_ID=$(gh run list --status failure --limit 1 --json databaseId --jq '.[0].databaseId' 2>/dev/null || echo "") | ||
| if [ -z "$RUN_ID" ]; then | ||
| echo "::error::No run ID provided and no failed runs found" | ||
| exit 1 | ||
| fi | ||
| echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT | ||
neubig marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| fi | ||
|
|
||
| - name: Check required configuration | ||
| shell: bash | ||
| env: | ||
| LLM_API_KEY: ${{ inputs.llm-api-key }} | ||
| GITHUB_TOKEN: ${{ inputs.github-token }} | ||
| run: | | ||
| if [ -z "$LLM_API_KEY" ]; then | ||
| echo "Error: llm-api-key is required." | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ -z "$GITHUB_TOKEN" ]; then | ||
| echo "Error: github-token is required." | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Run ID to debug: ${{ steps.get-run-id.outputs.run_id }}" | ||
| echo "Repository: ${{ github.repository }}" | ||
| echo "LLM model: ${{ inputs.llm-model }}" | ||
|
|
||
| - name: Run CI debug analysis | ||
| shell: bash | ||
| env: | ||
| LLM_MODEL: ${{ inputs.llm-model }} | ||
| LLM_BASE_URL: ${{ inputs.llm-base-url }} | ||
| LLM_API_KEY: ${{ inputs.llm-api-key }} | ||
| GITHUB_TOKEN: ${{ inputs.github-token }} | ||
| RUN_ID: ${{ steps.get-run-id.outputs.run_id }} | ||
| REPO_NAME: ${{ github.repository }} | ||
| # Marker to prevent recursive debugging - checked by agent_script.py | ||
| OH_DEBUG_WORKFLOW: 'true' | ||
| run: | | ||
| uv run --with openhands-sdk --with openhands-tools \ | ||
| python extensions/plugins/debug-github-ci/scripts/agent_script.py | ||
|
|
||
| - name: Upload logs as artifact | ||
| uses: actions/upload-artifact@v4 | ||
| if: always() | ||
| with: | ||
| name: openhands-ci-debug-logs | ||
| path: | | ||
| *.log | ||
| output/ | ||
| retention-days: 7 | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.