Skip to content

fix: improve error messaging for agent output validation failures#68

Open
Mr-Neutr0n wants to merge 1 commit intoKeygraphHQ:mainfrom
Mr-Neutr0n:fix/prerecon-error-messaging
Open

fix: improve error messaging for agent output validation failures#68
Mr-Neutr0n wants to merge 1 commit intoKeygraphHQ:mainfrom
Mr-Neutr0n:fix/prerecon-error-messaging

Conversation

@Mr-Neutr0n
Copy link

Summary

Improve error messages when agents fail to create required deliverable files, making it easier to diagnose and fix common issues.

Problem

When the pre-recon agent (or other agents) fail to create required deliverable files, users see a generic error:

Agent pre-recon failed output validation after 3 attempts

This doesn't help users understand what went wrong or how to fix it.

Solution

Add troubleshooting tips and more detailed logging:

  1. Troubleshooting tips in error message (for pre-recon agent):

    • Check if repository path is accessible
    • If using ROUTER mode, ensure model supports tool use
    • Check agent logs location
    • Try Claude models (recommended)
    • Ensure target URL is reachable from Docker
  2. Log expected file path when validation fails:

    Expected file: /path/to/repo/deliverables/code_analysis_deliverable.md
    

Common causes this helps diagnose

  • Using router mode with models that don't follow instructions well
  • Repository path not accessible from Docker container
  • Target URL not reachable (should use host.docker.internal for local apps)

Test plan

  • Run Shannon with an intentionally inaccessible repo path
  • Verify improved error message appears with troubleshooting tips
  • Verify expected file path is logged on validation failure

Fixes #62

Add troubleshooting tips and more detailed error messages when agents
fail to create required deliverable files, especially for pre-recon.

Changes:
- Add troubleshooting tips to OutputValidationError for pre-recon agent
  (check repo path, router mode model, agent logs, target URL accessibility)
- Log expected file path when validation fails to help debug issues
- Include guidance to check audit logs for detailed agent output

This helps users diagnose common issues like:
- Using router mode with models that don't follow instructions well
- Repository path not accessible from Docker
- Target URL not reachable

Fixes KeygraphHQ#62
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent pre recon failed

1 participant