Skip to content

Skill asset promotion: capture working wisps as typed skill resources #370

@rockfordlhotka

Description

@rockfordlhotka

Problem

The agent's self-improvement loop is asymmetric: failures get distilled into prose annotations on skills, but successes have no path to becoming a saved, reusable asset. The 10am operational-review run on 2026-05-08 made this visible — two subagents (acff89d7d383 communications, 744d17efba28 SocialAgent) spent 8 and 10 minutes respectively reconciling tool names, schemas, and download paths, converged on working sequences, and were then cancelled at their wallclock budget. Whatever they figured out died with the sessions. Tomorrow's run will start from zero.

The infrastructure to fix this is fully wired: SkillResource, Manifest, SaveSkill(resources=…), GetSkillResource, and the [Wisp, Python] index tag all exist; SkillResourceType.Wisp is an enum value; subagents already have SaveSkill in their tool list (SubagentRunner.cs:112). The capability is unused because no directive or dream pass tells the agent to use it.

Four underlying defects

  1. Wisp success bodies are not retained. WispExecutionRecord (src/RockBot.Host.Abstractions/WispExecutionRecord.cs) stores DefinitionHash but not the step definitions. Even when a hash repeats successfully, the JSON to attach is unrecoverable.
  2. promotionCandidates is dead-ended. The wisp-failure-dream.md directive emits promotionCandidates; the receiver in DreamService.cs:2739 only logs them. The DTO carries no targetSkillName, no resourceFilename, no body — nothing calls SaveSkill with resources=[…].
  3. The reflection system is failure-shaped. skill-optimize.md and wisp-failure-dream.md only fire on failure signals. There is no symmetric success-pass. Result: skills like patrol/wisp-mcp-params accumulate ten near-identical "negative example" paragraphs instead of one positive example file attached as a resource.
  4. Subagents promote prose, not assets. subagent-directives.md tells the subagent to update skill prose with verified specifics — text edits only. Nothing tells it to attach the working wisp/script body it just executed as a typed SkillResource.

Plan (five phases)

Full design: see design/skill-asset-promotion.md (PR #369).

Phase Summary Effort
1. Persist wisp bodies Add DefinitionBody to WispExecutionRecord; dedup-by-hash; bounded size; nullable on failed/old records Small
2. In-session promotion New promote_skill_asset LLM tool + subagent-directives.md section; SkillResource gains Provisional + CreatedAt; index shows [Wisp*] for provisional Small
3. Success-shaped dream pass New wisp-success-dream.md directive; receiver resolves canonical body from Phase 1 and calls SaveSkill(...) to attach non-provisionally; threshold freq >= 3 && successRate == 1.0 Medium
4. Wire promotionCandidates actionable Replace the dead-ended DTO with the actionable Promotion shape from Phase 3; close the dream-side loop Small
5. Provisional validation/demotion Read-side records (skill, filename, succeeded) events; N successes → flip non-provisional; M consecutive failures → remove and feed FailureClusterStore; 30d staleness sweep Medium

Suggested order: 1 + 2 in parallel → 3 → 4 → 5. Phase 2 alone gives the calendar/communications lanes a way to capture today's work today. Phases 1–4 close the dream-side loop. Phase 5 turns on the trust gradient.

Goals

  • A subagent that converges on a working tool sequence captures it as a typed skill resource before exiting, with a verify hint.
  • The dream system distinguishes failure-shaped reflection (already present) from success-shaped reflection (new).
  • Promoted resources self-evict when they stop working.
  • Skill index visibly distinguishes provisional from validated resources.

Non-goals

  • Replacing existing failure-driven skill-optimize / wisp-failure-dream passes. Additive complement.
  • General-purpose code generation. Promotion captures observed working assets only — never synthesizes from scratch.
  • Retroactive promotion of pre-shipped historical successes. Old hash-only records remain hash-only.

Relationship to existing self-repair series (#345#350)

This is a peer track to the self-repair work, not a phase of it. Where they overlap, this design reuses self-repair primitives:

Splitting into per-phase issues can come once the design is approved, mirroring the self-repair pattern.

🤖 Generated with Claude Code

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions