Skill asset promotion: full 5-phase implementation (#370)#376
Open
rockfordlhotka wants to merge 7 commits intomainfrom
Open
Skill asset promotion: full 5-phase implementation (#370)#376rockfordlhotka wants to merge 7 commits intomainfrom
rockfordlhotka wants to merge 7 commits intomainfrom
Conversation
Add DefinitionBody and BodyOmittedTooLarge to WispExecutionRecord so the success-shaped dream pass can recover the JSON of a repeating successful wisp for promotion to a skill resource. Bodies are retained only on success and only when they fit an 8 KiB cap; failures and oversize runs record null with the diagnostic flag. Add IWispExecutionLog.GetCanonicalBodyAsync(definitionHash) returning the earliest non-null body for a given hash so dedup-by-hash is cheap and subsequent retries do not need to re-store identical bodies. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…Async (#370) Extend SkillResource with Provisional, CreatedAt, VerifyHint, and DefinitionHash so the validation pass can distinguish trust levels, preserve advisory exercise notes, and cross-reference resources against the wisp execution log by content hash. Add ISkillStore.AttachResourceAsync (additive single-resource add), RemoveResourceAsync, and UpdateResourceMetadataAsync. The 2-arg SaveAsync now preserves Provisional/VerifyHint from the input and stamps CreatedAt/DefinitionHash on every entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add SkillTools.PromoteSkillAsset gated behind a new enablePromote ctor flag — currently set only by SubagentRunner. The main agent does not see the tool because it does not perform the exploratory discovery whose result is worth capturing as a typed asset; it consumes assets the dream pass has already promoted. The tool computes a content hash matching the wisp execution log's definition-hash scheme and writes the manifest entry as provisional with CreatedAt and an optional VerifyHint, leaving validation to the dream-cycle pass. FormatResourceTag now appends a trailing * to types that have at least one provisional entry, so the LLM sees the trust gradient at a glance in the skill index. The subagent directive describes when and how to call promote_skill_asset as the symmetric complement to skill-prose tightening. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add RunWispSuccessAnalysisPassAsync as the symmetric complement to the failure pass. It groups recent wisp records by definition hash, keeps groups that succeeded repeatedly with no failures, resolves the invoking skill via ISkillUsageStore, fetches the canonical body via Phase 1's GetCanonicalBodyAsync, and asks the LLM to choose promotions. Promotions land non-provisional via AttachResourceAsync because the dream pass operates on observed repetition rather than a hypothesis — the in-session promote_skill_asset path is the one that lands provisional. The apply loop is extracted as an internal static helper so unit tests can drive the attach logic without standing up the LLM. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…370) The promotionCandidates field on the failure-pass output was advisory-only — receiver in DreamService just logged each entry without calling SaveSkill. Now that Phase 3 ships a real success-shaped dream pass that actually attaches resources, the dead end is removed: the field comes out of the directive (file + built-in fallback), the DTO is deleted, and the trailing log loop is gone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…370) Extend the self-repair RepairTarget enum with SkillResource so the dream-cycle apply pass can mutate skill manifests as a first-class target, and add SkillResourceApplier with three ops: - attach: add a provisional resource (revert removes it, or restores the prior body if one existed at the same filename) - delete: remove a resource (revert restores body + manifest entry) - demote-provisional: flip Provisional=true on an existing entry, used when the dream pass loses confidence in a previously validated resource (revert restores Provisional=false) Both attach and delete capture pre-state at apply time so the verify phase can roll back without a second LLM round-trip. demote-provisional is no-op when the entry is already provisional. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the dream-cycle pass that turns Phase-2 provisional resources into validated ones, removes the broken ones, and de-emphasises the stale ones — closing the trust gradient. Decision rule (DecideProvisionalAction, internal-static for testing): - Distinct-session successful wisp records sharing the resource's DefinitionHash >= 3 → flip Provisional=false, keep VerifyHint per user pref. - Most-recent N wisp records all failures → remove resource and record a FailureClusterStore entry keyed (skill-resource, skillName, filename). - Resource older than 30d with zero activity → prefix description with "[stale]" so the LLM stops loading it (body kept on disk). - Otherwise → keep. For non-wisp resources (Python, JsonSchema), distinct-session checkouts recorded by the new ISkillResourceUsageStore (FileSkillResourceUsageStore JSONL) substitute as a soft positive signal. SkillTools.GetSkillResource fire-and-forgets a checkout event when the optional store is wired and a sessionId is in scope; SubagentRunner now wires it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #370.
Implements the full design from
design/skill-asset-promotion.md(PR #369). All five phases land in this PR per the plan; commits are split atomically per phase.Summary
The agent's self-improvement loop is no longer asymmetric. Subagents now capture the working artifacts they converge on (wisp definitions, scripts, schemas) as typed skill resources via a new
promote_skill_assettool. The dream cycle has a symmetric success-shaped pass that promotes repeating successful patterns autonomously. Provisional resources self-validate or self-evict based on observed use.WispExecutionRecord.DefinitionBody+IWispExecutionLog.GetCanonicalBodyAsyncmake a successful wisp's JSON recoverable for promotion. Bodies are size-capped at 8 KiB; failures and oversize runs recordnullwith a diagnostic flag.SkillResourcegainsProvisional,CreatedAt,VerifyHint,DefinitionHash.ISkillStoregains additiveAttachResourceAsync/RemoveResourceAsync/UpdateResourceMetadataAsyncso a single resource can be added or mutated without disturbing the rest of the manifest.SkillTools.PromoteSkillAssetis gated behind a newenablePromotector flag, set only bySubagentRunner. The main agent does not see the tool.subagent-directives.mddescribes when to use it.FormatResourceTagshows[Wisp*]when any entry of that type is provisional.RunWispSuccessAnalysisPassAsyncgroups recent records by definition hash, keeps groups withfrequency >= 3 && successRate == 1.0, resolves the invoking skill viaISkillUsageStore, fetches the canonical body via Phase 1, and asks the LLM to choose promotions. Promotions land non-provisional (observed-repetition validated). Newwisp-success-dream.mddirective plus a built-in fallback.promotionCandidateslog loop on the failure pass. The success pass is now the single source of truth for promotions.RepairTarget.SkillResource+SkillResourceApplierextend self-repair to mutate skill manifests as a first-class target. Three ops (attach,delete,demote-provisional), each with a revert callback so verify failures roll back cleanly.ISkillResourceUsageStorerecords resource-checkout events.RunProvisionalValidationPassAsyncuses both wisp-record cross-reference (byDefinitionHash) and checkout events to flip provisional resources to validated after distinct-session success, remove them after consecutive failure (with aFailureClusterStoreentry), and prefix[stale]to old unused ones. Decision logic is extracted asDreamService.DecideProvisionalActionfor testability.Test plan
dotnet build RockBot.slnxclean (no warnings on changed code)dotnet test RockBot.slnxclean — 1808 tests pass, 0 fail (15 skipped: RabbitMQ + Scripts.Container integration)WispExecutionLogTests— 16 tests, including newGetCanonicalBodyAsyncround-trip and oversize-body handlingFileSkillStoreTests— 57 tests, 8 new forAttachResourceAsync/RemoveResourceAsync/UpdateResourceMetadataAsyncSkillToolsTests— 35 tests, 10 new forPromoteSkillAsset,enablePromotegating, andFormatResourceTagprovisional markerDreamServiceWispSuccessTests— 6 tests forApplyWispSuccessPromotionsAsyncdecision boundariesSkillResourceApplierTests— 10 tests for attach / delete / demote-provisional + revert round-tripsProvisionalValidationPassTests— 11 tests forDecideProvisionalActionthresholds +FileSkillResourceUsageStoreround-tripMEMORY.md; not blocking the PRNotes
BuiltInWispFailureDirectiveconstant inDreamService.csand the on-diskwisp-failure-dream.mdboth havepromotionCandidatesremoved — Phase 4a — so they stay in sync.WispPromotionCandidateDtorecord is deleted (no consumers remain after Phase 4a).FileSkillStore.SaveAsync(skill, resources)(the bulk-replace path) now preservesProvisional,VerifyHint, and stampsCreatedAt+DefinitionHashon every entry it writes. This means the existingpatrol/wisp-mcp-paramsskill — currently the only one with a populated manifest — will gainCreatedAt+DefinitionHashthe next time it's resaved, with no migration required.🤖 Generated with Claude Code