feat: add azure-arc-servers skill#2359
Open
deankroker wants to merge 2 commits into
Open
Conversation
Adds a new `azure-arc-servers` skill that routes Azure Arc-enabled server (Connected Machine) intents — onboarding, troubleshooting, day-2 management — to the right workflow. The skill targets servers that live outside Azure (on-prem, AWS, GCP, edge) being projected into Azure via the Connected Machine agent (`azcmagent`) — ARM type `Microsoft.HybridCompute/machines`. It is explicitly disambiguated from `azure-compute` (which owns new VMs in Azure proper). ## Structure - `SKILL.md` — router (intent → workflow) - `references/arc-vs-azure-vm.md` — disambiguation cheatsheet - `references/arc-mcp-tools.md` — Azure MCP + `az` + `azcmagent` lookup - `workflows/arc-server-onboard/` — single + at-scale onboarding, prerequisites, connectivity (Public / Private Link / Arc Gateway / Edge / air-gapped), deployment methods (Basic / ConfigMgr / GPO / Ansible) - `workflows/arc-server-troubleshoot/` — agent status (Connected / Disconnected / Expired / Error) and common-issues lookup - `workflows/arc-server-manage/` — agent upgrade, Extended Security Updates (ESU), Hotpatch, Pay-as-you-go Windows Server Source content lifted from the HybridComputeExtension portal repo with references back to the canonical `Microsoft.HybridCompute` ARM types and the HCRP swagger. ## Tests - `tests/azure-arc-servers/triggers.test.ts` — 52 trigger / negative / edge-case cases across onboarding, at-scale, connectivity, troubleshoot, manage, ESU, Hotpatch, PayGo - `tests/azure-arc-servers/integration.test.ts` — 4 routing assertions (single onboard, on-prem disambiguation vs `azure-compute`, Disconnected troubleshoot, ESU manage) including a hard check that `azure-compute` is NOT invoked for a clearly-Arc prompt - Registered in `tests/skills.json` skills array and the 12pm integration schedule slot ## Char budget Bumped `DEFAULT_SKILL_CHAR_BUDGET` in `scripts/src/copilot-cli-char-budget.ts` from 20000 → 22000. The description (908 chars) pushed the existing total (19815) over the 20000 soft cap. The comment on that constant explicitly says it's a soft cap meant to be bumped when consumption grows; this bump gives ~1000 chars of headroom for the next contribution. New count: 20875. ## Validation - `npm run build` — stamps `azure-arc-servers` 1.0.0 via NBGV - `scripts: npm run frontmatter ../output/skills/azure-arc-servers/SKILL.md` — pass - `scripts: npm run references -- --skills-dir ../output/skills` — pass - `scripts: npm run checkCopilotCliCharBudget` — pass (20875 / 22000) - `scripts: npm run checkPluginVersionPr` — pass - `scripts: npm test` — 463 / 463 pass - `tests: npm run typecheck && npm run lint` — pass - `tests: SKIP_INTEGRATION_TESTS=true npm test -- --testPathPatterns=azure-arc-servers` — 52 / 52 pass, snapshot created, 4 integration tests appropriately skipped Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Introduces a new azure-arc-servers skill (workflows + references) and adds corresponding Jest trigger/integration tests, while updating test registration/scheduling and increasing the Copilot CLI skill char budget soft cap.
Changes:
- Added the
azure-arc-serversskill with onboarding, troubleshooting, and day-2 management workflows plus extensive reference docs. - Added trigger tests, integration tests, snapshots, and fixtures for
azure-arc-servers, and registered/scheduled the skill intests/skills.json. - Increased the default skill character budget soft cap from 20,000 to 22,000.
Reviewed changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/skills.json | Registers azure-arc-servers and includes it in the integration test schedule. |
| tests/azure-arc-servers/triggers.test.ts | Adds trigger coverage and snapshots for the new skill. |
| tests/azure-arc-servers/integration.test.ts | Adds agent-based integration tests validating workflow routing. |
| tests/azure-arc-servers/fixtures/sample.json | Adds a fixture file (currently template-like). |
| tests/azure-arc-servers/snapshots/triggers.test.ts.snap | Adds Jest snapshots for extracted keywords/description triggers. |
| tests/azure-arc-servers/README.md | Adds a README (currently appears to be a generic template). |
| scripts/src/copilot-cli-char-budget.ts | Raises the default skill char soft cap to 22,000. |
| plugin/skills/azure-arc-servers/workflows/arc-server-troubleshoot/references/common-issues.md | Adds troubleshooting lookup table for common Arc agent issues. |
| plugin/skills/azure-arc-servers/workflows/arc-server-troubleshoot/references/agent-status.md | Adds detailed reference for Connected/Disconnected/Expired/Error states. |
| plugin/skills/azure-arc-servers/workflows/arc-server-troubleshoot/arc-server-troubleshoot.md | Adds the troubleshooting workflow. |
| plugin/skills/azure-arc-servers/workflows/arc-server-onboard/references/prerequisites.md | Adds prerequisites reference for onboarding guidance. |
| plugin/skills/azure-arc-servers/workflows/arc-server-onboard/references/deployment-methods.md | Adds deployment method decision guide (Basic/ConfigMgr/GPO/Ansible). |
| plugin/skills/azure-arc-servers/workflows/arc-server-onboard/references/connectivity-options.md | Adds connectivity decision guide (public/proxy/PLS/gateway/air-gapped). |
| plugin/skills/azure-arc-servers/workflows/arc-server-onboard/references/at-scale-onboarding.md | Adds at-scale onboarding guidance and rollout strategy. |
| plugin/skills/azure-arc-servers/workflows/arc-server-onboard/arc-server-onboard.md | Adds the onboarding workflow with plan-card gating and routing. |
| plugin/skills/azure-arc-servers/workflows/arc-server-manage/references/pay-as-you-go.md | Adds PayGo licensing reference and CLI sketches. |
| plugin/skills/azure-arc-servers/workflows/arc-server-manage/references/hotpatch.md | Adds Hotpatch enablement guidance and troubleshooting. |
| plugin/skills/azure-arc-servers/workflows/arc-server-manage/references/extended-security-updates.md | Adds ESU licensing guidance and troubleshooting. |
| plugin/skills/azure-arc-servers/workflows/arc-server-manage/references/agent-upgrade.md | Adds agent upgrade/auto-upgrade reference. |
| plugin/skills/azure-arc-servers/workflows/arc-server-manage/arc-server-manage.md | Adds day-2 management workflow including policies/services/connecting. |
| plugin/skills/azure-arc-servers/version.json | Adds skill version metadata and path filters. |
| plugin/skills/azure-arc-servers/references/arc-vs-azure-vm.md | Adds conceptual disambiguation between Arc servers and Azure VMs. |
| plugin/skills/azure-arc-servers/references/arc-mcp-tools.md | Adds MCP/CLI/tooling reference for Arc-related operations. |
| plugin/skills/azure-arc-servers/SKILL.md | Adds skill manifest + routing overview for the new skill. |
Addresses Copilot reviewer feedback on microsoft#2359: - Delete `tests/azure-arc-servers/README.md` (was unmodified `_template` README that referenced `.js` test files). Only 2 of 28 skill test dirs ship a README; the dominant convention is no per-skill README. - Delete unused `tests/azure-arc-servers/fixtures/sample.json` (and empty `fixtures/` dir). It was the template's placeholder fixture (with an unrelated `azure__bicepschema` mock) and was not referenced by any test. - `integration.test.ts`: - Rename `invocationRateThreshold` → `INVOCATION_RATE_THRESHOLD` to match `SKILL_NAME` / `RUNS_PER_PROMPT` UPPER_SNAKE_CASE convention. - Drop `| undefined` from `expectPromptToInvokeWorkflow` return type — the function always returns an object; the dead - Keep `${SKILL_NAME}_ - Integration Tests` describe title; the underscore is required by the custom ESLint rule `integration-test-name` (pattern `/^[a-z0-9-]+_[a-z0-9-]* - Integration Tests$/`), which all other skills also satisfy. Reviewer flagged this as a typo, but removing it breaks lint. - `triggers.test.ts`: add a "Should NOT Trigger - unambiguous azure-compute" describe block with 2 near-neighbor prompts that the keyword pre-filter correctly rejects. A long comment explains the scope: `TriggerMatcher` is keyword-based and SKILL.md contributes generic keywords (`azure`, `machine`, `server`, `windows`) that any Azure-VM prompt will hit; true Arc-vs-Compute disambiguation is enforced by the LLM router and verified by the integration test that asserts `wrongSkillInvocations === 0`. The 2 included prompts have no Arc-keyword overlap and serve as a regression guard against keyword drift in SKILL.md. - `arc-server-manage.md`: fix misleading RDP section. The previous snippet labeled `az connectedmachine run-command create ...` as an RDP command — `run-command` is unrelated to interactive remoting. Rewritten to keep the verified SSH-over-Arc flow for Linux, direct Windows RDP users to the portal **Connect** blade (which sets up the tunnel and hands off to the local RDP client), and explicitly warn against improvising RDP-via-Arc with `run-command`. Validation: - `SKIP_INTEGRATION_TESTS=true npm test` → 1327/1327 pass, 54 in arc-servers - `scripts: npm run frontmatter -- ../output/skills/azure-arc-servers/SKILL.md` ✅ - `scripts: npm run references -- --skills-dir ../output/skills` ✅ - `scripts: npm run checkCopilotCliCharBudget` ✅ (20875 / 22000) - `tests: npm run typecheck && npm run lint` ✅ Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| @@ -0,0 +1,87 @@ | |||
| --- | |||
| name: azure-arc-servers | |||
| description: "Azure Arc-enabled servers router. WHEN: Arc Server, Arc-enabled server, Connected Machine agent, azcmagent, hybrid server, on-prem server, onboard to Arc, project a server into Azure, connect machine to Azure, install Connected Machine agent, generate onboarding script, at-scale onboarding, Group Policy onboarding, Configuration Manager onboarding, Ansible onboarding, Service Principal for Arc, Arc Private Link Scope, Arc Gateway, proxy for Arc, agent connectivity, agent status, Connected / Disconnected / Expired / Error, agent upgrade, automatic upgrade, manual upgrade, Extended Security Updates (ESU) for Arc, Hotpatch on Arc, Pay-as-you-go Windows Server, Software Assurance benefits, Microsoft.HybridCompute, hybridcompute/machines. PREFER OVER azure-compute when the user is talking about non-Azure (on-prem, other-cloud, edge) servers being projected into Azure - not about creating an Azure VM." | |||
Comment on lines
+67
to
+70
| - The **Azure VM agent** (`Microsoft.Azure.HybridCompute` extension on | ||
| an Azure VM) is mutually exclusive with `azcmagent` on the same OS | ||
| instance. If the machine **is** an Azure VM, the user should not be | ||
| onboarding it via Arc. |
Comment on lines
+36
to
+42
| # Read current setting (PATCH the desired state) | ||
| az resource patch \ | ||
| --resource-type "Microsoft.HybridCompute/machines" \ | ||
| --resource-group <rg> \ | ||
| --name <machine-name> \ | ||
| --api-version 2025-02-19-preview \ | ||
| --properties '{"agentUpgrade":{"enableAutomaticUpgrade":true}}' |
|
|
||
| const describeIntegration = skipTests ? describe.skip : describe; | ||
|
|
||
| describeIntegration(`${SKILL_NAME}_ - Integration Tests`, () => { |
Member
|
Hey Dean -- looks like we need a CODEOWNERS entry for the new skill. Every existing skill has a line in Can you add one for I'll get you Jackie's alias separately. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
azure-arc-serversskill that routes Azure Arc–enabled server (Connected Machine) intents — onboarding, troubleshooting, and day-2 management — to the right workflow.The skill targets servers that live outside Azure (on-prem, AWS, GCP, edge) being projected into Azure via the Connected Machine agent (
azcmagent) — ARM typeMicrosoft.HybridCompute/machines. It is explicitly disambiguated fromazure-compute(which owns brand-new VMs in Azure proper,Microsoft.Compute/virtualMachines).Structure
Content lifted from internal portal-team Arc skill source and adapted to this repo's conventions. All references point back to canonical
Microsoft.HybridCompute/{machines,licenses,gateways,privateLinkScopes}ARM types and public learn.microsoft.com docs.Disambiguation from
azure-computeThe
SKILL.mddescription (908 chars, within the 1024 limit) is engineered to fire on Arc-specific signals (on-prem,hybrid,azcmagent,Connected Machine,Arc-enabled, ESU, Hotpatch, PayGo) and not on generic "create a VM in Azure" prompts. The integration suite hard-assertsazure-computeis not also invoked for an unambiguous on-prem prompt.Tests
tests/azure-arc-servers/:triggers.test.ts— 52 cases across 9 buckets: onboard / at-scale / connectivity / troubleshoot / manage / ESU / Hotpatch / PayGo / negatives / edge casesintegration.test.ts— 4 routing assertions:tests/skills.json(skills array + 12pm integration schedule slot)Char budget
Bumped
DEFAULT_SKILL_CHAR_BUDGETinscripts/src/copilot-cli-char-budget.tsfrom20000→22000. Repo was already at 19815/20000 before this PR; the in-file comment explicitly says the constant is a soft cap meant to be bumped as new skills land. New total: 20875 / 22000 (≈1100 chars of headroom for the next contribution).Validation (all green locally)
npm run buildazure-arc-servers 1.0.0scripts: npm run frontmatter -- ../output/skills/azure-arc-servers/SKILL.mdscripts: npm run references -- --skills-dir ../output/skillsscripts: npm run checkCopilotCliCharBudgetscripts: npm run checkPluginVersionPrscripts: npm testtests: npm run typecheck && npm run linttests: SKIP_INTEGRATION_TESTS=true npm test -- --testPathPatterns=azure-arc-serversKnown soft warnings (non-blocking in CI)
npm run tokens checkflagsSKILL.md(1394 tok) andarc-server-onboard.md(>2000 tok) above the soft default. CI only reports this; several existing skills (azure-compute, microsoft-foundry, entra-*, azure-upgrade) sit similarly over their soft caps. Happy to split onboarding further if a reviewer wants strict compliance.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com