Skip to content

feat(samples): community contribution flow + generated pipeline registry #556

@staging-devin-ai-integration

Description

Summary

Epic: #555

First concrete deliverable of the community pipeline marketplace (epic: see linked issue). Stand up the contribution path + a generated, GitHub-hosted pipeline registry — without touching the server or UI yet. The output is a consumable, validated index.json (with a pipelines[] array) that a later phase wires into the server install flow.

Scope

  1. Contribution dir. Add a community/ location for community-submitted pipelines (decision in the epic: same-repo dir vs separate streamkit/pipelines repo). Each entry is a pipeline YAML plus the discovery metadata (group/variant/category/tags, landing via feat(samples): discovery UX with variant grouping and faceted search #544/Backfill explicit discovery metadata on sample YAMLs; thin the derivation heuristics #551) and a declared requires { plugins, models, features, assets }.
  2. CI validation (reuse, don't reinvent). Gate every submission with the existing checks: parse + compile + node-availability (the logic behind test(pipelines): add CI validation for official oneshot samples #542 / test(samples): guard dynamic samples with parse/compile + node-availability checks #543), so a contributed pipeline can't reference a nonexistent node kind or fail to compile. Validate that declared requires.plugins/features match the node kinds actually referenced in the YAML.
  3. Registry generator. Mirror scripts/marketplace/generate_official_plugins.py to emit per-version manifest.json for each pipeline and a registry index.json containing a pipelines[] array. Keep the catalog thin (metadata only); the YAML + requires{} live in the per-version manifest, fetched on demand.
  4. Publish. Build + publish the generated registry to GitHub Pages alongside the existing plugin registry (combined index.json vs dedicated URL — decision in the epic).

Schema (additive, backward-compatible)

RegistryIndex.plugins is already #[serde(default)] (apps/skit/src/marketplace.rs), so adding a sibling pipelines[] doesn't break existing registries or older servers. Pipeline manifest mirrors PluginManifest but carries inline yaml (or yaml_url), mode, discovery metadata, and requires{}. See the epic for the proposed shape.

Out of scope (phase 2, tracked off the epic)

  • Server-side list/fetch/install of pipelines and dependency resolution (marketplace.rs, marketplace_installer.rs, /api/v1/marketplace/pipelines*).
  • Any UI for browsing/installing pipelines.
  • Discovery/search service (Option C).

Acceptance

  • A PR adding a pipeline under community/ is validated by CI (parse/compile/node-availability + requires consistency).
  • The generator produces a signed-or-signable index.json with a populated pipelines[] and per-version manifests.
  • The registry is published and fetchable over HTTPS (GitHub Pages), gzip-compressed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions