Skip to content

Optimize library template discovery for large template collections #1502

@ChristianLempa

Description

@ChristianLempa

Problem

As the template library grows, the compose list command (and other module list commands) becomes noticeably slower. Currently, the CLI checks the draft status of every template directory by parsing YAML files, which scales linearly with template count.

Current performance:

  • 34 templates (20 drafts + 14 published): ~230ms
  • Projected with 100 templates: ~700ms
  • Projected with 300 templates: ~2+ seconds

The bottleneck occurs because we:

  1. Iterate through all template directories
  2. Open and parse each template.yaml file to check metadata.draft status
  3. Parse YAML for templates that won't even be displayed (drafts)

This happens on every list command, even though draft status rarely changes.

User Impact

  • Slower command response times as library grows
  • Poor user experience with large template collections
  • Wasted I/O and CPU on unchanged data
  • Scales poorly with multiple libraries

When to Address

This optimization should be implemented before the library reaches 100+ templates or when users report noticeable slowdown in list commands.

Context

Performance analysis and potential solutions have been documented in PERFORMANCE_OPTIMIZATION_PLAN.md. The refactoring should maintain backward compatibility while significantly improving template discovery performance.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions