discussion/standardize-docs-across-tools

[bwalsh]

Problem

The tools documentation under docs/tools/ is inconsistent across projects. Some tools place pages at docs/tools/<tool>/docs/*.md with an index.md redirect, others place pages directly under docs/tools/<tool>/. Navigation and section ordering vary, and there is no standard "quickstart/TL;DR" or "For bioinformaticians" snippet to help researchers quickly apply the tools in pipelines.

Why this matters

• Inconsistent navigation and page naming make it harder for users (especially bioinformaticians) to find quick, actionable examples.
• Manual updates are error-prone and time-consuming when we want to sync canonical docs from upstream projects.
• CI/automation should be able to run validation (presence of required pages, linkcheck) after sync.

Proposed change

Standardize per-tool docs layout and navigation, add short TL;DR quickstarts and "For bioinformaticians" sections, and provide Makefile automation to sync canonical docs from tool repos.

Goals

• Every tool uses a consistent layout and .nav.yml so the site nav is predictable.
• Every tool provides a short copy-paste TL;DR quickstart and a small "For bioinformaticians" example (Snakemake/Nextflow snippet) at the top of the overview or quickstart page.
• Provide a documented Makefile workflow to sync docs from upstream repos and a validation step (mkdocs build / linkcheck) to run during CI.

Deliverables / tasks

1. Audit (automated) — produce a report listing for each tool which of these pages exist:

   • docs/index.md (overview)
   • docs/quickstart.md
   • docs/installation.md
   • docs/configuration.md
   • docs/commands.md (or cli.md)
   • docs/integrations.md or examples.md
   • docs/troubleshooting.md
   • docs/developer/ (optional)

2. Create a canonical page template and a short "For bioinformaticians" TL;DR snippet in scripts/templates/.

3. For priority tools (start with git-drs, syfon, funnel, grip):

   • Ensure files are in docs/tools/<tool>/docs/ (or update nav to point to their existing locations consistently).
   • Add/insert the TL;DR and "For bioinformaticians" snippet to the overview or quickstart page.
   • Add/overwrite docs/tools/<tool>/.nav.yml using the standardized nav order.

4. Add validation steps in Makefile or CI:

   • make sync already exists (runs sync-syfon-docs, sync-git-drs-docs, update-branches). Extend to optionally run mkdocs build and mkdocs link-check on changed docs.
   • Add --dry-run or preview mode for sync targets.

5. Run mkdocs build locally to confirm rendering and correct nav; fix any front-matter/title issues.

6. Prepare a PR containing the audited report and the proposed changes; include before/after nav snippets for reviewers.

Acceptance criteria

• Each prioritized tool has the minimum pages present (Overview, Quickstart, Installation, Commands, Troubleshooting).
• Each prioritized tool contains a short TL;DR block and a "For bioinformaticians" small example.
• Top-level docs/tools/.nav.yml and per-tool .nav.yml files are consistent and produce a stable navigation order.
• make sync can be run and followed by mkdocs build with no fatal errors; if any remote/IO errors occur the sync or update script should exit non-zero so CI can detect failures.

Notes & hints for implementation

• Use the template below for per-tool .nav.yml (replace <tool> and filenames as needed):

title: <Tool Name>

nav:
  - Overview: docs/index.md
  - Quickstart: docs/quickstart.md
  - Installation: docs/installation.md
  - Configuration: docs/configuration.md
  - Commands: docs/commands.md
  - Integrations: docs/integrations.md
  - Troubleshooting: docs/troubleshooting.md
  - Developer: docs/developer/

• Example TL;DR snippet to place at the top of docs/quickstart.md:

::: tip TL;DR — copy & paste
Install, configure, and run:

```bash
# install (example)
curl -sSL https://github.com/calypr/<tool>/releases/latest/download/install.sh | bash

# minimal usage
<tool> init
<tool> add data/example.bam
<tool> push

# fetch on compute node
<tool> fetch data/example.bam

:::

- Add a small "For bioinformaticians" subsection with a Snakemake or Nextflow rule demonstrating how to consume a DRS or presigned URL.

Labels & priority
-----------------
- labels: `documentation`, `enhancement`, `help wanted`
- Priority: start with `git-drs` and `syfon` (they already have active docs and are high-impact for users).

If you want I can:
- Run the audit script and add the report to this issue.
- Apply the standardized nav and templates for `git-drs` and `syfon` and open a PR with the changes.

Please indicate which option you prefer (audit-only, apply-to-git-drs+syfon, or apply-site-wide) and I will proceed.


---

## Audit report (automated)

The following audit was generated by `scripts/audit_tools_docs.py` and appended here.

# Tools documentation audit report

Scanned: `/Users/walsbr/calypr/calypr.github.io/docs/tools` — found 7 tool directories

## git-drs

- ✅ Overview — `docs/index.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/git-drs)
- ✅ Quickstart — `docs/quickstart.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/git-drs)
- ❌ Installation — `docs/installation.md`
- ❌ Configuration — `docs/configuration.md`
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ✅ Troubleshooting — `docs/troubleshooting.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/git-drs)
- ❌ Developer docs (developer/)

**Missing pages:** Installation, Configuration, Commands/API, Integrations/Examples

## syfon

- ❌ Overview — `docs/index.md`
- ❌ Quickstart — `docs/quickstart.md`
- ❌ Installation — `docs/installation.md`
- ❌ Configuration — `docs/configuration.md`
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ❌ Troubleshooting — `docs/troubleshooting.md`
- ❌ Developer docs (developer/)

**Missing pages:** Overview, Quickstart, Installation, Configuration, Commands/API, Integrations/Examples, Troubleshooting

## data-client

- ✅ Overview — `docs/index.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/data-client)
- ❌ Quickstart — `docs/quickstart.md`
- ❌ Installation — `docs/installation.md`
- ❌ Configuration — `docs/configuration.md`
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ❌ Troubleshooting — `docs/troubleshooting.md`
- ❌ Developer docs (developer/)

**Missing pages:** Quickstart, Installation, Configuration, Commands/API, Integrations/Examples, Troubleshooting

## sifter

- ✅ Overview — `docs/index.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/sifter)
- ❌ Quickstart — `docs/quickstart.md`
- ❌ Installation — `docs/installation.md`
- ❌ Configuration — `docs/configuration.md`
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ❌ Troubleshooting — `docs/troubleshooting.md`
- ❌ Developer docs (developer/)

**Missing pages:** Quickstart, Installation, Configuration, Commands/API, Integrations/Examples, Troubleshooting

## forge

- ✅ Overview — `docs/index.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/forge)
- ❌ Quickstart — `docs/quickstart.md`
- ❌ Installation — `docs/installation.md`
- ✅ Configuration — `docs/configuration.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/forge)
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ❌ Troubleshooting — `docs/troubleshooting.md`
- ❌ Developer docs (developer/)

**Missing pages:** Quickstart, Installation, Commands/API, Integrations/Examples, Troubleshooting

## grip

- ✅ Overview — `docs/index.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/grip)
- ❌ Quickstart — `docs/quickstart.md`
- ❌ Installation — `docs/installation.md`
- ❌ Configuration — `docs/configuration.md`
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ❌ Troubleshooting — `docs/troubleshooting.md`
- ✅ Developer docs (developer/)

**Missing pages:** Quickstart, Installation, Configuration, Commands/API, Integrations/Examples, Troubleshooting

## funnel

- ✅ Overview — `docs/index.md` (in /Users/walsbr/calypr/calypr.github.io/docs/tools/funnel)
- ❌ Quickstart — `docs/quickstart.md`
- ❌ Installation — `docs/installation.md`
- ❌ Configuration — `docs/configuration.md`
- ❌ Commands/API — `docs/commands.md`
- ❌ Integrations/Examples — `docs/integrations.md`
- ❌ Troubleshooting — `docs/troubleshooting.md`
- ❌ Developer docs (developer/)

**Missing pages:** Quickstart, Installation, Configuration, Commands/API, Integrations/Examples, Troubleshooting

---

Summary: 7 tools scanned, total missing standard pages: 40

Tools with missing pages:

- git-drs: 4 missing
- syfon: 7 missing
- data-client: 6 missing
- sifter: 6 missing
- forge: 5 missing
- grip: 6 missing
- funnel: 6 missing

Generated by scripts/audit_tools_docs.py