Add all remaining production-grade scaffolding from copilot instructions#2
Conversation
- Add header comment blocks (author, license, GCP metadata) and 'use utf8' to all four Perl scripts (codon.pl, longORF.pl, translate.pl, hydropathy.pl) - Add CHANGELOG.md (Keep-a-Changelog format), CODE_OF_CONDUCT.md (Contributor Covenant v2.1), CODEOWNERS, CONTRIBUTING.md (nf-core-style), NOTICE, SECURITY.md, TERMS_OF_USE.md, CITATIONS.md - Add CITATION.cff (CFF v1.2.0) and .zenodo.json for software citation and Zenodo deposit - Add pyproject.toml (PEP 517/518 with hatchling) with doc dependencies and ruff lint config - Add .python-version (3.11), .perlcriticrc (severity=3), .perltidyrc - Add Makefile with lint, tidy, test, docs, all targets - Add .nf-core.yml template config with lint-ignore rules - Add .dockerignore, Dockerfile (multi-stage ubuntu:22.04), docker-compose.yml, docker-compose.dev.yml - Add Docker helper shell scripts: docker_up.sh, docker_down.sh, docker_interactive.sh, docker_push.sh, docker_test_local.sh - Add lib/Utils.groovy and lib/WorkflowMain.groovy (nf-core-style Groovy utilities) - Add tests/ directory: fixtures/test.fasta, unit tests for all four Perl scripts, integration/pipeline.t, run_tests.sh - Add docs/api/index.md listing all four Perl scripts - Add .github/workflows/docs.yml (lint-perl, test-perl, build-docs, deploy-docs jobs) - Add .github/workflows/release.yml (multi-arch Docker push, GitHub Release, Nextflow test) - Remove environment.yml (replaced by pyproject.toml) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: bibymaths <42838835+bibymaths@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: bibymaths <42838835+bibymaths@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds missing “production-grade” scaffolding around the existing Nextflow pipeline (CI/CD, Docker, docs, tests, metadata, and community/legal files) to align the repo with nf-core-style expectations and the referenced Copilot instructions.
Changes:
- Introduces CI workflows for docs + releases, plus Docker build/push and release automation.
- Adds Perl unit tests + a Nextflow integration test harness and minimal FASTA fixtures.
- Adds repo hygiene/scaffolding: Docker/compose helpers, MkDocs API index, citation/Zenodo metadata, and community/legal docs.
Reviewed changes
Copilot reviewed 42 out of 42 changed files in this pull request and generated 19 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/translate.t | Adds unit test coverage for translate.pl. |
| tests/unit/longORF.t | Adds unit test coverage for longORF.pl. |
| tests/unit/hydropathy.t | Adds unit test coverage for hydropathy.pl. |
| tests/unit/codon.t | Adds unit test coverage for codon.pl. |
| tests/run_tests.sh | Adds a combined unit + optional Nextflow integration test runner. |
| tests/integration/pipeline.t | Adds Nextflow smoke test for -profile test. |
| tests/fixtures/test.fasta | Adds minimal FASTA fixture for tests. |
| scripts/translate.pl | Adds standard header + use utf8; (logic unchanged). |
| scripts/longORF.pl | Adds standard header + use utf8; (logic unchanged). |
| scripts/hydropathy.pl | Adds standard header + use utf8; (logic unchanged). |
| scripts/codon.pl | Adds standard header + use utf8; (logic unchanged). |
| pyproject.toml | Adds Python packaging/tooling config for docs + ruff. |
| lib/WorkflowMain.groovy | Adds nf-core-style init + citation + completion logging helpers. |
| lib/Utils.groovy | Adds param validation and summary/banner helpers. |
| environment.yml | Removes conda environment file. |
| docs/api/index.md | Adds API index page for the four Perl scripts. |
| docker_up.sh | Adds helper to build/start compose stack. |
| docker_down.sh | Adds helper to stop/remove compose stack. |
| docker_interactive.sh | Adds helper for an interactive dev shell via compose. |
| docker_push.sh | Adds helper to tag/push images to GHCR. |
| docker_test_local.sh | Adds helper to run the pipeline test profile in a local image. |
| docker-compose.yml | Adds compose definition for running the pipeline container. |
| docker-compose.dev.yml | Adds compose definition for dev (bind-mount repo). |
| Dockerfile | Adds multi-stage container build for pipeline runtime. |
| .dockerignore | Adds ignore rules to slim Docker build context. |
| Makefile | Adds lint/tidy/test/docs convenience targets. |
| .perlcriticrc | Adds Perl::Critic configuration. |
| .perltidyrc | Adds perltidy configuration. |
| .python-version | Pins Python version for tooling. |
| .github/workflows/docs.yml | Adds docs CI/CD (perl lint/test + mkdocs build/deploy). |
| .github/workflows/release.yml | Adds release pipeline (multi-arch Docker push + GitHub release + NF test). |
| .nf-core.yml | Adds nf-core lint/template metadata. |
| CHANGELOG.md | Adds a changelog following Keep a Changelog. |
| CITATION.cff | Adds CFF citation metadata. |
| .zenodo.json | Adds Zenodo deposit metadata. |
| CITATIONS.md | Adds third-party citation list. |
| CODE_OF_CONDUCT.md | Adds Contributor Covenant-based CoC. |
| CONTRIBUTING.md | Adds contribution guide and local dev instructions. |
| CODEOWNERS | Adds default code owner mapping. |
| SECURITY.md | Adds security policy + reporting instructions. |
| TERMS_OF_USE.md | Adds plain-language terms of use. |
| NOTICE | Adds notice / attribution file. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| ARG DEBIAN_FRONTEND=noninteractive | ||
| ARG NEXTFLOW_VERSION=24.04.4 | ||
|
|
There was a problem hiding this comment.
NEXTFLOW_VERSION is declared but never used, so the Docker build isn’t actually pinned to that version. Either wire this ARG into the Nextflow install step or remove it to avoid a false sense of reproducibility.
| - ./data:/pipeline/data | ||
| - ./results:/pipeline/results | ||
| environment: | ||
| - NXF_OPTS=${NXF_OPTS:--Xms512m -Xmx2g} |
There was a problem hiding this comment.
${NXF_OPTS:--Xms512m -Xmx2g} isn’t valid Docker Compose variable-default syntax (Compose uses ${VAR:-default}). As written this can cause compose parsing errors or an empty value. Change to ${NXF_OPTS:-...} (and consider quoting the default because it contains spaces).
| - NXF_OPTS=${NXF_OPTS:--Xms512m -Xmx2g} | |
| - "NXF_OPTS=${NXF_OPTS:--Xms512m -Xmx2g}" |
| - ./data:/pipeline/data | ||
| - ./results:/pipeline/results | ||
| environment: | ||
| - NXF_OPTS=${NXF_OPTS:--Xms512m -Xmx2g} |
There was a problem hiding this comment.
${NXF_OPTS:--Xms512m -Xmx2g} is invalid in Compose; use ${NXF_OPTS:-...}. Without fixing this, docker compose may fail to start the dev service.
| - NXF_OPTS=${NXF_OPTS:--Xms512m -Xmx2g} | |
| - NXF_OPTS=${NXF_OPTS:-Xms512m -Xmx2g} |
| IMAGE="ghcr.io/bibymaths/codonanalyzer" | ||
| TAG_LATEST="${IMAGE}:latest" | ||
| TAG_VERSION="${IMAGE}:$(git describe --tags 2>/dev/null || echo 'dev')" | ||
|
|
||
| echo "==> Tagging image as ${TAG_LATEST} and ${TAG_VERSION}..." | ||
| docker tag codonanalyzer-pipeline "${TAG_LATEST}" | ||
| docker tag codonanalyzer-pipeline "${TAG_VERSION}" |
There was a problem hiding this comment.
docker tag codonanalyzer-pipeline ... assumes an image named codonanalyzer-pipeline exists, but neither docker compose nor docker build in these helper scripts creates that tag by default. Either build/tag the image with that name (e.g., docker build -t codonanalyzer-pipeline .) or set image: codonanalyzer-pipeline in the compose file and reference that consistently.
| IMAGE="ghcr.io/bibymaths/codonanalyzer" | |
| TAG_LATEST="${IMAGE}:latest" | |
| TAG_VERSION="${IMAGE}:$(git describe --tags 2>/dev/null || echo 'dev')" | |
| echo "==> Tagging image as ${TAG_LATEST} and ${TAG_VERSION}..." | |
| docker tag codonanalyzer-pipeline "${TAG_LATEST}" | |
| docker tag codonanalyzer-pipeline "${TAG_VERSION}" | |
| SOURCE_IMAGE="codonanalyzer-pipeline" | |
| IMAGE="ghcr.io/bibymaths/codonanalyzer" | |
| TAG_LATEST="${IMAGE}:latest" | |
| TAG_VERSION="${IMAGE}:$(git describe --tags 2>/dev/null || echo 'dev')" | |
| echo "==> Building local image ${SOURCE_IMAGE}..." | |
| docker build -t "${SOURCE_IMAGE}" . | |
| echo "==> Tagging image as ${TAG_LATEST} and ${TAG_VERSION}..." | |
| docker tag "${SOURCE_IMAGE}" "${TAG_LATEST}" | |
| docker tag "${SOURCE_IMAGE}" "${TAG_VERSION}" |
| [tool.hatch.build.targets.wheel] | ||
| packages = ["scripts"] |
There was a problem hiding this comment.
[tool.hatch.build.targets.wheel] packages = ["scripts"] will likely fail because scripts/ is not a Python package (no __init__.py, mostly Perl). This can break uv pip install -e ".[dev]" in the docs workflow. Either create a real Python package (e.g., src/codonanalyzer/) and point Hatch at it, or remove wheel packaging and install MkDocs deps without editable-installing the project.
| docker run --rm \ | ||
| -v "$(pwd)/data:/pipeline/data" \ | ||
| -v "$(pwd)/results:/pipeline/results" \ | ||
| "${IMAGE}" \ | ||
| -profile test,docker | ||
|
|
||
| EXIT_CODE=$? | ||
|
|
There was a problem hiding this comment.
Because set -e is enabled, a non-zero exit from docker run will terminate the script before you can record $? and print the custom pass/fail message. Capture the status via docker run ... || EXIT_CODE=$? (initialise EXIT_CODE=0) or disable -e just for this command.
| docker run --rm \ | |
| -v "$(pwd)/data:/pipeline/data" \ | |
| -v "$(pwd)/results:/pipeline/results" \ | |
| "${IMAGE}" \ | |
| -profile test,docker | |
| EXIT_CODE=$? | |
| EXIT_CODE=0 | |
| docker run --rm \ | |
| -v "$(pwd)/data:/pipeline/data" \ | |
| -v "$(pwd)/results:/pipeline/results" \ | |
| "${IMAGE}" \ | |
| -profile test,docker || EXIT_CODE=$? |
| - family-names: Mishra | ||
| given-names: Abhinav | ||
| email: mishraabhinav36@gmail.com | ||
| orcid: "" |
There was a problem hiding this comment.
orcid: "" is an empty value; CFF tooling commonly expects a valid ORCID format or the field to be omitted. Consider removing the field if unknown, or supplying a valid ORCID ID/URL to avoid metadata validation failures.
| orcid: "" |
| my $syntax = system($^X, '-c', $script, '2>/dev/null'); | ||
| is($syntax, 0, "hydropathy.pl passes syntax check (perl -c)"); | ||
|
|
||
| # 3. Script exits with error when called with no arguments | ||
| my $no_args = system($^X, $script, '2>/dev/null'); |
There was a problem hiding this comment.
List-form system doesn’t interpret shell redirections, so '2>/dev/null' is passed as an argv value rather than suppressing stderr. This can cause false failures. Switch to string-form system for redirection or capture STDERR from Perl.
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Install Nextflow | ||
| RUN curl -fsSL https://get.nextflow.io | bash \ |
There was a problem hiding this comment.
Installing Nextflow via curl ...get.nextflow.io | bash without passing a version means the image will pick up whatever Nextflow is current at build time. For reproducible pipeline containers, install the specific version you intend to support (e.g., using the NEXTFLOW_VERSION ARG).
| RUN curl -fsSL https://get.nextflow.io | bash \ | |
| RUN curl -fsSL https://get.nextflow.io | NXF_VER=${NEXTFLOW_VERSION} bash \ |
| VERSION="${GITHUB_REF_NAME#v}" | ||
| # Extract the section for this version from CHANGELOG.md | ||
| python3 - <<EOF | ||
| import re, sys | ||
| with open('CHANGELOG.md') as f: | ||
| content = f.read() | ||
| pattern = r'## \[' + re.escape(VERSION) + r'\].*?(?=\n## \[|\Z)' |
There was a problem hiding this comment.
The Python heredoc references VERSION in the regex construction, but VERSION isn’t defined in the Python process (it’s only a shell variable). This will raise NameError and break releases; pass the version into Python (env var or string interpolation) before building the regex.
| VERSION="${GITHUB_REF_NAME#v}" | |
| # Extract the section for this version from CHANGELOG.md | |
| python3 - <<EOF | |
| import re, sys | |
| with open('CHANGELOG.md') as f: | |
| content = f.read() | |
| pattern = r'## \[' + re.escape(VERSION) + r'\].*?(?=\n## \[|\Z)' | |
| export VERSION="${GITHUB_REF_NAME#v}" | |
| # Extract the section for this version from CHANGELOG.md | |
| python3 - <<EOF | |
| import os, re, sys | |
| version = os.environ['VERSION'] | |
| with open('CHANGELOG.md') as f: | |
| content = f.read() | |
| pattern = r'## \[' + re.escape(version) + r'\].*?(?=\n## \[|\Z)' |
|
@copilot apply changes based on the comments in this thread |
…h build step, pyproject wheel, perltidy backups, perlcriticrc comment, unit test system() calls, run_tests.sh/docker_test_local.sh exit-code capture, zenodo/CFF empty orcid, release.yml VERSION heredoc Agent-Logs-Url: https://github.com/bibymaths/codonanalyzer/sessions/a87fdc7c-fb8c-4a13-bdb1-bc880bad44a9 Co-authored-by: bibymaths <42838835+bibymaths@users.noreply.github.com>
Applied all 19 review suggestions in commit
|
|
@copilot Fix ALL perlcritic violations listed below in scripts/codon.pl, scripts/hydropathy.pl, STRICT CONSTRAINTS:
Apply the following fixes systematically to each file: FIX 1 — "die" → "croak" (Severity 3, PBP p.283)Affects: codon.pl lines 19,20 | hydropathy.pl lines 18,19,20 | longORF.pl lines 20,21,23 | translate.pl (none listed) Change: FIX 2 — Close filehandles as soon as possible (Severity 4, PBP p.209)Affects: codon.pl line 20 | hydropathy.pl lines 18,19,20 | longORF.pl lines 20,21,22 | translate.pl lines 43,45 Pattern to apply: BEFORE (filehandle opened, used across many lines, closed far away or never):open(my $fh, '<', ... many lines of code ...close($fh); # or no close at all AFTER — wrap in a scoped block so $fh goes out of scope immediately after last use:{ Do this for EVERY filehandle open() in each file. If the filehandle data is needed later, FIX 3 — "local" variable not initialized (Severity 3, PBP p.78)Affects: codon.pl line 23 Change: FIX 4 — Regular expressions missing "/x" flag (Severity 3, PBP p.236)Affects: codon.pl lines 26,27,30,46,168,192 | hydropathy.pl lines 44,49 | longORF.pl lines 56,57 | translate.pl lines 52,53,54,58,60 Rules for adding /x:
Examples: BEFOREif ($seq =~ /^>(\S+)/) { ... } AFTERif ($seq =~ / ^ > (\S+) /x) { ... } # match FASTA header For multi-character patterns, break across lines with indentation: FIX 5 — Reused variable name in lexical scope: $n (Severity 3)Affects: codon.pl lines 105, 122 Each inner $n that shadows an outer $n must be renamed to a unique, descriptive name. FIX 6 — Mixed high and low-precedence booleans (Severity 4, PBP p.70)Affects: codon.pl line 106 Replace mixing of 'and'/'or' with '&&'/'||' (or vice versa — pick one and be consistent). BEFORE (mixed)if ($a && $b or $c) { ... } AFTER (consistent high-precedence)if ( ($a && $b) || $c ) { ... } Or if using low-precedence throughout: FIX 7 — Map block should have a single statement (Severity 3, PBP p.113)Affects: codon.pl line 131 If the map block contains multiple statements, extract the logic into a named subroutine BEFOREmy @Result = map { my $x = $_; $x =~ s/foo/bar/; $x } @input; AFTERsub transform_item { FIX 8 — Expression form of eval (Severity 5, PBP p.161)Affects: codon.pl line 179 Replace string eval with block eval. This is a Severity 5 — highest priority fix. BEFORE (string eval — dangerous, hard to analyse)my $result = eval "$some_expression"; AFTER (block eval — safe, compiled at compile time)If the expression is dynamic, restructure the logic to avoid eval entirely.If the eval is used for error trapping:my $result = eval { some_function($arg) }; If the string eval cannot be trivially replaced (e.g., it evaluates a formula string), FIX 9 — "unless" with ">=" condition (Severity 3, PBP p.99)Affects: longORF.pl line 15 Replace negative conditional forms with positive equivalents: BEFOREunless ($len >= $min) { ... } AFTERif ($len < $min) { ... } FIX 10 — Magic variable "$/" should be local (Severity 4, PBP p.81-82)Affects: longORF.pl line 53 Always localize $/ before changing it to prevent side effects: BEFORE$/ = undef; # slurp mode — affects global state AFTER — scope the change{ FIX 11 — Subroutine does not end with "return" (Severity 4, PBP p.197)Affects: hydropathy.pl subroutine "process" at line 59 Add an explicit return statement as the final statement of the subroutine. BEFOREsub process { AFTERsub process { If the sub intentionally returns nothing, add: return; FIX 12 — Deeply nested code (Severity 3)Affects: longORF.pl line 90 Extract the innermost nested block into a well-named subroutine. BEFOREfor my $seq (@seqs) { AFTERsub process_codon { for my $seq (@seqs) { FIX 13 — High complexity score (Severity 3)Affects: codon.pl (score 58), longORF.pl (score 22) For codon.pl (score 58 — very high):
For longORF.pl (score 22 — moderate): VERIFICATION INSTRUCTIONSAfter making all changes to each file, mentally verify:
|
Agent-Logs-Url: https://github.com/bibymaths/codonanalyzer/sessions/a34f3b3f-c9fc-4c53-b5a3-d3160973132d Co-authored-by: bibymaths <42838835+bibymaths@users.noreply.github.com>
Applied all 13 fixes across the four scripts in commit codon.pl (was score 58):
hydropathy.pl: longORF.pl: translate.pl: Slurp+collect approach for ASAP FH close, |
|
@copilot Fix the pyproject.toml build error: Root cause: This is a Perl + Nextflow pipeline with NO Python source package directory. Apply this fix — choose ONE of the two approaches below based on what pyproject.toml is used for: APPROACH A (RECOMMENDED) — This project has no Python code to ship as a wheel. [build-system] [tool.hatch.build.targets.wheel] [tool.hatch.build.targets.sdist] This silences the error without pretending there is a Python package. APPROACH B — If the project must be installable (e.g., for a docs build hook or CLI entry point):
[tool.hatch.build.targets.wheel] WHICH TO USE:
ALSO CHECK: In the [project] table, ensure these fields are present to avoid [project] [dependency-groups] This separates pipeline runtime deps (none) from docs/dev tooling deps, |
Agent-Logs-Url: https://github.com/bibymaths/codonanalyzer/sessions/b8ecb27e-24be-411d-a06a-e1b779f15b14 Co-authored-by: bibymaths <42838835+bibymaths@users.noreply.github.com>
Fixed in commit
|
|
@copilot Fix the MkDocs deploy error: Root cause: pymdownx (PyMdown Extensions) is not listed as a dependency in pyproject.toml, FIX 1 — Add pymdownx to pyproject.toml docs dependency group: [dependency-groups] NOTE: mkdocs-material bundles pymdownx as a dependency, but uv's dependency resolver FIX 2 — Update the docs.yml GitHub Actions workflow to install the docs group explicitly: Change the uv install step from:
To:
This ensures all docs dependencies including pymdownx are installed before mkdocs runs. FIX 3 — Verify mkdocs.yml extensions block is correctly indented. markdown_extensions:
A YAML indentation error at that line can also trigger "cannot find module" misleadingly. PRIORITY ORDER:
|
Agent-Logs-Url: https://github.com/bibymaths/codonanalyzer/sessions/6812fc48-8f68-4360-ae47-cf38efcfb3d2 Co-authored-by: bibymaths <42838835+bibymaths@users.noreply.github.com>
Fixed in commit |
Completes the full nf-core-style pipeline implementation specified in
.github/workflows/copilot-instructions.md. All infrastructure, tooling, documentation, and community files were absent.Perl Scripts
Script,Author,Date,License,GCP) anduse utf8;to all 4.plscripts — logic untouchedCommunity & Legal
CHANGELOG.md,CODE_OF_CONDUCT.md,CODEOWNERS,CONTRIBUTING.md,NOTICE,SECURITY.md,TERMS_OF_USE.md,CITATIONS.mdCitation & Metadata
CITATION.cff(CFF 1.2.0),.zenodo.json— both wired to DOI10.5281/zenodo.15384943Python Tooling
pyproject.toml(hatchling build, mkdocs plugin deps, ruff),.python-version(3.11)environment.ymlPerl Tooling
.perlcriticrc(severity 3, core theme),.perltidyrc,Makefilewithlint/tidy/test/docstargetsnf-core Compatibility
.nf-core.yml,lib/Utils.groovy,lib/WorkflowMain.groovy(parameter validation, summary logging, citation display)Docker
Dockerfile(builder: Ubuntu 22.04 + Java 17 + Perl + Nextflow; runtime: minimal copy),.dockerignoredocker-compose.yml,docker-compose.dev.yml, and 5 helper scripts (docker_up,docker_down,docker_interactive,docker_push,docker_test_local)Tests
tests/fixtures/test.fasta— minimal 2-record DNA FASTAtests/unit/*.t— oneTest::Morefile per.plscript (syntax, missing-input, output-format checks)tests/integration/pipeline.t— full pipeline smoke test on-profile testtests/run_tests.sh—prove -v tests/unit/*.tthennextflow run main.nf -profile testCI/CD
.github/workflows/docs.yml— jobs:lint-perl,test-perl,build-docs,deploy-docs.github/workflows/release.yml— jobs:docker-build-push(multi-arch amd64/arm64 → GHCR),github-release(CHANGELOG-sourced notes + asset upload),nextflow-test-on-releaseDocs
docs/api/index.md— script index with one-line descriptions