picmaker rewrite: modernize repo, decompose god module, add tests/docs/CI#23
Merged
Conversation
Implements PR 1 of the four-PR modernization plan.
Files created:
- src/picmaker/__init__.py: module docstring; __version__ resolved via
importlib.metadata("rms-picmaker") with fallback to picmaker._version,
then to "0.0.0+unknown". Placeholder __all__ (re-exports come in PR 3).
- src/picmaker/py.typed: empty marker file already declared in
[tool.setuptools.package-data] but missing on disk.
- tests/fixtures/.baseline-help.txt + .baseline-flags.txt: snapshots of
the current optparse help and flag set, captured for PR 3's argparse
migration to diff against.
Files edited:
- README.md: replaced three TODO blocks with concrete CLI and library
usage examples, plus a pickle-trust warning (pickle dispatch is
preserved per scope guard).
- pyproject.toml:
- description and keywords get real values; pyroma now scores 10/10.
- Runtime dep floors set to currently installed versions:
astropy>=7.2.0, numpy>=2.4.4, pillow>=12.2.0, scipy>=1.17.1,
rms-pdsparser>=2.0.0, rms-tabulation>=2.0.0, rms-vicar>=1.2.1.
Rationale: pip-audit reports no advisories against any of these
versions; pillow 12.2 is well above the CVE-2023-50447 cutoff
(Pillow<10.2). Only pip itself shows advisories, which are not in
scope.
- Uncommented mypy>=1.0 and added pip-audit>=2.7 to dev extras.
- Added [[tool.mypy.overrides]] with ignore_missing_imports=true for
astropy.*, PIL.*, scipy.*, vicar, pdsparser, tabulation so the
first mypy run in PR 3 doesn't drown in stub-missing errors.
- Dropped [tool.pytest.ini_options].pythonpath=["src"] (no longer
needed once the package is installed editable).
- Changed addopts --cov=src -> --cov=picmaker to match
[tool.coverage.run].source.
- Added "*/picmaker.py" to coverage omit (anticipating PR 3 where
picmaker.py collapses to a BC re-export module whose
from-X-import-Y lines would otherwise inflate coverage).
- Set [tool.coverage.report].fail_under=0 temporarily; PR 4 ratchets
to floor(measured).
- src/picmaker/picmaker.py:
- Line 3373: return IOError(...) -> raise IOError(...). The original
code constructed but did not raise the exception, so the function
silently fell through to array_to_pil() with palette data lost.
- process_images proceed scope bug: in the --movie branch,
"if proceed:" referenced a nonexistent local. Changed to
"if option_dicts[0]['proceed']:" with an assertion that all dicts
in the list agree on proceed (main() forces this invariant by
writing the same value into every dict).
Verification:
- pip install -e ".[dev]" succeeds.
- python -c "import picmaker; print(picmaker.__version__)" prints
0.1.dev7.
- pyroma . -> 10/10.
- picmaker --help renders without error; flag set identical to the
captured baseline.
- scripts/run-all-checks.sh (pyroma + pymarkdown, the currently-enabled
defaults) passes.
- pip-audit --skip-editable reports no advisories on the project's
runtime dependencies.
- All previously-importable names from picmaker.picmaker remain
importable; py.typed is now installed alongside the package.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 1: add conftest.py + tests/fixtures/
Adds the shared test scaffolding for the PR 2 safety net described in
MODERNIZATION_PLAN.md:
- tests/conftest.py with fixtures_dir / expected_dir / tiny_array.
- tests/fixtures/*.recipe.py recipes (regenerable via
tests/fixtures/regenerate_all.py) plus their generated binary outputs:
- VICAR: cassini_iss, voyager_iss, galileo_ssi_a, galileo_ssi_b
- FITS: hst_wfc3, hst_acs (5 HDUs), hst_wfpc2 (5 HDUs), nh_mvic
- PDS3: pds3_sample.IMG (512-byte records)
- PIL: small_grayscale.png, small_rgb.png, small_tiff16.tiff
- Malformed: malformed_pickle.bin, malformed_numpy.bin,
corrupt_vicar.vic, corrupt_fits.fits
- tests/fixtures/two_versions.txt for the --versions round-trip test.
Detection paths verified by running read_one_image_array against each
fixture inside the dev venv (rms-vicar, astropy, pdsparser).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 2: add unit tests for paths/geometry/pil/enhance/color/zebra
86 passing tests, 3 xfailed (xfails gate PR 3's mutable-default fix).
Coverage:
- test_paths.py: find_common_path + get_outfile (replace policies, strip,
suffix, extension swap, directory tree creation).
- test_geometry.py: circle_mask (diameters 1/3/5/8/9), slice_array (band
average, slice axes, valid mask, NaN propagation), crop_array (interior,
all-zero, axis-selective).
- test_geometry_extra.py: wrap_image (horizontal/vertical sections, named
gap color), pad_image (interior pad, no-pad short-circuit, named color),
get_size (size/scale/frame/frame_max paths), resize_image (gray + RGB),
rotate_array_rgb (all 5 named rotations + unknown raises).
- test_pil_utils.py: array_to_pil↔pil_to_array round-trip for grayscale
and RGB; documents the rescale-flag dead-code bug at picmaker.py:3029.
- test_enhance.py: get_limits (integer half-pixel extension, explicit
limits, percentile bands); _percentile_lookup (below 0, above 100,
interpolation midpoint, exact match); apply_colormap (grayscale
no-colormap, 3-entry grayscale LUT, 3-entry color LUT).
- test_color.py: ColorNames.lookup positive cases (red/GhostWhite/
whitespace-tolerant), negative cases (unknown/empty/hex/non-str), RGB
parenthetical and bracket expressions.
- test_apply_gamma.py: gamma=1 identity (2D + 3D RGB), gamma=2.0 and 0.5
exact boundary values, NaN/Inf safety.
- test_mutable_defaults.py: xfail(strict=True) gates for the mutable
defaults `strip=[]`, `pointer=['IMAGE']` in images_to_pics/get_outfile
— PR 3's fix will flip these green.
- test_zebra.py: fill_zebra_stripes left edge, right edge, in-place
modification, no-op on fully-nonzero arrays.
Documents two pre-existing bugs in tests rather than fixing them (PR 3
scope): pil_to_array L-mode rescale flag is dead-code; ColorNames hex
codes are unsupported.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 3: add integration tests for io/pipeline/cli-flag semantics
90+ new tests covering:
- test_io.py: read_one_image_array end-to-end on every instrument fixture
(cassini, voyager, galileo_ssi_a/b, hst_wfc3, hst_acs, hst_wfpc2, nh_mvic);
malformed cascade reaches IOError("Unrecognized image file format ...").
- test_tinted_colormap.py: cassini substring chain (CL1+GRN/CL1+RED/CL1+UV/
BL+GRN combo/unknown), voyager dict lookup, galileo dict lookup, NH dict
lookup, HST F606W special case, F555W wavelength inference, unknown HST
filter returns None.
- test_hst_filter_tuple_normalization.py: CL1/CL2/N/A/CLEAR* normalization
paths at picmaker.py:2314-2327.
- test_unknown_filter_warning.py: captures current print()-to-stdout
behavior + xfail gate for PR 3's logger.warning() conversion.
- test_pipeline.py: images_to_pics with default options, gamma=2.0,
percentile stretch, tint, rot90, --movie (via process_images), 16-bit
TIFF extension, --hst (ACS/WFC + WFPC2 branches).
- test_frame_max.py: --frame=512,512 --frame_max=50 on 16x16 input caps
output dims at <=256.
- test_io_extra.py: read_array on PNG/RGB/16-bit TIFF, read_pil on
PNG/TIFF, write_pil quality round-trip, PDS3 reader marked skip
pending PR 3 (pdsparser API mismatch documented inline).
- test_warning_elevation.py: corrupt_fits.fits triggers astropy warning
→ elevated to error → cascade continues to PIL → final IOError.
- test_versions_override.py: --replace=none overrides versions-line
--replace=all; two_versions.txt produces two distinct output files.
- test_alt_strip_alias.py: every kebab/snake spelling parses identically
(--alt-strip / --alt_strip, --gap-size / --gapsize, --gap-color /
--gapcolor, --alt-pointer / --alt_pointer, --trim-zeros / --trimzeros).
- test_overlap_vs_overlaps.py: --overlap=0.1 ≡ --overlaps 0.1 0.1
(byte-identical outputs).
- test_pickle_iolost_propagates_to_final_error.py: documents current
pre-PR3 pickle-IOError propagation + xfail gate for the PR 3 fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 4: add subprocess-based test_cli
Five smoke tests cover the user-facing CLI surface:
- `picmaker --help` exits 0 and lists representative flags.
- `picmaker --help`'s flag set matches tests/fixtures/.baseline-flags.txt
byte-for-byte (gates PR 3's argparse migration: same flags must remain).
- `picmaker` with no args exits 0 (no work to do).
- `picmaker /nonexistent.IMG` exits non-zero and surfaces the underlying
FileNotFoundError via sys.excepthook (verifying picmaker.py:787-789).
- `picmaker --versions tests/fixtures/two_versions.txt cassini_iss.vic`
produces two output files (_v1.jpg + _v2.tif).
test_api_compat.py is deferred to PR 3 commit 5 per MODERNIZATION_PLAN.md
(the BC re-export module only exists then).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 5: drop tiff16 driver block, add tests/test_tiff16.py
Atomically replaces the legacy `__main__`-only test scaffolding at
tiff16.py:474-680 (the `from vicar import *`, `from optparse import
OptionParser`, dead `ROTATE_DICT`, and the four `*_test()` driver
functions) with a proper pytest module.
tiff16.py now ends at line 473 with `raise IOError("Not a recognized
TIFF16 file.")` — the last legitimate production line.
tests/test_tiff16.py covers:
- 16-bit grayscale round-trip (WriteTiff16 → ReadTiff16 → array_equal).
- 16-bit RGB round-trip.
- 16-bit palette round-trip with translate=True (write-as-RGB) and
translate=False (native palette TIFF) — both currently xfail(strict)
because tiff16.py:96 uses `palette != None` instead of `is not None`,
which raises ValueError under modern numpy. PR 3 will fix this.
- `up=True` flip round-trip.
- ReadTiff16 raises IOError on a non-TIFF file.
The deletion and the new test file land in a single commit so there is no
window where the driver block exists but is unreachable from tests; PR 3's
agent can rely on tiff16.py being trim-shaped from this commit forward.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 6: snapshot the unvectorized pipeline output (63 fixtures)
Adds the baseline that PR 3's vectorization commit will diff against.
Without this, "PR 3 is byte-identical to the legacy pipeline" is
unverifiable.
Files added:
- tests/generate_snapshots.py — regenerator script. Iterates over the
7 instrument fixtures × 9 representative option combos, calls
images_to_pics with the combo's kwargs, logs each as ok/FAIL, and
emits tests/snapshots_index.py with the surviving (fixture, slug, ext)
tuples. Failures don't halt the script — they're filtered out of the
emitted index.
- tests/fixtures/expected/ — 63 generated snapshots:
- 7 fixtures × 9 combos = 63 entries.
- Combos: default, gamma2, pct5_95, colormap_red_blue, tint, rot90,
frame_128_pad, frame_max_50, twobytes_tiff.
- tests/snapshots_index.py — auto-generated index (committed so test
collection doesn't require running the generator first).
- tests/test_snapshots.py — parametrized test asserting
produced_bytes == fixture_bytes for every entry in SNAPSHOTS.
All 63 tests pass against HEAD.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 commit 7: flip ENABLE_PYTEST=true in run-all-checks.sh
PR 2 now has a real test suite (235 collected, 232 pass, 2 xfail, 1 skip
documented). Enable pytest by default in the orchestrator so a bare
`scripts/run-all-checks.sh` exercises it without needing the env flag.
CLAUDE.md's note about ENABLE_PYTEST being off by default is now stale —
PR 3/4 will rewrite CLAUDE.md alongside the larger restructure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR2 cleanup: ruff --fix tests/ + manual lint cleanups
Final cleanup pass on the test suite so `ruff check tests/` is clean:
- Auto-fixes from `ruff --fix` (I001 import sort, F401 unused pytest
import, RUF100 redundant noqa).
- Split chained `assert a and b` into separate asserts (PT018) in
test_geometry, test_geometry_extra, test_overlap_vs_overlaps,
test_tinted_colormap.
- Drop unused tuple unpacking targets (RUF059) in test_geometry,
test_geometry_extra, test_io_extra.
- Move `import pytest` to the top of test_tiff16.py (E402).
- Replace U+00D7 (×) with `x` in test_versions_override docstring (RUF002).
All 248 tests still pass; ruff (default rules from pyproject.toml) is
clean against tests/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* New Pds3Label init
* Label fix
* Split fixture-generator scripts out of tests/fixtures/
`tests/fixtures/` now contains only data (the binary/text files tests
consume); the generator scripts live in `tests/fixture_recipes/`.
- tests/fixtures/ — *.vic, *.fits, *.IMG, *.bin, *.png, *.tiff,
two_versions.txt, .baseline-*.txt, expected/.
- tests/fixture_recipes/ — 16 *.recipe.py files, regenerate_all.py,
generate_snapshots.py.
Each recipe's `OUT` is rewritten from
Path(__file__).parent / '<name>.<ext>'
to
Path(__file__).parent.parent / 'fixtures' / '<name>.<ext>'
so the output still lands in tests/fixtures/ regardless of where the
recipe lives.
generate_snapshots.py now:
- reads inputs from tests/fixtures/<name> (via FIXTURES const),
- writes outputs to tests/fixtures/expected/ (via EXPECTED const),
- emits tests/snapshots_index.py (kept next to test_snapshots.py so the
`from snapshots_index import SNAPSHOTS` import resolves without
sys.path gymnastics).
test_snapshots.py docstring and error-message references updated to
point at the new path.
Verified: regenerate_all.py + generate_snapshots.py both produce
byte-identical outputs from the new location; 248 tests pass /
1 skipped / 7 xfailed; pyroma + pymarkdown green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Drop "*/picmaker.py" from coverage omit so PR 2's baseline is real
PR 1 added the omit entry forward-looking: after PR 3 commit 5 collapses
picmaker.py into a BC re-export module, its import-time-executed
`from X import Y` lines would otherwise count as covered for free
(via `test_api_compat`'s import side-effect), inflating coverage to
~100% without exercising any logic. See §A16.
The cost in PR 2, however, is that picmaker.py *is* the codebase — the
omit hides the entire safety-net's coverage signal and makes the
"PR 2 covers the legacy pipeline" claim unfalsifiable. With the omit
gone:
Name Stmts Miss Cover
src/picmaker/__init__.py 9 5 44%
src/picmaker/colornames.py 36 1 94%
src/picmaker/picmaker.py 1402 642 52%
src/picmaker/tiff16.py 187 21 86%
TOTAL 1634 669 56%
That's a meaningful baseline against which PR 3 commit 9's vectorization
can be judged.
PR 3 commit 5 (when picmaker.py becomes a re-export shim) must re-add
the entry; the plan now lists that as an explicit step under commit 5
and the §A16 description has been updated to reflect the temporary
absence.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: apply CodeRabbit auto-fixes
Fixed 21 file(s) based on 40 unresolved review comments.
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
* Tighten 8 PR2 tests from external code review
Triage of a stylistic-review pass dropped on the PR. Most findings
contradict project conventions (no docstrings on tests per CLAUDE.md;
Sphinx :func: roles in non-autodoc files; line-number refs are
deliberate PR3-navigation aids); see PR thread for the rejection list.
Eight findings were genuine correctness/strength improvements:
- conftest.py: drop "(populated by PR 2 commit 6)" mod-history note in
the expected_dir docstring.
- test_geometry.py: test_diameter_8 asserts mask.shape == (8, 8), not
just shape[0] == 8.
- test_geometry_extra.py: test_named_pad_color asserts the exact blue
tuple (0, 0, 255) instead of "not red"; test_named_rotations_run
parametrizes (name, expected_shape) and asserts equality, so
ROT90/ROT270 vs FLIP/ROT180 dimensions are individually checked.
- test_frame_max.py: tighten the loose `<= 256` check to the exact
produced size (8, 8), with a comment explaining the frame_max%-of-
input semantics that legacy get_size implements.
- test_hst_filter_tuple_normalization.py: replace the tautological
`result is not None or result is None` self-canceling assert with a
concrete equality assertion against the wavelength-inference output
([(0,0,0),(255,60,60),(255,255,255)] at 555 nm).
- test_alt_strip_alias.py: assert byte-equality of the two output
files, not just name equality — verified locally that every alias
pair produces byte-identical jpeg output.
- test_snapshots.py + generate_snapshots.py: remove the duplicated
slug→kwargs branch in the test. generate_snapshots.py now emits a
KWARGS_BY_SLUG dict next to SNAPSHOTS in tests/snapshots_index.py,
and the test imports both — single source of truth.
- test_io_extra.py: fix the unpack in the (currently-skipped)
read_pds_labeled_image_array test from a 2-tuple to the actual
3-tuple (arr, default_is_up, filter_info), so the test runs cleanly
when PR3 unskips it.
All 248 tests still pass (1 skipped, 7 xfailed); ruff clean; default
orchestrator green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Refresh PDS3 skip reason after Pds3Label constructor patch
Commits db0ea1e + 8f11fce switched read_pds_labeled_image_array's
constructor call from the removed PdsLabel.from_file() back to
PdsLabel() — but the body still iterates `for node in label`
expecting `node.name` and references `pdsparser.PdsOffsetPointer`,
which now raises AttributeError instead of the original symptom.
Update the test's skip reason so the inline rationale matches the
current failure mode; the test still skips for the same overall
"PR 3 will rewrite the reader" reason.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: CodeRabbit <noreply@coderabbit.ai>
…#7) * PR3: decompose picmaker into per-concern modules + argparse + cleanup Splits the ~3500-line `src/picmaker/picmaker.py` god module into purpose-specific leaf modules and an `instruments/` subpackage, swaps the CLI from optparse to argparse, and applies the hygiene fixes the modernization plan catalogues. The legacy import path `from picmaker.picmaker import X` still resolves to the same function object as `from picmaker import X` (identity-equal — gated by `tests/test_api_compat.py`). Modules created - cli.py — argparse rewrite of the optparse `main()`. Every flag spelling preserved, including the underscore aliases (`--alt_strip`, `--alt_pointer`, `--gapsize`, `--gapcolor`, `--trimzeros`, `--frame_max`). `--versions FILE` semantics, the 13-check mutex / value-validity chain, and `sys.excepthook`-based error reporting all match the legacy module byte for byte (verified against `tests/fixtures/.baseline-flags.txt`). - pipeline.py — `find_common_path`, `process_images`, `images_to_pics`. - io.py — `read_image_array`, `read_one_image_array`, `read_pds_labeled_image_array`, `read_pil`, `read_array`, `get_outfile`. The pickle branch's `except IOError as e: raise e` is gone; the VICAR / FITS magic-byte branches now also tolerate `OSError` so a missing-file path falls through to the final `OSError("Unrecognized image file format: ...")` instead of leaking the per-stage error message. - enhance.py — `fill_zebra_stripes`, `get_limits`, `_percentile_lookup`, `apply_colormap`, `apply_gamma`. Algorithms preserved verbatim so PR 2's byte-identical snapshots stay green. - geometry.py — `circle_mask`, `slice_array`, `crop_array`, `rotate_array_rgb`, `get_size`, `resize_image`, `wrap_image`, `pad_image`, and the private `_get_size_for_*`/`_resize_one_image` helpers. - color.py — `tinted_colormap` only; delegates to `instruments.lookup` and re-exports the `RGB_BY_NM`/`RFUNC`/`GFUNC`/`BFUNC` constants from `_rgb.py`. - pil_utils.py — `array_to_pil`, `pil_to_array`, `write_pil`, and the `_one_pil_to_array` helper. - _filters.py — `FILTER_DICT` + `filter_image`. - _rgb.py — leaf module holding the wavelength→RGB tables; placed outside `color.py` so `instruments/hst.py` can import without a cycle (`color → instruments → hst → _rgb (leaf)`). - instruments/{cassini,voyager,galileo,hst,nh}.py — each implements the 4-method protocol (`detect_vicar`, `detect_fits`, `matches`, `tint_for`) so the dispatch tables in `io.read_one_image_array` and `color.tinted_colormap` are data, not control flow. Backward compatibility - `src/picmaker/picmaker.py` collapses to a re-export shim with an explicit `__all__` covering 53 legacy names (every public function, the documented private helpers, the sibling-package re-exports `ColorNames` / `WriteTiff16` / `ReadTiff16` / `VicarImage` / `VicarError`, and the four instrument dicts `VOYAGER_ISS_DICT`, `NH_MVIC_DICT`, `GALILEO_SSI_DICT`, `GALILEO_SSI_NAMES`). - `src/picmaker/__init__.py` mirrors the public surface so `from picmaker import images_to_pics` works. - `tests/test_api_compat.py` asserts identity-equality between the two paths. Hygiene fixes (per MODERNIZATION_PLAN §"Bug fixes and autofixes") - Shebangs dropped from `tiff16.py` and `colornames.py`. - Mutable defaults flipped: `strip=None` and `pointer=None` with the list normalized inside `images_to_pics` / `get_outfile`. - `from struct import *` in `tiff16.py` → `from struct import pack, unpack`. - Pillow deprecation: `Image.FLIP_*` / `Image.ROTATE_*` → `Image.Transpose.*` (25 sites in `tiff16.py`). - Lose-traceback fix: `except IOError as e: raise e` deleted from the pickle branch of `read_one_image_array`. Behavior change documented in `test_pickle_iolost_propagates_to_final_error.py`. - `warnings.warn` call in `get_outfile` now passes `UserWarning` explicitly. - `print('******UNKNOWN FILTER:', ...)` in the HST tint dispatcher is now `logger.warning(...)`. CLI migration notes - Argparse uses lowercase `usage:` (optparse used capital `Usage:`). Updated `tests/test_cli.py::test_help_exits_zero_and_lists_flags` to accept either form. - Argparse does NOT accept the mixed `--option=N M` form for `nargs=2` options (it consumes only the `N` and leaves `M` as positional). Adjusted `tests/test_overlap_vs_overlaps.py` to use the space-separated form. - Flag set matches `tests/fixtures/.baseline-flags.txt` exactly. Documentation, packaging, and tooling - `pyproject.toml`: - `[project.scripts]` flipped from `picmaker.picmaker:main` to `picmaker.cli:main`. - `[tool.coverage.run].omit` adds `*/picmaker.py` so the BC re-export module's `from X import Y` lines don't inflate coverage. - `[tool.ruff.lint.per-file-ignores]` adds `A002` for the preserved `filter` kwarg, `N806` for the preserved camelCase locals in `pipeline.py`/`geometry.py`, and broad legacy-style ignores for `tiff16.py` and `colornames.py`. - `[[tool.mypy.overrides]]` adds permissive blocks for `picmaker.tiff16`, `picmaker.colornames`, and pytest test modules. - `scripts/run-all-checks.sh` defaults to `ENABLE_RUFF_CHECK=true` and `ENABLE_MYPY=true`. The mypy step runs only against `src/` because the bare `tests/test_*.py` filename pattern doesn't match mypy's module-pattern syntax without a `tests/__init__.py`. - `docs/module.rst` switches from a single `automodule:: picmaker` to per-leaf-module autodocs. Verification - `scripts/run-all-checks.sh -s` passes: ruff check, mypy, pytest (261 passed, 1 skipped, 2 xfailed), pyroma, pymarkdown. - `picmaker --help | grep -oE -- '--[a-z_-]+' | sort -u` matches `tests/fixtures/.baseline-flags.txt`. - `picmaker --versions tests/fixtures/two_versions.txt --directory /tmp/out tests/fixtures/cassini_iss.vic` produces both `cassini_iss_v1.jpg` and `cassini_iss_v2.tif`. Sphinx `-W` still fails on 4 pre-existing infrastructure warnings (`code_of_conduct.md` include path, `index.rst` title underline, two `toctree.not_included` entries); those are addressed in PR 4 along with `ENABLE_SPHINX=true`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Annotate legacy tiff16 + colornames; mypy now strict-clean on tests too The PR3 baseline hid the WriteTiff16 / ReadTiff16 / ColorNames.lookup untyped-call errors behind per-module mypy ignores. Replace those with real annotations so `mypy src tests` is strict-clean without `disable_error_code` for `no-untyped-call`. Source changes - src/picmaker/tiff16.py: type WriteTiff16, ReadTiff16, and my_assert. - src/picmaker/colornames.py: type ColorNames.lookup. - Remove the per-callsite `# type: ignore[no-untyped-call]` shims from io.py / pil_utils.py / enhance.py / geometry.py. - pyproject.toml: drop `no-untyped-call` from the picmaker.tiff16 / picmaker.colornames mypy override. - src/picmaker/_rgb.py: type RGB_BY_NM as NDArray[np.float64] and the three Tabulation splines as Tabulation instead of Any. - src/picmaker/cli.py: drop modification-history references from `_separate_files_and_dirs`, `_normalize_and_validate`, and `main()` docstrings; expand `_normalize_and_validate`'s docstring with proper Args: / Returns: sections. Test changes - tests/test_api_compat.py: add `-> None` to every test function. - tests/test_mutable_defaults.py: type the `_default` helper. - tests/test_enhance.py: annotate the `tiny_array` fixture parameters. - tests/test_io.py, tests/test_geometry_extra.py: replace bare `tuple` with `tuple[str, str, str]` / `tuple[int, ...]`. - tests/test_cli.py, tests/test_alt_strip_alias.py, tests/test_overlap_vs_overlaps.py, tests/test_versions_override.py: parametrize CompletedProcess with `[str]`. - tests/test_unknown_filter_warning.py: parametrize `pytest.CaptureFixture` with `[str]`. - tests/test_io_extra.py: split the tuple-unpack so the typed-None return doesn't trip mypy. - tests/test_color.py: add a localized `# type: ignore[arg-type]` for the intentional non-string lookup in `test_non_string_raises_typeerror`. - tests/snapshots_index.py + tests/fixture_recipes/generate_snapshots.py: type SNAPSHOTS / KWARGS_BY_SLUG / COMBOS / `_generate_one`. Build script - scripts/run-all-checks.sh: mypy step now reads `src tests`. Verification - `scripts/run-all-checks.sh -s` with ruff, mypy, pytest, pyroma, pymarkdown all green. - 261 tests pass (1 skipped, 2 xfailed); CLI baseline unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Docstrings: Args: -> Parameters: per project rules; add test docstrings Section 6 of `.cursor/rules/python_best_practices.mdc` explicitly mandates Google-style docstrings using ``Parameters:`` (not the ``Args:`` variant that Google's original guide allows). Bring every PR-3-touched docstring in line: Source modules (15 sites): - src/picmaker/{cli,color,enhance,geometry,io,pil_utils,pipeline}.py - src/picmaker/_filters.py - src/picmaker/instruments/__init__.py Tests: - tests/test_cli.py: docstrings for `_run`, every test function. - tests/test_overlap_vs_overlaps.py: expand `test_overlap_and_overlaps_byte_identical` to a black-box-testable docstring explaining the wrap-path invocation. - tests/test_unknown_filter_warning.py: docstrings for both test functions describing the logger expectations and the negative capsys assertion. - tests/test_api_compat.py: docstrings for every identity-equality assertion test, including the package-level smoke test. No behavior changes; ruff/mypy/pytest still green (261/1/2). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix Sphinx warnings + drop tiff16/colornames mypy override Sphinx - Restore CODE_OF_CONDUCT.md at the repo root (accidentally removed in commit 9f55adf). docs/code_of_conduct.md's `:include:` directive needed it. - docs/index.rst: extend the title underline to match "Welcome to the Documentation for rms-picmaker!" (46 chars, not 42) so docutils stops emitting WARNING: Title underline too short. - docs/index.rst: add `contributing` and `code_of_conduct` to the master toctree so those documents are no longer orphaned (toc.not_included warnings). - scripts/run-all-checks.sh: flip ENABLE_SPHINX from `false` to `true` so the documentation build is part of the default check set and any future warning is surfaced as a hard failure. Update the ENABLE_MYPY docstring entry to match the actual default (true). Mypy - Drop the per-module override for `picmaker.tiff16` and `picmaker.colornames`. The remaining errors that surfaced are real: - src/picmaker/colornames.py: the lookup() method reused the local variable name `test` for two incompatible types (`str` and `re.Match | None`). Renamed the regex result to `match` so each variable has a single, narrowable type. Also corrected `if test == None:` to `if match is None:`. - src/picmaker/geometry.py: - `Image.NEAREST` / `Image.LANCZOS` → `Image.Resampling.NEAREST` / `Image.Resampling.LANCZOS` (the non-deprecated PIL spelling). - `list.append(_resize_one_image(...))` was a real bug — calling the unbound `list` class method instead of `result.append(...)`. The line-level `# type: ignore[call-arg]` is gone; the call now appends to the local `result` list. Verification - `scripts/run-all-checks.sh -s` is fully green: ruff, mypy, pytest (261 / 1 skipped / 2 xfailed), pyroma, sphinx, pymarkdown. - Only one `# type: ignore` remains in the project, in tests/test_color.py for a deliberate non-string lookup that exercises the TypeError branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Remove "legacy" / "backwards compatibility" wording from documentation Project rule .cursor/rules/python_best_practices.mdc §4 forbids modification-history comments and §6 forbids backwards-compatibility references in docstrings. Bring every docstring / inline comment in the picmaker package and rendered docs in line with that. Source modules - src/picmaker/_rgb.py: docstring describes what the tables are for and why this module imports nothing from the rest of the package (cycle-prevention) rather than citing a pre-PR-3 line range. - src/picmaker/__init__.py: docstring describes the public surface and points at the two top-level entry points. - src/picmaker/picmaker.py: docstring states what the module is (an alternate import path that re-exports the public API) without the words "legacy" or "backward compatibility". - src/picmaker/cli.py: docstring describes the CLI entry point and the three error-handling layers; the obsolete file-level `# ruff: noqa: A002` is gone (pyproject already has the per-file ignore). - src/picmaker/pipeline.py: module + images_to_pics docstrings drop the BC framing; the `results` declaration comment now explains WHY `results` lives outside the per-filename loop (reuse path). - src/picmaker/color.py: module docstring drops the "callers that imported them from this module before PR 3" line; the filter normalization comment now describes the rule itself. - src/picmaker/pil_utils.py: 8-bit grayscale comment now describes why the rescale flag is ignored, not what the legacy code did. - src/picmaker/instruments/{cassini,voyager,galileo,hst,nh}.py: drop "Source: picmaker.py (pre-PR-3) lines N-M" headers; expand every public function's docstring with Parameters: and Returns: sections describing the actual behavior (and Raises: where it raises). HST's `tint_for` now documents the per-detector wavelength-scaling rules in a single bullet list. Documentation - docs/module.rst: drop the "legacy ... backward-compatibility re-export shim" sentence; the package overview now just lists what's documented below. No behavior changes; ruff, mypy, pytest, pyroma, sphinx (-W), pymarkdown all green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* PR4: activate CI, switch publish to OIDC, add User Guide
CI (.github/workflows/run-tests.yml)
- Re-enable pull_request / push / schedule / workflow_dispatch triggers.
- Lint job runs ruff check, mypy, sphinx -W, pymarkdown, pip-audit on
the project's declared deps. Python pinned to 3.10 to match
target-version = "py310".
- Test job re-enabled with matrix 3.10 / 3.11 / 3.12 / 3.13 and the
existing codecov upload (3.13 only, non-fork).
Publish workflows (publish_to_pypi.yml, publish_to_test_pypi.yml)
- Switch to PyPI Trusted Publishers / OIDC: add permissions.id-token:
write, drop password / user secrets, bind each job to its
environment (pypi / testpypi). A pre-merge manual step on the PyPI
side is required (documented in CONTRIBUTING.md).
Coverage (pyproject.toml)
- Ratchet [tool.coverage.report].fail_under from 0 to 59
(floor of the 59.24% measured against the full pytest run on the
post-PR-3 codebase).
User Guide (docs/user_guide.rst + docs/_static/user_guide/*)
- New top-level RST page wired into docs/index.rst toctree with all
12 required sections: overview, installation, quick start, full
CLI reference (one table per --help group), input-format cascade,
per-instrument tints (Cassini, Voyager, Galileo, HST, NH MVIC),
output formats, enhancement / geometry pipelines, --versions form,
programmatic usage, troubleshooting.
- 23 example thumbnails generated under docs/_static/user_guide/ by
tests/fixture_recipes/generate_user_guide_thumbnails.py, modelled
on PR 2's generate_snapshots.py so the gallery stays reproducible.
- docs/conf.py enables html_static_path so the gallery is copied
into the Sphinx build.
CLI / doc drift gate (tests/test_cli.py)
- New test_user_guide_documents_every_cli_flag asserts every long
flag in tests/fixtures/.baseline-flags.txt appears in the User
Guide. A new picmaker flag with no docs now fails CI.
CONTRIBUTING.md
- "Releasing to PyPI" section with the five-step OIDC trusted-publisher
setup (PyPI + TestPyPI + GitHub Environments + tag-push release).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Raise test coverage from 59% to 90% and ratchet fail_under
Coverage gains by module
- cli.py: 5% -> 91% (test_cli_unit.py — in-process tests for
_build_parser, _separate_files_and_dirs, _normalize_and_validate, main()).
- io.py: 50% -> 87% (test_io_branches.py + test_pds3_reader.py +
the read_pds_labeled_image_array port to the current pdsparser API).
- pipeline.py: 50% -> 87% (test_pipeline_branches.py — mutex checks,
movie mode, --proceed, HST mosaic, --versions PDS3 label branch).
- pil_utils.py: 66% -> 98% (test_misc_branches.py — 16-bit RGB path,
PIL list handling, parent-dir creation).
- enhance.py: 69% -> 90% (trim_zeros, footprint, below/above/invalid
paints, named two-stop colormap, uniform-input limit fallbacks).
- geometry.py: 70% -> 90% (wrap_ratio wide/tall, frame_max cap,
rot90/180/270/fliptb, crop_array 2-D + uniform input).
- instruments: per-module 67-83% -> 96-100% (parametrised Cassini
filter chain, Voyager/Galileo/NH detect-fail predicates, HST
POL0L/POL0S/FQUV/FQCH4 pinning paths).
- tiff16.py: 86% -> 88% (RGB write, 3-D grayscale, big-endian
byteorder, --up, transpose=ROTATE_90).
- __init__.py: 71% -> 100% (pragma: no cover on the version-metadata
fallback that only fires when the package isn't installed).
Code change: src/picmaker/io.py::read_pds_labeled_image_array
- Port to the current pdsparser API: the function previously referenced
`pdsparser.PdsOffsetPointer` and `node.name` / `node.value`, which no
longer exist (the function raised AttributeError on every PDS3 file
in the test suite). The rewrite uses the dict-style `label[key]` API
with the companion `<pname>_offset` / `<pname>_unit` keys that
pdsparser now emits for tuple pointers. Pointer-value forms
covered: attached int, detached str, tuple-with-filename. Unit is
RECORDS by default; `<BYTES>` is recognised.
Coverage gate
- pyproject.toml: ratchet [tool.coverage.report].fail_under from 59 to
90. Total project coverage is now 90.31% across 466 tests
(+204 over the previous baseline).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* CI: run tests on every branch, not just main
Drop the `branches: [main]` filter from both `pull_request` and `push`
triggers so the workflow fires on any branch. The schedule and
workflow_dispatch entries are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Drop Python 3.10; expand CI matrix to macOS and Windows
astropy 7.x requires Python >= 3.11, which collides with the
`requires-python = ">=3.10"` declaration and trips the resolver on
Python 3.10 ("Could not find a version that satisfies the requirement
astropy>=7.2.0"). Picmaker only uses `astropy.io.fits.open()` and the
standard HDU access surface, but the pin already captured 7.2.0 from
the dev venv and lowering it would be a step backwards on security
fixes.
pyproject.toml
- `requires-python = ">=3.11"` (was >=3.10).
- Drop `Programming Language :: Python :: 3.10` classifier.
- `[tool.ruff].target-version = "py311"` (was py310).
.github/workflows/run-tests.yml
- Lint job pinned to Python 3.11 to match target-version.
- Test matrix: drop 3.10; add macOS and Windows runners so the
cross-platform behaviour the README/classifiers promise is now
actually exercised. Matrix is 3.11 / 3.12 / 3.13 x
ubuntu / macos / windows (= 9 jobs).
CONTRIBUTING.md / docs/user_guide.rst
- Python 3.10+ -> Python 3.11+ in the compat copy.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Iterate label.dict instead of label.keys() in PDS3 reader
`pdsparser.Pds3Label` implements `__getitem__`, `__contains__`, `keys()`,
`items()`, etc. but not `__iter__`. Python's old-style iteration
protocol then falls back to `obj[0]`, `obj[1]`, …, which raises
`KeyError: 0` at the first probe — that's why `for key in label:`
crashes on a freshly constructed label.
The class exposes `label.dict` directly (a plain `dict` assigned in
`Pds3Label.__init__`), so iterating *it* is both clean and removes the
`noqa: SIM118` we were carrying for the `label.keys()` workaround.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Use label.dict directly throughout the PDS3 reader
`Pds3Label` proxies every dict operation through to its underlying
`.dict` attribute, which is a plain `dict`. Pull it out once into a
local `label_dict` and use it directly — `label_dict[k]`,
`label_dict.get(k, default)`, `pname in label_dict` — instead of going
through the Pds3Label proxy for each access. No behaviour change, just
clearer that the function is operating on a plain dict.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Move PyPI release docs out of CONTRIBUTING.md into RELEASING.md
CONTRIBUTING.md is contributor-facing — release-engineering setup is
maintainer-internal and doesn't belong there. Split the OIDC trusted-
publisher steps into a new repo-root RELEASING.md (the conventional
location for Python project release docs).
Also fix the URLs while moving: a brand-new project that has never
been uploaded has no per-project settings page on PyPI, so the
previous instructions linked to a 404. The pending-publisher form at
`pypi.org/manage/account/publishing/` (and the TestPyPI equivalent)
is the right entry point for a first release; PyPI promotes the
pending publisher to a regular one on the first successful upload.
- CONTRIBUTING.md: drop the "Releasing to PyPI" section.
- RELEASING.md: new file with the one-time-setup steps (using the
pending-publisher URL) plus the cut-a-release checklist.
- publish_to_pypi.yml / publish_to_test_pypi.yml: comment references
CONTRIBUTING.md -> RELEASING.md.
- scripts/run-all-checks.sh / run-tests.yml: include RELEASING.md in
the pymarkdown scan.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Review pass: spelling, Sphinx role, docstrings, regex
Apply the still-valid findings from the latest review and skip the
rest with rationale.
Applied
- docs/user_guide.rst: replace British "colour"/"coloured" with
American "color"/"colored" (9 occurrences) to match the rest of the
user-facing docs.
- tests/fixture_recipes/generate_user_guide_thumbnails.py: cross-
reference :func:`picmaker.pipeline.images_to_pics` from the
_generate_one docstring; add a docstring to main() covering purpose,
side effects, and return value; drop the now-redundant
`from __future__ import annotations` (requires-python is 3.11+ so
every annotation syntax in this file is native).
- tests/test_cli.py: widen the user-guide flag-extraction regex to
`r'--[a-z0-9_-]+'` so numeric flags such as `--16` are matched when
the baseline is regenerated to include them.
Skipped
- Sphinx role conversion for `vicar.VicarImage`, `astropy.io.fits.open`,
`PIL.Image.open`, `detect_vicar`, `detect_fits`, `rms-pdsparser`,
`^IMAGE`, `--pointer`, tuple field names. The project's
`intersphinx_mapping` covers only python / numpy / matplotlib, so
adding `:func:` / `:class:` roles for unresolvable symbols would emit
reference-target-not-found warnings that the `-W` Sphinx build
promotes to fatal errors.
- Renaming HERE / FIXTURES / OUT_DIR to _HERE / _FIXTURES / _OUT_DIR.
The file is a standalone script (not imported as a module) and the
module-level ALL_GALLERIES already uses the all-caps non-underscored
convention; splitting the convention adds noise without semantic
benefit. The _generate_one helper is underscored because it is
genuinely module-private.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* CI: include OS in the test job name; drop the py prefix
`Test rms-picmaker (py3.11)` becomes
`Test rms-picmaker (ubuntu-latest, 3.11)` so the matrix dimension is
visible at a glance in the checks list (relevant now that the matrix
runs on ubuntu/macos/windows).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix Windows test: use tmp_path for nonexistent-file probe
`test_falls_through_to_unrecognized_format_error` hardcoded the POSIX
path `/nonexistent/...`, which on Windows is not absolute. astropy's
FITS reader inside `read_one_image_array`'s cascade rejects the path
with "Local file URL is not absolute" before the cascade ends, so the
test fails with the wrong exception type.
Construct the non-existent path via `tmp_path / 'does_not_exist.IMG'`
so the absolute form is OS-native — same idiom already used in
`tests/test_io_branches.py::test_unrecognized_missing_file_raises`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Modernization pass from codebase + test critiques (closes #10)
Drives `CODEBASE_CRITIQUE.md` and `TEST_SUITE_CRITIQUE.md` to "fixed" or
"issue filed" on every item that didn't require a major-version bump.
Twelve follow-up issues (#9-#20) carry the deferred work.
## Production fixes
- pil_utils._one_pil_to_array now honours rescale=True for 'L' mode
PIL images, returning a float in [0, 1] instead of the raw uint8
array. Closes #10.
- tiff16:281: missing `raise` on the version-byte mismatch (the
function silently accepted non-TIFF files with a valid 'II'/'MM'
byte order but wrong version field). Regression test added.
- tiff16 WriteTiff16 `palette != None` → `palette is not None` (numpy
ambiguous-truth bug). Removed the two strict-xfail palette tests
that pinned the bug.
- ReadTiff16 + WriteTiff16 file-handle leaks: both functions now wrap
the open() in a `with` block. Surfaced by adding
filterwarnings = ["error"] to pytest (ResourceWarning was the
smoking gun).
- colornames.ColorNames.lookup uses ast.literal_eval (not `eval`),
raises KeyError on partial regex matches, and no longer shadows the
builtin `tuple` local. Two new tests pin the boundary.
- io.read_one_image_array chains the cascade-end OSError from
ExceptionGroup('No reader matched', cascade_errors) so per-reader
failure causes survive for diagnosis.
- pipeline.process_images: assert→raise ValueError for the movie-mode
proceed-consistency check (assert is stripped under python -O).
- pipeline.images_to_pics: traceback.print_tb → logger.exception
(deterministic ordering under pytest -n auto).
- pipeline.find_common_path: switched to os.path.commonpath so the
Windows path-separator bug is gone.
- cli.main: print(this_dir)/print(dirpath) → logger.info (matches the
pipeline verbose path).
- __init__.py: except Exception → except PackageNotFoundError on the
importlib.metadata fallback.
- io.get_outfile: suffix annotation widened to `str | None` to match
the documented None-accepting behaviour.
- _filters.filter_image docstring corrected to reflect the actual
KeyError-on-unknown-name semantics.
## API changes
- New picmaker.options.PicmakerOptions dataclass owns all
post-normalization mutex / value-validity checks via validate();
cli._normalize_and_validate and pipeline.images_to_pics both
delegate to it. The legacy kwargs surface on images_to_pics is
unchanged.
- New picmaker.io.ReadResult NamedTuple (array3d, default_is_up,
filter_info) returned by the reader cascade. Positional unpacking
still works; .array3d-style attribute access now also works.
- New picmaker.io.FilterInfo type alias for the
(inst_host, inst_id, filter_name) | None union.
- pipeline.images_to_pics `filter=` kwarg renamed to `filter_name=`;
the CLI --filter flag is unchanged (dest='filter_name'). The
PDS3-label local `filter_name` was renamed to `pds_filter_name` so
it no longer shadows the parameter.
## Ruff cleanups
- Removed per-file ignores for colornames.py (was E701, E711, N801,
W291, W293, RUF012, SIM102, A001) and most of tiff16.py (was E701,
N802, N806, E501, E711, W291, W293, SIM115; only N802 remains for
legacy WriteTiff16 / ReadTiff16 casing).
- Removed A001/A002/A003 ignores for cli.py / pipeline.py /
options.py once the `filter` → `filter_name` rename landed.
- All remaining per-file ignores have an inline reason.
## Tooling
- pytest filterwarnings = ["error"] (with one targeted Pillow
`Image.getdata` ignore tracked in #9).
- bandit and vulture uncommented in [project.optional-dependencies].dev
and wired into CI; both clean (bandit suppresses B301/B403 for the
documented pickle-fallback path).
- publish_to_pypi.yml / publish_to_test_pypi.yml now build on Python
3.11 (lowest supported) instead of 3.12.
- run-tests.yml push: trigger scoped to `branches: [main]` so feature
branches don't pay for the 9-cell matrix on every dev push.
- docs/module.rst gains a "Options" section for PicmakerOptions.
## Test-suite hygiene
- Split tests/test_misc_branches.py (842 lines) and
tests/test_io_branches.py (283 lines) into nine focused files
(test_pil_utils_branches.py, test_enhance_branches.py,
test_geometry_branches.py, test_filters_branches.py,
test_instruments_branches.py, test_tiff16_branches.py,
test_package_init.py, test_io_cascade.py,
test_pds3_reader_branches.py).
- Removed `from __future__ import annotations` from 24 files.
- Migrated `from picmaker.picmaker import X` to `from picmaker import
X` in 17 files (test_api_compat.py stays — it's the BC verifier).
- Tightened existence-only asserts: test_io.py / test_pipeline.py /
test_io_extra.py now check exact shapes instead of `.ndim == 3` or
`width > 0`.
- Added regression tests for the tiff16 version-byte fix and the
colornames partial-match / injection-payload rejection.
## Final check status
- pytest: 471 passed
- ruff: clean
- mypy strict: clean (79 source files)
- bandit: clean
- vulture: clean
- sphinx -W: clean
- All snapshot tests still byte-identical.
## Filed follow-up issues (with labels)
- #9 Cleanup / Useful / Medium — Pillow getdata migration + WriteTiff16 leak follow-up
- #10 Bug / Important / Easy — _one_pil_to_array rescale (FIXED HERE)
- #11 Big Project / Important / Medium — Prune picmaker.picmaker BC shim exports
- #12 Big Project / Important / Hard — Split images_to_pics + complete RORO migration
- #13 Cleanup / Minor / Easy — Derive VICAR/FITS_INSTRUMENTS from one list
- #14 Cleanup / Minor / Medium — Rename picmaker.io to avoid stdlib shadow
- #15 Cleanup / Minor / Easy — Make instrument FILTER_DICT private
- #16 Cleanup / Useful / Easy — Stop mutating argparse.Namespace
- #17 Cleanup / Useful / Medium — Standardize on pathlib.Path
- #18 Enhancement / Minor / Medium — Vectorize fill_zebra_stripes
- #19 Enhancement / Useful / Easy — Expand CI lint matrix to 3.11/3.12/3.13
- #20 Enhancement / Useful / Medium — Magic-byte dispatch for read_one_image_array
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add Developer Guide; move RELEASING content into it; add API source links
## New: docs/developer_guide.rst
A maintainer-facing complement to the User Guide. Eight sections:
1. Repository overview — purpose, layout, who reads what.
2. Module-by-module description — one paragraph per leaf module
under `src/picmaker/`, every public symbol linked via Sphinx
cross-reference.
3. Pipeline flowchart — Mermaid diagram from CLI entry through to
`write_pil`.
4. Major functions in prose — covers the CLI entry, option
validation, reader cascade, path planning, per-image pipeline,
enhancement / geometry / color / PIL bridges. Cross-refs all the
way through.
5. Running the test suite.
6. Adding a new instrument — full step-by-step:
- The four-function protocol (`detect_vicar`, `detect_fits`,
`matches`, `tint_for`).
- Per-step instructions: create module, register in
`instruments/__init__.py`, fixture recipe, cross-instrument
test, per-helper unit tests, snapshot append, user-guide
update, check-suite run.
- When-to-break-protocol notes covering Cassini's tint chain
helper and HST's wavelength-inference branches.
7. Releasing to PyPI — moved verbatim from RELEASING.md (one-time
trusted-publisher setup + cut-a-release steps).
8. Where to look next — pointers to user guide, module reference,
GitHub issues, critique audits, contributor workflow.
## docs/module.rst
Source-code links added at the top of each section, pointing at the
relevant file under github.com/SETI/rms-picmaker. New sections for
the previously-undocumented package surface:
- `picmaker` (top-level package)
- `picmaker.picmaker` (BC shim)
- `picmaker._rgb` (wavelength → RGB tables)
- `picmaker.instruments.*` (each per-mission module)
- `picmaker.tiff16`
- `picmaker.colornames`
CLI automodule gains `:private-members: _build_parser,
_separate_files_and_dirs, _normalize_and_validate` so the private
helpers exercised by `tests/test_cli_unit.py` appear in the rendered
API.
## docs/conf.py
- Intersphinx mappings for PIL, astropy, sphinx, and pytest so
external `:func:` / `:class:` / `:mod:` references resolve under
`sphinx-build -n`.
- `nitpick_ignore` for sibling-package types (`vicar.VicarImage`,
`vicar.VicarError`, `tabulation.Tabulation`), module-level dunders
(`__all__`, `__version__`), and a small set of internal data /
helper names that autodoc cannot pick up cleanly (legacy
`colornames.ColorNames`, the per-instrument `FILTER_DICT` /
`FILTER_NAMES` constants without docstrings, the `_rgb` splines,
the private `cli._normalize_and_validate`).
- `autodoc_default_options['show-inheritance'] = True`.
## RELEASING.md
Replaced with a five-line pointer to the new
`docs/developer_guide.rst` §7. CI workflow comments updated to point
at the new location. `pymarkdown scan` continues to lint the file.
## Source-side cross-reference fixes
To satisfy `sphinx-build -n -b html`, qualified bare names in
docstrings:
- `instruments/{galileo,nh,voyager,hst}.py`: `:data:`FILTER_DICT``
etc. → `:data:`picmaker.instruments.galileo.FILTER_DICT`` etc.
- `instruments/hst.py`: `:data:`RFUNC`` etc. →
`:data:`picmaker._rgb.RFUNC``.
- `_filters.py`: `:data:`FILTER_DICT`` →
`:data:`picmaker._filters.FILTER_DICT``.
Other doc-rule alignment:
- `tiff16.WriteTiff16` and `tiff16.ReadTiff16` docstrings converted
from the legacy `Inputs:` / `Return:` shape to Google style
(`Parameters:` / `Returns:`). Fixes the long-standing docutils
"Unexpected indentation" warning.
- American spelling pass: `colour` → `color`, `coloured` →
`colored`, `grey` (in prose) → `gray`, `honour(s)` → `honor(s)`,
`normalise(d)` → `normalize(d)`, `behaviour` → `behavior`,
`organis(ed)` → `organiz(ed)`. The X11 color-name table in
`colornames.py` keeps the legacy `grey` aliases (they are literal
X11 names, not English prose).
- `instruments/__init__.py`: `VICAR_INSTRUMENTS` / `FITS_INSTRUMENTS`
/ `ALL_INSTRUMENTS` gain `list[ModuleType]` annotations and
`#:`-style attribute docstrings so autodoc picks them up.
- `__init__.py` module docstring uses fully-qualified targets
(`picmaker.pipeline.images_to_pics`) so the references resolve
outside the `picmaker` namespace.
## Final check status
- pytest: 471 passed.
- ruff: clean.
- mypy strict: clean.
- bandit: clean.
- vulture: clean.
- `sphinx-build -W -b html`: clean.
- `sphinx-build -n -b html`: clean.
- pymarkdown: clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Split Developer Guide into per-section files; drop RELEASING.md
## New layout
The monolithic `docs/developer_guide.rst` is now an index page with a
nested toctree. The major sections live as separate files under
`docs/dev/`:
- `docs/dev/repository_overview.rst` — purpose + repository tree.
- `docs/dev/module_layout.rst` — module-by-module description with
per-symbol Sphinx cross-references.
- `docs/dev/pipeline.rst` — Mermaid flowchart + prose walk-through of
CLI → reader cascade → enhancement → geometry → write.
- `docs/dev/running_tests.rst` — pytest invocation, snapshot
regeneration.
- `docs/dev/adding_an_instrument.rst` — the four-function protocol
and the seven-step recipe.
- `docs/dev/releasing.rst` — moved from RELEASING.md verbatim (one-
time trusted-publisher setup + cut-a-release flow).
`docs/developer_guide.rst` is now a slim index page with the toctree
plus the "Where to look next" section (pointers to the API
reference, user guide, GitHub issues, and the two critique audits).
## RELEASING.md removed
The file is deleted. The CI step list in
`.github/workflows/run-tests.yml` and the optional-scan list in
`scripts/run-all-checks.sh` were updated to stop referencing it. The
two publish workflow comments now point at
`docs/dev/releasing.rst` instead.
## Critique against `.cursor/rules/documentation.mdc`
- American spelling: clean.
- Sphinx cross-references for every API symbol in narrative: present
in the new files (audited; the bare-name section titles like
`:mod:`picmaker.cli`` are kept because they remain useful links in
the rendered TOC).
- `sphinx-build -W -b html`: clean.
- `sphinx-build -n -b html`: zero warnings.
- `pymarkdown scan` over `docs/ .cursor/ README.md CONTRIBUTING.md`:
clean (the deleted `RELEASING.md` was dropped from the scan list).
- Per-page TOC entries appear correctly in the rendered
`developer_guide.html` (`toctree-l2` entries for each `dev/*.rst`).
## Full check status
- pytest: 471 passed.
- ruff: clean.
- mypy strict: clean.
- bandit: clean.
- vulture: clean.
- sphinx -W: clean.
- sphinx -n: zero warnings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Address reviewer findings
Verified each comment against current code; applied the still-valid
fixes. Skipped CODEBASE_CRITIQUE.md / TEST_SUITE_CRITIQUE.md items
and anything related to pickle or path traversal per reviewer
guidance. Skipped one further item (hst.py :data: → :func: for the
Tabulation splines) because Tabulation instances are module-level
data even though they are callable; :data: is the correct role.
## Production source
- `docs/user_guide.rst:458`: `--filter` table row now shows
`dest = filter_name` to match the actual argparse binding.
- `src/picmaker/colornames.py`: added a class docstring for
`ColorNames` and a full Google-style docstring for
`ColorNames.lookup` (parameters / returns / raises / accepted
input forms). Narrowed the return annotation from `Any` to
`tuple[int, int, int]` and normalized list-style `[r, g, b]`
parses to a tuple before returning so the two bracketing styles
give identical output. Removed the now-unused `typing.Any` import.
- `src/picmaker/instruments/galileo.py`, `.../nh.py`, `.../voyager.py`:
wrapped the long lines in the `FILTER_NAMES` description and the
`Raises: KeyError` bullets so every line fits in 90 columns.
- `src/picmaker/options.py`: dropped the redundant
`self.filter_name is not None` test in `validate()`; the field is
typed `str` with default `'NONE'`, so a direct case-folded compare
is enough.
- `src/picmaker/pipeline.py::process_images`: added an empty-list
guard in movie mode (`raise ValueError('movie mode requires at
least one option_dict')`) before indexing `option_dicts[0]`.
- `src/picmaker/pipeline.py::find_common_path`: replaced the
length-based root check with `os.path.splitdrive` so Windows
drive-only roots (`'C:\\'`) are recognized as "no useful prefix"
alongside POSIX `/`.
- `src/picmaker/tiff16.py::WriteTiff16`: validate `byteorder` against
`{'native', 'little', 'big'}` (case-insensitive) and raise
`ValueError` otherwise — previously any non-`'little'` value
silently produced a big-endian file.
- `src/picmaker/io.py::read_one_image_array`: open the FITS file via
`with pyfits.open(...) as hdulist:` so the handle is always
closed, even when downstream HST mosaic / `obj` branches raise.
Removed the now-redundant trailing `hdulist.close()`.
## Tests
- `tests/test_enhance_branches.py::test_get_limits_with_footprint_filter`:
replaced `hi <= 200 + 0.5` with the precise `lo == 0 and hi == 0`
(the 3×3 median of a single-outlier array is uniformly 0).
- `tests/test_filters_branches.py::test_filter_image_blur`: use a
non-uniform input (one bright pixel against black) and assert
blur actually changes pixel values, so a no-op `blur` would fail.
- `tests/test_pil_utils.py::test_rescale_grayscale_returns_float_in_unit_range`:
replaced the dtype/max checks with an exact
`np.testing.assert_array_equal` against the precisely-computed
expected normalized array (first pixel 0.0, the rest 1.0 due to
the input being deliberately outside the `[0, 1]` rescale
contract).
- `tests/test_pipeline_branches.py::test_process_images_movie_failure_raises`:
tightened `pytest.raises((OSError, Exception))` to
`pytest.raises(OSError, match='Unrecognized image file format')`.
- `tests/test_tiff16_branches.py`: every test now reads the file
back with `ReadTiff16` and asserts pixel-wise round-trip equality
via `np.testing.assert_array_equal`. Replaced the
`(8, 8) or (8, 8, 1)` shape OR with a single canonical shape via
`np.squeeze`. `transpose=ROTATE_90` asserts against
`np.rot90(arr, 1)`; `up=True` asserts against `np.flipud(arr)`.
- `tests/test_color.py::test_rgb_brackets_expression`: updated to
expect a tuple (matching the new `ColorNames.lookup` normalization)
and replaced the stale "eval result" comment with one that
references `ast.literal_eval`.
## Fixture-recipe internal-name convention
- `tests/fixture_recipes/generate_user_guide_thumbnails.py`:
`HERE` / `FIXTURES` / `OUT_DIR` renamed to `_HERE` / `_FIXTURES`
/ `_OUT_DIR` and every consumer updated. The script is invoked
only from the command line and from `regenerate_all.py`, so the
underscore prefix makes their module-private status explicit.
## Validation
- pytest: 471 passed.
- ruff: clean.
- mypy strict: clean.
- bandit / vulture: clean.
- `sphinx-build -W` and `sphinx-build -n`: clean (zero warnings).
- pymarkdown: clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix test_paths Windows separator expectations
`find_common_path` delegates to `os.path.commonpath`, which returns
the platform-native separator. On Windows
`os.path.commonpath(['/foo/bar/x', '/foo/bar/y'])` returns
`'\\foo\\bar'`, not `'/foo/bar'`, so the two `TestFindCommonPath`
assertions that hard-coded forward slashes were failing on the
windows-latest CI cell.
Wrap the expected values in `os.path.normpath(...)` so they match
the platform separator the function actually returns. The behaviour
is unchanged on POSIX (`os.path.normpath('/foo/bar') == '/foo/bar'`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Address reviewer findings (round 2)
Verified each comment against the current code. Every item the
reviewer flagged was still present.
Notable verification correction:
``test_get_limits_trim_zeros_with_mask`` — the reviewer suggested
`hi == 100, lo == 99`. Tracing `get_limits` shows the actual values
are `lo == 100, hi == 101` (after the all-zero exterior is trimmed,
the inner 6x6 is uniform 100 except for one masked cell, so
`array_min == array_max == 100` and the equal-values integer
branch returns `(value, value + 1)`). The exact-equality assertions
in the patch use the actual values.
## Documentation
- `docs/dev/module_layout.rst`: inline-literal API symbols in the
BC-shim paragraph (`ColorNames`, `ReadTiff16`, `WriteTiff16`,
`VicarError`, `VicarImage`, `GALILEO_SSI_DICT`,
`GALILEO_SSI_NAMES`, `NH_MVIC_DICT`, `VOYAGER_ISS_DICT`) replaced
with Sphinx `:class:` / `:func:` / `:data:` roles.
- `docs/dev/pipeline.rst`: the bare `fits.open` reference in the
reader-cascade paragraph now uses `:func:`astropy.io.fits.open``.
The bare `tint_for` and `write_pil` mentions in the "Color, filter,
and PIL bridges" paragraph were promoted to cross-references
(`:func:`picmaker.instruments.cassini.tint_for`` as the
canonical concrete example, and a `~picmaker.pil_utils.write_pil`
back-reference).
- `docs/user_guide.rst`: three "Colour used for..." table rows under
the enhancement options changed to American "Color used for...".
## Tests
- `tests/test_enhance_branches.py`:
* `test_get_limits_trim_zeros_with_mask`: the relational
`lo == hi - 1` is replaced with explicit `lo == 100, hi == 101`.
* `test_get_limits_with_trim`: the computed
`lo == arr[4:-4, 4:-4].min() - 0.5` style assertions are replaced
with literal `lo == 67.5, hi == 187.5` (the inner 8x8 block of
`arange(256).reshape(16, 16)` runs from 68..187).
* `test_apply_colormap_named_two_stop`: shape-only assertion is
replaced with `np.testing.assert_allclose` against the
analytically-computed expected RGB volume (red channel
`1 - arr/7`, blue channel `arr/7`, green channel zero) plus
explicit dtype and shape checks.
- `tests/test_pipeline_branches.py`: the two inline `import logging`
statements are removed; the module-level stdlib import block now
has a single `import logging` placed alphabetically with the
other stdlib imports.
## Fixture recipe
- `tests/fixture_recipes/generate_user_guide_thumbnails.py`: the
`_generate_one` helper's `ext` and `out_dir` parameters are now
keyword-only (`def _generate_one(slug, fixture_name, kwargs, *,
ext, out_dir)`); the single call site in `main()` passes them by
name.
## Validation
- pytest: 471 passed.
- ruff: clean.
- mypy strict: clean.
- bandit / vulture: clean.
- `sphinx-build -W` and `sphinx-build -n`: clean (zero warnings).
- pymarkdown: clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Address reviewer findings (round 3)
Tighten typing surface in picmaker.io, expose the public API in the
docs, and correct a stale comment that mis-described mypy's coverage of
tests/.
- src/picmaker/io.py: introduce ObjectSelector type alias and replace
Any in read_image_array / read_one_image_array /
read_pds_labeled_image_array signatures with concrete unions
(str/Sequence for filename, str|PathLike|None for labelfile). Narrow
read_pil's return to Image.Image | list[Image.Image] and read_array's
to NDArray[Any] (dtype genuinely varies); bridge the still-Any
array_to_pil / pil_to_array helpers with typing.cast. Replace the
broad TypeError handler in read_pds_labeled_image_array with an
early isinstance check that preserves the original "Invalid index
type" message.
- tests/test_io_cascade.py, tests/test_io_extra.py: assert
isinstance(img, Image.Image) where the tests treat read_pil's result
as a single PIL image, narrowing the new union.
- docs/module.rst: add a "Public API" section above the leaf-module
sections that lists every name in picmaker.__all__ via autofunction
/ autoclass, organized by topic. Bare data exports without their
own docstrings (FILTER_DICT, RGB_BY_NM, RFUNC/GFUNC/BFUNC) are
pointed at by prose to avoid autodata inheriting the type's stub
docstring (e.g. "dict() -> new empty dictionary...").
- scripts/run-all-checks.sh: rewrite the comment above the mypy
invocation, which incorrectly claimed tests/ is not type-checked.
Verified by the (src + tests) file count in the mypy success line.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Doc rewrite + critique-file untrack
- Remove CODEBASE_CRITIQUE.md, CODEBASE_REVIEW.md, and
TEST_SUITE_CRITIQUE.md from git tracking (the files stay in the
working tree for local reference) and add them, the local
modernization-plan drafts, and CLAUDE.md to .gitignore so they don't
re-enter the index. Drop the two doc cross-references that used to
point at them (docs/dev/repository_overview.rst tree listing and
docs/developer_guide.rst "where to look next" section).
- User Guide (docs/user_guide.rst) rewritten as an end-user document.
Drops the "without reading the source code" framing, fixes the Node
name to "PDS Ring-Moon Systems Node", adds the pipx install
alternative, removes the dev venv recipe and the dependency list
(those belong in the dev guide), converts the eleven wide CLI
reference tables to bullet lists and drops the dest column, strips
every code-level reference (module paths, function names, exception
types, return-value shapes, package-internal names), and uses bare
re-exported names like `images_to_pics` instead of full
picmaker.pipeline / picmaker.io paths. The 23 thumbnail image
references are gone -- the underlying _static/user_guide/*.jpg files
were 1x1-pixel placeholders that rendered as black squares.
- Developer Guide refreshed:
* docs/dev/repository_overview.rst: fixed Node name, added a
"Setting up a development environment" section with clone / venv
/ `pip install -e ".[dev]"` / `bash scripts/run-all-checks.sh`,
and neutralized the picmaker.py tree-comment to "Alternate import
path".
* docs/dev/module_layout.rst: dropped the "prose below" phrasing and
the picmaker.picmaker "BC shim" / "pre-PR-3 1.x API" section in
favor of a single line on the top-level package.
* docs/dev/pipeline.rst: removed the "prose walk-through" /
"surrounding prose" / "Major functions in prose" phrasings and
documented that the Mermaid diagram renders client-side as
natively zoomable inline SVG (mermaid_output_format = 'raw' in
conf.py); added :align: center.
- Cleanup of the orphan user-guide thumbnails:
docs/_static/user_guide/* (23 files) and
tests/fixture_recipes/generate_user_guide_thumbnails.py removed.
The recipe is no longer referenced from CI or regenerate_all.py.
- Test fixes from a fresh round of inline review comments:
* tests/test_enhance_branches.py: widen the below_color / above_color
sentinel assertions to compare the full RGB triplet via
np.testing.assert_allclose, not just one channel.
* tests/test_io_cascade.py: remove the two redundant inline
`from PIL import Image as PILImage` shadows; use the top-level
`Image` import for the `.new(...)` calls.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Revert .gitignore additions
The previous commit added local-only files (CLAUDE.md, the critique
docs, MODERNIZATION_PLAN drafts) to .gitignore. That was unrequested;
back it out. The files remain untracked but will now show up in
`git status` like any other untracked path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Clean up ruff per-file-ignores
Drop five of the six per-file ignores in pyproject.toml by fixing the
underlying violations. Behaviour is preserved; the public-API symbols
(rotate_array_rgb, the io.py reader cascade, the cli.py validator)
keep the same call shape, return types, and error messages, and the
full 472-test suite plus mypy --strict pass against both src/ and
tests/.
- RUF005 (io.py, pipeline.py): 8 sites of `(N,) + shape` rewritten as
`(N, *shape)` -- both forms produce byte-identical numpy shape
tuples.
- N806 (pipeline.py, geometry.py): the four camelCase locals
`arrayRGB` / `arraysRGB` / `panelsRGB` / `quadsRGB` renamed to
snake_case. They were function-local in `images_to_pics` and on
`rotate_array_rgb`, never exported, and every caller passes them
positionally. `rotate_array_rgb`'s param `arrayRGB` is renamed
array_rgb and the `# noqa: N803 -- preserved kwarg name` shim
alongside it is deleted (the package is pre-1.0 `0.1.dev20` so no
external kwarg-name contract exists).
- SIM102 (cli.py, geometry.py): 4 nested-if blocks collapsed into
single `if X and Y` guards. The cli.py case (the hst/band/bands
mutex chain) is preserved by the existing `pytest.raises(...,
match='hst and band'|'band and bands')` tests in
test_cli_unit.py.
The lone remaining ignore is `tiff16.py = ["N802"]`. The misleading
"modernisation pass" wording is replaced with the real reason:
WriteTiff16 / ReadTiff16 are public-API entries pinned by
tests/test_api_compat.py and are still PascalCase in the documented
call signature. Tracked for a future deprecation cycle in #22.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add happy-path no-warnings test and CI snapshot freshness check
- Replace mtime comparison with content comparison in test_versions_override
to avoid flakiness on coarse-mtime filesystems.
- Add test_happy_path_no_warnings to guard against spurious WARNING-level
log records on the default images_to_pics path.
- Add a Snapshot freshness CI step (ubuntu-latest + py3.13) that re-runs
the generator and asserts git diff is clean, catching silent drift in
tests/fixtures/expected/ and tests/snapshots_index.py.
- Emit the snapshots_index.py with type annotations so regeneration is
byte-identical to the committed file.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix Sphinx -W: drop html_static_path now that _static/ is gone
The user-guide thumbnail purge in 43239de removed every file under
docs/_static/, but conf.py still pointed html_static_path at the
directory. Sphinx then emits "html_static_path entry '_static' does not
exist", which the CI build promotes to an error via -W. Nothing in the
docs sources references _static/... any more, so the path is dropped
entirely rather than recreating an empty directory.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Caution Review failedPull request was closed or merged during review WalkthroughThe PR restructures rms-picmaker as a properly architected Python package with a public API, modular CLI, comprehensive documentation, and modern CI/CD. It raises the minimum Python version to 3.11, implements OIDC-based PyPI publishing, introduces modular CLI and pipeline components, defines a standardized instrument-detection interface across five providers (Cassini, Galileo, HST, New Horizons, Voyager), and adds extensive test coverage for all new functionality and branching paths. ChangesPackage API Definition and Core Refactoring
Documentation and Build Infrastructure
Test Suite and Fixtures
Estimated Code Review Effort🎯 5 (Critical) | ⏱️ ~120 minutes Possibly Related PRs
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR merges the cumulative work of branch
picmaker_rewriteintomain. Treat it as one changeset — every commit is part of the same modernization effort. The work spans repo packaging, a complete decomposition of the legacypicmaker.pygod module, a comprehensive test suite with byte-identical snapshot regression coverage, full user / developer / API documentation, and an activated CI pipeline that runs lint, type-checking, security scans, doc builds, and the test matrix across nine OS/Python combinations on every push.The legacy import path
from picmaker.picmaker import …continues to work and resolves to the same function objects asfrom picmaker import …(identity-equal, gated bytests/test_api_compat.py).Repo modernization
The repository now follows the standard SETI/PDS Rings Node
rms-*project layout with a realsrc/-layout package,pyproject.tomlas the single source of truth, and reproducible dev workflow.pyproject.tomlcarries the realdescription, keywords, runtime-dependency floors pinned to currently-installed versions (astropy>=7.2.0,numpy>=2.4.4,pillow>=12.2.0,scipy>=1.17.1, plus the sibling SETI packages), and dev extras (mypy, bandit, vulture, pip-audit).pyroma .scores 10/10.requires-pythonraised to 3.11+ (astropy 7.x requires it; thetarget-version = "py311"in ruff and the classifier list match).[project.scripts]flipped frompicmaker.picmaker:maintopicmaker.cli:main.setuptools_scmwrites the package version tosrc/picmaker/_version.pyat build time; the runtime resolves it viaimportlib.metadata.version('rms-picmaker')with a_version.pyfallback.src/picmaker/py.typedis now actually on disk (the marker was declared in[tool.setuptools.package-data]but missing).scripts/run-all-checks.shis the canonical orchestrator. Defaults are nowENABLE_RUFF_CHECK=true,ENABLE_MYPY=true,ENABLE_PYTEST=true,ENABLE_SPHINX=true, plus pyroma and pymarkdown.New directory structure / file structure for the old god module
src/picmaker/picmaker.pywas ~3,500 lines containing the entire library — CLI, pipeline, readers, writers, image enhancement, geometry, color handling, instrument-specific filter dispatch, and a TIFF16 driver. It has been decomposed into per-concern leaf modules and aninstruments/subpackage:The new instrument subpackage replaces the previous block of
if instrument == 'CASSINI': ... elif instrument == 'VOYAGER': ...control flow scattered through the god module. Each instrument is now a self-contained module implementing a uniform 4-method protocol —detect_vicar,detect_fits,matches,tint_for— and the dispatch tables inio.read_one_image_arrayandcolor.tinted_colormapare data rather than control flow. The protocol is documented indocs/dev/adding_an_instrument.rstalong with a step-by-step recipe for adding a new mission.src/picmaker/picmaker.pycollapsed from 3,493 lines to a 103-line re-export shim. The pipeline algorithms (limits, gamma, colormap, etc.) are preserved verbatim so the byte-identical snapshot tests stay green; the structural changes are the module split, the argparse migration, and the bug fixes (below).Documentation
The project now has a full Sphinx documentation set under
docs/:docs/user_guide.rst) — overview, install (pip / pipx), quick start, full CLI reference organized by--helpgroup, input-format cascade, per-instrument tints (Cassini, Voyager, Galileo, HST, NH MVIC), output formats, the enhancement and geometry pipelines, the--versionsform, programmatic usage examples, troubleshooting.docs/developer_guide.rst+docs/dev/*.rst) — index page with a nested toctree of six per-section files:dev/repository_overview.rst— purpose, layout, dev-environment setup.dev/module_layout.rst— module-by-module description with per-symbol Sphinx cross-references.dev/pipeline.rst— Mermaid flowchart and walk-through from CLI entry through the reader cascade, enhancement, geometry, and PIL write.dev/running_tests.rst— pytest invocation, snapshot regeneration.dev/adding_an_instrument.rst— the 4-method protocol + 7-step recipe.dev/releasing.rst— one-time PyPI trusted-publisher setup + cut-a-release flow.docs/module.rst) — autodoc per leaf module, plus a "Public API" section listing every name inpicmaker.__all__organized by topic. Source-code GitHub links at the top of each section.Parameters:/Returns:/Raises:sections wrapped at 90 columns (project convention from.cursor/rules/python_best_practices.mdc).docs/conf.pyconfigures intersphinx mappings (python, numpy, matplotlib, PIL, astropy, sphinx, pytest), nitpick handling for sibling-package types, andshow-inheritancefor autodoc.Both
sphinx-build -W -b htmlandsphinx-build -n -b htmlare zero-warning clean and run in CI.Test suite
[tool.coverage.report].fail_under = 90enforced bypytest --cov. Per-module coverage ranges from 87% (io.py, pipeline.py) to 100% (__init__.py).pytest filterwarnings = ["error"]is inpyproject.toml, so any newResourceWarning/DeprecationWarningfails CI.The test suite is organized in three layers, each pinning the rewrite against the pre-refactor behavior in a different way:
Black-box tests
Subprocess-driven CLI tests in
tests/test_cli.pyand the alias /--versions/--overlap/ mutex tests. The key compatibility gate istests/fixtures/.baseline-flags.txt— a snapshot of every long flag emitted by the legacyoptparsebuild ofpicmaker --help, captured before the rewrite. The argparse rewrite has to match this set byte for byte, andtests/test_cli.py::test_help_flags_match_baselineruns that check on every CI build..baseline-help.txtis a companion snapshot of the full help text.tests/test_cli.py::test_user_guide_documents_every_cli_flagalso asserts every flag appears in the User Guide, so a new CLI flag with no docs now fails CI.White-box tests
Per-module unit tests targeting each branch in each leaf module. The post-decomposition coverage gains are visible at the module level —
cli.py5%→91%,io.py50%→87%,pipeline.py50%→87%,pil_utils.py66%→98%,enhance.py69%→90%,geometry.py70%→90%, per-instrument modules 67–83%→96–100%. Tests assert exact numeric outputs (analytically-computed RGB volumes, exact(lo, hi)tuples fromget_limits) rather than loose shape checks, so future refactors cannot silently change pipeline math.Integration tests
tests/test_pipeline.pyexercisesimages_to_picsend-to-end against every instrument fixture with combinations of gamma, percentile stretch, tint, rotation,--movie, 16-bit TIFF output, and--hstACS/WFC/WFPC2 branches.tests/test_io.pyandtests/test_io_cascade.pyexercise the reader cascade across every supported format, including the malformed-input paths that must reach the cascade-endOSError("Unrecognized image file format ...").Snapshot regression — byte-identical compatibility with the original picmaker
tests/fixtures/expected/contains 63 byte-identical output snapshots (7 instrument fixtures × 9 representative option combos:default,gamma2,pct5_95,colormap_red_blue,tint,rot90,frame_128_pad,frame_max_50,twobytes_tiff). The snapshots were generated before the god-module decomposition.tests/test_snapshots.pyis parametrized acrossSNAPSHOTSand asserts byte equality of every produced output against the committed fixture. Any change to the enhancement, geometry, or color pipeline that perturbs a single output byte fails this test.The generator (
tests/fixture_recipes/generate_snapshots.py) is itself committed and re-run in CI by the Snapshot freshness check step. The check runs the generator and assertsgit diff --exit-codeis clean ontests/fixtures/expected/andtests/snapshots_index.py. Silent drift in the pipeline (e.g., a pillow-version-driven JPEG quantizer change) will fail this step.Compatibility enforcement going forward
tests/test_api_compat.py— asserts identity-equality betweenpicmaker.Xandpicmaker.picmaker.Xfor all 53 public names, so the legacy import path can never silently diverge.tests/fixtures/.baseline-flags.txt— pinned viatest_help_flags_match_baseline.tests/test_happy_path_no_warnings.py— runs the defaultimages_to_picspath and asserts no WARNING-level log records are emitted, so accidentally introducing a deprecation chirp in the default path fails CI.Bugs fixed
Latent / dormant bugs surfaced and fixed during the rewrite. None are scope creep; each was found by the new test suite, mypy strict, ruff, bandit, vulture, or a Sphinx build:
read_one_image_arraypalette branch:return IOError(...)constructed the exception but never raised it — fell through silently toarray_to_pil()with palette data lost. Now raises.process_images--moviebranch:if proceed:referenced a nonexistent local. Reworked to readoption_dicts[0]['proceed']with anassert-now-raise ValueErrorinvariant check (the previousassertwas stripped underpython -O).read_pds_labeled_image_arraywas raisingAttributeErroron every PDS3 file — it called the removedpdsparser.PdsOffsetPointerandnode.name/node.valueAPI. Rewritten to use the current dict-stylelabel[key]plus the<pname>_offset/<pname>_unitcompanion keys.tiff16.py: missingraiseon the version-byte mismatch (the function silently accepted non-TIFF files with a validII/MMbyte order but wrong version field). Regression test added.tiff16.WriteTiff16:palette != NoneraisedValueErrorunder modern numpy (ambiguous truth). Fixed tois not None.tiff16.WriteTiff16/ReadTiff16: file-handle leaks —open(...)was not wrapped inwith. Fix surfaced bypytest filterwarnings = ["error"]catching theResourceWarning.tiff16.WriteTiff16: silently produced a big-endian file for anybyteordervalue other than'little'. Now validated against{'native', 'little', 'big'}(case-insensitive) and raisesValueErrorotherwise.io.read_one_image_array: FITS handle was not closed when the HST mosaic /objbranches raised. Now opened in awith pyfits.open(...) as hdulist:block.colornames.ColorNames.lookupusedeval(...). Replaced withast.literal_eval. Also: stopped shadowing the builtintuplelocal; raisesKeyErroron partial regex matches instead of returning a misleading partial-color tuple. Two new tests pin the boundary.pil_utils._one_pil_to_arrayignoredrescale=Truefor'L'-mode PIL images. Now returns a float in[0, 1].pipeline.find_common_pathhad a Windows path-separator bug — switched toos.path.commonpath, then later switched toos.path.splitdrive-based root detection so Windows drive-only roots (C:\\) are recognized alongside POSIX/.geometry.py: a straylist.append(_resize_one_image(...))was calling the unbound class method onlistinstead ofresult.append(...). The errant# type: ignore[call-arg]covering this was removed.io.read_one_image_arraylose-traceback:except IOError as e: raise edeleted from the pickle branch (replaced with the cascade-endOSErrorchained fromExceptionGroup('No reader matched', cascade_errors)so per-reader failure causes survive for diagnosis).strip=[]andpointer=['IMAGE']inimages_to_pics/get_outfileare nowNonewith the list normalized inside the function.Image.FLIP_*/Image.ROTATE_*→Image.Transpose.*(25 call sites intiff16.py);Image.NEAREST/Image.LANCZOS→Image.Resampling.NEAREST/Image.Resampling.LANCZOS.from struct import *intiff16.pynarrowed tofrom struct import pack, unpack.warnings.warninget_outfilenow passesUserWarningexplicitly.print('******UNKNOWN FILTER:', ...)→logger.warning(...).traceback.print_tb(...)→logger.exception(...)in the pipeline (deterministic ordering underpytest -n auto).tiff16.pyandcolornames.py.Public API changes
from picmaker import Xis now the canonical import path.picmaker.__init__re-exports the 53 public names.from picmaker.picmaker import Xcontinues to work and resolves to the same object (identity-equal).picmaker.options.PicmakerOptionsdataclass — owns all post-normalization mutex / value-validity checks viavalidate(). Bothcli._normalize_and_validateandpipeline.images_to_picsdelegate to it.picmaker.io.ReadResultNamedTuple(array3d, default_is_up, filter_info)returned by the reader cascade. Positional unpacking still works;.array3dattribute access now also works.picmaker.io.FilterInfotype alias for the(inst_host, inst_id, filter_name) | Noneunion.picmaker.io.ObjectSelectortype alias replacesAnyinread_image_array/read_one_image_arraysignatures.pipeline.images_to_picsfilter=kwarg renamed tofilter_name=(it was shadowing the builtin). The CLI--filterflag is unchanged viadest='filter_name'.[project.scripts]entry point:picmaker = picmaker.cli:main(waspicmaker.picmaker:main).--alt_strip,--alt_pointer,--gapsize,--gapcolor,--trimzeros,--frame_max). Two argparse semantics differences are documented in tests:usage:(optparse usedUsage:).--option=N Mform fornargs=2; use the space-separated form.pil_to_arrayfor'L'-mode PIL images withrescale=Truenow returns a float array in[0, 1]instead of the rawuint8array. (Behavior change to match documented contract.)ColorNames.lookup: list-style[r, g, b]parses are now normalized to a tuple before returning, so the two bracketing styles ((r, g, b)and[r, g, b]) produce identical output. Return annotation narrowed fromAnytotuple[int, int, int].Issues filed during the process
All open in the GitHub issue tracker, labeled by
A-*(kind),Priority N,Effort N,B-*(area):pytest filterwarnings = ['error']; track Pillowget_flattened_datamigration (the remaining Pillowgetdataignore).rescale=Trueis ignored bypil_to_arrayfor'L'-mode PIL images. (Fix landed in this PR; tracker kept for visibility.)picmaker.picmakerBC shim exports once external callers have migrated.pipeline.images_to_picsandcli.maininto smaller helpers and complete the RORO migration.instruments.{VICAR,FITS}_INSTRUMENTSfrom a single canonical list rather than two parallel sequences.picmaker.ioto avoid shadowing the stdlibiomodule.FILTER_DICT/FILTER_NAMESprivate; rely on the BC aliases.argparse.Namespaceinsidecli._normalize_and_validate(return a new object instead).pathlib.Paththroughoutcli.py,pipeline.py,io.py.enhance.fill_zebra_stripes(replace the Python pixel loop).read_one_image_arrayto avoid the try-each-reader cascade (perf).tint_for: the NICMOS polariser branch is unreachable in practice; either remove or document.WriteTiff16/ReadTiff16to snake_case (the lone remaining ruffN802per-file ignore).CI
.github/workflows/run-tests.ymlis fully active onpull_request,pushtomain, a weeklyschedule, andworkflow_dispatch. Two jobs run on every event:Lint job (
ubuntu-latest, Python 3.11):ruff check src tests— explicit rule setE, F, W, I, UP, B, SIM, C4, A, N, PT, RUF.mypy src tests—strict = trueacross both source and test trees.bandit -c pyproject.toml -r src -q.vulture src.sphinx-build -W -b html docs docs/_build— promotes any Sphinx warning to a fatal error.pymarkdown scan docs/ .cursor/ README.md CONTRIBUTING.md.pip-audit .against the project's declared dependencies (not the active env).Test job — matrix of
[ubuntu-latest, macos-latest, windows-latest] × [3.11, 3.12, 3.13]= 9 jobs:pytest --cov=picmaker --cov-report=xml -n auto tests— enforcesfail_under = 90.tests/fixture_recipes/generate_snapshots.pyand assertsgit diff --exit-codeis clean.All currently green.
Release process
.github/workflows/publish_to_pypi.ymlandpublish_to_test_pypi.ymluse PyPI Trusted Publishers / OIDC — noPYPI_API_TOKENsecret is stored. The publish jobs fire onrelease: publishedand bind to GitHub Environments namedpypi/testpypiso PyPI can verify the OIDC claim.Outstanding manual steps before the first publish (documented in
docs/dev/releasing.rst):rms-picmaker, the GitHub owner / repo (case-sensitive), workflow filenamepublish_to_pypi.yml, environment namepypi. The pending-publisher form is needed because the project has never been uploaded — the per-project settings page does not exist yet and will return 404. PyPI promotes the pending publisher to a regular one on the first successful upload.publish_to_test_pypi.ymland environment nametestpypi. TestPyPI is a separate registry with its own name reservations.pypi(andtestpypi).git tag v0.1.0 && git push --tags, create a GitHub Release pointing at the tag — the workflow fires automatically. Dry-run against TestPyPI via the manualworkflow_dispatchtrigger ofpublish_to_test_pypi.ymlfirst if desired.The package version itself is provided by
setuptools_scm(driven by the git tag), so the tag is the only source of truth — no__version__literal to bump.Other things a future developer should know
.cursor/rules/*.mdc(allalwaysApply: true).python_best_practices.mdcis the load-bearing one: 100-col lines, Google-style docstrings wrapped at 90,pathliboveros.path,loggingoverprint, no mutable defaults, no shadowed builtins, ≤3 positional args then keyword-only,raise … fromto preserve tracebacks,__all__+py.typedfor any new public surface.scripts/run-all-checks.shis the canonical orchestrator. It knows the project'sRUN_* / ENABLE_*gating and runs the checks in parallel by default. Use-sfor sequential,-cfor code-only,-dfor docs-only, or pick a single check with--pytest,--ruff-check, etc.tool.ruff.format.quote-style = "single").ruff format --checkis configured but not enforced in CI — the project does not auto-runruff format.strict = trueand runs againstsrcandtests. The only remaining# type: ignorein the project is intests/test_color.pyfor a deliberate non-string lookup that exercises theTypeErrorbranch.docs/dev/adding_an_instrument.rst. The four-method protocol isdetect_vicar,detect_fits,matches,tint_for. Register the new module ininstruments/__init__.py; the dispatch table picks it up automatically.tests/fixture_recipes/(one*.recipe.pyper fixture, plusregenerate_all.pyandgenerate_snapshots.py); the generated binaries live intests/fixtures/. Recipes are committed so the binary blobs are reproducible from source.picmaker.picmakeris kept as a re-export shim so external scripts that importfrom picmaker.picmaker import images_to_pics(or any of the 53 public names) continue to work. Issue Prune the picmaker.picmaker BC shim exports #11 tracks pruning it after a migration period.CODEBASE_REVIEW.md,CODEBASE_CRITIQUE.md,TEST_SUITE_CRITIQUE.mdare local working notes and are no longer git-tracked. They remain in the working tree for reference but are not part of the deliverable.CODE_OF_CONDUCT.mdat the repo root is referenced bydocs/code_of_conduct.md's:include:directive — do not move it.Test plan
scripts/run-all-checks.sh -sclean locally.-Walready gates this in CI).from picmaker import images_to_picsandfrom picmaker.picmaker import images_to_picsboth resolve in a fresh interpreter andiseach other.docs/dev/releasing.rst.🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Documentation
Chores
Tests