ci: pin codegen tools + guard generated-file freshness; document preserve-on-conflict#877
ci: pin codegen tools + guard generated-file freshness; document preserve-on-conflict#877dougborg wants to merge 3 commits into
Conversation
Generated client/pydantic code is committed source, but the codegen tools were loose floors (`datamodel-code-generator>=0.27.0`, `openapi-python-client>=0.25.2`), so the resolved version could drift and silently change generated output with no signal. That's exactly what bit #873: datamodel-codegen 0.58.0 re-typed InventoryMovement's created_at/updated_at to bare `Any`, and it only surfaced because that PR happened to regenerate. - Pin both tools to exact current versions (==0.58.0 / ==0.28.4). Bumping is now a deliberate, reviewed change: edit the pin, run `uv run poe regenerate-all`, commit the diff. - Add a `generated-files` CI job that regenerates and `git diff --exit-code`s the generated paths (api/, models/, client.py, models_pydantic/_generated/, _auto_registry.py), failing if the committed output is stale. Gated on code/spec changes, mirroring the quality job. A fresh `regenerate-all` against current main is already clean, so no baseline regeneration is bundled here.
Capture the durable lesson from the service_id work (#870/#873/#876): a cache-only column populated by a *different* entity's `post_sync` hook (not the owning entity's own wire payload) must be listed in the owning spec's `preserve_columns_on_conflict`, or `_bulk_upsert`'s blanket `ON CONFLICT DO UPDATE SET` re-nulls it on the next delta. - typed_cache/README.md: new "Cache-only columns written cross-table need preserve-on-conflict" section — the general contract, attrs_postprocess vs post_sync, and the service_id example. - cookbook/catalog-search.md: a `service_id` subsection alongside the four attrs_postprocess fields, distinguishing the cross-table case. - CLAUDE.md: add the pattern to the typed-cache topical keywords for grep discoverability.
There was a problem hiding this comment.
Pull request overview
This PR makes generated client outputs more reproducible and documents the typed-cache durability contract that came out of the service parent_id work.
Changes:
- Pins OpenAPI/client codegen tools to exact versions.
- Adds a CI job that regenerates generated outputs and fails on drift.
- Documents when cross-table cache-only columns need
preserve_columns_on_conflict.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
pyproject.toml |
Pins codegen dependencies and explains the regeneration workflow. |
uv.lock |
Reflects the exact codegen dependency pins. |
.github/workflows/ci.yml |
Adds the generated-file freshness CI job. |
katana_mcp_server/docs/typed_cache/README.md |
Documents the cross-table cache-only column durability contract. |
katana_mcp_server/docs/cookbook/catalog-search.md |
Adds service_id guidance for catalog search denormalization. |
CLAUDE.md |
Adds discoverability keywords for the typed-cache pattern. |
| if ! git diff --exit-code -- \ | ||
| katana_public_api_client/api \ | ||
| katana_public_api_client/models \ | ||
| katana_public_api_client/client.py \ | ||
| katana_public_api_client/models_pydantic/_generated \ | ||
| katana_public_api_client/models_pydantic/_auto_registry.py; then |
There was a problem hiding this comment.
Good catch — fixed in c8b4c71. Scoped the diff to the whole katana_public_api_client/ package instead of an enumerated list, so errors.py, client_types.py, py.typed, and the templated __init__.py are all covered. The generators don't rewrite the hand-maintained files in the package (katana_client.py, helpers/, domain/, …), so there are no spurious diffs. Verified a clean regenerate-all produces an empty package-wide diff locally.
Copilot review of #877: the enumerated path list missed files regenerate_client.py also rewrites — errors.py, client_types.py (renamed from types.py), py.typed, and the templated __init__.py — so a stale one could slip through. Scope the diff to katana_public_api_client/ instead; the generators don't touch the hand-maintained files in the package, so there are no spurious diffs.
Closes #874. Two related follow-ups from the #870/#873/#876 service-
parent_idthread: the regen-freshness gap that thread surfaced, and the docs for the durability contract it produced.1. Generated-file freshness guard (#874)
Generated client/pydantic code is committed source, but the codegen tools were loose floors — so the resolved version could drift and silently change output with no signal. That's exactly what bit #873:
datamodel-codegen0.58.0 re-typedInventoryMovement.created_at/updated_atto bareAny, surfaced only because that PR happened to regenerate.datamodel-code-generator==0.58.0,openapi-python-client==0.28.4. Bumping is now deliberate: edit the pin,uv run poe regenerate-all, commit the diff.generated-filesCI job regenerates andgit diff --exit-codes the generated paths (api/,models/,client.py,models_pydantic/_generated/,_auto_registry.py), failing if committed output is stale. Gated on code/spec changes, mirroring thequalityjob.regenerate-allagainst current main is already clean (post-feat(mcp): denormalize service_id onto CachedVariant; drop search_items O(N) scan #873), so no baseline regeneration is bundled.2. Docs: the preserve-on-conflict contract
The durable lesson from the
service_idwork: a cache-only column populated by a different entity'spost_synchook (not the owning entity's own wire payload) must be in the owning spec'spreserve_columns_on_conflict, or_bulk_upsert's blanketON CONFLICT DO UPDATE SETre-nulls it on the next delta.typed_cache/README.md— new section stating the general contract (attrs_postprocessvspost_sync, theservice_idexample, the regression test that pins it).cookbook/catalog-search.md— aservice_idsubsection alongside the fourattrs_postprocessfields, distinguishing the cross-table case.CLAUDE.md— adds the pattern to the typed-cache topical keywords for grep discoverability.Test plan
uv run poe agent-checkclean (ruff/format/mdformat/ty)uv run poe testgreen (3780)regenerate-allproduces no drift (the new CI job's local equivalent passes)generated-filesjob runs green on this PR🤖 Generated with Claude Code