BC5 readiness: TypeScript live wire-capture canary by jeremy · Pull Request #294 · basecamp/basecamp-sdk

jeremy · 2026-05-01T22:43:58Z

Stacked on #293. Merge target: bc5-readiness-spec-foundations. Will be re-targeted to main once #293 lands.

Summary

The TypeScript runner becomes the canonical wire-capturer for the SDK's compatibility canary. It dispatches each live test through the SDK's typed surface, intercepts the raw HTTP response at the global fetch chokepoint, and validates the body against the OpenAPI response schema.

Strictly the TS layer of the multi-language canary plan: cross-language replay and pairwise BC4↔BC5 comparison are deferred to follow-up PRs.

What lands

Schema extension (conformance/schema.json): mode: "mock" | "live" (default mock); mockResponses conditionally required; liveAssertions[] with four assertion types; fixtureIds[] for placeholder resolution.
Live fixture (conformance/tests/live-my-surface.json): 10 operations covering the read surface BC5 forward-compat additions touch (GetMyNotifications, GetTodoset, GetMyProfile, etc.).
Mode gating: runner.test.ts loads only mock tests; live-runner.test.ts loads only live tests, skips entirely without BASECAMP_LIVE=1.
Wire capture (wire-capture.ts): wraps globalThis.fetch — single chokepoint for both openapi-fetch (page 1) and the fetchPage closure (page 2+). SDK behavior unchanged.
Schema validation (schema-validator.ts): ajv + ajv-formats. additionalProperties permissive (forward-compat doesn't break canary); required strict; $ref resolved against openapi.json. Per-page validation; extras union across pages and surface in the run summary.
Fixture-ID resolution (fixtures.ts): env-var → discovery → cache ladder (§5d). BASECAMP_BC4_PROJECT_ID → BASECAMP_PROJECT_ID → ListProjects → first project, same for TODOSET_ID, TODOLIST_ID, TODO_ID.
Dispatch coverage gate (live-dispatch.ts): assertDispatchCoverage() fires in beforeAll. Any operation in the fixture without a dispatch case fails the run with a pointer to where to add the mapping.
Snapshot persistence: when LIVE_RECORD_DIR is set, per-test snapshots write to <dir>/<backend>/wire/<test>.json for downstream replay (PR 3) and pairwise comparison (PR 4).
make conformance-typescript-live: opt-in target; not invoked by make check.
CONTRIBUTING.md "Live canary": env vars, snapshot dir layout, fixture/dispatch pairing rule.

Acceptance bar

Per the scope guard:

Mode-gated — BASECAMP_LIVE unset → mock only; set → live only.
Captures raw response bytes/headers at the global fetch boundary.
Validates with Ajv against the generated OpenAPI schema (per page, with extras-observed reporting).
Deterministic fixture naming — snapshots are namespaced by BASECAMP_BACKEND.
Runs against BC4 or BC5 without doing pairwise comparison yet.
Cross-language replay and BC4/BC5 diffing remain out.

Test plan

make conformance-typescript (default, no live env) — live-runner.test.ts skipped, runner.test.ts 63 mock tests pass + 5 pre-existing TS_SDK_SKIPS.
BASECAMP_LIVE=1 npx vitest run live-runner.test.ts (no token) — fails fast in beforeAll with a clear missing-env-var error.
Coverage gate: temporarily added NoSuchOperation to the fixture; runner refused to start with: Live runner is missing dispatch cases for: NoSuchOperation. Add a DispatchFn to LIVE_OPERATIONS in live-dispatch.ts. Reverted before commit.
Manual: BASECAMP_LIVE=1 BASECAMP_TOKEN=... BASECAMP_ACCOUNT_ID=... BASECAMP_HOST=... LIVE_RECORD_DIR=tmp/live-canary/bc4 BASECAMP_BACKEND=bc4 make conformance-typescript-live against a real BC4 sandbox. (Requires sandbox creds — reviewer to confirm or defer to follow-up.)

Out of scope (later PRs)

PR 3: cross-language wire-replay decoders (Ruby/Python/Go/Kotlin).
PR 4: pairwise BC4↔BC5 comparison + make check-bc5-compat orchestrator + scheduled CI.
PR 5: API gap detector tooling (scripts/detect-api-gaps.sh).

Summary by cubic

Adds a TypeScript live wire-capture canary that validates live API responses against the OpenAPI schema and records snapshots for cross‑language replay, keeping BC5 forward‑compat while leaving mock suites unchanged. Runs behind an opt‑in flag and tightens schema/runners/validator so live tests are reliable and safe.

New Features
- Conformance schema adds mode (mock/live) with per‑mode requirements; liveAssertions (liveCallSucceeds, liveResponseFieldsRequired, liveResponseFieldsExpected, liveSchemaValidate) and fixtureIds supported; new conformance/tests/live-my-surface.json (~10 read endpoints).
- TypeScript live runner wraps globalThis.fetch, validates with ajv + ajv-formats, and writes snapshots to LIVE_RECORD_DIR/<backend>/wire/*.json; gated by BASECAMP_LIVE=1 with BASECAMP_TOKEN/BASECAMP_ACCOUNT_ID; make conformance-typescript-live target and docs added.
- Offline runners (TS mock, Go, Ruby, Python, Kotlin) load only mode: "mock"; live tests are TS‑only wire‑capture. Live dispatch is coverage‑gated with Object.hasOwn; fixture IDs resolve before capture so discovery traffic doesn’t pollute snapshots.
Bug Fixes
- Schema validator: rewrites local $ref to openapi.json#/…, resolves multi‑level refs, finds non‑200 2xx schemas (e.g., 201), treats bodyless 2xx (e.g., 204‑only ops) as success, and collects extras recursively through arrays/objects; offline tests cover these paths.
- Live runner: fixes required‑field checks through arrays (correct path slicing with repeated segments), uses testCtx.skip for missing fixtures, and keeps the capture window free of pre‑resolution calls.

^{Written for commit 6e78222. Summary will update on new commits.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7e946013c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cubic-dev-ai

6 issues found across 12 files

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="conformance/runner/typescript/live-dispatch.ts">

<violation number="1" location="conformance/runner/typescript/live-dispatch.ts:106">
P2: Dispatch coverage check should not use `in` on a plain object; inherited keys can bypass missing-operation detection.</violation>
</file>

<file name="conformance/runner/typescript/schema-validator.ts">

<violation number="1" location="conformance/runner/typescript/schema-validator.ts:75">
P2: Response schema lookup ignores non-200 success responses (e.g. 201), causing false validation failures for valid operations.</violation>

<violation number="2" location="conformance/runner/typescript/schema-validator.ts:124">
P1: Compiling a response schema fragment leaves `#/components/...` refs without the OpenAPI root context, which can break AJV `$ref` resolution at runtime.</violation>

<violation number="3" location="conformance/runner/typescript/schema-validator.ts:182">
P2: `readObjectProperties` returns `null` for array response schemas, causing `collectExtras` to short-circuit and return `[]`. List endpoints (ListProjects, ListTodolists, ListTodos, etc.) that return top-level arrays will never report additive fields inside array items, silently missing BC5 forward-compat wire additions in the extras-observed summary. Consider unwrapping `items` for array schemas to inspect item-level properties.</violation>
</file>

<file name="conformance/runner/typescript/live-runner.test.ts">

<violation number="1" location="conformance/runner/typescript/live-runner.test.ts:203">
P2: Use Vitest's skip API here; rejecting the async test will mark missing fixtures as failures instead of skips.</violation>

<violation number="2" location="conformance/runner/typescript/live-runner.test.ts:284">
P1: Extras reporting does not work for array responses, so list endpoints won't surface additive wire fields.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e243c17ef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cubic-dev-ai

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="conformance/runner/typescript/schema-validator.ts">

<violation number="1" location="conformance/runner/typescript/schema-validator.ts:163">
P2: `collectExtras` resolves `$ref` only one level, so alias chains (common in response schemas) cause valid fields to be misreported as extras.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bc9c7e2b92

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 01ea18612f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebaac5db99

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

cubic-dev-ai

5 issues found across 15 files (changes from recent commits).

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="kotlin/conformance/src/main/kotlin/com/basecamp/sdk/conformance/SchemaWalker.kt">

<violation number="1" location="kotlin/conformance/src/main/kotlin/com/basecamp/sdk/conformance/SchemaWalker.kt:59">
P2: Include OpenAPI wildcard success code `"2XX"` in the fallback response-schema lookup; otherwise valid schemas can be missed and live responses go unvalidated.</violation>
</file>

<file name="conformance/runner/go/replay_runner.go">

<violation number="1" location="conformance/runner/go/replay_runner.go:252">
P2: Validate that each snapshot’s `operation` matches the fixture test’s expected operation before decoding. Without this check, swapped/mislabeled snapshot files can be decoded against the wrong schema and mask fixture mapping errors.</violation>
</file>

<file name="conformance/runner/ruby/replay-runner.rb">

<violation number="1" location="conformance/runner/ruby/replay-runner.rb:34">
P2: `SDK_DECODE` returns `nil` (success) for empty `body_text`, but the real SDK path calls `JSON.parse(body_text)` which raises `JSON::ParserError` on empty strings. This mismatch means the replay runner treats empty-body snapshots as successful decodes that would actually fail through the real SDK, producing false negatives in compatibility checks. Remove the early return or raise on empty body to match real SDK behavior.</violation>

<violation number="2" location="conformance/runner/ruby/replay-runner.rb:75">
P2: Validate that each snapshot file’s `operation` matches the fixture test’s expected operation; otherwise swapped/mislabeled snapshot files can be silently accepted and validated against the wrong schema.</violation>
</file>

<file name="conformance/runner/python/replay_runner.py">

<violation number="1" location="conformance/runner/python/replay_runner.py:163">
P2: Python's `or` treats empty string `""` as falsy, so when `bodyText` is empty this falls through to `json.dumps(page["body"])`. If `body` is `None`, that produces `"null"`, which `json.loads("null")` parses successfully — masking an actually-empty wire body as a successful decode. Compare with the Go equivalent which guards with `bodyText == "" && page.Body != nil`. Use an explicit `None` check instead of `or`:
`body_text = page.get("bodyText") if page.get("bodyText") is not None else json.dumps(page["body"])`
or handle empty bodyText as a decode boundary separately.</violation>
</file>

_{Tip: Review your code locally with the cubic CLI to iterate faster.}

Both come in transitively via openapi-typescript → @redocly/openapi-core (fast-uri through @redocly/ajv, brace-expansion through minimatch). npm audit on the affected versions reports: fast-uri ≤3.1.1 — high — path traversal + host confusion (GHSA-q3j6-qgpj-74h6 / GHSA-v39h-62p7-jpjc) brace-expansion 4.0.0–5.0.4 — moderate — zero-step sequence hangs the process (GHSA-f886-m6hf-6m8v) Lift both to the patched majors via overrides (alongside the existing minimatch pin). The override path is preferred over plain dependency bumps because the SDK doesn't depend on either package directly — the responsibility for forwarding the fix sits with this repo until the upstream packages cut new releases that pull the patches in transitively. Lands at the bottom of the BC5 stack so the wire-replay PRs above (#294, #301) inherit the fix on rebase.

* mode: "mock" | "live" (default mock) * mockResponses required when mode=mock; forbidden when mode=live * liveAssertions[] with liveCallSucceeds, liveResponseFieldsRequired, liveResponseFieldsExpected, liveSchemaValidate * fixtureIds[] for placeholder resolution (PROJECT_ID, TODOSET_ID, etc.) Initial live fixture (live-my-surface.json) covers the read surface exercised by the BC5 forward-compat additions: ListProjects, GetProject, GetMyAssignments + completed + due variants, GetMyNotifications, GetMyProfile, GetTodoset, ListTodolists, ListTodos. Live tests are gated by BASECAMP_LIVE=1 and never reach MSW in mock-mode runs (per the runner's mode filter, separate commit).

The TypeScript runner becomes the canonical wire-capturer for the SDK's compatibility canary: it dispatches each live test through the SDK's typed surface, intercepts the raw HTTP response (bytes + headers) at the global fetch chokepoint, and validates the body against the OpenAPI response schema. Forward-compat fields surface as "extras observed" — never as failures — so new BC5 fields don't break the canary while staying visible. Mode gating * runner.test.ts loads only mode=mock tests; live entries never reach MSW. * live-runner.test.ts loads only mode=live tests; entire suite skips unless BASECAMP_LIVE=1. Missing token/account-id error in beforeAll with a clear message before any test runs. Wire capture * wire-capture.ts wraps globalThis.fetch — single chokepoint for both openapi-fetch (page 1) and the fetchPage closure (page 2+). Cloned responses preserve raw bytes for snapshot persistence; SDK behavior unchanged. * Snapshots persist to <LIVE_RECORD_DIR>/<backend>/wire/<test>.json when the env var is set — feeds cross-language replay (PR 3) and pairwise BC4↔BC5 comparison (PR 4). Schema validation * schema-validator.ts uses ajv + ajv-formats. additionalProperties permissive (forward-compat must not break the canary); required strict; $ref resolved against openapi.json. * Per-page validation; extras union across pages and report in the run-end summary. Fixture-ID resolution * fixtures.ts walks the env-var → discovery → cache ladder per §5d: BASECAMP_BC4_PROJECT_ID / BASECAMP_BC5_PROJECT_ID → BASECAMP_PROJECT_ID → ListProjects → first project. Same pattern for TODOSET_ID, TODOLIST_ID, TODO_ID. Missing-fixture path skips with skipReason. Dispatch coverage gate * live-dispatch.ts maps operation → SDK call. assertDispatchCoverage() fires in beforeAll: any operation referenced in the fixture without a dispatch case fails the run with a clear error pointing at where to add the mapping. Make + docs * `make conformance-typescript-live` opt-in target (not invoked by `make check`). CONTRIBUTING.md documents env vars and the fixture/dispatch pairing rule. Out of scope per PR 2: cross-language wire replay (PR 3), pairwise BC4↔BC5 comparison (PR 4), API gap detector (PR 5).

Three fixes from PR review against the live-only path that offline checks didn't exercise: 1. Schema validator $ref resolution. The validator registered the full OpenAPI doc with key "openapi.json" but compiled the response-schema fragment directly. Refs like "#/components/schemas/ListProjectsResponseContent" resolved against the fragment root (not the registered doc), throwing "can't resolve reference ... from id #" on every live call. Fix: prepareForCompile rewrites local refs ("#/...") to fully-qualified refs ("openapi.json#/...") so Ajv resolves against the registered document. additionalProperties:false is still stripped in the same pass. 2. Vitest-native skip for fixture-missing. The previous code did Promise.reject(Object.assign(new Error("SKIP..."), { skip: true })), which Vitest reports as a failure regardless of any custom flag. A sandbox missing a todoset/todolist would have failed the canary despite the documented "skip with skipReason" behavior. Fix: use the test-context's runtime skip — testCtx.skip(reason) — so missing fixtures land as proper skipped tests. 3. Recursive extras collection. collectExtras() only inspected the root when it was an object, so list responses (ListProjects, ListTodos, etc.) yielded no extras even when items had unknown fields. Fix: walk body + schema together, descend through arrays (path segment "[]") and into known properties; emit dotted paths like "[].future_field" or "unreads[].future_envelope_field". Bounded depth as a cycle guard. Offline coverage: schema-validator.test.ts exercises validateResponse with crafted payloads — conformant ListProjects/Project, missing required field, forward-compat extras at root and item level, plus GetMyNotifications object-with-array properties. 10 tests, all pass. This is the regression bar for the live-only path: future schema wiring bugs surface on every PR via mock conformance, no live creds needed. Verified: BASECAMP_LIVE=1 BASECAMP_TOKEN=stub BASECAMP_HOST=<unreachable> runs report 4 fixture-needing tests as skipped (ctx.skip path) and 6 host-needing tests as expected-failed (network). No false failures.

Four issues from the cubic/codex review pass on the live-only path: 1. Schema lookup ignored non-200 success responses. findResponseSchema() only checked "200" then "default", missing operations that return 201 (Create*) or 204 (Delete*). Now scans 200/201/202/203/204/default first, then any 2xx code, before declaring "no schema found". 2. resolveRef only resolved one level. Alias chains (Foo → $ref Bar → $ref Baz) caused valid item-level fields to be misreported as extras. Now loops until non-ref or cycle detected. 3. assertDispatchCoverage used `op in LIVE_OPERATIONS`. Inherited Object.prototype keys (`toString`, `hasOwnProperty`, `constructor`) would have slipped past the gate. Switched to Object.hasOwn(). 4. Wire capture window included fixture-discovery traffic. dispatch(ctx) would call resolveFixtureId() inside the capture window, mixing the ListProjects discovery page into the snapshot for GetProject. Split the dispatch contract into DispatchSpec { fixtures, call }: the runner pre-resolves all required fixtures before installing capture, then runs the call() with already-resolved IDs. Discovery happens strictly before capture starts, so snapshots contain only the operation under test. Offline coverage: * schema-validator.test.ts gains a "non-200 success" case asserting CreateProject (201) finds a schema (vs the pre-fix "no schema found"). * live-dispatch.test.ts (new) covers the coverage gate including the Object.hasOwn semantics — fixture entries for `toString` / `hasOwnProperty` / `constructor` are correctly rejected. 15 offline tests pass; full TS conformance: 78 passed, 6 skipped.

The live-canary fixture (conformance/tests/live-my-surface.json) sits in the shared conformance/tests directory but only the TS runners know about the `mode` field. Without this fix: * Kotlin parses every JSON in the directory; ${PROJECT_ID} placeholders fail Int conversion and live-only operations are unknown — 9 hard failures in CI. * Go has the same loader pattern but its assertion checker silently ignores entries lacking `assertions` (live entries use `liveAssertions`), so the 10 live entries slip through as false passes (78/0 vs the expected 68/0). * Ruby and Python share the same shape; same risk. Make `mode` a cross-runner contract: the four offline runners (Go, Ruby, Python, Kotlin) now filter `mode == "live"` entries at load time. TS runners already filtered. Schema stays shared in conformance/tests/; no fixture relocation needed. Verified with `make conformance` (each runner separately): * Go: Passed: 68, Failed: 0 * Ruby: 59 passed, 9 SDK-specific skips * Python: 64 passed, 4 SDK-specific skips * Kotlin: Passed: 58, Skipped: 10 * TS: 3 mock test files passed, live-runner.test.ts skipped without BASECAMP_LIVE Each runner now reports 68 total (10 fewer than the pre-fix 78 — exactly matching the live-only entries that now correctly filter out).

…chema Ruby was the only offline runner expressed as "reject live" instead of "accept mock". With the schema enum bounded to {mock, live} the two are equivalent today, but accept-mock defends against any future mode (e.g. wire-replay) silently leaking into the offline runner. Also document in CONTRIBUTING.md that mode is a shared-schema contract across runners — fixtures live in conformance/tests/ for everyone, so offline runners must filter to mock before dispatch.

live-runner.test.ts `fieldExists` — when descending into an array, the remaining-path computation used `parts.indexOf(part)`, which returns the first occurrence of that segment name. For required-field paths with repeated names (`a.b.b.x`), recursion restarts from the wrong position and reports false missing-field failures. Iterate by index instead. schema-validator.ts `validateResponse` — when `findResponseSchema` returns null, the validator returned `ok: false` unconditionally. That broke schema validation for legitimately bodyless 2xx operations (e.g. `DeleteBoost` with only a 204 response), blocking them from inclusion in the live canary. Distinguish bodyless-by-design (`ok: true`, nothing to validate) from missing-operation (`ok: false`) via a new `operationHasBodylessSuccessOnly` helper that checks whether the operation declares any 2xx response without an `application/json` schema. New regression test pins the `DeleteBoost` case.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e78222a67

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-13T21:05:50Z

+              const firstStatus = snapshot.pages[0].status;
+              if (firstStatus < 200 || firstStatus >= 300) {
+                failures.push(`liveCallSucceeds: first page returned HTTP ${firstStatus}`);


Validate live call success from the final response

This check fails the test whenever the first captured HTTP response is non-2xx, even if the SDK transparently retries and the operation ultimately succeeds. In this codebase, createBasecampClient enables retry middleware by default, so transient 429/503 responses can be recovered but will still be marked as canary failures here. That makes the live suite flaky and incorrectly reports successful operations as failures in environments with intermittent retryable errors.

Useful? React with 👍 / 👎.

cubic-dev-ai

1 issue found across 3 files (changes from recent commits).

Prompt for AI agents (unresolved issues)


Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="conformance/runner/typescript/schema-validator.ts">

<violation number="1" location="conformance/runner/typescript/schema-validator.ts:150">
P2: Bodyless-operation shortcut returns success even when a response body is present, masking spec drift for 204-style endpoints.</violation>
</file>

_{Tip: Review your code locally with the cubic CLI to iterate faster.}

cubic-dev-ai · 2026-05-13T21:08:38Z

+    // valid by design — no schema means no body to validate. The latter
+    // is still a hard failure: the operation isn't covered by the spec.
+    if (operationHasBodylessSuccessOnly(doc, operationId)) {
+      return { ok: true, errors: [], extras: [] };


P2: Bodyless-operation shortcut returns success even when a response body is present, masking spec drift for 204-style endpoints.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At conformance/runner/typescript/schema-validator.ts, line 150: <comment>Bodyless-operation shortcut returns success even when a response body is present, masking spec drift for 204-style endpoints.</comment> <file context> @@ -142,6 +142,13 @@ export function validateResponse(operationId: string, body: unknown): Validation + // valid by design — no schema means no body to validate. The latter + // is still a hard failure: the operation isn't covered by the spec. + if (operationHasBodylessSuccessOnly(doc, operationId)) { + return { ok: true, errors: [], extras: [] }; + } return { </file context>

Suggested change

return { ok: true, errors: [], extras: [] };

return body === null || body === undefined

? { ok: true, errors: [], extras: [] }

: {

ok: false,

errors: [`Operation ${operationId} declares a bodyless success response, but a body was captured`],

extras: [],

};

_{Tip: Review your code locally with the cubic CLI to iterate faster.}

Copilot AI review requested due to automatic review settings May 1, 2026 22:43

github-actions Bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file conformance Conformance test suite enhancement New feature or request and removed documentation Improvements or additions to documentation labels May 1, 2026

jeremy force-pushed the bc5-readiness-spec-foundations branch from 5bd407c to 63205bf Compare May 1, 2026 22:49

jeremy force-pushed the bc5-readiness-ts-live-canary branch from b7e9460 to d776bfd Compare May 1, 2026 22:50

github-actions Bot added documentation Improvements or additions to documentation and removed documentation Improvements or additions to documentation labels May 1, 2026

chatgpt-codex-connector Bot reviewed May 1, 2026

View reviewed changes

Comment thread conformance/runner/typescript/live-runner.test.ts Outdated

Comment thread conformance/runner/typescript/schema-validator.ts Outdated

github-actions Bot added the documentation Improvements or additions to documentation label May 1, 2026

cubic-dev-ai Bot reviewed May 1, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed May 1, 2026

View reviewed changes

Comment thread conformance/runner/typescript/live-runner.test.ts Outdated

jeremy force-pushed the bc5-readiness-spec-foundations branch from 63205bf to 7e9c434 Compare May 1, 2026 23:03

cubic-dev-ai Bot reviewed May 1, 2026

View reviewed changes

Comment thread conformance/runner/typescript/schema-validator.ts Outdated

jeremy force-pushed the bc5-readiness-ts-live-canary branch from 0e243c1 to bc9c7e2 Compare May 1, 2026 23:09

chatgpt-codex-connector Bot reviewed May 1, 2026

View reviewed changes

Comment thread conformance/runner/typescript/live-runner.test.ts Outdated

github-actions Bot added the kotlin label May 1, 2026

chatgpt-codex-connector Bot reviewed May 2, 2026

View reviewed changes

Comment thread conformance/runner/typescript/schema-validator.ts

github-actions Bot added documentation Improvements or additions to documentation and removed documentation Improvements or additions to documentation labels May 2, 2026

chatgpt-codex-connector Bot reviewed May 4, 2026

View reviewed changes

Comment thread conformance/runner/python/replay_runner.py Outdated

Comment thread conformance/runner/ruby/replay-runner.rb Outdated

cubic-dev-ai Bot reviewed May 4, 2026

View reviewed changes

jeremy force-pushed the bc5-readiness-ts-live-canary branch from ebaac5d to 149ec69 Compare May 4, 2026 23:50

github-actions Bot added documentation Improvements or additions to documentation and removed documentation Improvements or additions to documentation labels May 4, 2026

jeremy mentioned this pull request May 4, 2026

BC5 readiness: cross-language wire-replay decoders #301

Open

7 tasks

jeremy added 6 commits May 13, 2026 13:54

jeremy force-pushed the bc5-readiness-ts-live-canary branch from 149ec69 to 447f806 Compare May 13, 2026 20:55

github-actions Bot added documentation Improvements or additions to documentation and removed documentation Improvements or additions to documentation labels May 13, 2026

chatgpt-codex-connector Bot reviewed May 13, 2026

View reviewed changes

cubic-dev-ai Bot reviewed May 13, 2026

View reviewed changes

-      return { ok: true, errors: [], extras: [] };
+      return body === null || body === undefined
+        ? { ok: true, errors: [], extras: [] }
+        : {
+            ok: false,
+            errors: [`Operation ${operationId} declares a bodyless success response, but a body was captured`],
+            extras: [],
+          };

Conversation

jeremy commented May 1, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What lands

Acceptance bar

Test plan

Out of scope (later PRs)

Summary by cubic

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot May 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jeremy commented May 1, 2026 •

edited by cubic-dev-ai Bot

Loading