From f23e2b2de0844887a46bdcae704205ebc8895a22 Mon Sep 17 00:00:00 2001 From: robbond Date: Wed, 12 Aug 2026 10:41:31 +0100 Subject: [PATCH] docs: record update-only previous-question fix --- docs/current-handoff.md | 6 +++++ docs/experiment-57j81.md | 51 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 docs/experiment-57j81.md diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 8de626e..41a27a4 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -183,6 +183,12 @@ _Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, --- +### Experiment 57J.81 — Update-Only Previous Question Fix + +**Objective:** Fix the 57J.80 apparatus defect where `FIXTURE_MODE=updateOnly` passes `previousQuestion = null` through to the production Zod validator, which rejects with `"Expected string, received null"` at `request_validation` stage before any model inference. + +**Classification: A — FIX VALIDATED (tooling only).** The pre-anchored fixture already contains the exact question text at `unresolved_question` ("Are the projected office savings from relocation realistic?"). Changed the harness to derive `previousQuestion` from that committed anchor (`fixtureData.unresolved_question ?? savingsNode.label`) instead of sending `null`. Source is the fixture anchor — no independent duplicate hardcoded question added. Zero production code changed. 49 harness tests pass (5 new deterministic regressions). The update-only apparatus can now reach the real production Update path without requiring Start. + --- ### Experiment 57J.53 — Structured Fidelity Multi-Turn Progress diff --git a/docs/experiment-57j81.md b/docs/experiment-57j81.md new file mode 100644 index 0000000..e71fee9 --- /dev/null +++ b/docs/experiment-57j81.md @@ -0,0 +1,51 @@ +# Experiment 57J.81 — Update-Only Previous Question Fix + +**Branch:** `feature/semantic-action-contract-v0.23` +**Starting HEAD:** `ce01e70` (tooling: add pre-anchored update-only mode to canonical harness) + +## Objective + +Fix the 57J.80 defect: update-only mode sends `previousQuestion = null`, which the production Zod validator rejects with `"Expected string, received null"` at `request_validation` stage, blocking all model inference. + +## Defect Source + +When `FIXTURE_MODE=updateOnly`, the harness sets `selectedQuestion = null` (line 242 of `reproduce-multi-turn-investigation.mjs`) and then passes it as `previousQuestion` to the Update request. The production validation schema (`lib/graph/schema.js:207`) requires `previousQuestion: z.string().min(1)`. + +## Fix + +**Source of previousQuestion:** FIXTURE ANCHOR (derived from committed fixture, not a hardcoded duplicate). + +The pre-anchored fixture already contains the exact question text in two locations: +- `unresolved_question` at the fixture level (snake_case, from JSON) +- The anchor node's `label` field on `n_savings_realism` + +**Change in harness** (`scripts/reproduce-multi-turn-investigation.mjs`): +```javascript +let selectedQuestion = fixtureData.unresolved_question ?? savingsNode.label; +``` + +This replaces `let selectedQuestion = null;` — both the harness and its test simulation now derive `previousQuestion` from the committed savings-realism anchor. + +**Exact previousQuestion:** `"Are the projected office savings from relocation realistic?"` + +## Classification: A — FIX VALIDATED (tooling only) + +One tooling commit fixes the boundary condition. The pre-anchored fixture already contains the required question string; no production code, prompts, or schemas changed. + +## Tests added + +1. Direct assertion that `previousQuestion` is a non-empty string +2. Direct assertion that `previousQuestion` matches the committed savings-realism anchor exactly +3. Direct assertion of exact fixture graph transmission (replaces indirect node-count check) +4. Direct assertion of exact ANSWER_2 in request body +5. Explicit zero-HTTP-call guard for missing ANSWER_2 + +All 49 harness tests pass. Zero Ollama calls. Zero live API calls. + +## What this enables + +The update-only apparatus can now reach the production Update path without requiring a prior Start call. Experiment 57J.80's incremental-meaning test case is unblocked and ready to run against the model inference stage. + +## Production code changed: NO +## Ollama calls: 0 +## Live API calls: 0