52 lines
2.4 KiB
Markdown
52 lines
2.4 KiB
Markdown
# Experiment 57J.81 — Update-Only Previous Question Fix
|
|
|
|
**Branch:** `feature/semantic-action-contract-v0.23`
|
|
**Starting HEAD:** `ce01e70` (tooling: add pre-anchored update-only mode to canonical harness)
|
|
|
|
## Objective
|
|
|
|
Fix the 57J.80 defect: update-only mode sends `previousQuestion = null`, which the production Zod validator rejects with `"Expected string, received null"` at `request_validation` stage, blocking all model inference.
|
|
|
|
## Defect Source
|
|
|
|
When `FIXTURE_MODE=updateOnly`, the harness sets `selectedQuestion = null` (line 242 of `reproduce-multi-turn-investigation.mjs`) and then passes it as `previousQuestion` to the Update request. The production validation schema (`lib/graph/schema.js:207`) requires `previousQuestion: z.string().min(1)`.
|
|
|
|
## Fix
|
|
|
|
**Source of previousQuestion:** FIXTURE ANCHOR (derived from committed fixture, not a hardcoded duplicate).
|
|
|
|
The pre-anchored fixture already contains the exact question text in two locations:
|
|
- `unresolved_question` at the fixture level (snake_case, from JSON)
|
|
- The anchor node's `label` field on `n_savings_realism`
|
|
|
|
**Change in harness** (`scripts/reproduce-multi-turn-investigation.mjs`):
|
|
```javascript
|
|
let selectedQuestion = fixtureData.unresolved_question ?? savingsNode.label;
|
|
```
|
|
|
|
This replaces `let selectedQuestion = null;` — both the harness and its test simulation now derive `previousQuestion` from the committed savings-realism anchor.
|
|
|
|
**Exact previousQuestion:** `"Are the projected office savings from relocation realistic?"`
|
|
|
|
## Classification: A — FIX VALIDATED (tooling only)
|
|
|
|
One tooling commit fixes the boundary condition. The pre-anchored fixture already contains the required question string; no production code, prompts, or schemas changed.
|
|
|
|
## Tests added
|
|
|
|
1. Direct assertion that `previousQuestion` is a non-empty string
|
|
2. Direct assertion that `previousQuestion` matches the committed savings-realism anchor exactly
|
|
3. Direct assertion of exact fixture graph transmission (replaces indirect node-count check)
|
|
4. Direct assertion of exact ANSWER_2 in request body
|
|
5. Explicit zero-HTTP-call guard for missing ANSWER_2
|
|
|
|
All 49 harness tests pass. Zero Ollama calls. Zero live API calls.
|
|
|
|
## What this enables
|
|
|
|
The update-only apparatus can now reach the production Update path without requiring a prior Start call. Experiment 57J.80's incremental-meaning test case is unblocked and ready to run against the model inference stage.
|
|
|
|
## Production code changed: NO
|
|
## Ollama calls: 0
|
|
## Live API calls: 0
|