113 lines
4.0 KiB
Markdown
113 lines
4.0 KiB
Markdown
# Experiment 57J.62 — Accepted-Update Capture Hardening
|
||
|
||
**Branch:** `feature/selected-question-contract-v0.22`
|
||
**Starting HEAD:** `929486c` (experiment: validate equivalent uncertainty identity live)
|
||
|
||
## Objective
|
||
|
||
Answer exactly:
|
||
|
||
> Why did the canonical harness fail to retain enough accepted Update 1 detail in 57J.61 to identify the persistent node that was added, and what is the smallest tooling change that makes accepted-update evidence reliable for the next live experiment?
|
||
|
||
## Classification: D — harness prints summary counts but not accepted proposal detail
|
||
|
||
## Exact capture failure cause
|
||
|
||
The harness's accepted-update output block (lines ~97–108 of `scripts/reproduce-multi-turn-investigation.mjs`) printed only:
|
||
|
||
```text
|
||
HTTP status
|
||
stage
|
||
proposal/apply success
|
||
selected question
|
||
node count
|
||
edge count
|
||
```
|
||
|
||
It did NOT print any of the response body fields that describe graph mutations:
|
||
|
||
- `answerMeaning.userSupportedMeaning` — absent
|
||
- `answerMeaning.possibleInference` — absent
|
||
- `answerMeaning.supportCategory` — absent
|
||
- `answerMeaning.resolutionGuidance` — absent
|
||
- `updatedProposal.updatedNodes[]` — absent
|
||
- `updatedProposal.resolvedUnknownNodeIds[]` — absent
|
||
- `updatedProposal.addedNodes[]` — absent
|
||
- `updatedProposal.addedEdges[]` — absent
|
||
- `selectedQuestion.nodeId` (node reference) — absent
|
||
- Resulting graph node/edge details — absent
|
||
|
||
After 57J.61's Update 1 returned HTTP 200 at `update_applied` with node count 6→7 and edge count 5→6, the harness produced no tooling-level evidence of **which** node was added or **what** it contained. The identity invariant ("equivalent unresolved meaning must not multiply graph state") cannot be tested when the evidence is missing.
|
||
|
||
A co-occurring bug: line ~102 referenced `startResult.status` instead of `updateResult.status`, printing the Start HTTP status in the Update block (cosmetic, not evidentiary).
|
||
|
||
## Changes made
|
||
|
||
### `scripts/reproduce-multi-turn-investigation.mjs`
|
||
|
||
Extended accepted-update output block to print:
|
||
|
||
```javascript
|
||
// answerMeaning fields
|
||
answerMeaning.userSupportedMeaning
|
||
answerMeaning.possibleInference
|
||
answerMeaning.supportCategory
|
||
answerMeaning.resolutionGuidance
|
||
|
||
// structural mutation fields
|
||
updatedProposal.updatedNodes[]
|
||
updatedProposal.resolvedUnknownNodeIds[]
|
||
updatedProposal.addedNodes[]
|
||
updatedProposal.addedEdges[]
|
||
|
||
// selectedQuestion node reference
|
||
selectedQuestion.nodeId
|
||
|
||
// Compact structural snapshot of resulting persistent graph
|
||
resulting graph: {id, kind, label/description, status} per node
|
||
: {from/to/relationship} per edge
|
||
```
|
||
|
||
Fixed `startResult.status` → `updateResult.status`.
|
||
|
||
### `tests/reproduce-multi-turn-investigation.harness.test.js`
|
||
|
||
Added 10 new deterministic harness tests via a companion simulation function (`runSimulationWithResponseShape`) that records capture outputs:
|
||
|
||
1. accepted Update exposes addedNodes details
|
||
2. accepted Update exposes updatedNodes details
|
||
3. accepted Update exposes resolvedUnknownNodeIds
|
||
4. accepted Update exposes selectedQuestion (question + nodeId)
|
||
5. accepted Update exposes answerMeaning structured fields
|
||
6. accepted Update exposes resulting persistent graph nodes/edges
|
||
7. rejected Update still exposes rejectedProposalSnapshot (existing behavior verified)
|
||
8. Update 1 accepted → Update 2 receives exactly that resulting graph state
|
||
9. no extra HTTP call is introduced for diagnostics
|
||
10. existing no-retry and call-accounting guarantees remain intact
|
||
|
||
All tests use mocked API responses only. Zero Ollama calls. Zero dev-server calls.
|
||
|
||
## Invariants preserved
|
||
|
||
- One Start invocation = one API call
|
||
- One Update invocation = one API call
|
||
- No semantic retries
|
||
- No transport retries
|
||
- Update failure stops the chain
|
||
- Call accounting remains exact
|
||
- RejectedProposalSnapshot path unchanged for rejected updates
|
||
|
||
## What this does NOT change
|
||
|
||
- Production API behavior
|
||
- Production reasoning code
|
||
- Prompt instructions
|
||
- Schema definitions
|
||
- Validator logic
|
||
- Provider/model integration
|
||
|
||
## Configured Ollama: none used. Dev server disturbed: NO.
|
||
|
||
## Tests
|
||
|
||
18 tests pass (8 existing + 10 new). 0 failed. |