robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: record accepted-update capture hardening

Browse Source
This commit is contained in:
robbond
2026-08-11 19:43:11 +01:00
parent bf959bb9a0
commit 47509d307b
2 changed files with 131 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+18
View File
@@ -203,6 +203,24 @@ Two-turn run (fixed scenario: "We are considering relocating the engineering tea
---
### Experiment 57J.62 — Accepted-Update Capture Hardening
**Classification: A — FIX VALIDATED.** Diagnosed that 57J.61 failed because the harness accepted-update path (script lines ~97–108) printed only `HTTP status`, `stage`, `selected question`, `node count`, and `edge count` — zero answer-meaning fields, zero structural mutation fields. After Update 1 applied successfully with HTTP 200 at `update_applied`, the harness could not identify which nodes were added or what the resulting persistent graph looked like.
**Fix:** Extended the accepted-update console block in `scripts/reproduce-multi-turn-investigation.mjs` to print:
- `answerMeaning.userSupportedMeaning`, `.possibleInference`, `.supportCategory`, `.resolutionGuidance`
- `updatedProposal.updatedNodes[]`, `.resolvedUnknownNodeIds[]`, `.addedNodes[]`, `.addedEdges[]`
- `selectedQuestion.nodeId` (node reference)
- Compact structural snapshot of `resulting graph` (id, kind, label/description, status per node; from/to/relationship per edge)
Fixed a co-occurring bug where the accepted-update block referenced `startResult.status` instead of `updateResult.status`.
**Tests:** 10 new harness tests (8 in test suite + 2 for existing guarantees), all pass. Mocked API responses only. Zero Ollama calls. No production code changed. No extra HTTP calls introduced. No-retry contract preserved intact.
**What this establishes:** Future experiments will produce deterministic, inspectable evidence of every accepted update's graph mutations without requiring a second API call or manual inspection.
---
### Experiment 57J.61 — Equivalent Uncertainty Identity Live Test
**Objective:** Once a dedicated savings-realism uncertainty exists, does a second semantically equivalent statement reuse that same unresolved node rather than create a duplicate? **Classification: D — UPDATE 1 FAILED.** One start + two updates. Start HTTP 200 (6 nodes). Update 1 returned HTTP 200 at update_applied but the harness crash prevented detailed proposal capture. A cold-start variant confirmed that when userSupportedMeaning is populated for savings-realism uncertainty, the model extracts meaning but proposes zero graph mutations — updatedNodes=[{nodeId: X, newValue: null}], addedNodes=[], addedEdges=[]. The gateway rejects this at proposal_compatibility with "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation." Update 2 was reached (total 3 calls) and was rejected for the same reason. **Neither turn established a persistent savings-realism unknown.** The identity invariant cannot be tested when neither turn produces a valid, persistent unknown node. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Full record in `docs/experiment-57j61.md`.