robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: diagnose null semantic mutation path

Browse Source
This commit is contained in:
robbond
2026-08-12 13:19:37 +01:00
parent eaf3194752
commit a40a3e343e
2 changed files with 241 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+4
View File
@@ -1675,3 +1675,7 @@ This satisfies 57J.77's boundary A recommendation: a committed update-only path
### Experiment 57J.83 — Direct Answer-Meaning Capture from updatedProposal
**Objective:** Verify the production path correctly reads `answerMeaning` and `structuralActionRequired` from inside `updatedProposal` (graphUpdate schema container) and that all test mock boundaries are coherent with this contract. **Classification: IMPLEMENTED.** Fixed mock boundary mismatch where some fixtures placed fields at root level while capture logic read from inside `updatedProposal`. All 49 harness tests pass. Full results in `docs/experiment-57j83.md`. No production code changed; only experiment apparatus (script + test harness).
### Experiment 57J.85 — Null Semantic/Action Architecture Diagnosis (Read-Only)
**Objective:** Diagnose why `answerMeaning=null + structuralActionRequired=null + meaningful mutation` is accepted through the current pipeline, and whether this compatibility path should remain open. Read-only diagnosis across prompt contract, schema, validator, and apply-proposal validation layers. **Classification: B — TRANSITION COMPATIBILITY.** The null/null/mutation path persists because: (1) Zod schema allows nullable fields for backward compatibility; (2) validator rules are scoped to only reject when meaning IS populated (rule 1), leaving null-meaning mutations unguarded; (3) the model produces useful data through this path (57J.84: £2m/year on reported_claim node); (4) tightening without a deterministic recovery/retry path would discard that information. Architectural choice A — keep the null transition path for now, pending deterministic recovery capability before tightening becomes safe. Smallest next boundary: implement a deterministic recovery mechanism for proposals with meaningful mutations but unpopulated semantic/action fields. No production code changed; no Ollama calls; documentation-only diagnosis.