robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: locate semantic-to-mutation contract gap

Browse Source
This commit is contained in:
robbond
2026-08-11 11:56:36 +01:00
parent b341c9cf2f
commit 77f5ea26d4
2 changed files with 190 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+18
View File
@@ -495,3 +495,21 @@ Rejected proposal snapshot: `answerMeaning.userSupportedMeaning` preserved both
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 2 live calls total. No production code changed.
### Experiment 57J.37 — Rejected Proposal Diagnostics: Semantic-to-Mutation Contract Gap (Read-Only Diagnosis)
**Objective:** Read-only analysis of whether the graph-update prompt/validator contract requires structural representation of newly introduced unresolved uncertainty, or whether an empty mutation with populated `answerMeaning` is permitted by the model contract and merely rejected later as a no-op.
**Method:** Analyzed prompt instructions (`lib/graph/prompt-builder.js`), schema defaults (`lib/graph/schema.js`), validator logic (`lib/graph/utils.js` line 868–885), application pipeline (`lib/graph/apply-proposal.js` line 3174, 3252–3270), and existing test coverage. No Ollama calls. No live API.
**Findings:**
- **Prompt contract is AMBIGUOUS:** Rule #6 requires inspecting for new uncertainty but rule #7 ("Add new unknown nodes only when...") is a restriction, not a requirement. Additional Guidance explicitly permits semantic-only proposals via `answerMeaning`.
- **Schema contract PERMITS the combination:** `graphUpdateSchema` allows populated `answerMeaning` + zero structural mutation (all array fields default to `[]`). No cross-field constraint exists.
- **Validator contract REJECTS it:** `hasMeaningfulChange` checks only structural fields (addedNodes, updatedNodes status/value changes, addedEdges, removedEdgeIds). `answerMeaning` is not considered meaningful change.
- **Test coverage NOT COVERED:** No test for "grounded answerMeaning introduces new unresolved uncertainty + zero structural changes." The closest tests verify schema validity of `{}` and validator rejection of all-empty arrays, but neither tests the populated `answerMeaning` case.
**Classification: E — MIXED.** Three independent contract boundaries contribute: (1) prompt ambiguity between inspection and materialization; (2) schema permissiveness vs validator rejection mismatch; (3) model receives permissive guidance that leads to a rejected downstream gate.
**Who owns the failure:** MIXED — Prompt Contract (ambiguity) + Validator Contract (schema/validator mismatch). Model does NOT own this failure.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Production code changed: NO. Prompt changed: NO. Tests changed: NO. Dev server disturbed: NO. Ollama calls: 0.