robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: validate equivalent uncertainty reuse live

Browse Source
This commit is contained in:
robbond
2026-08-11 18:29:02 +01:00
parent eb524d08e1
commit f25b1f550e
2 changed files with 204 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+13 -1
View File
@@ -1054,4 +1054,16 @@ Relevant unresolved unknowns at start: (1) `nx54wwa` — operating expenses (met
**Key finding:** The v0.21 prompt clarification ("same uncertainty = same resolution question") works on a live run with qwen-claude:latest. Unlike 57J.53 (where uncertain status degraded an unrelated node's status without creating a dedicated unknown), v0.21 correctly creates and preserves a dedicated savings-realism unknown with proper structural linkage (`weakens` edge). This resolves the prompt semantic-identity gap diagnosed in 57J.54.
**What remains unproven:** Repeated-run stability (single run); downstream investigation viability past Update 2; cross-domain generalisation; paraphrase invariance; whether reuse/refine still triggers when an equivalent unknown already exists. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. **Production code changed:** NO. **Ollama calls:** 1 live call (start + update via harness). APPEND_EOF
**What remains unproven:** Repeated-run stability (single run); downstream investigation viability past Update 2; cross-domain generalisation; paraphrase invariance; whether reuse/refine still triggers when an equivalent unknown already exists. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. **Production code changed:** NO. **Ollama calls:** 1 live call (start + update via harness).
### Experiment 57J.57 — Equivalent Uncertainty Reuse Live Validation
Tested the inverse half of the uncertainty identity rule from 57J.56: when a graph already contains a dedicated savings-realism uncertainty, does a second semantically equivalent paraphrase reuse/refine that node rather than create a duplicate? **Classification: D — UPDATE 1 FAILED.** Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Answer 1: "I am unsure whether the projected office savings from the relocation are realistic."
**Start:** HTTP 200 | unknown stage | 7 nodes | 4 edges. Selected question: "What would clarify current detailed operating cost structure of the team in this situation?" Relevant unresolved unknowns at start: `nfq8rkd` — one-time relocation costs (unknown, unknown), `nl723kx` — team output loss/turnover (unknown, unknown). Note `nx54wwa` (operating expenses) was known.
**Update 1:** HTTP 422 | proposal_compatibility stage. The model correctly interpreted Answer 1 semantically (`userSupportedMeaning: "The user is unsure whether the projected office savings from the relocation are realistic."`) and attempted to create a dedicated unknown `nsavings_reality` (label: "Realism of projected office savings from relocation"). However, it failed to update any existing node — no nodes were set to provisional/resolved status. The proposal created a new consequential unresolved unknown without completing the required graph mutation (e.g., downgrading `nx54wwa` from known to provisional). The system rejected the proposal: "selectedQuestion is required when consequential unresolved unknowns remain after resolving the answered unknown." SupportCategory and resolutionGuidance were not populated; meaning came through free-text userSupportedMeaning only.
**Key finding:** Semantic interpretation works correctly (distinct savings-realism uncertainty identified), but structural linkage failed silently — no existing node was updated to reflect the uncertainty, so the proposal lacked the graph state needed for downstream question selection. This is a different failure mode from 57J.56 which DID update nx54wwa successfully.
**What remains unproven:** Whether Answer 2 would reuse or duplicate; downstream investigation viability; cross-domain generalisation. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. **Production code changed:** NO. **Ollama calls:** 0 live calls (update rejected before mutation). APPEND_EOF