robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: validate uncertainty identity live

Browse Source
This commit is contained in:
robbond
2026-08-11 18:02:11 +01:00
parent a476431048
commit eb524d08e1
2 changed files with 211 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+14
View File
@@ -1041,3 +1041,17 @@ One fresh live run through the production `startCase()` → `updateCase()` path.
**Test results:** All 49 tests pass. No regression in existing prompt structure tests.
**Configured Ollama:** none used. **Production code changed:** NO. **Ollama calls:** 0.
### Experiment 57J.56 — Uncertainty Identity Live Validation
Tested whether v0.21 now preserves a focused savings-realism uncertainty separately from broader related cost unknowns. **Classification: A — V0.21 IDENTITY RULE WORKS LIVE.** Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic."
**Start:** HTTP 200 | unknown stage | 8 nodes | 5 edges | Selected question: "What would clarify how long it will take for ongoing savings to offset upfront expenses and productivity dips in this situation?"
Relevant unresolved unknowns at start: (1) `nx54wwa` — operating expenses (metric, known), (2) `nkmuu21` — one-time relocation costs (unknown, unknown), (3) `nt0asmb` — team output loss/turnover (unknown, unknown), (4) `n4j29jl` — payback period timing (unknown, unknown).
**Update 1:** HTTP 200 | update_applied stage | no errors/validation failures. New node `n_savings_realism` added with label "Validity of projected office savings estimates from relocation" (unknown/unknown) — distinct from all four start unknowns. `nx54wwa` updated: known → provisional. Added edge: `e-doubt-savings` weakens → `nx54wwa`. No equivalent pre-existing unknown existed; focused uncertainty added as a new node rather than absorbed into broader cost nodes. Identity result: **ADDED DISTINCT UNCERTAINTY**. Meaning: **FAITHFUL** (doubt preserved, not strengthened).
**Key finding:** The v0.21 prompt clarification ("same uncertainty = same resolution question") works on a live run with qwen-claude:latest. Unlike 57J.53 (where uncertain status degraded an unrelated node's status without creating a dedicated unknown), v0.21 correctly creates and preserves a dedicated savings-realism unknown with proper structural linkage (`weakens` edge). This resolves the prompt semantic-identity gap diagnosed in 57J.54.
**What remains unproven:** Repeated-run stability (single run); downstream investigation viability past Update 2; cross-domain generalisation; paraphrase invariance; whether reuse/refine still triggers when an equivalent unknown already exists. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. **Production code changed:** NO. **Ollama calls:** 1 live call (start + update via harness). APPEND_EOF