robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: test trade-off decomposition

Browse Source
This commit is contained in:
robbond
2026-08-12 16:51:39 +01:00
parent 8c5c4b5b75
commit c3c5351143
2 changed files with 250 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+29
View File
@@ -316,6 +316,35 @@ The engine correctly closed savings-realism, preserved verified evidence, repres
## Ollama calls beyond harness count: 0
## Dev server disturbed: NO
### Experiment 59A.2 — Trade-off Decomposition: Known Consequence vs Uncertain Consequence
**Branch:** `feature/question-formulation-v0.24`
**Date:** 2026-08-12
**Status:** Complete
**Following:** 59A.1 which established the boundary shift but bundled two consequences into one node. This tests whether the engine can separate a *known* consequence (engineers leaving) from an *uncertain* consequence (delivery impact).
**Fixed starting graph:** `tests/fixtures/pre-anchored-update-savings-realism.json`
**Fixed answer:** "I am comfortable that the £2 million annual saving is real. We know we would lose two senior engineers if we relocate. What I don't know is whether losing them would materially delay delivery, or by how much."
**Execution:** qwen-claude:latest at http://192.168.1.111:11434. startCalls=0, updateCalls=1, totalCalls=1.
**Result:** UPDATE 422 rejected at `proposal_compatibility` — "selectedQuestion must be a single non-compound question". No selectedQuestion was produced (null).
**Reasoning assessment from rejected snapshot:**
- Savings-realism: CLOSED CORRECTLY (resolved, in resolvedUnknownNodeIds)
- £2m annual saving: PRESERVED AS ACCEPTED EVIDENCE (in userSupportedMeaning)
- Two senior engineers leaving: REPRESENTED BUT LEFT UNCERTAIN (extracted in meaning but no separate structural node for the known fact)
- Delivery impact: REPRESENTED AS UNRESOLVED (dedicated unknown node n_delivery_delay_impact created)
- Relationship: PARTIALLY LINKED (causal link encoded in free text within one node, not as typed edge)
- Granularity: COLLAPSED INTO UMBRELLA UNKNOWN (both consequences bundled into one unknown node: "Impact of losing two senior engineers on delivery timelines")
- Next question: NONE (null — caused rejection)
**Classification: B — PARTIAL DECOMPOSITION**
The engine extracted all three semantic elements in userSupportedMeaning and created a dedicated unknown for delivery impact. However, the known-vs-uncertain distinction was not preserved structurally — both consequences were compressed into one unresolved-question frame. The selectedQuestion was null, triggering validator rejection. Full record in `docs/experiment-59a2.md`. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed.
---
### Experiment 58B.2 — Verified Uncertainty Resolution
**Objective:** When the user explicitly verifies a figure and states confidence in its realism, does the engine resolve the existing savings-realism uncertainty rather than merely changing its value?