confidence-engine/docs/experiment-58b2.md

# Experiment 58B.2 — Verified Uncertainty Resolution

**Branch:** `feature/question-formulation-v0.24`
**Date:** 2026-08-12
**Status:** Complete
**Previous context:** Follows 58B.1 which showed the engine preserves qualified evidence while keeping uncertainty open. This tests the opposite boundary: when the user explicitly verifies and confirms realism, does the engine resolve?

---

## Objective

When the user explicitly says the £2m figure has now been verified and is realistic, does the engine resolve the existing `n_savings_realism` uncertainty rather than merely changing its value or weakening its status?

---

## Fixed Starting Graph

Fixture: `tests/fixtures/pre-anchored-update-savings-realism.json`

```
id:   n_savings_realism
label: Are the projected office savings from relocation realistic?
kind: unknown
status: unknown
value: null
confidence: low
dependsOn: [n_relocation_state]
```

---

## Fixed Answer

> We have now verified the projected saving at about £2 million per year, including the relevant lease exit costs, and I am confident that estimate is realistic.

Components:
1. **Concrete value:** approximately £2 million per year
2. **Verification:** the estimate has now been checked
3. **Relevant dependency addressed:** lease exit costs included
4. **Explicit confidence:** user now believes the estimate is realistic

No new uncertainty introduced.

---

## Execution

```bash
FIXTURE_MODE=updateOnly \
ANSWER_2="We have now verified the projected saving at about £2 million per year, including the relevant lease exit costs, and I am confident that estimate is realistic." \
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
node scripts/reproduce-multi-turn-investigation.mjs
```

**Host/model:** qwen-claude:latest at http://192.168.1.111:11434
**Calls:** startCalls=0, updateCalls=1, totalCalls=1
**Retries:** 0

---

## Results

### Answer meaning

Not returned in the update response (updateOnly mode accepted without including answerMeaning in the output). The structural fields below are authoritative.

### Structural proposal

- `updatedNodes`: `[{"nodeId":"n_savings_realism","previousStatus":"unknown","newStatus":"resolved","previousValue":null,"newValue":"£2,000,000","reason":"User verified the projected annual savings including lease exit costs are realistic."}]`
- `resolvedUnknownNodeIds`: `["n_savings_realism"]`
- `addedNodes`: `[]`
- `addedEdges`: `[]`
- `structuralActionRequired`: null

### Persistent graph after update

```
node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional
node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=resolved
edge: n_savings_realism → n_relocation_state (depends_on)
```

### Selected question

None produced.

---

## Assessment

| Criterion | Result | Classification |
|-----------|--------|----------------|
| £2m figure | `"£2,000,000"` on n_savings_realism | PRESERVED AS VERIFIED |
| Existing uncertainty status | `unknown` → `resolved` | CORRECTLY RESOLVED |
| Identity of savings-realism node | Single node, unchanged id (`n_savings_realism`), no duplicates | ORIGINAL IDENTITY RESOLVED |
| Verification meaning | reason: "User verified the projected annual savings including lease exit costs are realistic." | PRESERVED |
| Next investigation | NONE — no consequential unresolved issue remains | GOOD |

### £2m figure analysis

The value `"£2,000,000"` preserves the core monetary figure. The "per year" unit is not explicit in `newValue` (unlike 58B.1 which had `"£2M/year (unverified)"`) but is preserved in the reason field ("projected **annual** savings"). This qualifies as PRESERVED AS VERIFIED — the amount is captured and the verification context survives.

### Uncertainty resolution analysis

Status clearly changed from `unknown` to `resolved`. The node id `n_savings_realism` appears in `resolvedUnknownNodeIds`. This is unambiguous correct resolution.

### Identity analysis

Exactly one savings-realism unknown node exists before and after the update. Same node id, same label, status transitions correctly. No duplicate created. ORIGINAL IDENTITY RESOLVED.

### Verification meaning analysis

The reason field on the updated node explicitly states: "User verified the projected annual savings including lease exit costs are realistic." This captures all four components of the user's answer (value, verification, lease costs, confidence). PRESERVED.

### Next investigation analysis

No selected question was produced. This is correct behavior — the existing uncertainty is resolved and no new consequential unresolved issue was introduced by the answer. GOOD.

---

## Classification: A — UNCERTAINTY CORRECTLY RESOLVED

- n_savings_realism correctly resolved (status → `resolved`)
- Included in `resolvedUnknownNodeIds`
- Verified £2m evidence survives as `"£2,000,000"` with full verification context in reason field
- No duplicate uncertainty created
- Same node id preserved (original identity resolved)
- No redundant question asked about realism
- No consequential unresolved issue remains to investigate

---

## What the engine understood correctly

1. **Resolution trigger:** The explicit "verified" and "confident...realistic" language triggered correct uncertainty resolution — status moved from `unknown` to `resolved`. This is the semantic boundary 58B.1 left open.
2. **Value extraction:** The figure was captured as `"£2,000,000"` — a clean monetary representation.
3. **Verification context:** The reason field captured all four answer components: value (£2m), verification status ("verified"), lease exit costs, and confidence ("realistic").
4. **No fabrication:** No new uncertainty nodes or edges were created from this answer that contained no new uncertainty.
5. **Identity preservation:** The original `n_savings_realism` was updated (not replaced or duplicated).
6. **Correct termination signal:** No selected question was produced, correctly reflecting that the existing investigation thread is complete.

---

## What it overstated, weakened, or lost

**Minor weakening of temporal unit:** The "per year" time unit is not explicit in `newValue` (which is `"£2,000,000"` rather than `"£2,000,000/year"`). However, the word "annual" in the reason field partially compensates. This does not affect the core resolution question — it is a secondary representation detail.

---

## What this establishes

1. When the user provides **explicit verification** AND **confidence about realism**, the engine correctly resolves the existing savings-realism uncertainty (status → `resolved` + inclusion in `resolvedUnknownNodeIds`).
2. This is the semantic opposite of 58B.1 and works correctly — the engine distinguishes between "unverified but plausible" (keep open) and "verified and confident" (resolve).
3. The verified £2m figure survives in persistent graph state with verification context captured.
4. No duplicate uncertainty is created during resolution.
5. The engine does not ask the resolved question again — it correctly terminates when no consequential unresolved issue remains.

---

## What this does NOT prove

1. **Single controlled case** — one answer, one model invocation. Stability across different answers or repeated runs is untested.
2. **Temporal unit preservation** — whether "per year" survives in `newValue` depends on answer phrasing and model behavior.
3. **answerMeaning fields** — the update response did not include answerMeaning, so whether userSupportedMeaning captured all four components independently of the reason field is unavailable.
4. **Multi-turn stability** — what happens when subsequent turns arrive after a resolved uncertainty is untested here.
5. **Cross-domain generalisation** — this is a single cost-savings domain case.

---

## Production code changed: NO
## Prompt changed: NO
## Validator changed: NO
## Harness changed: NO
## Vitest run: NO
## Ollama calls beyond harness count: 0
## Dev server disturbed: NO