confidence-engine/docs/experiment-59b2.md

# Experiment 59B.2 — Independent Decision Relevance Reasoning

**Branch:** `feature/question-formulation-v0.24`
**Date:** 2026-08-12
**Status:** Complete
**Following:** 59A.3 (known-vs-uncertain decomposition) and 59B.1 (user-hinted decision relevance). This removes the user's hint about precision relevance.

## Objective

When the benefit, known consequence, bounded downside, and do-nothing cost are all stated, does the engine independently reason about decision relevance rather than merely asking for greater precision?

Specifically: does the engine recognise when more precision may not change the decision?

## Context route

Read only:
- `docs/current-handoff.md` (latest section)
- `docs/experiment-59b1.md` (preceding experiment 59B.1 for context)
- Fixture: `tests/fixtures/pre-anchored-update-savings-realism.json`
- Harness: `scripts/reproduce-multi-turn-investigation.mjs`

Do not load older experiment history.

## Fixed starting graph

Fixture: `tests/fixtures/pre-anchored-update-savings-realism.json`

Existing unresolved question:
```
n_savings_realism — Are the projected office savings from relocation realistic? — status = unknown
```

## Fixed answer (exact, verbatim)

```text
The £2 million annual saving is real. We know two senior engineers would leave if we relocate. The worst-case delivery impact is about two months of delay. If we stay where we are, we keep paying the extra £2 million every year.
```

Contains:
- **KNOWN BENEFIT:** £2 million annual saving
- **KNOWN CONSEQUENCE:** two senior engineers would leave
- **BOUNDED DOWNSIDE:** worst-case delivery delay ≈ two months
- **DO-NOTHING BASELINE:** stay put → continue paying extra £2 million every year

Does **NOT** contain:
- "Would more precision change the decision?"
- "How exactly many weeks will delivery slip?"

The engine must decide what matters next.

## Execution

**Host/model:** qwen-claude:latest at http://192.168.1.111:11434
**Calls:** startCalls=0, updateCalls=1, totalCalls=1
**Retries:** 0

---

## Results

### UPDATE output (accepted)

```
HTTP status: 200
Stage: update_applied
Validation errors: none

updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: resolved, newValue: "confirmed £2M annual saving", reason: "User explicitly confirms the projected office savings are real."}]
resolvedUnknownNodeIds: ["n_savings_realism"]

addedNodes: [
  {
    id: n_talent_impact_clarity,
    label: "Impact of senior engineer departure on delivery",
    description: "Relocation causes two senior engineers to leave, creating a worst-case 2-month project delay.",
    kind: observation,
    status: supported,
    confidence: high
  },
  {
    id: n_net_relocation_benefit,
    label: "Whether relocation net benefit is positive",
    description: "Uncertainty about if the confirmed £2M annual savings outweigh the loss of two senior engineers and up to 2-month delay because it determines if the relocation achieves its core operational objective.",
    kind: unknown,
    status: unknown,
    confidence: medium
  }
]

addedEdges: [
  {fromNodeId: n_talent_impact_clarity, toNodeId: n_relocation_state, relationship: weakens},
  {fromNodeId: n_net_relocation_benefit, toNodeId: n_savings_realism, relationship: depends_on}
]

selectedQuestion: "What evidence would clarify relocation net benefit is positive?"
selectedQuestion.nodeId: "n_net_relocation_benefit"
```

Resulting persistent graph (4 nodes, 3 edges):
- `n_relocation_state` — Engineering team relocation consideration — status=provisional
- `n_savings_realism` — Are the projected office savings from relocation realistic? — status=resolved ✓
- `n_talent_impact_clarity` — Impact of senior engineer departure on delivery — status=supported, kind=observation
- `n_net_relocation_benefit` — Whether relocation net benefit is positive — status=unknown, kind=unknown

---

## Assessment

### 1. £2m annual saving
**PRESERVED AS KNOWN BENEFIT**

Node `n_savings_realism` resolved with value `"confirmed £2M annual saving"`. The exact figure (£2 million) and the unit (annual) are preserved on a resolved node.

### 2. Two-engineer departure
**PRESERVED AS KNOWN CONSEQUENCE**

New dedicated observation node `n_talent_impact_clarity`, kind=observation, status=supported, label="Impact of senior engineer departure on delivery", description explicitly states "Relocation causes two senior engineers to leave". This is structurally separate from uncertainty — an observation, not a question.

### 3. Two-month worst-case bound
**PRESERVED AS BOUNDED DOWNSIDE**

The same observation node's description includes "creating a worst-case 2-month project delay." The upper bound survives structurally within the observation node. It was **not** weakened to open-ended uncertainty.

### 4. Do-nothing baseline
**PRESERVED ONLY IN SEMANTIC/TEXT CONTEXT**

"If we stay where we are, we keep paying the extra £2 million every year" is not represented as a dedicated structural node in the graph. However, it is preserved semantically within `n_net_relocation_benefit`'s description which frames the comparison: "whether the confirmed £2M annual savings outweigh the loss of two senior engineers and up to 2-month delay because it determines if the relocation achieves its core operational objective." The baseline cost is implicit in the trade-off framing rather than explicit as a structural node.

### 5. Decision comparison
**TRADE-OFF PRESENT BUT BASELINE WEAK**

The engine created a net-benefit unknown (`n_net_relocation_benefit`) that inherently frames a trade-off. However, the do-nothing baseline ("stay put costs £2M/year") is not structurally represented as its own node. The comparison is present in the description but lacks structural grounding for the do-nothing side.

### 6. Remaining uncertainty chosen
**DECISION-CHANGING UNKNOWN**

"Whether relocation net benefit is positive" — this is the genuine decision boundary at this stage. Without knowing the net benefit (benefit minus consequences), no relocation decision can be made. Resolving this uncertainty would directly enable or prevent a go/no-go decision.

### 7. Independent decision relevance
**YES**

The engine did not ask for more precise delay information despite having only an approximate "about two months" figure. Instead, it:
- Resolved the known benefit (savings-realism → resolved)
- Preserved the known consequence structurally as a separate observation node (distinct epistemic state from uncertainty)
- Preserved the bounded downside within that observation
- Created a net-benefit trade-off unknown
- Asked about evidence for that trade-off

This demonstrates independent distinction between "uncertainty exists" and "this uncertainty is worth resolving."

### 8. Precision chasing
**NO**

The selected question asks "What evidence would clarify relocation net benefit is positive?" — this pursues the net-benefit trade-off, not a more precise delivery-delay figure. The two-month bound was preserved as-is within the observation node.

---

## Classification: A — INDEPENDENT DECISION-RELEVANCE REASONING

The engine preserves the key comparison inputs and independently focuses on information that could plausibly change the decision (net benefit of relocation), not on precision-chasing the bounded estimate.

### Why:

The update accepted all four factual elements from the answer:
1. **Known benefit preserved:** £2m annual saving resolved on `n_savings_realism`
2. **Known consequence preserved structurally:** `n_talent_impact_clarity` (kind=observation, status=supported) — separate epistemic node from uncertainty
3. **Bounded downside preserved:** "worst-case 2-month project delay" embedded in the observation node's description, not weakened to open-ended uncertainty
4. **Do-nothing baseline semantically preserved:** implicit in `n_net_relocation_benefit`'s trade-off framing ("whether... savings outweigh the loss... because it determines if the relocation achieves its core operational objective")

The engine independently chose to pursue a decision-changing unknown (net benefit) rather than asking for more precise delay information — exactly what this experiment was designed to test.

### What the engine understood correctly:

1. **Epistemic state separation:** The two senior engineers leaving is an *observation* (known), not an *unknown*. This is a distinct epistemic category from the delivery-delay bound, which is also preserved as bounded information within the same observation node — not treated as uncertain.
2. **Decision relevance over precision:** The engine did NOT reopen the "about two months" estimate to ask for exact figures. It recognised that the remaining question is whether the trade-off (savings vs consequences) is positive, not how precise the delay estimate is.
3. **Proper resolution of savings-realism:** The "The £2 million annual saving is real" language triggered correct resolution — no duplicate, no lingering uncertainty.
4. **Bounded downside carried forward:** The two-month upper bound survived in the observation node's description without being weakened or converted to open-ended uncertainty.

### What it lost or flattened:

**Do-nothing baseline is only semantic, not structural.** The explicit "If we stay where we are, we keep paying the extra £2 million every year" is not a dedicated node. It survives in the trade-off description but would be inaccessible to downstream structural queries that need the do-nothing cost as an independent reference point. This is the same class of loss seen in 59B.1 (baseline lost structurally) — here it's only slightly better because at least the trade-off framing preserves the *comparison logic*, even if not the explicit node.

### What uncertainty it chose to pursue next:

`n_net_relocation_benefit` — "Whether relocation net benefit is positive." This is the core decision question: does the £2M/year saving outweigh losing two engineers plus up to 2-month delay?

### Does that uncertainty materially affect whether relocation is worth doing: YES

Without knowing whether the net benefit is positive, no relocation decision can be made. The next question ("What evidence would clarify...") is appropriately broad at this stage — it invites identifying which specific evidence (quantified engineer departure cost, quantified delay cost, etc.) would tip the balance.

---

## Comparison to 59B.1

| Criterion | 59B.1 (user hinted) | 59B.2 (no hint) |
|-----------|---------------------|------------------|
| Savings preserved | YES | YES |
| Known consequence preserved | NO — LOST | YES — observation node |
| Bounded downside preserved | ABSENT from graph | YES — in observation description |
| Do-nothing baseline | STRUCTURALLY LOST | SEMANTICALLY PRESERVED (not structural) |
| Decision relevance question | YES — but user-provided | YES — independently generated |
| Precision chasing | NO | NO |

**Key improvement over 59B.1:** The engine preserves all four factual elements structurally (or semantically in the case of do-nothing baseline), whereas 59B.1 lost engineer departure and bounded downside entirely from the graph.

---

## What this establishes:

1. **The engine can independently distinguish "uncertainty exists" from "this uncertainty is worth resolving"** — even without an explicit user hint asking about decision relevance of precision, it chose a decision-changing unknown rather than precision-seeking.
2. **Known consequences are preserved as observation nodes** when the answer distinguishes them from uncertainty (59A.3's epistemic separation pattern survives into 59B.2).
3. **Bounded downside information is carried forward** within observation nodes without being weakened to open-ended uncertainty.
4. **The engine frames a net-benefit trade-off** as the remaining decision question, which is appropriate for this stage of investigation.

## What this does NOT prove:

1. **Stability** — one run only; cold-start variance may produce different outcomes on repeated runs.
2. **Do-nothing baseline structural representation** — the explicit recurring cost is still not a dedicated structural node; this remains a semantic-only preservation.
3. **Granularity of consequence investigation** — the observation node bundles both engineer departure and delay impact into one description; independent quantification of each would be needed for precise net-benefit analysis.
4. **Whether the engine would independently create the do-nothing cost node** if the answer didn't contain explicit "if we stay where we are" language that hints at it.
5. **Cross-domain generalisation** — single domain case only.

---

## Critical evidence rule check

Classification A requires the proposal/graph to preserve enough of benefit, known consequence, bounded downside, and do-nothing baseline for the question to be grounded in the decision:

- Benefit: ✓ resolved on `n_savings_realism` with "confirmed £2M annual saving"
- Known consequence: ✓ dedicated observation node `n_talent_impact_clarity`
- Bounded downside: ✓ preserved within observation description ("worst-case 2-month project delay")
- Do-nothing baseline: ⚠ semantic in trade-off description only, not structural

The question "What evidence would clarify relocation net benefit is positive?" is grounded in all four elements (three structural, one semantic). This qualifies as A with the noted caveat about do-nothing baseline.

Production code changed: NO