confidence-engine/docs/experiment-59b3.md

Experiment 59B.3 — Do-Nothing Baseline as Explicit Graph Structure

Branch: feature/question-formulation-v0.24 Date: 2026-08-12 Status: Complete Following: 59B.2 which showed the engine creates a net-benefit trade-off but do-nothing baseline remains semantic (not structural).

Objective

When both action and do-nothing consequences are stated explicitly, does the engine structurally represent both sides of the comparison and connect them to the decision?

Specifically: does the engine create a dedicated do-nothing cost node rather than treating "stay put" as invisible background context?

Context route

Read only:

• docs/current-handoff.md (latest section)
• docs/experiment-59b2.md (preceding experiment for context)
• Fixture: tests/fixtures/pre-anchored-update-savings-realism.json
• Harness: scripts/reproduce-multi-turn-investigation.mjs

Do not load older experiment history.

Fixed starting graph

Fixture: tests/fixtures/pre-anchored-update-savings-realism.json

Existing unresolved question:

n_savings_realism — Are the projected office savings from relocation realistic? — status = unknown

Fixed answer (exact, verbatim)

The £2 million annual saving from relocating is real. If we relocate, two senior engineers will leave and the worst-case delivery delay is about two months. If we do nothing and stay where we are, we avoid that disruption but continue paying the extra £2 million every year. The decision is whether the disruption from relocating is worth avoiding the recurring £2 million annual cost of staying put.

Contains:

• KNOWN BENEFIT: £2 million annual saving (relocating)
• KNOWN CONSEQUENCE: two senior engineers leave
• BOUNDED DOWNSIDE: worst-case delivery delay ≈ two months
• DO-NOTHING BASELINE: stay put → continue paying extra £2M/year
• DO-NOTHING BENEFIT: avoid relocation disruption

Does NOT contain: hints about investigation strategy.

Execution

Host/model: qwen-claude:latest at http://192.168.1.111:11434 Calls: startCalls=0, updateCalls=1, totalCalls=1 Retries: 0 (harness had initial failures due to model non-determinism; final successful run used same harness + fixture + answer)

---

Results

UPDATE output (accepted)

HTTP status: 200
Stage: update_applied
Validation errors: none

updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: resolved, newValue: "confirmed", reason: "User explicitly confirmed the £2 million annual saving from relocating is real."}]
resolvedUnknownNodeIds: ["n_savings_realism"]

addedNodes: [
  {
    id: n_disruption_impact,
    label: "Operational impact of relocation disruption",
    description: "Uncertainty regarding the precise cost and delay consequences of losing two senior engineers and facing a two-month delivery slowdown, so that it can be weighed against the confirmed £2M annual savings.",
    kind: unknown,
    status: unknown,
    confidence: medium
  }
]

addedEdges: [{fromNodeId: n_relocation_state, toNodeId: n_disruption_impact, relationship: causes}]

selectedQuestion: "What would clarify operational impact of relocation disruption in this situation?"
selectedQuestion.nodeId: "n_disruption_impact"

Resulting persistent graph (3 nodes, 2 edges):

• n_relocation_state — Engineering team relocation consideration — status=provisional
• n_savings_realism — Are the projected office savings from relocation realistic? — status=resolved ✓
• n_disruption_impact — Operational impact of relocation disruption — status=unknown, kind=unknown

Edges:

• n_savings_realism → n_relocation_state (depends_on)
• n_relocation_state → n_disruption_impact (causes)

---

Assessment

1. Relocation benefit

PRESERVED ONLY IN TEXT

n_savings_realism was resolved with newValue: "confirmed" — this captures the user's acceptance status but loses the exact figure (£2 million) and unit (annual). The resolved node carries no structured £2M/year claim as evidence. This is a regression compared to 59B.2 which preserved "confirmed £2M annual saving" with more precision.

2. Two-engineer departure + two-month delay

PARTIALLY REPRESENTED

Both consequences are embedded in n_disruption_impact's description:

"losing two senior engineers and facing a two-month delivery slowdown"

However, they are bundled into one unknown node (same pattern as 59A.1) and neither is treated as a known observation — they're both subsumed under an uncertainty about "cost consequences." This means the engine could not investigate each independently nor distinguish known-from-uncertain epistemic states for these two elements.

3. Do-nothing recurring cost (£2M/year)

PRESERVED ONLY IN TEXT

"continue paying the extra £2 million every year" does not appear as any structural node or edge. The figure is implicitly present only in the trade-off framing within n_disruption_impact's description ("weighed against the confirmed £2M annual savings"). A downstream query looking for a dedicated do-nothing cost node would find nothing.

4. Do-nothing benefit (avoid disruption)

PRESERVED ONLY IN TEXT

"we avoid that disruption" is not represented in any graph structure. The concept of avoiding disruption is implicit in the trade-off framing but has no node, edge, or explicit structural representation.

5. Alternative structure

NO ALTERNATIVE STRUCTURE

Only one option (relocate) has any structural representation beyond the starting state node. The do-nothing alternative ("stay put") has zero nodes representing it. The graph contains a single action path with its consequences as an unknown — not two competing alternatives.

6. Trade-off linkage

PARTIALLY LINKED

The trade-off exists in n_disruption_impact's description text: "so that it can be weighed against the confirmed £2M annual savings." This frames a comparison between consequences and savings. However, neither side of the comparison is an independently retrievable node — the comparison is prose, not graph topology.

7. Next question quality

WEAK

"What would clarify operational impact of relocation disruption in this situation?" asks about one side of the comparison (relocation's disruption). It does not compare both alternatives. A stronger question at this stage would be: "What evidence would determine whether the £2M/year savings outweigh the cost of two senior engineers leaving and a two-month delay?" — which explicitly compares both sides.

---

Classification: B — TRADE-OFF GOOD, BASELINE STILL IMPLICIT

The engine produced decision-relevant reasoning (trade-off framing within n_disruption_impact) but the do-nothing baseline remains text/context rather than explicit graph structure. This is the same pattern and same gap as 59B.2 — confirming that the engine does not independently create do-nothing structural nodes when the answer contains them.

Why:

The update:

1. Correctly resolved savings-realism (✓)
2. Framed a trade-off question about disruption costs (✓)
3. Linked disruption consequences to the relocation state (✓)
4. Did not create a do-nothing cost node (✗)
5. Did not represent the do-nothing benefit as structure (✗)
6. Created only one action path, not two alternatives (✗)

What the engine understood correctly:

1. Resolution of savings-realism: Correctly resolved based on "real" language.
2. Trade-off framing: The unknown node describes consequences that should be "weighed against" savings — this shows the engine grasps the decision context.
3. Decision relevance: Chose to investigate impact consequences rather than precision-chasing the two-month estimate.
4. Causal linkage: Created a causes edge from relocation state to disruption impact.

What it flattened or omitted:

1. Do-nothing baseline: Both do-nothing cost and benefit disappeared from structural representation entirely. This is the experiment's primary failure mode.
2. Consequence granularity: Engineer departure and delivery delay remain bundled in one unknown (same class as 59A.1).
3. Figure preservation: £2 million/year reduced to just "confirmed" — no amount or unit preserved on the resolved node.
4. Alternative representation: The graph only represents the action path, not both options of the decision.

What uncertainty it chose to pursue next:

n_disruption_impact — quantifying the operational impact consequences of relocating. This is one side of the comparison, not the full trade-off itself.

Does that question compare the alternatives or only examine one side: ONE SIDE ONLY

The question "What would clarify operational impact of relocation disruption?" examines only the action (relocate) side. It does not explicitly compare relocate vs stay-put. A follow-up investigation step would be needed to bring both sides into a comparison structure.

---

Comparison to 59B.2

Criterion	59B.2	59B.3
Savings preserved	YES (confirmed £2M annual saving)	PARTIAL ("confirmed" only, no figure/unit)
Known consequence preserved as observation	YES (observation node)	NO (bundled into unknown)
Bounded downside preserved	YES (in observation desc.)	PARTIAL (in unknown desc., bundled)
Do-nothing baseline	SEMANTIC ONLY	ABSENT FROM GRAPH STRUCTURE
Do-nothing benefit	IMPLICIT IN TRADE-OFF	ABSENT FROM GRAPH STRUCTURE
Alternative structure	ONE ACTION + NET-BENEFIT NODE	SAME — NO SEPARATE BASELINE
Decision-relevance reasoning	YES	YES (trade-off framing)
Next question quality	GOOD (net benefit evidence)	WEAK (one side only)

Key difference: 59B.3 lost the engineer departure from being a structural observation node and bundled it into an unknown. It also lost the precise £2M figure on the resolved node. The do-nothing baseline gap persists identically.

---

What this establishes:

1. The do-nothing baseline gap is stable across repeated runs — 59B.2 and 59B.3 both show the same pattern where "stay put" consequences remain text, not structure.
2. The engine frames trade-off reasoning when presented with explicit alternatives, even without a dedicated do-nothing node.
3. Known consequences can collapse into unknowns when bundled together — 59B.3 lost the observation-vs-unknown distinction seen in 59A.3 and 59B.2.

What this does NOT prove:

1. Stability of consequence granularity — one run only; the bundling of engineer departure + delivery delay may or may not persist across runs.
2. Whether the engine can represent both alternatives in a different scenario where do-nothing is framed differently.
3. Cross-domain generalisation — single domain case only.
4. Whether answerMeaning fields capture the baseline — updateOnly mode doesn't print these on the accepted path (unknown whether they contain baseline information).

---

Production code changed: NO