confidence-engine/docs/experiment-59b4.md

Experiment 59B.4 — Explicit Two-Option Decision Structure

Branch: feature/question-formulation-v0.24 Date: 2026-08-12 Status: Complete Following: 59B.3 which showed both action and do-nothing consequences stated explicitly but stay-put remained text, not graph structure. This names both "relocate" and "stay put" symmetrically as competing options with consequences on each side to test whether explicit dual-option framing overcomes the baseline-invisibility pattern from 59B.2/59B.3.

Objective

When the user explicitly defines "relocate" and "stay put" as two competing options with consequences on both sides, does the engine represent both alternatives structurally rather than modelling only the proposed action?

Context route

Read only:

• docs/current-handoff.md (latest section)
• Fixture: tests/fixtures/pre-anchored-update-savings-realism.json
• Harness: scripts/reproduce-multi-turn-investigation.mjs

Do not load older experiment history.

Fixed starting graph

Fixture: tests/fixtures/pre-anchored-update-savings-realism.json

Existing unresolved question:

n_savings_realism — Are the projected office savings from relocation realistic? — status = unknown

Fixed answer (exact, verbatim)

There are really two options now.

Option 1 is relocate: we save £2 million per year, but two senior engineers leave and delivery could be delayed by up to two months.

Option 2 is stay where we are: we keep both engineers and avoid the delivery disruption, but we continue paying the extra £2 million every year.

I am deciding which of those two options leaves us better off overall.

Contains:

• OPTION 1 — RELOCATE: benefit = £2m/year saving; costs = two senior engineers leave, up to two months delivery delay
• OPTION 2 — STAY PUT: benefits = retain both engineers, avoid delivery disruption; cost = continue paying extra £2m/year
• DECISION FRAME: explicit comparison request ("I am deciding which of those two options leaves us better off overall")

Does NOT contain: hints about investigation strategy.

Execution

Host/model: qwen-claude:latest at http://192.168.1.111:11434 Calls: startCalls=0, updateCalls=1, totalCalls=1 Retries: 0

---

Results

UPDATE output (accepted)

HTTP status: 200
Stage: update_applied
Validation errors: none

updatedNodes: []
resolvedUnknownNodeIds: []

addedNodes: [
  {
    id: n_relocation_operational_impact,
    label: "Uncertainty about operational impact of relocation on engineers and delivery",
    description: "Potential loss of two senior engineers and up to two months of delivery delay resulting from relocation matters because it determines whether the £2M annual savings justify the immediate project disruption.",
    kind: unknown,
    status: unknown,
    confidence: low
  }
]

addedEdges: [{fromNodeId: n_relocation_operational_impact, toNodeId: n_relocation_state, relationship: depends_on}]

selectedQuestion: "What would clarify operational impact of relocation on engineers and delivery in this situation?"
selectedQuestion.nodeId: "n_relocation_operational_impact"

Resulting persistent graph (3 nodes, 2 edges):

• n_relocation_state — Engineering team relocation consideration — status=provisional
• n_savings_realism — Are the projected office savings from relocation realistic? — status=unknown (UNRESOLVED)
• n_relocation_operational_impact — Uncertainty about operational impact of relocation on engineers and delivery — status=unknown, kind=unknown

Edges:

• n_savings_realism → n_relocation_state (depends_on)
• n_relocation_operational_impact → n_relocation_state (depends_on)

---

Assessment

1. Relocate option

UNAVAILABLE as structural entity. No dedicated node representing the relocate option or its consequences as an independent branch. The relocate facts (£2M saving, two engineers leaving, two-month delay) appear only in the description prose of one new unknown node (n_relocation_operational_impact), not as a retrievable option structure.

2. Stay-put option

UNAVAILABLE as structural entity. No node whatsoever representing "stay put" or its consequences (retain engineers, avoid disruption, continue paying £2M/year). Despite the user explicitly naming it as Option 2, the graph contains zero evidence of it.

3. Relocate consequences

For £2m/year saving, two senior engineers leave, up to two months delay: PARTIALLY REPRESENTED. The facts are extracted into description prose but not on any structural node. Notably, savings_realism remains status=unknown — the user's explicit confirmation of the £2M saving was not structurally captured as a resolved fact.

4. Stay-put consequences

For retain both engineers, avoid disruption, continue paying extra £2m/year: TEXT ONLY (if at all). The description references "£2M annual savings" as a comparison phrase in prose but contains no structural representation of any stay-put element.

5. Alternative separation

COLLAPSED INTO ONE TRADE-OFF NODE. Despite the answer explicitly framing two competing options with "I am deciding which of those two options", the engine produced a single undifferentiated unknown about relocation impact. The graph represents only one direction of inquiry (relocate's operational impact), not a structure containing both alternatives.

6. Decision linkage

TEXTUAL COMPARISON ONLY. The comparison appears only in description prose ("whether the £2M annual savings justify the immediate project disruption"). There is no structural node or edge that links two alternatives to an overall decision/comparison. The decision itself has no graph representation.

7. Later recoverability

Could a later graph-only reasoning step recover the relocate case? NO — relocate facts only exist embedded in prose of a single unknown node's description. No structured option branch to query.

Could a later graph-only reasoning step recover the stay-put case? NO — no structural representation exists for any stay-put element anywhere in the graph.

8. Next question

"What would clarify operational impact of relocation on engineers and delivery in this situation?" asks about one side only (relocate's disruption). It does not compare both alternatives, despite the user explicitly stating "I am deciding which of those two options leaves us better off overall." The answer frames a comparison; the question ignores it.

WEAK.

---

Classification: B — BOTH OPTIONS PRESENT, STRUCTURE INCOMPLETE

Both options appear in the description prose of one node (the model grasped both alternatives existed), but neither is represented as an independently recoverable structural entity. The stay-put baseline remains text despite being named explicitly and symmetrically. This continues the 59B.2/59B.3 pattern: explicit dual-option language does not cause the engine to create a two-option decision structure.

Why:

The update:

1. Did NOT resolve savings_realism (status remains unknown) — the user's confirmation of the £2M saving was structurally ignored
2. Created exactly one new unknown node about relocation operational impact
3. Did NOT create separate nodes for either option despite explicit dual-option framing
4. Did NOT represent any stay-put element as structure
5. Collapsed both alternatives into prose within a single unknown's description
6. Generated a question that investigates only the relocate side, ignoring the comparison the user just requested

What the engine understood correctly:

1. Material facts extraction: The model extracted "two senior engineers", "two months of delivery delay", and "£2M annual savings" into the description — the information is present in text.
2. Trade-off awareness: The description references whether savings justify disruption, showing the model grasps the decision context.

What it flattened or omitted:

1. Both alternatives collapsed into one unknown. Despite explicit "two options now" language and symmetric consequence listing, only relocate impact was structurally represented. Stay-put disappeared entirely from graph structure.
2. Savings confirmation ignored. updatedNodes: [] — the user's clear statement about saving £2M/year was not used to resolve or update any existing node.
3. No decision/comparison structure. The user explicitly framed a comparison ("deciding which of those two options leaves us better off overall"), but no decision node, comparison node, or dual-branch structure was created.
4. Stay-put consequences absent from graph. Retained engineers, avoided disruption, and continuing £2M/year — all gone from structural form.

What uncertainty it chose to pursue next:

n_relocation_operational_impact — whether the operational impact of relocating can be quantified. This continues investigating one option's consequences rather than addressing the user's explicitly stated need to compare two options.

Could resolving that uncertainty realistically distinguish the alternatives? DEBATABLE

Quantifying relocate's disruption could inform comparison, but it doesn't address what happens with stay-put. Without the stay-put side, resolution of this single unknown is insufficient to answer the decision. It advances comparison only partially and incompletely.

---

What this establishes:

1. Explicit dual-option framing does NOT overcome the stay-put baseline invisibility. Naming both options symmetrically with "Option 1" / "Option 2" and explicitly stating "I am deciding which of those two options leaves us better off overall" did not produce structural representation for the do-nothing alternative. This is the same outcome class as 59B.2/59B.3 despite significantly stronger explicit framing.
2. The engine can extract facts from dual-option prose and embed them in description text — it does not lose information from complex structured answers at the extraction level.
3. Savings_realism remains unresolved even after the user provides a clear relocation decision context with confirmed savings — the model does not automatically infer that confirmatory language applies to existing unknowns.

What this does NOT prove:

1. Whether stronger resolution triggers work. The answer did not use explicit resolution language ("the saving IS real", "I CONFIRM") — it stated the saving as a fact within an option description. This may explain why savings_realism wasn't resolved.
2. Cross-option reasoning capability. A single update call cannot test whether downstream reasoning steps would naturally create comparison structure once both sides exist.
3. Whether the issue is model limitation or prompt design. The model's behaviour may be consistent with its training rather than a prompt defect.
4. Stability across runs. Single run only.

---

Production code changed: NO Prompt changed during experiment: NO Validator changed during experiment: NO Harness changed during experiment: NO Vitest run: NO Ollama calls beyond harness count: 0 Dev server disturbed: NO