confidence-engine/docs/archive/experiments/decision-closure-integration/experiment-60b2.md

Experiment 60B.2 — Independent Decision Sufficiency Without Explicit Stopping Cue

Branch: feature/decision-options-v0.25
Date: 2026-08-13
Status: Complete
Type: LIVE RUN — Single bounded update to test whether the engine independently recognises decision sufficiency when both options have quantified material costs but the user does NOT explicitly state the investigation is complete.

Objective

When both options have clearly quantified material costs (£600k one-off relocation vs £2M/year stay-put) but the user does not say "there are no other material differences" or "we now have enough information", does the engine independently recognise decision sufficiency or identify a genuinely material missing factor?

Hypothesis

A strong result may do either:

Path 1 — independent sufficiency: The engine concludes that the supplied evidence is sufficient to resolve the current decision context.

Path 2 — justified continuation: The engine keeps the decision open but identifies a specific material factor already grounded in the existing graph or answer that could realistically change the comparison.

A weak result would:

• Ask a generic follow-up
• Invent a new risk
• Chase irrelevant precision
• Reopen already-known facts

Fixed Starting Graph

Fixture: tests/fixtures/pre-anchored-decision-options.json

Node	Kind	Status	Label
n_relocation_state	state	provisional	Engineering team relocation consideration
opt_relocate	option	known	Relocate to Manchester
opt_stay_put	option	known	Stay in London (Status Quo)
n_relocation_decision	unknown	unknown	Which option leaves us better off overall?

Edges: opt_relocate → n_relocation_decision (contained_in); opt_stay_put → n_relocation_decision (contained_in).

Configured Model

• Model: qwen-claude:latest
• Ollama base URL: http://127.0.0.1:3000 (dev server)

Fixed Answer (verbatim, exact)

We have now quantified the full financial impact of the relocation disruption, including replacing the two senior engineers and the delivery delay, at about £600,000 as a one-off cost. Staying put costs us an extra £2 million every year.

Note: Unlike 60B.1, this answer does NOT contain "there are no other material differences" or any equivalent explicit stopping cue.

Execution

Host/model: qwen-claude:latest at http://127.0.0.1:3000. startCalls=0, updateCalls=1, totalCalls=1. Hard one-call boundary.

Result

HTTP status: 200 — first call succeeds, no validation rejection.

Proposal mutations

updatedNodes: [
  {nodeId: "opt_relocate", previousStatus: "known", newStatus: "known", newValue: 600000,
   reason: "Quantified one-off disruption cost including engineer replacement and delivery delay."},
  {nodeId: "opt_stay_put", previousStatus: "known", newStatus: "known", newValue: 2000000,
   reason: "Quantified annual extra operational cost of remaining in London."},
  {nodeId: "n_relocation_decision", previousStatus: "unknown", newStatus: "supported",
   newValue: "Financial comparison now quantified; relocation yields net savings after ~3.6 months assuming delay remains ≤ 2 months.",
   reason: "Incorporated user-provided financial figures to enable comparative evaluation."}
]

resolvedUnknownNodeIds: []

addedNodes: []

addedEdges: []

selectedQuestion: "What outcome would demonstrate enough value to justify continuing?"
selectedQuestion.nodeId: "n_relocation_decision"

Resulting persistent graph (4 nodes, 2 edges)

Node	Kind	Status	Label
n_relocation_state	state	provisional	Engineering team relocation consideration
opt_relocate	option	known	Relocate to Manchester
opt_stay_put	option	known	Stay in London (Status Quo)
n_relocation_decision	unknown	supported	Which option leaves us better off overall?

Edges:

• opt_relocate → n_relocation_decision (contained_in)
• opt_stay_put → n_relocation_decision (contained_in)

No nodes added. No edges added. The decision node was not resolved — its status changed from unknown → supported.

Assessment

1. Decision identity: PRESERVED

The original n_relocation_decision node survived — same id, label "Which option leaves us better off overall?". Status transitioned from unknown → supported. Not duplicated or replaced. The decision context still exists as exactly one node.

2. Relocate identity: PRESERVED

opt_relocate survived unchanged as a kind=option node with status=known and label="Relocate to Manchester". newValue=600000 was set on the update list, but the original node was not replaced or duplicated. Count: 1.

3. Stay-put identity: PRESERVED

opt_stay_put survived unchanged as a kind=option node with status=known and label="Stay in London (Status Quo)". newValue=2000000 was set on the update list, but not replaced or duplicated. Count: 1.

4. £600k relocation cost: OPTION-OWNED DESCRIPTION (numeric)

The engine set newValue: 600000 directly on the opt_relocate option node — a numeric value field on the option itself, with reason text "Quantified one-off disruption cost including engineer replacement and delivery delay." This is better than the decision-options fixture's original null-value state. However, no dedicated metric node was created (unlike 60B.1). The value lives on the option node rather than as a first-class standalone graph entity with typed edges.

Classification: OPTION-OWNED DESCRIPTION — the number is attached to the option node but not elevated to independent structure.

5. £2m/year stay-put cost: OPTION-OWNED DESCRIPTION (numeric)

The engine set newValue: 2000000 directly on the opt_stay_put option node with reason text "Quantified annual extra operational cost of remaining in London." Same pattern as relocate — numeric value on the option, no separate metric node.

Classification: OPTION-OWNED DESCRIPTION — attached to the option node but not first-class structure.

6. Comparison completeness: PARTIAL

Both figures are present on their respective option nodes as numeric newValue fields. However:

• No dedicated metric/evidence nodes were created (unlike 60B.1)
• No edges connect these values between each other or to any comparison node
• The values are option-internal rather than independently queryable graph entities
• The time-unit distinction (one-off vs recurring) is lost — both have value type "number" with no unit field

The comparison exists implicitly in the two option newValue fields but lacks first-class structural representation.

Classification: PARTIAL

7. Decision treatment: KEPT OPEN GENERICALLY

The engine did not resolve the decision (resolvedUnknownNodeIds is empty). Status changed from unknown → supported, which indicates the evidence has some bearing on the decision but is insufficient for resolution. The selected question was "What outcome would demonstrate enough value to justify continuing?" targeting n_relocation_decision.

This is NOT a specific material reason for continuation — it does not identify any concrete missing factor grounded in the existing graph or answer. It is a generic request for additional justification evidence without naming what that evidence should be about.

8. Decision resolution: REMAINS OPEN WITHOUT JUSTIFICATION

The decision remained open (resolvedUnknownNodeIds = []). The status shifted to supported but no resolution occurred. The supporting newValue on the decision node ("Financial comparison now quantified; relocation yields net savings after ~3.6 months assuming delay remains ≤ 2 months.") shows the engine DID perform a preliminary financial comparison and computed an approximate payback period. However, it treated this as insufficient for resolution rather than sufficient.

9. Conclusion direction: FAVOURS RELOCATE (implicit)

The newValue on n_relocation_decision states "relocation yields net savings after ~3.6 months assuming delay remains ≤ 2 months" — this clearly favours relocate in its reasoning. The engine is keeping the decision open but has internally concluded that relocate is better if the delay constraint holds.

10. Precision chasing: NO

The engine did not ask for more precise figures for either £600k or £2m/year. It computed a rough ~3.6 month payback and accepted the comparison as partially sufficient. No precision-chasing behaviour detected.

11. Selected question quality

Question: "What outcome would demonstrate enough value to justify continuing?"

This is a generic meta-question about decision justification — it does not identify any specific missing factor in the graph or answer. It essentially says "tell me more about why you want to proceed" without acknowledging that both cost sides are already quantified and compared. This reopens the investigation at a higher level of abstraction rather than closing it (like 60B.1) or identifying a grounded missing factor.

Classification: WEAK — The question is not wrong per se but does not engage with the material state of the graph (both costs quantified, comparison computed). It's a generic continuation prompt.

Why the result matters

The engine demonstrated it CAN do the financial comparison (£600k vs £2M/year → ~3.6 month payback). This is genuine reasoning. But it treated this partially sufficient comparison as requiring more evidence rather than sufficient evidence — all without any explicit stopping cue from the user.

This reveals a systematic tendency: the engine does not independently recognise when quantified comparison data is sufficient for decision resolution. It defaults to keeping decisions open and asking generic follow-ups, even when both cost sides are clearly stated and numerically comparable.

Classification: C — GENERIC UNCERTAINTY CHASING

The decision stays open and the engine generates a generic meta-question ("What outcome would demonstrate enough value to justify continuing?") that does not identify any concrete grounded factor from the existing graph or answer. The engine demonstrated it can compute a rough comparison (~3.6 month payback) but treated partial evidence as insufficient without any material justification for needing more.

What the engine understood correctly:

1. Both costs are quantified and attributed — set numeric newValue on each option node (600000 on opt_relocate, 2000000 on opt_stay_put).
2. Financial comparison is possible — computed "~3.6 months assuming delay remains ≤ 2 months" as the payback period in the decision node's newValue.
3. Both identities preserved — opt_relocate and opt_stay_put survived unchanged; n_relocation_decision survived with status transition (unknown → supported).
4. No precision chasing — did not ask for more precise figures.
5. No fabricated risks — did not invent new unknowns or uncertainties.

What it unnecessarily reopened or lost:

1. Did not recognise evidence sufficiency — the engine computed a meaningful financial comparison (£600k one-off vs £2M/year recurring → ~3.6 month payback) but treated this as insufficient rather than sufficient to resolve the decision context.
2. Generic continuation question — "What outcome would demonstrate enough value to justify continuing?" does not identify any specific missing factor from the graph or answer. It is a generic justification request, not a material information gap identification.
3. Lost time-unit distinction — both £600k and £2M/year were stored as plain numeric values without distinguishing one-off (GBP) from recurring (GBP/year) units. This loses an important structural distinction for the comparison.
4. No first-class evidence nodes — unlike 60B.1, no metric/evidence nodes were created. The financial data lives only on option newValue fields with no independent queryable graph entities and no typed edges between them.

What this establishes:

1. Without an explicit stopping cue, the engine defaults to keeping decisions open rather than resolving them — even when it has computed a meaningful financial comparison.
2. The engine CAN perform rough financial comparisons (payback estimation) but does not use those computations as a sufficiency trigger.
3. The supported status transition is used instead of resolution when evidence partially supports a conclusion but falls short of the model's internal sufficiency threshold.

What this does NOT prove:

1. Whether the engine needs an explicit cue or whether any sufficient-evidence pattern would work — we only tested one specific gap (no "no other material differences" phrase). Different evidence structures might produce different results.
2. Stability across repeated runs — one run only; cold-start variance may produce different outcomes on repeated runs.
3. Whether the generic continuation is intentional behaviour or a model limitation — could be a design choice (always require explicit closing) or a gap in reasoning about sufficiency.
4. Cross-domain generalisation — single domain case only.
5. Whether the ~3.6 month payback computation reflects genuine understanding or pattern-matching — the rough approximation is plausible but not rigorously derived.

Production code changed: NO

Prompt changed: NO

Validator changed: NO

Harness changed: NO

Vitest run: NO

Ollama calls: 1

Direct API calls: 0

Dev server disturbed: NO