confidence-engine/docs/experiment-60b76.md

Experiment 60B.76 — Live Verification of Sufficiency Question Fix

Date: 2026-08-15
Branch: feature/sufficiency-decision-detection-v0.46
Preceded by: Experiment 60B.75 (fix applied + tests pass)

Objective

Does the 60B.75 fix make the exact no-confirmation production case select the focused sufficiency confirmation/discovery question while keeping the decision open?

Configured environment

• Model: qwen-claude:latest
• Ollama base URL: http://192.168.1.111:11434
• Confidence Engine base URL: http://127.0.0.1:3000

Input

• Fixture: tests/fixtures/pre-anchored-product-launch-customer-signing.json
  • Pre-anchored state: decision (n_product_launch_decision) unknown; customer signing (n_enterprise_customer_signing) unknown, activeUnknownNodeId = n_enterprise_customer_signing.
• Answer (exact, no paraphrase): "The enterprise customer has now confirmed in writing that they will not sign if we launch this year, so the £700,000 of expected annual revenue from them will not be received."
• Explicit sufficiency confirmation: NO

Run

FIXTURE_MODE=updateOnly \
FIXTURE_PATH=tests/fixtures/pre-anchored-product-launch-customer-signing.json \
ANSWER_2="The enterprise customer has now confirmed in writing that they will not sign if we launch this year, so the £700,000 of expected annual revenue from them will not be received." \
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
node scripts/reproduce-multi-turn-investigation.mjs

• startCalls: 0
• updateCalls: 1
• totalCalls: 1
• Retries: 0

Results

Proposal accepted: YES (HTTP 200)

updatedNodes:

[
  {
    "nodeId": "n_enterprise_customer_signing",
    "previousStatus": "unknown",
    "newStatus": "resolved",
    "reason": "Enterprise customer confirmed in writing they will not sign if we launch this year."
  },
  {
    "nodeId": "n_product_launch_decision",
    "previousStatus": "unknown",
    "newStatus": "resolved",
    "reason": "Material uncertainty resolved; evidence now sufficiently distinguishes net value of launch this year versus wait twelve months."
  }
]

resolvedUnknownNodeIds:

["n_enterprise_customer_signing", "n_product_launch_decision"]

addedNodes: []

addedEdges: []

structuralActionRequired: null

finalActiveUnknownNodeId: null

finalSelectedQuestion: null

Analysis

The sufficiency confirmation question did not fire. Instead, the decision was closed deterministically during apply-proposal.js processing — before formulateQuestion() was ever called.

Root cause: The deterministic sufficiency-closure gate (line 1673 of apply-proposal.js) fires when:

1. All material uncertainties are resolved → hasRemainingMaterialFactors() returns false
2. This triggers automatic decision closure with reason "sufficient evidence distinguishes net value"

The 60B.75 fix exposed State B in selectReasoningPattern() (so reasoningPattern === "decision" now reaches the sufficiency template selection), but did not address the earlier deterministic closure gate in apply-proposal.js. The decision resolves before question formulation can occur.

Structural impact:

• Customer: Resolves correctly → status = resolved, negative meaning preserved
• Decision: Prematurely closes → status = resolved (should be unknown)
• Active target: Gone → activeUnknownNodeId = null (decision was the only unknown)
• Question: Never formulated → finalSelectedQuestion = null
• No new nodes/edges

Classification: C — DECISION CLOSES

The deterministic sufficiency closure in apply-proposal.js resolves the decision before the question-formulation path is reached. This is also F — STRUCTURAL REGRESSION because meaning loss does not occur, but the structural behavior (premature closure) prevents testing of State B's question selection.

Comparison with 60B.74

Field	60B.74	60B.76
Customer status	resolved	resolved ✓
Customer meaning preserved	yes	yes ✓
Decision status	unknown (kept open)	resolved (closed prematurely) ✗
activeUnknownNodeId	n_product_launch_decision	null
finalSelectedQuestion	decision_threshold_outcome	null (never reached)
Classification	B	C + F

The decision closed in 60B.76 because hasRemainingMaterialFactors() correctly returns false after the customer factor resolves — and the deterministic closure gate fires before question formulation can evaluate sufficiency confirmation presence.

What this proves

1. The 60B.75 fix successfully exposes State B to reasoningPattern detection. The decision-context path is now reachable.
2. But an earlier gate prevents reaching that path: the apply-proposal.js deterministic closure fires before question formulation, closing the decision prematurely when no material factors remain.
3. The sufficiency confirmation check must either be lifted from deterministic closure or moved into the closure gate itself — checking for explicit confirmation before auto-closure.

What remains unproven

1. Whether State B's sufficiency template selection works correctly (question formulation is not reached).
2. Whether removing the kind gate in selectReasoningPattern causes any unintended side effects when question formulation IS reached.

Production code changed: NO

Prompt changed: NO

Schema changed: NO

Harness changed during experiment: NO

Vitest run: NO

Ollama calls: 1

Direct API calls: 0