confidence-engine/docs/experiment-60b27.md

Experiment 60B.27 — Clean Proposition Question Live Validation

Branch: feature/proposition-question-shape-v0.30
Date: 2026-08-13
Status: COMPLETE

Objective

Rerun the exact 60B.24 product-launch live case to verify that fix from 60B.26 preserves reasoning chain while producing a clean evidence-framed question without explanatory rationale.

Configured Model

qwen-claude:latest on http://192.168.1.111:11434

Hypothesis

A successful result should preserve the reasoning chain:

decision remains unresolved
customer-signing factor survives as first-class unknown
material target remains selected
no unrelated uncertainty invented

and improve only question shape:

final question uses evidence framing
final question contains the signing proposition
final question does NOT contain explanatory £700k / £1.2M rationale

Call Accounting

startCalls: 0
updateCalls: 1
totalCalls: 1
Retries: 0
Second live invocation: NO

UPDATE

HTTP: 200 (live update successful)
Stage: updateOnly
Validation errors: none

Proposal Applied

YES — ANSWER_2 injected as live answer containing the customer-signing unknown plus £700k/£1.2M financial context.

STRUCTURE

updatedNodes: []
addedNodes: [
  {
    "id": "unc_customer_signing_likelihood",
    "label": "Prospective enterprise customer signing likelihood",
    "description": "Unknown whether one prospective enterprise customer will sign if we launch this year, so that the remaining £700k of the expected £1.2M annual revenue is realized.",
    "kind": "unknown",
    "status": "unknown",
    "confidence": "medium"
  }
]
addedEdges: [
  {
    "id": "e-customer-to-launch-option",
    "fromNodeId": "unc_customer_signing_likelihood",
    "toNodeId": "opt_launch_this_year",
    "relationship": "may_cause",
    "confidence": "medium"
  }
]
resolvedUnknownNodeIds: []

Proposal selectedQuestion.nodeId: unc_customer_signing_likelihood

Final selectedQuestion.nodeId: unc_customer_signing_likelihood

Final selectedQuestion.question: "What evidence would clarify prospective enterprise customer signing likelihood?"

ASSESSMENT

Core reasoning chain

PRESERVED

• Decision remains unresolved (activeUnknownNodeId = n_product_launch_decision, status=unknown)
• Customer-signing factor survives as first-class unknown (kind=unknown, status=unknown)
• Material factor (customer-signing) is the selected target
• No unrelated uncertainty invented

Customer-signing factor

FIRST-CLASS UNKNOWN
Node unc_customer_signing_likelihood created with kind=unknown, status=unknown.

Preferred-target behaviour

MATERIAL FACTOR PRESERVED
The model selected the newly-created customer-signing unknown node — which IS the material factor identified by 60B.24's fix.

Question proposition

PRESERVED
The underlying proposition ("whether one prospective enterprise customer will sign if we launch this year") is preserved in spirit within the nominalized form "prospective enterprise customer signing likelihood." Both refer to the same decision variable.

Evidence framing

EVIDENCE FRAMED
Question uses "What evidence would clarify X?" pattern correctly activated by 60B.26's routing change.

Rationale contamination

NONE
The final question does NOT contain: £700k, £1.2M, expected annual revenue, resolving their intent, or financial impact. All explanatory rationale was successfully excluded from the user-facing formulation.

Source graph meaning

SOURCE DESCRIPTION PRESERVED
The node description preserves full rationale: "Unknown whether one prospective enterprise customer will sign if we launch this year, so that the remaining £700k of the expected £1.2M annual revenue is realized." — distinguishing user-facing cleanup from graph-state mutation.

Mechanism Note

Analysis of extractMeaning (lib/graph/question-formulator.js) reveals a gap: the targeted fix at line 121 checks /^whether\s+/i.test(strippedDescription) where strippedDescription only strips "uncertainty regarding/about" prefixes — NOT "unknown". For descriptions starting with "Unknown whether...", this check fails and falls through to generic label-based extraction, producing nominalized output ("Prospective Enterprise Customer Signing Likelihood") instead of a full whether... clause. The rationale stripping still works correctly because the extraction rule splits on the first semicolon within the matched text regardless. This gap is cosmetic: functionally equivalent meaning preserved, no rationale leakage.

60B.24 COMPARISON

Criterion	60B.24	60B.27
Final nodeId	uncertain_enterprise_customer_signing	unc_customer_signing_likelihood
Question shape	raw whether... + rationale + ?	clean "What evidence would clarify..."
Rationale in question	FULL (£700k, £1.2M)	NONE
Decision status	unresolved	unresolved
Customer factor presence	YES (node created)	YES (node created)

60B.24 produced: whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue, so that resolving their intent is needed to assess the financial impact...?

60B.27 produces: What evidence would clarify prospective enterprise customer signing likelihood?

Result Classification

A — LIVE CLEAN-QUESTION FIX CONFIRMED

Core reasoning chain preserved ✓
Target preserved ✓
Proposition preserved (functionally equivalent) ✓
Evidence framing used ✓
Explanatory rationale removed from final question ✓

Did 60B.26 Remove Rationale Contamination Live

YES

Did Evidence Framing Activate Live

YES — the "What evidence would clarify X?" template fired correctly through the decision_evidence path.

Did the Material Target Remain Stable

YES — customer-signing unknown remains selected as the question target.

What Improved Relative to 60B.24

1. Rationale removed: £700k/£1.2M financial context no longer leaks into user-facing question
2. Evidence framing active: "What evidence would clarify..." replaces raw proposition + "?" construction
3. Clean proposition: Question presents the signing decision variable without appended explanatory clauses

What Remains Weak or Unproven

1. Nominalized phrasing: Final question uses "prospective enterprise customer signing likelihood" (nominal) rather than a full whether... clause ("whether one prospective enterprise customer will sign if we launch this year"). The underlying proposition is preserved but the phrasing is less natural English. Root cause: extractMeaning description-start check (/^whether\s+/i) doesn't match "Unknown whether..." — a minor coverage gap in the targeted fix.
2. Cross-domain stability: Only one fixture tested. Nominalization behavior untested on other node-description patterns (e.g., "Uncertain whether...", bare "Whether...").

Production Code Changed

NO

Harness Used

scripts/reproduce-multi-turn-investigation.mjs in FIXTURE_MODE=updateOnly with exactly one update call.

Ollama calls: 1 MAXIMUM
Direct API calls: 0
Dev server disturbed: NO