confidence-engine/docs/experiment-60b37.md

Experiment 60B.37 — Customer-signing decision closure

Date: 2026-08-14 Branch: feature/customer-signing-followup-fixture-v0.35

Purpose

Test whether resolving the last material uncertainty of an existing unresolved decision updates that same factor in place and closes the decision cleanly, without duplication or unnecessary continuation.

Precondition

The pre-anchored fixture from 60B.36 already contains:

• n_product_launch_decision (unknown, status=unknown)
• opt_launch_this_year (option, status=known)
• opt_wait_twelve_months (option, status=known)
• n_enterprise_customer_signing (unknown, status=unknown, active target)

With edge n_enterprise_customer_signing -> opt_launch_this_year (contained_in).

Fixed input

Yes. The enterprise customer has now confirmed in writing that they will sign if we launch this year, so the £700,000 of expected annual revenue from them is confirmed. There are no other material uncertainties between launching this year and waiting twelve months.

Execution

FIXTURE_MODE=updateOnly \
FIXTURE_PATH=tests/fixtures/pre-anchored-product-launch-customer-signing.json \
ANSWER_2="Yes. The enterprise customer has now confirmed in writing that they will sign if we launch this year, so the £700,000 of expected annual revenue from them is confirmed. There are no other material uncertainties between launching this year and waiting twelve months." \
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
node scripts/reproduce-multi-turn-investigation.mjs

Configured model: qwen-claude:latest Ollama base URL: http://192.168.1.111:11434

Call accounting

• startCalls: 0
• updateCalls: 1
• totalCalls: 1
• Retries: 0

Results

HTTP / stage

Stage: updateOnly — single bounded update through the production pipeline.

Validation errors

None reported.

updatedNodes (2)

1. n_enterprise_customer_signing

   • previousStatus: unknown → newStatus: resolved
   • reason: "Confirmed in writing that the enterprise customer will sign if launched this year, removing uncertainty about the £700,000 revenue stream."

2. n_product_launch_decision

   • previousStatus: unknown → newStatus: known
   • reason: "Prerequisite uncertainty resolved and user confirms no other material uncertainties remain between the options."

resolvedUnknownNodeIds

["n_enterprise_customer_signing"]

addedNodes

[] — zero.

addedEdges

[] — zero.

Final graph state (5 nodes, 3 edges)

id	kind	label	status
n_product_launch_state	state	Product launch timing consideration	provisional
opt_launch_this_year	option	Launch this year	known
opt_wait_twelve_months	option	Wait twelve months	known
n_product_launch_decision	unknown	Which option leaves us better off overall?	known
n_enterprise_customer_signing	unknown	Prospective enterprise customer signing status	resolved

£700k evidence preservation

The reason field on the updated n_enterprise_customer_signing node contains the prose reference to "£700,000 revenue stream." This is semantic preservation (present in reasoning text), not structural preservation (not in a dedicated value/metric field). The original description ("approximately £700,000 of the £1.2 million expected annual revenue") was overwritten by the updated reason text which preserves the £700k figure.

Proposal / selectedQuestion

• selectedQuestion: "What outcome would demonstrate enough value to justify launching?"
• selectedQuestion.nodeId: "n_product_launch_decision"

Note: n_product_launch_decision's status is known. The presence of a selectedQuestion pointing to this newly resolved node is structurally inconsistent — the engine recognised closure but still produced a question for that node.

Assessment

Existing customer factor

RESOLVED IN PLACE

The original node id n_enterprise_customer_signing survived and transitioned from unknown → resolved. No duplicate was created.

£700k confirmation

PRESERVED SEMANTICALLY

The figure appears in the updated reason text: "removing uncertainty about the £700,000 revenue stream." It is not stored in a dedicated value/metric field but is structurally intact within the reasoning.

Decision identity

PRESERVED

Original id n_product_launch_decision survived. Status changed to known. No duplication or replacement.

Decision state

RESOLVED (structurally) / KEPT OPEN FOR SPECIFIC MATERIAL REASON (question artifact)

The node's status is known with rationale: "Prerequisite uncertainty resolved and user confirms no other material uncertainties remain between the options." However, a selectedQuestion still references this node. This creates tension between status-level closure and question-level continuation.

Decision direction

NO DIRECTION RECORDED

The decision rationale is structural (prerequisites met), not directional (which option is preferred). No explicit preference was recorded.

Option identities

• Launch option (opt_launch_this_year): PRESERVED — id intact, status=known
• Wait option (opt_wait_twelve_months): PRESERVED — id intact, status=known

Duplication

• Customer-signing factor: 1 (exactly one node with that id)
• Decision context: 1 (exactly one decision node)

New uncertainty discipline

NONE

No new nodes added. No edges added. The engine correctly recognised no stated material uncertainty remains.

Final question

• Proposal selectedQuestion.nodeId: n_product_launch_decision
• Final selectedQuestion.nodeId: n_product_launch_decision
• Final selectedQuestion.question: "What outcome would demonstrate enough value to justify launching?"

Classification: SPECIFIC MATERIAL FOLLOW-UP (technically present but for a resolved node)

Classification

C — FACTOR RESOLVES BUT GENERIC CONTINUATION REMAINS

The customer-signing uncertainty resolved correctly in place. The decision status became known with correct rationale. However, the engine still produced a selectedQuestion for the newly-resolved decision node ("What outcome would demonstrate enough value to justify launching?") — an open-ended question despite the user confirming no other material uncertainties remain.

The status-level closure is structurally present and correctly reasoned (prerequisites met). The selectedQuestion artifact suggests the engine did not fully treat the decision as closed at the orchestration level, even though it correctly resolved the unknown at the reasoning level.

Critical evidence

Criterion	Result
original customer unknown resolved in place	YES
no duplicate customer factor	YES
original decision preserved	YES
decision resolved (status=known)	YES
both options preserved	YES
no unrelated new unknown	YES
no further selected question	NO — question present for resolved node

What the engine understood correctly

1. Reused n_enterprise_customer_signing (no duplication)
2. Resolved that factor in place with correct reasoning
3. Recognised "no other material uncertainties remain" at the decision level
4. Resolved n_product_launch_decision status to known
5. Preserved both option identities
6. Created no new nodes or edges

What it duplicated, reopened, or lost

Nothing was duplicated or lost at the node level. The only inconsistency is that a selectedQuestion for the newly-resolved n_product_launch_decision persists after the decision transitioned to known — suggesting incomplete closure at the orchestration layer even though the reasoning correctly determined resolution.

What this establishes

• The engine can resolve an existing unknown in place via direct user answer
• The engine can propagate that resolution to an existing decision node's status
• The engine does not duplicate material factors on confirmation answers
• The engine preserves option identities across the update

What this does NOT prove

• That a resolved decision produces no follow-up question (it did)
• That the engine correctly treats a known-status decision as closed at the orchestration level (question artifact suggests it may not)
• That the engine would make a directional recommendation if prompted further (none was recorded)
• Full lifecycle behaviour of the decision-closure → next-turn path