confidence-engine/docs/experiment-60b47.md

Experiment 60B.47 — Negative-outcome decision closure

Date: 2026-08-14 Branch: feature/closure-metadata-capture-v0.39

Purpose

Test whether the same decision lifecycle closes cleanly when the material customer uncertainty resolves negatively instead of positively (60B.46).

Hypothesis: clean lifecycle handling should not depend on whether the answer is positive or negative.

Method

One bounded live update using the 60B.44/46 pre-anchored fixture:

• Fixture: tests/fixtures/pre-anchored-product-launch-customer-signing.json
• Mode: updateOnly (single Update, no Start)
• Answer: "No. The enterprise customer has now confirmed in writing that they will not sign if we launch this year, so the £700,000 of expected annual revenue from them will not be received. There are no other material uncertainties between launching this year and waiting twelve months."
• Model: qwen-claude:latest via Ollama (http://192.168.1.111:11434)

Call accounting

startCalls: 0
updateCalls: 1
totalCalls: 1
retries: 0

Results

HTTP

• HTTP status: 422
• Stage: proposal_compatibility — rejected (validation failure)
• Validation error: "Unknown node updated to resolved must also appear in resolvedUnknownNodeIds: \"n_product_launch_decision\""

Proposal content (from rejection snapshot)

The engine did produce a proposal before rejection. The rejected-snapshot reveals its structure:

{
  "answerMeaning": {
    "userSupportedMeaning": "The prospective enterprise customer will not sign if we launch this year, so £700,000 of expected annual revenue will not be received. There are no other material uncertainties between the two timing options.",
    "possibleInference": "This financial reduction materially weakens the immediate-launch option relative to waiting twelve months."
  },
  "updatedNodes": [
    { "nodeId": "n_enterprise_customer_signing", "newValue": null },
    { "nodeId": "n_product_launch_decision", "newValue": null }
  ],
  "resolvedUnknownNodeIds": [
    "n_enterprise_customer_signing"
  ],
  "addedNodes": [],
  "addedEdges": []
}

Structural analysis of the rejected proposal

What got right:

1. Negative meaning preserved correctly. userSupportedMeaning accurately reflects: "The prospective enterprise customer will not sign if we launch this year, so £700,000 of expected annual revenue will not be received."
2. Same customer factor identified. n_enterprise_customer_signing — the exact same node ID as 60B.46.
3. Same decision targeted. n_product_launch_decision — the exact same decision node as 60B.46.
4. Both nodes placed in updatedNodes for resolution.
5. No new unknowns invented. addedNodes: [].
6. No new edges created. addedEdges: [].

The defect:

• n_product_launch_decision appeared in updatedNodes (meaning the model proposed updating it to resolved), but was missing from resolvedUnknownNodeIds.
• The validator correctly caught this inconsistency and rejected the proposal.

Assessment

Criterion	Result
Customer factor identity	RESOLVED IN PROPOSAL (rejected before application)
Negative meaning preservation	PRESERVED — userSupportedMeaning accurately captures "will not sign" + £700k revenue lost
Decision state in proposal	RESOLVED (in updatedNodes)
Identity preservation	Same node IDs as 60B.46
New uncertainty invented	NONE
Structural validation	FAILED — resolvedUnknownNodeIds inconsistent with updatedNodes

Classification: G — DIFFERENT FIRST FAILURE

Structural validation failure: The proposal was rejected at proposal_compatibility because the model included n_product_launch_decision in updatedNodes (proposing to resolve it) but omitted it from resolvedUnknownNodeIds.

The engine's semantic reasoning was correct — same customer factor, opposite meaning preserved, same decision targeted. The failure is purely structural: an internal consistency gap between updatedNodes and resolvedUnknownNodeIds when the model proposes a multi-node resolution in one turn.

Why this matters

This is a different failure class from 60B.46 (which showed clean closure) but reveals an important asymmetry:

• 60B.46 (positive): The model apparently produced resolvedUnknownNodeIds that included both nodes — or the decision was resolved through a different mechanism (e.g., deterministic post-processing) — and the proposal passed validation cleanly.
• 60B.47 (negative): The model explicitly listed both nodes in updatedNodes but forgot to include the decision node in resolvedUnknownNodeIds, causing structural rejection.

The semantic path is symmetric (same factor, same decision, correct meaning). The structural path is not yet symmetric. This is a fixable gap: the model needs consistent output of resolvedUnknownNodeIds when resolving multiple unknowns in one turn.

60B.46 comparison

Field	60B.46 (positive)	60B.47 (negative)
Same customer factor reused	YES (n_enterprise_customer_signing)	YES (n_enterprise_customer_signing)
Opposite answer meaning preserved	N/A	YES — userSupportedMeaning correct
Decision closure attempted in proposal	YES	YES (but structurally inconsistent)
Validation outcome	PASSED (422 equivalent not triggered)	REJECTED 422
Added unknown count	0	0
Structural path symmetric?	—	NO

What this proves

The engine's semantic reasoning is robust to answer polarity — the negative answer correctly identified the same factor, preserved its meaning, and targeted the same decision. However, the structural output contract is not yet symmetric: when resolving multiple unknowns simultaneously in one turn under negative framing, the model fails to consistently populate resolvedUnknownNodeIds.

What remains weak or unproven

• Whether the same proposal would pass if structured correctly (i.e., whether n_product_launch_decision should also appear in resolvedUnknownNodeIds).
• Whether positive vs negative answers trigger different output-template paths in the model.
• A targeted fix for multi-node resolution consistency in resolvedUnknownNodeIds.

Production code changed: NO