confidence-engine/docs/experiment-58a2.md

Experiment 58A.2 — Evidence + Uncertainty with One Competing Demand

Branch: feature/semantic-action-contract-v0.23 Starting HEAD: 3218469cfb081d5f9e7e3ed1a763482fdd2a241f Experiment commit: pending

Objective

Can the engine preserve a concrete figure while also creating one new uncertainty?

58A.1 showed good uncertainty reasoning but lost the central £2m figure while creating two new unknowns. This experiment reduces the answer to one concrete figure + one qualification + one new assumption to test whether information loss was caused by competing structural demands rather than a fundamental inability to represent qualified evidence.

Fixed scenario (from fixture)

"We are considering relocating the engineering team to reduce operating costs."

Existing unresolved uncertainty: "Are the projected office savings from relocation realistic?"

Fixed answer

The projected saving is about £2 million per year, but I don't trust that figure until we know what penalty we would pay to exit the existing lease.

Configured model: qwen-claude:latest at http://192.168.1.111:11434

CALL ACCOUNTING

• startCalls: 0
• updateCalls: 1
• totalCalls: 1
• Retries: 0

UPDATE RESULT

• HTTP status: 200
• Stage: update_applied
• Validation errors: none
• structuralActionRequired: null (known gap from 57J.69)

Proposal mutations

updatedNodes: [
  {nodeId: n_savings_realism, previousStatus: unknown, newStatus: provisional,
   previousValue: null, newValue: "~£2,000,000/year (pending lease exit cost)",
   reason: "User provided a provisional savings projection that is contingent on unresolved lease termination costs."}
]
resolvedUnknownNodeIds: []
addedNodes: [
  {id: n_lease_penalty_uncertainty, label: "Lease exit penalty amount",
   description: "Financial penalty required to break the current London office lease before relocation,
                 because it directly determines whether the ~£2M/year savings projection will actually materialize.",
   kind: unknown, status: unknown}
]
addedEdges: [
  {fromNodeId: n_savings_realism, toNodeId: n_lease_penalty_uncertainty,
   relationship: depends_on, confidence: high,
   description: "Realizing the projected savings depends on quantifying the lease exit penalty."}
]
selectedQuestion.nodeId: "n_savings_realism"

Selected question

"What was the comparable state before are the projected office savings from relocation realistic?" nodeId: n_savings_realism

(Note: question text appears malformed — template injection failure producing grammatically broken sentence.)

Resulting persistent graph (3 nodes, 2 edges)

node: id=n_relocation_state,      kind=state,     label=Engineering team relocation consideration, status=provisional
node: id=n_savings_realism,       kind=unknown,   label=Are the projected office savings from relocation realistic?, status=provisional
node: id=n_lease_penalty_uncertainty, kind=unknown, label=Lease exit penalty amount, status=unknown
edge: from=n_savings_realism,     to=n_relocation_state,     relationship=depends_on
edge: from=n_savings_realism,     to=n_lease_penalty_uncertainty, relationship=depends_on

Reasoning assessment

£2m figure: PRESERVED AS QUALIFIED — the value ~£2,000,000/year (pending lease exit cost) survives on the savings-realism node. It is preserved with qualifier text, though not at full precision ("about £2 million" → "~£2,000,000") and not as a standalone reported_claim node.

Qualification: WEAKENED — The status change from unknown → provisional on n_savings_realism is the primary signal of weakening. "Provisional" suggests tentative acceptance rather than active investigation. The qualification survives in newValue text ("pending lease exit cost") and in the reason prose, but structurally the node no longer functions as an unresolved question — it functions as a tentative fact that needs verification.

Lease-exit uncertainty: STRUCTURALLY REPRESENTED — dedicated unknown node with kind=unknown, status=unknown, explicit description tying it to the savings figure, plus a depends_on edge from n_savings_realism to this node. The structural representation is stronger than 58A.1's lease-exit handling.

Original savings-realism uncertainty: WEAKENED — The node identity persists (n_savings_realism still exists, not duplicated). However, the status change from unknown → provisional means it no longer signals "unresolved investigation target" — it signals "tentatively accepted but needs verification." This is a degradation of uncertainty signaling that could mislead downstream question selection and Behaviour Selection.

Evidence / uncertainty linkage: CLEARLY LINKED — The depends_on edge from n_savings_realism to n_lease_penalty_uncertainty structurally encodes the dependency relationship: realizing savings depends on quantifying the penalty. Description reinforces this ("Realizing the projected savings depends on quantifying the lease exit penalty.").

Next question quality: WRONG — "What was the comparable state before are the projected office savings from relocation realistic?" is a grammatically broken template injection (combining "What was the comparable state before [X]?" with "[X]" = full unknown label). It does not materially help determine whether the £2m figure is realistic.

Classification: E — IDENTITY FAILURE

The original savings-realism uncertainty node's status was degraded from unknown to provisional, weakening its identity as an unresolved investigation target. This is not a correct resolution (status remains unknown-ish but with degraded semantics), nor is it simply "preserved." The uncertainty exists in a degraded state that could mislead downstream reasoning stages about the investigation's health.

Additionally, the selected question is malformed and fails to pursue any material unresolved issue.

What the engine preserved correctly:

1. The £2m/year figure survived as qualified evidence (newValue on existing node)
2. The lease-exit uncertainty was structurally represented with a dedicated unknown node
3. Evidence and new uncertainty are clearly linked via depends_on edge + description
4. No duplicate savings-realism uncertainty was created
5. No-resolve guard worked (resolvedUnknownNodeIds is empty)

What it lost or weakened:

1. The savings-realism uncertainty identity — degraded from unknown to provisional, losing its function as an active investigation target
2. Question quality — malformed sentence that does not pursue the material unresolved issue
3. Precision of the £2m figure ("about £2 million" → "~£2,000,000")

What uncertainty it chose to pursue next:

n_savings_realism (the existing savings-realism unknown), but the question text is broken and does not target the lease-exit penalty or any other material issue.

Was that the best available next uncertainty:

YES — n_savings_realism is the correct investigation target, but the execution of the question (malformed text) renders this moot.

Comparison with 58A.1:

58A.1 lost the £2m figure entirely but preserved savings-realism as unknown and produced a grammatically coherent (if debatable) next question. 58A.2 preserves both the figure and the new uncertainty, but at the cost of degrading the savings-realism node from unknown to provisional and producing a malformed question. The trade-off is clear: reducing competing demands (2 unknowns → 1 unknown) solved the evidence-loss problem but introduced a status-degradation failure. This establishes that evidence preservation and uncertainty preservation are not simply inverses of each other — there is a separate mechanism controlling node status that can degrade uncertainty identity even when evidence survives.

What this establishes:

1. Reducing competing structural demands (one unknown instead of two) allows the figure to survive as qualified evidence
2. The status-degradation failure (unknown → provisional) is a distinct failure mode from evidence loss — it occurs alongside successful evidence preservation
3. Question-formulation templates have a defect when targeting existing (non-newly-created) unknown nodes

What this does NOT prove:

• Whether the status-degradation issue would recur with different model configurations or repeated runs
• Whether a dedicated reported_claim node type would avoid the status conflict
• Whether question-formulation templates can be fixed to handle existing-node targets correctly
• Cross-domain generalisation of any finding

Production code changed: NO

Prompt changed during experiment: NO

Harness/tooling changed: NO

Ollama calls beyond harness count: 0

Dev server disturbed: NO