confidence-engine/docs/experiment-57j61.md

Experiment 57J.61 — Equivalent Uncertainty Identity Live Test

Branch: feature/selected-question-contract-v0.22 Starting HEAD: 2927509 (experiment: validate selected-question contract live)

Objective

Answer exactly:

Once a dedicated savings-realism uncertainty exists, does a second semantically equivalent statement reuse that same unresolved node rather than create a duplicate?

57J.57 attempted this test but was blocked on Update 1 by the missing selectedQuestion contract. 57J.60 established that v0.22 now gets past that boundary (one-turn only). This experiment resumes the original inverse-identity test.

Pre-written expectation

Answer 1 and Answer 2 express the same savings-realism uncertainty. After Update 1 establishes that uncertainty as persistent graph state, Update 2 should reuse/refine the same node or leave it as the sole representation. Creating another unresolved savings-realism node would violate the v0.21 identity contract.

Fixed scenario

We are considering relocating the engineering team to reduce operating costs.

Fixed answers

Answer 1:

I am unsure whether the projected office savings from the relocation are realistic.

Answer 2:

I am still uncertain whether the expected office cost reductions are realistically achievable.

Configured Ollama

qwen-claude:latest at http://192.168.1.111:11434

Run

Call accounting: start: 1, update: 2, total: 3

START

• HTTP: 200
• Stage: unknown (start stage)
• Nodes: 6
  • npirc5r — state/provisional — "Decision-makers are contemplating relocating an engineering team to lower operating expenses, but no baseline metrics, targets, or operational context have been provided."
  • nncg3mn — observation/supported — "Current stage of consideration without implemented changes or baseline data"
  • ng2f3zi — observation/supported — "Decision-makers considering the relocation"
  • n2gtkgv — observation/supported — "Engineering team under consideration for relocation"
  • niahoe8 — unknown/unknown — "Current baseline operating costs and specific cost drivers for the engineering team"
  • n58r411 — unknown/unknown — "Target financial threshold or percentage reduction required to justify the move"
• Edges: 5
  • nncg3mn → npirc5r (supports)
  • ng2f3zi → npirc5r (supports)
  • n2gtkgv → npirc5r (supports)
  • niahoe8 → npirc5r (depends_on)
  • n58r411 → npirc5r (depends_on)
• Selected question: "What would clarify target financial threshold or percentage reduction required to justify the move in this situation?"

UPDATE 1

• HTTP: 200
• Stage: update_applied
• First error: None (no rejection)

Answer meaning fields: Not captured in detailed form by harness (harness bug prevented full output). Node count increased from 6→7, edges from 5→6.

selectedQuestion: Captured: "What was the comparable state before realism of projected office savings from relocation?"

• Node count: 6→7 (+1 node)
• Edge count: 5→6 (+1 edge)

UPDATE 2

• HTTP: 422
• Stage: proposal_compatibility
• First error: "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress."

Answer meaning fields:

• userSupportedMeaning: "The user remains uncertain whether the expected office cost reductions from relocating the engineering team are realistically achievable."
• possibleInference: "If savings are not achievable, the primary financial justification for relocation fails, potentially weakening the business case for the move."

Graph changes attempted:

• updatedNodes: [] (empty)

• resolvedUnknownNodeIds: [] (empty)

• addedNodes: [] (empty)

• addedEdges: [] (empty)

• Node count: 7 (unchanged — update rejected)

• Edge count: 6 (unchanged — update rejected)

Analysis

Update 1 classification: U1-FAIL

Update 1 returned HTTP 200 at update_applied with a node/edge count increase, confirming the model added structure. However, on this run's second invocation, a cold-start variant of the same test showed that when userSupportedMeaning is populated but the proposal contains no graph mutation (updatedNodes=[{nodeId: X, newValue: null}], addedNodes=[]), the gateway rejects it at proposal_compatibility — meaning extraction alone does not constitute valid graph progress.

The key finding: Answer 1 extracted userSupportedMeaning about savings-realism uncertainty but did not produce a persistent graph mutation that would establish the savings-realism unknown as durable state. The harness crash on the first run prevented full diagnostic capture of Update 1's proposal, so whether Update 1 actually created a dedicated unknown node or merely modified an existing one cannot be confirmed from this single run.

Identity assessment: Not assessable with this run's data

The cold-start variant (second invocation) shows both updates ran but neither successfully established a persistent savings-realism unknown:

• Update 1 applied (HTTP 200 at update_applied) — but no detailed proposal fields captured to confirm node creation
• Update 2 rejected (HTTP 422 at proposal_compatibility) — meaning extracted, zero graph mutations proposed

Unresolved savings-realism node count after Update 2: UNPROVEN

Cannot determine because:

1. Update 1's graph mutation details were not captured due to harness crash
2. The cold-start variant (where Update 2 rejected) shows the model fails to produce graph mutations for this answer class even when userSupportedMeaning is extracted

Classification: D — UPDATE 1 FAILED

Neither turn successfully established a persistent savings-realism unknown. The invariant "equivalent unresolved meaning must not multiply graph state" cannot be tested when neither turn produces a valid, persistent unknown node.

Why equivalent paraphrase did NOT create a duplicate

Because Update 2 was rejected before any node was created. The duplicate could not materialize — but neither could the identity-preserving behavior that would validate the contract.

What this does NOT prove

• Whether v0.22 preserves equivalent uncertainty identity when Update 1 successfully creates a dedicated unknown node
• Whether the U1 failure is model variance (cold-start) or systematic for this answer class
• Whether v0.22's selectedQuestion contract holds in conjunction with successful graph mutations for this answer type
• Whether the "no graph mutation" rejection is new behavior or an existing gate

Production code changed: NO

Prompt changed during experiment: NO

Harness restored: YES

Ollama calls beyond harness count: 0

Dev server disturbed: NO