confidence-engine/docs/experiment-57j57.md

Experiment 57J.57 — Equivalent Uncertainty Reuse Live Validation

Branch: feature/uncertainty-identity-v0.21 Starting HEAD: eb524d0 (experiment: validate uncertainty identity live) Experiment commit: pending

Objective

Answer exactly:

After the graph contains a dedicated savings-realism uncertainty, does a second semantically equivalent expression of that uncertainty reuse/refine the existing node rather than create a duplicate?

57J.56 established the "distinct uncertainty" half of the identity rule.

This experiment tests the inverse half:

same resolution question
→ reuse/refine existing uncertainty
→ do not create duplicate

Fixed Inputs

Scenario:

We are considering relocating the engineering team to reduce operating costs.

Answer 1:

I am unsure whether the projected office savings from the relocation are realistic.

Answer 2:

I am still uncertain whether the expected office cost reductions are realistically achievable.

These two answers express the same resolution question. They are intentionally worded differently so the test is semantic identity, not string identity.

Pre-written Expectation

Answer 1 and Answer 2 express the same unresolved question: whether projected office savings are realistically achievable. Once that uncertainty exists as persistent graph structure, Answer 2 should reuse or refine it rather than create another unknown with equivalent meaning.

Run Results

Configured Ollama

• Base URL: http://192.168.1.111:11434 (from .env.local)
• Model: qwen-claude:latest

Dev Server

• Running on http://127.0.0.1:3000 (REUSE EXISTING)

CALL ACCOUNTING

startCalls: 1
updateCalls: 1
totalCalls: 2

Retries: 0 Supplementary scripts: NO

START

HTTP: 200 | Stage: unknown

Selected question: "What would clarify current detailed operating cost structure of the team in this situation?"

Nodes (7 total)

ID	Kind	Status	Label
ncwvq9x	state	provisional	Summary of the situation from the scenario text
nbb1z4m	observation	supported	Decision-making body ('We') evaluating the relocation
nduqivt	observation	supported	Engineering team targeted for relocation
nx54wwa	metric	known	Current and projected monthly/annual operating expenses for the engineering function
n20in8o	metric	known	Proposed physical or legal relocation of the engineering team to a new jurisdiction/location
nfq8rkd	unknown	unknown	Total one-time costs required for relocation (severance, hiring, infrastructure setup, legal/compliance)
nl723kx	unknown	unknown	Potential short- to medium-term loss in team output, morale, or turnover due to the move

Edges (4 total)

• e-sum-nbb1z4m supports → ncwvq9x
• e-sum-nduqivt supports → ncwvq9x
• e-unk-nfq8rkd depends_on → ncwvq9x
• e-unk-nl723kx depends_on → ncwvq9x

Relevant unresolved unknowns (costs/savings/relocation)

1. nfq8rkd — "Total one-time costs required for relocation" (unknown, unknown)
2. nl723kx — "Potential short- to medium-term loss in team output, morale, or turnover due to the move" (unknown, unknown)

Note: nx54wwa (current/projected operating expenses) is known, not unresolved.

UPDATE 1

HTTP: 422 | Stage: proposal_compatibility

First Error

selectedQuestion is required when consequential unresolved unknowns remain after resolving the answered unknown

Answer Meaning

{
  "userSupportedMeaning": "The user is unsure whether the projected office savings from the relocation are realistic.",
  "possibleInference": "If projections are unrealistic, the financial justification for relocating may be flawed, potentially leading to increased or unchanged operating costs."
}

Support Category / Resolution Guidance

supportCategory populated: NO (not present in answerMeaning) resolutionGuidance populated: NO (not present in answerMeaning) Structured path: NO — the structured field was not populated; meaning came through free-text userSupportedMeaning only.

Rejected Proposal Snapshot

{
  "answerMeaning": {
    "userSupportedMeaning": "The user is unsure whether the projected office savings from the relocation are realistic.",
    "possibleInference": "If projections are unrealistic, the financial justification for relocating may be flawed, potentially leading to increased or unchanged operating costs."
  },
  "updatedNodes": [],
  "resolvedUnknownNodeIds": [],
  "addedNodes": [
    {
      "id": "nsavings_reality",
      "kind": "unknown",
      "label": "Realism of projected office savings from relocation",
      "description": "Whether anticipated cost reductions match achievable financial outcomes, needed to decide if the relocation meets its core objective.",
      "parentId": null,
      "dependsOn": ["n20in8o"],
      "affects": [],
      "childIds": []
    }
  ],
  "addedEdges": [
    {
      "fromNodeId": "nsavings_reality",
      "toNodeId": "n20in8o",
      "relationship": "depends_on"
    }
  ]
}

Analysis of the Rejection

The model attempted to create a dedicated savings-realism unknown node (nsavings_reality) — which is the correct semantic interpretation. However, it also set updatedNodes: [] and resolvedUnknownNodeIds: [], meaning no existing nodes were updated or resolved. The proposal created a new consequential unresolved unknown without updating any existing node to reflect the uncertainty (e.g., downgrading nx54wwa from known to provisional as 57J.56 did).

The system then rejected the proposal because creating a new consequential unknown requires a selected question for follow-up, but the proposal lacked this linkage. The rejection is structural: the model produced valid semantics but failed to complete the required graph mutation (updating existing nodes to reflect uncertainty) that would normally trigger the selected-question path.

Update 1 Classification: U1-FAIL

No persistent savings-realism uncertainty was established in the graph because Update 1 was rejected. The proposed node (nsavings_reality) never entered the graph.

Reachable for Update 2?

NO. Update 1 failed, so by experiment protocol the run stops. Update 2 is not reached.

Classification: D — UPDATE 1 FAILED

The first turn never establishes the uncertainty needed for the inverse test. The model demonstrated correct semantic interpretation (it understood Answer 1 as savings-realism doubt and attempted to create a dedicated node), but failed at the structural linkage step: it did not update any existing node to reflect the uncertainty, leaving no selected-question trigger for downstream flow.

What This Establishes

1. Semantic interpretation works: The model correctly interprets both Answer 1 and would have interpreted Answer 2 (had Update 1 succeeded) as savings-realism doubt.
2. Dedicated node creation intent is correct: The model's attempt to create nsavings_reality confirms v0.21's prompt clarification successfully guides the model toward distinct unknown nodes rather than absorption.
3. Structural gap exposed: The rejection reveals a gap where semantic interpretation succeeds but graph mutation fails silently — no existing node was updated (e.g., nx54wwa remained known instead of provisional), so the proposal lacked the structural trigger needed for question selection.

What This Does NOT Prove

1. Whether Answer 2 would have reused or duplicated: We cannot answer the primary identity question because Update 1 never succeeded in establishing the persistent uncertainty that Update 2 would need to act upon.
2. Downstream investigation viability: The graph was not updated, so downstream investigation cannot be tested.
3. Cross-domain generalisation: Only tested on one scenario with one phrasing.
4. Whether the structural gap is specific to cold-start vs. mid-investigation: This occurred at cold start where nx54wwa (known) needed updating alongside new node creation — a different mutation pattern than 57J.56's update path which DID update nx54wwa.

Production code changed: NO

Prompt changed during experiment: NO

Canonical harness restored: YES

Hardened no-retry behaviour preserved: YES

Ollama calls beyond harness count: 0

Dev server disturbed: NO