confidence-engine/docs/experiment-57j47.md

Experiment 57J.47 — Convergence Test: Existing-First Uncertainty Fallback Live

Branch: feature/semantic-to-mutation-contract-v0.19 Starting HEAD: 94ca1b9 docs: experiment 57J.46 record and handoff update

Objective

Answer exactly:

For one explicit unresolved uncertainty, does v0.19 now produce a faithful structural graph action instead of another no-op?

This is the convergence test for the current prompt-tuning sequence.

If the same faithful no-op still occurs, do not diagnose or propose v0.20. Report it and stop.

Hypothesis

v0.19 gives the model a two-step structural action rule:

if equivalent unresolved unknown exists:
    reuse/refine it
otherwise:
    add a new unknown representing the uncertainty

Therefore faithful meaning should no longer end with zero meaningful graph mutation.

Pre-written expectation

A faithful interpretation of the explicit savings-realism uncertainty should now trigger one of two structural outcomes: reuse/refine an equivalent unresolved unknown if present, otherwise create a new unknown representing savings realism. A faithful proposal with no meaningful mutation means the current prompt-tuning approach has still not solved the boundary.

Configuration

Configured model: qwen-claude:latest at http://192.168.1.111:11434 Dev server: REUSED EXISTING (HTTP 200, Next.js running) Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic." Harness: scripts/reproduce-multi-turn-investigation.mjs (maxUpdates=1, answers=[fixed answer])

CALL ACCOUNTING

startCalls: 1 updateCalls: 1 totalCalls: 2

Supplementary scripts used: NO Retries: 0

START

HTTP status: 200 | stage: unknown

Nodes: 7 Edges: 4 Selected question: "What would clarify current detailed breakdown of engineering operating costs and geographic distribution in this situation?"

Start graph (cold-start from scenario):

• n-baseline-costs (unknown/observation) — baseline cost state
• nv759rs (state/knowledge) — relocation consideration state anchor
• n-engineering-operating-costs (unknown/observation) — engineering cost detail gap
• 4 additional nodes (scenario reconstruction scaffolding)

No analysis of cold-start variance.

UPDATE 1

HTTP status: 422 Stage: proposal_compatibility First error: "answerMeaning.userSupportedMeaning overstates a raw answer that remains uncertain."

Nodes: 7 (unchanged — mutation not applied) Edges: 4 (unchanged — mutation not applied) Selected question: null

rejectedProposalSnapshot

{
  "answerMeaning": {
    "userSupportedMeaning": "The user is currently uncertain whether the projected office savings from the relocation are realistic.",
    "possibleInference": null
  },
  "updatedNodes": [],
  "resolvedUnknownNodeIds": [],
  "addedNodes": [
    {
      "id": "ns82kx9",
      "kind": "unknown",
      "label": "Projected office savings from the relocation lack validation",
      "description": "The realism of projected office savings is unverified, so that the potential cost reduction cannot be validated for the relocation strategy.",
      "parentId": null,
      "dependsOn": [],
      "affects": [],
      "childIds": []
    }
  ],
  "addedEdges": [
    {
      "fromNodeId": "ns82kx9",
      "toNodeId": "nv759rs",
      "relationship": "depends_on"
    }
  ]
}

ANSWER MEANING

userSupportedMeaning: "The user is currently uncertain whether the projected office savings from the relocation are realistic." possibleInference: null

Meaning classification: STRENGTHENED

Raw answer: "I am unsure whether the projected office savings from the relocation are realistic." (first-person uncertainty statement) Extracted meaning: "The user is currently uncertain whether..." (third-person assertion about user's mental state + temporal specificity "currently")

The model converted first-person uncertainty ("I am unsure") into third-person assertion ("The user is currently uncertain"). This adds two elements beyond the raw answer:

1. Perspective shift: Asserting what "the user" feels/thinks, rather than preserving the first-person expression.
2. Temporal specificity: Adding "currently" — an explicit temporal qualifier not present in the raw answer.

This goes slightly beyond preserving "only uncertainty over whether projected office savings are realistic." The extracted meaning introduces a stronger claim (assertion about user mental state) that was not established by the raw answer.

STRUCTURAL PROPOSAL

updatedNodes: [] (none — empty array) resolvedUnknownNodeIds: [] (none — empty array) addedNodes: [{id: "ns82kx9", kind: "unknown", label: "Projected office savings from the relocation lack validation", description: "The realism of projected office savings is unverified, so that the potential cost reduction cannot be validated for the relocation strategy."}] addedEdges: [{fromNodeId: "ns82kx9", toNodeId: "nv759rs", relationship: "depends_on"}]

Meaningful updated/refined existing uncertainty: NO

updatedNodes is empty. No existing unknown was meaningfully modified.

Meaningful new uncertainty added: YES

A genuinely new unknown node (ns82kx9) was created, directly representing savings realism ("Projected office savings from the relocation lack validation"). The label and description are grounded in the answer's explicit concern. This represents exactly the user-supported uncertainty about whether projected savings are realistic.

Structural action: ADD NEW UNKNOWN

The proposal added a new unknown node (with one depends_on edge to the state anchor) representing savings realism. The existing-first rule found no equivalent existing unresolved unknown for savings realism, so the fallback-to-add path was correctly exercised.

Classification: D — STRENGTHENING BLOCKS TEST

Meaning is STRENGTHENED. The structural action (ADD NEW UNKNOWN) represents exactly the savings-realism uncertainty and is meaningful. However, the meaning extraction is not purely faithful (third-person assertion + temporal specificity overstates the raw first-person uncertainty). Therefore this test does not establish v0.19's effectiveness for the semantic-to-mutation boundary.

Why: The model produced a genuine new unknown node representing savings realism — this IS structural progress that was NOT present in prior experiments (57J.36-45 all showed faithful no-ops or empty proposals). However, the userSupportedMeaning contains third-person assertion ("The user is currently uncertain") that goes beyond the raw answer's first-person uncertainty statement. The proposal_compatibility validator caught this as semantic strengthening, rejecting the proposal before structural evaluation.

Did v0.19 solve the faithful semantic-to-mutation failure in this run: NO

The test cannot determine whether v0.19 solves the boundary because meaning was strengthened, which blocks the test. However, the evidence is directionally encouraging: the model DID produce a meaningful new unknown for savings realism — something none of the prior experiments (57J.36 through 57J.46) achieved in a single call. The faithful no-op pattern has been broken; the remaining blocker is the meaning extraction boundary, not the structural action selection.

What this establishes:

1. The existing-first uncertainty fallback rule works at the structural level. When no equivalent exists, the model adds a genuinely new unknown directly representing savings realism — exactly what the v0.19 prompt was designed to produce.
2. The faithful no-op is no longer the default output. This run produced one added node and one added edge. Prior experiments (57J.36-45) consistently returned zero structural mutations for the same type of uncertainty answer.
3. A new blocking issue emerges at the meaning extraction boundary: first-person uncertainty statements ("I am unsure") are being converted to third-person assertions ("The user is currently uncertain"), which the semantic fidelity guard correctly flags as strengthening.

What it does NOT prove:

• That v0.19 reliably produces faithful meaning from first-person uncertainty across repeated runs.
• That the new unknown node's label/description would survive if meaning were faithful.
• That later turns in the investigation remain productive after this type of rejection.
• That the "currently" temporal specificity issue generalizes to other answer types.

CONVERGENCE DECISION

Further prompt tuning justified by this run: NO

If result is non-A: Return to architecture discussion before any v0.20 change.

This is a convergence test for the current prompt-wording sequence (v0.17 → v0.18 → v0.19). Result is D (non-A), so the convergence rule applies: do not diagnose a new prompt tweak, propose v0.20, or continue prompt tuning. The next discussion should reconsider the architecture rather than automatically continuing prompt tuning.

Production code changed during experiment: NO Prompt changed during experiment: NO Canonical harness restored: YES Hardened no-retry behaviour preserved: YES Dev server disturbed: NO Ollama calls beyond harness count: 0

Documentation

• Created: docs/experiment-57j47.md
• Handoff updated: appended to docs/current-handoff.md

Git status after documentation: (dirty — doc file uncommitted)