confidence-engine/docs/experiment-57j47.md

Experiment 57J.47 — Convergence Test: Existing-First Uncertainty Fallback Live

Branch: feature/semantic-to-mutation-contract-v0.19 Starting HEAD: 94ca1b9 docs: experiment 57J.46 record and handoff update

Objective

Answer exactly:

For one explicit unresolved uncertainty, does v0.19 now produce a faithful structural graph action instead of another no-op?

This is the convergence test for the current prompt-tuning sequence.

If the same faithful no-op still occurs, do not diagnose or propose v0.20. Report it and stop.

Hypothesis

v0.19 gives the model a two-step structural action rule:

if equivalent unresolved unknown exists:
    reuse/refine it
otherwise:
    add a new unknown representing the uncertainty

Therefore faithful meaning should no longer end with zero meaningful graph mutation.

Pre-written expectation

A faithful interpretation of the explicit savings-realism uncertainty should now trigger one of two structural outcomes: reuse/refine an equivalent unresolved unknown if present, otherwise create a new unknown representing savings realism. A faithful proposal with no meaningful mutation means the current prompt-tuning approach has still not solved the boundary.

Configuration

Configured model: qwen-claude:latest at http://192.168.1.111:11434 Dev server: REUSED EXISTING (HTTP 200, Next.js running) Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic." Harness: scripts/reproduce-multi-turn-investigation.mjs (maxUpdates=1, answers=[fixed answer])

CALL ACCOUNTING

startCalls: 1 updateCalls: 1 totalCalls: 2

Supplementary scripts used: NO Retries: 0

START

HTTP status: 200 | stage: unknown

Nodes: 7 Edges: 4 Selected question: "What would clarify current detailed breakdown of engineering operating costs and geographic distribution in this situation?"

Start graph (cold-start from scenario):

• n-baseline-costs (unknown/observation) — baseline cost state
• nv759rs (state/knowledge) — relocation consideration state anchor
• n-engineering-operating-costs (unknown/observation) — engineering cost detail gap
• 4 additional nodes (scenario reconstruction scaffolding)

No analysis of cold-start variance.

UPDATE 1

HTTP status: 422 Stage: proposal_compatibility First error: "answerMeaning.userSupportedMeaning overstates a raw answer that remains uncertain."

Nodes: 7 (unchanged — mutation not applied) Edges: 4 (unchanged — mutation not applied) Selected question: null

rejectedProposalSnapshot

{
  "answerMeaning": {
    "userSupportedMeaning": "The user is currently uncertain whether the projected office savings from the relocation are realistic.",
    "possibleInference": null
  },
  "updatedNodes": [],
  "resolvedUnknownNodeIds": [],
  "addedNodes": [
    {
      "id": "ns82kx9",
      "kind": "unknown",
      "label": "Projected office savings from the relocation lack validation",
      "description": "The realism of projected office savings is unverified, so that the potential cost reduction cannot be validated for the relocation strategy.",
      "parentId": null,
      "dependsOn": [],
      "affects": [],
      "childIds": []
    }
  ],
  "addedEdges": [
    {
      "fromNodeId": "ns82kx9",
      "toNodeId": "nv759rs",
      "relationship": "depends_on"
    }
  ]
}

ANSWER MEANING

userSupportedMeaning: "The user is currently uncertain whether the projected office savings from the relocation are realistic." possibleInference: null

Meaning classification: HUMAN ASSESSMENT = MINOR PARAPHRASE | VALIDATOR REJECTION = LEXICAL MISMATCH

Raw answer: "I am unsure whether the projected office savings from the relocation are realistic." (first-person uncertainty statement) Extracted meaning: "The user is currently uncertain whether..." (third-person assertion about user's mental state + temporal specificity "currently")

Validator rejection reason: Lexical false positive. The deterministic category model (deriveAnswerMeaningProfile) detects "unsure" in the raw answer (category: uncertain) but does NOT detect "uncertain" in the extracted meaning (category: other). Both words express identical uncertainty semantics; "uncertain" is absent from the detection patterns ("not really sure" | "not sure" | "unsure" | "do not know" | "don't know"). The rejection was caused by keyword mismatch, not semantic strengthening.

Human semantic assessment (independent of validator): Neither element — the perspective shift nor the temporal qualifier "currently" — materially changes meaning beyond what the raw answer establishes. See 57J.48 for detailed deterministic analysis.

STRUCTURAL PROPOSAL

updatedNodes: [] (none — empty array) resolvedUnknownNodeIds: [] (none — empty array) addedNodes: [{id: "ns82kx9", kind: "unknown", label: "Projected office savings from the relocation lack validation", description: "The realism of projected office savings is unverified, so that the potential cost reduction cannot be validated for the relocation strategy."}] addedEdges: [{fromNodeId: "ns82kx9", toNodeId: "nv759rs", relationship: "depends_on"}]

Meaningful updated/refined existing uncertainty: NO

updatedNodes is empty. No existing unknown was meaningfully modified.

Meaningful new uncertainty added: YES

A genuinely new unknown node (ns82kx9) was created, directly representing savings realism ("Projected office savings from the relocation lack validation"). The label and description are grounded in the answer's explicit concern. This represents exactly the user-supported uncertainty about whether projected savings are realistic.

Structural action: ADD NEW UNKNOWN

The proposal added a new unknown node (with one depends_on edge to the state anchor) representing savings realism. The existing-first rule found no equivalent existing unresolved unknown for savings realism, so the fallback-to-add path was correctly exercised.

Classification: D — REJECTION BLOCKS TEST (LEXICAL FALSE POSITIVE)

Meaning extraction produced a semantically equivalent paraphrase that was lexically rejected. The structural action (ADD NEW UNKNOWN) represents exactly the savings-realism uncertainty and is meaningful. However, the meaning extraction used "uncertain" rather than "unsure" — identical semantics but absent from deriveAnswerMeaningProfile's detection patterns, causing a category mismatch (other instead of uncertain) that triggered rejection. This is not evidence of genuine semantic strengthening; it is evidence of incomplete keyword coverage. The faithful no-op pattern has been broken by the structural action, but the test cannot confirm v0.19's effectiveness because the meaning extraction boundary still produces lexically rejected paraphrases.

Why: The model produced a genuine new unknown node representing savings realism — this IS structural progress that was NOT present in prior experiments (57J.36-45 all showed faithful no-ops or empty proposals). However, the userSupportedMeaning contains third-person assertion ("The user is currently uncertain") that goes beyond the raw answer's first-person uncertainty statement. The proposal_compatibility validator caught this as semantic strengthening, rejecting the proposal before structural evaluation.

Did v0.19 solve the faithful semantic-to-mutation failure in this run: NO

The test cannot determine whether v0.19 solves the boundary because the meaning extraction produced a semantically faithful but lexically rejected paraphrase. However, the evidence is directionally encouraging: the model DID produce a meaningful new unknown for savings realism — something none of the prior experiments (57J.36 through 57J.46) achieved in a single call. The faithful no-op pattern has been broken; the remaining blocker is an incomplete lexical coverage gap in deriveAnswerMeaningProfile, not a structural action selection failure.

What this establishes:

1. The existing-first uncertainty fallback rule works at the structural level. When no equivalent exists, the model adds a genuinely new unknown directly representing savings realism — exactly what the v0.19 prompt was designed to produce.
2. The faithful no-op is no longer the default output. This run produced one added node and one added edge. Prior experiments (57J.36-45) consistently returned zero structural mutations for the same type of uncertainty answer.
3. A new blocking issue emerges at the meaning extraction boundary: the model produces semantically faithful paraphrases using words ("uncertain") that are absent from deriveAnswerMeaningProfile's detection patterns, causing false-positive rejection by the semantic fidelity guard. The guard is correct for genuine strengthening but incorrect here because of incomplete lexical coverage (detects "unsure" but not "uncertain").

What it does NOT prove:

• That v0.19 reliably produces faithful meaning from first-person uncertainty across repeated runs.
• That the new unknown node's label/description would survive if meaning were faithful.
• That later turns in the investigation remain productive after this type of rejection.
• That the "currently" temporal specificity issue generalizes to other answer types.

CONVERGENCE DECISION

Further prompt tuning justified by this run: NO

If result is non-A: Return to architecture discussion before any v0.20 change.

This is a convergence test for the current prompt-wording sequence (v0.17 → v0.18 → v0.19). Result is D (non-A), so the convergence rule applies: do not diagnose a new prompt tweak, propose v0.20, or continue prompt tuning. The next discussion should reconsider the architecture rather than automatically continuing prompt tuning.

Production code changed during experiment: NO Prompt changed during experiment: NO Canonical harness restored: YES Hardened no-retry behaviour preserved: YES Dev server disturbed: NO Ollama calls beyond harness count: 0

Documentation

• Created: docs/experiment-57j47.md
• Handoff updated: appended to docs/current-handoff.md

Git status after documentation: (dirty — doc file uncommitted)