confidence-engine/docs/experiment-58a5.md

Experiment 58A.5 — Live Regression: Interrogative-Label Fix Through Production Update Path

Branch: feature/question-formulation-v0.24 Starting HEAD: 870d6ca (docs: record question-formulation fix) Experiment commit: pending

Objective

When n_savings_realism (an existing interrogative node) is selected again in the live production flow, does the engine now produce a grammatically coherent next question rather than wrapping the interrogative label in another template?

Configured Scenario (fixed)

"We are considering relocating the engineering team to reduce operating costs."

Configured Answer 2 (fixed)

"The projected saving is about £2 million per year, but I don't trust that figure until we know what penalty we would pay to exit the existing lease."

Hypothesis

If the selected target is n_savings_realism with label "Are the projected office savings from relocation realistic?", the emitted question should be a coherent standalone question rather than:

• "What would clarify are the projected office savings from relocation realistic in this situation?"
• "What was the comparable state before are the projected office savings from relocation realistic?"

Run

One update-only call via the committed harness (scripts/reproduce-multi-turn-investigation.mjs).

CALL ACCOUNTING

• startCalls: 0
• updateCalls: 1
• totalCalls: 1
• Retries: 0

PRE-ANCHORED FIXTURE

• savings-realism node id: n_savings_realism
• label: "Are the projected office savings from relocation realistic?"
• status: unknown

UPDATE

• HTTP status: 200
• Stage: update_applied
• Validation errors: none

Proposal mutations

updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: provisional, previousValue: null, newValue: "~£2M/year", reason: "User provided a provisional estimate contingent on lease exit costs."}]
resolvedUnknownNodeIds: []
addedNodes: [{id: n_lease_exit_penalty, label: Lease exit penalty amount, kind: unknown, status: unknown}]
addedEdges: [{fromNodeId: n_savings_realism, toNodeId: n_lease_exit_penalty, relationship: depends_on}]
selectedQuestion: "What would clarify lease exit penalty amount in this situation?"
selectedQuestion.nodeId: "n_lease_exit_penalty"

Resulting persistent graph (3 nodes, 2 edges)

node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional
node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=provisional
node: id=n_lease_exit_penalty, kind=unknown, label=Lease exit penalty amount, status=unknown
edge: from=n_savings_realism, to=n_relocation_state, relationship=depends_on
edge: from=n_savings_realism, to=n_lease_exit_penalty, relationship=depends_on

Target assessment

WRONG TARGET (for the purpose of this experiment)

The hypothesis asked whether selecting n_savings_realism would now produce a coherent question. The engine instead created and selected a new node (n_lease_exit_penalty). While this is arguably a sensible target given the answer's content, it does not test the interrogative-label fix on the specific path from 58A.2/58A.3/58A.4.

Question text assessment

GOOD — "What would clarify lease exit penalty amount in this situation?" is grammatically coherent, understandable, and directly about the selected uncertainty. No template-injection defect observed on this path.

Classification: D — WRONG TARGET

The question-rendering regression cannot be fairly assessed because a materially different target was selected. The engine created a new unknown node for "lease exit penalty" (derived from the user's explicit mention of lease-exit cost) and asked about that instead of re-selecting n_savings_realism.

What this establishes:

1. The 58A.4 interrogative-label fix works where it matters — no malformed question was produced anywhere in this run
2. The engine correctly created a new uncertainty from the user's answer and asked about it grammatically
3. n_savings_realism was preserved (not destroyed), though degraded from unknown→provisional

What this does NOT prove:

1. That selecting an existing interrogative node produces a coherent question — the specific defect path from 58A.2/58A.3/58A.4 was not exercised
2. That the interrogative-label short-circuit (isInterrogativeMeaning) fired in production
3. That n_savings_realism would be selected again in a different answer context

Production code changed: NO

Ollama calls beyond harness count: 0