experiment: validate question-formulation fix live

2026-08-12 15:29:38 +01:00
parent 870d6caa05
commit 6f2c09cd94
2 changed files with 165 additions and 1 deletions
@@ -171,7 +171,7 @@ Answer before continuing:

 ---

-_Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to A–F scope._
+_Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: `feature/question-formulation-v0.24`. First-pass reasoning-fidelity v0.8 complete to A–F scope._

 ---

@@ -1719,3 +1719,70 @@ This satisfies 57J.77's boundary A recommendation: a committed update-only path
 **Ownership location:** `lib/graph/question-formulator.js` — functions `isInterrogativeMeaning`, `wrapInterrogativeForTemplate`, and modified paths in `buildNeutralClarificationQuestion`, `buildEvidenceFallbackQuestion`, `buildQuestionFromFamily`, `buildQuestionFromStrategy`.

 **Classification: PASS.** 20 new focused tests pass. No regressions. Two unrelated pre-existing failures explicitly out of scope: `question-priority-generalisation`, `selection-influence-diagnostic`.
+
+### Experiment 58A.5 — Live Regression: Interrogative-Label Fix Through Production Update Path
+
+**Objective:** Does the 58A.4 question-formulation fix work through the real `updateCase()` production path when the selected target is an existing interrogative node?
+
+**Classification: D — WRONG TARGET.** One update-only call via the committed harness. The engine did **not** select `n_savings_realism` as the next-question target. Instead it created a new unknown node `n_lease_exit_penalty` (about lease exit penalty) and selected that as the target.
+
+### Branch: `feature/question-formulation-v0.24`
+### Starting HEAD: `870d6ca` (docs: record question-formulation fix)
+
+### Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434
+
+### CALL ACCOUNTING
+
+- startCalls: 0
+- updateCalls: 1
+- totalCalls: 1
+- Retries: 0
+
+### PRE-ANCHORED FIXTURE
+
+- savings-realism node id: `n_savings_realism`
+- label: "Are the projected office savings from relocation realistic?"
+- status: unknown
+
+### UPDATE
+
+- HTTP status: 200
+- Stage: update_applied
+- Validation errors: none
+
+#### Proposal mutations
+
+```
+updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: provisional, previousValue: null, newValue: "~£2M/year", reason: "User provided a provisional estimate contingent on lease exit costs."}]
+resolvedUnknownNodeIds: []
+addedNodes: [{id: n_lease_exit_penalty, label: Lease exit penalty amount, kind: unknown, status: unknown}]
+addedEdges: [{fromNodeId: n_savings_realism, toNodeId: n_lease_exit_penalty, relationship: depends_on}]
+selectedQuestion: "What would clarify lease exit penalty amount in this situation?"
+selectedQuestion.nodeId: "n_lease_exit_penalty"
+```
+
+### Target assessment
+
+**WRONG TARGET** (for the purpose of this experiment)
+
+The hypothesis asked whether selecting `n_savings_realism` would now produce a coherent question. The engine instead created and selected a new node (`n_lease_exit_penalty`). While this is arguably a sensible target given the answer's content, it does not test the interrogative-label fix on the specific path from 58A.2/58A.3/58A.4.
+
+### Question text assessment
+
+**GOOD** — "What would clarify lease exit penalty amount in this situation?" is grammatically coherent, understandable, and directly about the selected uncertainty. No template-injection defect observed on this path.
+
+### What this establishes:
+
+1. The 58A.4 interrogative-label fix works where it matters — no malformed question was produced anywhere in this run (the new node's label is declarative, so it correctly got a template frame)
+2. The engine created a legitimate new uncertainty from the user's answer ("penalty we would pay to exit the existing lease") and asked about it grammatically
+3. `n_savings_realism` was preserved (not destroyed), though degraded from unknown→provisional
+
+### What this does NOT prove:
+
+1. That selecting an **existing interrogative** node (like `n_savings_realism`) produces a coherent question — the specific defect path from 58A.2/58A.3/58A.4 was not exercised
+2. That the interrogative-label short-circuit (`isInterrogativeMeaning`) fired in production
+3. That `n_savings_realism` would be selected again in a different answer context
+
+### Production code changed: NO
+
+### Ollama calls beyond harness count: 0