experiment: validate evidence versus clarification routing

docs/current-handoff.md

@@ -235,3 +235,7 @@ One live Ollama call (14,032 ms) returned `uncertaintyType: "user_clarification_
 **Classification: PASS.** The model correctly identified the ambiguity as user-owned, did not introduce spurious evidence gathering, and preserved the evidence-vs-user-meaning distinction cleanly.
 
 This experiment does not prove consistency across repeated runs, fidelity for other regression cases (A–E, G+), behavior in production reasoning paths, or downstream integration with Behaviour Selection or the SituationGraph. Branch: `feature/reasoning-fidelity-v0.8`. Files: `tests/reconstruction/semantic-regression-f-user-owned-ambiguity.test.js` and `docs/experiment-56l.md`. Status pending Rob's review.
+
+### Experiment 56M Summary — Production evidence vs clarification routing validation
+
+Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.

docs/experiment-56m.md

@@ -0,0 +1,93 @@
+# Experiment 56M — Validate Evidence vs Clarification Routing
+
+**Date:** 2026-08-09
+**Branch:** `feature/reasoning-fidelity-v0.8`
+**Codex refinement validated:** `f861e2c` (reasoning: preserve evidence versus clarification distinction)
+**Ollama calls:** 0
+
+## Objective
+
+Validate one production claim: after Codex commit `f861e2c`, does the production question-formulation boundary keep Regression E on an evidence route and Regression F on a user-clarification route?
+
+This experiment isolates whether the deterministic production boundary preserves the distinction. No live model call is required because Codex changed deterministic production logic, not semantic interpretation.
+
+## Method
+
+Exercised both regression cases against the real `formulateQuestion()` implementation via an inline Node.js session. Captured full output objects including reasoning pattern, investigation strategy, question family, template, and exact question text.
+
+No Ollama calls were made. Experiments 56K and 56L already established that the configured model can distinguish evidence-resolvable uncertainty from user-owned ambiguity.
+
+## Regression E — Evidence-resolvable disagreement
+
+**Input:**
+- `label`: "Possible causes of the delivery delay"
+- `description`: "Need to determine whether staff capacity or supplier lead times are responsible for the delivery delay."
+- `centralStatement`: "Delivery is delayed and the cause is still unknown."
+
+**Produced question:** "What evidence would clarify possible causes of the delivery delay?"
+
+**Reasoning pattern:** diagnosis (reason: "Selected diagnosis as the default because the active unknown needs clarifying evidence or mechanism-level investigation.")
+
+**Investigation strategy:** `evidence_gathering` (reason: "Selected because evidence about the practical limiting factor is needed before the unknown can be resolved.")
+
+**Question family:** diagnosis
+**Template:** diagnosis_evidence
+
+**Semantic assessment:**
+- The question clearly seeks evidence capable of distinguishing the competing external hypotheses.
+- It does NOT ask the user to settle which external cause is true.
+- Both reasoning pattern (diagnosis) and strategy (evidence_gathering) align with an evidence route.
+
+**Classification: PASS**
+
+## Regression F — User-owned ambiguity
+
+**Input:**
+- `label`: "Whether avoiding additional risk is a hard constraint"
+- `description`: "Need to know whether avoiding additional risk is a hard constraint or a preference/trade-off."
+
+**Produced question:** "Is avoiding additional risk a hard constraint or a preference/trade-off?"
+
+**Reasoning pattern:** prioritisation (reason: "Selected prioritisation because the active unknown is about ordering options or trade-offs.")
+- **Note:** This is correct — the `isPrioritisationPatternCandidate` check fires on "preference/trade-off" in the label, producing a valid reasoning pattern even though the question itself bypasses pattern-dependent template logic.
+
+**Investigation strategy:** null (intentionally — user-meaning boundary triggers early return before strategy selection)
+
+**Question family:** prioritisation
+**Template:** user_meaning_clarification
+
+**Semantic assessment:**
+- The question explicitly clarifies the hard-constraint versus preference/trade-off boundary.
+- It does NOT pretend external evidence can establish this distinction.
+- `rejectedQuestionFamilies` correctly excludes evidence_gathering, diagnosis, explanation, contradiction, and comparison.
+- `allowedQuestionFamilies` correctly includes only prioritisation variants.
+
+**Classification: PASS**
+
+## What This Validation Established
+
+1. After commit `f861e2c`, the production question-formulation code preserves the E/F distinction in deterministic reasoning:
+   - Regression E (competing causal hypotheses, resolvable by evidence) routes to `diagnosis` pattern + `evidence_gathering` strategy → evidence-seeking question.
+   - Regression F (constraint-versus-preference boundary, user-owned) triggers early-return at `isUserOwnedMeaningBoundaryUnknown()` → user-clarification question with null strategy.
+
+2. The routing mechanism is the `isUserOwnedMeaningBoundaryUnknown()` guard in `formulateQuestion()` (line ~1773), which fires before any investigation strategy or question family selection for node F inputs.
+
+3. The rejected/allowed question families confirm no evidence-adjacent families are permitted for user-owned boundary cases.
+
+4. All 19 existing tests in `tests/graph/question-formulator.test.js` continue to pass — no regression from the E/F routing change.
+
+## What This Validation Does NOT Prove
+
+1. Consistency of this behavior across repeated runs (no live model call was made).
+2. Fidelity for other regression cases (A–D, G+).
+3. Behavior when external evidence is later added to the graph and both routes remain available.
+4. Downstream integration with Behaviour Selection or the SituationGraph.
+5. Whether the wording of the produced questions is optimal for real users (that was covered in earlier experiments).
+
+## Production Files Modified
+
+None. This experiment reads production code only — no modification was made to any production file.
+
+---
+
+*Experiment 56M. Status: Rob's review.*