docs: consolidate reasoning refinement requirements

docs/current-handoff.md

@@ -160,11 +160,11 @@ Answer before continuing:
 
 ---
 
-*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55D. Branch: `feature/user-workspace-ux-v0.7`. Status pending Rob's review.*
+*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55E. Branch: `feature/user-workspace-ux-v0.7`. Status pending Rob's review.*
 
-### Return-to-Work Note (Experiment 55D)
+### Return-to-Work Note (Experiment 55E)
 
-55C showed preserved meaning can protect later resolution, but weak-priority meaning was already strengthened in Stage 1. 55D isolated that first interpretation step using a single-call stated-vs-inferred separation with four fixed answers across risk preference cases. Weak priority stayed as relative importance only (direct improvement over 55C's constraint-language strengthening). Conditionality survived through the conditional trade-off case. Explicit and uncertain controls stayed clean — no unsupported meaning leaked into statedMeaning. Stronger implications were kept separate in possibleInference for Cases 1 and 2, though Cases 3 and 4 showed unnecessary inference generation (hygiene issue, not leakage). This does not yet prescribe production architecture. Graph, Behaviour Selection, UI and production remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-stated-vs-inferred.test.js for the full experiment and results. Status pending Rob's review.
+This round of semantic experimentation is deliberately paused. The requirements synthesis for the next implementation pass lives in `docs/reasoning-refinement-requirements.md` — read it first. The key problem is preserving user-supported meaning, model inference, qualification, and unresolved uncertainty distinctly enough that later reasoning cannot silently convert one into another. Tomorrow should begin with code inspection against the requirements and regression pack rather than more semantic exploration. Production architecture is still undecided; no graph fields, Behaviour Selection changes, or UI layout have been prescribed. Branch: `feature/user-workspace-ux-v0.7`. This commit consolidates all findings into that single bounded document.
 
 ### Experiment 55A Summary — Clarification Uncertainty Preservation
 

docs/design-evolution-log.md

@@ -10147,7 +10147,7 @@ All 4 live inference calls completed successfully. Four answers tested through o
 
 ### Historical Comparison Result
 
-Compared to Experiment 55C's finding that Stage 1 strengthened "Risk matters more to me." into language about "preference/trade-off rather than absolute hard constraint," Experiment 55D shows the two-field separation can prevent that strengthening when it matters most: weak-priority answers stayed as relative importance in statedMeaning while stronger interpretations were placed separately in possibleInference. The mechanism works for the specific failure mode identified in 55C, but broader stability and downstream consumption remain untested.
+Compared to Experiment 55C's finding that Stage 1 strengthened "Risk matters more to me." into language about "preference/trade-off rather than absolute hard constraint," Experiment 55D shows the two-field separation avoided the specific weak-priority strengthening defect in this tested run: weak-priority answers stayed as relative importance in statedMeaning while stronger interpretations were placed separately in possibleInference. The mechanism handled the specific failure mode successfully in this probe, but broader stability and downstream consumption remain untested.
 
 ### Documentation Updated
 
@@ -10179,3 +10179,36 @@ This experiment created one new test file only. No stated-vs-inferred clarificat
 ### Return-to-Work Note (Experiment 55D)
 
 55C showed preserved meaning can protect later resolution, but weak-priority meaning was already strengthened in Stage 1. 55D isolated that first interpretation step using a single-call stated-vs-inferred separation with four fixed answers across risk preference cases. Weak priority stayed as relative importance only (direct improvement over 55C's constraint-language strengthening). Conditionality survived through the conditional trade-off case. Explicit and uncertain controls stayed clean — no unsupported meaning leaked into statedMeaning. Stronger implications were kept separate in possibleInference for Cases 1 and 2, though Cases 3 and 4 showed unnecessary inference generation (hygiene issue, not leakage). This does not yet prescribe production architecture. Graph, Behaviour Selection, UI and production remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-stated-vs-inferred.test.js for the full experiment and results. Status pending Rob's review.
+
+## Experiment 55E — Reasoning Refinement Requirements Synthesis (2026-08-08)
+
+### Objective
+
+Consolidate findings from Experiments 53–55D into a compact, implementation-ready reasoning contract for the next production pass. Stop opening new semantic sub-problems for this round.
+
+### Context Documents Reviewed
+
+- `docs/current-handoff.md` (current state and handoff);
+- Experiments 53, 54K–54Z, 55A–55D in `docs/design-evolution-log.md`;
+- Created: `docs/reasoning-refinement-requirements.md`.
+
+### Synthesis Output
+
+- **8 reasoning requirements** retained (R1–R8), all supported by at least one recorded experiment observation;
+- **7 known failure patterns** recorded from the experiment history;
+- **8 known good behaviours** recorded as regression expectations;
+- **6 regression cases** selected (A–F) covering: weak priority, conditional trade-off, non-answer/uncertainty, explicit hard constraint, evidence-resolvable disagreement, and user-owned ambiguity;
+- **10 open questions** explicitly retained to prevent premature architecture.
+
+### Key Unresolved Items
+
+- Stability across larger case sets and other models;
+- Exact production representation (graph integration pending);
+- Downstream consumption of inference fields;
+- Behaviour Selection and UI integration timing;
+- Performance/latency implications;
+- One call versus multiple calls for semantic separation.
+
+### Conclusion
+
+This round of semantic experimentation is closed. The requirements synthesis in `docs/reasoning-refinement-requirements.md` provides a bounded starting point for tomorrow's implementation pass. No live inference was performed. No production code, prompts, schemas, graph files, Behaviour Selection rules, or UI code were modified. The mechanism from Experiment 55D avoided the specific weak-priority strengthening defect in this tested run; broader stability remains unproven.

docs/reasoning-refinement-requirements.md

@@ -0,0 +1,205 @@
+# Reasoning Refinement Requirements — Experiment 55E Synthesis
+
+This document captures the evidence-backed requirements emerging from the current reasoning-experiment series. It is a handoff into the next implementation pass, not a final architecture and not a replacement for the design-evolution log.
+
+---
+
+## 1. Purpose
+
+The Confidence Engine's investigation has exposed recurring patterns where later reasoning silently converts user-supplied meaning into stronger model interpretation, flattens qualification, or over-resolves uncertainty. These experiments tested those patterns progressively across semantic separation, clarification chains, and resolution stability.
+
+This document records what can be relied on, which requirements follow from the evidence, which regression cases should constrain the first implementation pass, and what remains genuinely unresolved.
+
+---
+
+## 2. What We Can Rely On So Far
+
+### Source versus interpretation
+
+Experiments 54H–54J demonstrated that deterministic source identity (SHA-256 hashing of raw user input) can distinguish what the user supplied from what the model might infer. Multiple interpretations can share one deterministic source lineage. Automated semantic grounding (Experiment 54K/54L) showed promising directionality with one missed addition in a multi-addition case — it is viable but imperfect. These findings support the requirement to preserve source-versus-inference separation, though automated grounding alone cannot yet be relied upon for complete accuracy.
+
+### Stated meaning versus possible inference
+
+Experiment 55D tested explicit `statedMeaning` / `possibleInference` separation across four fixed answers (weak priority, conditional trade-off, explicit hard constraint, non-answer). All four preserved statedMeaning without strengthening (4/4). Inference was cleanly separated for Cases 1 and 2; Cases 3 and 4 received unnecessary inferences (hygiene issue, not leakage). No unsupported meaning leaked into statedMeaning. This establishes that a single-call two-field output contract can separate what the user directly established from plausible model interpretation — but broader stability across other models and answer patterns remains untested.
+
+Note: `statedMeaning` means meaning directly supported by the user's words, not necessarily verbatim text reproduction. Contextual framing of the clarification target in a non-answer (as seen in 55D Case 4) is acceptable; it does not constitute unsupported strengthening.
+
+### Uncertainty
+
+Experiments 54T, 55A Case 4, and 55C Case 3 confirmed that uncertainty can be preserved when explicitly tested. A user who says "I'm not really sure" can remain uncertain without the engine forcing resolution or a leaning, supported by repeated-identical-input testing in 54T.
+
+### Evidence versus clarification
+
+Experiment 54R showed the model can distinguish between uncertainty that observable evidence can resolve (e.g., competing delivery causes: staff capacity vs supplier lead times) and ambiguity that only the user can resolve (e.g., growth-versus-risk priority trade-off). All three fixed cases in 54R were correct. Experiment 54O/54Q showed that direct evidence-need comparison can succeed where consequence-detection alone failed (54N Case 3). These findings support distinguishing evidence-gathering need from user-clarification need, but broader reliability remains unproven.
+
+### Clarification chain
+
+Experiments 54S through 54W tested the sequence: `clarification need → clarification target → question → answer interpretation → target resolution`. Each individual step worked in isolated fixed-case tests. One small chained probe (54W) produced a correct end-to-end result; upstream distortion can propagate through to downstream resolution (55C). The stages worked individually and in one small chained probe; broader reliability remains untested.
+
+Experiments 54X–54Z showed target-specificity loss: the model sometimes broadens targets from material distinctions ("hard constraint vs preference/trade-off") to coarser priority ordering ("preferred priority between growth and risk"). With explicit answers this did not change resolved meaning (54Y); with weak answers it exposed differences between precise and broadened targets (54Z). Direction is unpredictable.
+
+### Meaning before judgement
+
+Experiments 55B and 55C tested whether extracting meaning before making resolution decisions preserves more nuance than resolving directly. Meaning-only extraction preserved all three tested answers in 55B; carrying that meaning forward protected conditionality in 55C Case 2. However, if Stage 1 distorts the answer, Stage 2 propagates that distortion (55C Case 1). Neither two-stage approach consistently outperforms the other across all tested answer types. This supports preserving faithful early meaning when available, but does not establish a required two-call architecture.
+
+---
+
+## 3. Reasoning Requirements for the First Implementation Pass
+
+Each requirement below is supported by at least one recorded experiment observation. Where wording is stronger than evidence, it has been weakened.
+
+### R1 — Preserve user-supplied meaning
+
+The engine must retain what the user actually established without silently strengthening it. Supported by: 55D (4/4 statedMeaning preserved), 55C Case 1 (distortion propagated downstream). Strongest-supported requirement.
+
+### R2 — Keep inference distinguishable
+
+Reasonable model inference must not become indistinguishable from user-supplied meaning. Supported by: 53, 55D (leakage = 0), 54J/54K (representation can separate source from interpretation).
+
+### R3 — Preserve qualification and conditionality
+
+Language such as "might," "normally," "depends," "for the right opportunity," "I'm not sure" must not be flattened into unconditional conclusions. Supported by: 54Z, 55A Case 3, 55D Case 2.
+
+### R4 — Preserve unresolved uncertainty
+
+A weak answer must be allowed to leave a target unresolved. Supported by: 54T (null-gating stable across three repeated calls), 55A Case 4, 55C Case 3.
+
+### R5 — Distinguish evidence need from clarification need
+
+Do not ask the user merely because the engine lacks external evidence. Supported by: 54R (3/3 correct for tested disagreement types), 54Q (explicit evidence distinction supports consequence judgment).
+
+### R6 — Clarify the actual unresolved distinction
+
+When clarification is needed, the target must reflect the specific user-owned ambiguity rather than only the general topic. Supported by: 54S (3/3 correct targets), 54X (broadening observed but not universal).
+
+### R7 — Avoid unnecessary clarification
+
+If clarification is explicitly not required, do not invent a clarification target. Supported by: 54T (null-gating stable for non-required cases), 54S instability in earlier runs.
+
+### R8 — Resolution must operate on preserved meaning
+
+Any later judgement must operate on preserved meaning rather than silently replacing it with stronger interpretation. Supported by: 55C Case 1 (distortion propagated), 55D (two-field separation prevented strengthening in the tested run). Partially supported: evidence shows operating on distorted meaning produces incorrect results, but the specific mechanism for consuming these fields has not been directly tested.
+
+---
+
+## 4. Known Failure Modes
+
+| Failure pattern | Example | What went wrong | Evidence status |
+|---|---|---|---|
+| Weak priority over-resolved to "not a constraint" | "Risk matters more to me." → targetResolved=true with inferred boundary meaning | Model converted relative importance into a negative hard-constraint assertion | Observed multiple times across 55A, 55B; varied across runs (run-specific) |
+| Conditional trade-off losing qualification | "I'd normally avoid more risk, but for the right opportunity I might accept some." → flattened to "preference or trade-off" | Conditionality removed during resolution judgement | Observed in 55A Case 3 and 55B; survived with two-field separation in 55D |
+| Target broadening replacing material distinction with priority ordering | "hard constraint vs preference/trade-off" → "preferred priority between growth and risk" | Clarification target lost the user-owned boundary type | Reproduced across runs (54X); consequence varies by answer strength (54Z) |
+| Unnecessary `possibleInference` generated for explicit or non-answers | Cases 3/4 in 55D received speculative implications where none was warranted | Model tends to always provide inference content rather than null | Observed once; hygiene issue, not leakage |
+| Stage 1 distortion propagating through Stage 2 resolution | 55C Case 1: weak-priority strengthening in Stage 1 carried into Stage 2 resolution | Preserved meaning was itself distorted upstream | Observed in one run; asymmetric with conditional answers (which improved) |
+| Uncertainty sometimes over-resolved to a leaning | "I'm not really sure" handled correctly in 55A/55C/54T, but other weak answers can produce implicit leanings | Varies by answer pattern and context | Not yet generalised; confirmed correct for non-answer in tested cases |
+| Automated grounding missing minor additions | 54K Case 2: model missed one source-supported content gap during multi-addition grounding | Grounding directionally viable but imperfect completeness | Observed once in multi-addition context |
+
+---
+
+## 5. Known Good Behaviours
+
+These have worked reliably enough in the tested runs to serve as regression expectations:
+
+- **Explicit hard constraint** remains a hard constraint (55D Case 3, 54Y, 54V Case 1).
+- **"I'm not really sure"** remains uncertain without forced resolution (54T, 55A Case 4, 55C Case 3).
+- **Operational cause disagreement** can be evidence-resolvable without user clarification (54R Case 1, 54Q).
+- **Affordability ambiguity** can produce a precise clarification target ("upfront cost versus long-term total cost") (54S Case 3).
+- **Fixed clarification target** can produce one neutral question without adding meaning (54U: 3/3 cases returned correct single neutral questions).
+- **Explicit hard-constraint answer** resolves under both precise and broadened clarification targets to materially equivalent resolved meaning (54Y).
+- **Weak-priority `statedMeaning` remains clean** with two-field separation (55D Case 1: "Risk matters more" stayed as relative importance, inference kept separate).
+- **Deterministic source identity** via SHA-256 is stable across repeated identical inputs (54H).
+
+Be precise about scope: each of these applies only within the tested answer patterns and model configuration. Broader reliability is not yet established.
+
+---
+
+## 6. Regression Pack for the First Production Refinement
+
+### Regression A — Weak priority (over-resolution)
+
+- **Source:** "I want the business to grow, but I don't want to take on more risk."
+- **Relevant answer:** "Risk matters more to me."
+- **Expected preserved meaning:** Risk is of greater relative importance than growth; no hard-constraint or non-hard-constraint boundary established.
+- **Expected uncertainty:** Hard-constraint status for avoiding additional risk is unresolved.
+- **Must not happen:** Inference that risk avoidance is "not a hard constraint" or equivalent negative assertion. The answer says nothing about whether it is a hard constraint, only about relative importance.
+
+### Regression B — Conditional trade-off
+
+- **Source:** "I want the business to grow, but I don't want to take on more risk."
+- **Relevant answer:** "I'd normally avoid more risk, but for the right opportunity I might accept some."
+- **Expected preserved meaning:** Normal preference to avoid additional risk; conditional willingness to accept some under specific circumstances.
+- **Expected uncertainty:** What constitutes "the right opportunity" remains undefined.
+- **Must not happen:** Removal of the conditional qualification ("for the right opportunity"). The answer establishes a two-sided conditional, not a flat stance.
+
+### Regression C — Non-answer (uncertainty)
+
+- **Source:** "I want the business to grow, but I don't want to take on more risk."
+- **Relevant answer:** "I'm not really sure."
+- **Expected preserved meaning:** User is uncertain about whether avoiding additional risk is a hard constraint or preference/trade-off.
+- **Expected uncertainty:** Full — no position taken.
+- **Must not happen:** Any leaning, inference about what the user likely prefers, or forced clarification target generation where the answer does not supply one.
+
+### Regression D — Explicit hard constraint
+
+- **Source:** "I want the business to grow, but I don't want to take on more risk."
+- **Relevant answer:** "It's a hard constraint. I don't want any increase in risk."
+- **Expected preserved meaning:** Avoiding additional risk is established as non-negotiable; no exception cases supplied.
+- **Expected uncertainty:** None regarding the constraint itself (user was explicit). Uncertainty may exist about what qualifies as "increase in risk" or how this interacts with other objectives.
+- **Must not happen:** Downgrading to preference/trade-off, adding conditional qualifications not present in the source.
+
+### Regression E — Evidence-resolvable disagreement
+
+- **Source:** Delivery delay concern.
+- **Relevant answer/competing causes:** "Staff capacity may be the issue" / "Supplier lead times are likely responsible."
+- **Expected preserved meaning:** Two distinct hypotheses about causation.
+- **Expected uncertainty:** Which hypothesis is correct — resolvable by evidence gathering, not user clarification.
+- **Must not happen:** Generating a user-facing clarification question when evidence sources can distinguish the hypotheses.
+
+### Regression F — User-owned ambiguity
+
+- **Source:** "I want the business to grow, but I don't want to take on more risk."
+- **Relevant answer:** (ambiguous statement about both)
+- **Expected preserved meaning:** User has not specified whether avoiding additional risk is a hard constraint or a strong preference/trade-off.
+- **Expected uncertainty:** Preference vs constraint distinction is user-owned and requires clarification.
+- **Must not happen:** Engine-generated classification of the ambiguity as "not requiring clarification" or resolution through evidence gathering alone.
+
+---
+
+## 7. What Is Still Unproven
+
+The following areas remain genuinely unresolved. Tomorrow's implementation should treat these as open questions, not settled decisions:
+
+- **Stability across larger case sets:** Six regression cases plus supporting observations cover key patterns, but the engine has not been tested against diverse answer patterns, domains, or multiple reasoning turns.
+- **Behaviour across other models:** All experiments used `qwen-claude:latest` on one instance. Different models (or different fine-tuning of the same model) may behave differently on these tasks.
+- **Exact production representation:** Whether statedMeaning/possibleInference, a provenance annotation layer, or another mechanism is appropriate for production has not been decided.
+- **Graph integration:** How these fields integrate into SituationGraph nodes and edges remains an open design question (Experiments 54A–54C confirmed the current graph lacks provenance fields).
+- **Downstream inference-field consumption:** How Behaviour Selection, UI, or later reasoning steps should consume `possibleInference` vs `statedMeaning` has not been tested.
+- **Clarification-target precision requirements:** The boundary between "materially different clarification target" and "coarser but workable framing" has been partially observed (54Y/54Z) but not generalised.
+- **Behaviour Selection integration:** No integration tests have been performed for the clarified-meaning pipeline with Behaviour Selection.
+- **UI timing and wording quality:** How and when clarification questions are presented to users, and whether the generated wording is appropriate, has not been tested beyond one neutral-question pass (54U).
+- **Performance/latency implications:** Adding semantic separation, grounding, or two-field interpretation calls will add latency. This has not been measured against production requirements.
+- **One call versus multiple calls:** The two-stage approach (55B/C) showed promise for conditionality but also propagation risk. The single-call two-field approach (55D) was cleaner but untested for resolution consumption. Which is appropriate remains undecided.
+
+---
+
+## 8. First Implementation Boundary
+
+The first production refinement should preserve user-supported meaning, model inference, qualification, and unresolved uncertainty distinctly enough that later reasoning cannot silently convert one into another. The engine must not strengthen relative importance into constraint boundaries, flatten conditional qualifications, or over-resolve uncertainty when the answer does not supply sufficient information.
+
+The implementation should be judged first against the regression pack above before expanding into Behaviour Selection, UI, or broader reasoning redesign.
+
+---
+
+## 9. Stop Conditions for the Next Implementation Pass
+
+The first implementation pass should stop and reassess if:
+
+- It requires redesigning the whole SituationGraph;
+- It requires broad UI changes;
+- It requires rewriting Behaviour Selection;
+- It requires rereading the complete historical experiment log (this document exists so this is not needed);
+- It introduces multiple new abstractions before passing the regression pack;
+- The implementation cannot be explained in a few paragraphs;
+- The first attempt starts changing unrelated engine behaviour.
+
+This section is scope-control guidance, not architecture design.