diff --git a/docs/current-handoff.md b/docs/current-handoff.md index b85748a..31613a5 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -170,7 +170,7 @@ Answer before continuing: --- -*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L. Branch: `feature/reasoning-fidelity-v0.8`. Status pending Rob's review.* +*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to A–F scope.* ### Return-to-Work Note (Experiment 55F) @@ -239,3 +239,36 @@ This experiment does not prove consistency across repeated runs, fidelity for ot ### Experiment 56M Summary — Production evidence vs clarification routing validation Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review. + +### Reasoning Fidelity v0.8 — First Pass Closeout + +**The first-pass reasoning-fidelity refinement is complete to its agreed scope.** + +Regression boundaries A–F have been investigated and the production defects identified from those boundaries have been addressed: + +- **A — weak priority:** supported against unsupported strengthening via pre-mutation compatibility guard; +- **B — conditional trade-off:** qualification preserved through deterministic derivation and normalisation; +- **C — unresolved uncertainty:** may remain unresolved when the user supplies no position; +- **D — explicit hard constraint:** explicit meaning preserved; +- **E — evidence-resolvable disagreement:** routed to evidence gathering; +- **F — user-owned ambiguity:** routed to clarification. + +No demonstrated production defect remains inside the A–F first-pass boundary. Deterministic production validation is passing (commit `ec398dc` validating evidence vs. clarification routing). + +**Current HEAD:** `ec398dc` — experiment: validate evidence versus clarification routing +**Key commits:** `f861e2c` (preserve evidence vs. clarification distinction), `ec398dc` (validate evidence vs. clarification routing) + +The two important production capabilities now present are: + +1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary; +2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation. + +**Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme.** These open questions remain for future evidence-driven investigation, not as current defects: + +- broader wording/domain/model robustness; +- clarification-target precision outside the tested cases; +- durable per-node provenance of user-supported meaning vs inference; +- whether rejected proposals should eventually be adapted rather than simply blocked; +- end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI; +- multilingual robustness; +- any future defect exposed by real use. diff --git a/docs/current-project-state.md b/docs/current-project-state.md index b0765e9..1c7e11d 100644 --- a/docs/current-project-state.md +++ b/docs/current-project-state.md @@ -39,7 +39,8 @@ These are what currently affect the working engine: - Unknown selection using atomicity and answerability checks; - Question formulation within a selected reasoning pattern; - Scenario API (analyseScenario / updateCase); -- Investigation turn cycle orchestration. +- Investigation turn cycle orchestration; +- **Reasoning-fidelity v0.8 (completed):** user-supported meaning cannot silently outrun the raw answer at the mutation boundary; evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation. A–F regression boundaries closed for this pass. See `docs/current-handoff.md` for closeout details. ### Passive experimental capabilities @@ -109,5 +110,5 @@ First document to read: `docs/current-project-state.md`. Then consult `.claude/a Implementation status last checked against source: Experiment 43. The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required. -**Branch:** `feature/user-workspace-ux-v0.7` -**Latest known commit before this experiment:** `544573a` (experiment: validate cross-boundary context routing, Exp 37) +**Branch:** `feature/reasoning-fidelity-v0.8` +**Current HEAD:** `ec398dc` (experiment: validate evidence versus clarification routing) diff --git a/docs/reasoning-refinement-requirements.md b/docs/reasoning-refinement-requirements.md index efc4730..c52ea69 100644 --- a/docs/reasoning-refinement-requirements.md +++ b/docs/reasoning-refinement-requirements.md @@ -12,6 +12,10 @@ This document records what can be relied on, which requirements follow from the --- +**Status:** First implementation pass completed against A–F regression boundaries on `feature/reasoning-fidelity-v0.8`; remaining open design questions are not claimed solved. + +--- + ## 2. What We Can Rely On So Far ### Source versus interpretation