docs: close reasoning fidelity v0.8 first pass
This commit is contained in:
+34
-1
@@ -170,7 +170,7 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L. Branch: `feature/reasoning-fidelity-v0.8`. Status pending Rob's review.*
|
||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to A–F scope.*
|
||||
|
||||
### Return-to-Work Note (Experiment 55F)
|
||||
|
||||
@@ -239,3 +239,36 @@ This experiment does not prove consistency across repeated runs, fidelity for ot
|
||||
### Experiment 56M Summary — Production evidence vs clarification routing validation
|
||||
|
||||
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
|
||||
|
||||
### Reasoning Fidelity v0.8 — First Pass Closeout
|
||||
|
||||
**The first-pass reasoning-fidelity refinement is complete to its agreed scope.**
|
||||
|
||||
Regression boundaries A–F have been investigated and the production defects identified from those boundaries have been addressed:
|
||||
|
||||
- **A — weak priority:** supported against unsupported strengthening via pre-mutation compatibility guard;
|
||||
- **B — conditional trade-off:** qualification preserved through deterministic derivation and normalisation;
|
||||
- **C — unresolved uncertainty:** may remain unresolved when the user supplies no position;
|
||||
- **D — explicit hard constraint:** explicit meaning preserved;
|
||||
- **E — evidence-resolvable disagreement:** routed to evidence gathering;
|
||||
- **F — user-owned ambiguity:** routed to clarification.
|
||||
|
||||
No demonstrated production defect remains inside the A–F first-pass boundary. Deterministic production validation is passing (commit `ec398dc` validating evidence vs. clarification routing).
|
||||
|
||||
**Current HEAD:** `ec398dc` — experiment: validate evidence versus clarification routing
|
||||
**Key commits:** `f861e2c` (preserve evidence vs. clarification distinction), `ec398dc` (validate evidence vs. clarification routing)
|
||||
|
||||
The two important production capabilities now present are:
|
||||
|
||||
1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary;
|
||||
2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation.
|
||||
|
||||
**Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme.** These open questions remain for future evidence-driven investigation, not as current defects:
|
||||
|
||||
- broader wording/domain/model robustness;
|
||||
- clarification-target precision outside the tested cases;
|
||||
- durable per-node provenance of user-supported meaning vs inference;
|
||||
- whether rejected proposals should eventually be adapted rather than simply blocked;
|
||||
- end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI;
|
||||
- multilingual robustness;
|
||||
- any future defect exposed by real use.
|
||||
|
||||
@@ -39,7 +39,8 @@ These are what currently affect the working engine:
|
||||
- Unknown selection using atomicity and answerability checks;
|
||||
- Question formulation within a selected reasoning pattern;
|
||||
- Scenario API (analyseScenario / updateCase);
|
||||
- Investigation turn cycle orchestration.
|
||||
- Investigation turn cycle orchestration;
|
||||
- **Reasoning-fidelity v0.8 (completed):** user-supported meaning cannot silently outrun the raw answer at the mutation boundary; evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation. A–F regression boundaries closed for this pass. See `docs/current-handoff.md` for closeout details.
|
||||
|
||||
### Passive experimental capabilities
|
||||
|
||||
@@ -109,5 +110,5 @@ First document to read: `docs/current-project-state.md`. Then consult `.claude/a
|
||||
Implementation status last checked against source: Experiment 43.
|
||||
The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required.
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
**Latest known commit before this experiment:** `544573a` (experiment: validate cross-boundary context routing, Exp 37)
|
||||
**Branch:** `feature/reasoning-fidelity-v0.8`
|
||||
**Current HEAD:** `ec398dc` (experiment: validate evidence versus clarification routing)
|
||||
|
||||
@@ -12,6 +12,10 @@ This document records what can be relied on, which requirements follow from the
|
||||
|
||||
---
|
||||
|
||||
**Status:** First implementation pass completed against A–F regression boundaries on `feature/reasoning-fidelity-v0.8`; remaining open design questions are not claimed solved.
|
||||
|
||||
---
|
||||
|
||||
## 2. What We Can Rely On So Far
|
||||
|
||||
### Source versus interpretation
|
||||
|
||||
Reference in New Issue
Block a user