docs: close reasoning fidelity v0.8 first pass
This commit is contained in:
+34
-1
@@ -170,7 +170,7 @@ Answer before continuing:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L. Branch: `feature/reasoning-fidelity-v0.8`. Status pending Rob's review.*
|
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to A–F scope.*
|
||||||
|
|
||||||
### Return-to-Work Note (Experiment 55F)
|
### Return-to-Work Note (Experiment 55F)
|
||||||
|
|
||||||
@@ -239,3 +239,36 @@ This experiment does not prove consistency across repeated runs, fidelity for ot
|
|||||||
### Experiment 56M Summary — Production evidence vs clarification routing validation
|
### Experiment 56M Summary — Production evidence vs clarification routing validation
|
||||||
|
|
||||||
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
|
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
|
||||||
|
|
||||||
|
### Reasoning Fidelity v0.8 — First Pass Closeout
|
||||||
|
|
||||||
|
**The first-pass reasoning-fidelity refinement is complete to its agreed scope.**
|
||||||
|
|
||||||
|
Regression boundaries A–F have been investigated and the production defects identified from those boundaries have been addressed:
|
||||||
|
|
||||||
|
- **A — weak priority:** supported against unsupported strengthening via pre-mutation compatibility guard;
|
||||||
|
- **B — conditional trade-off:** qualification preserved through deterministic derivation and normalisation;
|
||||||
|
- **C — unresolved uncertainty:** may remain unresolved when the user supplies no position;
|
||||||
|
- **D — explicit hard constraint:** explicit meaning preserved;
|
||||||
|
- **E — evidence-resolvable disagreement:** routed to evidence gathering;
|
||||||
|
- **F — user-owned ambiguity:** routed to clarification.
|
||||||
|
|
||||||
|
No demonstrated production defect remains inside the A–F first-pass boundary. Deterministic production validation is passing (commit `ec398dc` validating evidence vs. clarification routing).
|
||||||
|
|
||||||
|
**Current HEAD:** `ec398dc` — experiment: validate evidence versus clarification routing
|
||||||
|
**Key commits:** `f861e2c` (preserve evidence vs. clarification distinction), `ec398dc` (validate evidence vs. clarification routing)
|
||||||
|
|
||||||
|
The two important production capabilities now present are:
|
||||||
|
|
||||||
|
1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary;
|
||||||
|
2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation.
|
||||||
|
|
||||||
|
**Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme.** These open questions remain for future evidence-driven investigation, not as current defects:
|
||||||
|
|
||||||
|
- broader wording/domain/model robustness;
|
||||||
|
- clarification-target precision outside the tested cases;
|
||||||
|
- durable per-node provenance of user-supported meaning vs inference;
|
||||||
|
- whether rejected proposals should eventually be adapted rather than simply blocked;
|
||||||
|
- end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI;
|
||||||
|
- multilingual robustness;
|
||||||
|
- any future defect exposed by real use.
|
||||||
|
|||||||
@@ -39,7 +39,8 @@ These are what currently affect the working engine:
|
|||||||
- Unknown selection using atomicity and answerability checks;
|
- Unknown selection using atomicity and answerability checks;
|
||||||
- Question formulation within a selected reasoning pattern;
|
- Question formulation within a selected reasoning pattern;
|
||||||
- Scenario API (analyseScenario / updateCase);
|
- Scenario API (analyseScenario / updateCase);
|
||||||
- Investigation turn cycle orchestration.
|
- Investigation turn cycle orchestration;
|
||||||
|
- **Reasoning-fidelity v0.8 (completed):** user-supported meaning cannot silently outrun the raw answer at the mutation boundary; evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation. A–F regression boundaries closed for this pass. See `docs/current-handoff.md` for closeout details.
|
||||||
|
|
||||||
### Passive experimental capabilities
|
### Passive experimental capabilities
|
||||||
|
|
||||||
@@ -109,5 +110,5 @@ First document to read: `docs/current-project-state.md`. Then consult `.claude/a
|
|||||||
Implementation status last checked against source: Experiment 43.
|
Implementation status last checked against source: Experiment 43.
|
||||||
The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required.
|
The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required.
|
||||||
|
|
||||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
**Branch:** `feature/reasoning-fidelity-v0.8`
|
||||||
**Latest known commit before this experiment:** `544573a` (experiment: validate cross-boundary context routing, Exp 37)
|
**Current HEAD:** `ec398dc` (experiment: validate evidence versus clarification routing)
|
||||||
|
|||||||
@@ -12,6 +12,10 @@ This document records what can be relied on, which requirements follow from the
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
**Status:** First implementation pass completed against A–F regression boundaries on `feature/reasoning-fidelity-v0.8`; remaining open design questions are not claimed solved.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## 2. What We Can Rely On So Far
|
## 2. What We Can Rely On So Far
|
||||||
|
|
||||||
### Source versus interpretation
|
### Source versus interpretation
|
||||||
|
|||||||
Reference in New Issue
Block a user