docs: close reasoning fidelity v0.8 first pass

This commit is contained in:
2026-08-09 17:25:19 +01:00
parent ec398dcec9
commit e498bbcc63
3 changed files with 42 additions and 4 deletions
+34 -1
View File
@@ -170,7 +170,7 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 3853, 54A54Z, 55A55F, 56D56H, 56L. Branch: `feature/reasoning-fidelity-v0.8`. Status pending Rob's review.*
*Created by Experiment 34. Updated by Experiments 3853, 54A54Z, 55A55F, 56D56H, 56L56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to AF scope.*
### Return-to-Work Note (Experiment 55F)
@@ -239,3 +239,36 @@ This experiment does not prove consistency across repeated runs, fidelity for ot
### Experiment 56M Summary — Production evidence vs clarification routing validation
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
### Reasoning Fidelity v0.8 — First Pass Closeout
**The first-pass reasoning-fidelity refinement is complete to its agreed scope.**
Regression boundaries AF have been investigated and the production defects identified from those boundaries have been addressed:
- **A — weak priority:** supported against unsupported strengthening via pre-mutation compatibility guard;
- **B — conditional trade-off:** qualification preserved through deterministic derivation and normalisation;
- **C — unresolved uncertainty:** may remain unresolved when the user supplies no position;
- **D — explicit hard constraint:** explicit meaning preserved;
- **E — evidence-resolvable disagreement:** routed to evidence gathering;
- **F — user-owned ambiguity:** routed to clarification.
No demonstrated production defect remains inside the AF first-pass boundary. Deterministic production validation is passing (commit `ec398dc` validating evidence vs. clarification routing).
**Current HEAD:** `ec398dc` — experiment: validate evidence versus clarification routing
**Key commits:** `f861e2c` (preserve evidence vs. clarification distinction), `ec398dc` (validate evidence vs. clarification routing)
The two important production capabilities now present are:
1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary;
2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation.
**Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme.** These open questions remain for future evidence-driven investigation, not as current defects:
- broader wording/domain/model robustness;
- clarification-target precision outside the tested cases;
- durable per-node provenance of user-supported meaning vs inference;
- whether rejected proposals should eventually be adapted rather than simply blocked;
- end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI;
- multilingual robustness;
- any future defect exposed by real use.