docs: close reasoning fidelity v0.8 first pass
This commit is contained in:
+34
-1
@@ -170,7 +170,7 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L. Branch: `feature/reasoning-fidelity-v0.8`. Status pending Rob's review.*
|
||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to A–F scope.*
|
||||
|
||||
### Return-to-Work Note (Experiment 55F)
|
||||
|
||||
@@ -239,3 +239,36 @@ This experiment does not prove consistency across repeated runs, fidelity for ot
|
||||
### Experiment 56M Summary — Production evidence vs clarification routing validation
|
||||
|
||||
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
|
||||
|
||||
### Reasoning Fidelity v0.8 — First Pass Closeout
|
||||
|
||||
**The first-pass reasoning-fidelity refinement is complete to its agreed scope.**
|
||||
|
||||
Regression boundaries A–F have been investigated and the production defects identified from those boundaries have been addressed:
|
||||
|
||||
- **A — weak priority:** supported against unsupported strengthening via pre-mutation compatibility guard;
|
||||
- **B — conditional trade-off:** qualification preserved through deterministic derivation and normalisation;
|
||||
- **C — unresolved uncertainty:** may remain unresolved when the user supplies no position;
|
||||
- **D — explicit hard constraint:** explicit meaning preserved;
|
||||
- **E — evidence-resolvable disagreement:** routed to evidence gathering;
|
||||
- **F — user-owned ambiguity:** routed to clarification.
|
||||
|
||||
No demonstrated production defect remains inside the A–F first-pass boundary. Deterministic production validation is passing (commit `ec398dc` validating evidence vs. clarification routing).
|
||||
|
||||
**Current HEAD:** `ec398dc` — experiment: validate evidence versus clarification routing
|
||||
**Key commits:** `f861e2c` (preserve evidence vs. clarification distinction), `ec398dc` (validate evidence vs. clarification routing)
|
||||
|
||||
The two important production capabilities now present are:
|
||||
|
||||
1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary;
|
||||
2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation.
|
||||
|
||||
**Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme.** These open questions remain for future evidence-driven investigation, not as current defects:
|
||||
|
||||
- broader wording/domain/model robustness;
|
||||
- clarification-target precision outside the tested cases;
|
||||
- durable per-node provenance of user-supported meaning vs inference;
|
||||
- whether rejected proposals should eventually be adapted rather than simply blocked;
|
||||
- end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI;
|
||||
- multilingual robustness;
|
||||
- any future defect exposed by real use.
|
||||
|
||||
Reference in New Issue
Block a user