experiment: observe post-admission investigation progress

This commit is contained in:
2026-08-11 06:56:35 +01:00
parent fbbd271596
commit d5db3c3cd6
2 changed files with 138 additions and 0 deletions
+6
View File
@@ -280,6 +280,12 @@ This experiment does not prove consistency across repeated runs, fidelity for ot
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
### Experiment 57J.26 — Post-Admission Investigation Progress
Tested whether the engine makes genuine investigative progress after admitting two user-supported unknowns (Answer 1: savings realism + retention evidence). **Classification: D — NEW VALIDATION / REASONING FAILURE.** Both runs failed at Update 1's `proposal_compatibility` stage. Run 1 (57J.26 scenario): `"New unknown must be explicitly related to an answer-derived node"` — the old provenance-link gate returned, contradicting 57J.25's confirmed pass. Run 2 (exact 57J.25 scenario): `"Proposal cannot resolve beyond an unclassified answer by introducing unsupported stronger meaning"` — a different semantic compatibility error also blocks Update 2. The v0.15 unknown admission fix from 57J.25 cannot be reproduced under the same commit (`fbbd271`). Run-to-run variance in start graph construction (6→7→9 nodes) also observed. Status: unproven — needs regression investigation. No production code changed.
---
### Reasoning Fidelity v0.8 — First Pass Closeout
**The first-pass reasoning-fidelity refinement is complete to its agreed scope.**