robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: distinguish clarification from evidence needs

Browse Source
This commit is contained in:
robbond
2026-08-07 19:28:45 +01:00
parent dd3b8e505d
commit ee1391cd00
3 changed files with 526 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+5 -3
View File
@@ -19,6 +19,8 @@
- Keyword and phrase-based scope detection remains provisional scaffolding.
- `docs/current-project-state.md` is the main entry point for active project state.
- Experiment 54D confirmed the production update prompt explicitly separates the user answer (## User Answer section) but the proposal schema has no provenance field — source identity at prompt level is explicit, per-node provenance at output level is absent.
Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). The model did not collapse all ambiguity into clarification, did not generate any question or choose a winner. No production code changed. Status pending Rob's review.
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.
## 3. Why Work Is Paused
@@ -141,8 +143,8 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54Q. Branch: `feature/user-workspace-ux-v0.7`.*
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54R. Branch: `feature/user-workspace-ux-v0.7`.*
### Return-to-Work Note (Experiment 54Q)
### Return-to-Work Note (Experiment 54R)
Experiment 54N failed when moving directly from hypotheses to consequence on the staff-capacity versus supplier-lead-time case. Experiment 54O showed evidence discrimination works when asked directly. Experiment 54P showed explicit supplied evidence restores consequence judgement. Experiment 54Q tested whether one structured semantic call can first expose evidence needs and then preserve the consequence — it succeeded across all three cases including the original 54N failure. No winner or question generated; same host/model; no production behaviour changed. Whether this explains the 54N failure or only narrows it remains open. Branch: `feature/user-workspace-ux-v0.7`. First test/file to inspect when resuming: `tests/reconstruction/semantic-structured-evidence-consequence.test.js`.
54Q showed one structured hypothesis → evidence → consequence call worked across three tested cases but did not isolate which reformulation caused recovery. 54R tested whether a consequential disagreement actually requires the user or can be resolved through evidence — three cases: competing delivery causes (false), growth-versus-risk ambiguity (true), no-material-disagreement control (false). All three correct (3/3). The model distinguished missing evidence from missing user meaning without generating any question, choosing a winner, or changing Behaviour Selection. Same host/model; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`. First test/file to inspect when resuming: `tests/reconstruction/semantic-disagreement-resolution-source.test.js`.