robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: test scope coherence against unknown count

Browse Source
This commit is contained in:
robbond
2026-08-06 19:11:23 +01:00
parent 1d234acd8c
commit 0de2ffb4be
3 changed files with 577 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+2 -2
View File
@@ -101,8 +101,8 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41, 42, 43, 44, 45. Branch: `feature/user-workspace-ux-v0.7`.*
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41, 42, 43, 44, 45, 46. Branch: `feature/user-workspace-ux-v0.7`.*
### Return-to-Work Note (Experiment 45)
Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns; resolved-item gate works correctly (one stays too_broad, two clears it). The boundary appears mechanically clear but conceptually uncertain — real-scenario validation remains open. No production code changed. What remains: whether health should be healthy (not cannot_determine) for 2–3 unknowns with no question; whether the threshold needs widening. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/investigation-state-assessor.too-broad-boundary.test.js` for results, then `docs/design-evolution-log.md` Experiment 45 section for full data.
Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. What remains open: whether a coherence signal should be added, and what threshold would feel right in real-user data. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/investigation-state-assessor.scope-coherence.test.js` for results, then `docs/design-evolution-log.md` Experiment 46 section for full analysis.