docs: close experiment 42 selector refinement
This commit is contained in:
@@ -36,17 +36,16 @@ Experiment 38 tested whether a genuinely cold session (no prior conversation con
|
||||
|
||||
Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.
|
||||
|
||||
Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarive's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.
|
||||
Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarify's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.
|
||||
|
||||
Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:
|
||||
- Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
|
||||
- Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
|
||||
- Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause
|
||||
- No production code changed. Branch: `feature/user-workspace-ux-v0.7`.
|
||||
Experiment 42 implemented Variant B's narrow Acknowledge exclusion gate in the production selector (commit `05d3d96`). Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change. Clarify remains an open question. Branch: `feature/user-workspace-ux-v0.7`.
|
||||
|
||||
## 5. What Remains Open
|
||||
|
||||
- Implement Variant B's acknowledgment exclusion (recommended path from Exp 41);
|
||||
- Whether the `too_broad` health trigger needs widening so Clarify fires in more typical investigations;
|
||||
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states.
|
||||
|
||||
@@ -94,8 +93,8 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41, 42. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 41)
|
||||
### Return-to-Work Note (Experiment 42)
|
||||
|
||||
Experiments 39–41 diagnosed Acknowledge dominance in Behaviour Selection and compared two passive alternatives. **Experiment 41 concluded:** Variant B (Acknowledge exclusions via phase/progress/health gates) is recommended over Variant A (priority reordering, which introduces false-positive summarise). Both variants correctly identify the same two genuine changes: `concluding → summarise` and `stalled → pause`. No production code changed. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `docs/current-handoff.md`, then `docs/design-evolution-log.md` entry for Experiment 41, and `tests/behaviour-selection.counterfactual.test.js` for variant details. The next action is implementing Variant B's `isAcknowledgeExcluded()` gate as a test-only function, then evaluating against a wider range of assessment scenarios.
|
||||
Experiment 42 implemented the narrow Acknowledge exclusion in the production Behaviour Selection module. Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change. Clarify remains unresolved and was not modified in this experiment. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-handoff.md`, then `lib/behaviour-selection/behaviour-selector.js` for the exclusion gate, and `tests/behaviour-selection.reachability.test.js` for the final real-turn distribution assertions.
|
||||
|
||||
Reference in New Issue
Block a user