experiment: compare acknowledge priority alternatives
Experiment 41 compared two passive alternatives for reducing Acknowledge dominance: Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge - Converges on concluding→summarise and stalled→pause correctly - Introduces false-positive summarise at long-investigation t3 Variant B (Acknowledge exclusions): gate Acknowledge via phase/progress/health - Converges on the same two genuine changes without false-positives - Recommended: cleaner boundaries, preserves Acknowledge for healthy focus states Both variants produce identical results for 2 of 7 tested turns. Variant A diverges at long-investigation t3 (focusing phase with resolvedNodeCount=3). Variant B correctly preserves Acknowledge there via its exclusion list. Test files: - tests/behaviour-selection.counterfactual.test.js (44 tests, new) No production code changed.
This commit is contained in:
+10
-4
@@ -38,9 +38,15 @@ Experiment 39 resumed reasoning experiments with a passive validation of Behavio
|
||||
|
||||
Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarive's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.
|
||||
|
||||
Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:
|
||||
- Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
|
||||
- Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
|
||||
- Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause
|
||||
- No production code changed. Branch: `feature/user-workspace-ux-v0.7`.
|
||||
|
||||
## 5. What Remains Open
|
||||
|
||||
- Whether Acknowledge should be reordered past Summarise/Pause to resolve the priority conflict diagnosed in Exp 40;
|
||||
- Implement Variant B's acknowledgment exclusion (recommended path from Exp 41);
|
||||
- Whether the `too_broad` health trigger needs widening so Clarify fires in more typical investigations;
|
||||
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states.
|
||||
|
||||
@@ -88,8 +94,8 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38, 39, 40. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 39)
|
||||
### Return-to-Work Note (Experiment 41)
|
||||
|
||||
Experiment 40 audited why Clarify, Summarise, and Pause were absent in Experiment 39. Summarise and Pause are eligible_but_blocked: their rule conditions fire but Acknowledge (priority 1) always wins when health=healthy — a priority conflict. Clarify is never_eligible_in_tested_scenarios: its `too_broad` trigger requires activeUnknownCount > 3 AND resolved < 2 simultaneously, which no fixture reaches; the `orienting+low obs` path is never produced by the assessor for tested domains. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed. Status pending Rob's review. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `docs/current-handoff.md`, then `docs/design-evolution-log.md` entry for Experiment 40, and `tests/behaviour-selection.reachability.test.js` for diagnostic details. The next action is discussion about whether to reorder Acknowledge lower (past Summarise/Pause) or refine its trigger to exclude terminal phases.
|
||||
Experiments 39–41 diagnosed Acknowledge dominance in Behaviour Selection and compared two passive alternatives. **Experiment 41 concluded:** Variant B (Acknowledge exclusions via phase/progress/health gates) is recommended over Variant A (priority reordering, which introduces false-positive summarise). Both variants correctly identify the same two genuine changes: `concluding → summarise` and `stalled → pause`. No production code changed. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `docs/current-handoff.md`, then `docs/design-evolution-log.md` entry for Experiment 41, and `tests/behaviour-selection.counterfactual.test.js` for variant details. The next action is implementing Variant B's `isAcknowledgeExcluded()` gate as a test-only function, then evaluating against a wider range of assessment scenarios.
|
||||
|
||||
Reference in New Issue
Block a user