Feature/product platform foundation v0.62 #1
@@ -36,17 +36,16 @@ Experiment 38 tested whether a genuinely cold session (no prior conversation con
|
||||
|
||||
Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.
|
||||
|
||||
Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarive's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.
|
||||
Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarify's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.
|
||||
|
||||
Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:
|
||||
- Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
|
||||
- Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
|
||||
- Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause
|
||||
- No production code changed. Branch: `feature/user-workspace-ux-v0.7`.
|
||||
Experiment 42 implemented Variant B's narrow Acknowledge exclusion gate in the production selector (commit `05d3d96`). Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change. Clarify remains an open question. Branch: `feature/user-workspace-ux-v0.7`.
|
||||
|
||||
## 5. What Remains Open
|
||||
|
||||
- Implement Variant B's acknowledgment exclusion (recommended path from Exp 41);
|
||||
- Whether the `too_broad` health trigger needs widening so Clarify fires in more typical investigations;
|
||||
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states.
|
||||
|
||||
@@ -94,8 +93,8 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41, 42. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 41)
|
||||
### Return-to-Work Note (Experiment 42)
|
||||
|
||||
Experiments 39–41 diagnosed Acknowledge dominance in Behaviour Selection and compared two passive alternatives. **Experiment 41 concluded:** Variant B (Acknowledge exclusions via phase/progress/health gates) is recommended over Variant A (priority reordering, which introduces false-positive summarise). Both variants correctly identify the same two genuine changes: `concluding → summarise` and `stalled → pause`. No production code changed. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `docs/current-handoff.md`, then `docs/design-evolution-log.md` entry for Experiment 41, and `tests/behaviour-selection.counterfactual.test.js` for variant details. The next action is implementing Variant B's `isAcknowledgeExcluded()` gate as a test-only function, then evaluating against a wider range of assessment scenarios.
|
||||
Experiment 42 implemented the narrow Acknowledge exclusion in the production Behaviour Selection module. Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change. Clarify remains unresolved and was not modified in this experiment. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-handoff.md`, then `lib/behaviour-selection/behaviour-selector.js` for the exclusion gate, and `tests/behaviour-selection.reachability.test.js` for the final real-turn distribution assertions.
|
||||
|
||||
@@ -75,7 +75,7 @@ The following were built during Experiments 18–25B. They are isolated diagnost
|
||||
|
||||
## 6. Work Currently Paused
|
||||
|
||||
- Engine experiments are at Experiment 40 (behaviour reachability diagnostic — no rule changes).
|
||||
- Engine experiments advanced through Experiment 42 (narrow Acknowledge exclusion gate for behaviour selection; production selector changed but Behaviour Selection remains isolated with no runtime caller).
|
||||
- UI experiments are paused;
|
||||
- Knowledge-management experiments are complete (confirmed by Experiment 38 cold-start validation);
|
||||
- Nothing historical has been deleted or archived yet.
|
||||
@@ -100,7 +100,7 @@ Historical documents are retained under `docs/archive/` and should be opened onl
|
||||
|
||||
## 8. Return-to-Work Summary
|
||||
|
||||
Engine experiments advanced to Experiment 41, which compared two passive alternatives for reducing Acknowledge dominance. Variant B (Acknowledge exclusions via phase/progress/health gates) is recommended over Variant A (priority reordering with false-positive side effects). Both variants converge on the same two genuine changes: `concluding → summarise` and `stalled → pause`. No production code changed.
|
||||
Engine experiments advanced through Experiment 42, which implemented Variant B's narrow Acknowledge exclusion gate in the production selector. Summarise now appears at conclusion; Pause now appears when stalled. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change. Clarify remains unresolved. No active tests rerun as part of documentation closure.
|
||||
|
||||
First document to read: `docs/current-project-state.md`. Then consult `.claude/architecture-guardrails.md` before any code changes and `docs/project-knowledge-inventory.md` for task-specific references. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading.
|
||||
|
||||
|
||||
@@ -2494,3 +2494,91 @@ Both variants were implemented as test-only functions in `tests/behaviour-select
|
||||
- `docs/design-evolution-log.md` — this entry
|
||||
- `docs/current-handoff.md` — return-to-work note replaced
|
||||
|
||||
|
||||
## Experiment 41 — Conclusion
|
||||
|
||||
**Variant B was preferred because it changed only the two intended turns without introducing a false-positive in a focusing state. Variant A produced an early summarise in a focusing phase and was discarded. No production rule changed during Experiment 41.** The implementation of Variant B's exclusion gate is the subject of Experiment 42.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 42 — Implement Narrow Acknowledge Exclusion (Variant B) (2026-08-06)
|
||||
|
||||
### Hypothesis
|
||||
|
||||
Applying a narrow exclusion gate to Acknowledge — excluding it when phase is synthesising or concluding, progress is stalled, or conversation health is user_overloaded — will reduce the two identified false-Acknowledge selections (concluding → summarise, stalled → pause) without introducing any unintended behaviour changes in other tested turns.
|
||||
|
||||
### Exact Exclusion Rule
|
||||
|
||||
`isAcknowledgeExcluded(assessment)` returns `true` when:
|
||||
- `phase.value` is `synthesising` or `concluding`; OR
|
||||
- `progress.value` is `stalled`; OR
|
||||
- `conversationHealth.value` is `user_overloaded`.
|
||||
|
||||
When excluded, Acknowledge does not fire and the selector proceeds to the next priority rule. The gate qualifies the trigger; it does not replace it.
|
||||
|
||||
### Two Changed Turns
|
||||
|
||||
| Turn | Scenario | Phase | Progress | Health | Before | After |
|
||||
|------|----------|-------|----------|--------|--------|-------|
|
||||
| long-investigation t4 | concluding long-investigation | concluding(high) | steady | healthy | acknowledge | **summarise** |
|
||||
| contradictory-evidence t1 | stalled contradictory-evidence | focusing(high) | stalled | healthy | acknowledge | **pause** |
|
||||
|
||||
### Five Preserved Turns
|
||||
|
||||
| Turn | Scenario | Phase | Progress | Health | Behaviour (unchanged) |
|
||||
|------|----------|-------|----------|--------|----------------------|
|
||||
| long-investigation t0 | cannot_determine(low) | cannot_determine | too_narrow | continue |
|
||||
| long-investigation t3 | focusing(high) | steady | healthy | acknowledge |
|
||||
| contradictory-evidence t0 | focusing(high) | cannot_determine | healthy | acknowledge |
|
||||
| contradictory-evidence t2 | focusing(high) | steady | healthy | acknowledge |
|
||||
| short-early t0 | exploring(low) | cannot_determine | healthy | continue |
|
||||
|
||||
### Final Behaviour Distribution (7 real assessment turns)
|
||||
|
||||
- Acknowledge: 3
|
||||
- Summarise: 1
|
||||
- Pause: 1
|
||||
- Continue: 2
|
||||
- Clarify: 0
|
||||
|
||||
### Integration Status
|
||||
|
||||
The production Behaviour Selection module (`lib/behaviour-selection/behaviour-selector.js`) was changed to include the `isAcknowledgeExcluded()` gate. However, **active user-facing engine behaviour did not change** because Behaviour Selection remains isolated with no runtime caller — it is exported but never imported by any code in the repository.
|
||||
|
||||
### Clarify Status
|
||||
|
||||
`Clarify` remains unresolved and was not modified in this experiment. Its trigger conditions (`health=too_broad` or `phase=orienting + obs<3`) require states that no tested scenario produces. This remains an open question for future work.
|
||||
|
||||
### Selector Output Shape
|
||||
|
||||
The selector output shape did not change. The exclusion gate returns `null` from `selectAcknowledge`, which is the existing early-return mechanism used when a rule does not match. No new fields, no restructuring of the return object.
|
||||
|
||||
### Assessor and Fixtures
|
||||
|
||||
Assessor logic did not change. Fixtures did not change. Priority order did not change.
|
||||
|
||||
### Test Results
|
||||
|
||||
All 151 relevant tests passed across:
|
||||
- `tests/behaviour-selection.test.js`: 51 (no regressions)
|
||||
- `tests/behaviour-selection.reachability.test.js`: 33 (updated for new exclusion gate)
|
||||
- `tests/behaviour-selection.counterfactual.test.js`: 44 (from Exp 41, no changes)
|
||||
- `tests/behaviour-selection.real-assessment.test.js`: 16 (shared fixtures intact)
|
||||
|
||||
Tests were not rerun as part of this documentation-only closure. The recorded result comes from the implementation commit (05d3d96).
|
||||
|
||||
### Limitations
|
||||
|
||||
- Only seven real assessment turns across three scenarios were evaluated; other investigation domains may exhibit different patterns.
|
||||
- `health=user_overloaded` is excluded by rule but never produced by any current assessor fixture — it is untested in practice.
|
||||
- `Clarify` remains deferred because no scenario produces the narrow trigger conditions it requires.
|
||||
- The selector remains isolated with no runtime caller; there is no live user-facing validation.
|
||||
|
||||
### Result
|
||||
|
||||
**Confirmed within the tested scenarios.** Variant B correctly changes only the two intended turns and preserves all five others. No unintended side-effects were observed.
|
||||
|
||||
### Documents Updated
|
||||
|
||||
- `docs/design-evolution-log.md` — this entry
|
||||
- `docs/current-handoff.md` — return-to-work note replaced
|
||||
|
||||
Reference in New Issue
Block a user