experiment: validate behaviour selection against real assessments

docs/current-handoff.md

@@ -4,18 +4,18 @@
 
 ## 1. Where We Left It
 
-- Engine experiments are paused after Experiment 25B;
-- UI experiments are paused;
+- Engine experiments resumed with a passive validation;
+- UI experiments remain paused;
 - Knowledge-management experiments are complete;
-- Experiment 38 validated that a cold session recovers project state from three documents alone;
-- The working tree was clean at commit `544573a` (Exp 37).
+- Experiment 39 tested the existing Behaviour Selection module against real Investigation State Assessment outputs across three scenarios;
+- Acknowledge dominates (71% of selections) because it fires first when health=healthy, blocking Summarise/Pause/Clarify even in concluding or stalled states.
 
 > This handoff describes the latest stopping point only. When work moves on, replace stale current-work details rather than appending another historical note. Historical experiment and commit information belongs in `docs/design-evolution-log.md`.
 
 ## 2. What Is True Now
 
 - Main active engine path: deterministic reasoning pipeline (scenario reconstruction, graph update, unknown selection, question formulation, turn orchestration).
-- Passive experimental classifiers from Experiments 18–25B remain isolated diagnostic layers; none control the user-facing investigation.
+- Passive experimental classifiers from Experiments 18–25B remain isolated diagnostic layers; none control the user-facing investigation. Behaviour Selection was passively evaluated against real assessment outputs in Experiment 39 — it produced all valid behaviours but with skewed distribution (Acknowledge 71%).
 - Keyword and phrase-based scope detection remains provisional scaffolding.
 - `docs/current-project-state.md` is the main entry point for active project state.
 - `docs/task-context-packs.md` chooses the minimum context documents for each work type.
@@ -34,9 +34,13 @@ Experiment 38 tested whether a genuinely cold session (no prior conversation con
 
 **Commit:** pending (experiment: validate cold-start project recovery) — to be committed this session.
 
+Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.
+
 ## 5. What Remains Open
 
-- Whether tasks crossing pack boundaries can still stay concise without loading excessive context (not tested by this experiment).
+- Whether the Acknowledge→Summarise priority conflict can be resolved without breaking existing rule conditions;
+- Whether behaviour selection produces different patterns in investigation domains other than supplier procurement, market entry, and product comparison;
+- Which rules (Acknowledge, Clarify, Summarise, Pause) fire most frequently across a larger set of scenarios.
 
 ### When This Knowledge-Management Phase Is Complete
 
@@ -82,8 +86,8 @@ Answer before continuing:
 
 ---
 
-*Created by Experiment 34. Updated by Experiment 38. Branch: `feature/user-workspace-ux-v0.7`.*
+*Created by Experiment 34. Updated by Experiments 38, 39. Branch: `feature/user-workspace-ux-v0.7`.*
 
-### Return-to-Work Note (Experiment 38)
+### Return-to-Work Note (Experiment 39)
 
-Experiment 38 was performed from a cold session with no prior conversation context and confirmed that the reduced context system works as designed. A genuinely fresh session recovered the complete project state—active capabilities, passive diagnostic layers, paused work items, and context-pack routing—from just three documents without reading the full design-evolution log or any archived material. All seven knowledge-management completion criteria are confirmed met. The handoff itself was found to be accurate with no stale or ambiguous statements; one structural update was made (resolving the open item about handoff longevity). Engine and UI experiments remain paused pending Rob's review. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when returning: `docs/current-handoff.md`, then `docs/current-project-state.md` and the Engine Experiment pack per the routing table. The next action is discussion, not automatic implementation.
+Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs across three scenarios (long investigation, contradictory evidence, short early). No user-facing engine behaviour changed. The selector produced all valid behaviours but with skewed distribution: Acknowledge dominated at 71% because it fires first when health=healthy, even in terminal/concluding states where Summarise would be more useful — this is an acknowledged priority conflict, not a contract mismatch. The assessor→selector pipeline aligns cleanly. Clarify and Pause never fired (no too_broad health produced; other rules blocked). Status pending Rob's review. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when returning: `docs/current-handoff.md`, then `docs/design-evolution-log.md` entry for Experiment 39, and `tests/behaviour-selection.real-assessment.test.js` for evaluation details. The next action is discussion about whether to refine the selector or move on.

docs/current-project-state.md

@@ -46,7 +46,7 @@ These are what currently affect the working engine:
 The following were built during Experiments 18–25B. They are isolated diagnostic layers with no active integration into the user-facing investigation:
 
 - Investigation-state assessment (phase and progress classification);
-- Behaviour selection from assessed state;
+- Behaviour selection from assessed state — passively evaluated in Experiment 39 against real assessment outputs; produces all valid behaviours but Acknowledge dominates when health=healthy;
 - Decision condition status evaluation;
 - Question-to-condition relevance scoring;
 - Evidence direction classification (support, contradict, inform);
@@ -75,7 +75,7 @@ The following were built during Experiments 18–25B. They are isolated diagnost
 
 ## 6. Work Currently Paused
 
-- Engine experiments are paused after Experiment 25B;
+- Engine experiments are at Experiment 39 (passive Behaviour Selection validation, awaiting review);
 - UI experiments are paused;
 - Knowledge-management experiments are complete (confirmed by Experiment 38 cold-start validation);
 - Nothing historical has been deleted or archived yet.
@@ -100,12 +100,10 @@ Historical documents are retained under `docs/archive/` and should be opened onl
 
 ## 8. Return-to-Work Summary
 
-Engine experiments are paused after Experiment 25B, which established scope-aware condition status — distinguishing direct evidence from relevant-but-different claims by checking subject, timeframe and claim type. The passive classifier layers (Exps 18–25B) remain isolated; none yet control the user-facing investigation. Current focus is knowledge management: creating a short current-state entry point so future sessions do not need to read the full experiment history.
+Engine experiments advanced to Experiment 39, a passive validation of Behaviour Selection against real Investigation State Assessment outputs. The existing selector produces all valid behaviours but Acknowledge dominates (71%) because it fires first when health=healthy. No user-facing engine behaviour changed. Knowledge-management phase is complete: all seven criteria met. No backlog items deleted or promoted.
 
 First document to read: `docs/current-project-state.md`. Then consult `.claude/architecture-guardrails.md` before any code changes and `docs/project-knowledge-inventory.md` for task-specific references. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading.
 
-Engine experiments remain paused after Experiment 25B. UI experiments are paused. Knowledge-management phase is complete: all seven criteria met. No backlog items deleted or promoted. **First file to inspect when resuming:** `docs/current-project-state.md`.
-
 ## Verification Marker
 
 Implementation status last checked against source: Experiment 28.

docs/design-evolution-log.md

@@ -2264,3 +2264,63 @@ All seven knowledge-management completion criteria are confirmed met by this col
 The cold-start validation passed. A genuinely fresh session understood the project state, chose the correct context pack, verified the resume boundary, produced a valid engine-work resume brief, and found no handoff defects — all from three documents alone. No source code was read or changed. The reduced context system works for sessions that did not help create the documents.
 
 Engine and UI experiments remain paused pending Rob's review.
+
+---
+
+## Experiment 39 — Validate Behaviour Selection Against Real Assessment Outputs (2026-08-06)
+
+**Branch:** feature/user-workspace-ux-v0.7
+
+### Hypothesis
+The existing deterministic selector produces a useful rhythm across genuine assessment outputs without changing the active engine. If it repeatedly chooses one behaviour, chooses behaviours at the wrong time, or depends on signals the assessor does not actually produce, the experiment should expose that honestly.
+
+### Scenarios Evaluated (from `tests/investigation-state-assessor.test.js` fixture set)
+1. **Long investigation** (3 turns: early → deepening → complete terminal)
+2. **Contradictory evidence** (3 turns: two conflicting consultants, 0→1→2 resolved unknowns)
+3. **Short early** (1 turn: two observations, first unknown, no resolution)
+
+### Behaviour Distribution (7 turns total)
+- Acknowledge: 5 (71%)
+- Continue: 2 (29%)
+- Clarify: 0 (0%)
+- Summarise: 0 (0%)
+- Pause: 0 (0%)
+
+### Behaviour Sequence by Scenario
+**Long investigation:** continue → acknowledge → acknowledge
+- Turn 0: phase=cannot_determine, progress=cannot_determine, health=too_narrow → continue (no rule matched)
+- Turn 3: phase=focusing, progress=steady, health=healthy → acknowledge
+- Turn 4: phase=concluding, progress=steady, health=healthy → acknowledge
+
+**Contradictory evidence:** acknowledge → acknowledge → acknowledge
+- Turn 0: phase=focusing, progress=cannot_determine, health=healthy → acknowledge
+- Turn 1: phase=focusing, progress=stalled, health=healthy → acknowledge
+- Turn 2: phase=focusing, progress=steady, health=healthy → acknowledge
+
+**Short early:** continue
+- Turn 0: phase=exploring, progress=cannot_determine, health=healthy → continue
+
+### Sensible Selections (7 of 7)
+All selections were classified as sensible per the selection's stated conditions. Acknowledge fires because `health=healthy AND phase confidence≠low` across most states. Continue fires when no specific rule matches (early/cannot_determine/exploring phases).
+
+### Questionable or Inappropriate Selections
+**One notable pattern:** Summarise and Pause never fire, even in a concluding terminal state. This is not because the assessor fails to detect "concluding" — it does. It is because Acknowledge (priority 1) fires first when health=healthy, blocking Summarise (priority 3) from ever reaching its turn. This is an **acknowledgement/summarise priority conflict**: acknowledging a conclusion ("you've figured this out!") is not wrong, but "give me a summary" is more useful at terminal states. The current rule ordering does not distinguish "early healthy" from "concluding healthy."
+
+Clarify never fires because no test scenario produces `health=too_broad` — the assessor's "too_broad" trigger (activeUnknownCount > 3 AND resolved < 2) requires more nodes than any scenario in the fixture set has at that stage.
+
+Pause never fires because `health=user_overloaded` is never reached, and while contradictory-turn-1 has phase=focusing + progress=stalled, Acknowledge still blocks it.
+
+### Contract Alignment
+Assessor → Selector contract aligns cleanly. The assessor produces all three dimensions (phase, progress, conversationHealth) with the fields the selector expects. No transformation needed between pipeline stages.
+
+### Whether Selector Appears Useful Enough for Another Passive Experiment
+The existing selector works but its **behaviour variation is severely constrained by Acknowledge's priority position**. A next passive experiment should test whether reordering or refining the acknowledge condition (e.g., excluding concluding/terminal phases) produces more context-appropriate behaviour — without changing the assessor.
+
+### Status
+Pending Rob's review. Five behaviours are too narrow for this to be definitive, and only three scenarios were tested. The dominant pattern (acknowledge in healthy states) may change with different investigation domains.
+
+### Documents Updated
+- `docs/design-evolution-log.md` — this entry
+- `docs/current-handoff.md` — return-to-work note replaced
+
+---