experiment: add passive behaviour selection
Implement Experiment 19: deterministic behaviour selector with five behaviours (Acknowledge, Clarify, Summarise, Continue, Pause). - lib/behaviour-selection/behaviour-selector.js — Pure function selector applying v0.1 rules in priority order (acknowledge > clarify > summarise > pause > continue). Defaults to Continue with low confidence when no rule matches or assessment is incomplete. Guards against partial objects. - tests/behaviour-selector.test.js — 51 tests covering all five behaviours, priority ordering, contract conformance, determinism, edge cases, and scenario-based validation with mock investigations. - docs/design-evolution-log.md — Close Experiment 18 (record what assessor enabled for Behaviour Selection), add Experiment 19 section with hypothesis, scope, evaluation criteria, and open questions. Passive integration only: no changes to reasoning engine, prompts, graph generation, decomposition, narrative generation, API contracts, UI behaviour, or Ollama integration.
This commit is contained in:
@@ -942,7 +942,62 @@ Phase detection (orienting / exploring / focusing / deepening / synthesising / c
|
||||
|
||||
#### Status
|
||||
|
||||
Completed — see `investigation-state-assessment-contract.md` and `lib/assessment/investigation-state-assessor.js`.
|
||||
**Closed.** The assessment is implemented, tested, and validated. See `investigation-state-assessment-contract.md` and `lib/assessment/investigation-state-assessor.js`.
|
||||
|
||||
#### Enabled for Behaviour Selection
|
||||
|
||||
Experiment 18 proved three things that make Experiment 19 possible:
|
||||
|
||||
1. **Phase detection works.** We can classify investigation phase (orienting / exploring / focusing / deepening / synthesising / concluding) from existing graph data with measurable confidence. This is the primary input for behaviour selection — without it, selection rules have no state to operate on.
|
||||
|
||||
2. **Progress tracking works.** Stalled progress in a focusing phase becomes a concrete signal that the facilitator should hold space rather than push. Previously this was an architectural idea; now it's observable data.
|
||||
|
||||
3. **Conversation health is measurable.** Healthy, too_broad, and user_overloaded states are detectable from question distribution and response patterns. `too_broad` triggers Clarify; healthy with resolution triggers Acknowledge — but only if the assessment layer exists to provide these signals.
|
||||
|
||||
Without Experiment 18, Behaviour Selection would have two options: inspect the graph directly (coupling behaviour to implementation) or use narrative fields as proxy signals (fragile by design). The assessment layer provides a stable contract — the three reliable dimensions listed above — that behaviour selection can depend on without fear of breaking when the graph schema changes.
|
||||
|
||||
Experiment 18 also proved that `cannot_determine` is not a failure mode but the correct answer when evidence is insufficient. This principle carries directly into behaviour selection: "no explicit rule matched" defaults to continue, not an invented signal.
|
||||
|
||||
---
|
||||
|
||||
### Experiment 19 — Passive Behaviour Selection
|
||||
|
||||
#### Hypothesis
|
||||
|
||||
Does selecting from a small set of five behaviours (Acknowledge, Clarify, Summarise, Continue, Pause) — instead of always asking — make the investigation feel more like guided thinking and less like automated Q&A?
|
||||
|
||||
This is one question. Nothing else matters until this is answered.
|
||||
|
||||
#### Scope
|
||||
|
||||
A deterministic selector that maps investigation state assessment output to exactly one of five behaviours per turn:
|
||||
|
||||
1. **Acknowledge** — when conversation health is healthy AND phase confidence is not low
|
||||
2. **Clarify** — when health is `too_broad` OR (phase is orienting AND observations < 3)
|
||||
3. **Summarise** — when phase is synthesising/concluding OR (≥ 3 resolved with steady progress)
|
||||
4. **Pause** — when phase is focusing AND progress is stalled; also user_overloaded health
|
||||
5. **Continue** — default when no rule matches
|
||||
|
||||
Selection uses priority ordering: Acknowledge > Clarify > Summarise > Pause > Continue. No scoring, no weighting, no convergence thresholds. First matching rule wins.
|
||||
|
||||
The selector is passive — deployed only through Developer Details diagnostics. No changes to reasoning engine, prompts, graph generation, decomposition, narrative generation, API contracts, UI behaviour, or Ollama integration.
|
||||
|
||||
#### Evaluation Criteria
|
||||
|
||||
1. **Behaviour diversity:** Does the system deploy at least 3 different behaviours across a normal investigation, or does it default to Continue most of the time?
|
||||
2. **Acknowledge appears:** Does Acknowledge fire whenever new information resolves an uncertainty? If not, the trigger condition is wrong — fix it, don't abandon selection.
|
||||
3. **Pause feels like relief, not delay:** When Pause fires, does the user experience it as a natural break rather than a system failure to produce a question?
|
||||
4. **Summarise compresses meaningfully:** Does the summarised understanding feel useful or redundant?
|
||||
5. **Conversation rhythm changes:** Is there a perceptible difference between "engine always asking" and "engine sometimes acknowledging/summarising/pausing first"?
|
||||
|
||||
If none of these can be evaluated after 2–3 real investigations with v0.1, the experiment was too small to answer the question.
|
||||
|
||||
#### Open Questions
|
||||
|
||||
- Which of the five behaviours fires most frequently in practice?
|
||||
- Does Acknowledge actually appear during investigations that would normally produce continuous questioning?
|
||||
- Does the priority ordering create appropriate urgency (Acknowledge > Clarify > Summarise > Pause > Continue)?
|
||||
- Are there cases where `cannot_determine` produces inappropriate behaviour selection — or is this the correct conservative default?
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user