Files
confidence-engine/docs/facilitator-behaviour.md
robbond cb0c779019 architecture: introduce investigation state assessment
Close Experiment 15 (Facilitator Behaviour Specification).

Introduce Experiment 16 — Investigation State Assessment.

- Create docs/investigation-state-assessment.md with 7 assessment dimensions:
  Current Investigation Phase, Investigation Progress, Evidence Quality,
  Understanding Trajectory, Uncertainty Trend, Conversation Health,
  and Behaviour Readiness. Each dimension includes purpose, observable
  signals, possible values, and how behaviours may consume it.

- Document 6 assessment principles (Assess Not Decide, All Signals
  Traceable to Narrative, Descriptive Not Prescriptive, Convergence Over
  Single Signal, Stateful Across Turns, Uncertainty About Assessment Is
  Itself Assessable).

- Include exploratory decision matrix linking investigation states to
  likely behaviours with reasons.

- Prepend Behaviour Selection section to docs/facilitator-behaviour.md
  recording that behaviours are selected from Investigation State
  Assessment and do not inspect graph nodes directly.

- Update docs/design-evolution-log.md: close Experiment 15, add
  Experiment 16 closure, record emerging architecture with the new layer
  between Narrative and Behaviour Selection.

No implementation. Documentation only. No changes to reasoning engine,
graph generation, prompts, orchestrator, APIs, Ollama integration, or UI.
2026-08-05 16:09:24 +01:00

341 lines
18 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Facilitator Behaviour
> This is a behavioural specification, not an implementation. It describes what an expert facilitator *does* during an investigation — not what they say, not how the system implements it, and not a prompt.
This document records observed patterns from years of expert consulting practice and maps them onto the machine's capability. The goal is to make explicit the behaviour that was previously implicit in every interaction cycle.
---
## Behaviour Selection
Facilitator behaviours are selected from Investigation State Assessment.
Behaviours do not inspect graph nodes directly.
Behaviours consume assessment.
This keeps behaviour independent from reasoning implementation.
The assessment layer (introduced in Experiment 16) describes the investigation's current state across multiple analytical dimensions: phase, progress, evidence quality, understanding trajectory, uncertainty trend, conversation health, and behaviour readiness. Behaviour selection operates on this description — not on raw graph structure.
The decision of *which* behaviour to deploy is a separate concern from the inventory of *what* behaviours exist. This document defines the latter. The former depends on the assessment layer described in `docs/investigation-state-assessment.md`.
---
## What Experiment 14 proved
Experiment 14 proved architecturally that:
- The reasoning graph is an excellent *machine* representation of knowledge, uncertainty, and relationships.
- The narrative layer translates that graph into a human-understandable investigation state.
- The UI renders projections from the narrative — not directly from the graph.
What Experiment 14 did *not* address:
- How the facilitator *behaves* during an investigation turn.
- When to ask what.
- Why a particular question is chosen over alternatives.
- What patterns of behaviour distinguish a good facilitated investigation from a mechanical Q&A loop.
The architecture was sound. The behaviour remains implicit.
---
## What emerged (not anticipated)
During experiments 114, several unexpected patterns emerged:
### The interface stopped being the bottleneck before we finished optimising it
Every experiment focused on visual presentation — layout zones, semantic filtering, narrative translation, workspace structure. Each experiment confirmed that a calmer, more coherent workspace improves comprehension. But by Experiment 1314, further visual refinements yielded diminishing returns. The remaining gap was not visual: it was behavioural.
### The engine behaves mechanically where the facilitator should behave relationally
The current behaviour pattern is essentially:
```
Engine asks → User answers → Graph updates → Engine asks again
```
This is a valid reasoning loop. It is not how an expert consultant investigates. An expert consultant *listens*, observes patterns in what was said, notes gaps in understanding, and uses the next question to shape the direction of thinking — not simply to fill the next available unknown node.
### The graph captures state but not behaviour
The graph records *what* is known and *what remains uncertain*. It does not capture *how understanding developed* across the investigation. It cannot tell whether the last three questions followed a coherent investigative thread or randomly filled gaps.
### Conversation rhythm matters more than panel labels
Experiment 10 proved that a stable conversation column helps. Experiment 13 proved that semantic filtering improves clarity. But neither addresses: *does each turn feel like a natural step in an investigation, or does it feel like the system filling in a form?*
---
## Why the next phase is no longer primarily a UI problem
By Experiment 14, all remaining visual questions had been answered:
- Layout → two-column cognitive zones (confirmed)
- Hierarchy → strong question → response → history flow (confirmed)
- Facilitator panel → semantic translation of the graph (confirmed)
- Input sizing → lightweight observation, not report (confirmed)
- Loading states → context-aware processing messages (confirmed)
- Epistemic clarity → resolved vs. investigating clearly labelled (confirmed)
Further visual iteration will refine details but not change the core experience. The experience is defined by *what happens during a turn*, not by how panels are arranged.
The question now is:
> Given that the workspace layout is stable, what behaviour should the facilitator exhibit at each turn to make the investigation feel like genuine facilitated thinking rather than automated Q&A?
---
## Why conversation behaviour is the design focus
An expert consultant does not have a script. They have *behaviours* — recurring patterns of action that they deploy based on what they observe in the client's situation.
The same pattern applies to the machine facilitator:
- The engine *can* reason (graph).
- The engine *can* translate (narrative).
- What it should *do* differently at each stage depends on the investigation's current state and history — not just its graph.
The design focus shifts from "what should the panel show?" to "what should the facilitator do?"
---
## Facilitator Behavioural Model
This model describes behaviours as recurring patterns the facilitator deploys based on observable conditions in the investigation state. Each behaviour is triggered by a condition and produces a specific effect on the investigation's direction.
### 1. Orient
**Trigger:** Investigation begins (initial analysis complete).
**Behaviour:** The facilitator establishes what has been learned so far before anything else. It does not immediately ask questions. It presents the initial understanding as a shared baseline.
**Effect:** The user knows the engine has actually *heard* them, not just processed their input.
### 2. Acknowledge
**Trigger:** User provides new information that establishes or confirms something useful.
**Behaviour:** Before introducing any new uncertainty, the facilitator acknowledges what was gained. It integrates the information into the known state and explicitly notes what changed — what was resolved, what was strengthened, what remains the same.
**Effect:** Progress is visible to the user. Each turn adds value rather than just consuming input.
### 3. Observe pattern
**Trigger:** Multiple pieces of established information suggest a thematic connection or contradiction that has not yet been articulated.
**Behaviour:** The facilitator notes the pattern explicitly — without resolving it for the user. It does not infer conclusions. It surfaces what is *visible* in the current understanding.
**Effect:** The user gains meta-cognitive awareness of their own situation. They begin to see connections they had not made.
### 4. Clarify
**Trigger:** The user provides information that is partially useful but contains ambiguity, over-generalisation, or internal contradiction.
**Behaviour:** The facilitator isolates the ambiguous element and asks a narrow question targeted specifically at that element — not at filling a graph node.
**Effect:** The investigation gains precision without losing momentum.
### 5. Validate
**Trigger:** The user provides information that resolves an active uncertainty or strengthens a key understanding.
**Behaviour:** The facilitator explicitly marks the resolution and its consequence: "This tells us X, which means Y is no longer uncertain." It does not move on immediately — it pauses to integrate.
**Effect:** Understanding compounds. The user sees how pieces fit together rather than collecting facts.
### 6. Connect
**Trigger:** Two or more resolved items have a relationship that has not yet been explored (either explicitly by the graph or implicitly by the investigation).
**Behaviour:** The facilitator highlights the connection and proposes exploring it as a natural next step. It does not invent connections from weak evidence — only from established findings.
**Effect:** The investigation deepens organically rather than following a predetermined question list.
### 7. Challenge assumption
**Trigger:** An established fact or resolved unknown is used implicitly as a premise for further reasoning, but the evidence supporting it is thin, untested, or derived solely from user assertion.
**Behaviour:** The facilitator flags the assumption explicitly: "We are treating X as established. It has not yet been tested. Should we test it?" It does not dismiss the assumption — it exposes its status.
**Effect:** The user maintains epistemic integrity throughout the investigation. Weak foundations become visible before they undermine conclusions.
### 8. Refine understanding
**Trigger:** Multiple turns have passed and sufficient information exists to re-synthesise the current state more coherently than the last summary.
**Behaviour:** The facilitator restates the current understanding in a tighter, more integrated form — not as a repetition but as an evolution. It compresses without losing detail and surfaces implications that were implicit before.
**Effect:** The user gains confidence that their contributions are being meaningfully processed rather than mechanically stored.
### 9. Expose uncertainty
**Trigger:** The investigation has reached a point where understanding is partial — some areas are well-resolved, others remain uncertain, and the disparity is significant.
**Behaviour:** The facilitator makes the disparity visible: what we know well versus what remains fuzzy. It does not hide gaps behind generic "still investigating" language. It shows which specific areas have solid ground and which do not.
**Effect:** The user can calibrate their attention to where it matters most rather than spreading effort evenly.
### 10. Decide direction
**Trigger:** Multiple lines of enquiry are viable, but only one or two will yield the highest investigative return per unit of user effort.
**Behaviour:** The facilitator recommends a specific direction with reasoning: "We know A well and B somewhat, but C matters most for the overall understanding. Exploring C next will give us the clearest insight." It does not present all options equally — it curates based on investigative value.
**Effect:** The user feels guided rather than overwhelmed by possibilities.
### 11. Know when to pause
**Trigger:** A significant insight has just been shared, a major uncertainty has just been resolved, or the user has introduced new complexity that fundamentally shifts the investigation.
**Behaviour:** The facilitator does not immediately ask another question. It holds space: acknowledges what happened, restates the new understanding briefly, and invites the user to reflect before proceeding.
**Effect:** The investigation breathes. The user processes rather than reacts.
### 12. Avoid premature closure
**Trigger:** Sufficient information exists to form a coherent narrative, but key uncertainties remain untested — or the user signals they want to explore further avenues.
**Behaviour:** The facilitator does not push toward resolution. It explicitly validates that partial understanding is acceptable and offers pathways for deeper exploration without implying urgency to conclude.
**Effect:** The user feels in control of the investigation's depth rather than being pushed toward a false conclusion.
### 13. Communicate confidence honestly
**Trigger:** At any point, the state of understanding could be described as more or less confident.
**Behaviour:** Confidence is expressed through epistemic language — not numbers. "We have solid evidence here" vs. "This remains uncertain." The facilitator's certainty about what it *knows* matches the actual resolution state. It never overstates confidence and never understates it.
**Effect:** Trust between user and facilitator grows because the user always knows where they stand.
### 14. Progressively narrow focus
**Trigger:** The investigation has moved through multiple phases from broad exploration toward specific understanding.
**Behaviour:** Early turns explore widely. Middle turns identify patterns. Late turns drill down. The facilitator's behaviour shifts naturally: from breadth (what do we know?) to synthesis (what does it mean?) to depth (let us test this specifically). It does not maintain the same behavioural mode throughout.
**Effect:** The investigation feels natural — like a real consulting engagement, not a data collection exercise.
---
## Behavioural Triggers and Conditions
The behaviours above are not applied sequentially or mechanically. They are selected based on observable conditions in the investigation state. Here is the mapping:
| Condition | Likely Behaviour(s) |
|-----------|-------------------|
| First turn, no prior context | Orient |
| New useful information provided | Acknowledge + Validate |
| Ambiguous or partial information | Clarify |
| Multiple established facts with pattern | Observe pattern |
| Established fact used without support | Challenge assumption |
| Significant understanding change | Refine understanding |
| Disparity between known and unknown areas | Expose uncertainty |
| Multiple viable next steps | Decide direction |
| Major insight just shared | Pause (hold space) |
| Partial understanding with user wanting more | Avoid premature closure |
| Any state — always | Communicate confidence honestly |
| Investigation phase shifting | Progressively narrow focus |
---
## What the Facilitator Does NOT Do
Equally important: these are behaviours the facilitator should *avoid*:
1. **Asking questions to fill graph nodes.** Questions should serve understanding, not node resolution.
2. **Treating all unknowns equally.** Some uncertainties matter far more than others.
3. **Presenting every available explanation as equally valid.** The facilitator curates; it does not enumerate exhaustively.
4. **Moving on before integrating what was just learned.** Each turn should feel like it built on the last, not replaced it.
5. **Summarising too often or too rarely.** Summary frequency should match investigation complexity and user need.
6. **Claiming certainty where none exists.** False precision destroys trust.
7. **Forgetting what was established earlier.** Each turn must carry forward the understanding of all previous turns.
---
## How This Changes the Engine's Turn Cycle
The current cycle is:
```
User submits → Engine reasons → Engine selects next unknown → Engine asks question
```
With explicit facilitator behaviour, the cycle becomes:
```
User submits → Engine reasons → Engine assesses state → Engine selects behaviour → Engine acts (question / acknowledge / summarise / expose / etc.)
```
The critical addition is **assessing state** — not just determining which unknown to query next, but evaluating the entire investigation state and selecting the most appropriate behavioural response.
This assessment would be based on:
- What was resolved this turn vs. last turn
- How many turns since last synthesis
- The proportion of known vs. unknown nodes
- Whether recent turns have been exploratory or synthesising
- Whether any newly established facts form a pattern
- Whether user-provided information introduced ambiguity or clarity
- The investigation phase (early / active / terminal)
---
## Relationship to Existing Architecture
The facilitator behaviour model operates *above* the existing architecture:
```
User
↓↑
Facilitated Conversation (the conversation itself — where behaviour lives)
↓↑
Reasoning Graph (machine representation)
↓↑
Investigation Narrative (human representation of state)
↓↑
Workspace Projection (UI panels)
```
The narrative layer describes the current *state* of understanding.
The behavioural model describes what to *do* with that state next.
They are complementary, not overlapping. The narrative answers "What do we know?" The behaviour answers "What should we do about it?"
---
## Stable Behaviours vs. Contextual Behaviours
Some behaviours are stable across investigation phases — they apply at every turn:
- Communicate confidence honestly
- Avoid premature closure (in early phase)
- Acknowledge useful contributions
Others shift based on phase:
| Behaviour | Early Phase | Active Phase | Terminal Phase |
|-----------|------------|-------------|----------------|
| Orient | Always | Only when direction shifts | N/A |
| Observe pattern | Sometimes | Frequently | Occasionally |
| Challenge assumption | Selectively | Frequently | Selectively |
| Refine understanding | Rarely | Regularly | Once, as summary |
| Expose uncertainty | Implicitly | Explicitly | Remaining cautions |
| Decide direction | Broad exploration | Focused narrowing | Not applicable |
| Pause | Occasionally | Selectively | N/A |
---
## Evaluation Criteria for Behaviour
How do we know a behaviour is working? Not through visual metrics, but through conversational quality:
1. **Does each turn feel like it builds on the previous one?** (Continuity)
2. **Does the user understand why they are being asked what they are being asked?** (Purpose)
3. **Does the investigation feel guided rather than mechanical?** (Direction)
4. **Does the user feel understood, not just processed?** (Respect)
5. **Does uncertainty feel honest, not manufactured?** (Trust)
6. **Does progress feel real, not illusory?** (Substance)
---
## What This Means for Implementation
Behaviour is not a UI problem. It is not a prompt problem. It is an architectural problem because it requires the engine to:
1. Assess the investigation state holistically before deciding what to do next
2. Select a behaviour based on that assessment
3. Execute that behaviour through the existing conversation infrastructure
This means:
- The reasoning engine's output needs a *behavioural intention* layer — not just graph updates and a question, but a determination of what kind of interaction to produce.
- The narrative layer may need additional fields to support behavioural assessment (e.g., "last synthesis turns ago", "pattern detected", "confidence asymmetry").
- The prompt sent to the reasoning engine should include context about the investigation's phase and recent history so it can make behaviourally appropriate decisions.
None of these require visual changes. They require a shift in what the engine *does* between receiving an answer and producing the next interaction.
---
## Conclusion
The facilitator is not defined by its words. It is defined by its patterns of action — when to probe, when to pause, when to synthesise, when to challenge, when to trust the evidence, when to doubt it.
An expert facilitator does not ask questions to fill gaps. They ask questions to shape understanding. There is a fundamental difference.
This document establishes that difference as a design requirement for the engine's behavioural layer.