exp(18): implement investigation state assessment layer
Implement the three-dimensional assessment (phase, progress, conversation health) that sits between narrative and behaviour selection. Key changes: - lib/assessment/investigation-state-assessor.js: assessor module with countObservations, assessPhase, assessProgress, assessConversationHealth, assessInvestigationState — deterministic classifiers using known rules - tests/investigation-state-assessor.test.js: 51 tests covering phase classification (orienting→concluding), progress thresholds, health conditions, confidence aggregation, edge cases, and observation counting - lib/graph/orchestrator.js: integration calls passing correctly-shaped input to assessInvestigationState() at three call sites (~552, ~904, ~1013) Design decisions encoded in this iteration: - countObservations counts nodes with known/resolved status + high-confidence non-unknown non-state nodes (not just explicit observation-kind nodes) - Phase uses seven values including cannot_determine for insufficient data - Progress uses resolution ratio thresholds: accelerating (>0.6), steady (0.2-0.6), stalled (<0.2 with ≥1 resolved) - Overall confidence = minimum across all three dimensions (conservative) Also adds investigation-state-assessment-contract.md and updates design-evolution-log, investigation-state-assessment.md (status header), and investigation-turn-cycle.md (implementation status table).
This commit is contained in:
@@ -892,13 +892,57 @@ This architecture emerged from observation, not top-down design. It may still ch
|
||||
|
||||
A complete investigation can be described as a repeating turn cycle in which every architectural layer has a single responsibility.
|
||||
|
||||
Status
|
||||
Result
|
||||
|
||||
Architectural.
|
||||
Experiment validated that the investigation turn cycle is an *observation* about how existing layers interact rather than a new architectural layer. All eight stages (User Observation → Reasoning Graph → Investigation Narrative → State Assessment → Behaviour Selection → Conversation → Workspace → Wait) are supported by current architecture components, but only Stages 1–3 and 7 have working implementations. Stage 4 (State Assessment) and Stage 5 (Behaviour Selection) remain as architectural specifications without executable code.
|
||||
|
||||
Evaluation
|
||||
What did we learn?
|
||||
|
||||
Pending.
|
||||
- The turn cycle confirms that assessment sits between narrative and behaviour selection, not after the graph directly.
|
||||
- Every layer has one responsibility: each stage's purpose maps to an existing or specified component without overlap.
|
||||
- The cycle is deterministic in structure but adaptive in content — this is correct because the *sequence* of operations must be fixed while the *outputs* vary with investigation state.
|
||||
- Without a working Stage 4, all downstream stages (behaviour selection, conversation, workspace projection) operate on incomplete input. Phase 5 needs an executable assessment before behaviour can be validated experimentally.
|
||||
|
||||
Decision
|
||||
|
||||
The turn cycle architecture is confirmed as correct but requires implementation of Stage 4 (State Assessment) to move from observation to validation. The next step is the first deterministic evaluation function — not behaviour selection, which depends on assessment output. This becomes Experiment 18: First Executable Slice.
|
||||
|
||||
---
|
||||
|
||||
### Experiment 18 — First Executable Slice (Investigation State Assessment)
|
||||
|
||||
#### Hypothesis
|
||||
|
||||
A deterministic, conservative assessment of investigation phase and progress can be built from existing graph data without introducing new signals or modifying reasoning logic. The assessment should prefer `cannot_determine` over invented precision.
|
||||
|
||||
#### Scope
|
||||
|
||||
Phase detection (orienting / exploring / focusing / deepening / synthesising / concluding / cannot_determine), progress tracking (accelerating / steady / stalled / looping / spiralling / cannot_determine), and conversation health evaluation — using only data already present in the graph schema, orchestrator diagnostics, and facilitator-view outputs.
|
||||
|
||||
#### Constrained By
|
||||
|
||||
- Must use actual repo contracts (not assumptions about field names or structures).
|
||||
- Must be pure function — no network, LLM, mutation, or side effects.
|
||||
- Must handle missing fields gracefully — safe with absent data.
|
||||
- Must produce versioned assessment objects for future compatibility.
|
||||
- Passive integration only: add to diagnostics without changing public API or user-visible behaviour.
|
||||
|
||||
#### Questions
|
||||
|
||||
1. Can phase be reliably classified from node composition (kind/status ratio) alone?
|
||||
2. Does progress detection require turn history, or is a single-snapshot approximation sufficient for this first slice?
|
||||
3. What minimal conversation health signals can be extracted from existing graph metadata?
|
||||
|
||||
#### Evaluation
|
||||
|
||||
- Deterministic output across identical inputs.
|
||||
- Correct `cannot_determine` when data is insufficient (no false precision).
|
||||
- Handles all 11 mock scenarios at their turn points plus at least one live Ollama-shaped state.
|
||||
- Unsupported signals explicitly recorded in reasoning-contract-backlog.md.
|
||||
|
||||
#### Status
|
||||
|
||||
Completed — see `investigation-state-assessment-contract.md` and `lib/assessment/investigation-state-assessor.js`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user