experiment: assess decision condition status

This commit is contained in:
2026-08-06 09:04:06 +01:00
parent 32aa3f237a
commit 3119635211
4 changed files with 776 additions and 0 deletions
+60
View File
@@ -1215,6 +1215,66 @@ The classifier remains passive and is not in the active reasoning path.
---
## Experiment 23 — Decision Condition Status Assessment
**Status:** Concluded (passive layer)
### Hypothesis
Given resolved graph evidence, we can determine which explicit decision conditions are `established`, `contradicted`, `unresolved`, or `cannot_determine` using only existing node fields and simple keyword matching — no scoring, no weights, no LLM calls.
### Scope
- Pure passive classifier: reads `resolvedNodeIds`, `nodes[].label`, `nodes[].description`, `nodes[].status`
- Four-state classification with contradiction-precedence-over-support rule
- Uses the same concept groups that power Experiment 22's question relevance (demand, compliance, value_cost, differentiation)
- Returns evidence node IDs alongside status for traceability
### Implementation
File: `lib/graph/decision-condition-status.js`
Classification rules (evaluated in order):
1. **cannot_determine** — missing condition text or incomplete graph
2. **contradicted** — resolved evidence contains a contradiction phrase (e.g. "does not support", "not achievable")
3. **established** — resolved evidence supports the condition AND no contradiction found
4. **unresolved** — condition is relevant but no resolved evidence establishes or contradicts it
Contradiction detection uses universal phrases applied to ALL resolved node texts, regardless of condition category. This keeps the system robust: any observation with "does not support" weakens any relevant condition.
Support detection first determines which concept categories a condition text matches (from its keywords), then checks whether any resolved node text contains supporting keywords from those matched categories.
### Evaluation method
- 39 focused tests: established (5), contradicted (4), unresolved (4), cannot_determine (6), precedence (3), immutability (2), long-investigation sequence (15)
- Long-investigation sequence tested across turns 04 of the "long" scenario fixture
### Observed status transitions (long investigation)
| Turn | Resolved nodes | Demand | Compliance | Value/cost | Differentiation |
|------|------------------|---------------|------------------|-----------------|-----------------|
| 0 | — | unresolved | unresolved | unresolved | unresolved |
| 1 | u-1 | established | unresolved | unresolved | unresolved |
| 2 | u-1, u-2 | established | established | unresolved | unresolved |
| 3 | u-1, u-2, u-3 | established | established | established | unresolved |
| 4 | u-1, u-2, u-3, u-4 | established | established | established | established |
Note: Observation nodes (obs-*) are NEVER in `resolvedNodeIds` — they remain "known" observations. Only unknowns become resolved during investigation turns. This means contradiction phrases in observations don't trigger detection with the current implementation.
### Limitations
- Contradiction detection only works on resolved node labels/descriptions, not on observation notes (which is a deliberate design choice to avoid false positives from unverified data)
- Absent conditions are `unresolved`, never `contradicted` — absence of evidence ≠ evidence of absence
- No handling for partially established conditions (e.g. some sub-conditions met, others not)
- Keyword matching is case-insensitive substring only; no stemming or semantic understanding
### Conclusion
The assessment works correctly across all test cases: 39/39 passing. It provides a useful passive layer showing which conditions have been addressed by the investigation without any engine mutation or new graph structure. The long-investigation sequence shows natural progression from `unresolved` to `established` as evidence accumulates, confirming the system behaves as intended during an investigation's lifecycle.
---
## Current Open Questions
The following are active explorations rather than decisions.