experiment: assess decision condition status
This commit is contained in:
@@ -1215,6 +1215,66 @@ The classifier remains passive and is not in the active reasoning path.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 23 — Decision Condition Status Assessment
|
||||
|
||||
**Status:** Concluded (passive layer)
|
||||
|
||||
### Hypothesis
|
||||
|
||||
Given resolved graph evidence, we can determine which explicit decision conditions are `established`, `contradicted`, `unresolved`, or `cannot_determine` using only existing node fields and simple keyword matching — no scoring, no weights, no LLM calls.
|
||||
|
||||
### Scope
|
||||
|
||||
- Pure passive classifier: reads `resolvedNodeIds`, `nodes[].label`, `nodes[].description`, `nodes[].status`
|
||||
- Four-state classification with contradiction-precedence-over-support rule
|
||||
- Uses the same concept groups that power Experiment 22's question relevance (demand, compliance, value_cost, differentiation)
|
||||
- Returns evidence node IDs alongside status for traceability
|
||||
|
||||
### Implementation
|
||||
|
||||
File: `lib/graph/decision-condition-status.js`
|
||||
|
||||
Classification rules (evaluated in order):
|
||||
|
||||
1. **cannot_determine** — missing condition text or incomplete graph
|
||||
2. **contradicted** — resolved evidence contains a contradiction phrase (e.g. "does not support", "not achievable")
|
||||
3. **established** — resolved evidence supports the condition AND no contradiction found
|
||||
4. **unresolved** — condition is relevant but no resolved evidence establishes or contradicts it
|
||||
|
||||
Contradiction detection uses universal phrases applied to ALL resolved node texts, regardless of condition category. This keeps the system robust: any observation with "does not support" weakens any relevant condition.
|
||||
|
||||
Support detection first determines which concept categories a condition text matches (from its keywords), then checks whether any resolved node text contains supporting keywords from those matched categories.
|
||||
|
||||
### Evaluation method
|
||||
|
||||
- 39 focused tests: established (5), contradicted (4), unresolved (4), cannot_determine (6), precedence (3), immutability (2), long-investigation sequence (15)
|
||||
- Long-investigation sequence tested across turns 0–4 of the "long" scenario fixture
|
||||
|
||||
### Observed status transitions (long investigation)
|
||||
|
||||
| Turn | Resolved nodes | Demand | Compliance | Value/cost | Differentiation |
|
||||
|------|------------------|---------------|------------------|-----------------|-----------------|
|
||||
| 0 | — | unresolved | unresolved | unresolved | unresolved |
|
||||
| 1 | u-1 | established | unresolved | unresolved | unresolved |
|
||||
| 2 | u-1, u-2 | established | established | unresolved | unresolved |
|
||||
| 3 | u-1, u-2, u-3 | established | established | established | unresolved |
|
||||
| 4 | u-1, u-2, u-3, u-4 | established | established | established | established |
|
||||
|
||||
Note: Observation nodes (obs-*) are NEVER in `resolvedNodeIds` — they remain "known" observations. Only unknowns become resolved during investigation turns. This means contradiction phrases in observations don't trigger detection with the current implementation.
|
||||
|
||||
### Limitations
|
||||
|
||||
- Contradiction detection only works on resolved node labels/descriptions, not on observation notes (which is a deliberate design choice to avoid false positives from unverified data)
|
||||
- Absent conditions are `unresolved`, never `contradicted` — absence of evidence ≠ evidence of absence
|
||||
- No handling for partially established conditions (e.g. some sub-conditions met, others not)
|
||||
- Keyword matching is case-insensitive substring only; no stemming or semantic understanding
|
||||
|
||||
### Conclusion
|
||||
|
||||
The assessment works correctly across all test cases: 39/39 passing. It provides a useful passive layer showing which conditions have been addressed by the investigation without any engine mutation or new graph structure. The long-investigation sequence shows natural progression from `unresolved` to `established` as evidence accumulates, confirming the system behaves as intended during an investigation's lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## Current Open Questions
|
||||
|
||||
The following are active explorations rather than decisions.
|
||||
|
||||
Reference in New Issue
Block a user