experiment: compare evidence and condition scope

2026-08-06 11:36:42 +01:00
parent da291c715b
commit 1a9a9a94fe
3 changed files with 586 additions and 0 deletions
@@ -1375,6 +1375,74 @@ No changes to the active reasoning loop, prompt generation, or question-selectio

 ---

+### Experiment 25A — Evidence-Condition Scope Comparison
+
+**Status:** Completed (passive layer)
+
+#### Hypothesis
+
+Before evidence can support or contradict a condition, the engine must establish that both refer to the same:
+
+- subject;
+- timeframe;
+- type of claim.
+
+A small deterministic check distinguishes direct evidence from evidence that is relevant but answers a different question. Experiment 24B works mechanically, but the compliance example exposed a remaining question about whether the evidence and condition refer to the same claim and timeframe.
+
+#### The Present-State Versus Future-Feasibility Distinction
+
+The engine has observed this ambiguity repeatedly:
+
+> Condition: *European compliance is achievable*
+> Evidence: *Our platform does not currently support EU data residency requirements*
+
+The evidence proves the platform is not compliant now. It does not prove that compliance cannot be achieved. Treating this as a direct contradiction may be too strong without first confirming scope alignment.
+
+#### Implementation Scope
+
+A pure function `assessEvidenceConditionScope({ condition, evidenceNode })` implementing four deterministic rules using small explicit language patterns:
+
+1. **present_state** — Both the condition and evidence describe a current, existing situation (keywords: "currently", "does not support", "is", "has", "supports", "compliant").
+2. **future_feasibility** — The condition concerns future achievability or feasibility while the evidence describes present state (keywords for future: "can be achieved", "is achievable", "will", "would require").
+3. **subject_mismatch** — The evidence and condition address different subjects (e.g., compliance vs market demand). Detected via shared category from evidence-direction concept groups.
+4. **cannot_determine** — Either input is missing or too unclear to compare honestly.
+
+No LLM calls, no scoring, no weights, no graph schema changes, no mutation.
+
+#### Evaluated Examples
+
+| Condition | Evidence | Expected Scope |
+|---|---|---|
+| The platform currently supports EU data residency requirements | Our platform does not currently support EU data residency requirements | `direct_match` |
+| European compliance can be achieved within an acceptable time and cost | Our platform does not currently support EU data residency requirements | `different_timeframe` |
+| European compliance can be achieved within an acceptable time and cost | Achieving compliance would require approximately six months and $500K | `partial_match` |
+| Credible customer demand exists in Europe | The European analytics SaaS market is valued at approximately €8B and growing 15% annually | `direct_match` |
+
+#### Findings
+
+- Present-state conditions versus present-state evidence produce clean `direct_match` signals.
+- Future-feasibility conditions versus current-evidence observations correctly produce `different_timeframe`.
+- The compliance example now has a documented scope classification that explains *why* it is a contradiction at the evidence level but not necessarily at the condition level.
+- Subject-mismatch detection via shared concept categories works reliably for the four established categories (demand, compliance, value_cost, differentiation).
+
+#### Phrase list additions
+
+The future-feasibility phrase list was extended from `"can be achieved"` to also include `"can achieve"` and `"be achieved"`. This addresses a case where present-state evidence ("Our team currently has no EU regulatory expertise") and future-feasibility conditions ("We can achieve European compliance within 12 months") must be recognised as referring to different timeframes even though the condition uses "can achieve" rather than "can be achieved".
+
+#### Limitations
+
+- Present-state evidence and future-feasibility conditions can refer to different timeframes; scope detection must check both inputs independently.
+- Timeframe detection relies on explicit keyword patterns. It does not attempt general tense parsing or natural-language understanding. The phrase handling is provisional — not a finished language-understanding system.
+- Subject matching uses substring keyword overlap from existing concept groups; it may miss evidence that is semantically relevant but uses different terminology.
+- `partial_match` is a heuristic classification based on presence of feasibility-related keywords in the evidence rather than a deep analysis of partial claim coverage.
+- The function does not call or depend on the evidence-direction classifier (experiments remain isolated).
+
+#### Passive Status
+
+This experiment remains passive and isolated. It does not modify decision-condition-status.js, evidence-direction.js, graph schema, prompts, APIs, UI, or any active engine behaviour. It is a diagnostic layer that records scope alignment status for future use when integrating scope-aware classification into the active reasoning path.
+
+---
+
 ## Current Open Questions

 The following are active explorations rather than decisions.