Feature/product platform foundation v0.62 #1

Merged
robbond merged 683 commits from feature/product-platform-foundation-v0.62 into feature/emergent-unknowns-v0.5 2026-09-09 07:58:20 +01:00
3 changed files with 586 additions and 0 deletions
Showing only changes of commit 1a9a9a94fe - Show all commits
+68
View File
@@ -1375,6 +1375,74 @@ No changes to the active reasoning loop, prompt generation, or question-selectio
---
### Experiment 25A — Evidence-Condition Scope Comparison
**Status:** Completed (passive layer)
#### Hypothesis
Before evidence can support or contradict a condition, the engine must establish that both refer to the same:
- subject;
- timeframe;
- type of claim.
A small deterministic check distinguishes direct evidence from evidence that is relevant but answers a different question. Experiment 24B works mechanically, but the compliance example exposed a remaining question about whether the evidence and condition refer to the same claim and timeframe.
#### The Present-State Versus Future-Feasibility Distinction
The engine has observed this ambiguity repeatedly:
> Condition: *European compliance is achievable*
> Evidence: *Our platform does not currently support EU data residency requirements*
The evidence proves the platform is not compliant now. It does not prove that compliance cannot be achieved. Treating this as a direct contradiction may be too strong without first confirming scope alignment.
#### Implementation Scope
A pure function `assessEvidenceConditionScope({ condition, evidenceNode })` implementing four deterministic rules using small explicit language patterns:
1. **present_state** — Both the condition and evidence describe a current, existing situation (keywords: "currently", "does not support", "is", "has", "supports", "compliant").
2. **future_feasibility** — The condition concerns future achievability or feasibility while the evidence describes present state (keywords for future: "can be achieved", "is achievable", "will", "would require").
3. **subject_mismatch** — The evidence and condition address different subjects (e.g., compliance vs market demand). Detected via shared category from evidence-direction concept groups.
4. **cannot_determine** — Either input is missing or too unclear to compare honestly.
No LLM calls, no scoring, no weights, no graph schema changes, no mutation.
#### Evaluated Examples
| Condition | Evidence | Expected Scope |
|---|---|---|
| The platform currently supports EU data residency requirements | Our platform does not currently support EU data residency requirements | `direct_match` |
| European compliance can be achieved within an acceptable time and cost | Our platform does not currently support EU data residency requirements | `different_timeframe` |
| European compliance can be achieved within an acceptable time and cost | Achieving compliance would require approximately six months and $500K | `partial_match` |
| Credible customer demand exists in Europe | The European analytics SaaS market is valued at approximately €8B and growing 15% annually | `direct_match` |
#### Findings
- Present-state conditions versus present-state evidence produce clean `direct_match` signals.
- Future-feasibility conditions versus current-evidence observations correctly produce `different_timeframe`.
- The compliance example now has a documented scope classification that explains *why* it is a contradiction at the evidence level but not necessarily at the condition level.
- Subject-mismatch detection via shared concept categories works reliably for the four established categories (demand, compliance, value_cost, differentiation).
#### Phrase list additions
The future-feasibility phrase list was extended from `"can be achieved"` to also include `"can achieve"` and `"be achieved"`. This addresses a case where present-state evidence ("Our team currently has no EU regulatory expertise") and future-feasibility conditions ("We can achieve European compliance within 12 months") must be recognised as referring to different timeframes even though the condition uses "can achieve" rather than "can be achieved".
#### Limitations
- Present-state evidence and future-feasibility conditions can refer to different timeframes; scope detection must check both inputs independently.
- Timeframe detection relies on explicit keyword patterns. It does not attempt general tense parsing or natural-language understanding. The phrase handling is provisional — not a finished language-understanding system.
- Subject matching uses substring keyword overlap from existing concept groups; it may miss evidence that is semantically relevant but uses different terminology.
- `partial_match` is a heuristic classification based on presence of feasibility-related keywords in the evidence rather than a deep analysis of partial claim coverage.
- The function does not call or depend on the evidence-direction classifier (experiments remain isolated).
#### Passive Status
This experiment remains passive and isolated. It does not modify decision-condition-status.js, evidence-direction.js, graph schema, prompts, APIs, UI, or any active engine behaviour. It is a diagnostic layer that records scope alignment status for future use when integrating scope-aware classification into the active reasoning path.
---
## Current Open Questions
The following are active explorations rather than decisions.
+148
View File
@@ -0,0 +1,148 @@
/**
* Experiment 25A — Evidence-Condition Scope Comparison.
*
* Determines whether a piece of evidence and a decision condition refer to
* the same claim and timeframe before direction classification is applied.
*
* Returns one of:
* "direct_match" — same subject, present state in both
* "partial_match" — relevant but only addresses part of the condition
* "different_timeframe" — present evidence vs future feasibility (or vice versa)
* "unrelated" — different subjects entirely
* "cannot_determine" — missing or unclear input
*
* Uses four deterministic rules. No LLM calls, no scoring, no mutation.
*/
/* ── Shared concept groups (subset of evidence-direction.js categories) ── */
/* ── Normalisation ─────────────────────────────────────────── */
function normalise(value) {
return String(value || "").toLowerCase();
}
/* ── Present-state detection (condition) ──────────────────── */
const FUTURE_FEASIBILITY_PHRASES = [
"can be achieved", "can achieve", "would require", "will achieve",
"could achieve", "able to achieve", "be achieved", "feasible",
"worth the cost",
];
function isPresentState(text) {
/* Default: anything without future-feasibility markers is present-state */
return !isFutureFeasibility(text);
}
function isFutureFeasibility(text) {
return FUTURE_FEASIBILITY_PHRASES.some((p) => text.includes(p));
}
/* ── Concept family matching ──────────────────────────────── */
/**
* Each concept has two keyword lists:
* core — words that directly identify this concept (e.g. "demand", "compliance").
* related — words that typically co-occur with the concept in evidence text.
* A category is "shared" when the condition contains a core word AND the evidence
* contains either the same core word OR any related word from that concept family.
*/
const CONCEPT_FAMILIES = {
demand: {
core: ["demand", "need", "customer demand", "audience"],
related: ["market exists", "valued at", "growing market", "interest", "market size", "growing"],
},
compliance: {
core: ["compliance", "gdpr", "regulation", "data residency"],
related: ["eu compliance", "achieve compliance", "supports gdpr", "meets regulation", "supports data residency", "support eu"],
},
value_cost: {
core: ["cost", "investment", "viability", "financial viability"],
related: ["justifies the cost", "market value", "engineering investment", "return", "worth the cost", "affordable"],
},
differentiation: {
core: ["competitive differentiation", "unique feature", "differentiat"],
related: ["competitive advantage", "positioning", "no direct equivalent", "unique product", "competit"],
},
};
function findSharedCategories(condText, evText) {
const shared = [];
for (const [category, family] of Object.entries(CONCEPT_FAMILIES)) {
/* Condition must contain a core word for this category */
const condMatchesCore = family.core.some((kw) => condText.includes(kw));
if (!condMatchesCore) continue;
/* Evidence matches if it has either the same core word or any related word */
const evMatches = [...family.core, ...family.related].some((kw) => evText.includes(kw));
if (evMatches) {
shared.push(category);
}
}
return shared;
}
/* ── Core function ───────────────────────────────────────── */
/**
* Assess the scope alignment between a decision condition and evidence.
*
* @param {{ condition: object, evidenceNode: object }} input
* @returns {{ scope: "direct_match" | "partial_match" | "different_timeframe" | "unrelated" | "cannot_determine", reason: string }}
*/
export function assessEvidenceConditionScope({ condition, evidenceNode } = {}) {
if (!condition) return { scope: "cannot_determine", reason: "missing or null condition" };
if (!evidenceNode) return { scope: "cannot_determine", reason: "missing or null evidence node" };
const conditionText = normalise(condition.text ?? condition.label ?? "");
const evidenceText = normalise(evidenceNode.description ?? evidenceNode.text ?? evidenceNode.label ?? "");
if (conditionText.length === 0) return { scope: "cannot_determine", reason: "empty condition text" };
if (evidenceText.length === 0) return { scope: "cannot_determine", reason: "empty evidence node text" };
/* Rule 1 — Timeframe mismatch (applies across all subjects) */
const condIsPresent = isPresentState(conditionText);
const condIsFuture = isFutureFeasibility(conditionText);
const evIsPresent = isPresentState(evidenceText);
const evIsFuture = isFutureFeasibility(evidenceText);
if ((condIsPresent && evIsFuture) || (condIsFuture && evIsPresent)) {
return { scope: "different_timeframe", reason: "evidence describes present state while condition concerns future feasibility" };
}
/* Rule 2 — Shared category check */
const shared = findSharedCategories(conditionText, evidenceText);
if (shared.length === 0) {
/* Feasibility evidence without shared subject — both are feasibility-oriented */
if (condIsFuture && evIsFuture) {
return { scope: "partial_match", reason: "both express future-feasibility but address different subjects" };
}
if (condIsFuture || evIsFuture) {
return { scope: "different_timeframe", reason: "evidence describes present state while condition concerns future feasibility" };
}
return { scope: "unrelated", reason: "condition and evidence do not share a recognisable concept category" };
}
/* Rule 3 — Both present-state → direct match */
if (condIsPresent && evIsPresent) {
return { scope: "direct_match", reason: `both describe present state in ${shared.join(" / ")} category` };
}
/* Rule 4 — Future condition with feasibility evidence → partial match */
if ((condIsFuture || evIsFuture)) {
return { scope: "partial_match", reason: `evidence addresses feasibility for ${shared.join(" / ")} but does not fully answer the condition` };
}
/* Fallback — cannot determine when neither present nor future detected */
return { scope: "cannot_determine", reason: "neither present-state nor future-feasibility patterns detected in both texts" };
}
@@ -0,0 +1,370 @@
/**
* Experiment 25A — Tests for assessEvidenceConditionScope.
*
* Validates scope classification (direct_match, partial_match,
* different_timeframe, unrelated, cannot_determine) against explicit
* conditions and evidence nodes.
*/
import { describe, expect, it } from "vitest";
import { assessEvidenceConditionScope } from "@/lib/graph/evidence-condition-scope.js";
import { buildScenarioFixture } from "@/lib/mocks/scenarios.js";
/* ── Helpers ─────────────────────────────────────────────── */
function makeNode(id, text) {
return { id, description: text, label: text };
}
const DEMAND_CONDITION = "Credible customer demand exists in Europe";
const COMPLIANCE_PRESENT = "The platform currently supports EU data residency requirements";
const COMPLIANCE_FUTURE = "European compliance can be achieved within an acceptable time and cost";
const VALUE_COST_CONDITION = "The expected market value justifies the cost of entry";
/* ── direct_match: present-state condition versus present evidence ─── */
describe("assessEvidenceConditionScope — direct_match", () => {
it("returns direct_match for identical compliance subject in present state", () => {
const evidence = makeNode(
"obs-3",
"Our platform does not currently support EU data residency requirements",
);
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_PRESENT },
evidenceNode: evidence,
});
expect(result.scope).toBe("direct_match");
});
it("returns direct_match for demand condition versus market evidence", () => {
const evidence = makeNode(
"obs-2",
"The European analytics SaaS market is valued at approximately €8B and growing 15% annually",
);
const result = assessEvidenceConditionScope({
condition: { text: DEMAND_CONDITION },
evidenceNode: evidence,
});
expect(result.scope).toBe("direct_match");
});
it("returns direct_match when both condition and evidence use 'currently'", () => {
const evidence = makeNode("t-1", "The platform already supports GDPR requirements and data residency");
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_PRESENT },
evidenceNode: evidence,
});
expect(result.scope).toBe("direct_match");
});
it("returns direct_match using the long-investigation compliance example", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs3Node = graphNodes.find((n) => n.id === "obs-3");
expect(obs3Node).toBeDefined();
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_PRESENT },
evidenceNode: obs3Node,
});
expect(result.scope).toBe("direct_match");
});
it("returns direct_match for value-cost present-state condition", () => {
const evidence = makeNode("v-1", "The total addressable market provides sufficient return on investment");
const result = assessEvidenceConditionScope({
condition: { text: "The platform currently offers sufficient financial viability" },
evidenceNode: evidence,
});
expect(result.scope).toBe("direct_match");
});
});
/* ── different_timeframe: future condition versus present evidence ─── */
describe("assessEvidenceConditionScope — different_timeframe", () => {
it("returns different_timeframe for compliance future condition versus current evidence", () => {
const evidence = makeNode(
"obs-3",
"Our platform does not currently support EU data residency requirements",
);
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_FUTURE },
evidenceNode: evidence,
});
expect(result.scope).toBe("different_timeframe");
});
it("returns different_timeframe when evidence describes the present and condition uses 'can be achieved'", () => {
const evidence = makeNode("t-2", "Our team currently has no EU regulatory expertise");
const result = assessEvidenceConditionScope({
condition: { text: "Can credible customer demand be achieved in Europe" },
evidenceNode: evidence,
});
expect(result.scope).toBe("different_timeframe");
});
it("returns different_timeframe for value_cost future condition versus current evidence", () => {
const evidence = makeNode("t-3", "The platform currently has no pricing data for Europe");
const result = assessEvidenceConditionScope({
condition: { text: "Can market value be achieved within acceptable cost" },
evidenceNode: evidence,
});
expect(result.scope).toBe("different_timeframe");
});
});
/* ── partial_match: feasibility evidence versus future condition ─── */
describe("assessEvidenceConditionScope — partial_match", () => {
it("returns partial_match for compliance feasibility evidence against future condition", () => {
const evidence = makeNode(
"obs-4",
"Achieving compliance would require approximately six months and $500K",
);
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_FUTURE },
evidenceNode: evidence,
});
expect(result.scope).toBe("partial_match");
});
it("returns partial_match using the long-investigation value/cost example", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs4Node = graphNodes.find((n) => n.id === "obs-4");
expect(obs4Node).toBeDefined();
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_FUTURE },
evidenceNode: obs4Node,
});
expect(result.scope).toBe("partial_match");
});
it("returns partial_match for demand feasibility question", () => {
const evidence = makeNode("t-4", "Entry would require approximately €2M marketing investment over three years");
const result = assessEvidenceConditionScope({
condition: { text: "Credible customer demand can be achieved within acceptable cost" },
evidenceNode: evidence,
});
expect(result.scope).toBe("partial_match");
});
});
/* ── unrelated: different subjects ─────────────────────── */
describe("assessEvidenceConditionScope — unrelated", () => {
it("returns unrelated when evidence is about market size but condition is about compliance", () => {
const evidence = makeNode(
"t-5",
"European demand for analytics tools is increasing rapidly",
);
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_PRESENT },
evidenceNode: evidence,
});
expect(result.scope).toBe("unrelated");
});
it("returns unrelated when both subjects are completely different", () => {
const evidence = makeNode("t-6", "The office lease expires in December 2026");
const result = assessEvidenceConditionScope({
condition: { text: DEMAND_CONDITION },
evidenceNode: evidence,
});
expect(result.scope).toBe("unrelated");
});
});
/* ── cannot_determine: missing or unclear input ──────────── */
describe("assessEvidenceConditionScope — cannot_determine", () => {
it("returns cannot_determine when condition is missing", () => {
const result = assessEvidenceConditionScope({ evidenceNode: makeNode("t-1", "test") });
expect(result.scope).toBe("cannot_determine");
});
it("returns cannot_determine when condition is null", () => {
const result = assessEvidenceConditionScope({ condition: null, evidenceNode: makeNode("t-1", "test") });
expect(result.scope).toBe("cannot_determine");
});
it("returns cannot_determine when evidenceNode is missing", () => {
const result = assessEvidenceConditionScope({ condition: { text: DEMAND_CONDITION } });
expect(result.scope).toBe("cannot_determine");
});
it("returns cannot_determine when evidenceNode is null", () => {
const result = assessEvidenceConditionScope({
condition: { text: DEMAND_CONDITION },
evidenceNode: null,
});
expect(result.scope).toBe("cannot_determine");
});
it("returns cannot_determine when condition has empty text", () => {
const result = assessEvidenceConditionScope({
condition: { text: "" },
evidenceNode: makeNode("t-1", "test"),
});
expect(result.scope).toBe("cannot_determine");
});
it("returns cannot_determine when evidenceNode has empty description", () => {
const result = assessEvidenceConditionScope({
condition: { text: DEMAND_CONDITION },
evidenceNode: makeNode("t-1", ""),
});
expect(result.scope).toBe("cannot_determine");
});
it("returns cannot_determine when neither input is provided", () => {
const result = assessEvidenceConditionScope({});
expect(result.scope).toBe("cannot_determine");
});
});
/* ── Deterministic output ───────────────────────────────── */
describe("assessEvidenceConditionScope — determinism", () => {
it("returns identical results for identical inputs across multiple calls", () => {
const evidence = makeNode("det-1", "European analytics market valued at €8B");
const input = {
condition: { text: DEMAND_CONDITION },
evidenceNode: evidence,
};
const r1 = assessEvidenceConditionScope(input);
const r2 = assessEvidenceConditionScope(input);
expect(r1).toEqual(r2);
});
it("deterministic across 5 calls with long-investigation data", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs3Node = graphNodes.find((n) => n.id === "obs-3");
const input = {
condition: { text: COMPLIANCE_PRESENT },
evidenceNode: obs3Node,
};
const results = [1, 2, 3, 4, 5].map(() => assessEvidenceConditionScope(input));
for (let i = 1; i < results.length; i++) {
expect(results[i]).toEqual(results[0]);
}
});
});
/* ── Input immutability ─────────────────────────────────── */
describe("assessEvidenceConditionScope — input immutability", () => {
it("does not mutate the condition object", () => {
const cond = { text: DEMAND_CONDITION };
const originalText = cond.text;
const evidence = makeNode("imm-1", "European market valued at €8B");
assessEvidenceConditionScope({ condition: cond, evidenceNode: evidence });
expect(cond.text).toBe(originalText);
});
it("does not mutate the evidenceNode object", () => {
const node = makeNode("imm-2", "test description");
const originalDesc = node.description;
const condition = { text: DEMAND_CONDITION };
assessEvidenceConditionScope({ condition, evidenceNode: node });
expect(node.description).toBe(originalDesc);
});
});
/* ── Four long-investigation examples from the brief ─────── */
describe("assessEvidenceConditionScope — required cases from long investigation", () => {
it("compliance direct present-state match (condition vs obs-3)", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs3Node = graphNodes.find((n) => n.id === "obs-3");
expect(obs3Node).toBeDefined();
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_PRESENT },
evidenceNode: obs3Node,
});
expect(result.scope).toBe("direct_match");
});
it("future feasibility versus current compliance evidence", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs3Node = graphNodes.find((n) => n.id === "obs-3");
expect(obs3Node).toBeDefined();
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_FUTURE },
evidenceNode: obs3Node,
});
expect(result.scope).toBe("different_timeframe");
});
it("future feasibility versus implementation evidence (partial_match)", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs4Node = graphNodes.find((n) => n.id === "obs-4");
expect(obs4Node).toBeDefined();
const result = assessEvidenceConditionScope({
condition: { text: COMPLIANCE_FUTURE },
evidenceNode: obs4Node,
});
expect(result.scope).toBe("partial_match");
});
it("demand direct_match (obs-2)", () => {
const fixture = buildScenarioFixture("long", 3);
const graphNodes = fixture.situationGraph.nodes;
const obs2Node = graphNodes.find((n) => n.id === "obs-2");
expect(obs2Node).toBeDefined();
const result = assessEvidenceConditionScope({
condition: { text: DEMAND_CONDITION },
evidenceNode: obs2Node,
});
expect(result.scope).toBe("direct_match");
});
});