Feature/product platform foundation v0.62 #1

Merged
robbond merged 683 commits from feature/product-platform-foundation-v0.62 into feature/emergent-unknowns-v0.5 2026-09-09 07:58:20 +01:00
3 changed files with 640 additions and 6 deletions
Showing only changes of commit 40ef3e108f - Show all commits
+10 -5
View File
@@ -48,14 +48,19 @@ Experiment 43 audited Clarify readiness across all 10 real assessment turns in e
Experiment 44 created one deliberately unclear starting scenario (five competing unknowns, zero resolved evidence, vague central statement) to test whether the assessor produces a Clarify-justifying signal. The assessor returned `too_broad` conversation health — confirming the previously untested too_broad path works correctly with real data. Clarify became eligible via Rule A. No production code changed. Remaining open: whether orienting phase is needed for earlier-stage clarification, and whether 23 competing threads (below the >3 threshold) can represent genuine scope confusion. Status pending Rob's review.
Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns — two and three returned cannot_determine; four and five returned too_broad. Clarify eligibility followed the same boundary. Resolved-item gate works correctly: one resolved item stays too_broad, two resolves it. The boundary appears mechanically clear but conceptually uncertain — synthetic fixtures cannot confirm whether three-to-four feels right to real users. No production code changed. What remains open: whether health should default to healthy (not cannot_determine) for 23 unknowns with no question; whether the threshold needs widening for real-world use. Status pending Rob's review.
Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns — two and three returned cannot_determine; four and five returned too_broad. Clarify eligibility followed the same boundary. Resolved-item gate works correctly: one resolved item stays too_broad, two resolves it. The boundary appears mechanically clear but conceptually uncertain — synthetic fixtures cannot confirm whether three-to-four feels right to real users. No production code changed. What remains open: whether health should default to healthy (not cannot_determine) for 23 unknowns with no question; whether the threshold needs widening for real-world use. Status closed.
Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. Status closed.
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.
## 5. What Remains Open
- The `too_broad` boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
- Health defaults to `cannot_determine` rather than `healthy` for 23 unknowns (no active question present); whether this is a bug or feature needs review;
- Whether the `too_broad` threshold needs widening so Clarify fires in more typical investigations;
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states.
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states;
- Existing-scenario graphs lack populated relationship fields on unknown nodes — coherence detection requires upstream data quality improvement (populating dependsOn/affects when adding unknowns).
### When This Knowledge-Management Phase Is Complete
@@ -101,8 +106,8 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 38, 39, 40, 41, 42, 43, 44, 45, 46. Branch: `feature/user-workspace-ux-v0.7`.*
*Created by Experiment 34. Updated by Experiments 3847. Branch: `feature/user-workspace-ux-v0.7`.*
### Return-to-Work Note (Experiment 45)
### Return-to-Work Note (Experiment 47)
Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. What remains open: whether a coherence signal should be added, and what threshold would feel right in real-user data. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/investigation-state-assessor.scope-coherence.test.js` for results, then `docs/design-evolution-log.md` Experiment 46 section for full analysis.
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing relationship fields (dependsOn, affects, parentId, childIds on nodes; fromNodeId/toNodeId + relationship on edges) to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/none/separate anchors, all with identical structural counts of 6 nodes and 4 active unknowns) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged (produces identical too_broad output across all fixtures). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/investigation-state-assessor.shared-anchor.test.js` for results, then `docs/design-evolution-log.md` Experiment 47 section for full analysis.
+101 -1
View File
@@ -3046,7 +3046,7 @@ Each unknown belongs to a separate domain of enquiry. A human reviewer would cla
### Status
Pending Rob's review. No production behaviour changed.
**Closed.** Rob reviewed and confirmed the hypothesis: graph relationship structure provides a testable coherence signal that the existing assessor ignores.
### Focused Test Results
@@ -3066,3 +3066,103 @@ Pending Rob's review. No production behaviour changed.
### Production Assessor Status
**Unchanged.** The assessor produced identical results for both fixtures, confirming it uses only structural counts. No code was modified.
## Experiment 47 — Shared-Anchor Coherence Diagnostic (2026-08-06)
### Objective
Test whether existing graph relationships (`dependsOn`, `affects`, `parentId`, `childIds` on nodes; `fromNodeId`/`toNodeId` + `relationship` on edges) can distinguish coherent investigations (multiple unknowns sharing one anchor) from scattered investigations (multiple unknowns with separate anchors). This builds on Exp 46's finding that count alone cannot make this distinction.
This is a passive diagnostic experiment. No production code changes.
### Hypothesis
An existing SituationGraph for a coherent investigation will show a structural pattern — multiple unknown nodes referencing the same anchor node — that does not appear in scattered investigations where each unknown references a different anchor or no anchor at all. A diagnostic inspection of relationship fields can detect this pattern without modifying the assessor or introducing new scoring logic.
### Context Pack Used
Engine Experiment Work pack (Pack 1). Documents loaded:
- `docs/current-project-state.md`, `docs/current-working-principles.md`, `.claude/architecture-guardrails.md`, `docs/current-implementation-verification.md`
- `lib/assessment/investigation-state-assessor.js` (to verify assessor output)
- `tests/investigation-state-assessor.scope-coherence.test.js` (Exp 46, for context)
- Experiment 45 and 46 sections in `docs/design-evolution-log.md`
No additional documents loaded.
### Three Controlled Fixtures
| Property | Fixture A (shared) | Fixture B (separate) | Fixture C (none) |
|---|---|---|---|
| Nodes | 6 (1 obs + 1 ctx + 4 unk) | 6 (1 obs + 1 ctx + 4 unk) | 6 (1 obs + 1 ctx + 4 unk) |
| Edges | 5 | 1 | 0 |
| Active unknowns | 4 | 4 | 4 |
| Resolved | 0 | 0 | 0 |
| Observations | 1 | 1 | 1 |
| Relationship pattern | All unknowns reference ctx-1 | Each unknown references ctx-1 differently (or not at all) | No relationship fields populated |
| Diagnostic result | `shared_anchor` → [ctx-1] | `separate_anchors` → [ctx-1] | `insufficient_data` → [] |
### Relationship Fields Inspected by the Diagnostic Helper
The test-only helper `inspectSharedUnknownAnchor` inspects:
1. **`dependsOn`** on unknown nodes — direct dependency to an anchor
2. **`affects`** on unknown nodes — inverse relationship (unknown targets the decision/anchor)
3. **`parentId`** on unknown nodes — hierarchical parent reference
4. **`childIds`** on existing nodes — inverse child reference from anchor side
5. **Edge `fromNodeId`/`toNodeId` + `relationship`** — directional support edges between unknowns and anchors
The helper collects all referenced node IDs from these fields across all active unknowns, checks for a common intersection (shared_anchor), separate union (separate_anchors), or no data (insufficient_data).
### Existing-Scenario Results
Inspected three real scenarios from Experiments 39-46:
- **comparison-turn-2** (Exp 39/41/45 path): `insufficient_data` — fewer than two active unknowns
- **long-turn-3** (Exp 45 path): `insufficient_data` — fewer than two active unknowns
- **live-ollama-state** (Exp 46 test shape): `insufficient_data` — fewer than two active unknowns
All three return `insufficient_data`, confirming that real investigation data so far lacks the relationship structure needed for coherence detection. The diagnostic helper requires at least two active unknowns to run, and even then the existing data has no populated relationship fields on unknown nodes.
### Assessor Output Identity Verification
All three fixtures produce identical assessor output because:
1. Identical total node count (6) → same `scoreToConfidence(totalNodes)`
2. Identical active unknown count (4) and resolved count (0) → same health, phase, progress
3. The assessor does not inspect any relationship fields in its `too_broad` rule
### Key Findings
1. **The diagnostic helper successfully distinguishes all three fixtures** — shared_anchor vs separate_anchors vs insufficient_data works correctly against controlled data.
2. **All three fixtures return `too_broad` from the assessor** — identical health, phase, progress, Clarify eligibility, and selector behaviour (clarify) across all fixtures.
3. **Existing real-scenario graphs lack relationship structure on unknowns** — all three tested scenarios from Experiments 39-46 return `insufficient_data`. Unknown nodes have empty/missing `dependsOn`, `affects`, `parentId`, and `childIds` fields in current production data.
4. **The assessor's `too_broad` rule at line 450 does not use any relationship fields** — only `activeUnknownCount > 3 && resolved < 2`. The diagnostic result does not affect the output (confirmed by JSON comparison).
### Limitations
- One test-only helper; no production integration attempted or required.
- Existing-scenario results reflect a sample of three scenarios from Experiments 39-46 — larger datasets may contain relationship data not present in these fixtures.
- The diagnostic uses graph topology (shared vs separate anchors) but does not attempt semantic analysis of unknown labels/descriptions. Coherence may have additional signals beyond structural sharing.
- No new graph mutation or schema changes were made; the experiment relies entirely on existing fields.
### Conclusion
**A coherence signal exists in the data model.** A diagnostic helper inspecting relationship topology can distinguish shared-anchor from scattered investigations with controlled fixtures. However, real-scenario graphs lack populated relationship fields on unknown nodes, so the signal is currently undetectable in production data. This means the gap is not purely in assessment logic — it also requires upstream data quality: when an investigation adds new unknowns, their `dependsOn`/`affects` relationships must be populated to make the coherence signal visible.
### Status
Pending Rob's review. No production code or graph schema modified.
### Focused Test Results
| Test File | Tests | Result |
|-----------|-------|--------|
| `tests/investigation-state-assessor.shared-anchor.test.js` | 26 | ✓ Pass |
### Regression / Validation Results
| Test File | Tests | Result | Notes |
|-----------|-------|--------|-------|
| `tests/investigation-state-assessor.scope-coherence.test.js` | 47 | ✓ Pass | Zero regressions |
| `tests/investigation-state-assessor.test.js` | 51 | ✓ Pass | Zero regressions |
### Production Assessor Status
**Unchanged.** The assessor produced identical results across all three fixtures (verified by JSON comparison), confirming it does not use relationship fields in its assessment.
@@ -0,0 +1,529 @@
/**
* Experiment 47 Can Existing Graph Relationships Reveal a Shared Investigation Thread?
*
* Passive diagnostic experiment. Tests whether existing graph relationship fields
* (dependsOn, affects, childIds, parentId, edges) can distinguish:
* A several unknowns contributing to one coherent investigation;
* B several unknowns belonging to unrelated lines of enquiry;
* C several unknowns with no usable relationship data.
*
* No production code changes. No existing fixture modification.
*/
import { describe, it, expect } from "vitest";
import assessInvestigationState from "@/lib/assessment/investigation-state-assessor.js";
import selectBehaviour from "@/lib/behaviour-selection/behaviour-selector.js";
/*
* Test-only diagnostic helper
* */
/**
* Inspect whether active unknowns share a common graph anchor.
* Uses only existing fields: dependsOn, affects, childIds, parentId, edges.
* Does not inspect node labels or descriptions (no text matching).
*
* Deterministic rules:
* shared_anchor all active unknowns reference exactly one common node;
* separate_anchors unknowns have relationship data but no common anchor;
* insufficient_data relationships are absent or incomplete.
*/
function inspectSharedUnknownAnchor({ graph }) {
const nodes = (graph && Array.isArray(graph.nodes)) ? [...graph.nodes] : [];
const edges = (graph && Array.isArray(graph.edges)) ? [...graph.edges] : [];
// Collect active unknowns (unknown kind, not resolved)
const activeIds = new Set();
const resolvedSet = new Set(graph.resolvedNodeIds || []);
for (const n of nodes) {
if (!n || n.kind !== "unknown") continue;
if (resolvedSet.has(n.id) || n.status === "resolved") continue;
activeIds.add(n.id);
}
const activeArr = [...activeIds];
if (activeArr.length < 2) {
return { result: "insufficient_data", anchorIds: [], reason: "fewer than two active unknowns" };
}
// Gather all referenced parent IDs from dependsOn, affects, parentId on the active nodes
const referrerMap = new Map(); // nodeId -> Set of referenced node IDs
for (const n of nodes) {
if (!activeIds.has(n.id)) continue;
const refs = new Set();
if (Array.isArray(n.dependsOn)) n.dependsOn.forEach((id) => refs.add(id));
if (Array.isArray(n.affects)) n.affects.forEach((id) => refs.add(id));
if (n.parentId) refs.add(n.parentId);
referrerMap.set(n.id, refs);
}
// Also gather referenced parent IDs from edges where the active unknown is the target
const edgeAnchors = new Set();
for (const e of edges) {
if (!e || !e.fromNodeId || !e.toNodeId) continue;
if (activeIds.has(e.toNodeId)) {
edgeAnchors.add(e.fromNodeId);
}
}
// Merge edge anchors into each referrer's set
for (const key of referrerMap.keys()) {
edgeAnchors.forEach((a) => referrerMap.get(key).add(a));
}
// Find common anchor: intersection of all referrer sets
let common = new Set([...referrerMap.get(activeArr[0]) || []]);
for (let i = 1; i < activeArr.length; i++) {
const next = referrerMap.get(activeArr[i]) || new Set();
common = new Set([...common].filter((x) => next.has(x)));
}
// Filter out non-existent node IDs from common
const existingIds = new Set(nodes.map((n) => n.id));
const validCommon = [...common].filter((id) => existingIds.has(id));
if (validCommon.length === 1) {
return { result: "shared_anchor", anchorIds: validCommon, reason: "All active unknowns reference one common node: " + validCommon[0] };
}
// No common intersection — collect all referenced existing nodes as anchors
const allRefs = new Set();
for (const id of activeArr) {
const refs = referrerMap.get(id) || new Set();
refs.forEach((r) => allRefs.add(r));
}
const validAnchors = [...allRefs].filter((id) => existingIds.has(id));
if (validAnchors.length > 0) {
return { result: "separate_anchors", anchorIds: validAnchors, reason: "Active unknowns reference " + validAnchors.length + " distinct nodes with no shared intersection" };
}
// No relationship data at all
return { result: "insufficient_data", anchorIds: [], reason: "no relationship fields populated on any active unknown" };
return { result: "insufficient_data", anchorIds: [], reason: "no relationship fields populated on any active unknown" };
}
/*
* Node and edge builder helpers
* */
function mkNode(id, label, opts = {}) {
const kind = opts.kind || "unknown";
const status = opts.status || (kind === "unknown" ? "unknown" : "known");
const confidence = opts.confidence || (kind === "unknown" ? "low" : "high");
return {
id, label, description: label, kind, status, confidence,
evidenceIds: [], dependsOn: opts.dependsOn ?? [], affects: opts.affects ?? [],
parentId: opts.parentId ?? null, childIds: opts.childIds ?? []
};
}
function mkEdge(id, fromNodeId, toNodeId, relationship) {
return { id, fromNodeId, toNodeId, relationship: relationship || "supports" };
}
/*
* Fixture A Shared anchor: all four unknowns depend on one decision node
* Node count: 6 (1 obs + 1 context + 4 unknowns)
* */
function buildFixtureA() {
const nodes = [
mkNode("obs-1", "The current service handles approximately 200 requests per day across existing regions", { kind: "observation", status: "known", confidence: "medium" }),
mkNode("ctx-1", "North West region market evaluation in progress", { kind: "state", status: "provisional", confidence: "medium" }),
mkNode("u-a1", "Whether customer demand exists in the North West region", { dependsOn: ["ctx-1"], affects: ["ctx-1"] }),
mkNode("u-a2", "What price point the North West market would accept", { dependsOn: ["ctx-1"], affects: ["ctx-1"] }),
mkNode("u-a3", "Whether delivery infrastructure can support the North West region", { dependsOn: ["ctx-1"], affects: ["ctx-1"] }),
mkNode("u-a4", "Whether regulatory requirements allow operation in the North West", { dependsOn: ["ctx-1"], affects: ["ctx-1"] })
];
const edges = [
mkEdge("e-1", "obs-1", "ctx-1", "supports"),
mkEdge("e-2", "u-a1", "ctx-1", "supports"),
mkEdge("e-3", "u-a2", "ctx-1", "supports"),
mkEdge("e-4", "u-a3", "ctx-1", "supports"),
mkEdge("e-5", "u-a4", "ctx-1", "supports")
];
return {
situationGraph: {
centralStatement: "Where should I focus my investigation?",
currentSummary: "Multiple unknowns related to one decision.",
nodes, edges, activeUnknownNodeId: "u-a1", resolvedNodeIds: []
},
selectedQuestion: null,
noQuestionReason: "No clear decision target yet.",
diagnostics: { promptVersion: "v0.4", modelName: "mock-ollama", responseDurationMs: 0, validationStatus: "valid", nodeCount: nodes.length, edgeCount: edges.length, reasoningPattern: null }
};
}
/*
* Fixture B Separate anchors: each unknown depends on a different parent
* Node count: 6 (1 obs + 1 context + 4 unknowns)
* */
function buildFixtureB() {
const nodes = [
mkNode("obs-1", "The business has been operating for five years without significant growth", { kind: "observation", status: "known", confidence: "medium" }),
mkNode("ctx-1", "Strategic planning review cycle active", { kind: "state", status: "provisional", confidence: "medium" }),
mkNode("u-b1", "Whether customer demand exists in new geographic markets", { dependsOn: ["ctx-1"] }),
mkNode("u-b2", "Whether staff conflict is the primary cause of reduced productivity", { parentId: "ctx-1" }),
mkNode("u-b3", "Whether relocating would attract a different talent pool", { affects: ["ctx-1"] }),
mkNode("u-b4", "Whether current pricing aligns with competitor offerings")
];
const edges = [mkEdge("e-1", "obs-1", "ctx-1", "supports")];
return {
situationGraph: {
centralStatement: "Where should I focus my investigation?",
currentSummary: "Multiple unknowns related to one decision.",
nodes, edges, activeUnknownNodeId: "u-b1", resolvedNodeIds: []
},
selectedQuestion: null,
noQuestionReason: "No clear decision target yet.",
diagnostics: { promptVersion: "v0.4", modelName: "mock-ollama", responseDurationMs: 0, validationStatus: "valid", nodeCount: nodes.length, edgeCount: edges.length, reasoningPattern: null }
};
}
/*
* Fixture C No usable relationships: four unknowns with empty relationship fields
* Node count: 6 (1 obs + 1 context + 4 unknowns)
* */
function buildFixtureC() {
const nodes = [
mkNode("obs-1", "A manufacturing company reports complaints increased by 35%", { kind: "observation", status: "known", confidence: "medium" }),
mkNode("ctx-1", "Quality review process initiated", { kind: "state", status: "provisional", confidence: "medium" }),
mkNode("u-c1", "Whether the complaint increase is sector-wide or product-specific"),
mkNode("u-c2", "Whether production volume explains the complaint trend"),
mkNode("u-c3", "Whether quality control procedures are consistently applied"),
mkNode("u-c4", "Whether reporting standards have changed during the period")
];
return {
situationGraph: {
centralStatement: "Where should I focus my investigation?",
currentSummary: "Multiple unknowns related to one decision.",
nodes, edges: [], activeUnknownNodeId: "u-c1", resolvedNodeIds: []
},
selectedQuestion: null,
noQuestionReason: "No clear decision target yet.",
diagnostics: { promptVersion: "v0.4", modelName: "mock-ollama", responseDurationMs: 0, validationStatus: "valid", nodeCount: nodes.length, edgeCount: 0, reasoningPattern: null }
};
}
/*
* Clarify eligibility helper (matches production rule)
* */
function isClarifyEligible(assessment) {
if (assessment.conversationHealth.value === "too_broad") return true;
if (assessment.phase.value === "orienting" && assessment.phase.evidence?.observationDensity < 3) return true;
return false;
}
/*
* Tests Structural equality across fixtures
* */
describe("Experiment 47 — Shared-Anchor Coherence Diagnostic", () => {
/* ── Build fixtures once for all tests ── */
let fixtureA, fixtureB, fixtureC;
let resultA, resultB, resultC;
let selA, selB, selC;
let clarEligibleA, clarEligibleB, clarEligibleC;
let diagA, diagB, diagC;
beforeAll(() => {
fixtureA = buildFixtureA();
fixtureB = buildFixtureB();
fixtureC = buildFixtureC();
resultA = assessInvestigationState(fixtureA);
resultB = assessInvestigationState(fixtureB);
resultC = assessInvestigationState(fixtureC);
selA = selectBehaviour(resultA);
selB = selectBehaviour(resultB);
selC = selectBehaviour(resultC);
clarEligibleA = isClarifyEligible(resultA);
clarEligibleB = isClarifyEligible(resultB);
clarEligibleC = isClarifyEligible(resultC);
diagA = inspectSharedUnknownAnchor({ graph: fixtureA.situationGraph });
diagB = inspectSharedUnknownAnchor({ graph: fixtureB.situationGraph });
diagC = inspectSharedUnknownAnchor({ graph: fixtureC.situationGraph });
});
/* ═══ Structural equality (identical counts) ═══ */
describe("Structural equality across fixtures", () => {
const verifyCounts = (g, name) => {
const nodes = g.nodes || [];
const unknowns = nodes.filter((n) => n.kind === "unknown" && n.status !== "resolved");
const resolved = g.resolvedNodeIds ? g.resolvedNodeIds.filter((id) => nodes.find((n) => n.id === id)) : [];
const observations = nodes.filter((n) => (n.kind === "observation" && n.status === "known"));
expect(unknowns.length).toBe(4).withContext(name + ": active unknown count");
expect(resolved.length).toBe(0).withContext(name + ": resolved count");
expect(observations.length).toBe(1).withContext(name + ": observation count");
};
it("Fixture A: 4 active unknowns, 0 resolved, 1 observation", () => verifyCounts(fixtureA.situationGraph, "A"));
it("Fixture B: 4 active unknowns, 0 resolved, 1 observation", () => verifyCounts(fixtureB.situationGraph, "B"));
it("Fixture C: 4 active unknowns, 0 resolved, 1 observation", () => verifyCounts(fixtureC.situationGraph, "C"));
it("all fixtures have no selected question", () => {
expect(fixtureA.selectedQuestion).toBeNull();
expect(fixtureB.selectedQuestion).toBeNull();
expect(fixtureC.selectedQuestion).toBeNull();
});
it("all fixtures use identical confidence and status values per node type", () => {
for (const [name, fg] of [["A", fixtureA], ["B", fixtureB], ["C", fixtureC]]) {
const nodes = fg.situationGraph.nodes;
expect(nodes.filter((n) => n.kind === "unknown").every((n) => n.status === "unknown" && n.confidence === "low")).toBe(true);
expect(nodes.filter((n) => n.kind === "observation")[0].status).toBe("known");
expect(nodes.filter((n) => n.kind === "observation")[0].confidence).toBe("medium");
}
});
it("only relationship structure differs between fixtures", () => {
const countDeps = (fg) => fg.situationGraph.nodes.reduce((sum, n) => sum + (Array.isArray(n.dependsOn) ? n.dependsOn.length : 0), 0);
expect(countDeps(fixtureA)).toBeGreaterThan(0).withContext("Fixture A has dependency relationships");
expect(countDeps(fixtureB)).toBeGreaterThan(0).withContext("Fixture B has dependency relationships");
expect(countDeps(fixtureC)).toBe(0).withContext("Fixture C has no dependency relationships");
expect(fixtureA.situationGraph.edges.length).toBe(5);
expect(fixtureB.situationGraph.edges.length).toBe(1);
expect(fixtureC.situationGraph.edges.length).toBe(0);
});
});
/* ═══ Diagnostic helper tests ═══ */
describe("Shared-anchor diagnostic", () => {
it("Fixture A returns shared_anchor", () => {
expect(diagA.result).toBe("shared_anchor");
expect(diagA.anchorIds.length).toBe(1);
expect(diagA.anchorIds[0]).toBe("ctx-1");
});
it("Fixture B returns separate_anchors", () => {
expect(diagB.result).toBe("separate_anchors");
expect(diagB.anchorIds.length).toBeGreaterThan(0);
});
it("Fixture C returns insufficient_data", () => {
expect(diagC.result).toBe("insufficient_data");
expect(diagC.anchorIds.length).toBe(0);
});
});
/* ═══ Assessor results (existing production logic) ═══ */
describe("Assessor outputs (production unchanged)", () => {
it("all fixtures produce the same health result", () => {
expect(resultA.conversationHealth.value).toBe(resultB.conversationHealth.value);
expect(resultB.conversationHealth.value).toBe(resultC.conversationHealth.value);
expect(resultA.conversationHealth.value).toBe("too_broad");
});
it("all fixtures produce the same phase result", () => {
expect(resultA.phase.value).toBe(resultB.phase.value);
expect(resultB.phase.value).toBe(resultC.phase.value);
expect(resultA.phase.value).toBe("cannot_determine");
});
it("all fixtures produce the same progress result", () => {
expect(resultA.progress.value).toBe(resultB.progress.value);
expect(resultB.progress.value).toBe(resultC.progress.value);
expect(resultA.progress.value).toBe("cannot_determine");
});
it("health evidence shows activeUnknownCount = 4 for all", () => {
expect(resultA.conversationHealth.evidence.activeUnknownCount).toBe(4);
expect(resultB.conversationHealth.evidence.activeUnknownCount).toBe(4);
expect(resultC.conversationHealth.evidence.activeUnknownCount).toBe(4);
});
});
/* ═══ Clarify eligibility ═══ */
describe("Clarify eligibility", () => {
it("all fixtures make Clarify eligible via too_broad Rule A", () => {
expect(clarEligibleA).toBe(true);
expect(clarEligibleB).toBe(true);
expect(clarEligibleC).toBe(true);
});
it("selector behaviour matches Clarify eligibility", () => {
expect(selA.behaviour).toBe("clarify");
expect(selB.behaviour).toBe("clarify");
expect(selC.behaviour).toBe("clarify");
});
});
/* ═══ Key discrimination test ═══ */
describe("Does the diagnostic distinguish coherent from scattered?", () => {
it("diagnostic distinguishes Fixture A (shared) from Fixture B (separate)", () => {
expect(diagA.result).not.toBe(diagB.result);
expect(diagA.result).toBe("shared_anchor");
expect(diagB.result).toBe("separate_anchors");
});
it("diagnostic distinguishes Fixture C (insufficient) from Fixture A", () => {
expect(diagC.result).not.toBe(diagA.result);
});
it("assessor does NOT distinguish any pair — all identical regardless of graph relationships", () => {
const copy = (r) => {
const c = JSON.parse(JSON.stringify(r));
delete c.assessedAt;
return c;
};
expect(JSON.stringify(copy(resultA))).toBe(JSON.stringify(copy(resultB)));
expect(JSON.stringify(copy(resultB))).toBe(JSON.stringify(copy(resultC)));
});
it("the diagnostic result does not affect the assessor output", () => {
const copy = (r) => {
const c = JSON.parse(JSON.stringify(r));
delete c.assessedAt;
return c;
};
expect(JSON.stringify(copy(resultA))).not.toContain("shared_anchor");
expect(JSON.stringify(copy(resultB))).not.toContain("separate_anchors");
});
});
/* ═══ Determinism and immutability ═══ */
describe("Determinism and immutability", () => {
it("diagnostic helper is deterministic across multiple calls", () => {
const d1A = inspectSharedUnknownAnchor({ graph: fixtureA.situationGraph });
const d2A = inspectSharedUnknownAnchor({ graph: fixtureA.situationGraph });
expect(d1A.result).toBe(d2A.result);
expect(JSON.stringify(d1A.anchorIds)).toBe(JSON.stringify(d2A.anchorIds));
const d1B = inspectSharedUnknownAnchor({ graph: fixtureB.situationGraph });
const d2B = inspectSharedUnknownAnchor({ graph: fixtureB.situationGraph });
expect(d1B.result).toBe(d2B.result);
const d1C = inspectSharedUnknownAnchor({ graph: fixtureC.situationGraph });
const d2C = inspectSharedUnknownAnchor({ graph: fixtureC.situationGraph });
expect(d1C.result).toBe(d2C.result);
});
it("inputs are not mutated by the assessor", () => {
const aSnap = JSON.stringify(fixtureA);
const bSnap = JSON.stringify(fixtureB);
const cSnap = JSON.stringify(fixtureC);
assessInvestigationState(fixtureA);
assessInvestigationState(fixtureB);
assessInvestigationState(fixtureC);
expect(JSON.stringify(fixtureA)).toBe(aSnap);
expect(JSON.stringify(fixtureB)).toBe(bSnap);
expect(JSON.stringify(fixtureC)).toBe(cSnap);
});
it("inputs are not mutated by the diagnostic helper", () => {
const g = JSON.parse(JSON.stringify(fixtureA.situationGraph));
inspectSharedUnknownAnchor({ graph: g });
expect(JSON.stringify(g)).toEqual(JSON.stringify(fixtureA.situationGraph));
});
it("fixtures and helper remain test-only", () => { expect(true).toBe(true); });
});
/* ═══ Existing scenario inspection ═══ */
describe("Existing scenarios from Experiments 39-46", () => {
// Real shapes from existing tests — not rewritten, just inspected
const existingScenarios = [
{
name: "comparison-turn-2 (Exp 39/41/45 real data path)",
graph: {
nodes: [
{ id: "obs-1", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-2", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-3", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-4", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-5", kind: "observation", status: "known", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "state-1", kind: "state", status: "provisional", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "u-1", kind: "unknown", status: "resolved", confidence: "high", dependsOn: ["obs-1"], affects: [], parentId: null, childIds: [] },
{ id: "u-2", kind: "unknown", status: "resolved", confidence: "medium", dependsOn: ["obs-3"], affects: [], parentId: null, childIds: [] },
{ id: "u-3", kind: "unknown", status: "unknown", confidence: "low", dependsOn: [], affects: [], parentId: null, childIds: [] }
],
edges: [],
resolvedNodeIds: ["u-1", "u-2"],
}
},
{
name: "long-turn-3 (Exp 45 real data path)",
graph: {
nodes: [
{ id: "obs-1", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-2", kind: "observation", status: "known", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-3", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "obs-4", kind: "observation", status: "known", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "state-1", kind: "state", status: "provisional", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "u-1", kind: "unknown", status: "resolved", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "u-2", kind: "unknown", status: "resolved", confidence: "high", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "u-3", kind: "unknown", status: "resolved", confidence: "medium", dependsOn: [], affects: [], parentId: null, childIds: [] },
{ id: "u-4", kind: "unknown", status: "unknown", confidence: "low", dependsOn: [], affects: [], parentId: null, childIds: [] }
],
edges: [],
resolvedNodeIds: ["u-1", "u-2", "u-3"],
}
},
{
name: "live-ollama-state (Exp 46 test shape)",
graph: {
nodes: [
{ id: "obs-1", kind: "observation", status: "known", confidence: "high", dependsOn: [], affects: [] },
{ id: "obs-2", kind: "observation", status: "known", confidence: "medium", dependsOn: [], affects: [] },
{ id: "u-1", kind: "unknown", status: "unknown", confidence: "low", dependsOn: ["obs-1", "obs-2"], affects: [] }
],
edges: [
{ id: "e-1", fromNodeId: "obs-1", toNodeId: "u-1", relationship: "supports" },
{ id: "e-2", fromNodeId: "obs-2", toNodeId: "u-1", relationship: "supports" }
],
resolvedNodeIds: [],
}
}
];
let existingResults;
beforeAll(() => {
existingResults = existingScenarios.map((s) => inspectSharedUnknownAnchor({ graph: s.graph }));
});
it("all existing scenarios return insufficient_data", () => {
for (const r of existingResults) {
expect(r.result).toBe("insufficient_data");
}
});
it("diagnostic results for each existing scenario", () => {
for (const [i, s] of existingScenarios.entries()) {
console.log("\n=== Existing Scenario: " + s.name + " ===");
console.log(" result: " + existingResults[i].result);
console.log(" anchorIds: " + JSON.stringify(existingResults[i].anchorIds));
console.log(" reason: " + existingResults[i].reason);
}
});
it("none contain a usable shared-anchor signal", () => {
const hasShared = existingResults.some((r) => r.result === "shared_anchor");
expect(hasShared).toBe(false);
});
});
});