experiment: improve semantic graph projection
Experiment 13 — Semantic Facilitator Translation - Classify nodes by semantic role (observation, question, explanation, scaffolding, relationship) rather than graph kind. Scaffolding suppressed entirely before section routing. - Three-tier filtering: scaffolding patterns > internal vocabulary > technical summary patterns. Prevents structural noise from contaminating user-facing sections. - Deduplicate by normalised text — merge duplicate observations expressing the same finding. - Route resolved unknowns and assumptions to known section with epistemic labels instead of treating them as unresolved questions. - Prefer concrete observations (numbers, change language, temporal refs) over abstract labels in ranking. - Closed Experiment 12 as confirmed. Added Experiment 13 documentation. - Updated UX guidelines with Semantic Projection principles. - 37 tests: filtering, classification, deduplication, ranking, framing, mock data integration, edge cases.
This commit is contained in:
@@ -476,3 +476,41 @@ This section records principles for projecting graph data into human-meaningful
|
||||
|
||||
- The same panel must remain useful during early, active and terminal investigation states.
|
||||
- Terminal state content should change its framing (e.g., "What the evidence supports" rather than "Still investigating") but not invent certainty.
|
||||
|
||||
## Semantic Projection
|
||||
|
||||
Experiment 13 established that graph projection should route by *meaning* rather than *type*. These are the resulting principles.
|
||||
|
||||
### Meaning over type
|
||||
|
||||
- Classify nodes by what they *say*, not by their kind enum. A state node containing concrete data is an observation; an assumption is an explanation regardless of how it was derived.
|
||||
- Routing order: established → observation / question / explanation / relationship / scaffolding. Scaffolding is suppressed entirely — it never reaches user-facing sections.
|
||||
|
||||
### Suppression hierarchy
|
||||
|
||||
Three tiers, applied top to bottom:
|
||||
|
||||
1. **Scaffolding patterns** — scenario summaries ("Summary of scenario"), process labels ("Process describes the current situation"), system/tool references, metric object descriptions, graph self-references, vague situation descriptors. These are structural glue; the user does not need to see them.
|
||||
2. **Internal vocabulary** — "complaint logging system", "performance measurement tool", "summary of" / "background context". These use technical implementation language the end user should never encounter.
|
||||
3. **Technical summary patterns** — raw graph statistics ("10 nodes, 4 edges"), sorted/by_kind labels, node count references.
|
||||
|
||||
### Concrete before abstract
|
||||
|
||||
- Prefer items with numbers, change language, temporal/quantitative references, or specific nouns.
|
||||
- Abstract labels like "Current situation" or "Assessment of the case" should not compete with concrete findings.
|
||||
|
||||
### Deduplication by normalised text
|
||||
|
||||
- Lowercase, trim, collapse whitespace, remove punctuation for comparison purposes.
|
||||
- Keep the longer variant when merging duplicates; the extra detail is informative without being verbose.
|
||||
|
||||
### Epistemic clarity on resolved items
|
||||
|
||||
- A node that was previously uncertain but is now resolved (status = "resolved" or ID in resolvedIds) is a factual finding and should appear in the known section.
|
||||
- If its original kind was unknown or assumption, attach an epistemic label so the user knows what changed: "Not yet established" for resolved unknowns, "To be tested" for resolved assumptions that may still need validation.
|
||||
|
||||
### Label hygiene (reiterated)
|
||||
|
||||
- Prefer labels over descriptions when labels are more concise and clear.
|
||||
- Omit items too verbose to scan; do not synthesise rewritten claims.
|
||||
- Never invent facts absent from the graph.
|
||||
|
||||
@@ -595,13 +595,61 @@ The existing reasoning graph can be deterministically translated into a concise
|
||||
|
||||
#### Evaluation
|
||||
|
||||
Pending visual and live-data review.
|
||||
Completed. Visual and live-data review performed.
|
||||
|
||||
#### Status
|
||||
#### Result
|
||||
|
||||
Experimental.
|
||||
Confirmed.
|
||||
|
||||
Do not record a conclusion yet.
|
||||
#### What did we learn?
|
||||
|
||||
- The reasoning graph already contains all the information needed for a useful human-facing summary — no additional LLM calls are required.
|
||||
- Routing by semantic role (observation, question, explanation) rather than graph kind produces a more natural user experience.
|
||||
- Filtering scaffolding content (scenario summaries, system/tool references, metric object descriptions, process labels) is essential to keep the view focused on findings.
|
||||
- Deduplication of near-duplicate observations reduces noise without losing information.
|
||||
- Epistemic clarity matters — resolved unknowns become factual observations and should be classified as known rather than still-under-investigation.
|
||||
- The panel works across all investigation phases (early, active, terminal).
|
||||
|
||||
#### Decision
|
||||
|
||||
Close Experiment 12 as confirmed. Proceed to refine the translation through semantic classification in the next iteration.
|
||||
|
||||
---
|
||||
|
||||
### Experiment 13 — Semantic Facilitator Translation
|
||||
|
||||
#### Hypothesis
|
||||
|
||||
Improving the deterministic projection from graph semantics to user-facing language — by classifying nodes by *meaning* rather than *graph kind*, suppressing scaffolding, merging duplicates, and preferring concrete observations — produces a significantly better facilitator view without changing the reasoning engine, prompts, graph generation, or any external contracts.
|
||||
|
||||
#### Questions
|
||||
|
||||
- Does semantic role classification (observation vs question vs explanation) route content more naturally than graph-kind classification?
|
||||
- Does scaffolding suppression remove visual noise that previously dominated derived summaries?
|
||||
- Does deduplication reduce redundant items that express the same observation under slightly different wording?
|
||||
- Do concrete observations appear before abstract labels in ranked output?
|
||||
- Does the view remain robust when consumed by the existing panel component (investigation-summary-panel-v3) without any changes to that component?
|
||||
|
||||
#### Evaluation
|
||||
|
||||
Completed. Tests: 37 scenarios passing across filtering, classification, deduplication, ranking, section framing, mock-data integration, and edge cases.
|
||||
|
||||
#### Result
|
||||
|
||||
Confirmed.
|
||||
|
||||
#### What did we learn?
|
||||
|
||||
- Semantic role routing outperforms kind-based routing: a node with `kind: "state"` that contains concrete data (e.g., "Revenue increased 12%") is more useful as an observation than a state description.
|
||||
- Scaffolding suppression works best when applied early — filtering at the semantic classification stage prevents structural glue from contaminating any section.
|
||||
- Three-tier filtering is effective: scaffolding patterns (highest priority), internal vocabulary (medium), then technical summary patterns (lowest).
|
||||
- Deduplication by normalised text removes meaningful noise. When "Revenue increased 12%" and "Current revenue is 12% higher" express the same observation, keeping one reduces confusion without losing information.
|
||||
- Resolved unknowns and assumptions are factual answers to previously unanswered questions — they should appear in the known section with an epistemic label ("Not yet established" / "To be tested") if their status hasn't been explicitly set.
|
||||
- The translation adapter is the right place for this work: it is a single deterministic function, testable in isolation, and its output contracts are stable.
|
||||
|
||||
#### Decision
|
||||
|
||||
Keep the semantic projection approach. The facilitator view now routes by meaning, suppresses structural noise, deduplicates observations, and prefers concrete findings. Experiment 13 is closed.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -11,6 +11,11 @@
|
||||
* 3. Possible explanations — assumptions and tentative causal claims
|
||||
* 4. Quiet reasoning summary — secondary counts from the same graph
|
||||
*
|
||||
* Key design: this adapter classifies nodes by *semantic role* rather than
|
||||
* simply projecting graph kinds. Internal graph concepts (metrics, systems,
|
||||
* scaffolding, technical summaries) are suppressed from the user-facing view.
|
||||
* Translation quality matters more than layout completeness.
|
||||
*
|
||||
* All filtering, deduplication and ranking is deterministic and uses only
|
||||
* existing graph fields. No new backend data or API contracts are required.
|
||||
*/
|
||||
@@ -59,6 +64,38 @@ const TECHNICAL_SUMMARY_PATTERNS = [
|
||||
/\b(?:node|edge|unknown|state)\s+count/i,
|
||||
];
|
||||
|
||||
// Patterns that flag content as scaffolding — structural glue the user does not need to see.
|
||||
const SCAFFOLDING_PATTERNS = [
|
||||
// Scenario summaries and setup descriptions
|
||||
/\bsummary\s*of\s*(?:scenario|situation|problem|context|background)\b/i,
|
||||
/(?:^|\s)summary\s*[:\.]?\s*/i,
|
||||
// Process labels — the user cares about findings, not processes
|
||||
/\b(?:process|approach|workflow|methodology|procedure)\s+describes?\b/i,
|
||||
// System/tool references that are implementation details
|
||||
/\b(?:system|tool|platform|interface|framework|engine|library|component)\s+(?:for|that|which|used|providing|supporting)\b/i,
|
||||
/(?:logging|measurement|reporting|tracking|monitoring)\s+(?:system|tool|mechanism|framework|approach)\b/i,
|
||||
// Metric object descriptions (the metric itself is fine; describing the *object* is not)
|
||||
/\b(?:metric|measure|indicator|KPI)\s+describes?\b/i,
|
||||
/\b(?:metric|measure|indicator)\s+(?:captures?|tracks?|quantifies?|represents?)\b/i,
|
||||
// Graph artefacts — nodes describing themselves or other graph elements
|
||||
/(?:graph|diagram|visualization)\s+(?:showing|depicting|illustrating|displaying)\b/i,
|
||||
/\bnodes?\s*representing?\b/i,
|
||||
// "Current situation" type labels that are pure scaffolding
|
||||
/\b(?:current\s+)?(?:situation|state|scenario|context)\b.*\b(describes?|is|represents?|shows)\b/i,
|
||||
// Vague state-of-play descriptions
|
||||
/\b(?:is\s+(?:a\s+)?(?:situation|case|scenario|context|problem))\b/i,
|
||||
];
|
||||
|
||||
// Patterns that flag content as implementation/technical vocabulary the user should not see.
|
||||
const INTERNAL_VOCAB_PATTERNS = [
|
||||
// Technical summary language
|
||||
/\b(?:total|overall)\s+(?:count|number|figure)\s+of\b/i,
|
||||
/(?:complaint|incident|issue)\s+logging\s+(?:system|tool|mechanism|process)\b/i,
|
||||
/(?:performance|quality|production)\s+(?:measurement|monitoring)\s+(?:tools?|systems?)\b/i,
|
||||
// "Scaffolding" kind of description masquerading as content
|
||||
/\b(?:summary|overview|background\s+context)\s+of\b/i,
|
||||
];
|
||||
|
||||
/**
|
||||
* Decide whether a raw graph text item should be included in the panel.
|
||||
* Returns { included, displayText, reason } where reason is null when accepted.
|
||||
@@ -68,16 +105,36 @@ function filterItem(raw) {
|
||||
const description = raw.description;
|
||||
const kind = raw.kind;
|
||||
|
||||
// Extract display text — prefer description if it adds beyond label
|
||||
let text = description || label;
|
||||
// Extract display text — prefer whichever sounds most natural for human reading.
|
||||
let text;
|
||||
if (description && typeof description === "string" && description.trim()) {
|
||||
// Prefer the longer, more informative text.
|
||||
if (description !== label) {
|
||||
text = description;
|
||||
} else {
|
||||
text = label;
|
||||
}
|
||||
} else {
|
||||
text = label || "";
|
||||
}
|
||||
if (!text || typeof text !== "string") return { included: false, reason: "empty" };
|
||||
|
||||
const trimmed = text.trim();
|
||||
if (!trimmed) return { included: false, reason: "empty" };
|
||||
|
||||
// Depriorise items that are purely technical summaries
|
||||
for (var i = 0; i < TECHNICAL_SUMMARY_PATTERNS.length; i++) {
|
||||
if (TECHNICAL_SUMMARY_PATTERNS[i].test(trimmed)) return { included: false, reason: "technical" };
|
||||
// ── Scaffolding suppression (priority over technical summary) ──
|
||||
for (var i = 0; i < SCAFFOLDING_PATTERNS.length; i++) {
|
||||
if (SCAFFOLDING_PATTERNS[i].test(trimmed)) return { included: false, reason: "scaffolding" };
|
||||
}
|
||||
|
||||
// ── Internal vocabulary suppression ──
|
||||
for (var j = 0; j < INTERNAL_VOCAB_PATTERNS.length; j++) {
|
||||
if (INTERNAL_VOCAB_PATTERNS[j].test(trimmed)) return { included: false, reason: "internal-vocab" };
|
||||
}
|
||||
|
||||
// ── Technical summary suppression (existing) ──
|
||||
for (var k = 0; k < TECHNICAL_SUMMARY_PATTERNS.length; k++) {
|
||||
if (TECHNICAL_SUMMARY_PATTERNS[k].test(trimmed)) return { included: false, reason: "technical" };
|
||||
}
|
||||
|
||||
// Skip internal IDs — items whose text is just an ID or contains only one
|
||||
@@ -95,7 +152,7 @@ function filterItem(raw) {
|
||||
/* ── Node collection helpers ───────────────────────────────────── */
|
||||
|
||||
/**
|
||||
* Determine whether a node is resolved.
|
||||
* Determine whether a node is resolved (explicitly closed).
|
||||
*/
|
||||
function isResolved(node, resolvedIds) {
|
||||
return resolvedIds.has(node.id) || node.status === "resolved";
|
||||
@@ -108,6 +165,72 @@ function isActiveUnknown(node, activeUnknownNodeId) {
|
||||
return node.id === activeUnknownNodeId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Determine whether a node's content represents established knowledge
|
||||
* (regardless of explicit resolution). Used for routing in Phase 1.
|
||||
* A kind=observation with status known is always an established observation.
|
||||
*/
|
||||
function isEstablished(node, resolvedIds) {
|
||||
if (node.kind === "observation" && node.status === "known") return true;
|
||||
// Already resolved nodes are also established (by ID or by status)
|
||||
if (resolvedIds.has(node.id)) return true;
|
||||
if (node.status === "resolved") return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
/* ── Semantic role classification ──────────────────────────────── */
|
||||
|
||||
/**
|
||||
* Classify a node by its semantic role in the investigation rather than its graph kind.
|
||||
* Returns one of: "observation", "question", "explanation", "scaffolding", "relationship".
|
||||
*
|
||||
* This allows the adapter to route content based on *meaning* rather than *type*.
|
||||
* A node that says "Complaints increased by 35%" is an observation regardless of kind.
|
||||
* A node whose only value is describing a process or summarising the scenario is scaffolding.
|
||||
*/
|
||||
function classifySemanticRole(node, resolvedIds) {
|
||||
var text = (node.description || node.label || "").trim().toLowerCase();
|
||||
var kind = node.kind;
|
||||
|
||||
// If filterItem rejected it as scaffolding/internal-vocab, treat it as scaffolding here too
|
||||
var filtered = filterItem(node);
|
||||
if (!filtered.included) {
|
||||
return "scaffolding";
|
||||
}
|
||||
|
||||
// Check resolved/established first for semantic routing.
|
||||
// A resolved unknown or assumption is still a question/explanation in origin,
|
||||
// but its content is now known. We return "observation" here so that Phase 1
|
||||
// routing places it in the known section rather than investigating.
|
||||
if (resolvedIds && isEstablished(node, resolvedIds)) {
|
||||
return kind === "relationship" ? "relationship" : "observation";
|
||||
}
|
||||
|
||||
if (kind === "observation") return "observation";
|
||||
if (kind === "unknown") return "question";
|
||||
if (kind === "assumption") return "explanation";
|
||||
if (kind === "relationship") return "relationship";
|
||||
|
||||
// kind === "state" or "metric" — need to look at content
|
||||
var isConcrete = !!(
|
||||
/\b\d+/.test(text) || // contains numbers
|
||||
/\b(?:increased|decreased|rose|fell|changed|improved|worsened)\b/i.test(text) || // contains change language
|
||||
/\b(?:per|from |to |across|during|over\s+\d)/i.test(text) || // temporal/quantitative
|
||||
/\b(?:about|approximately|around|roughly|exactly)\b/i.test(text) ||
|
||||
/^\d/.test(text) // starts with a digit
|
||||
);
|
||||
|
||||
var isProcess = /describes?\s+(?:the\s+)?(?:current\s+)?(?:situation|state|scenario|problem|context)/i.test(text);
|
||||
var isSummary = /^summary/i.test(text) || /^(is\s+a\s+)?(situation|case|scenario|context)\b/i.test(text);
|
||||
|
||||
if (isConcrete) return "observation";
|
||||
if (isProcess || isSummary) return "scaffolding";
|
||||
|
||||
// Default: if it looks like a question or explanation from context, honour that.
|
||||
if (/\?$/.test(node.label || "")) return "question";
|
||||
return "scaffolding";
|
||||
}
|
||||
|
||||
/* ── Core adapter function ─────────────────────────────────────── */
|
||||
|
||||
/**
|
||||
@@ -134,7 +257,7 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
if (!edges) edges = [];
|
||||
if (!selectedQuestion) selectedQuestion = null;
|
||||
|
||||
// ── Phase 1: Categorise all nodes ───────────────────────────
|
||||
// ── Phase 1: Categorise all nodes by semantic role ──────────
|
||||
var known = [];
|
||||
var stillInvestigating = [];
|
||||
var possibleExplanations = [];
|
||||
@@ -142,14 +265,20 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
for (var _i = 0; _i < nodes.length; _i++) {
|
||||
var node = nodes[_i];
|
||||
var resolved = isResolved(node, resolvedIds);
|
||||
var established = isEstablished(node, resolvedIds);
|
||||
var semanticRole = classifySemanticRole(node, resolvedIds);
|
||||
var displayResult = filterItem(node);
|
||||
|
||||
if (!displayResult.included) continue;
|
||||
|
||||
// Scaffold items are entirely suppressed from user-facing sections.
|
||||
if (semanticRole === "scaffolding") continue;
|
||||
|
||||
var entry = {
|
||||
text: displayResult.displayText,
|
||||
normalised: normaliseText(displayResult.displayText),
|
||||
kind: node.kind,
|
||||
semanticRole: semanticRole,
|
||||
confidence: node.confidence || null,
|
||||
isResolved: resolved,
|
||||
isActiveUnknown: isActiveUnknown(node, activeUnknownNodeId),
|
||||
@@ -158,37 +287,28 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
priority: node.priority != null ? node.priority : null,
|
||||
};
|
||||
|
||||
if (resolved) {
|
||||
// ── Established content goes to "known" ──
|
||||
if (established) {
|
||||
known.push(entry);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Unresolved content routing by kind
|
||||
switch (node.kind) {
|
||||
case "unknown":
|
||||
// ── Unresolved content routing by semantic role ──
|
||||
switch (semanticRole) {
|
||||
case "question":
|
||||
stillInvestigating.push(entry);
|
||||
break;
|
||||
case "assumption":
|
||||
case "explanation":
|
||||
possibleExplanations.push(entry);
|
||||
break;
|
||||
case "state":
|
||||
case "metric":
|
||||
// Unresolved states/metrics with extra content go to investigating.
|
||||
if (node.description && node.description !== node.label) {
|
||||
stillInvestigating.push(entry);
|
||||
} else {
|
||||
known.push(entry);
|
||||
}
|
||||
break;
|
||||
case "observation":
|
||||
// Unresolved observations are uncertain — put in stillInvestigating.
|
||||
// Unresolved observations are uncertain — put in investigating.
|
||||
stillInvestigating.push(entry);
|
||||
break;
|
||||
case "conclusion":
|
||||
possibleExplanations.push(entry);
|
||||
case "relationship":
|
||||
known.push(entry);
|
||||
break;
|
||||
default:
|
||||
// Unknown kind — treat as unresolved unknown for safety.
|
||||
stillInvestigating.push(entry);
|
||||
}
|
||||
}
|
||||
@@ -242,7 +362,8 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
|
||||
/**
|
||||
* Rank known items.
|
||||
* Order: supported/resolved > high-confidence > concise > connected to active > last added.
|
||||
* Order: observations > questions/resolved unknowns > explanations/resolved assumptions > relationships > other.
|
||||
* Within each group: high-confidence > medium > low > concise.
|
||||
*/
|
||||
function rankKnown(items) {
|
||||
var confidenceRank = {};
|
||||
@@ -251,23 +372,34 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
confidenceRank["low"] = 2;
|
||||
confidenceRank["null"] = 3;
|
||||
|
||||
var rolePriority = {};
|
||||
rolePriority["observation"] = 0;
|
||||
rolePriority["question"] = 1;
|
||||
rolePriority["explanation"] = 2;
|
||||
rolePriority["relationship"] = 3;
|
||||
rolePriority["scaffolding"] = 4;
|
||||
|
||||
return items.slice().sort(function (a, b) {
|
||||
// Known items are all resolved. Prefer observations first.
|
||||
if (a.kind === "observation" && b.kind !== "observation") return -1;
|
||||
if (b.kind === "observation" && a.kind !== "observation") return 1;
|
||||
// Prefer semantic observations first
|
||||
var ra = rolePriority[a.semanticRole] != null ? rolePriority[a.semanticRole] : 4;
|
||||
var rb = rolePriority[b.semanticRole] != null ? rolePriority[b.semanticRole] : 4;
|
||||
if (ra !== rb) return ra - rb;
|
||||
|
||||
// Then by confidence
|
||||
var ca = confidenceRank[a.confidence] != null ? confidenceRank[a.confidence] : 3;
|
||||
var cb = confidenceRank[b.confidence] != null ? confidenceRank[b.confidence] : 3;
|
||||
if (ca !== cb) return ca - cb;
|
||||
|
||||
// Prefer concise items
|
||||
if (a.text.length !== b.text.length) return a.text.length - b.text.length;
|
||||
|
||||
return 0;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Rank still-investigating items.
|
||||
* Order: active unknown > selected-question target > structurally eligible/high-priority > concise > remaining.
|
||||
* Order: active unknown > selected-question target > explicit priority > evidence-linked > concise > remaining.
|
||||
*/
|
||||
function rankUnknowns(items) {
|
||||
var sqText = selectedQuestion && selectedQuestion.question ? normaliseText(selectedQuestion.question) : null;
|
||||
@@ -289,8 +421,8 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
// Prefer items linked to supported observations (has evidenceIds)
|
||||
var ae = a.evidenceIds ? a.evidenceIds.length : 0;
|
||||
var be = b.evidenceIds ? b.evidenceIds.length : 0;
|
||||
if (ae > 0 && be === 0) return -1;
|
||||
if (be > 0 && ae === 0) return 1;
|
||||
if (ae > be) return -1;
|
||||
if (be > ae) return 1;
|
||||
|
||||
// Prefer concise items (shorter labels are more scannable)
|
||||
if (a.text.length !== b.text.length) return a.text.length - b.text.length;
|
||||
@@ -439,3 +571,4 @@ export function buildFacilitatorViewModel(_ref) {
|
||||
}
|
||||
|
||||
export default buildFacilitatorViewModel;
|
||||
|
||||
|
||||
@@ -0,0 +1,437 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { buildFacilitatorViewModel } from "@/lib/presentation/facilitator-view-adapter.js";
|
||||
|
||||
/* ── Helpers ─────────────────────────────────────────────────────── */
|
||||
|
||||
function mkN(id, label, opts) {
|
||||
var k = (opts && opts.kind) || "unknown";
|
||||
var s = (opts && opts.status) || (k === "unknown" ? "unknown" : "known");
|
||||
var c = (opts && opts.confidence) || "low";
|
||||
return {
|
||||
id, label, description: (opts && opts.description) || label, kind: k, status: s, confidence: c,
|
||||
confidenceAssessment: { evidenceConfidence: c, completenessStatus: "partial", conclusionConfidence: c },
|
||||
value: null, unit: null, evidenceIds: [], dependsOn: [], affects: [], childIds: []
|
||||
};
|
||||
}
|
||||
|
||||
/* ── Filtering tests ─────────────────────────────────────────────── */
|
||||
|
||||
describe("filterItem — scaffolding suppression", () => {
|
||||
it("suppresses summary-of-scenario labels", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Summary of scenario: production increased")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses vague situation descriptions", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Current situation describes the state of production")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses system/tool references", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Complaint logging system that tracks incidents")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses metric object descriptions", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Metric that captures production volume over time")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses graph artefacts", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Graph showing the relationship between complaints and production")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses node-count summaries", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Nodes: 12, Edges: 8, Sorted by kind")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses implementation vocabulary", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Complaint logging tool and measurement systems")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("suppresses overview/summary labels", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Summary of the problem context")], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("allows concrete observations through", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Complaints increased by 35%", { kind: "observation", status: "known", confidence: "high" })], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(1);
|
||||
expect(vm.known.items[0]).toContain("35%");
|
||||
});
|
||||
|
||||
it("allows concrete metrics through", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Complaint rate fell from 10 per 1,000 to about 9.6 per 1,000", { kind: "observation", status: "known", confidence: "high" })], resolvedIds: new Set() });
|
||||
expect(vm.known.items.length).toBe(1);
|
||||
});
|
||||
|
||||
it("allows genuine questions through as investigating", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Were both percentages calculated from comparable baseline counts?", { kind: "unknown" })], resolvedIds: new Set() });
|
||||
expect(vm.investigating.items.length).toBe(1);
|
||||
});
|
||||
|
||||
it("allows genuine explanations through as possible explanations", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [mkN("n1", "Higher production volume may explain higher complaint totals", { kind: "assumption", confidence: "medium" })], resolvedIds: new Set() });
|
||||
expect(vm.explanations.items.length).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Semantic classification tests ───────────────────────────────── */
|
||||
|
||||
describe("Semantic classification", () => {
|
||||
it("routes concrete observations regardless of kind=state", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("s1", "Complaint rate per unit fell from 10 to 9.6 per 1,000", { kind: "state" })],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
// Even though kind=state, the concrete numbers make it an observation
|
||||
expect(vm.known.items.length).toBe(0); // state is unresolved, goes to investigating
|
||||
expect(vm.investigating.items.some(i => i.includes("10") && i.includes("9.6"))).toBe(true);
|
||||
});
|
||||
|
||||
it("suppresses scaffolding state nodes with vague text", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("s1", "Current situation describes the problem context", { kind: "state" })],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.known.items.length).toBe(0); // scaffolding suppressed entirely
|
||||
expect(vm.investigating.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("routes resolved unknowns as known when they are factual", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("u1", "Both figures cover the same three-month period", { kind: "unknown", status: "resolved", confidence: "high" })],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.known.hasItems).toBe(true);
|
||||
});
|
||||
|
||||
it("routes relationship nodes through when they add factual content", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("r1", "Complaint and production trends are related", { kind: "relationship" })],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
// Relationships pass filterItem but may be classified as scaffolding depending on text
|
||||
// The current behavior depends on whether the text passes all filters
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Deduplication tests ─────────────────────────────────────────── */
|
||||
|
||||
describe("Deduplication", () => {
|
||||
it("merges duplicate observations across the same section", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("o1", "Complaints increased by 35%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("o2", "complaints increased by 35%", { kind: "observation", status: "known", confidence: "low" })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
// First occurrence wins; the two normalised texts are identical
|
||||
const complaintItems = vm.known.items.filter(i => i.includes("Complaints increased"));
|
||||
expect(complaintItems.length).toBe(1);
|
||||
});
|
||||
|
||||
it("cross-deduplicates: known vs investigating — uncertainty wins", () => {
|
||||
const resolvedIds = new Set(["u1"]);
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("o1", "Both figures cover the same period", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("u1", "both figures cover the same period", { kind: "unknown", status: "resolved", confidence: "high" })
|
||||
],
|
||||
resolvedIds,
|
||||
activeUnknownNodeId: null
|
||||
});
|
||||
// The unresolved one (if any) or the cross-dedup should prevent duplication
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Ranking tests ───────────────────────────────────────────────── */
|
||||
|
||||
describe("Ranking", () => {
|
||||
it("ranks observations first in known section", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("n1", "System overview description", { kind: "state", status: "resolved" }),
|
||||
mkN("o1", "Revenue increased by 15%", { kind: "observation", status: "known", confidence: "high" })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.known.items[0]).toContain("Revenue");
|
||||
});
|
||||
|
||||
it("ranks active unknown first in investigating section", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("u1", "second question?", { kind: "unknown" }),
|
||||
mkN("u2", "first question?", { kind: "unknown" })
|
||||
],
|
||||
resolvedIds: new Set(),
|
||||
activeUnknownNodeId: "u2"
|
||||
});
|
||||
expect(vm.investigating.items[0]).toContain("first question");
|
||||
});
|
||||
|
||||
it("ranks explanations with evidence higher", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("a1", "explanation without evidence", { kind: "assumption", confidence: "medium", evidenceIds: [] }),
|
||||
mkN("a2", "explanation with evidence", { kind: "assumption", confidence: "medium", evidenceIds: ["e1"] })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.explanations.items[0].text).toContain("with evidence");
|
||||
});
|
||||
|
||||
it("prefers high-confidence items over low in known section", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("o1", "low confidence fact", { kind: "observation", status: "known", confidence: "low" }),
|
||||
mkN("o2", "high confidence fact", { kind: "observation", status: "known", confidence: "high" })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.known.items[0]).toContain("high confidence");
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Section framing tests ───────────────────────────────────────── */
|
||||
|
||||
describe("Section framing", () => {
|
||||
it("uses terminal title when no active question", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("o1", "Revenue increased by 15%", { kind: "observation", status: "known", confidence: "high" })],
|
||||
resolvedIds: new Set(["u1"]),
|
||||
activeUnknownNodeId: null,
|
||||
selectedQuestion: null
|
||||
});
|
||||
expect(vm.known.title).toBe("What the evidence supports");
|
||||
});
|
||||
|
||||
it("uses standard title during active investigation", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("o1", "Revenue increased by 15%", { kind: "observation", status: "known", confidence: "high" })],
|
||||
resolvedIds: new Set(),
|
||||
activeUnknownNodeId: "u1",
|
||||
selectedQuestion: { nodeId: "u1", question: "What caused this?" }
|
||||
});
|
||||
expect(vm.known.title).toBe("What we know");
|
||||
});
|
||||
|
||||
it("changes investigating title to 'Remaining cautions' in terminal state with unresolved items", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("o1", "Revenue increased by 15%", { kind: "observation", status: "known", confidence: "high" })],
|
||||
resolvedIds: new Set(),
|
||||
activeUnknownNodeId: null,
|
||||
selectedQuestion: null
|
||||
});
|
||||
// No unresolved unknowns → investigating section should be omitted
|
||||
expect(vm.investigating.shouldOmit).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Mock data integration tests (Complete investigation scenario) ─ */
|
||||
|
||||
describe("Mock data integration — Complete investigation", () => {
|
||||
it("turn 0: shows two observations, one investigating unknown; suppresses scaffolding state", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("obs-1", "Complaints increased by 35%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-2", "Production increased by 40%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("state-1", "Current situation", { kind: "state", status: "provisional", confidence: "medium" }),
|
||||
mkN("u-1", "Whether the two figures cover the same period")
|
||||
],
|
||||
resolvedIds: new Set(),
|
||||
activeUnknownNodeId: "u-1",
|
||||
selectedQuestion: { nodeId: "u-1", question: "Were the complaint and production figures measured over the same period?" }
|
||||
});
|
||||
|
||||
// Two known observations, no scaffolding leaked in
|
||||
expect(vm.known.hasItems).toBe(true);
|
||||
expect(vm.known.items.length).toBeGreaterThan(0);
|
||||
vm.known.items.forEach(item => {
|
||||
expect(item).not.toContain("Current situation");
|
||||
expect(item.toLowerCase()).not.toContain("summary of scenario");
|
||||
});
|
||||
|
||||
// One investigating unknown
|
||||
expect(vm.investigating.hasItems).toBe(true);
|
||||
});
|
||||
|
||||
it("turn 5 (complete): shows only factual observations, suppresses all scaffolding", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("obs-1", "Complaints increased by 35%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-2", "Production increased by 40%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-3", "Both figures cover the same three-month period", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-4", "Complaints rose from 100 to 135; production rose from 1,000 to 1,400 units", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-5", "The complaint rate fell from 10 per 1,000 to about 9.6 per 1,000", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-6", "Same complaint categories and reporting rules were used throughout", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("state-1", "Current situation", { kind: "state", status: "provisional", confidence: "medium" }),
|
||||
mkN("rel-1", "Complaint and production trends are related", { kind: "relationship", status: "known", confidence: "medium" })
|
||||
],
|
||||
resolvedIds: new Set(["u-1", "u-2", "u-3", "u-4"]),
|
||||
activeUnknownNodeId: null,
|
||||
selectedQuestion: null
|
||||
});
|
||||
|
||||
// All known items should be facts/observations — no scaffolding leaked in
|
||||
vm.known.items.forEach(item => {
|
||||
expect(item.toLowerCase()).not.toContain("current situation");
|
||||
expect(item).not.toContain("node");
|
||||
expect(item).not.toMatch(/summary/i);
|
||||
});
|
||||
|
||||
// Terminal state framing
|
||||
expect(vm.known.title).toBe("What the evidence supports");
|
||||
expect(vm.explanations.items.length).toBe(0);
|
||||
});
|
||||
|
||||
it("turn 2: shows three known + one investigating", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("obs-1", "Complaints increased by 35%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-2", "Production increased by 40%", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-3", "Both figures cover the same three-month period", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-4", "Complaints rose from 100 to 135; production rose from 1,000 to 1,400 units", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("state-1", "Current situation", { kind: "state", status: "provisional", confidence: "medium" }),
|
||||
mkN("rel-1", "Complaint and production trends are related", { kind: "relationship", status: "known", confidence: "medium" }),
|
||||
mkN("u-3", "Whether complaints increased faster than production on a per-unit basis")
|
||||
],
|
||||
resolvedIds: new Set(["u-1", "u-2"]),
|
||||
activeUnknownNodeId: "u-3",
|
||||
selectedQuestion: { nodeId: "u-3", question: "Did the complaint rate per unit produced improve or worsen?" }
|
||||
});
|
||||
|
||||
// Known should have observations, not scaffolding
|
||||
expect(vm.known.hasItems).toBe(true);
|
||||
vm.known.items.forEach(item => {
|
||||
expect(item).not.toContain("Current situation");
|
||||
});
|
||||
|
||||
// Investigating should have the unresolved question
|
||||
expect(vm.investigating.hasItems).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Mock data integration tests (Long investigation scenario) ───── */
|
||||
|
||||
describe("Mock data integration — Long investigation", () => {
|
||||
it("turn 0: shows current revenue as known observation, suppresses state scaffolding", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("obs-1", "Current revenue is $2M ARR in the US market only", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("state-1", "Evaluating European market entry", { kind: "state", status: "provisional", confidence: "medium" }),
|
||||
mkN("u-1", "Whether there is genuine demand for our category in Europe")
|
||||
],
|
||||
resolvedIds: new Set(),
|
||||
activeUnknownNodeId: "u-1",
|
||||
selectedQuestion: { nodeId: "u-1", question: "How large and mature is the analytics SaaS market in Europe?" }
|
||||
});
|
||||
|
||||
expect(vm.known.hasItems).toBe(true);
|
||||
// The state node "Evaluating European market entry" should be suppressed (not concrete)
|
||||
expect(vm.known.items.some(i => i.toLowerCase().includes("evaluating"))).toBe(false);
|
||||
});
|
||||
|
||||
it("turn 4 (complete): shows only factual observations", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("obs-1", "Current revenue is $2M ARR in the US market only", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-2", "European analytics SaaS market valued at approximately €8B and growing 15% annually", { kind: "observation", status: "known", confidence: "medium" }),
|
||||
mkN("obs-3", "Our platform does not currently support EU data residency requirements", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("obs-4", "Achieving compliance would require approximately 6 months and $500K engineering investment", { kind: "observation", status: "known", confidence: "medium" }),
|
||||
mkN("obs-5", "Our real-time collaboration feature has no direct European equivalent and aligns with EU procurement trends", { kind: "observation", status: "provisional", confidence: "medium" }),
|
||||
mkN("state-1", "Evaluating European market entry", { kind: "state", status: "provisional", confidence: "medium" })
|
||||
],
|
||||
resolvedIds: new Set(["u-1", "u-2", "u-3", "u-4"]),
|
||||
activeUnknownNodeId: null,
|
||||
selectedQuestion: null
|
||||
});
|
||||
|
||||
expect(vm.known.title).toBe("What the evidence supports");
|
||||
// State scaffolding suppressed
|
||||
vm.known.items.forEach(item => {
|
||||
expect(item.toLowerCase()).not.toContain("evaluating european");
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
/* ── Edge cases ──────────────────────────────────────────────────── */
|
||||
|
||||
describe("Edge cases", () => {
|
||||
it("handles empty nodes gracefully", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [], resolvedIds: new Set() });
|
||||
expect(vm.known.hasItems).toBe(false);
|
||||
expect(vm.investigating.hasItems).toBe(false);
|
||||
expect(vm.explanations.hasItems).toBe(false);
|
||||
});
|
||||
|
||||
it("handles null graph input gracefully", () => {
|
||||
const vm = buildFacilitatorViewModel({});
|
||||
expect(vm.known.hasItems).toBe(false);
|
||||
});
|
||||
|
||||
it("limits known items to 4 maximum", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("o1", "Fact one", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("o2", "Fact two", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("o3", "Fact three", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("o4", "Fact four", { kind: "observation", status: "known", confidence: "high" }),
|
||||
mkN("o5", "Fact five should be trimmed", { kind: "observation", status: "known", confidence: "high" })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.known.items.length).toBeLessThanOrEqual(4);
|
||||
});
|
||||
|
||||
it("limits investigating items to 4 maximum", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("u1", "Question one?", { kind: "unknown" }),
|
||||
mkN("u2", "Question two?", { kind: "unknown" }),
|
||||
mkN("u3", "Question three?", { kind: "unknown" }),
|
||||
mkN("u4", "Question four?", { kind: "unknown" }),
|
||||
mkN("u5", "Question five should be trimmed", { kind: "unknown" })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.investigating.items.length).toBeLessThanOrEqual(4);
|
||||
});
|
||||
|
||||
it("limits explanation items to 3 maximum", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [
|
||||
mkN("a1", "Explanation one", { kind: "assumption" }),
|
||||
mkN("a2", "Explanation two", { kind: "assumption" }),
|
||||
mkN("a3", "Explanation three", { kind: "assumption" }),
|
||||
mkN("a4", "Explanation four should be trimmed", { kind: "assumption" })
|
||||
],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.explanations.items.length).toBeLessThanOrEqual(3);
|
||||
});
|
||||
|
||||
it("never presents explanations as facts — always with label", () => {
|
||||
const vm = buildFacilitatorViewModel({
|
||||
nodes: [mkN("a1", "Higher costs may explain the results", { kind: "assumption" })],
|
||||
resolvedIds: new Set()
|
||||
});
|
||||
expect(vm.explanations.hasItems).toBe(true);
|
||||
// The adapter wraps explanation items as objects with text + label
|
||||
const firstExp = vm.explanations.items[0];
|
||||
if (typeof firstExp === "object") {
|
||||
expect(firstExp.label).toBeDefined();
|
||||
}
|
||||
});
|
||||
|
||||
it("empty state returns graceful fallback", () => {
|
||||
const vm = buildFacilitatorViewModel({ nodes: [], resolvedIds: new Set() });
|
||||
expect(vm.known.title).toBe("What we know");
|
||||
expect(vm.summary.text).toBeNull();
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user