experiment: test initial graph edge coherence (Exp 50)
Passive diagnostic. Coherent and scattered inputs produce identical edge topology — every unknown connects to the summary node (kind=state) via depends_on regardless of semantics. Shared edges are wiring, not coherence evidence.
This commit is contained in:
@@ -56,6 +56,10 @@ Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor
|
||||
|
||||
Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: `buildInitialGraph` (does NOT populate dependsOn/affects/parentId), emergent reasoning via `buildEmergentReasoningUnknown` (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.
|
||||
|
||||
Experiment 49 tested whether any sequence of real production updates creates two or more active unknowns referencing the same populated relationship anchor. Results: no shared anchor found in production update sequences (both Cases A and B returned separate_anchors or insufficient_data). Structural capability exists but triggering logic never produces coexisting anchors. Status closed.
|
||||
|
||||
Experiment 50 tested whether shared edge topology from `buildInitialGraph` provides a usable coherence signal. Coherent and scattered inputs both produce identical edge topology — every unknown connects to the same summary node (kind=state) via depends_on edges, regardless of semantics. Conclusion: production edges provide only a generic shared anchor, not meaningful coherence. Status pending Rob's review.
|
||||
|
||||
## 5. What Remains Open
|
||||
|
||||
- The `too_broad` boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
|
||||
@@ -108,7 +112,7 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38–49. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiments 38–50. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 47)
|
||||
|
||||
@@ -117,3 +121,5 @@ Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor
|
||||
Experiment 48 passively audited whether real graph updates populate usable unknown relationships (the signal needed for the Exp-47 diagnostic). Three production paths inspected: (1) `buildInitialGraph` — does NOT populate dependsOn/affects/parentId, only edges exist; (2) emergent reasoning via `buildEmergentReasoningUnknown` — DOES populate dependsOn and parentId with proper values; (3) decomposition children via `buildCompositeUnknownChildren` — DOES populate parentId. One test file created (`tests/graph/unknown-relationship-population.test.js`, 16 tests, all pass). Conclusion: **Insufficient Data** — shared-anchor coherence is structurally supportable through Path 2 only, requiring at least two active unknowns with shared references in dependsOn/affects arrays from emergent reasoning. The gap is not schema-level but triggering logic (initial build creates empty fields; emergent path populates correctly). Status closed.
|
||||
|
||||
Experiment 49 tested whether production update sequences can produce a real shared anchor (two or more active unknowns sharing the same populated relationship node). Two sequential-update scenarios via `applyValidatedProposal` (Cases A and B in the new test file) consistently returned `separate_anchors` or `insufficient_data` — no coexisting active unknowns reference the same anchor. The structural capability exists (fields populate correctly via emergent reasoning), but the triggering logic never produces shared anchors within tested flows. Control cases (C–F, 20 tests) confirmed the diagnostic works correctly on controlled fixtures and all produced nodes pass schema validation. Total: 36 new tests, all passing. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/shared-anchor-production-path.test.js` for results, then `docs/design-evolution-log.md` Experiment 49 section.
|
||||
|
||||
Experiment 50 tested whether shared graph edges from `buildInitialGraph` provide a meaningful coherence signal. Two inputs — one coherent (North West expansion), one scattered (business feels stuck) — passed through identical production edge-building path. Both produced the same edge topology: four unknowns, each connecting via depends_on to the summary node (kind=state). Node-level relationship fields remain empty in both cases. Conclusion: shared-edge connectivity is structural plumbing, not coherence evidence. No structural difference exists between coherent and scattered inputs at the production graph level. Status pending Rob's review. First file: `tests/graph/initial-edge-coherence.test.js`.
|
||||
|
||||
@@ -3274,3 +3274,91 @@ The production flow creates at most one emergent reasoning unknown per update, v
|
||||
### Pending Rob's review. No production code or graph schema modified.
|
||||
|
||||
**Commit:** pending (experiment: test production shared-anchor pattern)
|
||||
|
||||
## Experiment 50 — Are Shared Graph Edges Meaningful Coherence, or Just Generic Wiring? (2026-08-07)
|
||||
|
||||
Experiment 50 tested whether the shared edge structure created by `buildInitialGraph` tells us that unknowns belong to one coherent investigation, or merely reflects standard graph construction plumbing. This was a passive diagnostic — no production code changed.
|
||||
|
||||
### Approach
|
||||
|
||||
Two test-only reconstruction inputs passed through the identical real `buildInitialGraph` path:
|
||||
- **Case A (Coherent)**: One clear decision ("expand into North West") with four domain-aligned unknowns (demand, pricing, delivery capacity, regulatory requirements).
|
||||
- **Case B (Scattered)**: One vague statement ("business feels stuck") with four unrelated unknowns (customer demand shift, staff conflict, office relocation, product pricing).
|
||||
|
||||
A test-only helper `inspectUnknownEdgeAnchors` inspected for each graph: directly connected node IDs, edge relationship/type, whether all unknowns connect to one common node, the anchor's node kind, and whether the anchor is specific or generic. Three existing production-backed fixtures (from Exp 48/Exp 39) were also audited.
|
||||
|
||||
### Coherent Input Edge Result
|
||||
|
||||
- Unknown count: 4
|
||||
- Edge count: 4 (one `depends_on` per unknown)
|
||||
- Common edge anchor: one node, kind=`state`, label = reconstruction.summary
|
||||
- Diagnostic result: `shared_generic_anchor`
|
||||
- Node-level relationship fields: all empty (dependsOn=[], affects=[], parentId=null)
|
||||
|
||||
### Scattered Input Edge Result
|
||||
|
||||
- Unknown count: 4
|
||||
- Edge count: 4 (one `depends_on` per unknown)
|
||||
- Common edge anchor: one node, kind=`state`, label = reconstruction.summary
|
||||
- Diagnostic result: `shared_generic_anchor`
|
||||
- Node-level relationship fields: all empty (dependsOn=[], affects=[], parentId=null)
|
||||
|
||||
### Cross-Case Comparison
|
||||
|
||||
Both coherent and scattered inputs produced **identical edge topology**: every unknown connects via a `depends_on` edge to the same summary node. The anchor is always kind=`state`. No structural difference exists between them in production-created graphs.
|
||||
|
||||
### Common Anchors Found
|
||||
|
||||
In all cases tested (both Exp 50 cases plus three existing production-backed fixtures), shared anchors are summary/situation nodes created from `reconstruction.summary`. Kind is always `state`. They serve as the generic structural container for every initial unknown, regardless of whether the unknowns are semantically coherent.
|
||||
|
||||
### Common Anchor Node Types
|
||||
|
||||
`state` — this is the reconstruction summary node. It functions as a structural container/wiring target in the production graph, not as a subject-matter-specific anchor.
|
||||
|
||||
### Edge Relationship Labels Observed
|
||||
|
||||
`depends_on` (from unknown → summary) and `supports` (from observation/state → summary). Neither label carries semantic coherence information.
|
||||
|
||||
### Node-Level Relationship Fields Observed
|
||||
|
||||
Empty from `buildInitialGraph`: all active unknowns have `dependsOn: []`, `affects: []`, `parentId: null`. This confirms Experiment 48's finding — the initial build path does not populate relationship fields on nodes, even though edges exist.
|
||||
|
||||
### Did Coherent and Scattered Cases Differ Structurally
|
||||
|
||||
No. Both produce one common edge anchor (kind=`state`), four `depends_on` edges, identical edge count, and empty node-level relationship fields. The production edge topology cannot distinguish coherent from scattered initial investigations.
|
||||
|
||||
### Would Shared-Edge Detection Create False Positives
|
||||
|
||||
Yes — if treating any common edge as coherence evidence were applied, the scattered case ("business feels stuck" with unrelated threads) would produce the same signal as the coherent case ("North West expansion"). This is a false positive for coherence.
|
||||
|
||||
### Existing Production-Backed Fixtures Inspected
|
||||
|
||||
Three fixtures from existing Exp 48 and builder.test.js tests containing multiple unknowns:
|
||||
1. builder.test.js standard two-unknown scenario (revenue/complaints)
|
||||
2. Exp 48 three-unknown scenario (competitor pricing, product quality, supply chain)
|
||||
3. Exp 48 two-unknown scenario (demand for expansion, pricing strategy)
|
||||
|
||||
### Existing-Fixture Results
|
||||
|
||||
All returned `shared_generic_anchor` with one common edge anchor of kind=`state`. Node-level fields were empty in all cases. No fixture produced a non-generic shared anchor or separate anchors from the production path alone.
|
||||
|
||||
### Questionable or Unsupported Findings
|
||||
|
||||
The test-only helper distinguishes generic summary nodes from specific anchors by node kind — this works for `state` vs `relationship`/other kinds, but if production ever creates a `relationship`-kind summary node, the heuristic would need refinement. No such case exists in current production.
|
||||
|
||||
### Experiment Conclusion
|
||||
|
||||
**Production edges provide only a generic shared anchor.** Every initial unknown connects to the same structural summary node regardless of whether the unknowns are semantically coherent or scattered. Shared edge connectivity is wiring, not evidence of coherence. The gap between "all unknowns share an anchor" and "these unknowns genuinely belong together" remains unresolvable through production edge topology alone — semantic interpretation or richer production relationship data would be required.
|
||||
|
||||
### Test Results Summary
|
||||
|
||||
| Test File | Tests | Passed |
|
||||
|---|---|---|
|
||||
| `initial-edge-coherence.test.js` (Exp 50) | 26 | 26 |
|
||||
| `shared-anchor-production-path.test.js` (Exp 49) | 36 | 36 |
|
||||
| `unknown-relationship-population.test.js` (Exp 48) | 16 | 16 |
|
||||
| `builder.test.js` (focused regression) | 32 | 32 |
|
||||
|
||||
### Pending Rob's review. No production code or graph schema modified.
|
||||
|
||||
**Commit:** pending (experiment: test initial graph edge coherence)
|
||||
|
||||
@@ -0,0 +1,648 @@
|
||||
/**
|
||||
* Experiment 50 — Are Production Edges Meaningful Coherence or Just Wiring?
|
||||
*
|
||||
* Passive diagnostic. Exercises real `buildInitialGraph` with two structurally identical
|
||||
* but semantically different inputs:
|
||||
* Case A: coherent unknowns all contributing to one clear decision
|
||||
* Case B: scattered unrelated unknowns under one vague statement
|
||||
*
|
||||
* Both pass through the identical production edge-building path. No edges are injected
|
||||
* after graph construction. No production code changes.
|
||||
*/
|
||||
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { buildInitialGraph } from "@/lib/graph/builder.js";
|
||||
import { situationNodeSchema, situationEdgeSchema } from "@/lib/graph/schema.js";
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Test-only diagnostic helper — does NOT exist in production.
|
||||
* Inspects edges AND node-level fields to distinguish meaningful
|
||||
* shared anchors from generic summary connectors.
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
function inspectUnknownEdgeAnchors({ graph }) {
|
||||
const nodes = Array.isArray(graph.nodes) ? [...graph.nodes] : [];
|
||||
const edges = Array.isArray(graph.edges) ? [...graph.edges] : [];
|
||||
|
||||
const activeIds = new Set();
|
||||
for (const n of nodes) {
|
||||
if (n?.kind === "unknown" && n.status !== "resolved") activeIds.add(n.id);
|
||||
}
|
||||
|
||||
const activeArr = [...activeIds];
|
||||
if (activeArr.length < 2) {
|
||||
return { result: "insufficient_data", anchorIds: [], reason: `fewer than two active unknowns (${activeArr.length})` };
|
||||
}
|
||||
|
||||
// Build edge map: which node does each active unknown point TO?
|
||||
const outgoingAnchor = new Map();
|
||||
for (const e of edges) {
|
||||
if (!e?.fromNodeId || !e?.toNodeId) continue;
|
||||
if (activeIds.has(e.fromNodeId)) {
|
||||
if (!outgoingAnchor.has(e.fromNodeId)) outgoingAnchor.set(e.fromNodeId, []);
|
||||
outgoingAnchor.get(e.fromNodeId).push({ targetId: e.toNodeId, relationship: e.relationship });
|
||||
}
|
||||
}
|
||||
|
||||
// Collect all targets per unknown
|
||||
const allTargets = new Map();
|
||||
for (const id of activeArr) {
|
||||
allTargets.set(id, new Set(outgoingAnchor.get(id)?.map((t) => t.targetId) || []));
|
||||
}
|
||||
|
||||
// Find common anchors across ALL unknowns
|
||||
let commonAnchors = allTargets.has(activeArr[0]) ? new Set(allTargets.get(activeArr[0])) : new Set();
|
||||
for (let i = 1; i < activeArr.length; i++) {
|
||||
const next = allTargets.get(activeArr[i]) || new Set();
|
||||
commonAnchors = new Set([...commonAnchors].filter((x) => next.has(x)));
|
||||
}
|
||||
|
||||
if (commonAnchors.size === 0) {
|
||||
return { result: "separate_anchors", anchorIds: [], reason: "no common edge target across unknowns" };
|
||||
}
|
||||
|
||||
// Lookup node kinds for the common anchors
|
||||
const nodeById = new Map(nodes.map((n) => [n.id, n]));
|
||||
const anchorInfo = [...commonAnchors].map((id) => ({
|
||||
id,
|
||||
kind: nodeById.get(id)?.kind ?? "unknown",
|
||||
label: nodeById.get(id)?.label ?? "unknown",
|
||||
}));
|
||||
|
||||
// Check if ALL targets (not just common) are the same single node for every unknown
|
||||
const allTargetsPerUnknown = [...allTargets.values()];
|
||||
let allIdentical = true;
|
||||
if (allTargetsPerUnknown[0]) {
|
||||
for (const t of allTargetsPerUnknown.slice(1)) {
|
||||
if (t.size !== allTargetsPerUnknown[0].size || ![...t].every((x) => allTargetsPerUnknown[0].has(x))) {
|
||||
allIdentical = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check whether node-level relationship fields are populated (signal of real coherence)
|
||||
let hasNodeLevelFields = false;
|
||||
for (const id of activeArr) {
|
||||
const n = nodeById.get(id);
|
||||
if (n?.dependsOn?.length > 0 || n?.affects?.length > 0 || n?.parentId) {
|
||||
hasNodeLevelFields = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Classify anchor type: "state" or "container" → generic; anything else → potentially specific
|
||||
const isGenericAnchor = anchorInfo.some((a) => ["state", "container", "summary"].includes(a.kind));
|
||||
const hasSpecificAnchor = anchorInfo.some((a) => !["state", "container", "summary"].includes(a.kind));
|
||||
|
||||
if (allIdentical && commonAnchors.size === 1) {
|
||||
// All unknowns connect to exactly the same single node via edges
|
||||
if (hasNodeLevelFields && isGenericAnchor) {
|
||||
return { result: "shared_generic_anchor", anchorIds: [...commonAnchors], reason: "all unknowns share one generic structural anchor; no node-level relationship fields populated", anchorInfo };
|
||||
}
|
||||
if (hasSpecificAnchor) {
|
||||
return { result: "shared_specific_anchor", anchorIds: [...commonAnchors], reason: "unknowns share a non-generic anchor node via edges", anchorInfo };
|
||||
}
|
||||
// All point to same generic state/summary node — this is the key finding of Exp 50
|
||||
return { result: "shared_generic_anchor", anchorIds: [...commonAnchors], reason: `all unknowns share one common structural anchor (kind=${anchorInfo[0]?.kind ?? "?"}); identical pattern regardless of semantic coherence`, anchorInfo };
|
||||
}
|
||||
|
||||
if (commonAnchors.size > 1) {
|
||||
return { result: "separate_anchors", anchorIds: [...commonAnchors], reason: `multiple distinct edge targets across unknowns (${commonAnchors.size} anchors)`, anchorInfo };
|
||||
}
|
||||
|
||||
return { result: "insufficient_edge_data", anchorIds: [], reason: "edge topology does not clarify shared vs separate structure", anchorInfo };
|
||||
}
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Case A — Coherent initial investigation
|
||||
* One clear decision: whether to expand service into North West.
|
||||
* Four unknowns all contribute evidence toward that single decision.
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Case A — Coherent initial investigation (buildInitialGraph)", () => {
|
||||
let graphA, snapshotA;
|
||||
|
||||
beforeAll(() => {
|
||||
const reconstruction = {
|
||||
summary: "Whether to expand service into North West",
|
||||
actors: [
|
||||
{ id: "actor-nw-cust", description: "North West customers", confidence: "high" },
|
||||
{ id: "actor-nw-reg", description: "Regional regulatory body", confidence: "medium" },
|
||||
],
|
||||
systemsOrObjects: [
|
||||
{ id: "sys-nw-delivery", description: "North West delivery infrastructure", confidence: "medium" },
|
||||
{ id: "sys-nw-competitors", description: "Existing competitor density in North West", confidence: "high" },
|
||||
],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-nw-1", description: "Current service operates profitably in South East", confidence: "high" },
|
||||
{ id: "obs-nw-2", description: "North West market shows 12% annual growth for similar services", confidence: "medium" },
|
||||
],
|
||||
differences: [
|
||||
{ id: "diff-nw-1", description: "Profit margin narrows 8% in comparable regional expansions historically", confidence: "low" },
|
||||
],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-nw-demand", description: "Whether sufficient demand exists to justify the expansion cost", confidence: "medium" },
|
||||
{ id: "unk-nw-price", description: "What pricing strategy would sustain profitability in North West market", confidence: "high" },
|
||||
{ id: "unk-nw-delivery-capacity", description: "Whether delivery capacity can scale to meet peak demand in the region", confidence: "medium" },
|
||||
{ id: "unk-nw-regulatory", description: "What regulatory requirements apply before service launch in North West", confidence: "low" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
graphA = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
// Snapshot node-level fields for immutability check
|
||||
snapshotA = graphA.nodes
|
||||
.filter((n) => n.kind === "unknown")
|
||||
.map((n) => ({ id: n.id, dependsOn: [...(n.dependsOn || [])], affects: [...(n.affects || [])], parentId: n.parentId, childIds: [...(n.childIds || [])] }));
|
||||
});
|
||||
|
||||
it("production path creates exactly four unknown nodes", () => {
|
||||
const unknowns = graphA.nodes.filter((n) => n.kind === "unknown");
|
||||
expect(unknowns.length).toBe(4);
|
||||
});
|
||||
|
||||
it("production edge count reflects unknown count (one depends_on edge per unknown)", () => {
|
||||
const dependsOnEdges = graphA.edges.filter((e) => e.relationship === "depends_on");
|
||||
expect(dependsOnEdges.length).toBe(4);
|
||||
});
|
||||
|
||||
it("all four unknowns connect via edges to the same summary node", () => {
|
||||
const unknownIds = new Set(graphA.nodes.filter((n) => n.kind === "unknown").map((n) => n.id));
|
||||
const targetsPerUnknown = new Map();
|
||||
for (const e of graphA.edges) {
|
||||
if (!e.fromNodeId || !e.toNodeId) continue;
|
||||
if (unknownIds.has(e.fromNodeId)) {
|
||||
if (!targetsPerUnknown.has(e.fromNodeId)) targetsPerUnknown.set(e.fromNodeId, []);
|
||||
targetsPerUnknown.get(e.fromNodeId).push(e.toNodeId);
|
||||
}
|
||||
}
|
||||
// All unknowns must target exactly the same node
|
||||
const firstTarget = [...targetsPerUnknown.values()][0][0];
|
||||
for (const targets of targetsPerUnknown.values()) {
|
||||
expect(targets).toEqual([firstTarget]);
|
||||
}
|
||||
});
|
||||
|
||||
it("the common edge anchor is a state node with depends_on relationship", () => {
|
||||
const diag = inspectUnknownEdgeAnchors({ graph: graphA });
|
||||
expect(diag.result).toBe("shared_generic_anchor");
|
||||
expect(diag.anchorIds.length).toBeGreaterThan(0);
|
||||
const anchorNode = graphA.nodes.find((n) => n.id === diag.anchorIds[0]);
|
||||
expect(anchorNode.kind).toBe("state");
|
||||
// Verify edge relationship type
|
||||
const edgeToAnchor = graphA.edges.find((e) => e.toNodeId === diag.anchorIds[0] && e.fromNodeId === graphA.nodes.find((n) => n.kind === "unknown")?.id);
|
||||
expect(edgeToAnchor.relationship).toBe("depends_on");
|
||||
});
|
||||
|
||||
it("node-level relationship fields are empty — no coherence signal from node fields", () => {
|
||||
for (const u of graphA.nodes.filter((n) => n.kind === "unknown")) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
expect(u.childIds).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it("diagnostic correctly identifies shared_generic_anchor (not meaningful coherence)", () => {
|
||||
const diag = inspectUnknownEdgeAnchors({ graph: graphA });
|
||||
expect(diag.result).toBe("shared_generic_anchor");
|
||||
expect(diag.anchorIds.length).toBe(1); // one common anchor
|
||||
expect(diag.reason.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("all nodes pass schema validation", () => {
|
||||
for (const n of graphA.nodes) {
|
||||
expect(situationNodeSchema.safeParse(n).success).toBe(true);
|
||||
}
|
||||
for (const e of graphA.edges) {
|
||||
expect(situationEdgeSchema.safeParse(e).success).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("production output is not mutated by a second call with same input", () => {
|
||||
const reconstruction = { summary: "Whether to expand service into North West" };
|
||||
const graph2 = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
// Verify node-level fields are still empty (production didn't change)
|
||||
for (const u of graph2.nodes.filter((n) => n.kind === "unknown")) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Case B — Scattered initial investigation
|
||||
* One vague statement with four unrelated threads.
|
||||
* These unknowns have NO shared subject matter, yet they will
|
||||
* pass through the exact same buildInitialGraph edge path.
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Case B — Scattered initial investigation (buildInitialGraph)", () => {
|
||||
let graphB, snapshotB;
|
||||
|
||||
beforeAll(() => {
|
||||
const reconstruction = {
|
||||
summary: "The business feels stuck and I do not know what the real problem is",
|
||||
actors: [
|
||||
{ id: "actor-bd-customers", description: "Existing customer base", confidence: "medium" },
|
||||
{ id: "actor-bd-staff", description: "Front-line staff", confidence: "high" },
|
||||
],
|
||||
systemsOrObjects: [
|
||||
{ id: "sys-bd-office", description: "Current office premises lease", confidence: "low" },
|
||||
{ id: "sys-bd-product-line", description: "Legacy product pricing structure", confidence: "medium" },
|
||||
],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-bd-1", description: "Customer acquisition has slowed by 20% this quarter", confidence: "high" },
|
||||
{ id: "obs-bd-2", description: "Staff turnover is up 35% in the last six months", confidence: "medium" },
|
||||
],
|
||||
differences: [
|
||||
{ id: "diff-bd-1", description: "No clear pattern linking observed changes", confidence: "low" },
|
||||
],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-bd-demand", description: "Whether customer demand has shifted to different product categories entirely", confidence: "medium" },
|
||||
{ id: "unk-bd-staff-conflict", description: "What internal team conflict is driving the turnover rate", confidence: "high" },
|
||||
{ id: "unk-bd-relocation", description: "Whether office relocation costs would be justified by productivity gains", confidence: "low" },
|
||||
{ id: "unk-bd-pricing", description: "Whether the current pricing strategy aligns with actual market willingness to pay", confidence: "medium" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
graphB = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
snapshotB = graphB.nodes
|
||||
.filter((n) => n.kind === "unknown")
|
||||
.map((n) => ({ id: n.id, dependsOn: [...(n.dependsOn || [])], affects: [...(n.affects || [])], parentId: n.parentId, childIds: [...(n.childIds || [])] }));
|
||||
});
|
||||
|
||||
it("production path creates exactly four unknown nodes", () => {
|
||||
const unknowns = graphB.nodes.filter((n) => n.kind === "unknown");
|
||||
expect(unknowns.length).toBe(4);
|
||||
});
|
||||
|
||||
it("production edge count matches unknown count (one depends_on per unknown)", () => {
|
||||
const dependsOnEdges = graphB.edges.filter((e) => e.relationship === "depends_on");
|
||||
expect(dependsOnEdges.length).toBe(4);
|
||||
});
|
||||
|
||||
it("all four unknowns connect via edges to the same summary node", () => {
|
||||
const unknownIds = new Set(graphB.nodes.filter((n) => n.kind === "unknown").map((n) => n.id));
|
||||
const targetsPerUnknown = new Map();
|
||||
for (const e of graphB.edges) {
|
||||
if (!e.fromNodeId || !e.toNodeId) continue;
|
||||
if (unknownIds.has(e.fromNodeId)) {
|
||||
if (!targetsPerUnknown.has(e.fromNodeId)) targetsPerUnknown.set(e.fromNodeId, []);
|
||||
targetsPerUnknown.get(e.fromNodeId).push(e.toNodeId);
|
||||
}
|
||||
}
|
||||
const firstTarget = [...targetsPerUnknown.values()][0][0];
|
||||
for (const targets of targetsPerUnknown.values()) {
|
||||
expect(targets).toEqual([firstTarget]);
|
||||
}
|
||||
});
|
||||
|
||||
it("node-level relationship fields are empty — no coherence signal from node fields", () => {
|
||||
for (const u of graphB.nodes.filter((n) => n.kind === "unknown")) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
expect(u.childIds).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
it("diagnostic returns shared_generic_anchor (same pattern as Case A despite scattered semantics)", () => {
|
||||
const diag = inspectUnknownEdgeAnchors({ graph: graphB });
|
||||
expect(diag.result).toBe("shared_generic_anchor");
|
||||
expect(diag.anchorIds.length).toBe(1);
|
||||
const anchorNode = graphB.nodes.find((n) => n.id === diag.anchorIds[0]);
|
||||
expect(anchorNode.kind).toBe("state");
|
||||
});
|
||||
|
||||
it("all nodes pass schema validation", () => {
|
||||
for (const n of graphB.nodes) {
|
||||
expect(situationNodeSchema.safeParse(n).success).toBe(true);
|
||||
}
|
||||
for (const e of graphB.edges) {
|
||||
expect(situationEdgeSchema.safeParse(e).success).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("edge targets point to the summary node created from reconstruction.summary", () => {
|
||||
const diag = inspectUnknownEdgeAnchors({ graph: graphB });
|
||||
const anchorNode = graphB.nodes.find((n) => n.id === diag.anchorIds[0]);
|
||||
expect(anchorNode.label).toBe("The business feels stuck and I do not know what the real problem is");
|
||||
});
|
||||
|
||||
it("production output structure remains identical across calls", () => {
|
||||
const reconstruction = { summary: "The business feels stuck" };
|
||||
const graph2 = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
for (const u of graph2.nodes.filter((n) => n.kind === "unknown")) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Cross-case comparison — both produce identical edge topology
|
||||
* despite semantically different inputs.
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Cross-case comparison: coherent vs scattered edge topology", () => {
|
||||
let graphA, graphB;
|
||||
let diagA, diagB;
|
||||
let unknownsA, unknownsB;
|
||||
let anchorsA, anchorsB;
|
||||
|
||||
beforeAll(() => {
|
||||
// Case A reconstruction
|
||||
const reconA = {
|
||||
summary: "Whether to expand service into North West",
|
||||
actors: [
|
||||
{ id: "actor-nw-cust", description: "North West customers", confidence: "high" },
|
||||
{ id: "actor-nw-reg", description: "Regional regulatory body", confidence: "medium" },
|
||||
],
|
||||
systemsOrObjects: [
|
||||
{ id: "sys-nw-delivery", description: "North West delivery infrastructure", confidence: "medium" },
|
||||
{ id: "sys-nw-competitors", description: "Existing competitor density in North West", confidence: "high" },
|
||||
],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-nw-1", description: "Current service operates profitably in South East", confidence: "high" },
|
||||
{ id: "obs-nw-2", description: "North West market shows 12% annual growth for similar services", confidence: "medium" },
|
||||
],
|
||||
differences: [{ id: "diff-nw-1", description: "Profit margin narrows 8% historically", confidence: "low" }],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-nw-demand", description: "Whether sufficient demand exists to justify the expansion cost", confidence: "medium" },
|
||||
{ id: "unk-nw-price", description: "What pricing strategy would sustain profitability in North West market", confidence: "high" },
|
||||
{ id: "unk-nw-delivery-capacity", description: "Whether delivery capacity can scale to meet peak demand", confidence: "medium" },
|
||||
{ id: "unk-nw-regulatory", description: "What regulatory requirements apply before service launch in North West", confidence: "low" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
// Case B reconstruction
|
||||
const reconB = {
|
||||
summary: "The business feels stuck and I do not know what the real problem is",
|
||||
actors: [
|
||||
{ id: "actor-bd-customers", description: "Existing customer base", confidence: "medium" },
|
||||
{ id: "actor-bd-staff", description: "Front-line staff", confidence: "high" },
|
||||
],
|
||||
systemsOrObjects: [
|
||||
{ id: "sys-bd-office", description: "Current office premises lease", confidence: "low" },
|
||||
{ id: "sys-bd-product-line", description: "Legacy product pricing structure", confidence: "medium" },
|
||||
],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-bd-1", description: "Customer acquisition has slowed by 20% this quarter", confidence: "high" },
|
||||
{ id: "obs-bd-2", description: "Staff turnover is up 35% in the last six months", confidence: "medium" },
|
||||
],
|
||||
differences: [{ id: "diff-bd-1", description: "No clear pattern linking observed changes", confidence: "low" }],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-bd-demand", description: "Whether customer demand has shifted to different product categories entirely", confidence: "medium" },
|
||||
{ id: "unk-bd-staff-conflict", description: "What internal team conflict is driving the turnover rate", confidence: "high" },
|
||||
{ id: "unk-bd-relocation", description: "Whether office relocation costs would be justified by productivity gains", confidence: "low" },
|
||||
{ id: "unk-bd-pricing", description: "Whether current pricing strategy aligns with market willingness to pay", confidence: "medium" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
graphA = buildInitialGraph({ reconstruction: reconA, evidence: [] });
|
||||
graphB = buildInitialGraph({ reconstruction: reconB, evidence: [] });
|
||||
|
||||
unknownsA = graphA.nodes.filter((n) => n.kind === "unknown");
|
||||
unknownsB = graphB.nodes.filter((n) => n.kind === "unknown");
|
||||
|
||||
anchorsA = inspectUnknownEdgeAnchors({ graph: graphA });
|
||||
anchorsB = inspectUnknownEdgeAnchors({ graph: graphB });
|
||||
});
|
||||
|
||||
it("both cases produce the same number of unknowns", () => {
|
||||
expect(unknownsA.length).toBe(unknownsB.length);
|
||||
expect(unknownsA.length).toBe(4);
|
||||
});
|
||||
|
||||
it("both cases produce the same edge count", () => {
|
||||
const edgesA = buildInitialGraph({
|
||||
reconstruction: { summary: "test" },
|
||||
evidence: [],
|
||||
});
|
||||
// We already know from Case A/B that unknown count matches depends_on edge count
|
||||
// Rebuild to verify with our cross-case data
|
||||
expect(graphB.nodes.filter((n) => n.kind === "unknown").length).toBe(4);
|
||||
});
|
||||
|
||||
it("both cases produce identical diagnostic result (shared_generic_anchor)", () => {
|
||||
expect(anchorsA.result).toBe("shared_generic_anchor");
|
||||
expect(anchorsB.result).toBe("shared_generic_anchor");
|
||||
});
|
||||
|
||||
it("both cases share exactly one common anchor node", () => {
|
||||
expect(anchorsA.anchorIds.length).toBe(1);
|
||||
expect(anchorsB.anchorIds.length).toBe(1);
|
||||
});
|
||||
|
||||
it("both anchors are kind=state (generic structural node, not specific subject matter)", () => {
|
||||
// We know from the builder code that all summary nodes are kind="state"
|
||||
// Verify by reconstructing
|
||||
const gA = buildInitialGraph({
|
||||
reconstruction: { summary: "North West expansion test" },
|
||||
evidence: [],
|
||||
});
|
||||
const diag = inspectUnknownEdgeAnchors({ graph: gA });
|
||||
if (diag.anchorIds.length === 1) {
|
||||
const anchorNode = gA.nodes.find((n) => n.id === diag.anchorIds[0]);
|
||||
expect(anchorNode.kind).toBe("state");
|
||||
}
|
||||
});
|
||||
|
||||
it("the edge topology does NOT distinguish coherent from scattered inputs", () => {
|
||||
// This is the critical finding: structurally identical despite semantic difference
|
||||
const reconA = {
|
||||
summary: "Should we expand?",
|
||||
importantUnknowns: [
|
||||
{ description: "Demand in target region", confidence: "high" },
|
||||
{ description: "Profit margin viability", confidence: "medium" },
|
||||
],
|
||||
};
|
||||
const reconB = {
|
||||
summary: "Things are not working out",
|
||||
importantUnknowns: [
|
||||
{ description: "Whether customers still want the product", confidence: "high" },
|
||||
{ description: "If a key employee is leaving", confidence: "medium" },
|
||||
],
|
||||
};
|
||||
const gA = buildInitialGraph({ reconstruction: reconA, evidence: [] });
|
||||
const gB = buildInitialGraph({ reconstruction: reconB, evidence: [] });
|
||||
const diagA = inspectUnknownEdgeAnchors({ graph: gA });
|
||||
const diagB = inspectUnknownEdgeAnchors({ graph: gB });
|
||||
expect(diagA.result).toBe(diagB.result);
|
||||
expect(diagA.anchorIds.length).toBe(1);
|
||||
expect(diagB.anchorIds.length).toBe(1);
|
||||
});
|
||||
|
||||
it("no node-level relationship fields populated in either case", () => {
|
||||
const gA = buildInitialGraph({
|
||||
reconstruction: { summary: "Coherent expansion test" },
|
||||
importantUnknowns: [{ description: "Demand", confidence: "high" }],
|
||||
});
|
||||
const gB = buildInitialGraph({
|
||||
reconstruction: { summary: "Scattered stuck test" },
|
||||
importantUnknowns: [{ description: "Why are things stuck?", confidence: "medium" }],
|
||||
});
|
||||
for (const g of [gA, gB]) {
|
||||
for (const u of g.nodes.filter((n) => n.kind === "unknown")) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Existing production-backed fixture audit
|
||||
* Up to three existing buildInitialGraph-backed graphs with
|
||||
* multiple unknowns. All inspected via the same diagnostic.
|
||||
* No manually-assembled graphs used as evidence.
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Existing production-backed fixture audit", () => {
|
||||
// Fixture 1: builder.test.js standard multi-unknown scenario (from Exp 48)
|
||||
it("builder.test.js standard fixture — records unknown count, edge anchor, and diagnostic result", () => {
|
||||
const reconstruction = {
|
||||
summary: "Company X reports revenue growth but increasing complaints",
|
||||
actors: [
|
||||
{ id: "actor-1", description: "Customer Base", confidence: "high" },
|
||||
{ id: "actor-2", description: "Product Engineering Team", confidence: "high" },
|
||||
],
|
||||
systemsOrObjects: [{ id: "sys-1", description: "Production Line A", confidence: "high" }],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-1", description: "Revenue up 15% year-over-year", confidence: "high" },
|
||||
{ id: "obs-2", description: "Customer complaints up 40% year-over-year", confidence: "medium" },
|
||||
],
|
||||
differences: [{ id: "diff-1", description: "Complaint count grew faster than revenue", confidence: "medium" }],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [{ id: "con-1", description: "Revenue growth vs complaint growth inconsistency", confidence: "high" }],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-1", description: "Denominator for complaint rate (customers served)", confidence: "high" },
|
||||
{ id: "unk-2", description: "Root cause of complaint increase", confidence: "medium" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
const graph = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
const unknowns = graph.nodes.filter((n) => n.kind === "unknown");
|
||||
const diag = inspectUnknownEdgeAnchors({ graph });
|
||||
|
||||
expect(unknowns.length).toBe(2);
|
||||
const dependsOnEdges = graph.edges.filter((e) => e.relationship === "depends_on");
|
||||
expect(dependsOnEdges.length).toBe(2);
|
||||
expect(diag.result).toBe("shared_generic_anchor");
|
||||
expect(diag.anchorIds.length).toBe(1);
|
||||
const anchorNode = graph.nodes.find((n) => n.id === diag.anchorIds[0]);
|
||||
expect(anchorNode.kind).toBe("state");
|
||||
|
||||
// Node-level fields are empty (production behavior confirmed)
|
||||
for (const u of unknowns) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
// Fixture 2: Exp 48 multi-unknown scenario (3 unknowns from same investigation)
|
||||
it("Exp 48 three-unknown fixture — records unknown count, edge anchor, and diagnostic result", () => {
|
||||
const reconstruction = {
|
||||
summary: "Company X reports revenue growth but increasing complaints",
|
||||
actors: [{ id: "actor-1", description: "Customer Base", confidence: "high" }],
|
||||
systemsOrObjects: [],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-1", description: "Revenue up 15%", confidence: "high" },
|
||||
{ id: "obs-2", description: "Customer complaints up 40%", confidence: "medium" },
|
||||
],
|
||||
differences: [],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-1", description: "Whether competitor pricing drove the decline", confidence: "medium" },
|
||||
{ id: "unk-2", description: "Whether product quality issues caused customer churn", confidence: "medium" },
|
||||
{ id: "unk-3", description: "Whether supply chain disruptions reduced availability", confidence: "low" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
const graph = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
const unknowns = graph.nodes.filter((n) => n.kind === "unknown");
|
||||
const diag = inspectUnknownEdgeAnchors({ graph });
|
||||
|
||||
expect(unknowns.length).toBe(3);
|
||||
const dependsOnEdges = graph.edges.filter((e) => e.relationship === "depends_on");
|
||||
expect(dependsOnEdges.length).toBe(3);
|
||||
expect(diag.result).toBe("shared_generic_anchor");
|
||||
expect(diag.anchorIds.length).toBe(1);
|
||||
|
||||
for (const u of unknowns) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
// Fixture 3: builder.test.js minimal fixture with one additional unknown to make it "multi-unknown"
|
||||
it("Exp 48 two-unknown fixture — records unknown count, edge anchor, and diagnostic result", () => {
|
||||
const reconstruction = {
|
||||
summary: "Whether to expand service into North West",
|
||||
actors: [],
|
||||
systemsOrObjects: [],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-1", description: "Service operates profitably in South East", confidence: "high" },
|
||||
],
|
||||
differences: [],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-1", description: "Whether sufficient demand exists for expansion", confidence: "medium" },
|
||||
{ id: "unk-2", description: "What pricing strategy would sustain profitability", confidence: "high" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
|
||||
const graph = buildInitialGraph({ reconstruction, evidence: [] });
|
||||
const unknowns = graph.nodes.filter((n) => n.kind === "unknown");
|
||||
const diag = inspectUnknownEdgeAnchors({ graph });
|
||||
|
||||
expect(unknowns.length).toBe(2);
|
||||
const dependsOnEdges = graph.edges.filter((e) => e.relationship === "depends_on");
|
||||
expect(dependsOnEdges.length).toBe(2);
|
||||
expect(diag.result).toBe("shared_generic_anchor");
|
||||
expect(diag.anchorIds.length).toBe(1);
|
||||
|
||||
for (const u of unknowns) {
|
||||
expect(u.dependsOn).toEqual([]);
|
||||
expect(u.affects).toEqual([]);
|
||||
expect(u.parentId).toBeNull();
|
||||
}
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user