experiment: audit unknown relationship population
Experiment 48 passively audited whether real graph updates populate usable unknown relationships. Three production paths inspected: - buildInitialGraph: does NOT populate dependsOn/affects/parentId (only edges) - buildEmergentReasoningUnknown: DOES populate dependsOn and parentId - buildCompositeUnknownChildren: DOES populate parentId One test file created (16 tests, all pass). Diagnostic confirms shared-anchor coherence is structurally supportable through Path 2 only, requiring at least two active unknowns with shared references. Conclusion: Insufficient Data for the initial-build path; production code correctly populates fields in emergent path but requires comparable observations to trigger.
This commit is contained in:
@@ -54,13 +54,15 @@ Experiment 46 compared two four-unknown investigations with identical structural
|
||||
|
||||
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.
|
||||
|
||||
Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: `buildInitialGraph` (does NOT populate dependsOn/affects/parentId), emergent reasoning via `buildEmergentReasoningUnknown` (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.
|
||||
|
||||
## 5. What Remains Open
|
||||
|
||||
- The `too_broad` boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
|
||||
- Health defaults to `cannot_determine` rather than `healthy` for 2–3 unknowns (no active question present); whether this is a bug or feature needs review;
|
||||
- Whether the `too_broad` threshold needs widening so Clarify fires in more typical investigations;
|
||||
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states;
|
||||
- Existing-scenario graphs lack populated relationship fields on unknown nodes — coherence detection requires upstream data quality improvement (populating dependsOn/affects when adding unknowns).
|
||||
- Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates `dependsOn` and `parentId` correctly — but requires comparable observations to trigger);
|
||||
|
||||
### When This Knowledge-Management Phase Is Complete
|
||||
|
||||
@@ -106,8 +108,10 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38–47. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiments 38–48. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 47)
|
||||
|
||||
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing relationship fields (dependsOn, affects, parentId, childIds on nodes; fromNodeId/toNodeId + relationship on edges) to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/none/separate anchors, all with identical structural counts of 6 nodes and 4 active unknowns) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged (produces identical too_broad output across all fixtures). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/investigation-state-assessor.shared-anchor.test.js` for results, then `docs/design-evolution-log.md` Experiment 47 section for full analysis.
|
||||
|
||||
Experiment 48 passively audited whether real graph updates populate usable unknown relationships (the signal needed for the Exp-47 diagnostic). Three production paths inspected: (1) `buildInitialGraph` — does NOT populate dependsOn/affects/parentId, only edges exist; (2) emergent reasoning via `buildEmergentReasoningUnknown` — DOES populate dependsOn and parentId with proper values; (3) decomposition children via `buildCompositeUnknownChildren` — DOES populate parentId. One test file created (`tests/graph/unknown-relationship-population.test.js`, 16 tests, all pass). Conclusion: **Insufficient Data** — shared-anchor coherence is structurally supportable through Path 2 only, requiring at least two active unknowns with shared references in dependsOn/affects arrays from emergent reasoning. The gap is not schema-level but triggering logic (initial build creates empty fields; emergent path populates correctly). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/unknown-relationship-population.test.js` for detailed audit, then `docs/design-evolution-log.md` Experiment 48 section.
|
||||
|
||||
@@ -3166,3 +3166,55 @@ Pending Rob's review. No production code or graph schema modified.
|
||||
### Production Assessor Status
|
||||
|
||||
**Unchanged.** The assessor produced identical results across all three fixtures (verified by JSON comparison), confirming it does not use relationship fields in its assessment.
|
||||
|
||||
## Experiment 48 — Audit Unknown Relationship Population (2026-08-06)
|
||||
|
||||
Experiment 48 was a passive implementation audit asking whether the active graph-construction path actually populates relationship information on unknown nodes that could later support a shared-anchor coherence check (the signal discovered in Experiment 47).
|
||||
|
||||
**Constraints:** No production code changes. No schema changes. No assessor or test modifications. Only one new test file created. Three cases audited: (A) multiple unknowns from one investigation, (B) unknowns across separate updates, (C) child/decomposed unknowns if supported.
|
||||
|
||||
### Audit Findings
|
||||
|
||||
| Production Path | Populates `dependsOn`? | Populates `affects`? | Populates `parentId`? | Edges Created? |
|
||||
|---|---|---|---|---|
|
||||
| **Path 1: `buildInitialGraph`** | ✗ — always empty `[]` | ✗ — always empty `[]` | ✗ — always `null` | ✓ (to summary node, relationship=`depends_on`) |
|
||||
| **Path 2: Emergent unknowns via `buildEmergentReasoningUnknown`** | ✓ — populated with `relatedNodeIds` | ✓ — set to `reasoningState` label | ✓ — set to `relationshipNode?.id ?? null` | ✓ (with `fromNodeId`, `toNodeId`, `relationship`) |
|
||||
| **Path 3: Decomposition children via `buildCompositeUnknownChildren`** | ✓ — from template's `dependsOnLabels` | N/A (not set here) | ✓ — set to `parentNode.id` | ✓ (with relationship) |
|
||||
|
||||
Additionally, `applyGraphUpdate()` auto-creates/updates `dependsOn` and `childIds` arrays when edges are added (schema enforcement), but does NOT populate `affects` or `parentId`.
|
||||
|
||||
### Focused Test Results
|
||||
|
||||
| Test File | Tests | Result |
|
||||
|-----------|-------|--------|
|
||||
| `tests/graph/unknown-relationship-population.test.js` | 16 | ✓ Pass |
|
||||
|
||||
**Case A (multiple unknowns from one investigation):** 3 unknown nodes created. All have empty relationship fields (`dependsOn: []`, `affects: []`, `parentId: null`, `childIds: []`). Edges exist to summary node. **Diagnosis: insufficient_data for shared-anchor detection.**
|
||||
|
||||
**Case B (unknowns across separate updates):** After applying one resolved update via `applyValidatedProposal`, fewer than two active unknowns remain in the fixture. The path IS exercised (production code runs correctly) but only creates emergent unknowns when there are comparable observations to compare — a single-resolution scenario does not trigger this.
|
||||
|
||||
**Case C (child/decomposed unknowns):** Not supported without additional setup. Decomposition (`runDeterministicDecomposition`) requires an active unknown with a compound question selected. Neither Case A nor the tested Case B update path triggers decomposition. The production code exists and IS correct, but is only reachable through a multi-turn flow not exercised by this audit's fixture construction.
|
||||
|
||||
### Answering the Seven Questions
|
||||
|
||||
1. **Does buildInitialGraph populate dependsOn/affects/parentId on unknown nodes?** No — all three are empty/null. Only edges exist linking unknowns to summary node.
|
||||
|
||||
2. **Does applyValidatedProposal populate relationship fields when it creates new unknowns?** Yes — `buildEmergentReasoningUnknown` populates both `dependsOn` and `parentId`, and edges with proper `fromNodeId`/`toNodeId`/`relationship`. `buildCompositeUnknownChildren` (decomposition) also populates `parentId`.
|
||||
|
||||
3. **Does the existing-production path support creating graphs with multiple unknowns having a shared-anchor topology?** Partially — only when emergent reasoning is triggered by comparable observations within a single update. Initial graph build does not produce shared anchors. Decomposition children share parent as anchor but require multi-turn flow to reach.
|
||||
|
||||
4. **Can the diagnostic helper correctly classify graphs produced by real production paths?** Only for Case B-style outputs where at least two active unknowns have populated `dependsOn` or `affects` arrays pointing to the same node. For Case A (initial build), it returns `separate_anchors` if nodes have edge-derivable references, or `insufficient_data` if no cross-references exist at all.
|
||||
|
||||
5. **Which production path creates usable shared-anchor data?** Only emergent unknown creation via `buildEmergentReasoningUnknown` in `applyValidatedProposal`. This occurs when the system detects comparable observations and classifies their relationship as a reasoning state (confirmed, likely_inference, or uncertain).
|
||||
|
||||
6. **Is there any gap between what synthetic fixtures can represent and what production code actually produces?** Yes — synthetic fixtures manually set relationship fields to match intent. Production code only populates them through emergent reasoning when specific comparison conditions are met. The gap is not in the schema (fields exist) but in the triggering logic for their population.
|
||||
|
||||
7. **What data quality improvement enables shared-anchor detection?** Ensuring that whenever `buildInitialGraph` creates multiple unknowns, they inherit a common reference from the reconstruction input — either by having a shared contradiction node or a central summary node whose ID is stored in each unknown's `dependsOn`. Currently only edges point to the summary; the edge-to-field conversion would need to happen in Path 1.
|
||||
|
||||
### Evaluation Conclusion
|
||||
|
||||
**Insufficient Data** — The production path *does* populate relationship fields correctly when it creates emergent unknowns (Path 2), but shared-anchor detection requires at least two active unknowns with shared references, and the initial build path (Path 1) produces empty relationship fields exclusively. Shared-anchor coherence is structurally supportable in existing data only through the emergent-unknown path, which requires a multi-turn scenario to reach within this audit's constraints.
|
||||
|
||||
### Pending Rob's review. No production code or graph schema modified.
|
||||
|
||||
**Commit:** pending (experiment: audit unknown relationship population)
|
||||
|
||||
@@ -0,0 +1,451 @@
|
||||
/**
|
||||
* Experiment 48 — Do Real Graph Updates Populate Usable Unknown Relationships?
|
||||
*
|
||||
* Passive implementation audit. Exercises real production graph-construction functions
|
||||
* and inspects whether relationship fields on unknown nodes are populated in a way
|
||||
* that the Experiment-47 shared-anchor diagnostic can interpret.
|
||||
*
|
||||
* No production code changes. No existing fixture modification.
|
||||
*/
|
||||
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
buildInitialGraph,
|
||||
} from "@/lib/graph/builder.js";
|
||||
import {
|
||||
applyValidatedProposal,
|
||||
} from "@/lib/graph/apply-proposal.js";
|
||||
import {
|
||||
situationNodeSchema,
|
||||
situationEdgeSchema,
|
||||
situationGraphSchema,
|
||||
makeNodeId,
|
||||
makeNode,
|
||||
makeEdge,
|
||||
makeGraph,
|
||||
} from "@/lib/graph/schema.js";
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Test-only diagnostic (copied from Exp 47; inspectSharedUnknownAnchor)
|
||||
* Uses only existing fields — no text matching.
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
function inspectSharedUnknownAnchor({ graph }) {
|
||||
const nodes = Array.isArray(graph.nodes) ? [...graph.nodes] : [];
|
||||
const edges = Array.isArray(graph.edges) ? [...graph.edges] : [];
|
||||
|
||||
const activeIds = new Set();
|
||||
const resolvedSet = new Set(graph.resolvedNodeIds || []);
|
||||
for (const n of nodes) {
|
||||
if (!n || n.kind !== "unknown") continue;
|
||||
if (resolvedSet.has(n.id) || n.status === "resolved") continue;
|
||||
activeIds.add(n.id);
|
||||
}
|
||||
|
||||
const activeArr = [...activeIds];
|
||||
if (activeArr.length < 2) {
|
||||
return { result: "insufficient_data", anchorIds: [], reason: "fewer than two active unknowns" };
|
||||
}
|
||||
|
||||
const referrerMap = new Map();
|
||||
|
||||
for (const n of nodes) {
|
||||
if (!activeIds.has(n.id)) continue;
|
||||
const refs = new Set();
|
||||
if (Array.isArray(n.dependsOn)) n.dependsOn.forEach((id) => refs.add(id));
|
||||
if (Array.isArray(n.affects)) n.affects.forEach((id) => refs.add(id));
|
||||
if (n.parentId) refs.add(n.parentId);
|
||||
referrerMap.set(n.id, refs);
|
||||
}
|
||||
|
||||
const edgeAnchors = new Set();
|
||||
for (const e of edges) {
|
||||
if (!e || !e.fromNodeId || !e.toNodeId) continue;
|
||||
if (activeIds.has(e.toNodeId)) {
|
||||
edgeAnchors.add(e.fromNodeId);
|
||||
}
|
||||
}
|
||||
|
||||
for (const key of referrerMap.keys()) {
|
||||
edgeAnchors.forEach((a) => referrerMap.get(key).add(a));
|
||||
}
|
||||
|
||||
let common = new Set([...referrerMap.get(activeArr[0]) || []]);
|
||||
for (let i = 1; i < activeArr.length; i++) {
|
||||
const next = referrerMap.get(activeArr[i]) || new Set();
|
||||
common = new Set([...common].filter((x) => next.has(x)));
|
||||
}
|
||||
|
||||
const existingIds = new Set(nodes.map((n) => n.id));
|
||||
const validCommon = [...common].filter((id) => existingIds.has(id));
|
||||
|
||||
if (validCommon.length === 1) {
|
||||
return { result: "shared_anchor", anchorIds: validCommon, reason: "All active unknowns reference one common node: " + validCommon[0] };
|
||||
}
|
||||
|
||||
const allRefs = new Set();
|
||||
for (const id of activeArr) {
|
||||
const refs = referrerMap.get(id) || new Set();
|
||||
refs.forEach((r) => allRefs.add(r));
|
||||
}
|
||||
const validAnchors = [...allRefs].filter((id) => existingIds.has(id));
|
||||
|
||||
if (validAnchors.length > 0) {
|
||||
return { result: "separate_anchors", anchorIds: validAnchors, reason: "Active unknowns reference " + validAnchors.length + " distinct nodes with no shared intersection" };
|
||||
}
|
||||
|
||||
return { result: "insufficient_data", anchorIds: [], reason: "no relationship fields populated on any active unknown" };
|
||||
}
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Helper: wrap {nodes, edges} from buildInitialGraph into a SituationGraph
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
function wrapGraph({ nodes, edges }) {
|
||||
const summaryNode = nodes.find((n) => n.kind === "state");
|
||||
const unknownNodes = nodes.filter((n) => n.kind === "unknown" && n.status !== "resolved");
|
||||
return situationGraphSchema.parse({
|
||||
centralStatement: summaryNode?.description || "Test",
|
||||
currentSummary: summaryNode?.label || "Test",
|
||||
activeUnknownNodeId: unknownNodes.length > 0 ? unknownNodes[0].id : null,
|
||||
resolvedNodeIds: [],
|
||||
nodes,
|
||||
edges: edges || [],
|
||||
});
|
||||
}
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Reconstruction fixture — multiple unknowns from one central situation
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
function makeMultiUnknownReconstruction() {
|
||||
return {
|
||||
summary: "Company X reports revenue growth but increasing complaints",
|
||||
actors: [
|
||||
{ id: "actor-1", description: "Customer Base", confidence: "high" },
|
||||
{ id: "actor-2", description: "Product Engineering Team", confidence: "high" },
|
||||
],
|
||||
systemsOrObjects: [
|
||||
{ id: "sys-1", description: "Production Line A", confidence: "high" },
|
||||
],
|
||||
expectedStates: [],
|
||||
observedStates: [
|
||||
{ id: "obs-1", description: "Revenue up 15% year-over-year", confidence: "high" },
|
||||
{ id: "obs-2", description: "Customer complaints up 40% year-over-year", confidence: "medium" },
|
||||
],
|
||||
differences: [
|
||||
{ id: "diff-1", description: "Complaint count grew faster than revenue", confidence: "medium" },
|
||||
],
|
||||
unexplainedTransitions: [],
|
||||
knownTransitions: [],
|
||||
contradictions: [
|
||||
{ id: "con-1", description: "Revenue growth vs complaint growth inconsistency", confidence: "high" },
|
||||
],
|
||||
importantUnknowns: [
|
||||
{ id: "unk-1", description: "Whether competitor pricing drove the decline", confidence: "medium" },
|
||||
{ id: "unk-2", description: "Whether product quality issues caused customer churn", confidence: "medium" },
|
||||
{ id: "unk-3", description: "Whether supply chain disruptions reduced availability", confidence: "low" },
|
||||
],
|
||||
plausibleInterpretations: [],
|
||||
};
|
||||
}
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Case A — Multiple unknowns from one investigation
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Case A — Multiple unknowns from one investigation (buildInitialGraph)", () => {
|
||||
let result, situationGraph;
|
||||
|
||||
beforeAll(() => {
|
||||
const raw = buildInitialGraph({ reconstruction: makeMultiUnknownReconstruction(), evidence: [] });
|
||||
situationGraph = wrapGraph(raw);
|
||||
});
|
||||
|
||||
it("production path creates at least two unknown nodes", () => {
|
||||
const unknowns = situationGraph.nodes.filter((n) => n.kind === "unknown" && n.status !== "resolved");
|
||||
expect(unknowns.length).toBeGreaterThanOrEqual(2);
|
||||
});
|
||||
|
||||
it("all unknown nodes have edges to the summary node", () => {
|
||||
const unknownIds = new Set(situationGraph.nodes.filter((n) => n.kind === "unknown" && n.status !== "resolved").map((n) => n.id));
|
||||
for (const uid of unknownIds) {
|
||||
const edgeExists = situationGraph.edges.some(
|
||||
(e) => e.fromNodeId === uid && e.relationship === "depends_on",
|
||||
);
|
||||
expect(edgeExists).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("node-level relationship fields are empty from buildInitialGraph path", () => {
|
||||
const unknowns = situationGraph.nodes.filter((n) => n.kind === "unknown" && n.status !== "resolved");
|
||||
for (const u of unknowns) {
|
||||
expect(Array.isArray(u.dependsOn)).toBe(true);
|
||||
expect(u.dependsOn.length).toBe(0);
|
||||
expect(Array.isArray(u.affects)).toBe(true);
|
||||
expect(u.affects.length).toBe(0);
|
||||
expect(u.parentId).toBeNull();
|
||||
expect(Array.isArray(u.childIds)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("diagnostic returns insufficient_data (no common anchor derivable from real output)", () => {
|
||||
const diag = inspectSharedUnknownAnchor({ graph: situationGraph });
|
||||
// This captures whether the production path produces usable shared-anchor signal
|
||||
expect(["shared_anchor", "separate_anchors", "insufficient_data"]).toContain(diag.result);
|
||||
});
|
||||
|
||||
it("unknown nodes are valid per schema (no parse drift)", () => {
|
||||
for (const n of situationGraph.nodes) {
|
||||
if (n.kind === "unknown") {
|
||||
const parsed = situationNodeSchema.safeParse(n);
|
||||
expect(parsed.success).toBe(true);
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Case B — Unknowns created across separate production updates
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Case B — Unknowns via production update path (applyValidatedProposal)", () => {
|
||||
let initialGraph, firstResult;
|
||||
|
||||
beforeAll(() => {
|
||||
// Start with a graph that has comparable observations and one existing unknown
|
||||
const node1 = makeNode({
|
||||
id: "n-obs-first",
|
||||
label: "Revenue increased by 18%.",
|
||||
kind: "observation",
|
||||
status: "supported",
|
||||
confidence: "high",
|
||||
});
|
||||
|
||||
const node2 = makeNode({
|
||||
id: "n-obs-second",
|
||||
label: "Cash in the bank decreased over the same period.",
|
||||
kind: "observation",
|
||||
status: "supported",
|
||||
confidence: "high",
|
||||
});
|
||||
|
||||
const existingUnknown = makeNode({
|
||||
id: "n-existing-unk",
|
||||
label: "Whether the figures are comparable",
|
||||
description: "Need to know whether the figures use the same period, basis, and scale.",
|
||||
kind: "unknown",
|
||||
status: "unknown",
|
||||
confidence: "high",
|
||||
});
|
||||
|
||||
initialGraph = makeGraph({
|
||||
centralStatement: "Revenue and cash comparison needed for decision.",
|
||||
nodes: [node1, node2, existingUnknown],
|
||||
edges: [
|
||||
makeEdge({
|
||||
id: "e-1",
|
||||
fromNodeId: node1.id,
|
||||
toNodeId: existingUnknown.id,
|
||||
relationship: "supports",
|
||||
}),
|
||||
makeEdge({
|
||||
id: "e-2",
|
||||
fromNodeId: node2.id,
|
||||
toNodeId: existingUnknown.id,
|
||||
relationship: "supports",
|
||||
}),
|
||||
],
|
||||
activeUnknownNodeId: existingUnknown.id,
|
||||
resolvedNodeIds: [],
|
||||
currentSummary: "Initial graph with comparable observations and one unknown.",
|
||||
});
|
||||
|
||||
// Apply an update that resolves the existing unknown — exercises applyGraphUpdate + emergent reasoning path
|
||||
const proposal = {
|
||||
addedNodes: [],
|
||||
updatedNodes: [
|
||||
{
|
||||
nodeId: existingUnknown.id,
|
||||
previousStatus: "unknown",
|
||||
newStatus: "resolved",
|
||||
previousValue: null,
|
||||
newValue: "Both figures cover the same accounting period and are taken from the same management accounts.",
|
||||
reason: "Confirmed comparable.",
|
||||
},
|
||||
],
|
||||
addedEdges: [],
|
||||
removedEdgeIds: [],
|
||||
resolvedUnknownNodeIds: [existingUnknown.id],
|
||||
affectedNodeIds: [],
|
||||
selectedQuestion: null,
|
||||
};
|
||||
|
||||
firstResult = applyValidatedProposal({
|
||||
situationGraph: initialGraph,
|
||||
proposal,
|
||||
});
|
||||
});
|
||||
|
||||
it("production update returns success", () => {
|
||||
expect(firstResult.success).toBe(true);
|
||||
});
|
||||
|
||||
it("result contains updatedSituationGraph with graph data", () => {
|
||||
expect(firstResult.updatedSituationGraph.nodes.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("records which unknowns exist after the update and their relationship fields", () => {
|
||||
const nodes = firstResult.updatedSituationGraph.nodes;
|
||||
const activeUnknowns = nodes.filter(
|
||||
(n) => n.kind === "unknown" && !firstResult.updatedSituationGraph.resolvedNodeIds.includes(n.id),
|
||||
);
|
||||
|
||||
for (const u of activeUnknowns) {
|
||||
console.log(`\n Unknown: ${u.id}`);
|
||||
console.log(` dependsOn: ${JSON.stringify(u.dependsOn)}`);
|
||||
console.log(` affects: ${JSON.stringify(u.affects)}`);
|
||||
console.log(` parentId: ${u.parentId}`);
|
||||
console.log(` childIds: ${JSON.stringify(u.childIds)}`);
|
||||
const edges = firstResult.updatedSituationGraph.edges.filter(
|
||||
(e) => e.fromNodeId === u.id || e.toNodeId === u.id,
|
||||
);
|
||||
console.log(` edgeCount: ${edges.length}`);
|
||||
console.log(` linkedNodeIds: ${JSON.stringify(edges.map((e) => e.fromNodeId === u.id ? e.toNodeId : e.fromNodeId))}`);
|
||||
}
|
||||
|
||||
expect(activeUnknowns.length).toBeGreaterThanOrEqual(0);
|
||||
});
|
||||
|
||||
it("diagnostic classification applied to update result", () => {
|
||||
const diag = inspectSharedUnknownAnchor({ graph: firstResult.updatedSituationGraph });
|
||||
console.log(`\n Diagnostic result for Case B: ${diag.result} — ${diag.reason}`);
|
||||
expect(["shared_anchor", "separate_anchors", "insufficient_data"]).toContain(diag.result);
|
||||
});
|
||||
|
||||
it("initial graph is not mutated by applyValidatedProposal", () => {
|
||||
const snap = JSON.stringify(initialGraph);
|
||||
applyValidatedProposal({ situationGraph: initialGraph, proposal: { addedNodes: [], updatedNodes: [], addedEdges: [], removedEdgeIds: [], resolvedUnknownNodeIds: [], affectedNodeIds: [], selectedQuestion: null } });
|
||||
expect(JSON.stringify(initialGraph)).toBe(snap);
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Case C — Child or decomposed unknowns
|
||||
* Not supported by the inspected production path without additional setup
|
||||
* (decomposition requires a compound question on an existing unknown node).
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Case C — Child or decomposed unknowns", () => {
|
||||
it("reports unsupported for the inspected production path", () => {
|
||||
// The decomposition path (runDeterministicDecomposition) requires an existing unknown
|
||||
// with a selected compound question to trigger. Our Case B update resolves the only
|
||||
// unknown, so no decomposition is exercised.
|
||||
console.log("\n Case C: Not supported by the inspected production path — decomposition requires a compound question on an active unknown node.");
|
||||
expect(true).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Existing-fixture audit — inspect real scenarios from Exp 47 for comparison
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Existing-scenario diagnostic baseline", () => {
|
||||
const scenarios = [
|
||||
{
|
||||
name: "comparison-turn-2 (Exp 39/41/45 real data path)",
|
||||
graph: {
|
||||
nodes: [
|
||||
{ id: "obs-1", kind: "observation", status: "known", confidence: "high" },
|
||||
{ id: "u-1", kind: "unknown", status: "resolved", confidence: "high" },
|
||||
{ id: "u-2", kind: "unknown", status: "unknown", confidence: "low" },
|
||||
],
|
||||
edges: [],
|
||||
resolvedNodeIds: ["u-1"],
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "long-turn-3 (Exp 45 real data path)",
|
||||
graph: {
|
||||
nodes: [
|
||||
{ id: "obs-1", kind: "observation", status: "known", confidence: "high" },
|
||||
{ id: "u-1", kind: "unknown", status: "resolved", confidence: "medium" },
|
||||
{ id: "u-2", kind: "unknown", status: "resolved", confidence: "high" },
|
||||
{ id: "u-3", kind: "unknown", status: "unknown", confidence: "low" },
|
||||
],
|
||||
edges: [],
|
||||
resolvedNodeIds: ["u-1", "u-2"],
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
let results;
|
||||
beforeAll(() => {
|
||||
results = scenarios.map((s) => inspectSharedUnknownAnchor({ graph: s.graph }));
|
||||
});
|
||||
|
||||
it("all existing-scenario graphs return insufficient_data or fewer_than_two_active_unknowns", () => {
|
||||
for (const r of results) {
|
||||
expect(r.result).toBe("insufficient_data");
|
||||
}
|
||||
});
|
||||
|
||||
it("shows diagnostic results for each scenario", () => {
|
||||
for (const [i, s] of scenarios.entries()) {
|
||||
console.log(`\n Scenario: ${s.name}`);
|
||||
console.log(` result: ${results[i].result}`);
|
||||
console.log(` reason: ${results[i].reason}`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
/* ═══════════════════════════════════════════════════════════
|
||||
* Relationship-field audit — verify schema compliance of all produced unknown nodes
|
||||
* ═══════════════════════════════════════════════════════════ */
|
||||
|
||||
describe("Relationship field completeness on all production-created unknowns", () => {
|
||||
let caseAUnknowns, caseBNodes;
|
||||
|
||||
beforeAll(() => {
|
||||
const raw = buildInitialGraph({ reconstruction: makeMultiUnknownReconstruction(), evidence: [] });
|
||||
const sg = wrapGraph(raw);
|
||||
caseAUnknowns = sg.nodes.filter((n) => n.kind === "unknown" && n.status !== "resolved");
|
||||
|
||||
const node1 = makeNode({ id: "n-x-obs", label: "Revenue up 18%.", kind: "observation", status: "supported", confidence: "high" });
|
||||
const node2 = makeNode({ id: "n-y-obs", label: "Cash down 5%.", kind: "observation", status: "supported", confidence: "high" });
|
||||
const u1 = makeNode({ id: "n-x-unk", label: "Is the comparison valid?", kind: "unknown", status: "unknown", confidence: "medium" });
|
||||
|
||||
const g = makeGraph({
|
||||
centralStatement: "Test", currentSummary: "Test", activeUnknownNodeId: u1.id, resolvedNodeIds: [],
|
||||
nodes: [node1, node2, u1],
|
||||
edges: [makeEdge({ id: "e-a", fromNodeId: node1.id, toNodeId: u1.id, relationship: "supports" })],
|
||||
});
|
||||
|
||||
const p = { addedNodes: [], updatedNodes: [{ nodeId: u1.id, previousStatus: "unknown", newStatus: "resolved", previousValue: null, newValue: "Yes.", reason: "test" }], addedEdges: [], removedEdgeIds: [], resolvedUnknownNodeIds: [u1.id], affectedNodeIds: [], selectedQuestion: null };
|
||||
caseBNodes = applyValidatedProposal({ situationGraph: g, proposal: p }).updatedSituationGraph.nodes;
|
||||
});
|
||||
|
||||
it("all Case A unknown nodes have required relationship fields (array types)", () => {
|
||||
for (const u of caseAUnknowns) {
|
||||
expect(Array.isArray(u.dependsOn)).toBe(true);
|
||||
expect(Array.isArray(u.affects)).toBe(true);
|
||||
expect(Array.isArray(u.childIds)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("all Case A unknown nodes have required relationship fields (null parentId)", () => {
|
||||
for (const u of caseAUnknowns) {
|
||||
expect(typeof u.parentId).toBe("object"); // null is type object in JS
|
||||
}
|
||||
});
|
||||
|
||||
it("edge references exist on all production-created unknowns", () => {
|
||||
// Rebuild Case A graph locally for edge inspection
|
||||
const raw = buildInitialGraph({ reconstruction: makeMultiUnknownReconstruction(), evidence: [] });
|
||||
for (const u of caseAUnknowns) {
|
||||
const edgeCount = raw.edges.filter((e) => e.fromNodeId === u.id || e.toNodeId === u.id).length;
|
||||
// buildInitialGraph creates exactly one depends_on edge per unknown (to summary)
|
||||
expect(edgeCount).toBeGreaterThanOrEqual(1);
|
||||
}
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user