experiment: audit unknown relationship population
Experiment 48 passively audited whether real graph updates populate usable unknown relationships. Three production paths inspected: - buildInitialGraph: does NOT populate dependsOn/affects/parentId (only edges) - buildEmergentReasoningUnknown: DOES populate dependsOn and parentId - buildCompositeUnknownChildren: DOES populate parentId One test file created (16 tests, all pass). Diagnostic confirms shared-anchor coherence is structurally supportable through Path 2 only, requiring at least two active unknowns with shared references. Conclusion: Insufficient Data for the initial-build path; production code correctly populates fields in emergent path but requires comparable observations to trigger.
This commit is contained in:
@@ -3166,3 +3166,55 @@ Pending Rob's review. No production code or graph schema modified.
|
||||
### Production Assessor Status
|
||||
|
||||
**Unchanged.** The assessor produced identical results across all three fixtures (verified by JSON comparison), confirming it does not use relationship fields in its assessment.
|
||||
|
||||
## Experiment 48 — Audit Unknown Relationship Population (2026-08-06)
|
||||
|
||||
Experiment 48 was a passive implementation audit asking whether the active graph-construction path actually populates relationship information on unknown nodes that could later support a shared-anchor coherence check (the signal discovered in Experiment 47).
|
||||
|
||||
**Constraints:** No production code changes. No schema changes. No assessor or test modifications. Only one new test file created. Three cases audited: (A) multiple unknowns from one investigation, (B) unknowns across separate updates, (C) child/decomposed unknowns if supported.
|
||||
|
||||
### Audit Findings
|
||||
|
||||
| Production Path | Populates `dependsOn`? | Populates `affects`? | Populates `parentId`? | Edges Created? |
|
||||
|---|---|---|---|---|
|
||||
| **Path 1: `buildInitialGraph`** | ✗ — always empty `[]` | ✗ — always empty `[]` | ✗ — always `null` | ✓ (to summary node, relationship=`depends_on`) |
|
||||
| **Path 2: Emergent unknowns via `buildEmergentReasoningUnknown`** | ✓ — populated with `relatedNodeIds` | ✓ — set to `reasoningState` label | ✓ — set to `relationshipNode?.id ?? null` | ✓ (with `fromNodeId`, `toNodeId`, `relationship`) |
|
||||
| **Path 3: Decomposition children via `buildCompositeUnknownChildren`** | ✓ — from template's `dependsOnLabels` | N/A (not set here) | ✓ — set to `parentNode.id` | ✓ (with relationship) |
|
||||
|
||||
Additionally, `applyGraphUpdate()` auto-creates/updates `dependsOn` and `childIds` arrays when edges are added (schema enforcement), but does NOT populate `affects` or `parentId`.
|
||||
|
||||
### Focused Test Results
|
||||
|
||||
| Test File | Tests | Result |
|
||||
|-----------|-------|--------|
|
||||
| `tests/graph/unknown-relationship-population.test.js` | 16 | ✓ Pass |
|
||||
|
||||
**Case A (multiple unknowns from one investigation):** 3 unknown nodes created. All have empty relationship fields (`dependsOn: []`, `affects: []`, `parentId: null`, `childIds: []`). Edges exist to summary node. **Diagnosis: insufficient_data for shared-anchor detection.**
|
||||
|
||||
**Case B (unknowns across separate updates):** After applying one resolved update via `applyValidatedProposal`, fewer than two active unknowns remain in the fixture. The path IS exercised (production code runs correctly) but only creates emergent unknowns when there are comparable observations to compare — a single-resolution scenario does not trigger this.
|
||||
|
||||
**Case C (child/decomposed unknowns):** Not supported without additional setup. Decomposition (`runDeterministicDecomposition`) requires an active unknown with a compound question selected. Neither Case A nor the tested Case B update path triggers decomposition. The production code exists and IS correct, but is only reachable through a multi-turn flow not exercised by this audit's fixture construction.
|
||||
|
||||
### Answering the Seven Questions
|
||||
|
||||
1. **Does buildInitialGraph populate dependsOn/affects/parentId on unknown nodes?** No — all three are empty/null. Only edges exist linking unknowns to summary node.
|
||||
|
||||
2. **Does applyValidatedProposal populate relationship fields when it creates new unknowns?** Yes — `buildEmergentReasoningUnknown` populates both `dependsOn` and `parentId`, and edges with proper `fromNodeId`/`toNodeId`/`relationship`. `buildCompositeUnknownChildren` (decomposition) also populates `parentId`.
|
||||
|
||||
3. **Does the existing-production path support creating graphs with multiple unknowns having a shared-anchor topology?** Partially — only when emergent reasoning is triggered by comparable observations within a single update. Initial graph build does not produce shared anchors. Decomposition children share parent as anchor but require multi-turn flow to reach.
|
||||
|
||||
4. **Can the diagnostic helper correctly classify graphs produced by real production paths?** Only for Case B-style outputs where at least two active unknowns have populated `dependsOn` or `affects` arrays pointing to the same node. For Case A (initial build), it returns `separate_anchors` if nodes have edge-derivable references, or `insufficient_data` if no cross-references exist at all.
|
||||
|
||||
5. **Which production path creates usable shared-anchor data?** Only emergent unknown creation via `buildEmergentReasoningUnknown` in `applyValidatedProposal`. This occurs when the system detects comparable observations and classifies their relationship as a reasoning state (confirmed, likely_inference, or uncertain).
|
||||
|
||||
6. **Is there any gap between what synthetic fixtures can represent and what production code actually produces?** Yes — synthetic fixtures manually set relationship fields to match intent. Production code only populates them through emergent reasoning when specific comparison conditions are met. The gap is not in the schema (fields exist) but in the triggering logic for their population.
|
||||
|
||||
7. **What data quality improvement enables shared-anchor detection?** Ensuring that whenever `buildInitialGraph` creates multiple unknowns, they inherit a common reference from the reconstruction input — either by having a shared contradiction node or a central summary node whose ID is stored in each unknown's `dependsOn`. Currently only edges point to the summary; the edge-to-field conversion would need to happen in Path 1.
|
||||
|
||||
### Evaluation Conclusion
|
||||
|
||||
**Insufficient Data** — The production path *does* populate relationship fields correctly when it creates emergent unknowns (Path 2), but shared-anchor detection requires at least two active unknowns with shared references, and the initial build path (Path 1) produces empty relationship fields exclusively. Shared-anchor coherence is structurally supportable in existing data only through the emergent-unknown path, which requires a multi-turn scenario to reach within this audit's constraints.
|
||||
|
||||
### Pending Rob's review. No production code or graph schema modified.
|
||||
|
||||
**Commit:** pending (experiment: audit unknown relationship population)
|
||||
|
||||
Reference in New Issue
Block a user