robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: audit unknown relationship population

Browse Source 
Experiment 48 passively audited whether real graph updates populate usable
unknown relationships. Three production paths inspected:

- buildInitialGraph: does NOT populate dependsOn/affects/parentId (only edges)
- buildEmergentReasoningUnknown: DOES populate dependsOn and parentId
- buildCompositeUnknownChildren: DOES populate parentId

One test file created (16 tests, all pass). Diagnostic confirms shared-anchor
coherence is structurally supportable through Path 2 only, requiring at least
two active unknowns with shared references. Conclusion: Insufficient Data for
the initial-build path; production code correctly populates fields in emergent
path but requires comparable observations to trigger.
This commit is contained in:
robbond
2026-08-06 19:48:18 +01:00
parent 40ef3e108f
commit b1c69e9303
3 changed files with 509 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+6 -2
View File
@@ -54,13 +54,15 @@ Experiment 46 compared two four-unknown investigations with identical structural
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.
Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: `buildInitialGraph` (does NOT populate dependsOn/affects/parentId), emergent reasoning via `buildEmergentReasoningUnknown` (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.
## 5. What Remains Open
- The `too_broad` boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
- Health defaults to `cannot_determine` rather than `healthy` for 2–3 unknowns (no active question present); whether this is a bug or feature needs review;
- Whether the `too_broad` threshold needs widening so Clarify fires in more typical investigations;
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states;
- Existing-scenario graphs lack populated relationship fields on unknown nodes — coherence detection requires upstream data quality improvement (populating dependsOn/affects when adding unknowns).
- Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates `dependsOn` and `parentId` correctly — but requires comparable observations to trigger);
### When This Knowledge-Management Phase Is Complete
@@ -106,8 +108,10 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 38–47. Branch: `feature/user-workspace-ux-v0.7`.*
*Created by Experiment 34. Updated by Experiments 38–48. Branch: `feature/user-workspace-ux-v0.7`.*
### Return-to-Work Note (Experiment 47)
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing relationship fields (dependsOn, affects, parentId, childIds on nodes; fromNodeId/toNodeId + relationship on edges) to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/none/separate anchors, all with identical structural counts of 6 nodes and 4 active unknowns) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged (produces identical too_broad output across all fixtures). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/investigation-state-assessor.shared-anchor.test.js` for results, then `docs/design-evolution-log.md` Experiment 47 section for full analysis.
Experiment 48 passively audited whether real graph updates populate usable unknown relationships (the signal needed for the Exp-47 diagnostic). Three production paths inspected: (1) `buildInitialGraph` — does NOT populate dependsOn/affects/parentId, only edges exist; (2) emergent reasoning via `buildEmergentReasoningUnknown` — DOES populate dependsOn and parentId with proper values; (3) decomposition children via `buildCompositeUnknownChildren` — DOES populate parentId. One test file created (`tests/graph/unknown-relationship-population.test.js`, 16 tests, all pass). Conclusion: **Insufficient Data** — shared-anchor coherence is structurally supportable through Path 2 only, requiring at least two active unknowns with shared references in dependsOn/affects arrays from emergent reasoning. The gap is not schema-level but triggering logic (initial build creates empty fields; emergent path populates correctly). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/unknown-relationship-population.test.js` for detailed audit, then `docs/design-evolution-log.md` Experiment 48 section.