experiment: audit referential provenance

docs/design-evolution-log.md

@@ -5437,7 +5437,7 @@ However, the requested output schema (graphUpdateSchema in lib/graph/schema.js)
 This means:
 - Prompt-level source identity: explicit
 - Proposal-level provenance: absent (structural limitation of schema)
-- The gap is not a prompt-design problem; it is a schema-deficiency problem
+- The tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided.
 
 ### Limitations
 
@@ -5463,3 +5463,103 @@ This means:
 
 No test run required; Experiment 54D is a prompt-source audit.
 
+
+## Experiment 54E — Can Existing Evidence References Preserve Provenance Without Adding a Node Field? (2026-08-07)
+
+### Objective
+
+Inspect whether the current graph design already preserves referential provenance through existing evidence identities and references, without requiring a new node field. This is a source-inspection experiment only. Do not implement provenance.
+
+### Hypothesis
+
+Existing `evidenceIds` and evidence records may already provide enough referential structure to preserve provenance if: (1) evidence records survive beyond reconstruction; (2) their IDs remain resolvable later; (3) `evidenceType` remains attached to those records; (4) graph nodes can reliably link to the evidence records that justify them.
+
+### Files Inspected
+
+- `lib/reconstruction/schema.js` — evidenceRecordSchema definition, reconstructionV2Schema evidence field
+- `lib/graph/schema.js` — situationNodeSchema evidenceIds field, graphUpdateSchema structure
+- `lib/graph/builder.js` — how evidence IDs are assigned during initial graph build (buildInitialGraph)
+- `lib/graph/orchestrator.js` — startCase and updateCaseWithDependencies return values; evidence flow through the pipeline
+- `lib/reconstruction/schema.js` lines 113–127 — evidenceRecordSchema fields
+- `lib/graph/apply-proposal.js` — how evidenceIds are populated during graph updates (appendUniqueValue pattern)
+- `lib/graph/prompt-builder.js` — rule 14 "Do not invent evidence"
+
+### Evidence Record Identity
+
+Each reconstruction evidence record (v0.2 schema) has:
+- **Stable ID**: `id: z.string().min(1)` — deterministic identifier present in every record.
+- **evidenceType**: `z.enum(["direct_observation", "reported_statement", "interpretation", "assumption", "inferred_relationship"])` — five distinct values.
+- **Sufficient information to distinguish supplied from inferred**: Yes. `reported_statement` identifies user-supplied material; `direct_observation` identifies observed facts; `interpretation`, `assumption`, and `inferred_relationship` identify model-inferred or unverified material. The type field is explicitly designed for this purpose.
+
+Answer: **Yes, evidence records carry sufficient identity.**
+
+### Graph Reference Behaviour
+
+When a graph node contains an `evidenceId`:
+- During `buildInitialGraph` (builder.js:62–70), evidence records from the analysis output are placed in an `evidenceMap`. Nodes are created with `evidenceIds` populated from actual evidence record IDs (builder.js:100: `node.evidenceIds.push(obs.id)`).
+- These IDs refer to real evidence records during construction. However, after graph construction, the graph contains only string IDs — they would resolve if a lookup table existed, but they are opaque strings within the node object itself.
+
+Answer: **IDs refer to real evidence records at build time; become opaque strings in the graph post-construction.**
+
+### Evidence-Record Lifetime
+
+The critical flow is:
+1. `analyseScenario` (lib/analysis.js) produces evidence records via reconstructionV2Schema, returned as `data.evidence`.
+2. In `startCase` (orchestrator.js:378), `analysis.evidence` is passed to `buildInitialGraph` solely to populate node `evidenceIds`.
+3. **The evidence records themselves are NOT included in `startCase` return value.** The return at orchestrator.js:489-566 includes only `situationGraph`, `selectedQuestion`, `diagnostics`, and `assessment`. No `evidence` field exists in the return object.
+4. In `updateCaseWithDependencies`, no new evidence records are created anywhere in the pipeline. Rule 14 of the update prompt ("Do not invent evidence") explicitly forbids the model from creating them. The prompt-builder.js shows no mechanism for evidence generation during updates.
+5. There is **no persistence layer** in this codebase that stores case state to disk or a database. The API routes (cases/start/route.js, cases/update/route.js) return data to the client; they do not persist anything.
+
+The initial evidence records exist only during the startCase execution lifetime. They are consumed to populate graph node evidenceIds but never returned alongside the case state. Subsequent update cycles produce no evidence records at all.
+
+Answer: **not_retained_with_graph**
+
+### Provenance Through Reference
+
+If a later consumer receives normal persisted case state (which contains only `situationGraph` with nodes having string `evidenceIds`), it cannot:
+1. Read a graph node — yes, the node and its evidenceIds are present.
+2. Follow its evidenceIds — no matching evidence records exist anywhere in the persisted state.
+3. Resolve each ID to an evidence record — impossible; records do not exist.
+4. Inspect evidenceType — N/A; no records to inspect.
+
+Answer: **no** — The reference chain breaks at step 2/3 because evidence records are not part of the returned case state.
+
+### Node-to-Evidence Completeness
+
+- **Initial graph nodes (from buildInitialGraph)**: Nodes from observedStates do receive evidence references (builder.js:100). Other node types (actors, systemsOrObjects, differences, contradictions, unknowns, interpretations) are created WITHOUT evidence references — only observedStates nodes get `evidenceIds` populated.
+- **Nodes added during update**: No new evidence records are ever created during the update flow. Nodes created via `applyValidatedProposal` may have their `evidenceIds` field set (the schema allows it), but no source code in the update path populates them from actual evidence records. There is no mechanism to create or assign evidence record IDs during updates.
+
+Answer: **Incomplete by design** — Initial graph supplies evidenceRefs only for observedStates; all other nodes get none. Update-phase nodes get no evidence references at all.
+
+### Referential Recoverability Trace
+
+| Step | Result | Reason |
+|------|--------|--------|
+| Graph node → evidenceId | works | Nodes carry evidenceIds as string arrays |
+| evidenceId → evidence record | breaks | Evidence records are consumed during startCase and never returned; no persistence layer retains them |
+| Evidence record → evidenceType | N/A | Chain already broken at previous step |
+
+### Answered Questions
+
+1. **Is node identity already sufficient?** No — nodes exist and have evidenceIds, but without the referenced records they carry no provenance information.
+2. **Is evidence identity already sufficient?** The evidence records themselves (when they exist) carry sufficient identity (id + evidenceType). But they are not available in case state.
+3. **Are evidence records retained long enough to resolve references?** No — consumed during construction, not returned with graph.
+4. **Is evidenceType available after resolution?** N/A — cannot resolve the reference to get there.
+5. **Are node-to-evidence links populated consistently enough?** No — only observedStates nodes in initial build receive references; all update-phase nodes receive none.
+6. **Can supplied-versus-inferred provenance currently be recovered referentially?** No.
+7. **Is the problem primarily:** More than one of these: (a) evidence provenance exists upstream during reconstruction but is not persisted alongside graph state; (b) no new evidence records are created or retained during update cycles; (c) incomplete linkage even at construction time (only observedStates nodes get references).
+
+### Conclusion
+
+**Existing provenance exists but referential linkage is incomplete** — specifically: evidence records carry sufficient identity (id + evidenceType distinguishing supplied from inferred) when they exist, but the provenance chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state, making them unrecoverable in persisted case data; and (2) no evidence records are created or retained during update cycles at all. The problem is primarily missing persistence of existing provenance combined with a gap in evidence record creation during updates.
+
+### Limitations
+
+- Source-inspection audit only; no live execution tested
+- Client-side state management was not inspected — if the client retains evidence records alongside graph state, referential recovery may work on that layer
+- Did not evaluate whether the client could reconstruct provenance from UI-visible data
+- Conclusions apply to the server-side pipeline as currently implemented
+
+### Status
+
+**Pending Rob's review.** Source-inspection complete. No production code changed. Working tree clean before commit.