experiment: audit evidence type provenance

docs/current-handoff.md

@@ -73,7 +73,7 @@ Experiment 52C separated free-language semantic understanding from enum normalis
 
 Experiment 52D isolated enum normalisation from semantic understanding: five fixed meaning statements (no decision target or question in the input) were mapped to the existing four-category contract via one live model call each. Four of five normalised to the expected enum. The compliance boundary case persisted — the model classified a "supports" relationship as `could_change_decision`, exposing genuine ambiguity between these two categories under the current definitions. The existing contract appears clear enough for a separate normalisation step; the remaining problem lies in category definitions, not semantic understanding or normalisation mechanism. Same Qwen model (`qwen-claude:latest`) and host (`http://192.168.1.111:11434`) were retained throughout. No production behaviour changed. What remains uncertain: whether the `supports_decision` ↔ `could_change_decision` boundary can be clarified without restructuring the contract, and whether the discrepancy holds under repeated runs. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-normalisation.test.js` for results.
 
-Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: evidence records carry sufficient identity (id + evidenceType) but are consumed during startCase and never returned alongside graph state, so the reference chain breaks — existing provenance exists but referential linkage is incomplete.
+Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state — no persistence layer retains them; and (2) no evidence records are created or retained during update cycles. Experiment 54E did not validate how those values are assigned in production. Experiment 54F audited evidenceType assignment: the reconstruction prompt instructs the LLM to classify each evidence item into one of five types based on its own judgment; no production code deterministically derives evidenceType from source origin — even reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came directly from the user."
 
 ## 5. What Remains Open
 
@@ -127,8 +127,8 @@ Answer before continuing:
 
 ---
 
-*Created by Experiment 34. Updated by Experiments 38–53, 54A–54E. Branch: `feature/user-workspace-ux-v0.7`.*
+*Created by Experiment 34. Updated by Experiments 38–53, 54A–54F. Branch: `feature/user-workspace-ux-v0.7`.*
 
 ### Return-to-Work Note (Experiment 53 → 54A → 54B → 54C → 54D → 54E)
 
-Experiment 53 separated supplied meaning from inference in semantic output. Experiments 54A–54D located where provenance disappears from durable graph representation, confirming the prompt preserves user-source identity but the proposal schema carries no per-node provenance. Experiment 54E tested whether existing evidence IDs and evidence records could preserve provenance referentially without assuming a new node field. The reference chain works only during initial reconstruction: evidence records carry sufficient identity (id + evidenceType) at that point, but they are consumed during startCase and never returned with graph state — no persistence layer retains them. No production code changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect when resuming: lib/graph/orchestrator.js to evaluate where evidence records should be returned alongside case state so the referential chain does not break.
+Experiment 53 separated supplied meaning from inference in semantic output. Experiments 54A–54D located where provenance disappears from durable graph representation, confirming the prompt preserves user-source identity but the proposal schema carries no per-node provenance. Experiment 54E tested whether existing evidence IDs and evidence records could preserve provenance referentially without assuming a new node field. The reference chain works only during initial reconstruction: evidence records carry sufficient identity (id + evidenceType) at that point, but they are consumed during startCase and never returned with graph state — no persistence layer retains them. Experiment 54F audited how evidenceType values are actually assigned in production: the LLM classifies each evidence item into one of five categories based on its own judgment (CRITICAL RULE 7 in reconstruct-v0.3.md), not from deterministic derivation by production code. evidenceType is semantic model output, not reliable provenance — reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came from the user." The provenance gap has two independent causes: (1) evidence records are not retained alongside graph state; and (2) evidenceType itself is model-generated, not structurally derived from source origin. No production code changed. Branch: feature/user-workspace-ux-v0.7.