experiment: audit evidence source linkage

docs/current-handoff.md

@@ -73,7 +73,7 @@ Experiment 52C separated free-language semantic understanding from enum normalis
 
 Experiment 52D isolated enum normalisation from semantic understanding: five fixed meaning statements (no decision target or question in the input) were mapped to the existing four-category contract via one live model call each. Four of five normalised to the expected enum. The compliance boundary case persisted — the model classified a "supports" relationship as `could_change_decision`, exposing genuine ambiguity between these two categories under the current definitions. The existing contract appears clear enough for a separate normalisation step; the remaining problem lies in category definitions, not semantic understanding or normalisation mechanism. Same Qwen model (`qwen-claude:latest`) and host (`http://192.168.1.111:11434`) were retained throughout. No production behaviour changed. What remains uncertain: whether the `supports_decision` ↔ `could_change_decision` boundary can be clarified without restructuring the contract, and whether the discrepancy holds under repeated runs. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-normalisation.test.js` for results.
 
-Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state — no persistence layer retains them; and (2) no evidence records are created or retained during update cycles. Experiment 54E did not validate how those values are assigned in production. Experiment 54F audited evidenceType assignment: the reconstruction prompt instructs the LLM to classify each evidence item into one of five types based on its own judgment; no production code deterministically derives evidenceType from source origin — even reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came directly from the user."
+Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state — no persistence layer retains them; and (2) no evidence records are created or retained during update cycles. Experiment 54E did not validate how those values are assigned in production. Experiment 54F audited evidenceType assignment: the reconstruction prompt instructs the LLM to classify each evidence item into one of five types based on its own judgment; no production code deterministically derives evidenceType from source origin — even reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came directly from the user." Experiment 54G audited whether evidence records nevertheless retain deterministic linkage to user words: neither verbatim text nor structured location references (character offsets, turn IDs) survive in any record field; `source` and `attribution` are free-form model-generated strings that may be null; raw user statement is not returned alongside validated results. Existing evidence records do not contain deterministic source provenance.
 
 ## 5. What Remains Open
 
@@ -127,8 +127,8 @@ Answer before continuing:
 
 ---
 
-*Created by Experiment 34. Updated by Experiments 38–53, 54A–54F. Branch: `feature/user-workspace-ux-v0.7`.*
+*Created by Experiment 34. Updated by Experiments 38–53, 54A–54G. Branch: `feature/user-workspace-ux-v0.7`.*
 
-### Return-to-Work Note (Experiment 53 → 54A → 54B → 54C → 54D → 54E)
+### Return-to-Work Note (Experiment 53 → 54A → 54B → 54C → 54D → 54E → 54F → 54G)
 
-Experiment 53 separated supplied meaning from inference in semantic output. Experiments 54A–54D located where provenance disappears from durable graph representation, confirming the prompt preserves user-source identity but the proposal schema carries no per-node provenance. Experiment 54E tested whether existing evidence IDs and evidence records could preserve provenance referentially without assuming a new node field. The reference chain works only during initial reconstruction: evidence records carry sufficient identity (id + evidenceType) at that point, but they are consumed during startCase and never returned with graph state — no persistence layer retains them. Experiment 54F audited how evidenceType values are actually assigned in production: the LLM classifies each evidence item into one of five categories based on its own judgment (CRITICAL RULE 7 in reconstruct-v0.3.md), not from deterministic derivation by production code. evidenceType is semantic model output, not reliable provenance — reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came from the user." The provenance gap has two independent causes: (1) evidence records are not retained alongside graph state; and (2) evidenceType itself is model-generated, not structurally derived from source origin. No production code changed. Branch: feature/user-workspace-ux-v0.7.
+Experiment 53 separated supplied meaning from inference in semantic output. Experiments 54A–54D located where provenance disappears from durable graph representation, confirming the prompt preserves user-source identity but the proposal schema carries no per-node provenance. Experiment 54E tested whether existing evidence IDs and evidence records could preserve provenance referentially without assuming a new node field — the reference chain breaks because evidence records are consumed during startCase and never returned with graph state. Experiment 54F established that `evidenceType` is model classification, not trustworthy provenance: the LLM classifies each evidence item into five categories based on its own judgment, so reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came from the user." Experiment 54G tested whether evidence records nevertheless retain deterministic linkage to the user's actual words. Neither verbatim text nor structured location references (character offsets, turn IDs, source record IDs) survive in any evidence record field — `source` and `attribution` are free-form model-generated strings that may be null. The raw user statement is not returned with validated results. Deterministic code cannot independently verify evidence records against user input without trusting `evidenceType`. No production code changed. Branch: feature/user-workspace-ux-v0.7. Commit hashes: 54F closure `455d6f4`, 54G `pending`. First file to inspect when resuming: `lib/reconstruction/schema.js` (evidenceRecordSchema) or `prompts/reconstruct-v0.3.md` (evidence output format).