confidence-engine/docs/archive/experiments/vol-1-chapters/ch10/experiments-54a-to-54k-provenance-audit.md

Experiment 54A — Audit Existing Graph Provenance Only (2026-08-07)

Experiment 53 showed that the semantic model can keep supplied meaning and possible inference separate in its output. The active graph compatibility question remained unknown. Experiment 54A was an inspection-only experiment to determine whether the current validated SituationGraph distinguishes information supplied by the user or evidence from information inferred by the model.

Hypothesis

The current graph may distinguish known/provisional and supported/unsupported without actually recording where information came from. If true, current graph state can represent epistemic status but not reliably recover supplied-versus-inferred provenance.

Files Inspected

• docs/current-handoff.md
• lib/graph/schema.js — SituationGraph and node schema definitions
• lib/graph/builder.js — production initial graph builder (how nodes are populated from reconstruction)
• lib/graph/update-proposal.js — LLM output parsing for graph updates
• lib/reconstruction/schema.js — evidenceRecordSchema, reconstructionV2Schema

Graph Vocabulary Relevant to Provenance

Existing relevant node kinds:

• observation, reported_claim, metric, state, transition, relationship, assumption, unknown, conclusion

Existing relevant status fields:

• known, unknown, provisional, supported, weakened, contradicted, resolved

Existing confidence fields:

• low, medium, high

Existing evidence / relationship fields:

• evidenceIds: array of strings (graph reference IDs from reconstruction)
• dependsOn: array of node IDs
• affects: array of node IDs
• parentId: nullable string
• childIds: array of node IDs
• Edge types: supports, weakens, contradicts, depends_on, causes, may_cause, measures, compares_with, updates, other

Explicit Supplied-Information Provenance Exists: No

No field or combination of fields in the SituationNode schema has documented or implemented meaning that is "this content was supplied by the user or evidence source." The kind field distinguishes semantic categories (observation vs assumption vs unknown), not provenance. A node with kind=assumption describes what kind of claim it is, not who produced it.

Explicit Inferred-Information Provenance Exists: No

No field or combination has documented or implemented meaning that is "this content was inferred or proposed by the model and is not established evidence." The LLM-inferred nodes flow through proposal.addedNodes into the graph with kinds determined by the LLM — but those kinds are semantic labels, not provenance markers.

Status Versus Provenance Finding

Fields like provisional, supported, assumption (as a kind), and confidence describe epistemic status only — they classify how confident or well-supported a claim is. They do not record where the information originated. A node with kind=unknown, status=unknown, confidence=low could have come from user input, model inference, or evidence extraction.

Are evidenceIds Provenance or Graph References

Graph references. In buildInitialGraph, evidenceIds are populated from obs.id — IDs that originate from the LLM's reconstruction output (reconstruction.observedStates[].id). These are internal identifiers for model-generated evidence records, not user-supplied source identifiers. The same applies during graph updates: node relationships use string IDs that are graph-internal references.

Are Inferred Nodes Explicitly Marked as Model-Generated

No. Neither builder.js (initial build) nor the update-proposal path marks inferred nodes with any model-generated flag. Node kinds in the update path are set by the LLM's JSON output — there is no explicit "this was model-inferred" marker.

Recoverability Result: not_recoverable

A later consumer receiving only the validated graph (with no conversation history or LLM response) cannot determine which statements came from user/evidence and which were generated as model inference. All nodes produced by different paths (initial build, emergent reasoning, decomposition children) have identical schema shape. The evidenceIds field contains IDs referencing model-generated reconstruction records, not external source identifiers.

Production Population Finding

• No relevant source/provenance fields are populated in production graph-building code
• Node kinds (observation, assumption, unknown, etc.) are used for semantic typing, not provenance
• Evidence IDs are model-generated internal references (not user-supplied identifiers)
• No inferred nodes carry any explicit model-generated marker

Experiment Conclusion

The existing SituationGraph does not preserve supplied-versus-inferred provenance. It represents epistemic status (how confident or well-supported information is) but has no mechanism to record where information originated. This confirms the hypothesis from Experiment 53's open question: while semantic output can separate supplied meaning from inference, the graph layer cannot recover that separation because it lacks provenance tracking fields entirely.

Limitations

• Inspection-based; no live model run was performed
• Only source files directly relevant to node schema and construction were examined
• The non-strict Zod schema allows extra fields but none are used for provenance in production code
• Does not address whether a fix is needed — only whether the gap exists

Status

Pending Rob's review. The audit confirms a provenance gap. No production code was changed. Working tree clean before commit.

Production Unchanged

• lib/graph/schema.js: 0 lines changed
• lib/graph/builder.js: 0 lines changed
• lib/graph/apply-proposal.js: 0 lines changed
• lib/graph/update-proposal.js: 0 lines changed
• No production files modified
• Working tree clean before commit

Tests / Validation Run

No test run required for the inspection result. Source inspection alone is sufficient — the schema definition in lib/graph/schema.js is a static contract, and no runtime execution is needed to confirm the absence of provenance fields.

Documentation Updated

• docs/current-handoff.md — handoff line 127 and Return-to-Work Note updated
• docs/design-evolution-log.md — Experiment 54A section appended

Experiment 54B — Trace Provenance Loss Through Graph Pipeline (2026-08-07)

Experiment 54A proved the provenance gap exists in the graph. This experiment traced both flows to find exactly where upstream provenance is lost and whether it is recoverable at any point.

Method

Inspected file-by-file through the complete data flow of both paths, tracking the evidenceType field from its creation in the reconstruction layer through to the final graph state.

Source-traced paths:

• Flow A (initial): analysis.js → buildInitialGraph() → SituationGraph.nodes
• Flow B (update): orchestrator.updateCase() → prompt → parseGraphUpdateProposal() → applyValidatedProposal() → SituationGraph.nodes/edges

Trace Table

Step	File	Data Present?	Provenance Status
LLM output raw	analyseScenario() lib/analysis.js:28-134	reconstruction + evidence with evidenceType enum	PRESENT (supplied vs inferred explicit in evidence records)
Validation v0.2 schema	lib/reconstruction/schema.js:129-162	EvidenceRecordSchema includes evidenceType: ["direct_observation","reported_statement","interpretation","assumption","inferred_relationship"]	PRESENT — enum encodes the distinction upstream
buildSuccessResultV2 return	lib/analysis.js:173-184	{reconstruction, evidence} returned	PRESENT in both paths
buildInitialGraph receives data	lib/graph/builder.js:17-18	reconstruction + evidence map built (line 62)	UPSTREAM AVAILABLE
Nodes created with kind/status	lib/graph/builder.js:40-54 (ensureNode), lines 75-209	Nodes get kind/status/confidence from semantic mapping of reconstruction fields (observedStates→observation, actors→observation, systemsOrObjects→metric, etc.)	LOST — no provenance field on nodes
EvidenceMap built but unused	lib/graph/builder.js:61-64	evidenceMap populated with evidence records (ev.id → ev)	DEAD CODE — never queried after construction
addEvidenceToNode called	lib/graph/builder.js:66-70, 100	Only obs.id pushed to node.evidenceIds as string reference	LOST — ID only, no type metadata transferred
buildMinimalGraph fallback	lib/graph/builder.js:257-280	No evidence at all; nodes created from scenario text directly	NO UPSTREAM PROVENANCE AVAILABLE
Orchestrator startCase → makeGraph	lib/graph/orchestrator.js:383	centralStatement = scenario string preserved at graph root	PARTIAL — only the raw scenario survives as centralStatement
updateCase receives answer	lib/graph/orchestrator.js:569-758+	Answer parameter enters orchestrator, included in prompt to LLM	SUPPLIED ANSWER TEXT present in prompt
LLM proposes graph updates	based on prompt content including answer	No separation of user-supplied vs model-inferred in proposal	LOST at prompt construction boundary
parseGraphUpdateProposal output	lib/graph/update-proposal.js:100-157	GraphUpdateSchema with addedNodes using situationNodeSchema	NO provenance on added nodes or edges
applyValidatedProposal receives answer	lib/graph/apply-proposal.js:2723	answer parameter passed through (line 2723), used in deriveReasoningStateOverride (line 2875)	PRESENT but NOT used for provenance — only affects reasoning state derivation
applyGraphUpdate applies changes	lib/graph/apply-proposal.js, line 2882+	Graph modified; no new provenance fields added	LOST — nodes/edges created without source metadata

Provenance Loss Summary

Flow A (initial graph):

• Upstream of graph: evidenceType enum explicitly distinguishes supplied from inferred in evidenceRecordSchema
• First loss point: buildInitialGraph() at lib/graph/builder.js — nodes are created with semantic kind/status but NO provenance field. The evidenceMap is built (line 62) but never used. Only obs.id is added to node.evidenceIds as a bare string reference without type information.
• Recovery path: NOT from graph state alone. Would require the upstream reconstruction + evidence array that was consumed during initial build.

Flow B (update):

• User answer enters as answer parameter in orchestrator, flows through LLM prompt, becomes part of a proposal with no provenance metadata on nodes or edges
• No separation between "user supplied this text" and "model proposed these graph changes" anywhere in the update pipeline
• The answer reaches applyValidatedProposal (line 2723) and is used for reasoning state derivation (line 2875), but no provenance field is added to nodes/edges

EvidenceIds Clarification Conclusion

The node-level evidenceIds does NOT represent "the list of nodes from which this information was derived." Instead:

• evidenceIds is a reference list — each string in the array is an ID that references a specific record in the upstream evidence array
• The distinction between supplied and inferred information lives in the evidenceType field of those evidence records, not on the node itself
• A node's kind/status fields encode epistemic classification (what role does this node play and how confident are we), NOT provenance (where did this information come from)
• To recover whether a piece of information was user-supplied or model-inferred, you must look up each evidenceId in the original evidence array and check its evidenceType field — which means provenance recovery depends on access to the upstream evidence data, not on the graph state alone

This is actually useful: it clarifies that provenance IS recoverable from validated reconstruction output (the reconstructed data includes an evidence array where each record has evidenceType), but it is NOT recoverable from graph state alone. The evidenceIds array is a bridge to upstream provenance, not provenance itself.

Key Findings

1. Supplied-vs-inferred provenance EXISTS upstream. evidenceRecordSchema (lib/reconstruction/schema.js line 116-121) has an explicit enum: direct_observation, reported_statement (supplied categories) vs interpretation, assumption, inferred_relationship (inferred categories). This is the most important finding.

2. First provenance loss point is buildInitialGraph(). In lib/graph/builder.js lines 61-70, the evidenceMap is built but dead-coded — never queried. Only ID strings are added to node.evidenceIds without type metadata.

3. Update flow has no provenance preservation. The user's answer arrives as a parameter but becomes embedded in an LLM prompt with no traceability. No node receives source metadata during updates.

4. Provenance recovery requires upstream data, not graph state. Since the SituationGraph schema has no provenance fields and nodes only carry ID references to evidence records, any provenance determination must reference the original reconstruction or update evidence array — it cannot be derived from the graph alone.

Experiment Conclusion

The supplied-versus-inferred distinction is fully preserved in the upstream reconstruction/update pipeline output (the validated evidence arrays carry explicit evidenceType values for each record). However, this distinction is never encoded into the SituationGraph nodes during either initial build or update application. The evidenceIds field on nodes provides indirect access to provenance via ID references, but only if the original evidence data remains available downstream of the graph.

The core insight: provenance is not lost from the pipeline — it is preserved in the reconstruction output that feeds the builder. It IS lost when the builder converts that output into a SituationGraph because the node schema has no field to receive it. This means any future implementation would need to add a provenance-bearing field to the node schema and propagate evidenceType through buildInitialGraph and applyValidatedProposal — though the specific implementation approach (field name, placement, propagation mechanism) remains undecided.

Limitations

• Inspection-based; no live model run required
• Traced production paths only (builder.js, apply-proposal.js, update-proposal.js)
• Did not examine LLM prompt templates to determine if they preserve answer-supplied vs inference distinction in output formatting
• The analysis assumes evidence records remain accessible after graph construction — downstream usage patterns were not audited

Status

Pending Rob's review. Tracing complete. No production code changed. Working tree clean before commit.

Production Unchanged

• lib/reconstruction/schema.js: 0 lines changed
• lib/graph/builder.js: 0 lines changed
• lib/graph/apply-proposal.js: 0 lines changed
• lib/graph/orchestrator.js: 0 lines changed
• No production files modified
• Working tree clean before commit

Tests / Validation Run

No test run required — this is a source-trace audit confirming data flow paths, not a behavioral test.

Documentation Updated

• docs/current-handoff.md — Return-to-Work Note updated with 54B findings
• docs/design-evolution-log.md — Experiment 54B section appended

Report 54B — Trace Provenance Loss Through Graph Pipeline

Experiment Purpose

Trace where the supplied-versus-inferred provenance distinction is lost in both the initial graph build and update flows, determine whether it is recoverable at any point in the pipeline, and document what future work must do to address the gap.

Method

File-by-file source inspection of the complete data flow for both paths: Flow A (analyseScenario → buildInitialGraph → SituationGraph) and Flow B (updateCase → LLM prompt → parseGraphUpdateProposal → applyValidatedProposal).

Provenance Existence Upstream

Yes. The evidenceRecordSchema in lib/reconstruction/schema.js lines 116-121 defines an explicit enum: direct_observation, reported_statement, interpretation, assumption, inferred_relationship. Supplied categories (direct_observation, reported_statement) are separate from inferred categories (interpretation, assumption, inferred_relationship). This distinction is preserved in the validated reconstruction output returned by analyseScenario and consumed by buildInitialGraph.

First Provenance Loss Point

• Flow A (initial graph): lib/graph/builder.js lines 61-70. The evidenceMap is built at line 62 from all evidence records but never queried after construction. Only the raw ID string (e.g., obs.id) is added to node.evidenceIds via addEvidenceToNode — no type metadata or provenance classification is transferred to the node schema, which has no provenance field defined in lib/graph/schema.js lines 55-70.

• Flow B (update): lib/graph/orchestrator.js where updateCase passes the user answer into an LLM prompt without separating "user-supplied" from "model-inferred" content, and lib/graph/update-proposal.js where graphUpdateSchema builds addedNodes using situationNodeSchema which has no provenance field. The answer parameter reaches applyValidatedProposal (lib/graph/apply-proposal.js line 2723) and is used in deriveReasoningStateOverride (line 2875), but no provenance metadata is attached to nodes or edges during the update application.

EvidenceIds Clarification Conclusion

The node-level evidenceIds field does not represent "the list of nodes from which this information was derived." Instead, each string in evidenceIds is a reference ID that points to a specific record in the upstream evidence array. The supplied-versus-inferred distinction lives on those upstream records (their evidenceType enum field), not on the node itself. To recover whether a piece of information was user-supplied or model-inferred requires accessing the original reconstruction or update evidence array and checking each referenced record's evidenceType — provenance recovery therefore depends on access to upstream data, not on the graph state alone.

Update Flow Answer Handling

The user answer enters orchestrator.updateCase() as a parameter, gets embedded in an LLM prompt without source attribution markers, and the resulting proposal carries no provenance metadata onto nodes or edges. The answer survives as a raw string through to applyValidatedProposal (lib/graph/apply-proposal.js line 2723) where it influences reasoning state derivation (line 2875), but no node receives any indication that it was derived from user-supplied content versus model inference.

Provenance Recovery Feasibility

From graph alone: No — the SituationGraph schema has no provenance fields on nodes or edges, and evidenceIds only carries ID references without type metadata. From upstream data: Yes — the validated reconstruction output (returned by analyseScenario) includes a complete evidence array where each record has an explicit evidenceType enum field distinguishing supplied from inferred information.

Future Work Consideration (Beyond 54B Scope)

Any future fix would need to embed provenance in graph state rather than keeping it only in external upstream data — potentially via a node-level provenance field and evidenceType propagation through buildInitialGraph and applyValidatedProposal, while retaining the existing evidenceIds reference system as a cross-reference layer. The specific implementation approach remains undecided. 54C examines whether update provenance is deterministically knowable at the application boundary before any fix is designed.

Status

Committed. Report appended to design log. Branch: feature/user-workspace-ux-v0.7. Working tree clean before commit.

Experiment 54C — Is Update Provenance Deterministically Knowable Before Graph Application? (2026-08-07)

Objective

Determine whether production code can distinguish user-supplied material from model-proposed additions at the boundary before graph mutation occurs. This is source-trace only; no code changed, no tests run, no solution designed.

Hypothesis

The active update pipeline holds the raw user answer and the validated model proposal as distinct inputs immediately before graph mutation. If so, origin may be deterministically knowable at that boundary even though the current graph does not store it.

Files Actually Inspected

• lib/graph/orchestrator.js — lines 569–768 (updateCase / updateCaseWithDependencies)
• lib/graph/apply-proposal.js — lines 2719–2912 (applyValidatedProposal)
• lib/graph/update-proposal.js — lines 1–157 (parseGraphUpdateProposal, graphUpdateSchema field list)
• lib/graph/schema.js — lines 55–98 (situationNodeSchema, situationEdgeSchema), lines 155–163 (graphUpdateSchema), lines 174–179 (updateCaseRequestSchema)
• docs/design-evolution-log.md — Experiment 54B section (lines 5080–5178)
• docs/current-handoff.md — lines 129–131 (Return-to-Work Note)

Production files actually inspected: lib/graph/orchestrator.js, lib/graph/apply-proposal.js, lib/graph/update-proposal.js, lib/graph/schema.js. Reconstruction files NOT inspected: lib/reconstruction/schema.js, lib/analysis.js, lib/graph/builder.js (per budget constraints).

Update Flow Trace

Stage	What contains the user answer?	What contains model proposals?	Are they separate?	Origin deterministically knowable?
HTTP request body → updateCaseWithDependencies (orchestrator.js:594)	answer from parsedRequest.data.answer (schema: z.string().min(1).max(5000))	—	Yes	N/A — no proposal yet
LLM prompt construction (orchestrator.js:633-638)	answer embedded in prompt text as context	Model generates response with additions/proposals	Separated by mechanism: answer is context, model output is the new data	Only by knowing that answer was supplied and rawResponse came from the LLM. No metadata markers separate user text from model inference within the proposal itself.
parseGraphUpdateProposal (update-proposal.js:100-157)	Not present in parsed output — stripped during JSON parsing	parsedProposal.proposal (graphUpdateSchema: addedNodes, updatedNodes, addedEdges, etc.)	N/A — user answer is gone from the parsed proposal object	Merged_or_lost — the raw user answer is no longer part of the proposal object
Orchestrator call to applyValidatedProposal (orchestrator.js:683-688)	answer passed as separate function argument	proposal: parsedProposal.proposal passed as separate function argument	explicitly_separate — two distinct named parameters in one function call	implicitly_distinguishable — parameter names distinguish them, but the proposal object itself contains no metadata labeling which nodes/edges came from user input vs model inference
Inside applyValidatedProposal (apply-proposal.js:2719-2880)	answer used only in deriveReasoningStateOverride (line 2875-2878). No provenance metadata derived from it.	proposal (validated against graphUpdateSchema, reconciled via reconcileResolutionSemantics) — then snapshot at line 2869	Still explicitly_separate within the function scope	The two inputs are separate variables, but the proposal object carries no origin labels on its nodes/edges
applyGraphUpdate calls (apply-proposal.js:2882, 2912)	answer not passed to applyGraphUpdate	proposalSnapshot applied to graphSnapshot. Nodes created without source metadata.	N/A — applyGraphUpdate receives only the merged snapshot	merged_or_lost — origin information is not transmitted to the mutation function
Resulting SituationGraph (after line 2912)	No record of which nodes/edges came from user	All new nodes/edges carry no provenance field	N/A	The graph stores only structural data; origin is unrecoverable from graph state alone

Answers to Required Questions

1. Is the raw user answer still available immediately before proposal application? Yes. In orchestrator.js line 683-688, answer (from parsedRequest.data.answer) is passed as a named argument to applyValidatedProposal alongside proposal. Both exist as separate function arguments at the call site.

2. Is the validated model proposal a separate object at that same point? Yes. parsedProposal.proposal is a distinct object from answer. It is the output of parseGraphUpdateProposal, validated against graphUpdateSchema, and passed as the proposal argument. The answer and proposal are different values in the JavaScript call stack.

3. Does applyValidatedProposal receive both, or only the proposal/graph? Both. The function signature (line 2719) receives { situationGraph, proposal, previousQuestion, answer }. All four are separate destructured parameters.

4. Can deterministic code identify "user supplied" versus "model proposed" without asking the LLM? At the applyValidatedProposal call boundary: yes, by parameter identity. The answer argument contains user-supplied text; the proposal argument contains model-generated graph changes. These are distinguishable because they are different variables in the JavaScript runtime and come from different sources in orchestrator.js (user request vs LLM response).

However: within the proposal object itself, there is no metadata on individual nodes or edges indicating whether a specific node originated from user-supplied information or was model-inferred. The proposal schema (graphUpdateSchema) has no provenance field on addedNodes or addedEdges — situationNodeSchema contains only structural fields (id, label, description, kind, status, confidence, value, unit, evidenceIds, dependsOn, affects, parentId, childIds).

5. At what exact function boundary does that distinction cease to be recoverable? The distinction is knowable at the orchestrator.updateCaseWithDependencies call site (line 683) because both answer and proposal are separate named arguments. It becomes merged_or_lost at two points:

a) Inside applyValidatedProposal: the answer parameter is passed only to deriveReasoningStateOverride and never used to annotate nodes/edges with source metadata. Origin information exists in scope but is not applied to the graph mutation path.

b) At applyGraphUpdate calls (lines 2882, 2912): only graphSnapshot and proposalSnapshot are passed. The answer parameter is discarded — it never reaches the mutation function that creates nodes/edges.

6. Is the loss caused by which factor? More than one boundary. Specifically:

• Proposal schema (graphUpdateSchema/situationNodeSchema): no provenance field exists on addedNodes or addedEdges — this is the primary structural cause. If a provenance field existed, it could be populated.
• Application function signature (applyGraphUpdate at lines 2882/2912): answer is not passed to the mutation function that actually creates graph state. Even if nodes had provenance fields, the source information would need to be carried through to reach them.
• Graph schema: the resulting situationGraph has no provenance-aware structure (follow-on effect of proposal schema gap).

7. Does the update path already contain enough information to assign provenance deterministically before graph storage? No. While user answer and model proposal are separate at the applyValidatedProposal call boundary, the answer text is a free-form string with no structural mapping to specific nodes in the proposal. The orchestrator does not know which parts of the LLM response were derived from user input versus independently inferred by the model. Even though both inputs exist as distinct parameters, there is no deterministic mechanism within the data flow to map user-supplied content to specific graph nodes/edges in the proposal.

Experiment Conclusion

Input origin remains explicit before application, but per-node supplied-versus-inferred provenance is not represented in the validated proposal and is lost before durable graph mutation.

The raw user answer and validated model proposal are explicitly separate at the applyValidatedProposal function call boundary (orchestrator.js:683-688). Whole-input origin is explicit: answer = user supplied; proposal = model produced. However, per-node origin inside the proposal is not deterministically recoverable from the validated proposal alone. This distinction does not translate to per-node provenance because:

1. The graphUpdateSchema / situationNodeSchema has no provenance field on nodes or edges.
2. The raw user answer text has no structural mapping to proposal node boundaries — the LLM consumes the answer as context and generates additions independently, so there is no way to deterministically say "this node contains user information" versus "this node contains model inference."
3. The answer parameter is not forwarded to applyGraphUpdate, the function that actually mutates graph state.

Provenance loss occurs at the intersection of proposal schema (no provenance field) and application logic (answer discarded before mutation).

Limitations

• Source-trace audit only; no live model or parsing executed
• Did not examine prompt templates to determine if they carry answer-supplying markers
• Did not examine reconcileResolutionSemantics for any implicit origin tagging
• Did not examine applyGraphUpdate internals beyond the call signatures

Status

Pending Rob's review. Source-trace complete. No production code changed. Working tree clean before commit.

Production Unchanged

• lib/graph/orchestrator.js: 0 lines changed
• lib/graph/apply-proposal.js: 0 lines changed
• lib/graph/update-proposal.js: 0 lines changed
• lib/graph/schema.js: 0 lines changed
• No production files modified
• Working tree clean before commit

Tests / Validation Run

No test run required; Experiment 54C is a source-trace audit.

Experiment 54D — Does the Update Prompt Already Preserve User-vs-Model Origin? (2026-08-07)

Objective

Inspect the production graph-update prompt to determine whether it marks which content is the user's answer versus model-generated interpretation strongly enough that provenance could, in principle, be preserved downstream. This is a source-inspection experiment only. Do not implement provenance.

Two distinct questions:

• Prompt-level source identity: Can the model tell "this text is the user's answer"?
• Proposal-level provenance: Can downstream code tell "this particular proposed node directly represents supplied content rather than model inference"?

These may have different answers. The first may be explicit while the second is absent.

Hypothesis

The existing update prompt may already contain clearly separated sections such as: previous graph/context; current question; user answer; instructions for graph changes. If that structure is explicit, the upstream information needed to distinguish user-supplied input from model-generated additions may already exist at prompt time. If the prompt blends everything into undifferentiated text, provenance is weaker even before proposal parsing.

Files Actually Inspected

• lib/graph/orchestrator.js — lines 617-638 (caller passing answer to buildPrompt); line 20 (import statement)
• lib/graph/prompt-builder.js — full file (buildGraphUpdatePrompt function and its helpers: formatEnumValues, formatGraph, formatExampleAnswerBlock)

Production files inspected: lib/graph/orchestrator.js, lib/graph/prompt-builder.js.

Prompt Builder Function Inspected

Function: buildGraphUpdatePrompt in lib/graph/prompt-builder.js, exported as buildGraphUpdatePrompt (line 26), aliased as buildUpdatePrompt (line 134).

Arguments Passed Into Prompt Builder

• situationGraph — full current SituationGraph object
• previousQuestion — string (the previously selected question)
• answer — string (raw user answer, min 1 char, max 5000 chars per updateCaseRequestSchema)
• promptVersion — string, defaults to "v0.4"

The caller in orchestrator.js:633 passes these four arguments directly from function parameters and a config value. No provenance metadata is constructed or passed at the call site.

User Answer Source Identity in Prompt

Status: explicit

Section ## User Answer (line 48-49 of prompt-builder.js) contains the raw user answer as its entire content, separated by a Markdown header from everything above and below. The section header unambiguously identifies the block as user-supplied text. No other section contains this exact string.

Previous Question Separation in Prompt

Status: explicit

Section ## Previous Selected Question (line 45-46) contains only the previous question string, clearly separated by a Markdown header from both the graph above and the answer below.

Graph/Context Separation in Prompt

Status: explicit

Section ## Current Situation Graph (line 42-43) contains the full situation graph as formatted JSON, clearly separated by a Markdown header from all other content.

Instruction Versus User-Content Separation

Status: explicit

All instruction blocks use Markdown headers (## Proposal Rules, ## Allowed Node Kinds, ## Additional Guidance, etc.). These headers create visual and structural boundaries between user-provided sections (graph, question, answer) and system instructions. The prompt does not interleave instructions within user-content blocks.

Does Prompt Explicitly Identify User-Supplied Content

Status: explicit

Yes. The ## User Answer header unambiguously marks which text block is the user's contribution. Additionally, Proposal Rule 9 states "Every new unknown must be directly traceable to the user's answer," and Rule 13a references "the relevant answer-derived decision or context node" — both rules reinforce that the answer section represents the authoritative user-supplied source.

Does Prompt Explicitly Distinguish Supplied Meaning from Model Inference

Status: implicit

The prompt does not contain an explicit instruction telling the model to label or separate supplied meaning from inference in its output. However, implicit cues exist: Rule 9 requires traceability ("directly traceable to the user's answer"), Rule 9a requires a why-it-matters clause for new unknowns (implying the model must reason about what it derives versus what is given), and Rule 13a references "answer-derived" nodes. These create an expectation that the model should distinguish derived from supplied content, but there is no structural output mechanism to preserve that distinction in the JSON proposal.

Does Requested Proposal Output Contain Provenance

Status: absent

The graphUpdateSchema (defined in lib/graph/schema.js, lines 155-163) has no provenance or source fields on any of its node or edge schemas. The addedNodes schema uses situationNodeSchema which contains only structural fields (id, label, description, kind, status, confidence, value, unit, evidenceIds, dependsOn, affects, parentId, childIds). No field exists to tag content as "user-supplied," "model-inferred," or any equivalent origin marker.

Prompt-Level Source Identity Status

explicit — The model can clearly identify which input text came from the user (the ## User Answer section) and which sections contain context/instructions (the graph JSON, previous question, rules, guidance).

Proposal-Level Provenance Status

absent — The proposed output schema has no provenance fields. Even if the model understands which inputs were user-supplied, it has no mechanism to annotate its output nodes/edges with origin information.

Trace from User Answer to Validated Proposal

Stage	Source Identity	Supplied-vs-Inferred Meaning Explicit?
1. answer parameter in orchestrator.js:636	explicit (parameter name)	N/A — raw string
2. ## User Answer section in prompt (prompt-builder.js:49)	explicit (named Markdown section)	implicit — the answer is given; no instruction distinguishes parts of it as supplied vs inferred
3. LLM processes prompt and generates proposal	explicit (model can see which text is user answer)	implicit — rules require traceability but do not provide output mechanism for provenance
4. parsedProposal.proposal after parseGraphUpdateProposal	absent (no source metadata on nodes/edges)	absent — graphUpdateSchema has no provenance fields

First Point Where Per-Node Provenance Becomes Unavailable

The proposed output schema (graphUpdateSchema in lib/graph/schema.js) defines the JSON contract returned by the LLM. Since none of its node or edge schemas include any origin/provenance field, per-node provenance is unavailable at the output definition stage — i.e., the prompt's own requested format cannot carry provenance even if the model understands it internally. This is upstream of parsing and validation; even before parseGraphUpdateProposal runs, the schema itself forbids provenance encoding.

Could the Model Know Which Input Came from the User

Yes. The ## User Answer section makes the user's contribution unmistakably identifiable. Rules 9 and 13a further reinforce the distinction between answer-derived content and model-generated additions.

Could Downstream Deterministic Code Know Which Proposed Node Came from Supplied Meaning

No. The graphUpdateSchema has no provenance field on addedNodes, updatedNodes, or addedEdges. The validated proposal is a plain JSON object with no origin metadata. There is no deterministic mechanism to recover per-node provenance from the proposal alone.

Experiment Conclusion

Prompt clearly preserves user-source identity but proposal schema loses per-node provenance.

The production update prompt (prompt-builder.js) already separates the user answer into a distinct named section (## User Answer) with clear visual and structural boundaries from graph context, instructions, and constraints. The model can unambiguously identify which text is user-supplied. Rules 9 and 13a reinforce traceability expectations.

However, the requested output schema (graphUpdateSchema in lib/graph/schema.js) has no provenance fields on nodes or edges. Even if the model internally distinguishes derived from supplied content, the JSON output contract cannot encode that distinction. Provenance is lost at the output-definition stage — before any parsing or validation occurs.

This means:

• Prompt-level source identity: explicit
• Proposal-level provenance: absent (structural limitation of schema)
• The tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided.

Limitations

• Source-inspection audit only; no live model call executed
• Inspected the production prompt-builder and its immediate caller only
• Did not inspect whether evidenceType in reconstruction can carry origin information for the answer field itself
• Did not evaluate whether a schema change would be sufficient or whether additional upstream markers are needed
• Conclusions apply to the current prompt version (v0.4); earlier versions may differ

Status

Pending Rob's review. Source-inspection complete. No production code changed. Working tree clean before commit.

Production Unchanged

• lib/graph/orchestrator.js: 0 lines changed
• lib/graph/prompt-builder.js: 0 lines changed
• lib/graph/schema.js: 0 lines changed
• No production files modified
• Working tree clean before commit

Tests / Validation Run

No test run required; Experiment 54D is a prompt-source audit.

Experiment 54E — Can Existing Evidence References Preserve Provenance Without Adding a Node Field? (2026-08-07)

Objective

Inspect whether the current graph design already preserves referential provenance through existing evidence identities and references, without requiring a new node field. This is a source-inspection experiment only. Do not implement provenance.

Hypothesis

Existing evidenceIds and evidence records may already provide enough referential structure to preserve provenance if: (1) evidence records survive beyond reconstruction; (2) their IDs remain resolvable later; (3) evidenceType remains attached to those records; (4) graph nodes can reliably link to the evidence records that justify them.

Files Inspected

• lib/reconstruction/schema.js — evidenceRecordSchema definition, reconstructionV2Schema evidence field
• lib/graph/schema.js — situationNodeSchema evidenceIds field, graphUpdateSchema structure
• lib/graph/builder.js — how evidence IDs are assigned during initial graph build (buildInitialGraph)
• lib/graph/orchestrator.js — startCase and updateCaseWithDependencies return values; evidence flow through the pipeline
• lib/reconstruction/schema.js lines 113–127 — evidenceRecordSchema fields
• lib/graph/apply-proposal.js — how evidenceIds are populated during graph updates (appendUniqueValue pattern)
• lib/graph/prompt-builder.js — rule 14 "Do not invent evidence"

Evidence Record Identity

Each reconstruction evidence record (v0.2 schema) has:

• Stable ID: id: z.string().min(1) — deterministic identifier present in every record.
• evidenceType: z.enum(["direct_observation", "reported_statement", "interpretation", "assumption", "inferred_relationship"]) — five distinct values.
• Sufficient information to distinguish supplied from inferred: The evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but Experiment 54E did not validate how those values are assigned in production.

Answer: Yes, evidence records carry sufficient identity.

Graph Reference Behaviour

When a graph node contains an evidenceId:

• During buildInitialGraph (builder.js:62–70), evidence records from the analysis output are placed in an evidenceMap. Nodes are created with evidenceIds populated from actual evidence record IDs (builder.js💯 node.evidenceIds.push(obs.id)).
• These IDs refer to real evidence records during construction. However, after graph construction, the graph contains only string IDs — they would resolve if a lookup table existed, but they are opaque strings within the node object itself.

Answer: IDs refer to real evidence records at build time; become opaque strings in the graph post-construction.

Evidence-Record Lifetime

The critical flow is:

1. analyseScenario (lib/analysis.js) produces evidence records via reconstructionV2Schema, returned as data.evidence.
2. In startCase (orchestrator.js:378), analysis.evidence is passed to buildInitialGraph solely to populate node evidenceIds.
3. The evidence records themselves are NOT included in startCase return value. The return at orchestrator.js:489-566 includes only situationGraph, selectedQuestion, diagnostics, and assessment. No evidence field exists in the return object.
4. In updateCaseWithDependencies, no new evidence records are created anywhere in the pipeline. Rule 14 of the update prompt ("Do not invent evidence") explicitly forbids the model from creating them. The prompt-builder.js shows no mechanism for evidence generation during updates.
5. There is no persistence layer in this codebase that stores case state to disk or a database. The API routes (cases/start/route.js, cases/update/route.js) return data to the client; they do not persist anything.

The initial evidence records exist only during the startCase execution lifetime. They are consumed to populate graph node evidenceIds but never returned alongside the case state. Subsequent update cycles produce no evidence records at all.

Answer: not_retained_with_graph

Provenance Through Reference

If a later consumer receives normal persisted case state (which contains only situationGraph with nodes having string evidenceIds), it cannot:

1. Read a graph node — yes, the node and its evidenceIds are present.
2. Follow its evidenceIds — no matching evidence records exist anywhere in the persisted state.
3. Resolve each ID to an evidence record — impossible; records do not exist.
4. Inspect evidenceType — N/A; no records to inspect.

Answer: no — The reference chain breaks at step 2/3 because evidence records are not part of the returned case state.

Node-to-Evidence Completeness

• Initial graph nodes (from buildInitialGraph): Nodes from observedStates do receive evidence references (builder.js:100). Other node types (actors, systemsOrObjects, differences, contradictions, unknowns, interpretations) are created WITHOUT evidence references — only observedStates nodes get evidenceIds populated.
• Nodes added during update: No new evidence records are ever created during the update flow. Nodes created via applyValidatedProposal may have their evidenceIds field set (the schema allows it), but no source code in the update path populates them from actual evidence records. There is no mechanism to create or assign evidence record IDs during updates.

Answer: Incomplete by design — Initial graph supplies evidenceRefs only for observedStates; all other nodes get none. Update-phase nodes get no evidence references at all.

Referential Recoverability Trace

Step	Result	Reason
Graph node → evidenceId	works	Nodes carry evidenceIds as string arrays
evidenceId → evidence record	breaks	Evidence records are consumed during startCase and never returned; no persistence layer retains them
Evidence record → evidenceType	N/A	Chain already broken at previous step

Answered Questions

1. Is node identity already sufficient? No — nodes exist and have evidenceIds, but without the referenced records they carry no provenance information.
2. Is evidence identity already sufficient? The evidence records themselves (when they exist) carry sufficient identity (id + evidenceType). But they are not available in case state.
3. Are evidence records retained long enough to resolve references? No — consumed during construction, not returned with graph.
4. Is evidenceType available after resolution? N/A — cannot resolve the reference to get there.
5. Are node-to-evidence links populated consistently enough? No — only observedStates nodes in initial build receive references; all update-phase nodes receive none.
6. Can supplied-versus-inferred provenance currently be recovered referentially? No.
7. Is the problem primarily: More than one of these: (a) evidence provenance exists upstream during reconstruction but is not persisted alongside graph state; (b) no new evidence records are created or retained during update cycles; (c) incomplete linkage even at construction time (only observedStates nodes get references).

Conclusion

Existing provenance exists but referential linkage is incomplete — specifically: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material (Experiment 54E did not validate how those values are assigned), and the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state, making them unrecoverable in persisted case data; and (2) no evidence records are created or retained during update cycles at all. The problem is primarily missing persistence of existing provenance combined with a gap in evidence record creation during updates.

Limitations

• Source-inspection audit only; no live execution tested
• Client-side state management was not inspected — if the client retains evidence records alongside graph state, referential recovery may work on that layer
• Did not evaluate whether the client could reconstruct provenance from UI-visible data
• Conclusions apply to the server-side pipeline as currently implemented

Status

Pending Rob's review. Source-inspection complete. No production code changed. Working tree clean before commit.

Experiment 54F — Is evidenceType Actually Reliable Provenance, or Just a Model Label? (2026-08-07)

Objective

Inspect whether the current reconstruction path assigns evidenceType from a clear provenance rule, or whether the LLM itself decides whether something is a reported statement, interpretation, assumption, or inferred relationship. Source-audit only. Do not implement provenance.

Hypothesis

evidenceType may be semantically useful without being trustworthy provenance. If the LLM is asked to classify reconstructed content into those categories, then reported_statement may mean "the model thinks this looks like a reported statement" rather than "production code knows this came directly from the user."

Files Inspected

• prompts/reconstruct-v0.3.md — full file; CRITICAL RULE 7 at line 155 specifying how evidenceType should be classified
• lib/reconstruction/prompt.js — buildV3Prompt function (lines 67–78), which loads prompts/reconstruct-v0.3.md and substitutes the scenario via {{SCENARIO}}
• lib/reconstruction/schema.js — evidenceRecordSchema definition at lines 113–127; evidence field at line 185
• lib/reconstruction/compatibility.js — normaliseAnalysisResponse function (full file, lines 1–52); deterministic normalisation applied to parsed reconstruction results

Who Chooses evidenceType?

Answer: the LLM.

The prompt in prompts/reconstruct-v0.3.md line 155 states:

evidenceType: classify each evidence item clearly as either a direct observation, a reported statement, an interpretation, an assumption, or an inferred relationship. Do not treat raw counts as proof of causal relationships — they may be inferred relationships only when supported by explicit reasoning about denominators or rates.

This is a classification instruction directed at the model. No production code determines or constrains which evidenceType value a given piece of evidence should have. The LLM receives scenario text and chooses an evidenceType for each evidence record it creates in its output.

Relevant Reconstruction Instruction (Summarised)

The prompt instructs the model to:

• Produce a JSON object with inputClassification, reconstruction, evidence, and nextQuestion keys;
• For the evidence array, produce records each containing id, description, evidenceType, source, attribution, confidence, and importance;
• For evidenceType specifically, instructs the model to "classify each evidence item clearly as either a direct observation, a reported statement, an interpretation, an assumption, or an inferred relationship" based on its judgment of what each piece of evidence represents;
• No production-level rules restrict which category maps to which source origin.

Raw User Origin Status Before Reconstruction

Answer: model_classified.

The raw user scenario text is passed into the prompt via {{SCENARIO}} substitution at line 158 of reconstruct-v0.3.md. It is not wrapped, tagged, or structurally separated from any other content. Production code (lib/reconstruction/prompt.js) performs only a single string substitution: content.replace("{{SCENARIO}}", scenario). There is no structural marker in the prompt that tells the model "this text came from the user" — it simply appears at the end of the prompt as unlabelled text.

Note on Experiment 54D's finding: The update prompt (lib/graph/prompt-builder.js) does separate user-supplied answers via a Markdown header (## User Answer). However, Experiment 54F inspects the initial reconstruction path only, where no such structural separation exists. The update path uses a different prompt from a different builder and is not part of this audit's scope.

Raw User Origin Status Inside Reconstruction Output

Answer: model_classified.

After the LLM produces its evidence records, each record's evidenceType reflects the model's own judgment about what category that evidence belongs to — not any deterministic derivation from source origin. The only production-side manipulation of evidenceType is in lib/reconstruction/compatibility.js, which performs a single enum normalisation: if evidenceType === "reported_claim" (from an earlier schema version), it converts it to "reported_statement". This does not add provenance information; it only adjusts for schema versioning.

Is evidenceType Deterministic?

Answer: no. The model itself chooses each evidenceType value at generation time. No production code determines it from source structure.

Is reported_statement Trustworthy as User Provenance?

Answer: no. A record with evidenceType === "reported_statement" means "the model classified this evidence item as a reported statement" — not "production code knows this came directly from the user." The model makes this classification based on its understanding of the scenario text, which may conflate what it inferred about reported claims with what was actually stated by a person.

Are interpretation / assumption / inferred_relationship Trustworthy as Model Provenance?

Answer: partially. These values do indicate categories the model itself assigned to its own output, so they can serve as semantic labels for "model-generated content" in a loose sense. However, the boundary between these categories is defined by the LLM's judgment, not by production code — the model may classify something as an assumption when it was actually directly stated by the user, or vice versa. There is no deterministic gate separating user-supplied from model-generated content at classification time.

Trace: Raw User Statement → Reconstruction Prompt

Stage	User origin explicit?	Who assigns evidenceType?	Provenance strengthens/weakened?
Raw user scenario text	Not structurally marked (just pasted as {{SCENARIO}})	N/A	—
Reconstruction prompt (reconstruct-v0.3.md)	Scenario appears unlabelled at end of file, no structural distinction from system instructions	Model chooses per its judgment (CRITICAL RULE 7)	Weakened — user text is indistinguishable in structure from other prompt content

Trace: Reconstruction Prompt → LLM Evidence Record

Stage	User origin explicit?	Who assigns evidenceType?	Provenance strengthens/weakened?
Model processes prompt and generates evidence records	No structural marker in prompt distinguishes user content	Model (based on CRITICAL RULE 7 classification instruction)	Unchanged — model judgment, not deterministic derivation

Trace: LLM Evidence Record → Schema Validation

Stage	User origin explicit?	Who assigns evidenceType?	Provenance strengthens/weakened?
Raw model output (JSON)	N/A — already model-generated	Model's own choice	Unchanged
Compatibility normalisation (lib/reconstruction/compatibility.js)	No provenance added	reported_claim → reported_statement (schema versioning only)	Neutral — no provenance information added or removed; only enum compatibility
Zod validation against reconstructionV2Schema	N/A — schema validates structure, not origin	Schema accepts whatever evidenceType the model chose (enum valid)	Unchanged — schema enforces enum validity but not provenance correctness

Does Schema Validation Verify Origin or Only Enum Validity?

Answer: only enum validity. The Zod schema (lib/reconstruction/schema.js line 113-127) validates that evidenceType is one of five allowed strings. It does not and cannot verify what produced the value — whether it came from a user, was derived deterministically by production code, or was classified by the LLM.

Can Downstream Deterministic Code Safely Treat evidenceType as Provenance?

Answer: no. evidenceType values are model-generated labels reflecting semantic categories the model chose during reconstruction. They do not correspond to any deterministic derivation from source origin. Production code cannot safely interpret evidenceType === "reported_statement" as "this was definitely supplied by the user" without additional provenance infrastructure.

Does Experiment 54E's Referential Approach Depend on a Label That Is Itself Model-Generated?

Answer: yes. Experiment 54E found that evidence records carry sufficient identity (id + evidenceType) when they exist. But evidenceType is itself model-generated, not structurally derived from source origin. The referential approach depends on labels the LLM chose, meaning provenance inference at that point already rests on model classification rather than deterministic provenance — compounding the provenance gap rather than resolving it.

Primary Provenance Gap After This Audit

Answer: both. The provenance gap has two independent causes:

1. Evidence-record lifetime (54E): Records are consumed during startCase and never returned with graph state.
2. Evidence-type trustworthiness (54F): Even if records were retained, evidenceType is model classification, not deterministic provenance.

Experiment Conclusion

evidenceType is semantic model output, not reliable provenance. The reconstruction prompt instructs the LLM to classify each evidence item into one of five categories based on its own judgment. No production code deterministically derives evidenceType from source origin. The compatibility layer adds only a single enum normalisation (reported_claim → reported_statement) for schema versioning. Schema validation enforces enum validity but not provenance correctness. Experiment 54E's referential approach depends on a label that is itself model-generated, meaning the provenance chain was already unreliable before it broke due to missing persistence. The primary gap has two independent causes: evidence records are not retained alongside graph state, and evidenceType values themselves are model-classified rather than deterministically derived from source origin.

Limitations

• Source-inspection audit only; no live execution tested
• Inspected only the initial reconstruction path (v0.3 prompt), not the update path which uses a different prompt structure
• Did not evaluate whether source or attribution fields on evidence records carry any provenance value beyond what evidenceType does
• Did not test how reliable the LLM's evidenceType classification is in production (this would require validation, not inspection)
• Conclusions apply to the v0.3 reconstruction prompt as currently implemented

Status

Pending Rob's review. Source-inspection complete. No production code changed. Working tree clean before commit.

Experiment 54G — Do Evidence Records Contain Deterministic Source Linkage Back to User Words? (2026-08-07)

Objective

Answer one narrow provenance question: does the current reconstruction output contain enough source information to deterministically prove that an evidence record came from the user's actual words, without trusting the LLM's evidenceType label? Source-audit only. Do not implement provenance.

54F established that evidenceType is model classification, not reliable provenance. 54G tests whether evidence records nevertheless retain deterministic linkage to the user's actual words — such as exact text, character offsets, turn ID, source path, or other structural location data.

Hypothesis

An evidence record may already preserve enough source material — such as exact text, quote, source excerpt, source ID, character range, source path, turn ID, or input reference — for deterministic code to verify that a record is directly grounded in the user's supplied text. If no such information exists, then evidence-record identity alone cannot establish user provenance.

Files Inspected

• lib/reconstruction/schema.js — evidenceRecordSchema definition (lines 113–127); reconstructionV2Schema (line 182)
• prompts/reconstruct-v0.3.md — evidence record output format (lines 126–136); CRITICAL RULE 7 (line 155)
• lib/reconstruction/compatibility.js — normaliseAnalysisResponse function (full file); source null-handling (lines 31–38)
• tests/reconstruction/compatibility.test.js — real evidence-record shapes from tests (lines 36–44, 54–72, 107–115, 137–147)
• lib/analysis.js — analyseScenario function (full file); confirms raw scenario is NOT returned alongside evidence records (lines 173–188)

Evidence-Record Source-Related Fields

From evidenceRecordSchema:

Field	Type	Provenance potential
id	z.string().min(1)	None — free-form, LLM-generated. No structural reference to input.
description	z.string().min(1)	None — model-generated summary of the evidence, not user text.
evidenceType	enum (5 values)	None — model classification, per 54F. Not structural provenance.
source	z.string().optional()	Partially — prompt says "who/where this came from". But: it is free-form text generated by the LLM, not deterministic production code output. In tests, it appears as "report" (free label) or null (removed by normalisation). No character offsets, turn IDs, span data, or source record references are structured into it.
attribution	z.string().nullable().optional()	None — test fixtures show null by default. When populated, it is free-form model text describing who said what, not a deterministic production-code identifier back to input.
confidence	enum	None — subjective confidence level, not source data.
importance	enum	None — semantic weight, not source data.

Does Raw User Statement Coexist with Reconstruction Result?

Not retained alongside results. The raw user statement (scenario) is the input parameter to analyseScenario. It is available to production code while reconstruction is being performed, but it is NOT retained alongside the returned reconstruction/evidence state for later deterministic verification. At validation time, deterministic code has:

• the evidence records (with model-generated descriptions and types);
• but NOT the raw user statement itself.

Even if deterministic code had the scenario text available at validation time, the evidence records still lack any field containing verbatim user text or structural location back to it.

Does Evidence Record Preserve Exact User Wording?

No. The description field is a model-generated summary/paraphrase of the evidence — not the user's actual words. The source field, when non-null, is free-form text like "report" (a label, not a quote). No evidence record contains verbatim or near-verbatim user text.

Does Evidence Record Preserve Source Location / Span / Turn Identity?

No. None of the evidence record fields contain:

• character offsets within user input;
• sentence or line index;
• turn or message ID;
• source record ID owned by production code;
• any structural reference to a specific location in the original input.

The source field prompt description says "who/where this came from" but it is free-form, inconsistently populated (sometimes null), and produced by the LLM not deterministic code.

Who Creates Evidence Record IDs?

The LLM. The prompt template (line 128 of reconstruct-v0.3.md) says "id": "<any unique string>". The schema only requires z.string().min(1). No production code generates or constrains the ID beyond non-emptiness. The ID is opaque and carries no provenance semantics.

Can Evidence Records Be Verified Against Raw User Input?

No. Deterministic verification would require:

1. Raw user text available at validation time — NOT present in result;
2. Each evidence record containing verbatim user text or a deterministic structural reference to a location within it — neither present.

The description field is model-generated paraphrase, not verbatim text. The source field is free-form model output, not a structured pointer. No field survives from the raw input to the result in a form that code can verify.

Trace: Raw User Statement → Evidence Record → Source Verification

Stage	Raw source identity available?	Exact source wording/location?	Deterministic verification possible?
Raw user statement	explicit (input parameter)	explicit (raw text)	N/A — this is the ground truth
Reconstruction prompt	absent (scenario pasted as {{SCENARIO}} with no structural markers)	partial (present in prompt body but unlabelled and indistinguishable from system instructions)	No
LLM evidence record	absent (all source identity lost to model generation)	absent (description is model paraphrase; source/attribution are free-form model text or null)	No
Validated result (analysis.js return)	absent (scenario not returned alongside evidence)	absent	No

Trace: Evidence Record → Source Verification

Step	Status
Get evidence record fields	present
Check description against user text	impossible — no verbatim text to compare
Check source/attribution as structured location	impossible — free-form model output, not production-code identifiers
Check id for provenance semantics	impossible — opaque LLM-generated string
Verify evidenceType independently	impossible — no source text to verify against

Does Current Reconstruction Schema Contain Enough Information for Deterministic Provenance?

No. The schema fields source and attribution exist as free-form optional strings, but neither is deterministic (they are model-generated), nor do they contain structured location data. The raw user statement is not returned alongside the evidence records. Even if it were, no evidence record field contains verbatim text or structural reference to it.

Primary Source-Linkage Gap

Evidence records carry no verbatim user text and no deterministic structural reference (character offsets, turn/message IDs, source record references) back to the original input. The only candidate fields (source, attribution) are free-form model-generated strings that may be null, and which cannot be independently verified against user input. Additionally, the raw user statement itself is not returned with the validated reconstruction result at validation time.

Experiment 54G Conclusion

Bounded findings. Evidence records do not contain verbatim source text. Evidence records do not contain deterministic source locations (character offsets, turn/message IDs, or structural references). evidenceType is model classification, not production-code provenance. Current evidence records therefore cannot independently prove source provenance without trusting the LLM's label.

Limitations

• Source-inspection audit only; no live execution tested
• Inspected only the initial reconstruction path (v0.3 prompt) and the analysis pipeline
• Did not inspect whether evidence records are stored with graph nodes in a way that could later be recovered
• Did not inspect the update path which uses different prompts and potentially different provenance characteristics
• Conclusions apply to the v0.3 reconstruction output as currently implemented

Status

Pending Rob's review. Source-inspection complete. No production code changed. Working tree clean before commit.

Experiment 54H — Can Trustworthy Source Identity Be Established Before LLM Interpretation? (2026-08-07)

Objective

First, tighten Experiment 54G so it does not imply the raw user statement is unavailable during reconstruction.

Then test one small capability outside the active engine:

Can production code create a deterministic source record from the user's raw input before any LLM interpretation occurs, so later reasoning has something trustworthy to refer back to?

This is a passive test-only experiment. Do not integrate provenance into the graph or runtime.

Hypothesis

A trustworthy provenance root may not require the LLM to decide anything. Production code may be able to take the raw input and create a small immutable source record before reconstruction that provides stable identity, source type, and verbatim source text — without attempting to classify what individual reasoning claims mean. If this cannot be done cleanly without introducing hidden interpretation, record why.

Context Used

• docs/current-handoff.md (54G findings)
• Experiment 54G only in docs/design-evolution-log.md
• lib/reconstruction/schema.js — evidenceRecordSchema to avoid naming collisions
• Node.js built-in crypto.createHash('sha256') — standard library, deterministic

Test-Only Source Record Shape

{
  "sourceId": "...",
  "sourceType": "user_input",
  "verbatimText": "..."
}

This is deliberately small and does not include: evidence type, confidence, interpretation, node kind, relevance, inferred meaning, or summary.

Deterministic ID Method Used

SHA-256 hash of the verbatim input text using Node.js built-in crypto.createHash('sha256').

Why the ID Method Is Deterministic

• SHA-256 is a deterministic cryptographic hash function: same input always produces the same output; different inputs produce different outputs (with negligible collision probability).
• Uses Node.js standard library — no LLM, no random UUID, no clock/time dependency, no mutable global state.
• Input is the verbatim string in UTF-8 encoding with no transformation or normalisation.

Four Input Cases

Case	Text	Purpose
1	"The business has lost three major customers this year."	Baseline simple statement
2	"Three major customers have left the business this year."	Paraphrase — confirm identity tracks actual text, not meaning
3	"Revenue is down. I think pricing may be part of the problem, but I am not sure."	Multi-sentence — confirm full verbatim input survives unchanged
4	Same as Case 1	Confirm deterministic identity

Focused Test Results (9 tests, all pass)

#	Test	Result
1	Case 1: verbatimText equals the exact input string	Pass
2	Case 2 (paraphrase): sourceType is always user_input	Pass
3	Case 1 and Case 4 produce identical sourceId	Pass
4	Case 1 and Case 2 produce different sourceIds despite similar meaning	Pass
5	Case 3: multi-sentence verbatim text preserved unchanged	Pass
6	Input string is not mutated by createSourceRecord	Pass
7	Helper output is deterministic across repeated calls	Pass
8	No interpretation or summarisation occurs in the source record	Pass
9	Different inputs produce different IDs (distinct text)	Pass

Case Results Summary

• Case 1 result: verbatimText preserved exactly ✓; deterministic SHA-256 ID produced.
• Case 2 paraphrase result: Same meaning, different text → different sourceId ✓; verbatimText preserved exactly.
• Case 3 multi-sentence result: Full multi-sentence text preserved unchanged ✓; single deterministic ID.
• Case 4 repeated-input result: Identical to Case 1 text → identical sourceId ✓.

Same-Input Identity Result

Confirmed: Cases 1 and 4 produce exactly the same sourceId (sha256 of identical string = identical hash).

Different-Text Identity Result

Confirmed: Cases 1, 2, and 3 produce different sourceIds despite Case 1≈Case 2 in meaning.

Verbatim-Text Preservation Result

All three unique inputs preserved exactly as-is. No summarisation, normalisation, or transformation applied to verbatimText.

Input Immutability Result

Confirmed: the original input string is never mutated. createSourceRecord reads via String(rawInput) (copy) before hashing.

LLM / Network Dependency

None. The helper uses only Node.js built-in crypto and JavaScript primitives. Zero inference calls. Zero network calls.

Does the Helper Perform Semantic Interpretation

No. The helper outputs exactly: { sourceId, sourceType, verbatimText }. No meaning is extracted, classified, summarised, or inferred.

Distinction Between Source Identity and Semantic Provenance

This experiment establishes source identity — answering "what exact material entered the system, and what stable identity can we give that material?" It does not answer: what the text means, whether it is evidence, whether it was inferred, or whether graph nodes can link to it. Source identity ≠ claim provenance.

What Remains Explicitly Untested

• Linking evidence records to source records
• Linking graph nodes to evidence or source
• Update-turn source identity (not initial reconstruction)
• Persistence of source records alongside graph state
• Whether production code can reliably invoke this helper at the right boundary
• How downstream consumers discover and use the source record
• Multiple-input scenarios (multi-turn conversations)

Experiment Conclusion

Deterministic source identity is feasible before LLM interpretation. A deterministic SHA-256 hash of raw user text produces stable, verifiable identity without any semantic processing. The concept is straightforward because it operates purely at the input boundary — no classification, no inference, no graph contract required.

Limitations

• Test-only implementation; not integrated into any production path
• Single-input only (does not address multi-turn or batched input scenarios)
• Does not test whether production code can reliably invoke this at the correct reconstruction boundary
• Does not test persistence, retrieval, or downstream linking
• SHA-256 collision resistance is sufficient for source identity but does not guarantee uniqueness across all possible inputs in practice

Status

Closed. Bounded conclusion: deterministic source identity is feasible before LLM interpretation; identical tested text produces stable identity; source identity does not establish claim or graph provenance. No production code changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/reconstruction/deterministic-source-record.test.js.

Experiment 54I — Can Two Different Interpretations Stay Anchored to the Same User Source? (2026-08-07)

Objective

Take the deterministic source identity proved in Experiment 54H and test the next smallest step:

Can two different interpretations of the exact same user input remain separate while both retaining an explicit link back to that same source?

This experiment deliberately does not decide which interpretation is better. It tests whether the reasoning system could preserve one source, multiple interpretations, and clear lineage back to that source without silently replacing the original input or collapsing the interpretations together. This is a passive, test-only experiment.

Hypothesis

A very small lineage structure may be enough to demonstrate that:

• one source identity can anchor multiple interpretations;
• interpretations remain separately identifiable;
• neither interpretation changes or replaces the source;
• a later consumer can deterministically see that both interpretations came from the same exact input.

If even this cannot be represented cleanly without semantic ambiguity, record that honestly.

Context Used

• docs/current-handoff.md (Experiment 54H findings and Return-to-Work Note)
• Experiment 54H only in docs/design-evolution-log.md
• tests/reconstruction/deterministic-source-record.test.js (to verify the source-identity helper remains intact)
• Node.js built-in crypto.createHash('sha256') — deterministic, no LLM

Fixed Source

Raw user input:

Revenue is down. I think pricing may be part of the problem, but I am not sure.

One deterministic source record created from that exact text using the Experiment 54H method (SHA-256 of verbatim text).

Two Fixed Interpretations

Interpretation A:

Pricing may be contributing materially to the revenue decline.

Interpretation B:

The revenue decline may have causes other than pricing, and pricing has not yet been established as the main problem.

Both are plausible readings of the source. The experiment does not claim either is correct.

Interpretation Identity Method

Each interpretationId is generated deterministically from:

• the shared sourceId;
• plus the exact interpretation text (combined as concatenation of sourceId + "|" + interpretationText).

Same source + same interpretation → same interpretation ID. Same source + different interpretation → different interpretation ID. Interpretation ID changes if interpretation text changes. No random UUID, no clock, no LLM, no mutable global state.

Interpretation Record Shape (Test-Only)

{
  "interpretationId": "...",
  "sourceId": "...",
  "interpretationText": "..."
}

This is not a proposed production schema. It does not include: confidence, evidence type, status, scores, timestamps, model names, node kinds, or next-question information. The purpose is only identity and lineage.

Focused Test Results (15 tests, all pass)

#	Test	Result
1	Source retains exact verbatim user input	Pass
2	Interpretation A references the source's exact sourceId	Pass
3	Interpretation B references the same exact sourceId	Pass
4	A and B have different interpretationId values	Pass
5	Recreating A produces exactly the same interpretationId	Pass
6	Recreating B produces exactly the same interpretationId	Pass
7	Changing interpretation text changes the interpretation ID	Pass
8	Neither interpretation mutates the source record	Pass
9	Creating one interpretation does not mutate the other	Pass
10	Deterministic consumer identifies one shared source and two distinct interpretations	Pass
11	No semantic judgement chooses A over B	Pass
12	No LLM or network call occurs	Pass
Cross-check	Different source + same text → different interpretationId	Pass
Immutability	Source unchanged after multiple interpretation creations	Pass
Lineage traceability	Consumer traces both interpretations to exact verbatim source	Pass

Shared Source Lineage Result

Confirmed: both Interpretation A and Interpretation B reference the same sourceId. A deterministic consumer can recover that exactly one source anchors both interpretations.

Distinct Interpretation Identity Result

Confirmed: Interpretation A and Interpretation B have different interpretationId values despite sharing a source. The IDs remain stable across recreation and diverge when text changes.

Source Immutability Result

Confirmed: creating one or multiple interpretations from a source does not mutate the source record in any way (fields, structure, or content remain identical).

Interpretation Independence Result

Confirmed: creating an interpretation for A does not affect B's identity, and vice versa. Each interpretation is independently derived from the shared sourceId + its own text.

Deterministic Consumer Recovery

Confirmed: a consumer given the source record and both interpretation records can deterministically identify (1) one shared source, (2) two distinct interpretations, and (3) that both interpretations trace back to the same exact verbatim input.

Was Either Interpretation Selected as More Correct?

No. The experiment does not select, score, or prefer either interpretation.

Was Downstream Question Selection Tested?

No. The experiment explicitly excludes next-question derivation.

What This Experiment Establishes

• One source can deterministically anchor multiple interpretations.
• Those interpretations remain independently identifiable via stable interpretationId values.
• A later consumer can determine that two interpretations came from the same exact source.
• Preserving multiple interpretations does not require changing the original source record.
• The distinction between "the user supplied this" and "interpretation A/B says this may mean X/Y" survives representation cleanly in code.

What This Experiment Does Not Establish

• Which interpretation is more justified or better grounded.
• Whether different interpretations would lead to different next questions.
• Any semantic correctness claim about either interpretation.
• Graph integration, persistence, multi-turn history, or production architecture.

Explicitly Untested

• Deciding which interpretation is better grounded.
• Comparing interpretations against verbatim source wording.
• Comparing interpretations against other evidence.
• Contradictions between interpretations.
• Whether disagreement should reduce confidence.
• Whether disagreement should trigger clarification.
• Downstream question selection.
• Graph integration.
• Persistence.
• Multi-turn interpretation history.

Limitations

• Test-only implementation; not integrated into any production path.
• Only two fixed interpretations tested (not a general multi-interpretation protocol).
• No semantic analysis of whether either interpretation faithfully represents the source.
• The | separator in interpretationId derivation assumes the separator does not appear in user text; this is adequate for identity stability but would need review if adopted as production code.

Experiment Conclusion

Multiple interpretations can retain deterministic lineage to one source. One source identity successfully anchors two distinct interpretations while both remain separately identifiable and neither mutates the original source. The distinction between unchanged user evidence and changing model interpretation remains visible in the data structure.

Status

Closed. Bounded conclusion: multiple interpretations can retain deterministic lineage to one source. One source identity successfully anchors two distinct interpretations while both remain separately identifiable and neither mutates the original source. The distinction between unchanged user evidence and changing model interpretation remains visible in the data structure. No production code changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/reconstruction/source-interpretation-lineage.test.js.

Experiment 54J — Can Source-Grounded Meaning Be Separated From Interpretation-Added Meaning? (2026-08-07)

Objective

Take the lineage proved in Experiment 54I and test the next smallest reasoning capability:

Given one exact user source and two different interpretations linked to it, can deterministic test logic identify which parts of each interpretation are directly grounded in the source and which parts go beyond what the source actually states?

This experiment must not decide which interpretation is ultimately correct. It tests source-grounding comparison only.

Hypothesis

A small test-only grounding representation may be enough to distinguish:

• meaning directly supported by the source;
• meaning introduced by the interpretation.

If that distinction can be represented cleanly, later reasoning could compare interpretations without immediately selecting one. If deterministic comparison requires semantic judgement that cannot be justified from text alone, record that boundary honestly.

Context Used

• docs/current-handoff.md (Experiment 54I findings and Return-to-Work Note)
• Experiment 54I only in docs/design-evolution-log.md
• tests/reconstruction/source-interpretation-lineage.test.js (to confirm 54I remains intact — 15 tests pass)
• tests/reconstruction/deterministic-source-record.test.js (to confirm source identity helper remains intact — 9 tests pass)
• Node.js built-in crypto.createHash('sha256') — deterministic, no LLM

Fixed Source

Raw user input:

Revenue is down. I think pricing may be part of the problem, but I am not sure.

Two Fixed Interpretations

Interpretation A:

Pricing may be contributing materially to the revenue decline.

Interpretation B:

The revenue decline may have causes other than pricing, and pricing has not yet been established as the main problem.

Both are plausible readings of the source. The experiment does not claim either is correct.

Grounding Record Shape (Test-Only)

{
  "sourceId": "...",
  "interpretationId": "...",
  "supportedBySource": ["..."],
  "addedByInterpretation": ["..."]
}

This is not a proposed production schema. It does not include: confidence, evidence type, status, scores, timestamps, model names, node kinds, or next-question information.

Human-Fixed Grounding References (Pre-Written Test Data)

These are NOT generated dynamically. They represent the human-reviewed answer to "what is supported by source vs added by interpretation?"

Interpretation A reference:

Supported by source captures only:

• revenue is down;
• pricing may be part of the problem.

Added by interpretation captures that:

• pricing may be contributing materially to the decline.

The source does not establish material impact.

Interpretation B reference:

Supported by source captures only:

• revenue is down;
• pricing may be part of the problem;
• the user is unsure.

Added by interpretation captures that:

• there may be causes other than pricing;
• pricing is not established as the main problem.

These are plausible interpretations of uncertainty, but they are not directly stated as facts in the source.

Focused Test Results (13 tests, all pass)

#	Test	Result
1	Both grounding records reference the same exact sourceId	Pass
2	Each grounding record references its distinct interpretationId	Pass
3	Source-supported statements remain separate from interpretation-added	Pass
4	Interpretation A records 'materially' as added meaning	Pass
5	Interpretation B records alternative causes as added meaning	Pass
6	B's "not established as main problem" is interpretation-added, not user-stated fact	Pass
7	Neither grounding record mutates the source	Pass
8	Grounding creation does not affect interpretationId values	Pass
9	Recreating the same grounding record produces identical results	Pass
10	Consumer can inspect A and B: shared source / distinct interpretation / supported vs added	Pass
11	Neither interpretation is selected as a winner	Pass
12	No numeric scoring is present in grounding records	Pass
13	No LLM or network call occurs	Pass

Shared Source Lineage Result

Confirmed: both Interpretation A and Interpretation B grounding records reference the same sourceId. A deterministic consumer can recover that exactly one source anchors both interpretations' grounding.

Distinct Grounding Identity Result

Confirmed: Interpretation A and Interpretation B grounding records have different interpretationId values and distinct sets of supportedBySource and addedByInterpretation entries despite sharing a source.

Source-Added Meaning Comparison

Interpretation A adds: stronger causal/importance language ("materially") that the source does not contain. The source says pricing "may be part of the problem" without quantifying impact. Interpretation A strengthens this to material contribution.

Interpretation B adds: alternative explanations ("causes other than pricing") and framing around what has not yet been established, rather than what has been confirmed. This is a plausible interpretation of uncertainty but not directly stated as fact in the source.

Source Immutability Result

Confirmed: creating grounding records does not mutate the source or interpretation records in any way (fields, structure, or content remain identical).

Deterministic Consumer Recovery

Confirmed: a consumer given the grounding records can deterministically identify (1) one shared source, (2) two distinct interpretations with distinct grounding profiles, and (3) what each interpretation adds beyond the source.

Was Either Interpretation Selected as More Correct?

No. The experiment does not select, score, or prefer either interpretation.

Was Downstream Question Selection Tested?

No. The experiment explicitly excludes next-question derivation.

What This Experiment Establishes

• Source-supported meaning and interpretation-added meaning can be represented separately within a deterministic data structure.
• Two competing interpretations of the same source can each expose their own grounding profile while sharing the same source lineage.
• Interpretation A introduces stronger causal/importance language than the source.
• Interpretation B introduces alternative explanations not directly stated by the source.
• The representation preserves disagreement without treating either branch as fact.
• A later consumer can distinguish shared source content from distinct interpretation additions.

What This Experiment Does Not Establish

• Which interpretation is more justified or better grounded.
• Automated semantic extraction of grounding (grounding references are human-fixed test data, not dynamically extracted).
• Whether different interpretations would lead to different downstream questions.
• Any semantic correctness claim about either interpretation.
• Graph integration, persistence, multi-turn history, or production architecture.

Explicitly Untested

• Automated semantic extraction of grounding;
• choosing the better interpretation;
• evaluating interpretation correctness;
• comparing against other evidence;
• contradiction handling;
• confidence reduction;
• clarification behaviour;
• downstream question selection;
• graph integration;
• persistence;
• multi-turn reasoning.

Limitations

• Test-only implementation with human-fixed grounding references (not automated semantic extraction);
• Only two fixed interpretations tested (not a general multi-interpretation protocol);
• Grounding distinctions were written by human review, not derived algorithmically;
• The | separator in interpretationId derivation assumes the separator does not appear in user text; this is adequate for identity stability but would need review if adopted as production code;
• Does not test whether a consumer can use these grounding records to make downstream decisions about confidence or question selection.

Evaluation Questions

1. Can both interpretations retain the same source lineage while exposing different added meaning? Yes.
2. Can a later deterministic consumer see which claims were source-supported and which came from interpretation? Yes.
3. Does Interpretation A introduce stronger causal/importance language than the source? Yes — "materially" is not in the source.
4. Does Interpretation B introduce alternative explanations not explicitly stated by the source? Yes — "causes other than pricing" is plausible but not stated.
5. Does this representation preserve disagreement without treating either branch as fact? Yes — no correctness or status fields are assigned.
6. Does this experiment establish which interpretation is more justified? No.
7. Does this experiment establish which downstream question should be asked? No.

Evaluation Conclusion

Interpretation-added meaning can remain separate from source-supported meaning. The test primitive of fixed reference lists successfully preserves the distinction without collapsing either interpretation into the source or into each other.

Regression Tests

• source-interpretation-lineage.test.js: 15 tests, all pass
• deterministic-source-record.test.js: 9 tests, all pass

Status

Pending Rob's review. No production code changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/reconstruction/interpretation-source-grounding.test.js.

Experiment 54K — Can the Model Automatically Separate Source-Supported Meaning From Interpretation-Added Meaning? (2026-08-07)

Objective

Take the representation proved in Experiment 54J and test the smallest missing capability:

Given an exact user source and one interpretation of it, can the configured semantic model identify which meaning is supported by the source and which meaning was added by the interpretation?

Experiment 54J used human-fixed grounding references. Experiment 54K tests whether that grounding distinction can be produced semantically without changing production behaviour.

This is a passive test-only experiment.

Hypothesis

Given an exact verbatim source and one interpretation, the model may be able to separate:

1. what the source actually supports;
2. what the interpretation adds beyond the source.

If it can do this without strengthening, weakening, or rewriting the source meaning, then automated grounding is plausible enough for further investigation.

Context Used

• docs/current-handoff.md (Experiment 54J findings and Return-to-Work Note);
• Experiment 54J only in docs/design-evolution-log.md;
• Existing tests/reconstruction/interpretation-source-grounding.test.js to confirm the grounding record shape remains valid;
• Existing .env.local configuration (same Ollama host and model).

Configured Host and Model

• Ollama host: http://192.168.1.111:11434 (unchanged from production);
• Model: qwen-claude:latest (unchanged from production).

Semantic Output Contract

Each call receives { source, interpretation } and returns exactly:

{
  "supportedBySource": ["short factual statements"],
  "addedByInterpretation": ["short factual statements"]
}

No confidence. No scores. No explanation field. No chain-of-thought.

Semantic Instruction (identical for all cases)

Compare the interpretation with the exact source text. Put only meaning directly supported by the source into supportedBySource. Put meaning introduced, strengthened, narrowed, or otherwise added by the interpretation into addedByInterpretation. Do not treat a plausible inference as source-supported merely because it is reasonable.

Three Fixed Cases — Inputs and Semantic Outputs

Case 1 — Interpretation A

Source: "Revenue is down. I think pricing may be part of the problem, but I am not sure." Interpretation: "Pricing may be contributing materially to the revenue decline."

Semantic output (live model):

• supportedBySource: ["Revenue has declined.", "Pricing might be a factor in the problem.", "The speaker expresses uncertainty about pricing's role."]
• addedByInterpretation: ["Pricing has a significant or substantial impact on the revenue decline.", "The explicit expression of uncertainty is removed, strengthening the claim."]

Human-fixed reference: supported = [revenue is down, pricing may be part of the problem]; added = [pricing may be contributing materially to the decline]

Classification: grounding_correct

Case 2 — Interpretation B

Source: "Revenue is down. I think pricing may be part of the problem, but I am not sure." Interpretation: "The revenue decline may have causes other than pricing, and pricing has not yet been established as the main problem."

Semantic output (live model):

• supportedBySource: ["Revenue has declined.", "Pricing might be one contributing factor to the decline.", "The speaker is unsure whether pricing is a factor."]
• addedByInterpretation: ["Pricing has not yet been established as the main problem of the revenue decline."]

Human-fixed reference: supported = [revenue is down, pricing may be part of the problem, the user is unsure]; added = [there may be causes other than pricing, pricing has not yet been established as the main problem]

Classification: partial_grounding The model correctly captured "not established as main problem" on the added side. However, it did not capture "alternative causes besides pricing" on the added side — this is a genuine missed addition for Interpretation B, since alternative causality is a core element of that interpretation. The supportedBySource content was all correct including uncertainty.

Case 3 — Fully Grounded Control

Source: "Revenue is down. I think pricing may be part of the problem, but I am not sure." Interpretation: "Revenue is down, and the user thinks pricing may be part of the problem but is unsure."

Semantic output (live model):

• supportedBySource: ["Revenue is down.", "Pricing may be part of the problem.", "The speaker expresses uncertainty about whether pricing is contributing to the issue."]
• addedByInterpretation: []

Human-fixed reference: supported = [revenue is down, pricing may be part of the problem, the user is unsure]; added = []

Classification: grounding_correct The model produced no material additions for a faithful restatement. The output is an empty array on the added side, confirming it did not invent distinctions where none exist.

Required Questions — Answers

1. Did Case 1 correctly keep materially on the interpretation-added side? Yes. The model placed "significant or substantial" strengthening in addedByInterpretation and kept it out of supportedBySource.
2. Did Case 2 distinguish user uncertainty from the more specific interpretation layered onto it? Partially. It correctly kept uncertainty in supportedBySource but failed to capture one of two additions (alternative causes) on the added side.
3. Did Case 3 correctly produce no material added meaning? Yes. Empty addedByInterpretation array for a faithful restatement.
4. Did any plausible inference get incorrectly promoted into supportedBySource? No. The model did not promote any interpretation-specific content into the supported side across any case.
5. Did any genuinely source-supported meaning get incorrectly treated as interpretation-added? No. All three cases retained their core source-supported content in the supported side. The most notable was uncertainty — the model correctly identified it as source-supported (even in Case 1 where my reference didn't include it).
6. How many of three cases were: grounding_correct = 2, partial_grounding = 1, grounding_failed = 0.
7. Does the result suggest automated semantic grounding is plausible enough for further testing? Yes, with caution. The model kept strengthening (materially) on the correct side in every case and did not promote interpretation content into source-supported territory. One missed addition (alternative causes) suggests occasional under-detection of additions but no false positives on the critical dimension.
8. Does this experiment establish which interpretation is better? No. It does not select or score interpretations.
9. Does it establish what question should be asked next? No. That remains untested in this experiment.

Grounding Summary

Metric	Value
Grounding-correct count	2
Partial-grounding count	1
Grounding-failed count	0
Interpretation-added meaning leaked into supportedBySource	No
Source-supported meaning pushed into addedByInterpretation	No
Fully grounded control avoided invented additions	Yes

Evidence for Automated Semantic Grounding

The model correctly separated strengthening ("materially" → "significant or substantial") from source meaning in every case. No interpretation-specific content leaked into supportedBySource. The fully grounded control produced an empty added array. One partial result (Case 2) missed one of two expected additions but preserved all three pieces of source-supported content.

Inference Timing

Metric	Value
Number of live inference calls	3
Total inference time	96,372.37ms
Average	32,124.12ms per call
Fastest	25,575.87ms (Case 3)
Slowest	41,532.40ms (Case 2)

Timing is observational only. All three cases required ~25–42 seconds of model inference time on this host/model.

Limitations

• Single source text tested across all three cases — no cross-domain validation;
• Only one interpretation per source tested in each case — no multi-interpretation comparison in a single call;
• Model was qwen-claude:latest on host 192.168.1.111 — results may differ with other models or hosts;
• The partially correct Case 2 still captured the core supported content — the gap was in added-content completeness, not source-meaning accuracy;
• Evaluation used structured meaning checks (not keyword matching) but remains a heuristic approximation of semantic comparison;
• No repeated sampling — results reflect a single run per case.

Evaluation Conclusion

No interpretation-added meaning was observed in supportedBySource across the three tested cases. One missed addition in Case 2 and occasional paraphrase mismatches mean the result is not yet sufficient for production use because completeness, stability, and broader-domain behaviour remain untested. The direction is viable pending further testing.

Status

Pending Rob's review. No production code changed. No schemas modified. No active engine behaviour changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect when resuming: tests/reconstruction/semantic-interpretation-grounding.test.js.

---