confidence-engine/docs/current-handoff.md

Current Return-to-Work Handoff — Confidence Engine

This file describes only the latest stopping point. Replace its current-work sections when the project moves on. Historical evidence remains in the design log and archive.

1. Where We Left It

• Engine experiments resumed with a passive validation;
• UI experiments remain paused;
• Knowledge-management experiments are complete;
• Experiment 39 tested the existing Behaviour Selection module against real Investigation State Assessment outputs across three scenarios;
• Acknowledge dominates (71% of selections) because it fires first when health=healthy, blocking Summarise/Pause/Clarify even in concluding or stalled states.

This handoff describes the latest stopping point only. When work moves on, replace stale current-work details rather than appending another historical note. Historical experiment and commit information belongs in docs/design-evolution-log.md.

2. What Is True Now

• Main active engine path: deterministic reasoning pipeline (scenario reconstruction, graph update, unknown selection, question formulation, turn orchestration).
• Passive experimental classifiers from Experiments 18–25B remain isolated diagnostic layers; none control the user-facing investigation. Behaviour Selection was passively evaluated against real assessment outputs in Experiment 39 — it produced all valid behaviours but with skewed distribution (Acknowledge 71%).
• Keyword and phrase-based scope detection remains provisional scaffolding.
• docs/current-project-state.md is the main entry point for active project state.
• Experiment 54D confirmed the production update prompt explicitly separates the user answer (## User Answer section) but the proposal schema has no provenance field — source identity at prompt level is explicit, per-node provenance at output level is absent.

Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). Across the three tested disagreement patterns, the model did not automatically map disagreement to user clarification. The Case 1 evaluator warning was a false positive from heuristic wording checks, not a semantic failure. No production code changed. Status pending Rob's review.

Experiment 56D confirmed that Regression B (conditional trade-off resolution) works end-to-end through the real updateCase() production path. Deterministic derivation correctly identifies conditional semantics, passes all guards, and produces a valid graph update with emergent threshold unknown — no regression detected from commit 3e78d57. Status pending Rob's review.

Experiment 56E tested whether the weak-priority answer ("Risk matters more to me.") survives the full updateCase() production path without strengthening beyond relative importance. Result: FAIL - semantic interpretation. The LLM extracted userSupportedMeaning as "Avoiding additional risk is a preference/trade-off rather than a hard constraint" — asserting that risk is not a hard constraint, which goes beyond what the answer establishes (only relative importance). The deterministic guard passed because it saw the already-strengthened meaning. n-risk-constraint was incorrectly treated as resolved to "preference/trade-off". No emergent unknown created. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Status pending Rob's review.

Experiment 56F re-tested Regression A with the canonical live harness after Codex commit 4aa1492 (refine raw-answer boundary for answer meaning). Result: PASS - strengthening safely rejected. The LLM still produced semantic strengthening in userSupportedMeaning ("Avoiding additional risk is a strongly weighted preference/trade-off rather than a hard constraint") — the same class of over-resolution as 56E. However, the pre-mutation safeguard chain correctly rejected the proposal: deterministic derivation produced proposedMeaningCategory: hard_constraint which mismatched rawAnswerCategory: relative_importance, causing proposalValidation.success: false and preventing compatibility guard from passing. No graph mutation occurred — n-risk-constraint remained unresolved (status=unknown, value=null). One live call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed. Status pending Rob's review.

Experiment 56G tested Regression C (non-answer uncertainty: "I'm not really sure.") through the live production path to verify the risk-constraint distinction remains unresolved when the user expresses no position. BLOCKED - apparatus. The canonical helper (tests/graph/live-update-experiment-helper.cjs) contains a broken dynamic import path (../lib/graph/orchestrator.js resolves to tests/lib/graph/orchestrator.js, which does not exist — correct path is ../../lib/graph/orchestrator.js). No live calls were made. Full results in docs/experiment-56g.md. Status pending Rob's review.

Experiment 56H re-tested Regression C after harness repair (commit c40d8c6). Result: PASS - uncertainty preserved. The LLM did not invent any constraint or preference position from "I'm not really sure." — userSupportedMeaning was null. No graph mutation occurred; n-risk-constraint remained unknown with value=null. One live call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed. Status pending Rob's review.

Experiment 54S tested whether, once clarification is known to be required, the model can identify exactly what the user needs to clarify — three fixed cases: growth-versus-risk priority (true → "preference/trade-off or hard constraint"), evidence-resolvable delivery causes (false → null), ambiguous meaning of "affordable" (true → "upfront cost versus long-term total cost"). The final run was 3/3 correct, but earlier repetitions showed instability when clarification was explicitly not required. Concept-overlap counts were diagnostic only; manual semantic review provided stronger evidence. Case 2 instability is an observed behaviour, not merely a test warning. Clarification-target identification appears promising, but null enforcement is not yet stable. Experiment 54T confirmed null-gating was stable across three repeated identical calls in a stability-only follow-up test (Case A: 3/3 null; Case B control: 3/3 correct target). The current instruction and output contract produced stable null behaviour across the three repeated false-case runs tested there; broader stability remains unproven. Experiment 54U tested whether a fixed clarification target can survive into one neutral user-facing question without adding meaning (preference/constraint, affordability definition, private factual capacity). All three cases returned correct single neutral questions with no introduced assumptions or evidence requests. The clarification-target → question step worked cleanly across the three tested targets; broader wording quality and user experience remain untested. Same host/model (qwen-claude:latest on http://192.168.1.111:11434); no production code changed. Status pending Rob's review.

Experiment 54V tested whether the user's answer can resolve only that target without rewriting the rest of the source meaning. Three fixed cases: hard constraint resolved (true/null), affordability definition resolved (true/null), incomplete answer preserved (false/uncertainty). All three correct across boundary preservation, no forced interpretations, and no unsupported consequences or new questions generated. Clarification answers resolved only the intended target across all tested cases. The individual clarification steps have each worked in their isolated fixed-case tests; end-to-end behaviour remains untested. Graph updates, next-question choice, Behaviour Selection, and UI remain untested. Same host/model (qwen-claude:latest on http://192.168.1.111:11434); no production code changed. Status pending Rob's review.

• docs/task-context-packs.md chooses the minimum context documents for each work type.

Engine and UI work were deliberately paused because documentation had grown large enough to overload Claude and make returning across sessions difficult. The current phase is simplifying what a fresh session must load to understand the project, without losing evidential history. Historical material remains available under docs/archive/.

4. What Was Just Completed

Experiment 37 corrected the routing defect from Experiment 36 and tested a cross-boundary engine/UI task. It validated that two context packs can be combined deliberately while keeping working context small, explicit and accurate. All seven knowledge-management criteria are now met. No source code changed. No files moved or deleted.

Commit: pending (experiment: validate cold-start project recovery) — to be committed this session.

Experiment 54X isolated target specificity using three fixed clarification cases under the exact same instruction as Experiment 54S. Case 1 (preference/trade-off versus hard constraint) returned "preferred priority between business growth and risk avoidance" — broadened from the material distinction but usable. Case 2 (upfront versus long-term affordability) preserved the definition boundary. Case 3 (user's available time next month) preserved capacity specificity. The same broadening pattern was reproduced across two tested runs under the same model and configuration, making it a repeatable candidate behaviour rather than a one-off observation. No question generation, answer resolution, Behaviour Selection, graph, or UI integration was attempted. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-target-specificity.test.js for the full experiment and results. Status pending Rob's review.

Experiment 54Y tested whether that specificity loss actually changes downstream clarification in a tested scenario. Source: "I want the business to grow, but I don't want to take on more risk." Fixed answer: "It's a hard constraint. I don't want any increase in risk." Variant A (precise target) generated question asking whether avoiding risk is a hard constraint or preference/trade-off; Variant B (broadened target) generated question asking which to prioritize when growth and risk conflict. Both resolved the same answer with materially equivalent meaning. With the explicit hard-constraint answer used in this test, both target variants converged on materially equivalent resolved meaning. The broader target changed the clarification question but not the resolved meaning for the tested explicit answer; broader safety remains untested. Behaviour Selection, graph, UI, and production integration remained untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-specificity-consequence.test.js for the full experiment and results. Status pending Rob's review.

Experiment 54Z tested whether convergence between precise and broadened targets holds with weaker answers. Source same as 54Y. Two weak answers tested against both fixed variants: (1) "Risk matters more to me" — both variants produced materially equivalent meaning (risk not a hard constraint, but stronger than growth). (2) "I'd normally avoid more risk, but for the right opportunity I might accept some" — variants diverged: Variant A collapsed conditionality into flat preference; Variant B preserved conditional structure and remaining uncertainty. Unexpectedly, the broader target preserved more nuance for the conditional answer. Target broadening has material consequences with weaker answers, but direction is unpredictable. 4 live calls completed. Behaviour Selection, graph, UI, and production integration remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-weak-answer-consequence.test.js for the full experiment and results. Status pending Rob's review.

Experiment 55A isolated the answer-resolution step using one fixed target and four answers of varying strength (explicit hard constraint, weak priority, conditional trade-off, non-answer). Two of the four tested answers showed loss of nuance: one was over-resolved (weak priority set targetResolved=true with inferred "not a constraint" meaning) and one retained the correct target category while losing conditional qualification ("might accept some for the right opportunity" became "preference or trade-off rather than a hard constraint"). The same over-resolution reproduced with a fixed target, so target broadening is not required for the failure to occur. 4 live calls completed at ~62s total. The answer-resolution step appears biased toward resolution for weak priority statements. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-uncertainty-preservation.test.js for the full experiment and results. Status pending Rob's review.

Experiment 55B separated answer meaning from target-resolution judgement using independent calls per case. Three fixed answers tested (weak priority, conditional trade-off, non-answer) through two modes each: Mode A (meaning-only, no resolution decision) and Mode B (resolution via the same 54V/55A instruction). Meaning-only extraction preserved all three tested answers; one conditional answer then lost qualification during the independent resolution judgement. Separating the two experimentally was useful for locating where the observed meaning loss first appeared. Additionally, Case 1 (weak priority) resolved correctly in 55B but over-resolved in 55A — this does not establish that the weak-priority problem is solved; it indicates run-to-run variation. 6 live calls completed at ~104s total. No production code changed. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-answer-meaning-vs-resolution.test.js for the full experiment and results. Status pending Rob's review.

Experiment 55C chained actual preserved meaning from Stage 1 into Stage 2 resolution, testing whether carrying semantic state forward removes the conditionality loss observed in 55B. Three cases tested (weak priority, conditional trade-off, non-answer) through two stages each = 6 live calls at ~117s total. Case 2 conditional qualification survived through both stages and resolved correctly (targetResolved=true with condition retained). Case 3 non-answer uncertainty preserved through both stages. Case 1 over-resolved in Stage 2 because Stage 1 itself strengthened "risk matters more" into language about "preference/trade-off rather than absolute constraint." Compared to 55B, the weak-priority case did not remain honestly unresolved — If Stage 1 distorts the answer, Stage 2 may preserve and act on that distortion rather than correct it. No two-stage design is proven superior; meaning can be lost at either stage. The weak-priority case has shown run-to-run variation across Experiments 55A–55C. Graph, Behaviour Selection, UI and production remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-preserved-meaning-resolution.test.js for the full experiment and results. Status pending Rob's review.

Experiment 55D tested whether a first interpretation step can separate what the user established from what the model might infer, using a single-call two-field output contract (statedMeaning / possibleInference) across four fixed answers: weak priority, conditional trade-off, explicit hard constraint, and non-answer. Four live Ollama calls at http://192.168.1.111:11434 with qwen-claude:latest (~76.7s total). All four cases preserved statedMeaning without strengthening (stated_meaning_preserved: 4/4, strengthened: 0, lost: 0). Case 1's weak-priority answer stayed as relative importance only — direct improvement over 55C where the same answer was strengthened to constraint language. Conditionality survived in Case 2; explicit and uncertain controls stayed clean in Cases 3 and 4. Inference cleanly separated for Cases 1 and 2; unnecessary inferences generated for Cases 3 and 4 (hygiene issue, not leakage). No unsupported meaning leaked into statedMeaning. This does not yet prescribe production architecture. Graph, Behaviour Selection, UI and production remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-stated-vs-inferred.test.js for the full experiment and results. Status pending Rob's review.

Experiment 38 tested whether a genuinely cold session (no prior conversation context) can recover the project state from three documents alone. It recovered all capabilities, boundaries, and context-pack selection correctly without loading the full history or source code. All seven knowledge-management criteria confirmed met. One handoff update required: the open item "whether the handoff stays accurate after further advances" was resolved (handoff is accurate). The cold-start test passed.

Commit: pending (experiment: validate cold-start project recovery) — to be committed this session.

Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.

Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarify's triggers never activate in tested scenarios due to the too_broad health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.

Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:

• Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
• Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
• Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause Experiment 42 implemented Variant B's narrow Acknowledge exclusion gate in the production selector (commit 05d3d96). Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change.

Experiment 43 audited Clarify readiness across all 10 real assessment turns in existing fixtures. Zero turns produced Clarify-eligible states. Two findings: (1) the orienting-based Clarify rule is dead code because the assessor never produces phase=orienting, and (2) the too_broad trigger requires conditions no fixture exercises. Branch: feature/user-workspace-ux-v0.7.

Experiment 44 created one deliberately unclear starting scenario (five competing unknowns, zero resolved evidence, vague central statement) to test whether the assessor produces a Clarify-justifying signal. The assessor returned too_broad conversation health — confirming the previously untested too_broad path works correctly with real data. Clarify became eligible via Rule A. No production code changed. Remaining open: whether orienting phase is needed for earlier-stage clarification, and whether 2–3 competing threads (below the >3 threshold) can represent genuine scope confusion. Status pending Rob's review.

Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns — two and three returned cannot_determine; four and five returned too_broad. Clarify eligibility followed the same boundary. Resolved-item gate works correctly: one resolved item stays too_broad, two resolves it. The boundary appears mechanically clear but conceptually uncertain — synthetic fixtures cannot confirm whether three-to-four feels right to real users. No production code changed. What remains open: whether health should default to healthy (not cannot_determine) for 2–3 unknowns with no question; whether the threshold needs widening for real-world use. Status closed.

Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. Status closed.

Experiment 47 created a test-only diagnostic helper (inspectSharedUnknownAnchor) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.

Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: buildInitialGraph (does NOT populate dependsOn/affects/parentId), emergent reasoning via buildEmergentReasoningUnknown (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.

Experiment 49 tested whether any sequence of real production updates creates two or more active unknowns referencing the same populated relationship anchor. Results: no shared anchor found in production update sequences (both Cases A and B returned separate_anchors or insufficient_data). Structural capability exists but triggering logic never produces coexisting anchors. Status closed.

Experiment 50 tested whether shared edge topology from buildInitialGraph provides a usable coherence signal. Coherent and scattered inputs both produce identical edge topology — every unknown connects to the same summary node (kind=state) via depends_on edges, regardless of semantics. Initial shared edges are generic structural wiring, not coherence evidence. Closed (pending Rob's review).

Experiment 51 tested whether decision-relative relevance distinguishes coherent from scattered unknowns better than graph topology does. Within its training vocabulary, the classifier classified all four coherent unknowns as relevant and three of four scattered unknowns as irrelevant — but one scattered question was incorrectly flagged due to identical phrasing. Outside its vocabulary (different domain or paraphrased language), the classifier could not generalise: all four coherent unknowns received cannot_determine. The decision target never provided semantic context, only a binary action-keyword gate. No production code changed; no active engine behaviour changed; 70 tests pass (45 new + 25 Exp 21 regression). Status pending Rob's review.

Experiment 52 tested whether a small semantic interpretation step can judge decision relevance more reliably than keyword matching across paraphrases and domains. The semantic contract was implemented in tests/graph/decision-relevance-semantic.test.js. Live model comparison could not be completed because Ollama is not running on this machine — the test infrastructure uses the same /api/chat + format:json pattern as production. The deterministic keyword baseline continues to fail on paraphrases and new domains (confirmed via 15 passing guardrail tests). No semantic logic entered the active engine. The four-category decision-relevance contract remained unchanged. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-semantic.test.js for the full experiment and results.

Experiment 52H held domain constant (market-entry / customer demand) and varied ambiguous wording across five cases. Four phrasings were strengthened beyond their supplied meaning; only "connected to" preserved cannot_determine. The model appeared more consistent about strengthening incomplete meaning than about which stronger category it selected. Experiment 52I then tested one grounding rule rather than keyword patches: three of four ambiguous cases preserved cannot_determine under grounding without harming clear classifications, but "important to" remained strengthened — the model could classify correctly while still commenting on relationship strength. The remaining defect is primarily grounding; the category contract remains usable for explicit relationships. Same host and model retained; no production behaviour changed. Status pending Rob's review.

Experiment 52A recovered the semantic test infrastructure by correcting its configuration resolution. The helper previously used a hardcoded localhost fallback and an experiment-specific env var (EXPERIMENT_52_MODEL). Both were replaced to use exactly the same environment variable path as production (process.env.OLLAMA_BASE_URL / process.env.OLLAMA_MODEL) sourced from .env.local. Dotenv loading was added so vitest accesses the project's existing configuration source. Ollama at 192.168.1.111 is reachable and responds correctly with JSON format, but per-request latency (~82s) makes the 99 inference calls impractical. Configuration path verified correct; execution requires a faster inference host. No production code changed (0 lines in provider, config, analysis, orchestrator). Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-semantic.test.js lines 80–85 (helper).

Experiment 52C separated free-language semantic understanding from enum normalisation into two independent calls per case across five decision/question pairs. Meaning mode captured all five intended relationships correctly (5/5). Enum classification matched expected categories on four of five cases (4/5). One meaning-correct / enum-mismatch case: Case 2 (European regulatory compliance) was correctly described as supporting in both modes but classified as could_change_decision rather than supports_decision. Same Qwen model (qwen-claude:latest) and host were retained; no production behaviour changed. What remains uncertain: whether the meaning-enum gap generalises across decision domains, stability over repeated runs, and whether normalisation mechanisms can bridge the gap without altering interpretation. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-semantic-normalisation.test.js for results.

Experiment 52D isolated enum normalisation from semantic understanding: five fixed meaning statements (no decision target or question in the input) were mapped to the existing four-category contract via one live model call each. Four of five normalised to the expected enum. The compliance boundary case persisted — the model classified a "supports" relationship as could_change_decision, exposing genuine ambiguity between these two categories under the current definitions. The existing contract appears clear enough for a separate normalisation step; the remaining problem lies in category definitions, not semantic understanding or normalisation mechanism. Same Qwen model (qwen-claude:latest) and host (http://192.168.1.111:11434) were retained throughout. No production behaviour changed. What remains uncertain: whether the supports_decision ↔ could_change_decision boundary can be clarified without restructuring the contract, and whether the discrepancy holds under repeated runs. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-normalisation.test.js for results.

Experiment 54H tested whether trustworthy source identity can begin deterministically from raw user input before any LLM interpretation occurs. A test-only helper createSourceRecord(rawInput) hashes the verbatim text with SHA-256 to produce a stable sourceId, preserves verbatimText unchanged, and sets sourceType: "user_input". Nine focused tests confirm identical inputs produce identical IDs (Case 1 = Case 4), paraphrases produce different IDs (Case 1 ≠ Case 2), and multi-sentence input survives intact (Case 3). No semantic interpretation, summarisation, or LLM call occurs. Trustworthy source identity is feasible before reconstruction — the remaining gap is claim/node provenance and graph linkage, not source identity. Deterministic code can assign stable identity to raw material at the application boundary without any reasoning contract. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/reconstruction/deterministic-source-record.test.js.

Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state — no persistence layer retains them; and (2) no evidence records are created or retained during update cycles. Experiment 54E did not validate how those values are assigned in production. Experiment 54F audited evidenceType assignment: the reconstruction prompt instructs the LLM to classify each evidence item into one of five types based on its own judgment; no production code deterministically derives evidenceType from source origin — even reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came directly from the user." Experiment 54G audited whether evidence records nevertheless retain deterministic linkage to user words: neither verbatim text nor structured location references (character offsets, turn IDs) survive in any record field; source and attribution are free-form model-generated strings that may be null; the raw user statement is available to production code while reconstruction is being performed but is not retained alongside the returned reconstruction/evidence state for later deterministic verification. Evidence records do not contain verbatim source text or deterministic source locations; evidenceType is model classification, not trustworthy provenance. Current evidence records therefore cannot independently prove source provenance.

Experiment 54I showed multiple interpretations can share one deterministic source lineage via the Experiment 54H SHA-256 method. Both branches stayed traceable to the same source while remaining distinct in their reported additions. No interpretation was selected as better and no numeric scoring occurred. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/reconstruction/source-interpretation-lineage.test.js.

Experiment 54J proved the representation can separate source-supported from interpretation-added meaning using human-fixed references (13 tests, all pass). Grounding references were human-fixed; automated grounding remained untested. No production code or schemas changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/reconstruction/interpretation-source-grounding.test.js.

Experiment 54K tested whether the configured semantic model (qwen-claude:latest on 192.168.1.111:11434) can perform that grounding automatically. Three live Ollama calls (total ~96s): Case 1 (strengthening detection) = grounding_correct, Case 2 (multi-addition interpretation) = partial_grounding (missed one addition), Case 3 (faithful restatement control) = grounding_correct. Interpretation-added meaning did NOT leak into source-supported meaning in any case. One source-supported content gap: model missed "alternative causes" on the added side of Case 2. Automated semantic grounding is promising but imperfect — directionally viable but needs refinement before production use. Winner selection and downstream questions remain untested. No production code changed. Branch: feature/user-workspace-ux-v0.7. First file to inspect when resuming: tests/reconstruction/semantic-interpretation-grounding.test.js.

Experiment 54L repeated two identical grounding cases three times each to test stability across six live calls. The source-versus-added boundary was perfectly stable (zero leakage in all runs). Detection completeness appeared variable but manual analysis showed the instability came from the automated evaluator's paraphrase sensitivity, not the model itself. Case A strengthening identified in all 3 runs; Case B "other causes" and "not established as main problem" each identified in all 3 runs. Status pending Rob's review.

Experiment 54M tested whether two interpretations of one source can expose their substantive disagreement without deciding which is correct. Three live Ollama calls across three cases: real pricing attribution difference, paraphrase identity control, and competing causal explanations. All three classified as disagreement_correct by human semantic review. Paraphrase was correctly treated as agreement; shared meaning stayed separate; no invented disagreement or winner selection occurred. The comparison capability worked across the three tested patterns: substantive disagreement, paraphrase agreement, and competing causal explanations. Broader generalisation remains untested. Status pending Rob's review.

Experiment 54N tested whether an interpretation disagreement can be judged for material consequence on downstream information needs without generating a next question or choosing a winner. Three fixed cases: pricing ambiguity (consequence_correct), paraphrase identity control (consequence_correct), competing causes (consequence_failed — model returned false, missing that staff-capacity vs supplier evidence represent divergent investigation directions). 2/3 correct. Model did not choose a winner or generate an actual next question in any case. Branch: feature/user-workspace-ux-v0.7. First file to inspect when resuming: tests/reconstruction/semantic-disagreement-consequence.test.js.

5. What Remains Open

• The too_broad boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
• Health defaults to cannot_determine rather than healthy for 2–3 unknowns (no active question present); whether this is a bug or feature needs review;
• Whether the too_broad threshold needs widening so Clarify fires in more typical investigations;
• Whether user_overloaded health should be producible by the assessor for stalled/inconsistent evidence states;
• Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates dependsOn and parentId correctly — but requires comparable observations to trigger);

When This Knowledge-Management Phase Is Complete

Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):

1. A fresh session can resume from the handoff and one context pack; — met
2. Current state has been verified against implementation; — met
3. Historical material is outside default loading; — met
4. Current principles are separated from aspirational architecture; — met
5. Task-specific routing works for engine and UI tasks; — met
6. A cross-boundary task has been tested; — met (Experiment 37)
7. Maintaining the handoff does not require reading the full history. — met

Knowledge-management structure is ready for Rob's review before engine experiments resume.

6. How to Resume

1. Read docs/current-handoff.md.
2. Read docs/current-project-state.md.
3. Choose one pack from docs/task-context-packs.md.
4. Read .claude/architecture-guardrails.md before any code change.
5. Load extra context only for a named gap — record why.
6. Check Git status before continuing.

7. First Files by Work Type

Work type	Start with
Engine experiment	Engine Experiment pack
UI or mock work	UI and Mock pack
Architecture or contract review	Architecture or Contract pack
Knowledge management	Knowledge-Management pack

8. Resume Check

Answer before continuing:

1. What work is currently active?
2. What work is paused?
3. What was the latest completed experiment?
4. Which context pack applies to the next task?
5. Is there any uncommitted work?

---

Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, 56D–56H, 56L–56M, v0.8 closeout. Branch: feature/reasoning-fidelity-v0.8. First-pass reasoning-fidelity v0.8 complete to A–F scope.

Return-to-Work Note (Experiment 55F)

The first implementation pass against the reasoning refinement requirements is deferred one more round while we map how meaning actually flows through the production update path — before committing to any schema or architecture changes. A source-inspection exercise traced the full answer-to-reasoning chain from prompt building, through LLM response parsing and normalization, into graph mutation. The key finding: no provenance fields exist on nodes or edges in the current schema, meaning R1/R2 separation has no structural carrier. The answer string is used only for a narrow comparability check, not for semantic verification against proposed changes. A complete path map lives in docs/reasoning-production-path-map.md. Tomorrow should decide whether to add provenance fields to schemas, modify the prompt structure, or both — grounded in this accurate production trace rather than architectural speculation. Branch: feature/user-workspace-ux-v0.7.

Experiment 55A Summary — Clarification Uncertainty Preservation

Isolated the answer-resolution step using one fixed target (preference/trade-off or hard constraint) and four answers of different strength: fully explicit, weak priority, conditional trade-off, non-answer. Four live Ollama calls completed at http://192.168.1.111:11434 with qwen-claude:latest (~62s total). Case 1 (explicit hard constraint) resolved correctly. Case 2 (weak priority — "Risk matters more to me.") over-resolved: the model set targetResolved=true and inferred "not a rigid, non-negotiable constraint" — meaning stronger than the user supplied. Case 3 (conditional trade-off) resolved correctly on the target but flattened conditionality into flat "preference or trade-off" language without preserving the conditional qualification ("might accept some"). Case 4 (non-answer) correctly remained unresolved with appropriate remaining uncertainty. Two of the four tested answers showed loss of nuance: one was over-resolved and one retained the correct target category while losing conditional qualification. The same over-resolution reproduced with a fixed target, so target broadening is not required for the failure to occur. Broader generalisation across other models and answers remains untested. Behaviour Selection, graph, UI, and production integration remain untouched. Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-uncertainty-preservation.test.js for the full experiment and results. Status pending Rob's review.

Experiment 56A Summary — Regression B Proposal Validation Enum Mismatch

The first implementation pass added proposal-level answerMeaning with a pre-mutation compatibility guard. Deterministic regression tests A-D passed, but live Ollama runs showed Regression B failing at proposal_validation before the pre-mutation guard could execute. Experiment 56A traced this to a schema mismatch: Qwen returned supportCategory: "conditional_qualification" while the production Zod schema only accepts conditional_tradeoff among five values. The value survives normalization unchanged (normalize step handles node kind aliases, not supportCategory). The failure is at Zod validation — a proposal-contract issue, not a guard failure. Hypothesis confirmed. No fix was attempted. Branch: feature/reasoning-fidelity-v0.8. First file to inspect when resuming: lib/graph/schema.js line 165 (Zod enum for supportCategory) or the experiment record at docs/experiment-56a.md. Status pending Rob's review.

Experiment 56B Summary — Regression B Live Run After Normalisation

Commit 36faf70 added normalization for conditional_qualification → conditional_tradeoff, but a live Regression B run returned a different variant: supportCategory: "conditional_preference". The existing normalisation map does not cover this value. Two independent Zod errors occurred: (1) conditional_preference not in the supportCategory enum, and (2) resolutionGuidance was free-text instead of an enum value. Run-to-run model variation confirmed — the same fixed input produced conditional_qualification in Ex 56A and conditional_preference in Ex 56B. The pre-mutation guard remains unreachable because proposal_validation rejects first. Failure classification: FAIL — normalization / proposal contract. Branch: feature/reasoning-fidelity-v0.8. File to inspect when resuming: docs/experiment-56b.md. Status pending Rob's review.

Experiment 56D Summary — Regression B via Real Production Path

Tested whether deterministic derivation refinement from commit 3e78d57 (refine answer meaning derivation for negation and qualification) works end-to-end through the real updateCase() production path. Input: source "I want the business to grow, but I don't want to take on more risk." Answer "I'd normally avoid more risk, but for the right opportunity I might accept some." — the canonical conditional trade-off case (Regression B).

Result: PASS. Five of five checkpoints confirmed across one live Ollama call at http://192.168.1.111:11434 with qwen-claude:latest:

1. userSupportedMeaning correctly extracted conditional semantics — separated default preference (avoid risk) from qualification (override for right opportunity).
2. Deterministic profile derivation produced conditional_tradeoff category despite LLM returning null for supportCategory.
3. Pre-mutation guard passed with zero errors — the normalized/derived meaning is compatible.
4. Graph mutation proposed: n-risk-constraint resolved from unknown→resolved; emergent unknown n-opportunity-criteria created (unknown/unknown) capturing the threshold definition need.
5. Follow-up question correctly targets the emergent conditional/threshold unknown.

Key observation: The LLM does not auto-populate supportCategory — it is consistently null in answerMeaning. The deterministic derivation layer in readDiagnostics (and the inline pipeline) is the sole mechanism by which meaning profile category gets determined. This confirms the design: LLM produces raw meaning; deterministic logic categorizes it. No regression detected. Full results in docs/experiment-56d.md. Branch: feature/reasoning-fidelity-v0.8. Status pending Rob's review.

Experiment 56J Summary — Regression D Explicit Hard Constraint Semantic Probe

Tested whether the configured live Ollama model (qwen-claude:latest at http://192.168.1.111:11434) preserves explicit hard-constraint meaning from user answer "It's a hard constraint. I don't want any increase in risk." — Regression D from docs/reasoning-refinement-requirements.md.

One live Ollama call (19,343 ms) returned userSupportedMeaning: "Avoiding additional risk is a hard constraint, and no increase in risk is acceptable." with possibleInference: null.

Classification: PASS. The model preserved the explicit hard-constraint status without weakening it into preference/trade-off language and did not add unsupported interpretation. possibleInference is null, which is appropriate for a direct unambiguous answer.

This experiment does not prove fidelity for other regression cases (E, F), consistency across multiple runs, or behavior in production reasoning paths. Branch: feature/reasoning-fidelity-v0.8. Files: tests/reconstruction/semantic-regression-d-explicit-hard-constraint.test.js and docs/experiment-56j.md. Status pending Rob's review.

Experiment 56K Summary — Evidence-resolvable disagreement must not become user clarification

Tested whether the configured live Ollama model (qwen-claude:latest at http://192.168.1.111:11434) distinguishes evidence-resolvable uncertainty from user-owned ambiguity — Regression E from docs/reasoning-refinement-requirements.md.

Fixed case: Delivery delay concern with competing causes ("Staff capacity may be the issue" / "Supplier lead times are likely responsible.") — resolvable by evidence gathering, not user clarification.

One live Ollama call (18,580 ms) returned uncertaintyType: "evidence_needed" with specific evidence target: "Current internal staffing capacity levels and external supplier lead time records." No user clarification was introduced.

Classification: PASS. The model correctly identified the disagreement as requiring evidence rather than asking the user to settle an externally knowable question by clarification. It specified concrete, relevant evidence — demonstrating understanding of the causal structure rather than producing a generic classification. This confirms the model can preserve the distinction between "evidence needed to determine what is true" and "clarification needed because only the user can establish meaning/preference/intent/constraint" for this tested case.

This experiment does not prove fidelity for Regression F (user-owned ambiguity), consistency across domains/phrasings, downstream reasoning preservation, or end-to-end production flow. Branch: feature/reasoning-fidelity-v0.8. Files: tests/reconstruction/semantic-regression-e-evidence-vs-clarification.test.js and docs/experiment-56k.md. Status pending Rob's review.

Experiment 56L Summary — User-owned ambiguity requires clarification, not evidence

Tested whether the configured live Ollama model (qwen-claude:latest at http://192.168.1.111:11434) recognises that a preference-vs-constraint distinction belongs to the user's own meaning and requires clarification rather than external evidence — Regression F from docs/reasoning-refinement-requirements.md.

Fixed case: "I want the business to grow, but I don't want to take on more risk." — user has not specified whether avoiding additional risk is a hard constraint or a strong preference/trade-off.

One live Ollama call (14,032 ms) returned uncertaintyType: "user_clarification_needed" with evidenceNeeded: null and specific userClarificationNeeded describing the non-negotiable-versus-trade-off distinction only the user can establish. Matches pre-written human reference exactly at category level.

Classification: PASS. The model correctly identified the ambiguity as user-owned, did not introduce spurious evidence gathering, and preserved the evidence-vs-user-meaning distinction cleanly.

This experiment does not prove consistency across repeated runs, fidelity for other regression cases (A–E, G+), behavior in production reasoning paths, or downstream integration with Behaviour Selection or the SituationGraph. Branch: feature/reasoning-fidelity-v0.8. Files: tests/reconstruction/semantic-regression-f-user-owned-ambiguity.test.js and docs/experiment-56l.md. Status pending Rob's review.

Experiment 56M Summary — Production evidence vs clarification routing validation

Validated one production claim after Codex commit f861e2c: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic formulateQuestion() was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: feature/reasoning-fidelity-v0.8. File: docs/experiment-56m.md. Status pending Rob's review.

Reasoning Fidelity v0.8 — First Pass Closeout

The first-pass reasoning-fidelity refinement is complete to its agreed scope.

Regression boundaries A–F have been investigated and the production defects identified from those boundaries have been addressed:

• A — weak priority: supported against unsupported strengthening via pre-mutation compatibility guard;
• B — conditional trade-off: qualification preserved through deterministic derivation and normalisation;
• C — unresolved uncertainty: may remain unresolved when the user supplies no position;
• D — explicit hard constraint: explicit meaning preserved;
• E — evidence-resolvable disagreement: routed to evidence gathering;
• F — user-owned ambiguity: routed to clarification.

No demonstrated production defect remains inside the A–F first-pass boundary. Deterministic production validation is passing (commit ec398dc validating evidence vs. clarification routing).

Current HEAD: ec398dc — experiment: validate evidence versus clarification routing Key commits: f861e2c (preserve evidence vs. clarification distinction), ec398dc (validate evidence vs. clarification routing)

The two important production capabilities now present are:

1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary;
2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation.

Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme. These open questions remain for future evidence-driven investigation, not as current defects:

• broader wording/domain/model robustness;
• clarification-target precision outside the tested cases;
• durable per-node provenance of user-supported meaning vs inference;
• whether rejected proposals should eventually be adapted rather than simply blocked;
• end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI;
• multilingual robustness;
• any future defect exposed by real use.

Experiment 57B — Grounded Unclassified Affirmative Answer (Live)

Validated one live semantic probe: the configured Qwen model (qwen-claude:latest at http://192.168.1.111:11434) produced grounded user-supported meaning for a legitimate affirmative answer that falls into other (unclassified) — exactly the case blocked by v0.8's guard. Result: PASS. userSupportedMeaning: "The user directly states that cost reduction, specifically approximately £2M in annual office-overhead savings, is a primary stated reason for considering the relocation." No unsupported strengthening beyond the raw answer. One live call (20,351 ms). This confirms the live semantic premise behind Codex 4e4d0fa (semantic token overlap grounding). What remains unproven: token-overlap adequacy across diverse answers, stability across runs, production guard integration. Full results in docs/experiment-57b.md. Status pending Rob's review.

Experiment 57A — Contaminated After First Valid Observation

The first post-v0.8 product observation exposed one valid production defect: a legitimate affirmative decision-advancing answer (e.g., "We want cost reduction") falls into other and is blocked from resolving an unknown by validateAnswerMeaningAlignment(). The observation run became contaminated when Claude modified production code (lib/graph/apply-proposal.js, lib/graph/schema.js) adding four new categories and keyword detectors. Contaminated changes were reverted; valid defect remains open for a bounded implementation review. Repository production state is back at the merged v0.8 baseline (commit 14d68f1). Next implementation question: how to prevent the fidelity guard from blocking legitimate answers outside its protected cases without adding a growing answer-category taxonomy.