confidence-engine/docs/current-handoff.md

# Current Return-to-Work Handoff — Confidence Engine

> This file describes only the latest stopping point. Replace its current-work sections when the project moves on. Historical evidence remains in the design log and archive.

## 1. Where We Left It

- Engine experiments resumed with a passive validation;
- UI experiments remain paused;
- Knowledge-management experiments are complete;
- Experiment 39 tested the existing Behaviour Selection module against real Investigation State Assessment outputs across three scenarios;
- Acknowledge dominates (71% of selections) because it fires first when health=healthy, blocking Summarise/Pause/Clarify even in concluding or stalled states.

> This handoff describes the latest stopping point only. When work moves on, replace stale current-work details rather than appending another historical note. Historical experiment and commit information belongs in `docs/design-evolution-log.md`.

## 2. What Is True Now

- Main active engine path: deterministic reasoning pipeline (scenario reconstruction, graph update, unknown selection, question formulation, turn orchestration).
- Passive experimental classifiers from Experiments 18–25B remain isolated diagnostic layers; none control the user-facing investigation. Behaviour Selection was passively evaluated against real assessment outputs in Experiment 39 — it produced all valid behaviours but with skewed distribution (Acknowledge 71%).
- Keyword and phrase-based scope detection remains provisional scaffolding.
- `docs/current-project-state.md` is the main entry point for active project state.
- Experiment 54D confirmed the production update prompt explicitly separates the user answer (## User Answer section) but the proposal schema has no provenance field — source identity at prompt level is explicit, per-node provenance at output level is absent.
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.

## 3. Why Work Is Paused

Engine and UI work were deliberately paused because documentation had grown large enough to overload Claude and make returning across sessions difficult. The current phase is simplifying what a fresh session must load to understand the project, without losing evidential history. Historical material remains available under `docs/archive/`.

## 4. What Was Just Completed

Experiment 37 corrected the routing defect from Experiment 36 and tested a cross-boundary engine/UI task. It validated that two context packs can be combined deliberately while keeping working context small, explicit and accurate. All seven knowledge-management criteria are now met. No source code changed. No files moved or deleted.

**Commit:** `544573a` (experiment: validate cross-boundary context routing)

Experiment 38 tested whether a genuinely cold session (no prior conversation context) can recover the project state from three documents alone. It recovered all capabilities, boundaries, and context-pack selection correctly without loading the full history or source code. All seven knowledge-management criteria confirmed met. One handoff update required: the open item "whether the handoff stays accurate after further advances" was resolved (handoff is accurate). The cold-start test passed.

**Commit:** pending (experiment: validate cold-start project recovery) — to be committed this session.

Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.

Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarify's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.

Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:
- Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
- Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
- Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause
Experiment 42 implemented Variant B's narrow Acknowledge exclusion gate in the production selector (commit `05d3d96`). Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change.

Experiment 43 audited Clarify readiness across all 10 real assessment turns in existing fixtures. Zero turns produced Clarify-eligible states. Two findings: (1) the orienting-based Clarify rule is dead code because the assessor never produces phase=orienting, and (2) the too_broad trigger requires conditions no fixture exercises. Branch: `feature/user-workspace-ux-v0.7`.

Experiment 44 created one deliberately unclear starting scenario (five competing unknowns, zero resolved evidence, vague central statement) to test whether the assessor produces a Clarify-justifying signal. The assessor returned `too_broad` conversation health — confirming the previously untested too_broad path works correctly with real data. Clarify became eligible via Rule A. No production code changed. Remaining open: whether orienting phase is needed for earlier-stage clarification, and whether 2–3 competing threads (below the >3 threshold) can represent genuine scope confusion. Status pending Rob's review.

Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns — two and three returned cannot_determine; four and five returned too_broad. Clarify eligibility followed the same boundary. Resolved-item gate works correctly: one resolved item stays too_broad, two resolves it. The boundary appears mechanically clear but conceptually uncertain — synthetic fixtures cannot confirm whether three-to-four feels right to real users. No production code changed. What remains open: whether health should default to healthy (not cannot_determine) for 2–3 unknowns with no question; whether the threshold needs widening for real-world use. Status closed.

Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. Status closed.

Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.

Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: `buildInitialGraph` (does NOT populate dependsOn/affects/parentId), emergent reasoning via `buildEmergentReasoningUnknown` (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.

Experiment 49 tested whether any sequence of real production updates creates two or more active unknowns referencing the same populated relationship anchor. Results: no shared anchor found in production update sequences (both Cases A and B returned separate_anchors or insufficient_data). Structural capability exists but triggering logic never produces coexisting anchors. Status closed.

Experiment 50 tested whether shared edge topology from `buildInitialGraph` provides a usable coherence signal. Coherent and scattered inputs both produce identical edge topology — every unknown connects to the same summary node (kind=state) via depends_on edges, regardless of semantics. Initial shared edges are generic structural wiring, not coherence evidence. Closed (pending Rob's review).

Experiment 51 tested whether decision-relative relevance distinguishes coherent from scattered unknowns better than graph topology does. Within its training vocabulary, the classifier classified all four coherent unknowns as relevant and three of four scattered unknowns as irrelevant — but one scattered question was incorrectly flagged due to identical phrasing. Outside its vocabulary (different domain or paraphrased language), the classifier could not generalise: all four coherent unknowns received `cannot_determine`. The decision target never provided semantic context, only a binary action-keyword gate. No production code changed; no active engine behaviour changed; 70 tests pass (45 new + 25 Exp 21 regression). Status pending Rob's review.

Experiment 52 tested whether a small semantic interpretation step can judge decision relevance more reliably than keyword matching across paraphrases and domains. The semantic contract was implemented in `tests/graph/decision-relevance-semantic.test.js`. Live model comparison could not be completed because Ollama is not running on this machine — the test infrastructure uses the same `/api/chat` + `format:json` pattern as production. The deterministic keyword baseline continues to fail on paraphrases and new domains (confirmed via 15 passing guardrail tests). No semantic logic entered the active engine. The four-category decision-relevance contract remained unchanged. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic.test.js` for the full experiment and results.

Experiment 52H held domain constant (market-entry / customer demand) and varied ambiguous wording across five cases. Four phrasings were strengthened beyond their supplied meaning; only "connected to" preserved `cannot_determine`. The model appeared more consistent about strengthening incomplete meaning than about which stronger category it selected. Experiment 52I then tested one grounding rule rather than keyword patches: three of four ambiguous cases preserved `cannot_determine` under grounding without harming clear classifications, but "important to" remained strengthened — the model could classify correctly while still commenting on relationship strength. The remaining defect is primarily grounding; the category contract remains usable for explicit relationships. Same host and model retained; no production behaviour changed. Status pending Rob's review.

Experiment 52A recovered the semantic test infrastructure by correcting its configuration resolution. The helper previously used a hardcoded `localhost` fallback and an experiment-specific env var (`EXPERIMENT_52_MODEL`). Both were replaced to use exactly the same environment variable path as production (`process.env.OLLAMA_BASE_URL` / `process.env.OLLAMA_MODEL`) sourced from `.env.local`. Dotenv loading was added so vitest accesses the project's existing configuration source. Ollama at 192.168.1.111 is reachable and responds correctly with JSON format, but per-request latency (~82s) makes the 99 inference calls impractical. Configuration path verified correct; execution requires a faster inference host. No production code changed (0 lines in provider, config, analysis, orchestrator). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic.test.js` lines 80–85 (helper).

Experiment 52C separated free-language semantic understanding from enum normalisation into two independent calls per case across five decision/question pairs. Meaning mode captured all five intended relationships correctly (5/5). Enum classification matched expected categories on four of five cases (4/5). One meaning-correct / enum-mismatch case: Case 2 (European regulatory compliance) was correctly described as supporting in both modes but classified as `could_change_decision` rather than `supports_decision`. Same Qwen model (`qwen-claude:latest`) and host were retained; no production behaviour changed. What remains uncertain: whether the meaning-enum gap generalises across decision domains, stability over repeated runs, and whether normalisation mechanisms can bridge the gap without altering interpretation. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic-normalisation.test.js` for results.

Experiment 52D isolated enum normalisation from semantic understanding: five fixed meaning statements (no decision target or question in the input) were mapped to the existing four-category contract via one live model call each. Four of five normalised to the expected enum. The compliance boundary case persisted — the model classified a "supports" relationship as `could_change_decision`, exposing genuine ambiguity between these two categories under the current definitions. The existing contract appears clear enough for a separate normalisation step; the remaining problem lies in category definitions, not semantic understanding or normalisation mechanism. Same Qwen model (`qwen-claude:latest`) and host (`http://192.168.1.111:11434`) were retained throughout. No production behaviour changed. What remains uncertain: whether the `supports_decision` ↔ `could_change_decision` boundary can be clarified without restructuring the contract, and whether the discrepancy holds under repeated runs. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-normalisation.test.js` for results.

Experiment 54H tested whether trustworthy source identity can begin deterministically from raw user input before any LLM interpretation occurs. A test-only helper `createSourceRecord(rawInput)` hashes the verbatim text with SHA-256 to produce a stable `sourceId`, preserves `verbatimText` unchanged, and sets `sourceType: "user_input"`. Nine focused tests confirm identical inputs produce identical IDs (Case 1 = Case 4), paraphrases produce different IDs (Case 1 ≠ Case 2), and multi-sentence input survives intact (Case 3). No semantic interpretation, summarisation, or LLM call occurs. Trustworthy source identity is feasible before reconstruction — the remaining gap is claim/node provenance and graph linkage, not source identity. Deterministic code can assign stable identity to raw material at the application boundary without any reasoning contract. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/reconstruction/deterministic-source-record.test.js`.

Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state — no persistence layer retains them; and (2) no evidence records are created or retained during update cycles. Experiment 54E did not validate how those values are assigned in production. Experiment 54F audited evidenceType assignment: the reconstruction prompt instructs the LLM to classify each evidence item into one of five types based on its own judgment; no production code deterministically derives evidenceType from source origin — even reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came directly from the user." Experiment 54G audited whether evidence records nevertheless retain deterministic linkage to user words: neither verbatim text nor structured location references (character offsets, turn IDs) survive in any record field; `source` and `attribution` are free-form model-generated strings that may be null; the raw user statement is available to production code while reconstruction is being performed but is not retained alongside the returned reconstruction/evidence state for later deterministic verification. Evidence records do not contain verbatim source text or deterministic source locations; `evidenceType` is model classification, not trustworthy provenance. Current evidence records therefore cannot independently prove source provenance.

## 5. What Remains Open

- The `too_broad` boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
- Health defaults to `cannot_determine` rather than `healthy` for 2–3 unknowns (no active question present); whether this is a bug or feature needs review;
- Whether the `too_broad` threshold needs widening so Clarify fires in more typical investigations;
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states;
- Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates `dependsOn` and `parentId` correctly — but requires comparable observations to trigger);

### When This Knowledge-Management Phase Is Complete

Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):

1. A fresh session can resume from the handoff and one context pack; — **met**
2. Current state has been verified against implementation; — **met**
3. Historical material is outside default loading; — **met**
4. Current principles are separated from aspirational architecture; — **met**
5. Task-specific routing works for engine and UI tasks; — **met**
6. A cross-boundary task has been tested; — **met** (Experiment 37)
7. Maintaining the handoff does not require reading the full history. — **met**

> Knowledge-management structure is ready for Rob's review before engine experiments resume.

## 6. How to Resume

1. Read `docs/current-handoff.md`.
2. Read `docs/current-project-state.md`.
3. Choose one pack from `docs/task-context-packs.md`.
4. Read `.claude/architecture-guardrails.md` before any code change.
5. Load extra context only for a named gap — record why.
6. Check Git status before continuing.

## 7. First Files by Work Type

| Work type | Start with |
|---|---|
| Engine experiment | Engine Experiment pack |
| UI or mock work | UI and Mock pack |
| Architecture or contract review | Architecture or Contract pack |
| Knowledge management | Knowledge-Management pack |

## 8. Resume Check

Answer before continuing:

1. What work is currently active?
2. What work is paused?
3. What was the latest completed experiment?
4. Which context pack applies to the next task?
5. Is there any uncommitted work?

---

*Created by Experiment 34. Updated by Experiments 38–53, 54A–54H. Branch: `feature/user-workspace-ux-v0.7`.*

### Return-to-Work Note (Experiment 54H)

Experiments 54F and 54G established that current evidence metadata cannot prove provenance: `evidenceType` is model classification, not production-code origin. Experiment 54H tested whether trustworthy source identity can instead begin deterministically from the raw user input before any LLM interpretation occurs. Verbatim text preservation and stable identity both work — SHA-256 of raw input produces deterministic `sourceId` without semantic processing. Source identity is established; claim provenance and graph-node linkage remain explicitly untested. No production code changed. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/reconstruction/deterministic-source-record.test.js`.