confidence-engine/docs/current-handoff.md

Current Return-to-Work Handoff — Confidence Engine

This file describes only the latest stopping point. Replace its current-work sections when the project moves on. Historical evidence remains in the design log and archive.

1. Where We Left It

• Engine experiments resumed with a passive validation;
• UI experiments remain paused;
• Knowledge-management experiments are complete;
• Experiment 39 tested the existing Behaviour Selection module against real Investigation State Assessment outputs across three scenarios;
• Acknowledge dominates (71% of selections) because it fires first when health=healthy, blocking Summarise/Pause/Clarify even in concluding or stalled states.

This handoff describes the latest stopping point only. When work moves on, replace stale current-work details rather than appending another historical note. Historical experiment and commit information belongs in docs/design-evolution-log.md.

2. What Is True Now

• Main active engine path: deterministic reasoning pipeline (scenario reconstruction, graph update, unknown selection, question formulation, turn orchestration).
• Passive experimental classifiers from Experiments 18–25B remain isolated diagnostic layers; none control the user-facing investigation. Behaviour Selection was passively evaluated against real assessment outputs in Experiment 39 — it produced all valid behaviours but with skewed distribution (Acknowledge 71%).
• Keyword and phrase-based scope detection remains provisional scaffolding.
• docs/current-project-state.md is the main entry point for active project state.
• docs/task-context-packs.md chooses the minimum context documents for each work type.

3. Why Work Is Paused

Engine and UI work were deliberately paused because documentation had grown large enough to overload Claude and make returning across sessions difficult. The current phase is simplifying what a fresh session must load to understand the project, without losing evidential history. Historical material remains available under docs/archive/.

4. What Was Just Completed

Experiment 37 corrected the routing defect from Experiment 36 and tested a cross-boundary engine/UI task. It validated that two context packs can be combined deliberately while keeping working context small, explicit and accurate. All seven knowledge-management criteria are now met. No source code changed. No files moved or deleted.

Commit: 544573a (experiment: validate cross-boundary context routing)

Experiment 38 tested whether a genuinely cold session (no prior conversation context) can recover the project state from three documents alone. It recovered all capabilities, boundaries, and context-pack selection correctly without loading the full history or source code. All seven knowledge-management criteria confirmed met. One handoff update required: the open item "whether the handoff stays accurate after further advances" was resolved (handoff is accurate). The cold-start test passed.

Commit: pending (experiment: validate cold-start project recovery) — to be committed this session.

Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.

Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarify's triggers never activate in tested scenarios due to the too_broad health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.

Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:

• Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
• Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
• Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause Experiment 42 implemented Variant B's narrow Acknowledge exclusion gate in the production selector (commit 05d3d96). Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change.

Experiment 43 audited Clarify readiness across all 10 real assessment turns in existing fixtures. Zero turns produced Clarify-eligible states. Two findings: (1) the orienting-based Clarify rule is dead code because the assessor never produces phase=orienting, and (2) the too_broad trigger requires conditions no fixture exercises. Branch: feature/user-workspace-ux-v0.7.

Experiment 44 created one deliberately unclear starting scenario (five competing unknowns, zero resolved evidence, vague central statement) to test whether the assessor produces a Clarify-justifying signal. The assessor returned too_broad conversation health — confirming the previously untested too_broad path works correctly with real data. Clarify became eligible via Rule A. No production code changed. Remaining open: whether orienting phase is needed for earlier-stage clarification, and whether 2–3 competing threads (below the >3 threshold) can represent genuine scope confusion. Status pending Rob's review.

Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns — two and three returned cannot_determine; four and five returned too_broad. Clarify eligibility followed the same boundary. Resolved-item gate works correctly: one resolved item stays too_broad, two resolves it. The boundary appears mechanically clear but conceptually uncertain — synthetic fixtures cannot confirm whether three-to-four feels right to real users. No production code changed. What remains open: whether health should default to healthy (not cannot_determine) for 2–3 unknowns with no question; whether the threshold needs widening for real-world use. Status closed.

Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. Status closed.

Experiment 47 created a test-only diagnostic helper (inspectSharedUnknownAnchor) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.

Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: buildInitialGraph (does NOT populate dependsOn/affects/parentId), emergent reasoning via buildEmergentReasoningUnknown (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.

Experiment 49 tested whether any sequence of real production updates creates two or more active unknowns referencing the same populated relationship anchor. Results: no shared anchor found in production update sequences (both Cases A and B returned separate_anchors or insufficient_data). Structural capability exists but triggering logic never produces coexisting anchors. Status closed.

Experiment 50 tested whether shared edge topology from buildInitialGraph provides a usable coherence signal. Coherent and scattered inputs both produce identical edge topology — every unknown connects to the same summary node (kind=state) via depends_on edges, regardless of semantics. Initial shared edges are generic structural wiring, not coherence evidence. Closed (pending Rob's review).

Experiment 51 tested whether decision-relative relevance distinguishes coherent from scattered unknowns better than graph topology does. Within its training vocabulary, the classifier classified all four coherent unknowns as relevant and three of four scattered unknowns as irrelevant — but one scattered question was incorrectly flagged due to identical phrasing. Outside its vocabulary (different domain or paraphrased language), the classifier could not generalise: all four coherent unknowns received cannot_determine. The decision target never provided semantic context, only a binary action-keyword gate. No production code changed; no active engine behaviour changed; 70 tests pass (45 new + 25 Exp 21 regression). Status pending Rob's review.

Experiment 52 tested whether a small semantic interpretation step can judge decision relevance more reliably than keyword matching across paraphrases and domains. The semantic contract was implemented in tests/graph/decision-relevance-semantic.test.js. Live model comparison could not be completed because Ollama is not running on this machine — the test infrastructure uses the same /api/chat + format:json pattern as production. The deterministic keyword baseline continues to fail on paraphrases and new domains (confirmed via 15 passing guardrail tests). No semantic logic entered the active engine. The four-category decision-relevance contract remained unchanged. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-semantic.test.js for the full experiment and results.

Experiment 52H held domain constant (market-entry / customer demand) and varied ambiguous wording across five cases. Four phrasings were strengthened beyond their supplied meaning; only "connected to" preserved cannot_determine. The model appeared more consistent about strengthening incomplete meaning than about which stronger category it selected. Experiment 52I then tested one grounding rule rather than keyword patches: three of four ambiguous cases preserved cannot_determine under grounding without harming clear classifications, but "important to" remained strengthened — the model could classify correctly while still commenting on relationship strength. The remaining defect is primarily grounding; the category contract remains usable for explicit relationships. Same host and model retained; no production behaviour changed. Status pending Rob's review.

Experiment 52A recovered the semantic test infrastructure by correcting its configuration resolution. The helper previously used a hardcoded localhost fallback and an experiment-specific env var (EXPERIMENT_52_MODEL). Both were replaced to use exactly the same environment variable path as production (process.env.OLLAMA_BASE_URL / process.env.OLLAMA_MODEL) sourced from .env.local. Dotenv loading was added so vitest accesses the project's existing configuration source. Ollama at 192.168.1.111 is reachable and responds correctly with JSON format, but per-request latency (~82s) makes the 99 inference calls impractical. Configuration path verified correct; execution requires a faster inference host. No production code changed (0 lines in provider, config, analysis, orchestrator). Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-semantic.test.js lines 80–85 (helper).

Experiment 52C separated free-language semantic understanding from enum normalisation into two independent calls per case across five decision/question pairs. Meaning mode captured all five intended relationships correctly (5/5). Enum classification matched expected categories on four of five cases (4/5). One meaning-correct / enum-mismatch case: Case 2 (European regulatory compliance) was correctly described as supporting in both modes but classified as could_change_decision rather than supports_decision. Same Qwen model (qwen-claude:latest) and host were retained; no production behaviour changed. What remains uncertain: whether the meaning-enum gap generalises across decision domains, stability over repeated runs, and whether normalisation mechanisms can bridge the gap without altering interpretation. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-semantic-normalisation.test.js for results.

Experiment 52D isolated enum normalisation from semantic understanding: five fixed meaning statements (no decision target or question in the input) were mapped to the existing four-category contract via one live model call each. Four of five normalised to the expected enum. The compliance boundary case persisted — the model classified a "supports" relationship as could_change_decision, exposing genuine ambiguity between these two categories under the current definitions. The existing contract appears clear enough for a separate normalisation step; the remaining problem lies in category definitions, not semantic understanding or normalisation mechanism. Same Qwen model (qwen-claude:latest) and host (http://192.168.1.111:11434) were retained throughout. No production behaviour changed. What remains uncertain: whether the supports_decision ↔ could_change_decision boundary can be clarified without restructuring the contract, and whether the discrepancy holds under repeated runs. Branch: feature/user-workspace-ux-v0.7. First file to inspect: tests/graph/decision-relevance-normalisation.test.js for results.

5. What Remains Open

• The too_broad boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
• Health defaults to cannot_determine rather than healthy for 2–3 unknowns (no active question present); whether this is a bug or feature needs review;
• Whether the too_broad threshold needs widening so Clarify fires in more typical investigations;
• Whether user_overloaded health should be producible by the assessor for stalled/inconsistent evidence states;
• Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates dependsOn and parentId correctly — but requires comparable observations to trigger);

When This Knowledge-Management Phase Is Complete

Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):

1. A fresh session can resume from the handoff and one context pack; — met
2. Current state has been verified against implementation; — met
3. Historical material is outside default loading; — met
4. Current principles are separated from aspirational architecture; — met
5. Task-specific routing works for engine and UI tasks; — met
6. A cross-boundary task has been tested; — met (Experiment 37)
7. Maintaining the handoff does not require reading the full history. — met

Knowledge-management structure is ready for Rob's review before engine experiments resume.

6. How to Resume

1. Read docs/current-handoff.md.
2. Read docs/current-project-state.md.
3. Choose one pack from docs/task-context-packs.md.
4. Read .claude/architecture-guardrails.md before any code change.
5. Load extra context only for a named gap — record why.
6. Check Git status before continuing.

7. First Files by Work Type

Work type	Start with
Engine experiment	Engine Experiment pack
UI or mock work	UI and Mock pack
Architecture or contract review	Architecture or Contract pack
Knowledge management	Knowledge-Management pack

8. Resume Check

Answer before continuing:

1. What work is currently active?
2. What work is paused?
3. What was the latest completed experiment?
4. Which context pack applies to the next task?
5. Is there any uncommitted work?

---

Created by Experiment 34. Updated by Experiments 38–53, 54A. Branch: feature/user-workspace-ux-v0.7.

Return-to-Work Note (Experiment 53 → 54A)

Experiment 53 proved semantic separation was possible; 54A inspected whether the current SituationGraph can preserve that distinction from source code alone. The audit confirms the graph's kind/status/confidence fields describe epistemic status only — there is no provenance field recording where information came from. Supplied-versus-inferred provenance is not recoverable from validated graph state. No production code changed; branch: feature/user-workspace-ux-v0.7. What remains open: a compatibility layer to bridge the semantic separation proved in 53 with the graph's provenance gap identified in 54A. First file to inspect when resuming: lib/graph/schema.js for current node fields, then lib/graph/apply-proposal.js for the production update path that populates nodes during inference turns.