confidence-engine/docs/current-handoff.md

Current Handoff — Confidence Engine

Role: Concise operational snapshot for resuming work today. Not a historical diary. The design evolution archive index at docs/design-evolution/README.md provides progressive loading of experiment history; load the relevant chapter only when a specific historical question requires it.

CURRENT MVP DIRECTION

Initial-decomposition hardening is frozen for the current MVP stage.

Do not resume:

• repeated-same-input reconstruction experiments
• Qwen/Terra reconstruction comparison
• initial prompt refinement
• supplier/weekend-shift decomposition experiments
• relationship-preservation experiments
• causal-fidelity experiments

unless new end-to-end user-flow evidence reopens one of those boundaries.

Immediate next evidence question:

Measure one real browser investigation configured for OpenAI/Terra across every LLM stage, including user-visible latency, call sequence, and cost.

If YES, the next live experiment is one timed/costed OpenAI UI investigation measuring:

• user-visible latency
• OpenAI call count
• token usage where available
• approximate cost per investigation

Server-owned UI journey provider experiment

• Server-only CONFIDENCE_ENGINE_EXPERIMENT_PROVIDER=openai centrally resolves the existing OpenAI provider and gpt-5.6-terra; without it, normal production resolution remains Ollama/Qwen. An OpenAI key alone does not switch providers.
• Browser request contracts and client state remain unchanged. Deterministic coverage includes initial start, normal update, episode reconsideration, focused deconstruction, overview synthesis, and Current Understanding synthesis.
• Zero live calls occurred. Next boundary: one real Playwright-driven Terra investigation measuring user-visible latency, actual LLM-call sequence, and OpenAI usage/cost.

OpenAI alternate structured output

• The OpenAI provider now honors a caller-supplied structured-output schema, applying its existing strict-schema transport projection; absent an alternate schema, initial reconstruction retains its existing strict schema and transport normalization.
• The next live run exposed incomplete recursive strict projection: OpenAI rejected relationships.items because it lacked additionalProperties: false. The projector now recognizes every type: "object" node, including property-less objects in array items, and recursively enforces strict object schemas while preserving initial-reconstruction optionality/nullability behavior.
• The latest Terra request then exposed an inconsistent root properties/required contract. The projector now derives required after projection from the surviving property keys, and recursive tests verify properties, required, and additionalProperties consistency. Property-less object strictness and initial-reconstruction transport behavior remain preserved.
• Current deterministic final-fetch schema remains internally valid, yet the live rejection contradicts it. CONFIDENCE_ENGINE_EXPERIMENT_TRACE_OPENAI_SCHEMA=1 now emits one safe, server-side structural summary immediately before the OpenAI fetch—no prompt, answer, request body, secret, or model output.
• The focused-deconstruction route now emits complementary safe server diagnostics for start, provider success/failure, focused validation, and end status; the OpenAI schema trace remains provider-owned. No user content or secrets are logged.
• Canonical focused relationship items are now strict { from, to, type }: all required non-empty strings, free-text type, and no rationale; relationships remains required and may be []. Schema, prompt field names, and validator align; the stale rationale-bearing test fixture was corrected.
• Live Terra focused deconstruction now passes through the real UI, but the following Current Understanding synthesis exposed a separate caller/schema mismatch: its prompt and validator require { currentUnderstanding } while the provider received the default initial-reconstruction schema.
• Synthesis now supplies its own output schema as provider argument three; provider implementation is unchanged. Synthesis and provider deterministic suites pass with zero live calls. Next boundary: one isolated live synthesis request using the already captured situationGraph and findings payload.

Repository checkpoint

• Branch: feature/initial-decomposition-v0.61
• HEAD: 5878ce4 — experiment(confidence-engine): add reconstruction-only helper flag
• Working tree: clean after this session's commit

Initial reconstruction — current status

Semantically stable enough for current MVP stage. Exact graph topology is not stable and is not treated as an invariant. Trust-critical meaning must remain stable. Some compression is acceptable when meaning survives downstream. Missing meaning cannot be faithfully recovered downstream. Causal hypotheses must remain visibly provisional.

Current production default: reconstruct-v0.5 prompt + canonical reconstruction schema + Zod validation via z.toJSONSchema().

The /api/cases/start route returns validated initial reconstruction, situation graph, and selected question. Observability seam exposes the exact object used by buildInitialGraph() for comparison.

Frozen: initial decomposition, prompt refinement, Qwen/Terra comparison — see CURRENT MVP DIRECTION above.

Focused investigation — current status

Focused deconstruction plumbing fixes are complete:

• Schema mismatch resolved (focused route now supplies its own focusedDeconstructJsonSchema)
• Provider envelope no longer leaks into validator (inner .response unwrapped correctly)
• All 48 focused-investigation-boundary tests pass on first run

Focused deconstruction receives only:

• centralStatement
• targetLabel
• targetDescription
• question
• answer

Full SituationGraph / original scenario / previous findings are not supplied to that route. This is intentional epistemic separation.

Repeatability: supplier/weekend-shift epistemic separation repeated 3/3 on the fixed case after plumbing fix. Previous pre-fix semantic runs remain invalid (contaminated by provider-envelope misuse + wrong transport schema).

Canonical experiment apparatus — currently valid

scripts/start-case-experiment-helper.cjs    — canonical helper (tsx runtime)
tsx runtime                                  — alias-capable experiment execution
--file                                       — file-input mode (JSON fixture)
--reconstruction-only                        — stop after initial reconstruction, no question generation
import-only proof                            — prove startCase() import chain resolves without live calls
OpenAI reconstruction experiment seam        — START_CASE_EXPERIMENT_PROVIDER=openai (experiment-only)
focused-deconstruction schema fix            — plumbing verified, 48/48 tests pass

No other experimental apparatus is currently valid. Obsolete setups have been removed or superseded.

Current product architecture

Three distinct routes:

/                              → Portfolio (notebook index)
/investigations/{id}           → Investigation (working case/pages)
/investigations/{id}/report    → Investigation Report (derived summary)

Portfolio: investigation collection with actions per card (View report, Continue investigation, Restart). "+ Create new investigation" allocates durable ID via crypto.randomUUID() + navigates.

Investigation: ScenarioForm + ReasoningWorkspace. Handles focused turns, Done/Re-open semantics, Current Understanding synthesis.

Report: rendered persisted investigationReport snapshot. On-demand generation (one /api/cases/overview on first visit; zero on subsequent). Not canonical reasoning evidence — derived artefact.

Evidence discipline invariants

RAW USER EVIDENCE
≠ MODEL-DERIVED CONTRIBUTION SEMANTICS
≠ CURRENT CANONICAL FINDING
≠ IMMUTABLE SOURCE OBSERVATION
≠ USER DISPOSITION / AUTHORITY
≠ TURN CONTEXT / PROVENANCE

Finding dispositions: null (eligible working premise), agree (user-endorsed), not_relevant (excluded from eligible reasoning, provenance retained). User disposition never directly mutates authoritative graph state.

Persistence

• Owner: lib/storage/providers/local-storage.js (saveInvestigation / loadInvestigation)
• Key prefix: confidence-engine-investigation:<durable-id>
• Storage contract: lib/storage/investigation-storage.js (application-facing boundary)
• Identity: durable id allocated by application, not storage
• First persistence: when user produces meaningful state (scenario submitted), not on create-click
• Restart: preserves container/id/scenario; clears reasoning/report state

MVP boundaries

Implemented:

• Multi-investigation via localStorage with durable IDs
• Report freshness (Current / Update available) derived from revision comparison
• Manual Update report action
• Confirmation-gated restart
• No automatic regeneration
• Duplicate prevention guard

Deferred beyond MVP:

• Search, tag, archive, group within Portfolio
• Export/copy Reports to external document
• Report history / comparison
• Multi-provider production routing (OpenAI/Terra comparison is frozen)

Current development constraints

• Canonical dev server at http://localhost:3000. Never start/stop/restart/probe it. If unavailable → BLOCKED and stop.
• Playwright MCP: use Run Playwright code with semantic locators. Snapshot refs are observational only.
• For async/hydration states: use waitFor({ state: 'visible', timeout }) — not arbitrary sleeps.
• If a prescribed semantic locator cannot find its control → STOP. No fallback to CSS/XPath/DOM traversal.
• Live freeze: once Playwright verification begins, no production file edits until evidence is captured.
• Tests are instruments, not product truth. At first deterministic failure: classify PRODUCT vs APPARATUS, then stop.

Current genuinely open boundaries

Not yet implemented:

• Multi-investigation portfolio (search/tag/archive/group)
• Export/copy of Reports to external document
• Production multi-provider routing (Ollama/Qwen default remains)

Known limitations:

• Current Understanding and Investigation Report are architecturally distinct artefacts. Report interpretations remain explicit, not evidence.
• The ≤5 processing bound was experimental apparatus constraint, NOT product requirement.

HISTORICAL — initial-decomposition v0.61 status

Initial decomposition experiments (v0.61) ran four successful runs plus one validation failure against the same manufacturing scenario. Key findings:

• Consistently preserved across all runs: normalisation uncertainty, CRM comparability, supplier/shift ambiguity
• Variable: late-delivery vs defect distinction (1 PRESENT, 3 PARTIAL)
• Rarely surfaced: intervention-fit uncertainty (1 PRESENT, 3 ABSENT)
• Run 2 introduced speculative subdivisions not grounded in source text
• No steering language, unsupported causality, or action recommendations in any run
• Graph topology varies materially (14–21 nodes) across runs — this is expected, not a defect

Qwen/Terra matched evidence showed Terra with stronger stability on C and D1/D2 distinctions; Qwen preserved meaning but compressed more frequently. Both providers vary graph topology while preserving core dependencies.

These results are documented as historical experiment evidence. The v0.61 line is frozen — see CURRENT MVP DIRECTION above.

Provenance pointers

Need	Read
Product evolution v0.51–v0.60	docs/design-evolution/README.md (progressive loading)
Methodology / RTO axioms	docs/current-working-principles.md §0 (A1–A12)
Architecture guardrails	.claude/architecture-guardrails.md
Task routing by work type	docs/task-context-packs.md
Full experiment history (specific)	docs/design-evolution/README.md → relevant chapter

Consult docs/current-project-state.md for broader project context and passive classifier status.

Invalid semantic evidence — preserved as documented

Previous focused-deconstruction semantic runs before the plumbing fix remain invalid as semantic evidence. They produced all-zero semantic fields because:

1. Provider envelope buried semantic fields under wrapper (not .response)
2. Initial-reconstruction schema was supplied to focused-deconstruction call instead of its own contract

These are recorded as known contamination in the v0.61 archive chapter (docs/design-evolution/ch19/initial-decomposition-v0.61.md). The plumbing fix is complete and verified (48/48 tests).