confidence-engine/docs/current-project-state.md

# Current Project State — Confidence Engine

## v0.62d Production Docker Packaging

- `Dockerfile` created: multi-stage Alpine build (Node 22), Next.js standalone output, configurable port 3000, health-check boundary via `/api/health`
- `.dockerignore` created: excludes `node_modules`, `.next`, secrets (`env.*.local`), docs, IDE, OS artefacts
- `next.config.mjs`: added `output: 'standalone'` for lean production container (only config change required)
- `.env.example`: reorganized; Ollama variables now tracked as deployment-relevant env vars
- npm production build: PASS ✓
- Docker image build: blocked — no Docker runtime on development machine (apparatus limitation, not product defect)
- Production container start / `/api/health` smoke test: pending Docker availability
- No persistent application volume required
- Supabase remains external; Ollama remains private and server-reachable
- Public `NEXT_PUBLIC_*` variables may require build-time injection via `--build-arg` (baked into browser bundle)
- **No deployment performed**

> Created by Experiment 27. This document is the starting point for any fresh session working on the Confidence Engine. Read this first, then follow the routing table below to task-specific references.

## 1. What the Confidence Engine Is

The Confidence Engine helps people decide whether they have enough justified confidence to act on a complicated problem — one step at a time.

It does not simply answer the user's question. It:

- Reconstructs the situation;
- Separates observations, assumptions, relationships and unknowns;
- Builds a structured reasoning graph;
- Selects the most useful unresolved uncertainty;
- Asks one simple question;
- Evaluates whether remaining uncertainty justifies action;
- Repeats until action is justified or the remaining uncertainty is clear.

The user may already know the answer but needs confidence to act, may need to identify who to ask, may need to find where to look, or may need to determine how to test a claim. The engine carries the complexity of reasoning so the user does not have to manage graph theory, node IDs, internal enums, schemas, prompt versions or provider details.

## 2. Current Product Experience

The product direction is a **facilitated investigation** presented across three distinct routes:

```
/                              → Portfolio (investigator notebook index)
/investigations/{id}           → Investigation (working case / pages)
/investigations/{id}/report   → Investigation Report (readable derived summary)
```

**Portfolio:** Shows the persisted investigation collection. Actions on each card: *View report*, *Continue investigation*, *Restart investigation*. Below the cards: *+ Create new investigation* (allocates durable ID via `crypto.randomUUID()` + navigates to `/investigations/{id}`). Restart is confirmation-gated and preserves container while clearing reasoning/Report state.

**Investigation:** Contains `ScenarioForm` + `ReasoningWorkspace`. Handles graph reasoning, focused investigation turns, Done/Re-open semantics, Current Understanding synthesis. Report presentation is NOT part of this route — owned by the dedicated Report page.

**Report:** Renders persisted `investigationReport` snapshot. Generation is on-demand (exactly one `/api/cases/overview` call on first visit; zero on subsequent visits). The Report is a derived artefact, not canonical reasoning evidence.

**Authentication boundary:** Supabase Auth magic links gate product and CE API routes. Sessions are cookie-backed and `/auth/callback` exchanges the auth code before returning to `/`. Server-authoritative investigation persistence via authenticated browser HTTP provider; localStorage is legacy only.

**Database contract (v0.62c):** The applied `confidence_engine.investigations` schema sits outside `public`. Its platform metadata is `id`, `user_id`, and timestamps; the CE payload remains an opaque JSONB `snapshot`. Authenticated RLS ownership is `user_id = auth.uid()`, and external PostgREST configuration exposes the schema. Server persistence is now the production authority; localStorage is legacy only. No dual-write. No automatic legacy import.

The user controls which question to investigate, how deeply to investigate it, when to say Done for now, whether Current Understanding is sufficient, whether to reopen work, and when to review the Report. The engine facilitates — it does not steer or prioritise.

## September 8, 2026 Product Checkpoint

`docs/confidence-engine-product-checkpoint-2026-09-08.md` is the current checkpoint for the established working loop, multi-turn supplier evidence, Done-for-now/Re-open graph-state behavior, persistence, provider position, measured Terra economics, and the move toward commercially testing realistic end-to-end use. It distinguishes proven observations from provisional cost extrapolation and known future questions.

For the current stage, the primary question is increasingly whether this process leaves real people materially clearer about difficult situations, repeatedly enough that they will pay to use it. This does not weaken the invariant that the user owns investigation choice, depth, closure, reopening, sufficiency, confidence, and action; nor does it excuse trust-critical defects.

## 3. Current Engine Capabilities

### Active capabilities

These are what currently affect the working engine:

- Deterministic reasoning pipeline from scenario reconstruction through graph update, propagation and confidence/completeness calculation;
- Unknown selection using atomicity and answerability checks;
- **Question formulation** — remains an available capability (graph-backed question generation for selected nodes);
- Scenario API (analyseScenario / updateCase);
- Investigation turn cycle orchestration;
- **Reasoning-fidelity v0.8 (completed, frozen for current MVP):** user-supported meaning cannot silently outrun the raw answer at the mutation boundary; evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation. A–F regression boundaries closed for this pass. See `docs/current-handoff.md` for closeout details.

> **NOTE on investigation ownership:** The user currently owns which unresolved
> investigation/question to pursue. Selector-led compulsory next-question
> optimisation is paused under Return-to-Origin. Semantic-selector replacement
> is also paused. Question formulation remains available as a capability but its
> output is not automatically enforced as the user's required next step.

### Route architecture (UX/product lineage v0.51–v0.60)

Three distinct routes, each with clear ownership:

| Route | Owner | Presentation |
|---|---|---|
| `Portfolio` (`/`) | Portfolio page + storage | List of Investigation summaries; no Report presentation |
| `Investigation` (`/investigations/{id}`) | `ScenarioForm` + `ReasoningWorkspace` | Focused investigation turn cycle |
| `Report` (`/investigations/{id}/report`) | Report page (standalone) | Persisted derived artefact; on-demand generation |

**Key invariants:** ReasoningWorkspace no longer owns Report presentation. The Report is a distinct route/page, not an internal state of the Investigation.

### Persistence and report lifecycle

- **Server-authoritative:** Supabase `confidence_engine.investigations` via authenticated browser HTTP provider (`lib/storage/providers/server-http.js`). localStorage is legacy only — invisible to normal product flow. No dual-write. No automatic legacy import.
- `saveInvestigation()` / `loadInvestigation()` are the canonical storage seams, backed by server-HTTP provider with coalescing autosave in `lib/storage/investigation-storage.js`.
- `listInvestigations()` returns lightweight summaries for Portfolio rendering from the server API.
- Report generation: first visit → one synthesis call + persist; subsequent visits → zero calls, renders persisted snapshot.
- Restart is destructive and confirmation-gated (dialog → explicit second confirmation); uses shared transformation in `lib/storage/restart-investigation.js`.

### Reasoning-engine vs UX/product version lineage

The Confidence Engine tracks two independent version lineages:
- **Reasoning-engine experimental lineage** (v0.8+): reasoning-fidelity, investigation-state assessment, semantic selectors — under RTO pause.
- **UX/product development lineage** (v0.55+): Portfolio / Investigation / Report route separation, persisted report lifecycle, confirmation-gated restart, focused-presentation ownership, empty Done semantics. Do not conflate these lineages as describing one product version.

### Passive experimental capabilities

The following were built during Experiments 18–25B. They are isolated diagnostic layers with no active integration into the user-facing investigation:

- Investigation-state assessment (phase and progress classification);
- Behaviour selection from assessed state — passively evaluated in Experiments 39–41; all five behaviours reachable but Acknowledge dominates (71% on real data); Exp 41 recommends Variant B (Acknowledge exclusions via phase/progress/health gates) as the cleaner approach;
- Decision condition status evaluation;
- Question-to-condition relevance scoring;
- Evidence direction classification (support, contradict, inform);
- Evidence scope detection (direct_match, different_timeframe, subject_mismatch, partial_match, cannot_determine);
- Scope-aware condition status using phrase matching.

**These passive classifiers do not yet control the user-facing investigation.** They record signals for future use when integrated into the active reasoning path.

## 4. What Experiments 20–25B Established

- A decision's importance requires a destination — you cannot assess whether something matters without knowing what you are deciding between.
- Decision conditions explain what would make a decision justified; they are not the same as unresolved unknowns.
- Resolving a question does not automatically establish the condition that question might inform — there is a distinct gap between answering and establishing.
- Evidence can support, contradict or merely inform a condition depending on subject, timeframe and claim type alignment.
- Direction alone (support/contradict/inform) is insufficient without checking whether evidence and condition share subject, claim type and timeframe.
- Present-state evidence does not automatically settle future-feasibility conditions; scope detection must check both inputs independently.
- Keyword and phrase matching remains provisional experimental scaffolding — it is narrow, targeted and replaceable, not a finished language-understanding system.

## 5. What Remains Unresolved

### Not yet implemented (product capabilities)

Multi-investigation storage and portfolio are structurally complete in v0.60. Deferred feature work beyond MVP scope:
- Search, tag, archive, group behaviour within Portfolio
- Export/copy of Reports to Jira or external document

Multi-investigation identity is durable (`uuidv4`). Legacy singleton compatibility remains internally (unused by current product). Report freshness / versioning after investigation changes uses `generatedFromRevision` vs `investigationRevision`.

### Methodological unresolved

- How free language will be interpreted reliably without keyword scaffolding;
- Whether structured LLM interpretation should eventually replace current phrase-based detection;
- Whether passive classifiers generalise across domains or remain fixture-specific;
- How and when passive reasoning signals should enter the active turn cycle;
- Whether current architectural documents (v0.6-reasoning-architecture.md, etc.) still accurately match implementation after experiments 15–25B.

## 6. Work Currently Paused

- Engine experiments advanced through Experiment 43. Initial decomposition hardening (v0.61) completed and frozen for current MVP stage — see `docs/current-handoff.md` §CURRENT MVP DIRECTION.
- UI experiments are paused;
- Knowledge-management experiments are complete (confirmed by Experiment 38 cold-start validation);
- Nothing historical has been deleted or archived yet.

## 7. Context Loading Guide

| When you need | Read this |
|---|---|
| Returning after a break | **First:** `docs/current-handoff.md` (current state + methodology anchor), **then** `docs/Confidence_Engine_Return_to_Origin_Methodology_Context_2026-08-18.md` (methodology continuity) |
| Where we are now | `docs/current-project-state.md` (this file) |
| Current principles and reasoning guidance | `docs/current-working-principles.md` |
| What to keep from code changes during UX work | `.claude/architecture-guardrails.md` |
| Product direction and stage | `.claude/project-context.md` |
| Task-specific or historical references | `docs/project-knowledge-inventory.md` |
| Broader architectural intent | `docs/architectural-principles.md` (task-specific only) |
| Task-specific routing by work type | `docs/task-context-packs.md` (four minimal packs + common rules) |
| Historical evidence or a named experiment | `docs/design-evolution/README.md` → relevant chapter |

Do not read the full design-evolution log unless a specific experiment is required. Use the inventory to locate task-specific context, then load only what you need.

Historical documents are retained under `docs/archive/` and should be opened only when a named past decision, release or experiment requires them.

## 8. Return-to-Work Summary

Engine experiments advanced through Experiment 43 (Clarify readiness). UX/product development reached v0.60 (multi-investigation structurally complete) + v0.61 (initial decomposition hardening, frozen for MVP). Reasoning-fidelity v0.8 closed.

Key current state:
- Multi-investigation architecture structurally complete (v0.60): durable IDs, listInvestigations, restart container-preserve, report freshness
- Initial decomposition frozen (v0.61): semantically stable enough for MVP; exact topology not invariant; see CURRENT MVP DIRECTION in handoff
- Focused deconstruction plumbing fixed and verified (48/48 tests)
- Passive classifiers operational but isolated

First document to read: **`docs/current-handoff.md`** (methodology continuity + current state), then `docs/current-project-state.md`, then `docs/project-knowledge-inventory.md`. Consult `.claude/architecture-guardrails.md` before any code changes. The full experiment history remains available in `docs/design-evolution/README.md` but is no longer default reading — load only when a specific question requires it.

## Verification Marker

Implementation status last checked against source: Experiment 43 + v0.61 apparatus (tsx helper, reconstruction-only seam, focused-deconstruction schema fix). Multi-investigation architecture verified at v0.60g2+ and structurally complete. The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required.

## 10. Post-v0.8 Methodology Learning

### Durable methodology principles (2026-08-19)

The following principles have been durably codified in `docs/current-working-principles.md` as **Axiomatic Principles A1–A12**. They are now the authoritative methodological baseline for all future work:

| Classification | Principle |
|---|---|
| METHODOLOGY PRINCIPLE | A1 — Delivery-platform independence (methodology ≠ software) |
| METHODOLOGY PRINCIPLE | A2 — The Engine facilitates; the user directs investigation |
| METHODOLOGY PRINCIPLE | A3 — Meaning over prescribed vocabulary; no domain/language dictionaries |
| METHODOLOGY PRINCIPLE | A4 — Semantic interpretation as a facilitation capability (LLM optional) |
| METHODOLOGY PRINCIPLE | A5 — Deterministic code for structure/integrity only, not semantic judgment |
| METHODOLOGY PRINCIPLE | A6 — Investigation is non-linear; user moves between branches |
| METHODOLOGY PRINCIPLE | A7 — Background assistance must not redirect the user's current work |
| METHODOLOGY PRINCIPLE | A8 — Closure is user-owned and may be provisional (no mandatory thresholds) |
| METHODOLOGY PRINCIPLE | A9 — Graph-like reasoning structure does not mandate graph UI |
| METHODOLOGY PRINCIPLE | A10 — Progressive disclosure mirrors progressive reasoning |
| METHODOLOGY PRINCIPLE | A11 — Multilingual portability follows from meaning-based (not vocabulary-based) design |
| METHODOLOGY PRINCIPLE | A12 — Latency is an implementation trade-off, not a methodological constraint |
| IMPLEMENTATION HYPOTHESIS | Hybrid deterministic/semantic fast path (not selected; guardrail applies) |

> **New work must not contradict these axioms without explicit methodological review.**
> See `docs/current-working-principles.md` §0 for full wording and classification discipline.

### Carry-forward from Phase 1B audit evidence

The following material learnings are carried forward as durable context for safe later archival of experiment families:

- **Proposition phrasing materially influences deterministic scoring** (57-series / 60B.100):
  Even when underlying meaning is equivalent, the specific wording or lexical form of a
  proposition can change deterministic scoring outcomes. Phrase/keyword-driven semantic
  priority is therefore brittle and must not be treated as paraphrase-invariant.

- **Decision-sufficiency reasoning is passive capability**:
  Decision-sufficiency reasoning exists as a passive/experimental capability rather than
  current user-facing routing authority. (Already documented in §3 Passive experimental capabilities.)

- **Ownership survives rejection**:
  User-selected/active investigation ownership must survive substantive ties and
  question-formulation rejection. (Already documented in `docs/current-handoff.md`.)

### RTO learning from Experiments 14–17

Since the handoff document was written, further learning has emerged from Return-to-Origin work (RTO.14–17):

**What proved useful:** An LLM can retain prior focused knowledge, revise uncertainty, and carry coherent reasoning across turns inside a single investigation.

**What was exposed:** Pushing that approach further reproduced the original cumulative-state reconstruction pattern at a lower scope (`focused-investigation state + new answer → LLM rewrites focused state`). This is the same structural assumption we challenged at the global-graph level — just compressed.

**Current working hypothesis (not yet architecture):** The natural reasoning unit is one question → one answer → one interpretation/capture. Each accepted answer produces a small evidence-bearing reasoning fragment, remembered outside the LLM call. The larger investigation understanding emerges from composing those fragments over time. Only directly relevant prior knowledge needs to be supplied when qualifying a specific earlier fragment.

**Next experiment direction:** Test independent user-chosen question/answer deconstruction without accumulated focused state being passed between LLM calls. See `docs/Confidence_Engine_Return_to_Origin_Methodology_Context_2026-08-18.md` for full context.

> **Branch:** `feature/decision-closure-ownership-v0.47`
> **Current HEAD:** `85204f96aca7e4b350fe8db73150bf0683ebeefe` (experiment: checkpoint borderline relationship control)

### Current experimental context

The durable methodology principles (A1–A12 above) should be loaded before continuing experiments. For detailed current operational state, read `docs/current-handoff.md` first.