confidence-engine/docs/current-project-state.md

Current Project State — Confidence Engine

v0.62d Production Docker Packaging — LIVE PROVEN

• Dockerfile created: multi-stage Alpine build (Node 22), Next.js standalone output, configurable port 3000, health-check boundary via /api/health
• .dockerignore created: excludes node_modules, .next, secrets (env.*.local), docs, IDE, OS artefacts
• next.config.mjs: added output: 'standalone' for lean production container (only config change required)
• .env.example: reorganized; Ollama variables now tracked as deployment-relevant env vars
• npm production build: PASS ✓
• Docker image build on CT 112: PROVEN — builds successfully, standalone container starts successfully.
• Production container /api/health: PROVEN — {"healthy":true}, no private Ollama model name / raw internal error exposed.
• No persistent application volume required.
• Supabase remains external; Ollama remains private and server-reachable from deployment host.
• Public NEXT_PUBLIC_* variables may require build-time injection via --build-arg (baked into browser bundle).
• Deployment: LIVE PROVEN — manual Jenkins pipeline succeeded end-to-end.

External Deployment (LIVE)

Confidence Engine is externally reachable at https://confidence.rdbcloud.co.uk.

Topology: Internet → HTTPS → Nginx Proxy Manager → CT 112 / Confidence Engine Docker container.

CT 112: hostname confidence-engine, LAN 192.168.68.73, repo /opt/confidence-engine, container confidence-engine, port 3000.

External Magic-Link Login (LIVE)

Magic-link authentication proven through deployed instance. Callback-origin defect corrected; login returns to external portfolio rather than 0.0.0.0:3000. Email branding configured on external Supabase (sender name, subject, branded HTML body). SPF/DKIM/DMARC confirmed recipient-side.

Multi-user Boundary (LIVE)

Application-level isolation proven on deployed instance — User A does not see User B investigations and vice versa.

Created by Experiment 27. This document is the starting point for any fresh session working on the Confidence Engine. Read this first, then follow the routing table below to task-specific references.

1. What the Confidence Engine Is

The Confidence Engine helps people decide whether they have enough justified confidence to act on a complicated problem — one step at a time.

It does not simply answer the user's question. It:

• Reconstructs the situation;
• Separates observations, assumptions, relationships and unknowns;
• Builds a structured reasoning graph;
• Selects the most useful unresolved uncertainty;
• Asks one simple question;
• Evaluates whether remaining uncertainty justifies action;
• Repeats until action is justified or the remaining uncertainty is clear.

The user may already know the answer but needs confidence to act, may need to identify who to ask, may need to find where to look, or may need to determine how to test a claim. The engine carries the complexity of reasoning so the user does not have to manage graph theory, node IDs, internal enums, schemas, prompt versions or provider details.

2. Current Product Experience

The product direction is a facilitated investigation presented across three distinct routes:

/                              → Portfolio (investigator notebook index)
/investigations/{id}           → Investigation (working case / pages)
/investigations/{id}/report   → Investigation Report (readable derived summary)

Portfolio: Shows the persisted investigation collection. Actions on each card: View report, Continue investigation, Restart investigation. Below the cards: + Create new investigation (allocates durable ID via crypto.randomUUID() + navigates to /investigations/{id}). Restart is confirmation-gated and preserves container while clearing reasoning/Report state.

Investigation: Contains ScenarioForm + ReasoningWorkspace. Handles graph reasoning, focused investigation turns, Done/Re-open semantics, Current Understanding synthesis. Report presentation is NOT part of this route — owned by the dedicated Report page.

Report: Renders persisted investigationReport snapshot. Generation is on-demand (exactly one /api/cases/overview call on first visit; zero on subsequent visits). The Report is a derived artefact, not canonical reasoning evidence.

Authentication boundary: Supabase Auth magic links gate product and CE API routes. Sessions are cookie-backed and /auth/callback exchanges the auth code before returning to /. Server-authoritative investigation persistence via authenticated browser HTTP provider; localStorage is legacy only.

Database contract (v0.62c): The applied confidence_engine.investigations schema sits outside public. Its platform metadata is id, user_id, and timestamps; the CE payload remains an opaque JSONB snapshot. Authenticated RLS ownership is user_id = auth.uid(), and external PostgREST configuration exposes the schema. Server persistence is now the production authority; localStorage is legacy only. No dual-write. No automatic legacy import.

The user controls which question to investigate, how deeply to investigate it, when to say Done for now, whether Current Understanding is sufficient, whether to reopen work, and when to review the Report. The engine facilitates — it does not steer or prioritise.

September 8, 2026 Product Checkpoint

docs/confidence-engine-product-checkpoint-2026-09-08.md is the current checkpoint for the established working loop, multi-turn supplier evidence, Done-for-now/Re-open graph-state behavior, persistence, provider position, measured Terra economics, and the move toward commercially testing realistic end-to-end use. It distinguishes proven observations from provisional cost extrapolation and known future questions.

For the current stage, the primary question is increasingly whether this process leaves real people materially clearer about difficult situations, repeatedly enough that they will pay to use it. This does not weaken the invariant that the user owns investigation choice, depth, closure, reopening, sufficiency, confidence, and action; nor does it excuse trust-critical defects.

3. Current Engine Capabilities

Active capabilities

These are what currently affect the working engine:

• Deterministic reasoning pipeline from scenario reconstruction through graph update, propagation and confidence/completeness calculation;
• Unknown selection using atomicity and answerability checks;
• Question formulation — remains an available capability (graph-backed question generation for selected nodes);
• Scenario API (analyseScenario / updateCase);
• Investigation turn cycle orchestration;
• Reasoning-fidelity v0.8 (completed, frozen for current MVP): user-supported meaning cannot silently outrun the raw answer at the mutation boundary; evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation. A–F regression boundaries closed for this pass. See docs/current-handoff.md for closeout details.

NOTE on investigation ownership: The user currently owns which unresolved investigation/question to pursue. Selector-led compulsory next-question optimisation is paused under Return-to-Origin. Semantic-selector replacement is also paused. Question formulation remains available as a capability but its output is not automatically enforced as the user's required next step.

Route architecture (UX/product lineage v0.51–v0.60)

Three distinct routes, each with clear ownership:

Route	Owner	Presentation
Portfolio (/)	Portfolio page + storage	List of Investigation summaries; no Report presentation
Investigation (/investigations/{id})	ScenarioForm + ReasoningWorkspace	Focused investigation turn cycle
Report (/investigations/{id}/report)	Report page (standalone)	Persisted derived artefact; on-demand generation

Key invariants: ReasoningWorkspace no longer owns Report presentation. The Report is a distinct route/page, not an internal state of the Investigation.

Persistence and report lifecycle

• Server-authoritative: Supabase confidence_engine.investigations via authenticated browser HTTP provider (lib/storage/providers/server-http.js). localStorage is legacy only — invisible to normal product flow. No dual-write. No automatic legacy import.
• saveInvestigation() / loadInvestigation() are the canonical storage seams, backed by server-HTTP provider with coalescing autosave in lib/storage/investigation-storage.js.
• listInvestigations() returns lightweight summaries for Portfolio rendering from the server API.
• Report generation: first visit → one synthesis call + persist; subsequent visits → zero calls, renders persisted snapshot.
• Restart is destructive and confirmation-gated (dialog → explicit second confirmation); uses shared transformation in lib/storage/restart-investigation.js.

Reasoning-engine vs UX/product version lineage

The Confidence Engine tracks two independent version lineages:

• Reasoning-engine experimental lineage (v0.8+): reasoning-fidelity, investigation-state assessment, semantic selectors — under RTO pause.
• UX/product development lineage (v0.55+): Portfolio / Investigation / Report route separation, persisted report lifecycle, confirmation-gated restart, focused-presentation ownership, empty Done semantics. Do not conflate these lineages as describing one product version.

Passive experimental capabilities

The following were built during Experiments 18–25B. They are isolated diagnostic layers with no active integration into the user-facing investigation:

• Investigation-state assessment (phase and progress classification);
• Behaviour selection from assessed state — passively evaluated in Experiments 39–41; all five behaviours reachable but Acknowledge dominates (71% on real data); Exp 41 recommends Variant B (Acknowledge exclusions via phase/progress/health gates) as the cleaner approach;
• Decision condition status evaluation;
• Question-to-condition relevance scoring;
• Evidence direction classification (support, contradict, inform);
• Evidence scope detection (direct_match, different_timeframe, subject_mismatch, partial_match, cannot_determine);
• Scope-aware condition status using phrase matching.

These passive classifiers do not yet control the user-facing investigation. They record signals for future use when integrated into the active reasoning path.

4. What Experiments 20–25B Established

• A decision's importance requires a destination — you cannot assess whether something matters without knowing what you are deciding between.
• Decision conditions explain what would make a decision justified; they are not the same as unresolved unknowns.
• Resolving a question does not automatically establish the condition that question might inform — there is a distinct gap between answering and establishing.
• Evidence can support, contradict or merely inform a condition depending on subject, timeframe and claim type alignment.
• Direction alone (support/contradict/inform) is insufficient without checking whether evidence and condition share subject, claim type and timeframe.
• Present-state evidence does not automatically settle future-feasibility conditions; scope detection must check both inputs independently.
• Keyword and phrase matching remains provisional experimental scaffolding — it is narrow, targeted and replaceable, not a finished language-understanding system.

5. What Remains Unresolved

Not yet implemented (product capabilities)

Multi-investigation storage and portfolio are structurally complete in v0.60. Deferred feature work beyond MVP scope:

• Search, tag, archive, group behaviour within Portfolio
• Export/copy of Reports to Jira or external document

Multi-investigation identity is durable (uuidv4). Legacy singleton compatibility remains internally (unused by current product). Report freshness / versioning after investigation changes uses generatedFromRevision vs investigationRevision.

Methodological unresolved

• How free language will be interpreted reliably without keyword scaffolding;
• Whether structured LLM interpretation should eventually replace current phrase-based detection;
• Whether passive classifiers generalise across domains or remain fixture-specific;
• How and when passive reasoning signals should enter the active turn cycle;
• Whether current architectural documents (v0.6-reasoning-architecture.md, etc.) still accurately match implementation after experiments 15–25B.

6. Work Currently Paused

• Engine experiments advanced through Experiment 43. Initial decomposition hardening (v0.61) completed and frozen for current MVP stage — see docs/current-handoff.md §CURRENT MVP DIRECTION.
• UI experiments are paused;
• Knowledge-management experiments are complete (confirmed by Experiment 38 cold-start validation);
• Nothing historical has been deleted or archived yet.

7. Context Loading Guide

When you need	Read this
Returning after a break	First: docs/current-handoff.md (current state + methodology anchor), then docs/Confidence_Engine_Return_to_Origin_Methodology_Context_2026-08-18.md (methodology continuity)
Where we are now	docs/current-project-state.md (this file)
Current principles and reasoning guidance	docs/current-working-principles.md
What to keep from code changes during UX work	.claude/architecture-guardrails.md
Product direction and stage	.claude/project-context.md
Task-specific or historical references	docs/project-knowledge-inventory.md
Broader architectural intent	docs/architectural-principles.md (task-specific only)
Task-specific routing by work type	docs/task-context-packs.md (four minimal packs + common rules)
Historical evidence or a named experiment	docs/design-evolution/README.md → relevant chapter

Do not read the full design-evolution log unless a specific experiment is required. Use the inventory to locate task-specific context, then load only what you need.

Historical documents are retained under docs/archive/ and should be opened only when a named past decision, release or experiment requires them.

8. Return-to-Work Summary

Engine experiments advanced through Experiment 43 (Clarify readiness). UX/product development reached v0.60 (multi-investigation structurally complete) + v0.61 (initial decomposition hardening, frozen for MVP). Reasoning-fidelity v0.8 closed.

Key current state:

• Multi-investigation architecture structurally complete (v0.60): durable IDs, listInvestigations, restart container-preserve, report freshness
• Initial decomposition frozen (v0.61): semantically stable enough for MVP; exact topology not invariant; see CURRENT MVP DIRECTION in handoff
• Focused deconstruction plumbing fixed and verified (48/48 tests)
• Passive classifiers operational but isolated

First document to read: docs/current-handoff.md (methodology continuity + current state), then docs/current-project-state.md, then docs/project-knowledge-inventory.md. Consult .claude/architecture-guardrails.md before any code changes. The full experiment history remains available in docs/design-evolution/README.md but is no longer default reading — load only when a specific question requires it.

Verification Marker

Implementation status last checked against source: Experiment 43 + v0.61 apparatus (tsx helper, reconstruction-only seam, focused-deconstruction schema fix). Multi-investigation architecture verified at v0.60g2+ and structurally complete. The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required.

Deployment Automation — LIVE PROVEN

A manual Jenkins deployment pipeline has been established, version-controlled, and LIVE PROVEN end-to-end.

• Jenkinsfile (root) — Declarative Pipeline with three stages: Resolve → Deploy → Verify/result.
• Parameter: GIT_REF (string) — user-supplied Git ref (branch, tag, or SHA). Blank value fails clearly.
• Resolution: Jenkins resolves the ref to an exact commit SHA via git ls-remote origin before deployment. The SHA is deployed immutably.
• Target host: CT 112 (confidence-engine, 192.168.68.73) at /opt/confidence-engine.
• Docker image: tagged confidence-engine:<sha>, built on CT 112, no registry required.
• Environment: production values in /opt/confidence-engine/deploy.env on CT 112 (not in Git). deploy.env.example provided as reference.
• Health check: polls http://127.0.0.1:3000/api/health; requires {"healthy":true} within 60s.
• Rollback: one-step rollback to the previous container image on failure (if available). IMPLEMENTED, NOT LIVE PROVEN.
• Jenkins job: configured and operational. First successful deployment recorded with explicit "DEPLOYMENT SUCCEEDED" output.
• Jenkins SCM branch used to load the Jenkinsfile is conceptually separate from the GIT_REF chosen for deployment.

Pipeline design notes

• Manual trigger only — no automatic webhook deploy.
• No Docker registry introduced; SHA-tagged images retained on CT 112 for rollback support.
• No product behaviour changed by deployment automation.

9. First-Time Unauthenticated Landing/Login Framing

• login page now explains product purpose and investigation flow to first-time visitors without prior explanation
• user judgement/choice explicitly preserved ("It doesn't try to make the decision for you")
• existing magic-link authentication unchanged
• no broader onboarding/tutorial system introduced
• verified via Playwright semantic locators (heading, text content, form controls) and responsive viewport

10. Post-v0.8 Methodology Learning

Durable methodology principles (2026-08-19)

The following principles have been durably codified in docs/current-working-principles.md as Axiomatic Principles A1–A12. They are now the authoritative methodological baseline for all future work:

Classification	Principle
METHODOLOGY PRINCIPLE	A1 — Delivery-platform independence (methodology ≠ software)
METHODOLOGY PRINCIPLE	A2 — The Engine facilitates; the user directs investigation
METHODOLOGY PRINCIPLE	A3 — Meaning over prescribed vocabulary; no domain/language dictionaries
METHODOLOGY PRINCIPLE	A4 — Semantic interpretation as a facilitation capability (LLM optional)
METHODOLOGY PRINCIPLE	A5 — Deterministic code for structure/integrity only, not semantic judgment
METHODOLOGY PRINCIPLE	A6 — Investigation is non-linear; user moves between branches
METHODOLOGY PRINCIPLE	A7 — Background assistance must not redirect the user's current work
METHODOLOGY PRINCIPLE	A8 — Closure is user-owned and may be provisional (no mandatory thresholds)
METHODOLOGY PRINCIPLE	A9 — Graph-like reasoning structure does not mandate graph UI
METHODOLOGY PRINCIPLE	A10 — Progressive disclosure mirrors progressive reasoning
METHODOLOGY PRINCIPLE	A11 — Multilingual portability follows from meaning-based (not vocabulary-based) design
METHODOLOGY PRINCIPLE	A12 — Latency is an implementation trade-off, not a methodological constraint
IMPLEMENTATION HYPOTHESIS	Hybrid deterministic/semantic fast path (not selected; guardrail applies)

New work must not contradict these axioms without explicit methodological review. See docs/current-working-principles.md §0 for full wording and classification discipline.

Carry-forward from Phase 1B audit evidence

The following material learnings are carried forward as durable context for safe later archival of experiment families:

• Proposition phrasing materially influences deterministic scoring (57-series / 60B.100): Even when underlying meaning is equivalent, the specific wording or lexical form of a proposition can change deterministic scoring outcomes. Phrase/keyword-driven semantic priority is therefore brittle and must not be treated as paraphrase-invariant.

• Decision-sufficiency reasoning is passive capability: Decision-sufficiency reasoning exists as a passive/experimental capability rather than current user-facing routing authority. (Already documented in §3 Passive experimental capabilities.)

• Ownership survives rejection: User-selected/active investigation ownership must survive substantive ties and question-formulation rejection. (Already documented in docs/current-handoff.md.)

RTO learning from Experiments 14–17

Since the handoff document was written, further learning has emerged from Return-to-Origin work (RTO.14–17):

What proved useful: An LLM can retain prior focused knowledge, revise uncertainty, and carry coherent reasoning across turns inside a single investigation.

What was exposed: Pushing that approach further reproduced the original cumulative-state reconstruction pattern at a lower scope (focused-investigation state + new answer → LLM rewrites focused state). This is the same structural assumption we challenged at the global-graph level — just compressed.

Current working hypothesis (not yet architecture): The natural reasoning unit is one question → one answer → one interpretation/capture. Each accepted answer produces a small evidence-bearing reasoning fragment, remembered outside the LLM call. The larger investigation understanding emerges from composing those fragments over time. Only directly relevant prior knowledge needs to be supplied when qualifying a specific earlier fragment.

Next experiment direction: Test independent user-chosen question/answer deconstruction without accumulated focused state being passed between LLM calls. See docs/Confidence_Engine_Return_to_Origin_Methodology_Context_2026-08-18.md for full context.

Branch: feature/decision-closure-ownership-v0.47 Current HEAD: 85204f96aca7e4b350fe8db73150bf0683ebeefe (experiment: checkpoint borderline relationship control)

Current experimental context

The durable methodology principles (A1–A12 above) should be loaded before continuing experiments. For detailed current operational state, read docs/current-handoff.md first.