Confidence Engine

An experimental prototype that tests whether an LLM can build and maintain an evidence-based reconstruction of a situation over multiple turns.

Purpose

This is Milestone 1 — a technical vertical slice only. It demonstrates:

• Sending a scenario to a local Ollama model via a Next.js server route
• Receiving structured JSON output
• Validating the result with Zod schemas
• Displaying the reconstruction and diagnostic information in a plain UI

Prerequisites

• Node.js 18+ (LTS recommended)
• npm (or equivalent package manager)
• Ollama installed and running on your local network, with a model pulled (e.g., ollama pull llama3)

Installation

cd confidence-engine
npm install
cp .env.example .env.local

Edit .env.local and set:

• OLLAMA_BASE_URL — your Ollama server address (e.g., http://192.168.1.100:11434)
• OLLAMA_MODEL — the model name (e.g., llama3)

Development Commands

npm run dev        # Start development server on localhost:3000
npm run build      # Production build
npm run start      # Run production server
npm run lint       # ESLint check

Testing Commands

npm test           # Run all tests (one-shot)
npm run test:watch # Run tests in watch mode

Tests mock the Ollama network request. No real Ollama server is needed to run them.

Verifying Ollama Connectivity

1. Start the dev server: npm run dev
2. Open http://localhost:3000/api/health
3. You should see JSON with "reachable": true and your model name

Current Limitations

• Single-turn only — no conversation memory or multi-turn dialogue
• No persistence — results are not saved between requests
• Ollama only — the provider abstraction exists but only Ollama is implemented
• JSON mode reliance — output quality depends on the model's ability to produce valid structured JSON
• No question generation — no follow-up questions or uncertainty resolution yet
• No real-time streaming — waits for full response before displaying results
• Plain UI — no animations, theming, or responsive polish beyond basic layout

Deliberately Not Implemented

• Authentication / user accounts
• Billing / rate limiting
• Database or file storage
• Vector databases or embeddings
• Deployment configuration (Docker, Vercel, etc.)
• External cloud LLM providers (OpenAI, Anthropic, etc.)
• Agent frameworks or tool use
• Complex state management (Zustand, Redux, etc.)
• Multi-turn conversation history
• Question generation or ranking

Architecture Notes

The application uses a provider abstraction in lib/llm/provider.js. The server routes call the provider, which hides the Ollama-specific implementation. To add a new provider, implement the same interface:

{
  generateReconstruction(scenario, modelName): Promise<Reconstruction>
}
```READMEEOF