88 lines
2.8 KiB
Markdown
88 lines
2.8 KiB
Markdown
# Confidence Engine
|
|
|
|
An experimental prototype that tests whether an LLM can build and maintain an evidence-based reconstruction of a situation over multiple turns.
|
|
|
|
## Purpose
|
|
|
|
This is Milestone 1 — a technical vertical slice only. It demonstrates:
|
|
|
|
- Sending a scenario to a local Ollama model via a Next.js server route
|
|
- Receiving structured JSON output
|
|
- Validating the result with Zod schemas
|
|
- Displaying the reconstruction and diagnostic information in a plain UI
|
|
|
|
## Prerequisites
|
|
|
|
- **Node.js 18+** (LTS recommended)
|
|
- **npm** (or equivalent package manager)
|
|
- **Ollama** installed and running on your local network, with a model pulled (e.g., `ollama pull llama3`)
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
cd confidence-engine
|
|
npm install
|
|
cp .env.example .env.local
|
|
```
|
|
|
|
Edit `.env.local` and set:
|
|
|
|
- `OLLAMA_BASE_URL` — your Ollama server address (e.g., `http://192.168.1.100:11434`)
|
|
- `OLLAMA_MODEL` — the model name (e.g., `llama3`)
|
|
|
|
## Development Commands
|
|
|
|
```bash
|
|
npm run dev # Start development server on localhost:3000
|
|
npm run build # Production build
|
|
npm run start # Run production server
|
|
npm run lint # ESLint check
|
|
```
|
|
|
|
## Testing Commands
|
|
|
|
```bash
|
|
npm test # Run all tests (one-shot)
|
|
npm run test:watch # Run tests in watch mode
|
|
```
|
|
|
|
Tests mock the Ollama network request. No real Ollama server is needed to run them.
|
|
|
|
## Verifying Ollama Connectivity
|
|
|
|
1. Start the dev server: `npm run dev`
|
|
2. Open http://localhost:3000/api/health
|
|
3. You should see JSON with `"reachable": true` and your model name
|
|
|
|
## Current Limitations
|
|
|
|
- **Single-turn only** — no conversation memory or multi-turn dialogue
|
|
- **No persistence** — results are not saved between requests
|
|
- **Ollama only** — the provider abstraction exists but only Ollama is implemented
|
|
- **JSON mode reliance** — output quality depends on the model's ability to produce valid structured JSON
|
|
- **No question generation** — no follow-up questions or uncertainty resolution yet
|
|
- **No real-time streaming** — waits for full response before displaying results
|
|
- **Plain UI** — no animations, theming, or responsive polish beyond basic layout
|
|
|
|
## Deliberately Not Implemented
|
|
|
|
- Authentication / user accounts
|
|
- Billing / rate limiting
|
|
- Database or file storage
|
|
- Vector databases or embeddings
|
|
- Deployment configuration (Docker, Vercel, etc.)
|
|
- External cloud LLM providers (OpenAI, Anthropic, etc.)
|
|
- Agent frameworks or tool use
|
|
- Complex state management (Zustand, Redux, etc.)
|
|
- Multi-turn conversation history
|
|
- Question generation or ranking
|
|
|
|
## Architecture Notes
|
|
|
|
The application uses a **provider abstraction** in `lib/llm/provider.js`. The server routes call the provider, which hides the Ollama-specific implementation. To add a new provider, implement the same interface:
|
|
|
|
```js
|
|
{
|
|
generateReconstruction(scenario, modelName): Promise<Reconstruction>
|
|
}
|
|
```READMEEOF |