robbond
cb0c779019
Close Experiment 15 (Facilitator Behaviour Specification). Introduce Experiment 16 — Investigation State Assessment. - Create docs/investigation-state-assessment.md with 7 assessment dimensions: Current Investigation Phase, Investigation Progress, Evidence Quality, Understanding Trajectory, Uncertainty Trend, Conversation Health, and Behaviour Readiness. Each dimension includes purpose, observable signals, possible values, and how behaviours may consume it. - Document 6 assessment principles (Assess Not Decide, All Signals Traceable to Narrative, Descriptive Not Prescriptive, Convergence Over Single Signal, Stateful Across Turns, Uncertainty About Assessment Is Itself Assessable). - Include exploratory decision matrix linking investigation states to likely behaviours with reasons. - Prepend Behaviour Selection section to docs/facilitator-behaviour.md recording that behaviours are selected from Investigation State Assessment and do not inspect graph nodes directly. - Update docs/design-evolution-log.md: close Experiment 15, add Experiment 16 closure, record emerging architecture with the new layer between Narrative and Behaviour Selection. No implementation. Documentation only. No changes to reasoning engine, graph generation, prompts, orchestrator, APIs, Ollama integration, or UI.
Confidence Engine
An experimental prototype that tests whether an LLM can build and maintain an evidence-based reconstruction of a situation over multiple turns.
Purpose
This is Milestone 1 — a technical vertical slice only. It demonstrates:
- Sending a scenario to a local Ollama model via a Next.js server route
- Receiving structured JSON output
- Validating the result with Zod schemas
- Displaying the reconstruction and diagnostic information in a plain UI
Prerequisites
- Node.js 18+ (LTS recommended)
- npm (or equivalent package manager)
- Ollama installed and running on your local network, with a model pulled (e.g.,
ollama pull llama3)
Installation
cd confidence-engine
npm install
cp .env.example .env.local
Edit .env.local and set:
OLLAMA_BASE_URL— your Ollama server address (e.g.,http://192.168.1.100:11434)OLLAMA_MODEL— the model name (e.g.,llama3)
Development Commands
npm run dev # Start development server on localhost:3000
npm run build # Production build
npm run start # Run production server
npm run lint # ESLint check
Testing Commands
npm test # Run all tests (one-shot)
npm run test:watch # Run tests in watch mode
Tests mock the Ollama network request. No real Ollama server is needed to run them.
Verifying Ollama Connectivity
- Start the dev server:
npm run dev - Open http://localhost:3000/api/health
- You should see JSON with
"reachable": trueand your model name
Current Limitations
- Single-turn only — no conversation memory or multi-turn dialogue
- No persistence — results are not saved between requests
- Ollama only — the provider abstraction exists but only Ollama is implemented
- JSON mode reliance — output quality depends on the model's ability to produce valid structured JSON
- No question generation — no follow-up questions or uncertainty resolution yet
- No real-time streaming — waits for full response before displaying results
- Plain UI — no animations, theming, or responsive polish beyond basic layout
Deliberately Not Implemented
- Authentication / user accounts
- Billing / rate limiting
- Database or file storage
- Vector databases or embeddings
- Deployment configuration (Docker, Vercel, etc.)
- External cloud LLM providers (OpenAI, Anthropic, etc.)
- Agent frameworks or tool use
- Complex state management (Zustand, Redux, etc.)
- Multi-turn conversation history
- Question generation or ranking
Architecture Notes
The application uses a provider abstraction in lib/llm/provider.js. The server routes call the provider, which hides the Ollama-specific implementation. To add a new provider, implement the same interface:
{
generateReconstruction(scenario, modelName): Promise<Reconstruction>
}
```READMEEOF