robbond
57c9f2205e
Implementation of Candidate B (unknown+option) from decision architecture design in 60A.2. Adds two new primitives to the situation graph: Schema (lib/graph/schema.js): - SituationKind.option — a choice available within a decision context - SituationRelationship.contained_in — links option → its parent unknown context Prompt rules (lib/graph/prompt-builder.js): - Section added: Decision Option Structure Rules with 5 numbered instructions governing when/how to create option nodes, link them via contained_in, attach consequences to specific options, and handle do-nothing alternatives. Explicitly forbids alternative_to edges and is_baseline/is_default flags. Tests (446 new lines): - schema.test.js: +300 — enum completeness updates, option kind validation, contained_in edge validation, native two-option graph fixture (~25 new tests) - prompt-builder.test.js: +133 — focused rules verification for all 5 rule points, negative checks (no relocation/savings/example-specific wording, no alternative_to requirement, baseline flag prohibition context) No production code paths affected beyond the two enum additions; existing node and edge kinds remain unchanged. No Ollama calls, no live API calls.
Confidence Engine
An experimental prototype that tests whether an LLM can build and maintain an evidence-based reconstruction of a situation over multiple turns.
Purpose
This is Milestone 1 — a technical vertical slice only. It demonstrates:
- Sending a scenario to a local Ollama model via a Next.js server route
- Receiving structured JSON output
- Validating the result with Zod schemas
- Displaying the reconstruction and diagnostic information in a plain UI
Prerequisites
- Node.js 18+ (LTS recommended)
- npm (or equivalent package manager)
- Ollama installed and running on your local network, with a model pulled (e.g.,
ollama pull llama3)
Installation
cd confidence-engine
npm install
cp .env.example .env.local
Edit .env.local and set:
OLLAMA_BASE_URL— your Ollama server address (e.g.,http://192.168.1.100:11434)OLLAMA_MODEL— the model name (e.g.,llama3)
Development Commands
npm run dev # Start development server on localhost:3000
npm run build # Production build
npm run start # Run production server
npm run lint # ESLint check
Testing Commands
npm test # Run all tests (one-shot)
npm run test:watch # Run tests in watch mode
Tests mock the Ollama network request. No real Ollama server is needed to run them.
Verifying Ollama Connectivity
- Start the dev server:
npm run dev - Open http://localhost:3000/api/health
- You should see JSON with
"reachable": trueand your model name
Current Limitations
- Single-turn only — no conversation memory or multi-turn dialogue
- No persistence — results are not saved between requests
- Ollama only — the provider abstraction exists but only Ollama is implemented
- JSON mode reliance — output quality depends on the model's ability to produce valid structured JSON
- No question generation — no follow-up questions or uncertainty resolution yet
- No real-time streaming — waits for full response before displaying results
- Plain UI — no animations, theming, or responsive polish beyond basic layout
Deliberately Not Implemented
- Authentication / user accounts
- Billing / rate limiting
- Database or file storage
- Vector databases or embeddings
- Deployment configuration (Docker, Vercel, etc.)
- External cloud LLM providers (OpenAI, Anthropic, etc.)
- Agent frameworks or tool use
- Complex state management (Zustand, Redux, etc.)
- Multi-turn conversation history
- Question generation or ranking
Architecture Notes
The application uses a provider abstraction in lib/llm/provider.js. The server routes call the provider, which hides the Ollama-specific implementation. To add a new provider, implement the same interface:
{
generateReconstruction(scenario, modelName): Promise<Reconstruction>
}
```READMEEOF