test: establish canonical live reasoning experiment harness

This commit is contained in:
2026-08-09 11:56:10 +01:00
parent 4aa1492c8d
commit e6f784261b
3 changed files with 198 additions and 0 deletions
+42
View File
@@ -24,6 +24,48 @@
- the task requires an undocumented contract;
- the experiment begins expanding into several capabilities.
### Live experiment execution route
A canonical live-update harness exists at `tests/graph/live-update-experiment-helper.cjs`.
It loads `.env.local`, validates required variables, invokes the real `updateCase()` production
entry point, and returns standard reasoning checkpoints (userSupportedMeaning, possibleInference,
rawAnswerCategory, proposedMeaningCategory, proposalValidation, compatibilityGuard, graphMutation,
selectedQuestion, behaviourSelection, reasoningState).
**Execution pattern:**
```js
const { runLiveExperiment } = require("./tests/graph/live-update-experiment-helper.cjs");
const result = await runLiveExperiment({
graph: /* SituationGraph fixture *\/,
previousQuestion: "Is risk a hard constraint?",
answer: "Risk matters more to me.",
});
// Checkpoints available on `result`:
// result.userSupportedMeaning
// result.possibleInference
// result.rawAnswerCategory
// result.proposedMeaningCategory
// result.proposalValidation
// result.compatibilityGuard
// result.graphMutation
// result.selectedQuestion
// result.behaviourSelection
// result.reasoningState
```
**Required environment (from `.env.local`):**
- `process.env.OLLAMA_BASE_URL` — must be a real host (no localhost fallback)
- `process.env.OLLAMA_MODEL` — model name (e.g. `qwen-claude:latest`)
The harness fails clearly if either variable is missing or OLLAMA_BASE_URL points to localhost.
It makes exactly one live Ollama call per invocation unless the experiment explicitly specifies otherwise.
**Rule:** During normal reasoning experiments, never create a bespoke harness, enumerate `/api/tags`,
probe localhost, or discover/substitute another model. Use the canonical harness above.
## Pack 2 — UI and Mock Work
### Always read