test: establish canonical live reasoning experiment harness
This commit is contained in:
@@ -24,6 +24,48 @@
|
||||
- the task requires an undocumented contract;
|
||||
- the experiment begins expanding into several capabilities.
|
||||
|
||||
### Live experiment execution route
|
||||
|
||||
A canonical live-update harness exists at `tests/graph/live-update-experiment-helper.cjs`.
|
||||
It loads `.env.local`, validates required variables, invokes the real `updateCase()` production
|
||||
entry point, and returns standard reasoning checkpoints (userSupportedMeaning, possibleInference,
|
||||
rawAnswerCategory, proposedMeaningCategory, proposalValidation, compatibilityGuard, graphMutation,
|
||||
selectedQuestion, behaviourSelection, reasoningState).
|
||||
|
||||
**Execution pattern:**
|
||||
|
||||
```js
|
||||
const { runLiveExperiment } = require("./tests/graph/live-update-experiment-helper.cjs");
|
||||
|
||||
const result = await runLiveExperiment({
|
||||
graph: /* SituationGraph fixture *\/,
|
||||
previousQuestion: "Is risk a hard constraint?",
|
||||
answer: "Risk matters more to me.",
|
||||
});
|
||||
|
||||
// Checkpoints available on `result`:
|
||||
// result.userSupportedMeaning
|
||||
// result.possibleInference
|
||||
// result.rawAnswerCategory
|
||||
// result.proposedMeaningCategory
|
||||
// result.proposalValidation
|
||||
// result.compatibilityGuard
|
||||
// result.graphMutation
|
||||
// result.selectedQuestion
|
||||
// result.behaviourSelection
|
||||
// result.reasoningState
|
||||
```
|
||||
|
||||
**Required environment (from `.env.local`):**
|
||||
- `process.env.OLLAMA_BASE_URL` — must be a real host (no localhost fallback)
|
||||
- `process.env.OLLAMA_MODEL` — model name (e.g. `qwen-claude:latest`)
|
||||
|
||||
The harness fails clearly if either variable is missing or OLLAMA_BASE_URL points to localhost.
|
||||
It makes exactly one live Ollama call per invocation unless the experiment explicitly specifies otherwise.
|
||||
|
||||
**Rule:** During normal reasoning experiments, never create a bespoke harness, enumerate `/api/tags`,
|
||||
probe localhost, or discover/substitute another model. Use the canonical harness above.
|
||||
|
||||
## Pack 2 — UI and Mock Work
|
||||
|
||||
### Always read
|
||||
|
||||
Reference in New Issue
Block a user