68 lines
2.7 KiB
Markdown
68 lines
2.7 KiB
Markdown
# CU Synthesis Experiment Apparatus — Iteration 3
|
|
|
|
## Purpose
|
|
|
|
Deterministic experiment helper for invoking the production `synthesizeCurrentUnderstanding()` function with fixed local input fixtures, without live model calls.
|
|
|
|
## Files
|
|
|
|
| File | Role |
|
|
|------|------|
|
|
| `scripts/current-understanding-experiment-helper.cjs` | Standalone helper (CJS + dynamic import ESM) |
|
|
| `tests/graph/current-understanding-experiment-helper.test.js` | Deterministic verification (14 tests, 0 live calls) |
|
|
|
|
## Usage
|
|
|
|
### Helper (CLI)
|
|
|
|
```bash
|
|
node scripts/current-understanding-experiment-helper.cjs --file fixtures/cu-synthesis.json
|
|
```
|
|
|
|
Fixture contract: `{ situationGraph: object, findings: array }`
|
|
|
|
### Test
|
|
|
|
```bash
|
|
npx vitest run tests/graph/current-understanding-experiment-helper.test.js --environment=node
|
|
```
|
|
|
|
## Existing Mechanics Reused
|
|
|
|
- Dynamic `import()` seam from CJS to ESM (proven in `scripts/overview-experiment-helper.cjs` line 64)
|
|
- Environment loading via `@next/env` `loadEnvConfig` (proven in both existing helpers)
|
|
- `.env.local` ownership of `OLLAMA_BASE_URL` and `OLLAMA_MODEL` preserved — helper reads from `process.env`, never writes
|
|
- Deterministic inline test double provider pattern (mirrors `makeFakeProvider` in `tests/graph/current-understanding-synthesis.test.js`)
|
|
|
|
## Production Entry Point
|
|
|
|
`synthesizeCurrentUnderstanding()` from `lib/graph/current-understanding-synthesis.js`
|
|
|
|
Input contract: `{ situationGraph: object, findings: array }`
|
|
Optional deps: `{ provider: object, modelName: string }`
|
|
|
|
Output: `{ currentUnderstanding: string }`
|
|
|
|
## Verification Results
|
|
|
|
All 14 deterministic tests PASS. Zero live model calls. The following invariants verified:
|
|
- Fixed compatible input reaches `synthesizeCurrentUnderstanding()`
|
|
- Expected structured output `{ currentUnderstanding }` captured
|
|
- Prompt includes revised Rule 2 boundary text ("Do not introduce or describe open questions...")
|
|
- Exactly one synthesis invocation per run
|
|
- Input fixture is not mutated (deep clone + compare)
|
|
- No persistence/Supabase/DB operations
|
|
|
|
## Iteration 4 — Regression Fixture
|
|
|
|
Fixture path: `tests/fixtures/current-understanding-original-investigation.json`
|
|
|
|
Source: authenticated GET `/api/investigations/6b080d97-033e-477b-9d41-7b2f4f04bbed` (2026-10-08)
|
|
|
|
Contract: `{ situationGraph: object, findings: array }`
|
|
|
|
Contents:
|
|
- `situationGraph`: 11 nodes (1 state, 5 observation, 2 metric, 3 unknown), 10 edges, full `reasoningState`, `resolvedNodeIds`, `centralStatement`, `activeUnknownNodeId`
|
|
- `findings`: 9 findings total; all 7 previously verified IDs present (`hciikc`, `ybfsm3`, `k45vyt`, `vhftyr`, `9bgkmw`, `23192q`, `kkh39k`)
|
|
|
|
Verified: helper parses without invoking live synthesis; fixture immutability confirmed via deep-clone roundtrip. |