confidence-engine/docs/experiment-current-understanding-apparatus.md

# CU Synthesis Experiment Apparatus — Iteration 5

## Purpose

Deterministic experiment helper for invoking the production `synthesizeCurrentUnderstanding()` function with fixed local input fixtures, and an explicit live mode for production model execution. Zero live model calls by default.

## Files

| File | Role |
|------|------|
| `scripts/current-understanding-experiment-helper.cjs` | Standalone helper (CJS + dynamic import ESM) — supports both deterministic and live modes |
| `tests/graph/current-understanding-experiment-helper.test.js` | Deterministic verification (34 tests, 0 live calls) |

## Usage

### Helper (CLI)

```bash
# Deterministic mode (default — zero live model calls)
node scripts/current-understanding-experiment-helper.cjs --file fixtures/cu-synthesis.json

# Live production mode (uses configured OLLAMA_MODEL from .env.local)
node scripts/current-understanding-experiment-helper.cjs --live --file fixtures/cu-synthesis.json

# With output exposure (exposes exact currentUnderstanding narrative in structured result)
node scripts/current-understanding-experiment-helper.cjs --file fixtures/cu-synthesis.json --include-output

# Longitudinal checkpoint selection — uses findings from longitudinal manifest
node scripts/current-understanding-experiment-helper.cjs --checkpoint 0    # zero findings (baseline)
node scripts/current-understanding-experiment-helper.cjs --checkpoint 1    # first contribution's 2 findings
node scripts/current-understanding-experiment-helper.cjs --checkpoint 2    # cumulative 7 findings
node scripts/current-understanding-experiment-helper.cjs --checkpoint 3    # all 9 findings (full state)

# --checkpoint can be combined with --include-output
node scripts/current-understanding-experiment-helper.cjs --checkpoint 1 --include-output
```

Fixture contract: `{ situationGraph: object, findings: array }`

When `--checkpoint` is used, the helper reads `tests/fixtures/current-understanding-longitudinal-checkpoints.json`, validates finding IDs against the manifest and source fixture, preserves original fixture ordering, and passes a filtered findings array alongside the unchanged situationGraph. Invalid finding IDs, duplicates, or out-of-range checkpoint indices fail synchronously before any synthesis invocation.

### Test

```bash
npx vitest run tests/graph/current-understanding-experiment-helper.test.js --environment=node
```

## Existing Mechanics Reused

- Dynamic `import()` seam from CJS to ESM (proven in `scripts/overview-experiment-helper.cjs` line 64)
- Environment loading via `@next/env` `loadEnvConfig` (proven in both existing helpers)
- `.env.local` ownership of `OLLAMA_BASE_URL` and `OLLAMA_MODEL` preserved — helper reads from `process.env`, never writes
- Deterministic inline test double provider pattern (mirrors `makeFakeProvider` in `tests/graph/current-understanding-synthesis.test.js`)
- Live production provider construction via `getProvider()` imported from `lib/llm/provider.js` (canonical seam, first extraction at line 87-90 of experiment helper)

## Production Entry Point

`synthesizeCurrentUnderstanding()` from `lib/graph/current-understanding-synthesis.js`

Input contract: `{ situationGraph: object, findings: array }`
Optional deps: `{ provider: object, modelName: string }`

Output: `{ currentUnderstanding: string }`

## Verification Results

All 24 deterministic tests PASS. Zero live model calls. The following invariants verified:
- Fixed compatible input reaches `synthesizeCurrentUnderstanding()`
- Expected structured output `{ currentUnderstanding }` captured
- Prompt includes revised Rule 2 boundary text ("Do not introduce or describe open questions...")
- Exactly one synthesis invocation per run
- Input fixture is not mutated (deep clone + compare)
- No persistence/Supabase/DB operations
- Live-mode wiring verified via deterministic injected mock provider
- Deterministic mode behaviour preserved unchanged

## Iteration 5 — Live Provider Seam

### New seam: `createLiveProvider()` in `scripts/current-understanding-experiment-helper.cjs` (line 87-90)

Reuses the canonical provider construction from `lib/llm/provider.js`:
```js
const { getProvider } = await import(PROJECT_ROOT + "/lib/llm/provider.js");
return getProvider();
```

### New CLI flag: `--live`

When `--live` is specified alongside `--file`, the helper uses the production provider instead of the test double. Requires `OLLAMA_BASE_URL` and `OLLAMA_MODEL` from `.env.local`.

### Test coverage for live-mode (10 new tests)

All 10 tests use a deterministic injected mock provider (no network). They verify:
1. Fixture passes through unchanged to synthesis function
2. Provider seam invoked exactly once
3. Evidence projection passed as prompt with correct headers
4. Current Understanding text captured correctly
5. Schema matches `synthesisOutputSchema`
6. No persistence operations
7. Deterministic mode still works (regression)
8. Rule 2 boundary text present
9. Zero I/O in tests (mock is pure function)
10. Full apparatus output shape validated for live-mode path

## Iteration 4 — Regression Fixture

Fixture path: `tests/fixtures/current-understanding-original-investigation.json`

Source: authenticated GET `/api/investigations/6b080d97-033e-477b-9d41-7b2f4f04bbed` (2026-10-08)

Contract: `{ situationGraph: object, findings: array }`

Contents:
- `situationGraph`: 11 nodes (1 state, 5 observation, 2 metric, 3 unknown), 10 edges, full `reasoningState`, `resolvedNodeIds`, `centralStatement`, `activeUnknownNodeId`
- `findings`: 9 findings total; all 7 previously verified IDs present (`hciikc`, `ybfsm3`, `k45vyt`, `vhftyr`, `9bgkmw`, `23192q`, `kkh39k`)

Verified: helper parses without invoking live synthesis; fixture immutability confirmed via deep-clone roundtrip.

## Iteration 6 — Longitudinal Checkpoints (Stage 2)

### New files

| File | Role |
|------|------|
| `tests/fixtures/current-understanding-longitudinal-checkpoints.json` | Compact checkpoint manifest referencing original fixture; lists included contribution IDs and finding IDs per checkpoint |
| `tests/graph/current-understanding-experiment-helper.test.js` (extended) | 15 new longitudinal assertions in `Longitudinal checkpoints — manifest integrity` describe block |

### Checkpoint manifest structure

The manifest contains:
- `manifestVersion`, `description`, `sourceFixture`, `contaminationAssessment`
- `contributions[]` — ordered list with timestamps and ordering integers
- `findingsById` — verbatim copy of each original finding (by ID)
- `checkpoints[]` — four cumulative states: `{ id, includedContributions[], includedFindings[] }`

### Checkpoint state summary

| Checkpoint | Description | Contribs | Findings | Count |
|---|---|---|---|---|
| checkpoint-0 | Baseline (no findings) | none | none | 0 |
| checkpoint-1 | contrib-0001 | contrib-0001 | 3bw5f0, i5zdcq | 2 |
| checkpoint-2 | contrib-0001 + contrib-0002 | contrib-0001, contrib-0002 | 7 findings | 7 |
| checkpoint-3 | All contributions | all three | all 9 original findings | 9 |

### Graph-state contamination analysis

`synthesizeCurrentUnderstanding()` and `buildGraphEvidenceProjection()` project **only**:
- Nodes with `status: "known"` or `"supported"` (filtered)
- `centralStatement` (framing only)
- Findings passed as separate parameter

**Explicitly excluded from prompt:** `resolvedNodeIds`, `activeUnknownNodeId`, `reasoningState`, edges, `currentSummary`, nodes outside known/supported statuses.

**Conclusion: NONE — final-state graph fields do not leak into synthesis input.**

### Compatibility gap — resolved

The existing helper consumed a single `{ situationGraph, findings }` fixture via `--file`. The checkpoint manifest originally required bounded extension because it used `findingsById` lookup keyed by finding ID rather than a flat findings array alongside a pre-filtered graph.

**This gap is now resolved.** `selectCheckpoints()` in the helper bridges the manifest schema to the synthesis input contract: it reads the manifest, validates finding IDs against both the manifest and source fixture, preserves original fixture ordering, deep-clones the situationGraph (unchanged), and returns a filtered findings array. Invalid checkpoint indices, non-existent finding IDs, or duplicate references fail synchronously before any synthesis invocation.

Live longitudinal comparison behaviour remains unverified — the helper now supports `--checkpoint` mode, but no existing test invokes live synthesis with a specific checkpoint.

### Test assertions added (15 new tests)

1. Manifest has exactly 4 checkpoints
2. Checkpoint counts are 0, 2, 7, 9
3. checkpoint-0 has no included findings
4. checkpoint-1 includes exactly contrib-0001 findings
5. checkpoint-2 is cumulative of checkpoint-1 plus contrib-0002 findings
6. checkpoint-3 includes all 9 original findings
7. All checkpoint findings are verbatim from the original fixture
8. Contribution ordering matches recorded timestamps
9. Manifest ordering matches fixture finding order
10. checkpoint-3 is cumulative of all checkpoints before it
11. No findings appear before their contribution boundary
12. Original fixture unchanged (spot check)
13. Production synthesis contract accepts all checkpoint inputs
14. Manifest references the source fixture path
15. Manifest does not fabricate historical graph state

## Iteration 7 — Longitudinal Checkpoint Selection (Stage 3)

### Changes to `scripts/current-understanding-experiment-helper.cjs`

- **New CLI flag**: `--checkpoint <0|1|2|3>` selects findings from the longitudinal manifest.
- **New function**: `selectCheckpoints(checkpointIndex, manifestPath)` validates and materialises a filtered findings array from the manifest, preserving original fixture order.
- **Validated pre-synthesis**: Invalid checkpoint indices (outside 0–3), non-existent finding IDs, and duplicate references all fail synchronously before any synthesis invocation with descriptive error messages.
- **Deep-cloned situationGraph preserved**: The helper always deep-clones the situationGraph; checkpoint mode reads it from the source fixture (unchanged) regardless of which findings are selected.
- **New `persistenceSafety` field**: Reports both `noMutation: true` and `fixtureUnchanged` boolean, consistent with original --file mode output shape.
- **Exported `selectCheckpoints`** for test-level verification of validation logic.

### Checkpoint selection behaviour

| Flag | Findings selected | Validation |
|------|------------------|------------|
| `--checkpoint 0` | 0 (baseline) | manifest index range ✓ |
| `--checkpoint 1` | 2 (contrib-0001) | finding IDs in `findingsById` ✓, no duplicates ✓ |
| `--checkpoint 2` | 7 (cumulative contrib-0001 + contrib-0002) | same validation |
| `--checkpoint 3` | 9 (all findings) | same validation |

### Test coverage for checkpoint selection (15 new tests)

All tests in the `Longitudinal checkpoints — manifest integrity` describe block are deterministic. They verify:

1. Manifest has exactly 4 checkpoints
2. Checkpoint counts are 0, 2, 7, 9
3. checkpoint-0 has no included findings
4. checkpoint-1 includes exactly contrib-0001 findings
5. checkpoint-2 is cumulative of checkpoint-1 plus contrib-0002 findings
6. checkpoint-3 includes all 9 original findings
7. All checkpoint findings are verbatim from the original fixture
8. Contribution ordering matches recorded timestamps
9. Manifest ordering matches fixture finding order
10. checkpoint-3 is cumulative of all checkpoints before it
11. No findings appear before their contribution boundary
12. Original fixture unchanged (spot check)
13. Production synthesis contract accepts all checkpoint inputs
14. Manifest references the source fixture path
15. Manifest does not fabricate historical graph state

### What this establishes

Checkpoint selection is implemented and validated for deterministic mode only. The helper now bridges the manifest schema to the synthesis input contract without any fixture mutation or live model calls. Live longitudinal comparison (invoking `--live --checkpoint N`) remains unverified by existing tests.