feat(confidence-engine): add saveFocusedInvestigationEvidence persistence operation (Experiment 8)
Add exported saveFocusedInvestigationEvidence(checkpoint, identity, metadata, outputPath) to the canonical experiment helper. Validates output path, exactly one valid deconstruction contribution, required identity fields, and exact input values. Writes formatted JSON, reads back, verifies deep structural equality. Rejects ambiguous checkpoints instead of silently selecting a contribution. Deterministic test covers: positive file write, structured evidence completeness, input fidelity, identity fidelity, return values, missing path rejection, missing contribution rejection, ambiguous contribution rejection. All OS temp files cleaned in finally blocks. Zero live model calls. Zero browser calls. No production code modified.
This commit is contained in:
@@ -767,3 +767,27 @@ Returns JSON-serializable record with: five deconstruction arrays (`observations
|
||||
**Distinction:** This test proves that the serializer's JSON-serializable output survives write-to-disk and read-back without loss. It does NOT establish a production/live-run command for writing evidence to files — that remains an untested operational boundary for future work. The apparatus produces correct in-memory records; disk persistence of those records is now independently verified at the test level.
|
||||
|
||||
**Experiment 4's original structured output remains unavailable.** Experiment 4 was executed before `serializeFocusedInvestigationEvidence` existed; its raw deconstruction checkpoint was never captured. No amount of apparatus testing can retroactively reconstruct it. The PARTIAL semantic fidelity classification for Experiment 4 stands unchanged.
|
||||
|
||||
### Iterated Eight — Explicit Evidence Persistence Operation (Experiment 8, 2026-10-09)
|
||||
|
||||
**Type:** Apparatus persistence operation experiment (Iteration 8 of bounded iteration loop). Zero-live-call apparatus experiment.
|
||||
|
||||
**Objective:** Answer whether the existing canonical experiment apparatus can expose one explicit, reusable operation that saves a completed focused-investigation checkpoint to a durable JSON file without modifying production code or changing model execution.
|
||||
|
||||
**Exported function:** `saveFocusedInvestigationEvidence(checkpoint, identity, metadata, outputPath)` — `tests/graph/live-update-experiment-helper.cjs`.
|
||||
|
||||
**Required arguments:**
|
||||
- `checkpoint` — object returned by `composeDeepInvestigationCheckpoint()` (must contain `postEpisodeContributions` with exactly one valid deconstruction contribution; must contain `focusedInvestigationInput` with exact values)
|
||||
- `identity` — `{ experimentId, scenarioId, targetNodeId, evidenceDesignation }` (all four fields required, no nulls)
|
||||
- `metadata` — optional `{ modelName?, providerGenerateCalls?, modelElapsedMs?, endToEndElapsedMs?, success? }`
|
||||
- `outputPath` — explicit non-empty string file path
|
||||
|
||||
**Verified output contract:** Returns `{ output_path: string, verifiedEvidence: object }`. Writes complete structured evidence as formatted JSON to the specified file. Reads file back and verifies deep structural equality with serialized record before returning. Five structured arrays preserved verbatim (including legitimately empty ones). All identity fields preserved exactly. Exact input values preserved exactly.
|
||||
|
||||
**Ambiguous contribution handling:** Rejected — throws `Error` when multiple valid deconstruction-carrying contributions exist in the checkpoint. Does not silently select one.
|
||||
|
||||
**Production changes:** None. No production code modified. No live model calls. No Ollama invocation. No graph behaviour change. No new dependencies.
|
||||
|
||||
**Targeted test result:** `node --check` on helper — PASS. Full deterministic test suite — 0 failures across all sections including 6 new save-evidence assertions (positive: file write, structured content, input fidelity, identity fidelity, return values; negative: missing path rejection, missing contribution rejection, ambiguous contribution rejection). All OS temp files cleaned in `finally` blocks.
|
||||
|
||||
**Operational note:** Future live experiments must explicitly call `saveFocusedInvestigationEvidence()` after a completed checkpoint to persist the evidence record. The function is not invoked automatically. Experiment 4's raw output remains unavailable — it was never persisted by the apparatus at that time.
|
||||
|
||||
@@ -103,3 +103,5 @@ Preserve existing graph relationships, stable identities, provenance, persistenc
|
||||
- **Experiment 6 — Durable Focused-Investigation Evidence (2026-10-09, Iteration 6 of 10):** Verified that the existing apparatus can preserve a complete, auditable focused-deconstruction checkpoint. Added `serializeFocusedInvestigationEvidence()` to the canonical helper (`tests/graph/live-update-experiment-helper.cjs`). Pure function accepts a completed checkpoint + identity/metadata params and returns JSON-serializable evidence record: (1) five deconstruction arrays preserved verbatim — `observations`, `uncertainties`, `assumptions`, `relationships`, `possibleFollowUpQuestions`; (2) exact inputs — `canonicalTargetDescription`, `actualQuestion`, `exactAnswer`; (3) execution metadata — null for fields not exposed by the apparatus. Deterministic verification: syntax check passed, 79/79 assertions passed including 28 new evidence-ser tests. No production code modified. No live LLM calls. No new semantic evidence produced. Experiment 4's missing structured output still cannot be reconstructed (never persisted by the apparatus at that time). Evidence-capture limitation corrected for future experiments.
|
||||
|
||||
- **Experiment 7 — Durable Focused-Investigation Disk Persistence (2026-10-09, Iteration 7 of 10):** Verified that `serializeFocusedInvestigationEvidence()` output survives write-to-disk and read-back without loss. Extended deterministic test in `tests/graph/live-update-experiment-helper-checkpoint.test.cjs` writes serialized evidence to OS temp directory via `os.tmpdir()`, reads it back, parses recovered JSON, and asserts deep structural equality against the original. Explicitly verified: five structured arrays with correct lengths; exact synthetic inputs (`canonicalTargetDescription`, `actualQuestion`, `exactAnswer`) unchanged; identity fields preserved; execution metadata null-fields intact. Temp file removed in `finally` block. Zero live model calls. Zero browser calls. No production or canonical-helper modifications. Total test result: all assertions passed (0 failures). This test proves the serializer's JSON-serializable output survives disk persistence at the apparatus level — it does not establish a production/live-run command for writing evidence to files, which remains an untested operational boundary. Experiment 4's original structured output remains unavailable; never persisted by the apparatus at that time.
|
||||
|
||||
- **Experiment 8 — Explicit Evidence Persistence Operation (2026-10-09, Iteration 8 of 10):** Added `saveFocusedInvestigationEvidence(checkpoint, identity, metadata, outputPath)` to the canonical helper (`tests/graph/live-update-experiment-helper.cjs`). This is a small wrapper around `serializeFocusedInvestigationEvidence()` that performs explicit persistence via an explicitly supplied output path. Validates: (1) non-empty output path required; (2) exactly one valid deconstruction contribution in checkpoint (rejects ambiguous multi-candidate); (3) all four identity fields present (no nulls); (4) focused-investigation input values present. Writes formatted JSON, reads back, verifies deep structural equality. Returns `{ output_path, verifiedEvidence }`. Zero live model calls. Zero browser calls. No production code modified. Total deterministic test result: 0 failures across all assertions including positive tests (file write, structured evidence completeness, input fidelity, identity fidelity, return values) and negative tests (missing path rejection, missing contribution rejection, ambiguous contribution rejection). All OS temp files cleaned in `finally` blocks. Future live experiments must explicitly call this save function after a completed checkpoint. Experiment 4's original structured output remains unavailable; never persisted by the apparatus at that time.
|
||||
|
||||
Reference in New Issue
Block a user