robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(confidence-engine): add serializeFocusedInvestigationEvidence evidence serializer and deterministic test

Browse Source 
Apparatus-level pure serialization function that preserves all five
focused-deconstruction arrays (observations, uncertainties, assumptions,
relationships, possibleFollowUpQuestions) plus exact inputs and execution
metadata from the existing checkpoint — without modifying production code.

Deterministic verification: syntax check pass, 79/79 assertions pass
(including 28 new evidence-ser tests), zero live calls.

Experiment 4's missing structured output cannot be reconstructed (never
persisted by the apparatus). No new semantic evidence produced.
This commit is contained in:
robbond
2026-10-09 14:25:26 +01:00
parent 7fd05745e8
commit 7d7a9e8d29
4 changed files with 205 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+19
View File
@@ -730,3 +730,22 @@ Tested canonical uncertainty description as initial display instead of generated
**Input provenance:** Synthetic answer deliberately crafted for this controlled experiment as FICTIONAL TEST DATA. Not a real user response. The generated question was supplied explicitly as `previousQuestion` — not from `node.openQuestion`. This does not imply that the production UI always supplies `node.openQuestion` as `previousQuestion`.
**Remaining limitations:** Single scenario shape only (recruitment). Does not establish consistency across domains or models. A single run does not prove generalisability or user comprehension. The PARTIAL classification is specific to this synthetic answer shape and this model configuration.
## Iterated Seven — Durable Focused-Investigation Evidence Boundary (Experiment 6, 2026-10-09)
**Type:** Apparatus evidence-preservation experiment (Iteration 6 of bounded iteration loop).
**Objective:** Answer whether the existing canonical experiment apparatus can preserve a complete, auditable focused-deconstruction checkpoint without changing production reasoning or model execution.
**Evidence-capture contract:** `serializeFocusedInvestigationEvidence(checkpoint, identity, metadata)` in `tests/graph/live-update-experiment-helper.cjs` (line ~524). Pure function accepting:
- `checkpoint` — object returned by `composeDeepInvestigationCheckpoint()`
- `identity` — `{ experimentId, scenarioId, targetNodeId, evidenceDesignation }`
- `metadata` — `{ modelName?, providerGenerateCalls?, modelElapsedMs?, endToEndElapsedMs?, success? }`
Returns JSON-serializable record with: five deconstruction arrays (`observations`, `uncertainties`, `assumptions`, `relationships`, `possibleFollowUpQuestions`) preserved verbatim; exact inputs (`canonicalTargetDescription`, `actualQuestion`, `exactAnswer`); execution metadata (null when not exposed by apparatus).
**Deterministic verification:** Syntax check PASS. 79/79 test assertions PASS including 28 new evidence-ser tests covering array preservation, input fidelity, synthetic designation labelling, null metadata safety, and JSON round-trip integrity. Zero live LLM calls. Zero browser calls.
**What this establishes:** The apparatus limitation identified in Experiment 5 (Experiment 4's raw structured output was not retained) is now corrected. Future focused-investigation experiments can preserve their deconstruction checkpoint via `serializeFocusedInvestigationEvidence()`.
**What remains:** No new semantic evidence was produced. Experiment 4's missing structured output cannot be reconstructed — it was never persisted by the apparatus at that time. The PARTIAL semantic fidelity classification stands unchanged.