robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(confidence-engine): checkpoint plausible zero-state verification

Browse Source
This commit is contained in:
robbond
2026-10-02 07:12:36 +01:00
parent 0cc9d087c3
commit 30648ca5b3
2 changed files with 42 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+29 -2
View File
@@ -357,9 +357,36 @@ Live rendered-product verification via Playwright established that one actual in
- No production code or test changes during verification.
- **Evidence boundary:** establishes the implemented three-part boundary in the deterministic apparatus and the one observed rendered investigation. Does not generalise to universally solved, proven across all investigations, or production-complete.
**Unresolved — `Update report` placement/ownership:** OPEN PRODUCT QUESTION. A prescribed locator (`page.getByRole("button", { name: "Update report" })`) failed during verification, but this is a separate UI-ownership question (the control exists on the investigation detail page; its presence on the report page was not decided). The three-part report semantic/rendering increment is VERIFIED regardless.
**Update report placement/ownership (RESOLVED by Iteration 4):** `Report page + only when report is STALE`. The control exists on the report page and is invoked by the user when the lifecycle shows stale. This is established production behaviour, not an open UX decision.
**Report lifecycle generation/reuse:** not re-established by this browser run (zero-model-call budget). Existing lifecycle evidence remains authoritative unless separately retested. The inference that the report is "dynamically generated on page load" is not preserved from the verification record.
**Iterated Four — Zero-Plausible Interpretations Checkpoint (2026-10-02):**
Implementation commit: `0cc9d08` — "permit empty plausibleInterpretations for honest zero state"
Contract properties:
- `plausibleInterpretations` is a required string field;
- empty string (`""`) represents a genuine zero-plausible state;
- non-empty string represents a plausible narrative;
- `hasPlausibleInterpretations` derives from validated output content (no hard-coding);
- the model is explicitly permitted/required to return `""` when SECTION B is empty;
- zero-plausible reports omit "What remains plausible";
- non-zero behaviour remains deterministically covered.
Deterministic verification: 71/71 PASS.
Report lifecycle (established):
- absent → generate on first visit;
- current → reuse;
- investigation changes → stale;
- stale → user-owned Update report action;
- no automatic regeneration.
Fixed live investigation (`eae84f2d-acd8-4a92-bd43-347628b536a7`):
- precondition: `investigationRevision = 2`, `generatedFromRevision = 1`, report = STALE, plausible inputs = 0;
- exactly one live Update report action produced `plausibleInterpretations = ""`, `hasPlausibleInterpretations = false`, `investigationRevision = 2`, `generatedFromRevision = 2`, report = CURRENT;
- rendered result: "What we understand" visible, "What remains plausible" omitted, "What remains unresolved" visible, obsolete absence prose absent, canonical unresolved questions preserved.
**Evidence limitation:** This is specific to the one fixed zero-plausible regression case described above. The model correctly represented zero interpretations as empty string and rendered the report without a plausible section in this verified scenario. It does NOT prove universality across all scenario shapes or models.
**Next step:** product-level review of the Investigation Report experience — do not automatically change implementation.
docs/current-project-state.md
+13
View File
@@ -70,6 +70,19 @@ One actual Investigation Report demonstrated three distinct epistemic sections:
**Next:** product-level review of Investigation Report experience.
### Iteration 4 — Zero-Plausible Interpretations Checkpoint (2026-10-02)
Implementation commit: `0cc9d08` — "permit empty plausibleInterpretations for honest zero state"
The fixed zero-plausible regression case established:
- Canonical representation: `plausibleInterpretations = ""`, `hasPlausibleInterpretations = false`;
- Deterministic verification: 71/71 PASS;
- One live Update report call on investigation `eae84f2d` returned canonical zero state, moved report to CURRENT;
- Report rendered correctly: "What remains plausible" omitted, "What we understand" and "What remains unresolved" intact, obsolete absence prose absent;
- Update report ownership: Report page + only when stale (production behaviour).
Evidence is specific to this one fixed live regression case. Does not generalise universally.
#### Loop 2 (final semantic run — 2/2 valid calls)
Observed: