docs(confidence-engine): checkpoint plausible zero-state verification
This commit is contained in:
+29
-2
@@ -357,9 +357,36 @@ Live rendered-product verification via Playwright established that one actual in
|
|||||||
- No production code or test changes during verification.
|
- No production code or test changes during verification.
|
||||||
- **Evidence boundary:** establishes the implemented three-part boundary in the deterministic apparatus and the one observed rendered investigation. Does not generalise to universally solved, proven across all investigations, or production-complete.
|
- **Evidence boundary:** establishes the implemented three-part boundary in the deterministic apparatus and the one observed rendered investigation. Does not generalise to universally solved, proven across all investigations, or production-complete.
|
||||||
|
|
||||||
**Unresolved — `Update report` placement/ownership:** OPEN PRODUCT QUESTION. A prescribed locator (`page.getByRole("button", { name: "Update report" })`) failed during verification, but this is a separate UI-ownership question (the control exists on the investigation detail page; its presence on the report page was not decided). The three-part report semantic/rendering increment is VERIFIED regardless.
|
**Update report placement/ownership (RESOLVED by Iteration 4):** `Report page + only when report is STALE`. The control exists on the report page and is invoked by the user when the lifecycle shows stale. This is established production behaviour, not an open UX decision.
|
||||||
|
|
||||||
**Report lifecycle generation/reuse:** not re-established by this browser run (zero-model-call budget). Existing lifecycle evidence remains authoritative unless separately retested. The inference that the report is "dynamically generated on page load" is not preserved from the verification record.
|
**Iterated Four — Zero-Plausible Interpretations Checkpoint (2026-10-02):**
|
||||||
|
|
||||||
|
Implementation commit: `0cc9d08` — "permit empty plausibleInterpretations for honest zero state"
|
||||||
|
|
||||||
|
Contract properties:
|
||||||
|
- `plausibleInterpretations` is a required string field;
|
||||||
|
- empty string (`""`) represents a genuine zero-plausible state;
|
||||||
|
- non-empty string represents a plausible narrative;
|
||||||
|
- `hasPlausibleInterpretations` derives from validated output content (no hard-coding);
|
||||||
|
- the model is explicitly permitted/required to return `""` when SECTION B is empty;
|
||||||
|
- zero-plausible reports omit "What remains plausible";
|
||||||
|
- non-zero behaviour remains deterministically covered.
|
||||||
|
|
||||||
|
Deterministic verification: 71/71 PASS.
|
||||||
|
|
||||||
|
Report lifecycle (established):
|
||||||
|
- absent → generate on first visit;
|
||||||
|
- current → reuse;
|
||||||
|
- investigation changes → stale;
|
||||||
|
- stale → user-owned Update report action;
|
||||||
|
- no automatic regeneration.
|
||||||
|
|
||||||
|
Fixed live investigation (`eae84f2d-acd8-4a92-bd43-347628b536a7`):
|
||||||
|
- precondition: `investigationRevision = 2`, `generatedFromRevision = 1`, report = STALE, plausible inputs = 0;
|
||||||
|
- exactly one live Update report action produced `plausibleInterpretations = ""`, `hasPlausibleInterpretations = false`, `investigationRevision = 2`, `generatedFromRevision = 2`, report = CURRENT;
|
||||||
|
- rendered result: "What we understand" visible, "What remains plausible" omitted, "What remains unresolved" visible, obsolete absence prose absent, canonical unresolved questions preserved.
|
||||||
|
|
||||||
|
**Evidence limitation:** This is specific to the one fixed zero-plausible regression case described above. The model correctly represented zero interpretations as empty string and rendered the report without a plausible section in this verified scenario. It does NOT prove universality across all scenario shapes or models.
|
||||||
|
|
||||||
**Next step:** product-level review of the Investigation Report experience — do not automatically change implementation.
|
**Next step:** product-level review of the Investigation Report experience — do not automatically change implementation.
|
||||||
|
|
||||||
|
|||||||
@@ -70,6 +70,19 @@ One actual Investigation Report demonstrated three distinct epistemic sections:
|
|||||||
|
|
||||||
**Next:** product-level review of Investigation Report experience.
|
**Next:** product-level review of Investigation Report experience.
|
||||||
|
|
||||||
|
### Iteration 4 — Zero-Plausible Interpretations Checkpoint (2026-10-02)
|
||||||
|
|
||||||
|
Implementation commit: `0cc9d08` — "permit empty plausibleInterpretations for honest zero state"
|
||||||
|
|
||||||
|
The fixed zero-plausible regression case established:
|
||||||
|
- Canonical representation: `plausibleInterpretations = ""`, `hasPlausibleInterpretations = false`;
|
||||||
|
- Deterministic verification: 71/71 PASS;
|
||||||
|
- One live Update report call on investigation `eae84f2d` returned canonical zero state, moved report to CURRENT;
|
||||||
|
- Report rendered correctly: "What remains plausible" omitted, "What we understand" and "What remains unresolved" intact, obsolete absence prose absent;
|
||||||
|
- Update report ownership: Report page + only when stale (production behaviour).
|
||||||
|
|
||||||
|
Evidence is specific to this one fixed live regression case. Does not generalise universally.
|
||||||
|
|
||||||
#### Loop 2 (final semantic run — 2/2 valid calls)
|
#### Loop 2 (final semantic run — 2/2 valid calls)
|
||||||
|
|
||||||
Observed:
|
Observed:
|
||||||
|
|||||||
Reference in New Issue
Block a user