robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(confidence-engine): checkpoint verified three-part report

Browse Source
This commit is contained in:
robbond
2026-10-01 17:13:10 +01:00
parent 56b5f0099c
commit 8dc6e58d43
2 changed files with 47 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+27
View File
@@ -336,6 +336,33 @@ scripts/overview-experiment-helper.cjs — canonical overview live helper
- Both manufacturing (with findings) and product-launch (zero Findings) A/B controlled cases pass all determinism checks.
- **No live second-case A/B has yet been run.** The behavioural question — whether exposing unresolved uncertainty in the presence of zero Findings produces different model output than without — remains open.
### Three-Part Report Increment — Loop 4/2 (VERIFIED)
**Implementation commit:** `56b5f00` — "feat(confidence-engine): show unresolved report questions"
Deterministic apparatus verified that:
- Resolved/unknown nodes are excluded from admitted evidence at prompt construction time;
- Passing the full situationGraph with unresolved nodes vs a filtered version produces identical prompt text (18 deterministic tests, all PASS).
Live rendered-product verification via Playwright established that one actual investigation Report renders three distinct epistemic sections:
1. **What we understand** — contains evidence-backed understanding; unresolved Open Questions do not appear as supported understanding.
2. **What remains plausible** — visibly and semantically separate; section correctly displayed its empty/default state (no supplied plausible interpretations in the verified investigation).
3. **What remains unresolved** — derives from canonical unresolved Open Questions; rendered the canonical question text; remained visibly separate from understanding and plausible interpretations; did not answer, reinterpret, rank, prioritise, or imply blockers/readiness/confidence/evidence sufficiency/investigation completion.
- Deterministic verification: passed.
- Rendered-product verification: passed for three-part epistemic separation.
- Unresolved questions remained canonical non-evidence.
- Zero additional experimental model calls during live verification.
- No production code or test changes during verification.
- **Evidence boundary:** establishes the implemented three-part boundary in the deterministic apparatus and the one observed rendered investigation. Does not generalise to universally solved, proven across all investigations, or production-complete.
**Unresolved — `Update report` placement/ownership:** OPEN PRODUCT QUESTION. A prescribed locator (`page.getByRole("button", { name: "Update report" })`) failed during verification, but this is a separate UI-ownership question (the control exists on the investigation detail page; its presence on the report page was not decided). The three-part report semantic/rendering increment is VERIFIED regardless.
**Report lifecycle generation/reuse:** not re-established by this browser run (zero-model-call budget). Existing lifecycle evidence remains authoritative unless separately retested. The inference that the report is "dynamically generated on page load" is not preserved from the verification record.
**Next step:** product-level review of the Investigation Report experience — do not automatically change implementation.
## Current product architecture
Three distinct routes: