From 7c1d657761d6e21999d65f70b07071719bb8e39b Mon Sep 17 00:00:00 2001 From: robbond Date: Wed, 30 Sep 2026 17:22:52 +0100 Subject: [PATCH] =?UTF-8?q?docs(confidence-engine):=20record=20product-lau?= =?UTF-8?q?nch=20epistemic=20A/B=20result=20=E2=80=94=204-iteration=20loop?= =?UTF-8?q?=20complete?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Controlled A/B (zero-Findings scenario, Qwen): unresolved uncertainty exposure did not contaminate supported understanding or plausible interpretation. No production changes. No report semantic changes. --- docs/current-handoff.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/docs/current-handoff.md b/docs/current-handoff.md index d5ffec9..1dd5592 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -309,7 +309,22 @@ scripts/overview-experiment-helper.cjs — canonical overview live helper - **visible**: appends unresolved questions as separate NON-EVIDENCE section to baseline prompt - Apparatus invariant proven deterministically: A/B evidence, findings, plausibleInterpretations, scenario state are identical; only unresolved visibility differs. - Production `lib/graph/investigation-overview-synthesis.js` unchanged. -- **No live A/B calls executed yet.** Next boundary: Call A (hidden) with product-launch fixture. +- **Controlled A/B completed:** product-launch zero-Findings case, 2 live calls (51s hidden, 54s visible). + +**Result:** Supported understanding materially unchanged; both calls produce the same core finding (product ready enough for launch, one large enterprise client relevant). No plausible interpretations in either variant (helper called with `[]` plausibleInterpretations — correct for this fixture's initial state). Unresolved customer-signing and competitor uncertainty remained distinct, unanswered, non-evidence in B. Neither call promoted unresolved material into supported understanding or plausible interpretation. + +**Contamination classification:** NONE +- No supported claim added or removed +- No qualification/certainty shift beyond minor phrasing variation (~17 chars) +- Customer-signing uncertainty: NOT promoted to evidence, NOT interpreted/answered +- Competitor uncertainty: NOT promoted to evidence, NOT interpreted/answered +- Unresolved not prioritised in either call + +**What this establishes:** Under controlled product-launch conditions (zero Findings), exposing unresolved future-dependent Open Questions as a separate NON-EVIDENCE section does not cause the model to use that uncertainty as evidence or merge it into plausible interpretation. The epistemic boundary holds for Qwen under this scenario shape. + +**What it does not establish:** Does not prove universality across other scenarios, other models, or when plausibleInterpretations are non-empty. Does not generalize to manufacturing scenarios (with findings). One scenario shape only. + +### Zero-Finding Report Experiment Case (Loop 2/3) ### Zero-Finding Report Experiment Case (Loop 2/3)