docs(confidence-engine): record product-launch epistemic A/B result — 4-iteration loop complete
Controlled A/B (zero-Findings scenario, Qwen): unresolved uncertainty exposure did not contaminate supported understanding or plausible interpretation. No production changes. No report semantic changes.
This commit is contained in:
+16
-1
@@ -309,7 +309,22 @@ scripts/overview-experiment-helper.cjs — canonical overview live helper
|
||||
- **visible**: appends unresolved questions as separate NON-EVIDENCE section to baseline prompt
|
||||
- Apparatus invariant proven deterministically: A/B evidence, findings, plausibleInterpretations, scenario state are identical; only unresolved visibility differs.
|
||||
- Production `lib/graph/investigation-overview-synthesis.js` unchanged.
|
||||
- **No live A/B calls executed yet.** Next boundary: Call A (hidden) with product-launch fixture.
|
||||
- **Controlled A/B completed:** product-launch zero-Findings case, 2 live calls (51s hidden, 54s visible).
|
||||
|
||||
**Result:** Supported understanding materially unchanged; both calls produce the same core finding (product ready enough for launch, one large enterprise client relevant). No plausible interpretations in either variant (helper called with `[]` plausibleInterpretations — correct for this fixture's initial state). Unresolved customer-signing and competitor uncertainty remained distinct, unanswered, non-evidence in B. Neither call promoted unresolved material into supported understanding or plausible interpretation.
|
||||
|
||||
**Contamination classification:** NONE
|
||||
- No supported claim added or removed
|
||||
- No qualification/certainty shift beyond minor phrasing variation (~17 chars)
|
||||
- Customer-signing uncertainty: NOT promoted to evidence, NOT interpreted/answered
|
||||
- Competitor uncertainty: NOT promoted to evidence, NOT interpreted/answered
|
||||
- Unresolved not prioritised in either call
|
||||
|
||||
**What this establishes:** Under controlled product-launch conditions (zero Findings), exposing unresolved future-dependent Open Questions as a separate NON-EVIDENCE section does not cause the model to use that uncertainty as evidence or merge it into plausible interpretation. The epistemic boundary holds for Qwen under this scenario shape.
|
||||
|
||||
**What it does not establish:** Does not prove universality across other scenarios, other models, or when plausibleInterpretations are non-empty. Does not generalize to manufacturing scenarios (with findings). One scenario shape only.
|
||||
|
||||
### Zero-Finding Report Experiment Case (Loop 2/3)
|
||||
|
||||
### Zero-Finding Report Experiment Case (Loop 2/3)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user