robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(confidence-engine): support zero-finding report experiment case

Browse Source 
- Add nullish coalescing on fix.findings ?? [] so absent findings
  array in a fixture is represented as zero Findings [] for the A/B
  input apparatus.

- Append product-launch second controlled case (live-product-launch-
  start-response.json) with zero Findings to the epistemic comparison
  test suite. All 34 tests pass deterministically.

- Update docs/current-handoff.md to record the zero-Findings case.
This commit is contained in:
robbond
2026-09-30 13:25:22 +01:00
parent d387b66c03
commit 8d268efb12
2 changed files with 132 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+11 -1
View File
@@ -295,11 +295,21 @@ tsx runtime — alias-capable experiment execut
import-only proof — prove startCase() import chain resolves without live calls
OpenAI reconstruction experiment seam — START_CASE_EXPERIMENT_PROVIDER=openai (experiment-only)
focused-deconstruction schema fix — plumbing verified, 48/48 tests pass
tests/experimental/report-epistemic-ab-input-control.test.js — deterministic A/B input control apparatus (17 tests, zero live calls)
tests/experimental/report-epistemic-ab-input-control.test.js — deterministic A/B input control apparatus (34 tests across two cases, zero live calls)
```
**No other experimental apparatus is currently valid.** Obsolete setups have been removed or superseded.
### Zero-Finding Report Experiment Case (Loop 2/3)
- `tests/fixtures/live-product-launch-start-response.json` is the valid second controlled case.
- Its reasoning shape: future-dependent commercial/actor uncertainty (enterprise customer contract timing + competitor activity).
- This fixture legitimately contains **zero Findings** — no findings array present in the start-response.
- Apparatus now represents absent `findings` as `[]` via `(fix.findings ?? [])` nullish coalescing on line 20 of the test file.
- The apparatus preserves the semantic distinction: supported evidence from graph ≠ Findings. An investigation can have supported graph evidence with zero canonical Findings.
- Both manufacturing (with findings) and product-launch (zero Findings) A/B controlled cases pass all determinism checks.
- **No live second-case A/B has yet been run.** The behavioural question — whether exposing unresolved uncertainty in the presence of zero Findings produces different model output than without — remains open.
## Current product architecture
Three distinct routes: