robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

test(harness): support gated live investigation continuation

Browse Source
This commit is contained in:
robbond
2026-08-18 06:48:40 +01:00
parent 7dd4a956fb
commit 600b07d820
4 changed files with 693 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/experiment-60b97.md
+18 -11
View File
@@ -47,26 +47,33 @@ Per critical gate rules, Fixed Answer 1 was not submitted because the Start sele
## Update 1 Result
**Reached:** YES
**DISCARDED** — The canonical harness auto-continued with its preconfigured `answers[0]`, so the Update occurred outside the experiment's semantic gate. This evidence is invalid for 60B.97 conclusions.
**HTTP:** 500
**Stage:** result_validation
**First error:** `"Active unknown violates reasoning pattern consistency: \"n_fincomp12\" is explanation but active pattern is decision"`
This error indicates an internal graph-state inconsistency: node `n_fincomp12` was classified as an "explanation" type node but the active reasoning pattern requires a "decision" pattern. The Start path produced a situation graph with this structural violation.
The HTTP 500 is NOT established as a reasoning defect from 60B.97.
## Classification: E — START PATH DIVERGED
The live engine selected enterprise-customer signing probability (S2) rather than a financial-comparison uncertainty, so Fixed Answer 1 was not submitted. Additionally, Update 1 hit a result_validation error indicating a reasoning-pattern consistency violation in the graph state.
Valid 60B.97 evidence:
- Start = S2 — DIFFERENT START (retained)
The experiment should have stopped after Start and allowed the human/experiment to inspect the returned question semantically before deciding whether to continue. The canonical harness did not provide this capability at time of 60B.97 execution, so the Update portion of 60B.97 is invalid evidence.
### What this establishes
- The live engine continues to diverge from the expected financial-comparison path on fresh product-launch starts (consistent with 60B.95 pattern).
- A graph-state inconsistency exists: `n_fincomp12` (explanation) is active while the reasoning pattern is decision — a structural validity issue in the Start/Update pipeline.
### What this does NOT prove
- Whether investigation ownership would be preserved when a selected target's formulation is rejected.
- Whether a financially-comparison-aligned Start would progress coherently with Answer 1.
- Whether the explanation-vs-decision pattern inconsistency is deterministic or intermittent.
- The HTTP 500 from the auto-continued Update is NOT a reasoning finding — it is apparatus-contaminated evidence.
## Apparatus correction (60B.99)
The canonical harness (`scripts/reproduce-multi-turn-investigation.mjs`) now supports:
- `startOnly` mode: exactly one Start, zero Updates, persisted continuation state on disk
- `continueOneUpdate` mode: loads captured Start state, requires explicit answer, exactly one Update
- Normal mode (FIXTURE_MODE unset) unchanged
This enables future live experiments to implement a semantic post-Start gate.
## Compliance Checklist
- **Production code changed:** NO
@@ -76,5 +83,5 @@ The live engine selected enterprise-customer signing probability (S2) rather tha
- **Playwright runs:** 0
## Documentation
- `docs/experiment-60b97.md` — created (this file)
- `docs/current-handoff.md` — appended experiment result entry
- `docs/experiment-60b97.md` — updated with apparatus correction note
- `docs/current-handoff.md` — appended experiment result entry + apparatus note