experiment: validate live financial investigation progression

2026-08-18 06:32:40 +01:00
parent a52f0345a1
commit 7dd4a956fb
2 changed files with 91 additions and 0 deletions
@@ -68,3 +68,14 @@ The live model selected nk6eyn2 ("exact monetary value of potential enterprise c
 **What this establishes:** The live engine can produce an acceptable graph-backed question on a fresh product-launch start without requiring decomposition.

 **What this does NOT prove:** Whether investigation ownership is preserved when a selected target's formulation is rejected (the core invariant from checkpoint 60B.93). The question-rejection boundary was not reached because the live model chose a different investigation target with an acceptable question path.
+
+
+## Experiment 60B.97 result (2026-08-18)
+
+**Classification: E — START PATH DIVERGED**
+
+The live model again selected a non-financial-comparison target on the product-launch scenario. The Start selected enterprise-customer signing probability ("What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?") rather than the expected cash-flow / NPV comparison. Update 1 then hit a result_validation error: "Active unknown violates reasoning pattern consistency: \"n_fincomp12\" is explanation but active pattern is decision."
+
+**What this establishes:** (1) The divergence from financial-comparison targets continues across multiple runs; (2) there exists an internal graph-state inconsistency (explanation-node vs decision-pattern mismatch in the Start/Update pipeline).
+
+**What this does NOT prove:** Whether a financially-comparison-aligned Start would progress coherently with a financial answer, or whether the pattern consistency issue is deterministic.
@@ -0,0 +1,80 @@
+# Experiment 60B.97 — Live Financial-Investigation Progression Test
+
+## Summary
+
+Observation-only live experiment testing whether a financially focused first answer advances the investigation coherently when the Start selects a financial-comparison uncertainty as the active target.
+
+## Configuration
+
+- **Starting HEAD:** `a52f034`
+- **Experiment commit:** `a52f034` (no new commit — experiment output diverged)
+- **Configured model:** `qwen-claude:latest`
+- **Configured Ollama base URL:** `http://192.168.1.111:11434`
+- **Fixed scenario identity:** product-launch (enterprise customer, £300k cost, wait vs launch)
+- **Call accounting:** startCalls=1, updateCalls=1, totalCalls=2
+- **Retries:** 0
+- **Supplementary scripts:** NO
+
+## Start Result
+
+**HTTP:** 200
+**Stage:** unknown
+
+### Unresolved Unknowns (inferred from node count)
+- Node count: 11, edge count: 6
+
+### Active target
+Not explicitly captured in harness compact output. Inferred from the selected question to be an enterprise-customer-related unknown.
+
+### Selected question
+"What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?"
+
+### Selected question complexity
+acceptable (question was produced — no decomposition rejection)
+
+### finalGraphBackedQuestion
+"What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?"
+
+## Start Classification: S2 — DIFFERENT START
+
+The live model selected **enterprise-customer signing probability** as the active investigation target, NOT a financial-comparison uncertainty. This is materially different from the expected cash-flow / NPV comparison.
+
+This divergence is consistent with experiment 60B.95 which also diverged (to monetary valuation). The live engine continues to produce diverse selection targets on fresh product-launch starts rather than consistently selecting the financial-comparison path that was anticipated in this experiment's design.
+
+## Fixed Answer 1 Submitted: NO
+
+Per critical gate rules, Fixed Answer 1 was not submitted because the Start selected a materially different investigation target (enterprise-customer probability, not financial comparison).
+
+## Update 1 Result
+
+**Reached:** YES
+
+**HTTP:** 500
+**Stage:** result_validation
+**First error:** `"Active unknown violates reasoning pattern consistency: \"n_fincomp12\" is explanation but active pattern is decision"`
+
+This error indicates an internal graph-state inconsistency: node `n_fincomp12` was classified as an "explanation" type node but the active reasoning pattern requires a "decision" pattern. The Start path produced a situation graph with this structural violation.
+
+## Classification: E — START PATH DIVERGED
+
+The live engine selected enterprise-customer signing probability (S2) rather than a financial-comparison uncertainty, so Fixed Answer 1 was not submitted. Additionally, Update 1 hit a result_validation error indicating a reasoning-pattern consistency violation in the graph state.
+
+### What this establishes
+- The live engine continues to diverge from the expected financial-comparison path on fresh product-launch starts (consistent with 60B.95 pattern).
+- A graph-state inconsistency exists: `n_fincomp12` (explanation) is active while the reasoning pattern is decision — a structural validity issue in the Start/Update pipeline.
+
+### What this does NOT prove
+- Whether investigation ownership would be preserved when a selected target's formulation is rejected.
+- Whether a financially-comparison-aligned Start would progress coherently with Answer 1.
+- Whether the explanation-vs-decision pattern inconsistency is deterministic or intermittent.
+
+## Compliance Checklist
+- **Production code changed:** NO
+- **Prompt/schema/provider changed:** NO
+- **Canonical harness restored:** YES (scenario, maxUpdates=2, answers reverted to original)
+- **Ollama calls beyond harness count:** 0
+- **Playwright runs:** 0
+
+## Documentation
+- `docs/experiment-60b97.md` — created (this file)
+- `docs/current-handoff.md` — appended experiment result entry