Files
confidence-engine/docs/experiment-60b97.md
T

4.0 KiB

Experiment 60B.97 — Live Financial-Investigation Progression Test

Summary

Observation-only live experiment testing whether a financially focused first answer advances the investigation coherently when the Start selects a financial-comparison uncertainty as the active target.

Configuration

  • Starting HEAD: a52f034
  • Experiment commit: a52f034 (no new commit — experiment output diverged)
  • Configured model: qwen-claude:latest
  • Configured Ollama base URL: http://192.168.1.111:11434
  • Fixed scenario identity: product-launch (enterprise customer, £300k cost, wait vs launch)
  • Call accounting: startCalls=1, updateCalls=1, totalCalls=2
  • Retries: 0
  • Supplementary scripts: NO

Start Result

HTTP: 200 Stage: unknown

Unresolved Unknowns (inferred from node count)

  • Node count: 11, edge count: 6

Active target

Not explicitly captured in harness compact output. Inferred from the selected question to be an enterprise-customer-related unknown.

Selected question

"What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?"

Selected question complexity

acceptable (question was produced — no decomposition rejection)

finalGraphBackedQuestion

"What evidence would clarify probability or likelihood that the enterprise customer will sign within the current launch window?"

Start Classification: S2 — DIFFERENT START

The live model selected enterprise-customer signing probability as the active investigation target, NOT a financial-comparison uncertainty. This is materially different from the expected cash-flow / NPV comparison.

This divergence is consistent with experiment 60B.95 which also diverged (to monetary valuation). The live engine continues to produce diverse selection targets on fresh product-launch starts rather than consistently selecting the financial-comparison path that was anticipated in this experiment's design.

Fixed Answer 1 Submitted: NO

Per critical gate rules, Fixed Answer 1 was not submitted because the Start selected a materially different investigation target (enterprise-customer probability, not financial comparison).

Update 1 Result

Reached: YES

HTTP: 500 Stage: result_validation First error: "Active unknown violates reasoning pattern consistency: \"n_fincomp12\" is explanation but active pattern is decision"

This error indicates an internal graph-state inconsistency: node n_fincomp12 was classified as an "explanation" type node but the active reasoning pattern requires a "decision" pattern. The Start path produced a situation graph with this structural violation.

Classification: E — START PATH DIVERGED

The live engine selected enterprise-customer signing probability (S2) rather than a financial-comparison uncertainty, so Fixed Answer 1 was not submitted. Additionally, Update 1 hit a result_validation error indicating a reasoning-pattern consistency violation in the graph state.

What this establishes

  • The live engine continues to diverge from the expected financial-comparison path on fresh product-launch starts (consistent with 60B.95 pattern).
  • A graph-state inconsistency exists: n_fincomp12 (explanation) is active while the reasoning pattern is decision — a structural validity issue in the Start/Update pipeline.

What this does NOT prove

  • Whether investigation ownership would be preserved when a selected target's formulation is rejected.
  • Whether a financially-comparison-aligned Start would progress coherently with Answer 1.
  • Whether the explanation-vs-decision pattern inconsistency is deterministic or intermittent.

Compliance Checklist

  • Production code changed: NO
  • Prompt/schema/provider changed: NO
  • Canonical harness restored: YES (scenario, maxUpdates=2, answers reverted to original)
  • Ollama calls beyond harness count: 0
  • Playwright runs: 0

Documentation

  • docs/experiment-60b97.md — created (this file)
  • docs/current-handoff.md — appended experiment result entry