102 lines
6.2 KiB
Markdown
102 lines
6.2 KiB
Markdown
# Experiment 60B.95 — Live Product-Launch Start: Question-Rejection Ownership
|
|
|
|
## Summary
|
|
|
|
Observation-only live experiment testing whether the confidence engine preserves investigation ownership when a selected enterprise-customer uncertainty cannot produce an acceptable question on a fresh product-launch start.
|
|
|
|
## Configuration
|
|
|
|
- **Starting HEAD:** `7685a4f`
|
|
- **Experiment commit:** `7685a4f` (no new commit — experiment output diverged from deterministic capture)
|
|
- **Configured model:** `qwen-claude:latest`
|
|
- **Configured Ollama base URL:** `http://192.168.1.111:11434`
|
|
- **Fixed scenario identity:** product-launch (one large enterprise customer, £300k additional cost, wait vs launch)
|
|
- **Call accounting:** startCalls=1, updateCalls=0, totalCalls=1
|
|
- **Retries:** 0
|
|
- **Supplementary scripts:** NO
|
|
|
|
## Start Ownership Evidence
|
|
|
|
**HTTP:** 200
|
|
**Stage:** unknown (not present in response)
|
|
**First error:** none
|
|
|
|
### Central Statement
|
|
"I am deciding whether to launch a new software product this year or wait twelve months. The product is ready enough to launch, but one large enterprise customer could represent a significant part of the expected revenue and I do not yet know whether they will sign. Launching this year would also require around £300,000 of additional support and implementation cost. Waiting twelve months would reduce that immediate cost and give us more time to improve the product, but it would delay revenue and may allow competitors to move first. I need to decide whether there is enough evidence to launch this year or whether waiting is the safer decision."
|
|
|
|
### Unresolved Unknowns
|
|
- id: `npzfx36` — label: "The likelihood, negotiation stage, and targeted signing date for the large enterprise customer" (ENTERPRISE-CUSTOMER)
|
|
- id: `nk6eyn2` — label: "The exact monetary value of the potential enterprise contract relative to the £300k launch cost" (OTHER)
|
|
- id: `nn03k45` — label: "The probability and timeline for competitors to release a comparable product within the next twelve months" (COMPETITOR)
|
|
|
|
### Active Unknown
|
|
- id: `nk6eyn2`
|
|
- label: "The exact monetary value of the potential enterprise contract relative to the £300k launch cost"
|
|
|
|
### selectedUnknownNodeId
|
|
- id: `nk6eyn2`
|
|
- meaning: OTHER (monetary valuation, not probability/status)
|
|
|
|
### Deterministic Selection
|
|
Not directly exposed as `deterministicSelection.selectedNodeId` in the live response. The response structure uses `diagnostics.unknownSelectionExplanation.selected.nodeId` — this path was not captured by the harness diagnostic extraction (it returned "N/A" because the field name mismatch). Based on the overall response, deterministic selection also points to `nk6eyn2`.
|
|
|
|
### selectedQuestion
|
|
- nodeId: `nk6eyn2`
|
|
- selectedQuestionTemplate: `decision_threshold_outcome`
|
|
- question: "What outcome would demonstrate enough value to justify launching a software product now?"
|
|
- questionComplexity.acceptable: true
|
|
|
|
### selectedContainerUnknown: null
|
|
### selectedChildUnknown: nk6eyn2
|
|
|
|
### decompositionRequired: false
|
|
### decompositionAttempted: false
|
|
### decompositionAccepted: UNAVAILABLE
|
|
### decompositionStoppedReason: UNAVAILABLE
|
|
|
|
### finalGraphBackedQuestion
|
|
"What outcome would demonstrate enough value to justify launching a software product now?"
|
|
|
|
### noQuestionReason: null
|
|
|
|
## Ownership Analysis
|
|
|
|
**Active target meaning:** OTHER (monetary valuation of enterprise contract)
|
|
**Selected target meaning:** OTHER (same node nk6eyn2)
|
|
**Question target meaning:** OTHER (same node nk6eyn2, question about value justification)
|
|
**Backend ownership coherent:** YES (all three point to same unknown nk6eyn2)
|
|
**Question-rejection boundary reached:** NO
|
|
**Did question rejection transfer ownership:** UNPROVEN
|
|
|
|
## Classification: E — LIVE PATH DIVERGED
|
|
|
|
The live model reconstruction on a fresh start produced:
|
|
1. **Three** unresolved unknowns (not two as in the deterministic capture). The live model introduced nk6eyn2 (monetary valuation) as an additional unknown alongside npzfx36 (enterprise customer signing probability).
|
|
2. Selected `nk6eyn2` (OTHER — monetary value) rather than `npzfx36` (ENTERPRISE-CUSTOMER — probability/status).
|
|
3. Produced an **acceptable** question for nk6eyn2, bypassing the decomposition/rejection boundary entirely.
|
|
|
|
The live path diverged before reaching the question-rejection boundary. The selected unknown nk6eyn2 ("exact monetary value of potential enterprise contract relative to £300k launch cost") is materially different from the deterministic capture's target npzfx36/ntpt9ki ("probability or current status of the large enterprise customer signing").
|
|
|
|
This divergence is not automatically a regression — it could reflect legitimate model behavior where the live LLM identified monetary valuation as the strongest investigative priority. However, it means the key ownership-preservation question under rejection conditions was not tested in this run.
|
|
|
|
## What this establishes
|
|
- The live engine can produce an acceptable graph-backed question on a fresh product-launch start without requiring decomposition.
|
|
- Backend ownership is coherent within the selected node (no mismatch between activeUnknownNodeId, selectedUnknownNodeId, and selectedQuestion.nodeId).
|
|
- The response path for acceptable-question starts functions correctly through HTTP.
|
|
|
|
## What this does NOT prove
|
|
- Whether investigation ownership is preserved when a selected target's formulation is rejected (the core invariant from checkpoint 60B.93).
|
|
- Whether the live engine would produce decompositionRequired=true for npzfx36 (the enterprise-customer probability target) in scenarios where that uncertainty remains the strongest selection.
|
|
- The deterministic capture's two-unknown structure vs this three-unknown structure — whether the additional unknown is a regression or legitimate model interpretation.
|
|
|
|
## Compliance Checklist
|
|
- **Production code changed:** NO
|
|
- **Prompt/schema/provider changed:** NO
|
|
- **Canonical harness restored:** YES (scenario, maxUpdates=0 → 2, answers=[], diagnostic capture code reverted)
|
|
- **Ollama calls beyond harness count:** 1 (exactly one Start call)
|
|
- **Playwright runs:** 0
|
|
|
|
## Documentation
|
|
- `docs/experiment-60b95.md` — created (this file)
|
|
- `docs/current-handoff.md` — appended experiment result entry
|