238 lines
7.4 KiB
Markdown
238 lines
7.4 KiB
Markdown
# Experiment 60B.29 — Live Validation of Uncertainty Proposition Coverage
|
|
|
|
**Branch:** `feature/proposition-prefix-coverage-v0.31`
|
|
**Starting HEAD:** `f94d47d813fef0be062a263205a83fbedbafd7f3`
|
|
**Date:** 2026-08-14
|
|
**Status:** COMPLETE
|
|
|
|
## Objective
|
|
|
|
Run the exact bounded live continuation once to determine whether the full live path now preserves the explicit `whether ...` proposition for the product-launch customer-signing case after 60B.28's deterministic prefix-coverage extension.
|
|
|
|
## Configured Model
|
|
|
|
```text
|
|
qwen-claude:latest
|
|
```
|
|
|
|
## Configured Ollama Base URL
|
|
|
|
```text
|
|
http://192.168.1.111:11434
|
|
```
|
|
|
|
## Execution
|
|
|
|
Single committed harness invocation only:
|
|
|
|
```bash
|
|
FIXTURE_MODE=updateOnly \
|
|
FIXTURE_PATH=tests/fixtures/pre-anchored-product-launch-options.json \
|
|
ANSWER_2="The revenue and launch-cost estimates are good enough for the decision. The remaining issue is one prospective enterprise customer. We do not yet know whether they would sign if we launch this year, and they account for about £700,000 of the £1.2 million expected annual revenue." \
|
|
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
|
|
node scripts/reproduce-multi-turn-investigation.mjs
|
|
```
|
|
|
|
Call accounting from the same run:
|
|
|
|
```text
|
|
startCalls: 0
|
|
updateCalls: 1
|
|
totalCalls: 1
|
|
Retries: 0
|
|
Second live invocation: NO
|
|
```
|
|
|
|
## Recoverable First-Run Output
|
|
|
|
The recoverable terminal output from the single permitted invocation showed:
|
|
|
|
```text
|
|
updatedNodes: []
|
|
resolvedUnknownNodeIds: []
|
|
addedNodes: [
|
|
{
|
|
"id": "n_enterprise_customer_signing",
|
|
"label": "Prospective enterprise customer signing status",
|
|
"description": "Uncertainty over whether the prospective enterprise customer will sign if we launch this year, because their contract accounts for approximately £700,000 of the expected first-year revenue and could materially flip the net-value comparison.",
|
|
"kind": "unknown",
|
|
"status": "unknown",
|
|
"confidence": "medium",
|
|
"dependsOn": ["n_product_launch_decision"]
|
|
}
|
|
]
|
|
addedEdges: [
|
|
{
|
|
"id": "e-dec-to-customer-signing",
|
|
"fromNodeId": "n_product_launch_decision",
|
|
"toNodeId": "n_enterprise_customer_signing",
|
|
"relationship": "depends_on",
|
|
"confidence": "high"
|
|
}
|
|
]
|
|
structuralActionRequired: null
|
|
selectedQuestion: "What outcome would demonstrate enough value to justify launching?"
|
|
selectedQuestion.nodeId: "n_enterprise_customer_signing"
|
|
```
|
|
|
|
Resulting persistent graph from the same run:
|
|
|
|
```text
|
|
n_product_launch_decision remains unknown
|
|
n_enterprise_customer_signing exists as unknown/unknown
|
|
no unrelated new uncertainty appears
|
|
```
|
|
|
|
The recovered output did not include the earlier accepted-path `HTTP` and `stage` lines, so those values are not directly recoverable from the same captured artifact.
|
|
|
|
## Assessment
|
|
|
|
### Core reasoning chain
|
|
|
|
**PRESERVED**
|
|
|
|
- decision remains unresolved (`n_product_launch_decision` stays unknown)
|
|
- customer-signing factor survives as a first-class unknown (`n_enterprise_customer_signing`)
|
|
- material factor remains the final target (`selectedQuestion.nodeId = n_enterprise_customer_signing`)
|
|
- no unrelated uncertainty invented
|
|
|
|
### Customer-signing factor
|
|
|
|
**FIRST-CLASS UNKNOWN**
|
|
|
|
The live proposal created a dedicated unknown node with its own id, label, description, and structural edge.
|
|
|
|
### Prefix form actually exercised
|
|
|
|
**OTHER**
|
|
|
|
The live customer node description begins:
|
|
|
|
```text
|
|
Uncertainty over whether ...
|
|
```
|
|
|
|
This is not one of the newly supported 60B.28 prefixes (`Unknown whether...`, `Uncertain whether...`) and is also not one of the previously supported exact forms (`Uncertainty about whether...`, `Uncertainty regarding whether...`).
|
|
|
|
### Preferred target behaviour
|
|
|
|
**MATERIAL FACTOR PRESERVED**
|
|
|
|
The final target remained the customer-signing factor node.
|
|
|
|
### Full proposition preservation
|
|
|
|
**LOST**
|
|
|
|
The final question does not retain either of the required proposition components:
|
|
|
|
- `customer will sign`
|
|
- `if we launch this year`
|
|
|
|
Instead it asks a generic decision-threshold question:
|
|
|
|
```text
|
|
What outcome would demonstrate enough value to justify launching?
|
|
```
|
|
|
|
### Evidence framing
|
|
|
|
**WRONG**
|
|
|
|
The final question is neither:
|
|
|
|
- evidence-framed around the customer-signing proposition, nor
|
|
- a direct interrogative about customer signing,
|
|
|
|
but a generic launch-justification question despite the selected node being the customer-signing unknown.
|
|
|
|
### Rationale contamination
|
|
|
|
**NONE**
|
|
|
|
The final question contains no:
|
|
|
|
- `£700,000`
|
|
- `£1.2 million`
|
|
- `annual revenue`
|
|
- `financial impact`
|
|
- equivalent revenue rationale
|
|
|
|
### Source graph meaning
|
|
|
|
**SOURCE DESCRIPTION PRESERVED**
|
|
|
|
The live node description retains both the proposition and the explanatory rationale in graph state.
|
|
|
|
## 60B.27 Comparison
|
|
|
|
| Dimension | 60B.27 | 60B.29 |
|
|
| ------------------------ | ------------------------------------ | ------------------------------------- |
|
|
| Live node prefix | `Unknown whether...` | `Uncertainty over whether...` |
|
|
| Final target | customer-signing factor | customer-signing factor |
|
|
| Proposition preservation | nominalized, partial | lost in final question |
|
|
| Final question shape | evidence-framed nominalized question | generic launch-justification question |
|
|
| Rationale contamination | none | none |
|
|
|
|
60B.27 final question:
|
|
|
|
```text
|
|
What evidence would clarify prospective enterprise customer signing likelihood?
|
|
```
|
|
|
|
60B.29 final question:
|
|
|
|
```text
|
|
What outcome would demonstrate enough value to justify launching?
|
|
```
|
|
|
|
## Result Classification
|
|
|
|
### C — TARGET PRESERVED, PROPOSITION STILL PARTIAL
|
|
|
|
The live run preserved the correct material target and decision state, but the final question did not preserve the explicit customer-signing proposition at all.
|
|
|
|
This run does **not** confirm the 60B.28 prefix-extension live because the model did not produce either newly-supported prefix form. The live customer node used `Uncertainty over whether...`, so the new `Unknown whether...` / `Uncertain whether...` path was not exercised.
|
|
|
|
## Did 60B.28 Newly-Supported Prefix Handling Fire Live
|
|
|
|
**NO**
|
|
|
|
The live node description did not begin with `Unknown whether...` or `Uncertain whether...`.
|
|
|
|
## Did the Full Proposition Survive Live
|
|
|
|
**NO**
|
|
|
|
The final question lost both the signing action and the launch-condition clause.
|
|
|
|
## Did the Material Target Remain Stable
|
|
|
|
**YES**
|
|
|
|
The customer-signing factor remained the selected node id.
|
|
|
|
## What Improved Relative to 60B.27
|
|
|
|
- Nothing on proposition preservation can be claimed from this run.
|
|
- Rationale contamination remained absent.
|
|
- Material targeting remained stable.
|
|
|
|
## What Remains Weak or Unproven
|
|
|
|
- The exact 60B.28 live prefix extension remains unproven because the live node did not use `Unknown whether...` or `Uncertain whether...`.
|
|
- The full runtime path can still produce a generic final question even when the material customer-signing factor is selected.
|
|
- The divergence between selected target (`n_enterprise_customer_signing`) and generic final question wording remains unaddressed by this observation-only run.
|
|
|
|
## Production Boundary
|
|
|
|
Production code changed: **NO**
|
|
Prompt changed: **NO**
|
|
Validator changed: **NO**
|
|
Schema changed: **NO**
|
|
Harness changed: **NO**
|
|
Vitest run: **NO**
|
|
Ollama calls: **1 maximum**
|
|
Direct API calls: **0**
|
|
Dev server disturbed: **NO**
|