160 lines
7.0 KiB
Markdown
160 lines
7.0 KiB
Markdown
# Experiment 60B.56 — Negative Customer-Signing Clean Closure (Live)
|
|
|
|
**Date:** 2026-08-14
|
|
**Branch:** `feature/closure-selection-reconciliation-v0.41`
|
|
**Head commit:** 54e2e21 fix(reasoning): reconcile closure selection state
|
|
|
|
## Objective
|
|
|
|
Does the negative customer-signing outcome now close cleanly live — i.e., does the full production runtime resolve the same customer factor and decision with no stale active target or follow-up question?
|
|
|
|
## Hypothesis (from committed deterministic fix)
|
|
|
|
```
|
|
updated unknown -> resolved
|
|
=> mirrored into resolvedUnknownNodeIds
|
|
|
|
selectedQuestion targeting same-turn resolved node
|
|
=> cleared before strict validation
|
|
|
|
if no genuine unresolved unknown remains
|
|
=> activeUnknownNodeId = null
|
|
=> selectedQuestion = null
|
|
```
|
|
|
|
## Input
|
|
|
|
- **Fixture:** `tests/fixtures/pre-anchored-product-launch-customer-signing.json`
|
|
- Pre-anchored state: decision (`n_product_launch_decision`) in unknown status; enterprise customer signing (`n_enterprise_customer_signing`) in unknown status, activeUnknownNodeId = n_enterprise_customer_signing.
|
|
- **Answer:** "No. The enterprise customer has now confirmed in writing that they will not sign if we launch this year, so the £700,000 of expected annual revenue from them will not be received. There are no other material uncertainties between launching this year and waiting twelve months."
|
|
|
|
## Configured environment
|
|
|
|
- **Model:** qwen-claude:latest
|
|
- **Ollama base URL:** http://192.168.1.111:11434
|
|
- **Confidence Engine base URL:** http://127.0.0.1:3000
|
|
|
|
## Run
|
|
|
|
```bash
|
|
FIXTURE_MODE=updateOnly \
|
|
FIXTURE_PATH=tests/fixtures/pre-anchored-product-launch-customer-signing.json \
|
|
ANSWER_2="No. The enterprise customer has now confirmed in writing that they will not sign if we launch this year, so the £700,000 of expected annual revenue from them will not be received. There are no other material uncertainties between launching this year and waiting twelve months." \
|
|
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
|
|
node scripts/reproduce-multi-turn-investigation.mjs
|
|
```
|
|
|
|
- **startCalls:** 0
|
|
- **updateCalls:** 1
|
|
- **totalCalls:** 1
|
|
- **Retries:** 0
|
|
|
|
## Results
|
|
|
|
### Proposal accepted: YES (HTTP 200)
|
|
|
|
### updatedNodes:
|
|
```json
|
|
[{"nodeId":"n_enterprise_customer_signing","previousStatus":"unknown","newStatus":"resolved","previousValue":null,"newValue":null,"reason":"User confirmed in writing the customer will not sign if launched this year, resolving the active material uncertainty."}]
|
|
```
|
|
|
|
### resolvedUnknownNodeIds:
|
|
```json
|
|
["n_enterprise_customer_signing"]
|
|
```
|
|
|
|
### addedNodes:
|
|
```json
|
|
[]
|
|
```
|
|
|
|
### addedEdges:
|
|
```json
|
|
[]
|
|
```
|
|
|
|
### customer node final state:
|
|
- `n_enterprise_customer_signing`: status = **resolved**
|
|
|
|
### customer resolution meaning:
|
|
"User confirmed in writing the customer will not sign if launched this year, resolving the active material uncertainty." → Negative meaning **preserved**.
|
|
|
|
### decision node final state:
|
|
- `n_product_launch_decision`: status = **unknown** (still open)
|
|
|
|
### launch option final state:
|
|
- `opt_launch_this_year`: status = known
|
|
|
|
### wait option final state:
|
|
- `opt_wait_twelve_months`: status = known
|
|
|
|
### DIRECT CLOSURE METADATA
|
|
|
|
```
|
|
finalActiveUnknownNodeId: "n_product_launch_decision"
|
|
finalSelectedQuestion: {"nodeId":"n_product_launch_decision","question":"What outcome would demonstrate enough value to justify launching?","reason":"Formulated from graph context using the decision_threshold investigation strategy.",...}
|
|
```
|
|
|
|
## Assessment
|
|
|
|
### Customer factor: RESOLVED IN PLACE
|
|
The enterprise-customer-signing node was updated in place from `unknown` → `resolved`.
|
|
|
|
### Negative meaning: PRESERVED
|
|
The resolution reason explicitly states "customer will not sign" — the negative meaning is intact.
|
|
|
|
### Decision state: KEPT OPEN FOR SPECIFIC MATERIAL REASON
|
|
`n_product_launch_decision` remains `status=unknown` with `activeUnknownNodeId = n_product_launch_decision` and a non-null `selectedQuestion` targeting it. The fix's goal of closing the decision when all its dependency unknowns resolve was **not achieved**.
|
|
|
|
### Identity preservation:
|
|
- Decision node: PRESERVED
|
|
- Launch option: PRESERVED
|
|
- Wait option: PRESERVED
|
|
|
|
### Active lifecycle: GENUINE UNRESOLVED TARGET (but arguably stale)
|
|
`n_product_launch_decision` is still the active target. It has no remaining dependent unknowns — both `opt_launch_this_year` and `opt_wait_twelve_months` are known. Its resolution depends on evaluating the remaining evidence, which was the point of having the customer-signing unknown as a dependency.
|
|
|
|
### Final question: SPECIFIC MATERIAL FOLLOW-UP
|
|
The engine formulated a decision_threshold question ("What outcome would demonstrate enough value to justify launching?") targeting `n_product_launch_decision`.
|
|
|
|
### New uncertainty discipline: NONE (no new nodes created)
|
|
|
|
## 60B.47 comparison
|
|
|
|
| Field | 60B.47 | 60B.56 |
|
|
|---|---|---|
|
|
| Proposal accepted | NO (422 proposal_compatibility) | YES |
|
|
| resolvedUnknownNodeIds | UNAVAILABLE | ["n_enterprise_customer_signing"] |
|
|
| Customer final state | UNAVAILABLE | RESOLVED |
|
|
| Decision final state | UNAVAILABLE | UNKNOWN (kept open) |
|
|
| finalActiveUnknownNodeId | UNAVAILABLE | "n_product_launch_decision" |
|
|
| finalSelectedQuestion | UNAVAILABLE | non-null (decision_threshold) |
|
|
|
|
**Progress from 60B.47 → 60B.56:** The proposal-compatibility validation bug is fixed — the update is accepted. However, the clean-closure contract was not met.
|
|
|
|
## Classification: D — GRAPH CLOSES BUT CONVERSATION DOES NOT
|
|
|
|
The customer-signing factor resolves correctly in place, and negative meaning is preserved. No nodes or edges are added. But `finalActiveUnknownNodeId` is non-null (`"n_product_launch_decision"`) and `finalSelectedQuestion` is non-null (a decision_threshold question). The graph-level closure of the dependency succeeded, but the parent decision node was not resolved — it remains open with a new follow-up question rather than closing.
|
|
|
|
## What this proves
|
|
|
|
1. **The proposal-compatibility validation bug is fixed.** Experiment 60B.47's 422 rejection no longer occurs.
|
|
2. **Customer-signing resolves in place** with the correct status transition and meaning preserved.
|
|
3. **No spurious graph mutations** — zero addedNodes, zero addedEdges.
|
|
|
|
## What remains weak or unproven
|
|
|
|
1. **Decision-node auto-resolution when all dependencies resolve.** The deterministic fix's primary goal was to close `n_product_launch_decision` when its only dependency (`n_enterprise_customer_signing`) resolves. This did not happen.
|
|
2. **selectedQuestion handling after full resolution.** When the sole unresolved unknown in a decision context is resolved, the system should produce null for both `activeUnknownNodeId` and `selectedQuestion`. Instead, it generated a new investigation question targeting the decision node itself.
|
|
3. **The clean-closure contract** (null → null on all known options with no remaining unknowns) remains unverified in live runs.
|
|
|
|
## Production code changed: NO
|
|
## Prompt changed: NO
|
|
## Validator changed: NO
|
|
## Schema changed: NO
|
|
## Harness changed during experiment: NO
|
|
## Vitest run: NO
|
|
## Ollama calls: 1 MAXIMUM
|
|
## Direct API calls: 0
|
|
## Dev server disturbed: NO
|