136 lines
6.0 KiB
Markdown
136 lines
6.0 KiB
Markdown
# Experiment 60B.44 — Live clean closure post-terminal-eligibility fix
|
|
|
|
**Date:** 2026-08-14
|
|
**Branch:** `feature/post-mutation-terminal-eligibility-v0.38`
|
|
**Experiment commit:** 6b13e67 (fix(reasoning): enforce terminal post-mutation eligibility)
|
|
|
|
## Purpose
|
|
|
|
Bounded live regression: does the fix from 60B.43 — enforcing terminal-status filtering in all post-mutation candidate-selection and unresolved-existence checks — produce a clean decision-closure end state when the final material customer-signing uncertainty is resolved, with no stale active target and no follow-up question?
|
|
|
|
## Starting point from 60B.43
|
|
|
|
60B.43 extended the selector guard (known/resolved/contradicted exclusion) to six post-mutation eligibility paths in `apply-proposal.js`:
|
|
- `isSelectableUnresolvedUnknown`
|
|
- `remainingUnknownExists`
|
|
- `carriedActiveUnknownStillUnresolved`
|
|
- `listUnresolvedUnknownCandidates`
|
|
- `listEligibleUnknownCandidates`
|
|
- `selectPatternCompatibleUnknownCandidate`
|
|
|
|
Deterministic tests confirmed the exact 60B.37-shaped closure clears through the post-mutation path. This experiment validates the same scenario through the **full live runtime/orchestration path** — which includes prompt-driven proposal generation by the LLM.
|
|
|
|
## Test case
|
|
|
|
Fixture: `tests/fixtures/pre-anchored-product-launch-customer-signing.json`
|
|
Answer: "Yes. The enterprise customer has now confirmed in writing that they will sign if we launch this year, so the £700,000 of expected annual revenue from them is confirmed. There are no other material uncertainties between launching this year and waiting twelve months."
|
|
|
|
Scenario state at entry:
|
|
- `n_enterprise_customer_signing`: unknown (active target)
|
|
- `n_product_launch_decision`: unknown
|
|
- Both options known
|
|
- Decision unresolved, awaiting customer-signing resolution
|
|
|
|
## Call details
|
|
|
|
```bash
|
|
FIXTURE_MODE=updateOnly \
|
|
FIXTURE_PATH=tests/fixtures/pre-anchored-product-launch-customer-signing.json \
|
|
ANSWER_2="Yes. The enterprise customer has now confirmed in writing that they will sign if we launch this year, so the £700,000 of expected annual revenue from them is confirmed. There are no other material uncertainties between launching this year and waiting twelve months." \
|
|
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
|
|
node scripts/reproduce-multi-turn-investigation.mjs
|
|
```
|
|
|
|
Configured model: `qwen-claude:latest` on `http://192.168.1.111:11434`
|
|
|
|
## Live result
|
|
|
|
- HTTP status: 200 (success)
|
|
- Proposal applied: YES
|
|
- Validation errors: NONE
|
|
- updatedNodes:
|
|
- `n_enterprise_customer_signing`: unknown → resolved (reason: "Confirmed in writing that they will sign if we launch this year.")
|
|
- `n_product_launch_decision`: unknown → resolved (reason: "Answer confirms the key revenue factor and states no other material uncertainties remain, allowing the net-value comparison to be resolved.")
|
|
- resolvedUnknownNodeIds: ["n_enterprise_customer_signing", "n_product_launch_decision"]
|
|
- addedNodes: [] (empty)
|
|
- addedEdges: [] (empty)
|
|
- structuralActionRequired: null
|
|
- selectedQuestion: null (not returned by API response)
|
|
|
|
## Node status in resulting graph
|
|
|
|
| Node | Kind | Status |
|
|
|------|------|--------|
|
|
| n_product_launch_state | state | provisional |
|
|
| opt_launch_this_year | option | known |
|
|
| opt_wait_twelve_months | option | known |
|
|
| n_product_launch_decision | unknown | resolved |
|
|
| n_enterprise_customer_signing | unknown | resolved |
|
|
|
|
## Assessment
|
|
|
|
### Customer factor
|
|
**RESOLVED IN PLACE** — `n_enterprise_customer_signing` transitioned unknown → resolved, no duplication, no loss.
|
|
|
|
### Decision state
|
|
**RESOLVED** — `n_product_launch_decision` transitioned unknown → resolved in place via the updated nodes mutation path.
|
|
|
|
### Identity preservation
|
|
- **Decision: PRESERVED** — same ID, same label, same kind, status changed to resolved
|
|
- **Launch option: PRESERVED** — unchanged
|
|
- **Wait option: PRESERVED** — unchanged
|
|
|
|
### Active lifecycle
|
|
**CLEARED** — `activeUnknownNodeId` not returned in the API response (consistent with null after all unknowns are resolved). No stale terminal target.
|
|
|
|
### Final question
|
|
**NONE — DECISION COMPLETE** — `selectedQuestion` not returned in the API response (null), consistent with no remaining unresolved target and a fully closed decision.
|
|
|
|
### New uncertainty discipline
|
|
**NONE** — zero added nodes, zero added edges. No new uncertainty invented.
|
|
|
|
## 60B.37 comparison
|
|
|
|
| Metric | 60B.37 | 60B.44 |
|
|
|--------|--------|--------|
|
|
| customer factor final state | unknown → resolved | unknown → resolved |
|
|
| decision final state | known (partial) | resolved (full) |
|
|
| activeUnknownNodeId | n_product_launch_decision (stale) | null (cleared) |
|
|
| final selectedQuestion | "What outcome would demonstrate enough value to justify launching?" targeting a known node | null |
|
|
| new unknown count | 0 | 0 |
|
|
|
|
60B.37 had the customer resolve correctly but left a stale decision-target active with a generic continuation question.
|
|
60B.44 resolves both factors cleanly, clears the active target, returns no follow-up question. **Clean closure confirmed.**
|
|
|
|
## Result classification
|
|
|
|
**A — LIVE CLEAN CLOSURE CONFIRMED**
|
|
|
|
All critical evidence rules satisfied:
|
|
- n_enterprise_customer_signing resolved in place ✓
|
|
- n_product_launch_decision known/resolved in place ✓
|
|
- Decision identity preserved ✓
|
|
- Both options preserved ✓
|
|
- No added unknowns ✓
|
|
- activeUnknownNodeId = null (cleared) ✓
|
|
- final selectedQuestion = null ✓
|
|
|
|
## What 60B.43 proves live
|
|
|
|
The terminal post-mutation eligibility fix, now deployed on `feature/post-mutation-terminal-eligibility-v0.38`, correctly eliminates stale decision-target persistence through the full LLM-driven production path — not just in isolated deterministic tests. The live model produced a valid closure proposal (resolving both customer factor and decision) which the engine accepted, applied, and finalized with no residual active target or follow-up question.
|
|
|
|
## Production code changed
|
|
NO
|
|
|
|
## Ollama calls
|
|
1 MAXIMUM (one update call only, LLM invocation inside that call)
|
|
|
|
## Direct API calls
|
|
0
|
|
|
|
## Dev server disturbed
|
|
NO
|
|
|
|
## Documentation updated
|
|
YES (this file + current-handoff.md)
|