197 lines
9.5 KiB
Markdown
197 lines
9.5 KiB
Markdown
# Experiment 60A.8 — Downstream Option Evidence Update on Committed Fixture
|
|
|
|
**Branch:** `feature/decision-options-v0.25`
|
|
**Date:** 2026-08-13
|
|
**Status:** Complete
|
|
**Type:** LIVE RUN — Single bounded update to test whether the engine attaches new option-specific evidence to the correct existing option without rebuilding the decision.
|
|
|
|
## Objective
|
|
|
|
When new information applies specifically to the Relocate option ("£400,000 lost margin from two-month delivery delay"), does the engine attach that information to the existing Relocate branch while preserving the existing Stay-put option and shared decision context?
|
|
|
|
## Hypothesis
|
|
|
|
A strong result should:
|
|
- Preserve the existing decision-context unknown (n_relocation_decision)
|
|
- Preserve the existing Relocate option identity (opt_relocate)
|
|
- Preserve the existing Stay-put option identity (opt_stay_put)
|
|
- Represent the £400k lost-margin information as belonging to Relocate
|
|
- Avoid creating duplicate Relocate / Stay-put options
|
|
- Keep the overall decision unresolved
|
|
|
|
## Fixed Starting Graph
|
|
|
|
**Fixture:** `tests/fixtures/pre-anchored-decision-options.json`
|
|
|
|
Pre-existing structure (4 nodes, 2 edges):
|
|
| Node | Kind | Status | Label |
|
|
|------|------|--------|-------|
|
|
| n_relocation_state | state | provisional | Engineering team relocation consideration |
|
|
| opt_relocate | option | known | Relocate to Manchester |
|
|
| opt_stay_put | option | known | Stay in London (Status Quo) |
|
|
| n_relocation_decision | unknown | unknown | Which option leaves us better off overall? |
|
|
|
|
Edges: opt_relocate → n_relocation_decision (contained_in); opt_stay_put → n_relocation_decision (contained_in).
|
|
|
|
## Configured Model
|
|
|
|
- **Model:** qwen-claude:latest
|
|
- **Ollama base URL:** http://192.168.1.111:11434
|
|
|
|
## Execution
|
|
|
|
Host/model: qwen-claude:latest at http://192.168.1.111:11434. startCalls=0, updateCalls=1, totalCalls=1. Hard one-call boundary.
|
|
|
|
**Fixed Answer (verbatim):**
|
|
> We have now estimated that the two-month delivery delay from relocating would cost about £400,000 in lost margin. That cost applies only to the relocation option. I am still deciding whether relocating or staying put leaves us better off overall.
|
|
|
|
## Result
|
|
|
|
**HTTP status:** 200 — first call succeeds, no validation rejection.
|
|
|
|
### Proposal mutations
|
|
|
|
```
|
|
updatedNodes: []
|
|
resolvedUnknownNodeIds: []
|
|
|
|
addedNodes: [{
|
|
id: "n_delay_cost",
|
|
label: "Lost margin from relocation delay",
|
|
description: "Estimated £400,000 lost margin due to two-month delivery delay if relocated.",
|
|
kind: "metric",
|
|
status: "known",
|
|
confidence: "high",
|
|
value: 400000,
|
|
unit: "GBP",
|
|
evidenceIds: [],
|
|
dependsOn: [],
|
|
affects: [],
|
|
parentId: null,
|
|
childIds: ["opt_relocate"]
|
|
}]
|
|
|
|
addedEdges: [{
|
|
id: "e-cost-to-opt-relocate",
|
|
fromNodeId: "n_delay_cost",
|
|
toNodeId: "opt_relocate",
|
|
relationship: "measures",
|
|
confidence: "high",
|
|
description: "Quantifies the financial impact of the relocation option's delay."
|
|
}]
|
|
|
|
selectedQuestion.nodeId: "n_relocation_decision"
|
|
selectedQuestion: "What outcome would demonstrate enough value to justify continuing?"
|
|
```
|
|
|
|
### Resulting persistent graph (5 nodes, 3 edges)
|
|
|
|
| Node | Kind | Status | Label |
|
|
|------|------|--------|-------|
|
|
| n_relocation_state | state | provisional | Engineering team relocation consideration |
|
|
| opt_relocate | option | **known** | Relocate to Manchester |
|
|
| opt_stay_put | option | **known** | Stay in London (Status Quo) |
|
|
| n_relocation_decision | unknown | **unknown** | Which option leaves us better off overall? |
|
|
| n_delay_cost | **metric** | **known** | Lost margin from relocation delay |
|
|
|
|
| Edge | From | To | Relationship |
|
|
|------|------|----|-------------|
|
|
| e-opt-rel-to-dec | opt_relocate | n_relocation_decision | contained_in |
|
|
| e-opt-stay-to-dec | opt_stay_put | n_relocation_decision | contained_in |
|
|
| e-cost-to-opt-relocate | n_delay_cost | opt_relocate | measures |
|
|
|
|
## Assessment
|
|
|
|
### 1. Decision identity: PRESERVED
|
|
|
|
The original `n_relocation_decision` node survived untouched — same id, label "Which option leaves us better off overall?", status=unknown. Exactly one decision-context unknown. No duplicate created.
|
|
|
|
### 2. Relocate identity: PRESERVED
|
|
|
|
`opt_relocate` survived unchanged — kind=option, status=known, label="Relocate to Manchester". Not updated, not replaced, not duplicated.
|
|
|
|
### 3. Stay-put identity: PRESERVED
|
|
|
|
`opt_stay_put` survived unchanged — kind=option, status=known, label="Stay in London (Status Quo)". Not updated, not replaced, not duplicated.
|
|
|
|
### 4. £400k consequence: FIRST-CLASS STRUCTURE
|
|
|
|
The engine created `n_delay_cost` as a dedicated metric node with:
|
|
- **value:** 400000 (numeric, not prose)
|
|
- **unit:** "GBP" (structured unit field)
|
|
- **kind:** "metric"
|
|
- **status:** "known"
|
|
- **label:** "Lost margin from relocation delay"
|
|
- **description:** "Estimated £400,000 lost margin due to two-month delivery delay if relocated."
|
|
|
|
This is first-class graph structure with typed edges and numeric value — not description-only or embedded text.
|
|
|
|
### 5. Option ownership: CLEARLY OWNED BY RELOCATE
|
|
|
|
The `measures` edge connects n_delay_cost → opt_relocate. The `childIds` field on n_delay_cost contains ["opt_relocate"]. Both the edge relationship and the child reference unambiguously tie this metric to the Relocate option, not Stay-put. Graph-only reasoning can determine: £400k belongs only to Relocate.
|
|
|
|
### 6. Existing contained_in structure: BOTH PRESERVED
|
|
|
|
Both pre-existing edges remain intact:
|
|
- opt_relocate → n_relocation_decision (contained_in) ✓
|
|
- opt_stay_put → n_relocation_decision (contained_in) ✓
|
|
|
|
No edges were removed or altered.
|
|
|
|
### 7. Decision state: CORRECTLY REMAINS UNRESOLVED
|
|
|
|
`n_relocation_decision.status` is still "unknown". `resolvedUnknownNodeIds` is empty. The user's continued indecision ("I am still deciding") was correctly preserved — the engine did not prematurely resolve the overall decision.
|
|
|
|
### 8. Duplication: NO DUPLICATION
|
|
|
|
| Entity | Count | Node IDs |
|
|
|--------|-------|----------|
|
|
| Relocate option | 1 | opt_relocate |
|
|
| Stay-put option | 1 | opt_stay_put |
|
|
| Overall decision | 1 | n_relocation_decision |
|
|
|
|
### 9. Selected question: GOOD
|
|
|
|
"What outcome would demonstrate enough value to justify continuing?" targets `n_relocation_decision`. This is a genuinely decision-relevant next question — it pursues the missing trade-off evaluation rather than recreating already-known structure. It acknowledges that the cost figure has been added but net-value comparison still requires assessment.
|
|
|
|
## Classification: A — EXISTING OPTION GRAPH UPDATED CORRECTLY
|
|
|
|
The engine preserved all existing option identities and decision context, created a first-class metric node for the £400k consequence correctly owned by Relocate via both edge relationship (`measures`) and child reference (`childIds: ["opt_relocate"]`), added no duplicates to any entity, and kept the overall decision unresolved. This is a strong confirmation that downstream option evidence attaches cleanly to existing options without rebuilding the decision.
|
|
|
|
## What the engine understood correctly:
|
|
|
|
1. **Evidence ownership:** The £400k lost-margin fact belongs to Relocate specifically — represented via a `measures` edge from metric → opt_relocate and childIds containing only "opt_relocate".
|
|
2. **Non-resolution of overall decision:** Despite new evidence being added, the engine correctly kept n_relocation_decision unresolved. The user's continued indecision was respected.
|
|
3. **No option duplication:** Existing opt_relocate and opt_stay_put survived untouched — no duplicate nodes created for either option.
|
|
4. **First-class representation:** The consequence was not relegated to prose/description. It is a numeric metric node with value=400000, unit="GBP", kind="metric".
|
|
5. **Edge topology preserved:** Both original contained_in edges remain intact alongside the new measures edge.
|
|
|
|
## What it did NOT do:
|
|
|
|
1. **Did not update existing option nodes** — opt_relocate was added-to via a child reference but its own node fields were not modified (updatedNodes=[])
|
|
2. **Did not create dependent unknowns** — no new uncertainty nodes were generated from the consequence; the metric is stated as known
|
|
3. **Did not resolve n_savings_realism** — there was no such node in this fixture (this was a clean decision-options context, not savings-realism)
|
|
|
|
## What this establishes:
|
|
|
|
1. The engine can add first-class structural evidence (numeric metric nodes with typed edges) to an existing option branch in a single update call.
|
|
2. Evidence ownership by the correct option is achievable via both edge relationships and child references — enabling graph-only reasonability.
|
|
3. Downstream option evidence does not force premature resolution of the overall decision context.
|
|
4. Existing option identities are preserved without duplication during evidence updates.
|
|
|
|
## What this does NOT prove:
|
|
|
|
1. **Stability across repeated runs** — one run only; cold-start variance may produce different outcomes on repeated runs.
|
|
2. **Consequence directionality semantics** — `measures` edge goes from metric → opt_relocate; the semantic direction (cost as a property of the option vs. the option being measured by the cost) is correct but untested for alternative relationship types.
|
|
3. **Cross-domain generalisation** — single domain case only.
|
|
4. **Multiple consequences per option** — tested with one consequence fact; multiple concurrent facts on the same option were not tested.
|
|
|
|
## Production code changed: NO
|
|
## Prompt changed: NO
|
|
## Validator changed: NO
|
|
## Harness changed: NO
|
|
## Vitest run: NO
|
|
## Ollama calls: 1
|
|
## Direct API calls: 0
|
|
## Dev server disturbed: NO
|