experiment: test do-nothing baseline representation

2026-08-12 17:42:25 +01:00
parent 70688f91c9
commit 2cd346423d
2 changed files with 273 additions and 1 deletions
@@ -345,7 +345,64 @@ The engine extracted all three semantic elements in userSupportedMeaning and cre

 ---

-### Experiment 59A.3 — Known vs Uncertain Consequence Structure
+### Experiment 59B.3 — Do-Nothing Baseline as Explicit Graph Structure
+
+**Branch:** `feature/question-formulation-v0.24`
+**Date:** 2026-08-12
+**Status:** Complete
+**Following:** 59B.2 which showed the engine creates a net-benefit trade-off but does-nothing baseline remains semantic (not structural). This makes both action and do-nothing consequences explicit in the answer to test whether the engine structurally represents both sides.
+
+**Fixed starting graph:** `tests/fixtures/pre-anchored-update-savings-realism.json`
+**Fixed answer (exact, verbatim):** "The £2 million annual saving from relocating is real. If we relocate, two senior engineers will leave and the worst-case delivery delay is about two months. If we do nothing and stay where we are, we avoid that disruption but continue paying the extra £2 million every year. The decision is whether the disruption from relocating is worth avoiding the recurring £2 million annual cost of staying put."
+
+**Execution:** qwen-claude:latest at http://192.168.1.111:11434. startCalls=0, updateCalls=1, totalCalls=1.
+
+**Result:** HTTP 200, stage = update_applied, no validation errors.
+
+```
+updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: resolved, newValue: "confirmed", reason: "User explicitly confirmed the £2 million annual saving from relocating is real."}]
+resolvedUnknownNodeIds: ["n_savings_realism"]
+
+addedNodes: [
+  {
+    id: n_disruption_impact,
+    label: "Operational impact of relocation disruption",
+    description: "Uncertainty regarding the precise cost and delay consequences of losing two senior engineers and facing a two-month delivery slowdown, so that it can be weighed against the confirmed £2M annual savings.",
+    kind: unknown,
+    status: unknown,
+    confidence: medium
+  }
+]
+
+addedEdges: [{fromNodeId: n_relocation_state, toNodeId: n_disruption_impact, relationship: causes}]
+
+selectedQuestion: "What would clarify operational impact of relocation disruption in this situation?"
+selectedQuestion.nodeId: "n_disruption_impact"
+```
+
+Resulting persistent graph (3 nodes, 2 edges):
+- `n_relocation_state` — Engineering team relocation consideration — status=provisional
+- `n_savings_realism` — Are the projected office savings from relocation realistic? — status=resolved
+- `n_disruption_impact` — Operational impact of relocation disruption — status=unknown, kind=unknown
+
+### Assessment
+
+1. **Relocation benefit:** PRESERVED ONLY IN TEXT — `newValue: "confirmed"` on resolved node captures acceptance but omits the figure (£2 million) and unit (annual). No structural £2m/year claim survives as evidence.
+2. **Two-engineer departure:** PARTIALLY REPRESENTED — embedded in n_disruption_impact's description ("losing two senior engineers") but not a separate observation/known fact node. Same class of bundling as 59A.1.
+3. **Bounded downside:** PARTIALLY REPRESENTED — embedded in same description ("two-month delivery delay") but not structurally separated from engineer departure.
+4. **Do-nothing recurring cost:** PRESERVED ONLY IN TEXT — "continue paying the extra £2 million every year" is absent from any structural node. Only exists implicitly within the trade-off framing of n_disruption_impact's description ("weighed against the confirmed £2M annual savings").
+5. **Do-nothing benefit:** PRESERVED ONLY IN TEXT — "we avoid that disruption" does not appear in any graph structure.
+6. **Alternative structure:** NO ALTERNATIVE STRUCTURE — only one action path (relocate) represented as a node with consequences; do-nothing option has no structural presence whatsoever.
+7. **Trade-off linkage:** PARTIALLY LINKED — the trade-off exists in n_disruption_impact's description text ("so that it can be weighed against the confirmed £2M annual savings") but both sides are not independently retrievable as graph nodes.
+8. **Next question:** WEAK — asks about disruption impact (one side of the comparison only), not about comparing both alternatives.
+
+**Classification: B — TRADE-OFF GOOD, BASELINE STILL IMPLICIT**
+
+The engine produced decision-relevant reasoning (trade-off framing) but the do-nothing baseline remains text/context rather than explicit graph structure. This is the same pattern as 59B.2 and the same gap identified in this experiment's stated objective.
+
+Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Appending to handoff.
+
+---

 **Branch:** `feature/question-formulation-v0.24`
 **Date:** 2026-08-12