experiment: diagnose uncertainty identity vs relatedness

docs/current-handoff.md

@@ -191,6 +191,12 @@ Two-turn run (fixed scenario: "We are considering relocating the engineering tea
 
 ---
 
+### Experiment 57J.54 — Uncertainty Identity vs Relatedness Diagnosis
+
+**Diagnosed why** explicit savings-realism uncertainty in 57J.53 Update 1 was absorbed into a broader cost node (`noymlfr`) rather than represented as a dedicated unknown. **Classification: B — PROMPT SEMANTIC-IDENTITY GAP.** The root cause is not a validator defect (the exact-string duplicate detector works correctly for what it checks) or a graph-model defect (the schema can represent the distinction). The gap is in the prompt's Additional Guidance line 137: it instructs the model to distinguish "same uncertainty" from "merely related uncertainty" but provides no criterion for when a broad cost node covers a specific savings-valuation concern versus asks a different question about it. Concept A ("total costs at new location") and Concept B ("are projected office savings realistic?") are OVERLAPPING BUT DISTINCT — one is magnitude estimation across all categories; the other is assumption validity for one specific category. Resolving Concept A does not resolve Concept B. **Recommended boundary: prompt-only clarification** of rule #7's "unresolved term" criterion to explicitly treat uncertainty about a specific sub-component within an existing uncertain topic as a new unresolved term requiring its own node. No production code changed, no Ollama calls. Full record in `docs/experiment-57j54.md`.
+
+---
+
 ### Experiment 57E — Irrelevant Decomposition Question Boundary
 
 **Objective:** Identify the exact graph node that triggered the decomposition producing "How the two observations were measured", and determine whether the parent was genuinely about comparison/measurement/timing before decomposition.

docs/experiment-57j54.md

@@ -0,0 +1,130 @@
+# Experiment 57J.54 — Uncertainty Identity vs Relatedness Diagnosis
+
+**Branch:** `feature/structured-semantic-fidelity-v0.20`
+**Starting HEAD:** `19c00f3` (experiment: test structured-fidelity multi-turn progress)
+**Experiment commit:** pending
+
+## Objective
+
+Answer exactly: Under the current v0.20 graph-update contract, why can an explicit unresolved uncertainty such as "whether projected office savings are realistic" be represented by modifying a broader existing cost unknown instead of creating/refining a node that actually represents that uncertainty?
+
+## Controlled Distinction
+
+**Broad projected-cost uncertainty (Concept A):**
+"What will total costs at the new location be, including relocation, facilities and payroll?"
+
+**Savings-realism uncertainty (Concept B):**
+"Are the projected office savings realistic?"
+
+**Verdict: OVERLAPPING BUT DISTINCT**
+
+These are not fully distinct because Concept B is a *sub-question* of Concept A's domain. Concept A asks "how much will it cost?" and Concept B asks "is one component of the cost projection valid?" They overlap in that both concern projected financial outcomes at the new location. But they are not the same uncertainty: Concept A is about **magnitude/estimation accuracy** across all cost categories; Concept B is about **assumption validity** for a specific cost category (office savings). One can be fully resolved (we know total will be £X) while the other remains open (savings may be overstated).
+
+The graph cannot currently represent this without either:
+- A dedicated unknown for Concept B (direct), or
+- Absorbing it into Concept A's node (indirect, losing specificity).
+
+## Part 1 — Prompt Contract
+
+**Same-vs-related distinction explicit: PARTIAL**
+
+Relevant rules from `lib/graph/prompt-builder.js`:
+
+- Rule #6: structural mutation required for consequential information/uncertainty
+- Rule #7: new unknown only for "new decision, claim, object, measure, dependency, or unresolved term"
+- Rule #11: "Do not add duplicate unknowns."
+- Additional Guidance (line 137): "first check whether an existing unresolved node already represents the same uncertainty; if so, update/refine that existing structure rather than adding a duplicate; if no such node exists, add a new unknown"
+- Rule #12: "Do not expand unrelated branches."
+
+**Analysis:** The prompt instructs the model to distinguish "same uncertainty" from "merely related uncertainty" but provides no structural mechanism to enforce this distinction. Rule #11 says "do not add duplicate unknowns" — but this only triggers when the model *chooses* to add a new unknown node (which then gets checked against existing ones). When the model *chooses update/refine over add*, this rule never applies. The additional guidance line 137 tells the model to check for "the same uncertainty" but gives no criteria for distinguishing "same" from "related." Rule #7's list ("new decision, claim, object, measure, dependency, or unresolved term") is exhaustive in structure but not semantic — it lists categories that justify new nodes but does not define when an existing node already covers the uncertainty.
+
+## Part 2 — Validator Contract
+
+**Same-vs-related distinction enforced: NO**
+
+Mechanism: `validateSemanticDuplicateUnknowns()` in `lib/graph/apply-proposal.js` (line 378) compares added nodes against *unresolved existing unknowns* for exact normalized string overlap on label/description text. It uses `.includes(text)` — i.e., the added node's label or description must appear as a substring of the existing unknown's label or description.
+
+This mechanism **cannot** distinguish Concept A from Concept B because:
+1. The model chose `updatedNodes` (not `addedNodes`), so this function never runs for the savings-realism question.
+2. Even if it did run, exact string matching would not flag "savings realism" as a duplicate of "total costs at new location" since neither text contains the other.
+
+The validator has no invariant that says: "If an existing unknown is a superset concept and the answer introduces a sub-question within that superset but outside its direct scope, a new unknown may be required." This distinction is purely semantic and falls entirely on model compliance.
+
+## Part 3 — Structural Consequences
+
+**57J.53 Update 1 representation: MATERIAL INFORMATION LOSS**
+
+Why: The engine replaced a focused uncertainty ("is this specific savings assumption valid?") with a broad status flag ("this cost projection is provisional"). The node's semantic content did not change — only its status field changed from `unknown` to `provisional`. This means:
+
+1. **Query capability lost:** The graph cannot answer "What evidence bears on whether savings are realistic?" because the node's label/description still says "Projected total costs at the new location including relocation, facility, and payroll adjustments." The specific savings-realism question is not retrievable from any node field.
+2. **Dependency tracking lost:** If someone later adds evidence about savings realism (as Update 2 did), there is no structural target for that evidence beyond a "supports" edge to the central statement — not to the cost unknown node where the concern actually resides.
+3. **Scope drift possible:** Future reasoning steps might treat `provisional` status as meaning "this cost estimate needs verification" rather than "I specifically doubt whether these savings assumptions hold." The distinction is subtle but material for downstream investigation.
+
+**Can later reasoning directly ask what evidence bears on savings realism: PARTIAL**
+
+PARTIAL because the selected question from Update 1 ("What would clarify realism of projected office savings from the relocation in this situation?") preserved the specific language, so at least one textual anchor survives — but this is in the `selectedQuestion.label`, not in the graph structure itself. If the selectedQuestion field is transient, no persistent graph-level anchor for the savings-realism concern remains.
+
+## Part 4 — Evidence Linkage Consequence
+
+**Is Update 2's parallel £2M metric consequence of the same representation issue: YES — SAME ROOT CAUSE**
+
+Why structurally: Because Update 1 represented the savings-realism uncertainty via status degradation rather than a dedicated node, there was no structural anchor for the evidence to attach to. When Update 2 arrives with concrete savings data ("£2M/year based on lease/business rates/etc."), the model sees:
+- Central statement node (target of "supports" edge — but that's generic)
+- `noymlfr` node with only a `provisional` status flag (not a clear "savings realism unknown")
+- No dedicated savings-realism unknown
+
+The £2M metric was correctly added as evidence, but without a dedicated savings-realism unknown from Update 1, the model had no structurally obvious target for the linkage. It connected to the central statement instead — which is valid but incomplete. The lack of cross-linkage between the new evidence and the existing uncertainty is a direct downstream consequence of Update 1's weak representation.
+
+## Part 5 — Architecture Ownership
+
+**Classification: B — PROMPT SEMANTIC-IDENTITY GAP**
+
+Why: The prompt correctly instructs the model to distinguish "same uncertainty" from "merely related uncertainty" (Additional Guidance, line 137), but this instruction is fundamentally underspecified. It tells the model to *do* the right thing (check whether an existing node represents the same uncertainty) without giving it a criterion for when a broad cost node covers a specific savings-valuation concern. The gap is in the prompt's semantic identity definition — it conflates "overlapping topic domain" with "same uncertainty" without distinguishing them structurally or semantically.
+
+The prompt does NOT need keyword logic (anti-keyword rule confirmed: the distinction is inherently semantic). It needs clearer boundary conditions between:
+- "This broad node already covers my concern" (update/refine)
+- "This broad node overlaps my domain but asks a different question about it" (add new unknown)
+
+## Part 6 — Smallest Next Boundary
+
+**Smallest next boundary: B — prompt-only clarification**
+
+The single semantic distinction the prompt must make:
+
+> When the answer expresses uncertainty about a *specific assumption or sub-component* within an existing uncertain topic, treat this as a **new unresolved term** under rule #7 (the assumption itself is the unresolved term), even if the broader domain appears covered. "Same uncertainty" means the question being asked is structurally equivalent — both are asking for the same factual resolution. "Overlapping but distinct" means one asks about scope/magnitude while the other asks about a specific variable's validity or realism within that scope, and resolving the magnitude does not resolve the variable's validity.
+
+This can be stated as an addition to Additional Guidance under rule #6 for explicitly unresolved uncertainty — no schema change, validator change, or graph-model change required. It simply tightens the criterion the model uses to judge "same uncertainty" vs "related but distinct."
+
+---
+
+## Convergence
+
+**Does this require keyword/synonym logic: NO**
+
+**Does this require new semantic taxonomy: NO**
+
+**Does this require graph schema change: NO**
+
+**Does this require validator change: NO** (the current validator works correctly for what it checks — exact string duplicates. The gap is upstream in model instruction, not validation.)
+
+**Does this require prompt change: YES**
+
+**Should the candidate-tie stall be handled in this same change: NO** (explicitly excluded)
+
+**What this establishes:**
+- The root cause of 57J.53's Update 1 behavior is a prompt-level semantic-identity gap, not a validator or graph-model defect.
+- "Same uncertainty" and "overlapping but distinct" are both real distinctions the system needs to make, and the current contract does not distinguish them clearly enough to enforce consistently.
+
+**What this does NOT establish:**
+- Whether the model can actually comply with tighter prompt guidance (requires testing).
+- Whether similar gaps exist in other structured categories beyond uncertainty identity.
+- Any resolution of the question-selection tie stall from Update 2.
+- The full scope of information loss across all existing unknown nodes that might absorb sub-concerns.
+
+## Production code changed: NO
+## Prompt changed: NO
+## Validator changed: NO
+## Schema changed: NO
+## Tests changed: NO
+## Ollama calls: 0
+## Dev server disturbed: NO