# Experiment 56H — Validate Unresolved Uncertainty After Harness Repair

**Date**: 2026-08-09  
**Branch**: feature/reasoning-fidelity-v0.8  
**Starting reasoning commit**: e6f7842 (establish canonical live reasoning experiment harness)  
**Harness repair commit**: c40d8c6 (fix canonical live experiment harness import)  
**Type**: Live experiment — observation only  
**Status**: PASS

---

## Objective

When the user says "I'm not really sure.", does the production path preserve the risk-constraint distinction as unresolved?

## Hypothesis

The production path will preserve the user's uncertainty:
- `userSupportedMeaning` will not invent a preference or hard-constraint position;
- compatibility/guard logic will prevent unsupported resolution;
- the risk-constraint unknown will remain unresolved.

## Fixed Case — Regression C

- **Source**: "I want the business to grow, but I don't want to take on more risk."
- **Previous question**: "Is avoiding additional risk a hard constraint or a preference/trade-off?"
- **Answer**: "I'm not really sure."
- **Expected preserved meaning**: User is uncertain about whether avoiding additional risk is a hard constraint or preference/trade-off.
- **Expected uncertainty**: Full — no position taken.

## Graph Setup

Pre-update graph state:
- `n-risk-constraint` (unknown/unknown) — active unknown, status=unknown
- `obs-source-statement` (observation/supported) — source observation
- 1 edge connecting source to risk unknown

## Results

| # | Checkpoint | Result |
|---|-----------|--------|
| 1 | `userSupportedMeaning` extracted | **null** — no semantic content extracted from the non-answer |
| 2 | `possibleInference` derived | null |
| 3 | `rawAnswerCategory` (deterministic) | `cannot_determine` |
| 4 | `proposedMeaningCategory` (from userSupportedMeaning) | `none` |
| 5 | `proposalValidation.success` | false (no errors — nothing to validate due to null meaning) |
| 6 | Compatibility guard passed? | **false** — guard did not pass |
| 7 | Graph mutation applied? | **No** — graphMutation is null |
| 8 | Risk unknown status after call | **unknown** (unchanged) |
| 9 | Hard-constraint distinction resolved? | **NO** |

## Verdict

**PASS - uncertainty preserved**

The final authoritative graph state does **not** establish either:
- risk is a hard constraint; nor
- risk is not a hard constraint;

from "I'm not really sure." alone. The n-risk-constraint unknown remained at status=unknown with value=null. No graph mutation occurred.

## Analysis Against Regression C Contract

### What the model extracted (userSupportedMeaning)

> **null** — no semantic content extracted from a non-answer response ("I'm not really sure.").

The LLM did not invent any preference, constraint position, or leaning. This is the correct behaviour for a genuine non-answer. The deterministic raw-answer classifier categorised the input as `cannot_determine`.

### What prevented graph mutation

The pre-mutation safeguard chain rejected the proposal:
- **No meaningful userSupportedMeaning** was extracted from the non-answer (null).
- Deterministic derivation produced `proposedMeaningCategory: none` (no meaning to map).
- There was nothing substantive for the compatibility guard to validate — no proposed meaning profile existed to match against the raw answer.
- **No graph mutation occurred.** `n-risk-constraint` remained unknown with value=null.

### Key observation

The non-answer ("I'm not really sure.") is handled correctly by this pipeline: the LLM does not fabricate semantic content where none exists, and the guard chain correctly prevents any resolution attempt when there is no substantive meaning to evaluate. The risk-constraint distinction remains unresolved as expected.

## What this established

- After harness repair (commit c40d8c6), Regression C passes through the real production path. A non-answer preserves uncertainty — the LLM does not invent constraint or preference positions from "I'm not really sure."
- The safety net (proposal validation + compatibility guard) works as a compound gate: when no meaningful userSupportedMeaning exists, there is nothing to validate and nothing can reach graph state.
- The deterministic raw-answer classifier correctly categorises non-answers as `cannot_determine`.

## What remains untested

- Whether the LLM's handling of "I'm not really sure." is stable across repeated identical runs.
- Whether a near-answer (e.g., "I'm leaning toward..." or "It depends on...") would trigger different behaviour.
- Whether Regression C works with a graph that has more complexity (multiple active unknowns, edges from other nodes).
- Stability across other models — this test used only qwen-claude:latest.
- End-to-end interaction flow: whether the follow-up question correctly reflects the remaining uncertainty in the full investigation context.

## Configuration

- **Host**: http://192.168.1.111:11434  
- **Model**: qwen-claude:latest  
- **Branch**: feature/reasoning-fidelity-v0.8  
- **Harness**: tests/graph/live-update-experiment-helper.cjs (canonical)  
- **Runner**: experiment-56h-runner.mjs (temporary, removed after capture)  
- **Production code changed**: NO  
- **Live calls**: 1  

## Previous Attempt

Experiment 56G was blocked by apparatus failure (broken import path in the canonical helper). This repair was completed by commit c40d8c6. Experiment 56H succeeds where 56G could not.

---

*Status pending Rob's review.*