From 70db093cb102fed1d2925bbd364cc988fbbf4fa5 Mon Sep 17 00:00:00 2001 From: robbond Date: Wed, 12 Aug 2026 10:34:53 +0100 Subject: [PATCH] experiment: test incremental meaning on existing uncertainty --- docs/current-handoff.md | 16 ++++++++ docs/experiment-57j80.md | 82 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 98 insertions(+) create mode 100644 docs/experiment-57j80.md diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 852bfe7..8de626e 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -175,6 +175,16 @@ _Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, --- +### Experiment 57J.80 — Incremental Meaning on Existing Uncertainty (PRE-ANCHORED) + +**Objective:** When a savings-realism uncertainty already exists and the user supplies new concrete information relevant to it, does the model emit `structuralActionRequired=true`, preserve that new information structurally, and avoid creating a duplicate equivalent uncertainty? + +**Classification: I — BLOCKED.** The committed `FIXTURE_MODE=updateOnly` apparatus passed a null `previousQuestion` through to the production server, which rejected at `request_validation` stage with `"Expected string, received null"`. The rejection occurred before any model inference call. One update call was made (counted but not executed by the model). No graph mutation, no structural action output, no information preservation assessment possible. The pre-anchored fixture and harness from 57J.78 remain correct for their original intent; this experiment encountered a boundary condition where no prior Start means no selectedQuestion to carry forward. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. + +--- + +--- + ### Experiment 57J.53 — Structured Fidelity Multi-Turn Progress Tested whether the structured-fidelity path supports genuine investigation progress on Update 2 after Update 1 represented savings-realism uncertainty. **Classification: B — USEFUL PARTIAL PROGRESS.** @@ -1508,3 +1518,9 @@ FIXTURE_MODE=updateOnly ANSWER_2="I am unsure whether the projected office savin ``` This satisfies 57J.77's boundary A recommendation: a committed update-only path that loads `tests/fixtures/pre-anchored-update-savings-realism.json` and sends it as an Update request body without first running Start. + +--- + +### Experiment 57J.80 — Incremental Meaning on Existing Uncertainty (PRE-ANCHORED) + +**Objective:** When a savings-realism uncertainty already exists and the user supplies new concrete information relevant to it, does the model emit `structuralActionRequired=true`, preserve that new information structurally, and avoid creating a duplicate equivalent uncertainty? **Classification: I — BLOCKED.** The committed `FIXTURE_MODE=updateOnly` apparatus passed a null `previousQuestion` through to the production server, which rejected at `request_validation` stage with `"Expected string, received null"`. The rejection occurred before any model inference call. One update call was made (counted but not executed by the model). No graph mutation, no structural action output, no information preservation assessment possible. The pre-anchored fixture and harness from 57J.78 remain correct for their original intent; this experiment encountered a boundary condition where no prior Start means no selectedQuestion to carry forward. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. diff --git a/docs/experiment-57j80.md b/docs/experiment-57j80.md new file mode 100644 index 0000000..5f93613 --- /dev/null +++ b/docs/experiment-57j80.md @@ -0,0 +1,82 @@ +# Experiment 57J.80 — Incremental Meaning on Existing Uncertainty (Pre-Anchored) + +**Branch:** `feature/semantic-action-contract-v0.23` +**Starting HEAD:** `ce01e70` (tooling: add pre-anchored update-only mode to canonical harness) + +## Objective + +Answer exactly: + +> When the savings-realism uncertainty already exists and the user supplies new concrete information relevant to it, does the model emit `structuralActionRequired=true`, preserve that new information structurally, and avoid creating a duplicate equivalent uncertainty? + +## Hypothesis + +The answer contains both: +- **existing unresolved meaning:** uncertainty about whether the savings estimate is realistic +- **new supported information:** approximately £2 million per year, basis = eliminating the current lease cost + +Expected valid contract path: `structuralActionRequired = true`, meaningful mutation, new info preserved, existing uncertainty identity preserved (single node). + +## Configured scenario (from fixture) + +```text +"We are considering relocating the engineering team to reduce operating costs." +``` + +## Configured answer (fixed) + +```text +"The projected office saving is about £2 million per year based on eliminating the current lease cost, but I am still unsure whether that estimate is realistic." +``` + +## Pre-anchored fixture + +```text +tests/fixtures/pre-anchored-update-savings-realism.json +``` + +**Savings-realism node:** +- id: `n_savings_realism` +- label: "Are the projected office savings from relocation realistic?" +- status: `unknown` + +**Exactly one equivalent unresolved uncertainty before Update: YES** + +## Run + +### CALL ACCOUNTING + +- startCalls: 0 +- updateCalls: 1 +- totalCalls: 1 +- Retries: 0 +- Supplementary scripts: NO + +### UPDATE 1 + +- HTTP status: 400 +- Stage: `request_validation` +- Validation errors: `[{"path":["previousQuestion"],"message":"Expected string, received null","code":"invalid_type"}]` +- structuralActionRequired: UNAVAILABLE + +**Result:** Update rejected at request_validation before any model inference call. The harness passed `previousQuestion: null` (correct for update-only mode with no Start), but the production server's Zod validation requires `previousQuestion` to be a string. + +## Classification: I — BLOCKED + +The committed FIXTURE_MODE=updateOnly path fails before the model call due to a request_validation boundary condition: no prior Start means no selectedQuestion, and the server does not accept null for previousQuestion in update-only mode. The apparatus works (fixture loads, anchor verified, exactly one Update attempted), but cannot reach the model inference stage. + +## What this establishes + +- FIXTURE_MODE=updateOnly apparatus correctly verifies fixture integrity +- One update call is attempted even when blocked at validation +- Call accounting reports accurately +- Pre-anchored harness from 57J.78 requires a non-null previousQuestion string to reach the model inference stage + +## What this does NOT prove + +- Whether the model would produce structuralActionRequired=true for incremental meaning on existing uncertainty +- Whether new £2m/year information would be structurally preserved +- Whether lease-cost basis would be represented +- Whether duplicate uncertainty identity is avoided + +## Production code changed: NO