Files
confidence-engine/docs/experiment-57j80.md
T

3.2 KiB

Experiment 57J.80 — Incremental Meaning on Existing Uncertainty (Pre-Anchored)

Branch: feature/semantic-action-contract-v0.23 Starting HEAD: ce01e70 (tooling: add pre-anchored update-only mode to canonical harness)

Objective

Answer exactly:

When the savings-realism uncertainty already exists and the user supplies new concrete information relevant to it, does the model emit structuralActionRequired=true, preserve that new information structurally, and avoid creating a duplicate equivalent uncertainty?

Hypothesis

The answer contains both:

  • existing unresolved meaning: uncertainty about whether the savings estimate is realistic
  • new supported information: approximately £2 million per year, basis = eliminating the current lease cost

Expected valid contract path: structuralActionRequired = true, meaningful mutation, new info preserved, existing uncertainty identity preserved (single node).

Configured scenario (from fixture)

"We are considering relocating the engineering team to reduce operating costs."

Configured answer (fixed)

"The projected office saving is about £2 million per year based on eliminating the current lease cost, but I am still unsure whether that estimate is realistic."

Pre-anchored fixture

tests/fixtures/pre-anchored-update-savings-realism.json

Savings-realism node:

  • id: n_savings_realism
  • label: "Are the projected office savings from relocation realistic?"
  • status: unknown

Exactly one equivalent unresolved uncertainty before Update: YES

Run

CALL ACCOUNTING

  • startCalls: 0
  • updateCalls: 1
  • totalCalls: 1
  • Retries: 0
  • Supplementary scripts: NO

UPDATE 1

  • HTTP status: 400
  • Stage: request_validation
  • Validation errors: [{"path":["previousQuestion"],"message":"Expected string, received null","code":"invalid_type"}]
  • structuralActionRequired: UNAVAILABLE

Result: Update rejected at request_validation before any model inference call. The harness passed previousQuestion: null (correct for update-only mode with no Start), but the production server's Zod validation requires previousQuestion to be a string.

Classification: I — BLOCKED

The committed FIXTURE_MODE=updateOnly path fails before the model call due to a request_validation boundary condition: no prior Start means no selectedQuestion, and the server does not accept null for previousQuestion in update-only mode. The apparatus works (fixture loads, anchor verified, exactly one Update attempted), but cannot reach the model inference stage.

What this establishes

  • FIXTURE_MODE=updateOnly apparatus correctly verifies fixture integrity
  • One update call is attempted even when blocked at validation
  • Call accounting reports accurately
  • Pre-anchored harness from 57J.78 requires a non-null previousQuestion string to reach the model inference stage

What this does NOT prove

  • Whether the model would produce structuralActionRequired=true for incremental meaning on existing uncertainty
  • Whether new £2m/year information would be structurally preserved
  • Whether lease-cost basis would be represented
  • Whether duplicate uncertainty identity is avoided

Production code changed: NO