diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 2e87a40..b0cf186 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -1441,3 +1441,11 @@ New branch: `feature/semantic-action-contract-v0.23` **What this fixes:** The self-defeating constraint from 57J.73 where Answer 2 was unreachable because Update 1 failed at proposal_compatibility when declaring `structuralActionRequired=true` with zero mutation. Now the harness can inject a pre-anchored graph as the Update input, enabling direct testing of whether the live model produces `false + no meaningful mutation → accepted` on a graph that already contains the savings-realism anchor. **What remains unproven:** Whether the live model will actually produce `false + no meaningful mutation` for Answer 2 when given this pre-anchored graph (a separate live experiment is required). + +### Experiment 57J.75 — Pre-Anchored No-Op Update Live Test + +**Objective:** Run a single controlled live test: inject the pre-anchored savings-realism fixture directly as the Update input with an answer preserving existing uncertainty, and verify the model produces `false + zero mutation → accepted`. **Classification: A — INTENTIONAL NO-OP WORKS.** + +**Answer:** The one controlled run shows that with an equivalent savings-realism uncertainty already present, the model directly emitted `structuralActionRequired = false`, produced zero meaningful mutation (no added/updated nodes or edges), and the production validator accepted the proposal. All four required conditions met: structuralActionRequired=false observed directly, zero meaningful mutation, HTTP 200 / update_applied, exactly one equivalent savings-realism uncertainty remained after Update. + +**What remains unproven:** Whether `false + zero mutation` would also be accepted for non-anchored graphs; cold-start comparison via normal Start→Update; stability across repeated runs (only one run); generalization beyond savings-realism anchors. diff --git a/docs/experiment-57j75.md b/docs/experiment-57j75.md new file mode 100644 index 0000000..27fc99c --- /dev/null +++ b/docs/experiment-57j75.md @@ -0,0 +1,101 @@ +# Experiment 57J.75 — Pre-Anchored No-Op Update Live Test + +**Branch:** `feature/semantic-action-contract-v0.23` +**Starting HEAD:** `8184e05` (docs: record pre-anchored update apparatus) +**Experiment commit:** pending (`experiment: validate controlled structural no-op live`) + +## Objective + +Answer exactly: + +> Given a deterministic pre-existing graph fixture with a savings-realism anchor already present, does the live model produce `structuralActionRequired = false` + zero meaningful mutation when updating with an answer that preserves the existing uncertainty, and is that accepted? + +This is the direct follow-up to **57J.73** (Classification G), where the `false + no meaningful mutation → accepted` branch was unreachable because Update 1 failed at `proposal_compatibility`. The apparatus from **57J.74** enables this test via a pre-anchored fixture injected directly into the Update request's `situationGraph`, bypassing Start entirely. + +## Hypothesis + +If the model receives a pre-anchored graph where the savings-realism unknown already exists, and updates with an answer that preserves (rather than resolves) that uncertainty, it will: +1. Declare `structuralActionRequired = false` (no new structure needed) +2. Produce zero meaningful mutation (graph unchanged) +3. Be **accepted** (not rejected by the contract validator, because the v0.23 contract validates `false + zero mutation` as a valid intentional no-op) + +## Tooling + +The pre-anchored apparatus from **57J.74** consists of: +1. JSON fixture: `tests/fixtures/pre-anchored-update-savings-realism.json` — deterministic graph with one unresolved savings-realism unknown +2. Harness tests: `scripts/reproduce-multi-turn-investigation.harness.test.js` — 10 deterministic tests covering anchor count, schema validity, relationship integrity, exact graph injection + +The live test sends the fixture's graph directly as the Update request's `situationGraph`, with an answer that preserves (not resolves) the existing uncertainty. + +## Test Config + +- **Fixture:** Pre-existing graph with savings-realism unknown (`n_savings_realism`, kind=unknown, status=unknown) +- **Answer:** "I am still unsure whether the projected office savings from the relocation are realistic." +- **Model:** qwen-claude:latest at http://192.168.1.111:11434 +- **Previous Question:** "Are the projected office savings from relocation realistic?" (from fixture) + +## Results + +**HTTP status: 200 — update_applied** + +```json +{ + "success": true, + "stage": "update_applied", + "proposal": { + "structuralActionRequired": false, + "answerMeaning": { + "userSupportedMeaning": "The user remains unsure whether the projected office savings from relocation are realistic.", + "possibleInference": "Proceeding with relocation without validated savings projections carries unquantified financial risk.", + "supportCategory": "uncertain", + "resolutionGuidance": "may_resolve" + }, + "addedNodes": [], + "updatedNodes": [], + "addedEdges": [], + "resolvedUnknownNodeIds": [] + } +} +``` + +**Graph after Update:** Unchanged — still exactly 2 nodes (1 state, 1 unknown) and 1 edge. The savings-realism anchor persisted without modification. + +**Next Question generated:** "What would clarify are the projected office savings from relocation realistic in this situation?" + +## Evidence Summary + +This one controlled run shows that, with an equivalent savings-realism uncertainty already present in the pre-anchored graph, the model directly emitted `structuralActionRequired = false`, produced zero meaningful mutation (no added/updated nodes or edges), and the production validator accepted the proposal. + +## Classification: A — INTENTIONAL NO-OP WORKS + +All four conditions met: +1. `structuralActionRequired = false` observed directly in the Update response +2. Zero meaningful mutation (addedNodes=[], updatedNodes=[], addedEdges=[]) +3. HTTP 200 / update_applied +4. Exactly one equivalent savings-realism uncertainty remained after Update + +## What this establishes + +This controlled run demonstrates that when the model receives a pre-anchored graph with an existing savings-realism unknown and produces an answer preserving that uncertainty, it correctly declares no structural action needed and is accepted by the v0.23 contract validator. The `false + zero mutation → accepted` path does exist in the contract for pre-anchored inputs. + +## What this does NOT prove + +1. Whether `false + zero mutation` would also be accepted for non-anchored graphs (where the model might legitimately need to create structure) +2. Whether the same answer would produce different results starting from a clean Start (i.e., whether cold-start dynamics change the outcome) +3. Stability across repeated runs — only one test run was performed +4. Whether this generalizes to other types of anchors beyond savings-realism + +## What remains unproven + +1. **Cold-start comparison:** Run the same answer through normal Start→Update to compare whether the starting state changes the model's structural-action judgment. +2. **Different anchors:** Test with different pre-anchored graphs (e.g., risk-constraint anchor, timeline anchor) to verify generalization. + +## Production code changed: NO + +No production logic changed. Only deterministic harness tests from 57J.74 were used as the test apparatus. + +## Harness restored: YES + +Scenario, answers, and maxUpdates in `scripts/reproduce-multi-turn-investigation.mjs` remain at canonical defaults. No fixture mode was committed to the harness. + +## Dev server disturbed: NO