diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 1e17b22..abb3858 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -676,4 +676,28 @@ This run did not test the v0.17 contract's core question because the model never **Ollama calls beyond harness count:** 0 **Documentation updated:** YES -Now let me commit: \ No newline at end of file +--- + +### Experiment 57J.41 — Semantic-to-Mutation Contract Live Validation: Faithful Meaning Only + +**Objective:** When the user introduces one simple, explicit unresolved uncertainty with no conditional/constraint language, does v0.17 translate that faithful meaning into structural graph progress rather than a semantic-only no-op? + +**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs." +**Fixed answer:** "I am unsure whether the projected office savings from the relocation are realistic." + +**Live-call count:** start: 1, update: 1, total: 2 + +**START:** HTTP 200 | stage: unknown | nodes: 6 | edges: 3 | question: "What would clarify current annual operating costs and cost structure of the engineering team in this situation?" + +**UPDATE 1:** HTTP 422 | stage: proposal_compatibility | First error: "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress." + +**Rejected Proposal Snapshot:** +- `userSupportedMeaning`: "The user is unsure whether the projected office savings from the relocation are realistic." — **FAITHFUL** (preserves uncertainty, no conditional/constraint language) +- `possibleInference`: null +- All structural fields: empty (zero updatedNodes, zero resolvedUnknownNodeIds, zero addedNodes, zero addedEdges) + +**Classification: B — SAME SEMANTIC-ONLY NO-OP.** Meaning is FAITHFUL. All structural mutation fields are empty. The v0.17 MUST rule + validator diagnostic fires as designed, converting the original 57J.36 silent accepted no-op into an explicit rejected proposal with specific error. However, this does NOT prove that faithful meaning CAN produce graph progress — only that it cannot do so without structural mutation under v0.17's new contract. + +**Key finding:** v0.17 successfully blocks the original failure (accepted semantic-only no-op) but does not establish a positive path for faithful meaning to produce graph progress. The open question remains: is there any valid pathway where faithful meaning translates to structural mutation, or does the MUST rule universally block it? + +Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Harness restored. No-retry preserved. Dev server disturbed: NO. \ No newline at end of file diff --git a/docs/experiment-57j41.md b/docs/experiment-57j41.md new file mode 100644 index 0000000..9fdf715 --- /dev/null +++ b/docs/experiment-57j41.md @@ -0,0 +1,117 @@ +# Experiment 57J.41 — Semantic-to-Mutation Contract Live Validation: Faithful Meaning Only + +**Objective:** When the user introduces one simple, explicit unresolved uncertainty with no conditional/constraint language, does v0.17 translate that faithful meaning into structural graph progress rather than a semantic-only no-op? + +57J.40 could not test this because Qwen strengthened the original answer into a decision condition. This experiment deliberately removes that confound. + +**Branch:** `feature/semantic-to-mutation-contract-v0.17` +**Starting HEAD:** 39217b6 experiment: validate semantic-to-mutation contract live + +## Pre-written expectation + +The answer introduces one clear unresolved uncertainty: whether projected office savings are realistic. A faithful proposal should preserve that uncertainty and express its effect structurally, either by updating/refining equivalent existing graph structure or by adding justified new structure. `answerMeaning` alone with zero graph mutation is the specific failure under test. + +## Configuration + +- **Scenario:** "We are considering relocating the engineering team to reduce operating costs." +- **Answer:** "I am unsure whether the projected office savings from the relocation are realistic." +- **maxUpdates:** 1 +- **Configured model:** qwen-claude:latest at http://192.168.1.111:11434 +- **Dev server:** REUSED EXISTING (HTTP 200) + +## Run + +### Call Accounting + +startCalls: 1 +updateCalls: 1 +totalCalls: 2 + +### START + +``` +HTTP status: 200 +stage: unknown +selected question: "What would clarify current annual operating costs and cost structure of the engineering team in this situation?" +node count: 6 +edge count: 3 +``` + +### UPDATE 1 + +``` +HTTP status: 422 +stage: proposal_compatibility +First error: "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress." +selected question: null +node count: 6 (unchanged) +edge count: 3 (unchanged) +``` + +**Rejected Proposal Snapshot:** +```json +{ + "answerMeaning": { + "userSupportedMeaning": "The user is unsure whether the projected office savings from the relocation are realistic.", + "possibleInference": null + }, + "updatedNodes": [], + "resolvedUnknownNodeIds": [], + "addedNodes": [], + "addedEdges": [] +} +``` + +## Analysis + +### Meaning Fidelity + +**Classification: FAITHFUL** + +The `userSupportedMeaning` directly preserves the raw answer's uncertainty: +- **Raw:** "I am unsure whether the projected office savings from the relocation are realistic." +- **Produced:** "The user is unsure whether the projected office savings from the relocation are realistic." + +No conditional language. No constraint language. No decision requirement. No priority statement. The model preserved the simple uncertainty about savings realism without strengthening or degradation. + +`possibleInference` is null — appropriate for a direct, unambiguous single-dimension uncertainty. + +### Structural Mutation + +``` +updatedNodes: 0 +resolvedUnknownNodeIds: 0 +addedNodes: 0 +addedEdges: 0 +``` + +Zero structural mutation across all fields. This is a semantic-only no-op at the proposal level. + +The rejection occurred at `proposal_compatibility` because the v0.17 MUST rule triggers when `userSupportedMeaning` is populated with zero structural mutation. The rejection error exactly matches the new contract diagnostic: "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation." + +### Classification: B — SAME SEMANTIC-ONLY NO-OP + +Meaning is FAITHFUL. All structural mutation fields are empty. + +However, this is not a silent semantic-only no-op (which was the original 57J.36 problem). It is an **explicitly rejected** semantic-only no-op enforced by the v0.17 MUST rule + validator diagnostic. The model produced faithful meaning but zero structural progress, and the new contract boundary caught it before graph mutation could occur. + +## Did v0.17 remove the faithful semantic-only no-op failure? + +**UNPROVEN for positive outcome.** v0.17 successfully converts what would have been an accepted semantic-only no-op into a rejected proposal with a specific diagnostic error. This confirms the v0.17 contract fix (Option B) is working as designed — it blocks faithfulness-verified but structurally-empty proposals. + +However, v0.17 does NOT prove that faithful meaning CAN produce graph progress. It proves the opposite direction: that v0.17 prevents a semantically faithful proposal with zero structure from passing through. The open question remains unanswered — is there any valid pathway where faithful meaning translates to structural mutation under v0.17, or does the new constraint universally block it? + +## What this run establishes + +1. The configured model preserves the explicit uncertainty about savings realism without strengthening (direct improvement over 57J.40). +2. The v0.17 MUST rule + validator diagnostic fires exactly as designed: faithful meaning with zero structural mutation → rejected at proposal_compatibility with specific error. +3. The original 57J.36 failure pattern (accepted semantic-only no-op) is now blocked — the rejection is explicit and diagnostic. + +## What this does NOT prove + +1. That faithful meaning CAN produce graph progress under v0.17. +2. Whether the model can simultaneously preserve faithfulness AND add justified structure for this or other scenarios. +3. Whether the MUST rule is too aggressive — it may block both no-ops and legitimate partial-progress proposals. +4. That cold-start quality (6 nodes) affects the outcome — but cold-start variance was not the variable under test here. + +Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 2 live calls total. No production code changed. Harness restored to original scenario/answers. No-retry preserved. Dev server disturbed: NO.