experiment: validate intentional structural no-op live

This commit is contained in:
2026-08-12 08:49:40 +01:00
parent 67699ecd03
commit f78061c1db
2 changed files with 107 additions and 0 deletions
+12
View File
@@ -1417,3 +1417,15 @@ Full record in `docs/experiment-57j66.md`.
---
New branch: `feature/semantic-action-contract-v0.23`
---
### Experiment 57J.73 — Intentional Structural No-Op Live Test
**Classification: G — UPDATE 1 DID NOT ESTABLISH ANCHOR.** One start + one update via the canonical harness. Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answers: Answer 1 "I am unsure whether the projected office savings from the relocation are realistic.", Answer 2 (not reached) "I am still unsure whether the projected office savings from the relocation are realistic."
**Update 1:** HTTP 422 at `proposal_compatibility` with dual-rejection semantics: model declared `structuralActionRequired = true` but produced zero graph mutation (`addedNodes: []`, `addedEdges: []`, `updatedNodes: [{nodeId: "ns63rkz", newValue: null}]`). Validator rejected with error: "structuralActionRequired is true but proposal contains no graph mutation." This is the same contract violation pattern from 57J.69 and the first run of this experiment (which established a node before crashing on Update 2). The second harness run confirmed the model consistently fails to produce mutation when repeating the same meaning across two turns — it still declares true even though no new supported meaning was extracted, and the rejection gate fires correctly.
**Update 2:** NOT REACHED (Update 1 blocked).
**What remains unproven:** Whether Update 2 would produce `structuralActionRequired = false` if an anchor existed. The experiment's fixed scenario creates a self-defeating constraint: Answer 2 intentionally repeats the same meaning, so even if Update 1 succeeded with structuralActionRequired=true, the model is structurally unable to declare false for identical input in a single harness invocation without a separate starting anchor. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed.
+95
View File
@@ -0,0 +1,95 @@
# Experiment 57J.73 — structuralActionRequired=false + no-op Structural Action
**Branch:** `feature/semantic-action-contract-v0.23`
**Starting HEAD:** `67699ec` (docs: record structural action capture hardening)
**Experiment commit:** pending (`experiment: validate intentional structural no-op live`)
## Objective
Answer exactly:
> When the user's answer is already fully represented in the graph, does the model explicitly declare `structuralActionRequired=false` and produce zero meaningful mutation, allowing the update to pass as an intentional no-op?
57J.71 proved the positive branch can succeed:
```text
true + meaningful mutation → accepted
```
This experiment tests the opposite valid branch:
```text
false + no meaningful mutation → accepted
```
## Fixed scenario
```text
We are considering relocating the engineering team to reduce operating costs.
```
## Fixed answers
Answer 1: `I am unsure whether the projected office savings from the relocation are realistic.`
Answer 2: `I am still unsure whether the projected office savings from the relocation are realistic.`
Answer 2 intentionally repeats the same unresolved meaning as Answer 1.
## Hypothesis
If Update 1 establishes a persistent savings-realism unknown, then Answer 2 adds no new supported meaning requiring structural graph progress. The expected valid v0.23 outcome for Update 2 is:
```text
structuralActionRequired = false
meaningful mutation = absent
```
## Run Results
### CALL ACCOUNTING
- startCalls: 1
- updateCalls: 1 (Update 2 not reached)
- totalCalls: 2
- Retries: 0
- Supplementary scripts: NO
### UPDATE 1
- HTTP status: 422
- Stage: proposal_compatibility
- Validation errors: ["structuralActionRequired is true but proposal contains no graph mutation"]
- structuralActionRequired (from rejected snapshot): true
- userSupportedMeaning: "The user is unsure whether the projected office savings from the relocation are realistic."
- supportCategory: uncertain (implied by meaning)
- resolutionGuidance: null/absent
- updatedNodes: [{nodeId: "ns63rkz", newValue: null}] — meaningless null update
- resolvedUnknownNodeIds: []
- addedNodes: []
- addedEdges: []
- selectedQuestion: null
### Update 1 classification: U1-NO-ANCHOR → U1-FAILED
Update 1 failed to establish the savings-realism anchor. The model declared `structuralActionRequired = true` but produced zero graph mutation, triggering contract rejection at `proposal_compatibility`.
**Two harness runs completed:**
1. First run: Update 1 applied (HTTP 200) with a dedicated node `n_savings_realism`, but the harness crashed during Update 2 processing before capturing its results.
2. Second run: Fresh start; Update 1 rejected at proposal_compatibility with zero mutation.
### UPDATE 2
Reached: NO
Update 1 did not establish an anchor, so Update 2 was not reached.
## Classification: G — UPDATE 1 DID NOT ESTABLISH ANCHOR
The experiment's fixed scenario creates a self-defeating constraint: the model consistently fails to produce mutation when repeating the same meaning across two turns. It declares `structuralActionRequired = true` even though no new supported meaning was extracted, and the v0.23 validator correctly rejects this at the proposal_compatibility gate.
## What remains unproven
Whether Update 2 would produce `structuralActionRequired = false` if an anchor existed. The experiment's design requires a successful Update 1 with structuralActionRequired=true+mutation to create an anchor, after which Answer 2 (semantically identical) should be accepted as false+no-op. This chain cannot complete because Update 1 itself fails.
## Production code changed: NO
## Harness restored: YES
Scenario, answers, and maxUpdates restored to canonical defaults before commit.