96 lines
3.7 KiB
Markdown
96 lines
3.7 KiB
Markdown
# Experiment 57J.73 — structuralActionRequired=false + no-op Structural Action
|
|
|
|
**Branch:** `feature/semantic-action-contract-v0.23`
|
|
**Starting HEAD:** `67699ec` (docs: record structural action capture hardening)
|
|
**Experiment commit:** pending (`experiment: validate intentional structural no-op live`)
|
|
|
|
## Objective
|
|
|
|
Answer exactly:
|
|
|
|
> When the user's answer is already fully represented in the graph, does the model explicitly declare `structuralActionRequired=false` and produce zero meaningful mutation, allowing the update to pass as an intentional no-op?
|
|
|
|
57J.71 proved the positive branch can succeed:
|
|
```text
|
|
true + meaningful mutation → accepted
|
|
```
|
|
|
|
This experiment tests the opposite valid branch:
|
|
```text
|
|
false + no meaningful mutation → accepted
|
|
```
|
|
|
|
## Fixed scenario
|
|
|
|
```text
|
|
We are considering relocating the engineering team to reduce operating costs.
|
|
```
|
|
|
|
## Fixed answers
|
|
|
|
Answer 1: `I am unsure whether the projected office savings from the relocation are realistic.`
|
|
Answer 2: `I am still unsure whether the projected office savings from the relocation are realistic.`
|
|
|
|
Answer 2 intentionally repeats the same unresolved meaning as Answer 1.
|
|
|
|
## Hypothesis
|
|
|
|
If Update 1 establishes a persistent savings-realism unknown, then Answer 2 adds no new supported meaning requiring structural graph progress. The expected valid v0.23 outcome for Update 2 is:
|
|
```text
|
|
structuralActionRequired = false
|
|
meaningful mutation = absent
|
|
```
|
|
|
|
## Run Results
|
|
|
|
### CALL ACCOUNTING
|
|
|
|
- startCalls: 1
|
|
- updateCalls: 1 (Update 2 not reached)
|
|
- totalCalls: 2
|
|
- Retries: 0
|
|
- Supplementary scripts: NO
|
|
|
|
### UPDATE 1
|
|
|
|
- HTTP status: 422
|
|
- Stage: proposal_compatibility
|
|
- Validation errors: ["structuralActionRequired is true but proposal contains no graph mutation"]
|
|
- structuralActionRequired (from rejected snapshot): true
|
|
- userSupportedMeaning: "The user is unsure whether the projected office savings from the relocation are realistic."
|
|
- supportCategory: uncertain (implied by meaning)
|
|
- resolutionGuidance: null/absent
|
|
- updatedNodes: [{nodeId: "ns63rkz", newValue: null}] — meaningless null update
|
|
- resolvedUnknownNodeIds: []
|
|
- addedNodes: []
|
|
- addedEdges: []
|
|
- selectedQuestion: null
|
|
|
|
### Update 1 classification: U1-NO-ANCHOR → U1-FAILED
|
|
|
|
Update 1 failed to establish the savings-realism anchor. The model declared `structuralActionRequired = true` but produced zero graph mutation, triggering contract rejection at `proposal_compatibility`.
|
|
|
|
**Two harness runs completed:**
|
|
1. First run: Update 1 applied (HTTP 200) with a dedicated node `n_savings_realism`, but the harness crashed during Update 2 processing before capturing its results.
|
|
2. Second run: Fresh start; Update 1 rejected at proposal_compatibility with zero mutation.
|
|
|
|
### UPDATE 2
|
|
|
|
Reached: NO
|
|
|
|
Update 1 did not establish an anchor, so Update 2 was not reached.
|
|
|
|
## Classification: G — UPDATE 1 DID NOT ESTABLISH ANCHOR
|
|
|
|
The experiment's fixed scenario creates a self-defeating constraint: the model consistently fails to produce mutation when repeating the same meaning across two turns. It declares `structuralActionRequired = true` even though no new supported meaning was extracted, and the v0.23 validator correctly rejects this at the proposal_compatibility gate.
|
|
|
|
## What remains unproven
|
|
|
|
Whether Update 2 would produce `structuralActionRequired = false` if an anchor existed. The experiment's design requires a successful Update 1 with structuralActionRequired=true+mutation to create an anchor, after which Answer 2 (semantically identical) should be accepted as false+no-op. This chain cannot complete because Update 1 itself fails.
|
|
|
|
## Production code changed: NO
|
|
|
|
## Harness restored: YES
|
|
|
|
Scenario, answers, and maxUpdates restored to canonical defaults before commit.
|