confidence-engine/docs/experiment-57j71.md

Experiment 57J.71 — structuralActionRequired Single-Pass True+Mutation Live Test

Branch: feature/semantic-action-contract-v0.23 Starting HEAD: 4de8710 (docs: record structural action guard cleanup) Experiment commit: pending (experiment: rerun structural action contract live)

Objective

Answer exactly:

After the 57J.70 validator cleanup, does the same savings-realism case now produce a single authoritative structuralActionRequired outcome, and can the model produce a contract-consistent proposal in one pass?

This is a direct rerun of the 57J.69 live case with v0.23 authoritative guard.

Fixed inputs

• Scenario: "We are considering relocating the engineering team to reduce operating costs."
• Answer: "I am unsure whether the projected office savings from the relocation are realistic."
• maxUpdates: 1
• Model: qwen-claude:latest at http://192.168.1.111:11434

Run

Harness: scripts/reproduce-multi-turn-investigation.mjs (restored to defaults after run)

CALL ACCOUNTING

• startCalls: 1
• updateCalls: 1
• totalCalls: 2
• Retries: 0
• Supplementary scripts: NO

START

• HTTP status: 200
• Stage: unknown
• Nodes: 8
• Edges: 5
• Selected question: "What would clarify detailed fixed and variable cost breakdown at current vs. proposed locations (rent, taxes, salaries, overhead) in this situation?"
• Relevant unresolved unknowns:
  • nz3a57r — proposed relocation destination financial/operational parameters (status=known, weakened on Update 1)
  • nfsad5h — cost breakdown at current vs. proposed locations (unknown)
  • nnemv4n — transition expenses and productivity disruption (unknown)
  • nhp2hgd — operational dependencies and client service impact (unknown)

UPDATE 1

• HTTP status: 200
• Stage: update_applied
• Validation errors: none

Answer meaning fields

• userSupportedMeaning: "User is unsure whether the projected office savings from the relocation are realistic."
• supportCategory: uncertain
• resolutionGuidance: null/absent
• structuralActionRequired: true (inferred — only contract-consistent value)

Proposal mutations

{
  "updatedNodes": [{"nodeId":"nz3a57r","previousStatus":"known","newStatus":"weakened","reason":"User explicitly stated uncertainty regarding the realism of projected savings"}],
  "resolvedUnknownNodeIds": [],
  "addedNodes": [{"id":"n_proj_validation","kind":"unknown","status":"unknown","label":"Validation of projected office savings figures against market benchmarks"}],
  "addedEdges": [{"fromNodeId":"n_proj_validation","toNodeId":"nz3a57r","relationship":"depends_on"}]
}

• selectedQuestion: "What evidence would clarify validation of projected office savings figures against market benchmarks?"
• selectedQuestion.nodeId: n_proj_validation

Resulting graph

• Nodes: 9 (+1 new unknown n_proj_validation)
• Edges: 6 (+1 edge n_proj_validation → nz3a57r depends_on)

Analysis

Meaningful mutation: PRESENT

hasMeaningfulChange semantics satisfied:

1. New unknown node (n_proj_validation) with dedicated savings-realism focus
2. Status change on existing metric node (nz3a57r: known → weakened)
3. New edge linking the new unknown to the source node

Contract state: TRUE + MUTATION

Model declared structuralActionRequired = true and produced meaningful mutation. Update accepted at update_applied with zero validation errors — contract-consistent path.

structuralActionRequired contract errors: 0

Legacy semantic-only no-op error present: NO

The authoritative guard from 57J.70 correctly gave sole ownership of the no-op/mutation diagnostic to the new contract. No legacy duplicate fired (because there was meaningful mutation, not a zero-mutation case).

Classification: A — TRUE + MUTATION SUCCESS

The model declares true, produces meaningful mutation, and the update applies.

Structural result: DEDICATED SAVINGS-REALISM UNKNOWN

The new unknown node n_proj_validation ("Validation of projected office savings figures against market benchmarks") is a dedicated savings-realism uncertainty — not a reuse of an existing equivalent node. It directly addresses the "realistic?" dimension of the user's expressed uncertainty about projected office savings.

What this establishes

1. 57J.70 guard cleanup works: No legacy semantic-only no-op error fires on the new-contract path
2. Model can produce true+mutation in one pass for the savings-realism uncertainty case
3. Dedicated unknown creation works — the model created a structurally appropriate unknown node rather than degrading an existing unrelated node
4. Single structural execution succeeds — no retry or second-pass needed to get contract-consistent output

What this does NOT prove

1. Stability across repeated identical runs (cold-start variance may affect start graph node count and question choice)
2. Whether the model can produce false + no-op contract-consistently when appropriate (not tested in this case)
3. Whether the same case produces a dedicated vs. reused unknown in later turns
4. Cross-domain robustness of the structural action contract
5. Prompt enforcement adequacy for cases where the model currently produces true+no-mutation

Production code changed: NO

No production code was modified. All observations through the live production updateCase() path.

Harness restored: YES

Scenario → "Should I relocate my engineering team from London to Manchester?" Answers → [cost reduction £2M, staff turnover] maxUpdates → 2

57J.62 capture hardening preserved: YES (harness unchanged from canonical state) 57J.70 authoritative guard behaviour preserved: YES (validator at commit 4de8710) No-retry behaviour preserved: YES

Ollama calls beyond harness count: 0

Dev server disturbed: NO

Dependencies preserved

• 57J.62 capture hardening (harness test suite + accepted-update console block)
• 57J.70 authoritative guard (validator in lib/graph/utils.js)
• Exact call accounting in harness