confidence-engine/docs/experiment-57j65.md

Experiment 57J.65 — Smallest Enforceable Semantic-to-Mutation Contract

Branch: feature/selected-question-contract-v0.22 Starting HEAD: d7cb343 (experiment: diagnose semantic-to-mutation action ownership)

Objective

Answer one question:

What is the smallest structured contract that lets the model declare whether graph action is required, and lets deterministic code verify that the actual proposal fulfils that declaration?

57J.64 established that further prompt-only wording is not the next boundary. This experiment answers with data-contract analysis only.

---

Part 1 — Are Existing Fields Enough?

Classification: C — NEW ACTION DECLARATION REQUIRED

The existing fields provide these capabilities:

Field	What it expresses
userSupportedMeaning	Semantic content (text) of what the user supports
supportCategory	Category label for semantic content
resolutionGuidance	Resolution instruction
updatedNodes	Nodes modified
resolvedUnknownNodeIds	Unknowns resolved
addedNodes	New nodes created
addedEdges	New edges created
selectedQuestion	Follow-up question candidate

Why they are insufficient:

These fields encode what changed but not what was intended. When a model intends "I agree with the semantic content, no structural change is needed," it returns empty mutation arrays. There is no explicit field saying "I intentionally declare zero graph action." The validator's current check (line 886 of utils.js) derives intent from:

userSupportedMeaning populated + all mutation arrays empty → REJECT

This treats the model's silence as an error rather than accepting a valid intentional no-op declaration. It cannot distinguish between "model forgot to mutate" and "model intentionally chose no mutation."

---

Part 2 — Minimum Required Distinction

What deterministic validation actually needs:

The validator does not need to know why the model made its choice. It only needs to verify that the model's declared intent matches the proposal shape.

Intended Action	How validator checks	Classification
Reuse/refine existing	updatedNodes references existing node with status/value change	DERIVABLE FROM PROPOSAL SHAPE
Add new unknown	addedUnknownCount > 0	DERIVABLE FROM PROPOSAL SHAPE
Resolve existing	resolvedUnknownNodeIds.length > 0	DERIVABLE FROM PROPOSAL SHAPE
Other structural mutation	Any non-empty mutation array or addedEdges	DERIVABLE FROM PROPOSAL SHAPE
No structural change	All mutation arrays empty	MUST BE DECLARED (by the model)

Conclusion: The minimum distinction is MUTATION REQUIRED vs NO MUTATION REQUIRED.

Deterministic validation does not need to know which mutation type was intended because it checks the actual proposal shape for each possible mutation independently. The only gap is: when all arrays are empty, how do we know the model intentionally chose no-op vs failed to produce one?

---

Part 3 — Compare Three Designs

Option A — Boolean Contract

A single field: structuralActionRequired: true | false

Criterion	Answer
Prevents ambiguous semantic-only no-op	PARTIAL — declares intent, but model can always choose the "safe" value without verifying
Checks actual mutation	YES — validator compares declared value against proposal shape
Requires re-reading English semantics	NO — only compares structured field against structured arrays
New schema concept	BOOLEAN
Validator complexity	LOW — two boolean checks (true→non-empty, false→empty)
Model-compliance risk	MEDIUM — model may default to one value under pressure; binary choice is simplest for the model

Option B — Small Action Enum

A field: semanticAction: "add_new_unknown" | "reuse_or_refine_existing" | "resolve_existing" | "other_structural_mutation" | "no_change_needed"

Criterion	Answer
Prevents ambiguous semantic-only no-op	PARTIAL — more categories than validation needs, but declares explicit intent
Checks actual mutation	YES — validator maps each enum value to specific proposal shape requirements
Requires re-reading English semantics	NO — only compares structured field against structured arrays
New schema concept	SMALL ENUM (5 values)
Validator complexity	MEDIUM — five mapping rules plus cross-validation
Model-compliance risk	MEDIUM-HIGH — more categories increase noncompliance risk; model must pick from five options deterministically

Option C — Existing Fields Only

No new field. Use userSupportedMeaning populated + empty mutation arrays to mean "intentional semantic agreement, no graph change."

Criterion	Answer
Prevents ambiguous semantic-only no-op	PARTIAL — currently rejects this case; treating it as valid would accept noncompliant outputs silently
Checks actual mutation	YES — proposal shape is always checkable
Requires re-reading English semantics	NO — existing behavior already works without semantic parsing
New schema concept	NONE
Validator complexity	LOW — no new logic needed
Model-compliance risk	HIGH — treating empty-mutation-as-intentional would accept every noncompliant zero-mutation output, making the boundary unenforceable

---

Part 4 — The No-Change Case

Can no-change be verified without re-reading English?

YES — but only with a new structured declaration

With existing fields:

• userSupportedMeaning populated + all mutation arrays empty → current code REJECTS
• We cannot distinguish "model intended no-op" from "model forgot to mutate"
• This is NOT verifiable as intentional without knowing what the model meant

With a new declaration field:

• Model sets structuralActionRequired: false + all mutation arrays empty → validation PASSES (model explicitly declared no action)
• Model sets structuralActionRequired: true + all mutation arrays empty → validation REJECTS (contradiction between intent and proposal)
• The declaration itself is the verification mechanism

---

Part 5 — Relationship to supportCategory

Relationship: INDEPENDENT OF supportCategory

Reasoning:

• supportCategory = "uncertain" does NOT necessarily mean add new unknown
• An equivalent uncertainty may already exist and should be reused (v0.21 identity rule)
• supportCategory classifies the semantic content of the answer
• The structural action declaration classifies the proposed graph change
• These are orthogonal: the same supportCategory can map to different structural actions depending on current graph state

The v0.21 identity rule must be preserved: when an equivalent unresolved uncertainty already exists, reuse/refine that existing node — do not add a duplicate.

---

Part 6 — Deterministic Invariants

For Option A (boolean contract), the invariants are:

1. structuralActionRequired = true + all mutation arrays empty → REJECT The model declared intent for structural action but produced none.

2. structuralActionRequired = false + meaningful mutation present → ACCEPT (diagnostic note) Model declared no change but produced one. This is not a contradiction — it may be the model doing extra work beyond what was needed. Log a warning.

3. structuralActionRequired missing + userSupportedMeaning populated → REJECT Cannot verify intent when required field is absent.

4. No invariant needed for structuralActionRequired = false + empty mutations This is the valid "semantic agreement, no structural change" case. The model explicitly declared its intention; validation passes because it can do so deterministically without semantic parsing.

---

Part 7 — 57J.63 Walkthrough

Case A: Successful proposal with dedicated unknown

userSupportedMeaning: "uncertainty about projected office savings realism"
proposal: adds dedicated savings-realism unknown
Declaration: structuralActionRequired = true

Why validation passes:

• Model declares true → expects meaningful mutation
• addedNodes contains a new unknown node (non-empty)
• Validator compares: declared true + actual mutation present → PASS

Case B: Semantic-only no-op with same meaning

userSupportedMeaning: "uncertainty about projected office savings realism"
proposal: no meaningful graph mutation
Declaration options: structuralActionRequired = false (intentional) or structuralActionRequired = true (noncompliant)

What the model can declare:

• If equivalent uncertainty already exists in the graph → structuralActionRequired = false is valid. The model has legitimately determined no new structure is needed.
• If no equivalent exists and the answer introduces genuinely new material → structuralActionRequired = true is required by rule #6.

Exactly what deterministic validation does:

1. Check structuralActionRequired is populated (not null) because userSupportedMeaning is populated
2. Compare declared value against proposal shape:
   • true + empty mutations → REJECT (contradiction)
   • false + empty mutations → PASS (explicit no-op declaration validated against zero mutation)
   • false + non-empty mutations → ACCEPT with diagnostic note (model did more than declared)

Classification of preferred design:

A — ACTUAL CONTRACT ENFORCEMENT

This is a contract at the structural level: the model declares its intent in a structured field, and code verifies that the proposal shape matches. If the model declares false (no change needed), validation passes because it checks the actual empty mutation state — not semantic similarity. The boundary between "intentional no-op" and "noncompliant no-op" is enforced by requiring the explicit declaration.

This solves the boundary because:

• Noncompliant zero-mutation outputs cannot hide behind empty arrays (they must also declare true, which fails validation)
• Intentional no-ops are valid when equivalent structure already exists (model declares false, validation confirms empty mutation)

---

Part 8 — Recommendation

Recommended option: B — boolean structural-action contract

Exact new field

structuralActionRequired: boolean | null
nullable during transition: YES (but rejected if userSupportedMeaning is populated and field is null)

Location in schema

Add to answerMeaningSchema in lib/graph/schema.js:

export const answerMeaningSchema = z.object({
  userSupportedMeaning: z.string().min(1),
  possibleInference: z.string().nullable().optional(),
  supportCategory: z.enum(...).nullable().optional(),
  resolutionGuidance: z.enum(...).nullable().optional(),
  structuralActionRequired: z.boolean().nullable().optional(),  // NEW
});

Exact validator invariants (in lib/graph/utils.js, in validateGraphUpdate)

After the existing hasMeaningfulChange check (around line 876):

if (!hasMeaningfulChange) {
  if (update.answerMeaning?.structuralActionRequired === false) {
    // Intentional no-op — model declared no change needed, and proposal confirms it
    // PASS — this is the "semantic agreement, no structural change" case
  } else if (!update.answerMeaning?.structuralActionRequired) {
    errors.push(
      "answerMeaning.structuralActionRequired must be populated when userSupportedMeaning is present."
    );
  } else if (update.answerMeaning?.structuralActionRequired === true) {
    errors.push(
      "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress."
    );
  }
}

Transition policy: B — missing new field + populated userSupportedMeaning is rejected

Reason: The current architecture has userSupportedMeaning as a commitment signal. If we accept zero-mutation proposals without the new field, every noncompliant output becomes valid again. During transition, reject until the model produces the new field. After the field is present, allow it as the enforcement mechanism.

---

Convergence

This is ready for bounded implementation. The design is minimal: one boolean field and one invariant check. It does not invent new semantic taxonomies. It does not require keyword/synonym logic. It is provider-agnostic because it validates structured output fields, not model behavior.

The boundary it solves: the gap between "model understands the meaning" and "model declares its structural intent deterministically." With this contract, the validator checks a declared boolean against actual proposal shape — no semantic parsing needed.