confidence-engine/docs/experiment-57j66.md

Experiment 57J.66 — Where structuralActionRequired Belongs and What Contradictions Reject

Branch: feature/selected-question-contract-v0.22 Starting HEAD: 9425e7b (experiment: define semantic action contract)

Objective

Answer exactly:

Should structuralActionRequired belong inside answerMeaning or at the top-level graph-update proposal, and what exact invariant matrix makes it a real contract rather than advisory metadata?

57J.65 established the boolean declaration is required and recommended placing it inside answerMeaningSchema. This experiment re-examines that recommendation and resolves the contradiction semantics.

Classification: READ-ONLY ARCHITECTURE DECISION. No production code changed.

---

Part 1 — Field Ownership

Option A — inside answerMeaning

// Current answerMeaningSchema (schema.js line 161):
answerMeaningSchema = {
  userSupportedMeaning,     // string — semantic content
  possibleInference,        // string | null — model's own interpretation
  supportCategory,          // enum | null — category label
  resolutionGuidance,       // enum | null — resolution instruction
  structuralActionRequired, // boolean | null  ← proposed (57J.65)
}

PROS:

• Follows 57J.65's recommendation directly
• Keeps all model-derived answer fields in one sub-object
• Minimal schema change count (one file: schema.js)
• The prompt currently lists answerMeaning keys as a single group — adding there keeps the model seeing all answer-related fields together

CONS:

• structuralActionRequired is NOT about meaning — it is about graph-mutation intent
• The validator checks this field against structural arrays (addedNodes, updatedNodes, addedEdges), not against meaning fields. Having it nested under answerMeaning obscures what it actually validates against
• Conflates semantic analysis with structural action decision: these are conceptually orthogonal layers
• Future structural fields (if any, e.g., structuralReason, actionScope) would need to stay outside answerMeaning anyway
• The prompt's "Required JSON Field Names" section lists top-level fields separately from answerMeaning keys — placing a structurally-decisive field inside answerMeaning creates cognitive separation between the field and its structural consequences

Option B — top-level proposal field

// Current graphUpdateSchema (schema.js line 184):
graphUpdateSchema = {
  addedNodes,
  updatedNodes,
  addedEdges,
  removedEdgeIds,
  resolvedUnknownNodeIds,
  affectedNodeIds,
  selectedQuestion,       // node reference — structural decision
  answerMeaning,          // semantic content
  structuralActionRequired, // boolean | null ← proposed (57J.66)
}

PROS:

• Clearly separates what the user means (answerMeaning) from whether that meaning requires graph change (structuralActionRequired at top level)
• Aligns with where the validator actually evaluates it: the validator checks structural arrays and the boolean simultaneously
• selectedQuestion already sits at this level as another structural decision — structuralActionRequired is a peer, not an outlier
• Cleaner schema evolution: if we later add related structural fields (e.g., structuralReason), they stay with other structural decisions
• Avoids conflating meaning with action: the field's placement communicates its role

CONS:

• Moves away from 57J.65's specific recommendation
• The prompt's "Required JSON Field Names" and "Required Shapes" sections would need two additions (one to each list) instead of one
• Model sees this alongside mutation arrays, which is correct but may increase cognitive load slightly

Chosen placement: TOP_LEVEL

Why: structuralActionRequired expresses graph-mutation intent, not semantic meaning. The validator evaluates it against structural arrays (addedNodes, updatedNodes, addedEdges), not against meaning fields. Placing it at the proposal level keeps semantic analysis separate from structural action decisions, and aligns with where the field is actually used in validation logic. The distinction between "what the user means" and "whether that meaning requires graph change" should be architecturally visible in the schema itself, not only in documentation.

---

Part 2 — Meaning / Action Independence

Case A: userSupportedMeaning populated + structuralActionRequired = true

VALID

The model extracts user-supported meaning from the answer AND declares that this meaning requires graph action. This is the primary positive case: the answer introduces consequential, unresolved information not already in the graph, and the model both captures it and claims structural mutation is needed.

Case B: userSupportedMeaning populated + structuralActionRequired = false

VALID

The model extracts user-supported meaning but determines no graph change is needed because existing graph state already fully represents the user-supported meaning (e.g., v0.21 identity rule: an equivalent unresolved uncertainty already exists). The model intentionally declares a semantic agreement with zero structural change.

Case C: userSupportedMeaning = null + structuralActionRequired = true

VALID ONLY UNDER SPECIFIC EXISTING CASE

Conceptually possible when the model decides structural action is needed despite not extracting meaningful content from the answer. Examples: graph maintenance (cleaning orphaned structure), resolving an unknown that was already established in prior turns, or reacting to a non-answer prompt. However, in typical flow this would indicate the model should have populated userSupportedMeaning — it's valid only when there is a genuine reason for structural action independent of fresh meaning extraction.

Case D: userSupportedMeaning = null + structuralActionRequired = false

VALID

The simplest no-op case: nothing to extract from the answer and nothing to change in the graph. This covers neutral acknowledgments, non-informative answers, or cases where existing state fully suffices. Under transition policy B (see Part 5), if userSupportedMeaning is null and structuralActionRequired is missing/null, existing behavior is retained (reject with "no meaningful change").

---

Part 3 — Meaningful Mutation Definition

Current production definition (lib/graph/utils.js lines 869–881):

const statusChanged = update.updatedNodes.some(
  (u) => u.previousStatus !== null && u.newStatus !== u.previousStatus,
);
const valueChanged = update.updatedNodes.some(
  (u) => (u.previousValue ?? null) !== (u.newValue ?? null),
);

const hasMeaningfulChange =
  update.addedNodes.length > 0 ||
  statusChanged ||
  valueChanged ||
  update.addedEdges.length > 0 ||
  update.removedEdgeIds.length > 0;

This checks five conditions: (1) new nodes added, (2) node status changed, (3) node value changed, (4) edges added, (5) edges removed.

Contract should: REUSE EXISTING DEFINITION

Why: structuralActionRequired = true directly means "this proposal claims graph mutation is required." hasMeaningfulChange directly measures whether the proposal contains any graph mutation. These are the same boundary expressed at different abstraction levels:

• true → expects hasMeaningfulChange === true
• false → expects hasMeaningfulChange === false (or accepts it as advisory if model adds extra structure)

Creating a separate definition would split what is conceptually one check into two subtly different boundaries — precisely the kind of drift this contract was designed to prevent. No new mutation definition is needed or desirable.

---

Part 4 — Contradiction Matrix

1: structuralActionRequired = true + meaningful mutation = true

PASS

Model declares action needed, and proposal contains meaningful mutations. Contract fulfilled. The validator confirms the declaration matches reality.

2: structuralActionRequired = true + meaningful mutation = false

REJECT

Model claims graph action is required but produces zero mutations. This is a contract violation: the model either misunderstood the answer's implications or failed to execute on its own declaration. The deterministic error is "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation."

3: structuralActionRequired = false + meaningful mutation = false

PASS

Model declares no action needed, and proposal confirms zero mutations. This is the valid intentional no-op case. The model has explicitly declared its intention; validation passes because it can do so deterministically without semantic parsing.

4: structuralActionRequired = false + meaningful mutation = true

ACCEPT (advisory — not REJECT)

Model declares minimal action needed but the proposal contains more structure than declared. This is not a contradiction in the harmful sense:

• The model's declaration means "I believe at least this much change is needed"
• The actual proposal goes further, adding useful structure beyond what was declared
• There is no semantic loss, no misrepresentation, and no harm to the user

Contract semantics: ADVISORY

The boolean is a minimum intent declaration, not an exact action spec. The model declares "I need at least this much change" — producing more is acceptable because it still advances the investigation. A diagnostic warning should be logged but the proposal accepted.

Challenge of 57J.65's proposal (Part 6, invariant 2): 57J.65 proposed accepting false + mutation with a diagnostic note. This analysis confirms that recommendation but goes further: it explicitly classifies the contract as advisory rather than strict, which matters for future design decisions about what happens when declarations deviate from reality.

---

Part 5 — Missing/Null Field Transition

Chosen policy: B

missing/null + populated userSupportedMeaning → reject
missing/null + no userSupportedMeaning → retain existing behaviour

Why:

• Transition necessity: If we accept zero-mutation proposals without structuralActionRequired, every noncompliant output (model forgot to mutate) becomes valid again. During transition, the field must be required whenever there is meaningful content to justify structural action.
• Backward compatibility: When userSupportedMeaning is null/no-populated, existing behavior ("Update contains no meaningful change") covers the rejection case. The new contract only adds constraints on top of what already exists — it does not remove any existing checks.
• Legacy test impact: Existing tests that don't populate structuralActionRequired but have empty mutation arrays will behave identically to today when answerMeaning is null (rejected with "no meaningful change"). Tests with populated userSupportedMeaning will fail at validation until the field is added — this is intentional and correct.
• Live model transition: The prompt addition must explicitly require the field. Until the prompt changes, the validator's rejection of missing-field-with-meaning prevents silent degradation.

---

Part 6 — Semantic Truth Boundary

Can deterministic code verify that structuralActionRequired = false is semantically correct?

NO (deterministic code cannot prove semantic correctness)

What the boolean actually guarantees: Contract consistency, not semantic truth.

semantic truth:
  whether the user's meaning genuinely requires graph action

contract consistency:
  whether the proposal shape matches the model's declared action requirement

Deterministic code can only verify contract consistency: does the boolean match the mutation arrays? If false + zero mutations → the declaration is consistent. Code cannot independently prove the model was correct to declare false — that would require understanding what the user's answer genuinely demands, which means re-reading English semantics and making a semantic judgment. The boolean's purpose is precisely to avoid requiring that judgment: it delegates the semantic judgment to the model and only checks consistency.

What structuralActionRequired guarantees:

1. The model explicitly declared its structural intent (no more silence)
2. The proposal shape matches the declaration (or advisory note is logged)
3. Noncompliant zero-mutation outputs cannot hide behind empty arrays

It does NOT guarantee:

• The model made the correct semantic judgment about whether action was needed
• No useful graph structure was omitted
• The answer didn't warrant more than what was produced

---

Part 7 — Does false + Empty Become a Valid No-Op?

For non-meaning inputs: ACCEPTED

userSupportedMeaning = null (or not populated)
structuralActionRequired = false
zero meaningful mutation
→ ACCEPTED

Why: There is no populated meaning to evaluate. The model explicitly declared nothing requires graph change, and zero mutations confirm the declaration. Deterministic code trusts the model's structured declaration rather than independently proving it — which is appropriate because there is nothing independent to prove against.

Architectural meaning of acceptance: The contract shifts from "silence = error" to "explicit no-op = valid." This means deterministic code is trusting the model's structured declaration rather than independently proving correctness. For non-meaning inputs, this is safe: there are no semantics to get wrong. For meaning-populated inputs, the contract allows intentional no-ops only when structuralActionRequired = false (advisory if extra mutations present).

Architectural meaning of rejection: If we rejected all zero-mutation proposals regardless of content, we would force the model into one of two behaviors: either always propose mutation (even when unnecessary), or omit userSupportedMeaning (losing semantic fidelity to avoid structural pressure). The intentional no-op path preserves both semantic extraction and structural correctness.

---

Part 8 — Prompt Contract Implication

Minimum prompt obligation: SUFFICIENT

The prompt must tell the model two things:

1. Set true: when the answer requires any graph progress (new unknown, updated node, resolved node, added edge)
2. Set false: only when existing graph state already fully represents the user-supported meaning or no graph progress is justified

These two rules are sufficient. They cover every case:

• true covers all scenarios where structural action is needed
• false covers both "semantic agreement with existing state" and "nothing to do"
• The transition policy (B) handles the missing-field gap

No additional principle is required. Adding more rules would expand the prompt framework without improving clarity — the two-rule distinction maps cleanly to the boolean domain.

---

Part 9 — Final Implementation Decision

Chosen option: D

top-level field + advisory false/mutation handling

Exact schema shape and transition nullability:

New field in graphUpdateSchema:

structuralActionRequired: z.boolean().nullable().optional(),

Nullable during transition: YES. Once the prompt requires it, treat as mandatory when userSupportedMeaning is populated (validator rejects missing-field-with-meaning).

---

Convergence

READY FOR BOUNDED IMPLEMENTATION: YES

The design resolves all previously ambiguous decisions:

• Field location: top-level graphUpdateSchema (not inside answerMeaning)
• Semantics: advisory for false+mutation, strict for true+no-mutation
• Null transition: policy B — reject when meaning is populated, retain existing behavior otherwise
• Contradiction matrix: fully specified in Part 4 above

---

Required Implementation Boundary (if READY)

Files changed:

1. lib/graph/schema.js — add structuralActionRequired to graphUpdateSchema
2. lib/graph/utils.js — update validator logic around hasMeaningfulChange
3. lib/graph/prompt-builder.js — add field to required fields list + prompt rule for true/false
4. tests/graph/utils.test.js — new tests for the contract

New tests:

1. true + meaningful mutation → pass;
2. true + zero mutation → reject;
3. false + zero mutation → pass (intentional no-op);
4. false + meaningful mutation → accept with diagnostic note;
5. missing/null + populated userSupportedMeaning → reject (transition policy B);
6. missing/null + no userSupportedMeaning → retain existing "no meaningful change" rejection;
7. existing hasMeaningfulChange semantics remain unchanged for non-contract paths;
8. supportCategory remains independent of structuralActionRequired;
9. equivalent existing uncertainty can legitimately produce false when no mutation is required (v0.21 identity rule);
10. no keyword/synonym/raw-English semantic logic added anywhere.

Scope exclusions (intentionally out of scope):

• retry/regeneration
• mutation enums
• scoring
• evidence linkage
• provider-specific behaviour
• semantic similarity detection
• keyword classifiers

What this intentionally leaves unresolved:

• Whether the advisory false + mutation path should eventually become strict
• Whether structuralActionRequired should eventually carry additional fields (e.g., structuralReason)
• Whether the prompt rule needs refinement based on live model behavior under the contract

---

Classification: READ-ONLY ARCHITECTURE DECISION. No production code changed. No Ollama calls. No tests modified.