Files
confidence-engine/docs/experiment-57j39.md
T

4.3 KiB

Experiment 57J.39 — Semantic-to-Mutation Contract Implementation (Option B)

Objective: Implement the agreed Option B from 57J.38 with ownership correction: prompt owns structural materialization obligation, validator owns only the structural fact that answerMeaning alone is not graph progress.

Implementation boundary (strict):

  1. One MUST rule in prompt Additional Guidance (replaced rule #6 in prompt-builder.js)
  2. One deterministic check in validateGraphUpdate() after hasMeaningfulChange (utils.js)
  3. Focused tests proving each contract case

Changes to production code:

Prompt contract (lib/graph/prompt-builder.js)

Replaced ambiguous rule #6 ("Then inspect the answer for newly introduced consequential uncertainty.") with explicit MUST:

"If answerMeaning.userSupportedMeaning contains consequential information or unresolved uncertainty that is not already represented in the graph, you MUST express its effect through structural mutation. This may be an update/refinement of existing structure, resolution of an existing unknown, a genuinely new unknown, or a justified relationship. answerMeaning alone is not sufficient for a successful proposal."

Validator contract (lib/graph/utils.js)

Added specific diagnostic inside the existing !hasMeaningfulChange rejection path:

"answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress."

This diagnostic fires BEFORE the generic "Update contains no meaningful change" only when userSupportedMeaning is populated AND there is zero structural mutation. The generic error remains for all other structurally empty proposals.

Not changed:

  • hasMeaningfulChange definition (variable still computes the same structural fields)
  • Schema
  • Graph node/edge semantics
  • Provenance, answerability, decomposition, reasoning taxonomy
  • Semantic overlap rules or classifiers
  • Provider integration or Behaviour Selection
  • possibleInference handling

Tests added:

utils.test.js — semantic-to-mutation contract (8 tests):

  1. semantic-only no-op → REJECT with specific error (not generic)
  2. ordinary no-op (answerMeaning null) → REJECT with "no meaningful change"
  3. possibleInference only → does NOT trigger new error, generic no-op applies
  4. update existing structure (status change) → ACCEPT past guard
  5. resolve existing unknown → counts as structural progress
  6. add new structure (new unknown) → counts as structural progress
  7. duplicate avoidance preserved with populated userSupportedMeaning
  8. value-only change → counts as structural progress

prompt-builder.test.js — MUST rule verification (7 tests): 9-15. Verify prompt contains MUST rule, permits update/resolve/new unknown, states answerMeaning alone insufficient, does not force new node, references userSupportedMeaning not possibleInference

Test results:

  • utils.test.js: 68 passed (0 failed)
  • prompt-builder.test.js: 15 passed (0 failed)
  • cases-update-route.test.js: 13 passed (0 failed)
  • harness tests: 8 passed (0 failed)
  • rejected-proposal-snapshot.test.js: 7 passed (0 failed)
  • orchestrator.test.js: 31 passed, 1 pre-existing failure (unrelated)

What this implementation now guarantees:

  • A proposal with populated userSupportedMeaning and zero structural mutation receives a specific, actionable rejection error — not the generic no-op message
  • The prompt explicitly instructs the model that meaningful user-supported meaning must be expressed through graph structure, not just stated in answerMeaning
  • No new semantic classifier, schema change, or provider-specific logic is introduced
  • possibleInference alone does not trigger the specific diagnostic
  • Duplicate avoidance and all existing validation behavior is preserved

What it intentionally does NOT guarantee:

  • That userSupportedMeaning contains truly consequential meaning (validator doesn't judge that)
  • That the LLM will comply with the MUST rule in live use (that requires empirical verification)
  • Resolution of cold-start variance or other downstream defects

Classification: E — IMPLEMENTATION COMPLETE Configured Ollama: none used. Production code changed: prompt-builder.js, utils.js. Tests permanently changed: utils.test.js (+8), prompt-builder.test.js (+7). Dev server disturbed: NO. Ollama calls: 0.