72 lines
4.3 KiB
Markdown
72 lines
4.3 KiB
Markdown
### Experiment 57J.39 — Semantic-to-Mutation Contract Implementation (Option B)
|
|
|
|
**Objective:** Implement the agreed Option B from 57J.38 with ownership correction: prompt owns structural materialization obligation, validator owns only the structural fact that `answerMeaning` alone is not graph progress.
|
|
|
|
**Implementation boundary (strict):**
|
|
1. One MUST rule in prompt Additional Guidance (replaced rule #6 in prompt-builder.js)
|
|
2. One deterministic check in `validateGraphUpdate()` after `hasMeaningfulChange` (utils.js)
|
|
3. Focused tests proving each contract case
|
|
|
|
**Changes to production code:**
|
|
|
|
#### Prompt contract (lib/graph/prompt-builder.js)
|
|
Replaced ambiguous rule #6 ("Then inspect the answer for newly introduced consequential uncertainty.") with explicit MUST:
|
|
|
|
> "If answerMeaning.userSupportedMeaning contains consequential information or unresolved uncertainty that is not already represented in the graph, you MUST express its effect through structural mutation. This may be an update/refinement of existing structure, resolution of an existing unknown, a genuinely new unknown, or a justified relationship. answerMeaning alone is not sufficient for a successful proposal."
|
|
|
|
#### Validator contract (lib/graph/utils.js)
|
|
Added specific diagnostic inside the existing `!hasMeaningfulChange` rejection path:
|
|
|
|
> "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress."
|
|
|
|
This diagnostic fires BEFORE the generic "Update contains no meaningful change" only when `userSupportedMeaning` is populated AND there is zero structural mutation. The generic error remains for all other structurally empty proposals.
|
|
|
|
**Not changed:**
|
|
- `hasMeaningfulChange` definition (variable still computes the same structural fields)
|
|
- Schema
|
|
- Graph node/edge semantics
|
|
- Provenance, answerability, decomposition, reasoning taxonomy
|
|
- Semantic overlap rules or classifiers
|
|
- Provider integration or Behaviour Selection
|
|
- `possibleInference` handling
|
|
|
|
**Tests added:**
|
|
|
|
*utils.test.js — semantic-to-mutation contract (8 tests):*
|
|
1. semantic-only no-op → REJECT with specific error (not generic)
|
|
2. ordinary no-op (answerMeaning null) → REJECT with "no meaningful change"
|
|
3. possibleInference only → does NOT trigger new error, generic no-op applies
|
|
4. update existing structure (status change) → ACCEPT past guard
|
|
5. resolve existing unknown → counts as structural progress
|
|
6. add new structure (new unknown) → counts as structural progress
|
|
7. duplicate avoidance preserved with populated userSupportedMeaning
|
|
8. value-only change → counts as structural progress
|
|
|
|
*prompt-builder.test.js — MUST rule verification (7 tests):*
|
|
9-15. Verify prompt contains MUST rule, permits update/resolve/new unknown, states answerMeaning alone insufficient, does not force new node, references userSupportedMeaning not possibleInference
|
|
|
|
**Test results:**
|
|
- utils.test.js: 68 passed (0 failed)
|
|
- prompt-builder.test.js: 15 passed (0 failed)
|
|
- cases-update-route.test.js: 13 passed (0 failed)
|
|
- harness tests: 8 passed (0 failed)
|
|
- rejected-proposal-snapshot.test.js: 7 passed (0 failed)
|
|
- orchestrator.test.js: 31 passed, 1 pre-existing failure (unrelated)
|
|
|
|
**What this implementation now guarantees:**
|
|
- A proposal with populated `userSupportedMeaning` and zero structural mutation receives a specific, actionable rejection error — not the generic no-op message
|
|
- The prompt explicitly instructs the model that meaningful user-supported meaning must be expressed through graph structure, not just stated in answerMeaning
|
|
- No new semantic classifier, schema change, or provider-specific logic is introduced
|
|
- possibleInference alone does not trigger the specific diagnostic
|
|
- Duplicate avoidance and all existing validation behavior is preserved
|
|
|
|
**What it intentionally does NOT guarantee:**
|
|
- That `userSupportedMeaning` contains truly consequential meaning (validator doesn't judge that)
|
|
- That the LLM will comply with the MUST rule in live use (that requires empirical verification)
|
|
- Resolution of cold-start variance or other downstream defects
|
|
|
|
**Classification: E — IMPLEMENTATION COMPLETE**
|
|
Configured Ollama: none used. Production code changed: prompt-builder.js, utils.js. Tests permanently changed: utils.test.js (+8), prompt-builder.test.js (+7). Dev server disturbed: NO. Ollama calls: 0.
|
|
|
|
---
|