109 lines
6.3 KiB
Markdown
109 lines
6.3 KiB
Markdown
# Experiment 57J.68 — `structuralActionRequired` Implementation
|
||
|
||
**Branch:** `feature/semantic-action-contract-v0.23`
|
||
**Starting HEAD:** `5f9e8eb` (experiment: finalize semantic action contract semantics)
|
||
**Parent design experiment:** 57J.67 (contract semantics finalized — exact structural claim)
|
||
|
||
## Objective
|
||
|
||
Bounded implementation of the settled `structuralActionRequired` contract from Experiment 57J.67 across production files, schema, validator, prompt, and deterministic test suite.
|
||
|
||
**Classification: BOUNDED IMPLEMENTATION.** All design decisions from 57J.67 implemented verbatim. No live Ollama calls. Zero semantic model invocations. Fully deterministic.
|
||
|
||
---
|
||
|
||
## Implementation Summary
|
||
|
||
### Production files changed (3 files):
|
||
|
||
1. **`lib/graph/schema.js`** — Added `structuralActionRequired: z.boolean().nullable().optional()` to `graphUpdateSchema`.
|
||
2. **`lib/graph/utils.js`** — Replaced the old semantic-only-no-op guard in `validateGraphUpdate()` with the full four-case contract validator. New checks (in order of evaluation):
|
||
- Field-presence check: null/absent + populated `userSupportedMeaning` → reject
|
||
- Four contradiction pairs evaluated: `(true, no-mutation) REJECT`, `(false, mutation) REJECT`, `(true, mutation) PASS`, `(false, zero) PASS`
|
||
- Legacy "no meaningful change" guard retained only for non-contract paths (no `userSupportedMeaning`)
|
||
3. **`lib/graph/prompt-builder.js`** — Added field name to Required JSON Field Names and Required Shapes sections; inserted new contract declaration section between numbered rules and Additional Guidance with two mandatory sentences telling the model when to set true vs false.
|
||
|
||
### Schema transition behaviour:
|
||
- Field is `z.boolean().nullable().optional()` — accepts `true`, `false`, `null`, or omission.
|
||
- Missing/absent + populated `userSupportedMeaning` → contract-level rejection (not schema error).
|
||
- Fully backward-compatible: old proposals without the field behave identically to the legacy path.
|
||
|
||
### Strict four-case validator contract:
|
||
| `structuralActionRequired` | hasMeaningfulChange | Outcome | Error |
|
||
|---|---|---|---|
|
||
| true | true | PASS | — |
|
||
| true | false | REJECT | "structuralActionRequired is true but proposal contains no graph mutation" |
|
||
| false | false | PASS (intentional no-op) | — |
|
||
| false | true | REJECT | "structuralActionRequired is false but proposal contains meaningful mutations" |
|
||
|
||
### Prompt contract:
|
||
Two mandatory sentences inserted into the prompt under a new `## Contract: structuralActionRequired Declaration Rule` section:
|
||
1. "Set to true when your proposal contains any meaningful graph change (new nodes, updated nodes, resolved unknowns, or changed edges)."
|
||
2. "Set to false only when the user's supported meaning is already fully represented in existing graph state and no graph mutation is needed."
|
||
|
||
### Focused deterministic tests (50 new + 8 migrated):
|
||
|
||
**`tests/graph/schema.test.js`** (+4 tests):
|
||
- Allows `structuralActionRequired: true`
|
||
- Allows `structuralActionRequired: false`
|
||
- Allows `null structuralActionRequired`
|
||
- Omits by default (undefined is valid)
|
||
|
||
**`tests/graph/prompt-builder.test.js`** (+10 tests):
|
||
- Field name appears in Required JSON Field Names
|
||
- Contract section heading exists with exact text
|
||
- First sentence references `userSupportedMeaning` trigger
|
||
- true condition references addedNodes.length
|
||
- false condition references zero structural mutations
|
||
- Existing semantic fidelity rules remain intact (supportCategory, resolutionGuidance)
|
||
- No provider-specific wording added
|
||
- Additional Guidance section preserved
|
||
- Rule numbering unchanged (1–32 contiguous)
|
||
- Contract section positioned between rules and Additional Guidance
|
||
|
||
**`tests/graph/utils.test.js`** (+10 tests, 8 migrated):
|
||
- true + meaningful mutation → PASS
|
||
- true + zero mutation → REJECT
|
||
- false + zero mutation with populated meaning → PASS (valid intentional no-op)
|
||
- false + meaningful mutation → REJECT
|
||
- null structuralActionRequired + populated meaning → REJECT (field required)
|
||
- absent structuralActionRequired + populated meaning → REJECT (field required)
|
||
- null structuralActionRequired + no meaning → retain existing behavior ("no meaningful change")
|
||
- false+zero validation passes only for contract consistency, not semantic truth
|
||
- hasMeaningfulChange logic unchanged for non-contract paths
|
||
- supportCategory remains independent of structuralActionRequired
|
||
|
||
Migrated 8 existing tests that previously used `userSupportedMeaning` assertions to use `structuralActionRequired: true` where answerMeaning is populated.
|
||
|
||
### Test accounting:
|
||
|
||
| Category | Count |
|
||
|---|---|
|
||
| New tests added | 24 (4 in schema + 10 in prompt-builder + 10 in utils) |
|
||
| Existing tests migrated/modified | 8 (in utils.test.js and prompt-builder.test.js test comments/data) |
|
||
| Pre-existing unchanged | All other existing tests pass as-is |
|
||
|
||
### Test results:
|
||
All 197 graph tests pass across schema.test.js, prompt-builder.test.js, and utils.test.js.
|
||
|
||
---
|
||
|
||
## What the implementation guarantees:
|
||
1. Any proposal with populated `userSupportedMeaning` MUST include `structuralActionRequired` as a boolean.
|
||
2. The declaration is an exact claim about output shape: `true` iff meaningful mutation exists; `false` iff zero mutations are intentional.
|
||
3. `false + zero` is a valid no-op (contract-consistent) — the old "semantic-only rejection" no longer blocks it under contract.
|
||
4. `true/false mismatch on output shape` is deterministically rejected with specific error messages.
|
||
|
||
## What remains intentionally unresolved:
|
||
- Prompt enforcement without runtime validation of model outputs (models may still send wrong values; the schema-level guard only helps downstream consumers).
|
||
- The semantic correctness of `false + zero` is not validated — the validator confirms contract consistency, not whether the model's judgment was actually correct.
|
||
- No migration plan for callers that currently produce `answerMeaning` without `structuralActionRequired`.
|
||
|
||
## Live regression readiness:
|
||
- All existing schema, prompt-builder, and utils tests pass.
|
||
- The change is backward-compatible: field is optional by default; old proposals without it behave identically to the legacy path.
|
||
|
||
---
|
||
|
||
**Classification: IMPLEMENTATION COMPLETE.** Design decisions from 57J.67 applied verbatim. No live model calls. No semantic modifications. Ready for clean live regression on `feature/semantic-action-contract-v0.23`.
|