confidence-engine/docs/experiment-57j51.md

Experiment 57J.51 — Structured Semantic Fidelity Implementation

Branch: feature/structured-semantic-fidelity-v0.20 Starting HEAD: b6a232ff6f56b5f1af49d94bb2881190b5bf8345 Production commit: 7d06cd3c473cee64c2c371c1e1af1c466cdc32dd

Objective

Implement Option C from Experiment 57J.50:

Use existing structured semantic fields (supportCategory, resolutionGuidance) as the primary fidelity contract when populated, enforce their allowed enum values, validate only structured cross-field consistency, and retain current lexical derivation only as a temporary fallback when those fields are null.

Scope Implemented

1. Schema

lib/graph/schema.js

• Constrained answerMeaning.supportCategory to z.enum(Object.values(answerSupportCategory)).nullable().optional();
• Constrained answerMeaning.resolutionGuidance to z.enum(Object.values(answerResolutionGuidance)).nullable().optional();
• Preserved transitional nullability on both fields;
• Reused existing enum constants — no new taxonomy added.

2. Prompt

lib/graph/prompt-builder.js

• Exposed allowed values for both structured semantic fields in the output contract;
• Replaced the old “optional descriptive hints only” instruction with structured population guidance;
• Instructed the model to:
  • populate supportCategory whenever the answer fits an existing category,
  • use other when none of the protected categories applies,
  • avoid leaving supportCategory null merely because wording is uncertain,
  • populate resolutionGuidance when one of the existing resolution states genuinely applies,
  • keep resolutionGuidance null only when no existing state actually applies;
• Used the existing formatEnumValues() helper;
• Added no provider-specific wording.

3. Validator — structured first

lib/graph/apply-proposal.js

• Added getAnswerMeaningProfile(answerMeaning) to unify:
  • structured supportCategory / resolutionGuidance when populated,
  • lexical derivation only when those structured fields are null;
• Updated validateAnswerMeaningCompatibilityWithRawAnswer() so populated structured semantic fields bypass raw-text lexical category verification entirely;
• Updated validateAnswerMeaningAlignment() so:
  • structured fields are authoritative when populated,
  • lexical fallback remains active only for legacy null cases.

4. Non-lexical consistency

Implemented one deterministic structured consistency check now:

• resolutionGuidance = must_remain_unresolved + proposal resolves an unknown → reject with:
  • Proposal resolves an unknown even though answerMeaning.resolutionGuidance is must_remain_unresolved.

Deferred one check intentionally:

• must_resolve target-specific enforcement was deferred because the current proposal structure does not safely identify the answered/targeted unknown in every valid case without inventing new linkage.

5. possibleInference

• Preserved current behaviour: possibleInference remains non-authoritative;
• It does not independently justify mutation;
• No validator path was added that treats it as authoritative structure.

Captured False Positive

The exact unsure → uncertain populated structured-path false positive is now removed.

Captured case

raw answer:
I am unsure whether the projected office savings from the relocation are realistic.

userSupportedMeaning:
The user is currently uncertain whether the projected office savings from the relocation are realistic.

supportCategory:
uncertain

resolutionGuidance:
must_remain_unresolved

Outcome

• Passes on the populated structured path;
• Does not depend on synonym logic;
• unsure vs uncertain wording is irrelevant when structured category is present.

Tests Added / Updated

Focused deterministic coverage added or updated in:

• tests/graph/schema.test.js
• tests/graph/prompt-builder.test.js
• tests/graph/apply-proposal.test.js
• tests/graph/update-proposal.test.js (directly related parse-boundary suite due to new enum enforcement)

Required outcomes

1. raw unsure + structured supportCategory=uncertain does not produce old lexical mismatch rejection — PASS
2. equivalent paraphrase wording does not change category acceptance when structured category is populated — PASS
3. invalid supportCategory rejected by schema — PASS
4. invalid resolutionGuidance rejected by schema — PASS
5. must_remain_unresolved + relevant resolution mutation rejected — PASS
6. must_resolve + unresolved target rejected if safely implementable — DEFERRED
7. null structured fields still use existing lexical fallback — PASS
8. populated conditional_tradeoff and explicit_hard_constraint use structured path without lexical verification — PASS
9. possibleInference remains non-authoritative — PASS
10. no new synonym/regex/keyword logic was added — PASS

Commands Run

npx vitest run tests/graph/schema.test.js tests/graph/apply-proposal.test.js tests/graph/prompt-builder.test.js
npx vitest run tests/graph/update-proposal.test.js

What this now guarantees

1. Populated structured semantic fields are now the primary fidelity contract.
2. The engine no longer re-derives protected semantic categories lexically when those structured fields are populated.
3. Invalid structured category/resolution values fail at schema parse time.
4. must_remain_unresolved is enforced through deterministic structured consistency rather than English keyword matching.
5. Legacy null structured proposals still follow the old lexical fallback path during transition.

What remains intentionally unresolved

1. Safe deterministic enforcement of must_resolve against a specific target unknown without inventing new linkage.
2. Population reliability of structured fields in live model runs.
3. Full retirement of the lexical fallback path once structured population is proven reliable.

Constraints respected

• No new semantic taxonomy;
• No synonym or regex expansion;
• No new semantic classifier;
• No new LLM call;
• No provider integration changes;
• No Ollama calls;
• No graph redesign.