confidence-engine/docs/experiment-57j49.md

# Experiment 57J.49 — Can Existing Structured Semantic Fields Replace Keyword-Based Fidelity Inference?

**Branch:** `feature/semantic-to-mutation-contract-v0.19`
**Starting HEAD:** `a2c790e` experiment: diagnose uncertainty fidelity false positive

## Objective

Answer exactly:

> Can the current answerMeaning/schema contract carry enough structured semantic information to let fidelity validation compare meaning directly, instead of re-inferring uncertainty/constraint/trade-off semantics from English keywords?

This is a **read-only architecture diagnosis** following 57J.48's lexical false positive finding.
Do not call Ollama. Do not run the live API. Do not modify production code, prompts, validators, schema, or tests.

## Part 1 — Inventory of Existing Structured Semantics

For `answerMeaning` and directly related proposal fields:

### Field: userSupportedMeaning

```text
type: z.string().min(1)
required/optional: required (min length 1)
nullable: NO
populated by: LLM — model restates user meaning in third-person descriptive language
consumed by: deriveAnswerMeaningProfile() → keyword detection → category; validateAnswerMeaningCompatibilityWithRawAnswer(); validateAnswerMeaningAlignment()
survives proposal validation: YES (passes Zod schema parse as a required string field)
purpose: Primary carrier of what the user's answer semantically establishes; the sole structured semantic field actually populated by the model in production. All downstream category derivation flows through this text via keyword detection.
```

### Field: possibleInference

```text
type: z.string().nullable()
required/optional: optional (nullable)
nullable: YES — can be null or absent
populated by: LLM — when model wants to express a stronger interpretation beyond user meaning
consumed by: Only through rejectedProposalSnapshot passthrough in orchestrator.js. NOT consumed by any validator, classifier, or fidelity check. No production code examines possibleInference for any decision.
survives proposal validation: YES (passes Zod schema parse as optional nullable)
purpose: Intended for separating stronger model interpretations from user-supported meaning. Currently dead/pass-through — exists in schema and prompt but no validator inspects it.
```

### Field: supportCategory

```text
type: z.string().min(1).nullable() — FREE TEXT (no enum constraint enforced)
required/optional: optional (nullable)
nullable: YES
populated by: Prompt requests it, but LLM consistently produces null in all tested experiments. Confirmed by 56D: "the LLM does not auto-populate supportCategory." The deterministic derivation layer is the sole mechanism for meaning profile category determination.
consumed by: validateAnswerMeaningAlignment() would use it IF populated (lines 3010-3011). deriveAnswerMeaningProfile() derives category from text — NOT from this field.
survives proposal validation: YES (passes Zod as free text), but no schema constraint enforces valid values against answerSupportCategory enum
purpose: Intended as a structured semantic classification carrier that the model self-assigns. In production: never populated by model, so it carries no information. The enum answerSupportCategory exists at lines 147-153 of schema.js but is not used to constrain this field.
```

### Field: resolutionGuidance

```text
type: z.string().min(1).nullable() — FREE TEXT (no enum constraint enforced)
required/optional: optional (nullable)
nullable: YES
populated by: Prompt requests it, but LLM consistently produces null in all tested experiments. Same pattern as supportCategory.
consumed by: validateAnswerMeaningAlignment() would use it IF populated (line 3011, check at 3013). deriveAnswerMeaningProfile() derives guidance from text — NOT from this field.
survives proposal validation: YES (passes Zod as free text), but no schema constraint enforces valid values against answerResolutionGuidance enum
purpose: Intended to convey whether the semantic content requires remaining unresolved, may resolve, or must resolve. In production: never populated by model, so it carries no information.
```

### Field: uncertaintyType

```text
type: NOT PRESENT in production schema — only exists in experimental test fixtures (tests/reconstruction/semantic-regression-e-f*.test.js) and experiment documentation
required/optional: N/A — not in any production contract
populated by: N/A — not part of answerMeaningSchema or prompt instructions
consumed by: N/A — no production code references it
survives proposal validation: N/A
purpose: Experimental concept from 57J.48 documentation describing a potential structured uncertainty classification. Has never existed in the production schema or model contract.
```

## Part 2 — Captured-Case Representation

**Raw answer:** "I am unsure whether the projected office savings from the relocation are realistic."

**Semantically faithful model meaning:** "The user is currently uncertain whether the projected office savings from the relocation are realistic."

**Shared semantic fact to express:** `meaning remains unresolved uncertainty`

### Can current fields express this without lexical inference?

**PARTIAL**

The minimum existing field/value combination that would express it (if populated by the model):

```json
{
  "userSupportedMeaning": "The user is currently uncertain whether the projected office savings from the relocation are realistic.",
  "supportCategory": "uncertain",
  "resolutionGuidance": "must_remain_unresolved"
}
```

- `supportCategory: "uncertain"` — directly expresses the uncertainty classification (one of five values in answerSupportCategory enum)
- `resolutionGuidance: "must_remain_unresolved"` — directly expresses that resolution is not appropriate (one of three values in answerResolutionGuidance enum)

**Why PARTIAL, not YES:** These two fields (supportCategory and resolutionGuidance) are the correct carriers but are **never populated by the model** in production. The validator currently cannot consume them because they are null. The structured capability exists in the schema design but is unreachable — no code path populates these fields with actual classification values, only userSupportedMeaning carries information end-to-end.

Additionally:
- Both fields are free-text Zod types (no enum constraint enforcement), so even if populated, there is no structural guarantee they contain valid category values.
- `uncertaintyType` does not exist in the production schema at all — a dedicated structured uncertainty classifier field would need to be added or supportCategory used for that purpose.

## Part 3 — Current Population Path

### supportCategory: **B — schema exists but prompt does not clearly require population**

**Why:** The prompt (prompt-builder.js line 28) says "supportCategory and resolutionGuidance are optional descriptive hints only; if you are unsure of the exact wording, leave them null rather than inventing rigid category labels." This explicit permission to remain null explains why the LLM consistently produces null. The schema does not enforce population (optional + nullable + free-text). Combined: schema says "nullable," prompt says "leave null if unsure" — no mechanism drives model to populate it.

### resolutionGuidance: **B — schema exists but prompt does not clearly require population**

**Why:** Same mechanism as supportCategory. Prompt line 28 explicitly tells the model it can leave it null. Schema marks it optional + nullable. No enforcement.

### possibleInference: **D — field is derived/populated conditionally by model but has no downstream consumer**

**Why:** The model populates this when it wants to express a stronger interpretation beyond what the user stated. It survives validation as a pass-through field but is never examined by any validator, classifier, or fidelity check. Its existence is effectively cosmetic — it exists in the contract but carries no functional weight.

### uncertaintyType: **NOT PRESENT**

**Why:** This field has never existed in the production answerMeaning schema. It appears only in experimental test fixtures (57J.48 documentation references it as a potential structured signal, and tests for semantic-regression-e/f use it as a model output from inference calls, not from the graph-update contract).

## Part 4 — Current Validator Dependency

### Uncertainty

```text
current source: RAW TEXT
deriveAnswerMeaningProfile() lines 2877-2883: .includes() checks on ["not really sure", "not sure", "unsure", "do not know", "don't know"] → category = "uncertain"

existing structured replacement available: PARTIAL
supportCategory could carry the uncertainty classification (one of five enum values includes "uncertain"), but model never populates it. No other field carries uncertainty classification.

would replacement require new semantic taxonomy: NO
"uncertain" already exists in answerSupportCategory enum at line 150 of schema.js
```

### Conditional/trade-off

```text
current source: MIXED (hasConditionalQualification keyword detection + conditionalPreferenceStructure compound check)
deriveAnswerMeaningProfile() lines 2893-2904 uses hasConditionalQualification(text) [includes("might","normally","for the right opportunity","depends","conditional","under specific")] plus hasDefaultPreferenceSignal + hasExceptionOrOverrideSignal

existing structured replacement available: PARTIAL
supportCategory could carry "conditional_tradeoff" (enum value at line 149 of schema.js). But model never populates it.

would replacement require new semantic taxonomy: NO
"conditional_tradeoff" already exists in answerSupportCategory enum at line 149
```

### Hard constraint

```text
current source: RAW TEXT
mentionsHardConstraint(text) at line 2834: includes("hard constraint","constraint","non negotiable","non-negotiable")
mentionsNegatedHardConstraint(text) at line 2843: included for negation detection

existing structured replacement available: PARTIAL
supportCategory could carry "explicit_hard_constraint" (enum value at line 151 of schema.js). But model never populates it.

would replacement require new semantic taxonomy: NO
"explicit_hard_constraint" already exists in answerSupportCategory enum at line 151
```

### Resolution semantics

```text
current source: RAW TEXT → deriveAnswerMeaningProfile() resolutionGuidance derivation (lines 2886, 2902, 2909, 2922) or fallback null
Derived from text patterns: uncertainty phrases → "must_remain_unresolved", conditional → "may_resolve", hard constraint → "must_resolve", else null

existing structured replacement available: PARTIAL
resolutionGuidance field exists for this purpose, and three valid values exist in answerResolutionGuidance enum. But model never populates it, so deriveAnswerMeaningProfile() must re-derive from text.

would replacement require new semantic taxonomy: NO
"must_remain_unresolved", "may_resolve", "must_resolve" all exist in answerResolutionGuidance enum at lines 156-158
```

### Relative priority (not explicitly asked but relevant)

```text
current source: RAW TEXT
deriveAnswerMeaningProfile() lines 2913-2924: .includes() checks on ["matters more", "more important", "higher priority", "greater relative importance", "relative importance"] → category = "relative_priority_only"

existing structured replacement available: PARTIAL
supportCategory could carry "relative_priority_only" (enum value at line 148 of schema.js). But model never populates it.

would replacement require new semantic taxonomy: NO
"relative_priority_only" already exists in answerSupportCategory enum at line 148
```

## Part 5 — Trust-Boundary Problem

### Pattern A — trust model classification directly

Model supplies structured category; validator compares category to category.

```text
removes lexical dictionary dependence: YES (for all protected categories simultaneously, provided model populates supportCategory)
preserves fidelity protection: PARTIAL (depends on reliable model population; if model lies about its own classification, validator has no independent check — the current keyword inference provides that independent check but with lexical coverage gaps)
requires new schema fields: NO (supportCategory already exists; enum values cover all protected categories)
requires new semantic taxonomy: NO (all five categories + three resolution_guidance values already exist in enums)
```

### Pattern B — model classification + raw-text lexical verification

Structured category is populated, but current keyword inference remains the authority. Validator checks both: model says X, keywords say Y → mismatch flag.

```text
removes lexical dictionary dependence: NO (still uses keyword detection as one of two inputs)
preserves fidelity protection: YES (cross-checks model claim against independent text analysis; catches both lexical gaps AND model hallucination)
requires new schema fields: NO
requires new semantic taxonomy: NO
```

### Pattern C — structured model claim + independent deterministic consistency checks that do NOT attempt full English semantic inference

Examples: schema invariants, cross-field consistency, structural plausibility.

```text
removes lexical dictionary dependence: PARTIAL (removes keyword detection for uncertainty classification where supportCategory is populated; remaining categories still use keywords when supportCategory is null)
preserves fidelity protection: PARTIAL (deterministic checks like "resolutionGuidance=must_remain_unresolved AND resolved=true" catch some contradictions but not all semantic inconsistencies — e.g., a wrong category with compatible text could pass)
requires new schema fields: NO
requires new semantic taxonomy: NO

Specific deterministic consistency checks already possible from existing fields:
1. If resolutionGuidance = "must_remain_unresolved" AND proposal resolves any unknown → CONTRADICTION (currently validated via derived text, would be directly checkable if field populated)
2. If supportCategory = "explicit_hard_constraint" AND userSupportedMeaning contains "rather than a hard constraint" or "not a hard constraint" → CONTRADICTION (cross-field consistency between category and meaning text)
3. If possibleInference is populated but userSupportedMeaning carries no new uncertainty → INCONSISTENCY (inference without meaningful supporting meaning)
4. supportCategory value should be one of answerSupportCategory enum values — currently not enforced by schema
5. resolutionGuidance value should be one of answerResolutionGuidance enum values — currently not enforced by schema
```

## Part 6 — Architecture Decision

### **B — EXISTING STRUCTURE IS PARTIAL**

Current fields cover some protected semantics but cannot replace lexical inference cleanly without a small structured-contract extension.

**What this establishes:**

1. The `supportCategory` enum (answerSupportCategory) already contains all five required classification values: relative_priority_only, conditional_tradeoff, uncertain, explicit_hard_constraint, other.
2. The `resolutionGuidance` enum (answerResolutionGuidance) already contains all three required resolution states: must_remain_unresolved, may_resolve, must_resolve.
3. These fields exist in the production schema and are explicitly requested in the prompt — the structured capability is designed but not operationalized.
4. The missing piece is **reliable model population** (prompt says "optional" and "leave null if unsure") and **schema enforcement** (both are free-text Zod strings, not constrained to their respective enums).

**What it does NOT establish:**

1. That structured output alone solves the trust problem — Pattern A reveals that trusting model classification directly has no independent verification.
2. That the existing enum taxonomy is complete — `uncertaintyType` (evidence_needed / user_clarification_needed) used in tests for regression cases E/F does not exist in any production schema. If this distinction matters, it requires new fields.
3. That adding field requirements to the prompt is sufficient — model compliance with "please fill these fields" has never been proven across repeated runs and domains.

## Production code changed: NO
## Prompt changed: NO  
## Validator changed: NO
## Tests permanently changed: NO
## Temporary test removed: YES (none created for this read-only diagnosis)
## Ollama calls made: 0
## Dev server disturbed: NO

---