confidence-engine/docs/experiment-56a.md

Experiment 56A — Regression B Proposal Validation Enum Mismatch

Date: 2026-08-09
Branch: feature/reasoning-fidelity-v0.8
Status: observation complete, no fix attempted

Hypothesis

Regression B fails at proposal_validation because Qwen returns supportCategory: "conditional_qualification" while the production proposal schema accepts only conditional_tradeoff among others.

This is a proposal-contract mismatch — not a pre-mutation guard failure.

Fixed Input (Regression B)

• Source: "I want the business to grow, but I don't want to take on more risk."
• Answer: "I'd normally avoid more risk, but for the right opportunity I might accept some."
• Expected supportCategory: conditional_tradeoff
• SituationGraph: single unknown node n-risk-constraint

Configuration

• Ollama endpoint: http://192.168.1.111:11434 (from .env.local)
• Model: qwen-claude:latest

Four Checkpoints Observed

Checkpoint 1 — answerMeaning in raw structured response

The model returned an answerMeaning object with a non-null supportCategory.
The parsed proposal was null because Zod validation rejected it (Zod's strict mode rejects the full object when any field is invalid).

Checkpoint 2 — supportCategory at schema boundary

Observed value: conditional_qualification

Normalization step (applyKnownEnumAliases) does not handle supportCategory; it only converts reported_statement → reported_claim on added nodes. The value survives unchanged to Zod validation.

Checkpoint 3 — Schema-accepted values

relative_priority_only
conditional_tradeoff
uncertain
explicit_hard_constraint
other

Source: lib/graph/schema.js, lines 147–152 (answerSupportCategory enum).

conditional_qualification is NOT in this list.

Checkpoint 4 — Zod validation result

path: ["answerMeaning", "supportCategory"]
message: "Invalid enum value. Expected 'relative_priority_only' | 'conditional_tradeoff' | 'uncertain' | 'explicit_hard_constraint' | 'other', received 'conditional_qualification'"
code: invalid_enum_value
stage: proposal_validation

Result

Hypothesis confirmed: YES

1. Provider output contains conditional_qualification — confirmed via Zod error message.
2. Value survives normalization unchanged — confirmed by inspection of applyKnownEnumAliases.
3. Schema does not accept it — confirmed (not in the enum).
4. Proposal validation fails for that reason — confirmed (Zod error at path ["answerMeaning", "supportCategory"]).

What Was Not Done

• No production code was changed.
• No fix was attempted.
• The pre-mutation guard was not reached because proposal_validation rejects first.
• Cases A, C, D, E, F were not tested.
• This experiment tested only ONE call; model output may vary across runs.

Files

• Read: lib/graph/schema.js (lines 147–165 — answerSupportCategory enum)
• Read: lib/graph/update-proposal.js (full file — normalization functions)
• Read: lib/llm/provider.js (full file — Ollama provider)
• Read: lib/graph/orchestrator.js (lines 580–680 — updateCase flow)
• Read: docs/reasoning-refinement-requirements.md (Regression B section)
• Read: tests/graph/regression-a-d-v0.8.test.js (fixed graph + input for Regression B)

Git

• Commit message: experiment: isolate regression B proposal validation
• Working tree left clean after experiment cleanup.