Experiment 56E — Weak Priority Through Live Production Path

Date: 2026-08-09
Commit: 3e78d57 (refine answer meaning derivation for negation and qualification)
Type: Observation-only — no code changes
Objective: Validate that the production path preserves only what the weak-priority answer establishes (relative importance) without inventing whether risk is or is not a hard constraint.

Input (Fixed)

Source: "I want the business to grow, but I don't want to take on more risk."
Answer: "Risk matters more to me."

Graph Setup

Pre-update graph state matched Regression A fixture:

n-risk-constraint (unknown/unknown) — active unknown, status=unknown
No source observation node
0 edges

Results

#	Checkpoint	Result
1	`userSupportedMeaning` extracted	❌ "Avoiding additional risk is a preference/trade-off rather than a hard constraint." — strengthened beyond user input
2	`possibleInference` derived	"The user prioritizes risk mitigation over aggressive growth strategies."
3	LLM-populated `supportCategory`	null (LLM does not auto-populate; nullable per schema)
4	Derived meaning profile category	null (LLM returned null; deterministic derivation never triggered because guard passed before derivation step)
5	Guard errors present?	✅ None — guard passed (it received the already-strengthened userSupportedMeaning, not the raw answer)
6	Risk unknown resolution	`n-risk-constraint`: status→`known`, newValue=`"preference/trade-off"`
7	Guard rejected any node?	No guard errors; proposal accepted
8	New nodes created	None
9	Selected question proposed	null (risk unknown treated as resolved)

Analysis Against Regression A Contract

Expected preserved meaning

Risk is of greater relative importance than growth; no hard-constraint or non-hard-constraint boundary established.

What the model actually extracted

"Avoiding additional risk is a preference/trade-off rather than a hard constraint."

Violation

The user answered only "Risk matters more to me." — this establishes relative importance only. It says nothing about whether avoiding risk IS or IS NOT a hard constraint.

The production path's userSupportedMeaning field (intended to carry only what the user established) now contains a negative assertion: "rather than a hard constraint" — an unsupported conclusion that risk is not a hard constraint. This directly violates the Regression A "must not happen" requirement:

Must not happen: Inference that risk avoidance is "not a hard constraint" or equivalent negative assertion.

Failure location

The strengthening occurred at the semantic interpretation layer (the model's answer-meaning extraction). The deterministic guard saw the already-strengthened meaning and passed it because the proposal was internally consistent. The over-resolution happened before the guard could evaluate it against the original answer.

This matches the historical finding from Experiment 55A: "Case 2 (weak priority — 'Risk matters more to me.') over-resolved: the model set targetResolved=true and inferred 'not a rigid, non-negotiable constraint' — meaning stronger than the user supplied." The same failure pattern reproduced through the full production path.

Verdict

FAIL - semantic interpretation

For Regression A, the live model and production reasoning path did not preserve only what the answer establishes. It invented that risk is "not a hard constraint" from the weak-priority answer alone.

The PASS requirement is not met:

❌ userSupportedMeaning asserts "rather than a hard constraint" (negative assertion)
❌ The hard-constraint distinction was resolved to "preference/trade-off" rather than left unresolved
❌ The deterministic guard could not prevent this because the over-resolution happened before the guard

Key Findings

The strengthening defect persists through commit 3e78d57. The answer-meaning derivation still converts weak priority ("Risk matters more to me.") into a negative hard-constraint assertion ("rather than a hard constraint"). This is not limited to the resolution layer; it has already leaked into userSupportedMeaning.
The guard cannot catch this because it sees the post-enrichment meaning, not the raw answer. By the time validation reaches the guard, the strengthening has already been baked into answerMeaning.userSupportedMeaning.
Run-to-run variation in inference field. Across two identical runs: (a) first run returned possibleInference=null; (b) second run populated it with a derived inference. Both contained the over-resolution in userSupportedMeaning. The enrichment is unstable across runs for the weak-priority case.
No emergent unknown created. Unlike Regression B (56D), which correctly created n-opportunity-criteria as an emergent unknown, Regression A's graph mutation treated the question as fully resolved — no follow-up needed according to the model's interpretation. This is incorrect: the hard-constraint distinction should remain open.

What remains untested

Whether separating userSupportedMeaning from inference (as attempted in 55D) actually prevents this strengthening when the contract is enforced end-to-end
Whether the fix from 36faf70 (conditional_qualification normalisation) or 3e78d57 (negation/qualification refinement) addresses weak-priority specifically
Whether adding a post-guard verification layer that compares userSupportedMeaning against the original answer text can catch this class of over-resolution

Configuration

Host: http://192.168.1.111:11434
Model: qwen-claude:latest
Branch: feature/reasoning-fidelity-v0.8
Production code changed: NO
Temporary instrumentation: minimal Node script only — removed after capture

5.9 KiB Raw Blame History