confidence-engine/docs/archive/experiments/vol-1-chapters/ch16/55B-separate-answer-meaning-from-resolution-judgement.md

Experiment 55B — Separate Answer Meaning from Resolution Judgement (2026-08-08)

Objective

Experiment 55A showed one weak answer was over-resolved and one conditional answer lost nuance. The remaining question: is the distortion introduced when the model restates the answer's meaning, or only when it decides whether the clarification target is resolved?

55B separates these two steps using independent calls per case:

• Mode A (meaning-only): "State what the user's answer establishes. Preserve uncertainty and qualification exactly." No resolution decision.
• Mode B (resolution): The same instruction and contract as Experiment 54V/55A.

Hypothesis

The model may preserve weak and conditional meaning correctly when asked only to restate what the user established, and only become over-confident when asked whether the clarification target is resolved. If so, the problem lies in converting meaning into a targetResolved judgement, not in interpreting the answer itself. If the meaning-only output already strengthens or flattens the answer, then the problem occurs earlier.

Configuration

Host: http://192.168.1.111:11434 (same as all prior experiments) Model: qwen-claude:latest (same as all prior experiments)

Number of Live Inference Calls

Exactly 6 live Ollama calls — one meaning-only call and one resolution call per case, three cases. The calls are independent; Mode A output is never fed into Mode B.

Fixed Source / Target / Question (identical to 55A)

• Source: "I want the business to grow, but I don't want to take on more risk."
• Clarification target: whether avoiding additional risk is a preference/trade-off or a hard constraint.
• Clarification question: Do you view avoiding additional risk as a hard constraint, or as a preference or trade-off?

Mode A instruction

State only what the user's answer establishes in relation to the clarification question. Preserve uncertainty, conditionality, and qualification exactly as supplied. Do not decide whether the clarification target is resolved. Do not infer what the user did not say. Do not recommend action or generate another question.

Return valid JSON only in this shape: { "answerMeaning": "short statement" }

Mode A output contract

{
  "answerMeaning": "short statement"
}

Mode B instruction unchanged

Same as Experiment 54V / 55A resolution instruction. Output contract unchanged (resolvedMeaning, targetResolved, remainingUncertainty).

---

Case 1 — Weak Priority

Answer: "Risk matters more to me."

Mode A result:

• answerMeaning: "The user indicates that avoiding additional risk holds greater importance to them than other factors."
• Classification: meaning_preserved

The meaning preserved relative importance without deciding whether it is a hard constraint or not. No strengthening or loss detected.

Mode B result:

• resolvedMeaning: "Avoiding additional risk is established as a strongly prioritized preference over growth, rather than an explicitly stated boundary."
• targetResolved: false
• remainingUncertainty: "Whether this priority functions as a non-negotiable hard constraint or remains a flexible trade-off in certain situations."
• Classification: resolution_correct

The resolution stayed unresolved with appropriate uncertainty. Interestingly, the resolved meaning is slightly stronger than the Mode A output ("strongly prioritized preference" vs "greater importance"), but targetResolved was correctly false — no over-resolution occurred this time.

---

Case 2 — Conditional Trade-Off

Answer: "I'd normally avoid more risk, but for the right opportunity I might accept some."

Mode A result:

• answerMeaning: "The user indicates that avoiding additional risk is generally a preference that may become a trade-off under specific conditions, rather than an absolute constraint."
• Classification: meaning_preserved

Both the normal preference and the conditional exception ("under specific conditions") were preserved. The meaning captures all of what the user established without flattening.

Mode B result:

• resolvedMeaning: "Avoiding additional risk is a preference or trade-off rather than a hard constraint."
• targetResolved: true
• remainingUncertainty: null
• Classification: resolution_meaning_loss

The target was correctly resolved (targetResolved=true). However, the conditional qualification ("for the right opportunity I might accept some") was flattened into a flat "preference or trade-off" statement. Mode A preserved "generally a preference that may become a trade-off under specific conditions"; Mode B collapsed it to "a preference or trade-off."

This is a direct comparison within the same case: meaning was preserved in Mode A but lost during resolution.

---

Case 3 — Non-Answer

Answer: "I'm not really sure."

Mode A result:

• answerMeaning: "The user is uncertain whether avoiding additional risk is a hard constraint or a preference or trade-off."
• Classification: meaning_preserved

Simple uncertainty was preserved accurately. No position was invented.

Mode B result:

• resolvedMeaning: "The user is unsure whether avoiding additional risk is a hard constraint or a preference/trade-off."
• targetResolved: false
• remainingUncertainty: "Whether avoiding additional risk is a hard constraint or a preference/trade-off remains unclear."
• Classification: resolution_correct

Correctly unresolved. No meaning distortion detected in either mode.

---

Evaluation Summary

Case	Answer	Mode A	Mode A Class	Mode B	Mode B Class
1	"Risk matters more to me."	Preserved relative priority	meaning_preserved	Correctly unresolved	resolution_correct
2	"I'd normally avoid more risk, but for the right opportunity I might accept some."	Preserved conditional qualification	meaning_preserved	Resolved but flattened conditionality	resolution_meaning_loss
3	"I'm not really sure."	Preserved uncertainty	meaning_preserved	Correctly unresolved	resolution_correct

Meaning counts (Mode A):

• meaning_preserved: 3
• meaning_strengthened: 0
• meaning_lost: 0

Resolution counts (Mode B):

• resolution_correct: 2
• resolution_meaning_loss: 1
• resolution_overresolved: 0
• resolution_underresolved: 0

---

Questions Answered

1. Did Case 1 Mode A preserve only relative priority without deciding hard-constraint status? Yes — "greater importance to them than other factors" preserves the relative priority without declaring anything about hard constraint status.

2. Did Case 1 Mode B over-resolve the target again? No — this run returned targetResolved=false with appropriate remaining uncertainty. The resolved meaning was slightly stronger ("strongly prioritized preference") but did not cross into definitive classification. (Note: this differs from the 55A run on the same case, which had over-resolved to targetResolved=true. This may indicate some instability in the resolution step across runs.)

3. Did Case 2 Mode A preserve the conditional "for the right opportunity" qualification? Yes — "generally a preference that may become a trade-off under specific conditions" preserves both the normal stance and the conditional exception.

4. Did Case 2 Mode B preserve or flatten that same conditionality? Flattened. Mode B collapsed "I'd normally avoid more risk, but for the right opportunity I might accept some" into "a preference or trade-off rather than a hard constraint." The conditional qualification ("for the right opportunity") was lost during resolution.

5. Did Case 3 Mode A preserve simple uncertainty? Yes — "The user is uncertain whether avoiding additional risk is a hard constraint or a preference or trade-off" preserves the lack of position without inventing one.

6. Did Case 3 Mode B correctly remain unresolved? Yes — targetResolved=false with accurate remainingUncertainty. No meaning distortion in either mode.

7. In any case, was meaning already distorted before the resolution judgement? No — all three cases preserved their meaning accurately in Mode A (meaning_preserved: 3). The first material information loss appeared only during the resolution step.

8. In any case, did meaning remain accurate in Mode A but become stronger or flatter in Mode B? Yes — Case 2 is the clearest example. Mode A preserved "generally a preference that may become a trade-off under specific conditions"; Mode B flattened it to "a preference or trade-off."

9. Does this isolate the failure to the resolution judgement? Partially yes, for the case of conditional meaning loss. The comparison within Case 2 (same answer, same model, independent calls) shows that meaning can be preserved in isolation and then lost when a resolution decision is introduced. However, only one instance of meaning-preserved-but-resolution-flattened was observed; broader generalisation requires more tested cases.

10. Does this establish how production logic should be redesigned? No — the evidence from three answers is insufficient to justify specific production changes. Further testing with additional answer types and different models would be needed before redesigning any logic.

11. Does this establish graph representation or Behaviour Selection changes? No — this experiment did not integrate with graph, Behaviour Selection, or any other engine component.

---

Limitations

• Only three answers were tested. Different answer patterns may behave differently.
• The weak-priority case in 55B resolved correctly (unlike 55A which over-resolved it), suggesting the resolution step may have some instability across runs with the same configuration.
• Only one model configuration was used (qwen-claude:latest on 192.168.1.111:11434).
• The conditional trade-off answer is a specific pattern; other conditional phrasings may behave differently.
• Six live calls total — insufficient for broader generalisation.

---

Experiment Conclusion

Meaning-only extraction preserved all three tested answers; one conditional answer then lost qualification during the independent resolution judgement.

All three tested answers preserved their meaning correctly in Mode A (meaning_preserved: 3/3). The only information loss appeared in Case 2 when transitioning from meaning-only to resolution: the conditional qualification "for the right opportunity I might accept some" was present and preserved by Mode A, then flattened to a flat "preference or trade-off" statement during resolution.

Additionally, Case 1 produced different resolution outcomes across experiments (55A over-resolved; 55B correctly unresolved), suggesting the resolution step exhibits some run-to-run instability under the same configuration — an observation worth monitoring but not yet actionable without more data.

Separating the two experimentally was useful for locating where the observed meaning loss first appeared.

Focused Test Result

All 6 live inference calls completed successfully. Three answers tested independently through two modes each. Meaning preservation was perfect across Mode A (3/3). Resolution introduced one meaning-loss case (Case 2) and correctly handled the other two. Total: ~104s, average: ~17.3s per call, fastest: 11.5s, slowest: 24.1s.

Historical Comparison Result

Compared to Experiment 55A's conclusion that "the answer-resolution step appears biased toward resolution," Experiment 55B shows this bias is not universal: Case 1 did not over-resolve in the 55B run, and Case 3 was correct in both experiments. The specific loss pattern (conditional meaning preserved in isolation but flattened during resolution) appeared only in Case 2. This narrows the failure from "biased toward resolution" to a more specific pattern: conditional nuance is vulnerable to flattening specifically when the model is forced to make a target-resolution decision.

Documentation Updated

• docs/design-evolution-log.md — added full Experiment 55B entry; applied wording corrections to Experiment 55A
• docs/current-handoff.md — updated with Experiment 55B summary and new Return-to-Work note

Confirmation Host and Model Remained Unchanged

Host: http://192.168.1.111:11434. Model: qwen-claude:latest. Same as all prior experiments.

Confirmation Semantic Instruction and Output Contract Remained Unchanged (for Mode B)

Mode B instruction and output contract identical to Experiment 54V / 55A. No production code changed.

Mode A used a new minimal instruction and output contract specific to this experiment only. It does not replace any existing mechanism.

Confirmation Production Prompts and Schemas Remained Unchanged

No production prompts read or modified. No schemas changed. All inference calls used the experiment-specific semantic instructions defined in this test file.

Confirmation Behaviour Selection Remained Unchanged

Behaviour Selection was not called or referenced. No integration with the selector occurred.

Confirmation Graph and UI Remained Unchanged

No graph files read or modified. No UI code touched. The experiment is test-only.

Confirmation No Meaning-Resolution Separation Entered Active Runtime

This experiment created one new test file only. No meaning-resolution separation logic entered any active runtime path, production module, or behaviour selection output. Production continues using the pre-existing combined instruction and contract.

---