Experiment 56L — User-owned ambiguity boundary probe

Objective

Can the configured model recognise that an unresolved distinction belongs to the user's own meaning and therefore requires clarification rather than external evidence gathering?

Config

Ollama base URL: http://192.168.1.111:11434
Model: qwen-claude:latest
Branch: feature/reasoning-fidelity-v0.8
Start HEAD: [current HEAD at time of experiment]

Fixed Case (Regression F)

Source: "I want the business to grow, but I don't want to take on more risk."
Answer context: (ambiguous statement about both growth and risk — user has not specified preference/trade-off versus hard constraint)
Expected preserved meaning: User has not specified whether avoiding additional risk is a hard constraint or a strong preference/trade-off.
Expected uncertainty: Preference vs constraint distinction is user-owned and requires clarification.
Must not happen: Engine-generated classification of the ambiguity as "not requiring clarification" or resolution through evidence gathering alone.

Pre-written Human Reference (written before model output)

{
  "uncertaintyType": "user_clarification_needed",
  "reason": "The unresolved distinction (whether avoiding additional risk is a hard constraint or a strong preference/trade-off) belongs to the user's own meaning. External evidence cannot establish what the user means, prefers, intends, defines, or constrains on their own behalf.",
  "evidenceNeeded": null,
  "userClarificationNeeded": "Whether the user treats avoiding additional risk as a hard constraint (non-negotiable boundary) versus a strong preference/trade-off (weighted but potentially overrideable)."
}

Live-call Result

Call duration: 14,032 ms
Call count: 1

Raw Structured Response

{
  "uncertaintyType": "user_clarification_needed",
  "reason": "The ambiguity concerns whether avoiding additional risk is a strict hard constraint or a flexible preference, which reflects the user's private intent and priorities that external data cannot determine.",
  "evidenceNeeded": null,
  "userClarificationNeeded": "Whether avoiding additional risk is a non-negotiable hard constraint or a trade-off preference that can be balanced against business growth goals."
}

Human Semantic Classification: PASS

Rationale: The model clearly recognises the ambiguity belongs to the user's own meaning and requires user clarification rather than external evidence. It preserved the distinction cleanly: uncertaintyType is user_clarification_needed, evidenceNeeded is null (no spurious evidence target introduced), and userClarificationNeeded specifically describes the preference/trade-off versus hard-constraint distinction that only the user can establish.

Comparison with Pre-written Human Reference

Expected: user_clarification_needed
Actual: user_clarification_needed
Matches: YES

The model's answer matches the human reference at the category level and substantively agrees on both what is unclear and why (the distinction is private to user meaning, not externally determinable).

What This Experiment Established

The configured model (qwen-claude:latest) can distinguish user-owned ambiguity from evidence-resolvable uncertainty for Regression F's canonical case.
It correctly identified that the preference-vs-constraint distinction is user-owned and requires clarification, not evidence gathering.
It did not introduce unnecessary evidence targets where none apply.

What This Experiment Does NOT Prove

Consistency across repeated runs with this or other models.
Fidelity for other regression cases (A–E, G+).
Behavior in production reasoning paths or graph-update contexts.
Downstream integration with Behaviour Selection, UI, or the SituationGraph.
Whether clarification targeting is precise enough to generate a useful user-facing question (that was explicitly excluded from this experiment's scope per output contract).

Files

Test: tests/reconstruction/semantic-regression-f-user-owned-ambiguity.test.js
Document: docs/experiment-56l.md

4.1 KiB Raw Blame History Unescape Escape