confidence-engine/docs/experiment-56d.md

Experiment 56D — Regression B via Real Production Path

Date: 2026-08-09
Commit: 3e78d57 (refine answer meaning derivation for negation and qualification)
Type: Observation-only — no code changes
Objective: Verify that deterministic derivation refinement works end-to-end for conditional trade-off scenarios

---

Input (Fixed)

Source: "I want the business to grow, but I don't want to take on more risk."
Answer: "I'd normally avoid more risk, but for the right opportunity I might accept some."

Graph Setup

Pre-update graph state matched Regression B fixture:

• n-risk-constraint (unknown/unknown) — active unknown
• obs-source-statement (observation/supported) — source observation
• 1 edge connecting source to risk unknown

---

Results

#	Checkpoint	Result
1	userSupportedMeaning extracted	✅ "Risk avoidance is a strong default preference that can be overridden for specific opportunities deemed suitable, rather than an absolute hard constraint."
2	possibleInference derived	✅ "Growth strategy should focus on identifying and qualifying high-potential opportunities with clearly defined, bounded risk parameters instead of broad or unconditional expansion."
3	LLM-populated supportCategory	null (LLM does not auto-populate; nullable per schema)
4	Derived meaning profile category	conditional_tradeoff (derived from userSupportedMeaning via deterministic logic)
5	Guard errors present?	✅ None — guard passed successfully
6	Risk unknown resolved correctly	n-risk-constraint: status→resolved, newValue=null, reason=preference vs constraint distinction clarified
7	Proposed graph mutation valid	Updated n-risk-constraint as resolved; created new unknown n-opportunity-criteria (unknown/unknown) with dependsOn=[n-risk-constraint]
8	Newly proposed question	"What specific criteria define an acceptable 'right opportunity' that justifies taking on additional risk?" targeting the emergent unknown

Key Findings

1. Meaning derivation correctly identifies conditional tradeoff: The userSupportedMeaning extraction cleanly separated the default stance (avoid risk) from the qualification (override for right opportunity). This is precisely the Regression B scenario.

2. Deterministic profile categorization works end-to-end: Despite LLM returning null for supportCategory, our inline derivation logic (triggered by hasDefaultPref && hasException pattern matching on "normally" + "might/accept") correctly derives conditional_tradeoff.

3. Guard validation passes through: No guard errors — the resolved node and newly added unknown are both compatible with the source scenario.

4. Emergent conditional unknown created successfully: The system created n-opportunity-criteria (kind=unknown, status=unknown) with a description that directly operationalizes the conditional nature: "Needs explicit criteria to define when additional risk is justified." This confirms the pipeline correctly recognizes that a conditional tradeoff requires further exploration.

5. selectedQuestion targets emergent unknown: The proposal correctly includes selectedQuestion pointing to n-opportunity-criteria, maintaining conversation flow toward resolution of the remaining uncertainty.

6. LLM does not auto-populate supportCategory: Across runs, answerMeaning.supportCategory is consistently null. This confirms the derivation logic in readDiagnostics (and the inline pipeline) is the mechanism by which the meaning profile gets determined. This is expected design — the LLM produces the raw meaning; the deterministic layer categorizes it.

---

Verdict

Regression B PASSES via real production path. The full updateCase() pipeline correctly:

• Extracts conditional tradeoff semantics from userAnswer
• Derives conditional_tradeoff category via deterministic profile matching
• Resolves the active unknown while creating an emergent conditional/threshold unknown
• Passes all guard constraints
• Proposes a follow-up question targeting the remaining uncertainty

No regression detected. The meaning derivation refinement from commit 3e78d57 works as intended for conditional trade-off scenarios.