experiment: record 56D (Regression B) and 56E (weak-priority strengthening) results

2026-08-09 10:49:22 +01:00
parent 3e78d57aca
commit 168ef69074
2 changed files with 164 additions and 0 deletions
@@ -0,0 +1,62 @@
+# Experiment 56D — Regression B via Real Production Path
+
+**Date**: 2026-08-09  
+**Commit**: 3e78d57 (refine answer meaning derivation for negation and qualification)  
+**Type**: Observation-only — no code changes  
+**Objective**: Verify that deterministic derivation refinement works end-to-end for conditional trade-off scenarios
+
+---
+
+## Input (Fixed)
+
+**Source**: "I want the business to grow, but I don't want to take on more risk."  
+**Answer**: "I'd normally avoid more risk, but for the right opportunity I might accept some."
+
+## Graph Setup
+
+Pre-update graph state matched Regression B fixture:
+- `n-risk-constraint` (unknown/unknown) — active unknown
+- `obs-source-statement` (observation/supported) — source observation
+- 1 edge connecting source to risk unknown
+
+---
+
+## Results
+
+| # | Checkpoint | Result |
+|---|-----------|--------|
+| 1 | `userSupportedMeaning` extracted | ✅ `"Risk avoidance is a strong default preference that can be overridden for specific opportunities deemed suitable, rather than an absolute hard constraint."` |
+| 2 | `possibleInference` derived | ✅ `"Growth strategy should focus on identifying and qualifying high-potential opportunities with clearly defined, bounded risk parameters instead of broad or unconditional expansion."` |
+| 3 | LLM-populated `supportCategory` | null (LLM does not auto-populate; nullable per schema) |
+| 4 | Derived meaning profile category | **conditional_tradeoff** (derived from userSupportedMeaning via deterministic logic) |
+| 5 | Guard errors present? | ✅ None — guard passed successfully |
+| 6 | Risk unknown resolved correctly | `n-risk-constraint`: status→`resolved`, newValue=null, reason=preference vs constraint distinction clarified |
+| 7 | Proposed graph mutation valid | Updated n-risk-constraint as resolved; created new unknown `n-opportunity-criteria` (unknown/unknown) with dependsOn=[n-risk-constraint] |
+| 8 | Newly proposed question | `"What specific criteria define an acceptable 'right opportunity' that justifies taking on additional risk?"` targeting the emergent unknown |
+
+## Key Findings
+
+1. **Meaning derivation correctly identifies conditional tradeoff**: The `userSupportedMeaning` extraction cleanly separated the default stance (avoid risk) from the qualification (override for right opportunity). This is precisely the Regression B scenario.
+
+2. **Deterministic profile categorization works end-to-end**: Despite LLM returning null for `supportCategory`, our inline derivation logic (triggered by `hasDefaultPref && hasException` pattern matching on "normally" + "might/accept") correctly derives `conditional_tradeoff`.
+
+3. **Guard validation passes through**: No guard errors — the resolved node and newly added unknown are both compatible with the source scenario.
+
+4. **Emergent conditional unknown created successfully**: The system created `n-opportunity-criteria` (kind=unknown, status=unknown) with a description that directly operationalizes the conditional nature: *"Needs explicit criteria to define when additional risk is justified."* This confirms the pipeline correctly recognizes that a conditional tradeoff requires further exploration.
+
+5. **selectedQuestion targets emergent unknown**: The proposal correctly includes `selectedQuestion` pointing to `n-opportunity-criteria`, maintaining conversation flow toward resolution of the remaining uncertainty.
+
+6. **LLM does not auto-populate `supportCategory`**: Across runs, `answerMeaning.supportCategory` is consistently null. This confirms the derivation logic in `readDiagnostics` (and the inline pipeline) is the mechanism by which the meaning profile gets determined. This is expected design — the LLM produces the raw meaning; the deterministic layer categorizes it.
+
+---
+
+## Verdict
+
+**Regression B PASSES via real production path.** The full updateCase() pipeline correctly:
+- Extracts conditional tradeoff semantics from userAnswer
+- Derives `conditional_tradeoff` category via deterministic profile matching
+- Resolves the active unknown while creating an emergent conditional/threshold unknown
+- Passes all guard constraints
+- Proposes a follow-up question targeting the remaining uncertainty
+
+No regression detected. The meaning derivation refinement from commit 3e78d57 works as intended for conditional trade-off scenarios.