63 lines
4.2 KiB
Markdown
63 lines
4.2 KiB
Markdown
# Experiment 56D — Regression B via Real Production Path
|
|
|
|
**Date**: 2026-08-09
|
|
**Commit**: 3e78d57 (refine answer meaning derivation for negation and qualification)
|
|
**Type**: Observation-only — no code changes
|
|
**Objective**: Verify that deterministic derivation refinement works end-to-end for conditional trade-off scenarios
|
|
|
|
---
|
|
|
|
## Input (Fixed)
|
|
|
|
**Source**: "I want the business to grow, but I don't want to take on more risk."
|
|
**Answer**: "I'd normally avoid more risk, but for the right opportunity I might accept some."
|
|
|
|
## Graph Setup
|
|
|
|
Pre-update graph state matched Regression B fixture:
|
|
- `n-risk-constraint` (unknown/unknown) — active unknown
|
|
- `obs-source-statement` (observation/supported) — source observation
|
|
- 1 edge connecting source to risk unknown
|
|
|
|
---
|
|
|
|
## Results
|
|
|
|
| # | Checkpoint | Result |
|
|
|---|-----------|--------|
|
|
| 1 | `userSupportedMeaning` extracted | ✅ `"Risk avoidance is a strong default preference that can be overridden for specific opportunities deemed suitable, rather than an absolute hard constraint."` |
|
|
| 2 | `possibleInference` derived | ✅ `"Growth strategy should focus on identifying and qualifying high-potential opportunities with clearly defined, bounded risk parameters instead of broad or unconditional expansion."` |
|
|
| 3 | LLM-populated `supportCategory` | null (LLM does not auto-populate; nullable per schema) |
|
|
| 4 | Derived meaning profile category | **conditional_tradeoff** (derived from userSupportedMeaning via deterministic logic) |
|
|
| 5 | Guard errors present? | ✅ None — guard passed successfully |
|
|
| 6 | Risk unknown resolved correctly | `n-risk-constraint`: status→`resolved`, newValue=null, reason=preference vs constraint distinction clarified |
|
|
| 7 | Proposed graph mutation valid | Updated n-risk-constraint as resolved; created new unknown `n-opportunity-criteria` (unknown/unknown) with dependsOn=[n-risk-constraint] |
|
|
| 8 | Newly proposed question | `"What specific criteria define an acceptable 'right opportunity' that justifies taking on additional risk?"` targeting the emergent unknown |
|
|
|
|
## Key Findings
|
|
|
|
1. **Meaning derivation correctly identifies conditional tradeoff**: The `userSupportedMeaning` extraction cleanly separated the default stance (avoid risk) from the qualification (override for right opportunity). This is precisely the Regression B scenario.
|
|
|
|
2. **Deterministic profile categorization works end-to-end**: Despite LLM returning null for `supportCategory`, our inline derivation logic (triggered by `hasDefaultPref && hasException` pattern matching on "normally" + "might/accept") correctly derives `conditional_tradeoff`.
|
|
|
|
3. **Guard validation passes through**: No guard errors — the resolved node and newly added unknown are both compatible with the source scenario.
|
|
|
|
4. **Emergent conditional unknown created successfully**: The system created `n-opportunity-criteria` (kind=unknown, status=unknown) with a description that directly operationalizes the conditional nature: *"Needs explicit criteria to define when additional risk is justified."* This confirms the pipeline correctly recognizes that a conditional tradeoff requires further exploration.
|
|
|
|
5. **selectedQuestion targets emergent unknown**: The proposal correctly includes `selectedQuestion` pointing to `n-opportunity-criteria`, maintaining conversation flow toward resolution of the remaining uncertainty.
|
|
|
|
6. **LLM does not auto-populate `supportCategory`**: Across runs, `answerMeaning.supportCategory` is consistently null. This confirms the derivation logic in `readDiagnostics` (and the inline pipeline) is the mechanism by which the meaning profile gets determined. This is expected design — the LLM produces the raw meaning; the deterministic layer categorizes it.
|
|
|
|
---
|
|
|
|
## Verdict
|
|
|
|
**Regression B PASSES via real production path.** The full updateCase() pipeline correctly:
|
|
- Extracts conditional tradeoff semantics from userAnswer
|
|
- Derives `conditional_tradeoff` category via deterministic profile matching
|
|
- Resolves the active unknown while creating an emergent conditional/threshold unknown
|
|
- Passes all guard constraints
|
|
- Proposes a follow-up question targeting the remaining uncertainty
|
|
|
|
No regression detected. The meaning derivation refinement from commit 3e78d57 works as intended for conditional trade-off scenarios.
|