Feature/product platform foundation v0.62 #1
@@ -2869,3 +2869,7 @@ Experiment 60B.22 was a read-only diagnosis of why the correct material target (
|
||||
---
|
||||
|
||||
Experiment 60B.24 ran the exact product-launch live regression (same fixture as 60B.21) on the post-60B.23 branch to verify the proposition-specificity fix end-to-end. **Classification: A — LIVE QUESTION-SPECIFICITY FIX CONFIRMED.** The model produced `uncertain_enterprise_customer_signing` (kind=unknown, status=unknown) with depends_on edge from n_product_launch_decision and affects edge to opt_launch_this_year, preserving £700k significance in the node description. The final selected question ("whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue...") is proposition-specific — not generic audience wording. Core reasoning chain preserved, decision remains unresolved, no unrelated uncertainty invented. One live Ollama call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed.
|
||||
|
||||
---
|
||||
|
||||
Experiment 60B.25 was a read-only diagnosis of why the proposition-specific question produced "whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k..." (proposition-plus-rationale) instead of a clean conversational question. **Classification: C — BOTH A + B.** Two independent defects in `extractMeaning` (lib/graph/question-formulator.js): (A) **Rationale extraction too broad** at line 108 — when label contains "status" and description starts with "Whether", the function returns `sentenceCase(strippedDescription)` which includes the full description text after the semicolon, capturing explanatory rationale (~£700k context clause). No internal delimiter logic exists. (B) **No whether→direct-question conversion** — `isInterrogativeMeaning` at line 189 treats "whether" clauses as already interrogative and passes them through unchanged via `wrapInterrogativeForTemplate`, so the output is proposition+rationale with "?" appended, not a converted question. The first meaningful divergence from 60B.20 (which produced clean "will our largest client leave if we relocate?") is at extractMeaning line 108: 60B.20's interrogative label bypasses the status+whether branch entirely, while 60B.24's nominal "signing status" label triggers it, pulling in full description including rationale. **Smallest corrective boundary:** A+C — strip rationale on extraction (split on first semicolon), then let existing evidence framing produce "What evidence would clarify whether X?" or keep direct interrogative form via decision path line 1272-1273. No provider rewrite, no target selection change, no schema change needed. Implementation readiness: A. Full trace documented in docs/experiment-60b25.md.
|
||||
|
||||
@@ -0,0 +1,232 @@
|
||||
# Experiment 60B.25 — Diagnosis: Proposition-Plus-Rationale Instead of Clean Question
|
||||
|
||||
**Branch:** `feature/question-family-specificity-v0.29`
|
||||
**Date:** 2026-08-13
|
||||
**Status:** COMPLETE
|
||||
**Type:** READ-ONLY DIAGNOSIS (no code changes)
|
||||
|
||||
## Objective
|
||||
|
||||
Answer one measurable question:
|
||||
|
||||
> Why does deterministic formulation preserve the whole proposition-plus-rationale string instead of converting the unresolved proposition into a concise interrogative question?
|
||||
|
||||
Input node (from 60B.24):
|
||||
```
|
||||
label: "Prospective enterprise customer signing status"
|
||||
description: "Whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue, so that resolving their intent is needed to assess the financial impact of launching now."
|
||||
```
|
||||
|
||||
Produced question:
|
||||
```
|
||||
whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue, so that resolving their intent is needed to assess the financial impact of launching now?
|
||||
```
|
||||
|
||||
## Checkpoint 1 — Extraction Behaviour (full trace)
|
||||
|
||||
### `extractMeaning(node)` trace for the fixed node:
|
||||
|
||||
**Line 86:** `raw = "Prospective enterprise customer signing status Whether one prospective..."`
|
||||
|
||||
**Line 87-89:** `meaning = stripTrailingPunctuation("Prospective enterprise customer signing status")`
|
||||
→ `"Prospective enterprise customer signing status"` (no trailing punctuation to strip)
|
||||
|
||||
**Line 92:** `hasAudienceIdentityQuestion(lowered)` → **NO**
|
||||
None of the patterns match: "who is the customer", "target customer", "identifying the customer", etc. The word "enterprise customer" does not match any pattern — it's a noun modifier, not an audience-identity construct.
|
||||
|
||||
**Line 96-97:** `strippedDescription = stripTrailingPunctuation(description)`
|
||||
→ Full description text with no trailing punctuation change (it ends with period which gets stripped).
|
||||
|
||||
**Lines 102-108 — THE KEY BRANCH:**
|
||||
```js
|
||||
if (
|
||||
/\b(status|likelihood|probability|chance|risk|uncertainty)\b/i.test("Prospective enterprise customer signing status") && // MATCHES "status" ✓
|
||||
/^whether\s+/i.test(strippedDescription) // MATCHES "Whether..." ✓
|
||||
) {
|
||||
return sentenceCase(strippedDescription); // ← ALL text returned
|
||||
}
|
||||
```
|
||||
|
||||
Both conditions match. **This branch is taken.**
|
||||
|
||||
`sentenceCase("Whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k...")` →
|
||||
**lowercases first char, preserves everything else including rationale after semicolon**
|
||||
|
||||
### Checkpoint 1 Answers:
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| label considered? | **YES** — used to trigger the status regex |
|
||||
| description considered? | **YES** — full text passed to sentenceCase on line 108 |
|
||||
| returned meaning | `"whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue, so that resolving their intent is needed to assess the financial impact of launching now."` (first char lowered) |
|
||||
| rationale stripped? | **NO** — line 108 returns full `strippedDescription` |
|
||||
| semicolon boundary recognized? | **NO** — no split logic exists for description extraction |
|
||||
| "so that" rationale recognized? | **NO** — no rationale marker detection in extractMeaning |
|
||||
|
||||
## Checkpoint 2 — Interrogative Conversion (full trace)
|
||||
|
||||
### `isInterrogativeMeaning(meaning)` trace:
|
||||
|
||||
Input: `"whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k..."`
|
||||
|
||||
- Line 170 (`wh-questions`): No match — starts with "whether", not who/what/where/when/how
|
||||
- Lines 179-186 (aux inversion): No match — first word is "whether", not a modal auxiliary
|
||||
- **Line 189: `/^whether\b/i.test(trimmed)` → YES ✓**
|
||||
|
||||
Returns `true`. The engine recognises the meaning as already question-shaped.
|
||||
|
||||
### `wrapInterrogativeForTemplate(meaning)` trace:
|
||||
|
||||
Input: lowercased meaning string
|
||||
- Line 198 (wh-questions): No match — starts with "whether"
|
||||
- **Line 202: `isInterrogativeMeaning` → true**
|
||||
- Returns: `stripTrailingPunctuation(meaning).trim()` = full proposition + rationale, no trailing punctuation
|
||||
|
||||
### Decision evidence path trace (line 1271-1273):
|
||||
|
||||
```js
|
||||
if (isInterrogativeMeaning(meaning)) { // YES ✓
|
||||
return `${wrapInterrogativeForTemplate(meaning)}?`;
|
||||
}
|
||||
```
|
||||
|
||||
Result: `"whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue, so that resolving their intent is needed to assess the financial impact of launching now?"`
|
||||
|
||||
### Checkpoint 2 Answers:
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| Does engine recognise "whether..." as unresolved proposition? | **YES** — `isInterrogativeMeaning` returns true at line 189 |
|
||||
| Does it convert "whether X..." into "Will/Does/Is X...?" | **NO** — no conversion logic exists; "whether" is treated as already interrogative |
|
||||
| Does it merely append "?" | **YES** — direct from the full extracted string including rationale |
|
||||
| Evidence framing applied? | **CONDITIONAL** — `buildEvidenceFallbackQuestion` would add "What evidence would confirm or rule out...", but in the decision reasoning path (line 1272-1273), interrogative means bypass the evidence template and go straight to append "?" |
|
||||
|
||||
## Checkpoint 3 — Why 60B.20 Looked Better
|
||||
|
||||
### 60B.20 source shape:
|
||||
```
|
||||
label: "Will our largest client leave if we relocate?"
|
||||
description: "Uncertainty regarding whether our largest client would depart following a relocation to Manchester; matters because their departure would cost approximately £5M per year..."
|
||||
```
|
||||
|
||||
**Label is already interrogative:** "Will our largest client leave if we relocate?"
|
||||
|
||||
### 60B.24 source shape:
|
||||
```
|
||||
label: "Prospective enterprise customer signing status"
|
||||
description: "Whether one prospective enterprise customer will sign if we launch this year; they account for ~£700k of the £1.2M expected annual revenue, so that resolving their intent is needed to assess the financial impact of launching now."
|
||||
```
|
||||
|
||||
**Label is nominal (noun phrase); proposition in description starting with "Whether"**
|
||||
|
||||
### First meaningful divergence: `extractMeaning` line 108
|
||||
|
||||
In 60B.24, the condition at lines 102-107 fires because the label contains "status" AND the description starts with "Whether". This causes `extractMeaning` to return the **full** `strippedDescription` (proposition + rationale after semicolon).
|
||||
|
||||
In 60B.20, the label is already interrogive ("Will our largest client..."). The condition at lines 102-107 does NOT fire because:
|
||||
- Label contains no status/probability words (no "status" in "Will our largest client leave if we relocate?")
|
||||
- Even though it has no "status", the label itself IS interrogative
|
||||
|
||||
The extracted meaning for 60B.20 is the **label** ("Will our largest client leave if we relocate?"), not the description. This is already a clean question, so the template simply appends "?" to produce a valid output.
|
||||
|
||||
### Quality difference cause: `description extraction` + `rationale contamination`
|
||||
|
||||
The root divergence is in `extractMeaning`:
|
||||
1. **60B.20** — Label is interrogative → meaning = label (clean) → output = label + "?" ✓
|
||||
2. **60B.24** — Label triggers status condition → meaning = full description including rationale after semicolon → output = proposition+rationale + "?" ✗
|
||||
|
||||
The quality difference comes from **description extraction capturing rationale** and the **absence of "whether→direct-question conversion"**.
|
||||
|
||||
## Cause Classification: C (Both A + B)
|
||||
|
||||
### A — RATIONALE EXTRACTION TOO BROAD
|
||||
`extractMeaning` line 108 returns `sentenceCase(strippedDescription)` which includes everything after the semicolon. There is no internal delimiter logic for separating proposition from explanatory rationale.
|
||||
|
||||
### B — NO WHETHER→QUESTION CONVERSION
|
||||
The engine recognises "whether X" as already interrogative (line 189) and passes it through unchanged. No conversion to "Will/Does/Is X?" exists in the codebase. The `isInterrogativeMeaning` function treats "whether" clauses as complete interrogatives rather than treating them as unresolved propositions that need conversion.
|
||||
|
||||
## Current Semantic Contract
|
||||
|
||||
**For a selected unknown whose explicit meaning is "whether X", what should deterministic formulation ideally represent?**
|
||||
|
||||
### C — Direct Interrogative: "Will/Does/Is X?"
|
||||
|
||||
The current code's intent (lines 170-189, 202-203) is:
|
||||
- If the extracted meaning is already interrogative (wh- question, aux inversion, or whether-clause), pass it through unchanged.
|
||||
- The rationale for line 189 treating "whether" as complete interrogative was to prevent double-wrapping ("What would clarify are...").
|
||||
|
||||
However, this conflates two distinct semantic states:
|
||||
1. **Direct interrogative** (e.g., "Will X happen?") — ready as a question
|
||||
2. **Indirect interrogative / unresolved proposition** (e.g., "whether X will happen") — needs conversion
|
||||
|
||||
The current contract treats both identically, which is why 60B.24's output preserves the indirect form with rationale contamination.
|
||||
|
||||
## Candidate Evaluations
|
||||
|
||||
### Candidate A — Strip Rationale Only
|
||||
|
||||
Extract only: `"Whether one prospective enterprise customer will sign if we launch this year"` (before semicolon). Then preserve existing formulation behaviour.
|
||||
|
||||
| Assessment | Value |
|
||||
|---|---|
|
||||
| Improves concision | **HIGH** — removes the entire explanatory clause |
|
||||
| Produces conversational question | **NO** — "Whether one prospective enterprise customer will sign if we launch this year?" is still an indirect question (embedded/yes-no proposition form), not natural conversational English. The user would expect "Will one...?" |
|
||||
| Risk of losing context | **LOW** — rationale is explanatory, not material. Materiality lives in the graph structure (£700k on edge/unknown node) |
|
||||
|
||||
### Candidate B — Deterministic Whether→Interrogative Conversion
|
||||
|
||||
Convert simple explicit propositions:
|
||||
- Input: `"whether the customer will sign"`
|
||||
- Output: `"Will the customer sign?"`
|
||||
|
||||
| Assessment | Value |
|
||||
|---|---|
|
||||
| Semantic robustness | **MEDIUM** — works for straightforward propositions but fails on complex conditionals ("whether we should launch if X AND Y") |
|
||||
| Grammar complexity | **HIGH** — requires subject-auxiliary inversion, pronoun mapping, tense preservation, conditional clause handling |
|
||||
| Meaning-change risk | **LOW** — "whether X" is semantically equivalent to "Will/Does/Is X?" in decision context |
|
||||
|
||||
### Candidate C — Clean Proposition + Evidence Framing
|
||||
|
||||
Strip rationale → formulate: `"What evidence would clarify whether the customer will sign if we launch this year?"`
|
||||
|
||||
| Assessment | Value |
|
||||
|---|---|
|
||||
| Semantic robustness | **HIGH** — "whether" is preserved as-is (no conversion needed); framing adapts to any proposition form |
|
||||
| Conversational quality | **MEDIUM** — more formal than direct questions but still natural and decision-relevant. Standard in decision analysis literature |
|
||||
| Consistency with existing `decision_evidence` family | **HIGH** — aligns with the evidence-gathering intent of the family (see line 1275 template) |
|
||||
|
||||
### Candidate D — Minimum Combination
|
||||
|
||||
**A + C**: Strip rationale first (Candidate A's extraction fix), then let existing evidence framing apply (producing Candidate C output). This avoids Candidate B's grammar complexity entirely.
|
||||
|
||||
## Decision Criteria Assessment
|
||||
|
||||
| Criterion | A | B | C | D (A+C) |
|
||||
|---|---|---|---|---|
|
||||
| 1. Remove explanatory rationale from question | PARTIAL | NO | YES | **YES** ✓ |
|
||||
| 2. Preserve unresolved proposition | YES | YES | YES | **YES** ✓ |
|
||||
| 3. Remains deterministic | YES | PARTIAL | YES | **YES** ✓ |
|
||||
| 4. No provider rewriting | YES | YES | YES | **YES** ✓ |
|
||||
| 5. No target selection change | YES | YES | YES | **YES** ✓ |
|
||||
| 6. Preserve direct interrogative cases like 60B.20 | NO (breaks label path) | PARTIAL | YES | **YES** ✓ |
|
||||
| 7. Avoid domain-specific grammar rules | YES | NO | YES | **YES** ✓ |
|
||||
|
||||
## Final Choice: D — MINIMUM COMBINATION
|
||||
|
||||
### Smallest implementation boundary
|
||||
|
||||
**One change to `extractMeaning`:**
|
||||
On line 108, instead of returning the full `strippedDescription`, split on semicolons and return only the first segment (the proposition), before applying `sentenceCase`.
|
||||
|
||||
```
|
||||
Before: return sentenceCase(strippedDescription);
|
||||
After: return sentenceCase(strippedDescription.split(/;|[,]\s*(so\s+that|which\s+means)/i)[0].trim());
|
||||
```
|
||||
|
||||
**No other changes required.** The existing decision-evidence formulation path (line 1272-1273) will then receive a clean proposition, and the `isInterrogativeMeaning` detection on line 189 will still correctly handle "whether" clauses as interrogative.
|
||||
|
||||
**Alternative boundary:** If you want cleaner output than "Whether X?" for all cases, also modify the decision path (line 1272-1273) to use `buildEvidenceFallbackQuestion` instead of direct append-for-interrogative-meaning when the meaning starts with "whether". This produces "What evidence would clarify whether X?" which is both natural and consistent with the evidence family.
|
||||
|
||||
## Implementation Readiness: A — READY FOR BOUNDED IMPLEMENTATION
|
||||
|
||||
One line change to `extractMeaning` at line 108 plus (optionally) one additional refinement in the decision path formatting logic.
|
||||
Reference in New Issue
Block a user