diff --git a/docs/current-handoff.md b/docs/current-handoff.md index f18a3b2..0a7157c 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -152,6 +152,18 @@ Experiment 54N tested whether an interpretation disagreement can be judged for m - `npx vitest run tests/graph/apply-proposal.test.js tests/graph/reasoning-context-compatibility.test.js` → PASS (96/96) - **Still unproven live:** the exact 60B.12 client-retention continuation case under live model output. +### 60B.23 audience-question routing specificity + +- 60B.22 identified that `decision_audience` routing was too broad: any occurrence of `customer|user|buyer|stakeholder|recipient|audience` inside a decision unknown could hijack a proposition-specific question before proposition extraction ran. +- `decision_audience` is now gated by explicit audience-identity phrasing only; raw noun occurrence is no longer sufficient. +- `extractMeaning()` now prefers explicit `whether ...` propositions from descriptions when the label is a nominal status/risk/likelihood phrase, so proposition-specific decision unknowns preserve their actual unresolved proposition. +- Decision-pattern `whether ...` propositions now remain on `decision_evidence_clarification` instead of being pre-empted by `decision_threshold_outcome`. +- Legitimate audience-discovery cases remain preserved, and 60B.20-style direct interrogatives remain unchanged. +- Focused verification passed: + - `npx vitest run tests/graph/question-formulator.test.js` → PASS (25/25) + - `npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js` → PASS (45/45) +- **Still unproven live:** the exact 60B.21 product-launch regression in the full end-to-end path. + ### When This Knowledge-Management Phase Is Complete Provisional criteria for review (all confirmed met by Experiment 38 cold-start test): diff --git a/docs/experiment-60b23.md b/docs/experiment-60b23.md new file mode 100644 index 0000000..bfadbe4 --- /dev/null +++ b/docs/experiment-60b23.md @@ -0,0 +1,188 @@ +# Experiment 60B.23 — Narrow decision-audience routing to genuine audience-identity uncertainty + +**Branch:** `feature/question-family-specificity-v0.29` +**Date:** 2026-08-13 +**Status:** COMPLETE +**Type:** BOUNDED IMPLEMENTATION + +## Objective + +Implement the smallest deterministic narrowing so proposition-specific decision unknowns containing audience nouns do not get hijacked by the generic `decision_audience` route. + +The target regression from 60B.21 / 60B.22 was: + +```text +label: +Prospective enterprise customer signing status + +description: +Uncertainty about whether the prospective enterprise customer will sign if we launch this year, so that its resolution is needed to decide which timing option provides superior net value. +``` + +This should not be treated as audience discovery merely because `customer` appears in the text. + +## 60B.22 diagnosis carried forward + +60B.22 established two linked causes: + +1. `selectQuestionFamily()` routed any decision-pattern node containing `customer|user|buyer|stakeholder|recipient|audience` into `decision_audience` via first-match early return. +2. `extractMeaning()` also genericised any such noun occurrence into `the relevant customer, user, or value recipient`, bypassing the underlying unresolved proposition. + +The implementation boundary for 60B.23 was therefore: + +> audience-family routing depends on the semantic role of the audience term, not mere lexical presence. + +## Exact semantic narrowing implemented + +### 1. Audience-family routing now requires explicit audience-identity phrasing + +Added a shared deterministic helper in `lib/graph/question-formulator.js`: + +- `hasAudienceIdentityQuestion(text)` + +This fires only for explicit audience-identity forms such as: + +- `Who is the target customer?` +- `Which buyers are we building this for?` +- `Who would receive the value?` +- `Which audience should this serve?` +- `Who experiences this problem?` + +It does **not** fire merely because a customer/user/buyer/stakeholder/recipient noun appears as the subject of another proposition. + +### 2. Proposition-preserving extraction now prefers explicit `whether...` descriptions for status-like labels + +When the label is a nominal status phrase (`status`, `likelihood`, `probability`, `chance`, `risk`, `uncertainty`) and the description begins with: + +```text +Uncertainty about whether ... +``` + +`extractMeaning()` now returns the explicit `whether ...` proposition directly instead of the nominal label phrase. + +This preserves proposition-specific meaning for cases like: + +- customer will sign +- customer will renew +- users will adopt the change +- stakeholder will approve the plan + +without hardcoding any product-launch wording. + +### 3. Decision threshold routing no longer preempts proposition-specific `whether ...` decision unknowns + +Within decision-pattern family selection, if extracted meaning is already a `whether ...` proposition, the node stays on `decision_evidence_clarification` rather than being diverted into `decision_threshold_outcome`. + +### 4. Legitimate audience questions remain valid through post-build validation + +The formulated audience question still resolves to the `decision_audience` family and survives existing question validation. No prompt, schema, provider, or targeting logic changed. + +## Behaviour preserved + +### Legitimate audience-discovery cases preserved + +True audience-identity questions still route to `decision_audience`. + +### 60B.20 direct interrogative behaviour preserved + +Existing direct proposition-style interrogatives such as: + +```text +Will our largest client leave if we relocate? +``` + +remain unchanged and still produce direct proposition-specific wording. + +## Focused test results + +### Dedicated question-formulator suite + +```bash +npx vitest run tests/graph/question-formulator.test.js +``` + +Result: **PASS (25/25)** + +Covered: + +- 60B.21 signing-status regression +- legitimate audience identity preservation +- customer-as-subject non-audience case +- user-as-subject non-audience case +- buyer/stakeholder lexical mention non-audience case +- existing direct interrogative preservation + +### Smallest broader regression suite containing decision-family tests + +```bash +npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js +``` + +Result: **PASS (45/45)** + +## What is now guaranteed + +1. Raw audience/customer/user/etc. noun occurrence is no longer sufficient to trigger `decision_audience`. +2. `decision_audience` now requires that audience identity itself be unresolved. +3. Proposition-specific decision unknowns with audience nouns can preserve their unresolved proposition into the final question. +4. Existing audience-discovery questions still route to the audience family. +5. Existing interrogative direct-question behaviour remains unchanged. +6. No schema, prompt, apply-proposal, question-target selection, materiality, compatibility, provider, or harness logic changed. + +## What remains unproven until live rerun + +Still unproven until the exact 60B.21 live product-launch regression is rerun: + +- whether the live model-selected node + final deterministic wording path now yields the expected proposition-specific question in the full end-to-end launch-timing case. + +## Production code changed + +YES — `lib/graph/question-formulator.js` + +## Tests changed + +YES — `tests/graph/question-formulator.test.js` + +## Prompt changed + +NO + +## Schema changed + +NO + +## Apply-proposal changed + +NO + +## Question-target selection changed + +NO + +## Materiality changed + +NO + +## Reasoning-context compatibility changed + +NO + +## Provider changed + +NO + +## Harness changed + +NO + +## Ollama calls + +0 + +## Live API calls + +0 + +## Full suite run + +NO