docs: record audience question routing fix
This commit is contained in:
@@ -152,6 +152,18 @@ Experiment 54N tested whether an interpretation disagreement can be judged for m
|
|||||||
- `npx vitest run tests/graph/apply-proposal.test.js tests/graph/reasoning-context-compatibility.test.js` → PASS (96/96)
|
- `npx vitest run tests/graph/apply-proposal.test.js tests/graph/reasoning-context-compatibility.test.js` → PASS (96/96)
|
||||||
- **Still unproven live:** the exact 60B.12 client-retention continuation case under live model output.
|
- **Still unproven live:** the exact 60B.12 client-retention continuation case under live model output.
|
||||||
|
|
||||||
|
### 60B.23 audience-question routing specificity
|
||||||
|
|
||||||
|
- 60B.22 identified that `decision_audience` routing was too broad: any occurrence of `customer|user|buyer|stakeholder|recipient|audience` inside a decision unknown could hijack a proposition-specific question before proposition extraction ran.
|
||||||
|
- `decision_audience` is now gated by explicit audience-identity phrasing only; raw noun occurrence is no longer sufficient.
|
||||||
|
- `extractMeaning()` now prefers explicit `whether ...` propositions from descriptions when the label is a nominal status/risk/likelihood phrase, so proposition-specific decision unknowns preserve their actual unresolved proposition.
|
||||||
|
- Decision-pattern `whether ...` propositions now remain on `decision_evidence_clarification` instead of being pre-empted by `decision_threshold_outcome`.
|
||||||
|
- Legitimate audience-discovery cases remain preserved, and 60B.20-style direct interrogatives remain unchanged.
|
||||||
|
- Focused verification passed:
|
||||||
|
- `npx vitest run tests/graph/question-formulator.test.js` → PASS (25/25)
|
||||||
|
- `npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js` → PASS (45/45)
|
||||||
|
- **Still unproven live:** the exact 60B.21 product-launch regression in the full end-to-end path.
|
||||||
|
|
||||||
### When This Knowledge-Management Phase Is Complete
|
### When This Knowledge-Management Phase Is Complete
|
||||||
|
|
||||||
Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):
|
Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):
|
||||||
|
|||||||
@@ -0,0 +1,188 @@
|
|||||||
|
# Experiment 60B.23 — Narrow decision-audience routing to genuine audience-identity uncertainty
|
||||||
|
|
||||||
|
**Branch:** `feature/question-family-specificity-v0.29`
|
||||||
|
**Date:** 2026-08-13
|
||||||
|
**Status:** COMPLETE
|
||||||
|
**Type:** BOUNDED IMPLEMENTATION
|
||||||
|
|
||||||
|
## Objective
|
||||||
|
|
||||||
|
Implement the smallest deterministic narrowing so proposition-specific decision unknowns containing audience nouns do not get hijacked by the generic `decision_audience` route.
|
||||||
|
|
||||||
|
The target regression from 60B.21 / 60B.22 was:
|
||||||
|
|
||||||
|
```text
|
||||||
|
label:
|
||||||
|
Prospective enterprise customer signing status
|
||||||
|
|
||||||
|
description:
|
||||||
|
Uncertainty about whether the prospective enterprise customer will sign if we launch this year, so that its resolution is needed to decide which timing option provides superior net value.
|
||||||
|
```
|
||||||
|
|
||||||
|
This should not be treated as audience discovery merely because `customer` appears in the text.
|
||||||
|
|
||||||
|
## 60B.22 diagnosis carried forward
|
||||||
|
|
||||||
|
60B.22 established two linked causes:
|
||||||
|
|
||||||
|
1. `selectQuestionFamily()` routed any decision-pattern node containing `customer|user|buyer|stakeholder|recipient|audience` into `decision_audience` via first-match early return.
|
||||||
|
2. `extractMeaning()` also genericised any such noun occurrence into `the relevant customer, user, or value recipient`, bypassing the underlying unresolved proposition.
|
||||||
|
|
||||||
|
The implementation boundary for 60B.23 was therefore:
|
||||||
|
|
||||||
|
> audience-family routing depends on the semantic role of the audience term, not mere lexical presence.
|
||||||
|
|
||||||
|
## Exact semantic narrowing implemented
|
||||||
|
|
||||||
|
### 1. Audience-family routing now requires explicit audience-identity phrasing
|
||||||
|
|
||||||
|
Added a shared deterministic helper in `lib/graph/question-formulator.js`:
|
||||||
|
|
||||||
|
- `hasAudienceIdentityQuestion(text)`
|
||||||
|
|
||||||
|
This fires only for explicit audience-identity forms such as:
|
||||||
|
|
||||||
|
- `Who is the target customer?`
|
||||||
|
- `Which buyers are we building this for?`
|
||||||
|
- `Who would receive the value?`
|
||||||
|
- `Which audience should this serve?`
|
||||||
|
- `Who experiences this problem?`
|
||||||
|
|
||||||
|
It does **not** fire merely because a customer/user/buyer/stakeholder/recipient noun appears as the subject of another proposition.
|
||||||
|
|
||||||
|
### 2. Proposition-preserving extraction now prefers explicit `whether...` descriptions for status-like labels
|
||||||
|
|
||||||
|
When the label is a nominal status phrase (`status`, `likelihood`, `probability`, `chance`, `risk`, `uncertainty`) and the description begins with:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Uncertainty about whether ...
|
||||||
|
```
|
||||||
|
|
||||||
|
`extractMeaning()` now returns the explicit `whether ...` proposition directly instead of the nominal label phrase.
|
||||||
|
|
||||||
|
This preserves proposition-specific meaning for cases like:
|
||||||
|
|
||||||
|
- customer will sign
|
||||||
|
- customer will renew
|
||||||
|
- users will adopt the change
|
||||||
|
- stakeholder will approve the plan
|
||||||
|
|
||||||
|
without hardcoding any product-launch wording.
|
||||||
|
|
||||||
|
### 3. Decision threshold routing no longer preempts proposition-specific `whether ...` decision unknowns
|
||||||
|
|
||||||
|
Within decision-pattern family selection, if extracted meaning is already a `whether ...` proposition, the node stays on `decision_evidence_clarification` rather than being diverted into `decision_threshold_outcome`.
|
||||||
|
|
||||||
|
### 4. Legitimate audience questions remain valid through post-build validation
|
||||||
|
|
||||||
|
The formulated audience question still resolves to the `decision_audience` family and survives existing question validation. No prompt, schema, provider, or targeting logic changed.
|
||||||
|
|
||||||
|
## Behaviour preserved
|
||||||
|
|
||||||
|
### Legitimate audience-discovery cases preserved
|
||||||
|
|
||||||
|
True audience-identity questions still route to `decision_audience`.
|
||||||
|
|
||||||
|
### 60B.20 direct interrogative behaviour preserved
|
||||||
|
|
||||||
|
Existing direct proposition-style interrogatives such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Will our largest client leave if we relocate?
|
||||||
|
```
|
||||||
|
|
||||||
|
remain unchanged and still produce direct proposition-specific wording.
|
||||||
|
|
||||||
|
## Focused test results
|
||||||
|
|
||||||
|
### Dedicated question-formulator suite
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx vitest run tests/graph/question-formulator.test.js
|
||||||
|
```
|
||||||
|
|
||||||
|
Result: **PASS (25/25)**
|
||||||
|
|
||||||
|
Covered:
|
||||||
|
|
||||||
|
- 60B.21 signing-status regression
|
||||||
|
- legitimate audience identity preservation
|
||||||
|
- customer-as-subject non-audience case
|
||||||
|
- user-as-subject non-audience case
|
||||||
|
- buyer/stakeholder lexical mention non-audience case
|
||||||
|
- existing direct interrogative preservation
|
||||||
|
|
||||||
|
### Smallest broader regression suite containing decision-family tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js
|
||||||
|
```
|
||||||
|
|
||||||
|
Result: **PASS (45/45)**
|
||||||
|
|
||||||
|
## What is now guaranteed
|
||||||
|
|
||||||
|
1. Raw audience/customer/user/etc. noun occurrence is no longer sufficient to trigger `decision_audience`.
|
||||||
|
2. `decision_audience` now requires that audience identity itself be unresolved.
|
||||||
|
3. Proposition-specific decision unknowns with audience nouns can preserve their unresolved proposition into the final question.
|
||||||
|
4. Existing audience-discovery questions still route to the audience family.
|
||||||
|
5. Existing interrogative direct-question behaviour remains unchanged.
|
||||||
|
6. No schema, prompt, apply-proposal, question-target selection, materiality, compatibility, provider, or harness logic changed.
|
||||||
|
|
||||||
|
## What remains unproven until live rerun
|
||||||
|
|
||||||
|
Still unproven until the exact 60B.21 live product-launch regression is rerun:
|
||||||
|
|
||||||
|
- whether the live model-selected node + final deterministic wording path now yields the expected proposition-specific question in the full end-to-end launch-timing case.
|
||||||
|
|
||||||
|
## Production code changed
|
||||||
|
|
||||||
|
YES — `lib/graph/question-formulator.js`
|
||||||
|
|
||||||
|
## Tests changed
|
||||||
|
|
||||||
|
YES — `tests/graph/question-formulator.test.js`
|
||||||
|
|
||||||
|
## Prompt changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Schema changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Apply-proposal changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Question-target selection changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Materiality changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Reasoning-context compatibility changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Provider changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Harness changed
|
||||||
|
|
||||||
|
NO
|
||||||
|
|
||||||
|
## Ollama calls
|
||||||
|
|
||||||
|
0
|
||||||
|
|
||||||
|
## Live API calls
|
||||||
|
|
||||||
|
0
|
||||||
|
|
||||||
|
## Full suite run
|
||||||
|
|
||||||
|
NO
|
||||||
Reference in New Issue
Block a user