docs: record audience question routing fix
This commit is contained in:
@@ -152,6 +152,18 @@ Experiment 54N tested whether an interpretation disagreement can be judged for m
|
||||
- `npx vitest run tests/graph/apply-proposal.test.js tests/graph/reasoning-context-compatibility.test.js` → PASS (96/96)
|
||||
- **Still unproven live:** the exact 60B.12 client-retention continuation case under live model output.
|
||||
|
||||
### 60B.23 audience-question routing specificity
|
||||
|
||||
- 60B.22 identified that `decision_audience` routing was too broad: any occurrence of `customer|user|buyer|stakeholder|recipient|audience` inside a decision unknown could hijack a proposition-specific question before proposition extraction ran.
|
||||
- `decision_audience` is now gated by explicit audience-identity phrasing only; raw noun occurrence is no longer sufficient.
|
||||
- `extractMeaning()` now prefers explicit `whether ...` propositions from descriptions when the label is a nominal status/risk/likelihood phrase, so proposition-specific decision unknowns preserve their actual unresolved proposition.
|
||||
- Decision-pattern `whether ...` propositions now remain on `decision_evidence_clarification` instead of being pre-empted by `decision_threshold_outcome`.
|
||||
- Legitimate audience-discovery cases remain preserved, and 60B.20-style direct interrogatives remain unchanged.
|
||||
- Focused verification passed:
|
||||
- `npx vitest run tests/graph/question-formulator.test.js` → PASS (25/25)
|
||||
- `npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js` → PASS (45/45)
|
||||
- **Still unproven live:** the exact 60B.21 product-launch regression in the full end-to-end path.
|
||||
|
||||
### When This Knowledge-Management Phase Is Complete
|
||||
|
||||
Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
# Experiment 60B.23 — Narrow decision-audience routing to genuine audience-identity uncertainty
|
||||
|
||||
**Branch:** `feature/question-family-specificity-v0.29`
|
||||
**Date:** 2026-08-13
|
||||
**Status:** COMPLETE
|
||||
**Type:** BOUNDED IMPLEMENTATION
|
||||
|
||||
## Objective
|
||||
|
||||
Implement the smallest deterministic narrowing so proposition-specific decision unknowns containing audience nouns do not get hijacked by the generic `decision_audience` route.
|
||||
|
||||
The target regression from 60B.21 / 60B.22 was:
|
||||
|
||||
```text
|
||||
label:
|
||||
Prospective enterprise customer signing status
|
||||
|
||||
description:
|
||||
Uncertainty about whether the prospective enterprise customer will sign if we launch this year, so that its resolution is needed to decide which timing option provides superior net value.
|
||||
```
|
||||
|
||||
This should not be treated as audience discovery merely because `customer` appears in the text.
|
||||
|
||||
## 60B.22 diagnosis carried forward
|
||||
|
||||
60B.22 established two linked causes:
|
||||
|
||||
1. `selectQuestionFamily()` routed any decision-pattern node containing `customer|user|buyer|stakeholder|recipient|audience` into `decision_audience` via first-match early return.
|
||||
2. `extractMeaning()` also genericised any such noun occurrence into `the relevant customer, user, or value recipient`, bypassing the underlying unresolved proposition.
|
||||
|
||||
The implementation boundary for 60B.23 was therefore:
|
||||
|
||||
> audience-family routing depends on the semantic role of the audience term, not mere lexical presence.
|
||||
|
||||
## Exact semantic narrowing implemented
|
||||
|
||||
### 1. Audience-family routing now requires explicit audience-identity phrasing
|
||||
|
||||
Added a shared deterministic helper in `lib/graph/question-formulator.js`:
|
||||
|
||||
- `hasAudienceIdentityQuestion(text)`
|
||||
|
||||
This fires only for explicit audience-identity forms such as:
|
||||
|
||||
- `Who is the target customer?`
|
||||
- `Which buyers are we building this for?`
|
||||
- `Who would receive the value?`
|
||||
- `Which audience should this serve?`
|
||||
- `Who experiences this problem?`
|
||||
|
||||
It does **not** fire merely because a customer/user/buyer/stakeholder/recipient noun appears as the subject of another proposition.
|
||||
|
||||
### 2. Proposition-preserving extraction now prefers explicit `whether...` descriptions for status-like labels
|
||||
|
||||
When the label is a nominal status phrase (`status`, `likelihood`, `probability`, `chance`, `risk`, `uncertainty`) and the description begins with:
|
||||
|
||||
```text
|
||||
Uncertainty about whether ...
|
||||
```
|
||||
|
||||
`extractMeaning()` now returns the explicit `whether ...` proposition directly instead of the nominal label phrase.
|
||||
|
||||
This preserves proposition-specific meaning for cases like:
|
||||
|
||||
- customer will sign
|
||||
- customer will renew
|
||||
- users will adopt the change
|
||||
- stakeholder will approve the plan
|
||||
|
||||
without hardcoding any product-launch wording.
|
||||
|
||||
### 3. Decision threshold routing no longer preempts proposition-specific `whether ...` decision unknowns
|
||||
|
||||
Within decision-pattern family selection, if extracted meaning is already a `whether ...` proposition, the node stays on `decision_evidence_clarification` rather than being diverted into `decision_threshold_outcome`.
|
||||
|
||||
### 4. Legitimate audience questions remain valid through post-build validation
|
||||
|
||||
The formulated audience question still resolves to the `decision_audience` family and survives existing question validation. No prompt, schema, provider, or targeting logic changed.
|
||||
|
||||
## Behaviour preserved
|
||||
|
||||
### Legitimate audience-discovery cases preserved
|
||||
|
||||
True audience-identity questions still route to `decision_audience`.
|
||||
|
||||
### 60B.20 direct interrogative behaviour preserved
|
||||
|
||||
Existing direct proposition-style interrogatives such as:
|
||||
|
||||
```text
|
||||
Will our largest client leave if we relocate?
|
||||
```
|
||||
|
||||
remain unchanged and still produce direct proposition-specific wording.
|
||||
|
||||
## Focused test results
|
||||
|
||||
### Dedicated question-formulator suite
|
||||
|
||||
```bash
|
||||
npx vitest run tests/graph/question-formulator.test.js
|
||||
```
|
||||
|
||||
Result: **PASS (25/25)**
|
||||
|
||||
Covered:
|
||||
|
||||
- 60B.21 signing-status regression
|
||||
- legitimate audience identity preservation
|
||||
- customer-as-subject non-audience case
|
||||
- user-as-subject non-audience case
|
||||
- buyer/stakeholder lexical mention non-audience case
|
||||
- existing direct interrogative preservation
|
||||
|
||||
### Smallest broader regression suite containing decision-family tests
|
||||
|
||||
```bash
|
||||
npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js
|
||||
```
|
||||
|
||||
Result: **PASS (45/45)**
|
||||
|
||||
## What is now guaranteed
|
||||
|
||||
1. Raw audience/customer/user/etc. noun occurrence is no longer sufficient to trigger `decision_audience`.
|
||||
2. `decision_audience` now requires that audience identity itself be unresolved.
|
||||
3. Proposition-specific decision unknowns with audience nouns can preserve their unresolved proposition into the final question.
|
||||
4. Existing audience-discovery questions still route to the audience family.
|
||||
5. Existing interrogative direct-question behaviour remains unchanged.
|
||||
6. No schema, prompt, apply-proposal, question-target selection, materiality, compatibility, provider, or harness logic changed.
|
||||
|
||||
## What remains unproven until live rerun
|
||||
|
||||
Still unproven until the exact 60B.21 live product-launch regression is rerun:
|
||||
|
||||
- whether the live model-selected node + final deterministic wording path now yields the expected proposition-specific question in the full end-to-end launch-timing case.
|
||||
|
||||
## Production code changed
|
||||
|
||||
YES — `lib/graph/question-formulator.js`
|
||||
|
||||
## Tests changed
|
||||
|
||||
YES — `tests/graph/question-formulator.test.js`
|
||||
|
||||
## Prompt changed
|
||||
|
||||
NO
|
||||
|
||||
## Schema changed
|
||||
|
||||
NO
|
||||
|
||||
## Apply-proposal changed
|
||||
|
||||
NO
|
||||
|
||||
## Question-target selection changed
|
||||
|
||||
NO
|
||||
|
||||
## Materiality changed
|
||||
|
||||
NO
|
||||
|
||||
## Reasoning-context compatibility changed
|
||||
|
||||
NO
|
||||
|
||||
## Provider changed
|
||||
|
||||
NO
|
||||
|
||||
## Harness changed
|
||||
|
||||
NO
|
||||
|
||||
## Ollama calls
|
||||
|
||||
0
|
||||
|
||||
## Live API calls
|
||||
|
||||
0
|
||||
|
||||
## Full suite run
|
||||
|
||||
NO
|
||||
Reference in New Issue
Block a user