docs: record audience question routing fix

This commit is contained in:
2026-08-13 16:58:32 +01:00
parent 8cca70774c
commit 4767de30f7
2 changed files with 200 additions and 0 deletions
+12
View File
@@ -152,6 +152,18 @@ Experiment 54N tested whether an interpretation disagreement can be judged for m
- `npx vitest run tests/graph/apply-proposal.test.js tests/graph/reasoning-context-compatibility.test.js` → PASS (96/96) - `npx vitest run tests/graph/apply-proposal.test.js tests/graph/reasoning-context-compatibility.test.js` → PASS (96/96)
- **Still unproven live:** the exact 60B.12 client-retention continuation case under live model output. - **Still unproven live:** the exact 60B.12 client-retention continuation case under live model output.
### 60B.23 audience-question routing specificity
- 60B.22 identified that `decision_audience` routing was too broad: any occurrence of `customer|user|buyer|stakeholder|recipient|audience` inside a decision unknown could hijack a proposition-specific question before proposition extraction ran.
- `decision_audience` is now gated by explicit audience-identity phrasing only; raw noun occurrence is no longer sufficient.
- `extractMeaning()` now prefers explicit `whether ...` propositions from descriptions when the label is a nominal status/risk/likelihood phrase, so proposition-specific decision unknowns preserve their actual unresolved proposition.
- Decision-pattern `whether ...` propositions now remain on `decision_evidence_clarification` instead of being pre-empted by `decision_threshold_outcome`.
- Legitimate audience-discovery cases remain preserved, and 60B.20-style direct interrogatives remain unchanged.
- Focused verification passed:
- `npx vitest run tests/graph/question-formulator.test.js` → PASS (25/25)
- `npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js` → PASS (45/45)
- **Still unproven live:** the exact 60B.21 product-launch regression in the full end-to-end path.
### When This Knowledge-Management Phase Is Complete ### When This Knowledge-Management Phase Is Complete
Provisional criteria for review (all confirmed met by Experiment 38 cold-start test): Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):
+188
View File
@@ -0,0 +1,188 @@
# Experiment 60B.23 — Narrow decision-audience routing to genuine audience-identity uncertainty
**Branch:** `feature/question-family-specificity-v0.29`
**Date:** 2026-08-13
**Status:** COMPLETE
**Type:** BOUNDED IMPLEMENTATION
## Objective
Implement the smallest deterministic narrowing so proposition-specific decision unknowns containing audience nouns do not get hijacked by the generic `decision_audience` route.
The target regression from 60B.21 / 60B.22 was:
```text
label:
Prospective enterprise customer signing status
description:
Uncertainty about whether the prospective enterprise customer will sign if we launch this year, so that its resolution is needed to decide which timing option provides superior net value.
```
This should not be treated as audience discovery merely because `customer` appears in the text.
## 60B.22 diagnosis carried forward
60B.22 established two linked causes:
1. `selectQuestionFamily()` routed any decision-pattern node containing `customer|user|buyer|stakeholder|recipient|audience` into `decision_audience` via first-match early return.
2. `extractMeaning()` also genericised any such noun occurrence into `the relevant customer, user, or value recipient`, bypassing the underlying unresolved proposition.
The implementation boundary for 60B.23 was therefore:
> audience-family routing depends on the semantic role of the audience term, not mere lexical presence.
## Exact semantic narrowing implemented
### 1. Audience-family routing now requires explicit audience-identity phrasing
Added a shared deterministic helper in `lib/graph/question-formulator.js`:
- `hasAudienceIdentityQuestion(text)`
This fires only for explicit audience-identity forms such as:
- `Who is the target customer?`
- `Which buyers are we building this for?`
- `Who would receive the value?`
- `Which audience should this serve?`
- `Who experiences this problem?`
It does **not** fire merely because a customer/user/buyer/stakeholder/recipient noun appears as the subject of another proposition.
### 2. Proposition-preserving extraction now prefers explicit `whether...` descriptions for status-like labels
When the label is a nominal status phrase (`status`, `likelihood`, `probability`, `chance`, `risk`, `uncertainty`) and the description begins with:
```text
Uncertainty about whether ...
```
`extractMeaning()` now returns the explicit `whether ...` proposition directly instead of the nominal label phrase.
This preserves proposition-specific meaning for cases like:
- customer will sign
- customer will renew
- users will adopt the change
- stakeholder will approve the plan
without hardcoding any product-launch wording.
### 3. Decision threshold routing no longer preempts proposition-specific `whether ...` decision unknowns
Within decision-pattern family selection, if extracted meaning is already a `whether ...` proposition, the node stays on `decision_evidence_clarification` rather than being diverted into `decision_threshold_outcome`.
### 4. Legitimate audience questions remain valid through post-build validation
The formulated audience question still resolves to the `decision_audience` family and survives existing question validation. No prompt, schema, provider, or targeting logic changed.
## Behaviour preserved
### Legitimate audience-discovery cases preserved
True audience-identity questions still route to `decision_audience`.
### 60B.20 direct interrogative behaviour preserved
Existing direct proposition-style interrogatives such as:
```text
Will our largest client leave if we relocate?
```
remain unchanged and still produce direct proposition-specific wording.
## Focused test results
### Dedicated question-formulator suite
```bash
npx vitest run tests/graph/question-formulator.test.js
```
Result: **PASS (25/25)**
Covered:
- 60B.21 signing-status regression
- legitimate audience identity preservation
- customer-as-subject non-audience case
- user-as-subject non-audience case
- buyer/stakeholder lexical mention non-audience case
- existing direct interrogative preservation
### Smallest broader regression suite containing decision-family tests
```bash
npx vitest run tests/graph/question-formulator.test.js tests/graph/question-formulation-v0.24.test.js
```
Result: **PASS (45/45)**
## What is now guaranteed
1. Raw audience/customer/user/etc. noun occurrence is no longer sufficient to trigger `decision_audience`.
2. `decision_audience` now requires that audience identity itself be unresolved.
3. Proposition-specific decision unknowns with audience nouns can preserve their unresolved proposition into the final question.
4. Existing audience-discovery questions still route to the audience family.
5. Existing interrogative direct-question behaviour remains unchanged.
6. No schema, prompt, apply-proposal, question-target selection, materiality, compatibility, provider, or harness logic changed.
## What remains unproven until live rerun
Still unproven until the exact 60B.21 live product-launch regression is rerun:
- whether the live model-selected node + final deterministic wording path now yields the expected proposition-specific question in the full end-to-end launch-timing case.
## Production code changed
YES — `lib/graph/question-formulator.js`
## Tests changed
YES — `tests/graph/question-formulator.test.js`
## Prompt changed
NO
## Schema changed
NO
## Apply-proposal changed
NO
## Question-target selection changed
NO
## Materiality changed
NO
## Reasoning-context compatibility changed
NO
## Provider changed
NO
## Harness changed
NO
## Ollama calls
0
## Live API calls
0
## Full suite run
NO