49 lines
2.6 KiB
Markdown
49 lines
2.6 KiB
Markdown
# v0.5 Question Priority Generalisation
|
|
|
|
## Hypothesis
|
|
|
|
The current deterministic unknown selector and graph-context question formulator should generalise across several decision types by selecting a foundational unknown before downstream implementation or pricing leaves.
|
|
|
|
## Scenarios
|
|
|
|
1. Should we hire another engineer?
|
|
2. Should we replace the delivery vans?
|
|
3. Should we launch in another country?
|
|
4. Should we continue a project that is over budget?
|
|
5. Should we introduce a paid support tier?
|
|
|
|
## Results
|
|
|
|
| Scenario | Selected unknown | Strategy | Pass/Fail |
|
|
| ---------------------------- | --------------------------- | -------------------- | --------- |
|
|
| Hire another engineer | `hire-success-criteria` | `decision criterion` | Pass |
|
|
| Replace the delivery vans | `van-reliability-threshold` | `decision criterion` | Pass |
|
|
| Launch in another country | `country-value-threshold` | `actor/customer` | Pass |
|
|
| Continue over-budget project | `project-benefit-threshold` | `decision criterion` | Pass |
|
|
| Introduce paid support tier | `support-value-threshold` | `actor/customer` | Pass |
|
|
|
|
## Repeated failure patterns
|
|
|
|
Two repeated structural formulation failures appeared before the final pass:
|
|
|
|
1. **Constraint language in surrounding graph context outranked node-local decision-threshold language** in more than one case.
|
|
2. **Baseline language in surrounding graph context outranked node-local threshold language** in more than one case.
|
|
|
|
Both failures affected formulation strategy, not deterministic unknown selection.
|
|
|
|
## Code change made
|
|
|
|
A small deterministic change was made in `lib/graph/question-formulator.js`:
|
|
|
|
- prefer node-local `definition` language before broader criterion inference
|
|
- prefer node-local `decision criterion` language before context-only `constraint` inference
|
|
- only treat `baseline` or `constraint` as primary when the selected node itself carries that language, otherwise allow them as fallback strategies later
|
|
|
|
No architecture, UI, persistence, prompt, scoring, additional model turns, or provider calls were added.
|
|
|
|
## Remaining limitations
|
|
|
|
- In two passing cases, the selector chose a threshold-style foundational node while the formulator still used an `actor/customer` strategy because related context strongly referenced customers or recipients.
|
|
- This experiment is fixture-driven and deterministic; it is useful for regression protection, not scientific validation.
|
|
- The suite exercises the production path without model calls, but it does not prove behaviour over arbitrary real-world graph structures.
|