confidence-engine/docs/v0.5-question-priority-generalisation.md

v0.5 Question Priority Generalisation

Hypothesis

The current deterministic unknown selector and graph-context question formulator should generalise across several decision types by selecting a foundational unknown before downstream implementation or pricing leaves.

Scenarios

1. Should we hire another engineer?
2. Should we replace the delivery vans?
3. Should we launch in another country?
4. Should we continue a project that is over budget?
5. Should we introduce a paid support tier?

Results

Scenario	Selected unknown	Strategy	Pass/Fail
Hire another engineer	hire-success-criteria	decision criterion	Pass
Replace the delivery vans	van-reliability-threshold	decision criterion	Pass
Launch in another country	country-value-threshold	actor/customer	Pass
Continue over-budget project	project-benefit-threshold	decision criterion	Pass
Introduce paid support tier	support-value-threshold	actor/customer	Pass

Repeated failure patterns

Two repeated structural formulation failures appeared before the final pass:

1. Constraint language in surrounding graph context outranked node-local decision-threshold language in more than one case.
2. Baseline language in surrounding graph context outranked node-local threshold language in more than one case.

Both failures affected formulation strategy, not deterministic unknown selection.

Code change made

A small deterministic change was made in lib/graph/question-formulator.js:

• prefer node-local definition language before broader criterion inference
• prefer node-local decision criterion language before context-only constraint inference
• only treat baseline or constraint as primary when the selected node itself carries that language, otherwise allow them as fallback strategies later

No architecture, UI, persistence, prompt, scoring, additional model turns, or provider calls were added.

Remaining limitations

• In two passing cases, the selector chose a threshold-style foundational node while the formulator still used an actor/customer strategy because related context strongly referenced customers or recipients.
• This experiment is fixture-driven and deterministic; it is useful for regression protection, not scientific validation.
• The suite exercises the production path without model calls, but it does not prove behaviour over arbitrary real-world graph structures.