confidence-engine/docs/v0.7-question-simplicity-experiment.md

v0.7 Question Simplicity Experiment

Observed failure

The first v0.7 UI scenario exposed a reasoning failure where the selected unknown could still be directionally correct while the resulting question was too large to answer in one coherent response.

Example failure:

Have you measured the current financial or operational cost to users who lack justified confidence, and what baseline budget do they currently allocate for comparable decision-support methods?

This question bundled multiple investigations:

• cost
• user impact
• existing alternatives
• current budget

That violated the intended one-step reasoning discipline.

Principle

A correct unknown paired with an unanswerably broad question is still a reasoning failure.

The engine should ask one question about one primary concept at a time.

One-question / one-concept rule

Every user-facing question should:

• contain one question mark
• target one unresolved graph node
• ask for one primary concept
• request one coherent answer
• avoid joined investigations
• minimise cognitive effort while still reducing meaningful uncertainty

Deterministic cognitive-load rules

The new deterministic question-complexity assessment marks a question as too broad when it shows signals such as:

• multiple requested answers joined by and
• distinct measures combined in one prompt, such as cost plus budget
• comma-list phrasing that expands the request into several sub-questions
• more than one primary concept
• abstract noun chains that make the question hard to parse on first reading
• very long question length

The assessment returns:

• acceptable
• primaryConceptCount
• compoundQuestionSignals
• abstractTermCount
• cognitiveLoad
• reasons

Decomposition-before-rewording rule

The engine now treats broad commercial-validation unknowns as composite.

If the selected unknown still spans multiple validation dimensions, the system should not simply shorten the sentence. It should first decompose the unknown into smaller child unknowns and then select one foundational child.

For the current scenario, this meant creating child unknowns such as:

• who experiences the problem
• what happens when it is not resolved
• how often it happens
• how people deal with it today
• whether people actively look for help

The selector then reaches the first foundational child through prerequisite ordering encoded in the decomposition graph rather than through global scoring changes.

UI result

The long compound question no longer survives as the first follow-up in the tested path.

The new first-step question is:

Who experiences this problem?

This question:

• asks one thing
• is understandable immediately
• stays graph-backed
• avoids pricing or budget before problem existence is established

Remaining limitations

• question-complexity assessment is still conservative and pattern-based rather than semantic in a richer linguistic sense
• plain-language simplification currently uses a small deterministic replacement set
• broader prerequisite ordering is strongest for decomposition structures that explicitly encode those dependencies