confidence-engine/docs/v0.7-question-simplicity-experiment.md

v0.7 Question Simplicity Experiment

Observed failure

The first v0.7 UI scenario exposed a reasoning failure where the selected unknown could still be directionally correct while the resulting question was too large to answer in one coherent response.

Example failure:

Have you measured the current financial or operational cost to users who lack justified confidence, and what baseline budget do they currently allocate for comparable decision-support methods?

This question bundled multiple investigations:

• cost
• user impact
• existing alternatives
• current budget

That violated the intended one-step reasoning discipline.

Principle

A correct unknown paired with an unanswerably broad question is still a reasoning failure.

The engine should ask one question about one primary concept at a time.

The reconstruction model may suggest a question, but only the graph-backed deterministic pipeline may select the user-facing question.

One-question / one-concept rule

Every user-facing question should:

• contain one question mark
• target one unresolved graph node
• ask for one primary concept
• request one coherent answer
• avoid joined investigations
• minimise cognitive effort while still reducing meaningful uncertainty

Deterministic cognitive-load rules

The new deterministic question-complexity assessment marks a question as too broad when it shows signals such as:

• multiple requested answers joined by and
• distinct measures combined in one prompt, such as cost plus budget
• comma-list phrasing that expands the request into several sub-questions
• more than one primary concept
• abstract noun chains that make the question hard to parse on first reading
• very long question length

The assessment returns:

• acceptable
• primaryConceptCount
• compoundQuestionSignals
• abstractTermCount
• cognitiveLoad
• reasons

Decomposition-before-rewording rule

The engine now treats broad commercial-validation unknowns as composite.

If the selected unknown still spans multiple validation dimensions, the system should not simply shorten the sentence. It should first decompose the unknown into smaller child unknowns and then select one foundational child.

For the current scenario, this meant creating child unknowns such as:

• who experiences the problem
• what happens when it is not resolved
• how often it happens
• how people deal with it today
• whether people actively look for help

The selector then reaches the first foundational child through prerequisite ordering encoded in the decomposition graph rather than through global scoring changes.

UI result

The long compound question no longer survives as the first follow-up in the tested path.

The new first-step question is:

Who experiences this problem?

This question:

• asks one thing
• is understandable immediately
• stays graph-backed
• avoids pricing or budget before problem existence is established

Start-case authority rule

There were previously two question paths during initial analysis:

• reconstruction model nextQuestion
• graph-backed unknown selection and question formulation

The defect was that startCase copied the reconstruction nextQuestion directly into the normal UI.

That path is now closed.

Initial user-facing questioning now follows this pipeline:

reconstruction
→ graph build
→ unresolved unknown selection
→ atomicity assessment
→ decomposition if needed
→ investigation strategy
→ question formulation
→ complexity validation
→ selectedQuestion

The reconstruction question is still retained in diagnostics as provenance, but it is not authoritative.

Live result

Running the commercial-method scenario through the real environment now:

• succeeds without the enum compatibility failure
• does not show the broad reconstruction question in the UI path
• surfaces a graph-backed first question instead
• keeps the reconstruction question only in diagnostics

For the tested scenario, the user-facing first question remained:

Who experiences this problem?

Remaining limitations

• question-complexity assessment is still conservative and pattern-based rather than semantic in a richer linguistic sense
• plain-language simplification currently uses a small deterministic replacement set
• broader prerequisite ordering is strongest for decomposition structures that explicitly encode those dependencies