confidence-engine/docs/v0.7-question-simplicity-experiment.md

Post-update selection invariant

After every successful graph update, the full deterministic question-selection pipeline must run again whenever eligible unresolved unknowns remain.

The active reasoning pattern constrains which graph nodes may participate in reasoning.

That means the update path must not stop at graph mutation, child resolution, emergent unknown creation, decomposition, or upward propagation. It must continue through:

updated graph
→ rebuild reasoning state
→ identify unresolved candidates
→ select active unknown
→ atomicity assessment
→ answerability assessment
→ decompose if required
→ reselect
→ reasoning-pattern selection
→ investigation-strategy selection
→ question-family selection
→ question formulation
→ complexity validation
→ selectedQuestion

Returning no question is only valid when no eligible unresolved candidate remains, the case is complete, ambiguity cannot be safely resolved, or question formulation fails validation with an explicit deterministic reason.

Graph validity vs reasoning-pattern validity

These are separate requirements.

• Graph validity means references, IDs, node shapes, and update semantics are structurally correct.
• Reasoning-pattern validity means selectable investigation nodes are compatible with the current reasoning mode.

A graph can be structurally valid while still being reasoning-invalid.

Example: a decision investigation may still contain an unresolved comparison-style node such as How the two observations were measured. That node is structurally well-formed, but it is not allowed to participate as an active investigation target unless the reasoning pattern has actually shifted into comparison, contradiction, or explanation work.

The engine therefore needs both invariants:

1. the graph must be structurally valid
2. every selectable unknown must be compatible with the active reasoning pattern

v0.7 Question Simplicity Experiment

Observed failure

The first v0.7 UI scenario exposed a reasoning failure where the selected unknown could still be directionally correct while the resulting question was too large to answer in one coherent response.

Example failure:

Have you measured the current financial or operational cost to users who lack justified confidence, and what baseline budget do they currently allocate for comparable decision-support methods?

This question bundled multiple investigations:

• cost
• user impact
• existing alternatives
• current budget

That violated the intended one-step reasoning discipline.

Principle

A correct unknown paired with an unanswerably broad question is still a reasoning failure.

The engine should ask one question about one primary concept at a time.

The reconstruction model may suggest a question, but only the graph-backed deterministic pipeline may select the user-facing question.

A node is only questionable if it is independently answerable.

One-question / one-concept rule

Every user-facing question should:

• contain one question mark
• target one unresolved graph node
• ask for one primary concept
• request one coherent answer
• avoid joined investigations
• minimise cognitive effort while still reducing meaningful uncertainty

Deterministic cognitive-load rules

The new deterministic question-complexity assessment marks a question as too broad when it shows signals such as:

• multiple requested answers joined by and
• distinct measures combined in one prompt, such as cost plus budget
• comma-list phrasing that expands the request into several sub-questions
• more than one primary concept
• abstract noun chains that make the question hard to parse on first reading
• very long question length

The assessment returns:

• acceptable
• primaryConceptCount
• compoundQuestionSignals
• abstractTermCount
• cognitiveLoad
• reasons

Decomposition-before-rewording rule

The engine now treats broad commercial-validation unknowns as composite.

If the selected unknown still spans multiple validation dimensions, the system should not simply shorten the sentence. It should first decompose the unknown into smaller child unknowns and then select one foundational child.

For the current scenario, this meant creating child unknowns such as:

• who experiences the problem
• what happens when it is not resolved
• how often it happens
• how people deal with it today
• whether people actively look for help

The selector then reaches the first foundational child through prerequisite ordering encoded in the decomposition graph rather than through global scoring changes.

Atomicity vs answerability

These are different reasoning properties.

• Atomicity asks: does this node describe one investigation or several bundled investigations?
• Answerability asks: even if the wording looks singular, can this node be answered directly without first resolving multiple prerequisite dimensions?

A node can appear atomic in wording but still fail answerability.

Examples include broad evaluation containers such as product validation, customer value, business case, technical feasibility, or commercial justification. These often compress several prerequisite investigations into one conclusion-shaped unknown.

That means atomicity alone is not enough.

The engine now decomposes whenever either of these is true:

• the unknown is not atomic
• the unknown is not independently answerable

This prevents a broad container node from becoming the selected question target even when its wording looks grammatically singular.

Reasoning Pattern

The next failure exposed a deeper issue: even after atomicity and answerability were added, the engine could still choose a question template from the wrong reasoning family.

The live failure was an explanation-style prompt appearing in a commercial validation scenario:

What changed during the period that could help explain why ...

That was wrong not because of wording, but because the engine had selected an explanation family when the actual task was a decision investigation.

To correct that, the deterministic pipeline now explicitly inserts a reasoning-pattern stage:

selected unknown
→ atomicity
→ answerability
→ reasoning pattern
→ investigation strategy
→ question family
→ question

This matters because each stage must constrain the next.

• Reasoning Pattern decides what kind of reasoning is happening
• Investigation Strategy decides how to reduce uncertainty within that pattern
• Question Family decides what template space is allowed
• Question is the final concrete wording

Without this stage separation, strategy and template selection can leak across domains and reuse relationship/explanation prompts too broadly.

Deterministic reasoning-pattern vocabulary

The current deterministic pattern vocabulary is intentionally small:

• decision
• explanation
• contradiction
• definition
• diagnosis
• comparison
• prioritisation

Pattern selection uses graph structure rather than wording alone, including:

• node kind
• relationship / observation topology
• parent context
• reasoning state
• selected unknown role in the graph

Question-family mapping

Patterns now constrain which question families are allowed.

• decision
  • decision_foundation
  • decision_evidence
  • decision_threshold
  • definition
• explanation
  • explanation
  • comparison
• contradiction
  • contradiction
  • comparison
  • explanation
• definition
  • definition
• diagnosis
  • diagnosis
  • comparison
• comparison
  • comparison
• prioritisation
  • prioritisation
  • decision_threshold

Most importantly:

• explanation templates are only allowed for explanation or contradiction
• decision investigations cannot emit explanation-family questions

Live correction

For the commercial-method scenario, the engine now classifies the reasoning as a decision pattern rather than an explanation pattern.

That means explanation-family templates are explicitly rejected, and the selected child unknown must be questioned using a decision-compatible family instead.

UI result

The long compound question no longer survives as the first follow-up in the tested path.

The new first-step question is:

Who experiences this problem?

This question:

• asks one thing
• is understandable immediately
• stays graph-backed
• avoids pricing or budget before problem existence is established

Start-case authority rule

There were previously two question paths during initial analysis:

• reconstruction model nextQuestion
• graph-backed unknown selection and question formulation

The defect was that startCase copied the reconstruction nextQuestion directly into the normal UI.

That path is now closed.

Initial user-facing questioning now follows this pipeline:

reconstruction
→ graph build
→ unresolved unknown selection
→ atomicity assessment
→ decomposition if needed
→ investigation strategy
→ question formulation
→ complexity validation
→ selectedQuestion

The reconstruction question is still retained in diagnostics as provenance, but it is not authoritative.

Live result

Running the commercial-method scenario through the real environment now:

• succeeds without the enum compatibility failure
• does not show the broad reconstruction question in the UI path
• surfaces a graph-backed first question instead
• keeps the reconstruction question only in diagnostics

For the tested scenario, the user-facing first question remained:

Who experiences this problem?

Remaining limitations

• question-complexity assessment is still conservative and pattern-based rather than semantic in a richer linguistic sense
• plain-language simplification currently uses a small deterministic replacement set
• broader prerequisite ordering is strongest for decomposition structures that explicitly encode those dependencies