diff --git a/docs/v0.6-reasoning-architecture.md b/docs/v0.6-reasoning-architecture.md new file mode 100644 index 0000000..0c456d6 --- /dev/null +++ b/docs/v0.6-reasoning-architecture.md @@ -0,0 +1,375 @@ +# v0.6 Reasoning Architecture + +## Purpose + +This document describes the implemented deterministic reasoning architecture on branch `feature/question-strategy-alignment-v0.6`. + +It is written for future developers who need to understand how v0.6 actually executes, what invariants it relies on, where the recursive loops are, and what the system deliberately does **not** attempt to do. + +## End-to-end pipeline + +The implemented runtime pipeline is: + +```text +Scenario input +↓ +LLM analysis / reconstruction +↓ +Initial graph build +↓ +Deterministic unknown selection +↓ +Selected question +↓ +User answer +↓ +LLM graph-update proposal +↓ +Proposal parsing / normalisation +↓ +Proposal compatibility validation +↓ +Deterministic graph update application +↓ +Reasoning-state rebuild +↓ +Comparability assessment +↓ +Relationship classification +↓ +Explicit emergent unknown creation / reuse (if required) +↓ +Deterministic reselection +↓ +Atomicity assessment +↓ +Optional decomposition into child unknowns +↓ +Deterministic reselection +↓ +Resolved-child propagation upward +↓ +Confidence / completeness / corroboration update +↓ +Next active unknown +↓ +Question formulation +``` + +## Deterministic stages + +### 1. Scenario analysis / reconstruction + +- **Purpose**: obtain structured reconstruction material from scenario text +- **Input**: scenario, prompt version +- **Output**: analysis payload containing reconstruction, evidence, diagnostics, and optional next question +- **Why it exists**: provides the initial structured substrate from which the graph is built +- **What breaks if removed**: the graph builder has no structured reconstruction to convert into nodes and edges + +### 2. Initial graph build + +- **Purpose**: convert reconstruction output into an initial `SituationGraph` +- **Input**: reconstruction + evidence +- **Output**: graph nodes and edges, then `makeGraph(...)` wraps them with active/resolved/summary state +- **Why it exists**: all later reasoning is graph-based, not free text +- **What breaks if removed**: no explicit unknown nodes, no deterministic selection, no validated update loop + +### 3. Deterministic unknown selection + +- **Purpose**: choose the next active unknown from unresolved graph nodes +- **Input**: graph, resolved node IDs +- **Output**: selected candidate or explicit ambiguity result +- **Why it exists**: the system needs a deterministic next investigation target +- **What breaks if removed**: question ordering becomes arbitrary or hidden in prompts + +### 4. Selected question exposure + +- **Purpose**: expose the chosen unknown as the next question to the user +- **Input**: selected unknown + question formulation or tie-resolution logic +- **Output**: selected question object +- **Why it exists**: the user-facing loop must ask a concrete next question +- **What breaks if removed**: the system can build a graph but cannot continue interaction coherently + +### 5. LLM graph-update proposal + +- **Purpose**: transform a user answer into a proposed graph change set +- **Input**: current graph, previous question, answer, prompt version +- **Output**: raw JSON-like proposal +- **Why it exists**: the LLM is limited to proposing changes; it does not mutate the graph directly +- **What breaks if removed**: answers cannot affect the graph except through manual hard-coded logic + +### 6. Proposal parsing / normalisation + +- **Purpose**: parse JSON, remove null array items, apply known aliases, fill omitted optional fields +- **Input**: raw model response +- **Output**: validated `graphUpdateSchema` payload or structured parser failure +- **Why it exists**: model outputs are not trusted as-is +- **What breaks if removed**: malformed or partially missing model output would reach graph logic directly + +### 7. Proposal compatibility validation + +- **Purpose**: ensure the proposal is graph-safe and semantically valid before application +- **Input**: current graph + proposed update +- **Output**: accepted proposal or compatibility errors +- **Why it exists**: protects graph integrity and reasoning invariants +- **What breaks if removed**: duplicate IDs, missing references, fake selected questions, and no-op updates could corrupt the graph + +### 8. Deterministic graph update application + +- **Purpose**: apply only validated graph changes to a copied graph +- **Input**: graph + validated proposal +- **Output**: updated nodes, edges, resolved node IDs +- **Why it exists**: separates safe application from generation +- **What breaks if removed**: no explicit, replayable state transition exists + +### 9. Reasoning-state rebuild + +- **Purpose**: derive fresh comparability/relationship state from the updated graph +- **Input**: updated graph + optional override state +- **Output**: `reasoningState` +- **Why it exists**: reasoning stages are derived from graph state, not stored blindly +- **What breaks if removed**: comparability and relationship decisions drift from actual graph contents + +### 10. Comparability assessment + +- **Purpose**: decide whether supported observations are comparable enough for relationship reasoning +- **Input**: graph observations + central statement + optional stored override +- **Output**: comparability status/reason + contradiction permission +- **Why it exists**: relationship reasoning is gated by comparability +- **What breaks if removed**: contradiction or relationship reasoning would run over incomparable observations + +### 11. Relationship classification + +- **Purpose**: classify observation relationships once comparability permits it +- **Input**: graph + comparability result +- **Output**: relationship status, reason, whether a follow-up question is justified +- **Why it exists**: determines whether explanation-style follow-up is needed +- **What breaks if removed**: the system cannot distinguish compatible, duplicate, insufficient, and contradiction-adjacent observation sets + +### 12. Explicit emergent unknown creation / reuse + +- **Purpose**: ensure any justified relationship follow-up is represented by an explicit unresolved graph node +- **Input**: provisional graph + relationship assessment +- **Output**: reused or newly added explanation unknown and edges +- **Why it exists**: preserves the invariant that a question must originate from an explicit unknown +- **What breaks if removed**: relationship follow-up would revert to fallback-only question text not backed by the graph + +### 13. Atomicity assessment + +- **Purpose**: determine whether the selected unknown is directly investigable or too composite +- **Input**: selected unknown + graph context +- **Output**: `atomic` or `composite` decision with decomposition kind/reason +- **Why it exists**: prevents asking broad explanation unknowns directly +- **What breaks if removed**: the system asks high-level composite unknowns instead of decomposing them first + +### 14. Optional decomposition + +- **Purpose**: split a composite unknown into deterministic child unknowns +- **Input**: composite selected unknown + graph context +- **Output**: 2–5 child unknowns, edges, quality summary, rejection diagnostics +- **Why it exists**: narrows broad unknowns into explicit candidate dimensions +- **What breaks if removed**: recursive reasoning stops at broad parents and loses graph-backed substructure + +### 15. Resolved-child propagation upward + +- **Purpose**: move resolved child effects to parent and ancestor chain without prematurely resolving them +- **Input**: updated graph + proposal snapshot +- **Output**: parent/ancestor status and confidence updates, additional diagnostics +- **Why it exists**: decomposition requires deterministic reconstruction as well as decomposition +- **What breaks if removed**: child answers stay local and parents never become progressively better-supported + +### 16. Confidence / completeness / corroboration update + +- **Purpose**: derive parent-level `confidenceAssessment` from resolved children and branch interactions +- **Input**: parent child set + branch evidence/status interactions +- **Output**: `evidenceConfidence`, `completenessStatus`, `conclusionConfidence`, plus derived display `confidence` +- **Why it exists**: reasoning support must be separated from completion and contradiction state +- **What breaks if removed**: parent confidence collapses back into vague status-driven heuristics + +### 17. Next active unknown + question formulation + +- **Purpose**: reselect the next unresolved unknown and formulate a concrete next question +- **Input**: updated graph + selection state + graph context +- **Output**: next active unknown and question object +- **Why it exists**: closes the recursive interaction loop +- **What breaks if removed**: the system updates the graph but cannot continue investigation deterministically + +## Architectural invariants + +The current implementation enforces these invariants: + +1. **A question must originate from an explicit unresolved unknown node.** +2. **Unknown selection is deterministic.** +3. **Alphabetical ordering is not treated as reasoning.** +4. **Relationship reasoning does not precede comparability.** +5. **The LLM never mutates the graph directly; it only proposes updates.** +6. **All graph updates are schema-validated before application.** +7. **All node/edge references must resolve to existing nodes.** +8. **Duplicate node IDs are rejected.** +9. **Duplicate added edge IDs are rejected.** +10. **A selected question cannot target a resolved unknown.** +11. **A resolved unknown updated to `resolved` must also appear in `resolvedUnknownNodeIds`.** +12. **A proposal must contain a meaningful change.** +13. **Every newly added unknown must include why-it-matters language.** +14. **Every newly added unknown must be explicitly connected to answer-derived graph structure.** +15. **Composite selected unknowns are decomposed before direct questioning when atomicity rules require it.** +16. **Parent unknowns remain unresolved until completion rules are satisfied.** +17. **Confidence must not outrun completeness.** +18. **Duplicate evidence cannot increase confidence.** +19. **Conflicting evidence caps conclusion confidence.** +20. **Cross-branch corroboration only counts for distinct branches with distinct evidence keys.** +21. **Ambiguous leading unknowns remain explicit ambiguity, not silent forced choice.** + +## Recursive loops and stopping rules + +### Main investigation loop + +```text +Unknown +↓ +Question +↓ +Answer +↓ +Proposal +↓ +Graph update +↓ +Propagation +↓ +Next unknown +``` + +- **Exit condition**: no unresolved candidates remain, or no next question is justified, or proposal/application fails +- **Stopping rule**: deterministic selection returns `null` or explicit ambiguity, or update validation blocks progress +- **Completion behaviour**: continues only while the graph contains justified unresolved unknowns + +### Decomposition loop + +```text +Selected unknown +↓ +Atomicity assessment +↓ +If composite: decompose +↓ +Reselect child +↓ +Atomicity assessment again +``` + +- **Exit condition**: selected child is atomic; parent already has children; max decomposition depth reached; or decomposition quality fails +- **Stopping rule**: `MAX_DECOMPOSITION_DEPTH`, reuse instead of regeneration, or inability to produce enough valid child unknowns +- **Completion behaviour**: deterministic and bounded; no infinite recursive decomposition path is intentionally allowed + +### Propagation loop + +```text +Resolved child +↓ +Ancestor chain walk +↓ +Recompute parent state +↓ +Stop when no ancestor state changes +``` + +- **Exit condition**: no more parents in the ancestor chain or no state change +- **Stopping rule**: ancestor chain is explicit and finite; propagation does not invent new ancestors +- **Completion behaviour**: deterministic upward traversal with explicit stop on unchanged state + +### Potential infinite loops reviewed + +- **Unknown/question recursion**: bounded by unresolved unknown set, proposal validation, and explicit no-candidate states +- **Decomposition recursion**: bounded by max depth and child reuse rules +- **Propagation recursion**: bounded by finite ancestor chain and no-change stop condition + +No intentional infinite reasoning loop is present in the implemented architecture. + +## Graph lifecycle summary + +### Node lifecycle + +1. node created by `buildInitialGraph` or later proposal/decomposition/emergent-unknown logic +2. node validated by schema +3. node may become active unknown +4. node may be updated by proposal application +5. unknown node may become `resolved`, `provisional`, `contradicted`, or remain `unknown` +6. resolved unknown ID is tracked in `resolvedNodeIds` + +### Edge lifecycle + +1. edge created in initial graph or by deterministic proposal augmentation +2. edge validated against existing node IDs +3. edge may be removed only through explicit `removedEdgeIds` +4. edge relationships also update `dependsOn` / `childIds` projections during application + +### Unknown lifecycle + +1. initial unknown discovered from reconstruction +2. selected deterministically or left ambiguous +3. may be decomposed if composite +4. may be resolved directly by answer +5. may cause emergent reasoning unknown creation when relationship reasoning demands a new explicit question target + +### Resolved lifecycle + +1. proposal marks unresolved unknown resolved +2. reconciliation ensures resolution semantics are explicit +3. `resolvedUnknownNodeIds` feed graph application +4. propagation may resolve parent only when completion rule is met + +### Confidence lifecycle + +1. nodes begin with base `confidence` +2. parent/ancestor propagation derives `confidenceAssessment` +3. display `confidence` is derived from `conclusionConfidence` +4. completeness, duplicate evidence, contradiction, and corroboration constrain the result + +### Question lifecycle + +1. selected unknown becomes question target +2. `formulateQuestion` or tie-resolution logic produces question text +3. answer returns through update route +4. proposal may select a new question target or leave reselection to deterministic logic + +Every major transition above is explicit in the current codebase rather than implicit in model text alone. + +## Duplicated or overlapping concepts + +The following concepts are intentionally close and may look duplicated: + +- **status vs confidence**: status captures lifecycle/progression; confidence captures support strength +- **confidence vs confidenceAssessment**: `confidence` is now a derived display field, while `confidenceAssessment` carries separated reasoning dimensions +- **resolvedNodeIds vs node.status === resolved**: both are maintained; the first is a graph-level index, the second is node-local state +- **selectedQuestion in proposal vs selectedQuestion in final result**: proposal may omit or propose one, final result recomputes deterministic selection/questioning after graph logic +- **comparability state in reasoningState vs derived comparability from graph**: overrides may carry forward prior confirmed reasoning, but `buildReasoningState` still rebuilds from graph + override context + +These are not necessarily defects, but they are the main places where future simplification pressure is likely. + +## Known boundaries and deliberate exclusions + +v0.6 deliberately does **not** attempt the following: + +- probabilistic reasoning +- Bayesian inference +- persistence +- semantic embeddings +- fuzzy semantic similarity +- autonomous exploration outside explicit user answers +- multi-hop corroboration across unrelated subtrees without a shared direct parent +- expert-only jargon-specific reasoning modes +- UI-heavy reasoning visualisation beyond existing graph/update displays +- arbitrary non-deterministic tie breaking + +## Defects found during this review + +No new production defect was intentionally introduced or fixed as part of this architecture review. + +## Developer notes + +- `startCase` owns reconstruction → graph build → first deterministic selection. +- `updateCaseWithDependencies` owns proposal generation / parsing and delegates deterministic graph semantics to `applyValidatedProposal`. +- `applyValidatedProposal` is the main reasoning pipeline coordinator for update-time graph evolution. +- `question-formulator.js` owns comparability, relationship classification, atomicity assessment, investigation strategy selection, and question formulation. +- `utils.js` owns selection scoring, ordering, graph validation, and safe graph update application.