confidence-engine/docs/reasoning-contract-backlog.md

Reasoning Contract Backlog

This document tracks every field that the UI currently mocks because the reasoning engine does not yet provide it. Each row maps a UI need to the temporary workaround and the desired eventual contract.

Legend

Column	Purpose
Feature	The UX / component that needs this field
UI need	What the interface is trying to communicate
Temporary mock	How the UI fakes or derives the value today
Desired output	What the reasoning engine should eventually emit
Likely stage	Which reasoning phase would naturally produce this data
Notes	Context, constraints, open questions

---

Status / State

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
InvestigationSummaryPanel	Current status indicator (investigating / complete / evidence_limit)	Derives from selectedQuestion existence + resolvedNodeIds count	Explicit status enum: "investigating", "resolution_achieved", "evidence_limit_reached"	Post-investigation finalisation	Should be emitted after the engine decides there are no more useful questions
InvestigationSummaryPanel	Elapsed time since last update	Computes Date.now() - result.updatedAt	Engine-provided lastUpdatedAt on every turn	Every API response	UI already stores this; needs confirmation from reasoning

Understanding / Summaries

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
CurrentUnderstandingCard, InvestigationSummaryPanel	Durable plain-language synthesis of current state	result.summary → falls back to graph.currentSummary	A single summary string that represents the latest synthesis	Final summary step; updated at each turn end	Must be stable across refreshes; separate from graph data
CurrentUnderstandingCard	Filter technical summaries from plain-language ones	Heuristic regex against keywords (nodes, edges, by_kind)	Boolean isPlainLanguageSummary flag or guaranteed plain-language field	Every turn	Regex is fragile; engine should guarantee output quality

Questions & Unknowns

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
InvestigationSummaryPanel	Questions answered count	Counts unknown nodes with status === "resolved" or in resolvedNodeIds	Explicit list of resolvedUnknownIds from engine	Post-each turn	Current heuristic conflates structural resolution with questioning
InvestigationSummaryPanel	Still working on count	total unknowns - resolved	Total identified unknowns minus resolved	Finalisation	Should not imply 1 unknown = 1 question
ReasoningWorkspace	Active question (next useful)	selectedQuestion.question from start/update API	Same — but engine should guarantee a question exists when status === "investigating"	Question selection phase	If no question is available, engine should emit evidence_limit_reached instead
ScenarioForm	Selected question reason / "why this matters"	selectedQuestion.reason from fixture	Same — but guaranteed on every turn	Question selection	Already partially wired; just needs consistent coverage
ReasoningWorkspace	Question reasoning pattern metadata	selectedQuestion.reasoningPattern	Engine should emit the pattern class for UI display (e.g. "comparability_check")	Question selection	Used in Developer details; could also inform UI tooltips

Graph & Evidence

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
SituationGraphView	Node confidence values	Mock confidence ("low"/"medium"/"high")	Computed confidence per node from evidence weight	Graph construction	UI displays as indicators; needs numeric or ordinal source
SituationGraphView	Confidence assessment breakdown	confidenceAssessment.evidenceConfidence, completenessStatus, conclusionConfidence	Structured confidence assessment with sub-scores	Evidence analysis	Currently flat mock object
DeveloperDetails	Active unknown node ID	graph.activeUnknownNodeId from fixture	Explicit active target for next investigation step	Question selection	Internal reference; exposed via developer view only
ReasoningWorkspace	Newly surfaced unknown nodes	Scenarios provide proposal.addedNodes or mocks a static list	Engine emits newlySurfacedNodeIds per turn	Each update turn	UI highlights these to show what the investigation discovered
ScenarioForm	Node kind discrimination (observation / assumption / conclusion / unknown)	Hardcoded kind values in mock fixtures	Engine classifies each node correctly	Graph construction	Critical for correct display and reasoning traceability
ScenarioForm	Edge relationships	Mock relationship ("supports", "undermines")	Engine emits relationship type between nodes	Graph construction	Needed for developer view; affects UI if confidence model expands

Evidence & Resolution Tracking

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
DeveloperDetails	evidenceIds per node	Empty array [] in every mock node	List of evidence nodes supporting this node	Graph construction	Needed for traceability in developer view
DeveloperDetails	dependsOn / affects per node	Empty arrays [] in mock nodes	Dependency and effect edges	Graph construction	Shows reasoning structure; currently hidden in collapsed developer details
InvestigationSummaryPanel	Whether evidence limit has been reached (terminal state)	Infers from activeUnknownNodeId === null + unresolved unknowns present	Explicit terminal status flag from engine	Post-evaluation	UI shows "Current evidence limit reached" card
ReasoningWorkspace	resolvedNodeIds from update	Mocked from scenario fixture; mirrors resolved unknown IDs	Engine emits resolvedUnknownNodeIds per turn	Update response	Used to mark answered questions in history
ReasoningWorkspace	affectedNodeIds from update	Empty array in mock	List of nodes changed by this answer	Update response	Developer view; shows ripple effects

Diagnostics & Technical Metadata

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
DiagnosticsView	promptVersion	Hardcoded "v0.4" in mocks	Actual prompt version used for this turn	Every request	Useful for debugging and rollout tracking
DiagnosticsView	modelName	Hardcoded "mock-ollama"	Actual model identifier	Every request	Needed when multiple models are supported
DiagnosticsView	responseDurationMs	Zeroed in mocks	Actual response duration	Every request	Shows user how long reasoning took
DiagnosticsView	validationStatus	Hardcoded "valid"	Whether the output passed structured-validation	Post-processing	UI already uses this to decide if graph was parsed
DeveloperDetails	proposal details (addedNodes, updatedNodes)	Mocked from scenario fixture	Full proposal metadata from reasoning engine	Update response	Shows what changed and why

Recovery & Error States

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
ReasoningWorkspace (ProviderUnavailableCard)	Detect provider/network failure	Regex on error string (provider, unavailable)	Explicit providerAvailable: false flag or HTTP status	Request time	Should distinguish transient from permanent failures
ReasoningWorkspace (MalformedResponseCard)	Detect unstructured / invalid JSON response	Regex on error string (malformed, parse, structured)	Explicit validationError object with path details	Post-processing	UI needs to know the validation failure for debugging
ReasoningWorkspace (UnexpectedStateCard)	Detect internal engine error	stage === "unexpected" from mock	Engine-specific error code + recoverable flag	Any stage	Should distinguish recoverable vs unrecoverable errors

Investigation Lifecycle

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
InvestigationSummaryPanel	investigationStartedAt timestamp	Uses result.updatedAt (from session storage)	Engine-provided investigationStartedAt on start response	Start case	Currently uses last-updated time as fallback; inaccurate
InvestigationSummaryPanel	lastUpdatedAt timestamp	Session updatedAt persisted by UI	Engine-provided timestamp on every update response	Every turn	UI already tracks this via session hook
ReasoningWorkspace	Genuine completion detection	Heuristic: all unknowns resolved + no active question	Explicit genuineCompletion: true from engine	Post-evaluation	Should distinguish "everything resolved" from "stalled"
CompletionCard	Final summary for complete state	propUnderstanding or graph.currentSummary	Engine-emitted final conclusion when all unknowns are resolved	Finalisation	Distinct from intermediate summaries
EvidenceLimitCard	Final summary at evidence limit	Same as above	Engine-emitted terminal summary when no more questions are useful	Finalisation	UI card style differs from CompletionCard

Scenario & Central Statement

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
OriginalSituation	Central statement display	scenario prop (user input) or graph.centralStatement	Engine-derived central statement from user input	Start case	UI already handles both; engine should normalise
DeveloperDetails	Node descriptions	Mock nodes have label === description	Distinct, detailed description per node	Graph construction	Current mock uses label as description; separate fields needed

Investigation Map (Workspace UX)

Open design decision — final map shape intentionally unresolved

The current Investigation Map implementation exists only to validate:

• placement within the workspace;
• information density at preview scale;
• status presentation (established / current / unknown);
• responsive layout across viewports;
• interaction with surrounding components across turns.

It is NOT a committed design. The eventual map should be derived from the reasoning engine, not from hard-coded UI categories.

The following are unresolved design questions — do NOT treat them as agreed contract fields:

• Will the engine provide a flat topic list, hierarchy, branches, or grouped clusters?
• Who determines ordering — engine or user interaction?
• Will there be evidence counts, completion percentages, or path metadata?
• How does the map handle dynamic addition/removal of topics during investigation?

Current entry (temporary)

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
InvestigationMap	Visible investigation progress	mock-only placeholder: minimal set of neutral topic names (≤5) with manual turn-based status progression	Engine emits investigationTopics: [{ title, status, ordering?, evidenceCount? }]	Each turn — start and update response	UI displays topics in engine-determined order; statuses: "established" / "current" / "unknown"
InvestigationMap	Topic status evolution across turns	mock-only placeholder: Hardcoded PROGRESSION array indexed by investigationHistory.length	Engine determines which topics are established, active, or unknown at each turn	Question selection phase	Topics should not expose graph internals; plain-language labels only

The current adapter (lib/map/investigation-map-adapter.js) uses generic placeholder names (e.g. "Starting point", "Current focus") explicitly because they do NOT represent a domain-specific design decision.

Open Questions / Future Work

1. Structured confidence scores: The UI currently mocks ordinal confidence (low/medium/high). The reasoning engine should eventually emit numeric confidence values per node and a computed conclusion confidence, enabling richer visual indicators.

2. Evidence provenance: Nodes mock empty evidenceIds. The engine should emit which observation nodes support each assumption/conclusion, enabling the developer view to show full evidence chains.

3. Turn-level diagnostics: Currently only basic validation metadata is mocked. Full turn diagnostics (prompt used, model, duration, temperature, validation results) would help debugging and monitoring.

4. Terminal state semantics: The UI distinguishes "resolution_achieved" from "evidence_limit_reached" using heuristics. The engine should emit explicit terminal states so the UI can show the appropriate card without inference.

5. Session integrity: The session persistence hook (Phase 3) stores situationGraph + selectedQuestion + summary. If the engine later emits additional fields that affect the UI (e.g., investigationStartedAt, genuineCompletion), the persisted payload should expand to include them.

6. Recovery action granularity: The recovery cards currently offer a single "restart investigation" action. Future engine contracts could support partial recovery (e.g., retry with different parameters, switch models) rather than full restart.

7. Investigation duration tracking: The summary panel computes elapsed time from Date.now() - updatedAt. If the engine emits proper timestamps, the UI can show accurate elapsed duration and investigate stalls (>5 min between turns).

8. Layout independence (v0.7 workspace layout phase): Reasoning outputs must remain entirely independent of presentation layout. The UI's responsive workspace layout — which progressively reveals simultaneous context on wide screens — is a pure presentation concern. No reasoning contract field should be added, removed, or modified to accommodate layout changes. Future reasoning outputs should carry data semantically; how that data arranges itself visually is the responsibility of the presentation layer alone.

---

Behaviour Selection (Experiment 19)

Goal: Test whether selecting from a small set of behaviours — instead of always asking a question — makes the investigation feel like guided thinking rather than automated Q&A.

v0.1 Behaviour Set (5 Patterns)

Behaviour	When to deploy	What it does
Acknowledge	Any turn where user provided useful info (≥1 resolved node)	State what was learned; do not immediately ask
Clarify	Conversation health is too_broad OR phase is orienting with insufficient data	Ask for a single specific piece of context
Summarise	Phase is synthesising/concluding; or ≥3 turns without summarisation	Restate current understanding; compress without losing detail
Continue	Default — no other behaviour matches	Ask the next useful question (current engine behaviour)
Pause	Phase is focusing with stalled progress	Hold space; acknowledge what was learned; invite reflection

Selection Rules (No Scoring, No Weights)

Plain conditions. If multiple fire, priority is: Acknowledge > Clarify > Summarise > Pause > Continue.

1. Acknowledge if conversation health is healthy AND at least one node resolved
2. Clarify if conversation health is too_broad OR phase is orienting with <3 observations
3. Summarise if phase is synthesising/concluding OR ≥3 turns without summarisation
4. Pause if phase is focusing AND progress is stalled
5. Continue as default

Why v0.1 Is Deliberately Narrow

• No scoring or weighting (invented numbers, not observed signals)
• No convergence requirements (design preference, not discovery)
• Only the three assessment dimensions currently available (phase, progress, conversation health)
• One behaviour per turn — no combinations, no stable pairing
• No rationale output or developer view infrastructure (signal first, display later)

Future Considerations (Not In v0.1)

See docs/behaviour-selection.md section "Future Considerations" for: signal weighting, convergence thresholds, full 14-behaviour inventory, Behaviour Readiness derived dimension, rationale output.

---

Investigation State Assessment (Experiment 18)

The investigation state assessment layer introduces three new assessed dimensions that feed into behaviour selection: phase, progress, and conversationHealth. Each dimension has its own value enum, confidence level, descriptive signals, and evidence object. The overall assessment uses the minimum confidence across all dimensions.

Phase Assessment

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
Behaviour Selection (phase gate)	Determine which behaviours are appropriate now	Heuristic based on resolved node count + hasActiveQuestion	assessment.phase: { value, confidence, signals[], evidence } where value ∈ "orienting", "exploring", "focusing", "deepening", "synthesising", "concluding", "cannot_determine"	Per-turn assessment	Deterministic thresholds: conclusive (active=0, resolved≥2), synthesising (active≤1, ratio>0.5), focusing (active=1, observations≥3), exploring (observations≥2, ratio<0.4), deepening (active>1, resolved≥3)

Progress Assessment

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
Behaviour Selection (urgency gate)	Determine whether investigation is moving forward and at what velocity	Heuristic based on resolved count trend	assessment.progress: { value, confidence, signals[], evidence } where value ∈ "accelerating", "steady", "stalled", "looping", "spiralling", "cannot_determine"	Per-turn assessment	Single-snapshot approximation in v0.1 using resolution ratio thresholds: accelerating (>0.6), steady (0.2-0.6), stalled (<0.2 with ≥1 resolved)

Conversation Health Assessment

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
Behaviour Selection (interruption gate)	Determine whether the interaction pattern needs adjustment regardless of investigation state	Heuristic based on question type and unknown count	assessment.conversationHealth: { value, confidence, signals[], evidence } where value ∈ "healthy", "repetitive", "too_broad", "too_narrow", "user_overloaded", "user_under_informed", "cannot_determine"	Per-turn assessment	v0.1 rules: healthy (has unknown + has question), too_broad (>3 active, <2 resolved), too_narrow (≤1 observation with question)

Confidence Aggregation

Feature	UI need	Temporary mock	Desired reasoning output	Likely stage	Notes
Overall assessment trustworthiness	How much should we trust any individual dimension?	N/A — no prior equivalent	assessment.confidence = min(phase.confidence, progress.confidence, conversationHealth.confidence) where "high" < "medium" < "low" < "cannot_determine"	Per-turn assessment	Conservative: if ANY dimension is low/cannot_determine, overall drops. This prevents false precision in behaviour selection.

Evidence Objects (Source Mapping)

All evidence fields are derived from the situation graph and orchestrator diagnostics without direct graph traversal by the behaviour layer:

Evidence Field	Source	Available In
resolvedNodeCount	count of nodes with status "resolved" or in resolvedNodeIds	Every turn
activeUnknownCount	count of unknown-kinded unresolved nodes + active node fallback	Every turn
unknownResolutionRatio	resolvedNodeCount / totalNonEmptyNodes (null if total ≤ 0)	Every turn
observationDensity	observations = observation-kind known/resolved + high-confidence non-unknown non-state	Every turn
evidenceDepth	"shallow" (<2), "moderate" (2-3), "deep" (≥4 observations)	Every turn
turnCount	approximated as floor(totalNodes / 3)	Every turn
hasActiveQuestion	Boolean: selectedQuestion?.nodeId exists	Every turn
summaryLength	Length of situationGraph.currentSummary	Every turn

Current Limitations (Experiment 18 v0.1)

These are acknowledged constraints of the current implementation, not change requests:

• Single-snapshot progress: v0.1 uses a resolution ratio from the current snapshot only. Multi-turn trend detection (looping, spiralling) is planned but requires turn history data not yet available in the contract.
• No evidence quality dimension: This is specified in the architecture doc but requires per-node evidence confidence scoring across multiple sources — not yet implementable.
• No understanding trajectory dimension: Requires comparing narrative complexity across turns; depends on future narrative evolution.
• No uncertainty trend dimension: Requires tracking which unknowns resolve by what pattern across turns.
• No behaviour readiness layer: The final synthesis of all dimensions into behaviour signals is deferred to the behaviour selection experiment.

Implementation Status

Implemented. The assessor (lib/assessment/investigation-state-assessor.js) produces a deterministic assessment object matching this contract at v0.1 schema version. Integration call sites in lib/graph/orchestrator.js (lines ~552, ~904, ~1013) pass correctly shaped input to assessInvestigationState(). The 51-test suite validates all classification rules and edge cases.

---

Facilitator View Projection (Experiment 12)

Version C derives its content from existing graph fields without requiring new backend data. The following fields are used as inputs:

Input	Source
node type / kind	node.kind (observation, unknown, assumption, state, metric, conclusion)
node label or description	node.label, node.description
support / status	node.status, resolvedNodeIds
confidence where available	node.confidence
active unknown identity	graph.activeUnknownNodeId
resolution state	node.status === "resolved" or resolvedNodeIds.includes(id)
evidence references where available	node.evidenceIds (currently empty in mocks)
relationship relevance where available	edge.relevance, node.relationships

Current limitations (observations, not requests)

The following are observed constraints of the current graph output. They are documented here because they affect the adapter's filtering and ranking logic. They should NOT be treated as backend change requests during this experiment.

• Graph text may repeat the full original scenario verbatim in node labels or descriptions.
• Labels may be verbose relative to what a user can scan quickly.
• The selected question rationale and the selected question itself may diverge slightly in wording from the underlying unknown node.
• Ranking signals (relevance, priority) may not be sufficient for ideal user-facing ordering; the adapter uses deterministic fallbacks.
• Some assumptions may be too generic to be useful without context.
• Duplicate semantic content may occur across node types (e.g., an observation and an unknown restating the same scenario fragment).

The adapter handles these limitations through:

1. Length-based filtering of overly verbose items;
2. Normalised text deduplication across node kinds;
3. Deprioritisation of items matching known boilerplate patterns;
4. Deterministic ranking with explicit fallback ordering documented in code comments.