robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Phase 2-5 UX enhancements — recovery cards, session persistence, summary panel, contract backlog

Browse Source 
Phase 2: Recovery state components (ProviderUnavailableCard,
MalformedResponseCard, UnexpectedStateCard, ContinueLaterBanner) with
automatic error detection for provider/network/malformed/unexpected states.

Phase 3: Session persistence via sessionStorage — save after each
successful turn, restore on mount, clear on restart/reset. Continuelater banner shown when session is restored.

Phase 4: InvestigationSummaryPanel component displaying current status,
understanding summary, questions answered/remaining, investigation timestamps.

Phase 5: docs/reasoning-contract-backlog.md documenting all mocked
fields (60+ rows across 7 categories) with feature/UI need/mock/desired
output/stage/notes columns.

Also: wired onRestart through ReasoningWorkspace → ScenarioForm, fixed
getErrorType scope issues, removed broken window.__restartInvestigation.
This commit is contained in:
robbond
2026-08-05 06:48:59 +01:00
parent 28289bb4b7
commit c4f5744c30
6 changed files with 1007 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/reasoning-contract-backlog.md
+114
View File
@@ -0,0 +1,114 @@
# Reasoning Contract Backlog
This document tracks every field that the UI currently mocks because the
reasoning engine does not yet provide it. Each row maps a UI need to the
temporary workaround and the desired eventual contract.
## Legend
| Column | Purpose |
| ----------------- | --------------------------------------------------------------------------------------------------- |
| **Feature** | The UX / component that needs this field |
| **UI need** | What the interface is trying to communicate |
| **Temporary mock** | How the UI fakes or derives the value today |
| **Desired output** | What the reasoning engine should eventually emit |
| **Likely stage** | Which reasoning phase would naturally produce this data |
| **Notes** | Context, constraints, open questions |
---
## Status / State
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| InvestigationSummaryPanel | Current status indicator (investigating / complete / evidence_limit) | Derives from `selectedQuestion` existence + `resolvedNodeIds` count | Explicit `status` enum: `"investigating"`, `"resolution_achieved"`, `"evidence_limit_reached"` | Post-investigation finalisation | Should be emitted after the engine decides there are no more useful questions |
| InvestigationSummaryPanel | Elapsed time since last update | Computes `Date.now() - result.updatedAt` | Engine-provided `lastUpdatedAt` on every turn | Every API response | UI already stores this; needs confirmation from reasoning |
## Understanding / Summaries
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| CurrentUnderstandingCard, InvestigationSummaryPanel | Durable plain-language synthesis of current state | `result.summary` → falls back to `graph.currentSummary` | A single `summary` string that represents the latest synthesis | Final summary step; updated at each turn end | Must be stable across refreshes; separate from graph data |
| CurrentUnderstandingCard | Filter technical summaries from plain-language ones | Heuristic regex against keywords (`nodes`, `edges`, `by_kind`) | Boolean `isPlainLanguageSummary` flag or guaranteed plain-language field | Every turn | Regex is fragile; engine should guarantee output quality |
## Questions & Unknowns
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| InvestigationSummaryPanel | Questions answered count | Counts unknown nodes with `status === "resolved"` or in `resolvedNodeIds` | Explicit list of `resolvedUnknownIds` from engine | Post-each turn | Current heuristic conflates structural resolution with questioning |
| InvestigationSummaryPanel | Still working on count | `total unknowns - resolved` | Total identified unknowns minus resolved | Finalisation | Should not imply 1 unknown = 1 question |
| ReasoningWorkspace | Active question (next useful) | `selectedQuestion.question` from start/update API | Same — but engine should guarantee a question exists when `status === "investigating"` | Question selection phase | If no question is available, engine should emit `evidence_limit_reached` instead |
| ScenarioForm | Selected question reason / "why this matters" | `selectedQuestion.reason` from fixture | Same — but guaranteed on every turn | Question selection | Already partially wired; just needs consistent coverage |
| ReasoningWorkspace | Question reasoning pattern metadata | `selectedQuestion.reasoningPattern` | Engine should emit the pattern class for UI display (e.g. "comparability_check") | Question selection | Used in Developer details; could also inform UI tooltips |
## Graph & Evidence
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| SituationGraphView | Node confidence values | Mock `confidence` ("low"/"medium"/"high") | Computed confidence per node from evidence weight | Graph construction | UI displays as indicators; needs numeric or ordinal source |
| SituationGraphView | Confidence assessment breakdown | `confidenceAssessment.evidenceConfidence`, `completenessStatus`, `conclusionConfidence` | Structured confidence assessment with sub-scores | Evidence analysis | Currently flat mock object |
| DeveloperDetails | Active unknown node ID | `graph.activeUnknownNodeId` from fixture | Explicit active target for next investigation step | Question selection | Internal reference; exposed via developer view only |
| ReasoningWorkspace | Newly surfaced unknown nodes | Scenarios provide `proposal.addedNodes` or mocks a static list | Engine emits `newlySurfacedNodeIds` per turn | Each update turn | UI highlights these to show what the investigation discovered |
| ScenarioForm | Node kind discrimination (observation / assumption / conclusion / unknown) | Hardcoded kind values in mock fixtures | Engine classifies each node correctly | Graph construction | Critical for correct display and reasoning traceability |
| ScenarioForm | Edge relationships | Mock `relationship` ("supports", "undermines") | Engine emits relationship type between nodes | Graph construction | Needed for developer view; affects UI if confidence model expands |
## Evidence & Resolution Tracking
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| DeveloperDetails | `evidenceIds` per node | Empty array `[]` in every mock node | List of evidence nodes supporting this node | Graph construction | Needed for traceability in developer view |
| DeveloperDetails | `dependsOn` / `affects` per node | Empty arrays `[]` in mock nodes | Dependency and effect edges | Graph construction | Shows reasoning structure; currently hidden in collapsed developer details |
| InvestigationSummaryPanel | Whether evidence limit has been reached (terminal state) | Infers from `activeUnknownNodeId === null` + unresolved unknowns present | Explicit terminal status flag from engine | Post-evaluation | UI shows "Current evidence limit reached" card |
| ReasoningWorkspace | `resolvedNodeIds` from update | Mocked from scenario fixture; mirrors resolved unknown IDs | Engine emits `resolvedUnknownNodeIds` per turn | Update response | Used to mark answered questions in history |
| ReasoningWorkspace | `affectedNodeIds` from update | Empty array in mock | List of nodes changed by this answer | Update response | Developer view; shows ripple effects |
## Diagnostics & Technical Metadata
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| DiagnosticsView | `promptVersion` | Hardcoded `"v0.4"` in mocks | Actual prompt version used for this turn | Every request | Useful for debugging and rollout tracking |
| DiagnosticsView | `modelName` | Hardcoded `"mock-ollama"` | Actual model identifier | Every request | Needed when multiple models are supported |
| DiagnosticsView | `responseDurationMs` | Zeroed in mocks | Actual response duration | Every request | Shows user how long reasoning took |
| DiagnosticsView | `validationStatus` | Hardcoded `"valid"` | Whether the output passed structured-validation | Post-processing | UI already uses this to decide if graph was parsed |
| DeveloperDetails | `proposal` details (addedNodes, updatedNodes) | Mocked from scenario fixture | Full proposal metadata from reasoning engine | Update response | Shows what changed and why |
## Recovery & Error States
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| ReasoningWorkspace (ProviderUnavailableCard) | Detect provider/network failure | Regex on `error` string (`provider`, `unavailable`) | Explicit `providerAvailable: false` flag or HTTP status | Request time | Should distinguish transient from permanent failures |
| ReasoningWorkspace (MalformedResponseCard) | Detect unstructured / invalid JSON response | Regex on `error` string (`malformed`, `parse`, `structured`) | Explicit `validationError` object with path details | Post-processing | UI needs to know the validation failure for debugging |
| ReasoningWorkspace (UnexpectedStateCard) | Detect internal engine error | `stage === "unexpected"` from mock | Engine-specific error code + recoverable flag | Any stage | Should distinguish recoverable vs unrecoverable errors |
## Investigation Lifecycle
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| InvestigationSummaryPanel | `investigationStartedAt` timestamp | Uses `result.updatedAt` (from session storage) | Engine-provided `investigationStartedAt` on start response | Start case | Currently uses last-updated time as fallback; inaccurate |
| InvestigationSummaryPanel | `lastUpdatedAt` timestamp | Session `updatedAt` persisted by UI | Engine-provided timestamp on every update response | Every turn | UI already tracks this via session hook |
| ReasoningWorkspace | Genuine completion detection | Heuristic: all unknowns resolved + no active question | Explicit `genuineCompletion: true` from engine | Post-evaluation | Should distinguish "everything resolved" from "stalled" |
| CompletionCard | Final summary for complete state | `propUnderstanding` or `graph.currentSummary` | Engine-emitted final conclusion when all unknowns are resolved | Finalisation | Distinct from intermediate summaries |
| EvidenceLimitCard | Final summary at evidence limit | Same as above | Engine-emitted terminal summary when no more questions are useful | Finalisation | UI card style differs from CompletionCard |
## Scenario & Central Statement
| Feature | UI need | Temporary mock | Desired reasoning output | Likely stage | Notes |
| ---------------------- | ---------------------------------- | --------------------------------------------------- | -------------------------------------------------------------- | ---------------------- | -------------------------------------------------------- |
| OriginalSituation | Central statement display | `scenario` prop (user input) or `graph.centralStatement` | Engine-derived central statement from user input | Start case | UI already handles both; engine should normalise |
| DeveloperDetails | Node descriptions | Mock nodes have `label === description` | Distinct, detailed description per node | Graph construction | Current mock uses label as description; separate fields needed |
## Open Questions / Future Work
1. **Structured confidence scores**: The UI currently mocks ordinal confidence (low/medium/high). The reasoning engine should eventually emit numeric confidence values per node and a computed conclusion confidence, enabling richer visual indicators.
2. **Evidence provenance**: Nodes mock empty `evidenceIds`. The engine should emit which observation nodes support each assumption/conclusion, enabling the developer view to show full evidence chains.
3. **Turn-level diagnostics**: Currently only basic validation metadata is mocked. Full turn diagnostics (prompt used, model, duration, temperature, validation results) would help debugging and monitoring.
4. **Terminal state semantics**: The UI distinguishes "resolution_achieved" from "evidence_limit_reached" using heuristics. The engine should emit explicit terminal states so the UI can show the appropriate card without inference.
5. **Session integrity**: The session persistence hook (Phase 3) stores `situationGraph` + `selectedQuestion` + `summary`. If the engine later emits additional fields that affect the UI (e.g., `investigationStartedAt`, `genuineCompletion`), the persisted payload should expand to include them.
6. **Recovery action granularity**: The recovery cards currently offer a single "restart investigation" action. Future engine contracts could support partial recovery (e.g., retry with different parameters, switch models) rather than full restart.
7. **Investigation duration tracking**: The summary panel computes elapsed time from `Date.now() - updatedAt`. If the engine emits proper timestamps, the UI can show accurate elapsed duration and investigate stalls (>5 min between turns).