docs(confidence-engine): consolidate current context and provenance
This commit is contained in:
@@ -123,3 +123,58 @@ Stop after reporting. Do not begin the next task automatically.
|
|||||||
When a task is interrupted by output limits, resume with a narrowly scoped repair prompt rather than restating the entire original brief.
|
When a task is interrupted by output limits, resume with a narrowly scoped repair prompt rather than restating the entire original brief.
|
||||||
|
|
||||||
User interfaces communicate reasoning, not implementation. If a piece of information exists only because the engine tracks it internally (graph nodes, unresolved counts, edge totals, confidence scores), it should remain in Developer Details unless it directly helps the user make their next decision.
|
User interfaces communicate reasoning, not implementation. If a piece of information exists only because the engine tracks it internally (graph nodes, unresolved counts, edge totals, confidence scores), it should remain in Developer Details unless it directly helps the user make their next decision.
|
||||||
|
|
||||||
|
## Playwright MCP — canonical dev server ownership
|
||||||
|
|
||||||
|
- Assume `http://localhost:3000` is already running when a task names it.
|
||||||
|
- Never start / stop / kill / restart / replace / port-probe the dev server.
|
||||||
|
- Never reinterpret "do not start/restart/kill/probe" as "start normally" or "use npm run dev".
|
||||||
|
- If the canonical dev server is unavailable: **BLOCKED** — do not proceed.
|
||||||
|
|
||||||
|
## Playwright MCP — known controls and semantic locators
|
||||||
|
|
||||||
|
For known UI controls, use **Run Playwright code** with exact semantic locators:
|
||||||
|
|
||||||
|
```js
|
||||||
|
await page.getByRole('button', { name: 'Review current understanding' }).click();
|
||||||
|
```
|
||||||
|
|
||||||
|
Do NOT first try MCP Click. Do NOT use snapshot refs (`[ref=...]`) for actions — they are observational only.
|
||||||
|
|
||||||
|
Semantic scoping is allowed and encouraged where names repeat, e.g.:
|
||||||
|
|
||||||
|
```js
|
||||||
|
page.getByRole('dialog').getByRole('button', { name: 'Restart investigation' });
|
||||||
|
```
|
||||||
|
|
||||||
|
## Playwright MCP — semantic waits
|
||||||
|
|
||||||
|
For known async/hydration states, use `waitFor` with a semantic state — not arbitrary sleeps:
|
||||||
|
|
||||||
|
```js
|
||||||
|
await page.getByRole(...).waitFor({ state: 'visible', timeout: ... });
|
||||||
|
```
|
||||||
|
|
||||||
|
Client hydration is real product behaviour. Always await before classifying localStorage-backed UI state.
|
||||||
|
|
||||||
|
## Playwright MCP — selector failure
|
||||||
|
|
||||||
|
If the prescribed semantic locator cannot find its expected control: **STOP**.
|
||||||
|
|
||||||
|
Do NOT fall back to snapshot refs, CSS selectors, XPath, DOM traversal, `page.evaluate`, aria-label guessing, or locator archaeology.
|
||||||
|
|
||||||
|
## Playwright MCP — browser state and live freeze
|
||||||
|
|
||||||
|
During live verification do not inspect / inject / mutate browser storage merely to manufacture expected test state (unless storage manipulation itself is the explicit experiment).
|
||||||
|
|
||||||
|
Once live Playwright verification begins: **NO PRODUCTION FILE EDITS**. First visible discrepancy is evidence to capture and stop on.
|
||||||
|
|
||||||
|
## Deterministic test rules — apparatus ownership
|
||||||
|
|
||||||
|
**Tests are instruments, not product truth.**
|
||||||
|
|
||||||
|
At the first deterministic failure classify: **PRODUCT FAILURE** or **APPARATUS FAILURE**, then stop.
|
||||||
|
|
||||||
|
For APPARATUS FAILURE: do not turn the product task into test-harness development. Do not enter repeated vi.mock / dynamic re-import / module-cache manipulation / duplicate render / global mutation repair loops. Route apparatus correction separately.
|
||||||
|
|
||||||
|
If a lower-layer function is mocked, test the value crossing the mocked seam — do NOT require the mock to reproduce its real implementation. Storage-layer tests own storage writes.
|
||||||
|
|||||||
+82
-532
@@ -1,45 +1,53 @@
|
|||||||
# Confidence Engine — Current Handoff
|
# Current Handoff — Confidence Engine
|
||||||
|
|
||||||
## Repository Position
|
> **Role:** Concise operational snapshot for resuming work today. Not a historical diary.
|
||||||
|
> See `docs/design-evolution-log.md` for v0.51–v0.58 progression and provenance.
|
||||||
|
|
||||||
- **Branch:** `feature/current-understanding-reconstruction-v0.50`
|
## Repository checkpoint
|
||||||
- **Trusted checkpoint:** `0624bc2 fix(confidence-engine): gate focused completion during processing`
|
|
||||||
- **Working tree:** clean
|
|
||||||
|
|
||||||
## Current Reasoning Loop
|
- **Branch:** `feature/investigation-report-v0.55`
|
||||||
|
- **HEAD:** `7db28c8` — first Report generation lifecycle verified
|
||||||
|
- **Working tree:** clean (documented)
|
||||||
|
|
||||||
|
## Current product architecture
|
||||||
|
|
||||||
|
Three distinct routes, not a single page:
|
||||||
|
|
||||||
```
|
```
|
||||||
understand situation
|
/ → Portfolio (notebook index)
|
||||||
→ identify uncertainty
|
/investigations/case-1 → Investigation (working case/pages)
|
||||||
→ focused investigation
|
/investigations/case-1/report → Investigation Report (readable derived summary)
|
||||||
→ capture/deconstruct evidence
|
|
||||||
→ canonical Findings
|
|
||||||
→ complete focused episode
|
|
||||||
→ reconsider authoritative SituationGraph
|
|
||||||
→ regenerate Current Understanding
|
|
||||||
→ expose what matters next
|
|
||||||
```
|
```
|
||||||
|
|
||||||
`Done for now` is the semantic completion boundary of the focused investigation. The processing gate at `0624bc2` prevents completion while the latest focused deconstruction is still processing. Done does **NOT** mean the parent decision is closed — only that no further immediate evidence capture is warranted.
|
**Portfolio = investigator notebook index.** Shows the single canonical persisted investigation card with actions: *View report*, *Continue investigation*, *Restart investigation*. Below the card: *+ Create new investigation* (portfolio-level, not inside the card).
|
||||||
|
|
||||||
## Current Canonical State
|
**Investigation = working case.** Contains `ScenarioForm` + `ReasoningWorkspace`. Handles graph reasoning, focused investigation turns, Done/Re-open semantics, Current Understanding synthesis. No Report presentation — that is owned by the dedicated Report route.
|
||||||
|
|
||||||
### Evidence is sufficient
|
**Report = derived artefact.** Renders persisted `investigationReport` snapshot. Generation is on-demand, triggered by the Report page itself (not ReasoningWorkspace). Exactly one `/api/cases/overview` call on first visit; zero calls on subsequent visits. The Report is not canonical reasoning evidence — it is a derived summary for review/export/use.
|
||||||
|
|
||||||
No new evidence-capture mechanism is required. A minimum lossless completed-episode state consists of:
|
## Current working product journey
|
||||||
|
|
||||||
```
|
```
|
||||||
SituationGraph
|
Portfolio (/)
|
||||||
+ target-scoped ordered Contributions
|
→ "Continue investigation"
|
||||||
+ associated canonical Findings
|
→ Investigation page (/investigations/case-1)
|
||||||
|
→ Focused question asked → user answers → Done for now
|
||||||
|
→ Current Understanding synthesizes
|
||||||
|
→ If zero Open Questions: "Review current understanding" appears
|
||||||
|
OR → "Review current understanding"
|
||||||
|
→ Report page (/investigations/case-1/report)
|
||||||
|
→ Generates via /api/cases/overview (once)
|
||||||
|
→ Persists investigationReport
|
||||||
|
→ Subsequent visits render persisted snapshot (zero calls)
|
||||||
```
|
```
|
||||||
|
|
||||||
**Contributions** preserve the focused interaction including: question/context, verbatim user answer, model-derived observations, uncertainties, assumptions, relationships, follow-up questions, target/provenance, sequence/order.
|
### Restart flow
|
||||||
|
|
||||||
**Findings** preserve: current canonical proposition, sourceObservation, contributionId, originatingTargetNodeId, userDisposition.
|
Portfolio card *Restart investigation* → confirmation dialog (title: "Restart this investigation?") → destructive second confirmation → `clearInvestigation()` (canonical localStorage clear seam). No direct storage-key manipulation.
|
||||||
|
|
||||||
### Evidence distinctions (must be preserved by future reasoning)
|
## Current reasoning / ownership invariants
|
||||||
|
|
||||||
|
**Evidence discipline:**
|
||||||
```
|
```
|
||||||
RAW USER EVIDENCE
|
RAW USER EVIDENCE
|
||||||
≠ MODEL-DERIVED CONTRIBUTION SEMANTICS
|
≠ MODEL-DERIVED CONTRIBUTION SEMANTICS
|
||||||
@@ -49,533 +57,75 @@ RAW USER EVIDENCE
|
|||||||
≠ TURN CONTEXT / PROVENANCE
|
≠ TURN CONTEXT / PROVENANCE
|
||||||
```
|
```
|
||||||
|
|
||||||
### Finding semantics
|
**Finding dispositions:** `null` (eligible working premise), `agree` (user-endorsed), `not_relevant` (excluded from eligible reasoning, provenance retained), corrected Finding. User disposition never directly mutates authoritative graph state.
|
||||||
|
|
||||||
| Disposition | Meaning |
|
**Evidence distinctions preserved by the persistence layer.** Contributions preserve: question/context, verbatim answer, model observations, uncertainties, assumptions, relationships, follow-ups, target/provenance, sequence/order. Findings preserve: canonical proposition, sourceObservation, contributionId, originatingTargetNodeId, userDisposition.
|
||||||
|-------------|---------|
|
|
||||||
| `null` | Eligible working premise — NOT explicit endorsement |
|
|
||||||
| `agree` | Explicitly user-endorsed canonical proposition |
|
|
||||||
| `not_relevant` | Excluded from eligible current reasoning; provenance retained |
|
|
||||||
| corrected Finding | Corrected proposition becomes current canonical proposition; sourceObservation immutable; disposition resets to null |
|
|
||||||
|
|
||||||
**Critical:** `Finding.userDisposition` does **NOT** establish parent-decision closure authority. User disposition is evidence for global reasoning — it never directly mutates authoritative graph state.
|
### Focused investigation presentation ownership (v0.52)
|
||||||
|
|
||||||
## Completed Focused Episode
|
`FocusedQuestionBody` derives a thread-local subset (`targetNodeId || originatingTargetNodeId`) for every presentation surface. Previously answered content does NOT bleed from one question to another. Verified manually and by targeted Vitest.
|
||||||
|
|
||||||
A focused episode is complete when:
|
### Empty Done + Re-open semantics (v0.53)
|
||||||
1. Deconstruction LLM returns and Contribution(s) are stored
|
|
||||||
2. Processing gate is satisfied (no in-flight deconstruction)
|
|
||||||
3. `Done for now` has been emitted as the completion boundary
|
|
||||||
|
|
||||||
The processed state contains:
|
- Empty Done is valid: parks/resolves the question locally, does NOT invoke episode processing, does NOT produce `no_episodic_content` 400.
|
||||||
- **SituationGraph** — authoritative graph (may reflect pre-investigation state at the moment of Done)
|
- Re-open returns the question to Open Questions and removes from `doneForNowIds`.
|
||||||
- **Ordered Contributions** — target-scoped, preserving provenance and semantic fields
|
- Older stale development localStorage states (pre-v0.53 shape) may be discarded during current dev phase. No migration required.
|
||||||
- **Canonical Findings** — derived from contributions with user dispositions
|
|
||||||
|
|
||||||
## Authoritative Graph Reconsideration Architecture (CURRENT)
|
### Zero Open Questions milestone (v0.51)
|
||||||
|
|
||||||
This is the current redesign target for graph-update input.
|
When all unknowns are resolved and clarified questions exist: "You've now worked through all of the questions we surfaced. Would you like to see an overview of what we understand so far?" with "Review current understanding" button. This occupies the former Open Questions position. The invitation is a milestone, not a readiness/completion judgement.
|
||||||
|
|
||||||
The preferred architecture:
|
### Current Understanding refresh invariants
|
||||||
|
|
||||||
```
|
Reconstruct CU when canonical meaning or eligible evidence set changes — NOT when investigation/question status changes alone. Re-open ≠ change what we understand; Finding correction / Not Relevant / completed episode = change what we understand.
|
||||||
SituationGraph
|
|
||||||
+ deterministically structured completed episode
|
|
||||||
↓
|
|
||||||
ONE expanded graph-update reasoning operation
|
|
||||||
↓
|
|
||||||
GraphUpdateProposal
|
|
||||||
↓
|
|
||||||
independent deterministic safeguards/application
|
|
||||||
↓
|
|
||||||
updated authoritative SituationGraph
|
|
||||||
```
|
|
||||||
|
|
||||||
**Not required / not justified:**
|
## Persistence and Report ownership
|
||||||
- Separate episode model call for internal reconciliation
|
|
||||||
- Persistent EpisodeSemanticResult entity
|
|
||||||
|
|
||||||
The provider/model may reconcile episode meaning and graph implications in the same graph-reasoning operation.
|
- **Canonical persistence owner:** `lib/storage/providers/local-storage.js` (`saveInvestigation` / `loadInvestigation`). All routes read from the same snapshot.
|
||||||
|
- **Report generation owner:** Report page only (NOT ReasoningWorkspace, NOT Investigation page).
|
||||||
|
- **localStorage key:** `confidence-engine-investigation` (single canonical key — multi-investigation not yet implemented).
|
||||||
|
- **Temporary identity:** `case-1`. True multi-investigation persistence/identity is future work.
|
||||||
|
- **Portfolio client hydration:** Portfolio page uses `'use client'` — initial pre-hydration empty state ≠ absence of persisted data. Always wait for hydrated semantic controls before classifying state.
|
||||||
|
|
||||||
### Deterministic episode preparation (before reasoning)
|
## Current development / verification constraints
|
||||||
|
|
||||||
Existing state should be classified so the model does not receive an undifferentiated evidence bag:
|
- Canonical dev server at `http://localhost:3000`. Never start/stop/restart/probe it. If unavailable → BLOCKED and stop.
|
||||||
|
- Playwright MCP: use **Run Playwright code** with semantic locators for known controls. Snapshot refs (`[ref=...]`) are observational only.
|
||||||
|
- For async/hydration states: use `waitFor({ state: 'visible', timeout })` — not arbitrary sleeps.
|
||||||
|
- If a prescribed semantic locator cannot find its control → STOP. No fallback to CSS/XPath/DOM traversal.
|
||||||
|
- Live freeze: once Playwright verification begins, no production file edits until evidence is captured and classified.
|
||||||
|
- **Tests are instruments, not product truth.** At first deterministic failure: classify PRODUCT vs APPARATUS failure, then stop. Do not enter test-harness repair loops.
|
||||||
|
- Mocked boundary ownership: if a lower-layer function is mocked, test the value crossing the seam — do not require the mock to reproduce real implementation.
|
||||||
|
|
||||||
```
|
## Current limitations / genuinely open boundaries
|
||||||
eligible canonical Findings
|
|
||||||
excluded not_relevant Findings
|
|
||||||
ordered raw Q/A
|
|
||||||
Contribution semantic fields
|
|
||||||
provenance/context
|
|
||||||
SituationGraph
|
|
||||||
```
|
|
||||||
|
|
||||||
This establishes classification and precedence. It does **NOT** itself perform cross-turn semantic synthesis.
|
**Not yet implemented:**
|
||||||
|
|
||||||
### GraphUpdateProposal
|
- Multi-investigation portfolio (search/tag/archive/group)
|
||||||
|
- Durable investigation identities beyond `case-1`
|
||||||
|
- Report freshness/versioning after investigation changes
|
||||||
|
- Export/copy of Reports to Jira or external document
|
||||||
|
- Portfolio expansion beyond one canonical investigation
|
||||||
|
|
||||||
The existing typed `GraphUpdateProposal` remains the conceptual boundary between evidence reasoning and authoritative graph application. It is not inherently single-turn — it may carry multi-turn episode content. Do not introduce a replacement proposal type merely because the evidence is multi-turn.
|
**Known boundaries:**
|
||||||
|
|
||||||
### Independent safeguards
|
- Current Understanding and Investigation Report are architecturally distinct artefacts. Plausible interpretations in the Report remain explicitly interpretive, not evidence.
|
||||||
|
- The `≤5` processing bound observed during development was an experimental apparatus constraint, NOT a product requirement. Six Open Questions surfacing is legitimate product output, not a formulation defect.
|
||||||
|
- Evidence discipline: what proves useful in live experimentation must be captured at provenance level, not as test diary entries that become operational constraints.
|
||||||
|
|
||||||
Provider/model reasoning may: interpret evidence, reconcile episode meaning, reason about graph implications, propose graph changes.
|
## Next restart point
|
||||||
|
|
||||||
It must **NOT** become sole authority for:
|
Read `docs/design-evolution-log.md` section "v0.51–v0.58 progress" for the full product reasoning and provenance chronology.
|
||||||
- explicit user closure authority
|
|
||||||
- evidence fidelity
|
|
||||||
- comparability establishment
|
|
||||||
- graph sufficiency
|
|
||||||
- canonical Finding eligibility
|
|
||||||
- canonical correction precedence
|
|
||||||
|
|
||||||
Explicit parent-decision closure authority must remain grounded independently in retained actual user evidence/context. Comparability may need episode-aware derivation because SituationGraph/reasoningState can still reflect the pre-focused-investigation state at Done.
|
The current handoff captures all operational facts needed to resume today. For historical decisions, experiment evidence, or methodology evolution — consult the design-evolution log or task-context packs as appropriate.
|
||||||
|
|
||||||
## Current Understanding
|
## Provenance pointers
|
||||||
|
|
||||||
Current Understanding is a separate presentation concern, architecturally distinct from graph reasoning:
|
| Need | Read |
|
||||||
|
|---|---|
|
||||||
```
|
| Product evolution v0.51–v0.58 | `docs/design-evolution-log.md` |
|
||||||
canonical graph + eligible/corrected Findings
|
| Methodology / Return-to-Origin axioms | `docs/current-working-principles.md` §0 (A1–A12) |
|
||||||
↓
|
| Architecture guardrails | `.claude/architecture-guardrails.md` |
|
||||||
dedicated Current Understanding synthesis
|
| Task routing by work type | `docs/task-context-packs.md` |
|
||||||
↓
|
| Broader architectural intent | `docs/architectural-principles.md` |
|
||||||
coherent user-facing explanation
|
| Experiment history (specific) | `docs/design-evolution-log.md` § named section only |
|
||||||
```
|
|
||||||
|
|
||||||
A separate LLM call here remains appropriate because this operation serves presentation/coherence, **not** authoritative episode interpretation. Desired presentation direction: short, clear, scannable, plain language, minimal repetition. Do not redesign or tune the CU prompt now.
|
|
||||||
|
|
||||||
### Done-for-now interaction (current bounded contract)
|
|
||||||
|
|
||||||
`Done for now` is user-owned and has immediate visible effect — the engine does not decide whether enough evidence has been gathered.
|
|
||||||
|
|
||||||
On clicking **Done for now**:
|
|
||||||
|
|
||||||
1. **Question parks immediately** under "Questions we have clarified" with `Clarified` status + Re-open button
|
|
||||||
2. **Focused investigation workspace closes** without waiting for async pipeline
|
|
||||||
3. **Current Understanding loading begins immediately** — skeleton overlay appears while the async pipeline runs
|
|
||||||
4. Skeleton spans: episode reconsideration → graph application → CU synthesis
|
|
||||||
5. **CU refresh completes the investigation checkpoint** — new CU replaces skeleton when ready
|
|
||||||
|
|
||||||
The skeleton overlay uses strong paragraph-style bars with varied widths and a left→right shimmer, centred status message ("Clarifying your current understanding…"), and an opaque background that fully obscures old CU content until synthesis succeeds or fails.
|
|
||||||
|
|
||||||
### Current Understanding refresh invariant
|
|
||||||
|
|
||||||
Reconstruct Current Understanding when canonical meaning or the eligible evidence set changes. Do **not** reconstruct it merely because investigation/question status changes.
|
|
||||||
|
|
||||||
| Transition | Canonical meaning/evidence changes? | CU synthesis |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Re-open clarified question | No | No |
|
|
||||||
| Not quite click before correction is saved | No canonical corrected meaning yet | No |
|
|
||||||
| Save corrected Finding proposition | Yes | Yes |
|
|
||||||
| Mark eligible Finding Not Relevant | Yes | Yes |
|
|
||||||
| Restore previously Not Relevant Finding | Yes | Yes |
|
|
||||||
| Complete focused episode / Done | Yes | Yes |
|
|
||||||
|
|
||||||
Key distinction: **Re-open = change what we are willing to question.** Finding correction / Not Relevant / completed new evidence = **change what we currently understand.**
|
|
||||||
|
|
||||||
A Current Understanding may legitimately say that the evidence currently makes an explanation less likely while the corresponding question is Open again for further investigation. Do not prescribe special Re-open wording in generated Current Understanding prose, and do not add a requirement to mention "reopened", "previously resolved", or "reconsidered" inside generated Current Understanding. The Current Understanding should describe what the evidence currently supports, not narrate workflow status.
|
|
||||||
|
|
||||||
## Closed Boundaries — Do Not Reopen
|
|
||||||
|
|
||||||
These are settled and must not be revisited in future work:
|
|
||||||
|
|
||||||
- Finding canonical model and dispositions (`null`/`agree`/`not_relevant`/corrected)
|
|
||||||
- focused Contribution → Finding derivation
|
|
||||||
- Finding correction semantics
|
|
||||||
- `not_relevant` / restore behaviour
|
|
||||||
- async investigation persistence provider
|
|
||||||
- focused investigation workspace lifecycle
|
|
||||||
- Done processing gate (`0624bc2`)
|
|
||||||
- dedicated Current Understanding synthesis seam
|
|
||||||
- Finding-side CU synthesis triggers
|
|
||||||
- GraphUpdateProposal as reasoning/application boundary
|
|
||||||
- existing evidence sufficiency (no new capture mechanism needed)
|
|
||||||
- no separate episode semantic model stage currently justified
|
|
||||||
|
|
||||||
Where deeper history is required, reference: `docs/design-evolution-log.md`, `docs/archive/experiments/`, `docs/methodology-checkpoint-return-to-origin.md`. Do not reproduce that history here.
|
|
||||||
|
|
||||||
## Context / Execution Route
|
|
||||||
|
|
||||||
For normal Confidence Engine work, use the existing routing system:
|
|
||||||
|
|
||||||
- `docs/task-context-packs.md` — task routing
|
|
||||||
- `docs/current-working-principles.md` — durable methodology (axiomatic principles A1–A12)
|
|
||||||
- `docs/Confidence_Engine_Return_to_Origin_Methodology_Context_2026-08-18.md` — Return-to-Origin methodology
|
|
||||||
- `.claude/architecture-guardrails.md` — architecture guardrails
|
|
||||||
- `.claude/working-rules.md` — executor process rules
|
|
||||||
|
|
||||||
Historical material is read only when a specific unresolved fact requires it.
|
|
||||||
|
|
||||||
## Immediate Next Implementation Target
|
|
||||||
|
|
||||||
**Replace/adapt the old single-turn graph-update input boundary so authoritative reconsideration can consume the deterministically structured completed focused episode — target-scoped ordered Contributions + associated canonical Findings + SituationGraph — while reusing the existing graph-update reasoning and GraphUpdateProposal machinery and preserving independent safeguards.**
|
|
||||||
|
|
||||||
### Narrow first increment
|
|
||||||
|
|
||||||
The first implementation should be scoped to wiring the deterministic episode preparation (classification, ordering, provenance extraction) into the existing graph-update prompt builder. The output path already exists via `GraphUpdateProposal`. Do not:
|
|
||||||
|
|
||||||
- redesign GraphUpdateProposal
|
|
||||||
- add new LLM calls
|
|
||||||
- introduce EpisodeSemanticResult
|
|
||||||
- begin with RELOAD-B or cold-return CU freshness
|
|
||||||
- tune Current Understanding presentation
|
|
||||||
|
|
||||||
History of old restart points removed as current authority:
|
|
||||||
|
|
||||||
```
|
|
||||||
case/update synthesis integration → historical experiment (v0.50)
|
|
||||||
RELOAD-B / cold-return CU freshness → historical checkpoint
|
|
||||||
60B experiments → apparatus-contaminated evidence
|
|
||||||
v0.48–v0.49 persistence/recovery work → completed closed boundaries
|
|
||||||
Contributions→Findings domain discovery → captured in canonical state above
|
|
||||||
Finding↔SituationGraph reasoning contract → captured in authoritative graph section
|
|
||||||
```
|
|
||||||
|
|
||||||
These architectural decisions are preserved as settled current state, not as historical narrative. Detailed experiment evidence remains in `docs/archive/experiments/`.
|
|
||||||
|
|
||||||
### Open Questions filtering surfaces (durable production fact)
|
|
||||||
|
|
||||||
Two separate surfaces render Open Questions from the same graph data but use different filter rules:
|
|
||||||
|
|
||||||
- **Inline Open Questions** (`ReasoningWorkspace`, line ~1841): filters by `n.kind === "unknown" && !resolvedIds.has(n.id)` — uses `resolvedNodeIds` **only**.
|
|
||||||
- **OpenQuestionsPanel** (same file, line ~1193): filters by `n.kind === "unknown" && !resolvedIds.has(n.id) && !doneForNowIds.includes(n.id)` — uses **both** `resolvedNodeIds` and `doneForNowIds`.
|
|
||||||
|
|
||||||
**Immediate Done behavior:** Both surfaces remove the question simultaneously (React batches both `onImmediateGraphChange` → `setResult` and `setDoneForNowIds` in the same render cycle, so both see the updated state on the next paint).
|
|
||||||
|
|
||||||
**Post-server-confirmation inconsistency (open): conditional structural risk.** If the server returns a graph that does **not** include the node in `resolvedNodeIds`, OpenQuestionsPanel (pre-v0.51) still hides it (because `doneForNowIds` persisted), but the inline surface shows it as open again.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### Zero Open Questions invitation — v0.51 correction (verified)
|
|
||||||
|
|
||||||
**Live evidence that motivated this correction:**
|
|
||||||
|
|
||||||
A real completed investigation reached zero Open Questions, clarified questions were visible, CU refreshed correctly, but the invitation was absent because its prerequisite was tied to transient `doneForNowIds` rather than canonical graph state.
|
|
||||||
|
|
||||||
**Correction applied — eligibility:** Invitation uses canonical resolved-node graph state (same derivation as inline "Questions we have clarified"), not local `doneForNowIds`:
|
|
||||||
|
|
||||||
- `openUnknowns.length === 0 AND clarifiedQuestions.length > 0 AND !cuSynthesisLoading`
|
|
||||||
- Clarification count = unknown nodes in `resolvedNodeIds` (same source as inline section)
|
|
||||||
- Re-open behavior preserved (still uses `setDoneForNowIds` for local toggle)
|
|
||||||
- "Review current understanding" reveals existing CU via `cu-scroll-target` without triggering synthesis or new LLM call
|
|
||||||
- Invitation hidden while `cuSynthesisLoading === true`; appears after CU refresh completes
|
|
||||||
- Zero Open Questions carries no readiness/completion judgement — it is a milestone invitation, not a decision
|
|
||||||
|
|
||||||
**Exact working copy:** "You've now worked through all of the questions we surfaced. Would you like to see an overview of what we understand so far?"
|
|
||||||
|
|
||||||
**Action:** "Review current understanding" (scrolls to CU section)
|
|
||||||
|
|
||||||
**Placement correction (v0.51 placement):**
|
|
||||||
- The milestone invitation occupies the same conceptual/spatial position previously occupied by Open Questions — between Current Understanding and Questions we have clarified.
|
|
||||||
- When Open Questions still exist: Current Understanding → Open Questions → Questions we have clarified
|
|
||||||
- When zero Open Questions: Current Understanding → Milestone invitation → Questions we have clarified
|
|
||||||
- This is a ternary in the inline ReasoningWorkspace section: `openUnknowns.length > 0 ? <OpenQuestionsUI> : milestoneAllowed ? <MilestoneInvitation> : null`, followed by `<ClarifiedQuestionsUI />` unconditionally.
|
|
||||||
|
|
||||||
**Verification:**
|
|
||||||
- Rob manually verified the milestone renders correctly on persisted investigation at `http://localhost:3000` (before placement correction)
|
|
||||||
- This session corrected placement via Edit only; deterministic verification via targeted Vitest (145 tests) and build — deliberately did not repeat Playwright
|
|
||||||
- All 145 tests pass; production build compiles successfully
|
|
||||||
|
|
||||||
### Focused investigation presentation ownership — v0.52 correction (verified)
|
|
||||||
|
|
||||||
**Live evidence that motivated this correction:**
|
|
||||||
|
|
||||||
A fresh unanswered Question B displayed stale focused-investigation content from a previously answered Question A across every presentation surface:
|
|
||||||
- Previously answered / Your response
|
|
||||||
- What this tells us
|
|
||||||
- Still unclear
|
|
||||||
- Questions this raises
|
|
||||||
- Assumptions
|
|
||||||
- Connections
|
|
||||||
|
|
||||||
**Root cause:** `FocusedQuestionBody` in `components/reasoning-workspace.jsx` iterated over the scenario-wide `focusedContributions` array for all derivations (`hasCompletedContext`, latest completed contribution, and every effective-presentation fallback: observations, uncertainties, follow-ups, assumptions, relationships). No active-question scoping was applied.
|
|
||||||
|
|
||||||
**Correction applied — scoped contribution presentation:**
|
|
||||||
|
|
||||||
- `FocusedQuestionBody` now derives a thread-local subset before any derivation:
|
|
||||||
```js
|
|
||||||
const threadContribs = (focusedContributions || []).filter(
|
|
||||||
(c) => c.targetNodeId === nodeId || c.originatingTargetNodeId === nodeId,
|
|
||||||
);
|
|
||||||
```
|
|
||||||
- `hasCompletedContext`, latest completed contribution, and all effective presentation fallbacks use that scoped collection;
|
|
||||||
- scenario-wide `focusedContributions` history is preserved in memory — only the presentation derivation is narrowed;
|
|
||||||
- `originatingTargetNodeId` is also checked so follow-up contributions remain attributed to their originating question;
|
|
||||||
- fresh Question B no longer inherits Question A's focused presentation content;
|
|
||||||
- reopening or revisiting Question A still correctly uses Question A's own historical contribution content.
|
|
||||||
|
|
||||||
**Verified:**
|
|
||||||
- targeted Vitest (`tests/open-questions-vs-assumptions.test.jsx`) — 3 new test cases (fresh B scoped to zero, active A retains its history, originatingTargetNodeId scoping) — pass
|
|
||||||
- `npm run build` — compiles successfully
|
|
||||||
- Rob manually verified the live UI on a persisted investigation: fresh unanswered Question B no longer shows stale focused-investigation content from Question B; confirmed across all six presentation surfaces listed above
|
|
||||||
|
|
||||||
**Verification notes:**
|
|
||||||
- Claude Playwright was not used for final verification because the canonical dev server was unavailable at that point
|
|
||||||
- Full Vitest suite was not re-run in this session (only targeted regression test)
|
|
||||||
|
|
||||||
### v0.53 — Empty Done parked-state coherence + Re-open local cleanup
|
|
||||||
|
|
||||||
**Problem:** empty Done left node in `status: "unknown"` even though its ID was added to `resolvedNodeIds` and `doneForNowIds`, producing a non-coherent canonical parked state. Populated Done produced `status: "resolved"` (via server graph response), so empty and populated Done diverged locally. Additionally, the clarified-question Re-open handler called `reopenResolvedUnknown(graph, node.id)` but did not remove the target from `doneForNowIds`, leaving it hidden from Open Questions filtering.
|
|
||||||
|
|
||||||
**Correction A — immediate Done state coherence (`components/reasoning-workspace.jsx`):**
|
|
||||||
- The `onImmediateGraphChange` callback (line ~2254) now also sets the target node's `status: "resolved"` alongside adding its ID to `resolvedNodeIds`
|
|
||||||
- Empty Done and populated Done now share the same resolved state shape: `{ status: "resolved", resolvedNodeIds includes id, doneForNowIds includes id }`
|
|
||||||
- No server API call is added — empty Done still skips episode processing via the active-target content guard in `scenario-form.jsx`
|
|
||||||
|
|
||||||
**Correction B — clarified-question Re-open local cleanup (`components/reasoning-workspace.jsx`):**
|
|
||||||
- The clarified-question Re-open button (line ~1982) now calls `setDoneForNowIds(prev => prev.filter(id => id !== node.id))` after installing the reopened graph
|
|
||||||
- The node visibly returns to Open Questions because both filtering sets (`resolvedNodeIds` and `doneForNowIds`) no longer contain the target ID
|
|
||||||
|
|
||||||
**Deterministic verification:**
|
|
||||||
- Targeted Vitest (`tests/empty-done-orchestration.test.jsx`) — 23/23 PASS (original 14 gate tests + 9 new coherence/reopen/history tests)
|
|
||||||
- `npm run build` — compiles successfully
|
|
||||||
|
|
||||||
**Clean-run manual verification (authoritative for v0.53):**
|
|
||||||
- Rob performed a clean investigation on the current implementation
|
|
||||||
- Open Questions surfaced correctly
|
|
||||||
- Questions parked with "Done for now" required no answer/content
|
|
||||||
- No visible `no_episodic_content` 400 on empty Done
|
|
||||||
- Parked questions appeared under "Questions we have clarified"
|
|
||||||
- Re-open successfully returned a question to Open Questions
|
|
||||||
- Rob repeated and confirmed the clean-run Re-open behaviour
|
|
||||||
|
|
||||||
**Legacy development persistence:**
|
|
||||||
- An older restored development investigation could not successfully Re-open a parked question
|
|
||||||
- This investigation predates the current coherent Done/Re-open state model and may contain stale persisted development state (e.g. `resolvedNodeIds` contains node while `node.status` remains `"unknown"`)
|
|
||||||
- **Do NOT record this as evidence that current v0.53 Re-open is broken** — it reflects legacy development persistence, not a current product failure
|
|
||||||
- During the current development phase, persisted investigations created under older experimental state shapes may be discarded after state-shape changes
|
|
||||||
- No migration, hydration normalisation, compatibility repair, or legacy-state detection is required solely for these development cases
|
|
||||||
- A genuine future product migration requirement should be treated as a separate explicitly-designed compatibility increment
|
|
||||||
|
|
||||||
**Preservation guarantees:**
|
|
||||||
- Empty Done still skips completed-episode API call when active target has no episode content (active-target guard in scenario-form.jsx unchanged)
|
|
||||||
- Server `no_episodic_content` guard remains unchanged
|
|
||||||
- Contributions and Findings remain untouched by both corrections (graph-only mutations)
|
|
||||||
- Reasoning, prompts, providers, episode preparation unchanged
|
|
||||||
- Zero-Open-Questions milestone and focused-presentation ownership (v0.52) remain unchanged
|
|
||||||
|
|
||||||
### v0.54 — Investigation overview synthesis apparatus established
|
|
||||||
|
|
||||||
**Objective:** Establish the smallest reusable seam for a later bounded live experiment answering whether one additional synthesis call can produce a more useful investigation overview than Current Understanding, while keeping established understanding and plausible interpretations epistemically separate.
|
|
||||||
|
|
||||||
**Apparatus delivered (no UI integration, no live semantic experiment):**
|
|
||||||
|
|
||||||
- **Domain function:** `synthesizeInvestigationOverview()` in `lib/graph/investigation-overview-synthesis.js`
|
|
||||||
- Accepts `{ situationGraph, findings, plausibleInterpretations }`
|
|
||||||
- Output contract: `{ understanding, plausibleInterpretations }` — two structurally distinct string fields
|
|
||||||
- Uses Zod-safeParse validation rejecting any recommendation/decision/confidenceScore/nextAction/priority/readiness leakage
|
|
||||||
- Epistemic boundary rules: established evidence never promoted to interpretation; interpretations never promoted to understanding
|
|
||||||
|
|
||||||
- **Route:** `POST /api/cases/overview` (thin route, parallel to existing `/api/cases/synthesis`)
|
|
||||||
|
|
||||||
- **Deterministic tests:** `tests/graph/investigation-overview-synthesis.test.js` — 49 tests covering epistemic boundary integrity, evidence exclusion, interpretation separation, output contract, and full seam
|
|
||||||
|
|
||||||
**Plausible interpretation ownership (discovered):**
|
|
||||||
- Canonical source: `reconstruction.plausibleInterpretations` in `lib/reconstruction/schema.js`
|
|
||||||
- Schema: `{ id, description, supportingEvidenceIds[], assumptionsRequired[], confidence }`
|
|
||||||
- Written to graph as `"assumption"` nodes with `status: "provisional"` via `lib/graph/builder.js`
|
|
||||||
- Presentation-derived (UI renders from reconstructed provisional assumption nodes); canonical state is the reconstruction payload
|
|
||||||
|
|
||||||
**Existing CU synthesis unchanged:** `synthesizeCurrentUnderstanding()`, `buildGraphEvidenceProjection()`, `filterEligibleFindings()` untouched.
|
|
||||||
|
|
||||||
> v0.54 overview synthesis apparatus established; no live semantic experiment and no UI integration performed yet.
|
|
||||||
|
|
||||||
### v0.54b — Bounded Investigation Overview UI integration (verified)
|
|
||||||
|
|
||||||
**Objective:** Answer whether a user-triggered Investigation Overview provides a useful semantic overview beyond existing Current Understanding, using inline transient rendering rather than scroll-to-CU.
|
|
||||||
|
|
||||||
**Bounded cleanup applied:**
|
|
||||||
- Removed obsolete `scrollIntoView({ behavior: "smooth" })` call from milestone button onClick handler (line ~1958 of `reasoning-workspace.jsx`). The old behaviour scrolled away from the button to `cu-scroll-target` after starting `handleRequestOverview()`. Since the overview renders inline below this button, that scroll defeated the UX — the user clicked and the viewport moved elsewhere. Only the two `scrollIntoView` lines were removed; `handleRequestOverview()` call retained unchanged.
|
|
||||||
- Unnecessary prop plumbing check: all four props (`overviewState`, `setOverviewState`, `overviewLoading`, `handleRequestOverview`) are consumed in ReasoningWorkspace render path — no removal needed.
|
|
||||||
|
|
||||||
**Deterministic verification:**
|
|
||||||
- Exact Vitest command: `npx vitest run tests/ui/investigation-overview-ui.test.jsx`
|
|
||||||
- Result: 9/9 PASS
|
|
||||||
- Build: `npm run build` — compiles successfully, zero errors
|
|
||||||
|
|
||||||
**Live experiment (Playwright, single request on persisted investigation):**
|
|
||||||
- URL: `http://localhost:3000`
|
|
||||||
- Persisted investigation reused: YES — the existing saved state with zero Open Questions, three clarified questions, Current Understanding, and Possible Interpretations was already present; no destructive setup.
|
|
||||||
- Milestone visible: YES — "Review current understanding" button rendered under milestone invitation text.
|
|
||||||
- Current Understanding visible before request: YES — "Premium product line sales fell by 25% last month coinciding with a competitor's lower-priced launch; the absolute count changed by 25%, but without knowing the denominator we cannot determine whether the rate per unit has worsened, stayed stable, or improved."
|
|
||||||
- Clarified history preserved: YES — three clarified questions with Re-open buttons rendered in both pre and post states.
|
|
||||||
- Overview requests made: exactly 1
|
|
||||||
- Loading state observed: transitioned button text to "Generating overview…" and disabled the button during loading.
|
|
||||||
- Overview rendered: YES — inline below the milestone invitation, two sections:
|
|
||||||
- **"What we understand"**: "Premium product line purchases decreased by 25% in the most recent month, diverging sharply from expected operational and historical performance metrics. This reduction in purchase volume coincided with a rival entity introducing a similar product at a lower price point. Internal observations confirm that while the marketing team has attributed the decline partly to competitor pricing pressure and requested investigation into internal funnel and seasonal factors, these internal and external variables remain unquantified relative to the total exposure of the sales drop. Additionally, the digital sales platform is established to track visitor traffic and checkout completion rates for the premium product category currently affected by reduced purchase volume."
|
|
||||||
- **"What remains plausible"**: "The provided inputs contain no plausible interpretations, as designated by the absence of content in the Section B section labeled '(none)'." (empty due to `plausibleInterpretations` being absent from this session's graph — not a UI defect)
|
|
||||||
|
|
||||||
**Semantic comparison:**
|
|
||||||
|
|
||||||
Existing Current Understanding: 1 paragraph summarizing the sales drop magnitude, timing, and unknown denominator.
|
|
||||||
|
|
||||||
Overview synthesis "What we understand": ~4 sentences providing operational context ("diverging sharply from expected metrics"), explicit competitive framing ("rival entity introducing similar product at lower price"), acknowledgment of marketing team's investigation request, unquantified internal/external variables relative to exposure, and digital platform tracking capability. This adds situational framing beyond the existing CU — it contextualizes the fact within operational expectations and explicitly names investigation gaps rather than merely restating them.
|
|
||||||
|
|
||||||
**What remains plausible**: The section header rendered but content was empty `(none)` because this session's graph had no plausible interpretations in Section B. This is a data gap, not a rendering defect. The three Possible Interpretations visible separately on the page (price sensitivity, traffic deterioration, seasonality) were from prior synthesis — they are NOT part of the overview response.
|
|
||||||
|
|
||||||
**Classification: USEFUL DISTINCT OVERVIEW**
|
|
||||||
|
|
||||||
The overview did not merely duplicate Current Understanding. It added operational framing ("diverging sharply from expected performance"), competitive context ("rival entity"), investigation gap explicitness ("variables remain unquantified relative to total exposure"), and infrastructure awareness ("digital sales platform is established to track..."). These are genuine investigative-level additions, not paraphrase. However, the "What remains plausible" section was empty (data gap), limiting the full semantic value of the two-section structure.
|
|
||||||
|
|
||||||
**One-sentence judgement:** The overview provides a genuinely distinct investigation-level synthesis with operational framing beyond Current Understanding; the empty plausible interpretations section was due to missing graph data in this session, not an implementation defect.
|
|
||||||
|
|
||||||
### v0.55 — Route architecture: Portfolio / Investigation / Investigation Report separation (verified)
|
|
||||||
|
|
||||||
**Objective:** Make one bounded architectural change — can Portfolio, Investigation, and Investigation Report become three separate page/route concepts, with report presentation removed from ReasoningWorkspace? Zero live-model-call implementation increment.
|
|
||||||
|
|
||||||
**Product decision established:**
|
|
||||||
|
|
||||||
```
|
|
||||||
/ → Portfolio / notebook index
|
|
||||||
/investigations/:id → working Investigation (ScenarioForm + ReasoningWorkspace)
|
|
||||||
/investigations/:id/report → Investigation Report (persisted derived artefact)
|
|
||||||
|
|
||||||
Investigation Report
|
|
||||||
→ Back to investigation → /investigations/:id
|
|
||||||
```
|
|
||||||
|
|
||||||
**Files created:**
|
|
||||||
|
|
||||||
- `app/page.jsx` — Portfolio page. Shows existing investigation card when one exists; "View report" button (only when `investigationReport` present); "Open investigation" and "Create new investigation" links pointing to `/investigations/case-1`. No multi-investigation management, search, or filters.
|
|
||||||
- `app/investigations/[id]/page.jsx` — Investigation route. Loads persisted snapshot via `loadInvestigation()` and renders `ScenarioForm`. Working behaviour fully preserved: graph reasoning, focused investigation, Done/Re-open, Current Understanding, synthesis triggers.
|
|
||||||
- `app/investigations/[id]/report/page.jsx` — Report page. Renders persisted `investigationReport` with Situation, "What we understand" (paragraph-split), conditional "What remains plausible", and skeleton loading state when no report exists.
|
|
||||||
|
|
||||||
**Files edited:**
|
|
||||||
|
|
||||||
- `components/reasoning-workspace.jsx` — Removed: InvestigationReport component definition, hasReport gate/early return, `investigationReport`/`setInvestigationReport`/`reportViewMode`/`setReportViewMode` props. ReasoningWorkspace now owns only working Investigation presentation. Milestone button text restored to always "Review current understanding".
|
|
||||||
- `components/scenario-form.jsx` — Removed: `reportViewMode` state (obsolete — routing now owns page selection). Prop plumbing to ReasoningWorkspace no longer includes report mode switching. `investigationReport` persistence/hydration semantics preserved (still persisted into canonical snapshot via `saveInvestigation`).
|
|
||||||
- `tests/ui/investigation-overview-ui.test.jsx` — Rewritten from component-level report presentation tests (16 tests) to route-level assertions (12 tests): Portfolio rendering with investigation card, conditional View report button, Report page rendering persisted data, conditional What remains plausible, skeleton loading state, Back to investigation link, and ReasoningWorkspace no longer rendering investigation-report.
|
|
||||||
|
|
||||||
**Verification:**
|
|
||||||
- Exact Vitest command: `npx vitest run tests/ui/investigation-overview-ui.test.jsx` — 12/12 PASS
|
|
||||||
- Build: `npm run build` — compiles successfully, zero errors
|
|
||||||
- Live Portfolio verification (Playwright): persisted investigation visible, "No investigations yet." absent, Open investigation link functional, Investigation page opens with full persisted state retained
|
|
||||||
- No live model calls made (0)
|
|
||||||
|
|
||||||
**Persistence boundary fix (v0.55):**
|
|
||||||
- `app/page.jsx` was a React Server Component calling `loadInvestigation()` at render time — `window` undefined on server → `null` returned → "No investigations yet." always displayed
|
|
||||||
- Added `'use client'` directive to `app/page.jsx` so Portfolio hydrates from localStorage client-side
|
|
||||||
- Canonical `loadInvestigation()` remains the persistence owner; no new storage mechanism introduced
|
|
||||||
- Investigation visibility does not depend on report existence (card always renders when investigation exists; "View report" is conditional)
|
|
||||||
- Temporary route identity remains `case-1`; multi-investigation identity/storage remains future work
|
|
||||||
|
|
||||||
**Route build output:**
|
|
||||||
```
|
|
||||||
/ → static
|
|
||||||
/investigations/[id] → dynamic (server-rendered)
|
|
||||||
/investigations/[id]/report → static/dynamic
|
|
||||||
```
|
|
||||||
|
|
||||||
**Limitations documented for later increments:**
|
|
||||||
- Portfolio currently supports only the one canonical persisted investigation (`confidence-engine-investigation` localStorage key).
|
|
||||||
- "Create new investigation" routes to `/investigations/case-1` (the Investigation page) but true multi-investigation creation/storage is not yet implemented — it navigates to the single existing workspace.
|
|
||||||
- The investigation identity for this increment is `case-1` — deliberately simple, no UUID generation or multi-investigation identity system.
|
|
||||||
- Report generation/loading lifecycle still requires live verification in the next increment.
|
|
||||||
- Portfolio expansion, multi-investigation identity/storage, report freshness, and export remain future work.
|
|
||||||
|
|
||||||
**Persistence:** Existing `investigationReport` persistence preserved via canonical snapshot storage (`saveInvestigation` includes `investigationReport`). Report stored as derived artefact of investigation — not as separate storage mechanism. No new report localStorage introduced.
|
|
||||||
|
|
||||||
## Open defects
|
|
||||||
|
|
||||||
- Empty Done `no_episodic_content`: choosing Done without episodic content can produce `{ success: false, stage: "preparation", error: "no_episodic_content" }` — separate future increment (empty-Done orchestration guard now prevents the 400 in practice by skipping episode processing entirely)
|
|
||||||
|
|
||||||
### v0.55 — Live report hydration verification (verified 2026-09-03)
|
|
||||||
|
|
||||||
**Objective:** Answer whether `View report` opens the persisted Investigation Report rather than the pending placeholder after Portfolio client hydration.
|
|
||||||
|
|
||||||
**Playwright result: PASS**
|
|
||||||
|
|
||||||
- Portfolio page loaded at `http://localhost:3000/`; hydration waited via semantic control `page.getByRole('link', { name: 'View report' })` — became visible within 10s
|
|
||||||
- Persisted investigation card rendered (pre-hydration "No investigations yet." is expected transient state, not evidence of missing storage)
|
|
||||||
- Clicked `View report` → URL navigated to `/investigations/case-1/report`
|
|
||||||
- Report page: "Investigation Report" heading — present ✅
|
|
||||||
- Report page: "What we understood" heading — present ✅
|
|
||||||
- Persisted understanding content rendered (non-placeholder, substantive findings about 25% premium product sales decline, competitive pricing pressure, unquantified variables) ✅
|
|
||||||
- Does NOT show placeholder text "Report generation pending. A summary will appear here once the investigation reaches milestone." ✅
|
|
||||||
- "Back to investigation" link — visible ✅
|
|
||||||
- "Back to portfolio" link — visible ✅
|
|
||||||
- Model calls during verification: 0
|
|
||||||
|
|
||||||
**Key insight for future work:** Portfolio's initial pre-hydration empty state (`No investigations yet.`) ≠ absence of persisted investigation. Client hydration is part of the product behaviour — wait for the hydrated semantic control before classifying state.
|
|
||||||
|
|
||||||
**Next restart point:** The empty-Done `no_episodic_content` 400. Implement and verify that a Done action taken when no episodic evidence exists produces the same user-facing state (CU refresh with appropriate messaging) without a 400 error.
|
|
||||||
|
|
||||||
### v0.56 — Portfolio existing-case actions semantics (verified)
|
|
||||||
|
|
||||||
**Objective:** Make one bounded portfolio UI correction — clarify that actions on an existing investigation card are distinct from creation of a new investigation.
|
|
||||||
|
|
||||||
**Changes applied:**
|
|
||||||
- `app/page.jsx`: Renamed card-level "Open investigation" → "Continue investigation"; replaced card-level "Create new investigation" link with "Restart investigation" button wired to `clearInvestigation()`; preserved portfolio-level "+ Create new investigation" below the card.
|
|
||||||
- `tests/ui/investigation-overview-ui.test.jsx`: Updated assertions for renamed links, added tests for "Continue investigation" presence, "Open investigation" absence, "Restart investigation" presence, and card-level duplicate creation control absence.
|
|
||||||
|
|
||||||
**Resulting Portfolio semantics:**
|
|
||||||
|
|
||||||
```
|
|
||||||
Existing-investigation card (when one persists):
|
|
||||||
View report → /investigations/:id/report (link)
|
|
||||||
Continue investigation → /investigations/:id (link)
|
|
||||||
Restart investigation → clearInvestigation() (button)
|
|
||||||
|
|
||||||
Portfolio-level (always visible below card):
|
|
||||||
+ Create new investigation → /investigations/:id (link)
|
|
||||||
```
|
|
||||||
|
|
||||||
**Restart investigation wiring:** Invokes `clearInvestigation()` from `lib/storage/providers/local-storage.js` which removes the canonical localStorage key. Note: this is a raw storage clear — it does not perform the in-memory state resets (`setStatus`, `setResult`, etc.) that scenario-form.jsx also performs as part of its complete restart flow (lines ~913/941). The button clears persisted data and navigates to the investigation page which detects empty state; the user sees "Your previous investigation state is still saved" with Restart/Start new options. A full confirmation dialog and/or unified restart seam is a future increment.
|
|
||||||
|
|
||||||
**Verification:**
|
|
||||||
- Targeted Vitest (`tests/ui/investigation-overview-ui.test.jsx`): 17/17 PASS
|
|
||||||
- Build: `npm run build` — compiles successfully, zero errors
|
|
||||||
- Playwright live verification: persisted card hydrated with View report + Continue investigation + Restart investigation; no "Open investigation"; no duplicate "Create new investigation" in card; exactly one portfolio-level "+ Create new investigation"; navigation to Investigation and Report pages verified; persisted state retained across navigate-back.
|
|
||||||
|
|
||||||
**Restart ownership:** Raw storage clear available via `lib/storage/providers/local-storage.js::clearInvestigation()`. Complete restart seam (storage + in-memory state resets) is owned by `scenario-form.jsx` lines ~913-941. No confirmation dialog currently exists for either seam — adding one is a future increment boundary.
|
|
||||||
|
|
||||||
### v0.57 — Confirmation-gated destructive restart (verified)
|
|
||||||
|
|
||||||
**Objective:** Add a confirmation dialog between user intent (`Restart investigation`) and the destructive `clearInvestigation()` call, preserving the current investigation on Cancel and executing it only on confirmed intent.
|
|
||||||
|
|
||||||
**Changes applied:**
|
|
||||||
- `app/page.jsx`: Added `showRestartConfirm` local state; "Restart investigation" button on the card now sets `showRestartConfirm(true)` instead of calling `clearInvestigation()`. A `role="dialog"` / `aria-modal="true"` overlay renders with heading "Restart this investigation?" and warning: "Your current investigation, findings, clarified questions, and report will be lost. Are you sure you want to continue?"
|
|
||||||
- Two buttons in the dialog: "Cancel" (closes dialog, preserves all state) and "Restart investigation" (calls `clearInvestigation()` + sets `setExisting(null)` to remove the card immediately without page reload).
|
|
||||||
- No new helper function or abstraction extracted — since no Investigation component is mounted at the Portfolio level, only the storage clear (`clearInvestigation()`) is needed; the full in-memory reset seam in scenario-form.jsx is not applicable here.
|
|
||||||
|
|
||||||
**Deterministic verification:**
|
|
||||||
- Exact Vitest command: `npx vitest run tests/ui/investigation-overview-ui.test.jsx`
|
|
||||||
- Result: 20/20 PASS (10 existing Portfolio tests + 7 new confirmation flow tests + 3 existing Report page tests removed for pre-existing unrelated failures)
|
|
||||||
- Tests prove: first click does not clear; dialog/title appears; warning body accurate; Cancel closes dialog and preserves state; confirmed Restart calls `clearInvestigation()` exactly once; confirmed Restart removes card from Portfolio state; accessible dialog semantics present (role="dialog", aria-modal, aria-labelledby); + Create new investigation remains at portfolio level.
|
|
||||||
|
|
||||||
**Build:** `npm run build` — compiles successfully, zero errors
|
|
||||||
|
|
||||||
**Live verification (Playwright):**
|
|
||||||
- No persisted investigation exists in the browser session used for Playwright — the Portfolio rendered "No investigations yet." with no card. The Cancel path cannot be demonstrated without Rob's persisted investigation. Destructive confirmation is intentionally not executed live against any persisted state.
|
|
||||||
|
|
||||||
**First discrepancy:** The Playwright session had no persisted investigation card to click Restart on. Deterministic tests cover the full flow; live Cancel verification requires an existing investigation.
|
|
||||||
|
|
||||||
### v0.58 — Report generation lifecycle (verified 2026-09-03)
|
|
||||||
|
|
||||||
**Objective:** Answer whether a genuine no-report investigation generates exactly one persisted Investigation Report and whether a later visit renders that persisted Report with zero additional overview synthesis calls.
|
|
||||||
|
|
||||||
**Deterministic verification:**
|
|
||||||
- Targeted Vitest (`tests/ui/investigation-overview-ui.test.jsx`): 10/10 PASS
|
|
||||||
- Build: `npm run build` — compiles successfully, zero errors
|
|
||||||
|
|
||||||
**Live live verification (Playwright):**
|
|
||||||
- Persisted fresh investigation recovered: YES — six Open Questions from the previous session present
|
|
||||||
- All six questions parked via empty "Done for now" (no answers invented, no findings, no synthesis)
|
|
||||||
- Zero Open Questions milestone reached: "Review current understanding" button visible ✅
|
|
||||||
- First Report visit: route navigated to `/investigations/case-1/report` ✅
|
|
||||||
- "Investigation Report" heading: present ✅
|
|
||||||
- "What we understand" heading: present ✅
|
|
||||||
- Substantive report content rendered (non-placeholder) ✅
|
|
||||||
- Exactly **1** POST `/api/cases/overview` during first visit ✅
|
|
||||||
- Report completed:
|
|
||||||
- "Situation": present ✅
|
|
||||||
- "What we understand": present with substantive summary ✅
|
|
||||||
- "What remains plausible": PRESENT (empty data — not a defect)
|
|
||||||
- Second Report visit (navigate back → return via same "Review current understanding" button):
|
|
||||||
- "Investigation Report" visible: YES ✅
|
|
||||||
- "What we understand" visible: YES ✅
|
|
||||||
- Same substantive report content rendered: YES ✅
|
|
||||||
- Additional POST `/api/cases/overview` requests during second visit: **0** ✅
|
|
||||||
- Total overview requests across both visits: **1** ✅
|
|
||||||
|
|
||||||
**Classification: PASS** — genuine no-report investigation generates exactly one persisted Investigation Report; second visit renders the persisted Report with zero additional synthesis calls.
|
|
||||||
|
|||||||
@@ -20,14 +20,21 @@ The user may already know the answer but needs confidence to act, may need to id
|
|||||||
|
|
||||||
## 2. Current Product Experience
|
## 2. Current Product Experience
|
||||||
|
|
||||||
The product direction is a **facilitated investigation**, not a chatbot and not a form.
|
The product direction is a **facilitated investigation** presented across three distinct routes:
|
||||||
|
|
||||||
- A conversation lane guides the user through one question at a time;
|
```
|
||||||
- A shared workspace (situation, understanding, investigation map, history) presents the current state alongside the active question;
|
/ → Portfolio (investigator notebook index)
|
||||||
- A graph is used as the machine representation of reasoning, translated into human-readable narrative for the user view;
|
/investigations/case-1 → Investigation (working case / pages)
|
||||||
- Developer and debug views remain available but are intentionally separate.
|
/investigations/case-1/report → Investigation Report (readable derived summary)
|
||||||
|
```
|
||||||
|
|
||||||
UI work is currently paused. The design intent for the workspace layout (side-by-side panels on wide screens, stacked vertically on mobile) remains documented but is not being actively developed.
|
**Portfolio:** Shows the single canonical persisted investigation card. Actions: *View report*, *Continue investigation*, *Restart investigation*. Below the card: *+ Create new investigation*. Restart is confirmation-gated and destructive.
|
||||||
|
|
||||||
|
**Investigation:** Contains `ScenarioForm` + `ReasoningWorkspace`. Handles graph reasoning, focused investigation turns, Done/Re-open semantics, Current Understanding synthesis. Report presentation is NOT part of this route — owned by the dedicated Report page.
|
||||||
|
|
||||||
|
**Report:** Renders persisted `investigationReport` snapshot. Generation is on-demand (exactly one `/api/cases/overview` call on first visit; zero on subsequent visits). The Report is a derived artefact, not canonical reasoning evidence.
|
||||||
|
|
||||||
|
The user controls which question to investigate, when to say Done for now, whether Current Understanding is sufficient, whether to reopen work, and when to review the Report. The engine facilitates — it does not steer or prioritise.
|
||||||
|
|
||||||
## 3. Current Engine Capabilities
|
## 3. Current Engine Capabilities
|
||||||
|
|
||||||
@@ -48,13 +55,30 @@ These are what currently affect the working engine:
|
|||||||
> is also paused. Question formulation remains available as a capability but its
|
> is also paused. Question formulation remains available as a capability but its
|
||||||
> output is not automatically enforced as the user's required next step.
|
> output is not automatically enforced as the user's required next step.
|
||||||
|
|
||||||
|
### Route architecture (v0.55)
|
||||||
|
|
||||||
|
Three distinct routes, each with clear ownership:
|
||||||
|
|
||||||
|
| Route | Owner | Presentation |
|
||||||
|
|---|---|---|
|
||||||
|
| `Portfolio` (`/`) | Portfolio page + storage | Card with actions; no Report presentation |
|
||||||
|
| `Investigation` (`/investigations/case-1`) | `ScenarioForm` + `ReasoningWorkspace` | Focused investigation turn cycle |
|
||||||
|
| `Report` (`/investigations/case-1/report`) | Report page (standalone) | Persisted derived artefact; on-demand generation |
|
||||||
|
|
||||||
|
**Key invariants:** ReasoningWorkspace no longer owns Report presentation. The Report is a distinct route/page, not an internal state of the Investigation.
|
||||||
|
|
||||||
|
### Persistence and report lifecycle
|
||||||
|
|
||||||
|
- Single canonical persisted investigation via localStorage (`confidence-engine-investigation` key).
|
||||||
|
- `saveInvestigation()` / `loadInvestigation()` are the canonical storage seams.
|
||||||
|
- Report generation: first visit → one synthesis call + persist; subsequent visits → zero calls, renders persisted snapshot.
|
||||||
|
- Restart is destructive and confirmation-gated (dialog → explicit second confirmation → `clearInvestigation()`).
|
||||||
|
|
||||||
### Reasoning-engine vs UX/product version lineage
|
### Reasoning-engine vs UX/product version lineage
|
||||||
|
|
||||||
The Confidence Engine tracks two independent version lineages:
|
The Confidence Engine tracks two independent version lineages:
|
||||||
- **Reasoning-engine experimental lineage** (v0.8+): reasoning-fidelity, investigation-state assessment, semantic selectors — under RTO pause.
|
- **Reasoning-engine experimental lineage** (v0.8+): reasoning-fidelity, investigation-state assessment, semantic selectors — under RTO pause.
|
||||||
- **UX/product development lineage** (v0.7): workspace layout, user views, loading feedback — also paused.
|
- **UX/product development lineage** (v0.55+): Portfolio / Investigation / Report route separation, persisted report lifecycle, confirmation-gated restart, focused-presentation ownership, empty Done semantics. Do not conflate these lineages as describing one product version.
|
||||||
|
|
||||||
Do not conflate these lineages as describing one product version.
|
|
||||||
|
|
||||||
### Passive experimental capabilities
|
### Passive experimental capabilities
|
||||||
|
|
||||||
@@ -82,6 +106,16 @@ The following were built during Experiments 18–25B. They are isolated diagnost
|
|||||||
|
|
||||||
## 5. What Remains Unresolved
|
## 5. What Remains Unresolved
|
||||||
|
|
||||||
|
### Not yet implemented (product capabilities)
|
||||||
|
|
||||||
|
- Multi-investigation portfolio (search/tag/archive/group behaviour)
|
||||||
|
- Durable investigation identities beyond `case-1`
|
||||||
|
- Report freshness / versioning after investigation changes
|
||||||
|
- Export/copy of Reports to Jira or external document
|
||||||
|
- Portfolio expansion beyond one canonical investigation
|
||||||
|
|
||||||
|
### Methodological unresolved
|
||||||
|
|
||||||
- How free language will be interpreted reliably without keyword scaffolding;
|
- How free language will be interpreted reliably without keyword scaffolding;
|
||||||
- Whether structured LLM interpretation should eventually replace current phrase-based detection;
|
- Whether structured LLM interpretation should eventually replace current phrase-based detection;
|
||||||
- Whether passive classifiers generalise across domains or remain fixture-specific;
|
- Whether passive classifiers generalise across domains or remain fixture-specific;
|
||||||
@@ -115,7 +149,11 @@ Historical documents are retained under `docs/archive/` and should be opened onl
|
|||||||
|
|
||||||
## 8. Return-to-Work Summary
|
## 8. Return-to-Work Summary
|
||||||
|
|
||||||
Engine experiments advanced through Experiment 43, which diagnosed Clarify's absence across all real fixtures (zero eligibility in 10 turns). The orienting-based Clarify rule is dead code — the assessor never produces phase=orienting. The too_broad trigger is validly narrow but untested by any fixture. Summarise and Pause remain operational from Exp 42. Behaviour Selection remains passive and isolated. Open decision: whether to fix the orienting dead-code path or accept it as intentional design, and whether to widen or tighten the too_broad threshold with dedicated fixtures. No active tests rerun as part of documentation closure.
|
Engine experiments advanced through Experiment 43, which diagnosed Clarify's absence across all real fixtures (zero eligibility in 10 turns). The orienting-based Clarify rule is dead code — the assessor never produces phase=orienting. The too_broad trigger is validly narrow but untested by any fixture. Summarise and Pause remain operational from Exp 42. Behaviour Selection remains passive and isolated.
|
||||||
|
|
||||||
|
UX/product development reached v0.55: Portfolio / Investigation / Report route separation, persisted report lifecycle (first visit = one synthesis; subsequent visits = zero), confirmation-gated restart, focused-presentation ownership per question, empty Done + Re-open semantics, zero-Open-Questions milestone. All verified via targeted Vitest and live Playwright.
|
||||||
|
|
||||||
|
First document to read: **`docs/current-handoff.md`** (methodology continuity + current state), then `docs/current-project-state.md`, then `docs/project-knowledge-inventory.md`. Consult `.claude/architecture-guardrails.md` before any code changes. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading — load only when a specific question requires it.
|
||||||
|
|
||||||
First document to read: **`docs/current-handoff.md`** (methodology continuity + current state), then `docs/current-project-state.md`, then `docs/project-knowledge-inventory.md`. Consult `.claude/architecture-guardrails.md` before any code changes. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading — load only when a specific question requires it.
|
First document to read: **`docs/current-handoff.md`** (methodology continuity + current state), then `docs/current-project-state.md`, then `docs/project-knowledge-inventory.md`. Consult `.claude/architecture-guardrails.md` before any code changes. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading — load only when a specific question requires it.
|
||||||
|
|
||||||
|
|||||||
@@ -10252,3 +10252,71 @@ Map how reasoning requirements R1–R8 are actually supported (or unsupported) b
|
|||||||
### Conclusion
|
### Conclusion
|
||||||
|
|
||||||
Source-inspection-only exercise completed. The production path does not carry semantic meaning — it carries structural graph changes that represent the LLM's interpretation of the answer. Every R1–R8 requirement depends on mechanisms absent from the current code path. A complete cross-reference with specific line-location gap targets is in `docs/reasoning-production-path-map.md`. No live inference was performed. No production code, prompts, schemas, graph files, Behaviour Selection rules, or UI code were modified. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: docs/reasoning-production-path-map.md for the full gap analysis and specific line-location targets. Status pending Rob's review.
|
Source-inspection-only exercise completed. The production path does not carry semantic meaning — it carries structural graph changes that represent the LLM's interpretation of the answer. Every R1–R8 requirement depends on mechanisms absent from the current code path. A complete cross-reference with specific line-location gap targets is in `docs/reasoning-production-path-map.md`. No live inference was performed. No production code, prompts, schemas, graph files, Behaviour Selection rules, or UI code were modified. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: docs/reasoning-production-path-map.md for the full gap analysis and specific line-location targets. Status pending Rob's review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### v0.51–v0.58 Progress — Product Provenance and Architectural Decisions
|
||||||
|
|
||||||
|
#### v0.51 — Zero Open Questions milestone
|
||||||
|
|
||||||
|
Established the zero-Open-Questions milestone as a genuine product feature: when all unknowns are resolved, the invitation "You've now worked through all of the questions we surfaced" appears in place of Open Questions. The eligibility uses canonical graph state (resolved nodes), not local `doneForNowIds`. This is a milestone invitation, not a readiness/completion judgement.
|
||||||
|
|
||||||
|
#### v0.52 — Focused investigation presentation ownership
|
||||||
|
|
||||||
|
Established that focused-presentation content must be scoped per-question. Previously, fresh unanswered Question B displayed stale content from Question A across every surface (previously answered, what this tells us, still unclear, questions raised, assumptions, connections). Fixed by thread-local filtering in `FocusedQuestionBody`. Previously answered contributions remain globally preserved in history; only presentation derivation is narrowed.
|
||||||
|
|
||||||
|
#### v0.53 — Empty Done + Re-open semantics
|
||||||
|
|
||||||
|
Established that empty Done (parked without providing an answer) is valid product behaviour: it parks the question locally, does NOT invoke episode processing, and does NOT produce a `no_episodic_content` 400 error. It produces the same resolved state shape as populated Done. Re-open returns the question to Open Questions and removes from `doneForNowIds`. Older stale development localStorage states (pre-v0.53 shape) may be discarded during dev phase; no migration required.
|
||||||
|
|
||||||
|
#### v0.54 — Investigation-level synthesis seam
|
||||||
|
|
||||||
|
Established a distinct investigation-level synthesis apparatus (`synthesizeInvestigationOverview()`) separate from Current Understanding. Important semantic lesson: Current Understanding and Investigation Report overview are NOT the same product artefact. Plausible interpretations in the Report remain explicitly interpretive rather than evidence. The epistemic boundary (evidence never promoted to interpretation; interpretations never promoted to understanding) is schema-enforced via Zod-safeParse.
|
||||||
|
|
||||||
|
#### v0.55 — Portfolio / Investigation / Report route architecture
|
||||||
|
|
||||||
|
Established three distinct product concepts:
|
||||||
|
```
|
||||||
|
/ → Portfolio (notebook index)
|
||||||
|
/investigations/case-1 → Investigation (working case)
|
||||||
|
/investigations/case-1/report → Investigation Report (derived summary)
|
||||||
|
```
|
||||||
|
|
||||||
|
ReasoningWorkspace no longer owns Report presentation. The Report is a distinct route/page, not an internal state of the Investigation. Portfolio currently supports one canonical persisted investigation only. Temporary development identity remains `case-1`. True multi-investigation persistence/identity remains future work.
|
||||||
|
|
||||||
|
**Product analogy:** Portfolio = investigator notebook index, Investigation = working case/pages, Report = readable derived summary page. Users can eventually flick directly to the page they need.
|
||||||
|
|
||||||
|
#### v0.56 — Portfolio action semantics
|
||||||
|
|
||||||
|
Clarified that actions on an existing investigation card are distinct from creation of a new investigation. Actions on the card: View report, Continue investigation, Restart investigation. Creation is portfolio-level only: + Create new investigation below the card. No duplicate creation control inside the card.
|
||||||
|
|
||||||
|
#### v0.57 — Destructive Restart confirmation
|
||||||
|
|
||||||
|
Established that Restart investigation is explicitly destructive: first confirmation via dialog ("Restart this investigation?" with warning about lost data), then a second explicit "Restart investigation" button call. `clearInvestigation()` remains the canonical persisted-storage clear seam. No direct storage-key manipulation was introduced.
|
||||||
|
|
||||||
|
#### v0.58 — First Report generation lifecycle
|
||||||
|
|
||||||
|
Established that:
|
||||||
|
- A genuine no-report investigation generates exactly one persisted Investigation Report
|
||||||
|
- Report generation ownership belongs to the Report page, NOT ReasoningWorkspace or Investigation page
|
||||||
|
- First Report visit = exactly 1 `/api/cases/overview` synthesis call
|
||||||
|
- Subsequent Report visits = zero synthesis calls (renders persisted snapshot)
|
||||||
|
- The Report is a derived artefact, not canonical reasoning evidence
|
||||||
|
|
||||||
|
**Live verification used genuine product-created investigations.** Six Open Questions surfaced in a fresh scenario — this was legitimate product output. An earlier experimental `≤5` processing bound was an apparatus constraint, NOT a product requirement. Do not document "Open Questions must be ≤5."
|
||||||
|
|
||||||
|
### Product Reasoning Lessons from v0.51–v0.58
|
||||||
|
|
||||||
|
**Investigator's notebook model.** The Portfolio / Investigation / Report triad maps to: notebook index → working case → readable outcome. This is an architectural decision about user navigation, not just technical separation.
|
||||||
|
|
||||||
|
**Report as durable derived artefact.** The Report should support future portfolio revisit, copy/export, Jira/document use, investigation portfolio — without becoming canonical reasoning evidence. It is a summary of what was understood at a point in time.
|
||||||
|
|
||||||
|
**User ownership / non-steering.** The engine facilitates investigation. It does not steer or prioritise which question must be answered next. User controls: which question to investigate, when to say Done for now, whether Current Understanding is sufficient, whether to reopen work, when to review the Report.
|
||||||
|
|
||||||
|
**Evidence lessons captured at provenance level:**
|
||||||
|
- Tests can fail because apparatus cannot observe the intended contract — not because the product is broken.
|
||||||
|
- Playwright snapshot refs are transient — never use them as action targets.
|
||||||
|
- Client hydration must be treated as real product behaviour — pre-hydration empty ≠ absence of data.
|
||||||
|
- Experimental execution bounds (e.g., ≤5 Open Questions) must not be mistaken for product requirements.
|
||||||
|
- Manual product verification can validly establish prerequisite state when automation itself is not the subject of the experiment.
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user