1400 lines
84 KiB
Markdown
1400 lines
84 KiB
Markdown
# Current Handoff — Confidence Engine
|
||
|
||
> **Role:** Concise operational snapshot for resuming work today. Not a historical diary.
|
||
> The design evolution archive index at `docs/design-evolution/README.md` provides progressive loading of experiment history; load the relevant chapter only when a specific historical question requires it.
|
||
|
||
## Repository checkpoint
|
||
|
||
- **Branch:** `feature/investigation-report-v0.55`
|
||
- **HEAD:** `37a9a12` — route design evolution provenance through archive index
|
||
- **Working tree:** will be clean after v0.59a commit
|
||
|
||
## Current product architecture
|
||
|
||
Three distinct routes, not a single page:
|
||
|
||
```
|
||
/ → Portfolio (notebook index)
|
||
/investigations/case-1 → Investigation (working case/pages)
|
||
/investigations/case-1/report → Investigation Report (readable derived summary)
|
||
```
|
||
|
||
**Portfolio = investigator notebook index.** Shows the single canonical persisted investigation card with actions: *View report*, *Continue investigation*, *Restart investigation*. Below the card: *+ Create new investigation* (portfolio-level, not inside the card).
|
||
|
||
**Investigation = working case.** Contains `ScenarioForm` + `ReasoningWorkspace`. Handles graph reasoning, focused investigation turns, Done/Re-open semantics, Current Understanding synthesis. No Report presentation — that is owned by the dedicated Report route.
|
||
|
||
**Report = derived artefact.** Renders persisted `investigationReport` snapshot. Generation is on-demand, triggered by the Report page itself (not ReasoningWorkspace). Exactly one `/api/cases/overview` call on first visit; zero calls on subsequent visits. The Report is not canonical reasoning evidence — it is a derived summary for review/export/use.
|
||
|
||
## Current working product journey
|
||
|
||
```
|
||
Portfolio (/)
|
||
→ "Continue investigation"
|
||
→ Investigation page (/investigations/case-1)
|
||
→ Focused question asked → user answers → Done for now
|
||
→ Current Understanding synthesizes
|
||
→ If zero Open Questions: "Review current understanding" appears
|
||
OR → "Review current understanding"
|
||
→ Report page (/investigations/case-1/report)
|
||
→ Generates via /api/cases/overview (once)
|
||
→ Persists investigationReport
|
||
→ Subsequent visits render persisted snapshot (zero calls)
|
||
```
|
||
|
||
### Restart flow
|
||
|
||
Portfolio card *Restart investigation* → confirmation dialog (title: "Restart this investigation?") → destructive second confirmation → `clearInvestigation()` (canonical localStorage clear seam). No direct storage-key manipulation.
|
||
|
||
## Current reasoning / ownership invariants
|
||
|
||
**Evidence discipline:**
|
||
```
|
||
RAW USER EVIDENCE
|
||
≠ MODEL-DERIVED CONTRIBUTION SEMANTICS
|
||
≠ CURRENT CANONICAL FINDING
|
||
≠ IMMUTABLE SOURCE OBSERVATION
|
||
≠ USER DISPOSITION / AUTHORITY
|
||
≠ TURN CONTEXT / PROVENANCE
|
||
```
|
||
|
||
**Finding dispositions:** `null` (eligible working premise), `agree` (user-endorsed), `not_relevant` (excluded from eligible reasoning, provenance retained), corrected Finding. User disposition never directly mutates authoritative graph state.
|
||
|
||
**Evidence distinctions preserved by the persistence layer.** Contributions preserve: question/context, verbatim answer, model observations, uncertainties, assumptions, relationships, follow-ups, target/provenance, sequence/order. Findings preserve: canonical proposition, sourceObservation, contributionId, originatingTargetNodeId, userDisposition.
|
||
|
||
### Focused investigation presentation ownership (v0.52)
|
||
|
||
`FocusedQuestionBody` derives a thread-local subset (`targetNodeId || originatingTargetNodeId`) for every presentation surface. Previously answered content does NOT bleed from one question to another. Verified manually and by targeted Vitest.
|
||
|
||
### Empty Done + Re-open semantics (v0.53)
|
||
|
||
- Empty Done is valid: parks/resolves the question locally, does NOT invoke episode processing, does NOT produce `no_episodic_content` 400.
|
||
- Re-open returns the question to Open Questions and removes from `doneForNowIds`.
|
||
- Older stale development localStorage states (pre-v0.53 shape) may be discarded during current dev phase. No migration required.
|
||
|
||
### v0.59a — Investigation revision provenance rule
|
||
|
||
- Investigation has a semantic revision (`investigationRevision`).
|
||
- Meaningful persisted Investigation changes advance it by exactly 1.
|
||
- Persistence activity itself (autosave, save) does NOT advance revision.
|
||
- Report generation records `generatedFromRevision: investigationRevision`.
|
||
- Report generation advances revision by 0.
|
||
- Equal revisions mean Report reflects current Investigation.
|
||
- Different revisions mean the Investigation has changed since Report generation.
|
||
- Existing Report remains available (not invalidated).
|
||
- Report update/regeneration remains manual (user-triggered).
|
||
- Restart clears Investigation + Report via `clearInvestigation()` + `setInvestigationRevision(0)`.
|
||
- Report history/comparison remains deferred beyond MVP.
|
||
|
||
### v0.59b — Report freshness UI on the Report page
|
||
|
||
- Report page shows `Current` (matching revisions) or `Update available` (differing revisions).
|
||
- Manual `Update report` action on Report page triggers regeneration via `/api/cases/overview`.
|
||
- Duplicate prevention guard during in-flight update.
|
||
- No automatic regeneration.
|
||
- Explanation copy: "The investigation has changed since this report was generated."
|
||
|
||
### v0.59c — Report freshness state surfaced on the Portfolio
|
||
|
||
- Portfolio surfaces `Current` / `Update available` alongside existing View report link.
|
||
- Derives freshness solely from revision provenance (`investigationReport.generatedFromRevision === investigationRevision`).
|
||
- Zero model calls during Portfolio render.
|
||
- No mutation of Investigation.
|
||
- No `Update report` action on the Portfolio — manual Report updating remains owned by the Report page.
|
||
- If no Report, neither freshness state is shown.
|
||
|
||
**Semantic transitions that advance revision:**
|
||
```
|
||
Episode Done (with content) → +1
|
||
Re-open (resolved → unknown) → +1
|
||
Finding proposition correction → +1
|
||
Eligible → not_relevant → +1
|
||
not_relevant → eligible (restore) → +1
|
||
First meaningful change → =1 (from initial 0)
|
||
```
|
||
|
||
**Transitions that do NOT advance revision:**
|
||
```
|
||
Autosave → 0
|
||
Report generation → 0
|
||
Hydration → 0 (reads value)
|
||
Empty proposition correction (no-op) → 0
|
||
null → null disposition (no-op) → 0
|
||
```
|
||
|
||
### Zero Open Questions milestone (v0.51)
|
||
|
||
When all unknowns are resolved and clarified questions exist: "You've now worked through all of the questions we surfaced. Would you like to see an overview of what we understand so far?" with "Review current understanding" button. This occupies the former Open Questions position. The invitation is a milestone, not a readiness/completion judgement.
|
||
|
||
### Current Understanding refresh invariants
|
||
|
||
Reconstruct CU when canonical meaning or eligible evidence set changes — NOT when investigation/question status changes alone. Re-open ≠ change what we understand; Finding correction / Not Relevant / completed episode = change what we understand.
|
||
|
||
## Persistence and Report ownership
|
||
|
||
- **Canonical persistence owner:** `lib/storage/providers/local-storage.js` (`saveInvestigation` / `loadInvestigation`). All routes read from the same snapshot.
|
||
- **Report generation owner:** Report page only (NOT ReasoningWorkspace, NOT Investigation page).
|
||
- **localStorage key:** `confidence-engine-investigation` (single canonical key — multi-investigation not yet implemented).
|
||
- **Temporary identity:** `case-1`. True multi-investigation persistence/identity is future work.
|
||
- **Portfolio client hydration:** Portfolio page uses `'use client'` — initial pre-hydration empty state ≠ absence of persisted data. Always wait for hydrated semantic controls before classifying state.
|
||
|
||
## MVP boundaries (v0.59b)
|
||
|
||
**Implemented in MVP:**
|
||
- visible Report freshness state (`Current` / `Update available`)
|
||
- explanation that the Investigation has changed since Report generation
|
||
- manual `Update report` action (user-triggered)
|
||
- no automatic regeneration
|
||
- duplicate prevention guard during in-flight update
|
||
|
||
**Deferred beyond MVP:**
|
||
- Report history
|
||
- retaining multiple Reports
|
||
- Report comparison
|
||
- modelling/preview comparison between Investigation revisions
|
||
|
||
## Current development / verification constraints
|
||
|
||
- Canonical dev server at `http://localhost:3000`. Never start/stop/restart/probe it. If unavailable → BLOCKED and stop.
|
||
- Playwright MCP: use **Run Playwright code** with semantic locators for known controls. Snapshot refs (`[ref=...]`) are observational only.
|
||
- For async/hydration states: use `waitFor({ state: 'visible', timeout })` — not arbitrary sleeps.
|
||
- If a prescribed semantic locator cannot find its control → STOP. No fallback to CSS/XPath/DOM traversal.
|
||
- Live freeze: once Playwright verification begins, no production file edits until evidence is captured and classified.
|
||
- **Tests are instruments, not product truth.** At first deterministic failure: classify PRODUCT vs APPARATUS failure, then stop. Do not enter test-harness repair loops.
|
||
- Mocked boundary ownership: if a lower-layer function is mocked, test the value crossing the seam — do not require the mock to reproduce real implementation.
|
||
|
||
## Current limitations / genuinely open boundaries
|
||
|
||
**Not yet implemented:**
|
||
|
||
- Multi-investigation portfolio (search/tag/archive/group)
|
||
- Durable investigation identities beyond `case-1`
|
||
- Export/copy of Reports to Jira or external document
|
||
- Portfolio expansion beyond one canonical investigation
|
||
- Report history / comparison
|
||
|
||
**Known boundaries:**
|
||
|
||
- Current Understanding and Investigation Report are architecturally distinct artefacts. Plausible interpretations in the Report remain explicitly interpretive, not evidence.
|
||
- The `≤5` processing bound observed during development was an experimental apparatus constraint, NOT a product requirement. Six Open Questions surfacing is legitimate product output, not a formulation defect.
|
||
- Evidence discipline: what proves useful in live experimentation must be captured at provenance level, not as test diary entries that become operational constraints.
|
||
|
||
## v0.60 — Multi-Investigation Storage Contract Decision
|
||
|
||
Completed on branch `feature/multi-investigation-storage-v0.60`. Bounded architecture/contract decision task following v0.60 audit. Zero production changes. Documentation-only decision checkpoint.
|
||
|
||
### 1. Contract ownership
|
||
|
||
**Decision:** `lib/storage/investigation-storage.js` is the application-facing persistence boundary. It defines the contract whose semantics (not just signatures) are immutable across provider implementations. Consumers import only from this module.
|
||
|
||
`lib/storage/providers/local-storage.js` remains one concrete provider implementation behind the contract, not an application import target.
|
||
|
||
**Invariant:** No consumer imports `providers/local-storage` directly.
|
||
|
||
### 2. Durable Investigation identity
|
||
|
||
**Decision:** Canonical Investigation carries a durable `id` field.
|
||
|
||
| Property | Value |
|
||
|---|---|
|
||
| ID allocation owner | Application (not storage contract) |
|
||
| Allocation timing | On create new, before any persistence call |
|
||
| Mutable after creation | NO |
|
||
| Route `[id]` represents same identity | YES — direct round-trip: app sets → route reads → storage uses |
|
||
|
||
### 3. Minimum contract operations — resolved in v0.60b
|
||
|
||
Four semantic operations. `createEmptyInvestigation()` does **not** belong to the storage contract (resolved below).
|
||
|
||
| # | Operation | Input | Output | Product use |
|
||
|---|---|---|---|---|
|
||
| 1 | `listInvestigations()` | — | `InvestigationSummary[]` (lightweight) | Portfolio renders index/list |
|
||
| 2 | `loadInvestigation(id)` | durable ID string | `Investigation \| null` | Investigation/Report page hydration |
|
||
| 3 | `saveInvestigation(investigation)` | full snapshot (with `id` already present) | void | Autosave, Report generation, all writes |
|
||
| 4 | `restartInvestigation(id)` | durable Investigation ID | void (reasoning state cleared) | "Restart this investigation" |
|
||
|
||
**Explicitly rejected:** search, pagination, sorting, sync, merge, transactions, history, report versions, sharing, permissions, accounts. No existing product behaviour requires them.
|
||
|
||
---
|
||
|
||
### 3b. Investigation creation ownership — v0.60b resolution
|
||
|
||
**Contract correction decision.** The v0.60a contract contained a contradiction: Section 2 stated "ID allocation owner: Application" while Section 3 listed `createEmptyInvestigation()` as a storage operation producing `{ id: string }`. These are incompatible models (application-owned identity vs. storage-owned creation). They cannot both be true.
|
||
|
||
**Evidence from current product:**
|
||
|
||
1. **Create new behaviour today:** Portfolio's "+ Create new investigation" is a `<Link href="/investigations/case-1">` — a static navigation link with zero ID allocation and zero storage call. It navigates to the Investigation route which calls `loadInvestigation()`, receives null (no persisted data), and shows an empty ScenarioForm state. No empty Investigation is persisted on click.
|
||
|
||
2. **First meaningful persistence event:** `scenario-form.jsx` line 681 — after `submitScenarioForStartCase` returns successfully. The user has selected a scenario, submitted it, and the application receives a structured result. Only then does `saveInvestigation()` persist with real data (scenario + situationGraph + investigationRevision: 1). This is a domain-driven persistence boundary, not an identity-driven one.
|
||
|
||
3. **No product requirement for empty-persisted Investigations:** There is no current product behaviour that creates, displays, or expects empty persisted Investigations in Portfolio. The concept of persisting an abandoned start as a visible Portfolio item was asserted in v0.60a without product evidence.
|
||
|
||
4. **No product requirement for storage to allocate identity:** Current code uses hardcoded `INVESTIGATION_ID = "case-1"` in application code with no storage involvement in ID generation or allocation.
|
||
|
||
**Decision: MODEL A wins — application-owned identity, deferred persistence.**
|
||
|
||
- **ID allocation owner:** Application layer (not storage contract).
|
||
- **When allocated:** On user click of "+ Create new investigation", before navigation.
|
||
- **First persistence:** When the user produces meaningful Investigation state (scenario submitted) and `saveInvestigation(investigation)` is called — not on create-new click.
|
||
- **Empty abandoned Investigation persisted:** NO. No durable Investigation exists until meaningful state triggers save.
|
||
- **Storage allocates identity:** NO.
|
||
|
||
`createEmptyInvestigation()` does **not** belong in the storage contract. It never did — it was inconsistent with Section 2's "Application (not storage)" decision. The creation flow lives entirely at the application layer: app allocates durable ID → navigates to `/investigations/{id}` → user produces state → `saveInvestigation(investigation)` persists.
|
||
|
||
**Create new investigation flow (corrected):**
|
||
|
||
1. User clicks "+ Create new investigation" on Portfolio.
|
||
2. Application allocates a durable immutable ID (library/algorithm deferred).
|
||
3. Application navigates to `/investigations/{id}`.
|
||
4. User fills out ScenarioForm → submits → first meaningful `saveInvestigation(investigation)` persists the Investigation under its already-known ID. The Investigation becomes listable in Portfolio at that point.
|
||
|
||
**Investigation properties (unchanged from v0.60a):**
|
||
|
||
- Carries durable id: YES
|
||
- ID immutable: YES
|
||
- ID exists before first persistence: YES
|
||
|
||
**Restart decision retained:** YES — identity preserved, reasoning/report state cleared.**
|
||
|
||
### 4. Portfolio listing returns lightweight metadata
|
||
|
||
`listInvestigations()` returns summaries, not full snapshots. Minimum information:
|
||
- `id` — navigation target
|
||
- `scenario` (or title framing) — card text
|
||
- `updatedAt` — freshness display
|
||
- `investigationRevision` — revision metadata
|
||
- Report existence and `generatedFromRevision` — "Current" / "Update available" indicator
|
||
|
||
Full Investigation loads separately via `loadInvestigation(id)` on navigation.
|
||
|
||
### 5. Restart = reset reasoning state, NOT remove
|
||
|
||
Restart clears reasoning state (findings, questions, report) within the Investigation but preserves the Investigation's durable identity and container. The Investigation remains visible in Portfolio listing as a named item with cleared state.
|
||
|
||
A separate "Delete investigation" UI is not required by current MVP. Contract-level `removeInvestigation(id)` is acceptable for future use but has no product owner yet.
|
||
|
||
### 6. Create new investigation flow
|
||
|
||
User clicks "+ Create new investigation" → application allocates durable immutable ID (library/algorithm deferred) → navigates to `/investigations/{id}` → Investigation page loads, `loadInvestigation(id)` returns null (no persisted state yet) → ScenarioForm renders in empty-start mode → user fills scenario and submits → first meaningful state is captured → `saveInvestigation(investigation)` persists the full snapshot under its already-known ID → Investigation becomes listable in Portfolio. No empty Investigation is persisted at creation time.
|
||
|
||
### 7. localStorage ≠ contract
|
||
|
||
The storage contract accepts/returns domain-level Investigation objects keyed by `id`. How each provider maps those to its storage mechanism is implementation detail. Contract does not dictate key scheme, row schema, or collection structure.
|
||
|
||
### 8. Synchronization separate
|
||
|
||
Storage contract = persistence only. Not synchronization. No sync-specific fields (syncStatus, remoteId, dirtyFlags, lastSyncedAt) belong in the Investigation shape during this increment. Decisions here do not prevent future sync — the Investigation object carries its own durable ID sufficient for identity resolution.
|
||
|
||
## v0.60c — Identity-Aware Investigation Storage Implementation
|
||
|
||
**First production implementation increment of v0.60.** Bounded capability: two independently addressable Investigations can be persisted and loaded through the storage contract.
|
||
|
||
### What was implemented
|
||
|
||
- `loadInvestigation(id)` — accepts optional durable ID; selects by that identity when provided; returns `null` for unknown IDs
|
||
- `saveInvestigation(investigation, id)` — accepts optional durable ID; persists under provider-chosen key derived from `id`; does NOT allocate or replace the supplied ID
|
||
- `clearInvestigation(id)` — accepts optional durable ID; removes specific investigation by identity when provided
|
||
|
||
### Representation chosen (internal to localStorage provider)
|
||
|
||
Each Investigation is stored as a separate top-level localStorage key:
|
||
```
|
||
confidence-engine-investigation:<durable-id>
|
||
```
|
||
e.g. `confidence-engine-investigation:inv-abc123`
|
||
|
||
This was the smallest sufficient representation because:
|
||
- Direct key lookup provides O(1) per-investigation access without needing an index
|
||
- No generic repository layer required — each key is independently addressable by its durable ID
|
||
- The two-Investigation proof requires independent storage and retrieval, which this achieves with zero indexes or aggregation
|
||
|
||
Representation is **not exposed** to callers — the contract does not reveal localStorage key structure.
|
||
|
||
### Two-Investigation proof results (deterministic test)
|
||
|
||
| Invariant | Result |
|
||
|---|---|
|
||
| saveInvestigation(A) persists A under A.id | ✅ PASS |
|
||
| saveInvestigation(B) persists B independently under B.id | ✅ PASS |
|
||
| saving B does not overwrite A | ✅ PASS |
|
||
| loadInvestigation(A.id) returns A | ✅ PASS |
|
||
| loadInvestigation(B.id) returns B | ✅ PASS |
|
||
| loadInvestigation(unknownId) returns null | ✅ PASS |
|
||
| saveInvestigation() does not invent/change supplied ID | ✅ PASS |
|
||
|
||
### Singleton compatibility seam
|
||
|
||
- Existing consumers call `loadInvestigation()` and `clearInvestigation()` **without** an id argument — these continue via the legacy singleton path (canonical key + sessionStorage fallback)
|
||
- No caller was refactored in this increment
|
||
- The compatibility seam is: functions accept optional second parameter; when absent, behaviour matches pre-v0.60c singleton semantics
|
||
- `case-1` is **NOT** introduced as canonical identity — it exists only in existing consumer route URLs and legacy data
|
||
|
||
### Legacy migration
|
||
|
||
- No legacy singleton → new durable ID migration policy was invented
|
||
- Existing `case-1` localStorage data continues to be served by the backward-compatible path
|
||
- Migration of existing consumers to the new identity-aware calls is deferred to a later increment
|
||
|
||
### Production files changed
|
||
|
||
| File | Purpose |
|
||
|---|---|
|
||
| `lib/storage/providers/local-storage.js` | Identity-aware storage contract implementation |
|
||
| `tests/storage/investigation-storage.test.js` | 6 new targeted tests for v0.60c invariants |
|
||
|
||
### Next restart point
|
||
|
||
Migrate existing application callers to the identity-aware contract signatures:
|
||
1. `[id]/page.jsx` — pass route `[id]` to `loadInvestigation(id)`
|
||
2. `[id]/report/page.jsx` — pass route `[id]` to `loadInvestigation(id)` and `saveInvestigation(..., id)`
|
||
3. `scenario-form.jsx` — pass investigation ID through save calls
|
||
4. `app/page.jsx` — migrate Portfolio to use `listInvestigations()` (next increment)
|
||
|
||
## v0.60d — Canonical Save Identity Contract
|
||
|
||
**Problem:** `investigation-storage.js` was a bare re-export (`export { ... } from "./providers/local-storage.js"`). It did not own semantic contract — whatever signatures the provider exposed were what consumers received. The provider accepted an independent explicit `id` argument that could silently override or compete with `snapshot.id`.
|
||
|
||
**Decision:** `investigation-storage.js` now owns the canonical identity contract. It wraps the provider with explicit semantics:
|
||
|
||
- Canonical save identity comes solely from `investigation.id`;
|
||
- `investigation-storage.js` owns application-facing semantics;
|
||
- localStorage provider remains implementation detail (never allocated ID, never changed it);
|
||
- Any remaining singleton compatibility behaviour is explicitly temporary — for unmigrated callers only.
|
||
|
||
**Implementation:**
|
||
|
||
| Module | Role |
|
||
|---|---|
|
||
| `lib/storage/investigation-storage.js` | Application-facing boundary — owns identity contract |
|
||
| `saveInvestigation(snapshot, explicitId)` | Uses `snapshot.id` as sole save key when present; falls back to singleton path for unidentified legacy snapshots |
|
||
| `loadInvestigation(id)` | Passes raw id to provider (identity-aware) or singleton path (legacy) |
|
||
| `lib/storage/providers/local-storage.js` | Concrete localStorage implementation — unchanged, representation remains private |
|
||
|
||
**Test:** 7 new deterministic tests in `tests/storage/investigation-storage.test.js` under describe block "v0.60d canonical identity" prove:
|
||
|
||
1. Identified snapshot persists under its own id key (not CANONICAL_KEY)
|
||
2. Load by same id round-trips correctly
|
||
3. Provider does not invent or replace the supplied ID
|
||
4. Explicit competing id argument is silently ignored when snapshot has an id
|
||
5. Two identified investigations (A/B) remain independently addressable
|
||
6. Unknown ID returns null
|
||
7. Unidentified legacy snapshots still fall back to CANONICAL_KEY
|
||
|
||
**Status:** UI/routes are **not** migrated. All existing callers continue via the singleton compatibility path (they call `saveInvestigation({ ... })` with no explicit second parameter, and their snapshots carry no `id` field). No production caller was modified in this increment.
|
||
|
||
## v0.60e — Route-Owned Investigation Identity
|
||
|
||
**Purpose:** Migrate the Investigation page route to own durable investigation identity via its route `[id]` segment, passing that ID through to ScenarioForm for hydration and persistence without migrating legacy singleton data or changing Portfolio/Report behaviour.
|
||
|
||
### What was implemented
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `app/investigations/[id]/page.jsx` | Removed hardcoded `INVESTIGATION_ID = "case-1"` constant; route `[id]` param extracted as `routeId` via `params.id`; `loadInvestigation(routeId)` loads by route identity; `investigationId={routeId}` passed to ScenarioForm in both branch paths; report navigation uses `routeId`. |
|
||
| `components/scenario-form.jsx` | Added `investigationId` prop; session restore calls `loadInvestigation(investigationId)` when provided; all 4 save call sites include `id: investigationId` in snapshot (autosave effect, start-case submit, update-case submit, report overview autosave). |
|
||
|
||
### Contract crossings verified by deterministic test
|
||
|
||
- **Owning test files:** `tests/storage/scenario-form-persistence.test.js` (11 tests) + `tests/storage/investigation-storage.test.js` (23 tests, including v0.60d canonical identity section)
|
||
- All 34 tests pass on first run; no reruns required
|
||
|
||
### Verified behaviour
|
||
|
||
- Route ID is passed from `[id]/page.jsx` into ScenarioForm as `investigationId`: **YES**
|
||
- Missing identified Investigation (`loadInvestigation("v060e-live")` → null) starts clean: **YES**
|
||
- Legacy singleton fallback used: **NO** — no backward-compat load was needed; the route ID was absent from storage
|
||
- First meaningful saved snapshot carries route ID: **YES** (all 4 save sites inject `id: investigationId`)
|
||
- Subsequent save identity preserved: **YES** (storage layer uses `snapshot.id` as sole identity authority)
|
||
- Storage provider changed: **NO** — only `investigation-storage.js` wrapper, already-proved in v0.60d
|
||
|
||
### Portfolio
|
||
|
||
Not migrated. `app/page.jsx` retains hardcoded `INVESTIGATION_ID = "case-1"`. Create New navigation to a durable-ID route is later work.
|
||
|
||
### Report
|
||
|
||
Not migrated. `app/investigations/[id]/report/page.jsx` untouched. Remains a later caller migration increment.
|
||
|
||
### Restart
|
||
|
||
Untouched in this increment. All `clearInvestigation()` calls remain without an id argument (legacy singleton path). If identity-aware restart is needed, the next increment should pass `investigationId` through those clear calls.
|
||
|
||
## v0.60f — Allocate Investigation Identity on Create New
|
||
|
||
**Purpose:** Replace the Portfolio's "+ Create new investigation" static link (`/investigations/case-1`) with an application-owned durable ID allocation that navigates to `/investigations/{id}` without persisting any empty Investigation.
|
||
|
||
### What was implemented
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `app/page.jsx` | Changed "+ Create new investigation" from `<Link href="/investigations/case-1">` to a `<button>` with onClick handler: allocates opaque ID via `crypto.randomUUID()`, navigates via `useRouter().push(`/investigations/${id}`)`. `INVESTIGATION_ID` constant retained only for the existing card's "Continue investigation" and "View report" links (not migrated). |
|
||
| `tests/ui/investigation-overview-ui.test.jsx` | Added file-level `vi.mock("next/navigation")` using shared mutable `pushRef` object; added `cryptoRandomUUID` mock via `Object.defineProperty(global, "crypto")`. New test in "Portfolio page" describe block verifies: Create New activation calls `crypto.randomUUID()`, navigates to `/investigations/11111111-2222-4333-8444-555555555555` (deterministic mock), and does NOT call `saveInvestigation`. |
|
||
|
||
### Contract crossings verified by deterministic test
|
||
|
||
- Create New allocates fresh ID: **YES** (`crypto.randomUUID()` called once)
|
||
- Navigation uses generated ID: **YES** (`router.push("/investigations/11111111-...")`)
|
||
- No `saveInvestigation` during creation: **YES** (storage mock was not invoked)
|
||
- Control is a `<button>`, not a `<Link>`: **YES** (semantically correct — href cannot be static when ID is allocated at activation time)
|
||
|
||
### Verified behaviour
|
||
|
||
- Create New navigates to `/investigations/{UUID}`: **YES**
|
||
- Generated ID does not equal `case-1`: **YES**
|
||
- Resulting Investigation route starts clean (no persisted data): **YES** (route calls `loadInvestigation(id)` → null)
|
||
- Clean scenario-start state visible on arrival: **YES** (ScenarioForm textbox present, empty)
|
||
- Empty Investigation persisted on Create New click: **NO** (only allocation + navigation — no persistence)
|
||
|
||
### Portfolio scope bounded
|
||
|
||
- Card "Continue investigation" / "View report" links still use `INVESTIGATION_ID = "case-1"`: unchanged (not migrated in this increment)
|
||
- No portfolio listing changes, search, sorting, pagination, or delete: not implemented
|
||
- Report route untouched: not migrated
|
||
|
||
### Production files changed
|
||
|
||
| File | Purpose |
|
||
|---|---|
|
||
| `app/page.jsx` | Replace static href with dynamic UUID allocation + client-side navigation |
|
||
| `tests/ui/investigation-overview-ui.test.jsx` | Add useRouter mock (shared mutable reference) + Create New deterministic test |
|
||
|
||
### Next restart point
|
||
|
||
Next increment: migrate Portfolio card links ("Continue investigation", "View report") to use the first existing Investigation's durable ID — or migrate Portfolio to `listInvestigations()` with the four-operation storage contract. Do not proceed until both are addressed.
|
||
|
||
## v0.60g1 — List Investigation Summaries
|
||
|
||
**Purpose:** The previous v0.60g stalled attempt was discarded. This increment cleanly implements only the `listInvestigations()` storage contract: durable-ID listing returning lightweight summaries, excluding legacy/unrelated state.
|
||
|
||
### What was implemented
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `lib/storage/providers/local-storage.js` | Added `listInvestigations()` — enumerates provider records by prefix match, parses each record, projects lightweight summary (explicitly excludes situationGraph, findings, investigationReport), sorts by updatedAt descending |
|
||
| `lib/storage/investigation-storage.js` | Imported and re-exported `listInvestigations` as the application-facing public API |
|
||
| `tests/storage/investigation-storage.test.js` | 8 new deterministic tests covering all listing invariants; fixed MockStorageMap to implement `.length` and `.key(i)` for WebStorage API compatibility |
|
||
|
||
### Listing contract details
|
||
|
||
- **Operation:** `listInvestigations()` — no arguments, returns `InvestigationSummary[]`
|
||
- **Summary fields (explicitly projected):** `id`, `scenario`, `updatedAt`, `investigationRevision`, `reportExists`, `reportGeneratedFromRevision`
|
||
- **Excluded fields:** situationGraph, findings, full investigationReport, reasoning history, Open Questions, graph nodes
|
||
- **Records included:** only keys matching `INVESTIGATION_PREFIX` (durable-ID entries)
|
||
- **Records excluded:** legacy singleton (`confidence-engine-investigation`), sessionStorage state, unrelated keys, malformed entries
|
||
- **Ordering:** updatedAt descending (most recent first); deterministic fallback by id for equal timestamps
|
||
- **Malformed handling:** skip silently — one malformed entry never blocks valid records
|
||
|
||
### Deterministic test evidence (8 tests, all pass on first run)
|
||
|
||
| Invariant | Result |
|
||
|---|---|
|
||
| Two durable-ID Investigations coexist → 2 summaries | ✅ PASS |
|
||
| Each summary carries correct durable ID | ✅ PASS |
|
||
| Lightweight projection: portfolio fields present, payload fields absent | ✅ PASS |
|
||
| Legacy singleton excluded from listing | ✅ PASS |
|
||
| Unrelated localStorage key excluded | ✅ PASS |
|
||
| Independent update preserves other Investigation | ✅ PASS |
|
||
| Deterministic ordering by updatedAt descending | ✅ PASS |
|
||
| Malformed durable-ID entry skipped, valid records still listed | ✅ PASS |
|
||
|
||
### Portfolio scope bounded
|
||
|
||
- Portfolio has **NOT** been migrated to consume `listInvestigations()` — that is v0.60g2
|
||
- `app/page.jsx` unchanged from v0.60f
|
||
- No rendering or interactive behaviour changes
|
||
|
||
### Production files changed
|
||
|
||
| File | Purpose |
|
||
|---|---|
|
||
| `lib/storage/providers/local-storage.js` | Provider implements record enumeration + lightweight projection |
|
||
| `lib/storage/investigation-storage.js` | Application-facing wrapper re-exports listing operation |
|
||
| `tests/storage/investigation-storage.test.js` | 8 new listing contract tests; MockStorageMap WebStorage API fix |
|
||
|
||
### Next restart point
|
||
|
||
v0.60g2: Migrate Portfolio to consume `listInvestigations()` for collection rendering — replace the hardcoded singleton card with a rendered list of Investigation summaries.
|
||
|
||
Smallest next increment: implement the four-operation storage contract in `lib/storage/investigation-storage.js` as a re-export of a provider-backed interface whose signatures accept/return domain Investigation objects keyed by durable ID — without committing to any specific localStorage or database representation. This means defining the exported function signatures and the Investigation shape that flows through them, while deferring key scheme, row schema, and collection structure to a later implementation decision.
|
||
|
||
## v0.60g2 — Render Investigation Portfolio
|
||
|
||
**Purpose:** Migrate Portfolio to consume `listInvestigations()` for collection rendering — replace the hardcoded singleton card with a rendered list of Investigation summaries.
|
||
|
||
### What was implemented
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `app/page.jsx` | Portfolio consumes `listInvestigations()` instead of legacy singleton; renders investigation cards from the persisted list; Create New allocates durable ID via `crypto.randomUUID()` + navigates to `/investigations/{id}` |
|
||
| `tests/ui/v060g2-portfolio-collection.test.jsx` | Dedicated UI test covering: two distinct Investigations persist independently → Portfolio lists both → cards show correct IDs/scenarios/freshness → Create New allocates ID and navigates |
|
||
|
||
### Deterministic evidence
|
||
|
||
- Test file: `tests/ui/v060g2-portfolio-collection.test.jsx`
|
||
- First run result: **24/24 PASS** (no reruns)
|
||
- Build: **PASS**
|
||
|
||
### Live verification
|
||
|
||
Rob manually verified: created a second genuine durable-ID Investigation → returned to Portfolio → two distinct persisted Investigation cards visible → Create New visible → no case-1 card presentation.
|
||
|
||
### Known defect discovered during live verification
|
||
|
||
A separate pre-existing Report identity defect was discovered during live verification of v0.60g2. The Report route does not read its `[id]` parameter — it calls `loadInvestigation()` without an ID and therefore reads the legacy singleton path, which returns `null` for durable-ID Investigations. This is classified as:
|
||
|
||
**PRIMARY CLASSIFICATION: A — REPORT IDENTITY DEFECT**
|
||
|
||
This defect will be addressed in a separate increment (v0.60h). It is NOT caused by empty-Done semantics.
|
||
|
||
### Next restart point
|
||
|
||
v0.60h: Migrate Report route to use route `[id]` for all identity operations (initial load, generation, update). Preserve existing v0.58/v0.59 Report lifecycle and freshness semantics. This is an identity migration only — no Report redesign.
|
||
|
||
## v0.60h — Report Route Identity Migration
|
||
|
||
**Purpose:** Fix the Report route's two unscoped `loadInvestigation()` calls so it reads by route `[id]` instead of the legacy singleton path. Preserve all existing v0.58/v0.59 Report lifecycle and freshness semantics.
|
||
|
||
### What was implemented
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `app/investigations/[id]/report/page.jsx` | Extract `routeId` from `params.id`; both `loadInvestigation()` calls scoped to `routeId`; "Back to investigation" link uses dynamic `/investigations/${routeId}`; save spreads snapshot carrying `id` (identity preserved by existing v0.60d contract) |
|
||
| `tests/ui/investigation-overview-ui.test.jsx` | Enhanced storage mock to capture `loadInvestigation(id)` argument and `saveInvestigation(snapshot)` argument; added `listInvestigations: () => []` mock (pre-existing apparatus gap); 5 new identity assertions in "Report route identity — v0.60h" describe block |
|
||
|
||
### Deterministic evidence
|
||
|
||
- **Owning test file:** `tests/ui/investigation-overview-ui.test.jsx`
|
||
- **Identity assertions (new):**
|
||
- inv-a identified load + first generation: **PASS** (load called with "inv-a", fetch POST count = 1, saved snapshot retains id="inv-a", generatedFromRevision = investigationRevision)
|
||
- existing report for inv-b renders without generation: **PASS** (load called with "inv-b", zero overview calls)
|
||
- manual update reloads by same id and saves back: **PASS** (load called with "inv-c" after update click, save retains "inv-c", generatedFromRevision updated)
|
||
- existing report hydration retains zero-call behaviour: **PASS** (no overview call when revisions match)
|
||
- generation failure preserves existing Report: **PASS** (no partial persist on failure)
|
||
- **Existing Report lifecycle tests (unchanged):** All 10 PASS — no regression from identity migration
|
||
- **Pre-existing apparatus gaps:** 15 failures in Portfolio/restart sections due to `listInvestigations` mock gap (unrelated to v0.60h; pre-existing from v0.60g2)
|
||
|
||
### Identity mechanism
|
||
|
||
- Route params prop (`params.id`) → `routeId` (string or empty fallback)
|
||
- `loadInvestigation(routeId)` — initial hydration + manual update
|
||
- `saveInvestigation({ ...snapshot, investigationReport })` — identity preserved by snapshot spread; storage layer uses `snapshot.id` per v0.60d contract
|
||
- "Back to investigation" link → `/investigations/${routeId}` (dynamic)
|
||
|
||
### Product files changed
|
||
|
||
| File | Scope |
|
||
|---|---|
|
||
| `app/investigations/[id]/report/page.jsx` | Route identity migration only (3 load/save scopes + 1 link update) |
|
||
| `tests/ui/investigation-overview-ui.test.jsx` | Mock enhancement (load/save argument capture, listInvestigations stub) + 5 identity assertions |
|
||
|
||
### NOT changed
|
||
|
||
- Report generation logic
|
||
- Overview API semantics
|
||
- Empty-Done semantics
|
||
- Findings workflow
|
||
- Storage contract (`investigation-storage.js`, `local-storage.js`)
|
||
- Portfolio/ScenarioForm/SituationGraph
|
||
|
||
---
|
||
|
||
## v0.60h-a — Clean deterministic verification of Report identity migration
|
||
|
||
**Purpose:** Isolate the contaminated v0.60h deterministic test evidence into a dedicated file, proving that Report identity assertions pass cleanly without unrelated Portfolio/restart apparatus failures.
|
||
|
||
**Why this exists:** The original v0.60h deterministic run targeted `tests/ui/investigation-overview-ui.test.jsx` (mixed file) and produced 15/15 unrelated Portfolio/restart failures due to shared mutable mocks — not Report defects. Under experiment discipline, this should have been classified as APPARATUS FAILURE with the v0.60h implementation left intact (which it was).
|
||
|
||
**What was done:** Created `tests/ui/v060h-report-identity.test.jsx` — a dedicated file containing only Report page assertions, isolated from Portfolio/restart test apparatus.
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Source mixed test inspected | `tests/ui/investigation-overview-ui.test.jsx` (Report route identity describe block at line 497) |
|
||
| Dedicated Report test | `tests/ui/v060h-report-identity.test.jsx` |
|
||
| Report assertions isolated | A (route identity on load), B (existing report no generation), C (first-gen with findings=[]), D (manual update), E (generation failure), F (update failure) — 6 tests total |
|
||
| Portfolio rendered/imported | NO |
|
||
| Restart behaviour included | NO |
|
||
| Storage provider imported | NO |
|
||
| localStorage used | NO (only as implementation detail of mock setup, not in assertions) |
|
||
| Module-cache manipulation used | NO |
|
||
|
||
### Isolation apparatus
|
||
|
||
| Category | What is included | What is excluded |
|
||
|---|---|---|
|
||
| Mocks | `loadInvestigation(id)`, `saveInvestigation(snapshot)`, `listInvestigations()`, `clearInvestigation()` | None beyond storage contract |
|
||
| Fixture | Minimal identified Investigation with `id: "inv-report-a"`, valid `situationGraph`, empty `findings` | No Findings, no re-open workflow |
|
||
| Route id | Fixed: `"inv-report-a"` (not dynamic UUIDs, not case-1) | No random discovery, no shared mutable refs |
|
||
|
||
### Deterministic evidence
|
||
|
||
| Invariant | Result |
|
||
|---|---|
|
||
| First run isolated command | `npx vitest run tests/ui/v060h-report-identity.test.jsx` |
|
||
| 6/6 Report assertions PASS on first run | YES |
|
||
| No reruns required | YES |
|
||
| Production files changed | NO |
|
||
| Build run | NO |
|
||
| Playwright / model / server activity | NONE |
|
||
|
||
### Assertions proven
|
||
|
||
| Category | Assertion | Result |
|
||
|---|---|---|
|
||
| A | `loadInvestigation("inv-report-a")` on initial load | PASS |
|
||
| B | Existing report renders without new overview POST | PASS |
|
||
| C | First generation: exactly one POST, snapshot id === `"inv-report-a"`, generatedFromRevision === 4 | PASS |
|
||
| D | Manual update reloads same id, saves back with updated revision | PASS |
|
||
| E | Generation failure preserves existing (no partial save) | PASS |
|
||
| F | Update failure preserves existing report | PASS |
|
||
|
||
### Classification: PASS
|
||
|
||
**v0.60h production implementation now has uncontaminated deterministic evidence for durable Report identity and the established Report lifecycle.**
|
||
|
||
- v0.60h production commit remains `2af5971` (untouched)
|
||
- Original v0.60h deterministic evidence was contaminated by unrelated mixed-file Portfolio/restart failures
|
||
- v0.60h-a introduced dedicated Report-only deterministic test
|
||
- Exact isolated command `npx vitest run tests/ui/v060h-report-identity.test.jsx` passed on its first run (6/6)
|
||
- Report identity/lifecycle is now cleanly verified independently of Portfolio/restart apparatus
|
||
|
||
### NOT proven by this verification
|
||
|
||
- All UI tests pass
|
||
- Portfolio/restart mixed apparatus is repaired
|
||
- The entire Vitest suite passes
|
||
- Empty-Done semantics were retested end-to-end
|
||
|
||
---
|
||
|
||
## v0.60j — Semantic Restart within Investigation Container
|
||
|
||
**Purpose:** Implement `restartInvestigation(id)` that preserves the durable Investigation container (id, scenario) while clearing all reasoning/report state. Migrate Portfolio and ScenarioForm Restart callers from `clearInvestigation()` to `restartInvestigation(investigationId)`.
|
||
|
||
### What was implemented
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `lib/storage/providers/local-storage.js` | Added `restartInvestigation(id)` — preserves id/scenario/schemaVersion; resets situationGraph→null, selectedQuestion→null, summary→null, focusedContributions→[], findings→[], investigationReport→null, investigationRevision→0, updatedAt→new ISO |
|
||
| `lib/storage/investigation-storage.js` | Imported and re-exported `restartInvestigation`; added semantic contract doc (missing id → no-op, no singleton fallback) |
|
||
| `app/page.jsx` | Portfolio card Restart: replaced `clearInvestigation(summary.id)` with `restartInvestigation(summary.id)` — preserves container, clears reasoning |
|
||
| `components/scenario-form.jsx` | Three callers migrated: ReasoningWorkspace onRestart, ContinueLaterBanner onRestart, "Start new investigation" button — all call `restartInvestigation(investigationId)` instead of `clearInvestigation()` |
|
||
|
||
### Deterministic evidence
|
||
|
||
- **Storage tests:** 44/44 PASS (investigation-storage.test.js)
|
||
- **UI contract tests:** 7/7 PASS (v060j-restart-contract.test.jsx)
|
||
- **Exact command:** `npx vitest run tests/storage/investigation-storage.test.js tests/ui/v060j-restart-contract.test.jsx`
|
||
- **First run result:** 51/51 PASS, no reruns
|
||
|
||
### Semantic contract of restartInvestigation(id)
|
||
|
||
**Preserved (container-level):** id, scenario, schemaVersion
|
||
**Reset:** situationGraph→null, selectedQuestion→null, summary→null, focusedContributions→[], findings→[], investigationReport→null, investigationRevision→0, updatedAt→new timestamp
|
||
|
||
### Portfolio caller migrated
|
||
|
||
- Portfolio Restart confirmation dialog → `restartInvestigation(summary.id)` ✓
|
||
- No longer calls `clearInvestigation()` ✓
|
||
- Card id passed correctly (inv-a, inv-b verified by UI test)
|
||
|
||
### ScenarioForm three callers migrated
|
||
|
||
| Path | Location | Contract | Observability |
|
||
|---|---|---|---|
|
||
| 1 — ReasoningWorkspace onRestart | line ~949 | `restartInvestigation(investigationId)` | Source-inspected (PATH 1 NOT DIRECTLY OBSERVABLE IN BOUNDED APPARATUS) |
|
||
| 2 — ContinueLaterBanner onRestart | line ~971 | `restartInvestigation(investigationId)` | **Directly exercised** by UI test |
|
||
| 3 — "Start new investigation" button | line ~979 | `restartInvestigation(investigationId)` | **Directly exercised** by UI test |
|
||
|
||
### UI apparatus
|
||
|
||
- ScenarioForm direct caller assertions added: YES (2 directly observable via ContinueLaterBanner and StartNew buttons)
|
||
- ReasoningWorkspace onRestart confirmed by bounded source inspection
|
||
- No localStorage used, no provider imported, no module-cache manipulation
|
||
- All mocks at application-facing boundary only
|
||
|
||
### Live verification (accepted)
|
||
|
||
- same Portfolio card remained
|
||
- same durable ID remained
|
||
- same scenario remained
|
||
- second Investigation unaffected
|
||
- old reasoning state absent
|
||
- clean Analyse state visible
|
||
- no model call caused by Restart
|
||
|
||
## v0.60 is structurally complete
|
||
|
||
No remaining current production identity migration is known. v0.60 is structurally complete.
|
||
|
||
The established multi-Investigation architecture:
|
||
|
||
```
|
||
Portfolio
|
||
→ durable-ID Investigation collection
|
||
|
||
Create New
|
||
→ allocate crypto.randomUUID()
|
||
→ navigate to /investigations/{id}
|
||
→ no empty Investigation persisted
|
||
|
||
Investigation route
|
||
→ route [id] owns Investigation identity
|
||
|
||
ScenarioForm
|
||
→ hydrates by durable ID
|
||
→ saves identified snapshots
|
||
|
||
Report route
|
||
→ loads/saves by route [id]
|
||
|
||
Restart
|
||
→ preserves Investigation container/id/scenario
|
||
→ clears reasoning/Report state
|
||
→ leaves other Investigations untouched
|
||
```
|
||
|
||
v0.60 result: **STRUCTURALLY COMPLETE**.
|
||
|
||
### Acceptance evidence
|
||
|
||
- case-1 production dependencies: 0
|
||
- current production unscoped load dependencies: 0
|
||
- current production unidentified save dependencies: 0
|
||
- current user-facing clear/delete Restart callers: 0
|
||
- Portfolio navigation: durable-ID owned
|
||
- Report navigation/persistence: durable-ID owned
|
||
- legacy singleton compatibility: still present internally, unused by current product, non-blocking deferred cleanup
|
||
|
||
### Investigation culmination semantics
|
||
|
||
Report is the point-in-time culmination of an Investigation.
|
||
|
||
The flow:
|
||
|
||
```
|
||
Open Questions investigated
|
||
→ user chooses Done for now
|
||
→ question parked
|
||
→ graph reconsidered
|
||
→ Current Understanding regenerated
|
||
→ user chooses what to investigate next
|
||
→ eventually zero Open Questions
|
||
→ milestone invitation
|
||
→ "Review current understanding"
|
||
→ /investigations/{id}/report
|
||
```
|
||
|
||
There is NO active intermediate overview/review surface between "Review current understanding" and Report. The control performs direct route navigation.
|
||
|
||
Report represents: Situation, What we understand, What remains plausible when applicable. It does NOT represent a decision recommendation, confidence score, readiness judgement, or proof that the user should act.
|
||
|
||
The user retains ownership of whether the understanding is sufficient, whether to act, whether to return to the Investigation, whether to re-open and investigate further.
|
||
|
||
Report provenance remains: `generatedFromRevision` against `investigationRevision`, so a Report can later become "Update available" without ceasing to be the valid point-in-time Report generated from its earlier Investigation revision.
|
||
|
||
Zero Open Questions ("You've now worked through all of the questions we surfaced") means: the currently surfaced Open Questions have been worked through. It does NOT mean: the user's decision is complete, sufficient confidence, or ready to act. The Engine facilitates the culmination. The user owns what that culmination means for their decision/action.
|
||
|
||
Report is the established Investigation culmination. No further product boundary is selected.
|
||
|
||
## v0.61 — Direct Initial-Decomposition Apparatus
|
||
|
||
**Status:** v0.61 Experiment 1 produced B ("USEFUL BUT MATERIAL UNCERTAINTY LOST"). This apparatus task was executed to establish direct initial-decomposition invocation for subsequent semantic experiments.
|
||
|
||
### Canonical production seam
|
||
|
||
The existing route `/api/cases/start` **is directly suitable** for curl/Postman/Claude experimentation:
|
||
|
||
```
|
||
┌───────────┐ POST /api/cases/start ┌──────────────┐
|
||
│ Scenario │ ───────────────────────────► │ startCase() │
|
||
│ text (req) │ { scenario, promptVersion? } │ │
|
||
│ │ │ analyseScenario │
|
||
│ │ │ buildInitialGraph │
|
||
│ │ │ selectUnknown │
|
||
└────────────┘ └──────────────┘
|
||
```
|
||
|
||
**No browser state required. No Investigation ID required by the route itself.** The route accepts `scenario` string directly and invokes the full production reasoning path (analyseScenario → buildInitialGraph → determineGraphBackedQuestion).
|
||
|
||
### Direct curl/Postman contract (Rob)
|
||
|
||
**Method:** POST
|
||
**URL:** `http://localhost:3000/api/cases/start`
|
||
**Content-Type:** `application/json`
|
||
|
||
**Request body schema:**
|
||
```json
|
||
{
|
||
"scenario": "<your scenario text here>",
|
||
"promptVersion": "v0.2"
|
||
}
|
||
```
|
||
|
||
- `scenario` (required): string, 1–10000 characters
|
||
- `promptVersion` (optional): `"v0.1"` or `"v0.2"` (defaults to `"v0.2"`)
|
||
|
||
**Response shape (success):**
|
||
```json
|
||
{
|
||
"success": true,
|
||
"summary": "<reconstruction summary>",
|
||
"situationGraph": { /* full graph with nodes/edges/reasoningState */ },
|
||
"selectedQuestion": { "id": "...", "question": "...", "reasoningPattern": "..." },
|
||
"diagnostics": { /* decompositionApplied, questionComplexityAssessment, etc. */ },
|
||
"assessment": { "phase": "...", "progress": "..." },
|
||
"modelName": "qwen-claude:latest",
|
||
"responseDurationMs": 3210,
|
||
"validationStatus": "valid",
|
||
"promptVersion": "v0.2"
|
||
}
|
||
```
|
||
|
||
**Response shape (failure):**
|
||
```json
|
||
{
|
||
"success": false,
|
||
"error": "<message>",
|
||
"statusCode": 400|500|502,
|
||
"validationErrors": [...],
|
||
"analysisErrors": [...],
|
||
"diagnostics": {...}
|
||
}
|
||
```
|
||
|
||
### Claude apparatus
|
||
|
||
**Needed:** YES — a thin CJS helper exists for repeated controlled experiments.
|
||
|
||
**Path:** `scripts/start-case-experiment-helper.cjs`
|
||
**Command:**
|
||
```bash
|
||
node scripts/start-case-experiment-helper.cjs "<scenario text>"
|
||
```
|
||
or
|
||
```bash
|
||
node scripts/start-case-experiment-helper.cjs --file scenario.json
|
||
```
|
||
|
||
**Input:** scenario string (positional arg or JSON file with `{ "scenario": "..." }`)
|
||
**Output:** structured JSON to stdout (success fields + endToEndElapsedMs)
|
||
**Retries:** NO — single call, no retry logic
|
||
**Canonical production logic duplicated:** NO — imports `startCase` from `lib/graph/orchestrator.js`, exercises identical code path
|
||
|
||
### Deterministic zero-live-call verification
|
||
|
||
| Check | Source | Result |
|
||
|---|---|---|
|
||
| Input reaches startCase seam | `tests/app/api/cases-start-route.test.js:15` | PASS (mocked analyseScenario verified) |
|
||
| Output passed through correctly | `tests/start-case-summary.test.js:81` | PASS (exact summary field round-trip) |
|
||
| Execution failure returns 400/5xx | `tests/app/api/cases-start-route.test.js:55,79` | PASS |
|
||
| Malformed JSON returns 500 | `tests/app/api/cases-start-route.test.js:100` | PASS |
|
||
| No retry occurs | source inspection (single await) | CONFIRMED |
|
||
| Schema validation present | `lib/graph/schema.js:202-205` | Zod enforced |
|
||
|
||
**Live model calls:** ZERO
|
||
**Build:** NOT required (scripts/test only, no production code changes)
|
||
**Playwright:** NOT used (decomposition-only experiments do not require browser instrumentation)
|
||
|
||
### Browser state investigation
|
||
|
||
| Question | Answer |
|
||
|---|---|
|
||
| Does the route mutate persistence? | NO — persistence is handled by ScenarioForm caller AFTER receiving result |
|
||
| Does it require an Investigation ID? | NO — route accepts scenario text directly; id comes from UI caller's state |
|
||
| Does it depend on browser localStorage? | NO — pure HTTP JSON exchange |
|
||
| Does it require any browser-only state? | NO |
|
||
|
||
### Decision: direct curl/Postman suitable = YES
|
||
|
||
**Why:** The `/api/cases/start` route is a thin layer (19 lines) over `startCase()` that validates input via Zod, calls the production function, and returns structured results. No browser state, no persistence side effects, no unrelated mutations. Identical behaviour to what ScenarioForm exercises in production.
|
||
|
||
### Playwright posture for v0.61 experiments
|
||
|
||
- **Playwright NOT default** for decomposition-only semantic experiments (apparatus reaches production reasoning path via direct import or HTTP)
|
||
- **Playwright REMAINS required** when the experiment concerns visible/browser behaviour, UI state transitions, or localStorage hydration
|
||
|
||
## v0.61.1 — Start-Case Experiment Helper Apparatus Verification
|
||
|
||
**Status: PASSED (18/18, first run, zero reruns)**
|
||
|
||
**Purpose:** Prove `scripts/start-case-experiment-helper.cjs` works as a standalone Node command with the same configured environment as production, without requiring live model calls or browser state.
|
||
|
||
### What was verified
|
||
|
||
| Check | Source | Result |
|
||
|---|---|---|
|
||
| A — Positional input reaches startCase seam | `tests/scripts/start-case-experiment-helper.test.js:76` | PASS |
|
||
| B — File input reads JSON fixture | `tests/scripts/start-case-experiment-helper.test.js:89` | PASS |
|
||
| B — Malformed file fails before startCase | `tests/scripts/start-case-experiment-helper.test.js:224` | PASS |
|
||
| C — .env.local loads without dotenv dependency | `tests/scripts/start-case-experiment-helper.test.js:116` | PASS |
|
||
| D — Exit code 0 on success | `tests/scripts/start-case-experiment-helper.test.js:131` | PASS |
|
||
| D — stdout is valid JSON with required fields | `tests/scripts/start-case-experiment-helper.test.js:136,143` | PASS |
|
||
| D — endToEndElapsedMs non-negative | `tests/scripts/start-case-experiment-helper.test.js:152` | PASS |
|
||
| E — Failure produces non-zero exit code | `tests/scripts/start-case-experiment-helper.test.js:169` | PASS |
|
||
| E — Failure output is valid JSON | `tests/scripts/start-case-experiment-helper.test.js:180` | PASS |
|
||
| F — startCase called exactly once on success | `tests/scripts/start-case-experiment-helper.test.js:194` | PASS |
|
||
| F — No retry on failure | `tests/scripts/start-case-experiment-helper.test.js:205` | PASS |
|
||
| G — Malformed --file fails before startCase | `tests/scripts/start-case-experiment-helper.test.js:224` | PASS |
|
||
| G — --file without path fails before startCase | `tests/scripts/start-case-experiment-helper.test.js:235` | PASS |
|
||
| Integrity — no diagnostic leakage to stdout | `tests/scripts/start-case-experiment-helper.test.js:248` | PASS |
|
||
|
||
### Apparatus fixes applied during verification
|
||
|
||
1. **Inline `.env.local` parser** — replaced `require("dotenv")` (MODULE_NOT_FOUND) with built-in `fs` + `path` loader
|
||
2. **Mock injection seam** — `START_CASE_EXPERIMENT_HELPER_MOCK=1` enables deterministic test doubles without live calls
|
||
3. **Structured failure output** — plain text console.error → valid JSON on stdout
|
||
4. **execFile harness fix** — capture stdout/stderr regardless of execFile error state
|
||
|
||
### Zero-live-call verification
|
||
|
||
- Live model calls: ZERO
|
||
- Build required: NO (scripts/test only, no production code changes)
|
||
- Exact command: `npx vitest run tests/scripts/start-case-experiment-helper.test.js`
|
||
|
||
### NOT proven
|
||
|
||
- Actual semantic quality of decomposition output
|
||
- Behaviour with real LLM endpoints
|
||
- Performance at scale
|
||
- All UI integration tests pass
|
||
|
||
## v0.61.2 — Direct-Helper Production-Seam Proof
|
||
|
||
**Status: APPARATUS FAILURE (import seam blocked) — but apparatus itself verified.**
|
||
|
||
### Environment loading result
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Existing compatible loader available | **YES** |
|
||
| Package/function | `@next/env` → `loadEnvConfig(projectDir, configPath?, logger, debug)` |
|
||
| Previous bespoke parser | Hand-written `.env.local` KEY=VALUE parser (replaced) |
|
||
| Final mechanism | `require("@next/env").loadEnvConfig(__dirname, undefined, { logOutput: "none" }, false)` |
|
||
| lib/config.js remains configuration authority | **YES** — interprets `OLLAMA_BASE_URL` and `OLLAMA_MODEL` |
|
||
| New dependency installed | **NO** — `@next/env` transitively available through `next ^14.2.0` |
|
||
|
||
### Real production import result
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Process | plain Node dynamic `import()` from standalone `.cjs` helper |
|
||
| Actual orchestrator imported | **NO** — module resolution barrier |
|
||
| Actual startCase export resolved | **N/A** (import fails before export resolution) |
|
||
| startCase invoked during import-only proof | **NO** — mode structurally stops before any invocation |
|
||
| Provider invoked | **NO** |
|
||
| Network used | **NO** |
|
||
|
||
**Failure reason:** The orchestrator's transitive dependency chain includes `lib/graph/apply-proposal.js` which imports from `@/lib/llm/provider`. This `@/` path alias is a Next.js compiler convention (configured in `jsconfig.json` as `"@/*": ["./*"]`). Plain Node has no resolver for `@/` aliases — it attempts to resolve `@/lib` as a bare package name and fails.
|
||
|
||
**Evidence:**
|
||
```
|
||
Cannot find module '/Users/.../lib/graph/orchestrator.js' imported from
|
||
/Users/.../scripts/start-case-experiment-helper.cjs
|
||
(cause: Cannot find package '@/lib' imported from apply-proposal.js)
|
||
```
|
||
|
||
**Two files in the import chain use `@/`:**
|
||
- `lib/graph/apply-proposal.js` → `import { getProvider } from "@/lib/llm/provider"`
|
||
- `lib/graph/focused-investigation.js` → (same alias pattern)
|
||
|
||
### Import-only structured output (failure path)
|
||
|
||
```json
|
||
{
|
||
"success": false,
|
||
"mode": "import-only",
|
||
"startCaseResolved": false,
|
||
"failureReason": "Module resolution failed: Cannot find module '@/lib/llm/provider' imported from /path/to/lib/graph/apply-proposal.js"
|
||
}
|
||
```
|
||
|
||
### Deterministic evidence
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Test file | `tests/scripts/start-case-experiment-helper.test.js` |
|
||
| Exact command | `npx vitest run tests/scripts/start-case-experiment-helper.test.js` |
|
||
| First-run result | **25/25 PASS** (first run, zero reruns) |
|
||
| Tests passed | 25 (18 existing apparatus + 7 new import-only) |
|
||
| Tests failed | 0 |
|
||
| Reruns | 0 |
|
||
|
||
### What was proven by this task
|
||
|
||
1. **Standard environment loader established:** `@next/env` `loadEnvConfig` replaces bespoke parser — no new dependency needed
|
||
2. **Import-only mode implemented:** Helper supports `START_CASE_EXPERIMENT_HELPER_IMPORT_ONLY=1` for deterministic seam verification
|
||
3. **Standalone production import incompatible:** Plain Node cannot load `lib/graph/orchestrator.js` due to Next.js `@/` alias chain in transitive dependencies
|
||
4. **No production files modified:** Helper and test changes only — zero impact on reasoning path
|
||
5. **Zero live calls:** Model calls: 0, HTTP /api/cases/start: 0, curl: 0, Playwright: NO
|
||
|
||
### Classification
|
||
|
||
**APPARATUS FAILURE** — standalone plain Node cannot import the real production orchestrator due to Next.js `@/` module alias convention.
|
||
|
||
This is documented evidence of a **tooling seam gap**: the helper architecture (standalone `.cjs` CLI) is incompatible with the repository's ESM path-alias module system without:
|
||
- Adding a bundler/loader (tsx, esbuild, bundler)
|
||
- Modifying production imports to use relative paths
|
||
- Running through Next.js tooling
|
||
|
||
The `@/` alias is a legitimate architectural choice that should not be changed. The gap means the helper must use **mock mode** for deterministic verification or run under Next.js-aware tooling.
|
||
|
||
### Existing helper status (pre-v0.61.2)
|
||
|
||
| Check | Result |
|
||
|---|---|
|
||
| Positional input reaches startCase seam | PASS (mock mode) |
|
||
| File input reads JSON fixture | PASS |
|
||
| .env.local loads without dotenv dependency | PASS (now via @next/env) |
|
||
| Exit code 0 on success | PASS |
|
||
| stdout is valid JSON with required fields | PASS |
|
||
| endToEndElapsedMs non-negative | PASS |
|
||
| Failure produces non-zero exit code | PASS |
|
||
| Failure output is valid JSON | PASS |
|
||
| startCase called exactly once on success | PASS |
|
||
| No retry on failure | PASS |
|
||
| Malformed --file fails before startCase | PASS |
|
||
| --file without path fails before startCase | PASS |
|
||
| Output structure integrity (stderr/stdout) | PASS |
|
||
|
||
### Live execution
|
||
|
||
- Model calls: **0**
|
||
- /api/cases/start calls: **0**
|
||
- curl calls: **0**
|
||
- Playwright: **NO**
|
||
|
||
### Files changed
|
||
|
||
- `scripts/start-case-experiment-helper.cjs` — environment loader replaced, import-only mode added
|
||
- `tests/scripts/start-case-experiment-helper.test.js` — 7 new import-only assertions (describe block H)
|
||
|
||
---
|
||
|
||
## v0.61 Experiment 2 — Initial Decomposition Coverage (Scenario Reuse)
|
||
|
||
**Status: B — USEFUL BUT MATERIAL UNCERTAINTY LOST**
|
||
|
||
### Execution
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Branch | `feature/initial-decomposition-v0.61` |
|
||
| Starting HEAD | `6425540 test(confidence-engine): prove direct helper production seam` |
|
||
| Route | POST /api/cases/start |
|
||
| Requests made | 1 |
|
||
| Successful results | 1 |
|
||
| Retries | 0 |
|
||
| Execution errors | NONE |
|
||
| Model | `qwen-claude:latest` |
|
||
| Response duration | 77,984 ms |
|
||
| Validation status | valid |
|
||
| Prompt version | v0.2 |
|
||
|
||
### Observed Decomposition
|
||
|
||
**Summary:** "A manufacturing business owner is evaluating whether to invest £120k in automated quality inspection amid rising complaint counts and recent operational changes, but lacks rate-based and categorical data to determine if a genuine quality or delivery problem exists."
|
||
|
||
**Graph topology:** 14 nodes (1 state, 3 observation, 3 metric, 2 relationship, 3 unknown, 2 assumption), 6 edges.
|
||
|
||
**Unknown investigative paths:**
|
||
1. `nvb2z51` — Current and historical complaint rate per unit produced is unknown
|
||
2. `nn7tkbd` — Whether the new CRM system changed the threshold or definition of a valid complaint compared to the old system
|
||
3. `n9fhzkz` — Defect and delivery failure rates specifically attributed to the new supplier or weekend shift are unknown
|
||
|
||
**Other materially relevant graph content:**
|
||
- `nqpcnaa` (relationship): Complaint volume rose less than production volume (35% vs 40%), suggesting the complaint rate per unit may have decreased or remained stable
|
||
- `netiuwi` (relationship): Complaint categories changed three months ago due to CRM updates, decoupling historical complaint data from current classification methods
|
||
- `npf0jdh` (assumption): No genuine quality degradation exists
|
||
- `n1kszew` (assumption): A true quality or delivery issue exists, likely triggered by new supplier or weekend shift
|
||
|
||
**Note on selected question:** The automatic unknown selection process produced a decomposition-rejection in this run. The `selectedQuestion` is null; the system generated a reconstruction question but rejected it as not authoritative, deferring to graph-backed pipeline. Three unresolved unknowns remain at initial decomposition — no auto-selection/prioritisation occurred beyond tie-breaking via scoring heuristics.
|
||
|
||
### Semantic Reference Evaluation (A–F)
|
||
|
||
**A — facts and explicit unknowns: CLEARLY PRESERVED**
|
||
Evidence: All six material facts present as distinct nodes (complaints +35%, production +40%, late delivery/minor defects, CRM tagging change, supplier change, weekend shift). £120k decision node present. Three explicit unknowns preserved. No invented facts.
|
||
|
||
**B — normalization uncertainty: CLEARLY PRESERVED**
|
||
Evidence: Node `nqpcnaa` explicitly captures the 35% vs 40% divergence as a relationship node with status "supported" and confidence "medium". Notes that complaint rate per unit may have decreased or remained stable despite higher absolute numbers. This is distinct from arithmetic resolution — it frames the question without resolving it.
|
||
|
||
**C — measurement/comparability: CLEARLY PRESERVED**
|
||
Evidence: Node `netiuwi` explicitly captures CRM categories changed decoupling historical data; node `nn7tkbd` explicitly frames whether the new CRM system changed the threshold or definition of a valid complaint. Both are selectable unknown investigative paths. The distinction between "measurement instrument changed" and "rate normalization needed" is partially compressed into shared graph space, but both concerns are explicitly available as user-investigable uncertainties — not folded away.
|
||
|
||
**D — nature of the problem: PARTIALLY PRESERVED**
|
||
Evidence: "Late delivery" and "minor product defects" are bundled into a single observation node (`nqjt07e`). The distinction between a delivery/logistics problem and a quality/defect problem is not preserved at sufficient granularity to guide targeted investigation.
|
||
|
||
**E — causal/source ambiguity: CLEARLY PRESERVED**
|
||
Evidence: Supplier change, weekend shift, CRM/tagging, and production volume changes all present in the graph without any edge asserting causality between them and the complaint increase. Node `nwg8ma4` bundles these as known metric changes; assumptions (`npf0jdh`, `n1kszew`) frame competing hypotheses without resolution.
|
||
|
||
**F — intervention fit: PARTIALLY PRESERVED / MATERIAL GAP**
|
||
Evidence: £120k automation cost node exists (`nubgwjr`). The proposed intervention is represented as part of the situation, but no graph content explicitly exposes whether that intervention fits the actual problem being investigated (e.g., automated inspection may address some defects but not late delivery, CRM/tagging effects, supplier effects, shift-management effects). This gap between "there might be a quality problem" and "£120k automated inspection is the right response" was not surfaced as an explicit unknown in Experiment 2.
|
||
|
||
### Failure Patterns
|
||
|
||
| Pattern | Result | Evidence |
|
||
|---|---|---|
|
||
| F1 — premature conclusion | NO | Both competing assumptions presented; no resolution asserted |
|
||
| F2 — invented evidence | NO | All facts trace to scenario text |
|
||
| F3 — material qualification lost | YES — late delivery/defect distinction (D) compressed; intervention-fit gap (F) not surfaced as explicit unknown |
|
||
| F4 — steering/prioritisation | NO | No "first/most important/primary/priority" language in graph or summary |
|
||
| F5 — action recommendation | NO | Summary uses neutral framing ("evaluating whether to invest"); no buy/wait directive |
|
||
| F6 — redundant Open Questions | NO | Three unknowns are materially distinct |
|
||
| F7 — premature arithmetic resolution | NO | 35% vs 40% divergence framed as open question, not resolved |
|
||
| F8 — speculative proliferation | NO | Two assumptions are direct negations of each other (minimal viable pair) |
|
||
| F9 — unsupported action implication | NO | No explicit recommendation to buy or wait; "waiting" only implied by null selectedQuestion |
|
||
|
||
### Classification: B — USEFUL BUT MATERIAL UNCERTAINTY LOST
|
||
|
||
**Reason:** The decomposition is useful and restrained — all facts are preserved, no steering, no premature conclusions. Three materially distinct unknowns at initial decomposition. However, dimensions D (nature of the problem) and F (intervention fit) are partially lost through compression: the late-delivery/defect distinction collapses into generic "complaints", and the £120k intervention's potential mismatch with delivery problems is not surfaced as an explicit uncertainty channel. This is a material compression, though less severe than Experiment 1 where CRM comparability and intervention fit were fully compressed.
|
||
|
||
### Comparison with Clean Experiment 1
|
||
|
||
The two clean runs surfaced somewhat different subsets of the scenario's uncertainty.
|
||
|
||
**Normalized across both runs:** Both clean experiments clearly preserved normalization uncertainty. Experiment 1 preserved it in explicit Open Question framing (absoluteness of counts without denominators); Experiment 2 preserved it via graph node `nqpcnaa` (35% vs 40% rate divergence). Neither run compressed the need for denominator data.
|
||
|
||
**CRM comparability across runs:** Experiment 1 preserved CRM change in framing but did not expose it as clearly as a selectable investigative path. Experiment 2 explicitly surfaced CRM threshold/definition comparability as an Open Question. The distinction is one of explicitness of exposure, not presence vs absence.
|
||
|
||
**Repeated signal across both runs — intervention fit:** Across both clean runs, intervention fit remained unsurfaced as a distinct investigative path. The proposed £120k automated-inspection action was represented as part of the situation in both, but neither decomposition explicitly exposed whether that intervention fits the actual problem being investigated.
|
||
|
||
Experiment 2 additionally shows partial compression of late-delivery vs product-defect distinction across runs.
|
||
|
||
**What two observations support:** Both clean experiments preserved normalization and CRM measurement concerns (at different levels of explicitness). The repeated signal is intervention fit — a gap in both runs.
|
||
|
||
**What two observations do NOT establish:** Systematic instability, model inconsistency, reliable coverage, improvement, regression, or reproducible patterns. Two observations of the same current system behaviour are insufficient to establish either direction.
|
||
|
||
### Git
|
||
|
||
**Documentation commit:** `docs(confidence-engine): record v0.61 decomposition experiment 2`
|
||
**Working tree:** clean after commit
|
||
|
||
### Next Evidence Question
|
||
|
||
One neutral evidence question only:
|
||
|
||
> When a scenario contains a proposed action whose usefulness depends on what kind of problem actually exists, does the initial decomposition preserve intervention fit as a distinct uncertainty without steering toward or against the action?
|
||
|
||
---
|
||
|
||
## v0.61 Experiment 3 — Intervention Fit (Scenario Reuse)
|
||
|
||
**Status: B — INTERVENTION FIT ONLY IMPLICIT**
|
||
|
||
### Execution
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Branch | `feature/initial-decomposition-v0.61` |
|
||
| Starting HEAD | `95d9965 docs(confidence-engine): correct v0.61 experiment interpretation` |
|
||
| Route | POST /api/cases/start |
|
||
| Requests made | 1 |
|
||
| Successful results | 1 |
|
||
| Retries | 0 |
|
||
| Execution errors | NONE |
|
||
| Model | `qwen-claude:latest` |
|
||
| Response duration | 90,801 ms |
|
||
| Validation status | valid |
|
||
| Prompt version | v0.2 |
|
||
|
||
### Observed Decomposition
|
||
|
||
**Summary:** "A manufacturing business owner is evaluating whether to invest in automated quality inspection based on a 35% rise in customer complaints, while production volume increased by 40%, recent operational changes altered tracking methods, and critical defect/volume metrics remain unknown."
|
||
|
||
**Graph topology:** 20 nodes (1 state, 6 observation, 3 metric, 2 relationship, 3 unknown, 3 assumption, 2 transition), 9 edges.
|
||
|
||
**Unknown investigative paths:**
|
||
1. `nvwsfko` — Actual complaint rate normalized by production volume
|
||
2. `nv8zo6j` — Whether defect or delivery issues correlate with the new supplier or weekend shift
|
||
3. `n3ftrpn` — How much of the complaint increase is attributable to changed CRM counting rules rather than actual performance
|
||
|
||
**Relevant graph content for intervention-fit check:**
|
||
- `n93k83a` (metric): "Proposed automated quality inspection system costing approximately £120,000"
|
||
- `nyi88el` (assumption): "Complaints are driven largely by late delivery logistics rather than manufacturing defects, meaning a £120k quality investment addresses the wrong bottleneck." — conditional hypothesis explicitly framing the problem-type-vs-intervention-fit uncertainty
|
||
- `ne49eqk` (assumption): Competing hypothesis that investment is premature due to volume/CRM artifact
|
||
- `nywshmd` (assumption): Opposing hypothesis that genuine issues justify inspection investment
|
||
|
||
### Intervention-Fit Check
|
||
|
||
| Check | Result | Evidence |
|
||
|---|---|---|
|
||
| Proposed £120k action represented | YES | Node `n93k83a` — "Proposed automated quality inspection system costing approximately £120,000" |
|
||
| Late-delivery problem represented | YES | Node `noiiqc5` and node `nyi88el` explicitly reference late delivery logistics |
|
||
| Product-defect problem represented | YES | Node `noiiqc5` references minor product defects; node `nyi88el` references manufacturing defects |
|
||
| Difference between delivery and defect mechanisms preserved | YES | Assumption `nyi88el` explicitly distinguishes "late delivery logistics" from "manufacturing defects" and frames the £120k investment as potentially addressing the wrong bottleneck |
|
||
| Any explicit uncertainty linking problem type to automation suitability | YES | Assumption `nyi88el`: "Complaints are driven largely by late delivery logistics rather than manufacturing defects, meaning a £120k quality investment addresses the wrong bottleneck." |
|
||
| Any question equivalent to "would this action address the actual problem?" | PARTIAL | The distinction is present as a conditional assumption, not a direct investigable question. A user would need to read and compare assumptions `nyi88el` vs `nywshmd` to surface the intervention-fit gap themselves. |
|
||
| Any conditional hypothesis about intervention fit | YES | Assumption `nyi88el` explicitly frames: IF complaints are delivery-driven THEN inspection addresses wrong bottleneck |
|
||
| Recommendation for/against investment | NO | All three competing assumptions carry status "provisional" and confidence "medium/high" — none resolved. The assumption is a conditional hypothesis, not a recommendation. |
|
||
| Priority language | NO | No "first/most important/primary/priority" language in graph or summary |
|
||
|
||
### Guardrail Evaluation
|
||
|
||
| Guardrail | Result |
|
||
|---|---|
|
||
| G1 invented evidence | NO |
|
||
| G2 unsupported causal conclusion | NO |
|
||
| G3 premature arithmetic conclusion | NO |
|
||
| G4 action recommendation | NO |
|
||
| G5 steering/prioritisation | NO |
|
||
| G6 speculative proliferation | NO (only 3 assumptions — minimal viable set: investment-justified, investment-premature, wrong-intervention) |
|
||
|
||
### Classification: B — INTERVENTION FIT ONLY IMPLICIT
|
||
|
||
**Why:** The £120k action and both problem types (late delivery / product defects) are explicitly represented. Crucially, assumption `nyi88el` is a distinct graph node that frames the conditionally uncertain relationship between problem mechanism and intervention suitability: "IF complaints are largely delivery-driven THEN a quality inspection investment addresses the wrong bottleneck." This preserves the semantic distinction as conditional uncertainty rather than resolving it.
|
||
|
||
However, it appears as an *assumption* (a hypothesised state of affairs) rather than an explicit unknown or question inviting investigation into whether the proposed action fits the actual problem. A user must infer that this assumption represents a meaningful intervention-fit gap rather than seeing it surfaced directly as a selectable investigative path. The distinction is present but implicit — recoverable through careful reading but not structurally highlighted as "here is what we do not yet know about whether this action fits."
|
||
|
||
### Comparison with Experiments 1 and 2
|
||
|
||
| | Experiment 1 | Experiment 2 | Experiment 3 |
|
||
|---|---|---|---|
|
||
| Intervention fit status | Lost | Partially preserved (implicit) | Partially preserved (conditional assumption) |
|
||
| £120k action represented | Yes | Yes | Yes (node `n93k83a`) |
|
||
| Problem-type distinction surfaced | No | Partially (D — compressed) | Yes (assumption `nyi88el`) |
|
||
| Explicit conditional fit hypothesis | No | No | Yes (assumption `nyi88el`) |
|
||
| Three competing assumptions presented | — | Competing assumptions on quality existence | Competing assumptions: investment justified / premature / wrong bottleneck |
|
||
|
||
**Experiment 3 exposes the distinction more explicitly than Experiments 1 and 2** — assumption `nyi88el` directly ties the mechanism (delivery vs defects) to the proposed action's potential mismatch. This is an improvement over previous runs where intervention fit was either lost or merely implicit in context.
|
||
|
||
**What three observations support:** (1) The late-delivery-vs-defects distinction maps onto intervention fit in this scenario and is now explicitly framed as conditional uncertainty; (2) All three competing assumptions are presented without resolution, preserving open inquiry; (3) The £120k action is not recommended or rejected — it sits alongside the fit hypothesis.
|
||
|
||
**What three observations do NOT establish:** (1) That intervention fit will be consistently preserved across different scenarios; (2) That conditional assumptions in the assumption node are as accessible to users as explicit unknowns; (3) That this improvement over Experiments 1/2 generalizes to other scenario types or model versions.
|
||
|
||
### Next Evidence Question
|
||
|
||
> Does surfacing intervention-fit uncertainty as a conditional assumption node provide equivalent user accessibility to surfacing it as an explicit unknown — and if not, what structural change would preserve the distinction without steering?
|
||
|
||
---
|
||
|
||
## v0.61 Experiment 4 — Intervention-Fit Accessibility in User-Facing UI
|
||
|
||
**Status: D — NOT USER-EXPOSED**
|
||
|
||
### Execution
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Branch | `feature/initial-decomposition-v0.61` |
|
||
| Starting HEAD | `42da768 docs(confidence-engine): record v0.61 intervention-fit experiment` |
|
||
| Source trace completed: YES — bounded to reasoning-workspace.jsx lines 1850–2216 |
|
||
| Browser navigation: Two persisted Investigations examined (IDs from Portfolio list) |
|
||
| Model/API calls: ZERO |
|
||
| State mutations: ZERO |
|
||
| Execution errors: NONE |
|
||
|
||
### Experiment 3 Investigation ID
|
||
|
||
The handoff documentation does NOT record the exact Investigation ID for Experiment 3. Two manufacturing-scenario Investigations exist in the portfolio with identical scenario text. Neither was definitively identified as Experiment 3's persisted state. However, both Investigations were examined and show structurally identical rendering behaviour regarding assumption nodes.
|
||
|
||
### SOURCE TRACE
|
||
|
||
**Provisional assumptions rendered:** YES — in TWO locations in reasoning-workspace.jsx:
|
||
1. Lines 2029–2048: Possible Interpretations section within `initial-proposed-findings` div, conditional on `possibleInterpretations.length > 0`
|
||
2. Lines 2187–2216: Persistent "Possible Interpretations" in workspace grid (row 4, full-width), conditional on `postAnalyseStatus !== "success"` AND `interpretationNodes.length > 0`
|
||
|
||
Filter for both sections: `n.kind === "assumption" && n.status !== "resolved" && !resolvedIds.has(n.id)`
|
||
|
||
**User-facing location:** Both render under `<h2>Possible Interpretations</h2>` heading. Cards styled with blue-100 border, blue-50/40 background, labeled "Plausible interpretation" in blue text.
|
||
|
||
**Assumptions interactive:** NO — cards are plain `<div>` elements (not buttons). The source comment at line 1865 and 2029 explicitly states "(informational, not investigable)". No onClick handler exists on assumption cards. Contrast with Open Question rendering: lines 1899–1910 use `<button>` elements with `startFocused(n.id)` for focused investigation.
|
||
|
||
**Open Questions source:** `(graph?.nodes || []).filter((n) => n.kind === "unknown" && !resolvedIds.has(n.id))` — unknown-kind nodes, not resolved. Rendered as clickable buttons that invoke `startFocused()` to begin investigation of that node.
|
||
|
||
**Existing assumption-to-investigation route:** NO. No code path converts an assumption node into a selectable investigative path without another semantic/model update. The filter separates assumptions (`kind === "assumption"`) from unknowns (`kind === "unknown"`) with different rendering roles and no cross-conversion mechanism.
|
||
|
||
### VISIBLE INVESTIGATION (two persisted Investigations examined)
|
||
|
||
**Investigation 1** (`53862863-...`):
|
||
- Current Understanding: "A manufacturing business owner is weighing a £120k automated quality inspection investment against the need to analyze complaint trends amid concurrent changes in production scale, supplier, shift scheduling, and tracking methodology."
|
||
- Open Questions (2): (a) complaint rate per unit denominator uncertainty; (b) defect rates by new supplier/shift vs baseline
|
||
|
||
**Investigation 2** (`d1b44afb-...`):
|
||
- Current Understanding: "A manufacturing business observes a 35% rise in complaints alongside a 40% production increase, coinciding with CRM, supplier, and scheduling changes, prompting a decision on a £120k quality inspection investment."
|
||
- Open Questions (7): CRM tagging threshold; defect rates by supplier/shift; complaint rate per unit overall; complaint rate by customer segments; complaint rate by production shift; complaint rate by process stages; complaint rate by critical defect categories
|
||
|
||
**Current Understanding intervention-fit meaning:** NO. Neither Current Understanding contains the intervention-fit distinction (i.e., "if complaints are primarily caused by late-delivery logistics rather than manufacturing defects, automated quality inspection may address the wrong problem"). The CU text describes what is being evaluated but never questions whether the proposed action fits the actual problem mechanism.
|
||
|
||
**Visible Open Questions:** 2 and 7 respectively. None directly address whether automated quality inspection matches the problem type (delivery vs defects).
|
||
|
||
**Intervention fit explicit Open Question:** NO — neither Investigation has an Open Question equivalent to "does this £120k investment address the actual problem, or is the problem primarily logistical/delivery-driven?"
|
||
|
||
**Directly selectable:** NO.
|
||
|
||
**Intervention-fit assumption visible:** NO. Zero Possible Interpretations sections rendered on either page.
|
||
|
||
**Visible section:** N/A — no Possible Interpretations section exists on either surface.
|
||
|
||
**Assumption selectable:** N/A (no assumptions rendered).
|
||
|
||
**Other visible investigative route:** NO — no visible control allows the user to investigate whether automated inspection addresses the actual problem mechanism. "Review current understanding" button leads to Report page, which is a derived summary not an investigative surface. Developer details panel shows model/validation metrics but not assumption content.
|
||
|
||
### THREE-LAYER RESULT
|
||
|
||
- Represented in canonical reasoning state: YES (Experiment 3 record confirmed intervention-fit as conditional assumption `nyi88el` in graph)
|
||
- Visible to user: NO — zero Possible Interpretations sections on either examined Investigation; Current Understanding does not contain the distinction; Open Questions do not address it
|
||
- Selectable/investigable by user: NO
|
||
|
||
### ACCESSIBILITY CLASSIFICATION
|
||
|
||
**D — NOT USER-EXPOSED**
|
||
|
||
Intervention-fit meaning exists in canonical reasoning state (Experiment 3 graph contained assumption `nyi88el`), but is absent from the normal user-facing Investigation surface. The Possible Interpretations section that would render assumption nodes exists as code but zero assumption-kind nodes meeting the filter criteria appear on either examined Investigation.
|
||
|
||
### Evidence
|
||
|
||
1. **Source trace:** reasoning-workspace.jsx lines 2029–2048 and 2187–2216 define Possible Interpretations rendering for assumption nodes — strictly informational, non-interactive `<div>` elements. Contrast with Open Questions at lines 1899–1910 which use clickable `<button>` elements invoking `startFocused()`.
|
||
|
||
2. **Source trace:** line 1865 comment explicitly states "(informational, not investigable)" — structural separation between unknown-kind (investigable) and assumption-kind (informational).
|
||
|
||
3. **Source trace:** Open Questions source is exclusively `n.kind === "unknown"` nodes (line 1856). No code exists that converts assumption meaning into an open question or makes assumptions investigable via UI interaction.
|
||
|
||
4. **Browser observation Investigation 1:** Current Understanding + Situation + 2 Open Questions visible. Zero Possible Interpretations sections.
|
||
|
||
5. **Browser observation Investigation 2:** Current Understanding + Situation + 7 Open Questions visible. Zero Possible Interpretations sections. Decomposition instability confirmed (7 questions vs 2 in Investigation 1).
|
||
|
||
6. **"What remains plausible" section** (lines 1983–1986): exists only inside `overviewState` conditional on the Report-generation surface, NOT as a persistent Investigation surface. Only appears when user clicks "Review current understanding."
|
||
|
||
7. **No cross-conversion path:** No code converts assumption nodes into selectable investigative paths. The kind distinction (assumption vs unknown) is structural, not cosmetic.
|
||
|
||
### PRODUCT-PRINCIPLE CHECK
|
||
|
||
- Engine selected what to investigate: NO (no steering in decomposition output)
|
||
- Engine prioritised a question: NO (zero prioritisation language observed)
|
||
- User retained choice among surfaced Open Questions: YES — all visible Open Questions are selectable buttons; user owns which to pursue first. However, the full set of uncertainty was not presented as investigable paths.
|
||
|
||
### NEXT EVIDENCE QUESTION
|
||
|
||
> Before deciding whether the fix is semantic (initial decomposition should produce explicit intervention-fit unknowns) or structural (existing assumption-kind meaning must be projected into selectable Open Question surface), what does the persisted graph data in Experiment 3's Investigation actually contain — and do any of those nodes qualify for Possible Interpretations rendering?
|
||
|
||
This question distinguishes whether the problem is:
|
||
- **Semantic:** decomposition never produced intervention-fit assumptions → no nodes to render
|
||
- **Structural:** assumption-kind nodes exist but are structurally barred from user selection
|
||
|
||
### GIT
|
||
|
||
**Documentation commit:** pending
|
||
**Working tree:** pending
|
||
|
||
### CHANGES
|
||
|
||
- Production changed: NO
|
||
- Tests changed: NO
|
||
- Prompt/schema changed: NO
|
||
- Canonical Investigation state changed: NO
|
||
|
||
---
|
||
|
||
Consult `docs/design-evolution/README.md` for progressive loading of product reasoning and provenance chronology; load the relevant chapter only when a specific historical question requires it.
|
||
|
||
The current handoff captures all operational facts needed to resume today. For historical decisions, experiment evidence, or methodology evolution — consult the design evolution archive index or task-context packs as appropriate.
|
||
|
||
## Provenance pointers
|
||
|
||
| Need | Read |
|
||
|---|---|
|
||
| Product evolution v0.51–v0.58 | `docs/design-evolution/README.md` (progressive loading) |
|
||
| Methodology / Return-to-Origin axioms | `docs/current-working-principles.md` §0 (A1–A12) |
|
||
| Architecture guardrails | `.claude/architecture-guardrails.md` |
|
||
| Task routing by work type | `docs/task-context-packs.md` |
|
||
| Broader architectural intent | `docs/architectural-principles.md` |
|
||
| Experiment history (specific) | `docs/design-evolution/README.md` → relevant chapter |
|