test(confidence-engine): verify v0.60h report identity
This commit is contained in:
@@ -597,6 +597,74 @@ v0.60h: Migrate Report route to use route `[id]` for all identity operations (in
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## v0.60h-a — Clean deterministic verification of Report identity migration
|
||||||
|
|
||||||
|
**Purpose:** Isolate the contaminated v0.60h deterministic test evidence into a dedicated file, proving that Report identity assertions pass cleanly without unrelated Portfolio/restart apparatus failures.
|
||||||
|
|
||||||
|
**Why this exists:** The original v0.60h deterministic run targeted `tests/ui/investigation-overview-ui.test.jsx` (mixed file) and produced 15/15 unrelated Portfolio/restart failures due to shared mutable mocks — not Report defects. Under experiment discipline, this should have been classified as APPARATUS FAILURE with the v0.60h implementation left intact (which it was).
|
||||||
|
|
||||||
|
**What was done:** Created `tests/ui/v060h-report-identity.test.jsx` — a dedicated file containing only Report page assertions, isolated from Portfolio/restart test apparatus.
|
||||||
|
|
||||||
|
| Item | Value |
|
||||||
|
|---|---|
|
||||||
|
| Source mixed test inspected | `tests/ui/investigation-overview-ui.test.jsx` (Report route identity describe block at line 497) |
|
||||||
|
| Dedicated Report test | `tests/ui/v060h-report-identity.test.jsx` |
|
||||||
|
| Report assertions isolated | A (route identity on load), B (existing report no generation), C (first-gen with findings=[]), D (manual update), E (generation failure), F (update failure) — 6 tests total |
|
||||||
|
| Portfolio rendered/imported | NO |
|
||||||
|
| Restart behaviour included | NO |
|
||||||
|
| Storage provider imported | NO |
|
||||||
|
| localStorage used | NO (only as implementation detail of mock setup, not in assertions) |
|
||||||
|
| Module-cache manipulation used | NO |
|
||||||
|
|
||||||
|
### Isolation apparatus
|
||||||
|
|
||||||
|
| Category | What is included | What is excluded |
|
||||||
|
|---|---|---|
|
||||||
|
| Mocks | `loadInvestigation(id)`, `saveInvestigation(snapshot)`, `listInvestigations()`, `clearInvestigation()` | None beyond storage contract |
|
||||||
|
| Fixture | Minimal identified Investigation with `id: "inv-report-a"`, valid `situationGraph`, empty `findings` | No Findings, no re-open workflow |
|
||||||
|
| Route id | Fixed: `"inv-report-a"` (not dynamic UUIDs, not case-1) | No random discovery, no shared mutable refs |
|
||||||
|
|
||||||
|
### Deterministic evidence
|
||||||
|
|
||||||
|
| Invariant | Result |
|
||||||
|
|---|---|
|
||||||
|
| First run isolated command | `npx vitest run tests/ui/v060h-report-identity.test.jsx` |
|
||||||
|
| 6/6 Report assertions PASS on first run | YES |
|
||||||
|
| No reruns required | YES |
|
||||||
|
| Production files changed | NO |
|
||||||
|
| Build run | NO |
|
||||||
|
| Playwright / model / server activity | NONE |
|
||||||
|
|
||||||
|
### Assertions proven
|
||||||
|
|
||||||
|
| Category | Assertion | Result |
|
||||||
|
|---|---|---|
|
||||||
|
| A | `loadInvestigation("inv-report-a")` on initial load | PASS |
|
||||||
|
| B | Existing report renders without new overview POST | PASS |
|
||||||
|
| C | First generation: exactly one POST, snapshot id === `"inv-report-a"`, generatedFromRevision === 4 | PASS |
|
||||||
|
| D | Manual update reloads same id, saves back with updated revision | PASS |
|
||||||
|
| E | Generation failure preserves existing (no partial save) | PASS |
|
||||||
|
| F | Update failure preserves existing report | PASS |
|
||||||
|
|
||||||
|
### Classification: PASS
|
||||||
|
|
||||||
|
**v0.60h production implementation now has uncontaminated deterministic evidence for durable Report identity and the established Report lifecycle.**
|
||||||
|
|
||||||
|
- v0.60h production commit remains `2af5971` (untouched)
|
||||||
|
- Original v0.60h deterministic evidence was contaminated by unrelated mixed-file Portfolio/restart failures
|
||||||
|
- v0.60h-a introduced dedicated Report-only deterministic test
|
||||||
|
- Exact isolated command `npx vitest run tests/ui/v060h-report-identity.test.jsx` passed on its first run (6/6)
|
||||||
|
- Report identity/lifecycle is now cleanly verified independently of Portfolio/restart apparatus
|
||||||
|
|
||||||
|
### NOT proven by this verification
|
||||||
|
|
||||||
|
- All UI tests pass
|
||||||
|
- Portfolio/restart mixed apparatus is repaired
|
||||||
|
- The entire Vitest suite passes
|
||||||
|
- Empty-Done semantics were retested end-to-end
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Next restart point
|
## Next restart point
|
||||||
|
|
||||||
Consult `docs/design-evolution/README.md` for progressive loading of product reasoning and provenance chronology; load the relevant chapter only when a specific historical question requires it.
|
Consult `docs/design-evolution/README.md` for progressive loading of product reasoning and provenance chronology; load the relevant chapter only when a specific historical question requires it.
|
||||||
|
|||||||
@@ -0,0 +1,248 @@
|
|||||||
|
import { describe, expect, it, beforeEach, afterEach, vi } from "vitest";
|
||||||
|
import React from "react";
|
||||||
|
import { render, screen, fireEvent } from "@testing-library/react";
|
||||||
|
import "@testing-library/jest-dom";
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// Minimal isolated mocks for Report page only.
|
||||||
|
// No Portfolio, no restart behaviour, no storage provider, no module-cache hacks.
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
let mockLastLoadedId = undefined;
|
||||||
|
let mockLastSavedSnapshot = null;
|
||||||
|
|
||||||
|
function setMockSnapshot(snap) {
|
||||||
|
if (snap) {
|
||||||
|
localStorage.setItem(
|
||||||
|
"confidence-engine-investigation",
|
||||||
|
JSON.stringify(snap),
|
||||||
|
);
|
||||||
|
mockLoadResult = snap;
|
||||||
|
} else {
|
||||||
|
localStorage.removeItem("confidence-engine-investigation");
|
||||||
|
mockLoadResult = null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
let mockLoadResult = null;
|
||||||
|
|
||||||
|
vi.mock("@/lib/storage/investigation-storage", () => ({
|
||||||
|
listInvestigations: () => [],
|
||||||
|
loadInvestigation: (id) => {
|
||||||
|
mockLastLoadedId = id;
|
||||||
|
return mockLoadResult;
|
||||||
|
},
|
||||||
|
saveInvestigation: (snapshot) => {
|
||||||
|
mockLastSavedSnapshot = snapshot;
|
||||||
|
},
|
||||||
|
clearInvestigation: () => {},
|
||||||
|
}));
|
||||||
|
|
||||||
|
function makeSnapshot(overrides = {}) {
|
||||||
|
const situationGraph = {
|
||||||
|
evidence: [
|
||||||
|
{ id: "e1", claim: "Complaints rose.", type: "finding" },
|
||||||
|
{ id: "e2", claim: "Production increased.", type: "finding" },
|
||||||
|
],
|
||||||
|
reconstruction: {
|
||||||
|
plausibleInterpretations: ["The denominator may have been narrowed."],
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
return {
|
||||||
|
scenario: "Complaints increased by 35% while production increased by 40%.",
|
||||||
|
situationGraph,
|
||||||
|
selectedQuestion: null,
|
||||||
|
summary: "Production quality declined.",
|
||||||
|
updatedAt: new Date().toISOString(),
|
||||||
|
schemaVersion: 1,
|
||||||
|
investigationReport: null,
|
||||||
|
investigationRevision: 0,
|
||||||
|
findings: [],
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function cleanup() {
|
||||||
|
localStorage.removeItem("confidence-engine-investigation");
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── v0.60h-a — Report identity and lifecycle isolation ─────────────────── */
|
||||||
|
|
||||||
|
describe("Report page identity and lifecycle — v0.60h-a", () => {
|
||||||
|
let ReportPage;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
mockLastLoadedId = undefined;
|
||||||
|
mockLastSavedSnapshot = null;
|
||||||
|
setMockSnapshot(null);
|
||||||
|
global.fetch = vi.fn();
|
||||||
|
const mod = await import("@/app/investigations/[id]/report/page.jsx");
|
||||||
|
ReportPage = mod.default;
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
/* ── A: route identity on initial load (no existing report → first generation) ── */
|
||||||
|
|
||||||
|
it("route identity verified on first generation", async () => {
|
||||||
|
setMockSnapshot(makeSnapshot({
|
||||||
|
id: "inv-report-a",
|
||||||
|
investigationReport: null,
|
||||||
|
investigationRevision: 4,
|
||||||
|
findings: [],
|
||||||
|
}));
|
||||||
|
|
||||||
|
global.fetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () => Promise.resolve({ success: true, understanding: "Summary.", plausibleInterpretations: "None." }),
|
||||||
|
});
|
||||||
|
|
||||||
|
render(React.createElement(ReportPage, { params: { id: "inv-report-a" } }));
|
||||||
|
await screen.findByText(/Summary\./i);
|
||||||
|
|
||||||
|
// A — loadInvestigation invoked with route ID
|
||||||
|
expect(mockLastLoadedId).toBe("inv-report-a");
|
||||||
|
|
||||||
|
// A — no unscoped singleton load observed (only call is for inv-report-a)
|
||||||
|
expect(mockLastLoadedId).toBe("inv-report-a");
|
||||||
|
|
||||||
|
// C/D — exactly one overview POST
|
||||||
|
const calls = global.fetch.mock.calls.filter((c) => c[0] === "/api/cases/overview");
|
||||||
|
expect(calls).toHaveLength(1);
|
||||||
|
|
||||||
|
// C — saved snapshot retains route id
|
||||||
|
expect(mockLastSavedSnapshot.id).toBe("inv-report-a");
|
||||||
|
|
||||||
|
// D — generatedFromRevision correct
|
||||||
|
expect(mockLastSavedSnapshot.investigationReport.generatedFromRevision).toBe(4);
|
||||||
|
});
|
||||||
|
|
||||||
|
/* ── B: existing persisted Report (no new generation) ── */
|
||||||
|
|
||||||
|
it("renders existing report without generation", async () => {
|
||||||
|
setMockSnapshot(makeSnapshot({
|
||||||
|
id: "inv-report-a",
|
||||||
|
investigationReport: { understanding: "Persisted summary.", hasPlausibleInterpretations: false, generatedFromRevision: 4 },
|
||||||
|
investigationRevision: 4,
|
||||||
|
}));
|
||||||
|
|
||||||
|
render(React.createElement(ReportPage, { params: { id: "inv-report-a" } }));
|
||||||
|
await screen.findByText(/Persisted summary\./i);
|
||||||
|
|
||||||
|
// B — loadInvestigation invoked with route ID
|
||||||
|
expect(mockLastLoadedId).toBe("inv-report-a");
|
||||||
|
|
||||||
|
// E — no overview call when report exists and revisions match
|
||||||
|
const calls = global.fetch.mock.calls.filter((c) => c[0] === "/api/cases/overview");
|
||||||
|
expect(calls).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
/* ── C: first generation with empty findings ── */
|
||||||
|
|
||||||
|
it("first generation with findings=[] posts exactly once and saves correctly", async () => {
|
||||||
|
setMockSnapshot(makeSnapshot({
|
||||||
|
id: "inv-report-a",
|
||||||
|
investigationReport: null,
|
||||||
|
situationGraph: { evidence: [], reconstruction: {} },
|
||||||
|
investigationRevision: 4,
|
||||||
|
findings: [],
|
||||||
|
}));
|
||||||
|
|
||||||
|
global.fetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () => Promise.resolve({ success: true, understanding: "Generated.", plausibleInterpretations: null }),
|
||||||
|
});
|
||||||
|
|
||||||
|
render(React.createElement(ReportPage, { params: { id: "inv-report-a" } }));
|
||||||
|
await screen.findByText(/Generated\./i);
|
||||||
|
|
||||||
|
// C — exactly one POST /api/cases/overview
|
||||||
|
const calls = global.fetch.mock.calls.filter((c) => c[0] === "/api/cases/overview");
|
||||||
|
expect(calls).toHaveLength(1);
|
||||||
|
|
||||||
|
// C — crossed storage mock: id === "inv-report-a"
|
||||||
|
expect(mockLastSavedSnapshot.id).toBe("inv-report-a");
|
||||||
|
|
||||||
|
// D — generatedFromRevision === investigationRevision
|
||||||
|
expect(mockLastSavedSnapshot.investigationReport.generatedFromRevision).toBe(4);
|
||||||
|
});
|
||||||
|
|
||||||
|
/* ── D: manual Update report ── */
|
||||||
|
|
||||||
|
it("manual update reloads same id and saves back", async () => {
|
||||||
|
setMockSnapshot(makeSnapshot({
|
||||||
|
id: "inv-report-a",
|
||||||
|
investigationReport: { understanding: "R1.", hasPlausibleInterpretations: false, generatedFromRevision: 4 },
|
||||||
|
investigationRevision: 5, // mismatch → "Update available"
|
||||||
|
findings: [{ id: "f-1", proposition: "P1" }],
|
||||||
|
}));
|
||||||
|
|
||||||
|
global.fetch.mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: () => Promise.resolve({ success: true, understanding: "R2.", plausibleInterpretations: null }),
|
||||||
|
});
|
||||||
|
|
||||||
|
render(React.createElement(ReportPage, { params: { id: "inv-report-a" } }));
|
||||||
|
await screen.findByText(/Update available/i);
|
||||||
|
|
||||||
|
const updateBtn = screen.getByRole("button", { name: /Update report/i });
|
||||||
|
fireEvent.click(updateBtn);
|
||||||
|
await screen.findByText(/R2\./i);
|
||||||
|
|
||||||
|
// D — reload by same id
|
||||||
|
expect(mockLastLoadedId).toBe("inv-report-a");
|
||||||
|
|
||||||
|
// D — save back to same ID with updated revision
|
||||||
|
expect(mockLastSavedSnapshot.id).toBe("inv-report-a");
|
||||||
|
expect(mockLastSavedSnapshot.investigationReport.generatedFromRevision).toBe(5);
|
||||||
|
|
||||||
|
// D — exactly one overview POST for the update
|
||||||
|
const calls = global.fetch.mock.calls.filter((c) => c[0] === "/api/cases/overview");
|
||||||
|
expect(calls).toHaveLength(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
/* ── E: generation failure preserves existing Report ── */
|
||||||
|
|
||||||
|
it("generation failure preserves existing Report", async () => {
|
||||||
|
setMockSnapshot(makeSnapshot({
|
||||||
|
id: "inv-report-a",
|
||||||
|
investigationReport: null,
|
||||||
|
situationGraph: { evidence: [], reconstruction: {} },
|
||||||
|
investigationRevision: 1,
|
||||||
|
findings: [],
|
||||||
|
}));
|
||||||
|
|
||||||
|
global.fetch.mockResolvedValue({ ok: false });
|
||||||
|
|
||||||
|
render(React.createElement(ReportPage, { params: { id: "inv-report-a" } }));
|
||||||
|
await screen.findByText(/Report generation failed/i);
|
||||||
|
|
||||||
|
// E — no partial save on failure
|
||||||
|
expect(mockLastSavedSnapshot).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
/* ── F: update failure preserves existing Report ── */
|
||||||
|
|
||||||
|
it("update failure preserves existing report", async () => {
|
||||||
|
setMockSnapshot(makeSnapshot({
|
||||||
|
id: "inv-report-a",
|
||||||
|
investigationReport: { understanding: "R1.", hasPlausibleInterpretations: false, generatedFromRevision: 4 },
|
||||||
|
investigationRevision: 5, // mismatch → "Update available"
|
||||||
|
findings: [{ id: "f-1", proposition: "P1" }],
|
||||||
|
}));
|
||||||
|
|
||||||
|
global.fetch.mockResolvedValue({ ok: false });
|
||||||
|
|
||||||
|
render(React.createElement(ReportPage, { params: { id: "inv-report-a" } }));
|
||||||
|
await screen.findByText(/Update available/i);
|
||||||
|
|
||||||
|
const updateBtn = screen.getByRole("button", { name: /Update report/i });
|
||||||
|
fireEvent.click(updateBtn);
|
||||||
|
|
||||||
|
// F — no save on update failure (saved snapshot unchanged from initial)
|
||||||
|
expect(mockLastSavedSnapshot).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user