test(confidence-engine): add direct initial decomposition apparatus
Establishes reusable apparatus for asking: given scenario text X, what structured initial decomposition does current production path produce? - Direct curl/Postman via existing /api/cases/start route (no new API) - Thin CJS helper at scripts/start-case-experiment-helper.cjs for Claude experiments (imports startCase directly, zero code duplication) - Zero-live-call verification: all four seam checks confirmed by existing tests (cases-start-route.test.js, start-case-summary.test.js) - No browser state, no persistence mutation, no Investigation ID required by the route itself Files: + scripts/start-case-experiment-helper.cjs (new helper script) M docs/current-handoff.md (§v0.61 apparatus documentation)
This commit is contained in:
@@ -795,6 +795,123 @@ Zero Open Questions ("You've now worked through all of the questions we surfaced
|
||||
|
||||
Report is the established Investigation culmination. No further product boundary is selected.
|
||||
|
||||
## v0.61 — Direct Initial-Decomposition Apparatus
|
||||
|
||||
**Status:** v0.61 Experiment 1 produced B ("USEFUL BUT MATERIAL UNCERTAINTY LOST"). This apparatus task was executed to establish direct initial-decomposition invocation for subsequent semantic experiments.
|
||||
|
||||
### Canonical production seam
|
||||
|
||||
The existing route `/api/cases/start` **is directly suitable** for curl/Postman/Claude experimentation:
|
||||
|
||||
```
|
||||
┌───────────┐ POST /api/cases/start ┌──────────────┐
|
||||
│ Scenario │ ───────────────────────────► │ startCase() │
|
||||
│ text (req) │ { scenario, promptVersion? } │ │
|
||||
│ │ │ analyseScenario │
|
||||
│ │ │ buildInitialGraph │
|
||||
│ │ │ selectUnknown │
|
||||
└────────────┘ └──────────────┘
|
||||
```
|
||||
|
||||
**No browser state required. No Investigation ID required by the route itself.** The route accepts `scenario` string directly and invokes the full production reasoning path (analyseScenario → buildInitialGraph → determineGraphBackedQuestion).
|
||||
|
||||
### Direct curl/Postman contract (Rob)
|
||||
|
||||
**Method:** POST
|
||||
**URL:** `http://localhost:3000/api/cases/start`
|
||||
**Content-Type:** `application/json`
|
||||
|
||||
**Request body schema:**
|
||||
```json
|
||||
{
|
||||
"scenario": "<your scenario text here>",
|
||||
"promptVersion": "v0.2"
|
||||
}
|
||||
```
|
||||
|
||||
- `scenario` (required): string, 1–10000 characters
|
||||
- `promptVersion` (optional): `"v0.1"` or `"v0.2"` (defaults to `"v0.2"`)
|
||||
|
||||
**Response shape (success):**
|
||||
```json
|
||||
{
|
||||
"success": true,
|
||||
"summary": "<reconstruction summary>",
|
||||
"situationGraph": { /* full graph with nodes/edges/reasoningState */ },
|
||||
"selectedQuestion": { "id": "...", "question": "...", "reasoningPattern": "..." },
|
||||
"diagnostics": { /* decompositionApplied, questionComplexityAssessment, etc. */ },
|
||||
"assessment": { "phase": "...", "progress": "..." },
|
||||
"modelName": "qwen-claude:latest",
|
||||
"responseDurationMs": 3210,
|
||||
"validationStatus": "valid",
|
||||
"promptVersion": "v0.2"
|
||||
}
|
||||
```
|
||||
|
||||
**Response shape (failure):**
|
||||
```json
|
||||
{
|
||||
"success": false,
|
||||
"error": "<message>",
|
||||
"statusCode": 400|500|502,
|
||||
"validationErrors": [...],
|
||||
"analysisErrors": [...],
|
||||
"diagnostics": {...}
|
||||
}
|
||||
```
|
||||
|
||||
### Claude apparatus
|
||||
|
||||
**Needed:** YES — a thin CJS helper exists for repeated controlled experiments.
|
||||
|
||||
**Path:** `scripts/start-case-experiment-helper.cjs`
|
||||
**Command:**
|
||||
```bash
|
||||
node scripts/start-case-experiment-helper.cjs "<scenario text>"
|
||||
```
|
||||
or
|
||||
```bash
|
||||
node scripts/start-case-experiment-helper.cjs --file scenario.json
|
||||
```
|
||||
|
||||
**Input:** scenario string (positional arg or JSON file with `{ "scenario": "..." }`)
|
||||
**Output:** structured JSON to stdout (success fields + endToEndElapsedMs)
|
||||
**Retries:** NO — single call, no retry logic
|
||||
**Canonical production logic duplicated:** NO — imports `startCase` from `lib/graph/orchestrator.js`, exercises identical code path
|
||||
|
||||
### Deterministic zero-live-call verification
|
||||
|
||||
| Check | Source | Result |
|
||||
|---|---|---|
|
||||
| Input reaches startCase seam | `tests/app/api/cases-start-route.test.js:15` | PASS (mocked analyseScenario verified) |
|
||||
| Output passed through correctly | `tests/start-case-summary.test.js:81` | PASS (exact summary field round-trip) |
|
||||
| Execution failure returns 400/5xx | `tests/app/api/cases-start-route.test.js:55,79` | PASS |
|
||||
| Malformed JSON returns 500 | `tests/app/api/cases-start-route.test.js:100` | PASS |
|
||||
| No retry occurs | source inspection (single await) | CONFIRMED |
|
||||
| Schema validation present | `lib/graph/schema.js:202-205` | Zod enforced |
|
||||
|
||||
**Live model calls:** ZERO
|
||||
**Build:** NOT required (scripts/test only, no production code changes)
|
||||
**Playwright:** NOT used (decomposition-only experiments do not require browser instrumentation)
|
||||
|
||||
### Browser state investigation
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| Does the route mutate persistence? | NO — persistence is handled by ScenarioForm caller AFTER receiving result |
|
||||
| Does it require an Investigation ID? | NO — route accepts scenario text directly; id comes from UI caller's state |
|
||||
| Does it depend on browser localStorage? | NO — pure HTTP JSON exchange |
|
||||
| Does it require any browser-only state? | NO |
|
||||
|
||||
### Decision: direct curl/Postman suitable = YES
|
||||
|
||||
**Why:** The `/api/cases/start` route is a thin layer (19 lines) over `startCase()` that validates input via Zod, calls the production function, and returns structured results. No browser state, no persistence side effects, no unrelated mutations. Identical behaviour to what ScenarioForm exercises in production.
|
||||
|
||||
### Playwright posture for v0.61 experiments
|
||||
|
||||
- **Playwright NOT default** for decomposition-only semantic experiments (apparatus reaches production reasoning path via direct import or HTTP)
|
||||
- **Playwright REMAINS required** when the experiment concerns visible/browser behaviour, UI state transitions, or localStorage hydration
|
||||
|
||||
## Next restart point
|
||||
|
||||
> v0.60 is complete. Report is established as the culmination of an Investigation. No next product boundary is currently selected. Begin the next session by choosing the next unresolved user/product reasoning boundary from current product behaviour and founding principles, rather than continuing storage migration or assuming an old backlog item is next.
|
||||
|
||||
Reference in New Issue
Block a user