docs(confidence-engine): restore v0.61 decomposition objective
This commit is contained in:
+64
-4
@@ -795,9 +795,11 @@ Zero Open Questions ("You've now worked through all of the questions we surfaced
|
||||
|
||||
Report is the established Investigation culmination. No further product boundary is selected.
|
||||
|
||||
## v0.61 — Direct Initial-Decomposition Apparatus
|
||||
## v0.61 — Repeated-Run Initial Decomposition Stability
|
||||
|
||||
**Status:** v0.61 Experiment 1 produced B ("USEFUL BUT MATERIAL UNCERTAINTY LOST"). This apparatus task was executed to establish direct initial-decomposition invocation for subsequent semantic experiments.
|
||||
**Objective:** How stable is the initial semantic decomposition of the same fixed scenario across repeated runs using the same model, promptVersion and production API path?
|
||||
|
||||
**Status:** Experiments 1–3 produced evidence that different visible/semantic decomposition shapes arise for the same fixed scenario. The active next step is a formalised repeated-same-input experiment to quantify variation in materially important uncertainty coverage, question structure and interpretation.
|
||||
|
||||
### Canonical production seam
|
||||
|
||||
@@ -912,6 +914,43 @@ node scripts/start-case-experiment-helper.cjs --file scenario.json
|
||||
- **Playwright NOT default** for decomposition-only semantic experiments (apparatus reaches production reasoning path via direct import or HTTP)
|
||||
- **Playwright REMAINS required** when the experiment concerns visible/browser behaviour, UI state transitions, or localStorage hydration
|
||||
|
||||
### Repeated-same-input experiment — active design
|
||||
|
||||
**Input (fixed scenario):**
|
||||
```
|
||||
I run a small manufacturing business. Customer complaints have risen by 35% over the last six months, while production volume increased by 40%.
|
||||
|
||||
Most complaints mention late delivery or minor product defects, but our complaint categories changed when we introduced a new CRM tagging system three months ago. During the same period we also changed one supplier and introduced a weekend production shift.
|
||||
|
||||
I am deciding whether to spend about £120,000 on automated quality inspection now or wait until we understand whether there is actually a quality problem.
|
||||
|
||||
I do not yet know the complaint rate per unit, whether defect rates differ by shift or supplier, or whether the new tagging system changed what gets counted as a complaint.
|
||||
```
|
||||
|
||||
**Execution:** same /api/cases/start route · same promptVersion = v0.2 · same configured model · multiple independent runs · zero production changes.
|
||||
|
||||
**What is compared across runs (semantic structure, not wording):**
|
||||
- Open Question count range
|
||||
- unknown-node count range
|
||||
- assumption-node count range
|
||||
- material uncertainty channels present/absent: normalised complaint incidence · CRM measurement/comparability · late-delivery vs product-defect distinction · supplier/shift/source ambiguity · intervention fit
|
||||
- redundant questions
|
||||
- speculative subdivisions
|
||||
- premature interpretations
|
||||
- action implications / steering / prioritisation
|
||||
|
||||
**Raw-response policy:** responses only need to survive long enough for within-experiment comparison. Temporary files or in-memory capture are acceptable. Immutable per-run fixtures are NOT required.
|
||||
|
||||
**Run count:** to be fixed in the next experiment prompt.
|
||||
|
||||
**Active v0.61 question:**
|
||||
> For the same fixed scenario under the same current production configuration, how much does the initial decomposition vary in materially important uncertainty coverage, question structure and interpretation?
|
||||
|
||||
**Corresponding next evidence question:**
|
||||
> Across repeated independent decompositions of the same scenario, which material uncertainty channels are consistently preserved, intermittently omitted, or replaced by speculative/redundant structure?
|
||||
|
||||
**What is NOT the current v0.61 objective:** whether assumption-kind nodes render under Possible Interpretations · whether intervention-fit is user-selectable in the UI · whether experiments can be projected into persisted Investigations · whether raw API responses should become UI fixtures · downstream question answering · Current Understanding progression after answers · persistence · localStorage · new scenarios · prompt changes · schema changes · model changes
|
||||
|
||||
## v0.61.1 — Start-Case Experiment Helper Apparatus Verification
|
||||
|
||||
**Status: PASSED (18/18, first run, zero reruns)**
|
||||
@@ -1269,6 +1308,17 @@ However, it appears as an *assumption* (a hypothesised state of affairs) rather
|
||||
|
||||
**Status: BLOCKED — exact Experiment 3 persisted fixture unavailable**
|
||||
|
||||
### PAUSED / NOT CURRENT v0.61 OBJECTIVE
|
||||
|
||||
The intervention-fit accessibility / UI-projection line of inquiry from this experiment is **NOT the active next boundary** for v0.61. The active direction is repeated-same-input initial-decomposition stability (see the "active design" section at the top of §v0.61).
|
||||
|
||||
Valid source-level findings recorded in this experiment remain as historical context:
|
||||
- unknown nodes feed Open Questions
|
||||
- assumption nodes can feed Possible Interpretations
|
||||
- assumption-kind meaning and user-selectable unknown-kind meaning are structurally different surfaces
|
||||
|
||||
Any valid findings from this thread are preserved but the projection/accessibility line of inquiry is marked **PAUSED / NOT CURRENT v0.61 OBJECTIVE**.
|
||||
|
||||
### Execution
|
||||
|
||||
| Item | Value |
|
||||
@@ -1380,11 +1430,11 @@ Experiment 4 did not resolve this. The uncertainty is recorded; it is not chosen
|
||||
- Engine prioritised a question: NO (zero prioritisation language observed)
|
||||
- User retained choice among surfaced Open Questions: YES — for whatever Open Questions the specific Investigation happens to contain, they are selectable buttons. However, the full set of uncertainty from Experiment 3 was not demonstrated as present in any examined Investigation.
|
||||
|
||||
### NEXT EVIDENCE QUESTION
|
||||
### NEXT EVIDENCE QUESTION (HISTORICAL — NOT CURRENT)
|
||||
|
||||
> Given an exact captured `/api/cases/start` result containing an intervention-fit assumption, what is the smallest valid way to observe how that exact canonical graph projects into the existing Investigation UI without making another semantic/model call?
|
||||
|
||||
This is an apparatus/evidence question. No solution design included in this task.
|
||||
This question remains recorded as historical apparatus evidence. It is NOT selected as the active next direction for v0.61. The active next direction is the repeated-same-input decomposition stability experiment described at the top of §v0.61.
|
||||
|
||||
### GIT
|
||||
|
||||
@@ -1400,6 +1450,16 @@ This is an apparatus/evidence question. No solution design included in this task
|
||||
|
||||
---
|
||||
|
||||
## v0.61 active restart point
|
||||
|
||||
The current handoff direction is the repeated-same-input decomposition stability experiment described in the **active design** section at the top of §v0.61.
|
||||
|
||||
**Next task:** execute a formalised repeated-run experiment using the fixed manufacturing scenario via POST /api/cases/start, comparing semantic structure across runs.
|
||||
|
||||
**Previous active direction (drifted, now corrected):** intervention-fit accessibility in UI projection · assumption-kind node rendering · persistence/localStorage for experiment fixtures · downstream question-answering. These are **NOT current v0.61 objectives**.
|
||||
|
||||
---
|
||||
|
||||
Consult `docs/design-evolution/README.md` for progressive loading of product reasoning and provenance chronology; load the relevant chapter only when a specific historical question requires it.
|
||||
|
||||
The current handoff captures all operational facts needed to resume today. For historical decisions, experiment evidence, or methodology evolution — consult the design evolution archive index or task-context packs as appropriate.
|
||||
|
||||
Reference in New Issue
Block a user