From 65c5ded9ab4ab410535b41c8673b7189d6fdd0cb Mon Sep 17 00:00:00 2001 From: robbond Date: Fri, 4 Sep 2026 17:17:53 +0100 Subject: [PATCH] docs(confidence-engine): restore v0.61 decomposition objective --- docs/current-handoff.md | 68 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 4 deletions(-) diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 6559c52..f4d38b0 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -795,9 +795,11 @@ Zero Open Questions ("You've now worked through all of the questions we surfaced Report is the established Investigation culmination. No further product boundary is selected. -## v0.61 — Direct Initial-Decomposition Apparatus +## v0.61 — Repeated-Run Initial Decomposition Stability -**Status:** v0.61 Experiment 1 produced B ("USEFUL BUT MATERIAL UNCERTAINTY LOST"). This apparatus task was executed to establish direct initial-decomposition invocation for subsequent semantic experiments. +**Objective:** How stable is the initial semantic decomposition of the same fixed scenario across repeated runs using the same model, promptVersion and production API path? + +**Status:** Experiments 1–3 produced evidence that different visible/semantic decomposition shapes arise for the same fixed scenario. The active next step is a formalised repeated-same-input experiment to quantify variation in materially important uncertainty coverage, question structure and interpretation. ### Canonical production seam @@ -912,6 +914,43 @@ node scripts/start-case-experiment-helper.cjs --file scenario.json - **Playwright NOT default** for decomposition-only semantic experiments (apparatus reaches production reasoning path via direct import or HTTP) - **Playwright REMAINS required** when the experiment concerns visible/browser behaviour, UI state transitions, or localStorage hydration +### Repeated-same-input experiment — active design + +**Input (fixed scenario):** +``` +I run a small manufacturing business. Customer complaints have risen by 35% over the last six months, while production volume increased by 40%. + +Most complaints mention late delivery or minor product defects, but our complaint categories changed when we introduced a new CRM tagging system three months ago. During the same period we also changed one supplier and introduced a weekend production shift. + +I am deciding whether to spend about £120,000 on automated quality inspection now or wait until we understand whether there is actually a quality problem. + +I do not yet know the complaint rate per unit, whether defect rates differ by shift or supplier, or whether the new tagging system changed what gets counted as a complaint. +``` + +**Execution:** same /api/cases/start route · same promptVersion = v0.2 · same configured model · multiple independent runs · zero production changes. + +**What is compared across runs (semantic structure, not wording):** +- Open Question count range +- unknown-node count range +- assumption-node count range +- material uncertainty channels present/absent: normalised complaint incidence · CRM measurement/comparability · late-delivery vs product-defect distinction · supplier/shift/source ambiguity · intervention fit +- redundant questions +- speculative subdivisions +- premature interpretations +- action implications / steering / prioritisation + +**Raw-response policy:** responses only need to survive long enough for within-experiment comparison. Temporary files or in-memory capture are acceptable. Immutable per-run fixtures are NOT required. + +**Run count:** to be fixed in the next experiment prompt. + +**Active v0.61 question:** +> For the same fixed scenario under the same current production configuration, how much does the initial decomposition vary in materially important uncertainty coverage, question structure and interpretation? + +**Corresponding next evidence question:** +> Across repeated independent decompositions of the same scenario, which material uncertainty channels are consistently preserved, intermittently omitted, or replaced by speculative/redundant structure? + +**What is NOT the current v0.61 objective:** whether assumption-kind nodes render under Possible Interpretations · whether intervention-fit is user-selectable in the UI · whether experiments can be projected into persisted Investigations · whether raw API responses should become UI fixtures · downstream question answering · Current Understanding progression after answers · persistence · localStorage · new scenarios · prompt changes · schema changes · model changes + ## v0.61.1 — Start-Case Experiment Helper Apparatus Verification **Status: PASSED (18/18, first run, zero reruns)** @@ -1269,6 +1308,17 @@ However, it appears as an *assumption* (a hypothesised state of affairs) rather **Status: BLOCKED — exact Experiment 3 persisted fixture unavailable** +### PAUSED / NOT CURRENT v0.61 OBJECTIVE + +The intervention-fit accessibility / UI-projection line of inquiry from this experiment is **NOT the active next boundary** for v0.61. The active direction is repeated-same-input initial-decomposition stability (see the "active design" section at the top of §v0.61). + +Valid source-level findings recorded in this experiment remain as historical context: +- unknown nodes feed Open Questions +- assumption nodes can feed Possible Interpretations +- assumption-kind meaning and user-selectable unknown-kind meaning are structurally different surfaces + +Any valid findings from this thread are preserved but the projection/accessibility line of inquiry is marked **PAUSED / NOT CURRENT v0.61 OBJECTIVE**. + ### Execution | Item | Value | @@ -1380,11 +1430,11 @@ Experiment 4 did not resolve this. The uncertainty is recorded; it is not chosen - Engine prioritised a question: NO (zero prioritisation language observed) - User retained choice among surfaced Open Questions: YES — for whatever Open Questions the specific Investigation happens to contain, they are selectable buttons. However, the full set of uncertainty from Experiment 3 was not demonstrated as present in any examined Investigation. -### NEXT EVIDENCE QUESTION +### NEXT EVIDENCE QUESTION (HISTORICAL — NOT CURRENT) > Given an exact captured `/api/cases/start` result containing an intervention-fit assumption, what is the smallest valid way to observe how that exact canonical graph projects into the existing Investigation UI without making another semantic/model call? -This is an apparatus/evidence question. No solution design included in this task. +This question remains recorded as historical apparatus evidence. It is NOT selected as the active next direction for v0.61. The active next direction is the repeated-same-input decomposition stability experiment described at the top of §v0.61. ### GIT @@ -1400,6 +1450,16 @@ This is an apparatus/evidence question. No solution design included in this task --- +## v0.61 active restart point + +The current handoff direction is the repeated-same-input decomposition stability experiment described in the **active design** section at the top of §v0.61. + +**Next task:** execute a formalised repeated-run experiment using the fixed manufacturing scenario via POST /api/cases/start, comparing semantic structure across runs. + +**Previous active direction (drifted, now corrected):** intervention-fit accessibility in UI projection · assumption-kind node rendering · persistence/localStorage for experiment fixtures · downstream question-answering. These are **NOT current v0.61 objectives**. + +--- + Consult `docs/design-evolution/README.md` for progressive loading of product reasoning and provenance chronology; load the relevant chapter only when a specific historical question requires it. The current handoff captures all operational facts needed to resume today. For historical decisions, experiment evidence, or methodology evolution — consult the design evolution archive index or task-context packs as appropriate.