docs(confidence-engine): checkpoint design evolution archive tranche two
This commit is contained in:
@@ -65,9 +65,40 @@ Experiments 23–25B and their closeout/return-to-work material.
|
||||
Fidelity:
|
||||
Exact contiguous copy.
|
||||
|
||||
## Tranche 2
|
||||
|
||||
### Chapter 5
|
||||
Path:
|
||||
docs/archive/experiments/vol-1-chapters/ch5/context-inventory-routing-and-handoff-infrastructure.md
|
||||
|
||||
Original source:
|
||||
docs/design-evolution-log.md lines 1502–2052
|
||||
|
||||
Contents:
|
||||
Experiments 26–34 — context routing inventory and handoff infrastructure design.
|
||||
|
||||
Fidelity:
|
||||
Exact contiguous copy.
|
||||
|
||||
### Chapter 6
|
||||
Path:
|
||||
docs/archive/experiments/vol-1-chapters/ch6/handoff-validation-behavior-selection-and-assessor-audit.md
|
||||
|
||||
Original source:
|
||||
docs/design-evolution-log.md lines 2053–2503
|
||||
|
||||
Contents:
|
||||
Experiments 35–41 and Experiment 41 Conclusion — handoff validation, behavior selection, and assessor audit.
|
||||
|
||||
Fidelity:
|
||||
Exact contiguous copy.
|
||||
|
||||
Tranches 1 and 2 have now been extracted.
|
||||
The original monolithic log remains intact and authoritative while extraction is incomplete.
|
||||
|
||||
## Refactor status
|
||||
|
||||
Only tranche 1 has been extracted.
|
||||
Only tranches 1 and 2 have been extracted.
|
||||
|
||||
Do not remove these ranges from `docs/design-evolution-log.md` yet.
|
||||
|
||||
|
||||
+551
@@ -0,0 +1,551 @@
|
||||
|
||||
## Experiment 26 — Inventory Project Knowledge and Context Needs
|
||||
|
||||
**Status:** Pending review
|
||||
|
||||
### Hypothesis
|
||||
|
||||
The existing documentation can be separated into clear roles: current working context, task-specific references, historical evidence, and gaps to review. A simple inventory and loading map may reduce context without losing important knowledge.
|
||||
|
||||
### Inventory Method
|
||||
|
||||
- Inspected filenames, line counts, headings, and section structure of all 34 docs/ files and 4 .claude/ markdown files (38 documentation files total).
|
||||
- Did not print full contents of large documents (>100 lines).
|
||||
- Inspected headings via `grep`, file sizes via `wc -l`, and key sections (Experiments 23–25B, Return-to-Work notes) via targeted `sed`.
|
||||
- Created one inventory document: `docs/project-knowledge-inventory.md`.
|
||||
|
||||
### Proposed Minimum Context
|
||||
|
||||
For routine Confidence Engine work, Claude should normally load only:
|
||||
|
||||
1. `.claude/project-context.md` — entire file (product direction, current stage)
|
||||
2. `.claude/architecture-guardrails.md` — entire file (hard boundaries, invariants)
|
||||
3. `docs/design-evolution-log.md` — lines 1–90, 824–838, 889–910, 1218–1520 (phase overview + Experiments 16–25B history)
|
||||
4. `docs/03_Confidence_Engine_Language_Guide.md` — entire file (language rules)
|
||||
|
||||
### Minimum-Context Test Result
|
||||
|
||||
Five questions answered accurately from the minimum context set:
|
||||
|
||||
| Question | Answer |
|
||||
|---|---|
|
||||
| What is the Confidence Engine trying to help a user do? | Help people take justified next steps when a problem feels too big to know where to start — by breaking complexity into small pieces, building a reasoning graph, asking one question at a time, and updating until confidence is sufficient or remaining uncertainty is clear. |
|
||||
| What is the current engine experiment status? | Paused. Experiments concluded with Exp 25B (scope-aware condition status). Current focus: UX presentation improvements (v0.7 user workspace). |
|
||||
| What did Experiment 25B establish? | Scope-aware evidence-condition comparison: present-state evidence does not settle future-feasibility conditions. All 39+ tests pass across Exps 23–25B. |
|
||||
| What remains provisional? | Phrase-based scope detection (Exp 25A/B); passive classifiers not yet integrated into active reasoning; next-question selection pipeline needs re-evaluation. |
|
||||
| What work is intentionally paused? | All engine experiments beyond Exp 25B. No reasoning architecture changes. Current work: UX usability, presentation clarity, loading feedback. |
|
||||
|
||||
### Missing Context Discovered
|
||||
|
||||
None. The five questions were answered accurately from the minimum context set. No additional document was required.
|
||||
|
||||
### Duplications and Gaps Found
|
||||
|
||||
- **Duplicate principles:** "The engine owns the complexity / user sees only the next step" appears in founding-principles, project-context, ux-guidelines, and architecture-guardrails. Consider consolidating or cross-referencing.
|
||||
- **Buried current state:** Experiment 25B sits at line ~1,483 of a 1,542-line log. A developer must scroll past 14+ phases to find active status.
|
||||
- **No short entrypoint for active engine state:** project-context.md covers product direction but not experiment details (Exps 23–25B).
|
||||
- **Potentially stale architecture description:** v0.6-reasoning-architecture.md does not reference later additions from Experiments 15–25B.
|
||||
|
||||
### Status
|
||||
|
||||
Pending review. Nothing has been archived, moved, or deleted. The proposed context-loading plan is documented in `docs/project-knowledge-inventory.md`.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 27 — Create a Short Current-State Entry Point
|
||||
|
||||
**Status:** Pending Rob's review
|
||||
|
||||
### Hypothesis
|
||||
|
||||
A concise current-state document can replace the large experiment-log section as the normal starting point for future work. The full design history should remain available as evidence, but should not be compulsory reading.
|
||||
|
||||
### Documents Used
|
||||
|
||||
| Document | Sections |
|
||||
|---|---|
|
||||
| `docs/project-knowledge-inventory.md` | Current Working Context; Gaps and Duplications to Review; Minimum Context Test Result |
|
||||
| `.claude/project-context.md` | Entire file (~102 lines) |
|
||||
| `.claude/architecture-guardrails.md` | Entire file (~77 lines) |
|
||||
| `docs/design-evolution-log.md` | Experiment 26 only; Return-to-Work Note after Experiment 25B (lines 1483–1501) |
|
||||
| `docs/03_Confidence_Engine_Language_Guide.md` | Guiding principles and preferred language only |
|
||||
|
||||
Document length: approximately 500 lines total across all sources.
|
||||
|
||||
### Created File
|
||||
|
||||
`docs/current-project-state.md` — 252 lines. Organised by what is true now, not chronologically. Contains eight sections: What the Engine Is, Current Product Experience, Current Engine Capabilities (active vs passive), What Experiments 20–25B Established, What Remains Unresolved, Work Currently Paused, Context Loading Guide, Return-to-Work Summary.
|
||||
|
||||
### Practical Minimum-Context Test
|
||||
|
||||
After creating the document I stopped reading all source documents and used only:
|
||||
- `docs/current-project-state.md`
|
||||
- `.claude/architecture-guardrails.md`
|
||||
|
||||
To produce this briefing for a returning developer:
|
||||
|
||||
1. **Active:** Deterministic reasoning pipeline, unknown selection (atomicity/answerability), question formulation within reasoning patterns, scenario API, turn cycle orchestration. Nothing more from the engine itself.
|
||||
2. **Passive:** Investigation-state assessment, behaviour selection, decision condition status, question-to-condition relevance, evidence direction, evidence scope, scope-aware condition status — all isolated diagnostic layers with no active integration.
|
||||
3. **Paused:** Engine experiments (after 25B), UI experiments. Knowledge-management is active. Nothing archived or deleted.
|
||||
4. **Provisional:** Keyword/phrase matching for scope detection; passive classifier generalisability across domains; how passive reasoning enters the active cycle; whether architecture docs match implementation.
|
||||
5. **Next:** `docs/current-project-state.md` is the starting point. Use the inventory for task-specific context. Guardrails before code changes.
|
||||
|
||||
Result: The briefing was accurate and complete from these two files. No essential information was missing. The routing table in section 7 of the current-state document provided all necessary references without requiring additional documents.
|
||||
|
||||
### Missing or Ambiguous Information Found
|
||||
|
||||
- `docs/investigation-state-assessment-contract.md` (232 lines) describes a data contract that may no longer match implementation after experiments 15–25B; not verified.
|
||||
- The exact line count of the created document should be confirmed with `wc -l`.
|
||||
- Whether any of the passive classifiers have been partially integrated since Exp 25B was closed requires checking source code — this task did not read it.
|
||||
|
||||
### Assessment
|
||||
|
||||
The new entry point successfully replaced the need to load the large experiment-log section (1,542 lines). The current-state document conveys active vs passive capabilities, pause status, unresolved questions and loading instructions in a single short file. It can replace the large default log section as the normal starting point for future work.
|
||||
|
||||
The practical briefing was produced accurately from only two files without reading any source material beyond what was used to create it. This confirms the hypothesis that a concise current-state document is sufficient context for understanding where the project stands.
|
||||
|
||||
### Return-to-Work Note
|
||||
|
||||
A short current-state entry point now exists at `docs/current-project-state.md`. Future Claude sessions should begin there. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading. Nothing has been archived, moved or deleted yet. Before changing the documentation structure, review whether the new entry point reliably replaces the large log section and whether any historical documents should be formally archived. First file to inspect when resuming: `docs/current-project-state.md`. Branch: `feature/user-workspace-ux-v0.7`.
|
||||
|
||||
### Status
|
||||
|
||||
Pending Rob's review.
|
||||
|
||||
The following are active explorations rather than decisions.
|
||||
|
||||
- What is the right metaphor for the product?
|
||||
- Should the workspace resemble a facilitated workshop?
|
||||
- How should decomposition be represented?
|
||||
- What information belongs in shared understanding?
|
||||
- What should the Investigation Map eventually become?
|
||||
- How should wide thinking be reflected in the interface?
|
||||
|
||||
## Backlog — Experiment 05 Persistence Note
|
||||
|
||||
The "Don't show this introduction again" checkbox uses sessionStorage as a placeholder.
|
||||
|
||||
This preference should eventually be handled through user preferences or settings rather than local component state.
|
||||
|
||||
TODO: When user accounts are introduced, persist this preference to the user profile so it travels across devices and sessions.
|
||||
|
||||
## Future Note — Dark Mode
|
||||
|
||||
Dark mode is intentionally deferred.
|
||||
|
||||
Once the information architecture and visual hierarchy stabilise we will investigate whether an "Investigation Mode" (rather than a conventional dark mode) improves concentration.
|
||||
|
||||
This should be treated as a future UX experiment rather than an accessibility feature.
|
||||
|
||||
## Experiment 28 — Verify Current Project State Against Implementation
|
||||
|
||||
**Status:** Pending Rob's review
|
||||
|
||||
### Hypothesis
|
||||
|
||||
A focused code inspection can verify or correct the current-state document without requiring a fresh session to read the full experiment log. If the document is accurate, it can safely become the normal project entry point.
|
||||
|
||||
### Source Areas Inspected
|
||||
|
||||
- `docs/current-project-state.md` — entire file;
|
||||
- `.claude/architecture-guardrails.md` — entire file;
|
||||
- `docs/project-knowledge-inventory.md` — Current Working Context and Task-Specific References sections;
|
||||
- `app/api/*/route.js` — all API entry points (analyse, cases/start, cases/update, health);
|
||||
- `lib/graph/orchestrator.js` — imports (lines 6–32) and runtime calls at lines 376, 402, 552, 581, 622, 826, 904, 1013;
|
||||
- `lib/graph/*.js` — grep for imports of passive classifier modules (decision-condition-status, evidence-direction, evidence-condition-scope, question-decision-relevance, question-importance);
|
||||
- `lib/behaviour-selection/behaviour-selector.js` — cross-module import check;
|
||||
- `lib/assessment/investigation-state-assessor.js` — caller trace in orchestrator.
|
||||
|
||||
### Active / Passive Findings
|
||||
|
||||
**Active capabilities confirmed:**
|
||||
1. Scenario reconstruction (analyseScenario) — API entry at app/api/analyse/route.js → lib/analysis.js.
|
||||
2. Reasoning graph updates (startCase / updateCase) — API entries at app/api/cases/{start,update}/route.js → orchestrator.js → apply-proposal.js. Propagation, confidence cap, completeness calculated in apply-proposal.
|
||||
3. Unknown selection (atomicity + answerability) — selectActiveUnknownCandidate imported and called from orchestrator's determineGraphBackedQuestion within the active updateCase path.
|
||||
4. Question formulation — formulateQuestion / formulateTieResolutionQuestion imported and called from the active turn cycle.
|
||||
5. Turn orchestration — orchestrator.js updateCaseWithDependencies() is the active engine heart, coordinating unknown→question→answer→graph-update→propagation→next-unknown.
|
||||
|
||||
**Passive or isolated capabilities confirmed:**
|
||||
1. Investigation-state assessment (assessInvestigationState) — called at 3 sites in orchestrator but result only placed into a diagnostics field; not used for any control-flow decision. Classification: **diagnostic_only**.
|
||||
2. Behaviour selection (selectBehaviour) — exported from behaviour-selector.js; no callers anywhere in the repo. Classification: **isolated**.
|
||||
3. Question importance, question relevance to decision, evidence direction, evidence scope, scope-aware condition status — each exists as a standalone module or file with zero external callers. Evidence direction and scope are imported only by decision-condition-status.js, which itself has no callers.
|
||||
|
||||
### Corrections Made
|
||||
|
||||
None. The current-state document's active/passive classification is accurate as-is. Added verification marker to docs/current-project-state.md.
|
||||
|
||||
### Practical Context-Test Result
|
||||
|
||||
**Task:** A developer proposes connecting Behaviour Selection directly to the next user-facing response. Is it active today? What boundary exists? Which files would need inspection before future integration?
|
||||
|
||||
**Briefing:**
|
||||
1. **Active today?** No. `selectBehaviour` is exported from `lib/behaviour-selection/behaviour-selector.js` but has zero callers anywhere in the repository. It is not active, diagnostic, or accessible through any API.
|
||||
2. **Current boundary:** Behaviour Selection and Investigation-State Assessment exist as separate modules that were never wired into the orchestrator's turn cycle. The orchestrator returns an `assessment` field to clients but does not pass assessment results into its own decision logic. There is no data path from state assessment → behaviour selection → question/response.
|
||||
3. **Files to inspect before integration:** `lib/graph/orchestrator.js` (where the insertion point would be — between unknown selection and question formulation, or after propagation); `lib/assessment/investigation-state-assessor.js` (to understand what the assessment contract outputs); `lib/behaviour-selection/behaviour-selector.js` (to understand what behaviours it can produce); `docs/investigation-state-assessment-contract.md` and `docs/behaviour-selection.md` for the documented interfaces; `app/api/cases/update/route.js` to determine whether behaviour output would appear in the API response or remain internal.
|
||||
4. **Context sufficient?** Yes — the three-file set (current-project-state, verification file, guardrails) plus targeted code inspection of the modules above provides sufficient context for a designer to assess integration scope without reopening the full history.
|
||||
5. **Verdict:** Integration is feasible as a future experiment. The primary risk is that behaviour selection has no documented input contract from the assessment layer — these were built in parallel without an agreed handoff shape.
|
||||
|
||||
### Unresolved Questions
|
||||
|
||||
- Whether the assessment output from `assessInvestigationState` matches the documented `investigation-state-assessment-contract.md` (requires reading the assessor's internal logic, excluded per constraints).
|
||||
- Whether external API clients (not in this repo) call the orchestrator directly, bypassing the route files.
|
||||
- The exact integration sequence: should behaviour selection read from assessment output or from the graph state directly?
|
||||
|
||||
### Return-to-Work Note
|
||||
|
||||
The current-state briefing was checked against source code via targeted code inspection of API routes, orchestrator imports/calls, and cross-module traces for each passive classifier. Five active capabilities are confirmed (reconstruction, graph updates, unknown selection, question formulation, turn orchestration). Seven passive capabilities remain classified as diagnostic_only (investigation-state assessment) or isolated (behaviour selection, decision-condition status, evidence direction, evidence scope, question importance, question relevance to decision, scope-aware condition status). No corrections to the current-state document were required. Knowledge-management work remains active. Engine and UI experiments remain paused. Branch: feature/user-workspace-ux-v0.7. First file to inspect when resuming: `docs/current-project-state.md`, then `.claude/architecture-guardrails.md` before any code changes, then `lib/graph/orchestrator.js` for engine-resumption work.
|
||||
|
||||
Branch: feature/user-workspace-ux-v0.7
|
||||
Commit: 61c8a3a
|
||||
|
||||
|
||||
## Experiment 29 — Archive the History Without Losing the Trail
|
||||
|
||||
**Status:** Pending Rob's review
|
||||
|
||||
### Hypothesis
|
||||
|
||||
Historical documents can be moved into a clearly labelled archive without breaking links, losing evidence, or confusing future sessions. A fresh Claude session should still be able to understand the current system from the short entry point, locate historical material when specifically needed, and identify which documents are current versus retained only as evidence.
|
||||
|
||||
### Files Archived (5)
|
||||
|
||||
| Original Path | Archive Path | Reason |
|
||||
|---|---|---|
|
||||
| `docs/v0.4-handoff.md` | `docs/archive/v0.4-handoff.md` | Historical v0.4 handoff; architecture has evolved since. Referenced in `orchestrator-contract.md` (reference repaired). |
|
||||
| `docs/v0.4-route-status.md` | `docs/archive/v0.4-route-status.md` | Historical route tracking; current routes differ. |
|
||||
| `docs/v0.5-release-notes.md` | `docs/archive/v0.5-release-notes.md` | Historical release record; nothing active depends on it. |
|
||||
| `docs/v0.6-ambiguity-generalisation.md` | `docs/archive/v0.6-ambiguity-generalisation.md` | Superseded by later reasoning architecture decisions (Exp 15–25B). |
|
||||
| `docs/v0.7-observation-report.md` | `docs/archive/v0.7-observation-report.md` | Experimental observation snapshot; useful reference but not current guidance. UX work paused. |
|
||||
|
||||
### Files Deliberately Not Archived (2)
|
||||
|
||||
| Document | Reason |
|
||||
|---|---|
|
||||
| `docs/architectural-principles.md` | 14 architectural principles from experiments; may be needed when re-engaging with reasoning architecture. Status unclear — review before future archive. |
|
||||
| `docs/backlog info.md` | Mock fixture backlog useful if resuming UI development. Needs content verification before archiving. |
|
||||
|
||||
### Reference Repairs
|
||||
|
||||
- `docs/orchestrator-contract.md`: Updated reference from `docs/v0.4-handoff.md` to `docs/archive/v0.4-handoff.md` (line 78) and table entry (line 87).
|
||||
- `docs/project-knowledge-inventory.md`: Updated all five archive candidate entries with new paths and provenance notes; updated Return-to-Work section.
|
||||
- No other files contained active references to archived documents.
|
||||
|
||||
### Practical Archive Test
|
||||
|
||||
**Task:** A developer needs to find what v0.4 originally said about the case-orchestration API, without reading the full experiment log or archive directory.
|
||||
|
||||
**Execution:** From `docs/project-knowledge-inventory.md` (section 3) → identifies `docs/archive/v0.4-handoff.md` as the historical handoff for v0.4 architecture; from `docs/archive/README.md` → confirms file exists at that path and explains what it contains; verified file is accessible.
|
||||
|
||||
**Result:** The developer can locate the correct archived document in two steps: (1) inventory identifies which past document contains relevant evidence, (2) archive index confirms location and contents. The current project can be fully understood from `docs/current-project-state.md` alone without opening any archived file. No current task depends on archived files by default — they are consulted only when a named past decision or release is under investigation.
|
||||
|
||||
### Uncertain Candidates
|
||||
|
||||
- `docs/architectural-principles.md`: Should it be archived now, or reviewed first for accuracy against current implementation? Decision deferred to Rob's review.
|
||||
- `docs/backlog info.md`: Contains mock fixtures — may become irrelevant if the fixture strategy changes. Needs content verification before any future archive decision.
|
||||
|
||||
### Status
|
||||
|
||||
Pending Rob's review.
|
||||
|
||||
These are observations, not implementation tasks.
|
||||
|
||||
- Narrative adapter
|
||||
- Narrative quality heuristics
|
||||
- Narrative progression
|
||||
- Narrative completion state
|
||||
- Narrative confidence wording
|
||||
- Narrative testing
|
||||
- Narrative localisation
|
||||
- Multiple narrative projections
|
||||
|
||||
## Experiment 30 — Review Deferred Project Documents
|
||||
|
||||
**Status:** Pending Rob's review
|
||||
|
||||
### Hypothesis
|
||||
|
||||
Each deferred document can be classified by comparing it with the verified current project state without reopening the full experiment history or rewriting its contents. The result may be: keep as current guidance, keep as task-specific reference, archive as historical evidence, or retain temporarily pending revision. No additional categories should be invented.
|
||||
|
||||
### Review of architectural-principles.md
|
||||
|
||||
- **14 principles assessed against verified implementation:**
|
||||
- **6 current (match runtime or guardrails):** P1 (layer separation), P3 (user feedback loop), P4 (reasoning/UI separation), P6 (presentation renders, does not interpret), P8 (narrative never invents facts), P14 (user as first-class input).
|
||||
- **4 aspirational targets:** P5 (behaviour never reasons — module exists with zero callers), P10 (convergence over single signals — no mechanism), P11 (stateful assessment across turns — partially present), P12 (assessable uncertainty — absent).
|
||||
- **4 mixed/unclear:** P2 (information flows downward — partially matches but passive layers don't fit the cascade model), P7 (assessment never generates evidence — diagnostic_only but scope-aware condition status makes interpretive judgments), P9 (assessment describes not prescribes — signals descriptive, but decision-condition evaluation borders on prescription), P13 (progress qualitative not quantitative — product direction supports; unknown selection uses node status qualitatively but not verified).
|
||||
- **3 duplicated with guardrails:** P1 overlaps with architecture-guardrails' prohibition list. P4 overlaps with UX-task boundaries in guardrails. P8 overlaps with the explicit invariant "every question comes from a resolved graph node." Overlap adds value: guardrails state boundaries; principles explain why.
|
||||
|
||||
- **Role assigned:** Keep as task-specific reference. Six current principles and four aspirational targets make it valuable when resuming reasoning architecture work. Three duplications reduce (but don't eliminate) its independent value — the derived-from/implication context adds what guardrails lack. project-knowledge-inventory already listed it under "Review Before Archive"; confirmed as task-specific reference.
|
||||
|
||||
### Review of backlog info.md
|
||||
|
||||
- **Content analysis:**
|
||||
- **Still-relevant (≈20 lines):** Mock fixtures table — 15 scenario types with purposes and examples. Directly useful when UI work resumes.
|
||||
- **Historical/aspirational (≈370 lines):** UX roadmap phases 1–4 with wireframe text, animation specs, loading messages. Design intent is valid; specifics may change when UI resumes. Untracked — no commit/PR linkage.
|
||||
- **Duplicates:** Phase 4 "Mock Scenario Library" duplicates the fixtures table at top. "Deliberately Out of Scope" repeats pause decision in current-project-state and project-context.
|
||||
|
||||
- **Role assigned:** Retain temporarily pending revision. The mock fixtures table is too useful to lose in an archive, but the document's mixed role (useful reference + deferred planning) needs resolution when UI work resumes. Splitting the file or archiving portions requires revising content — constraints forbid this now.
|
||||
|
||||
### Practical Routing Test Result
|
||||
|
||||
**Task:** A future Claude session is about to work on UI mocks. Should it read architectural-principles.md, backlog info.md, both, or neither?
|
||||
|
||||
**Answer: Both.** Backlog info.md provides the mock fixtures table (direct reference). Architectural-principles.md provides boundaries (P4: reasoning never communicates directly with UI; P6: presentation never interprets) that prevent accidentally introducing reasoning logic into UI work. Three-document context (current-project-state, project-knowledge-inventory, document-role-review) is sufficient to route both documents correctly without reading the full experiment log or archive.
|
||||
|
||||
### Files Created / Modified
|
||||
|
||||
- `docs/document-role-review.md` — new (140 lines); classifies both candidates with evidence and routing test
|
||||
- `docs/project-knowledge-inventory.md` — updated "Review Before Archive" table (principle roles added), added "Knowledge management" section with document-role-review entry, updated Return-to-Work note
|
||||
- `docs/current-project-state.md` — updated Return-to-Work note to include Experiment 30 status
|
||||
- No files moved to archive (neither candidate qualifies as "archive as historical evidence")
|
||||
- No files deleted; no source code or tests changed
|
||||
|
||||
### Status
|
||||
|
||||
Pending Rob's review. Neither document moves. Both roles confirmed by evidence against verified implementation. When UI work resumes, backlog info.md's fixtures table will be the direct reference; architectural-principles.md is available for reasoning architecture context. Engine and UI experiments remain paused. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-project-state.md`, then Experiments 23–25B in design-evolution-log.md (lines 1218–1520).
|
||||
|
||||
These are observations, not implementation tasks.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 31 — Separate Useful UI Reference From Unstructured Backlog
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
The document `docs/backlog info.md` can be divided into:
|
||||
- a short task-specific mock/UI reference that remains in the normal documentation area;
|
||||
- a retained deferred backlog document that is excluded from default context loading.
|
||||
|
||||
This should make future UI work easier without losing previous ideas.
|
||||
|
||||
### Separation Method
|
||||
|
||||
Original file `docs/backlog info.md` (390 lines) was split into two new documents:
|
||||
|
||||
1. **`docs/ui-mock-reference.md`** (~62 lines) — practical mock-fixture reference extracted from the original lines 1–20, structured with available scenarios, fixture data locations, when-to-use guidance, and warnings.
|
||||
2. **`docs/archive/deferred-ux-backlog.md`** (376 lines) — deferred UX planning content from original lines 21–390, preserved with original header stating items are not commitments.
|
||||
|
||||
The original file was removed after complete accounting (every section accounted for in one of the two new documents).
|
||||
|
||||
### Content Accounting
|
||||
|
||||
| Original Section | Line Range | Destination | Treatment |
|
||||
|---|---|---|---|
|
||||
| Mock fixtures table + intro | 1–20 | `docs/ui-mock-reference.md` | Represented as structured reference (same scenarios, enhanced with fixture data locations and usage guidance) |
|
||||
| UI Roadmap header + intro | 21–26 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged |
|
||||
| Phase 1 – Core Investigation Experience | 27–118 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged |
|
||||
| Phase 2 – UX Polish | 119–169 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged |
|
||||
| Phase 3 – Developer Experience | 197–218 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged |
|
||||
| Phase 4 – Mock Scenario Library | 219–326 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged (scenarios listed twice — once in original fixtures table, once here — no duplication introduced) |
|
||||
| Backlog – Reasoning Replay | 328–378 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged |
|
||||
| Deliberately Out of Scope | 379–390 | `docs/archive/deferred-ux-backlog.md` | Copied unchanged |
|
||||
|
||||
**Material not transferred:** None. Every original section is represented in one of the two new documents.
|
||||
|
||||
### Files Created
|
||||
|
||||
- `docs/ui-mock-reference.md` (~62 lines) — mock fixture scenario reference
|
||||
- `docs/archive/deferred-ux-backlog.md` (376 lines) — deferred UX planning backlog
|
||||
|
||||
### Files Removed
|
||||
|
||||
- `docs/backlog info.md` (390 lines) — superseded by the split; all content accounted for above
|
||||
|
||||
### Files Modified
|
||||
|
||||
- `docs/archive/README.md` — added deferred-ux-backlog to Archived Files table; added Superseded Files section with backlog info.md entry
|
||||
- `docs/project-knowledge-inventory.md` — added ui-mock-reference to UI/UX task-specific references; added deferred-ux-backlog to archive candidates; updated backlog info.md role to "superseded"; updated Return-to-Work note
|
||||
- `docs/current-project-state.md` — updated Section 6 (Return-to-Work Summary) and section 8 header/note to reflect Experiment 31 split
|
||||
- `.claude/project-context.md` — added routing notes: UI mock work reads ui-mock-reference; deferred backlog only for named UX idea review
|
||||
|
||||
### Line Counts Before / After
|
||||
|
||||
| Document | Lines (before) | Lines (after) |
|
||||
|---|---|---|
|
||||
| Original combined document (`backlog info.md`) | 390 | removed |
|
||||
| New mock reference (`ui-mock-reference.md`) | — | ~62 |
|
||||
| New deferred backlog (`deferred-ux-backlog.md`) | — | 376 |
|
||||
| Total new content | — | 438 (62 + 376, including headers in both) |
|
||||
|
||||
### Practical Routing Test Result
|
||||
|
||||
**Scenario:** A developer wants to test the workspace against a long investigation and a contradictory-evidence scenario. Which mock scenarios should they use, and where is the fixture data defined?
|
||||
|
||||
**Answer:** They should use:
|
||||
- **Long investigation (10–15 turns)** — for testing history scrolling, collapsing, pacing;
|
||||
- **Contradiction** — for testing contradiction detection and user-facing messaging.
|
||||
|
||||
Fixture data is defined in `tests/e2e/fixtures/investigation-scenarios.js`. The mock client is in `lib/mocks/confidence-engine/mock-client.js`. Scenario names are set via `NEXT_PUBLIC_CONFIDENCE_ENGINE_MOCK_SCENARIO` env var in `components/scenario-form.jsx`. Reference details and usage guidance are in `docs/ui-mock-reference.md`.
|
||||
|
||||
**Was the deferred backlog necessary?** No. The practical routing test was answered entirely from `ui-mock-reference.md`, `project-knowledge-inventory.md`, `.claude/project-context.md`, and `architecture-guardrails.md`. The deferred backlog (376 lines of aspirational UX planning) was not required to answer a practical mock-scenario question.
|
||||
|
||||
**Was any practical mock information lost?** No. All 13 fixture scenarios are preserved in `ui-mock-reference.md` with enhanced guidance on where fixtures live and when to use each. The original fixtures table's content is fully represented.
|
||||
|
||||
### Gaps Found
|
||||
|
||||
- `docs/ui-mock-reference.md` references `tests/e2e/fixtures/investigation-scenarios.js` as the fixture definition location but does not list individual scenario keys or env var values (by design — those are implementation details that can be inspected directly in the fixture file).
|
||||
- The deferred backlog contains specific wireframe text and animation specifications that may still be useful when UI work resumes. The header note ("not commitments, priorities or active tasks") should prevent premature actioning.
|
||||
|
||||
### Status
|
||||
|
||||
Pending Rob's review. Both new documents contain all original content. Branch `feature/user-workspace-ux-v0.7` is clean after commit. Engine and UI experiments remain paused.
|
||||
|
||||
## Experiment 32 — Separate Current Principles From Aspirational Architecture
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
A short current-principles document can guide normal work while the original architectural-principles document remains available as the fuller historical and aspirational source. This should reduce ambiguity without deleting or rewriting the original reasoning.
|
||||
|
||||
### Source Documents Used
|
||||
|
||||
- `docs/current-project-state.md` — What the Confidence Engine Is; Current Engine Capabilities; Context Loading Guide
|
||||
- `.claude/architecture-guardrails.md` — entire file (77 lines)
|
||||
- `docs/document-role-review.md` — Architectural Principles Review (§2) and Recommended Actions (§4)
|
||||
- `docs/architectural-principles.md` — headings and the 14 principles only
|
||||
- `docs/03_Confidence_Engine_Language_Guide.md` — guiding principles only
|
||||
- `docs/current-implementation-verification.md` — Active Capabilities; Passive or Isolated Capabilities
|
||||
- Experiment 31 entry in `docs/design-evolution-log.md` (lines 1811–1893)
|
||||
|
||||
### Principles Included
|
||||
|
||||
**User Experience (5):** System carries complexity; steps are small enough to understand or investigate; engine guides without pretending certainty; first input is the hardest step; users may know answer/who to ask/where to look/how to test.
|
||||
|
||||
**Reasoning (5):** Resolved question ≠ established condition; evidence supports/contradicts/informs; present evidence does not settle future feasibility; uncertainty stated honestly; deterministic contracts separate from language interpretation.
|
||||
|
||||
**Building the System (6):** Build smallest thing that can be wrong; use evidence before architecture; every layer has one responsibility where applicable; presentation does not invent facts; current and aspirational labelled separately; load only needed context.
|
||||
|
||||
Total: 16 current principles, organized into three sections.
|
||||
|
||||
### Aspirational Material Deliberately Excluded
|
||||
|
||||
From `docs/architectural-principles.md`: P2 (Information Flows Downward — unresolved), P5 (Behaviour Never Reasons — aspirational), P7 (Assessment Never Generates Evidence — mixed), P9 (Assessment Describes Never Prescribes — mixed), P10 (Convergence Over Single Signals — aspirational), P11 (Assessment Is Stateful Across Turns — mixed/aspirational), P12 (Uncertainty About Assessment Is Itself Assessable — aspirational), P13 (Investigation Progress Is Qualitative Not Quantitative — mixed/aspirational). These remain in the original document for broader architectural review.
|
||||
|
||||
### Practical Principles-Test Result
|
||||
|
||||
**Task:** A developer proposes making every resolved question automatically increase confidence and close its related condition. Explain whether this fits current principles and why.
|
||||
|
||||
**Response from reduced context (current-project-state + current-working-principles + architecture-guardrails):**
|
||||
|
||||
1. **Resolving a question does not establish a condition.** current-working-principles §2 states: "A resolved question is not an established condition." Answer evidence must be inspected before any conclusion follows.
|
||||
2. **Answer evidence must be inspected.** current-working-principles §2 states direction alone (support/contradict/inform) is insufficient without checking subject, timeframe, and claim type alignment.
|
||||
3. **Confidence should not be manufactured.** architecture-guardrails invariants state "Confidence must not outrun evidence or completeness" and "Duplicate evidence must not increase confidence." current-project-state section 4 confirms: resolving a question does not automatically establish the condition.
|
||||
4. **Passive experimental logic is not automatically active behaviour.** current-project-state section 3 classifies passive classifiers (including decision-condition status evaluation) as diagnostic_only or isolated — they do not yet control the user-facing investigation.
|
||||
|
||||
**Was the three-document context sufficient?** Yes. All four points were answerable from `docs/current-working-principles.md` (principles §2), `.claude/architecture-guardrails.md` (reasoning invariants), and `docs/current-project-state.md` (section 3 passive classifier classification, section 4 what experiments established). No experiment history or source code was required.
|
||||
|
||||
### Unresolved Ambiguities
|
||||
|
||||
- The boundary between "current" and "aspirational" for P7 and P9 is inherently subjective; future sessions may interpret differently without the original document's reasoning context.
|
||||
- Some principles overlap with `.claude/architecture-guardrails.md` (e.g., "every layer has one responsibility" overlaps with guardrails' exhaustive prohibition list). No duplication was introduced deliberately, but a cross-reference could reduce redundancy in a future iteration.
|
||||
- The aspirational note points readers to the original document but does not provide a quick reference for which of the 14 principles are current versus aspirational. A summary table might be useful when architecture work resumes.
|
||||
|
||||
### Status
|
||||
|
||||
Pending Rob's review. No source code or tests changed. Engine and UI experiments remain paused. No files moved or deleted. Only documentation files were created or updated.
|
||||
|
||||
### Return-to-Work Note (80–150 words)
|
||||
|
||||
Current principles now live in `docs/current-working-principles.md`. This short document contains only guidance supported by verified implementation, current project direction, and established product philosophy — organised into three sections: user experience, reasoning, and building the system. Broader and aspirational architecture remains in `docs/architectural-principles.md` as a task-specific reference; it has not been rewritten or deleted. Future sessions should use `docs/current-working-principles.md` by default for product and reasoning work. Engine and UI experiments remain paused after Experiment 25B. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-project-state.md`, then `docs/current-working-principles.md` for current guidance.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 33 — Create Task-Specific Context Packs
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
A single concise context-pack guide can give each task type a minimal reading list, clear exclusions, and a stopping rule — reducing unnecessary context loading while preserving access to deeper material when a specific gap appears.
|
||||
|
||||
### Source Documents Used
|
||||
|
||||
- `docs/current-project-state.md` — Context Loading Guide; Current Engine Capabilities; Work Currently Paused
|
||||
- `docs/project-knowledge-inventory.md` — Current Working Context; Task-Specific References
|
||||
- `docs/current-implementation-verification.md` — Active Capabilities; Passive or Isolated Capabilities
|
||||
- `docs/current-working-principles.md` — entire file
|
||||
- `docs/ui-mock-reference.md` — headings and routing information only
|
||||
- `.claude/project-context.md` — routing notes only
|
||||
- `.claude/architecture-guardrails.md` — headings only
|
||||
- Experiment 32 entry in `docs/design-evolution-log.md` (lines 1895–1948)
|
||||
|
||||
### Deliverable
|
||||
|
||||
Created `docs/task-context-packs.md` (~110 lines) with four packs:
|
||||
- **Pack 1 — Engine Experiment Work:** current-project-state, current-working-principles, architecture-guardrails, current-implementation-verification.
|
||||
- **Pack 2 — UI and Mock Work:** current-project-state, current-working-principles, architecture-guardrails, ui-mock-reference.
|
||||
- **Pack 3 — Architecture or Contract Review:** current-project-state, current-implementation-verification, architecture-guardrails, current-working-principles + aspirational warning.
|
||||
- **Pack 4 — Knowledge-Management Work:** current-project-state, project-knowledge-inventory, task-context-packs, project-context.
|
||||
|
||||
Each pack lists what to always read, what to read only when relevant, and what to not load by default. Common rules prevent silent context inflation. Two routing tests verify sufficiency without loading history or source code.
|
||||
|
||||
### Routing Test A — Engine Task
|
||||
|
||||
**Task:** Verify whether Behaviour Selection currently affects the user-facing response.
|
||||
**Result:** Pack sufficient. `docs/current-implementation-verification.md` §3b states "Called by: None" for Behaviour Selection; `docs/current-project-state.md` §3 classifies it as isolated. No extra file required.
|
||||
|
||||
### Routing Test B — UI Task
|
||||
|
||||
**Task:** Choose the correct mock scenarios for testing a long investigation and contradictory evidence.
|
||||
**Result:** Pack sufficient. `docs/ui-mock-reference.md` lists "Long investigation (10–15 turns)" and "Contradiction" with matching purposes. Deferred UX backlog not needed.
|
||||
|
||||
### Validation
|
||||
|
||||
- All referenced files exist; no pack relies on fixed line numbers.
|
||||
- Each pack has a smaller default context than the full project documentation.
|
||||
- Active and passive capabilities remain clearly separated.
|
||||
- No source code or tests changed; no files moved or deleted.
|
||||
|
||||
### Return-to-Work Note
|
||||
|
||||
Task-specific context packs now exist in `docs/task-context-packs.md`, giving each work type a minimal four-document starting set plus targeted reading paths. Future sessions should start with `docs/current-project-state.md`, then choose one pack from `docs/task-context-packs.md`. Additional documents should be loaded only for a named gap, with the reason recorded. Engine and UI experiments remain paused. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-project-state.md`, then select the relevant pack from `docs/task-context-packs.md`.
|
||||
|
||||
---
|
||||
## Experiment 34 — Single Return-to-Work Handoff
|
||||
|
||||
**Date:** 2026-08-06
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
A single short handoff file can carry enough immediate context to resume work accurately while linking to deeper documents only when needed.
|
||||
|
||||
### Handoff Structure
|
||||
|
||||
Eight sections: Where We Left It, What Is True Now, Why Work Is Paused, What Was Just Completed, What Remains Open, How to Resume, First Files by Work Type (table), Resume Check (five questions). Plus a maintenance rule replacing current-work sections when the project moves on.
|
||||
|
||||
### Document Length
|
||||
|
||||
`docs/current-handoff.md`: 68 lines (target range: 60–100).
|
||||
|
||||
### Practical Resume-Test Result
|
||||
|
||||
**Task:** Return after two weeks, remember almost nothing. Explain where the project stands, what is paused, what was completed most recently, and what to read before an engine task — using only `docs/current-handoff.md` and `docs/task-context-packs.md`.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Identifies correct active phase (knowledge management) | Yes |
|
||||
| Identifies paused engine and UI work | Yes |
|
||||
| Identifies Experiment 33 as latest completed | Yes |
|
||||
| Chooses Engine Experiment pack for engine task | Yes |
|
||||
| Avoids opening full design history | Yes |
|
||||
| Does not confuse passive code with active behaviour | Yes |
|
||||
|
||||
**Verdict:** Pass. The handoff alone is sufficient to resume accurately.
|
||||
|
||||
### Missing Information
|
||||
|
||||
- "When knowledge-management work is complete enough to resume engine experiments" — no objective criterion exists yet; this is a judgment call for Rob.
|
||||
- "Whether tasks crossing pack boundaries can still stay concise" — unanswered in principle; requires testing with actual cross-boundary tasks.
|
||||
- Whether `docs/current-handoff.md` remains useful after several more knowledge-management experiments add to it.
|
||||
|
||||
### Can This Replace Scattered Current Return Notes?
|
||||
|
||||
Yes, for immediate resumption context. The handoff carries the latest stopping point without accumulating old notes. Historical return notes remain in `docs/current-project-state.md` and `docs/design-evolution-log.md` as evidence, not as current guidance. Rob should decide whether to purge older return notes once confident in the handoff model.
|
||||
|
||||
### Status
|
||||
|
||||
Pending Rob's review.
|
||||
|
||||
+451
@@ -0,0 +1,451 @@
|
||||
## Experiment 35 — Test Current Handoff Maintenance (2026-08-06)
|
||||
|
||||
**Hypothesis:** A current handoff can remain useful if it describes only the latest stopping point, replaces stale details rather than appending history, and identifies the latest confirmed experiment and commit unambiguously.
|
||||
|
||||
**Stale or ambiguous wording found:**
|
||||
- Section 1 named Experiment 33 and commit `b959cfa` as the current state — now stale after Experiments 34+35;
|
||||
- Section 4 described only Experiment 33's completion, giving no indication that a single handoff had been created in Experiment 34;
|
||||
- No explicit mention of commit `1d92aa0` anywhere in the handoff;
|
||||
- Footer said "Created by Experiment 34" without acknowledging this maintenance experiment.
|
||||
|
||||
**Corrections made:**
|
||||
- Section 1: updated to name Experiment 34 and commit `1d92aa0`; added the maintenance principle ("replace stale details rather than appending history");
|
||||
- Section 4: rewritten to describe Experiment 34's consolidation work;
|
||||
- Section 5: retained one genuinely open question about handoff longevity; added provisional KM completion criteria sub-section (7 criteria, marked provisional);
|
||||
- Footer: updated to reference Experiment 35; added Return-to-Work Note recording all current state.
|
||||
|
||||
**Fresh-return test result:** PASS — from `current-handoff.md` and `task-context-packs.md` only, a fresh session can determine:
|
||||
- Latest completed KM experiment: Experiment 34 ✓
|
||||
- Latest commit: `1d92aa0` ✓
|
||||
- Knowledge-management active, engine/UI paused ✓
|
||||
- Knowledge-Management context pack is the correct routing target ✓
|
||||
- No need to open full design history ✓
|
||||
- Older commits not mistaken for current stopping point ✓
|
||||
|
||||
**Provisional completion criteria added:** Seven criteria recorded in Section 5 (see above). Not yet declared complete — pending Rob's review.
|
||||
|
||||
**Handoff remained concise?** Yes. 86 lines (was 68). Increase justified by the maintenance principle paragraph, updated current-state wording, and provisional completion criteria section. No historical timeline appended.
|
||||
|
||||
**Status:** Pending Rob's review.
|
||||
|
||||
## Experiment 36 — Validate Reduced Context Routing
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
The documentation system (handoff + project-state + task-context-packs) is complete enough to support normal work without silently expanding into historical documentation. A fresh session can complete representative tasks using only routing instructions.
|
||||
|
||||
### Initial Documents Loaded (328 lines total)
|
||||
|
||||
1. `docs/current-handoff.md` — 86 lines
|
||||
2. `docs/current-project-state.md` — 132 lines
|
||||
3. `docs/task-context-packs.md` — 110 lines
|
||||
|
||||
### Additional Documents Loaded
|
||||
|
||||
| Document | Lines | Why Needed | Routing Should Include? |
|
||||
|---|---|---|---|
|
||||
| `docs/ui-mock-reference.md` | 63 | Task 2: verify mock scenarios for "long investigation" and "contradiction". Routing Test B claimed these were identifiable without loading it, but the specific scenario names do not appear in any initial document. | YES — routing defect found |
|
||||
| `docs/project-knowledge-inventory.md` | 215 | Task 4: confirm Engine Experiment pack's four always-read documents actually exist and understand KM phase outputs. | Debated — validated completeness but not strictly required by routing |
|
||||
| `docs/current-implementation-verification.md` | 111 | Cross-checked Behaviour Selection isolation against current-project-state §3. Provided corroboration but was not the sole basis for Task 1 answer. | Debated — useful corroboration; current-project-state alone sufficed |
|
||||
|
||||
### Tasks Completed Without Context Expansion
|
||||
|
||||
**Task 1 — Does Behaviour Selection affect engine behaviour?**
|
||||
No. Current project state §3 classifies it as isolated. Handoff §2 confirms passive classifiers don't control the investigation. Task-context-packs Routing Test A corroborates (current-implementation-verification §3b).
|
||||
|
||||
**Task 3 — Why passive classifiers are not yet in the active reasoning loop?**
|
||||
Passive classifiers record diagnostic signals for future use but have no integration into the turn cycle. Only investigation-state assessment is called (at 3 orchestrator sites), and its result goes into a diagnostics field — never checked by conditional branches. Others have zero callers.
|
||||
|
||||
### Tasks Requiring Extra Context
|
||||
|
||||
**Task 2 — Mock scenarios for long investigation and contradictory evidence**
|
||||
Required `docs/ui-mock-reference.md`. Routing Test B in task-context-packs claimed these were identifiable without loading it, but the specific scenario names ("Long investigation (10–15 turns)" and "Contradiction") do not appear in any initial document. The routing claim was unverifiable until the mock reference was loaded — this is a genuine routing defect.
|
||||
|
||||
**Task 4 — Where should a new developer begin for the next engine experiment?**
|
||||
Partially answered from initial documents (handoff → project-state → pack). Marginal need to verify that all four always-read pack documents actually exist, resolved by cross-referencing project-knowledge-inventory.
|
||||
|
||||
### Routing Failures Found
|
||||
|
||||
One genuine failure: **Routing Test B in task-context-packs.md**. The test states that mock scenarios for long investigation and contradiction are identifiable without loading ui-mock-reference.md. This was presented as a self-evident fact but the specific scenario names only exist in ui-mock-reference.md. The routing is incomplete — it should have included the mock reference file, or at minimum acknowledged that scenario names require verification.
|
||||
|
||||
### Documentation Changes Made
|
||||
|
||||
- Created `docs/context-routing-validation.md` (62 lines) — this experiment's record
|
||||
- Updated `docs/design-evolution-log.md` — appended Experiment 36 entry
|
||||
|
||||
**No source code or tests changed. No archive changes.**
|
||||
|
||||
### Overall Assessment: Mostly ready
|
||||
|
||||
Two of four tasks completed from initial context only. One routing defect found (Task 2; corrected by Experiment 37). After fixing Routing Test B to name ui-mock-reference.md as the scenario source, the reduced context system is ready for normal work.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 37 — Validate Cross-Boundary Context Routing
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
The context-pack system can support cross-boundary work if Claude:
|
||||
1. starts with one primary pack;
|
||||
2. adds a second pack only for a named boundary;
|
||||
3. records why each extra document was loaded;
|
||||
4. avoids loading the full history.
|
||||
|
||||
### Initial Documents Loaded (328 lines total)
|
||||
|
||||
1. `docs/current-handoff.md` — 85 lines; first return-to-work entry point
|
||||
2. `docs/current-project-state.md` — 131 lines; active state and capabilities
|
||||
3. `docs/task-context-packs.md` — 110 lines; routing for four work types
|
||||
|
||||
### Additional Documents Loaded
|
||||
|
||||
| Document | Lines | Why Needed | Routing Should Include? |
|
||||
|---|---|---|---|
|
||||
| `docs/ui-mock-reference.md` | 62 | Cross-boundary boundary: the task requires identifying a mock scenario for workspace display. This is the second pack (UI and Mock) needed because no other loaded document names scenarios or UI fixtures. Yes — it is part of the UI/Mock pack, not an ad-hoc addition. |
|
||||
|
||||
### Cross-Boundary Task Result
|
||||
|
||||
**Task:** Display passive condition-status information in the workspace for a mock investigation without changing the active reasoning loop.
|
||||
|
||||
| Finding | Details |
|
||||
|---|---|
|
||||
| Condition-status capability | Passive: decision-condition status evaluation records signals but has no integration into the turn cycle; never controls user-facing decisions or path selection |
|
||||
| Active reasoning loop | Unchanged: deterministic pipeline (scenario reconstruction → graph update → unknown selection → question formulation → turn orchestration); none of these pathways are affected by passive data |
|
||||
| Mock scenario | "Long investigation (10–15 turns)" from `ui-mock-reference.md`; workspace can display accumulated diagnostic signals over time without interrupting the active reasoning cycle |
|
||||
| Implementation areas to inspect later | decision-condition-status evaluation module; evidence scope detection module; UI workspace components for passive display integration |
|
||||
| Both packs genuinely needed? | Yes: Engine pack identifies which capabilities are active vs passive; UI pack identifies how the workspace presents state. Neither alone suffices |
|
||||
| Archive or full history required? | No |
|
||||
|
||||
**Context remained manageable:** Yes. 390 lines total (328 initial + 62 additional). Each document loaded for a specific named purpose. No blind expansion.
|
||||
|
||||
### Knowledge-Management Completion Criteria Review
|
||||
|
||||
| Criterion | Status |
|
||||
|---|---|
|
||||
| 1. Fresh session can resume from handoff + one pack | met |
|
||||
| 2. Current state verified against implementation | met |
|
||||
| 3. Historical material outside default loading | met |
|
||||
| 4. Current principles separated from aspirational architecture | met |
|
||||
| 5. Task-specific routing works for engine and UI tasks | met |
|
||||
| 6. Cross-boundary task tested | **met** |
|
||||
| 7. Maintaining handoff does not require reading full history | met |
|
||||
|
||||
All seven criteria are now met.
|
||||
|
||||
> Knowledge-management structure is ready for Rob's review before engine experiments resume.
|
||||
|
||||
### Routing Defects Discovered
|
||||
|
||||
None in this experiment. The correction to Routing Test B (naming `ui-mock-reference.md` as the scenario source) was applied before testing. No new defects found in the cross-boundary test.
|
||||
|
||||
### Overall Assessment: Ready
|
||||
|
||||
The context-pack system handled a genuine engine/UI cross-boundary task by combining two packs deliberately with full documentation of each loaded document and its purpose. Context remained small (390 lines). All knowledge-management criteria are met.
|
||||
|
||||
---
|
||||
|
||||
# Experiment 38 — Cold-Start Project Recovery Validation
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
**Type:** Knowledge-management / handoff validation (final KM experiment)
|
||||
**Objective:** Test whether a genuinely cold session can recover the project accurately from the reduced context system alone without reading the full history or any earlier experiment reports.
|
||||
|
||||
## Setup
|
||||
|
||||
Cold-start configuration: no prior conversation context, no past experiment reports loaded, repository documentation carries all context. Session was freshly created to simulate a real return-to-work scenario. Only `docs/current-handoff.md` was read first (per handoff §6 step 1), then the two documents specified by its resume instructions (§6 steps 2–3): `docs/current-project-state.md` and `docs/task-context-packs.md`.
|
||||
|
||||
## Documents Loaded
|
||||
|
||||
| Document | Reason |
|
||||
|---|---|
|
||||
| `docs/current-handoff.md` | Primary entry point (handoff §6 step 1) |
|
||||
| `docs/current-project-state.md` | Resume instruction (§6 step 2) and routing table (§6 step 7) |
|
||||
| `docs/task-context-packs.md` | Pack selection (§6 step 3) and pack contents for verification |
|
||||
|
||||
No additional documents were loaded. No blind expansion occurred. The full design-evolution log, archived documents, UI mock reference, source code, and tests were all excluded by design.
|
||||
|
||||
## Project-State Recovery Result
|
||||
|
||||
The cold session correctly recovered:
|
||||
- What the Confidence Engine does (facilitated investigation with structured reasoning graph).
|
||||
- Active capabilities: deterministic reasoning pipeline, unknown selection via atomicity/answerability, question formulation, scenario API, turn cycle orchestration.
|
||||
- Passive capabilities: seven diagnostic layers from Experiments 18–25B, all isolated, none control user-facing investigation.
|
||||
- Paused work: engine experiments (after Exp 25B), UI experiments.
|
||||
- Why KM phase was undertaken (documentation bloat blocking session recovery).
|
||||
|
||||
Recovery score: complete from three documents alone. No source code inspection required.
|
||||
|
||||
## Context-Pack Selection Result
|
||||
|
||||
Pack 1 — Engine Experiment Work selected correctly by the cold session. The three initial documents contained sufficient information to identify the pack, its default documents, and what to exclude without reading any additional material.
|
||||
|
||||
## Handoff Defects Found
|
||||
|
||||
None found in `docs/current-handoff.md`. The handoff accurately describes the stopping point, identifies all seven KM criteria as met, provides correct resume instructions, and includes accurate capability boundaries. One structural update was made: the open item "whether the handoff stays accurate after further advances" was resolved as no longer applicable (the cold-start test confirmed it is accurate).
|
||||
|
||||
## Completion-Criteria Result
|
||||
|
||||
All seven knowledge-management completion criteria are confirmed met by this cold-start validation:
|
||||
1. Fresh session can resume from handoff + one pack — met (Exp 38 demonstrates this)
|
||||
2. Current state verified against implementation — met (Exp 28+)
|
||||
3. Historical material outside default loading — met
|
||||
4. Current principles separated from aspirational architecture — met
|
||||
5. Task-specific routing works for engine and UI tasks — met (Exp 37)
|
||||
6. Cross-boundary task tested — met (Exp 37)
|
||||
7. Maintaining handoff does not require reading full history — met
|
||||
|
||||
> The knowledge-management phase is complete enough for Rob to choose when engine experiments resume.
|
||||
|
||||
## Documents Updated
|
||||
|
||||
- `docs/cold-start-validation.md` — created (this experiment's deliverable)
|
||||
- `docs/current-handoff.md` — Exp 38 commit placeholder, structural open-item resolution, return-to-work note replacement
|
||||
- `docs/current-project-state.md` — KM status update ("active" → "complete"), latest known commit correction
|
||||
- `docs/design-evolution-log.md` — this entry
|
||||
|
||||
## Overall Assessment: Ready
|
||||
|
||||
The cold-start validation passed. A genuinely fresh session understood the project state, chose the correct context pack, verified the resume boundary, produced a valid engine-work resume brief, and found no handoff defects — all from three documents alone. No source code was read or changed. The reduced context system works for sessions that did not help create the documents.
|
||||
|
||||
Engine and UI experiments remain paused pending Rob's review.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 39 — Validate Behaviour Selection Against Real Assessment Outputs (2026-08-06)
|
||||
|
||||
**Branch:** feature/user-workspace-ux-v0.7
|
||||
|
||||
### Hypothesis
|
||||
The existing deterministic selector produces a useful rhythm across genuine assessment outputs without changing the active engine. If it repeatedly chooses one behaviour, chooses behaviours at the wrong time, or depends on signals the assessor does not actually produce, the experiment should expose that honestly.
|
||||
|
||||
### Scenarios Evaluated (from `tests/investigation-state-assessor.test.js` fixture set)
|
||||
1. **Long investigation** (3 turns: early → deepening → complete terminal)
|
||||
2. **Contradictory evidence** (3 turns: two conflicting consultants, 0→1→2 resolved unknowns)
|
||||
3. **Short early** (1 turn: two observations, first unknown, no resolution)
|
||||
|
||||
### Behaviour Distribution (7 turns total)
|
||||
- Acknowledge: 5 (71%)
|
||||
- Continue: 2 (29%)
|
||||
- Clarify: 0 (0%)
|
||||
- Summarise: 0 (0%)
|
||||
- Pause: 0 (0%)
|
||||
|
||||
### Behaviour Sequence by Scenario
|
||||
**Long investigation:** continue → acknowledge → acknowledge
|
||||
- Turn 0: phase=cannot_determine, progress=cannot_determine, health=too_narrow → continue (no rule matched)
|
||||
- Turn 3: phase=focusing, progress=steady, health=healthy → acknowledge
|
||||
- Turn 4: phase=concluding, progress=steady, health=healthy → acknowledge
|
||||
|
||||
**Contradictory evidence:** acknowledge → acknowledge → acknowledge
|
||||
- Turn 0: phase=focusing, progress=cannot_determine, health=healthy → acknowledge
|
||||
- Turn 1: phase=focusing, progress=stalled, health=healthy → acknowledge
|
||||
- Turn 2: phase=focusing, progress=steady, health=healthy → acknowledge
|
||||
|
||||
**Short early:** continue
|
||||
- Turn 0: phase=exploring, progress=cannot_determine, health=healthy → continue
|
||||
|
||||
### Sensible Selections (7 of 7)
|
||||
All selections were classified as sensible per the selection's stated conditions. Acknowledge fires because `health=healthy AND phase confidence≠low` across most states. Continue fires when no specific rule matches (early/cannot_determine/exploring phases).
|
||||
|
||||
### Questionable or Inappropriate Selections
|
||||
**One notable pattern:** Summarise and Pause never fire, even in a concluding terminal state. This is not because the assessor fails to detect "concluding" — it does. It is because Acknowledge (priority 1) fires first when health=healthy, blocking Summarise (priority 3) from ever reaching its turn. This is an **acknowledgement/summarise priority conflict**: acknowledging a conclusion ("you've figured this out!") is not wrong, but "give me a summary" is more useful at terminal states. The current rule ordering does not distinguish "early healthy" from "concluding healthy."
|
||||
|
||||
Clarify never fires because no test scenario produces `health=too_broad` — the assessor's "too_broad" trigger (activeUnknownCount > 3 AND resolved < 2) requires more nodes than any scenario in the fixture set has at that stage.
|
||||
|
||||
Pause never fires because `health=user_overloaded` is never reached, and while contradictory-turn-1 has phase=focusing + progress=stalled, Acknowledge still blocks it.
|
||||
|
||||
### Contract Alignment
|
||||
Assessor → Selector contract aligns cleanly. The assessor produces all three dimensions (phase, progress, conversationHealth) with the fields the selector expects. No transformation needed between pipeline stages.
|
||||
|
||||
### Whether Selector Appears Useful Enough for Another Passive Experiment
|
||||
The existing selector works but its **behaviour variation is severely constrained by Acknowledge's priority position**. A next passive experiment should test whether reordering or refining the acknowledge condition (e.g., excluding concluding/terminal phases) produces more context-appropriate behaviour — without changing the assessor.
|
||||
|
||||
### Status
|
||||
Pending Rob's review. Five behaviours are too narrow for this to be definitive, and only three scenarios were tested. The dominant pattern (acknowledge in healthy states) may change with different investigation domains.
|
||||
|
||||
### Documents Updated
|
||||
- `docs/design-evolution-log.md` — this entry
|
||||
- `docs/current-handoff.md` — return-to-work note replaced
|
||||
|
||||
---
|
||||
|
||||
## Experiment 40 — Audit Behaviour Reachability and Blocking (2026-08-06)
|
||||
|
||||
### Objective
|
||||
|
||||
Why did Clarify, Summarise, and Pause not appear during Experiment 39? Acknowledge: 5 (71%), Continue: 2 (29%), others: 0. This is a passive diagnostic — no rule changes, no engine modifications.
|
||||
|
||||
### Method
|
||||
|
||||
One test file (`tests/behaviour-selection.reachability.test.js`) containing:
|
||||
- Diagnostic audit helper that evaluates every behaviour rule against one assessment object
|
||||
- Real-scenario audits across the same Experiment 39 turns (8 turns total)
|
||||
- Synthetic reachability checks for each behaviour in isolation
|
||||
|
||||
### Findings
|
||||
|
||||
#### Summarise — eligible_but_blocked
|
||||
|
||||
Eligible in 2 of 7 real turns:
|
||||
- long-investigation turn 1 (resolvedNodeCount ≥ 3 + progress=steady triggers summarise rule)
|
||||
- long-investigation turn 2 (phase=concluding triggers summarise rule)
|
||||
|
||||
In both cases, health=healthy simultaneously, so Acknowledge (priority 1) fires first. Summarise rules are met but its output is never returned because the selector returns early on priority ordering.
|
||||
|
||||
**Root cause: priority conflict, not assessor failure.** The phase evidence correctly identifies concluding/synthesising states; the problem is that Acknowledge's broader trigger condition (health=healthy is the most common state) fires first.
|
||||
|
||||
#### Clarify — never_eligible_in_tested_scenarios (reachable only in synthetic case)
|
||||
|
||||
Not eligible in any of 7 real turns because neither trigger condition is met:
|
||||
- `health=too_broad`: requires activeUnknownCount > 3 AND resolvedNodeCount < 2 — no fixture reaches this state
|
||||
- `phase=orienting + observationDensity < 3`: current assessor never produces phase=orienting for tested scenarios
|
||||
|
||||
Synthetic case confirms the rule fires correctly in isolation (with low-confidence phase to avoid Acknowledge blocking).
|
||||
|
||||
**Root cause: assessor health classification logic produces too few `too_broad` cases. The trigger condition is extremely narrow — needs activeUnknownCount > 3 AND resolved < 2 simultaneously.**
|
||||
|
||||
#### Pause — eligible_but_blocked
|
||||
|
||||
Eligible in 1 of 7 real turns:
|
||||
- contradictory-evidence turn 1 (phase=focusing + progress=stalled triggers pause rule)
|
||||
|
||||
In this case, health=healthy simultaneously, so Acknowledge blocks it. The second pause trigger (`health=user_overloaded`) is never met because the assessor never produces that state.
|
||||
|
||||
**Root cause: same priority conflict as Summarise. One of two rules fires in real data but gets blocked by Acknowledge's earlier position.**
|
||||
|
||||
### Synthetic Reachability Confirmation
|
||||
|
||||
All five behaviours are independently reachable when isolated from Acknowledge:
|
||||
- ✅ acknowledge — healthy + confident phase
|
||||
- ✅ clarify — too_broad health (with low-confidence phase to avoid Acknowledge)
|
||||
- ✅ summarise — synthesising/concluding phase (without healthy health)
|
||||
- ✅ pause — focusing+stalled or user_overloaded (without healthy health)
|
||||
- ✅ continue — no rules match
|
||||
|
||||
### Classifications
|
||||
|
||||
| Behaviour | Classification | Primary Cause |
|
||||
|---|---|---|
|
||||
| Summarise | eligible_but_blocked | Acknowledge priority 1 fires first when health=healthy |
|
||||
| Clarify | never_eligible_in_tested_scenarios (reachable only in synthetic) | `too_broad` trigger too narrow for test scenarios; `orienting+low obs` not produced by assessor |
|
||||
| Pause | eligible_but_blocked | Acknowledge priority 1 fires first when health=healthy; `user_overloaded` never produced |
|
||||
|
||||
### Impact on Prior Finding (Exp 39)
|
||||
|
||||
Experiment 39 concluded "the Acknowledge→Summarise priority conflict prevents Summarise from firing." Experiment 40 confirms this and adds that **Pause faces the same blocking** (1 eligible turn, blocked). Clarify's absence is fundamentally different: its rules are not triggered at all in tested scenarios.
|
||||
|
||||
This means any fix must address two distinct problems:
|
||||
1. Priority conflict affecting Summarise AND Pause (same cause)
|
||||
2. Narrow trigger conditions for Clarify and the `user_overloaded` health state
|
||||
|
||||
### Test Results
|
||||
|
||||
- `tests/behaviour-selection.reachability.test.js`: 33 passed (new diagnostic file)
|
||||
- `tests/behaviour-selection.test.js`: 51 passed (no regressions)
|
||||
- `tests/behaviour-selection.real-assessment.test.js`: 16 passed (shared fixtures intact)
|
||||
- `tests/investigation-state-assessor.test.js`: 51 passed (assessor unchanged)
|
||||
|
||||
### Documents Updated
|
||||
- `docs/design-evolution-log.md` — this entry
|
||||
- `docs/current-handoff.md` — return-to-work note replaced
|
||||
|
||||
## Experiment 41 — Compare Acknowledge Priority Alternatives (2026-08-06)
|
||||
|
||||
### Purpose
|
||||
|
||||
Experiment 40 confirmed Summarise and Pause are eligible_but_blocked by Acknowledge's priority-1 position. Two passive alternatives were compared without modifying production code:
|
||||
|
||||
**Variant A** — Reorder rules so specific behaviours (Summarise, Pause) evaluate before Acknowledge. The idea is that if a more specific behaviour fires first, it captures the terminal/stalled states where Acknowledge should not fire.
|
||||
|
||||
**Variant B** — Keep existing priority order but exclude Acknowledge from firing when phase=concluding/synthesising, progress=stalled, or health=user_overloaded. The idea is to gate Acknowledge rather than reorder everything.
|
||||
|
||||
### Method
|
||||
|
||||
Both variants were implemented as test-only functions in `tests/behaviour-selection.counterfactual.test.js`. Each variant was evaluated against the same 7 real assessment turns from Experiments 39/40 across 3 scenarios. All five behaviours confirmed independently reachable synthetically. No production rules changed.
|
||||
|
||||
### Assessor Outputs (7 real turns)
|
||||
|
||||
| # | Scenario | Turn | Phase (conf) | Progress | Health | Existing |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 1 | long-investigation | 0 | cannot_determine(low) | cannot_determine | too_narrow | continue |
|
||||
| 2 | long-investigation | 3 | focusing(high) | steady | healthy | acknowledge |
|
||||
| 3 | long-investigation | 4 | concluding(high) | steady | healthy | acknowledge |
|
||||
| 4 | contradictory-evidence | 0 | focusing(high) | cannot_determine | healthy | acknowledge |
|
||||
| 5 | contradictory-evidence | 1 | focusing(high) | stalled | healthy | acknowledge |
|
||||
| 6 | contradictory-evidence | 2 | focusing(high) | steady | healthy | acknowledge |
|
||||
| 7 | short-early | 0 | exploring(low) | cannot_determine | healthy | continue |
|
||||
|
||||
### Results on Real Scenarios
|
||||
|
||||
| Turn | Existing | Variant A | Variant B | Change? |
|
||||
|---|---|---|---|---|
|
||||
| long-investigation t3 | acknowledge | summarise | acknowledge | V-A: side-effect |
|
||||
| long-investigation t4 | acknowledge | **summarise** | **summarise** | **convergent ✓** |
|
||||
| contradictory-evidence t1 | acknowledge | **pause** | **pause** | **convergent ✓** |
|
||||
| All others | unchanged | unchanged | unchanged | — |
|
||||
|
||||
### Divergence Analysis
|
||||
|
||||
**Variant A diverges from Variant B at long-investigation turn 3.** Variant A produces `summarise` because its `resolvedNodeCount >= 3 && steady` rule fires at priority 1 without phase context. The assessor confirms this is a focusing-phase state (not synthesising/concluding) where the user needs acknowledgment, not compression. This is a false-positive for summarisation — a side-effect of Variant A's priority reordering.
|
||||
|
||||
**Variant B correctly preserves Acknowledge** at long-investigation t3 because:
|
||||
1. The exclusion list only includes `synthesising`, `concluding`, `stalled`, and `user_overloaded` — not focusing
|
||||
2. SummariseV2 itself has a phase gate (`phase.value === "synthesising"`) that prevents false-fire in focusing states
|
||||
3. Acknowledge at priority 1 wins because no exclusion applies
|
||||
|
||||
### Key Findings
|
||||
|
||||
1. **Both variants converge on the same two genuine changes:** `concluding → summarise` and `stalled → pause`. This was the experiment's primary question, and both approaches answer it correctly.
|
||||
|
||||
2. **Variant A introduces a false-positive:** The `resolvedNodeCount >= 3 && steady` rule fires in focusing-phase states without phase context, causing premature summarisation when Acknowledge would be more useful.
|
||||
|
||||
3. **Variant B has cleaner boundaries:** Explicit exclusion conditions prevent unwanted side-effects while preserving Acknowledge's role as the default healthy-state behaviour.
|
||||
|
||||
4. **Distribution shift (both variants):**
|
||||
- Existing: acknowledge 71%, continue 29%
|
||||
- Variant A: acknowledge 29%, summarise 29%, pause 14%, continue 29%
|
||||
- Variant B: acknowledge 43%, summarise 14%, pause 14%, continue 29%
|
||||
- Variant B preserves more Acknowledge because it doesn't remove the default healthy-state behaviour entirely
|
||||
|
||||
5. **Variant B is architecturally cleaner** for this problem space because it adds a targeted gate to one rule rather than reordering five priority levels — each of which would need individual review for side-effects.
|
||||
|
||||
### Test Results
|
||||
|
||||
- `tests/behaviour-selection.counterfactual.test.js`: 44 passed (new diagnostic file)
|
||||
- `tests/behaviour-selection.reachability.test.js`: 33 passed (no regressions)
|
||||
- `tests/behaviour-selection.real-assessment.test.js`: 16 passed (shared fixtures intact)
|
||||
- `tests/behaviour-selection.test.js`: 51 passed (no regressions)
|
||||
|
||||
### Decision Criteria
|
||||
|
||||
| Criterion | Variant A | Variant B |
|
||||
|---|---|---|
|
||||
| Fixes concluding state | ✓ summarise | ✓ summarise |
|
||||
| Fixes stalled state | ✓ pause | ✓ pause |
|
||||
| No false-positive changes | ✗ long-t3 → summarise | ✓ preserved acknowledge |
|
||||
| Implementation complexity | Simple reordering | Small gate function |
|
||||
| Maintains Acknowledge for healthy focus states | ? (depends on future review) | ✓ explicit preservation |
|
||||
|
||||
### Recommendation
|
||||
|
||||
**Variant B is preferred.** Both variants correctly identify the two genuine changes needed. Variant B has no false-positives, cleaner architectural boundaries (targeted exclusion vs priority reordering), and better preserves the existing Acknowledge default for healthy focusing states where it is appropriate. A recommended implementation would:
|
||||
|
||||
1. Keep existing priority order
|
||||
2. Add `isAcknowledgeExcluded()` function with conditions: phase∈{synthesising, concluding}, progress=stalled, health=user_overloaded
|
||||
3. Gate Acknowledge through this exclusion before selecting it at priority 1
|
||||
|
||||
### Documents Updated
|
||||
- `docs/design-evolution-log.md` — this entry
|
||||
- `docs/current-handoff.md` — return-to-work note replaced
|
||||
|
||||
|
||||
## Experiment 41 — Conclusion
|
||||
|
||||
**Variant B was preferred because it changed only the two intended turns without introducing a false-positive in a focusing state. Variant A produced an early summarise in a focusing phase and was discarded. No production rule changed during Experiment 41.** The implementation of Variant B's exclusion gate is the subject of Experiment 42.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user