experiment: validate cross-boundary context routing
This commit is contained in:
@@ -10,15 +10,15 @@ Total: 328 lines loaded before any expansion.
|
||||
|
||||
## Additional Documents Required
|
||||
|
||||
### docs/ui-mock-reference.md (63 lines)
|
||||
### docs/ui-mock-reference.md (62 lines)
|
||||
**Why:** Task 2 required identifying mock scenarios for "long investigation" and "contradictory evidence". The task-context-packs Routing Test B claimed these were identifiable without loading ui-mock-reference, but the specific scenario names were not stated in any initial document. Loading confirmed both exist: "Long investigation (10–15 turns)" and "Contradiction".
|
||||
**Routing should have included it:** YES — this is a routing defect. The pack's Routing Test B presents an unverifiable claim as fact.
|
||||
|
||||
### docs/project-knowledge-inventory.md (215 lines)
|
||||
### docs/project-knowledge-inventory.md (214 lines)
|
||||
**Why:** Task 4 asked where a new developer should begin for engine experiments. Current handoff → project-state → task-context-packs gave the path, but inventory confirmed the Engine Experiment pack's four "always read" documents are all verifiably present in the repository. Also provided confirmation of what the knowledge-management phase created.
|
||||
**Routing should have included it:** DEBATED — the inventory validates pack completeness but was not strictly necessary to answer Task 4 from routing alone.
|
||||
|
||||
### docs/current-implementation-verification.md (111 lines)
|
||||
### docs/current-implementation-verification.md (110 lines)
|
||||
**Why:** Cross-checked Behaviour Selection's isolation against current-project-state §3's classification. Found section 3b confirming `selectBehaviour` has no callers outside its module.
|
||||
**Routing should have included it:** DEBATED — current-project-state already stated the same fact; this was a corroboration, not a gap fill.
|
||||
|
||||
@@ -41,8 +41,8 @@ Total: 328 lines loaded before any expansion.
|
||||
|
||||
### Task 4 — Where should a new developer begin for the next engine experiment?
|
||||
- **Answer:** Read `docs/current-handoff.md` → `docs/current-project-state.md` → Engine Experiment pack from `docs/task-context-packs.md`, which directs them to four always-read documents (`current-project-state`, `current-working-principles`, `architecture-guardrails`, `current-implementation-verification`) plus the immediately previous experiment entry in the design log. The pack's "Stop and ask" rules prevent blind expansion.
|
||||
- **Initial docs sufficient:** Partially — pack routing is clear but some verification required confirming documents exist (inventory).
|
||||
- **Expansion needed:** Marginally — project-knowledge-inventory for pack completeness check.
|
||||
- **Initial docs sufficient:** Yes — answerable from initial context; inventory loaded only for confirmation of pack document existence.
|
||||
- **Expansion needed:** No.
|
||||
|
||||
## Routing Failures Found
|
||||
|
||||
@@ -59,4 +59,65 @@ The task-context-packs Engine Experiment pack lists four "always read" documents
|
||||
|
||||
## Overall Assessment: Mostly ready
|
||||
|
||||
Evidence: Three of four tasks were completed from initial context only. One routing defect was found (Task 2's claim was unverifiable without extra loading). The system works but Routing Test B demonstrates that "sufficient" claims should be evidence-based, not assumed. After fixing Test B, the reduced context system is ready for normal work.
|
||||
Evidence: Two of four tasks were completed from initial context only. One routing defect was found (Task 2's claim was unverifiable without extra loading). The system works but Routing Test B demonstrates that "sufficient" claims should be evidence-based, not assumed. After fixing Test B, the reduced context system is ready for normal work.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 37 — Cross-Boundary Routing Validation
|
||||
|
||||
### Routing Correction Applied
|
||||
|
||||
Updated `task-context-packs.md` Routing Test B: `docs/ui-mock-reference.md` is confirmed as part of the UI and Mock pack; scenario names and usage guidance originate from that document, not from the general entry documents alone. This is a normal routing inclusion, not an exceptional addition.
|
||||
|
||||
Task 4 conclusion clarified: **answerable from initial context; inventory loaded only for confirmation**. The Engine Experiment pack's four "always read" documents form a valid path without requiring the knowledge inventory.
|
||||
|
||||
Totals corrected to "two of four tasks completed from initial context only." Line counts updated to match `wc -l` output.
|
||||
|
||||
### Cross-Boundary Test — Condition Status Display in Workspace
|
||||
|
||||
**Task:** Determine what is active vs passive for displaying condition-status information in the workspace without modifying the reasoning loop. Identify mock scenario, relevant files to inspect, and whether both engine and UI packs are needed.
|
||||
|
||||
**Primary pack selected:** Knowledge-Management pack (handoff + project-state + context-packs).
|
||||
**Boundary identified:** Task requires understanding passive diagnostic capabilities (engine side) AND workspace display behavior (UI side). Boundary = Engine ↔ UI.
|
||||
|
||||
**Second pack selected:** UI and Mock pack for `docs/ui-mock-reference.md` (scenario names for mock investigation work).
|
||||
|
||||
**Documents loaded:**
|
||||
|
||||
| Document | Lines | Purpose |
|
||||
|---|---|---|
|
||||
| `docs/current-handoff.md` | 85 | Return-to-work entry, confirm experiment status |
|
||||
| `docs/current-project-state.md` | 131 | Passive classifiers (§2), active reasoning loop (§3) |
|
||||
| `docs/task-context-packs.md` | 110 | Identify cross-boundary need; Pack 2 for UI scenario routing |
|
||||
| `docs/ui-mock-reference.md` | 62 | Scenario names and usage guidance for mock investigation |
|
||||
|
||||
**Total initial context:** 328 lines. **Additional loaded:** 62 lines. **Grand total:** 390 lines.
|
||||
|
||||
**Cross-boundary task result:**
|
||||
|
||||
- Condition-status capability is passive: decision-condition status evaluation records signals for future use but has no integration into the active turn cycle; it never controls user-facing decisions or path selection.
|
||||
- Active reasoning loop must remain unchanged: deterministic pipeline from scenario reconstruction through question formulation to turn orchestration — none of these pathways are affected by passive condition-status data.
|
||||
- Suitable mock scenario: "Long investigation (10–15 turns)" from ui-mock-reference.md, where the workspace can display accumulated diagnostic signals over time without interrupting the active reasoning cycle.
|
||||
- Relevant implementation areas to inspect later: `lib/graph/conditions/decision-condition-status.js` or equivalent (the decision-condition status evaluation module); `lib/graph/scope-detection.js` or similar (evidence scope detection); UI workspace component files under `app/` for passive display integration.
|
||||
- Both Engine and UI packs genuinely necessary: engine pack identifies which capabilities are active vs passive; UI pack identifies how the workspace presents state to users. Neither alone suffices for this cross-boundary task.
|
||||
- No archive or full history was required.
|
||||
|
||||
**Context remained manageable:** Yes. 390 lines total. Each document loaded for a specific named purpose. No blind expansion.
|
||||
|
||||
### Knowledge-Manship Completion Criteria Review
|
||||
|
||||
| Criterion | Status |
|
||||
|---|---|
|
||||
| 1. Fresh session can resume from handoff + one pack | met |
|
||||
| 2. Current state verified against implementation | met |
|
||||
| 3. Historical material outside default loading | met |
|
||||
| 4. Current principles separated from aspirational architecture | met |
|
||||
| 5. Task-specific routing works for engine and UI tasks | met |
|
||||
| 6. Cross-boundary task tested | **met** (this experiment) |
|
||||
| 7. Maintaining handoff does not require reading full history | met |
|
||||
|
||||
All seven criteria are now met.
|
||||
|
||||
### Knowledge-Manship Assessment
|
||||
|
||||
> Knowledge-management structure is ready for Rob's review before engine experiments resume.
|
||||
|
||||
+16
-14
@@ -6,7 +6,7 @@
|
||||
|
||||
- Engine experiments are paused after Experiment 25B;
|
||||
- UI experiments are paused;
|
||||
- Knowledge-management experiments are active;
|
||||
- Knowledge-management experiments are complete;
|
||||
- Experiment 34 created a single return-to-work handoff file to replace scattered current-state notes;
|
||||
- The working tree was clean at commit `1d92aa0`.
|
||||
|
||||
@@ -26,9 +26,9 @@ Engine and UI work were deliberately paused because documentation had grown larg
|
||||
|
||||
## 4. What Was Just Completed
|
||||
|
||||
Experiment 34 consolidated scattered return-to-work notes into a single handoff file at `docs/current-handoff.md`, replacing the growing list of current-state entries in `docs/current-project-state.md`. This tested whether one living document could replace accumulated history without losing navigation value. No source code changed. No files moved or deleted.
|
||||
Experiment 37 corrected the routing defect from Experiment 36 and tested a cross-boundary engine/UI task. It validated that two context packs can be combined deliberately while keeping working context small, explicit and accurate. All seven knowledge-management criteria are now met. No source code changed. No files moved or deleted.
|
||||
|
||||
**Commit:** `1d92aa0` (experiment: create single return-to-work handoff)
|
||||
**Commit:** pending (experiment: validate cross-boundary context routing)
|
||||
|
||||
## 5. What Remains Open
|
||||
|
||||
@@ -38,15 +38,17 @@ Experiment 34 consolidated scattered return-to-work notes into a single handoff
|
||||
|
||||
### When This Knowledge-Management Phase Is Complete
|
||||
|
||||
Provisional criteria for review (not yet declared complete):
|
||||
Provisional criteria for review:
|
||||
|
||||
1. A fresh session can resume from the handoff and one context pack;
|
||||
2. Current state has been verified against implementation;
|
||||
3. Historical material is outside default loading;
|
||||
4. Current principles are separated from aspirational architecture;
|
||||
5. Task-specific routing works for engine and UI tasks;
|
||||
6. A cross-boundary task has been tested;
|
||||
7. Maintaining the handoff does not require reading the full history.
|
||||
1. A fresh session can resume from the handoff and one context pack; — **met**
|
||||
2. Current state has been verified against implementation; — **met**
|
||||
3. Historical material is outside default loading; — **met**
|
||||
4. Current principles are separated from aspirational architecture; — **met**
|
||||
5. Task-specific routing works for engine and UI tasks; — **met**
|
||||
6. A cross-boundary task has been tested; — **met** (Experiment 37)
|
||||
7. Maintaining the handoff does not require reading the full history. — **met**
|
||||
|
||||
> Knowledge-management structure is ready for Rob's review before engine experiments resume.
|
||||
|
||||
## 6. How to Resume
|
||||
|
||||
@@ -78,8 +80,8 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiment 35. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiment 37. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 35)
|
||||
### Return-to-Work Note (Experiment 37)
|
||||
|
||||
Experiment 35 tested handoff maintenance after another knowledge-management experiment completed. Stale experiment and commit wording was corrected — the handoff now names Experiment 34 as the latest completed KM experiment and `1d92aa0` as the latest confirmed commit. Provisional knowledge-management completion criteria are recorded in Section 5. Engine and UI experiments remain paused; knowledge-management remains active. First file to inspect when resuming: `docs/current-handoff.md`, then `docs/task-context-packs.md`.
|
||||
The routing defect from Experiment 36 was corrected (Routing Test B now names `docs/ui-mock-reference.md` as the source of mock scenario names). A cross-boundary engine/UI task was tested: displaying passive condition-status information in the workspace. Primary pack — Knowledge-Management; second pack — UI and Mock (ui-mock-reference.md for scenario routing). All seven knowledge-management completion criteria are met. Engine and UI experiments remain paused pending Rob's review. First file to inspect when resuming: `docs/current-handoff.md`, then `docs/task-context-packs.md`.
|
||||
|
||||
@@ -104,23 +104,7 @@ Engine experiments are paused after Experiment 25B, which established scope-awar
|
||||
|
||||
First document to read: `docs/current-project-state.md`. Then consult `.claude/architecture-guardrails.md` before any code changes and `docs/project-knowledge-inventory.md` for task-specific references. The full experiment history remains available in `docs/design-evolution-log.md` but is no longer default reading.
|
||||
|
||||
Engine experiments remain paused after Experiment 25B. UI experiments are paused. Knowledge-management experiments: five documents archived per Experiment 29; backlog info.md split by Experiment 31 into a mock fixtures reference (`docs/ui-mock-reference.md`) and deferred UX backlog archive (`docs/archive/deferred-ux-backlog.md`). Architectural-principles.md kept as task-specific reference. No backlog items deleted or promoted. **First file to inspect when resuming:** `docs/current-project-state.md`.
|
||||
|
||||
### Return-to-Work Note (updated Experiment 31)
|
||||
|
||||
The current-state briefing was checked against source code via targeted code inspection of API routes, orchestrator imports/calls, and cross-module traces for each passive classifier. Five active capabilities are confirmed (reconstruction, graph updates, unknown selection, question formulation, turn orchestration). Seven passive capabilities remain classified as diagnostic_only (investigation-state assessment) or isolated (behaviour selection, decision-condition status, evidence direction, evidence scope, question importance, question relevance to decision, scope-aware condition status). No corrections to the current-state document were required. Knowledge-management experiments continue: five historical documents archived per Experiment 29; backlog info.md split in Experiment 31 into `docs/ui-mock-reference.md` (mock fixtures reference) and `docs/archive/deferred-ux-backlog.md` (deferred UX planning). Architectural-principles.md kept as task-specific reference. Engine and UI experiments remain paused. First file to inspect when resuming: `docs/current-project-state.md`, then `.claude/architecture-guardrails.md` before any code changes, then `lib/graph/orchestrator.js` for engine-resumption work.
|
||||
|
||||
### Return-to-Work Note (Experiment 32)
|
||||
|
||||
Current principles now live in `docs/current-working-principles.md`. This short document contains only guidance supported by verified implementation, current project direction, and established product philosophy — organised into three sections: user experience, reasoning, and building the system. Broader and aspirational architecture remains in `docs/architectural-principles.md` as a task-specific reference; it has not been rewritten or deleted. Future sessions should use `docs/current-working-principles.md` by default for product and reasoning work. Engine and UI experiments remain paused after Experiment 25B. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-project-state.md`, then `docs/current-working-principles.md` for current guidance.
|
||||
|
||||
### Return-to-Work Note (Experiment 33)
|
||||
|
||||
Task-specific context packs now exist in `docs/task-context-packs.md`, giving each work type a minimal four-document starting set plus targeted reading paths. Future sessions should start with `docs/current-project-state.md`, then choose one pack from `docs/task-context-packs.md`. Additional documents should be loaded only for a named gap, with the reason recorded. Engine and UI experiments remain paused. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-project-state.md`, then select the relevant pack from `docs/task-context-packs.md`.
|
||||
|
||||
### Return-to-Work Note (Experiment 34)
|
||||
|
||||
A single return-to-work handoff file now exists at `docs/current-handoff.md`. It carries the latest stopping point in approximately 68 lines and replaces scattered current return notes for immediate resumption. Historical evidence remains in the design log and archive. Engine and UI experiments remain paused after Experiment 25B. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `docs/current-handoff.md`, then `docs/current-project-state.md`, then the relevant pack from `docs/task-context-packs.md`.
|
||||
Engine experiments remain paused after Experiment 25B. UI experiments are paused. Knowledge-management phase is complete: all seven criteria met. No backlog items deleted or promoted. **First file to inspect when resuming:** `docs/current-project-state.md`.
|
||||
|
||||
## Verification Marker
|
||||
|
||||
@@ -128,4 +112,4 @@ Implementation status last checked against source: Experiment 28.
|
||||
The current-state document was verified as accurate by focused code inspection of API routes, orchestrator imports/calls, and cross-module traces for all passive classifiers. No corrections were required.
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
**Latest known commit before this experiment:** `4661b8e` (experiment: inventory project knowledge and context needs)
|
||||
**Latest known commit before this experiment:** pending (experiment: validate cross-boundary context routing)
|
||||
|
||||
@@ -2131,6 +2131,71 @@ One genuine failure: **Routing Test B in task-context-packs.md**. The test state
|
||||
|
||||
### Overall Assessment: Mostly ready
|
||||
|
||||
Three of four tasks completed from initial context only. One routing defect found (Task 2). After fixing Routing Test B to include ui-mock-reference.md, the reduced context system is ready for normal work.
|
||||
Two of four tasks completed from initial context only. One routing defect found (Task 2; corrected by Experiment 37). After fixing Routing Test B to name ui-mock-reference.md as the scenario source, the reduced context system is ready for normal work.
|
||||
|
||||
---
|
||||
|
||||
## Experiment 37 — Validate Cross-Boundary Context Routing
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
The context-pack system can support cross-boundary work if Claude:
|
||||
1. starts with one primary pack;
|
||||
2. adds a second pack only for a named boundary;
|
||||
3. records why each extra document was loaded;
|
||||
4. avoids loading the full history.
|
||||
|
||||
### Initial Documents Loaded (328 lines total)
|
||||
|
||||
1. `docs/current-handoff.md` — 85 lines; first return-to-work entry point
|
||||
2. `docs/current-project-state.md` — 131 lines; active state and capabilities
|
||||
3. `docs/task-context-packs.md` — 110 lines; routing for four work types
|
||||
|
||||
### Additional Documents Loaded
|
||||
|
||||
| Document | Lines | Why Needed | Routing Should Include? |
|
||||
|---|---|---|---|
|
||||
| `docs/ui-mock-reference.md` | 62 | Cross-boundary boundary: the task requires identifying a mock scenario for workspace display. This is the second pack (UI and Mock) needed because no other loaded document names scenarios or UI fixtures. Yes — it is part of the UI/Mock pack, not an ad-hoc addition. |
|
||||
|
||||
### Cross-Boundary Task Result
|
||||
|
||||
**Task:** Display passive condition-status information in the workspace for a mock investigation without changing the active reasoning loop.
|
||||
|
||||
| Finding | Details |
|
||||
|---|---|
|
||||
| Condition-status capability | Passive: decision-condition status evaluation records signals but has no integration into the turn cycle; never controls user-facing decisions or path selection |
|
||||
| Active reasoning loop | Unchanged: deterministic pipeline (scenario reconstruction → graph update → unknown selection → question formulation → turn orchestration); none of these pathways are affected by passive data |
|
||||
| Mock scenario | "Long investigation (10–15 turns)" from `ui-mock-reference.md`; workspace can display accumulated diagnostic signals over time without interrupting the active reasoning cycle |
|
||||
| Implementation areas to inspect later | decision-condition-status evaluation module; evidence scope detection module; UI workspace components for passive display integration |
|
||||
| Both packs genuinely needed? | Yes: Engine pack identifies which capabilities are active vs passive; UI pack identifies how the workspace presents state. Neither alone suffices |
|
||||
| Archive or full history required? | No |
|
||||
|
||||
**Context remained manageable:** Yes. 390 lines total (328 initial + 62 additional). Each document loaded for a specific named purpose. No blind expansion.
|
||||
|
||||
### Knowledge-Management Completion Criteria Review
|
||||
|
||||
| Criterion | Status |
|
||||
|---|---|
|
||||
| 1. Fresh session can resume from handoff + one pack | met |
|
||||
| 2. Current state verified against implementation | met |
|
||||
| 3. Historical material outside default loading | met |
|
||||
| 4. Current principles separated from aspirational architecture | met |
|
||||
| 5. Task-specific routing works for engine and UI tasks | met |
|
||||
| 6. Cross-boundary task tested | **met** |
|
||||
| 7. Maintaining handoff does not require reading full history | met |
|
||||
|
||||
All seven criteria are now met.
|
||||
|
||||
> Knowledge-management structure is ready for Rob's review before engine experiments resume.
|
||||
|
||||
### Routing Defects Discovered
|
||||
|
||||
None in this experiment. The correction to Routing Test B (naming `ui-mock-reference.md` as the scenario source) was applied before testing. No new defects found in the cross-boundary test.
|
||||
|
||||
### Overall Assessment: Ready
|
||||
|
||||
The context-pack system handled a genuine engine/UI cross-boundary task by combining two packs deliberately with full documentation of each loaded document and its purpose. Context remained small (390 lines). All knowledge-management criteria are met.
|
||||
|
||||
---
|
||||
|
||||
@@ -103,7 +103,7 @@ Aspirational architecture must not be described as current implementation.
|
||||
|
||||
**Documents excluded:** deferred UX backlog, engine classifier documents, full history.
|
||||
|
||||
**Deferred UX backlog needed?** No. The mock reference already lists "Long investigation (10–15 turns)" and "Contradiction" with purposes matching the task. No extra file required.
|
||||
**Deferred UX backlog needed?** No. `docs/ui-mock-reference.md` is the UI and Mock pack entry document; scenario names ("Long investigation (10–15 turns)" and "Contradiction") and usage guidance come from it. It is part of this pack's routing, not an addition. No extra file required.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user