experiment: validate reduced context routing
This commit is contained in:
@@ -2079,3 +2079,58 @@ Pending Rob's review.
|
||||
**Handoff remained concise?** Yes. 86 lines (was 68). Increase justified by the maintenance principle paragraph, updated current-state wording, and provisional completion criteria section. No historical timeline appended.
|
||||
|
||||
**Status:** Pending Rob's review.
|
||||
|
||||
## Experiment 36 — Validate Reduced Context Routing
|
||||
|
||||
**Branch:** `feature/user-workspace-ux-v0.7`
|
||||
|
||||
### Hypothesis
|
||||
|
||||
The documentation system (handoff + project-state + task-context-packs) is complete enough to support normal work without silently expanding into historical documentation. A fresh session can complete representative tasks using only routing instructions.
|
||||
|
||||
### Initial Documents Loaded (328 lines total)
|
||||
|
||||
1. `docs/current-handoff.md` — 86 lines
|
||||
2. `docs/current-project-state.md` — 132 lines
|
||||
3. `docs/task-context-packs.md` — 110 lines
|
||||
|
||||
### Additional Documents Loaded
|
||||
|
||||
| Document | Lines | Why Needed | Routing Should Include? |
|
||||
|---|---|---|---|
|
||||
| `docs/ui-mock-reference.md` | 63 | Task 2: verify mock scenarios for "long investigation" and "contradiction". Routing Test B claimed these were identifiable without loading it, but the specific scenario names do not appear in any initial document. | YES — routing defect found |
|
||||
| `docs/project-knowledge-inventory.md` | 215 | Task 4: confirm Engine Experiment pack's four always-read documents actually exist and understand KM phase outputs. | Debated — validated completeness but not strictly required by routing |
|
||||
| `docs/current-implementation-verification.md` | 111 | Cross-checked Behaviour Selection isolation against current-project-state §3. Provided corroboration but was not the sole basis for Task 1 answer. | Debated — useful corroboration; current-project-state alone sufficed |
|
||||
|
||||
### Tasks Completed Without Context Expansion
|
||||
|
||||
**Task 1 — Does Behaviour Selection affect engine behaviour?**
|
||||
No. Current project state §3 classifies it as isolated. Handoff §2 confirms passive classifiers don't control the investigation. Task-context-packs Routing Test A corroborates (current-implementation-verification §3b).
|
||||
|
||||
**Task 3 — Why passive classifiers are not yet in the active reasoning loop?**
|
||||
Passive classifiers record diagnostic signals for future use but have no integration into the turn cycle. Only investigation-state assessment is called (at 3 orchestrator sites), and its result goes into a diagnostics field — never checked by conditional branches. Others have zero callers.
|
||||
|
||||
### Tasks Requiring Extra Context
|
||||
|
||||
**Task 2 — Mock scenarios for long investigation and contradictory evidence**
|
||||
Required `docs/ui-mock-reference.md`. Routing Test B in task-context-packs claimed these were identifiable without loading it, but the specific scenario names ("Long investigation (10–15 turns)" and "Contradiction") do not appear in any initial document. The routing claim was unverifiable until the mock reference was loaded — this is a genuine routing defect.
|
||||
|
||||
**Task 4 — Where should a new developer begin for the next engine experiment?**
|
||||
Partially answered from initial documents (handoff → project-state → pack). Marginal need to verify that all four always-read pack documents actually exist, resolved by cross-referencing project-knowledge-inventory.
|
||||
|
||||
### Routing Failures Found
|
||||
|
||||
One genuine failure: **Routing Test B in task-context-packs.md**. The test states that mock scenarios for long investigation and contradiction are identifiable without loading ui-mock-reference.md. This was presented as a self-evident fact but the specific scenario names only exist in ui-mock-reference.md. The routing is incomplete — it should have included the mock reference file, or at minimum acknowledged that scenario names require verification.
|
||||
|
||||
### Documentation Changes Made
|
||||
|
||||
- Created `docs/context-routing-validation.md` (62 lines) — this experiment's record
|
||||
- Updated `docs/design-evolution-log.md` — appended Experiment 36 entry
|
||||
|
||||
**No source code or tests changed. No archive changes.**
|
||||
|
||||
### Overall Assessment: Mostly ready
|
||||
|
||||
Three of four tasks completed from initial context only. One routing defect found (Task 2). After fixing Routing Test B to include ui-mock-reference.md, the reduced context system is ready for normal work.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user