experiment: validate reduced context routing

docs/context-routing-validation.md

@@ -0,0 +1,62 @@
+# Context Routing Validation — Experiment 36
+
+## Documents Initially Loaded
+
+1. `docs/current-handoff.md` (86 lines) — first return-to-work entry point
+2. `docs/current-project-state.md` (132 lines) — active state and capabilities
+3. `docs/task-context-packs.md` (110 lines) — routing for four work types
+
+Total: 328 lines loaded before any expansion.
+
+## Additional Documents Required
+
+### docs/ui-mock-reference.md (63 lines)
+**Why:** Task 2 required identifying mock scenarios for "long investigation" and "contradictory evidence". The task-context-packs Routing Test B claimed these were identifiable without loading ui-mock-reference, but the specific scenario names were not stated in any initial document. Loading confirmed both exist: "Long investigation (10–15 turns)" and "Contradiction".
+**Routing should have included it:** YES — this is a routing defect. The pack's Routing Test B presents an unverifiable claim as fact.
+
+### docs/project-knowledge-inventory.md (215 lines)
+**Why:** Task 4 asked where a new developer should begin for engine experiments. Current handoff → project-state → task-context-packs gave the path, but inventory confirmed the Engine Experiment pack's four "always read" documents are all verifiably present in the repository. Also provided confirmation of what the knowledge-management phase created.
+**Routing should have included it:** DEBATED — the inventory validates pack completeness but was not strictly necessary to answer Task 4 from routing alone.
+
+### docs/current-implementation-verification.md (111 lines)
+**Why:** Cross-checked Behaviour Selection's isolation against current-project-state §3's classification. Found section 3b confirming `selectBehaviour` has no callers outside its module.
+**Routing should have included it:** DEBATED — current-project-state already stated the same fact; this was a corroboration, not a gap fill.
+
+## Tasks Completed
+
+### Task 1 — Does Behaviour Selection affect engine behaviour?
+- **Answer:** No. It is isolated — no import or call exists in any file under lib/ or app/.
+- **Initial docs sufficient:** Yes (current-project-state §3 + handoff §2).
+- **Expansion needed:** No.
+
+### Task 2 — Correct mock scenarios for long investigation and contradictory evidence?
+- **Answer:** "Long investigation (10–15 turns)" and "Contradiction" from ui-mock-reference.md.
+- **Initial docs sufficient:** No. Routing Test B claimed they were, but the claim was unverifiable until ui-mock-reference was loaded.
+- **Expansion needed:** Yes — `docs/ui-mock-reference.md`.
+
+### Task 3 — Why passive classifiers are not part of active reasoning?
+- **Answer:** Passive classifiers (Experiments 18–25B) record diagnostic signals for future use but have no integration into the turn cycle. Investigation-state assessment is the only one called at all, and its result goes into a diagnostics field — never checked by conditional branches. Others have zero callers. None control user-facing decisions or path selection.
+- **Initial docs sufficient:** Yes (current-project-state §2–§5 + handoff §2).
+- **Expansion needed:** No.
+
+### Task 4 — Where should a new developer begin for the next engine experiment?
+- **Answer:** Read `docs/current-handoff.md` → `docs/current-project-state.md` → Engine Experiment pack from `docs/task-context-packs.md`, which directs them to four always-read documents (`current-project-state`, `current-working-principles`, `architecture-guardrails`, `current-implementation-verification`) plus the immediately previous experiment entry in the design log. The pack's "Stop and ask" rules prevent blind expansion.
+- **Initial docs sufficient:** Partially — pack routing is clear but some verification required confirming documents exist (inventory).
+- **Expansion needed:** Marginally — project-knowledge-inventory for pack completeness check.
+
+## Routing Failures Found
+
+**One genuine failure: Routing Test B in task-context-packs.md.**
+The test states that mock scenarios for "long investigation" and "contradiction" are identifiable without loading ui-mock-reference. This was presented as a self-evident fact but could not be verified from the stated documents alone — the specific scenario names exist only in ui-mock-reference.md. The routing is incomplete; it should have included the mock reference file.
+
+**One questionable exclusion: project-knowledge-inventory for Task 4.**
+The task-context-packs Engine Experiment pack lists four "always read" documents but does not themselves confirm all four exist. A cautious developer would load the inventory to verify, adding ~215 lines. This is acceptable cost but worth noting as a gap in the pack's self-validation.
+
+## Documentation Improvements Discovered
+
+1. **Routing Test B must include ui-mock-reference.md.** Remove the "No extra file required" claim and add the mock reference to the Engine Experiment pack's routing chain when tasks involve scenario selection.
+2. **Packs should confirm their listed documents exist.** Adding a verification check (or removing unverified entries) would prevent the need for inventory cross-referencing.
+
+## Overall Assessment: Mostly ready
+
+Evidence: Three of four tasks were completed from initial context only. One routing defect was found (Task 2's claim was unverifiable without extra loading). The system works but Routing Test B demonstrates that "sufficient" claims should be evidence-based, not assumed. After fixing Test B, the reduced context system is ready for normal work.