experiment: identify clarification target

This commit is contained in:
2026-08-07 20:03:27 +01:00
parent ee1391cd00
commit 6944f358a9
3 changed files with 470 additions and 7 deletions
+7 -6
View File
@@ -20,10 +20,11 @@
- `docs/current-project-state.md` is the main entry point for active project state.
- Experiment 54D confirmed the production update prompt explicitly separates the user answer (## User Answer section) but the proposal schema has no provenance field — source identity at prompt level is explicit, per-node provenance at output level is absent.
Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). The model did not collapse all ambiguity into clarification, did not generate any question or choose a winner. No production code changed. Status pending Rob's review.
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.
Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). Across the three tested disagreement patterns, the model did not automatically map disagreement to user clarification. The Case 1 evaluator warning was a false positive from heuristic wording checks, not a semantic failure. No production code changed. Status pending Rob's review.
## 3. Why Work Is Paused
Experiment 54S tested whether, once clarification is known to be required, the model can identify exactly what the user needs to clarify — three fixed cases: growth-versus-risk priority (true → "preference/trade-off or hard constraint"), evidence-resolvable delivery causes (false → null), ambiguous meaning of "affordable" (true → "upfront cost versus long-term total cost"). 3/3 correctly classified. The model can identify the specific unresolved user-owned distinction without generating a question, choosing a winner, or changing Behaviour Selection. One behavioral note: when requiresUserClarification is false, the model sometimes defaults to producing a target rather than respecting null — but this did not affect any of the three tested cases in this run. Same host/model; no production code changed. Status pending Rob's review.
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.
Engine and UI work were deliberately paused because documentation had grown large enough to overload Claude and make returning across sessions difficult. The current phase is simplifying what a fresh session must load to understand the project, without losing evidential history. Historical material remains available under `docs/archive/`.
@@ -143,8 +144,8 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 3853, 54A54R. Branch: `feature/user-workspace-ux-v0.7`.*
*Created by Experiment 34. Updated by Experiments 3853, 54A54S. Branch: `feature/user-workspace-ux-v0.7`.*
### Return-to-Work Note (Experiment 54R)
### Return-to-Work Note (Experiment 54S)
54Q showed one structured hypothesis → evidence → consequence call worked across three tested cases but did not isolate which reformulation caused recovery. 54R tested whether a consequential disagreement actually requires the user or can be resolved through evidence — three cases: competing delivery causes (false), growth-versus-risk ambiguity (true), no-material-disagreement control (false). All three correct (3/3). The model distinguished missing evidence from missing user meaning without generating any question, choosing a winner, or changing Behaviour Selection. Same host/model; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`. First test/file to inspect when resuming: `tests/reconstruction/semantic-disagreement-resolution-source.test.js`.
54R showed that evidence uncertainty and user-owned ambiguity can be distinguished across three tested patterns. 54S tested whether, once clarification is known to be required, the model can identify exactly what the user needs to clarify — growth-versus-risk priority correctly identified preference/trade-off versus hard constraint; delivery causes correctly returned null (evidence-resolvable); affordable ambiguity correctly identified upfront versus long-term cost meaning. 3/3 clarified-target classification correct. No clarification question was generated. Behaviour Selection remained untouched. Same host/model (qwen-claude:latest on 192.168.1.111:11434); no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`. First test/file to inspect when resuming: `tests/reconstruction/semantic-clarification-target.test.js`.