experiment: identify clarification target
This commit is contained in:
@@ -20,10 +20,11 @@
|
||||
- `docs/current-project-state.md` is the main entry point for active project state.
|
||||
- Experiment 54D confirmed the production update prompt explicitly separates the user answer (## User Answer section) but the proposal schema has no provenance field — source identity at prompt level is explicit, per-node provenance at output level is absent.
|
||||
|
||||
Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). The model did not collapse all ambiguity into clarification, did not generate any question or choose a winner. No production code changed. Status pending Rob's review.
|
||||
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.
|
||||
Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). Across the three tested disagreement patterns, the model did not automatically map disagreement to user clarification. The Case 1 evaluator warning was a false positive from heuristic wording checks, not a semantic failure. No production code changed. Status pending Rob's review.
|
||||
|
||||
## 3. Why Work Is Paused
|
||||
Experiment 54S tested whether, once clarification is known to be required, the model can identify exactly what the user needs to clarify — three fixed cases: growth-versus-risk priority (true → "preference/trade-off or hard constraint"), evidence-resolvable delivery causes (false → null), ambiguous meaning of "affordable" (true → "upfront cost versus long-term total cost"). 3/3 correctly classified. The model can identify the specific unresolved user-owned distinction without generating a question, choosing a winner, or changing Behaviour Selection. One behavioral note: when requiresUserClarification is false, the model sometimes defaults to producing a target rather than respecting null — but this did not affect any of the three tested cases in this run. Same host/model; no production code changed. Status pending Rob's review.
|
||||
|
||||
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.
|
||||
|
||||
Engine and UI work were deliberately paused because documentation had grown large enough to overload Claude and make returning across sessions difficult. The current phase is simplifying what a fresh session must load to understand the project, without losing evidential history. Historical material remains available under `docs/archive/`.
|
||||
|
||||
@@ -143,8 +144,8 @@ Answer before continuing:
|
||||
|
||||
---
|
||||
|
||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54R. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
*Created by Experiment 34. Updated by Experiments 38–53, 54A–54S. Branch: `feature/user-workspace-ux-v0.7`.*
|
||||
|
||||
### Return-to-Work Note (Experiment 54R)
|
||||
### Return-to-Work Note (Experiment 54S)
|
||||
|
||||
54Q showed one structured hypothesis → evidence → consequence call worked across three tested cases but did not isolate which reformulation caused recovery. 54R tested whether a consequential disagreement actually requires the user or can be resolved through evidence — three cases: competing delivery causes (false), growth-versus-risk ambiguity (true), no-material-disagreement control (false). All three correct (3/3). The model distinguished missing evidence from missing user meaning without generating any question, choosing a winner, or changing Behaviour Selection. Same host/model; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`. First test/file to inspect when resuming: `tests/reconstruction/semantic-disagreement-resolution-source.test.js`.
|
||||
54R showed that evidence uncertainty and user-owned ambiguity can be distinguished across three tested patterns. 54S tested whether, once clarification is known to be required, the model can identify exactly what the user needs to clarify — growth-versus-risk priority correctly identified preference/trade-off versus hard constraint; delivery causes correctly returned null (evidence-resolvable); affordable ambiguity correctly identified upfront versus long-term cost meaning. 3/3 clarified-target classification correct. No clarification question was generated. Behaviour Selection remained untouched. Same host/model (qwen-claude:latest on 192.168.1.111:11434); no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`. First test/file to inspect when resuming: `tests/reconstruction/semantic-clarification-target.test.js`.
|
||||
|
||||
Reference in New Issue
Block a user