experiment: validate decomposition relevance live

2026-08-10 08:04:59 +01:00
parent 7e4c506614
commit 4ea664d0d8
2 changed files with 95 additions and 0 deletions
@@ -320,3 +320,7 @@ Validated one live semantic probe: the configured Qwen model (`qwen-claude:lates
 ### Experiment 57A — Contaminated After First Valid Observation

 The first post-v0.8 product observation exposed one valid production defect: a legitimate affirmative decision-advancing answer (e.g., "We want cost reduction") falls into `other` and is blocked from resolving an unknown by `validateAnswerMeaningAlignment()`. The observation run became contaminated when Claude modified production code (`lib/graph/apply-proposal.js`, `lib/graph/schema.js`) adding four new categories and keyword detectors. Contaminated changes were reverted; valid defect remains open for a bounded implementation review. Repository production state is back at the merged v0.8 baseline (commit `14d68f1`). Next implementation question: how to prevent the fidelity guard from blocking legitimate answers outside its protected cases **without adding a growing answer-category taxonomy**.
+
+### Experiment 57F — Decomposition Relevance Fix Live Validation (BLOCKED)
+
+Tested whether v0.11 (`7e4c506` — prevent unsupported comparison decomposition) works on the live relocation scenario from Experiments 57E/57B. **Classification: BLOCKED.** Start returned HTTP 200 with a valid graph. Update 1 failed at `proposal_compatibility` stage with HTTP 422: "Proposal cannot resolve beyond an unclassified answer by introducing an unsupported constraint or preference/trade-off distinction." The semantic grounding gate (`validateAnswerMeaningAlignment()`) blocked cost-reduction answers classified as class "other" before any graph update could occur. No decomposition ran — prohibited children ("two observations", "measured", "different timing") cannot be confirmed absent. **New defect exposed:** the semantic grounding gate rejects legitimate cost-reduction/savings answers that fall into unclassified ("other") rather than protected categories, preventing any Update 1 evaluation of the v0.11 fix. This is a separate blocking issue from decomposition relevance. What remains unproven: whether v0.11 works when Update 1 does succeed (answers in supported classes), and whether it correctly allows appropriate comparison/measurement decomposition for genuine parents. Branch: `feature/decomposition-relevance-v0.11`. Status blocked by semantic grounding gate.