experiment: test wording effects on ambiguous relevance

docs/current-handoff.md

@@ -64,6 +64,8 @@ Experiment 51 tested whether decision-relative relevance distinguishes coherent
 
 Experiment 52 tested whether a small semantic interpretation step can judge decision relevance more reliably than keyword matching across paraphrases and domains. The semantic contract was implemented in `tests/graph/decision-relevance-semantic.test.js`. Live model comparison could not be completed because Ollama is not running on this machine — the test infrastructure uses the same `/api/chat` + `format:json` pattern as production. The deterministic keyword baseline continues to fail on paraphrases and new domains (confirmed via 15 passing guardrail tests). No semantic logic entered the active engine. The four-category decision-relevance contract remained unchanged. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic.test.js` for the full experiment and results.
 
+Experiment 52H held domain constant (market-entry / customer demand) and varied ambiguous wording across five cases. Three phrasings ("important to", "relevant to", "may matter for") became `could_change_decision`; one ("worth considering") became `supports_decision`; only "connected to" preserved `cannot_determine`. The evidence suggests the model strengthens vague relevance wording more generally, not just because of "important". Same host and model retained; no production behaviour changed. Status pending Rob's review.
+
 Experiment 52A recovered the semantic test infrastructure by correcting its configuration resolution. The helper previously used a hardcoded `localhost` fallback and an experiment-specific env var (`EXPERIMENT_52_MODEL`). Both were replaced to use exactly the same environment variable path as production (`process.env.OLLAMA_BASE_URL` / `process.env.OLLAMA_MODEL`) sourced from `.env.local`. Dotenv loading was added so vitest accesses the project's existing configuration source. Ollama at 192.168.1.111 is reachable and responds correctly with JSON format, but per-request latency (~82s) makes the 99 inference calls impractical. Configuration path verified correct; execution requires a faster inference host. No production code changed (0 lines in provider, config, analysis, orchestrator). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic.test.js` lines 80–85 (helper).
 
 Experiment 52C separated free-language semantic understanding from enum normalisation into two independent calls per case across five decision/question pairs. Meaning mode captured all five intended relationships correctly (5/5). Enum classification matched expected categories on four of five cases (4/5). One meaning-correct / enum-mismatch case: Case 2 (European regulatory compliance) was correctly described as supporting in both modes but classified as `could_change_decision` rather than `supports_decision`. Same Qwen model (`qwen-claude:latest`) and host were retained; no production behaviour changed. What remains uncertain: whether the meaning-enum gap generalises across decision domains, stability over repeated runs, and whether normalisation mechanisms can bridge the gap without altering interpretation. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic-normalisation.test.js` for results.
@@ -122,8 +124,8 @@ Answer before continuing:
 
 ---
 
-*Created by Experiment 34. Updated by Experiments 38–52G. Branch: `feature/user-workspace-ux-v0.7`.*
+*Created by Experiment 34. Updated by Experiments 38–52H. Branch: `feature/user-workspace-ux-v0.7`.*
 
-### Return-to-Work Note (Experiment 52G)
+### Return-to-Work Note (Experiment 52H)
 
-Experiment 52G tested equally ambiguous statements across four different domains (regulation, weather, employment references, customer feedback). Zero of four preserved `cannot_determine`; all four were forced into `could_change_decision`. No domain diverged from another — all used the same reasoning pattern: "important to [X]" → "go/no-go condition". External assumptions appeared uniformly but as linguistic (treating "important" as decisive) rather than domain-specific. Experiment 52F now looks like part of a broader behaviour: the model fills ambiguous meaning with domain expectations across all domains, not just strong-prior ones. Same model (`qwen-claude:latest`) and host retained; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`, commit pending. First file to inspect when resuming: `tests/graph/decision-relevance-domain-priors.test.js` for test results, then `docs/design-evolution-log.md` Experiment 52G section.
+Experiment 52G's conclusion was narrowed to match its evidence: the cross-domain effect of "important to [decision]" → `could_change_decision` is established, but this was phrasing-specific. Experiment 52H held domain constant (market-entry / customer demand) and varied only ambiguous wording across five cases. Three phrasings ("important to", "relevant to", "may matter for") became `could_change_decision`; one ("worth considering") settled at `supports_decision`; only "connected to" preserved `cannot_determine`. The evidence suggests the model strengthens vague relevance wording more generally — not just because of "important" — with intensity proportional to phrasing specificity. Same host (`http://192.168.1.111:11434`) and model (`qwen-claude:latest`) retained; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`, commit pending. First file to inspect when resuming: `docs/design-evolution-log.md` Experiment 52H section, then `tests/graph/decision-relevance-ambiguous-wording.test.js`.