robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: test domain priors in ambiguous relevance

Browse Source
This commit is contained in:
robbond
2026-08-07 10:28:43 +01:00
parent 9171844f5b
commit 69d0216250
3 changed files with 719 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+3 -3
View File
@@ -122,8 +122,8 @@ Answer before continuing:
---
*Created by Experiment 34. Updated by Experiments 38–52F. Branch: `feature/user-workspace-ux-v0.7`.*
*Created by Experiment 34. Updated by Experiments 38–52G. Branch: `feature/user-workspace-ux-v0.7`.*
### Return-to-Work Note (Experiment 52F)
### Return-to-Work Note (Experiment 52G)
Experiment 52F tested whether ambiguous relationship statements remain ambiguous during normalisation, rather than being forced into stronger categories. Two clear controls (explicit blocker and explicit supporting-evidence) confirmed the existing boundary still works cleanly for unambiguous inputs — both matched expected enums. Cases 3 and 4 (genuinely ambiguous regulatory-position statements) were tested against `cannot_determine`. The model's treatment of these cases — whether it preserved uncertainty or introduced external assumptions about regulation — determines whether the contract can honestly own ambiguity. Same model (`qwen-claude:latest`) and host retained; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`, commit pending. First file to inspect when resuming: `tests/graph/decision-relevance-ambiguity.test.js` for test results, then `docs/design-evolution-log.md` Experiment 52F section.
Experiment 52G tested equally ambiguous statements across four different domains (regulation, weather, employment references, customer feedback). Zero of four preserved `cannot_determine`; all four were forced into `could_change_decision`. No domain diverged from another — all used the same reasoning pattern: "important to [X]" → "go/no-go condition". External assumptions appeared uniformly but as linguistic (treating "important" as decisive) rather than domain-specific. Experiment 52F now looks like part of a broader behaviour: the model fills ambiguous meaning with domain expectations across all domains, not just strong-prior ones. Same model (`qwen-claude:latest`) and host retained; no production behaviour changed. Branch: `feature/user-workspace-ux-v0.7`, commit pending. First file to inspect when resuming: `tests/graph/decision-relevance-domain-priors.test.js` for test results, then `docs/design-evolution-log.md` Experiment 52G section.