robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: capture minimal clarification answerability

Browse Source
This commit is contained in:
robbond
2026-08-10 14:26:07 +01:00
parent 90e662397f
commit dcefb36f4d
2 changed files with 114 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+4
View File
@@ -332,3 +332,7 @@ Tested whether the cost-reduction answer (unclassified "other") now passes propo
### Experiment 57I — No-Structure Relationship Fallback Live Validation
**Objective:** Validate that commit `4c5666d` (reasoning: suppress explanation question without relationship structure) no longer creates the `Explanation for why...` parent when no meaningful relationship structure exists. **Classification: PASS.** Fixed scenario: "Should I relocate my engineering team from London to Manchester?" with answer "We're looking at this mainly for cost reduction — roughly £2M annual savings on office overhead." Update 1 returned HTTP 200 at stage `update_applied`. Nodes containing "Explanation for why": None. Nodes containing "why these observations differ": None. The reasoning pattern correctly shifted from "explanation" to "decision". The selected next question ("What would clarify team size, seniority levels, and willingness to relocate in this situation?") is grounded in a genuine unresolved relocation issue. No production code changed. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Branch: `feature/relationship-fallback-v0.13`. Status: PASS.
### Experiment 57J.2 — Minimal Clarification Answerability Diagnostics
Tested `{"scenario":"test"}` for exact graph node text and answerability diagnostics. **Classification: B — inconsistent diagnostics.** Single start produced `centralStatement="test"` with one unknown (`nlgonjv`): "The actual scenario, problem description, or data set intended for analysis." The API reported `prerequisiteConceptCount=3`, but only 1 of 7 prerequisite regex signals actually matched (rule 1: `\bproblem\b`). The count of 3 comes from conjunction-based amplification: two instances of "or" in the label+description → `conjunctionCount=2` → formula `Math.max(regex_matches=1, unresolved_deps=0, conjunctions+1=3)` = 3. The diagnostic name is misleading — it reports a maximum across three different amplification strategies (prerequisite regex, unresolved dependencies, conjunctions), not just prerequisite concept signals. No graph-backed question produced. One live Ollama call at qwen-claude:latest on http://192.168.1.111:11434 (27,109 ms). Full results in `docs/experiment-57j2.md`.