robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: validate relationship fallback live

Browse Source
This commit is contained in:
robbond
2026-08-10 13:10:21 +01:00
parent 4c5666dfb3
commit 90e662397f
2 changed files with 138 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+4
View File
@@ -328,3 +328,7 @@ Tested whether v0.11 (`7e4c506` — prevent unsupported comparison decomposition
### Experiment 57G — Semantic Compatibility Live Validation
Tested whether the cost-reduction answer (unclassified "other") now passes proposal compatibility through v0.12 (`69efc5d`). **Classification: PASS.** Both invocations returned HTTP 200 at `update_applied` — no more `proposal_compatibility` rejection. The selected next question ("What changed during that period...") is grounded in the relocation scenario (reasoning pattern: explanation, strategy: evidence_gathering). No prohibited decomposition children appeared. The v0.11 decomposition fix held on this successful update. **Cold-start observation:** second invocation started with 5 nodes instead of expected 8+ — inconsistent initial graph construction worth investigating separately. Branch: `feature/semantic-compatibility-v0.12`. Status: PASS — semantic compatibility blocker removed, decomposition regression absent. Full results in `docs/experiment-57g.md`.
### Experiment 57I — No-Structure Relationship Fallback Live Validation
**Objective:** Validate that commit `4c5666d` (reasoning: suppress explanation question without relationship structure) no longer creates the `Explanation for why...` parent when no meaningful relationship structure exists. **Classification: PASS.** Fixed scenario: "Should I relocate my engineering team from London to Manchester?" with answer "We're looking at this mainly for cost reduction — roughly £2M annual savings on office overhead." Update 1 returned HTTP 200 at stage `update_applied`. Nodes containing "Explanation for why": None. Nodes containing "why these observations differ": None. The reasoning pattern correctly shifted from "explanation" to "decision". The selected next question ("What would clarify team size, seniority levels, and willingness to relocate in this situation?") is grounded in a genuine unresolved relocation issue. No production code changed. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Branch: `feature/relationship-fallback-v0.13`. Status: PASS.