robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: observe guarded multi-turn progress

Browse Source
This commit is contained in:
robbond
2026-08-11 10:50:05 +01:00
parent bda3abf893
commit 06f67da1b4
2 changed files with 159 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+19
View File
@@ -415,3 +415,22 @@ Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No producti
**What it does NOT establish:** Whether "conditional" is the ideal trigger word across all contexts, whether "Before deciding" should itself have triggered conditional semantics, generalisation to other answers/domains, or whether cold-start node variance (57J.32) separately affects proposal quality.
Configured Ollama: none used. Production code changed: NO. Tests permanently changed: NO. Temporary test used and removed: YES.
---
### Experiment 57J.34 — Multi-Turn Investigation Progress After Accepted Update 1
**Objective:** On one fresh live run, if the first relocation answer passes the current reasoning safeguards, does answering the savings-realism question produce genuine investigation progress rather than repetition or irrelevant reasoning?
**Classification: D — VALIDATION FAILURE.** Two distinct outcomes from cold-start variance:
- **Run A (harness):** Update 1 rejected at `proposal_compatibility` with "answerMeaning introduces a stronger reasoning category" — same conditional-strengthening defect as 57J.33 R1. This is a correct fidelity guard.
- **Run B (pipeline capture, fresh case):** Update 1 applied successfully but produced **zero new unknown nodes** despite the user answer explicitly introducing two independent evidence dimensions (savings realism + retention impact). A single compressed merged unknown appeared ("specific criteria, budget constraints, talent retention implications, or timeline"). Edges decreased from 3→2. When Update 2 was then attempted with concrete savings evidence (£2M), it failed at `proposal_compatibility` by the same structural provenance-link gate: "New unknown must be explicitly related to an answer-derived node."
**Key finding:** Even when Update 1 passes semantic validation, the engine does NOT produce meaningful investigation progress. The savings-realism question from Answer 2 could not resolve because no dedicated savings realism unknown existed. This means the experiment's core question is answered negatively: accepting a "faithful" proposal does not guarantee downstream progress.
**What this establishes:** (1) Cold-start variance (5→8 nodes) directly affects whether Update 1's semantic fidelity guard fires; (2) Semantic acceptance ≠ structural validity — zero-node proposals can pass validation; (3) The provenance-link gate remains active in v0.16 and blocks legitimate new unknown creation; (4) The merged generic unknown instead of two distinct targets is a consistent pattern.
**What this does NOT prove:** That all cold-starts produce 5 nodes; that zero-node proposals are always produced on accepted updates; whether the merged-unknown behavior is intentional or a defect.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 4 live calls total (2 starts + 2 updates in final pipeline; 1 start + 1 update rejected from harness). No production code changed.