robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: validate prerequisite-aware question targeting live

Browse Source
This commit is contained in:
robbond
2026-08-13 09:45:13 +01:00
parent 3c6e436e89
commit 3a4dda9daf
2 changed files with 140 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+4
View File
@@ -2813,3 +2813,7 @@ Experiment 60B.7 diagnosed why selectedQuestion targets a generic parent decisio
---
Experiment 60B.8 compared three minimal mechanisms for preserving a material factor into final question selection (A: honour valid model-selected nodeId as preferred target, B: deterministic material-factor priority via scoring adjustment, C: explicit continuationTarget field). Analysis of actual code confirmed: (1) selectedQuestion.nodeId already means "candidate for next question" with full structural validation in place, (2) ignoring a valid model-selected node is semantically contradictory given the field's purpose, (3) existing validation (validateSelectedQuestion + isSelectableUnresolvedUnknown) covers all validity checks, and (4) existing fallback chain at lines 3418-3437 of apply-proposal.js handles the invalid-target case. Candidate A was chosen: honour valid model-selected nodeId as preferred target before deterministic scoring, with fallback to selectActiveImplementationCandidate when invalid. No new schema fields, no new validators, no new keyword logic. Smallest boundary: one Rule 172 clarification + one modification in buildSelectedQuestionResult. Implementation readiness: A — ready for bounded implementation. Status pending Rob's review.
---
Experiment 60B.12 reran the exact 60B.6 live case to verify prerequisite-aware targeting from 60B.11. **Classification: H — BLOCKED.** The update was rejected at stage `result_validation` with error: "Active unknown violates reasoning pattern consistency: 'n_client_retention_uncertainty' is diagnosis but active pattern is decision." Three differences from 60B.6: (1) node id changed from `n_client_retention` to `n_client_retention_uncertainty`, (2) node kind changed from `unknown` to `diagnosis`, (3) rejection occurred before any proposal applied — neither materiality nor targeting could be assessed. The blocker is a schema-level pattern-consistency mismatch between the model's output and the active decision pattern, not a question-targeting problem. Fix path: either constrain the model prompt to produce kind=unknown nodes or relax the active pattern validator to accept diagnosis nodes for decision-mode continuations.