Feature/product platform foundation v0.62 #1

Merged
robbond merged 683 commits from feature/product-platform-foundation-v0.62 into feature/emergent-unknowns-v0.5 2026-09-09 07:58:20 +01:00
2 changed files with 140 additions and 0 deletions
Showing only changes of commit 3a4dda9daf - Show all commits
+4
View File
@@ -2813,3 +2813,7 @@ Experiment 60B.7 diagnosed why selectedQuestion targets a generic parent decisio
---
Experiment 60B.8 compared three minimal mechanisms for preserving a material factor into final question selection (A: honour valid model-selected nodeId as preferred target, B: deterministic material-factor priority via scoring adjustment, C: explicit continuationTarget field). Analysis of actual code confirmed: (1) selectedQuestion.nodeId already means "candidate for next question" with full structural validation in place, (2) ignoring a valid model-selected node is semantically contradictory given the field's purpose, (3) existing validation (validateSelectedQuestion + isSelectableUnresolvedUnknown) covers all validity checks, and (4) existing fallback chain at lines 3418-3437 of apply-proposal.js handles the invalid-target case. Candidate A was chosen: honour valid model-selected nodeId as preferred target before deterministic scoring, with fallback to selectActiveImplementationCandidate when invalid. No new schema fields, no new validators, no new keyword logic. Smallest boundary: one Rule 172 clarification + one modification in buildSelectedQuestionResult. Implementation readiness: A — ready for bounded implementation. Status pending Rob's review.
---
Experiment 60B.12 reran the exact 60B.6 live case to verify prerequisite-aware targeting from 60B.11. **Classification: H — BLOCKED.** The update was rejected at stage `result_validation` with error: "Active unknown violates reasoning pattern consistency: 'n_client_retention_uncertainty' is diagnosis but active pattern is decision." Three differences from 60B.6: (1) node id changed from `n_client_retention` to `n_client_retention_uncertainty`, (2) node kind changed from `unknown` to `diagnosis`, (3) rejection occurred before any proposal applied — neither materiality nor targeting could be assessed. The blocker is a schema-level pattern-consistency mismatch between the model's output and the active decision pattern, not a question-targeting problem. Fix path: either constrain the model prompt to produce kind=unknown nodes or relax the active pattern validator to accept diagnosis nodes for decision-mode continuations.
+136
View File
@@ -0,0 +1,136 @@
# Experiment 60B.12 — Live Verification of Prerequisite-Aware Question Targeting
**Branch:** `feature/question-target-alignment-v0.27`
**Starting HEAD:** `3c6e436`
**Date:** 2026-08-13
**Status:** BLOCKED (apparatus failure)
**Type:** LIVE RUN — Single-call verification of 60B.11 prerequisite-aware targeting
## Objective
Run one bounded Update to answer:
> Does the engine now keep the decision open for the client-retention uncertainty AND make that same unknown the final selected question target?
## Following
Experiment 60B.6 (materiality rule with real unresolved factor)
Experiment 60B.11 (prerequisite-aware preferred targeting implemented in production code)
This is the **live regression** 60B.11 explicitly left unproven:
```text
the exact 60B.6 live continuation case,
where the model selects the newly exposed client-retention factor
and the final selected target preserves that same ready material unknown
```
## Fixed Starting Graph
**Fixture:** `tests/fixtures/pre-anchored-decision-options.json`
| Node | Kind | Status | Label |
|------|------|--------|-------|
| n_relocation_state | state | provisional | Engineering team relocation consideration |
| opt_relocate | option | known | Relocate to Manchester |
| opt_stay_put | option | known | Stay in London (Status Quo) |
| n_relocation_decision | unknown | unknown | Which option leaves us better off overall? |
## Configured Model
- **Model:** qwen-claude:latest
- **Ollama base URL:** http://192.168.1.111:11434 (from .env.local)
## Fixed Answer (verbatim, exact)
> We have now quantified the full financial impact of replacing the two senior engineers and the delivery delay at about £600,000 as a one-off relocation cost. Staying put costs us an extra £2 million every year. The remaining issue is our largest client: we do not yet know whether they would leave if we relocated, and losing them would cost us about £5 million per year.
## Execution
Exactly one update call through the production route via `reproduce-multi-turn-investigation.mjs` in `updateOnly` mode.
## Call Accounting
```
startCalls: 0
updateCalls: 1
totalCalls: 1
Retries: 0
```
## HTTP Response
- **Status:** 500 — rejected during validation
- **Stage:** result_validation
- **Proposal applied:** NO (rejected)
## Rejection Error
```
Active unknown violates reasoning pattern consistency: "n_client_retention_uncertainty" is diagnosis but active pattern is decision
```
The model attempted to create a node with id `n_client_retention_uncertainty` and kind `"diagnosis"`. The active reasoning pattern is `"decision"`, which does not allow the `"diagnosis"` kind for unknown nodes. This is a structural/pattern consistency validation failure — not a materiality or targeting question.
Note: 60B.6 used node id `n_client_retention` with kind `"unknown"`. The model in this run produced `n_client_retention_uncertainty` with kind `"diagnosis"` — different ID and different kind, which triggered the validator rejection before any proposal could be applied.
## Structural Action Required
UNAVAILABLE (rejection occurred before structural data was exposed)
## Assessment
### Materiality behaviour: UNAVAILABLE
Cannot assess — no proposal applied.
### Client-retention uncertainty: UNAVAILABLE
Cannot assess — model produced `n_client_retention_uncertainty` (kind=diagnosis) rather than a compatible unknown node.
### Client-risk ownership: UNAVAILABLE
Cannot assess.
### Preferred-target behaviour: UNAVAILABLE
Cannot assess — rejected before proposal application.
### Question text: NONE
No question returned.
### Prerequisite guard: UNAVAILABLE
Cannot assess — prerequisite checking occurs after proposal validation.
## 60B.6 vs 60B.12 Comparison
| Field | 60B.6 | 60B.12 |
|-------|-------|--------|
| Final nodeId | (created n_client_retention, but generic question) | UNAVAILABLE — rejected |
| Decision status | unresolved (PRESERVED) | UNAVAILABLE |
| Client-retention unknown | YES (n_client_retention) | UNAVAILABLE |
In 60B.6 the model produced `kind=unknown` with id `n_client_retention`. In 60B.12 the model produced `kind=diagnosis` with id `n_client_retention_uncertainty` — a different node name and an incompatible kind for the active decision pattern.
## Classification: H — BLOCKED
Apparatus (reasoning-pattern consistency validator) rejected the model's proposal before inference could be assessed. The blocker is not the 60B.11 targeting fix but a schema-level incompatibility between what the model produced (`kind=diagnosis`) and what the active pattern permits.
## Critical evidence
- No production code changed during this experiment
- No prompt changes to question-targeting logic — this failure is at the pattern-consistency layer
- The node id mismatch (60B.6 used `n_client_retention`; 60B.12 model produced `n_client_retention_uncertainty`) suggests stochastic variation in model output naming
- The kind mismatch (`unknown` vs `diagnosis`) is the actual validation blocker
## What this establishes
1. The live server was reachable and the update-only harness executed correctly.
2. The reasoning-pattern consistency validator catches kind mismatches between model output and active pattern before any proposal mutation.
3. Further live testing requires either (a) matching what 60B.6 did — producing `kind=unknown` with a compatible id — or (b) relaxing the active pattern to accept `diagnosis` nodes.
## Production code changed: NO
## Prompt changed during experiment: NO
## Validator changed: NO
## Schema changed: NO
## Harness changed: NO
## Vitest run: NO
## Ollama calls: 1
## Direct API calls: 0
## Dev server disturbed: NO