robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: record prerequisite-aware question targeting

Browse Source
This commit is contained in:
robbond
2026-08-13 09:36:10 +01:00
parent 54bc48342b
commit 3c6e436e89
3 changed files with 645 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+142 -73
View File
@@ -127,6 +127,19 @@ Experiment 54N tested whether an interpretation disagreement can be judged for m
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states;
- Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates `dependsOn` and `parentId` correctly — but requires comparable observations to trigger);
### 60B.10 / 60B.11 question-target alignment state
- **60B.10 completed the design diagnosis**: the winning rule is narrow prerequisite-aware preservation of a same-proposal model-selected unknown, not blanket trust in model selection.
- **60B.11 is now implemented** in `lib/graph/apply-proposal.js` only.
- The engine now prefers a same-proposal model-selected unresolved unknown **only when** it remains selectable after mutation and has **no unresolved same-proposal-added `depends_on` prerequisite**.
- If that bounded preference check fails, the existing deterministic selector remains authoritative and unchanged.
- The pricing regression remains preserved: model-selected `n_pricing` still falls back to deterministic selection of `n_commercial_value` when `n_pricing depends_on n_commercial_value`.
- Final wording still comes from the deterministic question formulator; model-provided question text is not final authority.
- Focused deterministic verification passed:
- `npx vitest run tests/graph/apply-proposal.test.js -t "60B.11"` → PASS (10/10)
- `npx vitest run tests/graph/apply-proposal.test.js tests/graph/prompt-builder.test.js` → PASS (174/174)
- **Still unproven live:** the exact 60B.6 continuation case where the model selects the newly exposed client-retention factor and the final target preserves that same ready material unknown in a live run.
### When This Knowledge-Management Phase Is Complete
Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):
@@ -191,6 +204,7 @@ When the financial saving is accepted as real but the user introduces retention
Fixture: `tests/fixtures/pre-anchored-update-savings-realism.json`
Existing uncertainty:
```
n_savings_realism — Are the projected office savings from relocation realistic? — status = unknown
```
@@ -239,6 +253,7 @@ n_relocation_net_value — "Does relocating provide net value despite potential
```
The node simultaneously represents both consequences mentioned in the answer:
1. **Key-engineer retention risk** — "potential staff loss" / "losing key engineers"
2. **Delivery slowdown** — "slowing engineering delivery for a year"
@@ -250,14 +265,14 @@ The node simultaneously represents both consequences mentioned in the answer:
## Assessment
| Criterion | Result | Classification |
|-----------|--------|----------------|
| Savings-realism uncertainty | `unknown` → `resolved`, in resolvedUnknownNodeIds | CLOSED CORRECTLY |
| £2m/year saving | `"Confirmed realistic: £2M annual saving is real"` on resolved node | PRESERVED AS ACCEPTED EVIDENCE |
| Key-engineer retention risk | Represented in node label + description ("losing key engineers") | STRUCTURALLY REPRESENTED |
| Delivery slowdown | Represented in node label + description ("slowing engineering delivery for a year") | STRUCTURALLY REPRESENTED |
| Decision shift | New unknown about net value, not savings realism | SHIFTED TO WORTH-IT / CONSEQUENCE REASONING |
| Next question quality | Asks the right meta-question: does benefit outweigh consequences? | GOOD |
| Criterion | Result | Classification |
| --------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------- |
| Savings-realism uncertainty | `unknown` → `resolved`, in resolvedUnknownNodeIds | CLOSED CORRECTLY |
| £2m/year saving | `"Confirmed realistic: £2M annual saving is real"` on resolved node | PRESERVED AS ACCEPTED EVIDENCE |
| Key-engineer retention risk | Represented in node label + description ("losing key engineers") | STRUCTURALLY REPRESENTED |
| Delivery slowdown | Represented in node label + description ("slowing engineering delivery for a year") | STRUCTURALLY REPRESENTED |
| Decision shift | New unknown about net value, not savings realism | SHIFTED TO WORTH-IT / CONSEQUENCE REASONING |
| Next question quality | Asks the right meta-question: does benefit outweigh consequences? | GOOD |
### Why:
@@ -309,11 +324,17 @@ The engine correctly closed savings-realism, preserved verified evidence, repres
---
## Production code changed: NO
## Prompt changed: NO
## Validator changed: NO
## Harness changed: NO
## Vitest run: NO
## Ollama calls beyond harness count: 0
## Dev server disturbed: NO
### Experiment 59A.2 — Trade-off Decomposition: Known Consequence vs Uncertain Consequence
@@ -321,7 +342,7 @@ The engine correctly closed savings-realism, preserved verified evidence, repres
**Branch:** `feature/question-formulation-v0.24`
**Date:** 2026-08-12
**Status:** Complete
**Following:** 59A.1 which established the boundary shift but bundled two consequences into one node. This tests whether the engine can separate a *known* consequence (engineers leaving) from an *uncertain* consequence (delivery impact).
**Following:** 59A.1 which established the boundary shift but bundled two consequences into one node. This tests whether the engine can separate a _known_ consequence (engineers leaving) from an _uncertain_ consequence (delivery impact).
**Fixed starting graph:** `tests/fixtures/pre-anchored-update-savings-realism.json`
**Fixed answer:** "I am comfortable that the £2 million annual saving is real. We know we would lose two senior engineers if we relocate. What I don't know is whether losing them would materially delay delivery, or by how much."
@@ -331,6 +352,7 @@ The engine correctly closed savings-realism, preserved verified evidence, repres
**Result:** UPDATE 422 rejected at `proposal_compatibility` — "selectedQuestion must be a single non-compound question". No selectedQuestion was produced (null).
**Reasoning assessment from rejected snapshot:**
- Savings-realism: CLOSED CORRECTLY (resolved, in resolvedUnknownNodeIds)
- £2m annual saving: PRESERVED AS ACCEPTED EVIDENCE (in userSupportedMeaning)
- Two senior engineers leaving: REPRESENTED BUT LEFT UNCERTAIN (extracted in meaning but no separate structural node for the known fact)
@@ -391,12 +413,12 @@ One update-only call via `scripts/reproduce-multi-turn-investigation.mjs`. (Firs
### CALL ACCOUNTING (successful run)
| Field | Value |
|-------|-------|
| startCalls | 0 |
| updateCalls | 1 |
| totalCalls | 1 |
| Retries | 1 (initial rejection at proposal_validation) |
| Field | Value |
| ----------- | -------------------------------------------- |
| startCalls | 0 |
| updateCalls | 1 |
| totalCalls | 1 |
| Retries | 1 (initial rejection at proposal_validation) |
## Results — Successful Run
@@ -443,46 +465,47 @@ One update-only call via `scripts/reproduce-multi-turn-investigation.mjs`. (Firs
### Resulting persistent graph (5 nodes, 4 edges)
| Node | Kind | Status | Label |
|------|------|--------|-------|
| n_relocation_state | state | provisional | Engineering team relocation consideration |
| n_savings_realism | unknown | resolved | Are the projected office savings from relocation realistic? |
| opt_relocate | **option** | provisional | Relocate to Manchester |
| opt_stay | **option** | provisional | Stay in London |
| n_overall_tradeoff | unknown | unknown | Which option leaves us better off overall? |
| Node | Kind | Status | Label |
| ------------------ | ---------- | ----------- | ----------------------------------------------------------- |
| n_relocation_state | state | provisional | Engineering team relocation consideration |
| n_savings_realism | unknown | resolved | Are the projected office savings from relocation realistic? |
| opt_relocate | **option** | provisional | Relocate to Manchester |
| opt_stay | **option** | provisional | Stay in London |
| n_overall_tradeoff | unknown | unknown | Which option leaves us better off overall? |
| Edge | From | To | Relationship |
|------|------|----|-------------|
| e-sr-to-state | n_savings_realism | n_relocation_state | depends_on |
| opt-rel-to-trad | opt_relocate | n_overall_tradeoff | **contained_in** |
| opt-stay-to-trad | opt_stay | n_overall_tradeoff | **contained_in** |
| trad-to-savings | n_overall_tradeoff | n_savings_realism | depends_on |
| Edge | From | To | Relationship |
| ---------------- | ------------------ | ------------------ | ---------------- |
| e-sr-to-state | n_savings_realism | n_relocation_state | depends_on |
| opt-rel-to-trad | opt_relocate | n_overall_tradeoff | **contained_in** |
| opt-stay-to-trad | opt_stay | n_overall_tradeoff | **contained_in** |
| trad-to-savings | n_overall_tradeoff | n_savings_realism | depends_on |
## Assessment
### 1. Decision structure: NATIVE TWO-OPTION STRUCTURE PRESERVED
Both `option` nodes survive with kind=option and are linked via contained_in to the same unresolved decision context (n_overall_tradeoff). This matches the 60A.4 pattern. Two minor differences from 60A.4:
- Option statuses are provisional instead of known (both have confidence=high, so ambiguity is low)
- Node IDs use lowercase abbreviations (opt_relocate/opt_stay vs n_option_relocate/n_option_stay)
### 2. Relocate consequences — ALL INDEPENDENTLY STRUCTURED (in descriptions)
| Consequence | Present? | Classification |
|-------------|----------|----------------|
| £2m/year saving | YES | "saves £2m/year" — exact figure and unit preserved |
| two senior engineers leave | YES | "loses two senior engineers" — exact headcount preserved |
| <= two months delivery delay | YES | "delays delivery by up to two months" — bounded phrasing preserved |
| Consequence | Present? | Classification |
| ---------------------------- | -------- | ------------------------------------------------------------------ |
| £2m/year saving | YES | "saves £2m/year" — exact figure and unit preserved |
| two senior engineers leave | YES | "loses two senior engineers" — exact headcount preserved |
| <= two months delivery delay | YES | "delays delivery by up to two months" — bounded phrasing preserved |
All three consequences present within opt_relocate's description. The description also serves as a structured field on an option node reachable via contained_in edge.
### 3. Stay-put consequences — ALL INDEPENDENTLY STRUCTURED (in descriptions)
| Consequence | Present? | Classification |
|-------------|----------|----------------|
| retain both engineers | YES | "retain both senior engineers" |
| avoid relocation delay | YES | "avoid relocation delay" |
| continue paying extra £2m/year | YES | "continue paying the extra £2m/year" |
| Consequence | Present? | Classification |
| ------------------------------ | -------- | ------------------------------------ |
| retain both engineers | YES | "retain both senior engineers" |
| avoid relocation delay | YES | "avoid relocation delay" |
| continue paying extra £2m/year | YES | "continue paying the extra £2m/year" |
All three consequences present within opt_stay's description.
@@ -496,26 +519,27 @@ All three consequences present within opt_stay's description.
### 5. Option attribution — CLEAR FOR BOTH OPTIONS
Graph makes it possible to tell which option each consequence belongs to:
- opt_relocate consequences are embedded in opt_relocate.description, reachable via contained_in edge from opt_relocate to n_overall_tradeoff
- opt_stay consequences are embedded in opt_stay.description, reachable via contained_in edge from opt_stay to n_overall_tradeoff
- No cross-contamination or ambiguity
### 6. Relationship direction — SEMANTICALLY CLEAR
| From | To | Relationship | Assessment |
|------|----|-------------|------------|
| opt_relocate | n_overall_tradeoff | contained_in | Clear — relocation is a candidate for the decision |
| opt_stay | n_overall_tradeoff | contained_in | Clear — staying put is a candidate for the decision |
| n_overall_tradeoff | n_savings_realism | depends_on | Workable but slightly odd — the tradeoff assessment depending on the savings figure makes semantic sense (you need the savings value to assess whether relocation is better), though the direction could be read as the unknown "needs" the resolved node, which is epistemically inverted. |
| From | To | Relationship | Assessment |
| ------------------ | ------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| opt_relocate | n_overall_tradeoff | contained_in | Clear — relocation is a candidate for the decision |
| opt_stay | n_overall_tradeoff | contained_in | Clear — staying put is a candidate for the decision |
| n_overall_tradeoff | n_savings_realism | depends_on | Workable but slightly odd — the tradeoff assessment depending on the savings figure makes semantic sense (you need the savings value to assess whether relocation is better), though the direction could be read as the unknown "needs" the resolved node, which is epistemically inverted. |
### 7. Graph-only recoverability
| Question | Answer |
|----------|--------|
| Recover Relocate option | YES — node kind=option, label="Relocate to Manchester" |
| Recover Stay-put option | YES — node kind=option, label="Stay in London" |
| Recover Relocate consequences | PARTIAL — present in opt_relocate.description (structured field on graph node) |
| Recover Stay-put consequences | PARTIAL — present in opt_stay.description (structured field on graph node) |
| Question | Answer |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| Recover Relocate option | YES — node kind=option, label="Relocate to Manchester" |
| Recover Stay-put option | YES — node kind=option, label="Stay in London" |
| Recover Relocate consequences | PARTIAL — present in opt_relocate.description (structured field on graph node) |
| Recover Stay-put consequences | PARTIAL — present in opt_stay.description (structured field on graph node) |
| Tell which consequence belongs to which option | YES — each description is attached to a distinct option node reached by its own contained_in edge |
### 8. Selected question — GOOD
@@ -592,12 +616,12 @@ Host/model: qwen-claude:latest at http://192.168.1.111:11434. startCalls=0, upda
### Structure (5 nodes, 3 edges)
| Node | Kind | Status |
|------|------|--------|
| n_savings_realism | unknown | resolved |
| opt_relocate | option | **known** |
| opt_stay_put | option | **known** |
| n_relocation_decision | unknown | unknown |
| Node | Kind | Status |
| --------------------- | ------- | --------- |
| n_savings_realism | unknown | resolved |
| opt_relocate | option | **known** |
| opt_stay_put | option | **known** |
| n_relocation_decision | unknown | unknown |
Edges: both options → n_relocation_decision via `contained_in`; n_savings_realism → n_relocation_state via `depends_on`.
@@ -618,12 +642,19 @@ First call succeeds; two-option structure confirmed on first attempt. Option sta
**What it does NOT prove:** Stability across repeated runs; whether consequences can be extracted as independent nodes without description parsing; cross-domain generalisation.
## Production code changed: NO
## Prompt changed: NO
## Validator changed: NO
## Harness changed: NO
## Vitest run: NO
## Ollama calls: 1
## Direct API calls: 0
## Dev server disturbed: NO
---
@@ -664,6 +695,7 @@ selectedQuestion.nodeId: "n_disruption_impact"
```
Resulting persistent graph (3 nodes, 2 edges):
- `n_relocation_state` — Engineering team relocation consideration — status=provisional
- `n_savings_realism` — Are the projected office savings from relocation realistic? — status=resolved
- `n_disruption_impact` — Operational impact of relocation disruption — status=unknown, kind=unknown
@@ -700,6 +732,7 @@ Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No producti
**Result:** UPDATE 422 rejected at `proposal_compatibility` — "selectedQuestion must be a single non-compound question". No selectedQuestion was produced (null).
**Rejected proposal snapshot assessment:**
- Known engineer loss: SEPARATE KNOWN STRUCTURE (dedicated observation node n_talent_departure, kind=observation)
- Delivery impact: SEPARATE UNRESOLVED STRUCTURE (dedicated unknown node n_delivery_delay_impact, kind=unknown, status=unknown)
- Epistemic separation: CLEARLY SEPARATED — two distinct nodes with different kinds and statuses
@@ -896,6 +929,7 @@ Two-turn run (fixed scenario: "We are considering relocating the engineering tea
**Classification: A — FIX VALIDATED.** Diagnosed that 57J.61 failed because the harness accepted-update path (script lines ~97–108) printed only `HTTP status`, `stage`, `selected question`, `node count`, and `edge count` — zero answer-meaning fields, zero structural mutation fields. After Update 1 applied successfully with HTTP 200 at `update_applied`, the harness could not identify which nodes were added or what the resulting persistent graph looked like.
**Fix:** Extended the accepted-update console block in `scripts/reproduce-multi-turn-investigation.mjs` to print:
- `answerMeaning.userSupportedMeaning`, `.possibleInference`, `.supportCategory`, `.resolutionGuidance`
- `updatedProposal.updatedNodes[]`, `.resolvedUnknownNodeIds[]`, `.addedNodes[]`, `.addedEdges[]`
- `selectedQuestion.nodeId` (node reference)
@@ -1901,6 +1935,7 @@ Implemented the bounded Option C from 57J.50 on branch `feature/structured-seman
One fresh live run through the production `startCase()` → `updateCase()` path. Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic."
**Key evidence:**
- `supportCategory = "uncertain"` (populated, valid enum) — STRUCTURED path active, not LEGACY FALLBACK.
- `resolutionGuidance = "may_resolve"` (populated).
- Meaning: FAITHFUL — model preserved uncertainty without strengthening or degrading.
@@ -1916,7 +1951,8 @@ One fresh live run through the production `startCase()` → `updateCase()` path.
**Objective:** Implement the correction from Experiment 57J.54: clarify that "same uncertainty" means the same resolution question, not topical overlap. Bounded prompt-only task — no schema, validator, graph-model, or provider changes.
**Changes:**
- `lib/graph/prompt-builder.js`: Added to Additional Guidance — *"Same uncertainty means the same resolution question: resolving the existing unknown would also resolve the uncertainty introduced by the user's answer. Mere topical overlap (concerning the same topic, object, decision, or domain) is not automatically the same uncertainty."*
- `lib/graph/prompt-builder.js`: Added to Additional Guidance — _"Same uncertainty means the same resolution question: resolving the existing unknown would also resolve the uncertainty introduced by the user's answer. Mere topical overlap (concerning the same topic, object, decision, or domain) is not automatically the same uncertainty."_
- `tests/graph/prompt-builder.test.js`: 10 new focused prompt tests verifying the definition, boundaries, and preserved contracts (49 total pass).
**Test results:** All 49 tests pass. No regression in existing prompt structure tests.
@@ -1956,11 +1992,13 @@ Tested the inverse half of the uncertainty identity rule from 57J.56: when a gra
**Objective:** Record durable handoff facts from `docs/experiment-57j58.md` which diagnosed why a proposal adding a new unresolved unknown was rejected due to missing `selectedQuestion`, and classified the root cause.
**validateQuestionSelectionRequirement() rejects when:**
- `selectedQuestion == null`
- AND
- proposal adds at least one non-resolved unknown
**requirement does NOT depend on:**
- `updatedNodes`
- actual `resolvedUnknownNodeIds`
- total existing unresolved-node count
@@ -1987,6 +2025,7 @@ Configured Ollama: none used. **Production code changed:** NO.
**Test results:** 59 tests pass (42 existing + 17 new = 59 total). Zero failures.
**Ownership split preserved:**
- MODEL: supply one valid candidate when new unresolved unknowns are added
- ENGINE: validate candidate, retain deterministic priority/scoring ownership
@@ -1994,13 +2033,14 @@ Configured Ollama: none used. **Production code changed:** prompt + tests only.
### Experiment 57J.65 — Smallest Enforceable Semantic-to-Mutation Contract (Read-Only Deterministic)
**Objective:** What is the smallest structured contract that lets the model declare whether graph action is required, and lets deterministic code verify that the actual proposal fulfils that declaration? **Classification: B — BOOLEAN STRUCTURAL-ACTION CONTRACT.**
**Objective:** What is the smallest structured contract that lets the model declare whether graph action is required, and lets deterministic code verify that the actual proposal fulfils that declaration? **Classification: B — BOOLEAN STRUCTURAL-ACTION CONTRACT.**
**Part 1 — Existing fields:** C — NEW ACTION DECLARATION REQUIRED. The existing fields encode *what changed* but not *what was intended*. When all mutation arrays are empty, there is no field saying "I intentionally declare zero graph action." The validator's current check (populated userSupportedMeaning + empty mutations → REJECT) treats model silence as error rather than accepting intentional no-op.
**Part 1 — Existing fields:** C — NEW ACTION DECLARATION REQUIRED. The existing fields encode _what changed_ but not _what was intended_. When all mutation arrays are empty, there is no field saying "I intentionally declare zero graph action." The validator's current check (populated userSupportedMeaning + empty mutations → REJECT) treats model silence as error rather than accepting intentional no-op.
**Part 2 — Minimum distinction:** `MUTATION REQUIRED` vs `NO MUTATION REQUIRED`. Reuse/refine existing, add new unknown, resolve existing, other structural mutation are all DERIVABLE FROM PROPOSAL SHAPE (validator checks each independently). Only "no structural change" MUST BE DECLARED because empty arrays alone cannot distinguish intent from omission.
**Part 3 — Three designs evaluated:**
- **Option A (boolean):** Prevents no-op = PARTIAL, checks mutation = YES, schema concept = BOOLEAN, validator complexity = LOW, model-compliance risk = MEDIUM
- **Option B (enum):** Prevents no-op = PARTIAL, checks mutation = YES, schema concept = SMALL ENUM, validator complexity = MEDIUM, model-compliance risk = MEDIUM-HIGH
- **Option C (existing fields only):** Prevents no-op = PARTIAL, but accepts every noncompliant zero-mutation output silently — HIGH model-compliance risk
@@ -2010,12 +2050,14 @@ Configured Ollama: none used. **Production code changed:** prompt + tests only.
**Part 5 — Relationship to supportCategory:** INDEPENDENT. `supportCategory = "uncertain"` does NOT mean `add new unknown`. Same category can map to different actions depending on graph state (v0.21 identity rule: reuse existing equivalent).
**Part 6 — Deterministic invariants:**
1. `structuralActionRequired = true` + empty mutations → REJECT
2. `structuralActionRequired = false` + meaningful mutation → ACCEPT with diagnostic note
3. Missing field + populated userSupportedMeaning → REJECT
4. `false` + empty mutations → PASS (valid "semantic agreement, no structural change")
**Part 7 — 57J.63 walkthrough:**
- Case A (successful): declares `true`, addedNodes non-empty → PASS by shape comparison
- Case B (no-op): declares `false`, empty arrays → PASS by explicit declaration + confirmed zero mutation
@@ -2035,9 +2077,10 @@ This is ACTUAL CONTRACT ENFORCEMENT because the validator compares a declared bo
**Part 1 — Field placement: TOP-LEVEL (graphUpdateSchema), NOT inside answerMeaning.**
`structuralActionRequired` expresses *graph-mutation intent*, not semantic meaning. It validates against structural arrays (addedNodes, updatedNodes, addedEdges), not meaning fields. Placing it at the proposal level keeps semantic analysis separate from structural action decisions and aligns with where the validator evaluates it. `selectedQuestion` already sits at this level as another structural decision — `structuralActionRequired` is a peer.
`structuralActionRequired` expresses _graph-mutation intent_, not semantic meaning. It validates against structural arrays (addedNodes, updatedNodes, addedEdges), not meaning fields. Placing it at the proposal level keeps semantic analysis separate from structural action decisions and aligns with where the validator evaluates it. `selectedQuestion` already sits at this level as another structural decision — `structuralActionRequired` is a peer.
**Part 2 — Meaning/action independence:**
- Meaning populated + true → VALID (extracting meaning AND claiming action needed)
- Meaning populated + false → VALID (extracting meaning AND deciding no action needed)
- Meaning null + true → VALID ONLY UNDER SPECIFIC EXISTING CASE (maintenance/cleanup scenarios where structural action is needed without fresh meaning extraction)
@@ -2046,6 +2089,7 @@ This is ACTUAL CONTRACT ENFORCEMENT because the validator compares a declared bo
**Part 3 — Meaningful mutation: REUSE existing definition.** `hasMeaningfulChange` (5 conditions: new nodes, status changes, value changes, added edges, removed edges) directly maps to the boolean. No separate definition needed or desirable.
**Part 4 — Contradiction matrix:**
- true + mutation → PASS (contract fulfilled)
- true + no mutation → REJECT (declaration violated)
- false + no mutation → PASS (intentional no-op)
@@ -2062,6 +2106,7 @@ The contract semantics are **ADVISORY** — the boolean is a minimum intent decl
**Part 8 — Prompt obligation: SUFFICIENT.** Two minimum rules suffice: "set true when answer requires graph progress" and "set false only when existing state fully represents user-supported meaning or no progress is justified." No additional principle needed.
**Implementation decision: D — top-level field + advisory false/mutation handling.**
- New field: `structuralActionRequired` in `graphUpdateSchema`, nullable during transition
- Nullable during transition: YES (mandatory once prompt adds it and when userSupportedMeaning is populated)
@@ -2073,7 +2118,7 @@ Full record in `docs/experiment-57j66.md`.
### Experiment 57J.67 — semanticActionRequired Contract Semantics Finalized
**Classification: A — SEMANTICS SETTLED.** Resolved the final ambiguity from 57J.66: is the boolean an *exact structural claim* (strict contract) or a *minimum-action claim* (advisory)? **Decision: EXACT STRUCTURAL CLAIM.** The field name "structuralActionRequired" semantically implies necessity, not suggestion. Definition A provides cleaner semantics, fully deterministic validation in all four cases, and prevents the most damaging error class (model declares no action but produces structure). Advisory (57J.66's recommendation) is rejected: `false + mutation` violates contract consistency — if the model declares "no structural action required" but produces meaningful mutations, it has either misunderstood the answer or over-produced unnecessary structure. This is not harmless.
**Classification: A — SEMANTICS SETTLED.** Resolved the final ambiguity from 57J.66: is the boolean an _exact structural claim_ (strict contract) or a _minimum-action claim_ (advisory)? **Decision: EXACT STRUCTURAL CLAIM.** The field name "structuralActionRequired" semantically implies necessity, not suggestion. Definition A provides cleaner semantics, fully deterministic validation in all four cases, and prevents the most damaging error class (model declares no action but produces structure). Advisory (57J.66's recommendation) is rejected: `false + mutation` violates contract consistency — if the model declares "no structural action required" but produces meaningful mutations, it has either misunderstood the answer or over-produced unnecessary structure. This is not harmless.
- **Boolean definition:** EXACT STRUCTURAL CLAIM (Definition A). true = meaningful mutation present; false = no meaningful mutation needed.
- **true + mutation:** PASS. true + no mutation: REJECT. false + no mutation: PASS. false + mutation: REJECT (under exact claim).
@@ -2095,22 +2140,26 @@ Full record in `docs/experiment-57j66.md`.
**Classification: IMPLEMENTATION COMPLETE.** All design decisions from 57J.67 implemented verbatim across three production files and validated by 50 deterministic tests (24 new + 8 migrated). No live Ollama calls.
#### Production changes:
1. `lib/graph/schema.js` — Added `structuralActionRequired: z.boolean().nullable().optional()` to graphUpdateSchema.
2. `lib/graph/utils.js` — Replaced semantic-only-no-op guard with four-case contract validator (true+mutation PASS, true+zero REJECT, false+zero PASS, false+mutation REJECT).
3. `lib/graph/prompt-builder.js` — Added field name to required lists; inserted contract declaration section between numbered rules and Additional Guidance.
#### Contract guarantees:
- Populated userSupportedMeaning requires boolean structuralActionRequired.
- Declaration is exact claim about output shape: true = meaningful mutation present; false = zero mutations intentional.
- false + zero is valid intentional no-op (contract-consistent).
- Mismatch between declaration and output shape deterministically rejected.
#### Test suite:
- 24 new tests across schema.test.js (+4), prompt-builder.test.js (+10), utils.test.js (+10).
- 8 existing tests migrated to use structuralActionRequired=true where answerMeaning is populated.
- All 197 graph tests pass (previous total preserved).
#### Live regression readiness:
- All existing schema, prompt-builder, and utils tests pass.
- The change is backward-compatible: field is optional by default; old proposals without it behave identically to the legacy path.
@@ -2175,6 +2224,7 @@ The 57J.74 apparatus commit ("docs: record pre-anchored update apparatus") expli
The 57J.75 experiment document ("experiment: validate controlled structural no-op live") records a live call that "sends the fixture's graph directly as the Update request's situationGraph" — but this capability does not exist in the committed mjs script. The live test depended on temporary uncommitted harness modifications (or direct manual production API invocation) at the time of execution. These were reverted before the apparatus was committed (which only added the fixture file and test-only helper).
**Apparatus classification:**
- `tests/fixtures/pre-anchored-update-savings-realism.json`: FIXTURE ONLY — valid deterministic graph, committed
- `PRE_ANCHORED_FIXTURE` constant + `runPreAnchoredSimulation()`: TEST-ONLY HELPER — synchronous mock simulator in test file, never makes real HTTP calls
- `scripts/reproduce-multi-turn-investigation.mjs`: PRODUCTION HARNESS PATH — but only standard start→update chain, no pre-anchored mode
@@ -2209,12 +2259,13 @@ The 57J.75 experiment document ("experiment: validate controlled structural no-o
- Pre-anchored rejected answerMeaning preservation
- Blocked mode verification (zero fixture load errors)
- Normal-mode isolation proof (accepted/rejected capture unchanged)
Updated `runPreAnchoredSimulation` mock to persist `rejectedProposalSnapshot` on rejection return values. Added `runPreAnchoredSimulationWithBlock()` helper.
**Evidence:** 46 harness tests pass (39 pre-existing + 7 new). No production code changed. No Ollama calls. No live API calls. Normal-mode Start→Update chain unmodified under guard.
**Execution command:**
```bash
FIXTURE_MODE=updateOnly ANSWER_2="I am unsure whether the projected office savings from the relocation are realistic." node scripts/reproduce-multi-turn-investigation.mjs
```
@@ -2262,6 +2313,7 @@ This satisfies 57J.77's boundary A recommendation: a committed update-only path
**Classification: D — WRONG TARGET.** One update-only call via the committed harness. The engine did **not** select `n_savings_realism` as the next-question target. Instead it created a new unknown node `n_lease_exit_penalty` (about lease exit penalty) and selected that as the target.
### Branch: `feature/question-formulation-v0.24`
### Starting HEAD: `870d6ca` (docs: record question-formulation fix)
### Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434
@@ -2356,6 +2408,7 @@ The hypothesis asked whether selecting `n_savings_realism` would now produce a c
**Result:** UPDATE accepted (HTTP 200 at update_applied). No validation errors.
**Assessment of four factual elements:**
- £2m annual saving: PRESERVED AS KNOWN BENEFIT (resolved on `n_savings_realism` with "confirmed £2M annual saving")
- Two-engineer departure: PRESERVED AS KNOWN CONSEQUENCE (dedicated observation node `n_talent_impact_clarity`, kind=observation, status=supported)
- Two-month worst-case bound: PRESERVED AS BOUNDED DOWNSIDE (embedded in observation description: "worst-case 2-month project delay")
@@ -2418,7 +2471,6 @@ Ollama calls: 0
Dev server disturbed: NO
Read-only diagnosis: YES
### Experiment 60A.2 — Choosing the Minimum Decision Representation
**Branch:** `feature/question-formulation-v0.24`
@@ -2439,6 +2491,7 @@ Option B (Stay put): retain engineers, avoid disruption, continue paying £2M/ye
```
Three models evaluated:
- **A** — DECISION + OPTION (new decision node kind + new option node kind)
- **B** — UNKNOWN + OPTION (reuse existing unknown as decision context + new option node kind)
- **C** — OPTION PAIR ONLY (option nodes linked by alternative_to, no decision context node)
@@ -2450,18 +2503,21 @@ Not implemented. No code changed. Read-only design evaluation.
## Results
### Candidate A (DECISION + OPTION)
- Semantic honesty: HIGH | Recoverability: FULL | Lifecycle: NATIVE | Question: CLEAN | Consequences: YES | Baseline: CLEAN
- New primitives: 2 node kinds + 1 edge type + 1 optional field = **4**
- Semantic overload: NONE
- Verdict: Satisfies all criteria but adds the most primitives
### Candidate B (UNKNOWN + OPTION) ✅ WINNER
- Semantic honesty: MEDIUM | Recoverability: FULL | Lifecycle: NATIVE | Question: CLEAN | Consequences: YES | Baseline: WORKABLE
- New primitives: 1 node kind + 1 edge type + 1 optional field = **3**
- Semantic overload: LOW (unknown carries both "uncertainty" and "decision context" — natural overlap, not contradictory)
- Verdict: Smallest model satisfying all five decision-rule conditions
### Candidate C (OPTION PAIR ONLY)
- Semantic honesty: LOW | Recoverability: POOR | Lifecycle: AWKWARD | Question: WORKABLE | Consequences: YES | Baseline: WORKABLE
- New primitives: 1 node kind + 1 edge type = **2**
- Semantic overload: LOW-MEDIUM
@@ -2472,21 +2528,25 @@ Not implemented. No code changed. Read-only design evaluation.
## Architectural Choice: B — UNKNOWN + OPTION
### What changes (exact boundary):
```javascript
// schema.js additions:
option: "option" // SituationKind enum value
contained_in: "contained_in" // SituationRelationship enum value
is_baseline: z.boolean().optional() // optional on option nodes (not required for v1)
option: "option"; // SituationKind enum value
contained_in: "contained_in"; // SituationRelationship enum value
is_baseline: z.boolean().optional(); // optional on option nodes (not required for v1)
```
### What does NOT change:
- `unknown` node kind retains its existing semantics; it now also serves as decision context via the new `option` children pattern
- All existing statuses, edge types, graph topology rules unchanged
- Question compatibility uses existing `selectedQuestion` mechanism without extension
- No migration of existing nodes required
### Decision lifecycle: NATIVE — open/resolved maps to unknown status transitions
### Additional questions answered:
1. Is `alternative_to` needed between options? **NO** — shared parent membership implies alternatives.
2. Is `is_baseline` flag required? **NOT NEEDED YET** — label/consequence patterns carry sufficient signal.
@@ -2543,6 +2603,7 @@ selectedQuestion: "What evidence would clarify which option leaves us better off
7. **Savings-realism:** REMAINS OPEN (correct — answer did not address it).
### What the engine understood correctly:
1. Dual-option decomposition into two `option` nodes
2. Shared decision context via single unresolved unknown
3. Correct use of `contained_in` for option → decision membership
@@ -2550,6 +2611,7 @@ selectedQuestion: "What evidence would clarify which option leaves us better off
5. Decision-question alignment with user's stated intent
### What this establishes:
The live model CAN create native two-option graph structure when the user explicitly presents two alternatives. Both options survive as first-class `option` nodes with structural membership to a shared decision context, and downstream graph-only reasoning can recover both branches and their distinct consequences.
### Classification: A — NATIVE TWO-OPTION STRUCTURE CONFIRMED
@@ -2573,13 +2635,15 @@ Dev server disturbed: NO
**Objective:** Add test-only support for loading the reusable decision-options fixture from its committed JSON file, enabling harness tests to verify pre-anchored update-only mode with non-default fixtures without inline data duplication.
**Methodology:**
**Methodology:**
- Load `tests/fixtures/pre-anchored-decision-options.json` directly via `fs.readFileSync` in the test harness
- Add `runPreAnchoredSimulationWithFixture()` helper that mirrors the production pre-anchored path (generic anchor validation, no Start call, exactly one Update, all hardened capture)
- Generalize script's anchor validation from savings-realism-specific to generic unresolved unknown check
- Run focused vitest harness test only
**Key findings:**
- All 63 harness tests pass (including 17 new decision-options fixture mode tests)
- Fixture loads correctly from committed JSON — no inline duplication needed
- Pre-anchored validation works generically across fixture types (savings-realism and decision-options)
@@ -2599,7 +2663,6 @@ Ollama calls: 0
Live API calls: 0
Vitest run: 1 focused command (63/63 pass)
### Experiment 60A.8 — Downstream Option Evidence Update on Committed Fixture
**Branch:** `feature/decision-options-v0.25`
@@ -2612,12 +2675,14 @@ Vitest run: 1 focused command (63/63 pass)
When new information applies specifically to the Relocate option ("£400,000 lost margin from two-month delivery delay"), does the engine attach that information to the existing Relocate branch while preserving the existing Stay-put option and shared decision context?
## Fixed Starting Graph
Fixture: `tests/fixtures/pre-anchored-decision-options.json`
Pre-existing: opt_relocate (option/known), opt_stay_put (option/known), n_relocation_decision (unknown/unknown)
## Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434
## Execution
startCalls=0, updateCalls=1, totalCalls=1. One live call at qwen-claude:latest. No retries.
## Result
@@ -2626,12 +2691,12 @@ startCalls=0, updateCalls=1, totalCalls=1. One live call at qwen-claude:latest.
### Structure (5 nodes, 3 edges)
| Node | Kind | Status |
|------|------|--------|
| opt_relocate | option | **known** (PRESERVED) |
| opt_stay_put | option | **known** (PRESERVED) |
| n_relocation_decision | unknown | **unknown** (PRESERVED, unresolved) |
| n_delay_cost | **metric** | known (NEW — first-class numeric node, value=400000, unit=GBP) |
| Node | Kind | Status |
| --------------------- | ---------- | -------------------------------------------------------------- |
| opt_relocate | option | **known** (PRESERVED) |
| opt_stay_put | option | **known** (PRESERVED) |
| n_relocation_decision | unknown | **unknown** (PRESERVED, unresolved) |
| n_delay_cost | **metric** | known (NEW — first-class numeric node, value=400000, unit=GBP) |
New edge: n_delay_cost → opt_relocate via `measures` relationship.
Both original contained_in edges preserved intact.
@@ -2645,6 +2710,7 @@ All three existing entity identities preserved without duplication; £400k conse
**What it does NOT prove:** Stability across repeated runs; multiple concurrent consequences per option; cross-domain generalisation.
## Production code changed: NO
Prompt changed: NO
Validator changed: NO
Harness changed: NO
@@ -2692,6 +2758,7 @@ Single-call live regression of experiment 60B.4's prompt-only decision materiali
The engine resolved the existing decision context independently. All three entity identities preserved. The generic continuation failure from 60B.2 is eliminated.
**Key changes vs 60B.2:**
- `n_relocation_decision` status: unknown → **resolved** (was supported in 60B.2)
- `resolvedUnknownNodeIds`: ["n_relocation_decision"] (was empty in 60B.2)
- No selected question generated — decision is complete (was generic continuation in 60B.2)
@@ -2728,6 +2795,7 @@ selectedQuestion: "What outcome would demonstrate enough value to justify contin
```
**What worked:**
- Decision identity preserved (status remained unknown)
- Both option identities preserved
- Client-retention uncertainty created as a first-class unknown node
@@ -2735,6 +2803,7 @@ selectedQuestion: "What outcome would demonstrate enough value to justify contin
- No fabricated factors or over-closure
**What was weak:**
- Generic question despite having just identified a specific material factor
- No observation node for the known financial comparison (unlike 60B.5)
- £5M/year not isolated from description text