Files
confidence-engine/docs/current-handoff.md
T

1142 lines
142 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Current Return-to-Work Handoff — Confidence Engine
> This file describes only the latest stopping point. Replace its current-work sections when the project moves on. Historical evidence remains in the design log and archive.
## 1. Where We Left It
- Engine experiments resumed with a passive validation;
- UI experiments remain paused;
- Knowledge-management experiments are complete;
- Experiment 39 tested the existing Behaviour Selection module against real Investigation State Assessment outputs across three scenarios;
- Acknowledge dominates (71% of selections) because it fires first when health=healthy, blocking Summarise/Pause/Clarify even in concluding or stalled states.
> This handoff describes the latest stopping point only. When work moves on, replace stale current-work details rather than appending another historical note. Historical experiment and commit information belongs in `docs/design-evolution-log.md`.
## 2. What Is True Now
- Main active engine path: deterministic reasoning pipeline (scenario reconstruction, graph update, unknown selection, question formulation, turn orchestration).
- Passive experimental classifiers from Experiments 1825B remain isolated diagnostic layers; none control the user-facing investigation. Behaviour Selection was passively evaluated against real assessment outputs in Experiment 39 — it produced all valid behaviours but with skewed distribution (Acknowledge 71%).
- Keyword and phrase-based scope detection remains provisional scaffolding.
- `docs/current-project-state.md` is the main entry point for active project state.
- Experiment 54D confirmed the production update prompt explicitly separates the user answer (## User Answer section) but the proposal schema has no provenance field — source identity at prompt level is explicit, per-node provenance at output level is absent.
Experiment 54R tested whether a consequential disagreement actually requires user clarification or can be resolved through evidence. Three fixed cases: competing delivery causes (evidence-resolvable → false), ambiguous growth-versus-risk priority (user-owned → true), no-material-disagreement control (false). All three correct (3/3) in one live inference call per case (~40s total). Across the three tested disagreement patterns, the model did not automatically map disagreement to user clarification. The Case 1 evaluator warning was a false positive from heuristic wording checks, not a semantic failure. No production code changed. Status pending Rob's review.
Experiment 56D confirmed that Regression B (conditional trade-off resolution) works end-to-end through the real `updateCase()` production path. Deterministic derivation correctly identifies conditional semantics, passes all guards, and produces a valid graph update with emergent threshold unknown — no regression detected from commit `3e78d57`. Status pending Rob's review.
Experiment 56E tested whether the weak-priority answer ("Risk matters more to me.") survives the full `updateCase()` production path without strengthening beyond relative importance. Result: **FAIL - semantic interpretation**. The LLM extracted userSupportedMeaning as "Avoiding additional risk is a preference/trade-off rather than a hard constraint" — asserting that risk is not a hard constraint, which goes beyond what the answer establishes (only relative importance). The deterministic guard passed because it saw the already-strengthened meaning. n-risk-constraint was incorrectly treated as resolved to "preference/trade-off". No emergent unknown created. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Status pending Rob's review.
Experiment 56F re-tested Regression A with the canonical live harness after Codex commit `4aa1492` (refine raw-answer boundary for answer meaning). Result: **PASS - strengthening safely rejected**. The LLM still produced semantic strengthening in `userSupportedMeaning` ("Avoiding additional risk is a strongly weighted preference/trade-off rather than a hard constraint") — the same class of over-resolution as 56E. However, the pre-mutation safeguard chain correctly rejected the proposal: deterministic derivation produced `proposedMeaningCategory: hard_constraint` which mismatched `rawAnswerCategory: relative_importance`, causing `proposalValidation.success: false` and preventing compatibility guard from passing. No graph mutation occurred — `n-risk-constraint` remained unresolved (status=unknown, value=null). One live call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed. Status pending Rob's review.
Experiment 56G tested Regression C (non-answer uncertainty: "I'm not really sure.") through the live production path to verify the risk-constraint distinction remains unresolved when the user expresses no position. **BLOCKED - apparatus**. The canonical helper (`tests/graph/live-update-experiment-helper.cjs`) contains a broken dynamic import path (`../lib/graph/orchestrator.js` resolves to `tests/lib/graph/orchestrator.js`, which does not exist — correct path is `../../lib/graph/orchestrator.js`). No live calls were made. Full results in `docs/experiment-56g.md`. Status pending Rob's review.
Experiment 56H re-tested Regression C after harness repair (commit c40d8c6). Result: **PASS - uncertainty preserved**. The LLM did not invent any constraint or preference position from "I'm not really sure." — `userSupportedMeaning` was null. No graph mutation occurred; `n-risk-constraint` remained unknown with value=null. One live call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed. Status pending Rob's review.
Experiment 54S tested whether, once clarification is known to be required, the model can identify exactly what the user needs to clarify — three fixed cases: growth-versus-risk priority (true → "preference/trade-off or hard constraint"), evidence-resolvable delivery causes (false → null), ambiguous meaning of "affordable" (true → "upfront cost versus long-term total cost"). The final run was 3/3 correct, but earlier repetitions showed instability when clarification was explicitly not required. Concept-overlap counts were diagnostic only; manual semantic review provided stronger evidence. Case 2 instability is an observed behaviour, not merely a test warning. Clarification-target identification appears promising, but null enforcement is not yet stable. Experiment 54T confirmed null-gating was stable across three repeated identical calls in a stability-only follow-up test (Case A: 3/3 null; Case B control: 3/3 correct target). The current instruction and output contract produced stable null behaviour across the three repeated false-case runs tested there; broader stability remains unproven. Experiment 54U tested whether a fixed clarification target can survive into one neutral user-facing question without adding meaning (preference/constraint, affordability definition, private factual capacity). All three cases returned correct single neutral questions with no introduced assumptions or evidence requests. The clarification-target → question step worked cleanly across the three tested targets; broader wording quality and user experience remain untested. Same host/model (qwen-claude:latest on http://192.168.1.111:11434); no production code changed. Status pending Rob's review.
Experiment 54V tested whether the user's answer can resolve only that target without rewriting the rest of the source meaning. Three fixed cases: hard constraint resolved (true/null), affordability definition resolved (true/null), incomplete answer preserved (false/uncertainty). All three correct across boundary preservation, no forced interpretations, and no unsupported consequences or new questions generated. Clarification answers resolved only the intended target across all tested cases. **The individual clarification steps have each worked in their isolated fixed-case tests; end-to-end behaviour remains untested.** Graph updates, next-question choice, Behaviour Selection, and UI remain untested. Same host/model (qwen-claude:latest on http://192.168.1.111:11434); no production code changed. Status pending Rob's review.
- `docs/task-context-packs.md` chooses the minimum context documents for each work type.
Engine and UI work were deliberately paused because documentation had grown large enough to overload Claude and make returning across sessions difficult. The current phase is simplifying what a fresh session must load to understand the project, without losing evidential history. Historical material remains available under `docs/archive/`.
## 4. What Was Just Completed
Experiment 37 corrected the routing defect from Experiment 36 and tested a cross-boundary engine/UI task. It validated that two context packs can be combined deliberately while keeping working context small, explicit and accurate. All seven knowledge-management criteria are now met. No source code changed. No files moved or deleted.
**Commit:** pending (experiment: validate cold-start project recovery) — to be committed this session.
Experiment 54X isolated target specificity using three fixed clarification cases under the exact same instruction as Experiment 54S. Case 1 (preference/trade-off versus hard constraint) returned "preferred priority between business growth and risk avoidance" — broadened from the material distinction but usable. Case 2 (upfront versus long-term affordability) preserved the definition boundary. Case 3 (user's available time next month) preserved capacity specificity. The same broadening pattern was reproduced across two tested runs under the same model and configuration, making it a repeatable candidate behaviour rather than a one-off observation. No question generation, answer resolution, Behaviour Selection, graph, or UI integration was attempted. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-target-specificity.test.js for the full experiment and results. Status pending Rob's review.
Experiment 54Y tested whether that specificity loss actually changes downstream clarification in a tested scenario. Source: "I want the business to grow, but I don't want to take on more risk." Fixed answer: "It's a hard constraint. I don't want any increase in risk." Variant A (precise target) generated question asking whether avoiding risk is a hard constraint or preference/trade-off; Variant B (broadened target) generated question asking which to prioritize when growth and risk conflict. Both resolved the same answer with materially equivalent meaning. With the explicit hard-constraint answer used in this test, both target variants converged on materially equivalent resolved meaning. The broader target changed the clarification question but not the resolved meaning for the tested explicit answer; broader safety remains untested. Behaviour Selection, graph, UI, and production integration remained untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-specificity-consequence.test.js for the full experiment and results. Status pending Rob's review.
Experiment 54Z tested whether convergence between precise and broadened targets holds with weaker answers. Source same as 54Y. Two weak answers tested against both fixed variants: (1) "Risk matters more to me" — both variants produced materially equivalent meaning (risk not a hard constraint, but stronger than growth). (2) "I'd normally avoid more risk, but for the right opportunity I might accept some" — variants diverged: Variant A collapsed conditionality into flat preference; Variant B preserved conditional structure and remaining uncertainty. Unexpectedly, the broader target preserved more nuance for the conditional answer. Target broadening has material consequences with weaker answers, but direction is unpredictable. 4 live calls completed. Behaviour Selection, graph, UI, and production integration remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-weak-answer-consequence.test.js for the full experiment and results. Status pending Rob's review.
Experiment 55A isolated the answer-resolution step using one fixed target and four answers of varying strength (explicit hard constraint, weak priority, conditional trade-off, non-answer). Two of the four tested answers showed loss of nuance: one was over-resolved (weak priority set targetResolved=true with inferred "not a constraint" meaning) and one retained the correct target category while losing conditional qualification ("might accept some for the right opportunity" became "preference or trade-off rather than a hard constraint"). The same over-resolution reproduced with a fixed target, so target broadening is not required for the failure to occur. 4 live calls completed at ~62s total. The answer-resolution step appears biased toward resolution for weak priority statements. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-uncertainty-preservation.test.js for the full experiment and results. Status pending Rob's review.
Experiment 55B separated answer meaning from target-resolution judgement using independent calls per case. Three fixed answers tested (weak priority, conditional trade-off, non-answer) through two modes each: Mode A (meaning-only, no resolution decision) and Mode B (resolution via the same 54V/55A instruction). Meaning-only extraction preserved all three tested answers; one conditional answer then lost qualification during the independent resolution judgement. Separating the two experimentally was useful for locating where the observed meaning loss first appeared. Additionally, Case 1 (weak priority) resolved correctly in 55B but over-resolved in 55A — this does not establish that the weak-priority problem is solved; it indicates run-to-run variation. 6 live calls completed at ~104s total. No production code changed. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-answer-meaning-vs-resolution.test.js for the full experiment and results. Status pending Rob's review.
Experiment 55C chained actual preserved meaning from Stage 1 into Stage 2 resolution, testing whether carrying semantic state forward removes the conditionality loss observed in 55B. Three cases tested (weak priority, conditional trade-off, non-answer) through two stages each = 6 live calls at ~117s total. Case 2 conditional qualification survived through both stages and resolved correctly (targetResolved=true with condition retained). Case 3 non-answer uncertainty preserved through both stages. Case 1 over-resolved in Stage 2 because Stage 1 itself strengthened "risk matters more" into language about "preference/trade-off rather than absolute constraint." Compared to 55B, the weak-priority case did not remain honestly unresolved — If Stage 1 distorts the answer, Stage 2 may preserve and act on that distortion rather than correct it. No two-stage design is proven superior; meaning can be lost at either stage. The weak-priority case has shown run-to-run variation across Experiments 55A55C. Graph, Behaviour Selection, UI and production remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-preserved-meaning-resolution.test.js for the full experiment and results. Status pending Rob's review.
Experiment 55D tested whether a first interpretation step can separate what the user established from what the model might infer, using a single-call two-field output contract (statedMeaning / possibleInference) across four fixed answers: weak priority, conditional trade-off, explicit hard constraint, and non-answer. Four live Ollama calls at http://192.168.1.111:11434 with qwen-claude:latest (~76.7s total). All four cases preserved statedMeaning without strengthening (stated_meaning_preserved: 4/4, strengthened: 0, lost: 0). Case 1's weak-priority answer stayed as relative importance only — direct improvement over 55C where the same answer was strengthened to constraint language. Conditionality survived in Case 2; explicit and uncertain controls stayed clean in Cases 3 and 4. Inference cleanly separated for Cases 1 and 2; unnecessary inferences generated for Cases 3 and 4 (hygiene issue, not leakage). No unsupported meaning leaked into statedMeaning. This does not yet prescribe production architecture. Graph, Behaviour Selection, UI and production remain untouched. Same host/model (qwen-claude:latest on http://192.168.1.111:11434). Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-stated-vs-inferred.test.js for the full experiment and results. Status pending Rob's review.
Experiment 38 tested whether a genuinely cold session (no prior conversation context) can recover the project state from three documents alone. It recovered all capabilities, boundaries, and context-pack selection correctly without loading the full history or source code. All seven knowledge-management criteria confirmed met. One handoff update required: the open item "whether the handoff stays accurate after further advances" was resolved (handoff is accurate). The cold-start test passed.
**Commit:** pending (experiment: validate cold-start project recovery) — to be committed this session.
Experiment 39 resumed reasoning experiments with a passive validation of Behaviour Selection against real Investigation State Assessment outputs. Seven turns across three scenarios were evaluated. Acknowledge dominated (71%) because it fires at priority 1 whenever health=healthy, even in terminal and stalled states where Summarise or Pause would be more useful. The assessor→selector contract aligns cleanly; no transformation is needed between pipeline stages. All five behaviours remain reachable but some never appear in typical scenarios (Clarify requires too_broad health which few fixtures produce). Status pending Rob's review.
Experiment 40 diagnosed the root causes: Summarise and Pause fire their rules in real data but are always blocked by Acknowledge's priority-1 position (priority conflict, not assessor failure). Clarify's triggers never activate in tested scenarios due to the `too_broad` health condition being extremely narrow. All five behaviours confirmed independently reachable in synthetic isolation. No rules changed.
Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:
- Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge — introduces false-positive summarise in focusing phase
- Variant B (Acknowledge exclusions): keep priority, gate Acknowledge when phase=concluding/synthesising or progress=stalled or health=user_overloaded — recommended
- Both variants converge on the same two genuine changes: concluding→summarise and stalled→pause
Experiment 42 implemented Variant B's narrow Acknowledge exclusion gate in the production selector (commit `05d3d96`). Summarise now appears at conclusion; Pause now appears when stalled. All other tested turns remain unchanged. Behaviour Selection remains passive and isolated with no runtime caller — active user-facing engine behaviour did not change.
Experiment 43 audited Clarify readiness across all 10 real assessment turns in existing fixtures. Zero turns produced Clarify-eligible states. Two findings: (1) the orienting-based Clarify rule is dead code because the assessor never produces phase=orienting, and (2) the too_broad trigger requires conditions no fixture exercises. Branch: `feature/user-workspace-ux-v0.7`.
Experiment 44 created one deliberately unclear starting scenario (five competing unknowns, zero resolved evidence, vague central statement) to test whether the assessor produces a Clarify-justifying signal. The assessor returned `too_broad` conversation health — confirming the previously untested too_broad path works correctly with real data. Clarify became eligible via Rule A. No production code changed. Remaining open: whether orienting phase is needed for earlier-stage clarification, and whether 23 competing threads (below the >3 threshold) can represent genuine scope confusion. Status pending Rob's review.
Experiment 45 tested the too_broad boundary from two to five competing unknowns using identical synthetic fixtures varying only in unknown count. The assessor switched at exactly three→four active unknowns — two and three returned cannot_determine; four and five returned too_broad. Clarify eligibility followed the same boundary. Resolved-item gate works correctly: one resolved item stays too_broad, two resolves it. The boundary appears mechanically clear but conceptually uncertain — synthetic fixtures cannot confirm whether three-to-four feels right to real users. No production code changed. What remains open: whether health should default to healthy (not cannot_determine) for 23 unknowns with no question; whether the threshold needs widening for real-world use. Status closed.
Experiment 46 compared two four-unknown investigations with identical structural counts — one coherent (four unknowns contributing to one decision) and one scattered (four unrelated threads). Both returned too_broad with Clarify eligible, confirming the assessor cannot distinguish semantic coherence from scatter using active-unknown count alone. No production behaviour changed. Status closed.
Experiment 47 created a test-only diagnostic helper (`inspectSharedUnknownAnchor`) that inspects existing graph relationship fields to distinguish shared-anchor investigations from scattered ones. Three controlled fixtures (shared/separate/none anchors, all with identical structural counts) confirmed the helper correctly distinguishes all three patterns. Inspecting three real scenarios from Experiments 39-46 returned insufficient_data for all — existing data lacks populated relationship fields on unknown nodes. The assessor remains unchanged. Status pending Rob's review.
Experiment 48 audited whether real graph updates populate usable unknown relationships. Three production paths inspected: `buildInitialGraph` (does NOT populate dependsOn/affects/parentId), emergent reasoning via `buildEmergentReasoningUnknown` (DOES populate dependsOn and parentId), decomposition children (DOES populate parentId). One test file created (16 tests, all pass). Conclusion: Insufficient Data — shared-anchor detection works through the emergent-unknown path only. Status closed.
Experiment 49 tested whether any sequence of real production updates creates two or more active unknowns referencing the same populated relationship anchor. Results: no shared anchor found in production update sequences (both Cases A and B returned separate_anchors or insufficient_data). Structural capability exists but triggering logic never produces coexisting anchors. Status closed.
Experiment 50 tested whether shared edge topology from `buildInitialGraph` provides a usable coherence signal. Coherent and scattered inputs both produce identical edge topology — every unknown connects to the same summary node (kind=state) via depends_on edges, regardless of semantics. Initial shared edges are generic structural wiring, not coherence evidence. Closed (pending Rob's review).
Experiment 51 tested whether decision-relative relevance distinguishes coherent from scattered unknowns better than graph topology does. Within its training vocabulary, the classifier classified all four coherent unknowns as relevant and three of four scattered unknowns as irrelevant — but one scattered question was incorrectly flagged due to identical phrasing. Outside its vocabulary (different domain or paraphrased language), the classifier could not generalise: all four coherent unknowns received `cannot_determine`. The decision target never provided semantic context, only a binary action-keyword gate. No production code changed; no active engine behaviour changed; 70 tests pass (45 new + 25 Exp 21 regression). Status pending Rob's review.
Experiment 52 tested whether a small semantic interpretation step can judge decision relevance more reliably than keyword matching across paraphrases and domains. The semantic contract was implemented in `tests/graph/decision-relevance-semantic.test.js`. Live model comparison could not be completed because Ollama is not running on this machine — the test infrastructure uses the same `/api/chat` + `format:json` pattern as production. The deterministic keyword baseline continues to fail on paraphrases and new domains (confirmed via 15 passing guardrail tests). No semantic logic entered the active engine. The four-category decision-relevance contract remained unchanged. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic.test.js` for the full experiment and results.
Experiment 52H held domain constant (market-entry / customer demand) and varied ambiguous wording across five cases. Four phrasings were strengthened beyond their supplied meaning; only "connected to" preserved `cannot_determine`. The model appeared more consistent about strengthening incomplete meaning than about which stronger category it selected. Experiment 52I then tested one grounding rule rather than keyword patches: three of four ambiguous cases preserved `cannot_determine` under grounding without harming clear classifications, but "important to" remained strengthened — the model could classify correctly while still commenting on relationship strength. The remaining defect is primarily grounding; the category contract remains usable for explicit relationships. Same host and model retained; no production behaviour changed. Status pending Rob's review.
Experiment 52A recovered the semantic test infrastructure by correcting its configuration resolution. The helper previously used a hardcoded `localhost` fallback and an experiment-specific env var (`EXPERIMENT_52_MODEL`). Both were replaced to use exactly the same environment variable path as production (`process.env.OLLAMA_BASE_URL` / `process.env.OLLAMA_MODEL`) sourced from `.env.local`. Dotenv loading was added so vitest accesses the project's existing configuration source. Ollama at 192.168.1.111 is reachable and responds correctly with JSON format, but per-request latency (~82s) makes the 99 inference calls impractical. Configuration path verified correct; execution requires a faster inference host. No production code changed (0 lines in provider, config, analysis, orchestrator). Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic.test.js` lines 8085 (helper).
Experiment 52C separated free-language semantic understanding from enum normalisation into two independent calls per case across five decision/question pairs. Meaning mode captured all five intended relationships correctly (5/5). Enum classification matched expected categories on four of five cases (4/5). One meaning-correct / enum-mismatch case: Case 2 (European regulatory compliance) was correctly described as supporting in both modes but classified as `could_change_decision` rather than `supports_decision`. Same Qwen model (`qwen-claude:latest`) and host were retained; no production behaviour changed. What remains uncertain: whether the meaning-enum gap generalises across decision domains, stability over repeated runs, and whether normalisation mechanisms can bridge the gap without altering interpretation. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-semantic-normalisation.test.js` for results.
Experiment 52D isolated enum normalisation from semantic understanding: five fixed meaning statements (no decision target or question in the input) were mapped to the existing four-category contract via one live model call each. Four of five normalised to the expected enum. The compliance boundary case persisted — the model classified a "supports" relationship as `could_change_decision`, exposing genuine ambiguity between these two categories under the current definitions. The existing contract appears clear enough for a separate normalisation step; the remaining problem lies in category definitions, not semantic understanding or normalisation mechanism. Same Qwen model (`qwen-claude:latest`) and host (`http://192.168.1.111:11434`) were retained throughout. No production behaviour changed. What remains uncertain: whether the `supports_decision``could_change_decision` boundary can be clarified without restructuring the contract, and whether the discrepancy holds under repeated runs. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/graph/decision-relevance-normalisation.test.js` for results.
Experiment 54H tested whether trustworthy source identity can begin deterministically from raw user input before any LLM interpretation occurs. A test-only helper `createSourceRecord(rawInput)` hashes the verbatim text with SHA-256 to produce a stable `sourceId`, preserves `verbatimText` unchanged, and sets `sourceType: "user_input"`. Nine focused tests confirm identical inputs produce identical IDs (Case 1 = Case 4), paraphrases produce different IDs (Case 1 ≠ Case 2), and multi-sentence input survives intact (Case 3). No semantic interpretation, summarisation, or LLM call occurs. Trustworthy source identity is feasible before reconstruction — the remaining gap is claim/node provenance and graph linkage, not source identity. Deterministic code can assign stable identity to raw material at the application boundary without any reasoning contract. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/reconstruction/deterministic-source-record.test.js`.
Experiment 53 proved semantic separation of supplied meaning from possible inference is achievable. Experiment 54A confirmed the SituationGraph cannot recover provenance from graph state alone. Experiment 54B traced supplied-versus-inferred distinction upstream to evidenceRecordSchema but found it lost at buildInitialGraph because the node schema has no provenance field. Experiment 54C inspected the normal answer-update boundary: whole-input origin is explicit (answer = user supplied; proposal = model produced) but per-node provenance inside the proposal is not deterministically recoverable from the validated proposal alone. Experiment 54D audited the production update prompt: it clearly separates the user answer (## User Answer section) and instructions, so prompt-level source identity is explicit; however the proposed output schema has no provenance fields on nodes or edges, so per-node provenance at output level is absent — the tested prompt already preserves user-source identity clearly; the blocking gap identified here is that the validated proposal does not carry per-node provenance forward. The eventual representation remains undecided. Experiment 54E audited whether existing evidence IDs and evidence records could preserve provenance referentially without a new node field: the evidence-record schema contains vocabulary capable of distinguishing supplied-like from inferred-like material, but the reference chain breaks because (1) evidence records are consumed during startCase and never returned alongside graph state — no persistence layer retains them; and (2) no evidence records are created or retained during update cycles. Experiment 54E did not validate how those values are assigned in production. Experiment 54F audited evidenceType assignment: the reconstruction prompt instructs the LLM to classify each evidence item into one of five types based on its own judgment; no production code deterministically derives evidenceType from source origin — even reported_statement means "the model thinks this looks like a reported statement" not "production code knows this came directly from the user." Experiment 54G audited whether evidence records nevertheless retain deterministic linkage to user words: neither verbatim text nor structured location references (character offsets, turn IDs) survive in any record field; `source` and `attribution` are free-form model-generated strings that may be null; the raw user statement is available to production code while reconstruction is being performed but is not retained alongside the returned reconstruction/evidence state for later deterministic verification. Evidence records do not contain verbatim source text or deterministic source locations; `evidenceType` is model classification, not trustworthy provenance. Current evidence records therefore cannot independently prove source provenance.
Experiment 54I showed multiple interpretations can share one deterministic source lineage via the Experiment 54H SHA-256 method. Both branches stayed traceable to the same source while remaining distinct in their reported additions. No interpretation was selected as better and no numeric scoring occurred. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/reconstruction/source-interpretation-lineage.test.js`.
Experiment 54J proved the representation can separate source-supported from interpretation-added meaning using human-fixed references (13 tests, all pass). Grounding references were human-fixed; automated grounding remained untested. No production code or schemas changed. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect: `tests/reconstruction/interpretation-source-grounding.test.js`.
Experiment 54K tested whether the configured semantic model (`qwen-claude:latest` on `192.168.1.111:11434`) can perform that grounding automatically. Three live Ollama calls (total ~96s): Case 1 (strengthening detection) = grounding_correct, Case 2 (multi-addition interpretation) = partial_grounding (missed one addition), Case 3 (faithful restatement control) = grounding_correct. Interpretation-added meaning did NOT leak into source-supported meaning in any case. One source-supported content gap: model missed "alternative causes" on the added side of Case 2. Automated semantic grounding is promising but imperfect — directionally viable but needs refinement before production use. Winner selection and downstream questions remain untested. No production code changed. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `tests/reconstruction/semantic-interpretation-grounding.test.js`.
Experiment 54L repeated two identical grounding cases three times each to test stability across six live calls. The source-versus-added boundary was perfectly stable (zero leakage in all runs). Detection completeness appeared variable but manual analysis showed the instability came from the automated evaluator's paraphrase sensitivity, not the model itself. Case A strengthening identified in all 3 runs; Case B "other causes" and "not established as main problem" each identified in all 3 runs. Status pending Rob's review.
Experiment 54M tested whether two interpretations of one source can expose their substantive disagreement without deciding which is correct. Three live Ollama calls across three cases: real pricing attribution difference, paraphrase identity control, and competing causal explanations. All three classified as disagreement_correct by human semantic review. Paraphrase was correctly treated as agreement; shared meaning stayed separate; no invented disagreement or winner selection occurred. The comparison capability worked across the three tested patterns: substantive disagreement, paraphrase agreement, and competing causal explanations. Broader generalisation remains untested. Status pending Rob's review.
Experiment 54N tested whether an interpretation disagreement can be judged for material consequence on downstream information needs without generating a next question or choosing a winner. Three fixed cases: pricing ambiguity (consequence_correct), paraphrase identity control (consequence_correct), competing causes (consequence_failed — model returned false, missing that staff-capacity vs supplier evidence represent divergent investigation directions). 2/3 correct. Model did not choose a winner or generate an actual next question in any case. Branch: `feature/user-workspace-ux-v0.7`. First file to inspect when resuming: `tests/reconstruction/semantic-disagreement-consequence.test.js`.
## 5. What Remains Open
- The `too_broad` boundary sits exactly between three and four active unknowns; it is mechanically clear but conceptually uncertain — whether it aligns with genuine user confusion requires real-scenario validation;
- Health defaults to `cannot_determine` rather than `healthy` for 23 unknowns (no active question present); whether this is a bug or feature needs review;
- Whether the `too_broad` threshold needs widening so Clarify fires in more typical investigations;
- Whether `user_overloaded` health should be producible by the assessor for stalled/inconsistent evidence states;
- Existing-scenario graphs lack populated relationship fields on unknown nodes from the initial-build path; coherence detection works through the emergent-unknown path only (Populates `dependsOn` and `parentId` correctly — but requires comparable observations to trigger);
### When This Knowledge-Management Phase Is Complete
Provisional criteria for review (all confirmed met by Experiment 38 cold-start test):
1. A fresh session can resume from the handoff and one context pack; — **met**
2. Current state has been verified against implementation; — **met**
3. Historical material is outside default loading; — **met**
4. Current principles are separated from aspirational architecture; — **met**
5. Task-specific routing works for engine and UI tasks; — **met**
6. A cross-boundary task has been tested; — **met** (Experiment 37)
7. Maintaining the handoff does not require reading the full history. — **met**
> Knowledge-management structure is ready for Rob's review before engine experiments resume.
## 6. How to Resume
1. Read `docs/current-handoff.md`.
2. Read `docs/current-project-state.md`.
3. Choose one pack from `docs/task-context-packs.md`.
4. Read `.claude/architecture-guardrails.md` before any code change.
5. Load extra context only for a named gap — record why.
6. Check Git status before continuing.
## 7. First Files by Work Type
| Work type | Start with |
| ------------------------------- | ----------------------------- |
| Engine experiment | Engine Experiment pack |
| UI or mock work | UI and Mock pack |
| Architecture or contract review | Architecture or Contract pack |
| Knowledge management | Knowledge-Management pack |
## 8. Resume Check
Answer before continuing:
1. What work is currently active?
2. What work is paused?
3. What was the latest completed experiment?
4. Which context pack applies to the next task?
5. Is there any uncommitted work?
---
_Created by Experiment 34. Updated by Experiments 3853, 54A54Z, 55A55F, 56D56H, 56L56M, v0.8 closeout. Branch: `feature/reasoning-fidelity-v0.8`. First-pass reasoning-fidelity v0.8 complete to AF scope._
---
### Experiment 57J.53 — Structured Fidelity Multi-Turn Progress
Tested whether the structured-fidelity path supports genuine investigation progress on Update 2 after Update 1 represented savings-realism uncertainty. **Classification: B — USEFUL PARTIAL PROGRESS.**
Two-turn run (fixed scenario: "We are considering relocating the engineering team to reduce operating costs.") with Answer 1 ("I am unsure whether the projected office savings from the relocation are realistic.") and Answer 2 ("The projected savings are based on the current London lease, business rates, service charges, utilities and facilities costs that would no longer be incurred at the same level after the move. The estimate is approximately £2M per year.")
**Update 1:** HTTP 200, supportCategory="uncertain", but did not create a dedicated savings-realism unknown node — instead degraded an existing unrelated node's status to provisional. Selected next question referenced "realism of projected office savings" correctly. Partial success: structured field populated but downstream structural action was weak.
**Update 2:** HTTP 200, supportCategory="other". Added new supported metric node (£2M/year savings) but did not act on the existing savings-realism uncertainty from Update 1. Evidence and uncertainty exist in parallel with no cross-linkage. No next question generated due to unknown-selection tie (all three original unknowns tied).
**Key finding:** Structured `supportCategory` works correctly across turns (model adapts category to answer semantics), but the downstream graph effect of "uncertain" status was not a dedicated unknown node as expected from 57J.52's earlier pattern. Evidence was added in Update 2 but remained structurally disconnected from the existing uncertainty. Cold-start variance (7 vs 8 nodes) may affect whether new-node creation triggers for this category.
**What remains unproven:** Stability of no-new-node behavior for uncertain status; whether cold-start node count determines structured-path outcomes; tie-breaking resolution mechanism that blocked next-question generation. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed.
---
### Experiment 57J.54 — Uncertainty Identity vs Relatedness Diagnosis
**Diagnosed why** explicit savings-realism uncertainty in 57J.53 Update 1 was absorbed into a broader cost node (`noymlfr`) rather than represented as a dedicated unknown. **Classification: B — PROMPT SEMANTIC-IDENTITY GAP.** The root cause is not a validator defect (the exact-string duplicate detector works correctly for what it checks) or a graph-model defect (the schema can represent the distinction). The gap is in the prompt's Additional Guidance line 137: it instructs the model to distinguish "same uncertainty" from "merely related uncertainty" but provides no criterion for when a broad cost node covers a specific savings-valuation concern versus asks a different question about it. Concept A ("total costs at new location") and Concept B ("are projected office savings realistic?") are OVERLAPPING BUT DISTINCT — one is magnitude estimation across all categories; the other is assumption validity for one specific category. Resolving Concept A does not resolve Concept B. **Recommended boundary: prompt-only clarification** of rule #7's "unresolved term" criterion to explicitly treat uncertainty about a specific sub-component within an existing uncertain topic as a new unresolved term requiring its own node. No production code changed, no Ollama calls. Full record in `docs/experiment-57j54.md`.
---
### Experiment 57J.60 — Selected-Question Contract Live Validation
**Objective:** Validate that v0.22's mandatory selectedQuestion rule (rule #16) works live when the model adds a new dedicated savings-realism unknown. **Classification: A — V0.22 FIX WORKS LIVE.** One start + one update through the production `updateCase()` path. Update 1 added one dedicated unknown node (`n_proj_savings_realism`, kind=unknown, status=unknown) and included a valid selectedQuestion candidate (nodeId=`n_proj_savings_realism`) referencing that same unresolved node. The old 57J.57 contract rejection — new unresolved unknown + selectedQuestion=null — did not recur. HTTP 200 at `update_applied`, no `proposal_compatibility` error, no rejectedProposalSnapshot. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Full record in `docs/experiment-57j60.md`.
---
### Experiment 57J.62 — Accepted-Update Capture Hardening
**Classification: A — FIX VALIDATED.** Diagnosed that 57J.61 failed because the harness accepted-update path (script lines ~97108) printed only `HTTP status`, `stage`, `selected question`, `node count`, and `edge count` — zero answer-meaning fields, zero structural mutation fields. After Update 1 applied successfully with HTTP 200 at `update_applied`, the harness could not identify which nodes were added or what the resulting persistent graph looked like.
**Fix:** Extended the accepted-update console block in `scripts/reproduce-multi-turn-investigation.mjs` to print:
- `answerMeaning.userSupportedMeaning`, `.possibleInference`, `.supportCategory`, `.resolutionGuidance`
- `updatedProposal.updatedNodes[]`, `.resolvedUnknownNodeIds[]`, `.addedNodes[]`, `.addedEdges[]`
- `selectedQuestion.nodeId` (node reference)
- Compact structural snapshot of `resulting graph` (id, kind, label/description, status per node; from/to/relationship per edge)
Fixed a co-occurring bug where the accepted-update block referenced `startResult.status` instead of `updateResult.status`.
**Tests:** 10 new harness tests (8 in test suite + 2 for existing guarantees), all pass. Mocked API responses only. Zero Ollama calls. No production code changed. No extra HTTP calls introduced. No-retry contract preserved intact.
**What this establishes:** Future experiments will produce deterministic, inspectable evidence of every accepted update's graph mutations without requiring a second API call or manual inspection.
---
### Experiment 57J.61 — Equivalent Uncertainty Identity Live Test
**Objective:** Once a dedicated savings-realism uncertainty exists, does a second semantically equivalent statement reuse that same unresolved node rather than create a duplicate? **Classification: D — UPDATE 1 FAILED.** One start + two updates. Start HTTP 200 (6 nodes). Update 1 returned HTTP 200 at update_applied but the harness crash prevented detailed proposal capture. A cold-start variant confirmed that when userSupportedMeaning is populated for savings-realism uncertainty, the model extracts meaning but proposes zero graph mutations — updatedNodes=[{nodeId: X, newValue: null}], addedNodes=[], addedEdges=[]. The gateway rejects this at proposal_compatibility with "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation." Update 2 was reached (total 3 calls) and was rejected for the same reason. **Neither turn established a persistent savings-realism unknown.** The identity invariant cannot be tested when neither turn produces a valid, persistent unknown node. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Full record in `docs/experiment-57j61.md`.
---
### Experiment 57E — Irrelevant Decomposition Question Boundary
**Objective:** Identify the exact graph node that triggered the decomposition producing "How the two observations were measured", and determine whether the parent was genuinely about comparison/measurement/timing before decomposition.
**Run results (one start + one update, scenario: team relocation London→Manchester, answer: cost reduction £2M savings):**
- **Parent node:** `nagtmgmg` — label: "Explanation for why Should I relocate my engineering team from London to Manchester", description: "Need to understand what change or event could explain why these observations differ..."
- **Parent kind/status:** unknown/unknown
- **Selected child (Update 1):** `np6zcaw` — label: "How the two observations were measured"
- **Reasoning pattern assigned to selected child:** comparison
- **Investigation strategy:** evidence_gathering
**Key finding:** The parent node `nagtmgmg` has NO semantics of comparison, measurement validity, or timing. Its description only references "these observations differ" in a generic explanatory sense (what change/event explains the difference between initial state and current state). It does not establish that there are two measured observations to compare. Yet decomposition produced five children including hardcoded "two observations" templates:
1. Whether the two observations reflect different timing
2. How the two observations were measured
3. Possible change mainly affecting [first observation]
4. Possible change mainly affecting [second observation]
5. Possible one-off event during the period
The parent itself is a generic "explanation for difference" unknown — structurally similar to any post-hoc explanation query — and does NOT contain comparison/measurement semantics. The "two observations" language in decomposition children originates from `buildDecompositionTemplates()` default template (line 14841510 of `lib/graph/apply-proposal.js`) which unconditionally injects these children for any unknown parent that doesn't match the special-case regex patterns.
**Classification: A — decomposition trigger defect.** The parent node is not genuinely about comparing observations, measurement validity, or timing comparability, yet deterministic decomposition generates measurement/comparison children. The decomposition mechanism fires on an inappropriate parent whose semantics are purely explanatory ("what change/event explains the state difference").
**What this establishes:** The "two observations" decomposition children are template-injected regardless of parent meaning. They appear whenever `buildDecompositionTemplates()` runs for a generic unknown node that doesn't match special-case regex patterns. This is not a case of semantically appropriate decomposition with overly-specific templates — the decomposition trigger itself is firing on an irrelevant parent.
**What this does NOT prove:** That every decomposition is irrelevant (some parents genuinely concern comparison/measurement). That fixing the trigger won't break valid decompositions elsewhere. Whether other template children (change affecting X/Y, one-off event) share the same defect pattern or have independent justification issues.
---
### Experiment 57C — Post-v0.9 Investigation Flow Observation
**STOPPED AT FIRST PRODUCTION-PATH FAILURE**
Attempted first-post-v0.9 multi-turn investigation through the real `startCase()``updateCase()` production path with a team-relocation scenario (London→Manchester). Turn 1 completed: selected question about "primary driver", user answered about cost savings/£400K funding. On Turn 2, a live model response produced `relationship: "affects"` as a graph edge value; the current production graph/update schema rejected that relationship string, preventing progression through the normal investigation path. Run was manually stopped before any workaround was accepted or any fix attempted. No `docs/experiment-57c.md` was written — this entry has been superseded by it.
---
### Return-to-Work Note (Experiment 55F)
The first implementation pass against the reasoning refinement requirements is deferred one more round while we map how meaning actually flows through the production update path — before committing to any schema or architecture changes. A source-inspection exercise traced the full answer-to-reasoning chain from prompt building, through LLM response parsing and normalization, into graph mutation. The key finding: no provenance fields exist on nodes or edges in the current schema, meaning R1/R2 separation has no structural carrier. The answer string is used only for a narrow comparability check, not for semantic verification against proposed changes. A complete path map lives in `docs/reasoning-production-path-map.md`. Tomorrow should decide whether to add provenance fields to schemas, modify the prompt structure, or both — grounded in this accurate production trace rather than architectural speculation. Branch: `feature/user-workspace-ux-v0.7`.
### Experiment 55A Summary — Clarification Uncertainty Preservation
Isolated the answer-resolution step using one fixed target (preference/trade-off or hard constraint) and four answers of different strength: fully explicit, weak priority, conditional trade-off, non-answer. Four live Ollama calls completed at http://192.168.1.111:11434 with qwen-claude:latest (~62s total). Case 1 (explicit hard constraint) resolved correctly. Case 2 (weak priority — "Risk matters more to me.") over-resolved: the model set targetResolved=true and inferred "not a rigid, non-negotiable constraint" — meaning stronger than the user supplied. Case 3 (conditional trade-off) resolved correctly on the target but flattened conditionality into flat "preference or trade-off" language without preserving the conditional qualification ("might accept some"). Case 4 (non-answer) correctly remained unresolved with appropriate remaining uncertainty. Two of the four tested answers showed loss of nuance: one was over-resolved and one retained the correct target category while losing conditional qualification. The same over-resolution reproduced with a fixed target, so target broadening is not required for the failure to occur. Broader generalisation across other models and answers remains untested. Behaviour Selection, graph, UI, and production integration remain untouched. Branch: feature/user-workspace-ux-v0.7. First test/file to inspect when resuming: tests/reconstruction/semantic-clarification-uncertainty-preservation.test.js for the full experiment and results. Status pending Rob's review.
### Experiment 56A Summary — Regression B Proposal Validation Enum Mismatch
The first implementation pass added proposal-level `answerMeaning` with a pre-mutation compatibility guard. Deterministic regression tests A-D passed, but live Ollama runs showed Regression B failing at `proposal_validation` before the pre-mutation guard could execute. Experiment 56A traced this to a schema mismatch: Qwen returned `supportCategory: "conditional_qualification"` while the production Zod schema only accepts `conditional_tradeoff` among five values. The value survives normalization unchanged (normalize step handles node kind aliases, not supportCategory). The failure is at Zod validation — a proposal-contract issue, not a guard failure. **Hypothesis confirmed.** No fix was attempted. Branch: `feature/reasoning-fidelity-v0.8`. First file to inspect when resuming: `lib/graph/schema.js` line 165 (Zod enum for supportCategory) or the experiment record at `docs/experiment-56a.md`. Status pending Rob's review.
### Experiment 56B Summary — Regression B Live Run After Normalisation
Commit `36faf70` added normalization for `conditional_qualification → conditional_tradeoff`, but a live Regression B run returned a _different_ variant: `supportCategory: "conditional_preference"`. The existing normalisation map does not cover this value. Two independent Zod errors occurred: (1) `conditional_preference` not in the supportCategory enum, and (2) `resolutionGuidance` was free-text instead of an enum value. **Run-to-run model variation confirmed** — the same fixed input produced `conditional_qualification` in Ex 56A and `conditional_preference` in Ex 56B. The pre-mutation guard remains unreachable because proposal_validation rejects first. Failure classification: `FAIL — normalization / proposal contract`. Branch: `feature/reasoning-fidelity-v0.8`. File to inspect when resuming: `docs/experiment-56b.md`. Status pending Rob's review.
### Experiment 56D Summary — Regression B via Real Production Path
Tested whether deterministic derivation refinement from commit `3e78d57` (refine answer meaning derivation for negation and qualification) works end-to-end through the real `updateCase()` production path. Input: source "I want the business to grow, but I don't want to take on more risk." Answer "I'd normally avoid more risk, but for the right opportunity I might accept some." — the canonical conditional trade-off case (Regression B).
**Result: PASS.** Five of five checkpoints confirmed across one live Ollama call at `http://192.168.1.111:11434` with `qwen-claude:latest`:
1. `userSupportedMeaning` correctly extracted conditional semantics — separated default preference (avoid risk) from qualification (override for right opportunity).
2. Deterministic profile derivation produced `conditional_tradeoff` category despite LLM returning null for `supportCategory`.
3. Pre-mutation guard passed with zero errors — the normalized/derived meaning is compatible.
4. Graph mutation proposed: `n-risk-constraint` resolved from unknown→resolved; emergent unknown `n-opportunity-criteria` created (unknown/unknown) capturing the threshold definition need.
5. Follow-up question correctly targets the emergent conditional/threshold unknown.
**Key observation**: The LLM does not auto-populate `supportCategory` — it is consistently null in `answerMeaning`. The deterministic derivation layer in `readDiagnostics` (and the inline pipeline) is the sole mechanism by which meaning profile category gets determined. This confirms the design: LLM produces raw meaning; deterministic logic categorizes it. No regression detected. Full results in `docs/experiment-56d.md`. Branch: `feature/reasoning-fidelity-v0.8`. Status pending Rob's review.
### Experiment 56J Summary — Regression D Explicit Hard Constraint Semantic Probe
Tested whether the configured live Ollama model (`qwen-claude:latest` at `http://192.168.1.111:11434`) preserves explicit hard-constraint meaning from user answer "It's a hard constraint. I don't want any increase in risk." — Regression D from `docs/reasoning-refinement-requirements.md`.
One live Ollama call (19,343 ms) returned `userSupportedMeaning: "Avoiding additional risk is a hard constraint, and no increase in risk is acceptable."` with `possibleInference: null`.
**Classification: PASS.** The model preserved the explicit hard-constraint status without weakening it into preference/trade-off language and did not add unsupported interpretation. `possibleInference` is null, which is appropriate for a direct unambiguous answer.
This experiment does not prove fidelity for other regression cases (E, F), consistency across multiple runs, or behavior in production reasoning paths. Branch: `feature/reasoning-fidelity-v0.8`. Files: `tests/reconstruction/semantic-regression-d-explicit-hard-constraint.test.js` and `docs/experiment-56j.md`. Status pending Rob's review.
### Experiment 56K Summary — Evidence-resolvable disagreement must not become user clarification
Tested whether the configured live Ollama model (`qwen-claude:latest` at `http://192.168.1.111:11434`) distinguishes evidence-resolvable uncertainty from user-owned ambiguity — Regression E from `docs/reasoning-refinement-requirements.md`.
Fixed case: Delivery delay concern with competing causes ("Staff capacity may be the issue" / "Supplier lead times are likely responsible.") — resolvable by evidence gathering, not user clarification.
One live Ollama call (18,580 ms) returned `uncertaintyType: "evidence_needed"` with specific evidence target: "Current internal staffing capacity levels and external supplier lead time records." No user clarification was introduced.
**Classification: PASS.** The model correctly identified the disagreement as requiring evidence rather than asking the user to settle an externally knowable question by clarification. It specified concrete, relevant evidence — demonstrating understanding of the causal structure rather than producing a generic classification. This confirms the model can preserve the distinction between "evidence needed to determine what is true" and "clarification needed because only the user can establish meaning/preference/intent/constraint" for this tested case.
This experiment does not prove fidelity for Regression F (user-owned ambiguity), consistency across domains/phrasings, downstream reasoning preservation, or end-to-end production flow. Branch: `feature/reasoning-fidelity-v0.8`. Files: `tests/reconstruction/semantic-regression-e-evidence-vs-clarification.test.js` and `docs/experiment-56k.md`. Status pending Rob's review.
### Experiment 56L Summary — User-owned ambiguity requires clarification, not evidence
Tested whether the configured live Ollama model (`qwen-claude:latest` at `http://192.168.1.111:11434`) recognises that a preference-vs-constraint distinction belongs to the user's own meaning and requires clarification rather than external evidence — Regression F from `docs/reasoning-refinement-requirements.md`.
Fixed case: "I want the business to grow, but I don't want to take on more risk." — user has not specified whether avoiding additional risk is a hard constraint or a strong preference/trade-off.
One live Ollama call (14,032 ms) returned `uncertaintyType: "user_clarification_needed"` with `evidenceNeeded: null` and specific `userClarificationNeeded` describing the non-negotiable-versus-trade-off distinction only the user can establish. Matches pre-written human reference exactly at category level.
**Classification: PASS.** The model correctly identified the ambiguity as user-owned, did not introduce spurious evidence gathering, and preserved the evidence-vs-user-meaning distinction cleanly.
This experiment does not prove consistency across repeated runs, fidelity for other regression cases (AE, G+), behavior in production reasoning paths, or downstream integration with Behaviour Selection or the SituationGraph. Branch: `feature/reasoning-fidelity-v0.8`. Files: `tests/reconstruction/semantic-regression-f-user-owned-ambiguity.test.js` and `docs/experiment-56l.md`. Status pending Rob's review.
### Experiment 56M Summary — Production evidence vs clarification routing validation
Validated one production claim after Codex commit `f861e2c`: does the deterministic question-formulation boundary preserve the E/F distinction? No live Ollama calls were made (0). Deterministic `formulateQuestion()` was exercised with both regression fixtures. Regression E (competing delivery-delay causes: "Staff capacity may be the issue" / "Supplier lead times are likely responsible.") produced question: "What evidence would clarify possible causes of the delivery delay?" — reasoning pattern=diagnosis, strategy=evidence_gathering, template=diagnosis_evidence. PASS. Regression F (preference vs constraint ambiguity: "Whether avoiding additional risk is a hard constraint") produced question: "Is avoiding additional risk a hard constraint or a preference/trade-off?" — reasoning pattern=prioritisation, strategy=null, template=user_meaning_clarification, with rejected families correctly excluding all evidence-adjacent families. PASS. Both cases maintain their distinct routes: E on an evidence route and F on user clarification. All 19 existing tests continue to pass. Branch: `feature/reasoning-fidelity-v0.8`. File: `docs/experiment-56m.md`. Status pending Rob's review.
### Experiment 57J.26 — Post-Admission Investigation Progress
Tested whether the engine makes genuine investigative progress after admitting two user-supported unknowns (Answer 1: savings realism + retention evidence). **Classification: D — NEW VALIDATION / REASONING FAILURE.** Both runs failed at Update 1's `proposal_compatibility` stage. Run 1 (57J.26 scenario): `"New unknown must be explicitly related to an answer-derived node"` — the old provenance-link gate returned, contradicting 57J.25's confirmed pass. Run 2 (exact 57J.25 scenario): `"Proposal cannot resolve beyond an unclassified answer by introducing unsupported stronger meaning"` — a different semantic compatibility error also blocks Update 2. The v0.15 unknown admission fix from 57J.25 cannot be reproduced under the same commit (`fbbd271`). Run-to-run variance in start graph construction (6→7→9 nodes) also observed. Status: unproven — needs regression investigation. No production code changed.
---
### Reasoning Fidelity v0.8 — First Pass Closeout
**The first-pass reasoning-fidelity refinement is complete to its agreed scope.**
Regression boundaries AF have been investigated and the production defects identified from those boundaries have been addressed:
- **A — weak priority:** supported against unsupported strengthening via pre-mutation compatibility guard;
- **B — conditional trade-off:** qualification preserved through deterministic derivation and normalisation;
- **C — unresolved uncertainty:** may remain unresolved when the user supplies no position;
- **D — explicit hard constraint:** explicit meaning preserved;
- **E — evidence-resolvable disagreement:** routed to evidence gathering;
- **F — user-owned ambiguity:** routed to clarification.
No demonstrated production defect remains inside the AF first-pass boundary. Deterministic production validation is passing (commit `ec398dc` validating evidence vs. clarification routing).
**Current HEAD:** `ec398dc` — experiment: validate evidence versus clarification routing
**Key commits:** `f861e2c` (preserve evidence vs. clarification distinction), `ec398dc` (validate evidence vs. clarification routing)
The two important production capabilities now present are:
1. User-supported meaning cannot silently outrun the raw answer at the mutation boundary;
2. Evidence-resolvable uncertainty and user-owned ambiguity are routed differently at question formulation.
**Next work should begin from a newly observed product or reasoning failure rather than automatically extending this regression programme.** These open questions remain for future evidence-driven investigation, not as current defects:
- broader wording/domain/model robustness;
- clarification-target precision outside the tested cases;
- durable per-node provenance of user-supported meaning vs inference;
- whether rejected proposals should eventually be adapted rather than simply blocked;
- end-to-end interaction behaviour across graph update, question choice, Behaviour Selection and UI;
- multilingual robustness;
- any future defect exposed by real use.
### Experiment 57B — Grounded Unclassified Affirmative Answer (Live)
Validated one live semantic probe: the configured Qwen model (`qwen-claude:latest` at `http://192.168.1.111:11434`) produced grounded user-supported meaning for a legitimate affirmative answer that falls into `other` (unclassified) — exactly the case blocked by v0.8's guard. **Result: PASS.** `userSupportedMeaning`: "The user directly states that cost reduction, specifically approximately £2M in annual office-overhead savings, is a primary stated reason for considering the relocation." No unsupported strengthening beyond the raw answer. One live call (20,351 ms). This confirms the live semantic premise behind Codex `4e4d0fa` (semantic token overlap grounding). **What remains unproven:** token-overlap adequacy across diverse answers, stability across runs, production guard integration. Full results in `docs/experiment-57b.md`. Status pending Rob's review.
### Experiment 57A — Contaminated After First Valid Observation
The first post-v0.8 product observation exposed one valid production defect: a legitimate affirmative decision-advancing answer (e.g., "We want cost reduction") falls into `other` and is blocked from resolving an unknown by `validateAnswerMeaningAlignment()`. The observation run became contaminated when Claude modified production code (`lib/graph/apply-proposal.js`, `lib/graph/schema.js`) adding four new categories and keyword detectors. Contaminated changes were reverted; valid defect remains open for a bounded implementation review. Repository production state is back at the merged v0.8 baseline (commit `14d68f1`). Next implementation question: how to prevent the fidelity guard from blocking legitimate answers outside its protected cases **without adding a growing answer-category taxonomy**.
### Experiment 57F — Decomposition Relevance Fix Live Validation (BLOCKED)
Tested whether v0.11 (`7e4c506` — prevent unsupported comparison decomposition) works on the live relocation scenario from Experiments 57E/57B. **Classification: BLOCKED.** Start returned HTTP 200 with a valid graph. Update 1 failed at `proposal_compatibility` stage with HTTP 422: "Proposal cannot resolve beyond an unclassified answer by introducing an unsupported constraint or preference/trade-off distinction." The semantic grounding gate (`validateAnswerMeaningAlignment()`) blocked cost-reduction answers classified as class "other" before any graph update could occur. No decomposition ran — prohibited children ("two observations", "measured", "different timing") cannot be confirmed absent. **New defect exposed:** the semantic grounding gate rejects legitimate cost-reduction/savings answers that fall into unclassified ("other") rather than protected categories, preventing any Update 1 evaluation of the v0.11 fix. This is a separate blocking issue from decomposition relevance. What remains unproven: whether v0.11 works when Update 1 does succeed (answers in supported classes), and whether it correctly allows appropriate comparison/measurement decomposition for genuine parents. Branch: `feature/decomposition-relevance-v0.11`. Status blocked by semantic grounding gate.
### Experiment 57G — Semantic Compatibility Live Validation
Tested whether the cost-reduction answer (unclassified "other") now passes proposal compatibility through v0.12 (`69efc5d`). **Classification: PASS.** Both invocations returned HTTP 200 at `update_applied` — no more `proposal_compatibility` rejection. The selected next question ("What changed during that period...") is grounded in the relocation scenario (reasoning pattern: explanation, strategy: evidence_gathering). No prohibited decomposition children appeared. The v0.11 decomposition fix held on this successful update. **Cold-start observation:** second invocation started with 5 nodes instead of expected 8+ — inconsistent initial graph construction worth investigating separately. Branch: `feature/semantic-compatibility-v0.12`. Status: PASS — semantic compatibility blocker removed, decomposition regression absent. Full results in `docs/experiment-57g.md`.
### Experiment 57I — No-Structure Relationship Fallback Live Validation
**Objective:** Validate that commit `4c5666d` (reasoning: suppress explanation question without relationship structure) no longer creates the `Explanation for why...` parent when no meaningful relationship structure exists. **Classification: PASS.** Fixed scenario: "Should I relocate my engineering team from London to Manchester?" with answer "We're looking at this mainly for cost reduction — roughly £2M annual savings on office overhead." Update 1 returned HTTP 200 at stage `update_applied`. Nodes containing "Explanation for why": None. Nodes containing "why these observations differ": None. The reasoning pattern correctly shifted from "explanation" to "decision". The selected next question ("What would clarify team size, seniority levels, and willingness to relocate in this situation?") is grounded in a genuine unresolved relocation issue. No production code changed. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Branch: `feature/relationship-fallback-v0.13`. Status: PASS.
### Experiment 57J.2 — Minimal Clarification Answerability Diagnostics
Tested `{"scenario":"test"}` for exact graph node text and answerability diagnostics. **Classification: B — inconsistent diagnostics.** Single start produced `centralStatement="test"` with one unknown (`nlgonjv`): "The actual scenario, problem description, or data set intended for analysis." The API reported `prerequisiteConceptCount=3`, but only 1 of 7 prerequisite regex signals actually matched (rule 1: `\bproblem\b`). The count of 3 comes from conjunction-based amplification: two instances of "or" in the label+description → `conjunctionCount=2` → formula `Math.max(regex_matches=1, unresolved_deps=0, conjunctions+1=3)` = 3. The diagnostic name is misleading — it reports a maximum across three different amplification strategies (prerequisite regex, unresolved dependencies, conjunctions), not just prerequisite concept signals. No graph-backed question produced. One live Ollama call at qwen-claude:latest on http://192.168.1.111:11434 (27,109 ms). Full results in `docs/experiment-57j2.md`.
### Experiment 57J.8 — Rejected Answerability Corroboration Candidate
Candidate commit `60048a5` (reasoning: require corroboration for conjunction compoundness) was reviewed and rejected. It crossed its explicit implementation stop condition: while it fixed the false-positive `"scenario, problem, or data set"` minimal clarification case by making it independently answerable, it also broke a genuinely compound fresh-evidence question (e.g., "What evidence supports the savings estimate **and** what evidence supports the retention assumption?") into independently answerable — because no graph structure existed yet. The candidate violated the rule that Case 1 and Case 2 must both be preserved with existing signals. It has been reverted to production baseline. Production reasoning is restored to pre-v0.14 answerability behaviour. The unresolved design question: surface conjunctions alone are an unreliable proxy for semantic compoundness, but the current graph also lacks sufficient structured information to distinguish every fresh single-concept alternative from every genuinely compound fresh unknown. **Do not resume by reintroducing conjunction corroboration or adding more English regex rules.** Full record in `docs/experiment-57j6.md`.
### Experiment 57J.11 — Live Unknown Dimensionality Representation
Tested whether a single answer containing two independent evidence dimensions is represented as two separate unknown nodes or collapsed into one compound unknown. **Classification: A — SEPARATE.** Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers." One live start + one live update. Update was rejected at `proposal_compatibility` (structural linkage error), but the rejection errors directly revealed two independently named proposed unknown nodes: `n-savings-realism` (Target A) and `n-retention-impact` (Target B). The model did not collapse them into a single compound node. Semantic separability existed in the model proposal before deterministic answerability/decomposition. **What remains unproven:** separation for implicit conjunctions, across models/repeated runs, and whether both nodes survive structural gating (`proposal_compatibility`) or question-selection after graph mutation. Full record in `docs/experiment-57j11.md`. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed.
### Experiment 57J.25 — Live User-Supported Unknown Admission Validation
**Objective:** Validate that the v0.15 candidate admits two user-supported unknowns from the 57J.11 case through the live production `updateCase()` path without requiring fake provenance edges.
**Pre-written expectation:** The answer explicitly introduces two independent uncertainties: savings realism and retention impact. If v0.15 works on the live production path, those user-supported unknowns should no longer be rejected solely because they lack an answer-derived provenance edge. No fake edge should be required or manufactured. A later failure at a different validation/reasoning boundary is acceptable evidence and must be recorded as the first new failure.
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Fixed answer:** "Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers."
**Live-call count:** start: 1, update: 1, total: 2
**START:** HTTP 200 | stage: unknown | nodes: 6 | edges: 3 | question: "What would clarify detailed breakdown of current engineering operating costs in this situation?"
**UPDATE 1:** HTTP 200 | stage: update_applied | nodes: 8 (+2) | edges: 5 (+2) | no errors/validation failures. Both proposed unknowns admitted: `n-oss-realistic` (savings realism) and `n-kr-loss` (retention impact), each with meaningful descriptions grounded in the answer semantics. Selected next question targets `n-oss-realistic` via evidence_gathering/diagnosis strategy.
**57J.11 provenance-link rejection:** ABSENT — no proposal_compatibility rejection occurred. Both target dimensions survived.
**Savings target:** PRESERVED
**Retention target:** PRESERVED
**Fake provenance edge:** NO
**Classification: A — PASS.** The v0.15 update path admits both user-supported evidence dimensions through the production path without rejection at the old 57J.11 provenance-link gate. No later failure occurred within this single update.
**What remains unproven:** Repeated-run stability; cross-domain generalisation; downstream investigation viability past Update 2+; implicit conjunction admission; whether `too_broad` conversation health eventually blocks later turns.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Branch: `feature/user-supported-unknown-admission-v0.15`.
### Experiment 57J.28 — Live Node-Support Semantic Inputs Capture
**Classification: C — GATE BEHAVES AS EXPECTED.** On a fresh live run, both user-supported unknowns (savings realism + retention impact) were admitted with HTTP 200 at `update_applied`, zero validation errors. The answerMeaning correctly captured both dimensions. The semantic gate (`rawAnswerSupportsUnclassifiedMeaning`) passed for both via the token-count clause (5 shared content tokens >= 3), not overlap ratio. Structural fallback also valid for both nodes. Static helper evaluation agrees with actual gate behavior — live admission outcome fully explained by captured inputs. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Start: 7 nodes, Update 1: 9 nodes (+2). What remains unproven: token-count adequacy for paraphrased/low-overlap answers, downstream investigation viability past Update 2+, run-to-run start graph stability. No production code changed.
### Experiment 57J.29 — Live Semantic Representation Stability (Repeated Identical Runs)
**Classification: D — DOWNSTREAM INSTABILITY SUSPECTED.** Three repeated identical live runs with the fixed scenario ("We are considering relocating the engineering team to reduce operating costs.") and fixed answer ("Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers.") through the production `startCase()``updateCase()` path. **key finding:** `userSupportedMeaning` was null/empty in ALL 3 trials — no semantic meaning was extracted by the model. Yet admission outcomes diverged: Trial 2 (start with 7 nodes) admitted both unknowns; Trials 1 & 3 (start with 6 nodes) rejected at `proposal_compatibility` with identical "stronger reasoning category" errors despite null diagnostics. This confirms that start graph quality (6 vs 7 nodes cold-start variance) directly affects admission outcomes, and when `userSupportedMeaning` is empty the gate may still process hidden semantic fields. The admission variance cannot be explained by upstream model representation because no meaningful semantic content was produced in any trial — instability is downstream of model representation. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 6 live calls total. No production code changed.
### Experiment 57J.31 — Rejected Proposal Diagnostics Integration
**Objective:** Address the blocking diagnostic visibility gap from Experiment 57J.30 — when `applyValidatedProposal` fails at `proposal_compatibility`, no pre-validation proposal fields are exposed to developers, making causal attribution of rejection outcomes impossible. **Classification: E — FIX VALIDATED.** Added a compact `rejectedProposalSnapshot` to the orchestrator's diagnostics object in the `!applicationResult.success` path (lines ~690725 of `lib/graph/orchestrator.js`). The snapshot is stage-gated (only for `proposal_compatibility`), contains only key proposal fields (`answerMeaning.userSupportedMeaning`, `answerMeaning.possibleInference`, `updatedNodes[].{nodeId,newValue}`, `resolvedUnknownNodeIds[]`, `addedNodes[].{id,kind,label,description,parentId,dependsOn,affects,childIds}`, `addedEdges[].{fromNodeId,toNodeId,relationship}`), and excludes raw model response/prompt content. No validation or mutation behavior changed — only the diagnostic surface is expanded. **Tested:** 7 new snapshot tests (all pass), 2 apply-proposal regression tests (both pass), 13 route-layer tests (all pass) = 84 passed, 0 failed by this change. Pre-existing orchestrator failure confirmed independent of this change. What remains unproven: whether the actual rejected trial from 57J.30 contained stronger `userSupportedMeaning` language vs. different structural elements — requires re-running Experiment 57J.30 now that the snapshot is visible in the HTTP response body. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 0 live calls needed (deterministic instrumentation). Production code changed: `lib/graph/orchestrator.js` only.
### Experiment 57J.32 — Inspect Rejected Proposal Live Variance (EXPERIMENT INTERRUPTED)
**Protocol breach: YES.** The execution harness drifted beyond the agreed budget. Only valid bounded evidence from the first 3 trials is retained in `docs/experiment-57j32.md`. All post-trial activity is excluded from conclusions. **Classification: F — MULTIPLE DIFFERENCES.** Using `rejectedProposalSnapshot` directly, accepted vs rejected proposals differ in both answerMeaning framing (prescriptive "decision is conditional on" vs neutral "requires evidence") AND structural content (accepted = 2 addedNodes + 2 addedEdges; rejected = 0 addedNodes + 0 addedEdges). Neither single cause can be independently isolated. The rejected trial's userSupportedMeaning did not faithfully preserve the raw answer: "The decision is conditional on" prescribes what the _decision_ requires rather than reporting the user's information need ("Before deciding, I need..."). **rejectedProposalSnapshot worked as designed** (57J.31 verified). Status: interrupted mid-execution; evidence retained in `docs/experiment-57j32.md`. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Production code unchanged.
### Experiment 57J.30 — Proposal-Boundary Live Variance
**Classification: I — INSUFFICIENT VISIBILITY.** Three repeated identical live runs with the fixed scenario ("We are considering relocating the engineering team to reduce operating costs.") and fixed answer through the production `startCase()``updateCase()` path. Mixed outcomes: Trial 1 (start=6 nodes) ACCEPTED, Trial 2 (start=8 nodes) REJECTED at `proposal_compatibility` with "answerMeaning.userSupportedMeaning introduces a stronger reasoning category", Trial 3 (start=5 nodes) ACCEPTED. **Cold-start instability confirmed at scale:** node count ranged from 5 to 8 across three identical inputs (60% variance). Accepted trials are structurally consistent: both produce exactly 2 unknown nodes (savings realism + engineer retention) with depends_on edges to state anchors. **Blocking gap:** the API does not surface parsed proposal fields (answerMeaning, addedNodes, etc.) in rejection responses — only error strings. Without pre-validation proposal visibility, causal attribution of the accepted-vs-rejected divergence is impossible: we cannot confirm whether the rejected trial's `userSupportedMeaning` contained stronger category language or whether a different structural element caused the rejection. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 6 live calls total. No production code changed.
### Experiment 57J.33 — Classify Captured Answer-Meaning Strengthening (Deterministic)
**Objective:** Determine whether the validator correctly rejected the captured Trial 2 `userSupportedMeaning` ("The decision is conditional on evidence...") as stronger than the raw answer ("Before deciding, I need evidence..."). Fully deterministic — zero Ollama calls.
**Method:** Replicated production `deriveAnswerMeaningProfile()` and `validateAnswerMeaningCompatibilityWithRawAnswer()` logic deterministically against all four captured strings (raw answer, rejected Trial 2, accepted comparisons A and B). Also ran existing apply-proposal tests (64 pass) and rejected-proposal-snapshot tests (7 pass).
**Key finding:** The rejected Trial 2 string contains the word "conditional" which triggers `hasConditionalQualification()` at line 2775 of `lib/graph/apply-proposal.js`, pushing it into the `conditional_tradeoff` category. Both accepted comparisons lack trigger words and correctly remain classified as `other`. The raw answer also profiles as `other`.
**Rejection mechanism:** `validateAnswerMeaningCompatibilityWithRawAnswer()` line 29822986: when raw answer = "other" and supported meaning ≠ "other", the validator adds the "stronger reasoning category" error. This is exactly what occurs with Trial 2 (other → conditional_tradeoff).
**Classification: A — VALIDATOR CORRECT.** The captured rejected meaning introduces `conditional_tradeoff` where the raw answer only establishes `other`. This is a real strengthening: "The decision is conditional on..." prescribes a requirement on the decision itself rather than reporting a personal information need. Both accepted comparison variants correctly remain in `other`.
**What this establishes:** The validator's rejection was correct for the captured pair. The mechanism is purely the keyword detector firing on "conditional" — not cold-start variance or structural elements.
**What it does NOT establish:** Whether "conditional" is the ideal trigger word across all contexts, whether "Before deciding" should itself have triggered conditional semantics, generalisation to other answers/domains, or whether cold-start node variance (57J.32) separately affects proposal quality.
Configured Ollama: none used. Production code changed: NO. Tests permanently changed: NO. Temporary test used and removed: YES.
---
### Experiment 57J.35 — No-Retry Live Experiment Harness Enforcement
**Objective:** Make the canonical live harness physically incapable of hidden retries. Enforce one-shot execution semantics: every requested Start maps to exactly one `/api/cases/start` call, every requested Update maps to exactly one `/api/cases/update` call, and rejections are returned immediately without implicit retry.
**Protocol breach prevention:** This change directly addresses the protocol breach from Experiment 57J.32 where an implicit retry loop consumed multiple Update calls per trial, contaminating evidence. Future prompts may rely on the canonical harness to enforce one-call/no-retry semantics; Claude must not create supplementary retry scripts during bounded experiments.
**Approach:** Bounded execution configuration (`maxUpdates`) + explicit call accounting (startCalls/updateCalls/totalCalls counters reflecting actual API invocations) + rejection-immediate-stop semantics + rejectedProposalSnapshot preservation for v0.16 diagnostic visibility.
**Changes to canonical harness (`scripts/reproduce-multi-turn-investigation.mjs`):**
- Hardcoded `Start → Update 1 → Update 2` sequence replaced with configurable bounded loop (`config.maxUpdates`)
- Call accounting added: `calls.startCalls`, `calls.updateCalls`, reported as `totalCalls`
- Rejection returns immediately; no retry path exists for any semantic outcome (proposal_compatibility, validation failure, etc.)
- `rejectedProposalSnapshot` preserved and logged when present in Update rejection diagnostics
- Every update call is explicit in the loop; `config.answers[i]` maps positionally to `Update i+1`
**Tests added (`tests/reproduce-multi-turn-investigation.harness.test.js`):** 8 deterministic cases via synchronous simulation mirror of harness logic — all pass (0 Ollama calls, no dev-server needed). Test cases:
1. Start success → exactly 1 Start call.
2. Start failure → exactly 1 Start call, no retry.
3. Update success → exactly 1 Update call.
4. `proposal_compatibility` rejection → exactly 1 Update call, rejection returned unchanged.
5. Update 1 rejection → Update 2 never called.
6. Update 1 success → Update 2 called exactly once when explicitly requested.
7. Call counters equal actual mocked API invocations.
8. No semantic retry after HTTP 422/valid rejection response.
**What this tooling change guarantees:** Future live experiment runs via the canonical harness are physically incapable of consuming more API calls than explicitly configured. Each Start request = exactly one call; each Update request = exactly one call; rejections stop the chain immediately without retry. Call accounting always reflects actual HTTP invocations, not inferred successes.
**What this does NOT guarantee:** That production reasoning is correct (no production code changed). That cold-start variance in node counts is resolved (start graph stability remains an open issue). That semantic validation outcomes change (only the harness wrapper changed). That transport-level failures are handled (not addressed by this tooling change).
**Configured Ollama:** none used. **Production code changed:** NO. **Tests run:** 8 passed, 0 failed.
---
### Experiment 57J.34 — Multi-Turn Investigation Progress After Accepted Update 1
**Objective:** On one fresh live run, if the first relocation answer passes the current reasoning safeguards, does answering the savings-realism question produce genuine investigation progress rather than repetition or irrelevant reasoning?
**Classification: D — VALIDATION FAILURE.** Two distinct outcomes from cold-start variance:
- **Run A (harness):** Update 1 rejected at `proposal_compatibility` with "answerMeaning introduces a stronger reasoning category" — same conditional-strengthening defect as 57J.33 R1. This is a correct fidelity guard.
- **Run B (pipeline capture, fresh case):** Update 1 applied successfully but produced **zero new unknown nodes** despite the user answer explicitly introducing two independent evidence dimensions (savings realism + retention impact). A single compressed merged unknown appeared ("specific criteria, budget constraints, talent retention implications, or timeline"). Edges decreased from 3→2. When Update 2 was then attempted with concrete savings evidence (£2M), it failed at `proposal_compatibility` by the same structural provenance-link gate: "New unknown must be explicitly related to an answer-derived node."
**Key finding:** Even when Update 1 passes semantic validation, the engine does NOT produce meaningful investigation progress. The savings-realism question from Answer 2 could not resolve because no dedicated savings realism unknown existed. This means the experiment's core question is answered negatively: accepting a "faithful" proposal does not guarantee downstream progress.
**What this establishes:** (1) Cold-start variance (5→8 nodes) directly affects whether Update 1's semantic fidelity guard fires; (2) Semantic acceptance ≠ structural validity — zero-node proposals can pass validation; (3) The provenance-link gate remains active in v0.16 and blocks legitimate new unknown creation; (4) The merged generic unknown instead of two distinct targets is a consistent pattern.
**What this does NOT prove:** That all cold-starts produce 5 nodes; that zero-node proposals are always produced on accepted updates; whether the merged-unknown behavior is intentional or a defect.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 4 live calls total (2 starts + 2 updates in final pipeline; 1 start + 1 update rejected from harness). No production code changed.
### Experiment 57J.36 — Multi-Turn Investigation Progress After Accepted Update 1 (Clean Run)
**Objective:** One clean guarded multi-turn case using the hardened canonical harness (bounded execution, no-retry) to determine whether a semantically faithful first answer produces genuine downstream progress on the second turn.
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Answer 1:** "Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers."
**Answer 2 (not reached):** "The projected savings are based on the current London lease, business rates, service charges, utilities and facilities costs that would no longer be incurred at the same level after the move. The estimate is approximately £2M per year."
**Live-call count:** start: 1, update: 1, total: 2
**START:** HTTP 200 | stage: unknown | nodes: 10 | edges: 5 | question: "What would clarify total projected costs at the new location, including one-time relocation expenses and long-term savings in this situation?"
**UPDATE 1:** HTTP 422 | stage: proposal_compatibility | error: "Update contains no meaningful change"
Rejected proposal snapshot: `answerMeaning.userSupportedMeaning` preserved both evidence dimensions faithfully; all structural fields empty (zero addedNodes, zero addedEdges, zero resolvedUnknownNodeIds, zero updatedNodes).
**Classification: B — DIFFERENT REJECTION.** Rejected for "Update contains no meaningful change" at `proposal_compatibility`, not for semantic strengthening. The LLM produced a structurally null proposal despite semantically faithful meaning extraction. U1-B (not U1-A because no strengthening present; not U1-C/U1-D because nothing was applied).
**Savings realism:** UNCLEAR
**Retention impact:** UNCLEAR
**What this establishes:** (1) When the LLM produces a structurally empty proposal with semantically faithful meaning, `proposal_compatibility` correctly rejects it as a no-op; (2) The gap between semantic extraction and structural creation exists independently of the semantic fidelity gate; (3) Cold-start produced 10 nodes (continuing the variance from 5→8 in 57J.34).
**What this does NOT prove:** That the LLM can produce a structurally non-empty faithful proposal; that downstream progress would occur with an accepted proposal; run-to-run stability of node counts or proposal structure.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 2 live calls total. No production code changed.
### Experiment 57J.37 — Rejected Proposal Diagnostics: Semantic-to-Mutation Contract Gap (Read-Only Diagnosis)
**Objective:** Read-only analysis of whether the graph-update prompt/validator contract requires structural representation of newly introduced unresolved uncertainty, or whether an empty mutation with populated `answerMeaning` is permitted by the model contract and merely rejected later as a no-op.
**Method:** Analyzed prompt instructions (`lib/graph/prompt-builder.js`), schema defaults (`lib/graph/schema.js`), validator logic (`lib/graph/utils.js` line 868885), application pipeline (`lib/graph/apply-proposal.js` line 3174, 32523270), and existing test coverage. No Ollama calls. No live API.
**Findings:**
- **Prompt contract is AMBIGUOUS:** Rule #6 requires inspecting for new uncertainty but rule #7 ("Add new unknown nodes only when...") is a restriction, not a requirement. Additional Guidance explicitly permits semantic-only proposals via `answerMeaning`.
- **Schema contract PERMITS the combination:** `graphUpdateSchema` allows populated `answerMeaning` + zero structural mutation (all array fields default to `[]`). No cross-field constraint exists.
- **Validator contract REJECTS it:** `hasMeaningfulChange` checks only structural fields (addedNodes, updatedNodes status/value changes, addedEdges, removedEdgeIds). `answerMeaning` is not considered meaningful change.
- **Test coverage NOT COVERED:** No test for "grounded answerMeaning introduces new unresolved uncertainty + zero structural changes." The closest tests verify schema validity of `{}` and validator rejection of all-empty arrays, but neither tests the populated `answerMeaning` case.
**Classification: E — MIXED.** Three independent contract boundaries contribute: (1) prompt ambiguity between inspection and materialization; (2) schema permissiveness vs validator rejection mismatch; (3) model receives permissive guidance that leads to a rejected downstream gate.
**Who owns the failure:** MIXED — Prompt Contract (ambiguity) + Validator Contract (schema/validator mismatch). Model does NOT own this failure.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. Production code changed: NO. Prompt changed: NO. Tests changed: NO. Dev server disturbed: NO. Ollama calls: 0.
### Experiment 57J.38 — Semantic-to-Mutation Contract Fix Selection
**Objective:** Choose the smallest safe contract change preventing faithful semantic-only no-op proposals (populated `answerMeaning` with consequential uncertainty + zero structural mutation).
**Three options evaluated:**
- **A (prompt-only):** Single MUST rule in Additional Guidance clarifying structural mutation requirement. Fixes ambiguity but relies entirely on model compliance.
- **B (prompt + validator):** Same prompt rule PLUS a deterministic validator check targeting `userSupportedMeaning` text + empty structural fields. Produces specific actionable error.
- **C (schema cross-field constraint):** Zod `.refine()` rejecting any populated `answerMeaning` object with zero structural mutation.
**Key finding on Option C:** Breaks Case 5 (possibleInference only). Schema cannot distinguish "new consequential meaning" from "inference-only" without semantic analysis. Any populated answerMeaning object triggers rejection regardless of content type.
**Controlled case results summary:**
- Case 1 (genuinely new uncertainty): All options reject as expected. B has best diagnostic visibility.
- Case 2 (already represented): All options correctly reject.
- Case 3 (update/resolve existing): All options correctly allow structural update to existing node.
- Case 4 (answerMeaning null): All options preserve existing behavior.
- Case 5 (possibleInference only): A✓ / B✓ / C✗ (breaks — schema sees populated object, cannot distinguish inference from meaning).
**Recommended option: B — PROMPT + VALIDATOR CONTRACT**
Why: Fixes 57J.36 completely (prompt ambiguity + enforcement gap). No new semantic classifier needed. Preserves provider-agnostic design. Does not break valid cases (null answerMeaning, possibleInference-only). Specific error message provides actionable diagnostic where option A relies entirely on model compliance (which the evidence from 57J.36 shows is unreliable for this pattern).
Configured Ollama: none used. Production code changed: NO. Prompt changed: NO. Tests changed: NO. Dev server disturbed: NO. Ollama calls: 0.
**READY FOR BOUNDED IMPLEMENTATION: YES**
Exact implementation boundary:
1. One MUST rule in prompt Additional Guidance (replaces line 132 of prompt-builder.js)
2. One deterministic check in `validateGraphUpdate()` after `hasMeaningfulChange` (utils.js)
3. Six regression tests: populated-meaning-zero-mutation rejection, prompt text verification, null-answerMeaning preserved, possibleInference-only not forced, update-existing-node valid, resolve-path valid
---
### Experiment 57J.39 — Semantic-to-Mutation Contract Implementation (Option B)
**Classification: E — IMPLEMENTATION COMPLETE.**
Implemented the bounded Option B from 57J.38 with ownership correction: prompt owns structural materialization obligation; validator owns only the structural fact that `answerMeaning` alone is not graph progress.
**Production changes:**
- **prompt-builder.js** rule #6 replaced: added explicit MUST structural-materialization rule. userSupportedMeaning must be expressed through graph structure — update/refine existing, resolve unknown, new unknown, or justified relationship. answerMeaning alone is insufficient.
- **utils.js** validateGraphUpdate(): added specific diagnostic "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation" inside the `!hasMeaningfulChange` path, ordered before generic no-op error for exact same proposal.
**Tests:** 15 new focused tests (8 in utils.test.js for all contract cases; 7 in prompt-builder.test.js for MUST rule verification). All 68 + 15 = 83 focused tests pass. No regressions in update-route (13), harness (8), or rejected-proposal-snapshot (7) tests. One pre-existing orchestrator failure unrelated to this work.
**Not changed:** hasMeaningfulChange definition, schema, node/edge semantics, provenance, answerability, decomposition, reasoning taxonomy, semantic classifiers, provider integration, Behaviour Selection. possibleInference alone does not trigger the new diagnostic.
Ollama calls: 0. Dev server disturbed: NO. Branch: feature/semantic-to-mutation-contract-v0.17 from HEAD 3b868b2.
---
### Experiment 57J.40 — Semantic-to-Mutation Contract Live Validation
**Objective:** On one fresh live run, does the v0.17 prompt contract cause a faithful `userSupportedMeaning` to produce meaningful structural graph mutation instead of a semantic-only no-op proposal?
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Fixed answer:** "Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers."
**Live-call count:** start: 1, update: 1, total: 2
**START:** HTTP 200 | stage: unknown | nodes: 8 | edges: 5 | question: "What was the comparable state before current baseline costs vs. projected costs at target location?"
**UPDATE 1:** HTTP 422 | stage: proposal_compatibility | error: "answerMeaning.userSupportedMeaning introduces a stronger reasoning category than the raw answer establishes."
Rejected proposal snapshot: `userSupportedMeaning`: "Proceeding with the relocation decision is explicitly conditional on obtaining verified evidence that projected office savings are realistic and that key engineer retention is preserved." — this converts the user's information-need statement ("I need evidence that...") into prescriptive conditional language about what the _decision_ requires, introducing a `conditional_qualification` meaning stronger than the raw answer. `possibleInference`: null. Structural proposal: 1 addedNode (`n_prereq_constraint`, kind=assumption) + 1 addedEdge (depends_on to state node).
**Meaning fidelity classification: STRENGTHENED.** The model transformed a neutral information-need ("Before deciding, I need evidence...") into prescriptive constraint language ("Proceeding...is explicitly conditional on obtaining verified evidence"). This introduces a stronger reasoning category not supported by the raw answer.
**Classification: C — CORRECT FIDELITY REJECTION.** The model strengthened the raw answer beyond what the user established, and the existing semantic-fidelity validator correctly rejected it at `proposal_compatibility`. This is not a v0.17 semantic-to-mutation failure — the strengthening was caught before any graph mutation could proceed.
**What this establishes:** (1) The configured model still maps "Before deciding, I need evidence..." to prescriptive conditional framing on this scenario; (2) The existing semantic-fidelity guard correctly catches this strengthening at proposal_compatibility; (3) v0.17's MUST rule did not trigger in this run because the meaning was strengthened before reaching the mutation boundary.
**What it does NOT prove:** That the configured model produces faithful `userSupportedMeaning` on this scenario under repeated runs; that v0.17's contract prevents the faithful semantic-only no-op when meaning is truly preserved; that strengthening avoidance would occur with different phrasing or in later turns.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 2 live calls total. No production code changed. Branch: `feature/semantic-to-mutation-contract-v0.17`.
---
### Experiment 57J.40 — Semantic-to-Mutation Contract Live Validation (Full Record)
**Objective:** Answer exactly: "On one fresh live run, does the v0.17 prompt contract cause a faithful `userSupportedMeaning` to produce meaningful structural graph mutation instead of a semantic-only no-op proposal?"
**Branch:** `feature/semantic-to-mutation-contract-v0.17`
**Starting HEAD:** 712c0c4 docs: experiment 57J.39 record and handoff update
**Pre-written expectation:**
> The raw answer contains two explicit unresolved evidence needs: projected savings realism and key-engineer retention impact. If v0.17 closes the semantic-to-mutation contract gap, a faithful `userSupportedMeaning` should no longer be accompanied by a completely empty structural proposal. The model should either update/refine existing relevant graph structure, resolve relevant structure, or add justified new structure.
> A semantic-strengthening rejection remains a valid protected outcome and does not count as failure of v0.17.
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Fixed answer:** "Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers."
**Configured model:** qwen-claude:latest at http://192.168.1.111:11434
**Dev server:** REUSED EXISTING (HTTP 200)
**CALL ACCOUNTING**
startCalls: 1
updateCalls: 1
totalCalls: 2
Supplementary scripts used: NO
Retries: 0
**START**
Nodes: 8
Edges: 5
Selected question: "What was the comparable state before current baseline costs vs. projected costs at target location?"
**UPDATE 1**
HTTP: 422
Stage: proposal_compatibility
First error: "answerMeaning.userSupportedMeaning introduces a stronger reasoning category than the raw answer establishes."
Nodes: 8 (unchanged)
Edges: 5 (unchanged)
Selected question: null
**ANSWER MEANING**
userSupportedMeaning: "Proceeding with the relocation decision is explicitly conditional on obtaining verified evidence that projected office savings are realistic and that key engineer retention is preserved."
possibleInference: null
**Meaning fidelity:** STRENGTHENED
The model transformed a neutral information-need ("Before deciding, I need evidence...") into prescriptive constraint language ("Proceeding...is explicitly conditional on obtaining verified evidence"). This introduces stronger reasoning category (conditional qualification) beyond what the raw answer supports. The transformation maps "I need evidence that X and Y" to "decision is conditionally dependent on X and Y" — a non-trivial semantic strengthening.
**STRUCTURAL PROPOSAL**
updatedNodes: []
resolvedUnknownNodeIds: []
addedNodes: 1 (`n_prereq_constraint`, kind=assumption, label="Prerequisite condition for proceeding")
addedEdges: 1 (depends_on from added node to state anchor `nqylvkl`)
updatedNodes count: 0
resolvedUnknownNodeIds count: 0
addedNodes count: 1
addedEdges count: 1
**Structural mutation present:** YES
Mutation type: ADD NODE
**Classification:** C — CORRECT FIDELITY REJECTION
**Why:** The model strengthened the raw answer beyond what it supports (information-need → prescriptive constraint), and the existing semantic-fidelity validator correctly rejected this at `proposal_compatibility`. The rejection was triggered by the strengthening detection, not by v0.17's semantic-to-mutation contract. The model did produce 1 added node and 1 edge, but these are downstream artifacts of the strengthened meaning — they do not represent faithful structural translation of the raw answer.
**Did v0.17 remove the faithful semantic-only no-op failure:** UNPROVEN
This run did not test the v0.17 contract's core question because the model never produced a faithful `userSupportedMeaning` to begin with. The strengthening occurred before reaching the mutation boundary, so v0.17's MUST rule was never exercised in this run. A faithful semantic-only no-op (if it exists under v0.17) is neither reproduced nor disproved here.
**What this clean run establishes:**
1. The configured model maps "Before deciding, I need evidence..." to prescriptive conditional framing on this scenario — a repeatable strengthening pattern observed in earlier experiments (57J.32, 57J.33).
2. The existing semantic-fidelity guard catches this class of strengthening at proposal_compatibility.
3. When meaning is strengthened and rejected, the model's structural proposal is built on the strengthened meaning (not a faithful translation), making mutation analysis irrelevant to v0.17.
**What it does NOT prove:**
1. That the configured model produces faithful `userSupportedMeaning` on this scenario under repeated runs.
2. That v0.17's MUST rule prevents the faithful semantic-only no-op when meaning is genuinely preserved.
3. That strengthening avoidance would occur with different phrasing, domain, or model.
4. That v0.17 works in any case where the model does produce faithful meaning.
**Production code changed:** NO
**Prompt changed during experiment:** NO
**Canonical harness restored:** YES
**Hardened no-retry behaviour preserved:** YES
**Dev server disturbed:** NO
**Ollama calls beyond harness count:** 0
**Documentation updated:** YES
---
### Experiment 57J.41 — Semantic-to-Mutation Contract Live Validation: Faithful Meaning Only
**Objective:** When the user introduces one simple, explicit unresolved uncertainty with no conditional/constraint language, does v0.17 translate that faithful meaning into structural graph progress rather than a semantic-only no-op?
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Fixed answer:** "I am unsure whether the projected office savings from the relocation are realistic."
**Live-call count:** start: 1, update: 1, total: 2
**START:** HTTP 200 | stage: unknown | nodes: 6 | edges: 3 | question: "What would clarify current annual operating costs and cost structure of the engineering team in this situation?"
**UPDATE 1:** HTTP 422 | stage: proposal_compatibility | First error: "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress."
**Rejected Proposal Snapshot:**
- `userSupportedMeaning`: "The user is unsure whether the projected office savings from the relocation are realistic." — **FAITHFUL** (preserves uncertainty, no conditional/constraint language)
- `possibleInference`: null
- All structural fields: empty (zero updatedNodes, zero resolvedUnknownNodeIds, zero addedNodes, zero addedEdges)
**Classification: B — SAME SEMANTIC-ONLY NO-OP.** Meaning is FAITHFUL. All structural mutation fields are empty. The v0.17 MUST rule + validator diagnostic fires as designed, converting the original 57J.36 silent accepted no-op into an explicit rejected proposal with specific error. However, this does NOT prove that faithful meaning CAN produce graph progress — only that it cannot do so without structural mutation under v0.17's new contract.
**Key finding:** v0.17 successfully blocks the original failure (accepted semantic-only no-op) but does not establish a positive path for faithful meaning to produce graph progress. The open question remains: is there any valid pathway where faithful meaning translates to structural mutation, or does the MUST rule universally block it?
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed. Harness restored. No-retry preserved. Dev server disturbed: NO.
### Experiment 57J.42 — Structural-Mutation MUST Rule Prompt Conflict Diagnosis
**Objective:** Diagnose why the model produces faithful `userSupportedMeaning` with zero structural mutation despite the v0.17 MUST rule. Read-only prompt-contract analysis of assembled prompt + tests. No Ollama calls.
**Method:** Inspected all prompt rules from `lib/graph/prompt-builder.js`, analyzed action-selection ambiguity, checked six conflict patterns (AF), reviewed test adequacy in `tests/graph/prompt-builder.test.js`.
**Relevant rule conflicts found:**
- Rule #6 (MUST) vs Additional Guidance bullet B (PERMIT empty arrays if "answer does not justify a change"): direct MUST vs PERMIT conflict. Rule #7's restrictive enumeration + rules #4 ("genuinely new concepts") + #11 (no duplicates) help the model decide nothing justifies a change, triggering the escape hatch in Additional Guidance.
- Additional Guidance bullet C: "Use answerMeaning to preserve the answer's direct meaning even when the graph change remains unresolved" explicitly encourages semantic-only output — the exact pattern v0.17 sought to eliminate.
- Action-selection ambiguity under rule #6: four acceptable forms listed without decision criteria or fallback ordering.
- Pattern D fidelity paralysis: high-traceability requirements make structural creation non-trivial for simple uncertainty about known topics.
**Classification: C — PROMPT CONFLICT**
Primary owner of 57J.41 failure: PROMPT CONFLICT (not MODEL)
Provider-agnostic concern: YES — CONTRACT LEVEL
Smallest correction boundary: Two Additional Guidance bullets (~10 words total) at lines ~132 of prompt-builder.js must be removed or conditioned on rule #6 not triggering. Text-preservation-only tests do not prove prompt coherence.
Configured Ollama: none used. Production code changed: NO. Prompt changed: NO. Tests changed: NO. Dev server disturbed: NO.
### Experiment 57J.43 — Remove Surviving Semantic-Only/No-Op Prompt Conflict
**Objective:** Apply the smallest proven correction from 57J.42: replace two Additional Guidance bullets that conflicted with rule #6's MUST rule, plus add one clarifying bullet and seven focused tests. Bounded prompt-contract task only. Not solving update-vs-add action selection.
**Production changes:**
- `lib/graph/prompt-builder.js` — Replaced two conflicting Additional Guidance bullets (line ~131-132):
- Old: "If the answer does not justify a change, return empty arrays for every category." → Now conditioned on rule #6 not applying.
- Old: "Use answerMeaning to preserve the answer's direct meaning even when the graph change remains unresolved." → Replaced with explicit semantic-fidelity vs structural-mutation separation.
- Added: Edge-connection requirement for new unknown nodes (clarifying bullet, consistent with existing rule #13a).
- `tests/graph/prompt-builder.test.js` — Added 7 focused tests covering all seven required coverage areas.
**What this guarantees:**
1. Empty-array permission is explicitly subordinate to rule #6 — MUST vs PERMIT contradiction eliminated.
2. `answerMeaning` can no longer substitute for graph mutation per the corrected bullet text.
3. All existing contracts preserved: duplicate avoidance, genuinely-new-concepts protection, fidelity/inference separation, traceability, update/refine preference.
**What is intentionally left unresolved:**
1. Action selection under rule #6 (update vs add) — confirmed ambiguous by 57J.42.
2. Live production validation — requires next experiment pass.
**Test results:**
- prompt-builder.test.js: 22/22 pass (7 new + 15 pre-existing)
- utils.test.js: 68/68 pass
- apply-proposal.test.js: 64/64 pass
- Total: 154 tests, 0 failures
**Classification: E — IMPLEMENTATION COMPLETE (prompt correction only)**
Configured Ollama: none used. Production code changed: prompt + tests only. Dev server disturbed: NO.
### Experiment 57J.44 — Direct Live Test of Conflict-Free Mutation Prompt
**Objective:** Answer whether the v0.18 corrected Additional Guidance bullets now produce structural graph mutation from faithful uncertainty (one simple unresolved question about savings realism).
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Fixed answer:** "I am unsure whether the projected office savings from the relocation are realistic."
**Live-call count:** start: 1, update: 1, total: 2
**START:** HTTP 200 | stage: unknown | nodes: 6 | edges: 3 | question: "What would clarify current operating costs for the present location versus projected post-relocation costs and one-time relocation expenses in this situation?"
**UPDATE 1:** HTTP 422 | stage: proposal_compatibility | error: "answerMeaning.userSupportedMeaning is populated, but the proposal contains no graph mutation. answerMeaning alone does not constitute graph progress."
- Nodes: 6 | Edges: 3 (unchanged)
**Rejected Proposal Snapshot:** `userSupportedMeaning`: "The user is unsure whether the projected office savings from the relocation are realistic." — FAITHFUL (no strengthening). `possibleInference`: about inflated projections negating benefits. All structural mutation fields empty (zero addedNodes, zero addedEdges, zero resolvedUnknownNodeIds, zero updatedNodes with meaningful change).
**Meaning classification:** FAITHFUL — v0.18 eliminated the conditional-strengthening defect from 57J.40 (v0.17).
**Classification: B — SAME FAITHFUL NO-OP.** The prompt contradiction was removed but faithful uncertainty alone still produced zero structural mutation for this model on this scenario. The action-selection gap under rule #6 (identified by 57J.42 as intentionally left unresolved) remains unaddressed.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No production code changed.
### Experiment 57J.45 — Choose Structural Action-Selection Rule
**Objective:** When rule #6 requires structural progress for a faithful unresolved uncertainty, choose the smallest provider-agnostic instruction that distinguishes update-of-existing from add-new-unknown.
**Pre-written expectation:** The current prompt (v0.18) has the ingredients to distinguish Case A (equivalent unknown exists → update/refine) from Case B (no equivalent → add new unknown), but lacks an explicit fallback sequence. Rule #6's four structural options are listed without decision criteria; rules #4, #7, and #11 collectively push toward "nothing justifies a change." Result: NO — the prompt does not distinguish Case A from Case B.
**Option A — EXISTING-FIRST FALLBACK:**
- Case A correct: YES (explicit first step checks existing unresolved nodes)
- Case B correct: YES ("otherwise" clause creates new unknown)
- Duplicate risk: LOW (validator safety net + deterministic order)
- Related-node overwrite risk: MEDIUM (semantic judgment required but bounded by existing contracts #4/#11)
- No-action risk: LOW (eliminates ambiguity that caused 57J.41/57J.44 failures)
- New semantic classifier: NO | New schema/state: NO | Validator change: NO | Provider-specific: NO
**Option B — NEW-UNKNOWN DEFAULT:**
- Case A correct: NO ("exact duplicate" requires undefined mechanism)
- Case B correct: YES (default-to-add works)
- Duplicate risk: HIGH (no "exact vs merely related" mechanism exists)
- Related-node overwrite risk: LOW (only creates, does not overwrite)
- No-action risk: MEDIUM (less than current but "exact duplicate" is underspecified)
- New semantic classifier: YES (required for "exact duplicate" definition) | New schema/state: NO | Validator change: YES | Provider-specific: NO
**Option C — GENERAL STRUCTURAL CHOICE:**
- Case A correct: PARTIAL (depends on model judgment across 4 unweighted options)
- Case B correct: PARTIAL (model may choose any option; evidence shows no-action preference)
- Duplicate risk: MEDIUM | Overwrite risk: MEDIUM | No-action risk: HIGH (essentially current state — confirmed failure in 57J.41/57J.44)
**Action-space narrowing for controlled case:**
- Edge-only sufficient: NO (edges connect nodes but cannot represent new uncertainty without a node)
- Resolution applicable: NO (user expresses uncertainty, not resolution of existing unknown)
- Relevant actions: update/refine (Case A) or add unknown (Case B)
**Recommended option: A — EXISTING-FIRST FALLBACK**
One explicit instruction-order rule in Additional Guidance: check existing unresolved nodes first; if none represents the same uncertainty, create a new unknown. Deterministic order replaces ambiguous choice-list. No new classifiers, schema state, or validator changes required. Existing rule #11 and deterministic validator remain as safety net.
**Status:** Design complete. READY FOR BOUNDED IMPLEMENTATION: YES. Implementation requires one Additional Guidance bullet in prompt-builder.js plus 8 focused deterministic regressions in tests. Full record in `docs/experiment-57j45.md`.
### Experiment 57J.46 — Existing-First Uncertainty Fallback: Prompt Implementation
**Branch:** `feature/semantic-to-mutation-contract-v0.19`
**Starting HEAD:** `acd1928` (HEAD of v0.18 at branch point)
**What changed:**
- `lib/graph/prompt-builder.js` — Added one Additional Guidance bullet: "When rule #6 applies to explicitly unresolved uncertainty: first check whether an existing unresolved node already represents the same uncertainty; if so, update/refine that existing structure rather than adding a duplicate; if no such node exists, add a new unknown that directly represents the unresolved uncertainty; do not use an edge alone to represent a previously unrepresented uncertainty."
- `tests/graph/prompt-builder.test.js` — Added 14 focused tests verifying: existing-first ordering, reuse path, fallback-to-add, related-node-insufficient, edge-only-prohibited, possibleInference separation, resolution path preserved, duplicate contract preserved, scope uncertainty-only, fidelity/traceability preserved, noop validator untouched, no semantic classifier added.
**Test results:**
- prompt-builder.test.js: 37 tests pass (23 existing + 14 new)
- utils.test.js: 68 tests pass (regression confirmation)
- Total: 105 tests pass, 0 failures
**What this guarantees:** When rule #6 fires for explicitly unresolved uncertainty, the prompt gives a deterministic instruction order: check existing first → reuse if equivalent → otherwise add new. Rule is scoped to unresolved uncertainty only. All existing contracts preserved (duplicate avoidance, fidelity/inference separation, traceability, noop validator, structural-materialization MUST rule).
**What is intentionally left unresolved:** Semantic threshold for "represents the same uncertainty" (relies on model capability + validator safety net). Live production validation (requires next experiment pass). Multi-turn tuning. Edge-connection strategy for Case B new unknowns.
Configured Ollama: none used. Production code changed: prompt + tests only. Dev server disturbed: NO.
### Experiment 57J.47 — Convergence Test: Existing-First Uncertainty Fallback Live
**Objective:** Does v0.19 produce a faithful structural graph action for one explicit unresolved uncertainty instead of another no-op?
**Fixed scenario:** "We are considering relocating the engineering team to reduce operating costs."
**Fixed answer:** "I am unsure whether the projected office savings from the relocation are realistic."
**Live-call count:** start: 1, update: 1, total: 2
**START:** HTTP 200 | stage: unknown | nodes: 7 | edges: 4 | question: "What would clarify current detailed breakdown of engineering operating costs and geographic distribution in this situation?"
**UPDATE 1:** HTTP 422 | stage: proposal_compatibility | error: "answerMeaning.userSupportedMeaning overstates a raw answer that remains uncertain."
rejectedProposalSnapshot:
- userSupportedMeaning: "The user is currently uncertain whether the projected office savings from the relocation are realistic." (HUMAN ASSESSMENT: minor non-material paraphrase — see 57J.48; VALIDATOR: lexically rejected because "uncertain" absent from deriveAnswerMeaningProfile detection patterns)
- possibleInference: null
- addedNodes: [{id: "ns82kx9", kind: unknown, label: "Projected office savings from the relocation lack validation"}] (genuinely new — ADD NEW UNKNOWN structural action)
- addedEdges: [{fromNodeId: "ns82kx9", toNodeId: "nv759rs", relationship: depends_on}]
- updatedNodes: [] | resolvedUnknownNodeIds: []
**Meaning classification:** LEXICAL MISMATCH in deterministic derivation (human assessment: minor non-material paraphrase; validator mechanism: "uncertain" absent from deriveAnswerMeaningProfile patterns while raw answer's "unsure" is detected)
**Structural action:** ADD NEW UNKNOWN — a genuinely new unknown directly representing savings realism was created. This is the FIRST time in experiments 57J.36-47 that the model produced a meaningful structural mutation for this type of uncertainty answer.
**Classification: D — REJECTION BLOCKS TEST (LEXICAL FALSE POSITIVE).** The rejection was caused by `"uncertain"` being absent from `deriveAnswerMeaningProfile`'s uncertainty detection patterns (which include `"unsure"`, `"not sure"`, `"do not know"`, etc.), not by genuine semantic strengthening. Both words express identical uncertainty semantics. Structural progress was achieved (one new unknown + one edge) — the faithful no-op pattern has been broken. The remaining blocker is incomplete lexical coverage in the deterministic category model, not a meaning extraction defect.
**Key finding:** v0.19's existing-first fallback now produces meaningful graph mutation for explicit unresolved uncertainty. New blocking boundary: semantically faithful paraphrases are being rejected because `deriveAnswerMeaningProfile` detects `"unsure"` but not `"uncertain"` (identical semantics). The guard's lexical gap causes false-positive "overstates a raw answer that remains uncertain" errors for semantically equivalent third-person uncertainty statements. Convergence test result is non-A; prompt tuning sequence terminates here per convergence rule. 57J.48 provides the full deterministic diagnosis.
Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 2 live calls total. No production code changed. No prompt change during experiment. Harness restored to original scenario/answers.
### Experiment 57J.48 — Uncertainty Fidelity False Positive Diagnosis
**Objective:** Determine whether the captured rejection of "The user is currently uncertain..." → rejected as overstatement when raw answer was "I am unsure..." is semantically justified or a lexical false positive. Fully deterministic — zero Ollama calls.
**Captured pair:**
- Raw answer: `"I am unsure whether the projected office savings from the relocation are realistic."`
- userSupportedMeaning: `"The user is currently uncertain whether the projected office savings from the relocation are realistic."`
**Deterministic trace:**
1. `deriveAnswerMeaningProfile(raw)` → category=`uncertain` (matches pattern `"unsure"` at line 2880)
2. `deriveAnswerMeaningProfile(userSupportedMeaning)` → category=`other` (pattern gap: `"uncertain"` not detected)
3. `validateAnswerMeaningCompatibilityWithRawAnswer`: line 2966-2970 fires because `raw=uncertain` && `supported≠uncertain` → error: "overstates a raw answer that remains uncertain."
**Inverse test (same semantics, different word):** Using `"unsure"` instead of `"uncertain"` in userSupportedMeaning → category=`uncertain`, errors=none. Both words express identical uncertainty.
**Semantic equivalence check:**
- Uncertainty preserved: YES
- Decision condition added: NO
- Hard constraint added: NO
- Priority added: NO
- Conclusion added: NO
- "currently" material temporal claim: NEGLIGIBLE
- Perspective shift: REPRESENTATIONAL NORMALISATION (not substantive)
- Pair classification: MINOR NON-MATERIAL PARAPHRASE
**Architecture classification:** B — LEXICAL FALSE POSITIVE
The meanings are semantically equivalent; the rejection depends on `"unsure"` being in `deriveAnswerMeaningProfile`'s detection patterns while `"uncertain"` is not. Both denote "lacking sufficient knowledge or certainty."
**Keyword-dictionary risk: YES.** The validator's semantic fidelity decision depends entirely on whether the LLM uses one of ~15-20 hardcoded English surface forms (`"unsure"`, `"not sure"`, etc.) rather than equivalent terms (`"uncertain"`). This is raw-language keyword detection, not structured semantic contract validation.
**Existing structured signals: PARTIAL.** The schema defines `supportCategory` (enum of five values) and `resolutionGuidance` (enum of three values), but both are free-text Zod strings without enum enforcement, and the LLM consistently produces null for both (per 56D). They can carry the needed semantics in theory but not in practice — a small contract extension (prompt requirement + schema constraint) is needed to make them operational. `possibleInference` exists but has no validator consumer. No production field carries a dedicated uncertainty-type classification (`evidence_needed`/`user_clarification_needed`) used in tests.
**57J.47 documentation correction applied:** Previous wording stated meaning was "STRENGTHENED" as fact. Corrected to distinguish: human semantic assessment (minor non-material paraphrase) from validator mechanism (lexical gap in detection patterns). The actual rejection was a false positive caused by incomplete keyword coverage, not genuine semantic strengthening.
**Production code changed: NO. Prompt changed: NO. Validator changed: NO. Tests permanently changed: NO. Ollama calls: 0.**
Configured Ollama: none (zero live calls).
---
### Experiment 57J.49 — Structured Semantic Fidelity Boundary Assessment
**Classification: B — EXISTING STRUCTURE IS PARTIAL**
Read-only architecture diagnosis of whether existing `answerMeaning` fields can replace keyword-based fidelity inference without a contract extension.
**Key finding:** All five classification values (`relative_priority_only`, `conditional_tradeoff`, `uncertain`, `explicit_hard_constraint`, `other`) and three resolution states (`must_remain_unresolved`, `may_resolve`, `must_resolve`) already exist in production enums (answerSupportCategory, answerResolutionGuidance at lines 147-158 of lib/graph/schema.js). The structured capability is fully designed.
**Missing piece — two independent gaps:**
1. **Population gap:** Both `supportCategory` and `resolutionGuidance` are free-text Zod strings not constrained to their respective enums, AND the prompt explicitly tells the model "if you are unsure of the exact wording, leave them null" (prompt-builder.js line 28). Result: LLM never populates them in production. The validator has no structured data to consume — it must re-infer from `userSupportedMeaning` text via keyword detection.
2. **Enforcement gap:** Neither field uses enum constraint enforcement. Even if populated, there is zero structural guarantee they contain valid category values.
**What existing structure CAN express (if populated):** `{supportCategory: "uncertain", resolutionGuidance: "must_remain_unresolved"}` — this exact combination expresses "remains unresolved uncertainty" without any lexical inference.
**Why B not A:** The fields are never reliably populated by the model, and without enum-constrained schema enforcement, they carry no trustworthy information in production today.
**What it does NOT establish:** That structured output solves the trust problem in general (Pattern A has no independent verification). That a dedicated `uncertaintyType` field (evidence_needed / user_clarification_needed) is unnecessary — this exists only in test fixtures, not any production schema.
Full results in `docs/experiment-57j49.md`.
### Experiment 57J.50 — Structured Fidelity Migration Choice
**Classification: C — STRUCTURED PRIMARY + NON-LEXICAL CONSISTENCY**
Read-only design decision selecting the smallest safe migration path from keyword-based fidelity inference to structured semantic fields as primary authority.
**Decision rationale:** Option C eliminates lexical dictionary dependence for all protected categories where structured fields are populated, while preserving backwards compatibility through null-to-lexical fallback for legacy proposals. It retains model-trust safeguards via cross-field deterministic consistency checks (resolutionGuidance vs proposal state, enum validity) without requiring the validator to re-infer English semantics from raw text or invent new classifiers.
**Exact next implementation boundary:** One branch — `feature/structured-semantic-fidelity-v0.20`. Four file modifications:
1. `lib/graph/prompt-builder.js` — rule 28 replacement (enforce + instruct) + output contract addition (enum value listing for the two fields);
2. `lib/graph/schema.js``z.enum(Object.values(answerSupportCategory))` and `z.enum(Object.values(answerResolutionGuidance))` constraints on the two zod fields;
3. `lib/graph/apply-proposal.js` — migrate `validateAnswerMeaningAlignment()` to read structured values first, lexical fallback for null legacy proposals only; add schema-invariant consistency checks;
4. `tests/graph/apply-proposal.test.js` — nine regression tests matching the required list below.
No new taxonomy. No provider-specific code. No embeddings. No synonym lists. No new LLM call pattern.
Full results in `docs/experiment-57j50.md`.
### Experiment 57J.51 — Structured Semantic Fidelity Implementation
**Classification: E — IMPLEMENTATION COMPLETE**
Implemented the bounded Option C from 57J.50 on branch `feature/structured-semantic-fidelity-v0.20`.
**Production changes:**
- `lib/graph/schema.js` now enum-constrains `answerMeaning.supportCategory` and `answerMeaning.resolutionGuidance` while preserving nullability.
- `lib/graph/prompt-builder.js` now instructs the model to populate the structured semantic fields using the existing allowed values and exposes those exact values in the prompt contract.
- `lib/graph/apply-proposal.js` now treats populated structured semantic fields as the primary fidelity authority, skips lexical category verification on that populated path, and retains lexical derivation only as legacy fallback when structured fields are null.
- Structured consistency now deterministically rejects `resolutionGuidance = must_remain_unresolved` when the proposal resolves an unknown.
**Captured false positive resolved:** The exact `unsure` raw answer + structured `supportCategory=uncertain` case no longer fails on the populated structured path. No synonym rule was added.
**Deferred intentionally:** `must_resolve` target-specific enforcement remains deferred because the current proposal structure does not safely identify the answered/targeted unknown in every case without inventing new linkage.
**Tests:** Required deterministic suites pass, plus one directly related proposal-parse suite updated for the new schema boundary. No live model calls. No Ollama. Full record in `docs/experiment-57j51.md`.
### Experiment 57J.52 — Structured Semantic Fidelity Live Verification
**Classification: A — V0.20 STRUCTURED PATH WORKS**
One fresh live run through the production `startCase()``updateCase()` path. Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic."
**Key evidence:**
- `supportCategory = "uncertain"` (populated, valid enum) — STRUCTURED path active, not LEGACY FALLBACK.
- `resolutionGuidance = "may_resolve"` (populated).
- Meaning: FAITHFUL — model preserved uncertainty without strengthening or degrading.
- New unknown node `nf3g7m2` ("Realism of projected office savings from relocation") added with `depends_on` edge to summary — meaningful graph structure produced.
- The old `unsure``uncertain` lexical mismatch does NOT determine the outcome: structured `supportCategory` is authoritative, bypassing lexical derivation entirely.
**What this establishes:** v0.20's structured semantic fidelity path executes live on the real production path. The model populates structured fields from a "I am unsure..." answer, the validator trusts those populated fields over lexical comparison, and meaningful graph structure is produced.
**What this does NOT prove:** Stability across repeated runs; behavior for answers outside existing categories; `must_remain_unresolved` enforcement in practice; end-to-end investigation viability past Update 2+. Full record in `docs/experiment-57j52.md`.
### Experiment 57J.55 — Uncertainty Identity Clarification (Prompt-Only)
**Objective:** Implement the correction from Experiment 57J.54: clarify that "same uncertainty" means the same resolution question, not topical overlap. Bounded prompt-only task — no schema, validator, graph-model, or provider changes.
**Changes:**
- `lib/graph/prompt-builder.js`: Added to Additional Guidance — *"Same uncertainty means the same resolution question: resolving the existing unknown would also resolve the uncertainty introduced by the user's answer. Mere topical overlap (concerning the same topic, object, decision, or domain) is not automatically the same uncertainty."*
- `tests/graph/prompt-builder.test.js`: 10 new focused prompt tests verifying the definition, boundaries, and preserved contracts (49 total pass).
**Test results:** All 49 tests pass. No regression in existing prompt structure tests.
**Configured Ollama:** none used. **Production code changed:** NO. **Ollama calls:** 0.
### Experiment 57J.56 — Uncertainty Identity Live Validation
Tested whether v0.21 now preserves a focused savings-realism uncertainty separately from broader related cost unknowns. **Classification: A — V0.21 IDENTITY RULE WORKS LIVE.** Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic."
**Start:** HTTP 200 | unknown stage | 8 nodes | 5 edges | Selected question: "What would clarify how long it will take for ongoing savings to offset upfront expenses and productivity dips in this situation?"
Relevant unresolved unknowns at start: (1) `nx54wwa` — operating expenses (metric, known), (2) `nkmuu21` — one-time relocation costs (unknown, unknown), (3) `nt0asmb` — team output loss/turnover (unknown, unknown), (4) `n4j29jl` — payback period timing (unknown, unknown).
**Update 1:** HTTP 200 | update_applied stage | no errors/validation failures. New node `n_savings_realism` added with label "Validity of projected office savings estimates from relocation" (unknown/unknown) — distinct from all four start unknowns. `nx54wwa` updated: known → provisional. Added edge: `e-doubt-savings` weakens → `nx54wwa`. No equivalent pre-existing unknown existed; focused uncertainty added as a new node rather than absorbed into broader cost nodes. Identity result: **ADDED DISTINCT UNCERTAINTY**. Meaning: **FAITHFUL** (doubt preserved, not strengthened).
**Key finding:** The v0.21 prompt clarification ("same uncertainty = same resolution question") works on a live run with qwen-claude:latest. Unlike 57J.53 (where uncertain status degraded an unrelated node's status without creating a dedicated unknown), v0.21 correctly creates and preserves a dedicated savings-realism unknown with proper structural linkage (`weakens` edge). This resolves the prompt semantic-identity gap diagnosed in 57J.54.
**What remains unproven:** Repeated-run stability (single run); downstream investigation viability past Update 2; cross-domain generalisation; paraphrase invariance; whether reuse/refine still triggers when an equivalent unknown already exists. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. **Production code changed:** NO. **Ollama calls:** 1 live call (start + update via harness).
### Experiment 57J.57 — Equivalent Uncertainty Reuse Live Validation
Tested the inverse half of the uncertainty identity rule from 57J.56: when a graph already contains a dedicated savings-realism uncertainty, does a second semantically equivalent paraphrase reuse/refine that node rather than create a duplicate? **Classification: D — UPDATE 1 FAILED.** Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Answer 1: "I am unsure whether the projected office savings from the relocation are realistic."
**Start:** HTTP 200 | unknown stage | 7 nodes | 4 edges. Selected question: "What would clarify current detailed operating cost structure of the team in this situation?" Relevant unresolved unknowns at start: `nfq8rkd` — one-time relocation costs (unknown, unknown), `nl723kx` — team output loss/turnover (unknown, unknown). Note `nx54wwa` (operating expenses) was known.
**Update 1:** HTTP 422 | proposal_compatibility stage. The model correctly interpreted Answer 1 semantically (`userSupportedMeaning: "The user is unsure whether the projected office savings from the relocation are realistic."`) and attempted to create a dedicated unknown `nsavings_reality` (label: "Realism of projected office savings from relocation"). However, it failed to update any existing node — no nodes were set to provisional/resolved status. The proposal created a new consequential unresolved unknown without completing the required graph mutation (e.g., downgrading `nx54wwa` from known to provisional). The system rejected the proposal: "selectedQuestion is required when consequential unresolved unknowns remain after resolving the answered unknown." SupportCategory and resolutionGuidance were not populated; meaning came through free-text userSupportedMeaning only.
**Key finding:** Semantic interpretation works correctly (distinct savings-realism uncertainty identified), but structural linkage failed silently — no existing node was updated to reflect the uncertainty, so the proposal lacked the graph state needed for downstream question selection. This is a different failure mode from 57J.56 which DID update nx54wwa successfully.
**What remains unproven:** Whether Answer 2 would reuse or duplicate; downstream investigation viability; cross-domain generalisation. Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. **Production code changed:** NO. **Ollama calls:** 0 live calls (update rejected before mutation).
---
### Experiment 57J.58 — Selected-Question Ownership Diagnosis Confirmation
**Objective:** Record durable handoff facts from `docs/experiment-57j58.md` which diagnosed why a proposal adding a new unresolved unknown was rejected due to missing `selectedQuestion`, and classified the root cause.
**validateQuestionSelectionRequirement() rejects when:**
- `selectedQuestion == null`
- AND
- proposal adds at least one non-resolved unknown
**requirement does NOT depend on:**
- `updatedNodes`
- actual `resolvedUnknownNodeIds`
- total existing unresolved-node count
**prompt rule #16 says selectedQuestion "may" identify a candidate**
**validator treats it as mandatory for added unresolved unknowns**
**classification = PROMPT CONTRACT GAP**
**smallest next boundary = prompt-only clarification**
Configured Ollama: none used. **Production code changed:** NO.
---
### Experiment 57J.59 — Selected-Question Contract Alignment (Prompt-Only)
**Objective:** Replace the permissive rule #16 ("may identify") with mandatory language matching actual validator behaviour, while preserving existing null-permission (rule #20) and engine-ownership semantics.
**Production changes:**
- `lib/graph/prompt-builder.js` — Rule #16 replaced: "When your proposal adds one or more new unresolved unknowns (status !== 'resolved'), you MUST include a selectedQuestion identifying one of those as a candidate unknown node. The engine validates your candidate and retains deterministic final-priority selection; your candidate does not need to be the highest-scoring unknown — it only needs to be a valid unresolved unknown that exists in the graph or in addedNodes."
- `lib/graph/prompt-builder.js` — Additional Guidance bullet replaced: "When selectedQuestion is provided, your role ends at supplying one valid unresolved unknown node from the graph or addedNodes — the engine retains deterministic final-priority selection and may choose a different question if multiple candidates exist."
- `tests/graph/prompt-builder.test.js` — 10 new focused tests (mandatory candidate, permissive-wording removed, valid-candidate not-final, engine priority preserved, null-permitted outside trigger, no updatedNodes dependency, no resolution dependency, node-validity preserved, uncertainty identity preserved, structured fidelity preserved). Updated existing test to match new rule #16 wording.
**Test results:** 59 tests pass (42 existing + 17 new = 59 total). Zero failures.
**Ownership split preserved:**
- MODEL: supply one valid candidate when new unresolved unknowns are added
- ENGINE: validate candidate, retain deterministic priority/scoring ownership
Configured Ollama: none used. **Production code changed:** prompt + tests only. **Ollama calls:** 0.