diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 41a27a4..8b81940 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -191,6 +191,135 @@ _Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F, --- +### Experiment 57J.82 — Incremental Supported Information on Anchored Uncertainty + +**Objective:** With the savings-realism uncertainty already present, does new supported information (£2m/year from lease elimination with continued realism uncertainty) cause the model to emit `structuralActionRequired=true`, preserve that information structurally, and keep a single savings-realism uncertainty identity? + +**Starting HEAD:** `8526aa4` (tooling: supply anchored previous question in update-only mode) +**Branch:** `feature/semantic-action-contract-v0.23` + +**Configured Ollama:** qwen-claude:latest at http://192.168.1.111:11434 + +### CALL ACCOUNTING + +- startCalls: 0 +- updateCalls: 1 +- totalCalls: 1 +- Retries: 0 +- Supplementary scripts: NO + +### PRE-ANCHORED FIXTURE + +- savings-realism node id: `n_savings_realism` +- label: "Are the projected office savings from relocation realistic?" +- status: unknown +- Exactly one equivalent unresolved uncertainty before Update: YES +- previousQuestion sent: "Are the projected office savings from relocation realistic?" + +### UPDATE + +- HTTP status: 200 +- Stage: update_applied +- Validation errors: none +- userSupportedMeaning: (not printed by harness; model extracted "~£2M/year (lease elimination)" as newValue on existing node) +- possibleInference: not printed +- supportCategory: not printed +- resolutionGuidance: not printed +- structuralActionRequired: null + +#### Proposal mutations + +``` +updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: unknown, previousValue: null, newValue: "~£2M/year (lease elimination)", reason: "User provided a specific projected savings figure but explicitly maintained uncertainty about its realism, so the question remains unresolved."}] +resolvedUnknownNodeIds: [] +addedNodes: [] +addedEdges: [] +selectedQuestion: "What would clarify are the projected office savings from relocation realistic in this situation?" +selectedQuestion.nodeId: "n_savings_realism" +``` + +#### Resulting persistent graph (2 nodes, 1 edge) + +``` +node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional +node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=unknown +edge: from=n_savings_realism, to=n_relocation_state, relationship=depends_on +``` + +### Meaning fidelity: INCOMPLETE + +The model extracted partial information as `newValue: "~£2M/year (lease elimination)"` — it captured the approximate figure and acknowledged lease basis in parenthetical form but collapsed these into a single value string rather than structuring them as separate fields. The continued uncertainty about realism was preserved in the reason text but not as a structured field. + +### Meaningful mutation: PRESENT + +The model produced a non-trivial update: it wrote `newValue: "~£2M/year (lease elimination)"` onto an existing node with value=null, changing from null to populated. However this is a soft/value-level update, not a dedicated structural change (no new node, no edge). + +### £2m/year information: PARTIAL + +Captured as `"~£2M/year"` in the newValue — approximate figure present but not at full precision ("about £2 million" → "~£2M"). Not inventing or omitting. + +### Lease-cost basis: NOT REPRESENTED (in structured value) + +The lease-elimination basis appears only inside parentheses within the value string `"(lease elimination)"`, not as a separate structured field. In the reason text it is contextualised but this is prose, not structural representation. + +### Savings-realism identity: EXISTING IDENTITY PRESERVED + +Equivalent unresolved savings-realism node count: 1 + +Exactly one equivalent unresolved savings-realism uncertainty remains. No duplicate created. The original `n_savings_realism` persisted with status=unknown throughout. + +### Contract state: MISSING + +`structuralActionRequired` is null — neither true nor false. The model did not populate this required field. + +### Classification: G — FIELD MISSING + +`structuralActionRequired` is null/absent. Cannot assess TRUE+MUTATION or FALSE+NO-MUTATION because the declaring boolean was never produced. + +### Why: + +The model produced meaningful mutation (populating a previously-null node value with "~£2M/year (lease elimination)") and preserved exactly one savings-realism uncertainty identity — but did not populate `structuralActionRequired`. The update was accepted by the production path (HTTP 200 at update_applied) because mutation was present, even though the structural action declaration field was null. This is the same gap observed in 57J.64/57J.69 where the model knows to act but omits the boolean declaration. + +### Did model directly emit structuralActionRequired=true: NO + +### Was meaningful new information structurally preserved: PARTIAL + +£2m/year captured approximately in newValue; lease-cost basis not separately structured. + +### Did equivalent uncertainty duplicate: NO + +### Did exactly one savings-realism identity remain: YES + +### What this establishes: + +1. The 57J.81 previousQuestion fix works — update-only mode reaches model inference without validation rejection +2. The model can extract and partially represent new supported information (£2m/year with lease basis) on an existing unresolved uncertainty node +3. No duplicate uncertainty is created — identity preservation holds in update-only mode +4. The structuralActionRequired field remains consistently null/unpopulated by this model on this prompt + +### What this does NOT prove: + +- Whether `structuralActionRequired` can ever be populated true on this prompt/model +- Whether the newValue format ("~£2M/year (lease elimination)") would survive full end-to-end graph queries +- Whether this behavior is stable across repeated runs +- Cross-domain generalisation + +### Production code changed: NO + +### Prompt changed during experiment: NO + +### Harness/tooling changed: NO + +### Canonical committed update-only mode used: YES + +### 57J.81 previousQuestion fix exercised: YES — fixture-derived `"Are the projected office savings from relocation realistic?"` was sent as previousQuestion + +### Ollama calls beyond harness count: 0 + +### Dev server disturbed: NO + +--- + ### Experiment 57J.53 — Structured Fidelity Multi-Turn Progress Tested whether the structured-fidelity path supports genuine investigation progress on Update 2 after Update 1 represented savings-realism uncertainty. **Classification: B — USEFUL PARTIAL PROGRESS.** diff --git a/docs/experiment-57j82.md b/docs/experiment-57j82.md new file mode 100644 index 0000000..3a36d10 --- /dev/null +++ b/docs/experiment-57j82.md @@ -0,0 +1,153 @@ +# Experiment 57J.82 — Incremental Supported Information on Anchored Uncertainty + +**Branch:** `feature/semantic-action-contract-v0.23` +**Starting HEAD:** `8526aa4` (tooling: supply anchored previous question in update-only mode) + +## Objective + +Answer exactly: + +> With the savings-realism uncertainty already present, does new supported information cause the model to emit `structuralActionRequired=true`, preserve that information structurally, and keep a single savings-realism uncertainty identity? + +## Configured scenario (fixed from fixture) + +"We are considering relocating the engineering team to reduce operating costs." + +## Configured answer (fixed) + +"The projected office saving is about £2 million per year based on eliminating the current lease cost, but I am still unsure whether that estimate is realistic." + +## Hypothesis + +The answer contains: +- existing unresolved meaning: whether the projected savings estimate is realistic +- new supported information: approximately £2 million per year, basis = eliminating the current lease cost + +Expected contract-consistent path: `structuralActionRequired = true`, meaningful mutation present. The existing savings-realism uncertainty should remain the sole persistent representation. + +## Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434 + +### CALL ACCOUNTING + +- startCalls: 0 +- updateCalls: 1 +- totalCalls: 1 +- Retries: 0 +- Supplementary scripts: NO + +### PRE-ANCHORED FIXTURE + +Savings-realism node: +``` +id: n_savings_realism +label: Are the projected office savings from relocation realistic? +status: unknown +``` + +Exactly one equivalent unresolved uncertainty before Update: YES + +previousQuestion sent: `"Are the projected office savings from relocation realistic?"` (derived from fixture anchor per 57J.81 fix) + +### UPDATE + +HTTP status: 200 +Stage: update_applied + +Validation errors: none + +#### Answer meaning (captured via mutation evidence, not explicit answerMeaning output) + +The model extracted new supported information and wrote it onto the existing uncertainty node: + +- newValue: `"~£2M/year (lease elimination)"` +- reason: "User provided a specific projected savings figure but explicitly maintained uncertainty about its realism, so the question remains unresolved." + +#### structuralActionRequired: null + +The model did not populate `structuralActionRequired`. This field was neither true nor false — it was absent from the model's output. + +#### Proposal mutations + +``` +updatedNodes: [ + { + nodeId: "n_savings_realism", + previousStatus: "unknown", + newStatus: "unknown", + previousValue: null, + newValue: "~£2M/year (lease elimination)", + reason: "User provided a specific projected savings figure but explicitly maintained uncertainty about its realism, so the question remains unresolved." + } +] +resolvedUnknownNodeIds: [] +addedNodes: [] +addedEdges: [] +selectedQuestion: "What would clarify are the projected office savings from relocation realistic in this situation?" +selectedQuestion.nodeId: "n_savings_realism" +``` + +#### Resulting persistent graph (2 nodes, 1 edge) + +``` +node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional +node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=unknown +edge: from=n_savings_realism, to=n_relocation_state, relationship=depends_on +``` + +### Meaning fidelity: INCOMPLETE + +The model extracted partial information as `newValue: "~£2M/year (lease elimination)"` — it captured the approximate figure and acknowledged lease basis in parenthetical form but collapsed these into a single value string rather than structuring them as separate fields. The continued uncertainty about realism was preserved in the reason text but not as a structured field. + +### Meaningful mutation: PRESENT + +The model produced a non-trivial update: it wrote `newValue: "~£2M/year (lease elimination)"` onto an existing node with value=null, changing from null to populated. However this is a soft/value-level update, not a dedicated structural change (no new node, no edge). + +### £2m/year information: PARTIAL + +Captured as `"~£2M/year"` in the newValue — approximate figure present but not at full precision ("about £2 million" → "~£2M"). Not inventing or omitting. + +### Lease-cost basis: NOT REPRESENTED (in structured value) + +The lease-elimination basis appears only inside parentheses within the value string `"(lease elimination)"`, not as a separate structured field. In the reason text it is contextualised but this is prose, not structural representation. + +### Savings-realism identity: EXISTING IDENTITY PRESERVED + +Equivalent unresolved savings-realism node count: 1 + +Exactly one equivalent unresolved savings-realism uncertainty remains. No duplicate created. The original `n_savings_realism` persisted with status=unknown throughout. + +### Contract state: MISSING + +`structuralActionRequired` is null — neither true nor false. The model did not populate this required field. + +## Classification: G — FIELD MISSING + +`structuralActionRequired` is null/absent. Cannot assess TRUE+MUTATION or FALSE+NO-MUTATION because the declaring boolean was never produced. + +## Why + +The model produced meaningful mutation (populating a previously-null node value with "~£2M/year (lease elimination)") and preserved exactly one savings-realism uncertainty identity — but did not populate `structuralActionRequired`. The update was accepted by the production path (HTTP 200 at update_applied) because mutation was present, even though the structural action declaration field was null. This is the same gap observed in 57J.64/57J.69 where the model knows to act but omits the boolean declaration. + +## What this establishes + +1. The 57J.81 previousQuestion fix works — update-only mode reaches model inference without validation rejection +2. The model can extract and partially represent new supported information (£2m/year with lease basis) on an existing unresolved uncertainty node +3. No duplicate uncertainty is created — identity preservation holds in update-only mode +4. The structuralActionRequired field remains consistently null/unpopulated by this model on this prompt + +## What this does NOT prove + +- Whether `structuralActionRequired` can ever be populated true on this prompt/model +- Whether the newValue format ("~£2M/year (lease elimination)") would survive full end-to-end graph queries +- Whether this behavior is stable across repeated runs +- Cross-domain generalisation + +## Production code changed: NO + +No production code was modified during this experiment. + +## Harness restored: YES + +Scenario, answers, and mode were set as environment variables for the single run; no harness modifications were made. + +---