experiment: rerun incremental meaning on anchored uncertainty
This commit is contained in:
@@ -191,6 +191,135 @@ _Created by Experiment 34. Updated by Experiments 38–53, 54A–54Z, 55A–55F,
|
||||
|
||||
---
|
||||
|
||||
### Experiment 57J.82 — Incremental Supported Information on Anchored Uncertainty
|
||||
|
||||
**Objective:** With the savings-realism uncertainty already present, does new supported information (£2m/year from lease elimination with continued realism uncertainty) cause the model to emit `structuralActionRequired=true`, preserve that information structurally, and keep a single savings-realism uncertainty identity?
|
||||
|
||||
**Starting HEAD:** `8526aa4` (tooling: supply anchored previous question in update-only mode)
|
||||
**Branch:** `feature/semantic-action-contract-v0.23`
|
||||
|
||||
**Configured Ollama:** qwen-claude:latest at http://192.168.1.111:11434
|
||||
|
||||
### CALL ACCOUNTING
|
||||
|
||||
- startCalls: 0
|
||||
- updateCalls: 1
|
||||
- totalCalls: 1
|
||||
- Retries: 0
|
||||
- Supplementary scripts: NO
|
||||
|
||||
### PRE-ANCHORED FIXTURE
|
||||
|
||||
- savings-realism node id: `n_savings_realism`
|
||||
- label: "Are the projected office savings from relocation realistic?"
|
||||
- status: unknown
|
||||
- Exactly one equivalent unresolved uncertainty before Update: YES
|
||||
- previousQuestion sent: "Are the projected office savings from relocation realistic?"
|
||||
|
||||
### UPDATE
|
||||
|
||||
- HTTP status: 200
|
||||
- Stage: update_applied
|
||||
- Validation errors: none
|
||||
- userSupportedMeaning: (not printed by harness; model extracted "~£2M/year (lease elimination)" as newValue on existing node)
|
||||
- possibleInference: not printed
|
||||
- supportCategory: not printed
|
||||
- resolutionGuidance: not printed
|
||||
- structuralActionRequired: null
|
||||
|
||||
#### Proposal mutations
|
||||
|
||||
```
|
||||
updatedNodes: [{nodeId: n_savings_realism, previousStatus: unknown, newStatus: unknown, previousValue: null, newValue: "~£2M/year (lease elimination)", reason: "User provided a specific projected savings figure but explicitly maintained uncertainty about its realism, so the question remains unresolved."}]
|
||||
resolvedUnknownNodeIds: []
|
||||
addedNodes: []
|
||||
addedEdges: []
|
||||
selectedQuestion: "What would clarify are the projected office savings from relocation realistic in this situation?"
|
||||
selectedQuestion.nodeId: "n_savings_realism"
|
||||
```
|
||||
|
||||
#### Resulting persistent graph (2 nodes, 1 edge)
|
||||
|
||||
```
|
||||
node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional
|
||||
node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=unknown
|
||||
edge: from=n_savings_realism, to=n_relocation_state, relationship=depends_on
|
||||
```
|
||||
|
||||
### Meaning fidelity: INCOMPLETE
|
||||
|
||||
The model extracted partial information as `newValue: "~£2M/year (lease elimination)"` — it captured the approximate figure and acknowledged lease basis in parenthetical form but collapsed these into a single value string rather than structuring them as separate fields. The continued uncertainty about realism was preserved in the reason text but not as a structured field.
|
||||
|
||||
### Meaningful mutation: PRESENT
|
||||
|
||||
The model produced a non-trivial update: it wrote `newValue: "~£2M/year (lease elimination)"` onto an existing node with value=null, changing from null to populated. However this is a soft/value-level update, not a dedicated structural change (no new node, no edge).
|
||||
|
||||
### £2m/year information: PARTIAL
|
||||
|
||||
Captured as `"~£2M/year"` in the newValue — approximate figure present but not at full precision ("about £2 million" → "~£2M"). Not inventing or omitting.
|
||||
|
||||
### Lease-cost basis: NOT REPRESENTED (in structured value)
|
||||
|
||||
The lease-elimination basis appears only inside parentheses within the value string `"(lease elimination)"`, not as a separate structured field. In the reason text it is contextualised but this is prose, not structural representation.
|
||||
|
||||
### Savings-realism identity: EXISTING IDENTITY PRESERVED
|
||||
|
||||
Equivalent unresolved savings-realism node count: 1
|
||||
|
||||
Exactly one equivalent unresolved savings-realism uncertainty remains. No duplicate created. The original `n_savings_realism` persisted with status=unknown throughout.
|
||||
|
||||
### Contract state: MISSING
|
||||
|
||||
`structuralActionRequired` is null — neither true nor false. The model did not populate this required field.
|
||||
|
||||
### Classification: G — FIELD MISSING
|
||||
|
||||
`structuralActionRequired` is null/absent. Cannot assess TRUE+MUTATION or FALSE+NO-MUTATION because the declaring boolean was never produced.
|
||||
|
||||
### Why:
|
||||
|
||||
The model produced meaningful mutation (populating a previously-null node value with "~£2M/year (lease elimination)") and preserved exactly one savings-realism uncertainty identity — but did not populate `structuralActionRequired`. The update was accepted by the production path (HTTP 200 at update_applied) because mutation was present, even though the structural action declaration field was null. This is the same gap observed in 57J.64/57J.69 where the model knows to act but omits the boolean declaration.
|
||||
|
||||
### Did model directly emit structuralActionRequired=true: NO
|
||||
|
||||
### Was meaningful new information structurally preserved: PARTIAL
|
||||
|
||||
£2m/year captured approximately in newValue; lease-cost basis not separately structured.
|
||||
|
||||
### Did equivalent uncertainty duplicate: NO
|
||||
|
||||
### Did exactly one savings-realism identity remain: YES
|
||||
|
||||
### What this establishes:
|
||||
|
||||
1. The 57J.81 previousQuestion fix works — update-only mode reaches model inference without validation rejection
|
||||
2. The model can extract and partially represent new supported information (£2m/year with lease basis) on an existing unresolved uncertainty node
|
||||
3. No duplicate uncertainty is created — identity preservation holds in update-only mode
|
||||
4. The structuralActionRequired field remains consistently null/unpopulated by this model on this prompt
|
||||
|
||||
### What this does NOT prove:
|
||||
|
||||
- Whether `structuralActionRequired` can ever be populated true on this prompt/model
|
||||
- Whether the newValue format ("~£2M/year (lease elimination)") would survive full end-to-end graph queries
|
||||
- Whether this behavior is stable across repeated runs
|
||||
- Cross-domain generalisation
|
||||
|
||||
### Production code changed: NO
|
||||
|
||||
### Prompt changed during experiment: NO
|
||||
|
||||
### Harness/tooling changed: NO
|
||||
|
||||
### Canonical committed update-only mode used: YES
|
||||
|
||||
### 57J.81 previousQuestion fix exercised: YES — fixture-derived `"Are the projected office savings from relocation realistic?"` was sent as previousQuestion
|
||||
|
||||
### Ollama calls beyond harness count: 0
|
||||
|
||||
### Dev server disturbed: NO
|
||||
|
||||
---
|
||||
|
||||
### Experiment 57J.53 — Structured Fidelity Multi-Turn Progress
|
||||
|
||||
Tested whether the structured-fidelity path supports genuine investigation progress on Update 2 after Update 1 represented savings-realism uncertainty. **Classification: B — USEFUL PARTIAL PROGRESS.**
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
# Experiment 57J.82 — Incremental Supported Information on Anchored Uncertainty
|
||||
|
||||
**Branch:** `feature/semantic-action-contract-v0.23`
|
||||
**Starting HEAD:** `8526aa4` (tooling: supply anchored previous question in update-only mode)
|
||||
|
||||
## Objective
|
||||
|
||||
Answer exactly:
|
||||
|
||||
> With the savings-realism uncertainty already present, does new supported information cause the model to emit `structuralActionRequired=true`, preserve that information structurally, and keep a single savings-realism uncertainty identity?
|
||||
|
||||
## Configured scenario (fixed from fixture)
|
||||
|
||||
"We are considering relocating the engineering team to reduce operating costs."
|
||||
|
||||
## Configured answer (fixed)
|
||||
|
||||
"The projected office saving is about £2 million per year based on eliminating the current lease cost, but I am still unsure whether that estimate is realistic."
|
||||
|
||||
## Hypothesis
|
||||
|
||||
The answer contains:
|
||||
- existing unresolved meaning: whether the projected savings estimate is realistic
|
||||
- new supported information: approximately £2 million per year, basis = eliminating the current lease cost
|
||||
|
||||
Expected contract-consistent path: `structuralActionRequired = true`, meaningful mutation present. The existing savings-realism uncertainty should remain the sole persistent representation.
|
||||
|
||||
## Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434
|
||||
|
||||
### CALL ACCOUNTING
|
||||
|
||||
- startCalls: 0
|
||||
- updateCalls: 1
|
||||
- totalCalls: 1
|
||||
- Retries: 0
|
||||
- Supplementary scripts: NO
|
||||
|
||||
### PRE-ANCHORED FIXTURE
|
||||
|
||||
Savings-realism node:
|
||||
```
|
||||
id: n_savings_realism
|
||||
label: Are the projected office savings from relocation realistic?
|
||||
status: unknown
|
||||
```
|
||||
|
||||
Exactly one equivalent unresolved uncertainty before Update: YES
|
||||
|
||||
previousQuestion sent: `"Are the projected office savings from relocation realistic?"` (derived from fixture anchor per 57J.81 fix)
|
||||
|
||||
### UPDATE
|
||||
|
||||
HTTP status: 200
|
||||
Stage: update_applied
|
||||
|
||||
Validation errors: none
|
||||
|
||||
#### Answer meaning (captured via mutation evidence, not explicit answerMeaning output)
|
||||
|
||||
The model extracted new supported information and wrote it onto the existing uncertainty node:
|
||||
|
||||
- newValue: `"~£2M/year (lease elimination)"`
|
||||
- reason: "User provided a specific projected savings figure but explicitly maintained uncertainty about its realism, so the question remains unresolved."
|
||||
|
||||
#### structuralActionRequired: null
|
||||
|
||||
The model did not populate `structuralActionRequired`. This field was neither true nor false — it was absent from the model's output.
|
||||
|
||||
#### Proposal mutations
|
||||
|
||||
```
|
||||
updatedNodes: [
|
||||
{
|
||||
nodeId: "n_savings_realism",
|
||||
previousStatus: "unknown",
|
||||
newStatus: "unknown",
|
||||
previousValue: null,
|
||||
newValue: "~£2M/year (lease elimination)",
|
||||
reason: "User provided a specific projected savings figure but explicitly maintained uncertainty about its realism, so the question remains unresolved."
|
||||
}
|
||||
]
|
||||
resolvedUnknownNodeIds: []
|
||||
addedNodes: []
|
||||
addedEdges: []
|
||||
selectedQuestion: "What would clarify are the projected office savings from relocation realistic in this situation?"
|
||||
selectedQuestion.nodeId: "n_savings_realism"
|
||||
```
|
||||
|
||||
#### Resulting persistent graph (2 nodes, 1 edge)
|
||||
|
||||
```
|
||||
node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional
|
||||
node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=unknown
|
||||
edge: from=n_savings_realism, to=n_relocation_state, relationship=depends_on
|
||||
```
|
||||
|
||||
### Meaning fidelity: INCOMPLETE
|
||||
|
||||
The model extracted partial information as `newValue: "~£2M/year (lease elimination)"` — it captured the approximate figure and acknowledged lease basis in parenthetical form but collapsed these into a single value string rather than structuring them as separate fields. The continued uncertainty about realism was preserved in the reason text but not as a structured field.
|
||||
|
||||
### Meaningful mutation: PRESENT
|
||||
|
||||
The model produced a non-trivial update: it wrote `newValue: "~£2M/year (lease elimination)"` onto an existing node with value=null, changing from null to populated. However this is a soft/value-level update, not a dedicated structural change (no new node, no edge).
|
||||
|
||||
### £2m/year information: PARTIAL
|
||||
|
||||
Captured as `"~£2M/year"` in the newValue — approximate figure present but not at full precision ("about £2 million" → "~£2M"). Not inventing or omitting.
|
||||
|
||||
### Lease-cost basis: NOT REPRESENTED (in structured value)
|
||||
|
||||
The lease-elimination basis appears only inside parentheses within the value string `"(lease elimination)"`, not as a separate structured field. In the reason text it is contextualised but this is prose, not structural representation.
|
||||
|
||||
### Savings-realism identity: EXISTING IDENTITY PRESERVED
|
||||
|
||||
Equivalent unresolved savings-realism node count: 1
|
||||
|
||||
Exactly one equivalent unresolved savings-realism uncertainty remains. No duplicate created. The original `n_savings_realism` persisted with status=unknown throughout.
|
||||
|
||||
### Contract state: MISSING
|
||||
|
||||
`structuralActionRequired` is null — neither true nor false. The model did not populate this required field.
|
||||
|
||||
## Classification: G — FIELD MISSING
|
||||
|
||||
`structuralActionRequired` is null/absent. Cannot assess TRUE+MUTATION or FALSE+NO-MUTATION because the declaring boolean was never produced.
|
||||
|
||||
## Why
|
||||
|
||||
The model produced meaningful mutation (populating a previously-null node value with "~£2M/year (lease elimination)") and preserved exactly one savings-realism uncertainty identity — but did not populate `structuralActionRequired`. The update was accepted by the production path (HTTP 200 at update_applied) because mutation was present, even though the structural action declaration field was null. This is the same gap observed in 57J.64/57J.69 where the model knows to act but omits the boolean declaration.
|
||||
|
||||
## What this establishes
|
||||
|
||||
1. The 57J.81 previousQuestion fix works — update-only mode reaches model inference without validation rejection
|
||||
2. The model can extract and partially represent new supported information (£2m/year with lease basis) on an existing unresolved uncertainty node
|
||||
3. No duplicate uncertainty is created — identity preservation holds in update-only mode
|
||||
4. The structuralActionRequired field remains consistently null/unpopulated by this model on this prompt
|
||||
|
||||
## What this does NOT prove
|
||||
|
||||
- Whether `structuralActionRequired` can ever be populated true on this prompt/model
|
||||
- Whether the newValue format ("~£2M/year (lease elimination)") would survive full end-to-end graph queries
|
||||
- Whether this behavior is stable across repeated runs
|
||||
- Cross-domain generalisation
|
||||
|
||||
## Production code changed: NO
|
||||
|
||||
No production code was modified during this experiment.
|
||||
|
||||
## Harness restored: YES
|
||||
|
||||
Scenario, answers, and mode were set as environment variables for the single run; no harness modifications were made.
|
||||
|
||||
---
|
||||
Reference in New Issue
Block a user