confidence-engine/docs/experiment-57j84.md

Experiment 57J.84 — Direct Meaning/Action Field Observation on Anchored £2m Update

Branch: feature/semantic-action-contract-v0.23 Starting HEAD: 6a04d62 (docs: record accepted answer-meaning capture)

Objective

For the anchored savings-realism case with new £2m/year information, what does the model directly populate for userSupportedMeaning and structuralActionRequired, and is the accepted proposal contract-consistent?

Reruns the 57J.82 case only to replace inference with direct observation.

Fixed Starting Graph

Pre-anchored fixture (tests/fixtures/pre-anchored-update-savings-realism.json):

id: n_savings_realism
label: Are the projected office savings from relocation realistic?
status: unknown
kind: unknown
value: null

Exactly one equivalent unresolved savings-realism uncertainty before Update: YES

Fixed Answer

"The projected office saving is about £2 million per year based on eliminating the current lease cost, but I am still unsure whether that estimate is realistic."

Configured Model

• Ollama base URL: http://192.168.1.111:11434
• Model: qwen-claude:latest

Run

FIXTURE_MODE=updateOnly \
ANSWER_2="The projected office saving is about £2 million per year based on eliminating the current lease cost, but I am still unsure whether that estimate is realistic." \
CONFIDENCE_ENGINE_BASE_URL=http://127.0.0.1:3000 \
node scripts/reproduce-multi-turn-investigation.mjs

CALL ACCOUNTING

• startCalls: 0
• updateCalls: 1
• totalCalls: 1
• Retries: 0
• Supplementary scripts: NO

PRE-ANCHORED FIXTURE

• savings-realism node id: n_savings_realism
• label: "Are the projected office savings from relocation realistic?"
• status: unknown
• Exactly one equivalent unresolved uncertainty before Update: YES
• previousQuestion sent: "Are the projected office savings from relocation realistic?" (derived from fixture.unresolved_question)

UPDATE

HTTP status: 200 (update succeeded — no rejection; mutation applied) Stage: UNAVAILABLE (harness updateOnly path does not explicitly print stage for accepted updates, but successful response with mutations confirms update_applied) Validation errors: none

Direct field capture

• userSupportedMeaning: null (answerMeaning present at top level of response but all fields — userSupportedMeaning, possibleInference, supportCategory, resolutionGuidance — resolved to null)
• possibleInference: null
• supportCategory: null
• resolutionGuidance: null
• structuralActionRequired: null

Proposal mutations

updatedNodes: []
resolvedUnknownNodeIds: []
addedNodes: [{"id":"n_lease_savings_claim","label":"Projected savings from lease elimination claim","description":"Claim that eliminating the current London lease yields approximately £2 million in annual office savings.","kind":"reported_claim","status":"provisional","confidence":"medium","value":2000000,"unit":"GBP/year","dependsOn":[],"affects":["n_savings_realism"],"parentId":null,"childIds":["n_savings_realism"]}]
addedEdges: [{"id":"e-claim-to-realism","fromNodeId":"n_lease_savings_claim","toNodeId":"n_savings_realism","relationship":"supports","confidence":"medium","description":"The specific savings claim directly feeds into the uncertainty regarding its realism."}]
selectedQuestion: "What evidence would clarify are the projected office savings from relocation realistic?"
selectedQuestion.nodeId: "n_savings_realism"

Resulting persistent graph (3 nodes, 2 edges)

node: id=n_relocation_state, kind=state, label=Engineering team relocation consideration, status=provisional
node: id=n_savings_realism, kind=unknown, label=Are the projected office savings from relocation realistic?, status=unknown
node: id=n_lease_savings_claim, kind=reported_claim, label=Projected savings from lease elimination claim, status=provisional
edge: from=n_savings_realism, to=n_relocation_state, relationship=depends_on
edge: from=n_lease_savings_claim, to=n_savings_realism, relationship=supports

Meaning fidelity

UNAVAILABLE — userSupportedMeaning was not directly populated. The model implicitly captured answer meaning through structural graph changes (new reported_claim node) rather than explicit semantic field population.

Meaningful mutation: PRESENT

The model produced a non-trivial structural change: new reported_claim node with structured value (£2,000,000 GBP/year) and supporting edge to the existing savings-realism unknown node. This satisfies hasMeaningfulChange semantics.

Direct contract state: D — MUTATION WITHOUT USER-SUPPORTED MEANING

userSupportedMeaning = null/UNAVAILABLE
structuralActionRequired = null
meaningful mutation = PRESENT

£2m/year information: REPRESENTED

Captured as value=2,000,000 with unit=GBP/year on the new reported_claim node. Full precision preserved ("£2 million" → 2,000,000).

Lease-cost basis: REPRESENTED

Description explicitly states "Claim that eliminating the current London lease yields approximately £2 million in annual office savings." — both £2m value and lease-elimination basis are structured on the node.

Savings-realism identity: EXISTING IDENTITY PRESERVED

Equivalent unresolved savings-realism node count: 1

Exactly one equivalent unresolved savings-realism uncertainty remains. The original n_savings_realism persisted unchanged (status=unknown, value=null). No duplicate created.

Classification: D — MUTATION WITHOUT USER-SUPPORTED MEANING

The model produced a meaningful structural mutation (new evidence node with £2m/year + lease-basis data) but did not populate any explicit semantic extraction fields (userSupportedMeaning, possibleInference, supportCategory all null). The structural action boolean (structuralActionRequired) was also not populated.

Why:

The model implicitly represented the user's answer through graph structure rather than explicit meaning fields. It created a new reported_claim node capturing the £2m/year savings figure and lease-elimination basis, then linked it as supporting evidence to the existing savings-realism uncertainty. This is a structurally faithful representation of the answer — but without userSupportedMeaning or other semantic field population, there is no direct observable meaning extraction to evaluate for fidelity.

Did model directly emit userSupportedMeaning: NO

Did model directly emit structuralActionRequired: NO (null)

Was meaningful new information structurally preserved: YES

£2m/year and lease-cost basis both fully represented on the new reported_claim node.

Did equivalent uncertainty duplicate: NO

Did exactly one savings-realism identity remain: YES

What this establishes:

1. The model can implicitly capture answer meaning through structural graph mutation even when explicit semantic fields are not populated
2. A new evidence node with structured numeric value (£2M GBP/year) and descriptive lease-basis was created as supporting evidence for the existing savings-realism uncertainty
3. Identity preservation holds — the original n_savings_realism node remains untouched
4. The model can produce contract-consistent true+mutation (through implicit representation) even without explicit userSupportedMeaning field population
5. The structuralActionRequired null gap persists: the model creates meaningful mutation but does not populate the boolean declaration

What this does NOT prove:

• Whether the model can explicitly populate userSupportedMeaning alongside structural mutation in a single response
• Whether the implicit-meaning-through-mutation pattern is stable across repeated runs
• Whether this behavior generalizes to other domains or answer types
• Whether the £2M/year structured value survives full end-to-end graph queries (no downstream query tested)

Production code changed: NO

Prompt changed during experiment: NO

Harness/tooling changed: NO

Canonical committed update-only mode used: YES

57J.83 direct accepted meaning capture exercised: YES

The harness correctly captured answerMeaning from the production response path — fields were present at top level but all null, confirming the model did not populate them.

Ollama calls beyond harness count: 0

Dev server disturbed: NO

Documentation updated: YES (docs/experiment-57j84.md + docs/current-handoff.md)