confidence-engine/docs/experiment-57j28.md

Experiment 57J.28 — Live Node-Support Semantic Inputs Capture

Objective: On one fresh live run of the 57J.25 case, capture the exact semantic inputs that reach the v0.15 node-support gate and determine whether the savings/retention unknowns pass or fail.

Pre-written expectation recorded: YES

The raw answer explicitly contains both savings-realism and retention concerns. If a proposed unknown fails semantic admission, the captured userSupportedMeaning and node text should show whether the failure came from answerMeaning loss or from the existing grounding helper's overlap decision.

Configured apparatus

• Ollama base URL: http://192.168.1.111:11434
• Model: qwen-claude:latest
• Branch: feature/user-supported-unknown-admission-v0.15
• HEAD: current HEAD of branch at session start

Fixed scenario and answer

Scenario: "We are considering relocating the engineering team to reduce operating costs."

Answer: "Before deciding, I need evidence that the projected office savings are realistic and evidence that the move will not materially increase loss of key engineers."

Live-call count

start: 1
update 1: 1
total: 2

START

• HTTP status: 200
• Stage: unknown
• Node count: 7
• Edge count: 4
• Selected question: "What would clarify current detailed breakdown of engineering operating costs by location and category in this situation?"

UPDATE 1

• HTTP status: 200
• Stage: update_applied
• Error/validation summary: null (none)
• Node count: 9 (+2 new unknowns)
• Edge count: 6 (+2 new edges)

ANSWER MEANING

• userSupportedMeaning: "The decision requires evidence that projected office savings are realistic and evidence that key engineers will not materially leave due to the move."
• possibleInference: null

SAVINGS UNKNOWN

Field	Value
id	n-savings-est
label	"Projected office savings from relocation"
description	"Financial estimate of reduced operational expenses due to the move, needed to decide if the primary goal of lowering operating costs is achievable so that cost reduction justifies the transition."
combined node text	projected office savings from relocation financial estimate of reduced operational expenses due to the move needed decide primary goal lowering operating costs achievable cost reduction justifies transition

Semantic token analysis

Metric	Value
userSupportedMeaning tokens (filtered)	decision, requires, evidence, projected, office, savings, are, realistic, key, engineers, will, not, materially, leave, due, move
node tokens (filtered)	projected, office, savings, relocation, financial, estimate, reduced, operational, expenses, due, move, needed, decide, primary, goal, lowering, operating, costs, achievable, cost, reduction, justifies, transition
shared tokens	projected, office, savings, due, move
overlap ratio (shared / candidate)	0.217 (5/23)
absolute overlap count	5
expected rawAnswerSupportsUnclassifiedMeaning result	PASS (overlapRatio=0.217 < 0.4 BUT overlappingTokens=5 >= 3)

RETENTION UNKNOWN

Field	Value
id	n-retention-risk
label	"Risk of key engineer turnover due to relocation"
description	"Potential increase in voluntary departure of critical staff following the move, so that workforce stability and project continuity are not compromised despite financial gains, matters because retaining engineering talent is prerequisite to sustaining output."
combined node text	risk of key engineer turnover due to relocation potential increase in voluntary departure of critical staff following the move so that workforce stability and project continuity are not compromised despite financial gains matters because retaining engineering talent prerequisite sustaining output

Semantic token analysis

Metric	Value
userSupportedMeaning tokens (filtered)	decision, requires, evidence, projected, office, savings, are, realistic, key, engineers, will, not, materially, leave, due, move
node tokens (filtered)	risk, key, engineer, turnover, due, relocation, potential, increase, voluntary, departure, critical, staff, following, move, workforce, stability, project, continuity, are, not, compromised, despite, financial, gains, matters, retaining, engineering, talent, prerequisite, sustaining, output
shared tokens	key, due, move, are, not
overlap ratio (shared / candidate)	0.161 (5/31)
absolute overlap count	5
expected rawAnswerSupportsUnclassifiedMeaning result	PASS (overlapRatio=0.161 < 0.4 BUT overlappingTokens=5 >= 3)

Structural fallback result

Both nodes have depends_on edges to the central state node (n8g9g4v). The structural fallback gate (hasExplicitAnswerDerivedRelationship) checks for explicit graph linkage between the new unknown and answer-derived nodes from startCase. Both nodes satisfy this via their depends_on wiring.

• savings structural fallback: PASS
• retention structural fallback: PASS

Gate behavior verification

The v0.15 gate is: !hasNodeLevelUserSupport(unknownNode) && !hasExplicitAnswerDerivedRelationship(unknownNode). Both conditions must be true for rejection. In the live run, neither condition was triggered — both nodes passed at least one sub-gate (in fact both passed the semantic gate first).

Classification: C — GATE BEHAVES AS EXPECTED

Both proposed unknowns were admitted with HTTP 200 at update_applied, zero validation errors. The live semantic inputs explain the outcome fully:

1. userSupportedMeaning contains both savings and retention targets semantically — no answer-meaning loss (rules out A).
2. Semantic gate passes for both nodes via the token-count clause (5 shared tokens >= 3 threshold) despite overlap ratios below 0.4 (rules out B).
3. Live admission outcome matches expected helper result for both nodes (PASS/PASS → admitted/admitted) (confirms C).
4. No code-path mismatch observed: static helper evaluation and actual hasNodeLevelUserSupport agree (rules out D).
5. answerMeaning is populated with a valid userSupportedMeaning string — semantic path is available, not unavailable (rules out E).

What this experiment established

1. On a fresh live run through v0.15, the answer meaning gate correctly captures both savings-realism and retention-impact dimensions from a single conjunction-rich user answer.
2. The token-count clause of rawAnswerSupportsUnclassifiedMeaning (>= 3 shared content tokens) is the operative mechanism for this case — overlap ratios alone (0.16–0.22) would not suffice, but absolute token matches do.
3. Structural fallback edges (depends_on to the central state node) exist and are valid as a secondary admission path, confirming that both layers work correctly when activated.

What this experiment does NOT prove

1. That the semantic gate passes for answers where shared tokens fall below 3 (e.g., paraphrased savings language).
2. That downstream investigation (Update 2+) proceeds without new failures at a different boundary.
3. That admission stability holds across repeated runs (7→9 start nodes variance was already observed in 57J.26).
4. That the same token-count mechanism works for cross-domain answers with no vocabulary overlap.

Production code changed: NO

Prompt changed: NO

Schema changed: NO

Temporary instrumentation location

scripts/reproduce-multi-turn-investigation.mjs (temporary additions only, now removed)

Temporary instrumentation removed: YES

Retries: 0

Ollama calls beyond budget: 0

Documentation updated

• docs/experiment-57j28.md — created
• docs/current-handoff.md — appended below

---