experiment: diagnose uncertainty fidelity false positive

docs/current-handoff.md

@@ -846,18 +846,57 @@ Configured Ollama: none used. Production code changed: prompt + tests only. Dev
 **UPDATE 1:** HTTP 422 | stage: proposal_compatibility | error: "answerMeaning.userSupportedMeaning overstates a raw answer that remains uncertain."
 
 rejectedProposalSnapshot:
-- userSupportedMeaning: "The user is currently uncertain whether the projected office savings from the relocation are realistic." (STRENGTHENED — third-person assertion of mental state + temporal specificity "currently" beyond first-person "I am unsure")
+- userSupportedMeaning: "The user is currently uncertain whether the projected office savings from the relocation are realistic." (HUMAN ASSESSMENT: minor non-material paraphrase — see 57J.48; VALIDATOR: lexically rejected because "uncertain" absent from deriveAnswerMeaningProfile detection patterns)
 - possibleInference: null
 - addedNodes: [{id: "ns82kx9", kind: unknown, label: "Projected office savings from the relocation lack validation"}] (genuinely new — ADD NEW UNKNOWN structural action)
 - addedEdges: [{fromNodeId: "ns82kx9", toNodeId: "nv759rs", relationship: depends_on}]
 - updatedNodes: [] | resolvedUnknownNodeIds: []
 
-**Meaning classification:** STRENGTHENED (third-person assertion of user mental state + temporal specificity overstates first-person uncertainty)
+**Meaning classification:** LEXICAL MISMATCH in deterministic derivation (human assessment: minor non-material paraphrase; validator mechanism: "uncertain" absent from deriveAnswerMeaningProfile patterns while raw answer's "unsure" is detected)
 
 **Structural action:** ADD NEW UNKNOWN — a genuinely new unknown directly representing savings realism was created. This is the FIRST time in experiments 57J.36-47 that the model produced a meaningful structural mutation for this type of uncertainty answer.
 
-**Classification: D — STRENGTHENING BLOCKS TEST.** Meaning is strengthened; the test cannot establish v0.19's effectiveness for the semantic-to-mutation boundary. However, structural progress was achieved (one new unknown + one edge) — the faithful no-op pattern has been broken. The remaining blocker is meaning extraction (first-person uncertainty → third-person assertion), not the existing-first rule.
+**Classification: D — REJECTION BLOCKS TEST (LEXICAL FALSE POSITIVE).** The rejection was caused by `"uncertain"` being absent from `deriveAnswerMeaningProfile`'s uncertainty detection patterns (which include `"unsure"`, `"not sure"`, `"do not know"`, etc.), not by genuine semantic strengthening. Both words express identical uncertainty semantics. Structural progress was achieved (one new unknown + one edge) — the faithful no-op pattern has been broken. The remaining blocker is incomplete lexical coverage in the deterministic category model, not a meaning extraction defect.
 
-**Key finding:** v0.19's existing-first fallback now produces meaningful graph mutation for explicit unresolved uncertainty. New blocking boundary: first-person uncertainty statements are being converted to third-person assertions, caught by proposal_compatibility as semantic strengthening. Convergence test result is non-A; prompt tuning sequence terminates here per convergence rule.
+**Key finding:** v0.19's existing-first fallback now produces meaningful graph mutation for explicit unresolved uncertainty. New blocking boundary: semantically faithful paraphrases are being rejected because `deriveAnswerMeaningProfile` detects `"unsure"` but not `"uncertain"` (identical semantics). The guard's lexical gap causes false-positive "overstates a raw answer that remains uncertain" errors for semantically equivalent third-person uncertainty statements. Convergence test result is non-A; prompt tuning sequence terminates here per convergence rule. 57J.48 provides the full deterministic diagnosis.
 
 Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. 2 live calls total. No production code changed. No prompt change during experiment. Harness restored to original scenario/answers.
+
+### Experiment 57J.48 — Uncertainty Fidelity False Positive Diagnosis
+
+**Objective:** Determine whether the captured rejection of "The user is currently uncertain..." → rejected as overstatement when raw answer was "I am unsure..." is semantically justified or a lexical false positive. Fully deterministic — zero Ollama calls.
+
+**Captured pair:**
+- Raw answer: `"I am unsure whether the projected office savings from the relocation are realistic."`
+- userSupportedMeaning: `"The user is currently uncertain whether the projected office savings from the relocation are realistic."`
+
+**Deterministic trace:**
+1. `deriveAnswerMeaningProfile(raw)` → category=`uncertain` (matches pattern `"unsure"` at line 2880)
+2. `deriveAnswerMeaningProfile(userSupportedMeaning)` → category=`other` (pattern gap: `"uncertain"` not detected)
+3. `validateAnswerMeaningCompatibilityWithRawAnswer`: line 2966-2970 fires because `raw=uncertain` && `supported≠uncertain` → error: "overstates a raw answer that remains uncertain."
+
+**Inverse test (same semantics, different word):** Using `"unsure"` instead of `"uncertain"` in userSupportedMeaning → category=`uncertain`, errors=none. Both words express identical uncertainty.
+
+**Semantic equivalence check:**
+- Uncertainty preserved: YES
+- Decision condition added: NO
+- Hard constraint added: NO
+- Priority added: NO
+- Conclusion added: NO
+- "currently" material temporal claim: NEGLIGIBLE
+- Perspective shift: REPRESENTATIONAL NORMALISATION (not substantive)
+- Pair classification: MINOR NON-MATERIAL PARAPHRASE
+
+**Architecture classification:** B — LEXICAL FALSE POSITIVE
+
+The meanings are semantically equivalent; the rejection depends on `"unsure"` being in `deriveAnswerMeaningProfile`'s detection patterns while `"uncertain"` is not. Both denote "lacking sufficient knowledge or certainty."
+
+**Keyword-dictionary risk: YES.** The validator's semantic fidelity decision depends entirely on whether the LLM uses one of ~15-20 hardcoded English surface forms (`"unsure"`, `"not sure"`, etc.) rather than equivalent terms (`"uncertain"`). This is raw-language keyword detection, not structured semantic contract validation.
+
+**Existing structured signals: SUFFICIENT.** The schema already carries `supportCategory` (enum), `resolutionGuidance` (nullable string distinguishing "must_remain_unresolved"/"may_resolve"), and `possibleInference`. These fields exist in the production contract but are consistently null from the LLM (per 56D), forcing reliance on keyword inference.
+
+**57J.47 documentation correction applied:** Previous wording stated meaning was "STRENGTHENED" as fact. Corrected to distinguish: human semantic assessment (minor non-material paraphrase) from validator mechanism (lexical gap in detection patterns). The actual rejection was a false positive caused by incomplete keyword coverage, not genuine semantic strengthening.
+
+**Production code changed: NO. Prompt changed: NO. Validator changed: NO. Tests permanently changed: NO. Ollama calls: 0.**
+
+Configured Ollama: none (zero live calls).