confidence-engine/docs/experiment-60b63.md

# Experiment 60B.63 — Closure Confirmation Signal Source

## Status: PASSED (design-only, no production code changes)

### Objective
Determine exactly what is the safest existing deterministic signal for explicit user confirmation that no other material uncertainty remains: the raw user answer, model-generated meaning/reason text, or a combination thereof.

**Answer: RAW USER ANSWER should own confirmation — via Candidate A (RAW ANSWER ONLY) with a narrow bounded phrase-family matcher.**

---

## Pre-check Confirmations

- Branch: `feature/decision-sufficiency-v0.42`
- Working tree: clean
- HEAD includes: `5ef2b5a`, `100dfa2`, `7ee9b19` ✓

---

## RAW ANSWER — Checkpoint 1

**Available post-propagation:** YES

The `answer` parameter is a direct function argument at line 3485 of `applyValidatedProposal`. It flows through the entire function scope as an unchanged string. At Candidate D (post-propagation, ~line 3770+), it is still in scope as the original `answer` variable.

**Unchanged user input:** YES — no sanitisation, normalisation, or model transformation has been applied to this parameter between reception at line 3481 and any downstream read.

**Requires model interpretation:** NO — it is the raw literal string the user typed/said.

**Exact variable/argument:** `answer` (parameter of `applyValidatedProposal`, available as a local variable throughout the function scope).

---

## EXISTING TEXT HANDLING — Checkpoint 2

### Bounded raw-answer matcher exists: NO

There is no existing helper that detects confirmation, sufficiency, or "no other material uncertainty" patterns in any text source (raw answer or model output). The 60B.56 test at line 4530 of `apply-proposal.test.js` shows the phrase *"With customer signing confirmed and no other material uncertainties remaining, the decision is closed."* appearing in a `reason` string — but this is test fixture data, not an existing detection helper.

### Reusable normalisation helper: PARTIAL

Three bounded normalisation helpers exist in `apply-proposal.js`:

1. **`normaliseText(value)`** (line 54): lowercases, strips non-alphanumeric, replaces runs with single space. Very aggressive tokenisation — destroys phrase structure.
2. **`normaliseSemanticText(value)`** (line 3001): lowercases, normalises whitespace. Preserves words but loses punctuation cues.
3. **`normalise(value)` in evidence-direction.js** (line 74): simply `.toLowerCase()`. Minimal.

None of these are *semantic detectors* — they are preprocessors for downstream matching. The `answerConfirmsComparability` function (line 2989) demonstrates an existing bounded matcher pattern: it applies `normaliseSemanticText`, then checks for `"yes"` plus specific phrase inclusions using regex and `.includes()`. This is the closest precedent for a confirmation detector.

### Existing deterministic raw-answer precedent: PARTIAL

Several functions demonstrate bounded phrase-family detection on text derived from answers:

- **`deriveAnswerMeaningProfile`** (line 3178): detects `"not sure"`, `"unsure"`, `"matters more"`, `"hard constraint"`, etc. via `.includes()` chains — but this operates on `userSupportedMeaning` (model-extracted), not raw answer.
- **`hasConditionalQualification`** (line 3072): detects `"might"`, `"depends"`, `"conditional"` etc. — same source limitation.
- **`containsConstraintBoundaryLanguage`** (line 3084): detects `"constraint"`, `"non-negotiable"`, `"preference"` etc. — same source.
- **`rawAnswerSupportsUnclassifiedMeaning`** (line 3063): uses `semanticOverlapRatio` between raw answer and model meaning for cross-validation — this IS raw-answer but is a semantic similarity check, not deterministic phrase detection.

No existing helper performs deterministic confirmation sufficiency detection on any text source.

---

## RAW-ANSWER CANDIDATE — Checkpoint 3

Proposed narrow policy: explicit confirmation only when raw user answer directly contains a bounded statement equivalent to *"no other material uncertainty remains"*.

| Criterion | Rating | Reasoning |
|-----------|--------|-----------|
| User-grounding | **HIGH** | Direct literal user words, zero model mediation |
| Model dependence | **LOW** | Pure regex/string match; no inference |
| False-positive risk | **MEDIUM** | A bounded phrase family could catch non-confirmations if too broad (e.g., "no other material issue I know of" in a different context). Exact-match-only would be very low but is overly restrictive. |
| False-negative risk | **HIGH** | The 60B.56 reference answer uses *"There are no other material uncertainties between launching this year and waiting twelve months."* — the phrase family would need to match both singular and plural ("uncertainty"/"uncertainties"), prepositions ("between X and Y"/implicit), and related synonyms ("differences"/"residuals"/"remaining"). |
| Deterministic | **YES** | Regex/string matching is deterministic by nature |
| Schema change | **NO** | Uses existing `answer` parameter |

**Principal weakness:** The 60B.56 answer's confirmation clause ("There are no other material uncertainties between launching this year and waiting twelve months.") uses a long, context-specific construction with the prepositional phrase "between X and Y" as part of the uncertainty scope. A narrow phrase family like `["no other material", "uncertainties? (?: remain|remains)"]` would match this but could be brittle — different users will use many constructions ("I don't see anything else uncertain", "everything's settled", "that's it", etc.). The breadth needed for low false-negative rate increases the risk that the pattern becomes too broad to be truly deterministic.

---

## USER-SUPPORTED MEANING — Checkpoint 4

Assessing `validatedProposal.answerMeaning.userSupportedMeaning`:

| Criterion | Rating | Reasoning |
|-----------|--------|-----------|
| Directly grounded in answer | **PARTIAL** | It is derived FROM the answer but is model-extracted meaning, not the user's words. The model may add, remove, or paraphrase content during extraction. |
| Model generated | **YES** | LLM determines its exact content |
| Can model omit qualification | **YES** | Unproven guarantee — 60B.56 showed the model can fail to produce critical closure language (which is exactly why this experiment exists). If it can miss parent-resolution in 60B.56, there is no basis for assuming it will always include "no remaining uncertainty" in userSupportedMeaning. |
| Can model paraphrase correctly | **NO** | Cannot guarantee — the model might express sufficiency as "all resolved", "everything settled", "sufficient to decide", etc., each requiring different detection logic. This defeats deterministic matching. |
| Suitable as closure owner | **NO** | Model-generated content cannot be deterministically trusted for a binary structural gate that controls system state mutation. |

---

## PARENT REASON — Checkpoint 5

Assessing `updatedNodes[].reason` on the parent decision node:

| Criterion | Rating | Reasoning |
|-----------|--------|-----------|
| Model generated | **YES** | Produced by LLM in response to prompt instructions |
| Guaranteed to exist | **CONDITIONAL** | It is standard output for every node update, but could be missing if the model returns malformed proposal (e.g., empty reason). The 60B.56 case shows it exists — but that's one data point. |
| Guaranteed parent-targeted | **NO** | Must search `updatedNodes[]` by node ID; not guaranteed to be present without iteration. |
| Could reintroduce model-compliance failure (60B.56) | **YES** | **CRITICAL** — 60B.56's entire finding was that the LLM produced correct factor resolution but *failed to close the parent decision*. Relying on `updatedNodes[].reason` for closure confirmation would be using the exact same model output channel that 60B.56 proved unreliable. If the model can miss parent closure in one context, there is no theoretical basis for assuming it will reliably emit sufficiency language in another. |

---

## CANDIDATE ASSESSMENT — Checkpoint 6

### Candidate A — RAW ANSWER ONLY
```
graph helper=false AND narrow raw-answer confirmation => close
```
| Criterion | Rating |
|-----------|--------|
| Fixes 60B.56 | **YES** — the user explicitly wrote "There are no other material uncertainties..." in their answer; bounded detection on this literal text is deterministic |
| User grounding | **HIGH** — direct user words, zero mediation |
| Model dependence | **LOW** — pure text matching |
| False-positive risk | **MEDIUM** — depends on phrase family breadth. Exact matches: very low. Family of 4-6 phrases: medium but acceptable with careful curation. |
| False-negative risk | **MEDIUM-HIGH** — users will use varied constructions. A bounded family of 4-6 phrases catches the reference case but misses others. This is inherent to raw-text matching and cannot be eliminated without model help (which defeats the point). |
| Schema change | **NO** |
| Principal weakness | **Bounded phrase families for "no remaining uncertainty" are inherently narrow in coverage.** Users express this concept in many ways. The breadth needed for low false-negative rate increases false-positive risk, creating a tension that bounded regex alone cannot fully resolve. |

### Candidate B — USER-SUPPORTED MEANING ONLY
| Criterion | Rating |
|-----------|--------|
| Fixes 60B.56 | **CONDITIONAL** — only if the model happened to include sufficiency language in userSupportedMeaning, which is unproven |
| User grounding | **MEDIUM** — derived from answer but model-filtered |
| Model dependence | **HIGH** — entirely depends on model output |
| False-positive risk | **LOW-MEDIUM** — false positives are unlikely because the pattern would be in model-generated text; if it's there, the model intended it. But this is a different kind of risk: what if the model includes sufficiency language without user having stated it? |
| False-negative risk | **HIGH** — unproven whether the model will always include sufficiency phrasing |
| Schema change | **NO** |
| Principal weakness | **Cannot guarantee presence or absence of sufficiency language.** Exactly the failure mode 60B.56 documented. |

### Candidate C — PARENT REASON ONLY
| Criterion | Rating |
|-----------|--------|
| Fixes 60B.56 | **CONDITIONAL** — only if reason contains explicit closure language (the 60B.56 proposal does, but the prompt doesn't guarantee it) |
| User grounding | **LOW** — model-extracted rationale, not user words |
| Model dependence | **HIGH** |
| False-positive risk | **LOW-MEDIUM** |
| False-negative risk | **HIGH** |
| Schema change | **NO** |
| Principal weakness | **Relies on the exact same model output channel that 60B.56 proved fails.** If the LLM can fail to close a parent decision in one case, there is no basis for assuming it will reliably emit sufficiency confirmation in another. |

### Candidate D — RAW ANSWER OR USER-SUPPORTED MEANING
```
graph helper=false AND either direct user wording OR faithful model-normalised meaning explicitly confirms => close
```
| Criterion | Rating |
|-----------|--------|
| Fixes 60B.56 | **YES** — raw answer matches; model meaning may or may not match (OR makes it succeed) |
| User grounding | **HIGH** — primary signal is user words |
| Model dependence | **MEDIUM** — OR condition means if raw answer doesn't match but model meaning does, we close. This lowers false-negative rate but introduces partial model dependence. |
| False-positive risk | **LOW-MEDIUM** — lower than A alone because the model's confirmation language acts as a cross-check (if both agree, very low FP risk; if only model agrees, medium) |
| False-negative risk | **MEDIUM-LOW** — significantly reduced by OR condition. Catches cases where user phrasing doesn't match the bounded family but model meaning does. |
| Schema change | **NO** |
| Principal weakness | **The OR condition means closure can fire based on model-generated text alone (when raw answer doesn't match). This partially reintroduces 60B.56's failure mode: we close because a model said "sufficient" when the user didn't actually state it.** The risk is lower than pure model-based approaches but is not eliminated. |

### Candidate E — RAW ANSWER AND MODEL CONFIRMATION
```
graph helper=false AND both raw answer AND model confirmation present => close
```
| Criterion | Rating |
|-----------|--------|
| Fixes 60B.56 | **CONDITIONAL** — requires BOTH to match. If model omits confirmation (as in 60B.56), closure doesn't fire even though user confirmed it. This is the exact opposite failure mode from 60B.56: delayed rather than premature. |
| User grounding | **HIGH** — user words required |
| Model dependence | **MEDIUM-HIGH** — model must also produce confirmation text, meaning a model omission blocks closure even when user confirmed it |
| False-positive risk | **VERY LOW** — both signals must agree; extremely unlikely for false positives |
| False-negative risk | **VERY HIGH** — any one signal missing prevents closure. User didn't phrase it right? No closure. Model omitted confirmation text? No closure. Both can happen simultaneously. |
| Schema change | **NO** |
| Principal weakness | **Reintroduces model dependence for a signal that shouldn't need it.** If the user explicitly confirmed "no other material uncertainties remain" in their answer but the model didn't echo it in userSupportedMeaning or reason, closure is blocked. This violates criterion 3 (model omission must not prevent closure when user explicitly confirmed). |

---

## NO-CONFIRMATION CASES — Checkpoint 7

### Case 1 — Factor resolves but user does NOT say "no uncertainty remains"

**Source:** `apply-proposal.test.js` line 4563+ (test: "discards a proposal-selected target that becomes known and falls back to another genuine unresolved candidate"). The test fixture at line 4578 uses reason: *"The active customer-signing uncertainty is resolved."* — no sufficiency language.

If the raw user answer were something like *"Customer signing confirmed"* (without any "no other" clause), a bounded confirmation matcher on raw text would return `false`. The decision remains open (correct).

**Confirmation result: `false`** — correctly keeps decision open because user did not state sufficiency.

### Case 2 — User says uncertainty remains elsewhere

Hypothetical answer shape from the same 60B.56 scenario: *"The enterprise customer has confirmed signing, but I'm still unsure about regulatory approval timing."*

A bounded confirmation matcher looking for "no other material" patterns would not match this text. The decision correctly remains open because uncertainty explicitly remains.

**Confirmation result: `false`** — correctly keeps decision open because user stated remaining uncertainty.

Both cases demonstrate that a raw-answer-only bounded approach correctly returns `confirmation = false`.

---

## PARAPHRASE TOLERANCE — Checkpoint 8

Assessed phrase family options for bounded detection of *"no other material uncertainty remains"*:

**Choice: B — SMALL BOUNDED PHRASE FAMILY**

A narrow family of 4-6 canonical phrases is recommended. Examples:
- `/\bno (?:other|further) material (uncertainties?|differences?)\b/`
- `/\bno (?:other|remaining) uncertainty\s+(?:remains?|left)\b/`
- `/\bnothing (?:else )?material is uncertain\b/`

This balances:
- **Low false-positive risk:** each phrase contains multiple content words that jointly confirm sufficiency intent ("no" + "material" + "uncertainty")
- **Manageable false-negative rate:** catches the reference case and its grammatical variants (singular/plural, "other"/"remaining", present/absent forms)
- **Deterministic:** exact regex/string matching
- **No schema change**

Choice A (exact phrase only) has unacceptably high false-negative risk. Choice C (model normalisation) reintroduces the 60B.56 model-compliance dependency. Choice D (raw text unsafe) is overly conservative — bounded phrase families have worked elsewhere in the codebase (see `deriveAnswerMeaningProfile`, `answerConfirmsComparability`).

---

## CRITICAL DISTINCTION — Checkpoint Final

**Choice: A — RAW USER ANSWER SHOULD OWN CONFIRMATION**

**Why:** The raw user answer is the only existing signal that satisfies ALL seven decision criteria simultaneously:

1. **60B.56 can close** ✓ — user wrote "There are no other material uncertainties..." in their answer; bounded detection catches it
2. **User meaning remains primary** ✓ — user words, not model interpretation
3. **Model omission does not prevent closure when user confirmed** ✓ — no model signal required; raw text is sufficient alone
4. **Model paraphrase does not create closure when user did not confirm** ✓ — model output is never the gate
5. **No schema change** ✓ — `answer` parameter already exists and flows through
6. **No broad NLP parsing** ✓ — bounded phrase family (~4-6 entries) using regex `.test()` or string `.includes()`
7. **No-confirmation cases remain open** ✓ — cases 1 and 2 correctly produce `confirmation = false`

Comparing against the rejected alternatives:
- **B (userSupportedMeaning)** violates criterion 3 (model omission blocks closure) and criterion 4 (model paraphrase may not be matchable).
- **C (parent reason)** is the exact same model-compliance channel that failed in 60B.56 — rejecting for this reason alone.
- **D (RAW + MODEL share)** partially violates criterion 3 because the OR path means closure can fire on model text alone when raw answer doesn't match.
- **E (current architecture lacks signal)** is false — we have `answer` parameter and existing bounded-matching precedents (`answerConfirmsComparability`, `deriveAnswerMeaningProfile`).
- **F (one more design question)** is not needed — the decision criteria uniquely identify raw answer as the correct signal.

---

## MINIMUM CORRECTIVE BOUNDARY

**Choice: A — add narrow raw-answer confirmation helper**

**Why:** The only missing piece is a bounded phrase-family detector on the `answer` parameter. This requires:
- 1 new helper function (bounded regex/array of `.includes()` checks)
- 0 schema changes
- 0 prompt changes
- 0 production mutation logic changes (the integration point was already identified in 60B.62)

No other approach satisfies all seven criteria with lower corrective boundary.

---

## VERIFICATION AGAINST DECISION CRITERIA

| Criterion | Status | Mechanism |
|-----------|--------|-----------|
| 1. 60B.56 can close | YES | Raw answer contains "no other material uncertainties"; bounded family matches it |
| 2. User meaning remains primary | YES | Raw text is the sole confirmation signal; model output is never consulted for confirmation |
| 3. Model omission does not prevent closure | YES | No model signal required; user words alone are sufficient |
| 4. Model paraphrase does not create closure | YES | Only raw answer is checked; model output is irrelevant to confirmation gate |
| 5. No schema change | YES | `answer` parameter flows through existing function signature |
| 6. No broad NLP parsing | YES | Bounded phrase family (~4-6 entries) using regex or `.includes()` chains |
| 7. No recommendation/direction inference | YES | Confirmation detects "no remaining uncertainty" only — no option preference is inferred |
| 8. No-confirmation cases remain open | YES | Case 1 (factor resolves, no sufficiency statement) → false; Case 2 (uncertainty stated) → false |

---

## IMPLEMENTATION READINESS

**Choice: A — READY FOR BOUNDED IMPLEMENTATION**

One unresolved question only at the implementation layer: determining the precise phrase family breadth. The boundary between "narrow enough for low FP risk" and "broad enough for acceptable FN rate" is a design detail, not a structural design question.

The exact phrase family can be derived from:
1. The 60B.56 reference answer (canonical source)
2. Standard English constructions for expressing sufficiency of remaining factors
3. Existing precedent in `deriveAnswerMeaningProfile` and `answerConfirmsComparability`

**Smallest implementation boundary:**
```
1 new helper: isUserConfirmationOfNoRemainingUncertainty(answer) => boolean
  - normaliseSemanticText(answer)
  - check against bounded phrase family array (4-6 entries)
1 integration at Candidate D (post-propagation, ~3 lines):
  if (hasRemainingMaterialFactors(decisionId, graph) === false && isUserConfirmationOfNoRemainingUncertainty(answer)) { /* close */ }
0 schema changes
0 prompt changes
0 test changes needed for this experiment (design-only)
```

---

## PRODUCTION CODE CHANGED: NO
## TESTS CHANGED: NO
## PROMPT CHANGED: NO
## SCHEMA CHANGED: NO
## OLLAMA CALLS: 0
## LIVE API CALLS: 0
## VITEST RUN: NO
## JEST RUN: NO
## WATCHMAN USED: NO