confidence-engine/docs/archive/experiments/decision-closure-integration/experiment-60b63.md

Experiment 60B.63 — Closure Confirmation Signal Source

Status: PASSED (design-only, no production code changes)

Objective

Determine exactly what is the safest existing deterministic signal for explicit user confirmation that no other material uncertainty remains: the raw user answer, model-generated meaning/reason text, or a combination thereof.

Answer: RAW USER ANSWER should own confirmation — via Candidate A (RAW ANSWER ONLY) with a narrow bounded phrase-family matcher.

---

Pre-check Confirmations

• Branch: feature/decision-sufficiency-v0.42
• Working tree: clean
• HEAD includes: 5ef2b5a, 100dfa2, 7ee9b19 ✓

---

RAW ANSWER — Checkpoint 1

Available post-propagation: YES

The answer parameter is a direct function argument at line 3485 of applyValidatedProposal. It flows through the entire function scope as an unchanged string. At Candidate D (post-propagation, ~line 3770+), it is still in scope as the original answer variable.

Unchanged user input: YES — no sanitisation, normalisation, or model transformation has been applied to this parameter between reception at line 3481 and any downstream read.

Requires model interpretation: NO — it is the raw literal string the user typed/said.

Exact variable/argument: answer (parameter of applyValidatedProposal, available as a local variable throughout the function scope).

---

EXISTING TEXT HANDLING — Checkpoint 2

Bounded raw-answer matcher exists: NO

There is no existing helper that detects confirmation, sufficiency, or "no other material uncertainty" patterns in any text source (raw answer or model output). The 60B.56 test at line 4530 of apply-proposal.test.js shows the phrase "With customer signing confirmed and no other material uncertainties remaining, the decision is closed." appearing in a reason string — but this is test fixture data, not an existing detection helper.

Reusable normalisation helper: PARTIAL

Three bounded normalisation helpers exist in apply-proposal.js:

1. normaliseText(value) (line 54): lowercases, strips non-alphanumeric, replaces runs with single space. Very aggressive tokenisation — destroys phrase structure.
2. normaliseSemanticText(value) (line 3001): lowercases, normalises whitespace. Preserves words but loses punctuation cues.
3. normalise(value) in evidence-direction.js (line 74): simply .toLowerCase(). Minimal.

None of these are semantic detectors — they are preprocessors for downstream matching. The answerConfirmsComparability function (line 2989) demonstrates an existing bounded matcher pattern: it applies normaliseSemanticText, then checks for "yes" plus specific phrase inclusions using regex and .includes(). This is the closest precedent for a confirmation detector.

Existing deterministic raw-answer precedent: PARTIAL

Several functions demonstrate bounded phrase-family detection on text derived from answers:

• deriveAnswerMeaningProfile (line 3178): detects "not sure", "unsure", "matters more", "hard constraint", etc. via .includes() chains — but this operates on userSupportedMeaning (model-extracted), not raw answer.
• hasConditionalQualification (line 3072): detects "might", "depends", "conditional" etc. — same source limitation.
• containsConstraintBoundaryLanguage (line 3084): detects "constraint", "non-negotiable", "preference" etc. — same source.
• rawAnswerSupportsUnclassifiedMeaning (line 3063): uses semanticOverlapRatio between raw answer and model meaning for cross-validation — this IS raw-answer but is a semantic similarity check, not deterministic phrase detection.

No existing helper performs deterministic confirmation sufficiency detection on any text source.

---

RAW-ANSWER CANDIDATE — Checkpoint 3

Proposed narrow policy: explicit confirmation only when raw user answer directly contains a bounded statement equivalent to "no other material uncertainty remains".

Criterion	Rating	Reasoning
User-grounding	HIGH	Direct literal user words, zero model mediation
Model dependence	LOW	Pure regex/string match; no inference
False-positive risk	MEDIUM	A bounded phrase family could catch non-confirmations if too broad (e.g., "no other material issue I know of" in a different context). Exact-match-only would be very low but is overly restrictive.
False-negative risk	HIGH	The 60B.56 reference answer uses "There are no other material uncertainties between launching this year and waiting twelve months." — the phrase family would need to match both singular and plural ("uncertainty"/"uncertainties"), prepositions ("between X and Y"/implicit), and related synonyms ("differences"/"residuals"/"remaining").
Deterministic	YES	Regex/string matching is deterministic by nature
Schema change	NO	Uses existing answer parameter

Principal weakness: The 60B.56 answer's confirmation clause ("There are no other material uncertainties between launching this year and waiting twelve months.") uses a long, context-specific construction with the prepositional phrase "between X and Y" as part of the uncertainty scope. A narrow phrase family like ["no other material", "uncertainties? (?: remain|remains)"] would match this but could be brittle — different users will use many constructions ("I don't see anything else uncertain", "everything's settled", "that's it", etc.). The breadth needed for low false-negative rate increases the risk that the pattern becomes too broad to be truly deterministic.

---

USER-SUPPORTED MEANING — Checkpoint 4

Assessing validatedProposal.answerMeaning.userSupportedMeaning:

Criterion	Rating	Reasoning
Directly grounded in answer	PARTIAL	It is derived FROM the answer but is model-extracted meaning, not the user's words. The model may add, remove, or paraphrase content during extraction.
Model generated	YES	LLM determines its exact content
Can model omit qualification	YES	Unproven guarantee — 60B.56 showed the model can fail to produce critical closure language (which is exactly why this experiment exists). If it can miss parent-resolution in 60B.56, there is no basis for assuming it will always include "no remaining uncertainty" in userSupportedMeaning.
Can model paraphrase correctly	NO	Cannot guarantee — the model might express sufficiency as "all resolved", "everything settled", "sufficient to decide", etc., each requiring different detection logic. This defeats deterministic matching.
Suitable as closure owner	NO	Model-generated content cannot be deterministically trusted for a binary structural gate that controls system state mutation.

---

PARENT REASON — Checkpoint 5

Assessing updatedNodes[].reason on the parent decision node:

Criterion	Rating	Reasoning
Model generated	YES	Produced by LLM in response to prompt instructions
Guaranteed to exist	CONDITIONAL	It is standard output for every node update, but could be missing if the model returns malformed proposal (e.g., empty reason). The 60B.56 case shows it exists — but that's one data point.
Guaranteed parent-targeted	NO	Must search updatedNodes[] by node ID; not guaranteed to be present without iteration.
Could reintroduce model-compliance failure (60B.56)	YES	CRITICAL — 60B.56's entire finding was that the LLM produced correct factor resolution but failed to close the parent decision. Relying on updatedNodes[].reason for closure confirmation would be using the exact same model output channel that 60B.56 proved unreliable. If the model can miss parent closure in one context, there is no theoretical basis for assuming it will reliably emit sufficiency language in another.

---

CANDIDATE ASSESSMENT — Checkpoint 6

Candidate A — RAW ANSWER ONLY

graph helper=false AND narrow raw-answer confirmation => close

Criterion	Rating
Fixes 60B.56	YES — the user explicitly wrote "There are no other material uncertainties..." in their answer; bounded detection on this literal text is deterministic
User grounding	HIGH — direct user words, zero mediation
Model dependence	LOW — pure text matching
False-positive risk	MEDIUM — depends on phrase family breadth. Exact matches: very low. Family of 4-6 phrases: medium but acceptable with careful curation.
False-negative risk	MEDIUM-HIGH — users will use varied constructions. A bounded family of 4-6 phrases catches the reference case but misses others. This is inherent to raw-text matching and cannot be eliminated without model help (which defeats the point).
Schema change	NO
Principal weakness	Bounded phrase families for "no remaining uncertainty" are inherently narrow in coverage. Users express this concept in many ways. The breadth needed for low false-negative rate increases false-positive risk, creating a tension that bounded regex alone cannot fully resolve.

Candidate B — USER-SUPPORTED MEANING ONLY

Criterion	Rating
Fixes 60B.56	CONDITIONAL — only if the model happened to include sufficiency language in userSupportedMeaning, which is unproven
User grounding	MEDIUM — derived from answer but model-filtered
Model dependence	HIGH — entirely depends on model output
False-positive risk	LOW-MEDIUM — false positives are unlikely because the pattern would be in model-generated text; if it's there, the model intended it. But this is a different kind of risk: what if the model includes sufficiency language without user having stated it?
False-negative risk	HIGH — unproven whether the model will always include sufficiency phrasing
Schema change	NO
Principal weakness	Cannot guarantee presence or absence of sufficiency language. Exactly the failure mode 60B.56 documented.

Candidate C — PARENT REASON ONLY

Criterion	Rating
Fixes 60B.56	CONDITIONAL — only if reason contains explicit closure language (the 60B.56 proposal does, but the prompt doesn't guarantee it)
User grounding	LOW — model-extracted rationale, not user words
Model dependence	HIGH
False-positive risk	LOW-MEDIUM
False-negative risk	HIGH
Schema change	NO
Principal weakness	Relies on the exact same model output channel that 60B.56 proved fails. If the LLM can fail to close a parent decision in one case, there is no basis for assuming it will reliably emit sufficiency confirmation in another.

Candidate D — RAW ANSWER OR USER-SUPPORTED MEANING

graph helper=false AND either direct user wording OR faithful model-normalised meaning explicitly confirms => close

Criterion	Rating
Fixes 60B.56	YES — raw answer matches; model meaning may or may not match (OR makes it succeed)
User grounding	HIGH — primary signal is user words
Model dependence	MEDIUM — OR condition means if raw answer doesn't match but model meaning does, we close. This lowers false-negative rate but introduces partial model dependence.
False-positive risk	LOW-MEDIUM — lower than A alone because the model's confirmation language acts as a cross-check (if both agree, very low FP risk; if only model agrees, medium)
False-negative risk	MEDIUM-LOW — significantly reduced by OR condition. Catches cases where user phrasing doesn't match the bounded family but model meaning does.
Schema change	NO
Principal weakness	The OR condition means closure can fire based on model-generated text alone (when raw answer doesn't match). This partially reintroduces 60B.56's failure mode: we close because a model said "sufficient" when the user didn't actually state it. The risk is lower than pure model-based approaches but is not eliminated.

Candidate E — RAW ANSWER AND MODEL CONFIRMATION

graph helper=false AND both raw answer AND model confirmation present => close

Criterion	Rating
Fixes 60B.56	CONDITIONAL — requires BOTH to match. If model omits confirmation (as in 60B.56), closure doesn't fire even though user confirmed it. This is the exact opposite failure mode from 60B.56: delayed rather than premature.
User grounding	HIGH — user words required
Model dependence	MEDIUM-HIGH — model must also produce confirmation text, meaning a model omission blocks closure even when user confirmed it
False-positive risk	VERY LOW — both signals must agree; extremely unlikely for false positives
False-negative risk	VERY HIGH — any one signal missing prevents closure. User didn't phrase it right? No closure. Model omitted confirmation text? No closure. Both can happen simultaneously.
Schema change	NO
Principal weakness	Reintroduces model dependence for a signal that shouldn't need it. If the user explicitly confirmed "no other material uncertainties remain" in their answer but the model didn't echo it in userSupportedMeaning or reason, closure is blocked. This violates criterion 3 (model omission must not prevent closure when user explicitly confirmed).

---

NO-CONFIRMATION CASES — Checkpoint 7

Case 1 — Factor resolves but user does NOT say "no uncertainty remains"

Source: apply-proposal.test.js line 4563+ (test: "discards a proposal-selected target that becomes known and falls back to another genuine unresolved candidate"). The test fixture at line 4578 uses reason: "The active customer-signing uncertainty is resolved." — no sufficiency language.

If the raw user answer were something like "Customer signing confirmed" (without any "no other" clause), a bounded confirmation matcher on raw text would return false. The decision remains open (correct).

Confirmation result: false — correctly keeps decision open because user did not state sufficiency.

Case 2 — User says uncertainty remains elsewhere

Hypothetical answer shape from the same 60B.56 scenario: "The enterprise customer has confirmed signing, but I'm still unsure about regulatory approval timing."

A bounded confirmation matcher looking for "no other material" patterns would not match this text. The decision correctly remains open because uncertainty explicitly remains.

Confirmation result: false — correctly keeps decision open because user stated remaining uncertainty.

Both cases demonstrate that a raw-answer-only bounded approach correctly returns confirmation = false.

---

PARAPHRASE TOLERANCE — Checkpoint 8

Assessed phrase family options for bounded detection of "no other material uncertainty remains":

Choice: B — SMALL BOUNDED PHRASE FAMILY

A narrow family of 4-6 canonical phrases is recommended. Examples:

• /\bno (?:other|further) material (uncertainties?|differences?)\b/
• /\bno (?:other|remaining) uncertainty\s+(?:remains?|left)\b/
• /\bnothing (?:else )?material is uncertain\b/

This balances:

• Low false-positive risk: each phrase contains multiple content words that jointly confirm sufficiency intent ("no" + "material" + "uncertainty")
• Manageable false-negative rate: catches the reference case and its grammatical variants (singular/plural, "other"/"remaining", present/absent forms)
• Deterministic: exact regex/string matching
• No schema change

Choice A (exact phrase only) has unacceptably high false-negative risk. Choice C (model normalisation) reintroduces the 60B.56 model-compliance dependency. Choice D (raw text unsafe) is overly conservative — bounded phrase families have worked elsewhere in the codebase (see deriveAnswerMeaningProfile, answerConfirmsComparability).

---

CRITICAL DISTINCTION — Checkpoint Final

Choice: A — RAW USER ANSWER SHOULD OWN CONFIRMATION

Why: The raw user answer is the only existing signal that satisfies ALL seven decision criteria simultaneously:

1. 60B.56 can close ✓ — user wrote "There are no other material uncertainties..." in their answer; bounded detection catches it
2. User meaning remains primary ✓ — user words, not model interpretation
3. Model omission does not prevent closure when user confirmed ✓ — no model signal required; raw text is sufficient alone
4. Model paraphrase does not create closure when user did not confirm ✓ — model output is never the gate
5. No schema change ✓ — answer parameter already exists and flows through
6. No broad NLP parsing ✓ — bounded phrase family (~4-6 entries) using regex .test() or string .includes()
7. No-confirmation cases remain open ✓ — cases 1 and 2 correctly produce confirmation = false

Comparing against the rejected alternatives:

• B (userSupportedMeaning) violates criterion 3 (model omission blocks closure) and criterion 4 (model paraphrase may not be matchable).
• C (parent reason) is the exact same model-compliance channel that failed in 60B.56 — rejecting for this reason alone.
• D (RAW + MODEL share) partially violates criterion 3 because the OR path means closure can fire on model text alone when raw answer doesn't match.
• E (current architecture lacks signal) is false — we have answer parameter and existing bounded-matching precedents (answerConfirmsComparability, deriveAnswerMeaningProfile).
• F (one more design question) is not needed — the decision criteria uniquely identify raw answer as the correct signal.

---

MINIMUM CORRECTIVE BOUNDARY

Choice: A — add narrow raw-answer confirmation helper

Why: The only missing piece is a bounded phrase-family detector on the answer parameter. This requires:

• 1 new helper function (bounded regex/array of .includes() checks)
• 0 schema changes
• 0 prompt changes
• 0 production mutation logic changes (the integration point was already identified in 60B.62)

No other approach satisfies all seven criteria with lower corrective boundary.

---

VERIFICATION AGAINST DECISION CRITERIA

Criterion	Status	Mechanism
1. 60B.56 can close	YES	Raw answer contains "no other material uncertainties"; bounded family matches it
2. User meaning remains primary	YES	Raw text is the sole confirmation signal; model output is never consulted for confirmation
3. Model omission does not prevent closure	YES	No model signal required; user words alone are sufficient
4. Model paraphrase does not create closure	YES	Only raw answer is checked; model output is irrelevant to confirmation gate
5. No schema change	YES	answer parameter flows through existing function signature
6. No broad NLP parsing	YES	Bounded phrase family (~4-6 entries) using regex or .includes() chains
7. No recommendation/direction inference	YES	Confirmation detects "no remaining uncertainty" only — no option preference is inferred
8. No-confirmation cases remain open	YES	Case 1 (factor resolves, no sufficiency statement) → false; Case 2 (uncertainty stated) → false

---

IMPLEMENTATION READINESS

Choice: A — READY FOR BOUNDED IMPLEMENTATION

One unresolved question only at the implementation layer: determining the precise phrase family breadth. The boundary between "narrow enough for low FP risk" and "broad enough for acceptable FN rate" is a design detail, not a structural design question.

The exact phrase family can be derived from:

1. The 60B.56 reference answer (canonical source)
2. Standard English constructions for expressing sufficiency of remaining factors
3. Existing precedent in deriveAnswerMeaningProfile and answerConfirmsComparability

Smallest implementation boundary:

1 new helper: isUserConfirmationOfNoRemainingUncertainty(answer) => boolean
  - normaliseSemanticText(answer)
  - check against bounded phrase family array (4-6 entries)
1 integration at Candidate D (post-propagation, ~3 lines):
  if (hasRemainingMaterialFactors(decisionId, graph) === false && isUserConfirmationOfNoRemainingUncertainty(answer)) { /* close */ }
0 schema changes
0 prompt changes
0 test changes needed for this experiment (design-only)

---

PRODUCTION CODE CHANGED: NO

TESTS CHANGED: NO

PROMPT CHANGED: NO

SCHEMA CHANGED: NO

OLLAMA CALLS: 0

LIVE API CALLS: 0

VITEST RUN: NO

JEST RUN: NO

WATCHMAN USED: NO