experiment: define closure confirmation signal

This commit is contained in:
2026-08-14 13:55:41 +01:00
parent 7ee9b197ab
commit 02b7c292a5
2 changed files with 412 additions and 0 deletions
+109
View File
@@ -3186,3 +3186,112 @@ Both must agree: structural guarantee + model's stated recognition of sufficienc
**Verification against decision criteria:** 8/8 pass. Positive closure valid, genuine remaining factors keep decision open, no-confirmation keeps decision open, direction remains separate.
### Implementation Readiness: A — READY FOR BOUNDED IMPLEMENTATION
---
## Experiment 60B.63 — Closure Confirmation Signal Source (read-only design analysis)
**Status: PASSED (design-only, no production code changes)**
### Objective
Determine exactly what is the safest existing deterministic signal for explicit user confirmation that no other material uncertainty remains: the raw user answer, model-generated meaning/reason text, or a combination.
**Answer: RAW USER ANSWER should own confirmation — via bounded phrase-family detection on the `answer` parameter.**
### RAW ANSWER
Available post-propagation: **YES**
Exact variable: `answer` (parameter of `applyValidatedProposal`, available throughout function scope as local variable)
Unchanged user input: **YES** — no sanitisation or model transformation applied between reception (line 3481) and any downstream read
### EXISTING TEXT HANDLING
Bounded raw-answer matcher exists: **NO** — no existing helper detects confirmation, sufficiency, or "no other material uncertainty" patterns in any text source.
Reusable normalisation helper: **PARTIAL**`normaliseText` (line 54), `normaliseSemanticText` (line 3001), and `normalise()` in evidence-direction.js exist but are preprocessors only (lowercase/whitespace), not semantic detectors.
Existing deterministic raw-answer precedent: **PARTIAL**`deriveAnswerMeaningProfile` (line 3178) and `answerConfirmsComparability` (line 2989) demonstrate bounded phrase-family detection patterns exist in the codebase, but they operate on model-extracted meaning or comparability intent, not confirmation sufficiency.
### RAW-ANSWER CANDIDATE
User-grounding: **HIGH** — direct user words, zero mediation
Model dependence: **LOW** — pure text matching; no inference required
False-positive risk: **MEDIUM** — phrase family breadth trade-off; exact matches very low, 4-6 entry family medium but acceptable with careful curation
False-negative risk: **MEDIUM-HIGH** — users express sufficiency in many constructions; bounded family catches reference case and grammatical variants but misses others. Inherent to raw-text matching.
Deterministic: **YES** — regex/string matching is deterministic
Schema change: **NO**
Principal weakness: Bounded phrase families for "no remaining uncertainty" are inherently narrow in coverage. The breadth needed for low false-negative rate increases false-positive risk; this tension cannot be fully resolved without model assistance (which defeats the purpose).
### USER-SUPPORTED MEANING
Directly grounded: **PARTIAL** — derived FROM answer but model-extracted meaning, not user words
Model generated: **YES**
May omit qualification: **YES** — unproven guarantee; 60B.56 showed model can fail to produce critical closure language
Suitable as closure owner: **NO** — model-generated content cannot be deterministically trusted for binary structural gate
### PARENT REASON
Model generated: **YES**
Guaranteed present: **CONDITIONAL** — standard output but could be missing from malformed proposals
Guaranteed parent-targeted: **NO**
Could reintroduce model-compliance failure (60B.56): **YES** — critical rejection reason. Relying on `updatedNodes[].reason` uses the exact same model output channel that 60B.56 proved unreliable.
### CANDIDATE ASSESSMENT
| Candidate | Fixes 60B.56 | User grounding | Model dependence | FP risk | FN risk | Schema | Principal weakness |
|-----------|-------------|----------------|------------------|---------|---------|--------|--------------------|
| A (RAW ANSWER ONLY) | YES | HIGH | LOW | MEDIUM | MEDIUM-HIGH | NO | Narrow coverage of user phrasings for sufficiency |
| B (USER-SUPPORTED MEANING ONLY) | CONDITIONAL | MEDIUM | HIGH | LOW-MEDIUM | HIGH | NO | Unproven presence of sufficiency language; exactly the 60B.56 failure mode |
| C (PARENT REASON ONLY) | CONDITIONAL | LOW | HIGH | LOW-MEDIUM | HIGH | NO | Same model-compliance channel that failed in 60B.56 |
| D (RAW OR MODEL) | YES | HIGH | MEDIUM | LOW-MEDIUM | MEDIUM-LOW | NO | OR path means closure can fire on model text alone when raw answer does not match |
| E (RAW AND MODEL) | CONDITIONAL | HIGH | MEDIUM-HIGH | VERY LOW | VERY HIGH | NO | Model omission blocks closure even when user confirmed; violates criterion 3 |
### NO-CONFIRMATION CASES
Case 1 (factor resolves, no sufficiency statement): Confirmation result = **false** — correctly keeps decision open.
Case 2 (user states uncertainty remains): Confirmation result = **false** — correctly keeps decision open.
Both derived from existing test data: apply-proposal.test.js line 4563+ shows factor-resolve-only scenario without sufficiency language.
### PARAPHRASE TOLERANCE
Choice: **B — SMALL BOUNDED PHRASE FAMILY** (~4-6 entries via regex/.includes())
Why: A is too narrow (unacceptably high FN), C reintroduces 60B.56 dependency, D is overly conservative given existing bounded-matching precedents in `deriveAnswerMeaningProfile` and `answerConfirmsComparability`.
### CRITICAL DISTINCTION
Choice: **A — RAW USER ANSWER SHOULD OWN CONFIRMATION**
Why: Only signal satisfying all seven decision criteria. Satisfies criterion 3 (model omission does not prevent closure when user explicitly confirmed) which B/C/D/E violate to varying degrees. Criterion 4 (model paraphrase does not create closure when user did not confirm) is satisfied because only raw answer is checked.
### MINIMUM CORRECTIVE BOUNDARY
Choice: **A — add narrow raw-answer confirmation helper**
Why: The only missing piece is a bounded phrase-family detector on the `answer` parameter. 1 new helper function, 0 schema/prompt/test changes. All seven criteria satisfied with lowest corrective boundary.
### WOULD 60B.56 CLOSE: YES (both graph sufficiency + user confirmation in raw answer are present)
### WOULD EXPLICIT USER WORDING REMAIN PRIMARY: YES (raw text is sole confirmation signal; model output irrelevant to confirmation gate)
### WOULD MODEL OMISSION STOP CLOSURE: NO (no model signal required for confirmation)
### WOULD NO-CONFIRMATION CASE REMAIN OPEN: YES (bounded family returns false when no match)
### IMPLEMENTATION READINESS: A — READY FOR BOUNDED IMPLEMENTATION
One unresolved question at implementation layer only: precise phrase family breadth balancing FP vs FN risk. This is a design detail, not a structural design question.
**Smallest implementation boundary:**
```
1 new helper: isUserConfirmationOfNoRemainingUncertainty(answer) => boolean
1 integration at candidate D (post-propagation, ~3 lines):
if (hasRemainingMaterialFactors(decisionId, graph) === false && isUserConfirmationOfNoRemainingUncertainty(answer)) { /* close */ }
0 schema/prompt/test changes
```
**Verification against decision criteria:** 8/8 pass.
### PRODUCTION CODE CHANGED: NO
### TESTS CHANGED: NO
### PROMPT CHANGED: NO
### SCHEMA CHANGED: NO
### OLLAMA CALLS: 0
### LIVE API CALLS: 0
### VITEST RUN: NO
### JEST RUN: NO
### WATCHMAN USED: NO