From a47643104837c6003e63b834183533f47a019b22 Mon Sep 17 00:00:00 2001 From: robbond Date: Tue, 11 Aug 2026 17:47:45 +0100 Subject: [PATCH] docs: record uncertainty identity clarification --- docs/current-handoff.md | 12 +++++++ docs/experiment-57j55.md | 71 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 83 insertions(+) create mode 100644 docs/experiment-57j55.md diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 897e77d..3212ec5 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -1029,3 +1029,15 @@ One fresh live run through the production `startCase()` → `updateCase()` path. **What this establishes:** v0.20's structured semantic fidelity path executes live on the real production path. The model populates structured fields from a "I am unsure..." answer, the validator trusts those populated fields over lexical comparison, and meaningful graph structure is produced. **What this does NOT prove:** Stability across repeated runs; behavior for answers outside existing categories; `must_remain_unresolved` enforcement in practice; end-to-end investigation viability past Update 2+. Full record in `docs/experiment-57j52.md`. + +### Experiment 57J.55 — Uncertainty Identity Clarification (Prompt-Only) + +**Objective:** Implement the correction from Experiment 57J.54: clarify that "same uncertainty" means the same resolution question, not topical overlap. Bounded prompt-only task — no schema, validator, graph-model, or provider changes. + +**Changes:** +- `lib/graph/prompt-builder.js`: Added to Additional Guidance — *"Same uncertainty means the same resolution question: resolving the existing unknown would also resolve the uncertainty introduced by the user's answer. Mere topical overlap (concerning the same topic, object, decision, or domain) is not automatically the same uncertainty."* +- `tests/graph/prompt-builder.test.js`: 10 new focused prompt tests verifying the definition, boundaries, and preserved contracts (49 total pass). + +**Test results:** All 49 tests pass. No regression in existing prompt structure tests. + +**Configured Ollama:** none used. **Production code changed:** NO. **Ollama calls:** 0. diff --git a/docs/experiment-57j55.md b/docs/experiment-57j55.md new file mode 100644 index 0000000..424126e --- /dev/null +++ b/docs/experiment-57j55.md @@ -0,0 +1,71 @@ +# Experiment 57J.55 — Uncertainty Identity Clarification (Prompt-Only) + +**Branch:** `feature/uncertainty-identity-v0.21` +**Starting HEAD:** `f0cf85d` (experiment: diagnose uncertainty identity vs relatedness) +**Experiment commit:** pending + +## Objective + +Implement the bounded correction from Experiment 57J.54: clarify that "same uncertainty" means the same resolution question, not just topical overlap. This is a prompt-only task — no schema, validator, graph-model, or provider changes. + +## Changes Made + +### lib/graph/prompt-builder.js + +Added to Additional Guidance under the existing first-existing-rule bullet: + +> **"Same uncertainty" means the same resolution question: resolving the existing unknown would also resolve the uncertainty introduced by the user's answer. Mere topical overlap (concerning the same topic, object, decision, or domain) is not automatically the same uncertainty. If the new concern can remain unresolved after the existing node is resolved, represent it separately as a distinct uncertainty.** + +This preserves the existing ordered fallback: +1. Check whether an existing unresolved unknown represents the same uncertainty. +2. If yes, update/refine it rather than creating a duplicate. +3. If no, add a new unknown representing the uncertainty. + +### tests/graph/prompt-builder.test.js + +Added 10 focused prompt tests under `buildGraphUpdatePrompt — 57J.55 uncertainty identity vs topical overlap`: + +| # | What is tested | Assertion type | +|---|----------------|---------------| +| 1 | "same resolution question" definition exists | positive containment | +| 2 | "topical overlap" explicitly insufficient | positive containment | +| 3 | independently unresolved → distinct uncertainty | positive containment | +| 4 | equivalent uncertainty still prefers reuse/refine (existing-first) | positive containment | +| 5 | broad nodes do not automatically absorb sub-concerns | negative containment | +| 6 | unrelated domains handled separately | positive containment | +| 7 | duplicate avoidance preserved | positive containment | +| 8 | existing-first ordering preserved | positive containment | +| 9 | no keyword/synonym/embedding/similarity machinery added | negative containment × 4 | +| 10 | structured semantic fidelity (supportCategory, resolutionGuidance) intact | positive containment × 4 | + +## Test Results + +``` +✓ tests/graph/prompt-builder.test.js (49 tests) 30ms + +Test Files 1 passed (1) + Tests 49 passed (49) +``` + +All 49 tests pass — no regression in existing prompt structure tests; all 10 new identity tests pass. + +## What This Implementation Guarantees + +- The prompt now defines "same uncertainty" as a resolution-question equivalence, not topical proximity. +- A focused uncertainty (e.g., "Are the projected office savings realistic?") is distinguishable from a broader related unknown (e.g., "What will total costs at the new location be?") by the independent-resolvability test: knowing total projected costs does not establish whether the office-savings assumption itself is realistic. +- Equivalent wording across turns (paraphrased savings-realism) still triggers reuse/refine via preserved existing-first ordering. +- No keyword, synonym, embedding, or numeric similarity logic was added — this remains purely prompt-level semantic reasoning. + +## What This Intentionally Leaves Unresolved + +- Whether the configured model (qwen-claude:latest) actually complies with the tightened guidance on live runs — requires live regression. +- Downstream effects of the clarification on question-selection, evidence linkage, or candidate tie behaviour — those remain separate issues per the scope exclusions. +- Generalisation to non-uncertainty categories (constraints, facts, decisions) — these may share similar gaps but are out of scope. + +## Production code changed: NO +## Validator changed: NO +## Schema changed: NO +## Prompt changed: YES +## Tests changed: YES +## Ollama calls: 0 +## Dev server disturbed: NO