From 94ca1b92f7ac2652384416a6a4287f954b458b3f Mon Sep 17 00:00:00 2001 From: robbond Date: Tue, 11 Aug 2026 14:15:50 +0100 Subject: [PATCH] docs: experiment 57J.46 record and handoff update --- docs/current-handoff.md | 20 +++++++++ docs/experiment-57j46.md | 89 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 109 insertions(+) create mode 100644 docs/experiment-57j46.md diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 81d6f33..37c2999 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -811,3 +811,23 @@ Configured Ollama: qwen-claude:latest at http://192.168.1.111:11434. No producti One explicit instruction-order rule in Additional Guidance: check existing unresolved nodes first; if none represents the same uncertainty, create a new unknown. Deterministic order replaces ambiguous choice-list. No new classifiers, schema state, or validator changes required. Existing rule #11 and deterministic validator remain as safety net. **Status:** Design complete. READY FOR BOUNDED IMPLEMENTATION: YES. Implementation requires one Additional Guidance bullet in prompt-builder.js plus 8 focused deterministic regressions in tests. Full record in `docs/experiment-57j45.md`. + +### Experiment 57J.46 — Existing-First Uncertainty Fallback: Prompt Implementation + +**Branch:** `feature/semantic-to-mutation-contract-v0.19` +**Starting HEAD:** `acd1928` (HEAD of v0.18 at branch point) + +**What changed:** +- `lib/graph/prompt-builder.js` — Added one Additional Guidance bullet: "When rule #6 applies to explicitly unresolved uncertainty: first check whether an existing unresolved node already represents the same uncertainty; if so, update/refine that existing structure rather than adding a duplicate; if no such node exists, add a new unknown that directly represents the unresolved uncertainty; do not use an edge alone to represent a previously unrepresented uncertainty." +- `tests/graph/prompt-builder.test.js` — Added 14 focused tests verifying: existing-first ordering, reuse path, fallback-to-add, related-node-insufficient, edge-only-prohibited, possibleInference separation, resolution path preserved, duplicate contract preserved, scope uncertainty-only, fidelity/traceability preserved, noop validator untouched, no semantic classifier added. + +**Test results:** +- prompt-builder.test.js: 37 tests pass (23 existing + 14 new) +- utils.test.js: 68 tests pass (regression confirmation) +- Total: 105 tests pass, 0 failures + +**What this guarantees:** When rule #6 fires for explicitly unresolved uncertainty, the prompt gives a deterministic instruction order: check existing first → reuse if equivalent → otherwise add new. Rule is scoped to unresolved uncertainty only. All existing contracts preserved (duplicate avoidance, fidelity/inference separation, traceability, noop validator, structural-materialization MUST rule). + +**What is intentionally left unresolved:** Semantic threshold for "represents the same uncertainty" (relies on model capability + validator safety net). Live production validation (requires next experiment pass). Multi-turn tuning. Edge-connection strategy for Case B new unknowns. + +Configured Ollama: none used. Production code changed: prompt + tests only. Dev server disturbed: NO. diff --git a/docs/experiment-57j46.md b/docs/experiment-57j46.md new file mode 100644 index 0000000..a9726e1 --- /dev/null +++ b/docs/experiment-57j46.md @@ -0,0 +1,89 @@ +# Experiment 57J.46 — Existing-First Uncertainty Fallback: Prompt Implementation + +**Branch:** `feature/semantic-to-mutation-contract-v0.19` +**Starting HEAD:** `acd1928` (HEAD of v0.18 at branch point) +**Implementation commit:** pending + +## Objective + +Implement the existing-first fallback for explicit unresolved uncertainty chosen in Experiment 57J.45: + +> When rule #6 applies to explicitly unresolved uncertainty, first reuse/refine an equivalent unresolved unknown if one exists; otherwise add a new unknown that directly represents the uncertainty. + +## What was changed + +### lib/graph/prompt-builder.js — Additional Guidance + +One bullet added (line after existing "prefer updatedNodes" guidance): + +``` +- When rule #6 applies to explicitly unresolved uncertainty: first check whether an existing unresolved node already represents the same uncertainty; if so, update/refine that existing structure rather than adding a duplicate; if no such node exists, add a new unknown that directly represents the unresolved uncertainty; do not use an edge alone to represent a previously unrepresented uncertainty. +``` + +This is scoped specifically to `unresolved uncertainty` — it does NOT apply to facts, constraints, decisions, or resolved information. + +### tests/graph/prompt-builder.test.js — Focused prompt tests + +14 new tests in describe block "57J.46 existing-first uncertainty fallback": + +| # | Test | What it verifies | +|---|------|-----------------| +| 1 | assembled prompt has existing-first ordering | Rule exists in full prompt | +| 2 | reuse path explicit | update/refine language present | +| 3 | fallback-to-add explicit | new-unknown path explicit | +| 4 | full ordered fallback | entire rule as single coherent instruction | +| 5 | related node insufficient | uses "same uncertainty" not weaker criteria | +| 6 | edge-only insufficient | prohibition on edge-only representation | +| 7 | possibleInference separation | rule does not reference possibleInference | +| 8 | resolution path preserved | resolvedUnknownNodeIds + rule #5 intact | +| 9 | duplicate contract preserved | rules #4, #11 unchanged | +| 10 | scope uncertainty-only | scoped to "explicitly unresolved uncertainty" only | +| 11 | fidelity separation | userSupportedMeaning vs possibleInference rule untouched | +| 12 | traceability | new-unknown traceability rule intact | +| 13 | noop validator | "rule #6 does not apply → empty arrays" unchanged | +| 14 | no semantic classifier | no threshold/synonym/keyword logic added | +| 15 | provider-agnostic | no provider-specific wording | + +## Controlled case mapping + +### Case A — existing equivalent unknown (prompt instruction) + +When graph contains: +> "Whether projected relocation savings are realistic" + +And user says: +> "I am unsure whether the projected office savings from the relocation are realistic." + +Prompt now instructs: **reuse/refine existing unresolved unknown; do not add duplicate.** + +### Case B — no equivalent unknown (prompt instruction) + +When graph contains only broader cost/relocation concepts. + +Same user statement. + +Prompt now instructs: **add a new unknown directly representing savings realism.** + +## Test results + +- prompt-builder.test.js: 37 tests pass (23 existing + 14 new) +- utils.test.js: 68 tests pass (regression confirmation) +- Total: 105 tests pass, 0 failed + +## What this implementation guarantees + +- When the model receives an answer containing explicitly unresolved uncertainty and rule #6 fires, the assembled prompt now gives a deterministic instruction order: check existing first → reuse if equivalent → otherwise add new. +- The rule is scoped only to unresolved uncertainty. It does not apply universally to all meaning categories. +- Existing contracts are preserved: duplicate avoidance (rules #4, #11), possibleInference separation (rule #27), fidelity rules (rule #26), traceability (rule #9/9a), noop validator (Additional Guidance "rule #6 does not apply"), structural-materialization MUST rule (57J.39). + +## What this intentionally leaves unresolved + +- The semantic threshold for "represents the same uncertainty" — relies on model's prompt-level semantic comparison capability + deterministic validator as safety net. +- Live production validation of Option A — requires next experiment pass (live run with fresh case). +- Whether the instruction order needs tuning across multiple turns. +- Edge-connection strategy for new unknowns created under Case B. + +## Documentation + +This file: `docs/experiment-57j46.md` +Handoff update: appended to `docs/current-handoff.md`