4.4 KiB
Experiment 57J.46 — Existing-First Uncertainty Fallback: Prompt Implementation
Branch: feature/semantic-to-mutation-contract-v0.19
Starting HEAD: acd1928 (HEAD of v0.18 at branch point)
Implementation commit: pending
Objective
Implement the existing-first fallback for explicit unresolved uncertainty chosen in Experiment 57J.45:
When rule #6 applies to explicitly unresolved uncertainty, first reuse/refine an equivalent unresolved unknown if one exists; otherwise add a new unknown that directly represents the uncertainty.
What was changed
lib/graph/prompt-builder.js — Additional Guidance
One bullet added (line after existing "prefer updatedNodes" guidance):
- When rule #6 applies to explicitly unresolved uncertainty: first check whether an existing unresolved node already represents the same uncertainty; if so, update/refine that existing structure rather than adding a duplicate; if no such node exists, add a new unknown that directly represents the unresolved uncertainty; do not use an edge alone to represent a previously unrepresented uncertainty.
This is scoped specifically to unresolved uncertainty — it does NOT apply to facts, constraints, decisions, or resolved information.
tests/graph/prompt-builder.test.js — Focused prompt tests
14 new tests in describe block "57J.46 existing-first uncertainty fallback":
| # | Test | What it verifies |
|---|---|---|
| 1 | assembled prompt has existing-first ordering | Rule exists in full prompt |
| 2 | reuse path explicit | update/refine language present |
| 3 | fallback-to-add explicit | new-unknown path explicit |
| 4 | full ordered fallback | entire rule as single coherent instruction |
| 5 | related node insufficient | uses "same uncertainty" not weaker criteria |
| 6 | edge-only insufficient | prohibition on edge-only representation |
| 7 | possibleInference separation | rule does not reference possibleInference |
| 8 | resolution path preserved | resolvedUnknownNodeIds + rule #5 intact |
| 9 | duplicate contract preserved | rules #4, #11 unchanged |
| 10 | scope uncertainty-only | scoped to "explicitly unresolved uncertainty" only |
| 11 | fidelity separation | userSupportedMeaning vs possibleInference rule untouched |
| 12 | traceability | new-unknown traceability rule intact |
| 13 | noop validator | "rule #6 does not apply → empty arrays" unchanged |
| 14 | no semantic classifier | no threshold/synonym/keyword logic added |
| 15 | provider-agnostic | no provider-specific wording |
Controlled case mapping
Case A — existing equivalent unknown (prompt instruction)
When graph contains:
"Whether projected relocation savings are realistic"
And user says:
"I am unsure whether the projected office savings from the relocation are realistic."
Prompt now instructs: reuse/refine existing unresolved unknown; do not add duplicate.
Case B — no equivalent unknown (prompt instruction)
When graph contains only broader cost/relocation concepts.
Same user statement.
Prompt now instructs: add a new unknown directly representing savings realism.
Test results
- prompt-builder.test.js: 37 tests pass (23 existing + 14 new)
- utils.test.js: 68 tests pass (regression confirmation)
- Total: 105 tests pass, 0 failed
What this implementation guarantees
- When the model receives an answer containing explicitly unresolved uncertainty and rule #6 fires, the assembled prompt now gives a deterministic instruction order: check existing first → reuse if equivalent → otherwise add new.
- The rule is scoped only to unresolved uncertainty. It does not apply universally to all meaning categories.
- Existing contracts are preserved: duplicate avoidance (rules #4, #11), possibleInference separation (rule #27), fidelity rules (rule #26), traceability (rule #9/9a), noop validator (Additional Guidance "rule #6 does not apply"), structural-materialization MUST rule (57J.39).
What this intentionally leaves unresolved
- The semantic threshold for "represents the same uncertainty" — relies on model's prompt-level semantic comparison capability + deterministic validator as safety net.
- Live production validation of Option A — requires next experiment pass (live run with fresh case).
- Whether the instruction order needs tuning across multiple turns.
- Edge-connection strategy for new unknowns created under Case B.
Documentation
This file: docs/experiment-57j46.md
Handoff update: appended to docs/current-handoff.md