90 lines
4.4 KiB
Markdown
90 lines
4.4 KiB
Markdown
# Experiment 57J.46 — Existing-First Uncertainty Fallback: Prompt Implementation
|
|
|
|
**Branch:** `feature/semantic-to-mutation-contract-v0.19`
|
|
**Starting HEAD:** `acd1928` (HEAD of v0.18 at branch point)
|
|
**Implementation commit:** pending
|
|
|
|
## Objective
|
|
|
|
Implement the existing-first fallback for explicit unresolved uncertainty chosen in Experiment 57J.45:
|
|
|
|
> When rule #6 applies to explicitly unresolved uncertainty, first reuse/refine an equivalent unresolved unknown if one exists; otherwise add a new unknown that directly represents the uncertainty.
|
|
|
|
## What was changed
|
|
|
|
### lib/graph/prompt-builder.js — Additional Guidance
|
|
|
|
One bullet added (line after existing "prefer updatedNodes" guidance):
|
|
|
|
```
|
|
- When rule #6 applies to explicitly unresolved uncertainty: first check whether an existing unresolved node already represents the same uncertainty; if so, update/refine that existing structure rather than adding a duplicate; if no such node exists, add a new unknown that directly represents the unresolved uncertainty; do not use an edge alone to represent a previously unrepresented uncertainty.
|
|
```
|
|
|
|
This is scoped specifically to `unresolved uncertainty` — it does NOT apply to facts, constraints, decisions, or resolved information.
|
|
|
|
### tests/graph/prompt-builder.test.js — Focused prompt tests
|
|
|
|
14 new tests in describe block "57J.46 existing-first uncertainty fallback":
|
|
|
|
| # | Test | What it verifies |
|
|
|---|------|-----------------|
|
|
| 1 | assembled prompt has existing-first ordering | Rule exists in full prompt |
|
|
| 2 | reuse path explicit | update/refine language present |
|
|
| 3 | fallback-to-add explicit | new-unknown path explicit |
|
|
| 4 | full ordered fallback | entire rule as single coherent instruction |
|
|
| 5 | related node insufficient | uses "same uncertainty" not weaker criteria |
|
|
| 6 | edge-only insufficient | prohibition on edge-only representation |
|
|
| 7 | possibleInference separation | rule does not reference possibleInference |
|
|
| 8 | resolution path preserved | resolvedUnknownNodeIds + rule #5 intact |
|
|
| 9 | duplicate contract preserved | rules #4, #11 unchanged |
|
|
| 10 | scope uncertainty-only | scoped to "explicitly unresolved uncertainty" only |
|
|
| 11 | fidelity separation | userSupportedMeaning vs possibleInference rule untouched |
|
|
| 12 | traceability | new-unknown traceability rule intact |
|
|
| 13 | noop validator | "rule #6 does not apply → empty arrays" unchanged |
|
|
| 14 | no semantic classifier | no threshold/synonym/keyword logic added |
|
|
| 15 | provider-agnostic | no provider-specific wording |
|
|
|
|
## Controlled case mapping
|
|
|
|
### Case A — existing equivalent unknown (prompt instruction)
|
|
|
|
When graph contains:
|
|
> "Whether projected relocation savings are realistic"
|
|
|
|
And user says:
|
|
> "I am unsure whether the projected office savings from the relocation are realistic."
|
|
|
|
Prompt now instructs: **reuse/refine existing unresolved unknown; do not add duplicate.**
|
|
|
|
### Case B — no equivalent unknown (prompt instruction)
|
|
|
|
When graph contains only broader cost/relocation concepts.
|
|
|
|
Same user statement.
|
|
|
|
Prompt now instructs: **add a new unknown directly representing savings realism.**
|
|
|
|
## Test results
|
|
|
|
- prompt-builder.test.js: 37 tests pass (23 existing + 14 new)
|
|
- utils.test.js: 68 tests pass (regression confirmation)
|
|
- Total: 105 tests pass, 0 failed
|
|
|
|
## What this implementation guarantees
|
|
|
|
- When the model receives an answer containing explicitly unresolved uncertainty and rule #6 fires, the assembled prompt now gives a deterministic instruction order: check existing first → reuse if equivalent → otherwise add new.
|
|
- The rule is scoped only to unresolved uncertainty. It does not apply universally to all meaning categories.
|
|
- Existing contracts are preserved: duplicate avoidance (rules #4, #11), possibleInference separation (rule #27), fidelity rules (rule #26), traceability (rule #9/9a), noop validator (Additional Guidance "rule #6 does not apply"), structural-materialization MUST rule (57J.39).
|
|
|
|
## What this intentionally leaves unresolved
|
|
|
|
- The semantic threshold for "represents the same uncertainty" — relies on model's prompt-level semantic comparison capability + deterministic validator as safety net.
|
|
- Live production validation of Option A — requires next experiment pass (live run with fresh case).
|
|
- Whether the instruction order needs tuning across multiple turns.
|
|
- Edge-connection strategy for new unknowns created under Case B.
|
|
|
|
## Documentation
|
|
|
|
This file: `docs/experiment-57j46.md`
|
|
Handoff update: appended to `docs/current-handoff.md`
|