robbond/confidence-engine
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

experiment: validate structured semantic fidelity live

Browse Source
This commit is contained in:
robbond
2026-08-11 17:18:59 +01:00
parent f156bf5ef8
commit 5947ccb642
2 changed files with 122 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/current-handoff.md
+17
View File
@@ -990,3 +990,20 @@ Implemented the bounded Option C from 57J.50 on branch `feature/structured-seman
**Deferred intentionally:** `must_resolve` target-specific enforcement remains deferred because the current proposal structure does not safely identify the answered/targeted unknown in every case without inventing new linkage.
**Tests:** Required deterministic suites pass, plus one directly related proposal-parse suite updated for the new schema boundary. No live model calls. No Ollama. Full record in `docs/experiment-57j51.md`.
### Experiment 57J.52 — Structured Semantic Fidelity Live Verification
**Classification: A — V0.20 STRUCTURED PATH WORKS**
One fresh live run through the production `startCase()` → `updateCase()` path. Fixed scenario: "We are considering relocating the engineering team to reduce operating costs." Fixed answer: "I am unsure whether the projected office savings from the relocation are realistic."
**Key evidence:**
- `supportCategory = "uncertain"` (populated, valid enum) — STRUCTURED path active, not LEGACY FALLBACK.
- `resolutionGuidance = "may_resolve"` (populated).
- Meaning: FAITHFUL — model preserved uncertainty without strengthening or degrading.
- New unknown node `nf3g7m2` ("Realism of projected office savings from relocation") added with `depends_on` edge to summary — meaningful graph structure produced.
- The old `unsure` → `uncertain` lexical mismatch does NOT determine the outcome: structured `supportCategory` is authoritative, bypassing lexical derivation entirely.
**What this establishes:** v0.20's structured semantic fidelity path executes live on the real production path. The model populates structured fields from a "I am unsure..." answer, the validator trusts those populated fields over lexical comparison, and meaningful graph structure is produced.
**What this does NOT prove:** Stability across repeated runs; behavior for answers outside existing categories; `must_remain_unresolved` enforcement in practice; end-to-end investigation viability past Update 2+. Full record in `docs/experiment-57j52.md`.