feat(confidence-engine): preserve focused deconstruction semantic fidelity

This commit is contained in:
2026-08-24 10:11:06 +01:00
parent 2c108df5a9
commit 9d670822a3
3 changed files with 157 additions and 6 deletions
+82
View File
@@ -176,3 +176,85 @@ RTO.1417 experiments proved that an LLM can retain focused knowledge and carr
See also:
- `docs/methodology-checkpoint-return-to-origin.md` — repository-facing checkpoint summary
---
## Focused Semantic Deconstruction — Reasoning Status (2026-08-24)
### Current state
- **Branch:** `feature/decision-closure-ownership-v0.47`
- **HEAD:** `2c108df5a92ec4cb0c4fa7878002cb7eb0aea4fd`
- **Working tree:**
```
M lib/graph/focused-investigation.js
?? tests/graph/live-focused-deconstruct-experiment-helper.mjs
```
- No unexpected temporary files or evidence directories.
### Canonical focused helper
```
tests/graph/live-focused-deconstruct-experiment-helper.mjs
```
Exports `runLiveFocusedDeconstructExperiment`. Future focused semantic experiments should: import it directly; use the real production `buildFocusedDeconstructPrompt`; use the production validator/provider through the helper; configure `.env.local` for `OLLAMA_BASE_URL` and `OLLAMA_MODEL`; make the explicitly budgeted number of live calls; never inline prompt/provider/schema logic; never replace the helper ad hoc.
### Product flow context
```
initial scenario → initial/global SituationGraph → user selects original open question/thread → focused formulate → user answer → focused deconstruct → contribution retained against original targetNodeId → subsequent focused formulate/deconstruct rounds
```
Focused deconstruction accumulates focused contributions rather than rewriting the initial global SituationGraph. Reuse/integration of accumulated focused learning is a later question, not part of this checkpoint.
### Semantic rules currently under review (validated in the cumulative diff)
The production `buildFocusedDeconstructPrompt` encodes six semantic principles:
1. **Observation fidelity** — only meaning directly supported by what the user's answer states becomes an observation; do not strengthen implications into observations.
2. **Uncertainty fidelity** — preserve uncertainty at the narrowest scope justified by evidence; when one factor is established but nothing else is, keep remaining uncertainty broad rather than inventing specific additional factors/deficits/causes/interventions.
3. **Assumption attribution** — attribute only propositions that the user's answer would cease to make sense if false; do not import plausible interpretations from investigation context or model analysis into the assumptions field; return `assumptions: []` when no genuine assumption exists.
4. **Relationship strength** — only connections the user's answer directly establishes become relationships; co-mentioned facts do not create causal/constraint/dependency links.
5. **Proposition ownership** — track who owns each proposition (user's comfort, willingness, threshold, belief, judgement, etc.); do not elevate subjective stance into objective requirement or situational constraint.
6. **Evidence-first / action-appropriate follow-ups** — before formulating follow-ups, check whether the question tests a proposition against current epistemic state; if an explanation/deficit/cause/intervention has not been established by evidence, phrase the question to test whether that proposition is true rather than assuming it; do not jump to implementation details until intervention is the relevant next issue.
A supporting negative principle also holds: *Do not solve semantic interpretation with an ever-growing deterministic dictionary/regex system. Structure may be deterministic; meaning remains semantic/LLM-based.*
### Validated learning (bounded evidence)
Bounded focused experiments on this branch showed:
- The original real UI failure was reproduced on the actual focused-deconstruct path.
- Observations could remain faithful while assumptions/relationships were weakened beyond what the user stated.
- Tightening the assumptions contract materially reduced model-generated assumptions being attributed to the user, supported by bounded contrast.
- Genuine implicit dependencies can still be preserved through the tightened boundary.
- Proposition ownership preserves subjective criteria without weakening genuine factual reports.
- Tentative factual reports can retain their uncertainty.
- Evidence-first follow-ups avoid premature remedy; practical follow-ups remain possible once a blocker is established.
- Uncertainty specificity follows user-supplied evidence rather than defaulting to generic broadness.
**Do not treat these as generalisation:** each finding is supported by the bounded contrast of tested cases only.
### EXP13 status: CONTAMINATED
```
EXP13 - CONTAMINATED — excluded from evidence
```
Reasons: iterative production prompt refinement; best-supported live-call count = 4; not valid bounded experiment evidence; tuned EXP13 uncertainty wording was removed during the diff. EXP13 must not be cited as proof of uncertainty behaviour.
### EXP13R / EXP14 (valid but conservative)
- **EXP13R:** materially reduced invented uncertainty specificity but retained some residual operational specialisation.
- **EXP14:** showed the same rule could preserve explicit user-supplied BACS/failed-payment specificity while leaving unidentified additional gaps broad.
Do not conclude that the uncertainty boundary is fully solved. The contrast between these two cases does not establish generalisation.
### Current restart point
The immediate semantic pass has established a coherent working principle:
> **The Engine should preserve what was said, how certain it was, who owns the proposition, and how strongly evidence supports relationships; then investigate missing evidence before inventing explanations or remedies.**
Deterministic tests (22/22) verify the structural boundaries. Live-call evidence is bounded. No new experiment is run in this checkpoint.