67 lines
3.3 KiB
Markdown
67 lines
3.3 KiB
Markdown
# Uncertainty Information Flow Audit — Programme Handoff
|
||
|
||
**Working title:** Confidence Engine — Uncertainty Information Flow Audit
|
||
**Date prepared:** 2026-10-14
|
||
**Prepared for:** Next independent investigation (following the completed 10-iteration uncertainty recognition quality investigation)
|
||
|
||
---
|
||
|
||
## Primary Question
|
||
|
||
"At which point, if any, does Confidence Engine constrain, transform, discard or fail to preserve meaningful uncertainty recognition?"
|
||
|
||
## Context
|
||
|
||
The previous 10-iteration programme demonstrated a quality gap between minimal direct-model uncertainty recognition and Confidence Engine-mediated uncertainty recognition. The precise limiting mechanism remains unresolved (see `uncertainty-recognition-investigation-report.md` sections D–G). This new programme is designed to map the actual production information flow and identify specific intervention points without presuming failure locations.
|
||
|
||
## Investigation Boundaries (Not Presumed Failure Points)
|
||
|
||
1. User input and scenario representation
|
||
2. Prompt construction
|
||
3. Raw LLM request and response
|
||
4. Parsing and validation
|
||
5. Internal graph representation
|
||
6. User-facing presentation
|
||
|
||
These boundaries define investigation scope, not expectations of where problems will be found.
|
||
|
||
## Recommended Approach
|
||
|
||
- Up to 10 iterations
|
||
- One bounded investigative question per iteration
|
||
- ChatGPT controls programme progression between iterations
|
||
- Claude Code executes one task per invocation and reports evidence
|
||
- Each `/rdbHandoff` invocation is a fresh session — work from repository evidence and this handoff
|
||
- Read-only investigation before any interventions
|
||
- Controlled experiments only when justified by evidence from prior iterations
|
||
- Preserve raw evidence and request provenance for every iteration
|
||
- No speculative product changes
|
||
|
||
## First Iteration Guidance
|
||
|
||
The first iteration should establish the actual production information flow and available capture boundaries:
|
||
|
||
1. Map the complete code path from scenario file input to model request construction
|
||
2. Document what data is accessible at each stage of the pipeline (raw scenario, post-parsing, pre-template, post-template, pre-request, post-response, post-parsing)
|
||
3. Identify any existing hooks, logs, or capture mechanisms for raw requests/responses
|
||
4. Determine which stages can produce evidence without source modifications
|
||
|
||
The output should be a factual map of available capture points and data accessibility, not an analysis of what is "wrong."
|
||
|
||
## Existing Evidence to Consider
|
||
|
||
- `docs/uncertainty-recognition-investigation-report.md` — Full report from the previous programme
|
||
- `.evidence-temp/iteration10/architecture-finding.txt` — Definitive CE prompt-path analysis showing template applicability at every invocation point
|
||
- `.evidence-temp/headless-semantic/` — Extensive supporting evidence from HS series experiments
|
||
|
||
## What This Programme Is Not
|
||
|
||
- A redesign of Confidence Engine
|
||
- A directive to change the model or provider configuration
|
||
- An assumption that any specific boundary is broken or deficient
|
||
- A continuation of the previous programme's quality-gap investigation (that closed at Iteration 10)
|
||
|
||
## Start Signal
|
||
|
||
Do not begin this programme from this document alone. Wait for an explicit `/rdbHandoff` invocation containing the start signal and first iteration directive.
|