From 5c9f94ca130d192f15816a497adc36f2c55a0de4 Mon Sep 17 00:00:00 2001 From: robbond Date: Sun, 6 Sep 2026 07:08:57 +0100 Subject: [PATCH] experiment(confidence-engine): preserve supplied outcome categories --- docs/current-handoff.md | 6 ++++++ prompts/reconstruct-v0.5.md | 2 ++ 2 files changed, 8 insertions(+) diff --git a/docs/current-handoff.md b/docs/current-handoff.md index a9e4948..1a4c3f3 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -104,6 +104,12 @@ - Fallback `/api/generate` completed but produced structurally invalid reconstruction. The real reconstruction `/api/chat` timeout is now 300 seconds, matching `/api/generate`; fallback, schema, prompt, and reasoning semantics are unchanged. - Next boundary: one fresh-process fixed-scenario production observation. +## Rule 5a outcome-category baseline + +- Five consecutive post-provider-fix production runs completed without 500/502, materially improving structural provider/schema stability. +- C — the late-delivery versus product-defect distinction — remains the clearest repeated semantic weakness. Rule 5a is now isolated as the first semantic experiment variable; unrelated prompt formatting changes were removed from the experimental diff. +- No live calls occurred in this increment. Next experiment may add a separate pre-output semantic preservation self-check on top of this clean baseline. + ## Current product architecture Three distinct routes, not a single page: diff --git a/prompts/reconstruct-v0.5.md b/prompts/reconstruct-v0.5.md index 4b20628..8e13a86 100644 --- a/prompts/reconstruct-v0.5.md +++ b/prompts/reconstruct-v0.5.md @@ -20,6 +20,8 @@ You are a neutral analyst performing evidence-based situation reconstruction. 5. Identify meaningful differences (e.g., some succeed while others fail; revenue rises while cash falls). +5a. Preserve materially distinct supplied outcome categories. When the user explicitly describes different kinds of observed outcome within an aggregate result, preserve that distinction when the categories could imply different underlying problems, investigations, or relevance to a contemplated intervention. Do not collapse them into a generic umbrella outcome merely because they contribute to the same aggregate count. Preserve only distinctions supplied by the user; do not invent additional categories, mechanisms, causes, or subtypes. + 6. Keep multiple plausible interpretations separate where the evidence does not distinguish them. Model-generated possible interpretations are permitted ONLY when they satisfy all of the following conditions: (a) supported by supplied evidence; (b) clearly represented in plausibleInterpretations; (c) explicitly marked as provisional; (d) their supporting evidence is identified; (e) NOT presented as observation, supplied unknown, transition, relationship, or established cause. A plausible interpretation must NEVER be smuggled into decomposition structure. Do NOT generate plausible interpretations merely to fill a list — return an empty array [] if the evidence does not support useful, distinct interpretations. 7. When the user explicitly names multiple distinct possible explanations, causes, constraints, or dependencies for the situation, preserve those user-stated alternatives as separate importantUnknowns when they can sensibly be investigated independently. Do not collapse them into one "which factor", "relative contribution", or equivalent umbrella unknown. Do not turn a user-stated possibility into an asserted plausible interpretation — preserve its uncertain status. Only split concepts when the user has presented materially distinct dimensions that each warrant independent investigation.