docs: archive historical Confidence Engine evidence

2026-08-19 12:07:17 +01:00
parent a12f9555af
commit e6d0327641
79 changed files with 27 additions and 9 deletions
@@ -0,0 +1,48 @@
+# v0.6 Comparability Experiment
+
+## Hypothesis
+
+The engine should confirm that observations are comparable before treating their difference as a contradiction that needs explanatory follow-up.
+
+## Fixtures
+
+1. Revenue increased by 18%, but cash in the bank fell over the same period.
+2. Complaints increased. Production increased.
+3. Average delivery time decreased by 25%, but order cancellations increased.
+4. Customer satisfaction increased, but complaints increased.
+5. Temperature increased. Ice melted.
+6. Sales doubled. Sales doubled.
+
+## Results
+
+- The first four scenarios repeated the same failure pattern: contradiction-level investigation could begin before comparability was established.
+- A deterministic comparability gate corrected that by producing one comparison question first.
+- Confirmed comparability did not by itself imply contradiction.
+- Temperature increased / Ice melted was reclassified as a compatible relationship, so no contradiction question was asked.
+- Sales doubled / Sales doubled was reclassified as duplicate observations, so no follow-up question was asked.
+
+## Relationship classification stage
+
+After comparability assessment, observations now pass through a deterministic relationship classification stage:
+
+- `contradictory`
+- `compatible`
+- `potentially_related`
+- `duplicate`
+- `insufficient_information`
+
+## Whether comparability should become a permanent reasoning stage
+
+Yes, in minimal deterministic form.
+
+The repeated pattern appeared in four scenarios, so a small pre-contradiction comparability assessment is justified.
+
+## Two-step experiment result
+
+A comparison question is useful only if its answer advances the reasoning stage rather than merely adding more text.
+
+In the revenue-versus-cash scenario, the first question now confirms whether the figures are comparable, and the answer resolves that existing uncertainty instead of creating a parallel note. After that update, the engine progresses from comparability assessment to cautious relationship assessment and can select one broad non-expert follow-up question.
+
+Every justified next question should correspond to an explicit unresolved graph node.
+
+The earlier fallback-only path has now been removed from the normal successful progression. After comparability is resolved and a further investigation question is justified, the engine creates or reuses an explicit unresolved reasoning unknown and lets deterministic selection and question formulation proceed through the standard graph pipeline. A fallback is now only acceptable as an explicit failure case, not as the normal source of the next question.