Commit Graph
15 Commits
Author SHA1 Message Date
robbond fe0f337a99 docs(experiments): checkpoint uncertainty recognition investigation 2026-10-14 09:39:11 +01:00
robbond e595ee6e02 feat(experiments): integrate Condition D/E apparatus 2026-10-13 16:24:00 +01:00
robbond 5af3e62ddd feat(uncertainty-led-investigation): add Condition D freeform deconstruct treatment selector
- Add reconstruct-v0.5-d-deconstruct-no21.md prompt (removes rule 21)
  from original deconstruct-first prompt: 'Do not create artificial
  uncertainties to increase their number.'
- Extend start-case-experiment-helper.cjs with --condition D and
  --treatment no21 CLI flags for treatment selector apparatus
- Add programmatic treatment validation (rejects invalid treatments
  cross-condition, validates only 'no21' is a valid treatment value)
- Condition D output uses freeformText field instead of reconstruction
  JSON — avoids production JSON schema enforcement on freeform text
- Mock double supports freeform mode: returns deterministic raw text
  instead of structured importantUnknowns when isFreeform=true
- Add 23 targeted tests: J (Condition D prompt selection),
  J.1 (treatment validation), J.2 (prompt isolation verification)
- Fix existing B.1 assertions to use matchObject for execution field
2026-10-12 18:45:53 +01:00
robbond d5a06b9f28 docs(confidence-engine): record CE-ULI-11 domain-preservation intervention — explicit instruction did NOT prevent structural compression (Condition B=1 unknown, Condition C=1 unknown) 2026-10-12 09:41:43 +01:00
robbondandClaude Code 3b04d05355 feat(confidence-engine): add CE-ULI-08 two-condition uncertainty comparison apparatus
Adds experimental apparatus for testing whether removing the question field
from importantUnknowns in reconstruction output improves semantic fidelity.

New files:
- lib/reconstruction/schema-variant.js (conditionB schema + production passthrough)
- prompts/reconstruct-v0.5-b-noreq.md (alt prompt, identical line count to prod)

Changed files:
- lib/graph/orchestrator.js (pass generationSchema through dependencies)
- lib/analysis.js (accept opts.generationSchema fallback to original schema)
- scripts/start-case-experiment-helper.cjs (--condition B/A flag support)
- tests/scripts/start-case-experiment-helper.test.js (8 new deterministic tests)

43 targeted deterministic tests PASS. Production callers use original prompt/schema unchanged.
No live model calls. No full suite evidence available.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-12 07:33:31 +01:00
robbond fd5241c6af feat(experiments): expose reconstruction open-question evidence 2026-10-09 08:12:06 +01:00
robbond 5878ce45ec experiment(confidence-engine): add reconstruction-only helper flag 2026-09-06 18:22:20 +01:00
robbond 0bb2f01100 experiment(confidence-engine): isolate reconstruction observation 2026-09-06 10:31:45 +01:00
robbond 215c783d11 experiment(confidence-engine): complete openai reconstruction apparatus 2026-09-06 10:06:06 +01:00
robbond 1daf2bb6ce experiment(confidence-engine): expose reconstruction provider seam 2026-09-06 08:01:29 +01:00
robbond a59d60262e fix(confidence-engine): correct experiment helper project root 2026-09-05 06:15:05 +01:00
robbond f2a761d26f test(confidence-engine): expose reconstruction experiment seam 2026-09-04 19:42:27 +01:00
robbond 642554038e test(confidence-engine): prove direct helper production seam 2026-09-04 14:33:02 +01:00
robbond 928954ee4a test(confidence-engine): verify direct decomposition helper 2026-09-04 14:02:36 +01:00
robbond 41ea2cb6b9 test(confidence-engine): add direct initial decomposition apparatus
Establishes reusable apparatus for asking: given scenario text X,
what structured initial decomposition does current production path produce?

- Direct curl/Postman via existing /api/cases/start route (no new API)
- Thin CJS helper at scripts/start-case-experiment-helper.cjs for Claude
  experiments (imports startCase directly, zero code duplication)
- Zero-live-call verification: all four seam checks confirmed by existing
  tests (cases-start-route.test.js, start-case-summary.test.js)
- No browser state, no persistence mutation, no Investigation ID required
  by the route itself

Files:
  + scripts/start-case-experiment-helper.cjs (new helper script)
  M docs/current-handoff.md (§v0.61 apparatus documentation)
2026-09-04 13:36:25 +01:00