Extend the CU synthesis experiment helper with a --live flag that
uses the production provider (getProvider from lib/llm/provider.js)
for live model invocation. Preserves deterministic mode unchanged.
Adds 10 new deterministic tests verifying live-mode wiring via
injected mock provider: fixture pass-through, single invocation,
schema correctness, immutability, persistence safety, and regression
of existing deterministic behaviour. Zero network calls in all tests.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Controlled A/B (zero-Findings scenario, Qwen): unresolved uncertainty exposure
did not contaminate supported understanding or plausible interpretation.
No production changes. No report semantic changes.
- Helper now supports --variant visible which appends unresolved Open Questions
as a separate NON-EVIDENCE section to the baseline prompt (built via production
buildOverviewSynthesisPrompt)
- Default (--variant hidden, or no flag) remains production path
- No production code changed; apparatus-only extension
- Deterministic A/B control proven by existing 34/34 test suite
synthesizeInvestigationOverview() called provider.generateReconstruction(prompt, modelName)
without an output contract. Without a structured-output schema the provider defaults to its
initial-reconstruction format which lacks { understanding, plausibleInterpretations }, causing
'Missing or invalid required fields' validation failure.
Add overviewOutputSchema (JSON-schema matching validateOverviewResponse contract) and pass as
third argument, matching the proven pattern in synthesizeCurrentUnderstanding().
53/53 tests PASS. Co-Authored-By: Claude Code <noreply@anthropic.com>
Production: synthesizeInvestigationOverview() now passes providerResult.response
to validateOverviewResponse() instead of the raw envelope, matching the proven
pattern in current-understanding-synthesis.js.
Regression: fakeProvider returns real-provider envelope shape { response, providerApiPath }
and four new regression tests verify unwrapping succeeds with valid response
and fails correctly with invalid response inside the envelope.
Handoff: documented defect confirmed, fix applied, and owning regression added.
- Add nullish coalescing on fix.findings ?? [] so absent findings
array in a fixture is represented as zero Findings [] for the A/B
input apparatus.
- Append product-launch second controlled case (live-product-launch-
start-response.json) with zero Findings to the epistemic comparison
test suite. All 34 tests pass deterministically.
- Update docs/current-handoff.md to record the zero-Findings case.
Retained deterministic input-control test proving that A/B apparatus
correctly holds supportedEvidence, findings, plausibleInterpretations,
scenarioState, and providerConfig constant while only varying unresolved
question visibility. Added buildOverviewSynthesisPrompt identity proof
that full vs filtered graph produce identical prompt text.
Corrected handoff semantics: deterministic input filtering does NOT prove
model output behaviour; live two-call semantic comparison remains required.
Preserved prior imperfect live evidence with correct qualification about
uncontrolled plausibleInterpretations divergence.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Deterministic test (17 cases) proves A/B inputs are correctly controlled:
- supportedEvidence identical in A and B
- findings identical in A and B
- plausibleInterpretations identical (fixed canonical interpretation) in A and B
- scenarioState identical in A and B
- providerConfig identical in A and B
- only unresolvedQuestions visibility differs (A=hidden, B=canonical)
- unresolved questions excluded from evidence and plausible interpretations
- fixed interpretation carries NON-EVIDENCE epistemic status
Also corrects Loop 2 documentation to reflect that three-category
invariance was NOT established (plausibleInterpretations differed
between calls despite identical input), retaining only the narrower
valid finding: no unresolved Open Question was promoted to evidence.
Co-Authored-By: Claude Code <noreply@anthropic.com>
- Reduce docs/current-handoff.md from 2003 → 185 lines (90% reduction)
- Move all initial-decomposition v0.61 experiment narrative to
docs/archive/experiments/vol-1-chapters/ch19/initial-decomposition-v0.61.md
- Update design-evolution/README.md with ch19 Era 8 entry
- Add CURRENT MVP DIRECTION section (frozen; OpenAI investigation is next question)
- Fix stale branch reference in current-project-state.md
- Update Return-to-Work Summary to reflect v0.61 completion
- Update Verification Marker for v0.60/v0.61 status
robbondandClaude Code