Commit Graph
100 Commits
Author SHA1 Message Date
robbond d5a06b9f28 docs(confidence-engine): record CE-ULI-11 domain-preservation intervention — explicit instruction did NOT prevent structural compression (Condition B=1 unknown, Condition C=1 unknown) 2026-10-12 09:41:43 +01:00
robbondandClaude Code 14ee66e82b docs(confidence-engine): record CE-ULI-10 paired observation — question-free reconstruction does not preserve broader uncertainty (single paired comparison, HS-03C)
Iteration 5 of bounded iteration loop. Two live calls per condition
using fixed HS-03C input (new-service-launch scenario). Model:
qwen-claude:latest via Ollama.

Condition A (question-required): 2 importantUnknowns with broad
descriptions + narrowing questions, 8 nodes, valid validation.

Condition B (question-free): 1 importantUnknown with structurally
narrowed combined-timeline description, 7 nodes, partial validation.

Neither condition preserved the original four-domain structure of
HS-03C. Narrowing occurred through different mechanisms in each
condition — hypothesis unconfirmed for this scenario shape.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-12 07:54:35 +01:00
robbond 742d7c319b docs(confidence-engine): record CE-ULI-09 paired observation — question-free reconstruction valid through canonical path (single paired comparison) 2026-10-12 07:45:13 +01:00
robbondandClaude Code 3b04d05355 feat(confidence-engine): add CE-ULI-08 two-condition uncertainty comparison apparatus
Adds experimental apparatus for testing whether removing the question field
from importantUnknowns in reconstruction output improves semantic fidelity.

New files:
- lib/reconstruction/schema-variant.js (conditionB schema + production passthrough)
- prompts/reconstruct-v0.5-b-noreq.md (alt prompt, identical line count to prod)

Changed files:
- lib/graph/orchestrator.js (pass generationSchema through dependencies)
- lib/analysis.js (accept opts.generationSchema fallback to original schema)
- scripts/start-case-experiment-helper.cjs (--condition B/A flag support)
- tests/scripts/start-case-experiment-helper.test.js (8 new deterministic tests)

43 targeted deterministic tests PASS. Production callers use original prompt/schema unchanged.
No live model calls. No full suite evidence available.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-12 07:33:31 +01:00
robbond a54b36a2b2 test(confidence-engine): extract focused target input for deterministic verification 2026-10-09 16:44:09 +01:00
robbondandClaude Code acf0665f23 docs(confidence-engine): record Experiment 9 evidence — explicit dependency preservation observed (PARTIAL fidelity)
HS-03C experiment with explicitly raised cross-area dependencies showed the model preserved relationships between recruitment, software integration and customer commitments while retaining the broader feasibility uncertainty. Result: PARTIAL semantic fidelity. Evidence at .evidence-temp/headless-semantic/HS-03C-focused-E9.json.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-09 14:41:51 +01:00
robbond 0698a1a61e feat(confidence-engine): add saveFocusedInvestigationEvidence persistence operation (Experiment 8)
Add exported saveFocusedInvestigationEvidence(checkpoint, identity, metadata,
outputPath) to the canonical experiment helper. Validates output path, exactly
one valid deconstruction contribution, required identity fields, and exact input
values. Writes formatted JSON, reads back, verifies deep structural equality.
Rejects ambiguous checkpoints instead of silently selecting a contribution.

Deterministic test covers: positive file write, structured evidence completeness,
input fidelity, identity fidelity, return values, missing path rejection, missing
contribution rejection, ambiguous contribution rejection. All OS temp files cleaned
in finally blocks.

Zero live model calls. Zero browser calls. No production code modified.
2026-10-09 14:33:33 +01:00
robbondandClaude Code aa57d8589c feat(confidence-engine): verify disk persistence of focused-investigation evidence serializer (Experiment 7, Iteration 7)
Extended deterministic test to write serialized evidence record to
OS temp directory, read back from disk, parse JSON, and assert deep
structural equality. Verified: five structured arrays preserved with
correct lengths; exact inputs unchanged; identity fields intact; null
metadata fields survive round-trip. Temp file cleaned in finally block.

Zero live model calls. Zero browser calls. No production code or
canonical-helper modifications. Experiment 4 original structured
output remains unavailable.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Signed-off-by: robbond <rob@rdbsolutions.co.uk>
2026-10-09 14:29:36 +01:00
robbond 7d7a9e8d29 feat(confidence-engine): add serializeFocusedInvestigationEvidence evidence serializer and deterministic test
Apparatus-level pure serialization function that preserves all five
focused-deconstruction arrays (observations, uncertainties, assumptions,
relationships, possibleFollowUpQuestions) plus exact inputs and execution
metadata from the existing checkpoint — without modifying production code.

Deterministic verification: syntax check pass, 79/79 assertions pass
(including 28 new evidence-ser tests), zero live calls.

Experiment 4's missing structured output cannot be reconstructed (never
persisted by the apparatus). No new semantic evidence produced.
2026-10-09 14:25:26 +01:00
robbond 7fd05745e8 feat(confidence-engine): record Experiment 4 synthetic HS-03C focused investigation results — PARTIAL fidelity, semantic narrowing observed
Controlled synthetic experiment (Iteration 4): fixed recruitment scenario with
synthetic answer showed partial preservation of the broader feasibility uncertainty.
The model correctly preserved timing estimates as observations but narrowed the
investigation to hiring pipeline logistics rather than cross-area launch feasibility.
2026-10-09 14:12:35 +01:00
robbond d0f357877d feat(confidence-engine): present canonical uncertainty description as initial display
Changed the facilitator-view-adapter to prefer node.description over
node.openQuestion for unresolved unknown items. This is a reversible
presentation change only — graph identity, reasoning-state ownership,
openQuestion generation, and resolved-unknown presentation are unchanged.

Owning test assertions updated to reflect new display contract:
- open-question-presentation.test.js: assertion target shifts from
  question text to description text
- facilitator-view-adapter.test.js: deduplication tests use distinct
  descriptions so nodes remain distinguishable after the change

Documentation: hypothesis experiment record and handoff return note.
2026-10-09 11:37:43 +01:00
robbond b71e3a8267 docs(confidence-engine): record uncertainty-led investigation hypothesis 2026-10-09 10:26:49 +01:00
robbond d89e0816b6 refactor(reconstruction): strengthen open-question semantic fidelity 2026-10-09 09:12:41 +01:00
robbond fd5241c6af feat(experiments): expose reconstruction open-question evidence 2026-10-09 08:12:06 +01:00
robbond 6bc8be6edb wip(confidence-engine): checkpoint open-question presentation work 2026-10-09 07:46:19 +01:00
robbond f74fe6de10 fix(focused-investigation): suppress incompatible question strategies 2026-10-08 17:48:53 +01:00
robbond 92ae463d10 feat(llm): propagate cancellation through reconstruction providers 2026-10-08 17:45:02 +01:00
robbondandClaude Code 35a6303ce8 test(confidence-engine): enable longitudinal synthesis checkpoint selection
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 10:49:51 +01:00
robbondandClaude Code 57c88b713a test(confidence-engine): establish longitudinal synthesis checkpoints
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 10:33:59 +01:00
robbondandClaude Code 8d9df8c939 test(confidence-engine): expose current understanding experiment output
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 10:16:34 +01:00
robbondandClaude Code ce7df91a42 test(confidence-engine): enable live current understanding experiment execution
Extend the CU synthesis experiment helper with a --live flag that
uses the production provider (getProvider from lib/llm/provider.js)
for live model invocation. Preserves deterministic mode unchanged.

Adds 10 new deterministic tests verifying live-mode wiring via
injected mock provider: fixture pass-through, single invocation,
schema correctness, immutability, persistence safety, and regression
of existing deterministic behaviour. Zero network calls in all tests.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 09:59:03 +01:00
robbondandClaude Code 425e1e7805 test(confidence-engine): capture original current understanding regression fixture
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 09:52:57 +01:00
robbondandClaude Code dd875b5535 test(confidence-engine): add current understanding synthesis experiment helper
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 09:49:09 +01:00
robbondandClaude Code 279b9a2a67 fix(confidence-engine): permit evidenced knowledge gaps in current understanding
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-08 09:41:24 +01:00
robbond f1fae621f1 test(confidence-engine): remove Iteration 16 unsubstantiated target-scoped contribution behaviour section
Remove the 140-line Iteration 16 test section (commit 5d4e7b5) that
represented manually supplied checkpoint values as evidence of
target-scoped contribution behaviour. The section used extractLiveExperimentCheckpoint
to construct mock result objects simulating multi-episode state transitions
but could not observe contributions — the deterministic seam has no
contributions field in its return shape.

Preserves: all pre-existing checkpoint tests, Iteration 15 source-level
assertions, synthetic CU synthesis tests, and existing helper imports.
2026-10-08 07:41:40 +01:00
robbond 5d4e7b5e94 test(confidence-engine): verify target-scoped contribution behaviour via deterministic seam
Add Iteration 16 test extension that documents the behavioral evidence
boundary for target-scoped contribution filtering:

- Observable via extractLiveExperimentCheckpoint:
  * Node A pre-episode graph state (graphCheckpoint.nodeCount)
  * B's fresh deconstruction trigger (providerGenerateCalls increments)
  * Graph/findings continuity (reasoningState turnCount advanced)

- Blocked from deterministic seam (no contributions field in checkpoint):
  * Historical contribution retention for non-current targets
  * B contribution associated with targetNodeId
  * Both A and B surviving in postEpisodeContributions
  * B prep scope receiving B-specific contribution

Uses mock result objects to simulate the two-target sequence
(n_enterprise_customer_signing -> n_product_launch_decision) and
explicitly documents which assertions can and cannot be verified.
2026-10-08 07:39:22 +01:00
robbond 99f200303a fix(confidence-engine): scope contribution filtering to targetNodeId in composeDeepInvestigationCheckpoint
Historical contributions for a different Open Question node no longer
suppress fresh focused deconstruction of the current target. Only
contributions whose targetNodeId matches the current target are counted
towards suppressing redundant deconstruction.
2026-10-08 07:34:28 +01:00
robbond ddd1c7bd6f test(confidence-engine): cover CU diagnostics and investigation checkpoints 2026-10-08 06:45:37 +01:00
robbond 20b416dcb6 test(confidence-engine): report final deep provider count 2026-10-07 07:43:08 +01:00
robbond 44db12d9c0 test(confidence-engine): forward deep provider schemas 2026-10-07 07:25:52 +01:00
robbond 0b84f8ad28 test(confidence-engine): expose CU synthesis failure shape 2026-10-07 06:59:43 +01:00
robbond d6dbc17bdf fix(confidence-engine): await focused deconstruction provider 2026-10-06 18:18:10 +01:00
robbond 814360e368 test(confidence-engine): use canonical focused deconstruction 2026-10-06 17:28:40 +01:00
robbond 9362c78f81 refactor(confidence-engine): extract focused deconstruction seam 2026-10-06 17:06:44 +01:00
robbond 8e3f4388d6 test(confidence-engine): preserve focused investigation evidence seam 2026-10-05 09:35:04 +01:00
robbond e996a542b3 test(confidence-engine): align deep investigation episode seam 2026-10-05 08:06:28 +01:00
robbond ca284d9c04 test(confidence-engine): preserve deep investigation episode state 2026-10-05 07:54:17 +01:00
robbond 90649c0495 test(confidence-engine): compose deep CU checkpoints 2026-10-05 06:44:29 +01:00
robbond 72b10d0050 test(confidence-engine): capture deep-investigation checkpoints 2026-10-05 06:40:28 +01:00
robbond 97c7482177 docs(confidence-engine): checkpoint report learning semantics 2026-10-02 17:20:29 +01:00
robbondandClaude Code b8c2909b5f feat(confidence-engine): show investigation learning in report
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-02 08:26:07 +01:00
robbond b448f522f6 feat(confidence-engine): deriveInvestigationLearning deterministic learning projection from canonical provenance
Adds deriveInvestigationLearning to episode-preparation.js: pure function that maps resolvedNodeIds + Contributions + Findings to investigation learning propositions, preserving node.label question text and contribution-provenance ordering.

Owning tests prove: correct provenance, cross-question isolation, eligibility rule (null/agree eligible; not_quite/not_relevant excluded), unresolved exclusion, no-eligible-material omission, text preservation, and provenance authority over originatingTargetNodeId shortcut.
2026-10-02 08:18:19 +01:00
robbond 30648ca5b3 docs(confidence-engine): checkpoint plausible zero-state verification 2026-10-02 07:12:36 +01:00
robbond 0cc9d087c3 fix(confidence-engine): permit empty plausibleInterpretations for honest zero state
- Zod validator: plausibleInterpretations accepts empty string (zero interpretations) while remaining required and string-typed
- Prompt instruction: SECTION B now explicitly requests "" instead of absence prose when no plausible inputs exist
- Report generation/update paths derive hasPlausibleInterpretations from actual synthesized data rather than hard-coding true
- Tests: updated validation contract, added hasPlausibleInterpretations derivation assertions, updated seam tests to use derived values
2026-10-02 06:48:24 +01:00
robbond 8dc6e58d43 docs(confidence-engine): checkpoint verified three-part report 2026-10-01 17:13:10 +01:00
robbondandClaude Code 56b5f0099c feat(confidence-engine): show unresolved report questions
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-01 14:20:01 +01:00
robbond 7c1d657761 docs(confidence-engine): record product-launch epistemic A/B result — 4-iteration loop complete
Controlled A/B (zero-Findings scenario, Qwen): unresolved uncertainty exposure
did not contaminate supported understanding or plausible interpretation.
No production changes. No report semantic changes.
2026-09-30 17:22:52 +01:00
robbond cf09a72739 feat(confidence-engine): add --variant hidden|visible to overview experiment helper for controlled A/B
- Helper now supports --variant visible which appends unresolved Open Questions
  as a separate NON-EVIDENCE section to the baseline prompt (built via production
  buildOverviewSynthesisPrompt)
- Default (--variant hidden, or no flag) remains production path
- No production code changed; apparatus-only extension
- Deterministic A/B control proven by existing 34/34 test suite
2026-09-30 17:20:28 +01:00
robbond 9a56176323 fix(confidence-engine): supply overview-specific output schema to generateReconstruction
synthesizeInvestigationOverview() called provider.generateReconstruction(prompt, modelName)
without an output contract. Without a structured-output schema the provider defaults to its
initial-reconstruction format which lacks { understanding, plausibleInterpretations }, causing
'Missing or invalid required fields' validation failure.

Add overviewOutputSchema (JSON-schema matching validateOverviewResponse contract) and pass as
third argument, matching the proven pattern in synthesizeCurrentUnderstanding().

53/53 tests PASS. Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-30 17:05:47 +01:00
robbond 6d9a7e7603 test(confidence-engine): add canonical overview experiment helper 2026-09-30 16:14:59 +01:00
robbond 3d0c90e64c fix(confidence-engine): unwrap overview provider response
Production: synthesizeInvestigationOverview() now passes providerResult.response
to validateOverviewResponse() instead of the raw envelope, matching the proven
pattern in current-understanding-synthesis.js.

Regression: fakeProvider returns real-provider envelope shape { response, providerApiPath }
and four new regression tests verify unwrapping succeeds with valid response
and fails correctly with invalid response inside the envelope.

Handoff: documented defect confirmed, fix applied, and owning regression added.
2026-09-30 16:06:14 +01:00
robbond 8d268efb12 test(confidence-engine): support zero-finding report experiment case
- Add nullish coalescing on fix.findings ?? [] so absent findings
  array in a fixture is represented as zero Findings [] for the A/B
  input apparatus.

- Append product-launch second controlled case (live-product-launch-
  start-response.json) with zero Findings to the epistemic comparison
  test suite. All 34 tests pass deterministically.

- Update docs/current-handoff.md to record the zero-Findings case.
2026-09-30 13:25:22 +01:00
robbondandClaude Code d387b66c03 test(confidence-engine): clarify controlled report experiment boundary
Retained deterministic input-control test proving that A/B apparatus
correctly holds supportedEvidence, findings, plausibleInterpretations,
scenarioState, and providerConfig constant while only varying unresolved
question visibility. Added buildOverviewSynthesisPrompt identity proof
that full vs filtered graph produce identical prompt text.

Corrected handoff semantics: deterministic input filtering does NOT prove
model output behaviour; live two-call semantic comparison remains required.
Preserved prior imperfect live evidence with correct qualification about
uncontrolled plausibleInterpretations divergence.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-30 12:30:36 +01:00
robbondandClaude Code ada83de1db test(confidence-engine): control report epistemic comparison inputs
Deterministic test (17 cases) proves A/B inputs are correctly controlled:
- supportedEvidence identical in A and B
- findings identical in A and B
- plausibleInterpretations identical (fixed canonical interpretation) in A and B
- scenarioState identical in A and B
- providerConfig identical in A and B
- only unresolvedQuestions visibility differs (A=hidden, B=canonical)
- unresolved questions excluded from evidence and plausible interpretations
- fixed interpretation carries NON-EVIDENCE epistemic status

Also corrects Loop 2 documentation to reflect that three-category
invariance was NOT established (plausibleInterpretations differed
between calls despite identical input), retaining only the narrower
valid finding: no unresolved Open Question was promoted to evidence.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-30 11:44:51 +01:00
robbondandClaude Code ef6592cd91 test(confidence-engine): add progressed investigation experiment fixture
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-30 10:55:13 +01:00
robbond 2dc1cb0fe6 docs(confidence-engine): record tester readiness findings 2026-09-11 10:34:53 +01:00
robbond 2349ef9d59 docs(confidence-engine): close pre-tester security review 2026-09-10 14:21:10 +01:00
robbond a6796c6f73 fix(confidence-engine): bound focused investigation input 2026-09-10 13:23:39 +01:00
robbond 2cb2d556fd fix(confidence-engine): sanitize reasoning error responses 2026-09-10 11:39:13 +01:00
robbond b9d49ff277 fix(confidence-engine): correct legal footer layout 2026-09-10 06:40:30 +01:00
robbond 27ded883e5 feat(confidence-engine): add tester legal pages 2026-09-10 06:18:28 +01:00
robbond 96b47d16a3 fix(confidence-engine): decouple app health from provider 2026-09-09 20:00:52 +01:00
robbond 00d7593ffd fix(confidence-engine): preserve scenario during outage 2026-09-09 19:45:55 +01:00
robbond 60c90bdd6a fix(confidence-engine): propagate start outage classification 2026-09-09 19:36:29 +01:00
robbond 707fe1b3c0 fix(confidence-engine): sanitize start reasoning outage 2026-09-09 18:02:29 +01:00
robbond ed033e71d5 fix(confidence-engine): sanitize focused reasoning outage 2026-09-09 17:34:56 +01:00
robbond 6974b710de fix(confidence-engine): refine landing page layout 2026-09-09 16:33:06 +01:00
robbond b93aad9667 fix(confidence-engine): simplify responsive login layout 2026-09-09 16:15:55 +01:00
robbond 5f423d2f5f fix(login): correct desktop layout — sections 1+3 form left column, sign-in card right at 420px; mobile order: proposition → sign-in → HOW IT WORKS 2026-09-09 15:21:26 +01:00
robbond 4d78c81773 fix(confidence-engine): prioritise mobile sign-in 2026-09-09 14:20:27 +01:00
robbond efdb0a29d1 docs(confidence-engine): record first-time framing increment 2026-09-09 14:04:36 +01:00
robbond 3bfc1c5c4b feat(confidence-engine): add first-time product framing 2026-09-09 14:04:08 +01:00
robbond fdb6c86ee2 docs(confidence-engine): checkpoint deployed product platform 2026-09-09 13:37:03 +01:00
robbond e18de817b4 build(confidence-engine): ignore deployment environment 2026-09-09 13:30:22 +01:00
robbond 607a2ad98b fix(confidence-engine): use deployment host address in quotes 2026-09-09 13:23:22 +01:00
robbond 3e473dcf55 fix(confidence-engine): use deployment host address 2026-09-09 13:21:47 +01:00
robbond dafb9ae9b7 fix(confidence-engine): bind Jenkins deployment SSH context 2026-09-09 13:19:21 +01:00
robbond b964f2f172 fix(confidence-engine): give Jenkins pipeline node context 2026-09-09 13:02:19 +01:00
robbond 2c535080d2 build(confidence-engine): add manual Jenkins deployment 2026-09-09 12:44:55 +01:00
robbond bb62a65174 fix(confidence-engine): preserve external auth callback origin 2026-09-09 11:26:57 +01:00
robbond 0125975ccb fix(confidence-engine): sanitize public health response 2026-09-09 10:33:54 +01:00
robbond f557175bfb fix(confidence-engine): keep health endpoint public 2026-09-09 10:01:10 +01:00
robbond 896851e68d build(confidence-engine) remove copy line from dockerfile 2026-09-09 09:52:55 +01:00
robbond 2079be690f build(confidence-engine) fix dockerfile env var 2026-09-09 09:47:45 +01:00
robbond e6c78f87fe build(confidence-engine): add production container packaging 2026-09-09 06:35:15 +01:00
robbond d6df1d210e feat(confidence-engine): use server investigation persistence 2026-09-08 19:21:05 +01:00
robbond 6dd447e56a feat(confidence-engine): add authenticated investigation persistence 2026-09-08 17:15:46 +01:00
robbond b949eea831 feat(confidence-engine): define investigation persistence schema 2026-09-08 16:56:07 +01:00
robbond 30bf44f2e5 feat(confidence-engine): establish authenticated product boundary 2026-09-08 16:30:07 +01:00
robbond 1c17452bee feat(confidence-engine): add dark mode 2026-09-08 11:04:49 +01:00
robbond 24d9e466f6 docs(confidence-engine): checkpoint commercially testable product loop 2026-09-08 10:40:21 +01:00
robbond 85b9f4411f fix(confidence-engine): reopen resolved unknowns by graph state 2026-09-08 09:42:46 +01:00
robbond 0b5a38f83d fix(confidence-engine): persist done-for-now episode closure 2026-09-07 18:36:38 +01:00
robbond 7af708159e docs(confidence-engine): record Terra journey boundary fixes 2026-09-07 15:51:18 +01:00
robbond 949a7024b3 fix(confidence-engine): allow no-op episode reconsideration 2026-09-07 15:50:50 +01:00
robbond ae00e70ced fix(confidence-engine): unwrap synthesis provider response 2026-09-07 15:50:50 +01:00
robbond 7548a6af59 fix(confidence-engine): supply synthesis output schema 2026-09-07 13:44:57 +01:00
robbond 3f2e2e05ae fix(confidence-engine): define focused relationship contract 2026-09-07 13:28:38 +01:00
robbond 14630cf6b7 experiment(confidence-engine): trace focused deconstruction failures 2026-09-07 13:08:08 +01:00
robbond 0cbe49913e experiment(confidence-engine): trace live openai schema boundary 2026-09-07 12:51:13 +01:00