Commit Graph
557 Commits
Author SHA1 Message Date
robbond 854c3aa002 experiment: choose material-factor question alignment 2026-08-13 07:57:26 +01:00
robbond d1fe4ca087 experiment: diagnose material-factor question targeting 2026-08-13 07:49:37 +01:00
robbond 721f1ccb6e experiment: test materiality rule against real unresolved factor 2026-08-13 07:41:09 +01:00
robbond e8e6986d15 experiment: validate decision materiality rule live 2026-08-13 07:33:52 +01:00
robbond b671681ddc docs: record decision materiality rule 2026-08-13 07:24:18 +01:00
robbond 5ce5e7349b feat(reasoning): add decision materiality rule 2026-08-13 07:24:16 +01:00
robbond 5dcaed39df experiment: diagnose decision sufficiency rule
Read-only inspection of 8 files (prompt-builder.js, schema.js, utils.js,
apply-proposal.js, orchestrator.js, experiment-60b1.md, experiment-60b2.md,
current-handoff.md). No code changes.

Key findings:
- Prompt has no independent materiality/sufficiency rule (Rule 20 says null
  selectedQuestion when 'no consequential unresolved unknown' but doesn't define
  what makes an unknown non-consequential)
- Validator performs structural checks only, no evidence sufficiency evaluation
- No cross-option comparison logic in propagateResolvedChildEvidence
- Schema has no materiality or couldChangeDecision field
- 60B.1 resolved WITH 'no other material differences' cue; 60B.2 continued
  WITHOUT it, despite internally computing ~3.6 month payback

Classification: C — NO SUFFICIENCY RULE + CONTINUATION BIAS
Missing distinction: MATERIALITY / DECISION-RELEVANCE RULE
2026-08-13 07:15:01 +01:00
robbond 306f8392a1 experiment: test independent decision sufficiency 2026-08-13 06:47:10 +01:00
robbond 60a1befff7 experiment: test decision sufficiency on option graph 2026-08-13 06:39:12 +01:00
robbond 18979e229b experiment: test downstream option evidence update 2026-08-13 06:30:24 +01:00
robbond 4c25faaa01 feat(60A.7): add reusable decision-options fixture loading in test harness
- Load decisions-options fixture from committed JSON (tests/fixtures/
  pre-anchored-decision-options.json) instead of inline duplicate
- Add runPreAnchoredSimulationWithFixture() helper for decision-options
  mode tests
- Generalize anchor validation from savings-realism-specific to generic
  unresolved unknown check in reproduce-multi-turn-investigation.mjs
- Add experiment documentation (experiment-60a7.md) and handoff note
- All 63 harness tests pass; no production reasoning code changed
2026-08-13 06:24:12 +01:00
robbond 2016a024c5 experiment: rerun option consequence structure once 2026-08-13 06:08:23 +01:00
robbond 56a04ddd0e experiment: test option-specific consequence structure 2026-08-13 05:53:20 +01:00
robbond 3db6f40fdc experiment: validate native option structure live 2026-08-12 19:49:02 +01:00
robbond 57c9f2205e feat: add 'option' node kind and 'contained_in' edge — 60A.3
Implementation of Candidate B (unknown+option) from decision architecture
design in 60A.2. Adds two new primitives to the situation graph:

Schema (lib/graph/schema.js):
- SituationKind.option — a choice available within a decision context
- SituationRelationship.contained_in — links option → its parent unknown context

Prompt rules (lib/graph/prompt-builder.js):
- Section added: Decision Option Structure Rules with 5 numbered instructions
  governing when/how to create option nodes, link them via contained_in,
  attach consequences to specific options, and handle do-nothing alternatives.
  Explicitly forbids alternative_to edges and is_baseline/is_default flags.

Tests (446 new lines):
- schema.test.js: +300 — enum completeness updates, option kind validation,
  contained_in edge validation, native two-option graph fixture (~25 new tests)
- prompt-builder.test.js: +133 — focused rules verification for all 5 rule points,
  negative checks (no relocation/savings/example-specific wording, no alternative_to
  requirement, baseline flag prohibition context)

No production code paths affected beyond the two enum additions; existing node and
edge kinds remain unchanged. No Ollama calls, no live API calls.
2026-08-12 19:39:58 +01:00
robbond 6dd9afbf6b experiment: choose minimum decision representation 2026-08-12 18:34:18 +01:00
robbond e6cf973d2a exp 60A.1: read-only vocabulary adequacy diagnosis for alternatives and decisions
Diagnoses the root cause of the persistent pattern from 59B.2-59B.4
where explicit dual-option input collapsed into a single undifferentiated
unknown node. Concludes the graph vocabulary lacks first-class primitives
for options/decisions (not primarily a prompt issue). Identifies three
missing primitives: option node kind, decision node kind, alternative_of
edge type. Recommends ~25-line schema addition for 60A.2 implementation.
2026-08-12 18:19:29 +01:00
robbond ec32713c31 experiment: test explicit two-option decision structure 2026-08-12 17:52:38 +01:00
robbond 2cd346423d experiment: test do-nothing baseline representation 2026-08-12 17:42:25 +01:00
robbond 70688f91c9 experiment: test independent decision relevance 2026-08-12 17:18:18 +01:00
robbond 7f27fccadc experiment: test decision relevance and do-nothing baseline 2026-08-12 17:10:01 +01:00
robbond 8856e66147 experiment: test known-vs-uncertain consequence structure 2026-08-12 16:59:56 +01:00
robbond c3c5351143 experiment: test trade-off decomposition 2026-08-12 16:51:39 +01:00
robbond 8c5c4b5b75 experiment: test shift into trade-off reasoning 2026-08-12 16:37:52 +01:00
robbond 3f1bf7bcb0 experiment: test verified uncertainty resolution 2026-08-12 16:26:43 +01:00
robbond 52c529a688 experiment: test qualified evidence uncertainty status 2026-08-12 16:02:28 +01:00
robbond 201f259326 experiment: exercise interrogative question rendering live 2026-08-12 15:35:00 +01:00
robbond 6f2c09cd94 experiment: validate question-formulation fix live 2026-08-12 15:29:38 +01:00
robbond 870d6caa05 docs: record question-formulation fix 2026-08-12 15:07:13 +01:00
robbond fa42a2643a fix(graph): preserve grammar for question-like unknown labels 2026-08-12 15:02:56 +01:00
robbond b1914f5da7 experiment: test next-question formulation 2026-08-12 13:53:03 +01:00
robbond 20e4b58440 experiment: test evidence preservation with one uncertainty 2026-08-12 13:46:29 +01:00
robbond 32184694c5 experiment: test qualified-answer reasoning 2026-08-12 13:35:40 +01:00
robbond a78f3edb10 experiment: choose declaration recovery boundary 2026-08-12 13:27:02 +01:00
robbond a40a3e343e experiment: diagnose null semantic mutation path 2026-08-12 13:19:37 +01:00
robbond eaf3194752 experiment: observe direct meaning/action fields on anchored update 2026-08-12 12:09:32 +01:00
robbond 6a04d62800 docs: record accepted answer-meaning capture 2026-08-12 12:01:16 +01:00
robbond c4431997b1 tooling: capture accepted answer meaning directly 2026-08-12 12:00:18 +01:00
robbond d2891730af experiment: rerun incremental meaning on anchored uncertainty 2026-08-12 11:19:47 +01:00
robbond f23e2b2de0 docs: record update-only previous-question fix 2026-08-12 10:41:31 +01:00
robbond 8526aa4b69 tooling: supply anchored previous question in update-only mode 2026-08-12 10:41:02 +01:00
robbond 70db093cb1 experiment: test incremental meaning on existing uncertainty 2026-08-12 10:34:53 +01:00
robbond ce01e70010 tooling: add pre-anchored update-only mode to canonical harness
Add FIXTURE_MODE=updateOnly support that bypasses Start and sends the
committed fixture (tests/fixtures/pre-anchored-update-savings-realism.json)
directly as an Update request body through production HTTP route.

scripts/reproduce-multi-turn-investigation.mjs:
  - Added ESM imports for deterministic fixture loading (fs, fileURLToPath, path)
  - Added FIXTURE_PATH constant pointing to committed fixture
  - Added fixtureMode env-var selector and runUpdateOnlyMode() function
  - Validates ANSWER_2 before any live call (zero calls if missing)
  - Verifies single savings-realism anchor invariant on load
  - Preserves all hardened capture fields in pre-anchored mode
  - Normal-mode Start→Update chain preserved under guard clause

tests/reproduce-multi-turn-investigation.harness.test.js:
  - Added 7 new harness tests for pre-anchored scenarios (46 total, all pass)
  - Updated runPreAnchoredSimulation to persist rejectedProposalSnapshot on rejection
  - Added runPreAnchoredSimulationWithBlock() helper

docs/:
  - New docs/experiment-57j78.md with full apparatus description
  - Updated docs/current-handoff.md with 57J.78 section
2026-08-12 10:19:09 +01:00
robbond 9b7721c610 experiment: audit pre-anchored live apparatus 2026-08-12 09:56:11 +01:00
robbond 85fb2b4256 experiment: validate controlled structural no-op live 2026-08-12 09:35:05 +01:00
robbond 8184e050c8 docs: record pre-anchored update apparatus 2026-08-12 09:13:40 +01:00
robbond d77a1ff04d tooling: add pre-anchored update fixture 2026-08-12 09:11:20 +01:00
robbond f78061c1db experiment: validate intentional structural no-op live 2026-08-12 08:49:40 +01:00
robbond 67699ecd03 docs: record structural action capture hardening 2026-08-12 08:33:06 +01:00
robbond beef434a6f tooling: capture structural action declaration in live harness 2026-08-12 08:32:21 +01:00