Commit Graph
100 Commits
Author SHA1 Message Date
robbond ea7f227974 experiment: diagnose material-question specificity 2026-08-13 13:15:22 +01:00
robbond 229fbfbfd9 experiment: test decision chain across product launch 2026-08-13 13:04:09 +01:00
robbond 43b9e5a35c experiment: validate bounded structural context admission live 2026-08-13 12:56:20 +01:00
robbond a7ca8d712d docs: record bounded structural context admission 2026-08-13 12:49:42 +01:00
robbond d871a8c5c4 fix(reasoning): scope structural context admission 2026-08-13 12:49:42 +01:00
robbond 7f97268f68 experiment: define structural reasoning-context embedding 2026-08-13 10:26:54 +01:00
robbond 48de8b6ce7 experiment: choose reasoning-pattern inheritance boundary 2026-08-13 10:14:19 +01:00
robbond 32e668969e experiment: diagnose decision-pattern kind mismatch 2026-08-13 09:54:42 +01:00
robbond 3a4dda9daf experiment: validate prerequisite-aware question targeting live 2026-08-13 09:45:13 +01:00
robbond 3c6e436e89 docs: record prerequisite-aware question targeting 2026-08-13 09:36:10 +01:00
robbond 54bc48342b fix(reasoning): preserve ready material question target 2026-08-13 09:36:10 +01:00
robbond 854c3aa002 experiment: choose material-factor question alignment 2026-08-13 07:57:26 +01:00
robbond d1fe4ca087 experiment: diagnose material-factor question targeting 2026-08-13 07:49:37 +01:00
robbond 721f1ccb6e experiment: test materiality rule against real unresolved factor 2026-08-13 07:41:09 +01:00
robbond e8e6986d15 experiment: validate decision materiality rule live 2026-08-13 07:33:52 +01:00
robbond b671681ddc docs: record decision materiality rule 2026-08-13 07:24:18 +01:00
robbond 5ce5e7349b feat(reasoning): add decision materiality rule 2026-08-13 07:24:16 +01:00
robbond 5dcaed39df experiment: diagnose decision sufficiency rule
Read-only inspection of 8 files (prompt-builder.js, schema.js, utils.js,
apply-proposal.js, orchestrator.js, experiment-60b1.md, experiment-60b2.md,
current-handoff.md). No code changes.

Key findings:
- Prompt has no independent materiality/sufficiency rule (Rule 20 says null
  selectedQuestion when 'no consequential unresolved unknown' but doesn't define
  what makes an unknown non-consequential)
- Validator performs structural checks only, no evidence sufficiency evaluation
- No cross-option comparison logic in propagateResolvedChildEvidence
- Schema has no materiality or couldChangeDecision field
- 60B.1 resolved WITH 'no other material differences' cue; 60B.2 continued
  WITHOUT it, despite internally computing ~3.6 month payback

Classification: C — NO SUFFICIENCY RULE + CONTINUATION BIAS
Missing distinction: MATERIALITY / DECISION-RELEVANCE RULE
2026-08-13 07:15:01 +01:00
robbond 306f8392a1 experiment: test independent decision sufficiency 2026-08-13 06:47:10 +01:00
robbond 60a1befff7 experiment: test decision sufficiency on option graph 2026-08-13 06:39:12 +01:00
robbond 18979e229b experiment: test downstream option evidence update 2026-08-13 06:30:24 +01:00
robbond 4c25faaa01 feat(60A.7): add reusable decision-options fixture loading in test harness
- Load decisions-options fixture from committed JSON (tests/fixtures/
  pre-anchored-decision-options.json) instead of inline duplicate
- Add runPreAnchoredSimulationWithFixture() helper for decision-options
  mode tests
- Generalize anchor validation from savings-realism-specific to generic
  unresolved unknown check in reproduce-multi-turn-investigation.mjs
- Add experiment documentation (experiment-60a7.md) and handoff note
- All 63 harness tests pass; no production reasoning code changed
2026-08-13 06:24:12 +01:00
robbond 2016a024c5 experiment: rerun option consequence structure once 2026-08-13 06:08:23 +01:00
robbond 56a04ddd0e experiment: test option-specific consequence structure 2026-08-13 05:53:20 +01:00
robbond 3db6f40fdc experiment: validate native option structure live 2026-08-12 19:49:02 +01:00
robbond 57c9f2205e feat: add 'option' node kind and 'contained_in' edge — 60A.3
Implementation of Candidate B (unknown+option) from decision architecture
design in 60A.2. Adds two new primitives to the situation graph:

Schema (lib/graph/schema.js):
- SituationKind.option — a choice available within a decision context
- SituationRelationship.contained_in — links option → its parent unknown context

Prompt rules (lib/graph/prompt-builder.js):
- Section added: Decision Option Structure Rules with 5 numbered instructions
  governing when/how to create option nodes, link them via contained_in,
  attach consequences to specific options, and handle do-nothing alternatives.
  Explicitly forbids alternative_to edges and is_baseline/is_default flags.

Tests (446 new lines):
- schema.test.js: +300 — enum completeness updates, option kind validation,
  contained_in edge validation, native two-option graph fixture (~25 new tests)
- prompt-builder.test.js: +133 — focused rules verification for all 5 rule points,
  negative checks (no relocation/savings/example-specific wording, no alternative_to
  requirement, baseline flag prohibition context)

No production code paths affected beyond the two enum additions; existing node and
edge kinds remain unchanged. No Ollama calls, no live API calls.
2026-08-12 19:39:58 +01:00
robbond 6dd9afbf6b experiment: choose minimum decision representation 2026-08-12 18:34:18 +01:00
robbond e6cf973d2a exp 60A.1: read-only vocabulary adequacy diagnosis for alternatives and decisions
Diagnoses the root cause of the persistent pattern from 59B.2-59B.4
where explicit dual-option input collapsed into a single undifferentiated
unknown node. Concludes the graph vocabulary lacks first-class primitives
for options/decisions (not primarily a prompt issue). Identifies three
missing primitives: option node kind, decision node kind, alternative_of
edge type. Recommends ~25-line schema addition for 60A.2 implementation.
2026-08-12 18:19:29 +01:00
robbond ec32713c31 experiment: test explicit two-option decision structure 2026-08-12 17:52:38 +01:00
robbond 2cd346423d experiment: test do-nothing baseline representation 2026-08-12 17:42:25 +01:00
robbond 70688f91c9 experiment: test independent decision relevance 2026-08-12 17:18:18 +01:00
robbond 7f27fccadc experiment: test decision relevance and do-nothing baseline 2026-08-12 17:10:01 +01:00
robbond 8856e66147 experiment: test known-vs-uncertain consequence structure 2026-08-12 16:59:56 +01:00
robbond c3c5351143 experiment: test trade-off decomposition 2026-08-12 16:51:39 +01:00
robbond 8c5c4b5b75 experiment: test shift into trade-off reasoning 2026-08-12 16:37:52 +01:00
robbond 3f1bf7bcb0 experiment: test verified uncertainty resolution 2026-08-12 16:26:43 +01:00
robbond 52c529a688 experiment: test qualified evidence uncertainty status 2026-08-12 16:02:28 +01:00
robbond 201f259326 experiment: exercise interrogative question rendering live 2026-08-12 15:35:00 +01:00
robbond 6f2c09cd94 experiment: validate question-formulation fix live 2026-08-12 15:29:38 +01:00
robbond 870d6caa05 docs: record question-formulation fix 2026-08-12 15:07:13 +01:00
robbond fa42a2643a fix(graph): preserve grammar for question-like unknown labels 2026-08-12 15:02:56 +01:00
robbond b1914f5da7 experiment: test next-question formulation 2026-08-12 13:53:03 +01:00
robbond 20e4b58440 experiment: test evidence preservation with one uncertainty 2026-08-12 13:46:29 +01:00
robbond 32184694c5 experiment: test qualified-answer reasoning 2026-08-12 13:35:40 +01:00
robbond a78f3edb10 experiment: choose declaration recovery boundary 2026-08-12 13:27:02 +01:00
robbond a40a3e343e experiment: diagnose null semantic mutation path 2026-08-12 13:19:37 +01:00
robbond eaf3194752 experiment: observe direct meaning/action fields on anchored update 2026-08-12 12:09:32 +01:00
robbond 6a04d62800 docs: record accepted answer-meaning capture 2026-08-12 12:01:16 +01:00
robbond c4431997b1 tooling: capture accepted answer meaning directly 2026-08-12 12:00:18 +01:00
robbond d2891730af experiment: rerun incremental meaning on anchored uncertainty 2026-08-12 11:19:47 +01:00
robbond f23e2b2de0 docs: record update-only previous-question fix 2026-08-12 10:41:31 +01:00
robbond 8526aa4b69 tooling: supply anchored previous question in update-only mode 2026-08-12 10:41:02 +01:00
robbond 70db093cb1 experiment: test incremental meaning on existing uncertainty 2026-08-12 10:34:53 +01:00
robbond ce01e70010 tooling: add pre-anchored update-only mode to canonical harness
Add FIXTURE_MODE=updateOnly support that bypasses Start and sends the
committed fixture (tests/fixtures/pre-anchored-update-savings-realism.json)
directly as an Update request body through production HTTP route.

scripts/reproduce-multi-turn-investigation.mjs:
  - Added ESM imports for deterministic fixture loading (fs, fileURLToPath, path)
  - Added FIXTURE_PATH constant pointing to committed fixture
  - Added fixtureMode env-var selector and runUpdateOnlyMode() function
  - Validates ANSWER_2 before any live call (zero calls if missing)
  - Verifies single savings-realism anchor invariant on load
  - Preserves all hardened capture fields in pre-anchored mode
  - Normal-mode Start→Update chain preserved under guard clause

tests/reproduce-multi-turn-investigation.harness.test.js:
  - Added 7 new harness tests for pre-anchored scenarios (46 total, all pass)
  - Updated runPreAnchoredSimulation to persist rejectedProposalSnapshot on rejection
  - Added runPreAnchoredSimulationWithBlock() helper

docs/:
  - New docs/experiment-57j78.md with full apparatus description
  - Updated docs/current-handoff.md with 57J.78 section
2026-08-12 10:19:09 +01:00
robbond 9b7721c610 experiment: audit pre-anchored live apparatus 2026-08-12 09:56:11 +01:00
robbond 85fb2b4256 experiment: validate controlled structural no-op live 2026-08-12 09:35:05 +01:00
robbond 8184e050c8 docs: record pre-anchored update apparatus 2026-08-12 09:13:40 +01:00
robbond d77a1ff04d tooling: add pre-anchored update fixture 2026-08-12 09:11:20 +01:00
robbond f78061c1db experiment: validate intentional structural no-op live 2026-08-12 08:49:40 +01:00
robbond 67699ecd03 docs: record structural action capture hardening 2026-08-12 08:33:06 +01:00
robbond beef434a6f tooling: capture structural action declaration in live harness 2026-08-12 08:32:21 +01:00
robbond fc06ff02e4 experiment: rerun structural action contract live 2026-08-12 08:26:58 +01:00
robbond 4de871092f docs: record structural action guard cleanup 2026-08-12 08:17:28 +01:00
robbond bd3c7d59ae fix(graph): make structural action contract authoritative 2026-08-12 08:15:41 +01:00
robbond c899ad620c experiment: validate structural action contract live 2026-08-12 08:04:41 +01:00
robbond 1b3bbd59aa docs: record structural action contract implementation 2026-08-12 07:29:28 +01:00
robbond 6aef806845 feat(graph): add structuralActionRequired contract (57J.67)
- Add structuralActionRequired field to graphUpdateSchema (optional boolean nullable)
- Validate declaration consistency in validateGraphUpdate():
  - true requires meaningful mutation (addedNodes/updatedNodes/addedEdges)
  - false permits intentional no-op when userSupportedMeaning populated
  - null/absent with meaning → reject
  - true/false mismatch on output shape → reject
  - preserve legacy no-op guard for non-contract paths
- Update prompt-builder: add field to required list, insert contract section between rules and Additional Guidance with two mandatory sentences
- 50 new tests: schema validation (4), prompt builder content checks (10), utils contract matrix (10), plus 26 existing suite migrations

All 197 graph tests pass.
2026-08-12 07:26:26 +01:00
robbond 5f9e8ebe33 experiment: finalize semantic action contract semantics 2026-08-12 06:30:40 +01:00
robbond 51da4b973f experiment: define semantic action contract placement 2026-08-12 06:23:21 +01:00
robbond 9425e7b2d6 experiment: define semantic action contract 2026-08-12 06:13:15 +01:00
robbond d7cb343838 experiment: diagnose semantic-to-mutation action ownership 2026-08-12 05:53:58 +01:00
robbond f022d6f4ad experiment: rerun equivalent uncertainty identity with hardened capture 2026-08-12 05:48:18 +01:00
robbond 47509d307b docs: record accepted-update capture hardening 2026-08-11 19:43:11 +01:00
robbond bf959bb9a0 tooling: retain accepted update experiment evidence 2026-08-11 19:42:43 +01:00
robbond 929486c354 experiment: validate equivalent uncertainty identity live 2026-08-11 19:30:08 +01:00
robbond 2927509ca5 experiment: validate selected-question contract live 2026-08-11 19:15:51 +01:00
robbond 3baa77eb72 docs: record selected-question contract alignment 2026-08-11 18:51:51 +01:00
robbond bf1f219256 prompt: require candidate question for new unknowns 2026-08-11 18:51:22 +01:00
robbond a8732288eb docs: record selected-question ownership diagnosis 2026-08-11 18:49:51 +01:00
robbond f0e0fd54de experiment: diagnose selected-question ownership 2026-08-11 18:45:00 +01:00
robbond f25b1f550e experiment: validate equivalent uncertainty reuse live 2026-08-11 18:29:02 +01:00
robbond eb524d08e1 experiment: validate uncertainty identity live 2026-08-11 18:02:11 +01:00
robbond a476431048 docs: record uncertainty identity clarification 2026-08-11 17:47:45 +01:00
robbond b8e6745c15 prompt: distinguish uncertainty identity from topical overlap 2026-08-11 17:47:12 +01:00
robbond f0cf85d2b6 experiment: diagnose uncertainty identity vs relatedness 2026-08-11 17:41:23 +01:00
robbond 19c00f3bf3 experiment: test structured-fidelity multi-turn progress 2026-08-11 17:34:54 +01:00
robbond 5947ccb642 experiment: validate structured semantic fidelity live 2026-08-11 17:18:59 +01:00
robbond f156bf5ef8 docs: record structured semantic fidelity implementation 2026-08-11 16:46:30 +01:00
robbond 7d06cd3c47 reasoning: use structured semantic fidelity contract 2026-08-11 16:45:00 +01:00
robbond b6a232ff6f experiment: choose structured fidelity migration 2026-08-11 16:17:55 +01:00
robbond f330421294 experiment: assess structured semantic fidelity boundary
- Experiment 57J.49: read-only architecture diagnosis showing EXISTING STRUCTURE IS PARTIAL
- All classification enums (supportCategory) and resolution enums (resolutionGuidance) already exist in production schema
- Gap is population (prompt says leave null if unsure) + enforcement (no enum constraint on Zod fields)
- Corrected 57J.48 overstatement of SUFFICIENT → PARTIAL sufficiency in handoff
2026-08-11 15:04:47 +01:00
robbond a2c790ee80 experiment: diagnose uncertainty fidelity false positive 2026-08-11 14:56:15 +01:00
robbond 174e581c23 experiment: convergence-test uncertainty action selection 2026-08-11 14:35:50 +01:00
robbond 94ca1b92f7 docs: experiment 57J.46 record and handoff update 2026-08-11 14:15:50 +01:00
robbond a5dd9d3f1a prompt: add existing-first uncertainty fallback
Add one explicit action-order rule in Additional Guidance for when
rule #6 applies to explicitly unresolved uncertainty:

1. First check whether an existing unresolved node already represents
   the same uncertainty.
2. If so, update/refine that existing structure rather than creating
   a duplicate.
3. If no such node exists, add a new unknown that directly represents
   the unresolved uncertainty.
4. Do not use an edge alone to represent a previously unrepresented
   uncertainty.

14 focused prompt tests verify: existing-first ordering, reuse path,
fallback-to-add, related-node-insufficient, edge-only-prohibited,
possibleInference separation, resolution path preserved, duplicate
contract preserved, scope uncertainty-only, fidelity/traceability
preserved, noop validator untouched, no semantic classifier added.
2026-08-11 14:12:42 +01:00
robbond 96b855b8e7 experiment: choose structural action-selection rule 2026-08-11 14:01:42 +01:00
robbond acd1928ebb experiment: validate conflict-free mutation prompt live 2026-08-11 13:36:03 +01:00
robbond 45d7b96827 docs: experiment 57J.43 record and handoff update 2026-08-11 13:23:14 +01:00
robbond 359ccc4ba9 prompt: remove semantic-only mutation conflict 2026-08-11 13:22:29 +01:00
robbond 0c477adea9 experiment: diagnose structural-mutation prompt compliance 2026-08-11 13:15:27 +01:00