Commit Graph
82 Commits
Author SHA1 Message Date
robbond a5dd9d3f1a prompt: add existing-first uncertainty fallback
Add one explicit action-order rule in Additional Guidance for when
rule #6 applies to explicitly unresolved uncertainty:

1. First check whether an existing unresolved node already represents
   the same uncertainty.
2. If so, update/refine that existing structure rather than creating
   a duplicate.
3. If no such node exists, add a new unknown that directly represents
   the unresolved uncertainty.
4. Do not use an edge alone to represent a previously unrepresented
   uncertainty.

14 focused prompt tests verify: existing-first ordering, reuse path,
fallback-to-add, related-node-insufficient, edge-only-prohibited,
possibleInference separation, resolution path preserved, duplicate
contract preserved, scope uncertainty-only, fidelity/traceability
preserved, noop validator untouched, no semantic classifier added.
2026-08-11 14:12:42 +01:00
robbond 359ccc4ba9 prompt: remove semantic-only mutation conflict 2026-08-11 13:22:29 +01:00
robbond 6adcd817e1 reasoning: require structural progress for supported meaning 2026-08-11 12:32:03 +01:00
robbond 0348921542 experiment: add rejected proposal diagnostics to failure path
Adds rejectedProposalSnapshot to orchestrator diagnostics for
proposal_compatibility rejections — exposing answerMeaning (userSupportedMeaning,
possibleInference), addedNodes structural fields, addedEdges structural fields,
updatedNodes summaries, and resolvedUnknownNodeIds. Diagnostic evidence only;
does not alter validation, mutation, or error messages. Stage-gated to
proposal_compatibility only.
2026-08-11 08:33:36 +01:00
robbond 2e20d30890 correction: delegate hasNodeLevelUserSupport to rawAnswerSupportsUnclassifiedMeaning
v0.15 duplicated the overlap helper logic in hasNodeLevelUserSupport with
reversed argument orientation (unknownText as source, userSupportedMeaning
as candidate) compared to rawAnswerSupportsUnclassifiedMeaning (USM as
source, unknownText as candidate). This produced different accept/reject
outcomes when the two texts have very different token counts.

The fix replaces the independent reconstruction with a single call to the
canonical helper, ensuring node-level and answer-meaning alignment use
identical semantics. Four boundary regression tests verify:

- Boundary A: overlap ratio < 0.4 but >= 3 shared tokens → accept (token rule)
- Boundary B: short candidate / long source accepted via structural linkage
- Boundary B control: unrelated unknown rejected with no structural edge
- Boundary C: ratio exactly at 0.4 threshold accepts via ratio rule
2026-08-10 20:09:39 +01:00
robbond 9e869cbcb8 reasoning: admit verified user-supported unknowns without provenance edges 2026-08-10 19:27:54 +01:00
robbond 0d15dd1f42 Revert "reasoning: require corroboration for conjunction compoundness"
This reverts commit 60048a5636.
2026-08-10 15:07:30 +01:00
robbond 60048a5636 reasoning: require corroboration for conjunction compoundness 2026-08-10 14:53:57 +01:00
robbond 4c5666dfb3 reasoning: suppress explanation question without relationship structure 2026-08-10 10:17:35 +01:00
robbond 69efc5d1b9 reasoning: ground unclassified answers without category expansion 2026-08-10 08:20:00 +01:00
robbond 7e4c506614 reasoning: prevent unsupported comparison decomposition 2026-08-10 07:44:40 +01:00
robbond 46503b4507 reasoning: normalise graph relationship contract at proposal boundary 2026-08-10 06:16:47 +01:00
robbond 4e4d0fa732 reasoning: stop answer fidelity guard blocking valid unclassified answers 2026-08-09 19:55:35 +01:00
robbond f861e2cac0 reasoning: preserve evidence versus clarification distinction 2026-08-09 16:20:06 +01:00
robbond c40d8c6d49 test: fix canonical live experiment harness import 2026-08-09 12:29:28 +01:00
robbond e6f784261b test: establish canonical live reasoning experiment harness 2026-08-09 11:56:10 +01:00
robbond 4aa1492c8d refine raw-answer boundary for answer meaning 2026-08-09 11:31:27 +01:00
robbond 3e78d57aca refine answer meaning derivation for negation and qualification 2026-08-09 09:48:08 +01:00
robbond 7965375aff refine answerMeaning contract around user-supported meaning 2026-08-09 08:50:37 +01:00
robbond 36faf70a08 refine answerMeaning support category normalisation 2026-08-09 08:06:38 +01:00
robbond 0d7ad5775c reasoning: add proposal-level answer meaning guard
- Add answerMeaning schema with supportCategory and resolutionGuidance enums
- Add pre-mutation guard that validates proposal alignment with answerMeaning
- Update prompt builder to instruct the model on answerMeaning contract
- Add tests for schema, guard logic, parsing defaults, and regression cases A-D
2026-08-09 07:07:21 +01:00
robbond d0b9be5fe6 experiment: separate stated meaning from model inference 2026-08-07 12:33:49 +01:00
robbond 6af9418eeb experiment: test grounding of decision relevance 2026-08-07 12:00:10 +01:00
robbond db5af97c98 experiment: test wording effects on ambiguous relevance 2026-08-07 11:03:33 +01:00
robbond 69d0216250 experiment: test domain priors in ambiguous relevance 2026-08-07 10:28:43 +01:00
robbond 9171844f5b experiment: test decision-relevance ambiguity handling 2026-08-07 10:15:40 +01:00
robbond 08c8f74bde experiment: test decision-relevance category boundary 2026-08-07 10:02:03 +01:00
robbond 34f06f2919 experiment: test decision-relevance normalisation 2026-08-07 09:46:21 +01:00
robbond b42a1ff244 experiment: separate semantic meaning from relevance labels 2026-08-07 09:00:46 +01:00
robbond 8ee1f575f7 experiment: run small semantic decision-relevance probe 2026-08-07 08:28:06 +01:00
robbond 690d4920d2 experiment: recover semantic evaluation configuration 2026-08-07 07:54:01 +01:00
robbond a88b9410a7 experiment: test semantic decision relevance 2026-08-07 07:24:35 +01:00
robbond 60dbbc0635 experiment: test decision-relative coherence 2026-08-07 07:13:54 +01:00
robbond 85fd90af1d experiment: test initial graph edge coherence (Exp 50)
Passive diagnostic. Coherent and scattered inputs produce identical
edge topology — every unknown connects to the summary node (kind=state)
via depends_on regardless of semantics. Shared edges are wiring, not
coherence evidence.
2026-08-07 06:51:13 +01:00
robbond 6f00a5e567 experiment: test production shared-anchor pattern (Exp 49)
Creates tests/graph/shared-anchor-production-path.test.js (36 tests, all pass).

Experiment 49 asks whether any sequence of real production updates via
applyValidatedProposal creates two or more active unknowns sharing the same
populated relationship anchor. Two sequential-update scenarios (Cases A & B)
consistently returned separate_anchors or insufficient_data — no shared
anchor observed in tested flows.

Control cases C–F confirm: diagnostic correctly distinguishes shared vs
separated patterns on controlled fixtures; all produced nodes/edges pass
schema validation; resolving one node does not mutate another (immunity);
decomposition children share parent anchor correctly.

Combined regression suite: 78 tests across Exp 47 (26), Exp 48 (16),
Exp 49 (36) — all passing, no production code modified.
2026-08-07 06:25:17 +01:00
robbond b1c69e9303 experiment: audit unknown relationship population
Experiment 48 passively audited whether real graph updates populate usable
unknown relationships. Three production paths inspected:

- buildInitialGraph: does NOT populate dependsOn/affects/parentId (only edges)
- buildEmergentReasoningUnknown: DOES populate dependsOn and parentId
- buildCompositeUnknownChildren: DOES populate parentId

One test file created (16 tests, all pass). Diagnostic confirms shared-anchor
coherence is structurally supportable through Path 2 only, requiring at least
two active unknowns with shared references. Conclusion: Insufficient Data for
the initial-build path; production code correctly populates fields in emergent
path but requires comparable observations to trigger.
2026-08-06 19:48:18 +01:00
robbond 273f715ae0 fix: complete scope-aware condition status evaluation
Handle the actual long-investigation fixture wording without rewriting conditions or evidence.

Fixes:
- Add 'is achievable' to future-feasibility phrase list so present-state evidence correctly leaves future conditions unresolved (different_timeframe scope)
- Add 'european equivalent' to differentiation related keywords so observation-5 evidence directly shares the differentiation concept with the condition (direct_match scope)

Updates:
- decision-condition-status tests to use present-state condition text where needed, and correct expectations for the two actual fixture cases
- Evidence-condition-scope tests for both actual fixture examples
- Design evolution log with Experiment 25B findings confirming long-investigation statuses
2026-08-06 13:05:29 +01:00
robbond eb12a9ce49 experiment: qualify condition status by evidence scope 2026-08-06 12:40:07 +01:00
robbond 1a9a9a94fe experiment: compare evidence and condition scope 2026-08-06 11:36:42 +01:00
robbond da291c715b experiment: derive condition status from answer evidence 2026-08-06 11:17:17 +01:00
robbond aabb797e5d experiment: classify answer evidence direction
Move EVIDENCE_DIRECTION_GROUPS out of the mock fixture library into
lib/graph/evidence-direction.js where it belongs. Remove unused
DECISION_CONDITIONS and CONTRADICTION_KEYWORDS exports from scenarios.

Add Experiment 24A entry to the design log.
2026-08-06 10:32:48 +01:00
robbond 3119635211 experiment: assess decision condition status 2026-08-06 09:04:06 +01:00
robbond 32aa3f237a experiment: test questions against decision conditions 2026-08-06 08:36:44 +01:00
robbond 89650b44df experiment: test question relevance against decision target 2026-08-06 06:19:09 +01:00
robbond 7c3d1e7355 experiment: evaluate question importance across long investigation 2026-08-05 19:56:06 +01:00
robbond 445aaa7b37 experiment: add passive question importance test 2026-08-05 19:48:11 +01:00
robbond fe6a9925cb fix: stabilise multi-turn question progression 2026-08-03 13:55:39 +01:00
robbond 34c25fcb43 fix: reselect after reasoning pattern filtering 2026-08-03 12:39:01 +01:00
robbond c27320984c feat: enforce reasoning pattern consistency 2026-08-03 12:10:57 +01:00
robbond 3e2edd2edc fix: continue question selection after graph updates 2026-08-03 11:28:49 +01:00