Commit Graph
100 Commits
Author SHA1 Message Date
robbond 7e4c506614 reasoning: prevent unsupported comparison decomposition 2026-08-10 07:44:40 +01:00
robbond 513c483501 experiment: locate irrelevant decomposition question boundary 2026-08-10 07:19:52 +01:00
robbond 62593eea14 docs: record canonical live multi-turn product-observation route 2026-08-10 06:49:03 +01:00
robbond 7533e471d4 test: establish minimal live multi-turn investigation route 2026-08-10 06:48:51 +01:00
robbond 46503b4507 reasoning: normalise graph relationship contract at proposal boundary 2026-08-10 06:16:47 +01:00
robbond cf6c5cb57f docs: record stopped experiment 57c contract failure 2026-08-10 06:04:46 +01:00
robbond 371ab0f52f merge(feature/reasoning-guard-generality-v0.9): integrate reasoning-guard generality v0.9 into main 2026-08-09 20:07:05 +01:00
robbond 19a42ca7f7 experiment: validate grounded unclassified answer live 2026-08-09 20:03:30 +01:00
robbond 4e4d0fa732 reasoning: stop answer fidelity guard blocking valid unclassified answers 2026-08-09 19:55:35 +01:00
robbond 7d94c6f73a docs: record contaminated experiment 57a observation 2026-08-09 19:40:49 +01:00
robbond 14d68f1ab7 merge: integrate reasoning-fidelity v0.8 first pass into main 2026-08-09 17:36:58 +01:00
robbond e498bbcc63 docs: close reasoning fidelity v0.8 first pass 2026-08-09 17:25:19 +01:00
robbond ec398dcec9 experiment: validate evidence versus clarification routing 2026-08-09 16:33:14 +01:00
robbond f861e2cac0 reasoning: preserve evidence versus clarification distinction 2026-08-09 16:20:06 +01:00
robbond e884b02e7c experiment: probe user-owned ambiguity boundary 2026-08-09 15:21:02 +01:00
robbond 11882bfaae experiment: probe evidence versus clarification boundary 2026-08-09 15:11:54 +01:00
robbond 85ee4bed30 experiment: probe explicit hard constraint semantic fidelity 2026-08-09 14:13:48 +01:00
robbond 23bfe5f756 experiment: validate unresolved uncertainty after harness repair 2026-08-09 12:55:10 +01:00
robbond c40d8c6d49 test: fix canonical live experiment harness import 2026-08-09 12:29:28 +01:00
robbond 144b7c53f5 experiment: validate unresolved uncertainty live path 2026-08-09 12:20:57 +01:00
robbond b06538ee91 experiment: validate raw-answer safeguard for weak priority 2026-08-09 12:06:50 +01:00
robbond e6f784261b test: establish canonical live reasoning experiment harness 2026-08-09 11:56:10 +01:00
robbond 4aa1492c8d refine raw-answer boundary for answer meaning 2026-08-09 11:31:27 +01:00
robbond c06aecc3f7 docs: update reasoning refinement handoff after experiment 56E 2026-08-09 11:25:43 +01:00
robbond 168ef69074 experiment: record 56D (Regression B) and 56E (weak-priority strengthening) results 2026-08-09 10:49:22 +01:00
robbond 3e78d57aca refine answer meaning derivation for negation and qualification 2026-08-09 09:48:08 +01:00
robbond 7965375aff refine answerMeaning contract around user-supported meaning 2026-08-09 08:50:37 +01:00
robbond 869afee1ab experiment: validate regression B after normalization 2026-08-09 08:25:59 +01:00
robbond 36faf70a08 refine answerMeaning support category normalisation 2026-08-09 08:06:38 +01:00
robbond b2329d8608 experiment: isolate regression B proposal validation 2026-08-09 07:55:11 +01:00
robbond 0d7ad5775c reasoning: add proposal-level answer meaning guard
- Add answerMeaning schema with supportCategory and resolutionGuidance enums
- Add pre-mutation guard that validates proposal alignment with answerMeaning
- Update prompt builder to instruct the model on answerMeaning contract
- Add tests for schema, guard logic, parsing defaults, and regression cases A-D
2026-08-09 07:07:21 +01:00
robbond 8c1036ecd0 docs: map reasoning requirements to production path 2026-08-08 11:08:37 +01:00
robbond 162ead2d69 docs: consolidate reasoning refinement requirements 2026-08-08 09:08:39 +01:00
robbond fcb7218407 experiment: separate stated clarification meaning from inference 2026-08-08 08:57:59 +01:00
robbond 3af623a7d5 experiment: test resolution from preserved answer meaning 2026-08-08 08:42:28 +01:00
robbond 22325d5ac3 experiment: separate answer meaning from resolution 2026-08-08 08:23:19 +01:00
robbond 8c12931b43 experiment: test clarification uncertainty preservation 2026-08-08 07:53:30 +01:00
robbond 67f2b084a5 experiment: test clarification broadening with weak answers 2026-08-08 07:43:19 +01:00
robbond 1d1dceefa3 experiment: test consequence of clarification target broadening 2026-08-08 07:29:36 +01:00
robbond 4302e3c435 experiment: test clarification target specificity 2026-08-08 07:16:55 +01:00
robbond 86c04d0fd8 experiment: test end-to-end clarification chain 2026-08-08 06:54:08 +01:00
robbond 5d1cba80cd experiment: test clarification answer resolution 2026-08-08 06:37:32 +01:00
robbond 8b1d69279f experiment: test clarification question wording 2026-08-08 06:26:17 +01:00
robbond c8ead0f690 experiment: test clarification null stability 2026-08-08 06:13:43 +01:00
robbond 6944f358a9 experiment: identify clarification target 2026-08-07 20:03:27 +01:00
robbond ee1391cd00 experiment: distinguish clarification from evidence needs 2026-08-07 19:28:45 +01:00
robbond dd3b8e505d experiment: test structured evidence to consequence reasoning 2026-08-07 19:02:37 +01:00
robbond cd9328ef8e experiment: test consequence from explicit evidence needs 2026-08-07 18:32:47 +01:00
robbond 10e87d0d44 experiment: test evidence needs across competing hypotheses 2026-08-07 18:14:03 +01:00
robbond 5daee5e911 experiment: test consequence of interpretation disagreement 2026-08-07 18:01:47 +01:00
robbond b9f737a293 experiment: test semantic interpretation disagreement 2026-08-07 17:30:10 +01:00
robbond fb5368ec5f experiment: test semantic grounding stability 2026-08-07 17:13:41 +01:00
robbond 118c5a789f experiment: test semantic grounding of interpretations 2026-08-07 16:32:56 +01:00
robbond 0f7457d9c1 experiment: separate source support from interpretation additions 2026-08-07 16:12:29 +01:00
robbond c2debad66d experiment: test interpretation lineage to user source 2026-08-07 15:58:39 +01:00
robbond 3537aa1b7a experiment: test deterministic user source identity 2026-08-07 15:24:47 +01:00
robbond d2d84656fe experiment: audit evidence source linkage 2026-08-07 15:11:57 +01:00
robbond 455d6f4c84 experiment: audit evidence type provenance 2026-08-07 15:10:03 +01:00
robbond 23f02d6169 experiment: audit referential provenance 2026-08-07 14:39:11 +01:00
robbond 90472de766 experiment: audit provenance in update prompt 2026-08-07 14:07:28 +01:00
robbond ec167f2689 experiment: trace update provenance boundary 2026-08-07 13:39:09 +01:00
robbond 7e74ad86f2 experiment: trace provenance loss through graph pipeline 2026-08-07 13:29:51 +01:00
robbond 5892a9b0f6 experiment: audit graph provenance 2026-08-07 13:01:25 +01:00
robbond d0b9be5fe6 experiment: separate stated meaning from model inference 2026-08-07 12:33:49 +01:00
robbond 6af9418eeb experiment: test grounding of decision relevance 2026-08-07 12:00:10 +01:00
robbond db5af97c98 experiment: test wording effects on ambiguous relevance 2026-08-07 11:03:33 +01:00
robbond 69d0216250 experiment: test domain priors in ambiguous relevance 2026-08-07 10:28:43 +01:00
robbond 9171844f5b experiment: test decision-relevance ambiguity handling 2026-08-07 10:15:40 +01:00
robbond 08c8f74bde experiment: test decision-relevance category boundary 2026-08-07 10:02:03 +01:00
robbond 34f06f2919 experiment: test decision-relevance normalisation 2026-08-07 09:46:21 +01:00
robbond b42a1ff244 experiment: separate semantic meaning from relevance labels 2026-08-07 09:00:46 +01:00
robbond 8ee1f575f7 experiment: run small semantic decision-relevance probe 2026-08-07 08:28:06 +01:00
robbond 690d4920d2 experiment: recover semantic evaluation configuration 2026-08-07 07:54:01 +01:00
robbond a88b9410a7 experiment: test semantic decision relevance 2026-08-07 07:24:35 +01:00
robbond 60dbbc0635 experiment: test decision-relative coherence 2026-08-07 07:13:54 +01:00
robbond 85fd90af1d experiment: test initial graph edge coherence (Exp 50)
Passive diagnostic. Coherent and scattered inputs produce identical
edge topology — every unknown connects to the summary node (kind=state)
via depends_on regardless of semantics. Shared edges are wiring, not
coherence evidence.
2026-08-07 06:51:13 +01:00
robbond 6f00a5e567 experiment: test production shared-anchor pattern (Exp 49)
Creates tests/graph/shared-anchor-production-path.test.js (36 tests, all pass).

Experiment 49 asks whether any sequence of real production updates via
applyValidatedProposal creates two or more active unknowns sharing the same
populated relationship anchor. Two sequential-update scenarios (Cases A & B)
consistently returned separate_anchors or insufficient_data — no shared
anchor observed in tested flows.

Control cases C–F confirm: diagnostic correctly distinguishes shared vs
separated patterns on controlled fixtures; all produced nodes/edges pass
schema validation; resolving one node does not mutate another (immunity);
decomposition children share parent anchor correctly.

Combined regression suite: 78 tests across Exp 47 (26), Exp 48 (16),
Exp 49 (36) — all passing, no production code modified.
2026-08-07 06:25:17 +01:00
robbond b1c69e9303 experiment: audit unknown relationship population
Experiment 48 passively audited whether real graph updates populate usable
unknown relationships. Three production paths inspected:

- buildInitialGraph: does NOT populate dependsOn/affects/parentId (only edges)
- buildEmergentReasoningUnknown: DOES populate dependsOn and parentId
- buildCompositeUnknownChildren: DOES populate parentId

One test file created (16 tests, all pass). Diagnostic confirms shared-anchor
coherence is structurally supportable through Path 2 only, requiring at least
two active unknowns with shared references. Conclusion: Insufficient Data for
the initial-build path; production code correctly populates fields in emergent
path but requires comparable observations to trigger.
2026-08-06 19:48:18 +01:00
robbond 40ef3e108f experiment: test shared-anchor coherence signal
Experiment 47: created a test-only diagnostic helper that inspects existing
graph relationship fields (dependsOn, affects, parentId, childIds on nodes;
fromNodeId/toNodeId + relationship on edges) to distinguish coherent
investigations (multiple unknowns sharing one anchor) from scattered ones.

Three controlled fixtures confirm the helper works: shared_anchor vs
separate_anchors vs insufficient_data — all with identical structural counts
(6 nodes, 4 active unknowns). All three produce identical too_broad output
from the existing assessor, confirming no production code changes needed.

Existing-scenario inspection (3 real scenarios from Exp 39-46) all return
insufficient_data — current data lacks populated relationship fields on
unknown nodes. This means the gap is not purely in assessment logic but also
in upstream data quality.

Closed Experiment 46. Updated design-evolution-log and handoff.
2026-08-06 19:31:43 +01:00
robbond 0de2ffb4be experiment: test scope coherence against unknown count 2026-08-06 19:11:23 +01:00
robbond 1d234acd8c experiment: test too-broad assessment boundary
Experiment 45 — passive boundary experiment measuring the existing
assessor's too_broad threshold from two to five competing unknowns.

Key findings:
- Boundary switches exactly between three and four active unknowns
- Clarify eligibility follows the same boundary
- Resolved-item gate works correctly (1 stays too_broad, 2 clears it)
- 2–3 unknowns return cannot_determine health (not healthy or too_broad)
- Boundary appears mechanically clear but conceptually uncertain

No production code changed. Synthetic fixtures only.
2026-08-06 18:52:40 +01:00
robbond ca71e79618 experiment: test assessor against unclear starting point 2026-08-06 18:31:15 +01:00
robbond ae2d1d9c52 experiment: audit clarify readiness signals
Passive diagnostic: zero Clarify-eligible turns across 10 real-scenario
assessments. Two findings — (1) orienting-based rule is dead code because
assessor never produces phase=orienting, (2) too_broad trigger validly narrow
but untested by any fixture. Created focused test file with 31 assertions.
All regression tests pass: 51 behaviour-selection + 33 reachability + 51
assessor = 166 total.
2026-08-06 18:22:11 +01:00
robbond fc310e77e1 docs: close experiment 42 selector refinement 2026-08-06 18:00:13 +01:00
robbond 05d3d96014 experiment: narrow acknowledge behaviour eligibility 2026-08-06 17:44:37 +01:00
robbond eee8c6b1e4 experiment: compare acknowledge priority alternatives
Experiment 41 compared two passive alternatives for reducing Acknowledge dominance:

Variant A (priority reordering): evaluate Summarise/Pause before Acknowledge
- Converges on concluding→summarise and stalled→pause correctly
- Introduces false-positive summarise at long-investigation t3

Variant B (Acknowledge exclusions): gate Acknowledge via phase/progress/health
- Converges on the same two genuine changes without false-positives
- Recommended: cleaner boundaries, preserves Acknowledge for healthy focus states

Both variants produce identical results for 2 of 7 tested turns.
Variant A diverges at long-investigation t3 (focusing phase with resolvedNodeCount=3).
Variant B correctly preserves Acknowledge there via its exclusion list.

Test files:
- tests/behaviour-selection.counterfactual.test.js (44 tests, new)
No production code changed.
2026-08-06 17:19:02 +01:00
robbond a4731d908f experiment: audit behaviour reachability and blocking 2026-08-06 16:48:41 +01:00
robbond da3d35f437 experiment: validate behaviour selection against real assessments 2026-08-06 16:30:10 +01:00
robbond 51648e4b8f experiment: validate cold-start project recovery 2026-08-06 16:11:37 +01:00
robbond 544573af75 experiment: validate cross-boundary context routing 2026-08-06 15:58:41 +01:00
robbond 7849b2f215 experiment: validate reduced context routing 2026-08-06 15:47:49 +01:00
robbond 73c375d116 experiment: test current handoff maintenance 2026-08-06 15:42:55 +01:00
robbond 1d92aa0b07 experiment: create single return-to-work handoff 2026-08-06 15:37:12 +01:00
robbond b959cfa7a7 experiment: create task-specific context packs 2026-08-06 15:30:14 +01:00
robbond 51f0c11bc8 experiment: separate current principles from architectural aspirations 2026-08-06 15:07:06 +01:00
robbond a4bbe0ef3f experiment: separate ui mock reference from deferred backlog 2026-08-06 14:47:34 +01:00
robbond 78c98fb973 experiment: review deferred project documents
Experiment 30 classified two deferred documents against verified current state:
- architectural-principles.md → keep as task-specific reference (6 current, 4 aspirational, 3 duplicates)
- backlog info.md → retain temporarily pending revision (mixed mock fixtures + deferred UX planning)
Neither moved to archive — both contain material with potential near-term utility.
Created docs/document-role-review.md with evidence, routing test, and return-to-work note.
2026-08-06 14:33:10 +01:00
robbond 97e4f3029e experiment: archive historical project documents 2026-08-06 14:24:52 +01:00
robbond 354ba26aad experiment: verify current project state against implementation 2026-08-06 14:16:57 +01:00
robbond 61c8a3adbd experiment: create current project state entry point 2026-08-06 14:06:20 +01:00