Commit Graph
100 Commits
Author SHA1 Message Date
robbond bda3abf893 experiment: classify captured answer-meaning strengthening 2026-08-11 10:19:19 +01:00
robbond a00f7b170d experiment: inspect rejected proposal live variance 2026-08-11 10:03:35 +01:00
robbond 48e9bcf3eb docs: add Experiment 57J.31 entry to handoff 2026-08-11 08:34:51 +01:00
robbond 0348921542 experiment: add rejected proposal diagnostics to failure path
Adds rejectedProposalSnapshot to orchestrator diagnostics for
proposal_compatibility rejections — exposing answerMeaning (userSupportedMeaning,
possibleInference), addedNodes structural fields, addedEdges structural fields,
updatedNodes summaries, and resolvedUnknownNodeIds. Diagnostic evidence only;
does not alter validation, mutation, or error messages. Stage-gated to
proposal_compatibility only.
2026-08-11 08:33:36 +01:00
robbond 79377670e2 experiment: capture proposal-boundary live variance 2026-08-11 08:19:21 +01:00
robbond 1c15b2b123 experiment: measure live semantic representation stability 2026-08-11 07:36:21 +01:00
robbond 25f56d75e2 experiment: capture live node-support semantic inputs 2026-08-11 07:12:30 +01:00
robbond d5db3c3cd6 experiment: observe post-admission investigation progress 2026-08-11 06:56:35 +01:00
robbond fbbd271596 experiment: validate user-supported unknown admission live 2026-08-11 06:38:30 +01:00
robbond 2e20d30890 correction: delegate hasNodeLevelUserSupport to rawAnswerSupportsUnclassifiedMeaning
v0.15 duplicated the overlap helper logic in hasNodeLevelUserSupport with
reversed argument orientation (unknownText as source, userSupportedMeaning
as candidate) compared to rawAnswerSupportsUnclassifiedMeaning (USM as
source, unknownText as candidate). This produced different accept/reject
outcomes when the two texts have very different token counts.

The fix replaces the independent reconstruction with a single call to the
canonical helper, ensuring node-level and answer-meaning alignment use
identical semantics. Four boundary regression tests verify:

- Boundary A: overlap ratio < 0.4 but >= 3 shared tokens → accept (token rule)
- Boundary B: short candidate / long source accepted via structural linkage
- Boundary B control: unrelated unknown rejected with no structural edge
- Boundary C: ratio exactly at 0.4 threshold accepts via ratio rule
2026-08-10 20:09:39 +01:00
robbond 9e869cbcb8 reasoning: admit verified user-supported unknowns without provenance edges 2026-08-10 19:27:54 +01:00
robbond 1fe3cec4bd experiment: observe live unknown dimensionality representation 2026-08-10 15:41:49 +01:00
robbond 15f2433151 docs: record rejected answerability corroboration candidate 2026-08-10 15:08:23 +01:00
robbond 0d15dd1f42 Revert "reasoning: require corroboration for conjunction compoundness"
This reverts commit 60048a5636.
2026-08-10 15:07:30 +01:00
robbond 60048a5636 reasoning: require corroboration for conjunction compoundness 2026-08-10 14:53:57 +01:00
robbond dcefb36f4d experiment: capture minimal clarification answerability 2026-08-10 14:26:07 +01:00
robbond 90e662397f experiment: validate relationship fallback live 2026-08-10 13:10:21 +01:00
robbond 4c5666dfb3 reasoning: suppress explanation question without relationship structure 2026-08-10 10:17:35 +01:00
robbond 1a31949a48 experiment: validate semantic compatibility live 2026-08-10 08:46:31 +01:00
robbond 69efc5d1b9 reasoning: ground unclassified answers without category expansion 2026-08-10 08:20:00 +01:00
robbond 4ea664d0d8 experiment: validate decomposition relevance live 2026-08-10 08:04:59 +01:00
robbond 7e4c506614 reasoning: prevent unsupported comparison decomposition 2026-08-10 07:44:40 +01:00
robbond 513c483501 experiment: locate irrelevant decomposition question boundary 2026-08-10 07:19:52 +01:00
robbond 62593eea14 docs: record canonical live multi-turn product-observation route 2026-08-10 06:49:03 +01:00
robbond 7533e471d4 test: establish minimal live multi-turn investigation route 2026-08-10 06:48:51 +01:00
robbond 46503b4507 reasoning: normalise graph relationship contract at proposal boundary 2026-08-10 06:16:47 +01:00
robbond cf6c5cb57f docs: record stopped experiment 57c contract failure 2026-08-10 06:04:46 +01:00
robbond 371ab0f52f merge(feature/reasoning-guard-generality-v0.9): integrate reasoning-guard generality v0.9 into main 2026-08-09 20:07:05 +01:00
robbond 19a42ca7f7 experiment: validate grounded unclassified answer live 2026-08-09 20:03:30 +01:00
robbond 4e4d0fa732 reasoning: stop answer fidelity guard blocking valid unclassified answers 2026-08-09 19:55:35 +01:00
robbond 7d94c6f73a docs: record contaminated experiment 57a observation 2026-08-09 19:40:49 +01:00
robbond 14d68f1ab7 merge: integrate reasoning-fidelity v0.8 first pass into main 2026-08-09 17:36:58 +01:00
robbond e498bbcc63 docs: close reasoning fidelity v0.8 first pass 2026-08-09 17:25:19 +01:00
robbond ec398dcec9 experiment: validate evidence versus clarification routing 2026-08-09 16:33:14 +01:00
robbond f861e2cac0 reasoning: preserve evidence versus clarification distinction 2026-08-09 16:20:06 +01:00
robbond e884b02e7c experiment: probe user-owned ambiguity boundary 2026-08-09 15:21:02 +01:00
robbond 11882bfaae experiment: probe evidence versus clarification boundary 2026-08-09 15:11:54 +01:00
robbond 85ee4bed30 experiment: probe explicit hard constraint semantic fidelity 2026-08-09 14:13:48 +01:00
robbond 23bfe5f756 experiment: validate unresolved uncertainty after harness repair 2026-08-09 12:55:10 +01:00
robbond c40d8c6d49 test: fix canonical live experiment harness import 2026-08-09 12:29:28 +01:00
robbond 144b7c53f5 experiment: validate unresolved uncertainty live path 2026-08-09 12:20:57 +01:00
robbond b06538ee91 experiment: validate raw-answer safeguard for weak priority 2026-08-09 12:06:50 +01:00
robbond e6f784261b test: establish canonical live reasoning experiment harness 2026-08-09 11:56:10 +01:00
robbond 4aa1492c8d refine raw-answer boundary for answer meaning 2026-08-09 11:31:27 +01:00
robbond c06aecc3f7 docs: update reasoning refinement handoff after experiment 56E 2026-08-09 11:25:43 +01:00
robbond 168ef69074 experiment: record 56D (Regression B) and 56E (weak-priority strengthening) results 2026-08-09 10:49:22 +01:00
robbond 3e78d57aca refine answer meaning derivation for negation and qualification 2026-08-09 09:48:08 +01:00
robbond 7965375aff refine answerMeaning contract around user-supported meaning 2026-08-09 08:50:37 +01:00
robbond 869afee1ab experiment: validate regression B after normalization 2026-08-09 08:25:59 +01:00
robbond 36faf70a08 refine answerMeaning support category normalisation 2026-08-09 08:06:38 +01:00
robbond b2329d8608 experiment: isolate regression B proposal validation 2026-08-09 07:55:11 +01:00
robbond 0d7ad5775c reasoning: add proposal-level answer meaning guard
- Add answerMeaning schema with supportCategory and resolutionGuidance enums
- Add pre-mutation guard that validates proposal alignment with answerMeaning
- Update prompt builder to instruct the model on answerMeaning contract
- Add tests for schema, guard logic, parsing defaults, and regression cases A-D
2026-08-09 07:07:21 +01:00
robbond 8c1036ecd0 docs: map reasoning requirements to production path 2026-08-08 11:08:37 +01:00
robbond 162ead2d69 docs: consolidate reasoning refinement requirements 2026-08-08 09:08:39 +01:00
robbond fcb7218407 experiment: separate stated clarification meaning from inference 2026-08-08 08:57:59 +01:00
robbond 3af623a7d5 experiment: test resolution from preserved answer meaning 2026-08-08 08:42:28 +01:00
robbond 22325d5ac3 experiment: separate answer meaning from resolution 2026-08-08 08:23:19 +01:00
robbond 8c12931b43 experiment: test clarification uncertainty preservation 2026-08-08 07:53:30 +01:00
robbond 67f2b084a5 experiment: test clarification broadening with weak answers 2026-08-08 07:43:19 +01:00
robbond 1d1dceefa3 experiment: test consequence of clarification target broadening 2026-08-08 07:29:36 +01:00
robbond 4302e3c435 experiment: test clarification target specificity 2026-08-08 07:16:55 +01:00
robbond 86c04d0fd8 experiment: test end-to-end clarification chain 2026-08-08 06:54:08 +01:00
robbond 5d1cba80cd experiment: test clarification answer resolution 2026-08-08 06:37:32 +01:00
robbond 8b1d69279f experiment: test clarification question wording 2026-08-08 06:26:17 +01:00
robbond c8ead0f690 experiment: test clarification null stability 2026-08-08 06:13:43 +01:00
robbond 6944f358a9 experiment: identify clarification target 2026-08-07 20:03:27 +01:00
robbond ee1391cd00 experiment: distinguish clarification from evidence needs 2026-08-07 19:28:45 +01:00
robbond dd3b8e505d experiment: test structured evidence to consequence reasoning 2026-08-07 19:02:37 +01:00
robbond cd9328ef8e experiment: test consequence from explicit evidence needs 2026-08-07 18:32:47 +01:00
robbond 10e87d0d44 experiment: test evidence needs across competing hypotheses 2026-08-07 18:14:03 +01:00
robbond 5daee5e911 experiment: test consequence of interpretation disagreement 2026-08-07 18:01:47 +01:00
robbond b9f737a293 experiment: test semantic interpretation disagreement 2026-08-07 17:30:10 +01:00
robbond fb5368ec5f experiment: test semantic grounding stability 2026-08-07 17:13:41 +01:00
robbond 118c5a789f experiment: test semantic grounding of interpretations 2026-08-07 16:32:56 +01:00
robbond 0f7457d9c1 experiment: separate source support from interpretation additions 2026-08-07 16:12:29 +01:00
robbond c2debad66d experiment: test interpretation lineage to user source 2026-08-07 15:58:39 +01:00
robbond 3537aa1b7a experiment: test deterministic user source identity 2026-08-07 15:24:47 +01:00
robbond d2d84656fe experiment: audit evidence source linkage 2026-08-07 15:11:57 +01:00
robbond 455d6f4c84 experiment: audit evidence type provenance 2026-08-07 15:10:03 +01:00
robbond 23f02d6169 experiment: audit referential provenance 2026-08-07 14:39:11 +01:00
robbond 90472de766 experiment: audit provenance in update prompt 2026-08-07 14:07:28 +01:00
robbond ec167f2689 experiment: trace update provenance boundary 2026-08-07 13:39:09 +01:00
robbond 7e74ad86f2 experiment: trace provenance loss through graph pipeline 2026-08-07 13:29:51 +01:00
robbond 5892a9b0f6 experiment: audit graph provenance 2026-08-07 13:01:25 +01:00
robbond d0b9be5fe6 experiment: separate stated meaning from model inference 2026-08-07 12:33:49 +01:00
robbond 6af9418eeb experiment: test grounding of decision relevance 2026-08-07 12:00:10 +01:00
robbond db5af97c98 experiment: test wording effects on ambiguous relevance 2026-08-07 11:03:33 +01:00
robbond 69d0216250 experiment: test domain priors in ambiguous relevance 2026-08-07 10:28:43 +01:00
robbond 9171844f5b experiment: test decision-relevance ambiguity handling 2026-08-07 10:15:40 +01:00
robbond 08c8f74bde experiment: test decision-relevance category boundary 2026-08-07 10:02:03 +01:00
robbond 34f06f2919 experiment: test decision-relevance normalisation 2026-08-07 09:46:21 +01:00
robbond b42a1ff244 experiment: separate semantic meaning from relevance labels 2026-08-07 09:00:46 +01:00
robbond 8ee1f575f7 experiment: run small semantic decision-relevance probe 2026-08-07 08:28:06 +01:00
robbond 690d4920d2 experiment: recover semantic evaluation configuration 2026-08-07 07:54:01 +01:00
robbond a88b9410a7 experiment: test semantic decision relevance 2026-08-07 07:24:35 +01:00
robbond 60dbbc0635 experiment: test decision-relative coherence 2026-08-07 07:13:54 +01:00
robbond 85fd90af1d experiment: test initial graph edge coherence (Exp 50)
Passive diagnostic. Coherent and scattered inputs produce identical
edge topology — every unknown connects to the summary node (kind=state)
via depends_on regardless of semantics. Shared edges are wiring, not
coherence evidence.
2026-08-07 06:51:13 +01:00
robbond 6f00a5e567 experiment: test production shared-anchor pattern (Exp 49)
Creates tests/graph/shared-anchor-production-path.test.js (36 tests, all pass).

Experiment 49 asks whether any sequence of real production updates via
applyValidatedProposal creates two or more active unknowns sharing the same
populated relationship anchor. Two sequential-update scenarios (Cases A & B)
consistently returned separate_anchors or insufficient_data — no shared
anchor observed in tested flows.

Control cases C–F confirm: diagnostic correctly distinguishes shared vs
separated patterns on controlled fixtures; all produced nodes/edges pass
schema validation; resolving one node does not mutate another (immunity);
decomposition children share parent anchor correctly.

Combined regression suite: 78 tests across Exp 47 (26), Exp 48 (16),
Exp 49 (36) — all passing, no production code modified.
2026-08-07 06:25:17 +01:00
robbond b1c69e9303 experiment: audit unknown relationship population
Experiment 48 passively audited whether real graph updates populate usable
unknown relationships. Three production paths inspected:

- buildInitialGraph: does NOT populate dependsOn/affects/parentId (only edges)
- buildEmergentReasoningUnknown: DOES populate dependsOn and parentId
- buildCompositeUnknownChildren: DOES populate parentId

One test file created (16 tests, all pass). Diagnostic confirms shared-anchor
coherence is structurally supportable through Path 2 only, requiring at least
two active unknowns with shared references. Conclusion: Insufficient Data for
the initial-build path; production code correctly populates fields in emergent
path but requires comparable observations to trigger.
2026-08-06 19:48:18 +01:00
robbond 40ef3e108f experiment: test shared-anchor coherence signal
Experiment 47: created a test-only diagnostic helper that inspects existing
graph relationship fields (dependsOn, affects, parentId, childIds on nodes;
fromNodeId/toNodeId + relationship on edges) to distinguish coherent
investigations (multiple unknowns sharing one anchor) from scattered ones.

Three controlled fixtures confirm the helper works: shared_anchor vs
separate_anchors vs insufficient_data — all with identical structural counts
(6 nodes, 4 active unknowns). All three produce identical too_broad output
from the existing assessor, confirming no production code changes needed.

Existing-scenario inspection (3 real scenarios from Exp 39-46) all return
insufficient_data — current data lacks populated relationship fields on
unknown nodes. This means the gap is not purely in assessment logic but also
in upstream data quality.

Closed Experiment 46. Updated design-evolution-log and handoff.
2026-08-06 19:31:43 +01:00