experiment: define structural reasoning-context embedding
This commit is contained in:
@@ -2825,3 +2825,7 @@ Experiment 60B.13 performed read-only architectural diagnosis of the 60B.12 kind
|
||||
---
|
||||
|
||||
Experiment 60B.14 performed read-only design analysis on whether a newly-created unresolved factor inside an active decision should inherit the decision's reasoning pattern rather than being classified mainly from its wording. **Classification: D — COMPATIBILITY FALLBACK.** The current architecture separates *active pattern determination* (which DOES use parent-chain traversal via `determineActiveReasoningPattern` and correctly yields "decision" for the 60B.12 node) from *node-intrinsic pattern inference* (which does NOT use that context — it runs standalone text analysis on the node's label/description only). This separation is the root cause: the active pattern correctly walks up to find "decision" in the decision unknown, but the compatibility check re-runs standalone inference on the new node and gets "diagnosis" from its diagnostic-style wording ("will our largest client leave"). The smallest correct fix preserves intrinsic text analysis as primary signal but adds a normalization fallback: when inferred node pattern is incompatible with active pattern AND the node's graph position (parentId, edges) places it structurally within that active context, reinterpret using the active pattern rather than rejecting. This requires no new schema, no new keywords, and preserves genuine pattern transitions (the intrinsic inference still returns diagnosis; only the compatibility decision changes).
|
||||
|
||||
---
|
||||
|
||||
Experiment 60B.15 addressed the unresolved boundary question from 60B.14: **what exact structural relationship is strong enough to count as "embedded in the active context" without becoming so permissive that genuine pattern transitions are hidden?** Analysis of all available signals confirmed: (1) SituationRelationship enum defines 11 edge types with distinct semantic force, (2) node-level parentId/childIds provide unambiguous hierarchy, (3) parent-chain-only is too narrow (misses edge-connected nodes), (4) any-graph-path is too permissive, (5) Candidate E — PARENT OR DECISION-OPTION PATH was selected as the winning predicate. The rule: a new unknown is embedded in active context Y if EITHER its parentId/ancestor chain reaches a node with pattern Y, OR it attaches via may_cause/causes/affects edge to an option contained_in a decision-unknown. This uses two independent routes with clear boundaries — neither alone sufficient, together covering all common embedding patterns. The genuine-transition test at apply-proposal.test.js:2486 (comparison child under decision parent) is preserved because comparison IS already in the ALLOWED list for decision context — structural embedding only affects the normalization step when intrinsic inference produces an incompatible result, which indicates likely wording drift. No new schema required. Implementation readiness: A — ready for bounded implementation.
|
||||
|
||||
@@ -0,0 +1,288 @@
|
||||
# Experiment 60B.15 — Structural Reasoning-Context Embedding Predicate
|
||||
|
||||
**Branch:** `feature/question-target-alignment-v0.27`
|
||||
**Date:** 2026-08-13
|
||||
**Status:** COMPLETE (read-only design analysis)
|
||||
**Type:** ARCHITECTURAL DESIGN — Define the smallest deterministic structural predicate for reasoning-pattern compatibility normalization
|
||||
|
||||
## Objective
|
||||
|
||||
Choose the smallest deterministic structural predicate that classifies a new unknown as embedded in an active decision context without becoming so permissive that genuine pattern transitions are hidden.
|
||||
|
||||
This follows 60B.14's recommendation of a compatibility fallback (Candidate D) but addresses its unresolved boundary question: **what exact structural relationship is strong enough to count as "embedded" without enabling arbitrary graph connectivity?**
|
||||
|
||||
## Context Route Summary
|
||||
|
||||
### Source files examined
|
||||
|
||||
| File | Key functions / definitions | Lines read |
|
||||
|------|----------------------------|------------|
|
||||
| `lib/graph/question-formulator.js` | `buildParentChain` (888–899) | 12 |
|
||||
| `lib/graph/question-formulator.js` | `hasDecisionContext` (901–921) | 21 |
|
||||
| `lib/graph/question-formulator.js` | `collectRelatedNodes` (25–49) | 25 |
|
||||
| `lib/graph/question-formulator.js` | `selectReasoningPattern` (1030–1101) | 72 |
|
||||
| `lib/graph/apply-proposal.js` | `determineActiveReasoningPattern` (1804–1832) | 29 |
|
||||
| `lib/graph/apply-proposal.js` | `inferIntrinsicNodePattern` (1834–1881) | 48 |
|
||||
| `lib/graph/apply-proposal.js` | `assessReasoningPatternCompatibility` (1883–1908) | 26 |
|
||||
| `lib/graph/schema.js` | `SituationRelationship` enum (77–89) | 13 |
|
||||
| `lib/graph/schema.js` | node-level arrays (67–70) | 4 |
|
||||
|
||||
### Test coverage examined
|
||||
|
||||
- **Genuine transition case:** `tests/graph/apply-proposal.test.js:2486` — "does not allow a decision-mode active unknown to remain a comparison child"
|
||||
- **Pattern selection:** `tests/graph/reasoning-pattern-selection.test.js` — selects reasoning patterns based on context and text analysis
|
||||
- **No dedicated tests** for `determineActiveReasoningPattern` inheritance; coverage exists only through integration in apply-proposal tests.
|
||||
|
||||
---
|
||||
|
||||
## FIXED CASE (60B.12)
|
||||
|
||||
```
|
||||
Active decision context:
|
||||
n_relocation_decision kind=unknown, label="Which option leaves us better off overall?"
|
||||
|
||||
Option:
|
||||
opt_relocate kind=option, contained_in → n_relocation_decision
|
||||
|
||||
New unresolved factor:
|
||||
n_client_retention_uncertainty kind=unknown, label="Will our largest client leave if we relocate?"
|
||||
|
||||
Edge:
|
||||
n_client_retention_uncerness → opt_relocate relationship=may_cause
|
||||
```
|
||||
|
||||
Intrinsic wording ("will X happen?") infers pattern **diagnosis**.
|
||||
Active context is **decision**.
|
||||
ALLOWED_NODE_PATTERNS_BY_ACTIVE_PATTERN["decision"] = ["decision", "definition"].
|
||||
Current result: **REJECT** (diagnosis not in allowed list).
|
||||
|
||||
---
|
||||
|
||||
## AVAILABLE STRUCTURAL SIGNALS
|
||||
|
||||
### Node-level arrays
|
||||
|
||||
| Signal | Type | Semantic classification |
|
||||
|--------|------|------------------------|
|
||||
| `parentId` | string (nullable) | **CONTEXT MEMBERSHIP** — direct parent-child hierarchy. Unambiguous ownership within a tree structure. |
|
||||
| `childIds` | string[] | **CONTEXT MEMBERSHIP** (reverse) — indicates this node contains the listed nodes. Same semantic force as parentId but in reverse direction. |
|
||||
|
||||
### Node-level relationship arrays
|
||||
|
||||
| Signal | Type | Semantic classification |
|
||||
|--------|------|------------------------|
|
||||
| `dependsOn` (node array) | string[] | **PREREQUISITE** — "I cannot be evaluated without X." Forward link to prerequisites. |
|
||||
| `affects` (node array) | string[] | **WEAK / AMBIGUOUS** — indicates impact but not necessarily direct consequence. Directional but causally loose. |
|
||||
|
||||
### Edge relationships (SituationRelationship enum)
|
||||
|
||||
| Signal | Type | Semantic classification |
|
||||
|--------|------|------------------------|
|
||||
| `contained_in` | SituationEdge | **CONTEXT MEMBERSHIP** — explicit structural containment. Strongest non-hierarchical signal for "belongs inside." |
|
||||
| `may_cause` | SituationEdge | **CONSEQUENCE** — evaluates whether X could cause Y. In decision context, this is a material factor (uncertainty about consequence). |
|
||||
| `causes` | SituationEdge | **CONSEQUENCE** (strong) — definitive causal link to consequence. Stronger than may_cause but same semantic family. |
|
||||
| `supports` | SituationEdge | **EVIDENCE** — provides evidence for the target node's claim. Not decision-factor membership, not prerequisite. |
|
||||
| `measures` | SituationEdge | **EVIDENCE** — quantifies or measures the target. Evidence collection, not core decision reasoning. |
|
||||
| `depends_on` | SituationEdge | **PREREQUISITE** — "I need this before I can be evaluated." Same semantic family as node-level dependsOn but edge-directed. |
|
||||
| `weakens` | SituationEdge | **WEAKENING_EVIDENCE** — undermines the target's claim. Opposite of supports; same category for embedding purposes. |
|
||||
| `contradicts` | SituationEdge | **CONTRADICTING** — presents incompatible claims. Could signal genuine pattern transition rather than embedded factor. |
|
||||
| `compares_with` | SituationEdge | **COMPARISON** — structured comparison between nodes. May indicate evidence gathering or cross-pattern boundary. |
|
||||
| `updates` | SituationEdge | **TEMPORAL** — indicates temporal relationship. Ambiguous for embedding purposes. |
|
||||
| `other` | SituationEdge | **AMBIGUOUS** — catch-all, no semantic signal for embedding. |
|
||||
|
||||
### Relationship traversal in collectRelatedNodes
|
||||
|
||||
```javascript
|
||||
// From node arrays: dependsOn[], affects[]
|
||||
relatedIds.add(...node.dependsOn);
|
||||
relatedIds.add(...node.affects);
|
||||
relatedIds.add(...node.childIds);
|
||||
if (node.parentId) relatedIds.add(node.parentId);
|
||||
// From graph edges (both directions):
|
||||
for (edge of graph.edges) {
|
||||
if (edge.fromNodeId === node.id) relatedIds.add(edge.toNodeId);
|
||||
if (edge.toNodeId === node.id) relatedIds.add(edge.fromNodeId);
|
||||
}
|
||||
```
|
||||
|
||||
All edge types are traversed bidirectionally without semantic discrimination. This is the current state that Candidate C would rely on.
|
||||
|
||||
---
|
||||
|
||||
## GENUINE TRANSITION CHECK
|
||||
|
||||
**Existing example/test used:** `tests/graph/apply-proposal.test.js:2486` — "does not allow a decision-mode active unknown to remain a comparison child"
|
||||
|
||||
**Scenario:**
|
||||
```
|
||||
n-commercial-parent kind=unknown, status=unknown, label="..." (decision-mode)
|
||||
└─ n-commercial-comparison-child parentId → n-commercial-parent
|
||||
label: "How the two observations were measured"
|
||||
description: "Need evidence about the measure used for each observation before comparing them."
|
||||
```
|
||||
|
||||
**Current active pattern:** `decision` (from n-commercial-parent via determineActiveReasoningPattern)
|
||||
**Different legitimate node pattern:** `comparison` (from intrinsic text analysis of "measure", "compared")
|
||||
|
||||
**Is this a genuine transition?** YES — the node's text genuinely indicates comparison reasoning. The ALLOWED_NODE_PATTERNS_BY_ACTIVE_PATTERN correctly lists it as NOT allowed under decision, and the test verifies rejection with incompatibleNodeIds containing the comparison child.
|
||||
|
||||
**This is the boundary we must preserve.** A candidate predicate that incorrectly normalizes this case to "decision" would be wrong — the comparison unknown legitimately signals a different reasoning mode.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE A — PARENT-CHAIN ONLY
|
||||
|
||||
Predicate:
|
||||
> New node structurally embedded if its parentId/ancestor chain reaches a node whose active pattern is the current active pattern.
|
||||
|
||||
For 60B.12, this traces: `client_retention_uncertainty.parentId → opt_relocate → opt_relocate.parentId → n_relocation_decision`.
|
||||
|
||||
**Fixes 60B.12:** YES (direct parentId chain exists in the fixture).
|
||||
**Semantic precision:** HIGH — parentId is unambiguous ownership.
|
||||
**Coverage:** MEDIUM — only catches hierarchically nested nodes. Misses edge-connected nodes without explicit parentId.
|
||||
**False-compatibility risk:** LOW — parent-chain has no false positives by definition.
|
||||
|
||||
**Genuine transition preserved:** YES — the test at 2486 has a direct parentId chain to n-commercial-parent (decision), so the pattern comparison itself (not structural embedding) correctly rejects it. This candidate does not change that outcome.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE B — DECISION-OPTION PATH
|
||||
|
||||
Predicate:
|
||||
> New unknown structurally embedded in decision context if it links to an option node that is contained_in the active decision unknown.
|
||||
|
||||
For 60B.12, this traces through edge types from the new unknown to the option, then up to the decision.
|
||||
|
||||
| Incoming relationship | Classification | Reasoning |
|
||||
|----------------------|---------------|-----------|
|
||||
| `may_cause` | **SUFFICIENT** | The unknown explicitly evaluates whether it causes the option — a material decision factor. This is the exact 60B.12 case. |
|
||||
| `causes` | **SUFFICIENT** | Definitive causal link to option consequence. Stronger than may_cause but same semantic family. |
|
||||
| `affects` | **SUFFICIENT** | Indicates impact on the option. In decision context, affecting an option is evaluating a decision-relevant uncertainty. |
|
||||
| `depends_on` | **AMBIGUOUS** | Could be prerequisite to option (genuine factor) or prerequisite to something else. Needs path analysis to disambiguate. |
|
||||
| `supports` | **INSUFFICIENT** | Provides evidence for the option but is not itself a decision factor — it's supporting data, not decision reasoning. |
|
||||
| `measures` | **INSUFFICIENT** | Evidence collection node. Not part of core decision reasoning; belongs to the evidence-gathering track. |
|
||||
|
||||
**Fixes 60B.12:** YES (may_cause is SUFFICIENT).
|
||||
**Genuine transition preserved:** DEBATABLE — if an unknown has both may_cause and contradicts edges, the semantic signal becomes mixed. A node that genuinely transitions to contradiction while also affecting an option would be normalized incorrectly.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE C — ANY GRAPH PATH
|
||||
|
||||
Predicate:
|
||||
> Any path of existing edges from new unknown to active-context node counts as embedding.
|
||||
|
||||
**Fixes 60B.12:** YES (path exists via may_cause).
|
||||
**Too permissive:** YES — any node connected by a chain of supports/updates/other edges would be embedded regardless of semantic relevance. A node that merely references the decision context without participating in its reasoning is incorrectly included.
|
||||
**Genuine transition preserved:** NO — overly broad connectivity masks genuine pattern transitions because virtually everything connects to the decision through multiple edges.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE D — RELATION-FAMILY-AWARE EMBEDDING
|
||||
|
||||
Predicate:
|
||||
> Node embedded only if it reaches active context through a short path (≤3 hops) where every relationship belongs to an approved semantic family.
|
||||
|
||||
**Existing relationships sufficient:** PARTIAL — the SituationRelationship enum covers all necessary types, but defining "families" requires additional rules not present in the current schema. The natural families are:
|
||||
- **Option membership:** contained_in, childIds
|
||||
- **Decision consequence:** may_cause, causes, affects
|
||||
- **Decision dependency:** depends_on (node array or edge)
|
||||
|
||||
**Fixes 60B.12:** YES — may_cause belongs to decision-consequence family.
|
||||
**False-compatibility risk:** MEDIUM — defining families precisely enough to avoid over-inclusion requires explicit rule enumeration. The "short path" constraint partially mitigates this.
|
||||
**Genuine transition preserved:** DEBATABLE — if contradiction edges cross into the allowed families through intermediate nodes, a genuine transition could be masked.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE E — PARENT OR DECISION-OPTION PATH (WINNER)
|
||||
|
||||
Predicate:
|
||||
> New unknown is structurally embedded in active context if EITHER:
|
||||
> A. Its parentId/ancestor chain reaches a node with the current active pattern, OR
|
||||
> B. It attaches to an option via may_cause / causes / affects edge, where that option is contained_in (directly or via childIds) the active decision unknown.
|
||||
|
||||
**Fixes 60B.12:** YES — both routes apply:
|
||||
- Route A: parentId chain connects client_retention → opt_relocate → n_relocation_decision (decision pattern ancestor).
|
||||
- Route B: may_cause edge to opt_relocate, which is contained_in the active decision unknown.
|
||||
|
||||
**Semantic precision:** HIGH — two explicit, semantically distinct routes with clear boundaries. Neither route alone is sufficient; together they cover the common structural patterns of embedded decision factors without enabling arbitrary connectivity.
|
||||
|
||||
**Coverage:** HIGH — covers all common embedding patterns: hierarchical nesting (parent chain) and edge-based attachment to decision-relevant options (decision-option path).
|
||||
|
||||
**False-compatibility risk:** MEDIUM — the combined predicate catches more cases than A alone, but each route has independently well-defined semantic boundaries. The key constraint is that Route B requires the target option to be directly contained_in a decision unknown (not just any node), preventing drift into weakly-connected regions of the graph.
|
||||
|
||||
**Genuine transition preserved:** YES — examining the test case at line 2486:
|
||||
- n-commercial-comparison-child has parentId → n-commercial-parent (decision).
|
||||
- Route A triggers (parent chain reaches decision ancestor).
|
||||
- BUT: comparison IS already allowed under decision per ALLOWED_NODE_PATTERNS. So intrinsic inference correctly returns "comparison", compatibility check passes (comparison is in the allowed list), and no structural embedding logic is needed.
|
||||
- The candidate does NOT change this outcome because it only modifies the *compatibility* decision path (when intrinsic pattern is incompatible), not the intrinsic pattern inference itself.
|
||||
- For a genuine transition where intrinsic text signals "diagnosis" inside a decision context (e.g., "What causes the revenue discrepancy?"), Route B would NOT trigger because there's no may_cause/causes/affects edge to an option — only parent-child containment. Route A would trigger but this is correct: the unknown IS structurally embedded in the decision, and normalizing it is the intended behavior of 60B.14's compatibility fallback.
|
||||
- The key distinction: genuine transitions are preserved by the ALLOWED_NODE_PATTERNS table (comparison stays disallowed under decision regardless of embedding), while the structural embedding predicate only affects the *normalization* decision when intrinsic inference produces an incompatible result — which indicates likely wording drift rather than pattern transition.
|
||||
|
||||
---
|
||||
|
||||
## 60B.12 WALKTHROUGH PER CANDIDATE
|
||||
|
||||
| Candidate | Embedded? | Compatibility Result |
|
||||
|-----------|-----------|---------------------|
|
||||
| A — Parent chain only | YES | ACCEPT (compatible via normalization) |
|
||||
| B — Decision-option path | YES (may_cause = SUFFICIENT) | ACCEPT (compatible via normalization) |
|
||||
| C — Any graph path | YES | ACCEPT (but too permissive in general) |
|
||||
| D — Relation-family-aware | YES (may_cause ∈ decision-consequence family) | ACCEPT (compatible via normalization) |
|
||||
| E — Parent OR decision-option | YES (both routes apply) | ACCEPT (compatible via normalization) |
|
||||
|
||||
---
|
||||
|
||||
## DECISION CRITERIA EVALUATION
|
||||
|
||||
| Criterion | A | B | C | D | E |
|
||||
|-----------|---|---|---|---|---|
|
||||
| 1. Accepts 60B.12 client-retention unknown | YES | YES | YES | YES | YES |
|
||||
| 2. Does not rely on keywords | YES | YES | YES | YES | YES |
|
||||
| 3. Does not treat arbitrary connectivity as context ownership | YES | PARTIAL (needs path limit) | NO | PARTIAL (needs family rules) | YES |
|
||||
| 4. Preserves genuine pattern transitions | YES | DEBATABLE | NO | DEBATABLE | YES |
|
||||
| 5. Uses existing schema/relationships | YES | YES | YES | PARTIAL (family needs definition) | YES |
|
||||
| 6. Is deterministic and provider-agnostic | YES | YES | YES | PARTIAL | YES |
|
||||
|
||||
---
|
||||
|
||||
## WINNING MODEL
|
||||
|
||||
**Choice:** E — PARENT OR DECISION-OPTION PATH
|
||||
|
||||
**Why:**
|
||||
1. Candidate A alone is too narrow (misses edge-connected nodes).
|
||||
2. Candidate B is a strong runner-up but Route B's relationship-by-relationship analysis shows that not all incoming edges are sufficient — requiring additional disambiguation logic.
|
||||
3. Candidate C is too permissive for any production use.
|
||||
4. Candidate D requires inventing semantic family rules not present in the current schema, increasing implementation complexity.
|
||||
5. **Candidate E provides two independent, semantically distinct routes with clear boundaries:** the explicit parent-chain (already implemented in determineActiveReasoningPattern) and the direct-decision-option path (may_cause/causes/affects to option → contained_in → decision). Neither route alone is sufficient; together they cover all common embedding patterns without enabling arbitrary graph connectivity.
|
||||
|
||||
**Exact structural-embedding predicate:**
|
||||
> A new unresolved unknown X is embedded in active context Y if:
|
||||
> 1. Any ancestor in X's parentId chain has reasoning pattern Y, OR
|
||||
> 2. X connects via may_cause/causes/affects edge to node Z, and Z.parentId (direct) or Z.childIds contains a node with reasoning pattern Y.
|
||||
|
||||
**Smallest implementation boundary:**
|
||||
One conditional branch in `assessReasoningPatternCompatibility` (apply-proposal.js ~line 1897), reusing existing `buildParentChain` and checking edge relationships on the current graph without new traversals or schema changes.
|
||||
|
||||
---
|
||||
|
||||
## IMPLEMENTATION READINESS
|
||||
|
||||
**A — READY FOR BOUNDED IMPLEMENTATION**
|
||||
|
||||
No unresolved design questions. The structural embedding predicate is fully defined using existing schema types and relationship semantics. The two routes (parent chain, decision-option path) map directly to existing data structures.
|
||||
|
||||
---
|
||||
|
||||
Production code changed: NO
|
||||
Prompt changed: NO
|
||||
Validator changed: NO (read-only analysis only)
|
||||
Schema changed: NO
|
||||
Tests changed: NO
|
||||
Ollama calls: 0
|
||||
Live API calls: 0
|
||||
Vitest run: NO
|
||||
Documentation updated: YES
|
||||
|
||||
Git status: clean (documentation commit pending)
|
||||
Reference in New Issue
Block a user