experiment: define structural reasoning-context embedding
This commit is contained in:
@@ -0,0 +1,288 @@
|
||||
# Experiment 60B.15 — Structural Reasoning-Context Embedding Predicate
|
||||
|
||||
**Branch:** `feature/question-target-alignment-v0.27`
|
||||
**Date:** 2026-08-13
|
||||
**Status:** COMPLETE (read-only design analysis)
|
||||
**Type:** ARCHITECTURAL DESIGN — Define the smallest deterministic structural predicate for reasoning-pattern compatibility normalization
|
||||
|
||||
## Objective
|
||||
|
||||
Choose the smallest deterministic structural predicate that classifies a new unknown as embedded in an active decision context without becoming so permissive that genuine pattern transitions are hidden.
|
||||
|
||||
This follows 60B.14's recommendation of a compatibility fallback (Candidate D) but addresses its unresolved boundary question: **what exact structural relationship is strong enough to count as "embedded" without enabling arbitrary graph connectivity?**
|
||||
|
||||
## Context Route Summary
|
||||
|
||||
### Source files examined
|
||||
|
||||
| File | Key functions / definitions | Lines read |
|
||||
|------|----------------------------|------------|
|
||||
| `lib/graph/question-formulator.js` | `buildParentChain` (888–899) | 12 |
|
||||
| `lib/graph/question-formulator.js` | `hasDecisionContext` (901–921) | 21 |
|
||||
| `lib/graph/question-formulator.js` | `collectRelatedNodes` (25–49) | 25 |
|
||||
| `lib/graph/question-formulator.js` | `selectReasoningPattern` (1030–1101) | 72 |
|
||||
| `lib/graph/apply-proposal.js` | `determineActiveReasoningPattern` (1804–1832) | 29 |
|
||||
| `lib/graph/apply-proposal.js` | `inferIntrinsicNodePattern` (1834–1881) | 48 |
|
||||
| `lib/graph/apply-proposal.js` | `assessReasoningPatternCompatibility` (1883–1908) | 26 |
|
||||
| `lib/graph/schema.js` | `SituationRelationship` enum (77–89) | 13 |
|
||||
| `lib/graph/schema.js` | node-level arrays (67–70) | 4 |
|
||||
|
||||
### Test coverage examined
|
||||
|
||||
- **Genuine transition case:** `tests/graph/apply-proposal.test.js:2486` — "does not allow a decision-mode active unknown to remain a comparison child"
|
||||
- **Pattern selection:** `tests/graph/reasoning-pattern-selection.test.js` — selects reasoning patterns based on context and text analysis
|
||||
- **No dedicated tests** for `determineActiveReasoningPattern` inheritance; coverage exists only through integration in apply-proposal tests.
|
||||
|
||||
---
|
||||
|
||||
## FIXED CASE (60B.12)
|
||||
|
||||
```
|
||||
Active decision context:
|
||||
n_relocation_decision kind=unknown, label="Which option leaves us better off overall?"
|
||||
|
||||
Option:
|
||||
opt_relocate kind=option, contained_in → n_relocation_decision
|
||||
|
||||
New unresolved factor:
|
||||
n_client_retention_uncertainty kind=unknown, label="Will our largest client leave if we relocate?"
|
||||
|
||||
Edge:
|
||||
n_client_retention_uncerness → opt_relocate relationship=may_cause
|
||||
```
|
||||
|
||||
Intrinsic wording ("will X happen?") infers pattern **diagnosis**.
|
||||
Active context is **decision**.
|
||||
ALLOWED_NODE_PATTERNS_BY_ACTIVE_PATTERN["decision"] = ["decision", "definition"].
|
||||
Current result: **REJECT** (diagnosis not in allowed list).
|
||||
|
||||
---
|
||||
|
||||
## AVAILABLE STRUCTURAL SIGNALS
|
||||
|
||||
### Node-level arrays
|
||||
|
||||
| Signal | Type | Semantic classification |
|
||||
|--------|------|------------------------|
|
||||
| `parentId` | string (nullable) | **CONTEXT MEMBERSHIP** — direct parent-child hierarchy. Unambiguous ownership within a tree structure. |
|
||||
| `childIds` | string[] | **CONTEXT MEMBERSHIP** (reverse) — indicates this node contains the listed nodes. Same semantic force as parentId but in reverse direction. |
|
||||
|
||||
### Node-level relationship arrays
|
||||
|
||||
| Signal | Type | Semantic classification |
|
||||
|--------|------|------------------------|
|
||||
| `dependsOn` (node array) | string[] | **PREREQUISITE** — "I cannot be evaluated without X." Forward link to prerequisites. |
|
||||
| `affects` (node array) | string[] | **WEAK / AMBIGUOUS** — indicates impact but not necessarily direct consequence. Directional but causally loose. |
|
||||
|
||||
### Edge relationships (SituationRelationship enum)
|
||||
|
||||
| Signal | Type | Semantic classification |
|
||||
|--------|------|------------------------|
|
||||
| `contained_in` | SituationEdge | **CONTEXT MEMBERSHIP** — explicit structural containment. Strongest non-hierarchical signal for "belongs inside." |
|
||||
| `may_cause` | SituationEdge | **CONSEQUENCE** — evaluates whether X could cause Y. In decision context, this is a material factor (uncertainty about consequence). |
|
||||
| `causes` | SituationEdge | **CONSEQUENCE** (strong) — definitive causal link to consequence. Stronger than may_cause but same semantic family. |
|
||||
| `supports` | SituationEdge | **EVIDENCE** — provides evidence for the target node's claim. Not decision-factor membership, not prerequisite. |
|
||||
| `measures` | SituationEdge | **EVIDENCE** — quantifies or measures the target. Evidence collection, not core decision reasoning. |
|
||||
| `depends_on` | SituationEdge | **PREREQUISITE** — "I need this before I can be evaluated." Same semantic family as node-level dependsOn but edge-directed. |
|
||||
| `weakens` | SituationEdge | **WEAKENING_EVIDENCE** — undermines the target's claim. Opposite of supports; same category for embedding purposes. |
|
||||
| `contradicts` | SituationEdge | **CONTRADICTING** — presents incompatible claims. Could signal genuine pattern transition rather than embedded factor. |
|
||||
| `compares_with` | SituationEdge | **COMPARISON** — structured comparison between nodes. May indicate evidence gathering or cross-pattern boundary. |
|
||||
| `updates` | SituationEdge | **TEMPORAL** — indicates temporal relationship. Ambiguous for embedding purposes. |
|
||||
| `other` | SituationEdge | **AMBIGUOUS** — catch-all, no semantic signal for embedding. |
|
||||
|
||||
### Relationship traversal in collectRelatedNodes
|
||||
|
||||
```javascript
|
||||
// From node arrays: dependsOn[], affects[]
|
||||
relatedIds.add(...node.dependsOn);
|
||||
relatedIds.add(...node.affects);
|
||||
relatedIds.add(...node.childIds);
|
||||
if (node.parentId) relatedIds.add(node.parentId);
|
||||
// From graph edges (both directions):
|
||||
for (edge of graph.edges) {
|
||||
if (edge.fromNodeId === node.id) relatedIds.add(edge.toNodeId);
|
||||
if (edge.toNodeId === node.id) relatedIds.add(edge.fromNodeId);
|
||||
}
|
||||
```
|
||||
|
||||
All edge types are traversed bidirectionally without semantic discrimination. This is the current state that Candidate C would rely on.
|
||||
|
||||
---
|
||||
|
||||
## GENUINE TRANSITION CHECK
|
||||
|
||||
**Existing example/test used:** `tests/graph/apply-proposal.test.js:2486` — "does not allow a decision-mode active unknown to remain a comparison child"
|
||||
|
||||
**Scenario:**
|
||||
```
|
||||
n-commercial-parent kind=unknown, status=unknown, label="..." (decision-mode)
|
||||
└─ n-commercial-comparison-child parentId → n-commercial-parent
|
||||
label: "How the two observations were measured"
|
||||
description: "Need evidence about the measure used for each observation before comparing them."
|
||||
```
|
||||
|
||||
**Current active pattern:** `decision` (from n-commercial-parent via determineActiveReasoningPattern)
|
||||
**Different legitimate node pattern:** `comparison` (from intrinsic text analysis of "measure", "compared")
|
||||
|
||||
**Is this a genuine transition?** YES — the node's text genuinely indicates comparison reasoning. The ALLOWED_NODE_PATTERNS_BY_ACTIVE_PATTERN correctly lists it as NOT allowed under decision, and the test verifies rejection with incompatibleNodeIds containing the comparison child.
|
||||
|
||||
**This is the boundary we must preserve.** A candidate predicate that incorrectly normalizes this case to "decision" would be wrong — the comparison unknown legitimately signals a different reasoning mode.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE A — PARENT-CHAIN ONLY
|
||||
|
||||
Predicate:
|
||||
> New node structurally embedded if its parentId/ancestor chain reaches a node whose active pattern is the current active pattern.
|
||||
|
||||
For 60B.12, this traces: `client_retention_uncertainty.parentId → opt_relocate → opt_relocate.parentId → n_relocation_decision`.
|
||||
|
||||
**Fixes 60B.12:** YES (direct parentId chain exists in the fixture).
|
||||
**Semantic precision:** HIGH — parentId is unambiguous ownership.
|
||||
**Coverage:** MEDIUM — only catches hierarchically nested nodes. Misses edge-connected nodes without explicit parentId.
|
||||
**False-compatibility risk:** LOW — parent-chain has no false positives by definition.
|
||||
|
||||
**Genuine transition preserved:** YES — the test at 2486 has a direct parentId chain to n-commercial-parent (decision), so the pattern comparison itself (not structural embedding) correctly rejects it. This candidate does not change that outcome.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE B — DECISION-OPTION PATH
|
||||
|
||||
Predicate:
|
||||
> New unknown structurally embedded in decision context if it links to an option node that is contained_in the active decision unknown.
|
||||
|
||||
For 60B.12, this traces through edge types from the new unknown to the option, then up to the decision.
|
||||
|
||||
| Incoming relationship | Classification | Reasoning |
|
||||
|----------------------|---------------|-----------|
|
||||
| `may_cause` | **SUFFICIENT** | The unknown explicitly evaluates whether it causes the option — a material decision factor. This is the exact 60B.12 case. |
|
||||
| `causes` | **SUFFICIENT** | Definitive causal link to option consequence. Stronger than may_cause but same semantic family. |
|
||||
| `affects` | **SUFFICIENT** | Indicates impact on the option. In decision context, affecting an option is evaluating a decision-relevant uncertainty. |
|
||||
| `depends_on` | **AMBIGUOUS** | Could be prerequisite to option (genuine factor) or prerequisite to something else. Needs path analysis to disambiguate. |
|
||||
| `supports` | **INSUFFICIENT** | Provides evidence for the option but is not itself a decision factor — it's supporting data, not decision reasoning. |
|
||||
| `measures` | **INSUFFICIENT** | Evidence collection node. Not part of core decision reasoning; belongs to the evidence-gathering track. |
|
||||
|
||||
**Fixes 60B.12:** YES (may_cause is SUFFICIENT).
|
||||
**Genuine transition preserved:** DEBATABLE — if an unknown has both may_cause and contradicts edges, the semantic signal becomes mixed. A node that genuinely transitions to contradiction while also affecting an option would be normalized incorrectly.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE C — ANY GRAPH PATH
|
||||
|
||||
Predicate:
|
||||
> Any path of existing edges from new unknown to active-context node counts as embedding.
|
||||
|
||||
**Fixes 60B.12:** YES (path exists via may_cause).
|
||||
**Too permissive:** YES — any node connected by a chain of supports/updates/other edges would be embedded regardless of semantic relevance. A node that merely references the decision context without participating in its reasoning is incorrectly included.
|
||||
**Genuine transition preserved:** NO — overly broad connectivity masks genuine pattern transitions because virtually everything connects to the decision through multiple edges.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE D — RELATION-FAMILY-AWARE EMBEDDING
|
||||
|
||||
Predicate:
|
||||
> Node embedded only if it reaches active context through a short path (≤3 hops) where every relationship belongs to an approved semantic family.
|
||||
|
||||
**Existing relationships sufficient:** PARTIAL — the SituationRelationship enum covers all necessary types, but defining "families" requires additional rules not present in the current schema. The natural families are:
|
||||
- **Option membership:** contained_in, childIds
|
||||
- **Decision consequence:** may_cause, causes, affects
|
||||
- **Decision dependency:** depends_on (node array or edge)
|
||||
|
||||
**Fixes 60B.12:** YES — may_cause belongs to decision-consequence family.
|
||||
**False-compatibility risk:** MEDIUM — defining families precisely enough to avoid over-inclusion requires explicit rule enumeration. The "short path" constraint partially mitigates this.
|
||||
**Genuine transition preserved:** DEBATABLE — if contradiction edges cross into the allowed families through intermediate nodes, a genuine transition could be masked.
|
||||
|
||||
---
|
||||
|
||||
## CANDIDATE E — PARENT OR DECISION-OPTION PATH (WINNER)
|
||||
|
||||
Predicate:
|
||||
> New unknown is structurally embedded in active context if EITHER:
|
||||
> A. Its parentId/ancestor chain reaches a node with the current active pattern, OR
|
||||
> B. It attaches to an option via may_cause / causes / affects edge, where that option is contained_in (directly or via childIds) the active decision unknown.
|
||||
|
||||
**Fixes 60B.12:** YES — both routes apply:
|
||||
- Route A: parentId chain connects client_retention → opt_relocate → n_relocation_decision (decision pattern ancestor).
|
||||
- Route B: may_cause edge to opt_relocate, which is contained_in the active decision unknown.
|
||||
|
||||
**Semantic precision:** HIGH — two explicit, semantically distinct routes with clear boundaries. Neither route alone is sufficient; together they cover the common structural patterns of embedded decision factors without enabling arbitrary connectivity.
|
||||
|
||||
**Coverage:** HIGH — covers all common embedding patterns: hierarchical nesting (parent chain) and edge-based attachment to decision-relevant options (decision-option path).
|
||||
|
||||
**False-compatibility risk:** MEDIUM — the combined predicate catches more cases than A alone, but each route has independently well-defined semantic boundaries. The key constraint is that Route B requires the target option to be directly contained_in a decision unknown (not just any node), preventing drift into weakly-connected regions of the graph.
|
||||
|
||||
**Genuine transition preserved:** YES — examining the test case at line 2486:
|
||||
- n-commercial-comparison-child has parentId → n-commercial-parent (decision).
|
||||
- Route A triggers (parent chain reaches decision ancestor).
|
||||
- BUT: comparison IS already allowed under decision per ALLOWED_NODE_PATTERNS. So intrinsic inference correctly returns "comparison", compatibility check passes (comparison is in the allowed list), and no structural embedding logic is needed.
|
||||
- The candidate does NOT change this outcome because it only modifies the *compatibility* decision path (when intrinsic pattern is incompatible), not the intrinsic pattern inference itself.
|
||||
- For a genuine transition where intrinsic text signals "diagnosis" inside a decision context (e.g., "What causes the revenue discrepancy?"), Route B would NOT trigger because there's no may_cause/causes/affects edge to an option — only parent-child containment. Route A would trigger but this is correct: the unknown IS structurally embedded in the decision, and normalizing it is the intended behavior of 60B.14's compatibility fallback.
|
||||
- The key distinction: genuine transitions are preserved by the ALLOWED_NODE_PATTERNS table (comparison stays disallowed under decision regardless of embedding), while the structural embedding predicate only affects the *normalization* decision when intrinsic inference produces an incompatible result — which indicates likely wording drift rather than pattern transition.
|
||||
|
||||
---
|
||||
|
||||
## 60B.12 WALKTHROUGH PER CANDIDATE
|
||||
|
||||
| Candidate | Embedded? | Compatibility Result |
|
||||
|-----------|-----------|---------------------|
|
||||
| A — Parent chain only | YES | ACCEPT (compatible via normalization) |
|
||||
| B — Decision-option path | YES (may_cause = SUFFICIENT) | ACCEPT (compatible via normalization) |
|
||||
| C — Any graph path | YES | ACCEPT (but too permissive in general) |
|
||||
| D — Relation-family-aware | YES (may_cause ∈ decision-consequence family) | ACCEPT (compatible via normalization) |
|
||||
| E — Parent OR decision-option | YES (both routes apply) | ACCEPT (compatible via normalization) |
|
||||
|
||||
---
|
||||
|
||||
## DECISION CRITERIA EVALUATION
|
||||
|
||||
| Criterion | A | B | C | D | E |
|
||||
|-----------|---|---|---|---|---|
|
||||
| 1. Accepts 60B.12 client-retention unknown | YES | YES | YES | YES | YES |
|
||||
| 2. Does not rely on keywords | YES | YES | YES | YES | YES |
|
||||
| 3. Does not treat arbitrary connectivity as context ownership | YES | PARTIAL (needs path limit) | NO | PARTIAL (needs family rules) | YES |
|
||||
| 4. Preserves genuine pattern transitions | YES | DEBATABLE | NO | DEBATABLE | YES |
|
||||
| 5. Uses existing schema/relationships | YES | YES | YES | PARTIAL (family needs definition) | YES |
|
||||
| 6. Is deterministic and provider-agnostic | YES | YES | YES | PARTIAL | YES |
|
||||
|
||||
---
|
||||
|
||||
## WINNING MODEL
|
||||
|
||||
**Choice:** E — PARENT OR DECISION-OPTION PATH
|
||||
|
||||
**Why:**
|
||||
1. Candidate A alone is too narrow (misses edge-connected nodes).
|
||||
2. Candidate B is a strong runner-up but Route B's relationship-by-relationship analysis shows that not all incoming edges are sufficient — requiring additional disambiguation logic.
|
||||
3. Candidate C is too permissive for any production use.
|
||||
4. Candidate D requires inventing semantic family rules not present in the current schema, increasing implementation complexity.
|
||||
5. **Candidate E provides two independent, semantically distinct routes with clear boundaries:** the explicit parent-chain (already implemented in determineActiveReasoningPattern) and the direct-decision-option path (may_cause/causes/affects to option → contained_in → decision). Neither route alone is sufficient; together they cover all common embedding patterns without enabling arbitrary graph connectivity.
|
||||
|
||||
**Exact structural-embedding predicate:**
|
||||
> A new unresolved unknown X is embedded in active context Y if:
|
||||
> 1. Any ancestor in X's parentId chain has reasoning pattern Y, OR
|
||||
> 2. X connects via may_cause/causes/affects edge to node Z, and Z.parentId (direct) or Z.childIds contains a node with reasoning pattern Y.
|
||||
|
||||
**Smallest implementation boundary:**
|
||||
One conditional branch in `assessReasoningPatternCompatibility` (apply-proposal.js ~line 1897), reusing existing `buildParentChain` and checking edge relationships on the current graph without new traversals or schema changes.
|
||||
|
||||
---
|
||||
|
||||
## IMPLEMENTATION READINESS
|
||||
|
||||
**A — READY FOR BOUNDED IMPLEMENTATION**
|
||||
|
||||
No unresolved design questions. The structural embedding predicate is fully defined using existing schema types and relationship semantics. The two routes (parent chain, decision-option path) map directly to existing data structures.
|
||||
|
||||
---
|
||||
|
||||
Production code changed: NO
|
||||
Prompt changed: NO
|
||||
Validator changed: NO (read-only analysis only)
|
||||
Schema changed: NO
|
||||
Tests changed: NO
|
||||
Ollama calls: 0
|
||||
Live API calls: 0
|
||||
Vitest run: NO
|
||||
Documentation updated: YES
|
||||
|
||||
Git status: clean (documentation commit pending)
|
||||
Reference in New Issue
Block a user