exp 60A.1: read-only vocabulary adequacy diagnosis for alternatives and decisions
Diagnoses the root cause of the persistent pattern from 59B.2-59B.4 where explicit dual-option input collapsed into a single undifferentiated unknown node. Concludes the graph vocabulary lacks first-class primitives for options/decisions (not primarily a prompt issue). Identifies three missing primitives: option node kind, decision node kind, alternative_of edge type. Recommends ~25-line schema addition for 60A.2 implementation.
This commit is contained in:
@@ -2092,3 +2092,45 @@ Full record in `docs/experiment-59b2.md`. Configured Ollama: qwen-claude:latest
|
||||
|
||||
### Experiment 59B.4 — Explicit Two-Option Decision Structure (full record in docs/experiment-59b4.md)
|
||||
|
||||
### Experiment 60A.1 — Read-Only Vocabulary Adequacy Diagnosis for Alternatives and Decisions
|
||||
|
||||
**Branch:** `feature/question-formulation-v0.24`
|
||||
**Date:** 2026-08-12
|
||||
**Type:** READ-ONLY ARCHITECTURE DIAGNOSIS
|
||||
**Full record:** `docs/experiment-60a1.md`
|
||||
|
||||
**Context route:** This diagnosis follows experiments 59B.2–59B.4 which showed the persistent pattern: engine understood trade-offs semantically but could not preserve two competing options as independently recoverable structural entities in the graph.
|
||||
|
||||
**Objective:** Answer whether the model fails to use decision structure that already exists, or whether the current graph vocabulary lacks an adequate first-class representation for alternatives and decisions.
|
||||
|
||||
**Methodology:** Read-only analysis of `lib/graph/schema.js`, `lib/graph/prompt-builder.js`, `lib/graph/utils.js`, `lib/graph/apply-proposal.js`, and experiment history. No production code changes, no API calls, no test runs.
|
||||
|
||||
**Key findings from 8-section diagnosis:**
|
||||
|
||||
1. **Vocabulary inventory (Section 1):** SituationKind has 9 node kinds (observation, reported_claim, metric, state, transition, relationship, assumption, unknown, conclusion) — zero kinds represent choices, options, or decisions. SituationRelationship has 10 edge types including `compares_with` but with no alternative-specific semantics.
|
||||
|
||||
2. **Representational adequacy test (Section 2):** The schema cannot express a "decision" node as a first-class entity. The schema cannot express "these two options are alternatives for the same decision." Consequences can attach only if option nodes exist first — a chicken-and-egg problem. The do-nothing baseline has no dedicated representation.
|
||||
|
||||
3. **Concept mapping for 59B.4 (Section 3):** User input had 8 distinct structured components (2 options, 6 consequences, comparison intent). Schema mapped 0 of these as structure. All 8 collapsed into one undifferentiated unknown node's prose description.
|
||||
|
||||
4. **Prompt-vs-schema diagnosis (Section 4):** The schema lacks the primitives AND the prompt lacks rules to use them — but root cause is schema-level. No existing primitive semantically means "an available choice." All four prompt-level reuse candidates (`state`, `compares_with`, `contradicts`, `transition`) fail because no rule references them for alternatives.
|
||||
|
||||
5. **Reuse strategy evaluation (Section 5):** Four candidate strategies tested (`state` as options, `observation` for consequences, repurpose `compares_with`, `transition` for outcomes). All are either semantically incorrect or incomplete without schema-level support. No reuse strategy is viable without adding new primitives.
|
||||
|
||||
6. **Minimum architectural distinction (Section 6):** Three missing primitives identified: DECISION node kind, OPTION node kind, and ALTERNATIVE-OF edge type. These three additions would enable full representation of the 59B.4 decision scenario. Smallest improved graph: ~25 lines total.
|
||||
|
||||
7. **Smallest improved graph (Section 7):** Concrete proposal — add `option` and `decision` to SituationKind, add `alternative_of` and `contains_option` to SituationRelationship, add 4 prompt rules for option creation. No breaking changes. Existing contract unchanged. ~25 lines of code.
|
||||
|
||||
8. **Implementation readiness (Section 8):** Schema additions are low-risk trivial enum extensions. Prompt rule additions are straightforward. Recommendation: next experiment should implement the three-primitive addition and re-run the 59B.4 scenario to verify structural option representation.
|
||||
|
||||
**Classification: DIAGNOSIS COMPLETE — SCHEMA-LEVEL ROOT CAUSE IDENTIFIED**
|
||||
|
||||
The current graph vocabulary lacks first-class primitives for alternatives and decisions. The model cannot represent what the schema does not define. Schema-level additions are necessary; prompt-only changes would not resolve the root cause.
|
||||
|
||||
Production code changed: NO
|
||||
Prompt changed during experiment: NO
|
||||
Validator changed during experiment: NO
|
||||
Vitest run: NO
|
||||
Ollama calls: 0
|
||||
Dev server disturbed: NO
|
||||
Read-only diagnosis: YES
|
||||
|
||||
Reference in New Issue
Block a user