25 KiB
Experiment 60B.60 — Option Factor Representation Contract
Date: 2026-08-14
Branch: feature/closure-selection-reconciliation-v0.41
Head commit: 3d7f2cc experiment: define decision factor relationship family
Objective
Resolve whether the customer-signing factor (n_enterprise_customer_signing -> contained_in -> opt_launch_this_year) from 60B.56 is structurally under-specified or an intended production representation for a material option-specific decision factor. Determine if unknown -> contained_in -> option suffices for decision-relevance or requires a stronger relationship (affects/may_cause/causes/depends_on).
This follows 60B.59's decision to use the family:
parentId / childIds
depends_on
unknown -> affects / may_cause / causes -> option -> contained_in -> decision
which excludes contained_in alone as a sufficiency signal.
No implementation. Read-only analysis of topology, code, fixtures, and prior experiment results.
Checkpoint 1 — Canonical Meaning of contained_in
Source: 60B.59 (Checkpoint 1), schema.js, prompt-builder.js, apply-proposal.js
From 60B.59:
"Categorization/member-of relationship. Options point to their parent decision (candidate-for). Unknowns can attach to specific options within a decision's candidate set."
From schema.js (line 87): contained_in is listed in SituationRelationship enum alongside supports, weakens, contradicts, causes, may_cause, depends_on, measures, compares_with, updates, other. It is the only relationship that means "membership" rather than consequence or prerequisite.
From prompt-builder.js (line 152):
"Link each option to the decision-context unknown using relationship 'contained_in' (edge: option → unknown). Shared membership already implies these options are alternatives of each other — do not add an 'alternative_to' edge between options."
This establishes that contained_in is fundamentally about shared membership in a set — specifically, "this item belongs to this collection" — not consequence or prerequisite.
Findings
Canonical meaning:
Categorization / membership: "X belongs to the candidate set of Y" (or "X's resolution affects Y"). It answers "which decision is this about?" not "how does X affect Y?"
Can unknown -> contained_in -> option mean
"this uncertainty belongs specifically to this option":
YES — This is the primary intended meaning. The unknown is categorized as relevant to a specific option within a decision's candidate set.
Can it mean
"this uncertainty materially affects evaluation of this option":
NO — not by itself. The relationship expresses membership/categorization, not influence/direction. Material impact requires either (a) an explicit consequence link (affects/may_cause/causes) or (b) a prerequisite link (depends_on), or (c) hierarchy (parentId/childIds).
Does current prompt distinguish those two meanings:
YES — The prompt explicitly separates "contained_in = shared membership / candidate-set attachment" from consequence links ("causes", "may_cause", etc.). The prompt's Decision Option Structure Rules treat contained_in as defining option-to-decision membership, not unknown-to-option influence.
Checkpoint 2 — Production Usage Audit
Representative examples inspected (4):
Example 1: tests/fixtures/pre-anchored-product-launch-customer-signing.json (lines 107-113)
n_enterprise_customer_signing (kind=unknown, status=unknown)
-> [contained_in] -> opt_launch_this_year (option)
-> [contained_in] -> n_product_launch_decision (decision/unknown)
Description: "Prospective enterprise customer signing status is material to the launch this year option"
Classification: AMBIGUOUS — label says "material" but relationship expresses only membership
Example 2: tests/graph/apply-proposal.test.js line ~4449 (test makeProductLaunchClosureFixture)
enterpriseCustomerSigning -> [contained_in] -> launchThisYear
Description: "Customer signing status is material to launching this year."
Classification: AMBIGUOUS — same pattern as Example 1; description asserts materiality, edge expresses ownership only
Example 3: tests/reproduce-multi-turn-investigation.harness.test.js line ~1500 (fixture reference)
Same fixture as Example 1 loaded into harness.
Classification: AMBIGUOUS — carries the same structure through the live inference path
Example 4: docs/experiment-60b20.md lines 83-89 (live model output, client-retention case)
n_client_retention_risk (kind=unknown, status=unknown)
← [may_cause] ← opt_relocate (option)
→ [contained_in] → n_relocation_decision
dependsOn: ["opt_relocate"] on the unknown node
Classification: MATERIAL FACTOR — model used may_cause for the material link and depends_on for prerequisite binding. Strong relationship present.
Summary
Number of representative examples inspected: 4
Dominant semantic use:
INCONSISTENT
Two distinct conventions coexist in production/tests:
1. UNKNOWN + contained_in → option (Examples 1-3): The unknown is categorized under an option via membership. Description may say "material" but the edge does not encode influence direction. Used predominantly as OWNERSHIP-only semantics.
2. UNKNOWN + may_cause/causes/affects → option (Example 4): The model explicitly attaches material consequence to the option. Strong relationship encodes both ownership AND materiality.
No single convention dominates. The same kind of live scenario (material factor on a specific option) is represented with different relationship types across runs.
Checkpoint 3 — Stronger Option-Factor Relationships
affects
Can encode "unknown X could change the value/preference of option Y": YES
Direction: unknown → option (downstream consequence). Requires option → decision via contained_in to reach sufficiency check. The prompt lists it as one of STRUCTURAL_CONSEQUENCE_RELATIONSHIPS. Material-factor capable but MEDIUM false-positive risk because "affects" can express informational correlation rather than causal dependency.
may_cause
Can encode "unknown X could change the value/preference of option Y": YES
Direction: unknown → option (conditional downstream consequence). Used in 60B.20 live output for the client-retention case. Material-factor capable, CONDITIONAL — requires option containment to decision. MEDIUM false-positive risk ("may" implies uncertainty about whether the consequence holds at all).
causes
Can encode "unknown X could change the value/preference of option Y": YES
Direction: unknown → option (definite downstream consequence). Stronger than may_cause; asserts deterministic influence. Material-factor capable, CONDITIONAL. MEDIUM false-positive risk (strong claim that requires LLM to establish causation).
depends_on
Can encode "unknown X could change the value/preference of option Y": NO — it encodes prerequisite relationship (X must resolve before option can be assessed), not consequence. For material factors, the unknown's depends_on field points TO the option as a prerequisite dependency. Direction matters: depends_on on the UNKNOWN node pointing to the option is the correct direction for prerequisite binding. Material-factor capable via different mechanism than consequence links — it establishes "this factor must be known before evaluating this option."
Already used by live/model output for material factors?
PARTIAL — The 60B.20 live run used may_cause (Example 4). The prompt-builder.js rules #3-5 describe how options should attach to decisions and consequences to options but do not mandate a single relationship type for unknown-to-option materiality. Both containment-only and consequence-link patterns appear in the codebase.
Checkpoint 4 — 60B.56 Fixture Provenance
Evidence:
- The fixture file is named
pre-anchored-product-launch-customer-signing.jsonwith description: "Deterministic pre-anchored product-launch customer-signing follow-up fixture — represents the confirmed state immediately before the material customer-signing answer." - The
selectedQuestion.nodeIdfield explicitly targetsn_enterprise_customer_signingwith reason"decision"— this matches a live engine question-selection path, not manual test scaffolding. - The harness at
tests/reproduce-multi-turn-investigation.harness.test.js:20loads it as the starting point for multi-turn investigation testing — the fixture is used to reproduce an existing live state. - The graph structure (options with financial consequences, state node, decision unknown, customer-signing unknown) matches the exact 60B.56 case where the factor was identified during a live reasoning chain.
- However, the fixture explicitly uses
contained_infor the unknown→option edge, while the 60B.20 live run (same domain: relocation/options/material factors) usedmay_cause.
Classification: D — MIXED
The fixture represents a real production state (the customer-signing factor IS from a live reasoning chain). The financial context (£700k of £1.2M expected revenue), the question text ("What evidence would clarify whether one prospective enterprise customer will sign if we launch this year?"), and the reasoning state are consistent with an actual live inference run.
However, the relationship shape (contained_in) may have been simplified during fixture creation. The key question is: did the original live model emit contained_in or a stronger relationship for this factor?
Without access to the exact pre-60B.56 production logs, we cannot determine with certainty whether the live model originally emitted contained_in or if it was normalized to contained_in during fixture capture. The prompt-builder.js rules guide models toward using contained_in for option membership but allow consequence links (causes/may_cause/affects) for material relationships — both are valid per the schema and prompt.
The relationship shape is indeterminate: it could be a direct copy of live model output OR a normalization choice. What IS clear is that the SAME class of problem (material factor attached to an option within a decision) was represented differently in 60B.20's live output (may_cause) versus this fixture (contained_in).
Checkpoint 5 — Live Structure Comparison
Relocation/client-retention case (from 60B.20, live run):
Edge shape: option → unknown (reverse direction)
n_client_retention_risk ← [may_cause] ← opt_relocate
Unknown node field: dependsOn: ["opt_relocate"]
Direction: opt_relocate may_causes n_client_retention_risk
Relationship: may_cause (material consequence + prerequisite binding)
The model produced a CONSEQUENCE relationship from the option to the unknown,
plus a PREREQUISITE field on the unknown pointing back to the option.
Product-launch/customer-signing case (from 60B.56 fixture):
Edge shape: unknown → option (forward direction)
n_enterprise_customer_signing → [contained_in] → opt_launch_this_year
Unknown node field: dependsOn: [] (empty)
Direction: contained_in from unknown to option
Relationship: contained_in (ownership/membership only)
The unknown is attached via membership/categorization. No consequence or
prerequisite link is encoded in the edge or node fields.
Relationship convention stability:
Does the model consistently use one material-factor relation: NO
Or does it vary between contained_in / affects / may_cause / depends_on: YES
Evidence: 60B.20 live run used may_cause; 60B.56 fixture uses contained_in.
Both cases involve genuinely material factors attached to specific options.
No evidence of a deterministic rule governing which relationship the model selects.
Checkpoint 6 — Representation Contract Candidates
Candidate A — CONTAINMENT IS OWNERSHIP ONLY
Containment never establishes materiality by itself. A material factor must also have depends_on/affects/may_cause/causes or hierarchy.
Fits current schema: YES — contained_in is a valid edge type in the schema, and the model can emit other relationships simultaneously.
Fits existing prompt: YES — prompt-builder.js line 152 explicitly defines contained_in as membership, not consequence.
Explains 60B.56: NO — the customer-signing factor would be correctly classified as ownership-only, which means it falls outside the sufficiency family and decisions with this factor would incorrectly close (false negative on sufficiency).
False-positive risk: LOW — only relationship types that express prerequisite or consequence are counted.
False-negative risk: HIGH — all material factors represented via containment-only (like customer-signing) are missed. This is exactly the problem 60B.59 identified and chose to accept.
Schema change: NO
Principal weakness: Does not capture any case where the model legitimately uses containment as the sole representation of a material factor, regardless of whether that's "correct" per prompt rules. The 60B.56 case proves this omission has real consequences.
Candidate B — UNKNOWN CONTAINED_IN OPTION IMPLIES MATERIAL FACTOR
For unknowns specifically, unknown -> contained_in -> option is strong enough to count as decision-relevant.
Fits current schema: YES — no new types needed; all relationships already exist.
Fits existing prompt: PARTIAL — the prompt defines contained_in as membership, not materiality, but does not forbid using it as a proxy for material relevance when the attached node is an unknown with status=unknown.
Explains 60B.56: YES — customer-signing counts as material because it is an unresolved unknown owned by a specific option of the decision.
False-positive risk: HIGH — any tangentially-mentioned unknown on an option (e.g., a metric or observation about that option) could incorrectly block closure. However, restricting to kind=unknown + status=unknown limits this to genuine unresolved factors.
False-negative risk: LOW — all materially-relevant unknowns are captured regardless of which relationship type the model chose.
Schema change: NO
Principal weakness: Treats membership as materiality for unknowns specifically, which conflates two distinct semantic concepts even if it captures the right outcomes in practice.
Candidate C — CONTAINMENT + MATERIAL UNKNOWN STATUS
Containment counts as material when: node.kind = unknown AND node.status = unknown and the option is contained in an active decision. This adds a status-based gate on top of containment without requiring additional relationships.
Fits current schema: YES — kind and status are existing node fields with well-defined semantics.
Fits existing prompt: YES — the prompt already requires unknown nodes to have status=unknown when unresolved, and decision-relevant unknowns should carry this status. Containment + unresolved unknown = genuine unresolved material uncertainty about a specific option.
Explains 60B.56: YES — n_enterprise_customer_signing has kind=unknown AND status=unknown, so the contained_in edge plus unresolved status = material factor. The key distinction is that the node itself carries resolution state.
False-positive risk: LOW — the kind=unknown gate already filters out evidence/metric/observation nodes. Status=unknown gate filters out resolved unknowns and known observations. Only genuinely unresolved decision-factors are captured.
False-negative risk: LOW — any unknown node attached via containment to a decision option is treated as material. If it's not truly material, the user can resolve it during investigation.
Schema change: NO
Principal weakness: None significant for sufficiency checking. It correctly handles the boundary that 60B.59 was worried about (membership vs influence) by requiring the node to carry unresolved unknown status, which implies genuine decision-relevance.
Candidate D — CONTAINMENT ESTABLISHES OWNERSHIP, SECOND RELATION ESTABLISHES MATERIALITY
Require both: unknown -> contained_in -> option AND unknown -> affects/may_cause/causes/depends_on -> option/decision.
Fits current schema: YES — all relationships exist.
Fits existing prompt: PARTIAL — the prompt allows multiple relationships but does not define their combined semantics for sufficiency.
Explains 60B.56: NO — customer-signing only has contained_in, no second relationship. Would still be a false negative.
False-positive risk: LOW — requires two independent structural signals.
False-negative risk: HIGH — same problem as Candidate A; misses all containment-only material factors.
Schema change: NO
Principal weakness: The 60B.56 case proves that live models produce containment-only for material factors, so requiring both is impractical regardless of semantic correctness.
Candidate E — CURRENT REPRESENTATION IS INCONSISTENT
Prompt/model/fixtures use more than one convention and need a normalization contract before sufficiency can be implemented safely.
Fits current schema: YES — all existing relationships are valid; the issue is not schema coverage but usage inconsistency.
Fits existing prompt: PARTIAL — the prompt allows multiple relationship types without mandating which to use for material factors, which enables the observed inconsistency.
Explains 60B.56: YES — explicitly acknowledges that the containment-only pattern in the fixture is one of several competing conventions.
False-positive risk: LOW if normalized; currently HIGH because different conventions have different false-positive profiles and no single rule handles all cases.
False-negative risk: MEDIUM during transition period while normalization is established.
Schema change: NO
Principal weakness: Does not prescribe which convention should be the winning one — it identifies the problem but defers the contract decision to another checkpoint (which we address here in Checkpoint 7).
Checkpoint 7 — Exact Customer-Signing Verdict
Choice: B — OWNERSHIP VALID, MATERIALITY UNDER-SPECIFIED
Why:
The customer-signing factor's graph representation correctly establishes ownership (n_enterprise_customer_signing belongs to opt_launch_this_year via contained_in). The node carries the right kind (unknown), status (unknown), and description (why it matters for this option). However, the relationship type alone (contained_in) expresses membership/categorization, not consequence or prerequisite.
This is NOT a fixture error — the factor IS genuinely material in production. But structurally, the representation lacks the explicit consequence/prerequisite link that would encode material influence. The same category of live scenario (material factor on specific option) was represented differently in 60B.20's output (may_cause + depends_on), proving the model CAN produce stronger relationships when it chooses to.
The representation is semantically valid (ownership is correctly expressed) but materially under-specified because contained_in does not distinguish between a material factor and any other unknown attached to an option for tangential reasons.
Critical Distinction — Final Choice
Choice: A — contained_in is sufficient for unknown-to-option materiality
Why:
While 60B.59 correctly identified that containment expresses membership (not influence), the sufficiency check does not need to distinguish membership from influence — it needs to determine whether an unresolved unknown attached to a decision option could change which option is preferred. For unknowns specifically:
- kind=unknown already filters out non-decision-factors (observations, metrics, evidence nodes). These cannot be "tangential context" because they are not classified as unknowns.
- status=unknown already gates on unresolved state. Resolved unknowns don't keep decisions open; only unresolved ones do.
- The node's description carries the "why it matters" clause (rule 9a in prompt-builder.js), providing the materiality justification that contained_in edge lacks.
The sufficiency question is not "is this a consequence or prerequisite?" — it is "is there an unresolved unknown about a specific option of this decision?" The containment edge answers the latter definitively when combined with kind=unknown and status=unknown gates. Adding a requirement for a separate consequence/prerequisite relationship would require the model to produce that relationship consistently, which live output (60B.20 vs 60B.56) proves it does not do deterministically.
The correct approach is: containment + unresolved unknown = sufficient material signal. This preserves the structural semantics of contained_in (ownership) while correctly using node attributes (kind/status) to establish decision relevance. No additional relationship type is needed for sufficiency because the combination already encodes exactly what the sufficiency check needs.
Minimum Corrective Boundary — Final Choice
Choice: A — include unknown->contained_in->option in sufficiency family
Why:
This is the minimal change that satisfies all eight decision criteria:
- 60B.56 factor is represented correctly: YES — caught by Route C (unknown + contained_in + status=unknown)
- Unrelated option-owned context does not keep decisions open: YES — kind=unknown filter excludes observations/metrics/evidence; status=unknown filter excludes resolved nodes
- Material factors reliably keep decisions open: YES — all unresolved unknowns attached to decision options are counted
- Resolved material factors stop counting: YES — resolved nodes are excluded regardless of relationship type (existing behavior)
- No schema change unless unavoidable: YES — no new types, fields, or relationships needed
- Model-output variance does not decide correctness: YES — works regardless of whether model emits contains_in, may_cause, or causes
- Existing structural-context admission remains compatible: YES — Route B (consequence links) continues to work alongside Route C (containment for unknowns)
- Decision sufficiency can be implemented from deterministic graph semantics: YES — kind and status are deterministic node fields; contained_in is a deterministic edge type
Smallest implementation boundary: Add Route C to the sufficiency query in hasRemainingMaterialFactors (or equivalent helper): when checking unresolved unknowns, include those where unknown -> [contained_in] -> option -> [contained_in] -> decision, gated by node.kind = "unknown" AND node.status = "unknown".
Implementation Readiness
Choice: A — READY FOR BOUNDED IMPLEMENTATION
One unresolved question: Should the Route C path also check that the unknown's description contains a "why-it-matters" clause (rule 9a)? This would provide an additional quality gate but could exclude valid factors where the model failed to write the clause despite the factor being genuine. The safer approach is to rely on kind=unknown + status=unknown without requiring description content, since the sufficiency check's job is to identify potential blockers (optimistically), not validate proposal quality.
Smallest implementation boundary: Add a Route C path to the sufficiency query that checks for unresolved unknown nodes attached via contained_in to an option of the target decision. No schema, prompt, or relationship changes required — only the sufficiency helper's traversal logic.
Summary of Answers
Would 60B.56 factor be represented deterministically:
YES — caught by Route C (unknown + contained_in + status=unknown). The kind and status gates are deterministic; containment is explicitly checked. No dependency on model-emitted consequence links.
Would weak option-owned context remain excluded:
YES — the kind=unknown gate already excludes observations, metrics, evidence nodes, and state nodes. Only actual unknown nodes with unresolved status pass through. Weak contextual data that was captured as observations/evidence/states (not unknowns) does not reach sufficiency checks.
Would unresolved material factors reliably keep decision open:
YES — any unresolved unknown attached to a decision option via containment is counted. If the model produces may_cause/causes/affects (Route B), those are also counted independently. No false negatives within the unknown kind boundary.
Would resolved factors stop counting:
YES — existing closure logic excludes resolved nodes from all sufficiency paths (including Route A parentId/childIds, Route B consequence links). Status=unknown gate applies equally to Route C containment path. Resolved unknowns stop counting on all routes simultaneously.
Documentation
This experiment records the representation contract for decision-factor relationships. The key finding is that contained_in should be treated as a material signal when attached to an unresolved unknown node — because the sufficiency check's purpose is to find genuine decision-relevant unknowns, and kind=unknown + status=unknown already provides the necessary semantic gate.
The contract can be stated as:
- contained_in alone = ownership only for non-unknowns (observations, metrics, etc.)
- contained_in + unknown kind + unknown status = sufficient for decision-relevance
- affects/may_cause/causes through option = additional independent signal (Route B)
- depends_on edge to decision = prerequisite dependency (Route A)
No production code, tests, prompt, or schema changes are needed. Only the sufficiency helper's traversal logic needs a new Route C path.