Feature/product platform foundation v0.62 #1
@@ -2928,3 +2928,7 @@ Experiment 60B.31 was a live regression of 60B.29 on the post-60B.30 branch to v
|
||||
---
|
||||
|
||||
Experiment 60B.32 was a code-path diagnosis of why the full runtime produces generic decision-justification questions for nodes whose descriptions start with bare "Whether..." — despite deterministic tests producing correct evidence-framed propositions. **Classification: D — ACTIVE CONTEXT DRIFT.** Three compounding factors in question-formulator.js cause the divergence: (1) extractMeaning's proposition detection requires label keywords (likelihood/status/probability/chance/risk/uncertainty) that "Enterprise customer signing decision" lacks, so the meaning strips "Whether" and isWhetherPropositionMeaning returns false; (2) hasDecisionValueLanguage becomes true because the description contains "value" within a parent-driven decision context; (3) extractActionPhrase scans ALL related nodes including the parent product-launch node, which supplies an action phrase ("launch") embedded in the generic template. The focused test avoids all three because its graph has no parent nodes and its label "Supplier renewal likelihood" contains the keyword "likelihood". Minimum corrective boundary: make extractMeaning recognize bare "Whether..." descriptions regardless of label keywords, so isWhetherPropositionMeaning returns true and blocks decision_threshold override. No production code changed. 0 Ollama calls. Pure code inspection.
|
||||
|
||||
---
|
||||
|
||||
Experiment 60B.34 was a live regression on `feature/bare-whether-proposition-v0.33` to verify that bare `Whether...` description preservation survives the full update route end-to-end. **Classification: B — FULL PROPOSITION PRESERVED BUT DIFFERENT SUPPORTED PREFIX EXERCISED.** One bounded update call produced: new unknown node `n_prospective_customer_signing` with description starting `"Unknown whether..."` (not bare `"Whether..."`). Decision remains unresolved (`n_product_launch_decision` still unknown). Final question: `"What evidence would clarify whether one prospective enterprise customer will sign if we launch this year?"` — full proposition preserved, evidence-framed, zero rationale contamination. The exact 60B.33 targeted fix (bare Whether...) was not exercised live; the model produced `Unknown whether...` instead, a different but functionally equivalent supported prefix path from the same code base. Equivalent outcome to 60B.31's deterministic regression. One live Ollama call at qwen-claude:latest on http://192.168.1.111:11434. No production code changed.
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
# Experiment 60B.34 — Live Bare `Whether` Proposition Preservation
|
||||
|
||||
**Branch:** `feature/bare-whether-proposition-v0.33`
|
||||
**Starting HEAD:** `2996c30` (feature/bare-whether-proposition-v0.33)
|
||||
**Date:** 2026-08-14
|
||||
**Status:** COMPLETE
|
||||
|
||||
## Objective
|
||||
|
||||
Answer whether the live bare `Whether...` case now preserves the full proposition end to end through the production update route, while keeping the material target and decision state intact.
|
||||
|
||||
This is an observation-only live regression against the deterministic fix recorded in 60B.33.
|
||||
|
||||
## Configured Model
|
||||
|
||||
- **Model:** `qwen-claude:latest`
|
||||
- **Ollama base URL:** `http://192.168.1.111:11434`
|
||||
- **Host:** `127.0.0.1:3000` (confidence-engine dev server)
|
||||
|
||||
## Call Budget
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| startCalls | 0 |
|
||||
| updateCalls | 1 |
|
||||
| totalCalls | 1 |
|
||||
| Retries | 0 |
|
||||
|
||||
## Fixed Input
|
||||
|
||||
```text
|
||||
The revenue and launch-cost estimates are good enough for the decision. The remaining issue is one prospective enterprise customer. We do not yet know whether they would sign if we launch this year, and they account for about £700,000 of the £1.2 million expected annual revenue.
|
||||
```
|
||||
|
||||
## Fixed Fixture
|
||||
|
||||
`tests/fixtures/pre-anchored-product-launch-options.json`
|
||||
|
||||
## Live Result
|
||||
|
||||
### HTTP / Stage
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| HTTP status | 200 (success) |
|
||||
| Stage | ACCEPTED (update applied) |
|
||||
| Validation errors | None |
|
||||
|
||||
### Structural Mutation
|
||||
|
||||
- **updatedNodes:** `[]`
|
||||
- **resolvedUnknownNodeIds:** `[]`
|
||||
- **addedNodes:** `1` (`n_prospective_customer_signing`)
|
||||
- **addedEdges:** `1` (`e-signing-to-option`, `depends_on`)
|
||||
|
||||
### Customer Node (new)
|
||||
|
||||
- **id:** `n_prospective_customer_signing`
|
||||
- **kind:** `unknown`
|
||||
- **label:** `"Prospective enterprise customer signing status"`
|
||||
- **status:** `unknown`
|
||||
- **description:** `"Unknown whether one prospective enterprise customer will sign if we launch this year, because they account for approximately £700,000 of the £1.2 million expected annual revenue, so that we can determine if launching this year remains net-positive."`
|
||||
|
||||
### Proposal Targeting
|
||||
|
||||
- **Proposal selectedQuestion.nodeId:** `"n_prospective_customer_signing"`
|
||||
- **Final selectedQuestion.nodeId:** `"n_prospective_customer_signing"`
|
||||
- **Final selectedQuestion.question:** `"What evidence would clarify whether one prospective enterprise customer will sign if we launch this year?"`
|
||||
|
||||
### Assessment
|
||||
|
||||
| Category | Classification |
|
||||
|----------|---------------|
|
||||
| Core reasoning chain | PRESERVED |
|
||||
| Customer-signing factor | FIRST-CLASS UNKNOWN |
|
||||
| Prefix form exercised | UNKNOWN WHETHER |
|
||||
| Preferred-target behaviour | MATERIAL FACTOR PRESERVED |
|
||||
| Full proposition preservation | FULL |
|
||||
| Evidence framing | EVIDENCE FRAMED |
|
||||
| Rationale contamination | NONE |
|
||||
| Source graph meaning | SOURCE DESCRIPTION PRESERVED |
|
||||
|
||||
### Key Observations
|
||||
|
||||
1. **Decision status preserved:** `n_product_launch_decision` remains `kind=unknown, status=unknown`.
|
||||
2. **Customer-signing factor created as first-class unknown node** (`n_prospective_customer_signing`), with proper edge to the material option.
|
||||
3. **Final question preserves the full proposition:**
|
||||
- `"whether one prospective enterprise customer will sign if we launch this year"` — both the commitment condition and the timeframe are intact.
|
||||
4. **Evidence framing used:** `"What evidence would clarify..."` prefix.
|
||||
5. **No rationale contamination** in the final question (no `£700k`, `£1.2M`, `annual revenue`).
|
||||
6. **Source description preserved** — the new node's description retains the full original text including financial figures (rationale correctly kept in source graph, stripped from question).
|
||||
|
||||
### Prefix Form Analysis
|
||||
|
||||
The live model produced `"Unknown whether"` as the prefix form, NOT bare `"Whether"`.
|
||||
|
||||
This is a different but already-supported prefix from 60B.33's change set. The 60B.33 fix specifically targeted bare `Whether...` at the description-start boundary; however, the "Unknown whether..." path was also supported and remains functional (it predates or runs in parallel to the bare Whether fix).
|
||||
|
||||
### 60B.31 Comparison
|
||||
|
||||
| Dimension | 60B.31 | 60B.34 |
|
||||
|-----------|--------|--------|
|
||||
| Prefix form | BARE WHETHER | UNKNOWN WHETHER |
|
||||
| Final target | Correct (customer) | Correct (customer) |
|
||||
| Full proposition preservation | FULL (deterministic fixture) | FULL (live) |
|
||||
| Final question shape | `"What evidence would clarify whether the prospective enterprise customer will commit this year?"` | `"What evidence would clarify whether one prospective enterprise customer will sign if we launch this year?"` |
|
||||
| Rationale contamination | NONE | NONE |
|
||||
|
||||
Both 60B.31 (deterministic) and 60B.34 (live) produce the same question shape pattern: **evidence-framed interrogative preserving the full proposition with no rationale contamination.** The prefix form differs, but the downstream behaviour is identical.
|
||||
|
||||
## Classification: B — FULL PROPOSITION PRESERVED BUT DIFFERENT SUPPORTED PREFIX EXERCISED
|
||||
|
||||
The outcome is correct and fully preserved, but the live model produced `Unknown whether...` rather than bare `Whether...`. The exact 60B.33 branch fix was not directly exercised in this live run, though its parallel-supported prefix path produces identical downstream results.
|
||||
|
||||
## What Improved Relative to 60B.31
|
||||
|
||||
None — 60B.34 shows equivalent behaviour to the 60B.31 deterministic regression. The question shape, proposition preservation, rationale stripping, and evidence framing are all consistent across both runs.
|
||||
|
||||
## What Remains Weak or Unproven
|
||||
|
||||
- The exact bare `Whether...` prefix was not directly exercised live. It works in the deterministic fixture (60B.33), but this run did not confirm it fires in production under this specific model/host combination.
|
||||
- No cross-model verification (qwen-claude:latest only).
|
||||
- The "Unknown whether..." path, while functionally correct, is distinct from the targeted 60B.33 fix and was never the focus of that change.
|
||||
|
||||
## Production Code Changed
|
||||
|
||||
NO
|
||||
|
||||
## Harness Modified
|
||||
|
||||
NO (used existing `FIXTURE_MODE=updateOnly`)
|
||||
|
||||
## Vitest Run
|
||||
|
||||
NO
|
||||
|
||||
## Ollama Calls
|
||||
|
||||
1 MAXIMUM
|
||||
|
||||
## Direct API Calls
|
||||
|
||||
0
|
||||
|
||||
## Dev Server Disturbed
|
||||
|
||||
NO
|
||||
|
||||
## Documentation Updated
|
||||
|
||||
docs/experiment-60b34.md created
|
||||
docs/current-handoff.md updated (append)
|
||||
Reference in New Issue
Block a user