Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
59631f2e72 | ||
|
|
b73760d5a7 | ||
|
|
fe6a9925cb | ||
|
|
34c25fcb43 | ||
|
|
c27320984c | ||
|
|
3e2edd2edc | ||
|
|
b00928d6fb | ||
|
|
42d4da3496 | ||
|
|
db994d7764 | ||
|
|
ef04b9e494 | ||
|
|
3c0f7f5a45 | ||
|
|
449cf996dc |
@@ -0,0 +1,136 @@
|
|||||||
|
# v0.7 Observation Report
|
||||||
|
|
||||||
|
**Date**: 2026-08-03 | **Commit**: c273209 | **Branch**: feature/reasoning-pattern-memory-v0.7
|
||||||
|
|
||||||
|
## Summary Table
|
||||||
|
|
||||||
|
| Scenario | Name | Start | Update | Nodes | Unknowns | Rating |
|
||||||
|
|----------|------|-------|--------|-------|----------|--------|
|
||||||
|
| scenario-1 | Confidence Engine commercial validation | pass | fail(400) | 9 | 3 | flow failure |
|
||||||
|
| scenario-2 | Hiring | pass | fail(400) | 18 | 8 | flow failure |
|
||||||
|
| scenario-3 | Vehicle replacement | pass | fail(400) | 15 | 8 | flow failure |
|
||||||
|
| scenario-4 | Welsh Government-style programme decision | pass | fail(400) | 10 | 3 | flow failure |
|
||||||
|
| scenario-5 | Operational contradiction | pass | fail(400) | 7 | 2 | flow failure |
|
||||||
|
| scenario-6 | Personal decision | fail | skipped | 0 | 0 | flow failure |
|
||||||
|
|
||||||
|
## Per-Scenario Findings
|
||||||
|
|
||||||
|
### scenario-1: Confidence Engine commercial validation
|
||||||
|
|
||||||
|
- **Overall**: Start=pass, Update=fail(400), Rating=flow failure
|
||||||
|
- Pattern: N/A | Nodes: 9 | Edges: 0
|
||||||
|
- Validation: valid | Duration: 63386ms
|
||||||
|
- Unknown IDs: nirkgb4, n36c0cc, nzeyzkz
|
||||||
|
- Error: [N/A] Invalid update-case request
|
||||||
|
|
||||||
|
- **Assessment**:
|
||||||
|
- reasoning-pattern fit: fail
|
||||||
|
- one-concept simplicity: fail
|
||||||
|
- plain-language clarity: fail
|
||||||
|
- logical progression: fail (No question generated)
|
||||||
|
- graph-backed: fail
|
||||||
|
- premature-specialism avoided: fail
|
||||||
|
|
||||||
|
### scenario-2: Hiring
|
||||||
|
|
||||||
|
- **Overall**: Start=pass, Update=fail(400), Rating=flow failure
|
||||||
|
- Pattern: N/A | Nodes: 18 | Edges: 5
|
||||||
|
- Validation: valid | Duration: 146476ms
|
||||||
|
- Unknown IDs: n7yonyv, npci7a7, nug9wj2, nz0vpey, nz8pwyc, newxmzu, nw14mjj, n25mnp3
|
||||||
|
- Error: [N/A] Invalid update-case request
|
||||||
|
|
||||||
|
- **Assessment**:
|
||||||
|
- reasoning-pattern fit: fail
|
||||||
|
- one-concept simplicity: fail
|
||||||
|
- plain-language clarity: fail
|
||||||
|
- logical progression: fail (No question generated)
|
||||||
|
- graph-backed: fail
|
||||||
|
- premature-specialism avoided: fail
|
||||||
|
|
||||||
|
### scenario-3: Vehicle replacement
|
||||||
|
|
||||||
|
- **Overall**: Start=pass, Update=fail(400), Rating=flow failure
|
||||||
|
- Pattern: N/A | Nodes: 15 | Edges: 5
|
||||||
|
- Validation: valid | Duration: 81460ms
|
||||||
|
- Unknown IDs: ng5yr11, nogqips, n499gin, n8fbv3p, nf2f6zx, n4feiap, nvwthlt, nqrxjli
|
||||||
|
- Error: [N/A] Invalid update-case request
|
||||||
|
|
||||||
|
- **Assessment**:
|
||||||
|
- reasoning-pattern fit: fail
|
||||||
|
- one-concept simplicity: fail
|
||||||
|
- plain-language clarity: fail
|
||||||
|
- logical progression: fail (No question generated)
|
||||||
|
- graph-backed: fail
|
||||||
|
- premature-specialism avoided: fail
|
||||||
|
|
||||||
|
### scenario-4: Welsh Government-style programme decision
|
||||||
|
|
||||||
|
- **Overall**: Start=pass, Update=fail(400), Rating=flow failure
|
||||||
|
- Pattern: N/A | Nodes: 10 | Edges: 0
|
||||||
|
- Validation: valid | Duration: 129682ms
|
||||||
|
- Unknown IDs: nrrm3qn, nefmpat, n6rtwg1
|
||||||
|
- Error: [N/A] Invalid update-case request
|
||||||
|
|
||||||
|
- **Assessment**:
|
||||||
|
- reasoning-pattern fit: fail
|
||||||
|
- one-concept simplicity: fail
|
||||||
|
- plain-language clarity: fail
|
||||||
|
- logical progression: fail (No question generated)
|
||||||
|
- graph-backed: fail
|
||||||
|
- premature-specialism avoided: fail
|
||||||
|
|
||||||
|
### scenario-5: Operational contradiction
|
||||||
|
|
||||||
|
- **Overall**: Start=pass, Update=fail(400), Rating=flow failure
|
||||||
|
- Pattern: N/A | Nodes: 7 | Edges: 0
|
||||||
|
- Validation: valid | Duration: 70579ms
|
||||||
|
- Unknown IDs: n6gm2cv, nylhu9g
|
||||||
|
- Error: [N/A] Invalid update-case request
|
||||||
|
|
||||||
|
- **Assessment**:
|
||||||
|
- reasoning-pattern fit: fail
|
||||||
|
- one-concept simplicity: fail
|
||||||
|
- plain-language clarity: fail
|
||||||
|
- logical progression: fail (No question generated)
|
||||||
|
- graph-backed: fail
|
||||||
|
- premature-specialism avoided: fail
|
||||||
|
|
||||||
|
### scenario-6: Personal decision
|
||||||
|
|
||||||
|
- **Overall**: Start=fail, Update=skipped, Rating=flow failure
|
||||||
|
- Pattern: N/A | Nodes: 0 | Edges: 0
|
||||||
|
- Validation: invalid | Duration: 72547ms
|
||||||
|
|
||||||
|
- **Assessment**:
|
||||||
|
- reasoning-pattern fit: fail
|
||||||
|
- one-concept simplicity: fail
|
||||||
|
- plain-language clarity: fail
|
||||||
|
- logical progression: fail (No question generated)
|
||||||
|
- graph-backed: fail
|
||||||
|
- premature-specialism avoided: fail
|
||||||
|
|
||||||
|
## Failure Pattern Analysis
|
||||||
|
|
||||||
|
### Start Phase
|
||||||
|
- **5/6 succeeded**, 1/6 failed
|
||||||
|
- scenario-6: Scenario analysis failed
|
||||||
|
|
||||||
|
### Update Phase
|
||||||
|
- **0/6 succeeded**, 5/6 failed, 1/6 skipped
|
||||||
|
|
||||||
|
- **N/A** (5 failures):
|
||||||
|
- scenario-1: Invalid update-case request
|
||||||
|
- scenario-2: Invalid update-case request
|
||||||
|
- scenario-3: Invalid update-case request
|
||||||
|
- scenario-4: Invalid update-case request
|
||||||
|
- scenario-5: Invalid update-case request
|
||||||
|
|
||||||
|
## What's Stable
|
||||||
|
|
||||||
|
- ✅ **Graph construction**: 5/6 start success across all scenario types (commercial, operational, personal, policy)
|
||||||
|
|
||||||
|
## Recommendations
|
||||||
|
|
||||||
|
1. **Fix update failures** (5/6): Primary focus area. Most failures in proposal_compatibility and delta detection.
|
||||||
|
- Monitor reasoning pattern inference reliability across different scenario domains.
|
||||||
|
- Consider adding timeout guards for long-running LLM calls (some exceeded 60s).
|
||||||
@@ -0,0 +1,288 @@
|
|||||||
|
## Post-update selection invariant
|
||||||
|
|
||||||
|
After every successful graph update, the full deterministic question-selection pipeline must run again whenever eligible unresolved unknowns remain.
|
||||||
|
|
||||||
|
The active reasoning pattern constrains which graph nodes may participate in reasoning.
|
||||||
|
|
||||||
|
That means the update path must not stop at graph mutation, child resolution, emergent unknown creation, decomposition, or upward propagation. It must continue through:
|
||||||
|
|
||||||
|
```text
|
||||||
|
updated graph
|
||||||
|
→ rebuild reasoning state
|
||||||
|
→ identify unresolved candidates
|
||||||
|
→ select active unknown
|
||||||
|
→ atomicity assessment
|
||||||
|
→ answerability assessment
|
||||||
|
→ decompose if required
|
||||||
|
→ reselect
|
||||||
|
→ reasoning-pattern selection
|
||||||
|
→ investigation-strategy selection
|
||||||
|
→ question-family selection
|
||||||
|
→ question formulation
|
||||||
|
→ complexity validation
|
||||||
|
→ selectedQuestion
|
||||||
|
```
|
||||||
|
|
||||||
|
Returning no question is only valid when no eligible unresolved candidate remains, the case is complete, ambiguity cannot be safely resolved, or question formulation fails validation with an explicit deterministic reason.
|
||||||
|
|
||||||
|
## Graph validity vs reasoning-pattern validity
|
||||||
|
|
||||||
|
These are separate requirements.
|
||||||
|
|
||||||
|
- **Graph validity** means references, IDs, node shapes, and update semantics are structurally correct.
|
||||||
|
- **Reasoning-pattern validity** means selectable investigation nodes are compatible with the current reasoning mode.
|
||||||
|
|
||||||
|
A graph can be structurally valid while still being reasoning-invalid.
|
||||||
|
|
||||||
|
Example: a decision investigation may still contain an unresolved comparison-style node such as `How the two observations were measured`. That node is structurally well-formed, but it is not allowed to participate as an active investigation target unless the reasoning pattern has actually shifted into comparison, contradiction, or explanation work.
|
||||||
|
|
||||||
|
The engine therefore needs both invariants:
|
||||||
|
|
||||||
|
1. the graph must be structurally valid
|
||||||
|
2. every selectable unknown must be compatible with the active reasoning pattern
|
||||||
|
|
||||||
|
# v0.7 Question Simplicity Experiment
|
||||||
|
|
||||||
|
## Observed failure
|
||||||
|
|
||||||
|
The first v0.7 UI scenario exposed a reasoning failure where the selected unknown could still be directionally correct while the resulting question was too large to answer in one coherent response.
|
||||||
|
|
||||||
|
Example failure:
|
||||||
|
|
||||||
|
> Have you measured the current financial or operational cost to users who lack justified confidence, and what baseline budget do they currently allocate for comparable decision-support methods?
|
||||||
|
|
||||||
|
This question bundled multiple investigations:
|
||||||
|
|
||||||
|
- cost
|
||||||
|
- user impact
|
||||||
|
- existing alternatives
|
||||||
|
- current budget
|
||||||
|
|
||||||
|
That violated the intended one-step reasoning discipline.
|
||||||
|
|
||||||
|
## Principle
|
||||||
|
|
||||||
|
**A correct unknown paired with an unanswerably broad question is still a reasoning failure.**
|
||||||
|
|
||||||
|
The engine should ask one question about one primary concept at a time.
|
||||||
|
|
||||||
|
**The reconstruction model may suggest a question, but only the graph-backed deterministic pipeline may select the user-facing question.**
|
||||||
|
|
||||||
|
**A node is only questionable if it is independently answerable.**
|
||||||
|
|
||||||
|
## One-question / one-concept rule
|
||||||
|
|
||||||
|
Every user-facing question should:
|
||||||
|
|
||||||
|
- contain one question mark
|
||||||
|
- target one unresolved graph node
|
||||||
|
- ask for one primary concept
|
||||||
|
- request one coherent answer
|
||||||
|
- avoid joined investigations
|
||||||
|
- minimise cognitive effort while still reducing meaningful uncertainty
|
||||||
|
|
||||||
|
## Deterministic cognitive-load rules
|
||||||
|
|
||||||
|
The new deterministic question-complexity assessment marks a question as too broad when it shows signals such as:
|
||||||
|
|
||||||
|
- multiple requested answers joined by `and`
|
||||||
|
- distinct measures combined in one prompt, such as cost plus budget
|
||||||
|
- comma-list phrasing that expands the request into several sub-questions
|
||||||
|
- more than one primary concept
|
||||||
|
- abstract noun chains that make the question hard to parse on first reading
|
||||||
|
- very long question length
|
||||||
|
|
||||||
|
The assessment returns:
|
||||||
|
|
||||||
|
- `acceptable`
|
||||||
|
- `primaryConceptCount`
|
||||||
|
- `compoundQuestionSignals`
|
||||||
|
- `abstractTermCount`
|
||||||
|
- `cognitiveLoad`
|
||||||
|
- `reasons`
|
||||||
|
|
||||||
|
## Decomposition-before-rewording rule
|
||||||
|
|
||||||
|
The engine now treats broad commercial-validation unknowns as composite.
|
||||||
|
|
||||||
|
If the selected unknown still spans multiple validation dimensions, the system should not simply shorten the sentence. It should first decompose the unknown into smaller child unknowns and then select one foundational child.
|
||||||
|
|
||||||
|
For the current scenario, this meant creating child unknowns such as:
|
||||||
|
|
||||||
|
- who experiences the problem
|
||||||
|
- what happens when it is not resolved
|
||||||
|
- how often it happens
|
||||||
|
- how people deal with it today
|
||||||
|
- whether people actively look for help
|
||||||
|
|
||||||
|
The selector then reaches the first foundational child through prerequisite ordering encoded in the decomposition graph rather than through global scoring changes.
|
||||||
|
|
||||||
|
## Atomicity vs answerability
|
||||||
|
|
||||||
|
These are different reasoning properties.
|
||||||
|
|
||||||
|
- **Atomicity** asks: does this node describe one investigation or several bundled investigations?
|
||||||
|
- **Answerability** asks: even if the wording looks singular, can this node be answered directly without first resolving multiple prerequisite dimensions?
|
||||||
|
|
||||||
|
A node can appear atomic in wording but still fail answerability.
|
||||||
|
|
||||||
|
Examples include broad evaluation containers such as product validation, customer value, business case, technical feasibility, or commercial justification. These often compress several prerequisite investigations into one conclusion-shaped unknown.
|
||||||
|
|
||||||
|
That means atomicity alone is not enough.
|
||||||
|
|
||||||
|
The engine now decomposes whenever either of these is true:
|
||||||
|
|
||||||
|
- the unknown is not atomic
|
||||||
|
- the unknown is not independently answerable
|
||||||
|
|
||||||
|
This prevents a broad container node from becoming the selected question target even when its wording looks grammatically singular.
|
||||||
|
|
||||||
|
## Reasoning Pattern
|
||||||
|
|
||||||
|
The next failure exposed a deeper issue: even after atomicity and answerability were added, the engine could still choose a question template from the wrong reasoning family.
|
||||||
|
|
||||||
|
The live failure was an explanation-style prompt appearing in a commercial validation scenario:
|
||||||
|
|
||||||
|
> What changed during the period that could help explain why ...
|
||||||
|
|
||||||
|
That was wrong not because of wording, but because the engine had selected an **explanation family** when the actual task was a **decision investigation**.
|
||||||
|
|
||||||
|
To correct that, the deterministic pipeline now explicitly inserts a reasoning-pattern stage:
|
||||||
|
|
||||||
|
```text
|
||||||
|
selected unknown
|
||||||
|
→ atomicity
|
||||||
|
→ answerability
|
||||||
|
→ reasoning pattern
|
||||||
|
→ investigation strategy
|
||||||
|
→ question family
|
||||||
|
→ question
|
||||||
|
```
|
||||||
|
|
||||||
|
This matters because each stage must constrain the next.
|
||||||
|
|
||||||
|
- **Reasoning Pattern** decides what kind of reasoning is happening
|
||||||
|
- **Investigation Strategy** decides how to reduce uncertainty within that pattern
|
||||||
|
- **Question Family** decides what template space is allowed
|
||||||
|
- **Question** is the final concrete wording
|
||||||
|
|
||||||
|
Without this stage separation, strategy and template selection can leak across domains and reuse relationship/explanation prompts too broadly.
|
||||||
|
|
||||||
|
## Deterministic reasoning-pattern vocabulary
|
||||||
|
|
||||||
|
The current deterministic pattern vocabulary is intentionally small:
|
||||||
|
|
||||||
|
- decision
|
||||||
|
- explanation
|
||||||
|
- contradiction
|
||||||
|
- definition
|
||||||
|
- diagnosis
|
||||||
|
- comparison
|
||||||
|
- prioritisation
|
||||||
|
|
||||||
|
Pattern selection uses graph structure rather than wording alone, including:
|
||||||
|
|
||||||
|
- node kind
|
||||||
|
- relationship / observation topology
|
||||||
|
- parent context
|
||||||
|
- reasoning state
|
||||||
|
- selected unknown role in the graph
|
||||||
|
|
||||||
|
## Question-family mapping
|
||||||
|
|
||||||
|
Patterns now constrain which question families are allowed.
|
||||||
|
|
||||||
|
- **decision**
|
||||||
|
- decision_foundation
|
||||||
|
- decision_evidence
|
||||||
|
- decision_threshold
|
||||||
|
- definition
|
||||||
|
- **explanation**
|
||||||
|
- explanation
|
||||||
|
- comparison
|
||||||
|
- **contradiction**
|
||||||
|
- contradiction
|
||||||
|
- comparison
|
||||||
|
- explanation
|
||||||
|
- **definition**
|
||||||
|
- definition
|
||||||
|
- **diagnosis**
|
||||||
|
- diagnosis
|
||||||
|
- comparison
|
||||||
|
- **comparison**
|
||||||
|
- comparison
|
||||||
|
- **prioritisation**
|
||||||
|
- prioritisation
|
||||||
|
- decision_threshold
|
||||||
|
|
||||||
|
Most importantly:
|
||||||
|
|
||||||
|
- explanation templates are only allowed for `explanation` or `contradiction`
|
||||||
|
- decision investigations cannot emit explanation-family questions
|
||||||
|
|
||||||
|
## Live correction
|
||||||
|
|
||||||
|
For the commercial-method scenario, the engine now classifies the reasoning as a **decision** pattern rather than an explanation pattern.
|
||||||
|
|
||||||
|
That means explanation-family templates are explicitly rejected, and the selected child unknown must be questioned using a decision-compatible family instead.
|
||||||
|
|
||||||
|
## UI result
|
||||||
|
|
||||||
|
The long compound question no longer survives as the first follow-up in the tested path.
|
||||||
|
|
||||||
|
The new first-step question is:
|
||||||
|
|
||||||
|
> Who experiences this problem?
|
||||||
|
|
||||||
|
This question:
|
||||||
|
|
||||||
|
- asks one thing
|
||||||
|
- is understandable immediately
|
||||||
|
- stays graph-backed
|
||||||
|
- avoids pricing or budget before problem existence is established
|
||||||
|
|
||||||
|
## Start-case authority rule
|
||||||
|
|
||||||
|
There were previously two question paths during initial analysis:
|
||||||
|
|
||||||
|
- reconstruction model `nextQuestion`
|
||||||
|
- graph-backed unknown selection and question formulation
|
||||||
|
|
||||||
|
The defect was that `startCase` copied the reconstruction `nextQuestion` directly into the normal UI.
|
||||||
|
|
||||||
|
That path is now closed.
|
||||||
|
|
||||||
|
Initial user-facing questioning now follows this pipeline:
|
||||||
|
|
||||||
|
```text
|
||||||
|
reconstruction
|
||||||
|
→ graph build
|
||||||
|
→ unresolved unknown selection
|
||||||
|
→ atomicity assessment
|
||||||
|
→ decomposition if needed
|
||||||
|
→ investigation strategy
|
||||||
|
→ question formulation
|
||||||
|
→ complexity validation
|
||||||
|
→ selectedQuestion
|
||||||
|
```
|
||||||
|
|
||||||
|
The reconstruction question is still retained in diagnostics as provenance, but it is not authoritative.
|
||||||
|
|
||||||
|
## Live result
|
||||||
|
|
||||||
|
Running the commercial-method scenario through the real environment now:
|
||||||
|
|
||||||
|
- succeeds without the enum compatibility failure
|
||||||
|
- does not show the broad reconstruction question in the UI path
|
||||||
|
- surfaces a graph-backed first question instead
|
||||||
|
- keeps the reconstruction question only in diagnostics
|
||||||
|
|
||||||
|
For the tested scenario, the user-facing first question remained:
|
||||||
|
|
||||||
|
> Who experiences this problem?
|
||||||
|
|
||||||
|
## Remaining limitations
|
||||||
|
|
||||||
|
- question-complexity assessment is still conservative and pattern-based rather than semantic in a richer linguistic sense
|
||||||
|
- plain-language simplification currently uses a small deterministic replacement set
|
||||||
|
- broader prerequisite ordering is strongest for decomposition structures that explicitly encode those dependencies
|
||||||
+1352
-16
File diff suppressed because it is too large
Load Diff
+350
-25
@@ -13,10 +13,14 @@ import {
|
|||||||
updateCaseRequestSchema,
|
updateCaseRequestSchema,
|
||||||
} from "./schema.js";
|
} from "./schema.js";
|
||||||
import { buildInitialGraph, describeGraph } from "./builder.js";
|
import { buildInitialGraph, describeGraph } from "./builder.js";
|
||||||
import { applyValidatedProposal } from "./apply-proposal.js";
|
import {
|
||||||
|
applyValidatedProposal,
|
||||||
|
determineGraphBackedQuestion,
|
||||||
|
} from "./apply-proposal.js";
|
||||||
import { buildGraphUpdatePrompt } from "./prompt-builder.js";
|
import { buildGraphUpdatePrompt } from "./prompt-builder.js";
|
||||||
import {
|
import {
|
||||||
buildReasoningState,
|
buildReasoningState,
|
||||||
|
formulateQuestion,
|
||||||
formulateTieResolutionQuestion,
|
formulateTieResolutionQuestion,
|
||||||
} from "./question-formulator.js";
|
} from "./question-formulator.js";
|
||||||
import { parseGraphUpdateProposal } from "./update-proposal.js";
|
import { parseGraphUpdateProposal } from "./update-proposal.js";
|
||||||
@@ -41,6 +45,33 @@ function buildDiagnostics({
|
|||||||
graph,
|
graph,
|
||||||
graphReferenceValidation,
|
graphReferenceValidation,
|
||||||
unknownSelectionExplanation,
|
unknownSelectionExplanation,
|
||||||
|
reconstructionQuestion,
|
||||||
|
reconstructionQuestionAccepted,
|
||||||
|
reconstructionQuestionRejectionReasons,
|
||||||
|
finalGraphBackedQuestion,
|
||||||
|
selectedUnknownNodeId,
|
||||||
|
decompositionApplied,
|
||||||
|
questionComplexityAssessment,
|
||||||
|
answerabilityAssessment,
|
||||||
|
independentlyAnswerable,
|
||||||
|
prerequisiteConceptCount,
|
||||||
|
decompositionTriggeredByAnswerability,
|
||||||
|
decompositionReason,
|
||||||
|
selectedContainerUnknown,
|
||||||
|
selectedChildUnknown,
|
||||||
|
reasoningPattern,
|
||||||
|
questionFamily,
|
||||||
|
allowedQuestionFamilies,
|
||||||
|
rejectedQuestionFamilies,
|
||||||
|
selectedQuestionTemplate,
|
||||||
|
reasoningPatternReason,
|
||||||
|
reasoningPatternValidation,
|
||||||
|
patternCompatibleNodeCount,
|
||||||
|
incompatibleNodeIds,
|
||||||
|
compatibilityFailures,
|
||||||
|
replacementActions,
|
||||||
|
graphReasoningIntegrity,
|
||||||
|
noQuestionReason,
|
||||||
}) {
|
}) {
|
||||||
return {
|
return {
|
||||||
promptVersion: analysis?.promptVersion ?? null,
|
promptVersion: analysis?.promptVersion ?? null,
|
||||||
@@ -54,6 +85,56 @@ function buildDiagnostics({
|
|||||||
compatibilityChanges: analysis?.compatibilityChanges ?? [],
|
compatibilityChanges: analysis?.compatibilityChanges ?? [],
|
||||||
compatibilityWarnings: analysis?.compatibilityWarnings ?? [],
|
compatibilityWarnings: analysis?.compatibilityWarnings ?? [],
|
||||||
unknownSelectionExplanation: unknownSelectionExplanation ?? null,
|
unknownSelectionExplanation: unknownSelectionExplanation ?? null,
|
||||||
|
reconstructionQuestion: reconstructionQuestion ?? null,
|
||||||
|
reconstructionQuestionAccepted: reconstructionQuestionAccepted ?? null,
|
||||||
|
reconstructionQuestionRejectionReasons:
|
||||||
|
reconstructionQuestionRejectionReasons ?? [],
|
||||||
|
finalGraphBackedQuestion: finalGraphBackedQuestion ?? null,
|
||||||
|
selectedUnknownNodeId: selectedUnknownNodeId ?? null,
|
||||||
|
decompositionApplied: decompositionApplied ?? false,
|
||||||
|
questionComplexityAssessment: questionComplexityAssessment ?? null,
|
||||||
|
answerabilityAssessment: answerabilityAssessment ?? null,
|
||||||
|
independentlyAnswerable: independentlyAnswerable ?? null,
|
||||||
|
prerequisiteConceptCount: prerequisiteConceptCount ?? null,
|
||||||
|
decompositionTriggeredByAnswerability:
|
||||||
|
decompositionTriggeredByAnswerability ?? false,
|
||||||
|
decompositionReason: decompositionReason ?? null,
|
||||||
|
selectedContainerUnknown: selectedContainerUnknown ?? null,
|
||||||
|
selectedChildUnknown: selectedChildUnknown ?? null,
|
||||||
|
reasoningPattern: reasoningPattern ?? null,
|
||||||
|
questionFamily: questionFamily ?? null,
|
||||||
|
allowedQuestionFamilies: allowedQuestionFamilies ?? [],
|
||||||
|
rejectedQuestionFamilies: rejectedQuestionFamilies ?? [],
|
||||||
|
selectedQuestionTemplate: selectedQuestionTemplate ?? null,
|
||||||
|
reasoningPatternReason: reasoningPatternReason ?? null,
|
||||||
|
reasoningPatternValidation: reasoningPatternValidation ?? null,
|
||||||
|
patternCompatibleNodeCount: patternCompatibleNodeCount ?? 0,
|
||||||
|
incompatibleNodeIds: incompatibleNodeIds ?? [],
|
||||||
|
compatibilityFailures: compatibilityFailures ?? [],
|
||||||
|
replacementActions: replacementActions ?? [],
|
||||||
|
graphReasoningIntegrity: graphReasoningIntegrity ?? null,
|
||||||
|
noQuestionReason: noQuestionReason ?? null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function fallbackStartCaseReasoningPatternValidation(
|
||||||
|
selectedQuestion,
|
||||||
|
existingValidation,
|
||||||
|
) {
|
||||||
|
if (existingValidation) {
|
||||||
|
return existingValidation;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!selectedQuestion?.reasoningPattern) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
activePattern: selectedQuestion.reasoningPattern,
|
||||||
|
valid: Boolean(selectedQuestion.question),
|
||||||
|
reason: selectedQuestion.question
|
||||||
|
? "Initial graph-backed selection produced a reasoning-pattern-compatible question."
|
||||||
|
: "Initial graph-backed selection did not produce a valid question for the inferred reasoning pattern.",
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -133,6 +214,33 @@ function buildUpdateDiagnostics({
|
|||||||
childUnknownCount,
|
childUnknownCount,
|
||||||
childNodeIds,
|
childNodeIds,
|
||||||
atomicityReason,
|
atomicityReason,
|
||||||
|
questionComplexityAccepted,
|
||||||
|
primaryConceptCount,
|
||||||
|
cognitiveLoad,
|
||||||
|
complexityReasons,
|
||||||
|
decompositionTriggeredByQuestionComplexity,
|
||||||
|
previousQuestion,
|
||||||
|
finalQuestion,
|
||||||
|
selectedUnknownBefore,
|
||||||
|
selectedUnknownAfter,
|
||||||
|
plainLanguageNormalisations,
|
||||||
|
reasoningPattern,
|
||||||
|
questionFamily,
|
||||||
|
allowedQuestionFamilies,
|
||||||
|
rejectedQuestionFamilies,
|
||||||
|
selectedQuestionTemplate,
|
||||||
|
reasoningPatternReason,
|
||||||
|
unresolvedCandidateCount,
|
||||||
|
eligibleCandidateCount,
|
||||||
|
candidateNodeIds,
|
||||||
|
resolvedCurrentTurnNodeIds,
|
||||||
|
noQuestionReason,
|
||||||
|
reasoningPatternValidation,
|
||||||
|
patternCompatibleNodeCount,
|
||||||
|
incompatibleNodeIds,
|
||||||
|
compatibilityFailures,
|
||||||
|
replacementActions,
|
||||||
|
graphReasoningIntegrity,
|
||||||
}) {
|
}) {
|
||||||
return {
|
return {
|
||||||
promptVersion: promptVersion ?? "v0.4",
|
promptVersion: promptVersion ?? "v0.4",
|
||||||
@@ -202,6 +310,34 @@ function buildUpdateDiagnostics({
|
|||||||
childUnknownCount: childUnknownCount ?? 0,
|
childUnknownCount: childUnknownCount ?? 0,
|
||||||
childNodeIds: childNodeIds ?? [],
|
childNodeIds: childNodeIds ?? [],
|
||||||
atomicityReason: atomicityReason ?? null,
|
atomicityReason: atomicityReason ?? null,
|
||||||
|
questionComplexityAccepted: questionComplexityAccepted ?? null,
|
||||||
|
primaryConceptCount: primaryConceptCount ?? null,
|
||||||
|
cognitiveLoad: cognitiveLoad ?? null,
|
||||||
|
complexityReasons: complexityReasons ?? [],
|
||||||
|
decompositionTriggeredByQuestionComplexity:
|
||||||
|
decompositionTriggeredByQuestionComplexity ?? false,
|
||||||
|
previousQuestion: previousQuestion ?? null,
|
||||||
|
finalQuestion: finalQuestion ?? null,
|
||||||
|
selectedUnknownBefore: selectedUnknownBefore ?? null,
|
||||||
|
selectedUnknownAfter: selectedUnknownAfter ?? null,
|
||||||
|
plainLanguageNormalisations: plainLanguageNormalisations ?? [],
|
||||||
|
reasoningPattern: reasoningPattern ?? null,
|
||||||
|
questionFamily: questionFamily ?? null,
|
||||||
|
allowedQuestionFamilies: allowedQuestionFamilies ?? [],
|
||||||
|
rejectedQuestionFamilies: rejectedQuestionFamilies ?? [],
|
||||||
|
selectedQuestionTemplate: selectedQuestionTemplate ?? null,
|
||||||
|
reasoningPatternReason: reasoningPatternReason ?? null,
|
||||||
|
unresolvedCandidateCount: unresolvedCandidateCount ?? 0,
|
||||||
|
eligibleCandidateCount: eligibleCandidateCount ?? 0,
|
||||||
|
candidateNodeIds: candidateNodeIds ?? [],
|
||||||
|
resolvedCurrentTurnNodeIds: resolvedCurrentTurnNodeIds ?? [],
|
||||||
|
noQuestionReason: noQuestionReason ?? null,
|
||||||
|
reasoningPatternValidation: reasoningPatternValidation ?? null,
|
||||||
|
patternCompatibleNodeCount: patternCompatibleNodeCount ?? 0,
|
||||||
|
incompatibleNodeIds: incompatibleNodeIds ?? [],
|
||||||
|
compatibilityFailures: compatibilityFailures ?? [],
|
||||||
|
replacementActions: replacementActions ?? [],
|
||||||
|
graphReasoningIntegrity: graphReasoningIntegrity ?? null,
|
||||||
unknownSelectionExplanation: unknownSelectionExplanation ?? null,
|
unknownSelectionExplanation: unknownSelectionExplanation ?? null,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -242,23 +378,11 @@ export async function startCase(body) {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const currentSummary = describeGraph(initialGraph);
|
const currentSummary = describeGraph(initialGraph);
|
||||||
const deterministicSelection = selectActiveUnknownCandidate(
|
const initialSituationGraph = makeGraph({
|
||||||
{
|
|
||||||
...initialGraph,
|
|
||||||
resolvedNodeIds: [],
|
|
||||||
},
|
|
||||||
[],
|
|
||||||
);
|
|
||||||
const activeUnknownNodeId =
|
|
||||||
deterministicSelection?.status === "selected"
|
|
||||||
? deterministicSelection.nodeId
|
|
||||||
: null;
|
|
||||||
|
|
||||||
const situationGraph = makeGraph({
|
|
||||||
centralStatement: scenario,
|
centralStatement: scenario,
|
||||||
nodes: initialGraph.nodes,
|
nodes: initialGraph.nodes,
|
||||||
edges: initialGraph.edges,
|
edges: initialGraph.edges,
|
||||||
activeUnknownNodeId,
|
activeUnknownNodeId: null,
|
||||||
resolvedNodeIds: [],
|
resolvedNodeIds: [],
|
||||||
currentSummary,
|
currentSummary,
|
||||||
reasoningState: buildReasoningState({
|
reasoningState: buildReasoningState({
|
||||||
@@ -269,17 +393,20 @@ export async function startCase(body) {
|
|||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
|
|
||||||
situationGraphSchema.parse(situationGraph);
|
situationGraphSchema.parse(initialSituationGraph);
|
||||||
|
|
||||||
const graphReferenceValidation = validateGraphReferences(situationGraph);
|
const graphReferenceValidation = validateGraphReferences(
|
||||||
const selectedQuestion =
|
initialSituationGraph,
|
||||||
deterministicSelection?.status === "ambiguous"
|
);
|
||||||
? {
|
const initialQuestionResult = determineGraphBackedQuestion({
|
||||||
id: "q_tie_resolution",
|
situationGraph: initialSituationGraph,
|
||||||
...formulateTieResolutionQuestion({ graph: situationGraph }),
|
});
|
||||||
tiedCandidateIds: deterministicSelection.tiedCandidateIds,
|
const situationGraph = initialQuestionResult.success
|
||||||
}
|
? initialQuestionResult.updatedSituationGraph
|
||||||
: (analysis.nextQuestion ?? null);
|
: initialSituationGraph;
|
||||||
|
const selectedQuestion = initialQuestionResult.success
|
||||||
|
? initialQuestionResult.selectedQuestion
|
||||||
|
: null;
|
||||||
const unknownSelectionExplanation = buildUnknownSelectionDiagnostics(
|
const unknownSelectionExplanation = buildUnknownSelectionDiagnostics(
|
||||||
situationGraph,
|
situationGraph,
|
||||||
[],
|
[],
|
||||||
@@ -294,6 +421,64 @@ export async function startCase(body) {
|
|||||||
graph: situationGraph,
|
graph: situationGraph,
|
||||||
graphReferenceValidation,
|
graphReferenceValidation,
|
||||||
unknownSelectionExplanation,
|
unknownSelectionExplanation,
|
||||||
|
reconstructionQuestion: analysis.nextQuestion?.question ?? null,
|
||||||
|
reconstructionQuestionAccepted: false,
|
||||||
|
reconstructionQuestionRejectionReasons:
|
||||||
|
analysis.nextQuestion?.question != null
|
||||||
|
? [
|
||||||
|
"reconstruction_question_not_authoritative",
|
||||||
|
"graph_backed_pipeline_required",
|
||||||
|
]
|
||||||
|
: [],
|
||||||
|
finalGraphBackedQuestion: selectedQuestion?.question ?? null,
|
||||||
|
selectedUnknownNodeId:
|
||||||
|
initialQuestionResult.selectedUnknownAfter ?? null,
|
||||||
|
decompositionApplied:
|
||||||
|
initialQuestionResult.decompositionPerformed ?? false,
|
||||||
|
questionComplexityAssessment:
|
||||||
|
initialQuestionResult.questionComplexityAssessment ?? null,
|
||||||
|
answerabilityAssessment:
|
||||||
|
initialQuestionResult.answerabilityAssessment ?? null,
|
||||||
|
independentlyAnswerable:
|
||||||
|
initialQuestionResult.independentlyAnswerable ?? null,
|
||||||
|
prerequisiteConceptCount:
|
||||||
|
initialQuestionResult.prerequisiteConceptCount ?? null,
|
||||||
|
decompositionTriggeredByAnswerability:
|
||||||
|
initialQuestionResult.decompositionTriggeredByAnswerability ?? false,
|
||||||
|
decompositionReason:
|
||||||
|
initialQuestionResult.selectedQuestion?.reason ?? null,
|
||||||
|
selectedContainerUnknown:
|
||||||
|
initialQuestionResult.selectedContainerUnknown ?? null,
|
||||||
|
selectedChildUnknown:
|
||||||
|
initialQuestionResult.selectedChildUnknown ?? null,
|
||||||
|
reasoningPattern:
|
||||||
|
initialQuestionResult.selectedQuestion?.reasoningPattern ?? null,
|
||||||
|
questionFamily:
|
||||||
|
initialQuestionResult.selectedQuestion?.questionFamily ?? null,
|
||||||
|
allowedQuestionFamilies:
|
||||||
|
initialQuestionResult.selectedQuestion?.allowedQuestionFamilies ?? [],
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
initialQuestionResult.selectedQuestion?.rejectedQuestionFamilies ??
|
||||||
|
[],
|
||||||
|
selectedQuestionTemplate:
|
||||||
|
initialQuestionResult.selectedQuestion?.selectedQuestionTemplate ??
|
||||||
|
null,
|
||||||
|
reasoningPatternReason:
|
||||||
|
initialQuestionResult.selectedQuestion?.reasoningPatternReason ??
|
||||||
|
null,
|
||||||
|
reasoningPatternValidation: fallbackStartCaseReasoningPatternValidation(
|
||||||
|
initialQuestionResult.selectedQuestion,
|
||||||
|
initialQuestionResult.reasoningPatternValidation,
|
||||||
|
),
|
||||||
|
patternCompatibleNodeCount:
|
||||||
|
initialQuestionResult.patternCompatibleNodeCount ?? 0,
|
||||||
|
incompatibleNodeIds: initialQuestionResult.incompatibleNodeIds ?? [],
|
||||||
|
compatibilityFailures:
|
||||||
|
initialQuestionResult.compatibilityFailures ?? [],
|
||||||
|
replacementActions: initialQuestionResult.replacementActions ?? [],
|
||||||
|
graphReasoningIntegrity:
|
||||||
|
initialQuestionResult.graphReasoningIntegrity ?? null,
|
||||||
|
noQuestionReason: initialQuestionResult.noQuestionReason ?? null,
|
||||||
}),
|
}),
|
||||||
validationErrors: graphReferenceValidation.errors,
|
validationErrors: graphReferenceValidation.errors,
|
||||||
statusCode: 500,
|
statusCode: 500,
|
||||||
@@ -309,6 +494,59 @@ export async function startCase(body) {
|
|||||||
graph: situationGraph,
|
graph: situationGraph,
|
||||||
graphReferenceValidation,
|
graphReferenceValidation,
|
||||||
unknownSelectionExplanation,
|
unknownSelectionExplanation,
|
||||||
|
reconstructionQuestion: analysis.nextQuestion?.question ?? null,
|
||||||
|
reconstructionQuestionAccepted: false,
|
||||||
|
reconstructionQuestionRejectionReasons:
|
||||||
|
analysis.nextQuestion?.question != null
|
||||||
|
? [
|
||||||
|
"reconstruction_question_not_authoritative",
|
||||||
|
"graph_backed_pipeline_required",
|
||||||
|
]
|
||||||
|
: [],
|
||||||
|
finalGraphBackedQuestion: selectedQuestion?.question ?? null,
|
||||||
|
selectedUnknownNodeId: initialQuestionResult.selectedUnknownAfter ?? null,
|
||||||
|
decompositionApplied:
|
||||||
|
initialQuestionResult.decompositionPerformed ?? false,
|
||||||
|
questionComplexityAssessment:
|
||||||
|
initialQuestionResult.questionComplexityAssessment ?? null,
|
||||||
|
answerabilityAssessment:
|
||||||
|
initialQuestionResult.answerabilityAssessment ?? null,
|
||||||
|
independentlyAnswerable:
|
||||||
|
initialQuestionResult.independentlyAnswerable ?? null,
|
||||||
|
prerequisiteConceptCount:
|
||||||
|
initialQuestionResult.prerequisiteConceptCount ?? null,
|
||||||
|
decompositionTriggeredByAnswerability:
|
||||||
|
initialQuestionResult.decompositionTriggeredByAnswerability ?? false,
|
||||||
|
decompositionReason:
|
||||||
|
initialQuestionResult.selectedQuestion?.reason ?? null,
|
||||||
|
selectedContainerUnknown:
|
||||||
|
initialQuestionResult.selectedContainerUnknown ?? null,
|
||||||
|
selectedChildUnknown: initialQuestionResult.selectedChildUnknown ?? null,
|
||||||
|
reasoningPattern:
|
||||||
|
initialQuestionResult.selectedQuestion?.reasoningPattern ?? null,
|
||||||
|
questionFamily:
|
||||||
|
initialQuestionResult.selectedQuestion?.questionFamily ?? null,
|
||||||
|
allowedQuestionFamilies:
|
||||||
|
initialQuestionResult.selectedQuestion?.allowedQuestionFamilies ?? [],
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
initialQuestionResult.selectedQuestion?.rejectedQuestionFamilies ?? [],
|
||||||
|
selectedQuestionTemplate:
|
||||||
|
initialQuestionResult.selectedQuestion?.selectedQuestionTemplate ??
|
||||||
|
null,
|
||||||
|
reasoningPatternReason:
|
||||||
|
initialQuestionResult.selectedQuestion?.reasoningPatternReason ?? null,
|
||||||
|
reasoningPatternValidation: fallbackStartCaseReasoningPatternValidation(
|
||||||
|
initialQuestionResult.selectedQuestion,
|
||||||
|
initialQuestionResult.reasoningPatternValidation,
|
||||||
|
),
|
||||||
|
patternCompatibleNodeCount:
|
||||||
|
initialQuestionResult.patternCompatibleNodeCount ?? 0,
|
||||||
|
incompatibleNodeIds: initialQuestionResult.incompatibleNodeIds ?? [],
|
||||||
|
compatibilityFailures: initialQuestionResult.compatibilityFailures ?? [],
|
||||||
|
replacementActions: initialQuestionResult.replacementActions ?? [],
|
||||||
|
graphReasoningIntegrity:
|
||||||
|
initialQuestionResult.graphReasoningIntegrity ?? null,
|
||||||
|
noQuestionReason: initialQuestionResult.noQuestionReason ?? null,
|
||||||
}),
|
}),
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
@@ -495,6 +733,27 @@ async function updateCaseWithDependencies(body, dependencies = {}) {
|
|||||||
childUnknownCount: 0,
|
childUnknownCount: 0,
|
||||||
childNodeIds: [],
|
childNodeIds: [],
|
||||||
atomicityReason: null,
|
atomicityReason: null,
|
||||||
|
questionComplexityAccepted: null,
|
||||||
|
primaryConceptCount: null,
|
||||||
|
cognitiveLoad: null,
|
||||||
|
complexityReasons: [],
|
||||||
|
decompositionTriggeredByQuestionComplexity: false,
|
||||||
|
previousQuestion,
|
||||||
|
finalQuestion: null,
|
||||||
|
selectedUnknownBefore: null,
|
||||||
|
selectedUnknownAfter: null,
|
||||||
|
unresolvedCandidateCount: 0,
|
||||||
|
eligibleCandidateCount: 0,
|
||||||
|
candidateNodeIds: [],
|
||||||
|
resolvedCurrentTurnNodeIds: [],
|
||||||
|
noQuestionReason: null,
|
||||||
|
reasoningPatternValidation: null,
|
||||||
|
patternCompatibleNodeCount: 0,
|
||||||
|
incompatibleNodeIds: [],
|
||||||
|
compatibilityFailures: [],
|
||||||
|
replacementActions: [],
|
||||||
|
graphReasoningIntegrity: null,
|
||||||
|
plainLanguageNormalisations: [],
|
||||||
unknownSelectionExplanation: explainUnknownSelection(
|
unknownSelectionExplanation: explainUnknownSelection(
|
||||||
situationGraph,
|
situationGraph,
|
||||||
situationGraph.resolvedNodeIds || [],
|
situationGraph.resolvedNodeIds || [],
|
||||||
@@ -582,6 +841,45 @@ async function updateCaseWithDependencies(body, dependencies = {}) {
|
|||||||
childUnknownCount: applicationResult.childUnknownCount,
|
childUnknownCount: applicationResult.childUnknownCount,
|
||||||
childNodeIds: applicationResult.childNodeIds,
|
childNodeIds: applicationResult.childNodeIds,
|
||||||
atomicityReason: applicationResult.atomicityReason,
|
atomicityReason: applicationResult.atomicityReason,
|
||||||
|
questionComplexityAccepted:
|
||||||
|
applicationResult.questionComplexityAccepted,
|
||||||
|
primaryConceptCount: applicationResult.primaryConceptCount,
|
||||||
|
cognitiveLoad: applicationResult.cognitiveLoad,
|
||||||
|
complexityReasons: applicationResult.complexityReasons,
|
||||||
|
decompositionTriggeredByQuestionComplexity:
|
||||||
|
applicationResult.decompositionTriggeredByQuestionComplexity,
|
||||||
|
previousQuestion: applicationResult.previousQuestion,
|
||||||
|
finalQuestion: applicationResult.finalQuestion,
|
||||||
|
selectedUnknownBefore: applicationResult.selectedUnknownBefore,
|
||||||
|
selectedUnknownAfter: applicationResult.selectedUnknownAfter,
|
||||||
|
unresolvedCandidateCount: applicationResult.unresolvedCandidateCount,
|
||||||
|
eligibleCandidateCount: applicationResult.eligibleCandidateCount,
|
||||||
|
candidateNodeIds: applicationResult.candidateNodeIds,
|
||||||
|
resolvedCurrentTurnNodeIds:
|
||||||
|
applicationResult.resolvedCurrentTurnNodeIds,
|
||||||
|
noQuestionReason: applicationResult.noQuestionReason,
|
||||||
|
reasoningPatternValidation:
|
||||||
|
applicationResult.reasoningPatternValidation,
|
||||||
|
patternCompatibleNodeCount:
|
||||||
|
applicationResult.patternCompatibleNodeCount,
|
||||||
|
incompatibleNodeIds: applicationResult.incompatibleNodeIds,
|
||||||
|
compatibilityFailures: applicationResult.compatibilityFailures,
|
||||||
|
replacementActions: applicationResult.replacementActions,
|
||||||
|
graphReasoningIntegrity: applicationResult.graphReasoningIntegrity,
|
||||||
|
plainLanguageNormalisations:
|
||||||
|
applicationResult.plainLanguageNormalisations,
|
||||||
|
reasoningPattern:
|
||||||
|
applicationResult.selectedQuestion?.reasoningPattern ?? null,
|
||||||
|
questionFamily:
|
||||||
|
applicationResult.selectedQuestion?.questionFamily ?? null,
|
||||||
|
allowedQuestionFamilies:
|
||||||
|
applicationResult.selectedQuestion?.allowedQuestionFamilies ?? [],
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
applicationResult.selectedQuestion?.rejectedQuestionFamilies ?? [],
|
||||||
|
selectedQuestionTemplate:
|
||||||
|
applicationResult.selectedQuestion?.selectedQuestionTemplate ?? null,
|
||||||
|
reasoningPatternReason:
|
||||||
|
applicationResult.selectedQuestion?.reasoningPatternReason ?? null,
|
||||||
unknownSelectionExplanation: buildUnknownSelectionDiagnostics(
|
unknownSelectionExplanation: buildUnknownSelectionDiagnostics(
|
||||||
applicationResult.updatedSituationGraph,
|
applicationResult.updatedSituationGraph,
|
||||||
applicationResult.updatedSituationGraph.resolvedNodeIds || [],
|
applicationResult.updatedSituationGraph.resolvedNodeIds || [],
|
||||||
@@ -650,6 +948,33 @@ async function updateCaseWithDependencies(body, dependencies = {}) {
|
|||||||
childUnknownCount: 0,
|
childUnknownCount: 0,
|
||||||
childNodeIds: [],
|
childNodeIds: [],
|
||||||
atomicityReason: null,
|
atomicityReason: null,
|
||||||
|
questionComplexityAccepted: null,
|
||||||
|
primaryConceptCount: null,
|
||||||
|
cognitiveLoad: null,
|
||||||
|
complexityReasons: [],
|
||||||
|
decompositionTriggeredByQuestionComplexity: false,
|
||||||
|
previousQuestion,
|
||||||
|
finalQuestion: null,
|
||||||
|
selectedUnknownBefore: null,
|
||||||
|
selectedUnknownAfter: null,
|
||||||
|
unresolvedCandidateCount: 0,
|
||||||
|
eligibleCandidateCount: 0,
|
||||||
|
candidateNodeIds: [],
|
||||||
|
resolvedCurrentTurnNodeIds: [],
|
||||||
|
noQuestionReason: null,
|
||||||
|
reasoningPatternValidation: null,
|
||||||
|
patternCompatibleNodeCount: 0,
|
||||||
|
incompatibleNodeIds: [],
|
||||||
|
compatibilityFailures: [],
|
||||||
|
replacementActions: [],
|
||||||
|
graphReasoningIntegrity: null,
|
||||||
|
plainLanguageNormalisations: [],
|
||||||
|
reasoningPattern: null,
|
||||||
|
questionFamily: null,
|
||||||
|
allowedQuestionFamilies: [],
|
||||||
|
rejectedQuestionFamilies: [],
|
||||||
|
selectedQuestionTemplate: null,
|
||||||
|
reasoningPatternReason: null,
|
||||||
unknownSelectionExplanation: buildUnknownSelectionDiagnostics(
|
unknownSelectionExplanation: buildUnknownSelectionDiagnostics(
|
||||||
situationGraph,
|
situationGraph,
|
||||||
situationGraph.resolvedNodeIds || [],
|
situationGraph.resolvedNodeIds || [],
|
||||||
|
|||||||
@@ -498,8 +498,42 @@ function buildBroadInvestigationQuestion(graph) {
|
|||||||
return `What changed during that period that could help explain why ${central}?`;
|
return `What changed during that period that could help explain why ${central}?`;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function buildFoundationalDirectQuestion(node) {
|
||||||
|
const label = stripTrailingPunctuation(node?.label || "");
|
||||||
|
|
||||||
|
if (/^who experiences this problem$/i.test(label)) {
|
||||||
|
return "Who experiences this problem?";
|
||||||
|
}
|
||||||
|
if (/^whether other people experience this problem$/i.test(label)) {
|
||||||
|
return "What makes you think other people experience this problem too?";
|
||||||
|
}
|
||||||
|
if (/^what happens when this problem is not resolved$/i.test(label)) {
|
||||||
|
return "What happens when this problem is not resolved?";
|
||||||
|
}
|
||||||
|
if (/^how often this problem happens$/i.test(label)) {
|
||||||
|
return "How often does this problem happen?";
|
||||||
|
}
|
||||||
|
if (/^how people deal with this problem today$/i.test(label)) {
|
||||||
|
return "How do people deal with this problem today?";
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
/^whether people actively look for help with this problem$/i.test(label)
|
||||||
|
) {
|
||||||
|
return "Do people actively look for help with this problem?";
|
||||||
|
}
|
||||||
|
if (/^whether people would pay to solve this problem$/i.test(label)) {
|
||||||
|
return "Would people pay to solve this problem?";
|
||||||
|
}
|
||||||
|
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
function isRelationshipExplanationUnknown(node, graph) {
|
function isRelationshipExplanationUnknown(node, graph) {
|
||||||
const text = normaliseText(`${node?.label || ""} ${node?.description || ""}`);
|
const text = normaliseText(`${node?.label || ""} ${node?.description || ""}`);
|
||||||
|
if (isDirectlyAnswerableObservationChildText(text)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
collectObservationNodes(graph).length >= 2 &&
|
collectObservationNodes(graph).length >= 2 &&
|
||||||
/\b(explain|explanation|divergence|moved differently|difference between|change or event|what changed|why the observations)/.test(
|
/\b(explain|explanation|divergence|moved differently|difference between|change or event|what changed|why the observations)/.test(
|
||||||
@@ -514,6 +548,12 @@ function isBroadCompositeUnknownText(text) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function isCommercialValidationUnknownText(text) {
|
||||||
|
return /\b(genuine problem|people would value|pay for it|commercially justified|commercial justification|commercial value|justified confidence|decision support methods|decision support|budget do they currently allocate|willingness to pay|problem existence|seek help)\b/.test(
|
||||||
|
text,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
function hasCompoundAbstractSignals(text) {
|
function hasCompoundAbstractSignals(text) {
|
||||||
return (
|
return (
|
||||||
/\b(timing|measurement|basis|cost|costs|debt|stock|tax|capital spending|mix|segment)\s+(and|or)\s+\b/.test(
|
/\b(timing|measurement|basis|cost|costs|debt|stock|tax|capital spending|mix|segment)\s+(and|or)\s+\b/.test(
|
||||||
@@ -536,6 +576,93 @@ function isDirectlyAnswerableObservationChildText(text) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function countPrerequisiteConceptSignals(text) {
|
||||||
|
let count = 0;
|
||||||
|
|
||||||
|
if (/\bproblem\b/.test(text)) count += 1;
|
||||||
|
if (/\b(audience|customer|user|buyer|stakeholder|recipient)\b/.test(text))
|
||||||
|
count += 1;
|
||||||
|
if (/\b(demand|seek help|actively look for help)\b/.test(text)) count += 1;
|
||||||
|
if (/\b(pay|willingness to pay|price|pricing)\b/.test(text)) count += 1;
|
||||||
|
if (
|
||||||
|
/\b(compare|comparison|different from|alternatives|alternative|existing alternatives|existing tools|better than)\b/.test(
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
count += 1;
|
||||||
|
if (/\b(value|viability|justified|business case|commercial)\b/.test(text))
|
||||||
|
count += 1;
|
||||||
|
if (/\b(feasibility|technical)\b/.test(text)) count += 1;
|
||||||
|
|
||||||
|
return count;
|
||||||
|
}
|
||||||
|
|
||||||
|
function countIndependentAnswerDimensions(node, graph) {
|
||||||
|
const text = normaliseText(`${node?.label || ""} ${node?.description || ""}`);
|
||||||
|
const relatedNodes = collectRelatedNodes(node, graph);
|
||||||
|
const unresolvedDependencies = relatedNodes.filter(
|
||||||
|
(relatedNode) =>
|
||||||
|
relatedNode.kind === "unknown" && relatedNode.status !== "resolved",
|
||||||
|
).length;
|
||||||
|
const conjunctionCount = (text.match(/\b(and|or)\b/g) || []).length;
|
||||||
|
const prerequisiteConceptCount = countPrerequisiteConceptSignals(text);
|
||||||
|
const implicitConclusion =
|
||||||
|
/\b(commercially justified|commercial justification|commercial viability|business case|customer value|market demand|product validation|technical feasibility)\b/.test(
|
||||||
|
text,
|
||||||
|
) ||
|
||||||
|
(/\bevidence that\b/.test(text) &&
|
||||||
|
/\b(problem|need|demand|audience|customer|user|alternatives|better than)\b/.test(
|
||||||
|
text,
|
||||||
|
)) ||
|
||||||
|
(/\bwhether\b/.test(text) &&
|
||||||
|
/\b(addresses|solve|solves|justifies|supports|demonstrates)\b/.test(
|
||||||
|
text,
|
||||||
|
) &&
|
||||||
|
/\b(problem|value|need|demand|audience|customer|user)\b/.test(text));
|
||||||
|
|
||||||
|
return {
|
||||||
|
prerequisiteConceptCount,
|
||||||
|
unresolvedDependencies,
|
||||||
|
conjunctionCount,
|
||||||
|
implicitConclusion,
|
||||||
|
multipleEvidenceDimensions:
|
||||||
|
prerequisiteConceptCount >= 2 || conjunctionCount >= 2,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export function assessUnknownAnswerability({ node, graph }) {
|
||||||
|
const text = normaliseText(`${node?.label || ""} ${node?.description || ""}`);
|
||||||
|
const dimensionSummary = countIndependentAnswerDimensions(node, graph);
|
||||||
|
const independentlyAnswerable =
|
||||||
|
!dimensionSummary.implicitConclusion &&
|
||||||
|
!dimensionSummary.multipleEvidenceDimensions &&
|
||||||
|
dimensionSummary.unresolvedDependencies === 0 &&
|
||||||
|
dimensionSummary.prerequisiteConceptCount <= 1;
|
||||||
|
|
||||||
|
if (independentlyAnswerable) {
|
||||||
|
return {
|
||||||
|
independentlyAnswerable: true,
|
||||||
|
reason:
|
||||||
|
"This unknown can be answered directly without first resolving several prerequisite investigations.",
|
||||||
|
prerequisiteConceptCount: dimensionSummary.prerequisiteConceptCount,
|
||||||
|
decompositionRequired: false,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
independentlyAnswerable: false,
|
||||||
|
reason: dimensionSummary.implicitConclusion
|
||||||
|
? "This unknown asks for a higher-level conclusion that depends on several smaller investigations."
|
||||||
|
: "This unknown still bundles multiple prerequisite evidence dimensions, so it should be decomposed before it becomes the selected question.",
|
||||||
|
prerequisiteConceptCount: Math.max(
|
||||||
|
dimensionSummary.prerequisiteConceptCount,
|
||||||
|
dimensionSummary.unresolvedDependencies,
|
||||||
|
dimensionSummary.conjunctionCount + 1,
|
||||||
|
),
|
||||||
|
decompositionRequired: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
export function assessUnknownAtomicity({ node, graph }) {
|
export function assessUnknownAtomicity({ node, graph }) {
|
||||||
const nodeText = normaliseText(
|
const nodeText = normaliseText(
|
||||||
`${node?.label || ""} ${node?.description || ""}`,
|
`${node?.label || ""} ${node?.description || ""}`,
|
||||||
@@ -592,6 +719,15 @@ export function assessUnknownAtomicity({ node, graph }) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (isCommercialValidationUnknownText(nodeText)) {
|
||||||
|
return {
|
||||||
|
atomicity: "composite",
|
||||||
|
reason:
|
||||||
|
"This unknown combines multiple problem-validation or commercial-validation dimensions, so it should be decomposed before asking a direct question.",
|
||||||
|
decompositionKind: "commercial_validation",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
return {
|
return {
|
||||||
atomicity: "atomic",
|
atomicity: "atomic",
|
||||||
reason:
|
reason:
|
||||||
@@ -600,6 +736,433 @@ export function assessUnknownAtomicity({ node, graph }) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const ALL_REASONING_PATTERNS = [
|
||||||
|
"decision",
|
||||||
|
"explanation",
|
||||||
|
"contradiction",
|
||||||
|
"definition",
|
||||||
|
"diagnosis",
|
||||||
|
"comparison",
|
||||||
|
"prioritisation",
|
||||||
|
];
|
||||||
|
|
||||||
|
const QUESTION_FAMILIES_BY_PATTERN = {
|
||||||
|
decision: [
|
||||||
|
"decision_foundation",
|
||||||
|
"decision_evidence",
|
||||||
|
"decision_threshold",
|
||||||
|
"definition",
|
||||||
|
],
|
||||||
|
explanation: ["explanation", "comparison"],
|
||||||
|
contradiction: ["contradiction", "comparison", "explanation"],
|
||||||
|
definition: ["definition"],
|
||||||
|
diagnosis: ["diagnosis", "comparison"],
|
||||||
|
comparison: ["comparison"],
|
||||||
|
prioritisation: ["prioritisation", "decision_threshold"],
|
||||||
|
};
|
||||||
|
|
||||||
|
const STRATEGIES_BY_PATTERN = {
|
||||||
|
decision: [
|
||||||
|
"decision_threshold",
|
||||||
|
"evidence_gathering",
|
||||||
|
"definition",
|
||||||
|
"baseline_reconstruction",
|
||||||
|
],
|
||||||
|
explanation: ["evidence_gathering", "baseline_reconstruction"],
|
||||||
|
contradiction: [
|
||||||
|
"contradiction_resolution",
|
||||||
|
"baseline_reconstruction",
|
||||||
|
"evidence_gathering",
|
||||||
|
],
|
||||||
|
definition: ["definition"],
|
||||||
|
diagnosis: ["evidence_gathering", "baseline_reconstruction"],
|
||||||
|
comparison: ["baseline_reconstruction", "evidence_gathering"],
|
||||||
|
prioritisation: ["decision_threshold", "evidence_gathering"],
|
||||||
|
};
|
||||||
|
|
||||||
|
function buildParentChain(node, graph) {
|
||||||
|
const nodesById = buildNodeMap(graph);
|
||||||
|
const chain = [];
|
||||||
|
let current = node?.parentId ? nodesById.get(node.parentId) : null;
|
||||||
|
|
||||||
|
while (current) {
|
||||||
|
chain.push(current);
|
||||||
|
current = current.parentId ? nodesById.get(current.parentId) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
return chain;
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasDecisionContext(node, graph, relatedNodes = []) {
|
||||||
|
const ancestry = buildParentChain(node, graph);
|
||||||
|
const contextText = normaliseText(
|
||||||
|
[
|
||||||
|
graph?.centralStatement,
|
||||||
|
node?.label,
|
||||||
|
node?.description,
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.label),
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.description),
|
||||||
|
...ancestry.map((ancestor) => ancestor.label),
|
||||||
|
...ancestry.map((ancestor) => ancestor.description),
|
||||||
|
...collectResolvedContextValues(graph),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" "),
|
||||||
|
);
|
||||||
|
|
||||||
|
return /\b(whether to|build|launch|continue|proceed|invest|commercially justified|commercial justification|commercial value|business case|viability)\b/.test(
|
||||||
|
contextText,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function hasObservationRelationshipTopology(node, graph, relatedNodes = []) {
|
||||||
|
const ancestry = buildParentChain(node, graph);
|
||||||
|
const topologyNodes = [node, ...relatedNodes, ...ancestry].filter(Boolean);
|
||||||
|
|
||||||
|
return (
|
||||||
|
collectObservationNodes(graph).length >= 2 &&
|
||||||
|
topologyNodes.some(
|
||||||
|
(candidate) =>
|
||||||
|
candidate.kind === "observation" || candidate.kind === "relationship",
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function isDefinitionPatternCandidate(node, graph) {
|
||||||
|
const text = normaliseText(`${node?.label || ""} ${node?.description || ""}`);
|
||||||
|
return isDefinitionLikeUnknown(
|
||||||
|
text,
|
||||||
|
`${text} ${graph?.centralStatement || ""}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function isContradictionPatternCandidate(node, graph, relatedNodes = []) {
|
||||||
|
const relationshipStatus = graph?.reasoningState?.relationshipStatus ?? null;
|
||||||
|
const text = normaliseText(
|
||||||
|
[
|
||||||
|
node?.label,
|
||||||
|
node?.description,
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.label),
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.description),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" "),
|
||||||
|
);
|
||||||
|
|
||||||
|
return (
|
||||||
|
relationshipStatus === "contradictory" ||
|
||||||
|
/\b(contradiction|contradict|conflict|inconsistent|mismatch|opposing)\b/.test(
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function isComparisonPatternCandidate(node, graph, relatedNodes = []) {
|
||||||
|
if (isRelationshipExplanationUnknown(node, graph)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
const text = normaliseText(
|
||||||
|
[
|
||||||
|
node?.label,
|
||||||
|
node?.description,
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.label),
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.description),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" "),
|
||||||
|
);
|
||||||
|
|
||||||
|
return (
|
||||||
|
(hasObservationRelationshipTopology(node, graph, relatedNodes) ||
|
||||||
|
collectObservationNodes(graph).length >= 2 ||
|
||||||
|
graph?.reasoningState?.comparabilityStatus === "uncertain") &&
|
||||||
|
/\b(compare|comparison|different timing|measured|measurement|basis|scale|period|alternative|alternatives|better than)\b/.test(
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function isExplanationPatternCandidate(node, graph, relatedNodes = []) {
|
||||||
|
if (isRelationshipExplanationUnknown(node, graph)) return true;
|
||||||
|
|
||||||
|
const text = normaliseText(
|
||||||
|
[
|
||||||
|
node?.label,
|
||||||
|
node?.description,
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.label),
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.description),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" "),
|
||||||
|
);
|
||||||
|
|
||||||
|
return (
|
||||||
|
hasObservationRelationshipTopology(node, graph, relatedNodes) &&
|
||||||
|
/\b(explain|explanation|why .* but|difference between|divergence|what changed|moved differently)\b/.test(
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function isPrioritisationPatternCandidate(node, graph, relatedNodes = []) {
|
||||||
|
const text = normaliseText(
|
||||||
|
[
|
||||||
|
node?.label,
|
||||||
|
node?.description,
|
||||||
|
graph?.centralStatement,
|
||||||
|
...relatedNodes.map((relatedNode) => relatedNode.label),
|
||||||
|
]
|
||||||
|
.filter(Boolean)
|
||||||
|
.join(" "),
|
||||||
|
);
|
||||||
|
|
||||||
|
return /\b(prioritise|prioritize|priority|rank|ranking|trade off|tradeoff|which first)\b/.test(
|
||||||
|
text,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function selectReasoningPattern({ node, graph, context = {} }) {
|
||||||
|
const relatedNodes = collectRelatedNodes(node, graph);
|
||||||
|
const patternContext = {
|
||||||
|
hasDecisionContext: hasDecisionContext(node, graph, relatedNodes),
|
||||||
|
hasObservationRelationshipTopology: hasObservationRelationshipTopology(
|
||||||
|
node,
|
||||||
|
graph,
|
||||||
|
relatedNodes,
|
||||||
|
),
|
||||||
|
};
|
||||||
|
|
||||||
|
if (isDefinitionPatternCandidate(node, graph)) {
|
||||||
|
return {
|
||||||
|
pattern: "definition",
|
||||||
|
reason:
|
||||||
|
"Selected definition because the active unknown is about meaning, scope, or term boundaries.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (isContradictionPatternCandidate(node, graph, relatedNodes)) {
|
||||||
|
return {
|
||||||
|
pattern: "contradiction",
|
||||||
|
reason:
|
||||||
|
"Selected contradiction because the graph indicates opposing claims or incompatible observations.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (isComparisonPatternCandidate(node, graph, relatedNodes)) {
|
||||||
|
return {
|
||||||
|
pattern: "comparison",
|
||||||
|
reason:
|
||||||
|
"Selected comparison because the active unknown is about distinguishing measurements, timing, basis, or alternatives.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (isExplanationPatternCandidate(node, graph, relatedNodes)) {
|
||||||
|
return {
|
||||||
|
pattern: "explanation",
|
||||||
|
reason:
|
||||||
|
"Selected explanation because the active unknown is about accounting for a relationship between observations.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (patternContext.hasDecisionContext) {
|
||||||
|
return {
|
||||||
|
pattern: "decision",
|
||||||
|
reason:
|
||||||
|
"Selected decision because the active unknown sits inside a build, continue, invest, or commercial-justification decision context.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (isPrioritisationPatternCandidate(node, graph, relatedNodes)) {
|
||||||
|
return {
|
||||||
|
pattern: "prioritisation",
|
||||||
|
reason:
|
||||||
|
"Selected prioritisation because the active unknown is about ordering options or trade-offs.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
pattern: "diagnosis",
|
||||||
|
reason:
|
||||||
|
"Selected diagnosis as the default because the active unknown needs clarifying evidence or mechanism-level investigation.",
|
||||||
|
context: patternContext,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function allowedQuestionFamiliesForPattern(pattern) {
|
||||||
|
return QUESTION_FAMILIES_BY_PATTERN[pattern] || [pattern];
|
||||||
|
}
|
||||||
|
|
||||||
|
function rejectedQuestionFamiliesForPattern(pattern) {
|
||||||
|
const allowed = new Set(allowedQuestionFamiliesForPattern(pattern));
|
||||||
|
return ALL_REASONING_PATTERNS.flatMap((candidatePattern) =>
|
||||||
|
(
|
||||||
|
QUESTION_FAMILIES_BY_PATTERN[candidatePattern] || [candidatePattern]
|
||||||
|
).filter((family) => !allowed.has(family)),
|
||||||
|
).filter((family, index, list) => list.indexOf(family) === index);
|
||||||
|
}
|
||||||
|
|
||||||
|
function constrainStrategyToReasoningPattern(strategy, reasoningPattern) {
|
||||||
|
if (!strategy) return null;
|
||||||
|
const allowedStrategies = STRATEGIES_BY_PATTERN[reasoningPattern] || [];
|
||||||
|
return allowedStrategies.includes(strategy.key) ? strategy : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function selectQuestionFamily({
|
||||||
|
node,
|
||||||
|
graph,
|
||||||
|
reasoningPattern,
|
||||||
|
investigationStrategy,
|
||||||
|
}) {
|
||||||
|
const text = normaliseText(`${node?.label || ""} ${node?.description || ""}`);
|
||||||
|
|
||||||
|
if (reasoningPattern === "definition") {
|
||||||
|
return { family: "definition", template: "definition_meaning" };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "contradiction") {
|
||||||
|
if (/\b(period|timing|basis|scale|measure|measured)\b/.test(text)) {
|
||||||
|
return {
|
||||||
|
family: "comparison",
|
||||||
|
template: "comparison_reconcile_measurement",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
family: "contradiction",
|
||||||
|
template: "contradiction_resolve_opposition",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "explanation") {
|
||||||
|
return {
|
||||||
|
family: "explanation",
|
||||||
|
template: "explanation_broad_investigation",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "comparison") {
|
||||||
|
return /\b(period|timing)\b/.test(text) &&
|
||||||
|
!/\bhow the two observations were measured|measurement\b/.test(text)
|
||||||
|
? { family: "comparison", template: "comparison_timing_basis" }
|
||||||
|
: { family: "comparison", template: "comparison_measurement_basis" };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "decision") {
|
||||||
|
if (
|
||||||
|
/\b(audience|customer|user|buyer|stakeholder|recipient|who experiences)\b/.test(
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
) {
|
||||||
|
return { family: "decision_foundation", template: "decision_audience" };
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
/\b(alternative|alternatives|better than|different from|deal with)\b/.test(
|
||||||
|
text,
|
||||||
|
)
|
||||||
|
) {
|
||||||
|
return {
|
||||||
|
family: "decision_foundation",
|
||||||
|
template: "decision_current_alternatives",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (/\b(problem|need|demand)\b/.test(text)) {
|
||||||
|
return {
|
||||||
|
family: "decision_foundation",
|
||||||
|
template: "decision_problem_existence",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
if (investigationStrategy?.key === "decision_threshold") {
|
||||||
|
return {
|
||||||
|
family: "decision_threshold",
|
||||||
|
template: "decision_threshold_outcome",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
family: "decision_evidence",
|
||||||
|
template: "decision_evidence_clarification",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "prioritisation") {
|
||||||
|
return { family: "prioritisation", template: "prioritisation_tradeoff" };
|
||||||
|
}
|
||||||
|
|
||||||
|
return investigationStrategy?.key === "baseline_reconstruction"
|
||||||
|
? { family: "comparison", template: "diagnosis_baseline_comparison" }
|
||||||
|
: { family: "diagnosis", template: "diagnosis_evidence" };
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildQuestionFromFamily({
|
||||||
|
node,
|
||||||
|
graph,
|
||||||
|
reasoningPattern,
|
||||||
|
questionFamily,
|
||||||
|
selectedQuestionTemplate,
|
||||||
|
investigationStrategy,
|
||||||
|
}) {
|
||||||
|
const meaning = extractMeaning(node);
|
||||||
|
|
||||||
|
if (reasoningPattern === "decision") {
|
||||||
|
if (selectedQuestionTemplate === "decision_audience") {
|
||||||
|
return "Who experiences this problem?";
|
||||||
|
}
|
||||||
|
if (selectedQuestionTemplate === "decision_current_alternatives") {
|
||||||
|
return "How do people deal with this today?";
|
||||||
|
}
|
||||||
|
if (selectedQuestionTemplate === "decision_problem_existence") {
|
||||||
|
return "What makes you think this is a real problem?";
|
||||||
|
}
|
||||||
|
if (selectedQuestionTemplate === "decision_threshold_outcome") {
|
||||||
|
return buildQuestionFromStrategy(
|
||||||
|
investigationStrategy || {
|
||||||
|
key: "decision_threshold",
|
||||||
|
meaning,
|
||||||
|
actionPhrase: null,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
return `What evidence would clarify ${stripTrailingPunctuation(meaning)}?`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (questionFamily === "definition") {
|
||||||
|
return `What does ${meaning} mean in this situation?`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "comparison") {
|
||||||
|
if (selectedQuestionTemplate === "comparison_timing_basis") {
|
||||||
|
return `What evidence would clarify whether ${stripTrailingPunctuation(meaning)}?`;
|
||||||
|
}
|
||||||
|
if (selectedQuestionTemplate === "comparison_measurement_basis") {
|
||||||
|
return `What evidence would clarify ${stripTrailingPunctuation(meaning)}?`;
|
||||||
|
}
|
||||||
|
return `What evidence would clarify ${stripTrailingPunctuation(meaning)}?`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "contradiction") {
|
||||||
|
return investigationStrategy?.key === "contradiction_resolution"
|
||||||
|
? buildQuestionFromStrategy(investigationStrategy)
|
||||||
|
: `What fact would resolve the contradiction about ${stripTrailingPunctuation(meaning)}?`;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "explanation") {
|
||||||
|
return buildBroadInvestigationQuestion(graph);
|
||||||
|
}
|
||||||
|
|
||||||
|
if (reasoningPattern === "prioritisation") {
|
||||||
|
return `Which option should be investigated first, and why?`;
|
||||||
|
}
|
||||||
|
|
||||||
|
return investigationStrategy
|
||||||
|
? buildQuestionFromStrategy(investigationStrategy)
|
||||||
|
: buildNeutralClarificationQuestion(meaning);
|
||||||
|
}
|
||||||
|
|
||||||
export function formulateTieResolutionQuestion({ graph }) {
|
export function formulateTieResolutionQuestion({ graph }) {
|
||||||
const comparability = assessComparability(graph);
|
const comparability = assessComparability(graph);
|
||||||
if (comparability.comparabilityStatus === "uncertain") {
|
if (comparability.comparabilityStatus === "uncertain") {
|
||||||
@@ -618,6 +1181,14 @@ export function formulateTieResolutionQuestion({ graph }) {
|
|||||||
relationshipStatus: deferredRelationship.relationshipStatus,
|
relationshipStatus: deferredRelationship.relationshipStatus,
|
||||||
relationshipReason: deferredRelationship.reason,
|
relationshipReason: deferredRelationship.reason,
|
||||||
relationshipAssessed: deferredRelationship.relationshipAssessed,
|
relationshipAssessed: deferredRelationship.relationshipAssessed,
|
||||||
|
reasoningPattern: "comparison",
|
||||||
|
reasoningPatternReason:
|
||||||
|
"Tie resolution is using the comparison family because comparability is still unresolved.",
|
||||||
|
allowedQuestionFamilies: allowedQuestionFamiliesForPattern("comparison"),
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
rejectedQuestionFamiliesForPattern("comparison"),
|
||||||
|
questionFamily: "comparison",
|
||||||
|
selectedQuestionTemplate: "comparison_tie_resolution",
|
||||||
questionRequired: true,
|
questionRequired: true,
|
||||||
reasoningStages: deferredRelationship.reasoningStages,
|
reasoningStages: deferredRelationship.reasoningStages,
|
||||||
};
|
};
|
||||||
@@ -637,6 +1208,14 @@ export function formulateTieResolutionQuestion({ graph }) {
|
|||||||
relationshipReason: relationship.reason,
|
relationshipReason: relationship.reason,
|
||||||
relationshipAssessed: relationship.relationshipAssessed,
|
relationshipAssessed: relationship.relationshipAssessed,
|
||||||
contradictionReasoningAllowed: relationship.contradictionReasoningAllowed,
|
contradictionReasoningAllowed: relationship.contradictionReasoningAllowed,
|
||||||
|
reasoningPattern: "comparison",
|
||||||
|
reasoningPatternReason:
|
||||||
|
"Tie resolution remains in the comparison family because no distinct winning unknown exists.",
|
||||||
|
allowedQuestionFamilies: allowedQuestionFamiliesForPattern("comparison"),
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
rejectedQuestionFamiliesForPattern("comparison"),
|
||||||
|
questionFamily: "comparison",
|
||||||
|
selectedQuestionTemplate: "comparison_no_question_required",
|
||||||
questionRequired: relationship.questionRequired,
|
questionRequired: relationship.questionRequired,
|
||||||
questionSuppressedReason: relationship.questionSuppressedReason,
|
questionSuppressedReason: relationship.questionSuppressedReason,
|
||||||
reasoningStages: relationship.reasoningStages,
|
reasoningStages: relationship.reasoningStages,
|
||||||
@@ -657,6 +1236,14 @@ export function formulateTieResolutionQuestion({ graph }) {
|
|||||||
relationshipReason: relationship.reason,
|
relationshipReason: relationship.reason,
|
||||||
relationshipAssessed: relationship.relationshipAssessed,
|
relationshipAssessed: relationship.relationshipAssessed,
|
||||||
contradictionReasoningAllowed: relationship.contradictionReasoningAllowed,
|
contradictionReasoningAllowed: relationship.contradictionReasoningAllowed,
|
||||||
|
reasoningPattern: "explanation",
|
||||||
|
reasoningPatternReason:
|
||||||
|
"Tie resolution is using the explanation family because the observations appear related and need a neutral explanation question.",
|
||||||
|
allowedQuestionFamilies: allowedQuestionFamiliesForPattern("explanation"),
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
rejectedQuestionFamiliesForPattern("explanation"),
|
||||||
|
questionFamily: "explanation",
|
||||||
|
selectedQuestionTemplate: "explanation_broad_investigation",
|
||||||
questionRequired: relationship.questionRequired,
|
questionRequired: relationship.questionRequired,
|
||||||
reasoningStages: relationship.reasoningStages,
|
reasoningStages: relationship.reasoningStages,
|
||||||
};
|
};
|
||||||
@@ -683,6 +1270,14 @@ export function formulateTieResolutionQuestion({ graph }) {
|
|||||||
relationshipReason: relationship.reason,
|
relationshipReason: relationship.reason,
|
||||||
relationshipAssessed: relationship.relationshipAssessed,
|
relationshipAssessed: relationship.relationshipAssessed,
|
||||||
contradictionReasoningAllowed: relationship.contradictionReasoningAllowed,
|
contradictionReasoningAllowed: relationship.contradictionReasoningAllowed,
|
||||||
|
reasoningPattern: "contradiction",
|
||||||
|
reasoningPatternReason:
|
||||||
|
"Tie resolution is using the contradiction family because the graph is trying to distinguish incompatible explanations.",
|
||||||
|
allowedQuestionFamilies: allowedQuestionFamiliesForPattern("contradiction"),
|
||||||
|
rejectedQuestionFamilies:
|
||||||
|
rejectedQuestionFamiliesForPattern("contradiction"),
|
||||||
|
questionFamily: "contradiction",
|
||||||
|
selectedQuestionTemplate: "contradiction_distinguishing_change",
|
||||||
questionRequired: relationship.questionRequired,
|
questionRequired: relationship.questionRequired,
|
||||||
reasoningStages: relationship.reasoningStages,
|
reasoningStages: relationship.reasoningStages,
|
||||||
};
|
};
|
||||||
@@ -770,6 +1365,9 @@ function buildInvestigationStrategy({
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
||||||
|
const effectiveReasoningPattern =
|
||||||
|
context.reasoningPattern ||
|
||||||
|
selectReasoningPattern({ node, graph, context }).pattern;
|
||||||
const relatedNodes = collectRelatedNodes(node, graph);
|
const relatedNodes = collectRelatedNodes(node, graph);
|
||||||
const meaning = extractMeaning(node);
|
const meaning = extractMeaning(node);
|
||||||
const combinedText = [
|
const combinedText = [
|
||||||
@@ -820,8 +1418,10 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
const hasPrimaryBaselineLanguage =
|
const hasPrimaryBaselineLanguage =
|
||||||
/\b(before|previous|baseline|prior|comparable state)\b/.test(nodeText);
|
/\b(before|previous|baseline|prior|comparable state)\b/.test(nodeText);
|
||||||
|
|
||||||
|
let selectedStrategy = null;
|
||||||
|
|
||||||
if (hasBaselineLanguage && hasPrimaryBaselineLanguage) {
|
if (hasBaselineLanguage && hasPrimaryBaselineLanguage) {
|
||||||
return buildInvestigationStrategy({
|
selectedStrategy = buildInvestigationStrategy({
|
||||||
key: "baseline_reconstruction",
|
key: "baseline_reconstruction",
|
||||||
reason:
|
reason:
|
||||||
"Selected because the unknown explicitly references a missing previous or baseline state.",
|
"Selected because the unknown explicitly references a missing previous or baseline state.",
|
||||||
@@ -862,8 +1462,11 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
relatedNode.kind === "conclusion",
|
relatedNode.kind === "conclusion",
|
||||||
);
|
);
|
||||||
|
|
||||||
if (hasPrimaryDefinitionLanguage || hasDefinitionLanguage) {
|
if (
|
||||||
return buildInvestigationStrategy({
|
!selectedStrategy &&
|
||||||
|
(hasPrimaryDefinitionLanguage || hasDefinitionLanguage)
|
||||||
|
) {
|
||||||
|
selectedStrategy = buildInvestigationStrategy({
|
||||||
key: "definition",
|
key: "definition",
|
||||||
reason:
|
reason:
|
||||||
"Selected because the unknown is primarily about clarifying what a term means in this case.",
|
"Selected because the unknown is primarily about clarifying what a term means in this case.",
|
||||||
@@ -875,8 +1478,8 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
if (hasDecisionValueLanguage || hasCriteriaLanguage) {
|
if (!selectedStrategy && (hasDecisionValueLanguage || hasCriteriaLanguage)) {
|
||||||
return buildInvestigationStrategy({
|
selectedStrategy = buildInvestigationStrategy({
|
||||||
key: "decision_threshold",
|
key: "decision_threshold",
|
||||||
reason:
|
reason:
|
||||||
"Selected because the unknown determines the threshold for making or justifying a decision.",
|
"Selected because the unknown determines the threshold for making or justifying a decision.",
|
||||||
@@ -888,8 +1491,11 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
if (hasPrimaryBaselineLanguage || hasBaselineLanguage) {
|
if (
|
||||||
return buildInvestigationStrategy({
|
!selectedStrategy &&
|
||||||
|
(hasPrimaryBaselineLanguage || hasBaselineLanguage)
|
||||||
|
) {
|
||||||
|
selectedStrategy = buildInvestigationStrategy({
|
||||||
key: "baseline_reconstruction",
|
key: "baseline_reconstruction",
|
||||||
reason:
|
reason:
|
||||||
"Selected because reconstructing the prior state is the most direct way to resolve the unknown.",
|
"Selected because reconstructing the prior state is the most direct way to resolve the unknown.",
|
||||||
@@ -901,8 +1507,8 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
if (hasContradictionLanguage) {
|
if (!selectedStrategy && hasContradictionLanguage) {
|
||||||
return buildInvestigationStrategy({
|
selectedStrategy = buildInvestigationStrategy({
|
||||||
key: "contradiction_resolution",
|
key: "contradiction_resolution",
|
||||||
reason:
|
reason:
|
||||||
"Selected because the graph context indicates conflicting claims or inconsistent states that must be reconciled.",
|
"Selected because the graph context indicates conflicting claims or inconsistent states that must be reconciled.",
|
||||||
@@ -914,8 +1520,11 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
if (hasEvidenceLanguage || hasMeasurementLanguage || hasConstraintLanguage) {
|
if (
|
||||||
return buildInvestigationStrategy({
|
!selectedStrategy &&
|
||||||
|
(hasEvidenceLanguage || hasMeasurementLanguage || hasConstraintLanguage)
|
||||||
|
) {
|
||||||
|
selectedStrategy = buildInvestigationStrategy({
|
||||||
key: "evidence_gathering",
|
key: "evidence_gathering",
|
||||||
reason:
|
reason:
|
||||||
hasConstraintLanguage && hasPrimaryConstraintLanguage
|
hasConstraintLanguage && hasPrimaryConstraintLanguage
|
||||||
@@ -929,7 +1538,10 @@ export function selectInvestigationStrategy({ node, graph, context = {} }) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
return null;
|
return constrainStrategyToReasoningPattern(
|
||||||
|
selectedStrategy,
|
||||||
|
effectiveReasoningPattern,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
function buildQuestionFromStrategy(strategy) {
|
function buildQuestionFromStrategy(strategy) {
|
||||||
@@ -998,24 +1610,181 @@ function validateFormulatedQuestion(question, meaning) {
|
|||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function countPrimaryConcepts(question) {
|
||||||
|
let concepts = 1;
|
||||||
|
if (/\band what\b/i.test(question)) concepts += 1;
|
||||||
|
if (/\bhow often\b.*\bwhat\b/i.test(question)) concepts += 1;
|
||||||
|
if (/\bcost\b.*\bbudget\b|\bbudget\b.*\bcost\b/i.test(question)) {
|
||||||
|
concepts += 1;
|
||||||
|
}
|
||||||
|
if (/\bwho\b.*\bwhat\b|\bwhat\b.*\bwho\b/i.test(question)) concepts += 1;
|
||||||
|
return concepts;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function assessQuestionComplexity({ question, selectedUnknown, graph }) {
|
||||||
|
const text = String(question || "").trim();
|
||||||
|
const lower = text.toLowerCase();
|
||||||
|
const reasons = [];
|
||||||
|
const compoundQuestionSignals = [];
|
||||||
|
const abstractMatches =
|
||||||
|
lower.match(
|
||||||
|
/\b(justified confidence|financial or operational cost|comparable decision support methods|commercial justification|decision support methods|value recipient)\b/g,
|
||||||
|
) || [];
|
||||||
|
const primaryConceptCount = countPrimaryConcepts(lower);
|
||||||
|
|
||||||
|
if ((text.match(/\?/g) || []).length !== 1) {
|
||||||
|
reasons.push("multiple_question_marks");
|
||||||
|
compoundQuestionSignals.push("multiple_question_marks");
|
||||||
|
}
|
||||||
|
if (/\band what\b|\bwhat .* and .* what\b/i.test(text)) {
|
||||||
|
reasons.push("multiple_requested_answers");
|
||||||
|
compoundQuestionSignals.push("joined_requests");
|
||||||
|
}
|
||||||
|
if (/,[^,]{0,60},/.test(text) || /,\s*(and|or)\b/i.test(text)) {
|
||||||
|
reasons.push("list_like_question");
|
||||||
|
compoundQuestionSignals.push("comma_list");
|
||||||
|
}
|
||||||
|
if (/\bcost\b.*\bbudget\b|\bbudget\b.*\bcost\b/i.test(text)) {
|
||||||
|
reasons.push("cost_and_budget_combined");
|
||||||
|
compoundQuestionSignals.push("distinct_measures_combined");
|
||||||
|
}
|
||||||
|
if (primaryConceptCount > 1) {
|
||||||
|
reasons.push("multiple_primary_concepts");
|
||||||
|
}
|
||||||
|
if (text.split(/\s+/).length > 20) {
|
||||||
|
reasons.push("very_long_question");
|
||||||
|
}
|
||||||
|
if (abstractMatches.length > 1) {
|
||||||
|
reasons.push("abstract_term_chain");
|
||||||
|
}
|
||||||
|
|
||||||
|
const cognitiveLoad =
|
||||||
|
reasons.length >= 3 ? "high" : reasons.length === 2 ? "medium" : "low";
|
||||||
|
|
||||||
|
return {
|
||||||
|
acceptable: reasons.length === 0,
|
||||||
|
primaryConceptCount,
|
||||||
|
compoundQuestionSignals: [...new Set(compoundQuestionSignals)],
|
||||||
|
abstractTermCount: abstractMatches.length,
|
||||||
|
cognitiveLoad,
|
||||||
|
reasons,
|
||||||
|
selectedUnknownId: selectedUnknown?.id ?? null,
|
||||||
|
graphCentralStatement: graph?.centralStatement ?? null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyPlainLanguageNormalisations(question) {
|
||||||
|
const normalisations = [];
|
||||||
|
let next = String(question || "");
|
||||||
|
const replacements = [
|
||||||
|
[
|
||||||
|
/individuals experiencing insufficient justified confidence/gi,
|
||||||
|
"people who struggle to feel confident about a decision",
|
||||||
|
"simplified_justified_confidence_phrase",
|
||||||
|
],
|
||||||
|
[
|
||||||
|
/current financial or operational cost/gi,
|
||||||
|
"current cost",
|
||||||
|
"simplified_cost_phrase",
|
||||||
|
],
|
||||||
|
[
|
||||||
|
/comparable decision support methods/gi,
|
||||||
|
"other ways they deal with the problem",
|
||||||
|
"simplified_decision_support_phrase",
|
||||||
|
],
|
||||||
|
[
|
||||||
|
/the relevant customer, user, or value recipient/gi,
|
||||||
|
"the people affected",
|
||||||
|
"simplified_actor_phrase",
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
for (const [pattern, replacement, code] of replacements) {
|
||||||
|
if (pattern.test(next)) {
|
||||||
|
next = next.replace(pattern, replacement);
|
||||||
|
normalisations.push(code);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
next = sanitizeQuestionText(next);
|
||||||
|
return { question: next, normalisations };
|
||||||
|
}
|
||||||
|
|
||||||
export function formulateQuestion({ node, graph, context = {} }) {
|
export function formulateQuestion({ node, graph, context = {} }) {
|
||||||
if (context.selectionState?.status === "ambiguous") {
|
if (context.selectionState?.status === "ambiguous") {
|
||||||
return formulateTieResolutionQuestion({ graph });
|
return formulateTieResolutionQuestion({ graph });
|
||||||
}
|
}
|
||||||
|
|
||||||
const investigationStrategy = selectInvestigationStrategy({
|
const reasoningPatternSelection = selectReasoningPattern({
|
||||||
node,
|
node,
|
||||||
graph,
|
graph,
|
||||||
context,
|
context,
|
||||||
});
|
});
|
||||||
|
const allowedQuestionFamilies = allowedQuestionFamiliesForPattern(
|
||||||
|
reasoningPatternSelection.pattern,
|
||||||
|
);
|
||||||
|
const rejectedQuestionFamilies = rejectedQuestionFamiliesForPattern(
|
||||||
|
reasoningPatternSelection.pattern,
|
||||||
|
);
|
||||||
|
const foundationalDirectQuestion = buildFoundationalDirectQuestion(node);
|
||||||
|
if (
|
||||||
|
foundationalDirectQuestion &&
|
||||||
|
reasoningPatternSelection.pattern === "decision"
|
||||||
|
) {
|
||||||
|
const plainLanguage = applyPlainLanguageNormalisations(
|
||||||
|
sanitizeQuestionText(foundationalDirectQuestion),
|
||||||
|
);
|
||||||
|
const questionComplexity = assessQuestionComplexity({
|
||||||
|
question: plainLanguage.question,
|
||||||
|
selectedUnknown: node,
|
||||||
|
graph,
|
||||||
|
});
|
||||||
|
|
||||||
let question = investigationStrategy
|
return {
|
||||||
? buildQuestionFromStrategy(investigationStrategy)
|
question: plainLanguage.question,
|
||||||
: isRelationshipExplanationUnknown(node, graph)
|
reason:
|
||||||
? buildBroadInvestigationQuestion(graph)
|
"Formulated as a direct foundational question because this child unknown should be answered one step at a time.",
|
||||||
: buildNeutralClarificationQuestion(extractMeaning(node));
|
strategy: null,
|
||||||
|
investigationStrategy: null,
|
||||||
|
reasoningPattern: reasoningPatternSelection.pattern,
|
||||||
|
reasoningPatternReason: reasoningPatternSelection.reason,
|
||||||
|
questionFamily: "decision_foundation",
|
||||||
|
allowedQuestionFamilies,
|
||||||
|
rejectedQuestionFamilies,
|
||||||
|
selectedQuestionTemplate: "decision_foundation_direct_child",
|
||||||
|
questionComplexity,
|
||||||
|
plainLanguageNormalisations: plainLanguage.normalisations,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const investigationStrategy = selectInvestigationStrategy({
|
||||||
|
node,
|
||||||
|
graph,
|
||||||
|
context: {
|
||||||
|
...context,
|
||||||
|
reasoningPattern: reasoningPatternSelection.pattern,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const questionFamilySelection = selectQuestionFamily({
|
||||||
|
node,
|
||||||
|
graph,
|
||||||
|
reasoningPattern: reasoningPatternSelection.pattern,
|
||||||
|
investigationStrategy,
|
||||||
|
});
|
||||||
|
|
||||||
|
let question = buildQuestionFromFamily({
|
||||||
|
node,
|
||||||
|
graph,
|
||||||
|
reasoningPattern: reasoningPatternSelection.pattern,
|
||||||
|
questionFamily: questionFamilySelection.family,
|
||||||
|
selectedQuestionTemplate: questionFamilySelection.template,
|
||||||
|
investigationStrategy,
|
||||||
|
});
|
||||||
|
|
||||||
question = sanitizeQuestionText(question);
|
question = sanitizeQuestionText(question);
|
||||||
|
const plainLanguage = applyPlainLanguageNormalisations(question);
|
||||||
|
question = plainLanguage.question;
|
||||||
|
|
||||||
const fallbackMeaning = extractMeaning(node);
|
const fallbackMeaning = extractMeaning(node);
|
||||||
if (
|
if (
|
||||||
@@ -1044,6 +1813,12 @@ export function formulateQuestion({ node, graph, context = {} }) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const questionComplexity = assessQuestionComplexity({
|
||||||
|
question,
|
||||||
|
selectedUnknown: node,
|
||||||
|
graph,
|
||||||
|
});
|
||||||
|
|
||||||
return {
|
return {
|
||||||
question,
|
question,
|
||||||
reason: investigationStrategy
|
reason: investigationStrategy
|
||||||
@@ -1051,5 +1826,13 @@ export function formulateQuestion({ node, graph, context = {} }) {
|
|||||||
: "Formulated as a neutral clarification question because no narrower investigation strategy clearly applied.",
|
: "Formulated as a neutral clarification question because no narrower investigation strategy clearly applied.",
|
||||||
strategy: investigationStrategy?.key ?? null,
|
strategy: investigationStrategy?.key ?? null,
|
||||||
investigationStrategy,
|
investigationStrategy,
|
||||||
|
reasoningPattern: reasoningPatternSelection.pattern,
|
||||||
|
reasoningPatternReason: reasoningPatternSelection.reason,
|
||||||
|
questionFamily: questionFamilySelection.family,
|
||||||
|
allowedQuestionFamilies,
|
||||||
|
rejectedQuestionFamilies,
|
||||||
|
selectedQuestionTemplate: questionFamilySelection.template,
|
||||||
|
questionComplexity,
|
||||||
|
plainLanguageNormalisations: plainLanguage.normalisations,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
+1
-1
@@ -870,7 +870,7 @@ export function validateGraphUpdate(graph, update) {
|
|||||||
(u) => u.previousStatus !== null && u.newStatus !== u.previousStatus,
|
(u) => u.previousStatus !== null && u.newStatus !== u.previousStatus,
|
||||||
);
|
);
|
||||||
const valueChanged = update.updatedNodes.some(
|
const valueChanged = update.updatedNodes.some(
|
||||||
(u) => u.previousValue !== null && u.newValue !== u.previousValue,
|
(u) => (u.previousValue ?? null) !== (u.newValue ?? null),
|
||||||
);
|
);
|
||||||
|
|
||||||
const hasMeaningfulChange =
|
const hasMeaningfulChange =
|
||||||
|
|||||||
@@ -16,6 +16,18 @@ export function normaliseAnalysisResponse(input) {
|
|||||||
normalised.evidence = normalised.evidence.map((record, index) => {
|
normalised.evidence = normalised.evidence.map((record, index) => {
|
||||||
if (!record || typeof record !== "object") return record;
|
if (!record || typeof record !== "object") return record;
|
||||||
|
|
||||||
|
if (record.evidenceType === "reported_claim") {
|
||||||
|
changesApplied.push({
|
||||||
|
path: ["evidence", index, "evidenceType"],
|
||||||
|
change: "Converted reported_claim to reported_statement",
|
||||||
|
});
|
||||||
|
|
||||||
|
record = {
|
||||||
|
...record,
|
||||||
|
evidenceType: "reported_statement",
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
if (record.source === null) {
|
if (record.source === null) {
|
||||||
changesApplied.push({
|
changesApplied.push({
|
||||||
path: ["evidence", index, "source"],
|
path: ["evidence", index, "source"],
|
||||||
|
|||||||
@@ -229,6 +229,53 @@ function makeApplicationFixture() {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const COMMERCIAL_SCENARIO =
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.";
|
||||||
|
|
||||||
|
function makeCommercialUpdateFixture() {
|
||||||
|
const parent = makeNode({
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
return makeGraph({
|
||||||
|
centralStatement: COMMERCIAL_SCENARIO,
|
||||||
|
nodes: [parent],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: parent.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Commercial update fixture",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeMeaningfulNoOpProposal() {
|
||||||
|
return {
|
||||||
|
addedNodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-anchor",
|
||||||
|
label: "Update anchor",
|
||||||
|
description:
|
||||||
|
"Anchor state introduced by the answer because the update must contain a meaningful change.",
|
||||||
|
kind: "state",
|
||||||
|
status: "known",
|
||||||
|
confidence: "low",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
updatedNodes: [],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
describe("applyValidatedProposal", () => {
|
describe("applyValidatedProposal", () => {
|
||||||
it("applies a valid proposal successfully", () => {
|
it("applies a valid proposal successfully", () => {
|
||||||
const { graph, proposal, ids } = makeApplicationFixture();
|
const { graph, proposal, ids } = makeApplicationFixture();
|
||||||
@@ -1318,4 +1365,287 @@ describe("applyValidatedProposal", () => {
|
|||||||
),
|
),
|
||||||
).toHaveLength(firstResult.childNodeIds.length);
|
).toHaveLength(firstResult.childNodeIds.length);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("reselects a remaining commercial sibling after resolving the first child", () => {
|
||||||
|
const graph = makeCommercialUpdateFixture();
|
||||||
|
|
||||||
|
const firstResult = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeMeaningfulNoOpProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(firstResult.success).toBe(true);
|
||||||
|
expect(firstResult.selectedQuestion?.question).toBe(
|
||||||
|
"Who experiences this problem?",
|
||||||
|
);
|
||||||
|
|
||||||
|
const secondResult = applyValidatedProposal({
|
||||||
|
situationGraph: firstResult.updatedSituationGraph,
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [
|
||||||
|
{
|
||||||
|
nodeId: firstResult.selectedQuestion.nodeId,
|
||||||
|
previousStatus: "unknown",
|
||||||
|
newStatus: "resolved",
|
||||||
|
previousValue: null,
|
||||||
|
newValue:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
reason: "The answer confirms a self-observed instance.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [firstResult.selectedQuestion.nodeId],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
previousQuestion: firstResult.selectedQuestion.question,
|
||||||
|
answer:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(secondResult.success).toBe(true);
|
||||||
|
expect(secondResult.resolvedUnknownNodeIds).toContain(
|
||||||
|
firstResult.selectedQuestion.nodeId,
|
||||||
|
);
|
||||||
|
expect(secondResult.newActiveUnknownNodeId).toBe(
|
||||||
|
secondResult.selectedQuestion?.nodeId,
|
||||||
|
);
|
||||||
|
expect(secondResult.selectedQuestion?.question).toBe(
|
||||||
|
"What makes you think other people experience this problem too?",
|
||||||
|
);
|
||||||
|
expect(secondResult.selectedQuestion?.reasoningPattern).toBe("decision");
|
||||||
|
expect(secondResult.selectedQuestion?.questionFamily).toBe(
|
||||||
|
"decision_foundation",
|
||||||
|
);
|
||||||
|
expect(secondResult.selectedQuestion?.nodeId).not.toBe(
|
||||||
|
firstResult.selectedQuestion.nodeId,
|
||||||
|
);
|
||||||
|
expect(secondResult.unresolvedCandidateCount).toBeGreaterThan(0);
|
||||||
|
expect(secondResult.eligibleCandidateCount).toBeGreaterThan(0);
|
||||||
|
expect(secondResult.candidateNodeIds).toContain(
|
||||||
|
secondResult.selectedQuestion?.nodeId,
|
||||||
|
);
|
||||||
|
expect(secondResult.resolvedCurrentTurnNodeIds).toContain(
|
||||||
|
firstResult.selectedQuestion.nodeId,
|
||||||
|
);
|
||||||
|
expect(secondResult.noQuestionReason).toBeNull();
|
||||||
|
expect(secondResult.selectedQuestion?.question.toLowerCase()).not.toMatch(
|
||||||
|
/price|budget|market size|pilot metrics|benchmark|technical differentiation/,
|
||||||
|
);
|
||||||
|
expect(secondResult.reasoningPatternValidation).toMatchObject({
|
||||||
|
activePattern: "decision",
|
||||||
|
valid: true,
|
||||||
|
});
|
||||||
|
expect(secondResult.graphReasoningIntegrity).toBe("valid");
|
||||||
|
expect(secondResult.incompatibleNodeIds).toEqual([]);
|
||||||
|
expect(secondResult.compatibilityFailures).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reselects a non-repeated comparison sibling instead of asking the same evidence question again", () => {
|
||||||
|
const { graph, proposal } = makeComparabilityUpdateFixture();
|
||||||
|
|
||||||
|
const firstResult = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal,
|
||||||
|
previousQuestion:
|
||||||
|
"Were these figures measured on the same basis and at the same scale?",
|
||||||
|
answer:
|
||||||
|
"Yes. Both figures cover the same accounting period and are taken from the same management accounts.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(firstResult.success).toBe(true);
|
||||||
|
expect(firstResult.selectedQuestion?.question).toBe(
|
||||||
|
"What evidence would clarify how the two observations were measured?",
|
||||||
|
);
|
||||||
|
|
||||||
|
const secondResult = applyValidatedProposal({
|
||||||
|
situationGraph: firstResult.updatedSituationGraph,
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [
|
||||||
|
{
|
||||||
|
nodeId: firstResult.selectedQuestion.nodeId,
|
||||||
|
previousStatus: "unknown",
|
||||||
|
newStatus: "resolved",
|
||||||
|
previousValue: null,
|
||||||
|
newValue:
|
||||||
|
"Both measures come from the same monthly reporting pack and use the same source system.",
|
||||||
|
reason: "The answer resolves the measurement clarification child.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [firstResult.selectedQuestion.nodeId],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
previousQuestion: firstResult.selectedQuestion.question,
|
||||||
|
answer:
|
||||||
|
"Both measures come from the same monthly reporting pack and use the same source system.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(secondResult.success).toBe(true);
|
||||||
|
expect(secondResult.selectedQuestion?.nodeId).not.toBe(
|
||||||
|
firstResult.selectedQuestion.nodeId,
|
||||||
|
);
|
||||||
|
expect(secondResult.selectedQuestion?.question).not.toBe(
|
||||||
|
firstResult.selectedQuestion.question,
|
||||||
|
);
|
||||||
|
expect(secondResult.finalQuestion).not.toBe(
|
||||||
|
firstResult.selectedQuestion.question,
|
||||||
|
);
|
||||||
|
expect(secondResult.noQuestionReason).toBeNull();
|
||||||
|
expect(secondResult.selectedQuestion?.nodeId).toBe(
|
||||||
|
secondResult.newActiveUnknownNodeId,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects a compound selected question before returning it", () => {
|
||||||
|
const graph = makeCommercialUpdateFixture();
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: {
|
||||||
|
...makeMeaningfulNoOpProposal(),
|
||||||
|
selectedQuestion: {
|
||||||
|
nodeId: "n-commercial-parent",
|
||||||
|
question:
|
||||||
|
"What changed during the period that could explain why work is taking longer, and how were the two observations measured?",
|
||||||
|
reason: "Invalid compound follow-up.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(false);
|
||||||
|
expect(result.stage).toBe("proposal_compatibility");
|
||||||
|
expect(result.errors).toContain(
|
||||||
|
"selectedQuestion must be a single non-compound question",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("allows a legitimate single-concept or-question", () => {
|
||||||
|
const graph = makeCommercialUpdateFixture();
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: {
|
||||||
|
...makeMeaningfulNoOpProposal(),
|
||||||
|
selectedQuestion: {
|
||||||
|
nodeId: "n-commercial-parent",
|
||||||
|
question: "Is the problem caused by timing or measurement basis?",
|
||||||
|
reason: "Single concept contrast.",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns no question when the graph is truly complete after resolution", () => {
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement: "A single missing fact needs confirmation.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-only-unknown",
|
||||||
|
label: "Missing fact",
|
||||||
|
description:
|
||||||
|
"Need the missing fact because the conclusion depends on it.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: "n-only-unknown",
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Single unknown fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [
|
||||||
|
{
|
||||||
|
nodeId: "n-only-unknown",
|
||||||
|
previousStatus: "unknown",
|
||||||
|
newStatus: "resolved",
|
||||||
|
previousValue: null,
|
||||||
|
newValue: "Confirmed.",
|
||||||
|
reason: "The answer resolves the only unknown.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: ["n-only-unknown"],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
previousQuestion: "What is the missing fact?",
|
||||||
|
answer: "Confirmed.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.selectedQuestion).toBeNull();
|
||||||
|
expect(result.finalQuestion).toBeNull();
|
||||||
|
expect(result.newActiveUnknownNodeId).toBeNull();
|
||||||
|
expect(result.noQuestionReason).toBe(
|
||||||
|
"No unresolved unknown candidates remain after this update.",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not allow a decision-mode active unknown to remain a comparison child", () => {
|
||||||
|
const graph = makeCommercialUpdateFixture();
|
||||||
|
graph.nodes.push(
|
||||||
|
makeNode({
|
||||||
|
id: "n-commercial-comparison-child",
|
||||||
|
label: "How the two observations were measured",
|
||||||
|
description:
|
||||||
|
"Need evidence about the measure used for each observation before comparing them.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
parentId: "n-commercial-parent",
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
graph.activeUnknownNodeId = "n-commercial-comparison-child";
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeMeaningfulNoOpProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.reasoningPatternValidation).toMatchObject({
|
||||||
|
activePattern: "decision",
|
||||||
|
valid: true,
|
||||||
|
});
|
||||||
|
expect(result.graphReasoningIntegrity).toBe("valid");
|
||||||
|
expect(result.incompatibleNodeIds).toContain(
|
||||||
|
"n-commercial-comparison-child",
|
||||||
|
);
|
||||||
|
expect(result.compatibilityFailures).toEqual(
|
||||||
|
expect.arrayContaining([
|
||||||
|
expect.objectContaining({
|
||||||
|
nodeId: "n-commercial-comparison-child",
|
||||||
|
activePattern: "decision",
|
||||||
|
nodePattern: "comparison",
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
expect(result.replacementActions).toEqual(
|
||||||
|
expect.arrayContaining([
|
||||||
|
expect.objectContaining({
|
||||||
|
rejectedNodeId: "n-commercial-comparison-child",
|
||||||
|
replacementNodeId: result.selectedQuestion?.nodeId,
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion?.nodeId).not.toBe(
|
||||||
|
"n-commercial-comparison-child",
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion?.reasoningPattern).toBe("decision");
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||||
|
import { applyValidatedProposal } from "@/lib/graph/apply-proposal.js";
|
||||||
import { validateGraphReferences } from "@/lib/graph/utils.js";
|
import { validateGraphReferences } from "@/lib/graph/utils.js";
|
||||||
import { makeGraph, makeNode } from "@/lib/graph/schema.js";
|
import { makeGraph, makeNode } from "@/lib/graph/schema.js";
|
||||||
|
|
||||||
@@ -56,6 +57,107 @@ function makeAnalysisResult(overrides = {}) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function makeCommercialAnalysisResult(overrides = {}) {
|
||||||
|
return makeAnalysisResult({
|
||||||
|
reconstruction: {
|
||||||
|
summary:
|
||||||
|
"A new reasoning method may become a commercial product, but problem existence and value remain unresolved.",
|
||||||
|
actors: [],
|
||||||
|
systemsOrObjects: [],
|
||||||
|
expectedStates: [],
|
||||||
|
observedStates: [],
|
||||||
|
differences: [],
|
||||||
|
knownTransitions: [],
|
||||||
|
unexplainedTransitions: [],
|
||||||
|
contradictions: [],
|
||||||
|
importantUnknowns: [
|
||||||
|
{
|
||||||
|
id: "unk-commercial",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
confidence: "high",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
plausibleInterpretations: [],
|
||||||
|
},
|
||||||
|
nextQuestion: {
|
||||||
|
id: "q-commercial",
|
||||||
|
question:
|
||||||
|
"What specific validation metrics, pilot feedback, or competitive benchmarking results have you collected to measure whether the method solves a recognized problem and how target users evaluate its practical utility compared to existing tools?",
|
||||||
|
reason: "Model-proposed broad validation question",
|
||||||
|
},
|
||||||
|
...overrides,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeCommercialTieAnalysisResult(overrides = {}) {
|
||||||
|
return makeAnalysisResult({
|
||||||
|
reconstruction: {
|
||||||
|
summary:
|
||||||
|
"A new reasoning method may become a commercial product, but multiple broad decision unknowns remain unresolved.",
|
||||||
|
actors: [],
|
||||||
|
systemsOrObjects: [],
|
||||||
|
expectedStates: [],
|
||||||
|
observedStates: [],
|
||||||
|
differences: [],
|
||||||
|
knownTransitions: [],
|
||||||
|
unexplainedTransitions: [],
|
||||||
|
contradictions: [],
|
||||||
|
importantUnknowns: [
|
||||||
|
{
|
||||||
|
id: "unk-fit-pay",
|
||||||
|
label:
|
||||||
|
"Evidence of genuine problem-solution fit and actual willingness to pay among target users",
|
||||||
|
description:
|
||||||
|
"Need to know whether there is real problem-solution fit and willingness to pay among target users before continuing development.",
|
||||||
|
confidence: "high",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "unk-distinction",
|
||||||
|
label:
|
||||||
|
"Clear, measurable distinction between the method and existing AI tools that justifies separate commercial value",
|
||||||
|
description:
|
||||||
|
"Need to know whether there is a clear measurable distinction from existing AI tools before continuing development.",
|
||||||
|
confidence: "high",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
plausibleInterpretations: [],
|
||||||
|
},
|
||||||
|
nextQuestion: {
|
||||||
|
id: "q-commercial-tie",
|
||||||
|
question:
|
||||||
|
"What specific validation metrics, pilot feedback, or competitive benchmarking results have you collected?",
|
||||||
|
reason: "Model-proposed broad validation question",
|
||||||
|
},
|
||||||
|
...overrides,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeCommercialUpdateGraph() {
|
||||||
|
const parent = makeNode({
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
return makeGraph({
|
||||||
|
centralStatement:
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.",
|
||||||
|
nodes: [parent],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: parent.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Commercial update scenario",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
function makeUpdateGraph() {
|
function makeUpdateGraph() {
|
||||||
const unknown = makeNode({
|
const unknown = makeNode({
|
||||||
id: "n-unknown",
|
id: "n-unknown",
|
||||||
@@ -401,9 +503,15 @@ describe("lib/graph/orchestrator startCase", () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it("returns null selectedQuestion when analysis has no nextQuestion", async () => {
|
it("returns null selectedQuestion when neither analysis nor graph path yields a question", async () => {
|
||||||
mockAnalyseScenario.mockResolvedValue(
|
mockAnalyseScenario.mockResolvedValue(
|
||||||
makeAnalysisResult({ nextQuestion: undefined }),
|
makeAnalysisResult({
|
||||||
|
nextQuestion: undefined,
|
||||||
|
reconstruction: {
|
||||||
|
...makeAnalysisResult().reconstruction,
|
||||||
|
importantUnknowns: [],
|
||||||
|
},
|
||||||
|
}),
|
||||||
);
|
);
|
||||||
const { startCase } = await import("@/lib/graph/orchestrator.js");
|
const { startCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
|
|
||||||
@@ -413,6 +521,73 @@ describe("lib/graph/orchestrator startCase", () => {
|
|||||||
expect(result.selectedQuestion).toBeNull();
|
expect(result.selectedQuestion).toBeNull();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("uses the graph-backed question path instead of the reconstruction nextQuestion on startCase", async () => {
|
||||||
|
mockAnalyseScenario.mockResolvedValue(makeCommercialAnalysisResult());
|
||||||
|
const { startCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
|
|
||||||
|
const result = await startCase({
|
||||||
|
scenario:
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.selectedQuestion?.question).toBe(
|
||||||
|
"Who experiences this problem?",
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion?.nodeId).toBe(
|
||||||
|
result.situationGraph.activeUnknownNodeId,
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion?.question).not.toContain(
|
||||||
|
"validation metrics, pilot feedback, or competitive benchmarking",
|
||||||
|
);
|
||||||
|
expect(result.diagnostics.reconstructionQuestion).toContain(
|
||||||
|
"validation metrics, pilot feedback, or competitive benchmarking",
|
||||||
|
);
|
||||||
|
expect(result.diagnostics.reconstructionQuestionAccepted).toBe(false);
|
||||||
|
expect(result.diagnostics.reconstructionQuestionRejectionReasons).toContain(
|
||||||
|
"graph_backed_pipeline_required",
|
||||||
|
);
|
||||||
|
expect(result.diagnostics.finalGraphBackedQuestion).toBe(
|
||||||
|
"Who experiences this problem?",
|
||||||
|
);
|
||||||
|
expect(result.diagnostics.selectedUnknownNodeId).toBe(
|
||||||
|
result.situationGraph.activeUnknownNodeId,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reselects and decomposes a tied commercial start-case candidate instead of returning a silent null question", async () => {
|
||||||
|
mockAnalyseScenario.mockResolvedValue(makeCommercialTieAnalysisResult());
|
||||||
|
const { startCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
|
|
||||||
|
const result = await startCase({
|
||||||
|
scenario:
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(
|
||||||
|
result.situationGraph.nodes.some((node) => node.kind === "unknown"),
|
||||||
|
).toBe(true);
|
||||||
|
expect(result.selectedQuestion).not.toBeNull();
|
||||||
|
expect(result.selectedQuestion?.question).toBe(
|
||||||
|
"Who experiences this problem?",
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion?.nodeId).toBe(
|
||||||
|
result.situationGraph.activeUnknownNodeId,
|
||||||
|
);
|
||||||
|
expect(result.diagnostics.noQuestionReason).toBeNull();
|
||||||
|
expect(result.diagnostics.selectedContainerUnknown).toBeTruthy();
|
||||||
|
expect(result.diagnostics.selectedChildUnknown).toBeTruthy();
|
||||||
|
expect(result.diagnostics.decompositionApplied).toBe(true);
|
||||||
|
expect(result.diagnostics.reasoningPatternValidation).toMatchObject({
|
||||||
|
activePattern: "decision",
|
||||||
|
valid: true,
|
||||||
|
});
|
||||||
|
expect(result.selectedQuestion?.question.toLowerCase()).not.toMatch(
|
||||||
|
/pay|price|pricing|budget|benchmark/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
it("includes compatibility diagnostics when provided by analysis", async () => {
|
it("includes compatibility diagnostics when provided by analysis", async () => {
|
||||||
mockAnalyseScenario.mockResolvedValue(
|
mockAnalyseScenario.mockResolvedValue(
|
||||||
makeAnalysisResult({
|
makeAnalysisResult({
|
||||||
@@ -908,6 +1083,77 @@ describe("lib/graph/orchestrator startCase", () => {
|
|||||||
expect(result.selectedQuestion?.nodeId).toBe("n-value");
|
expect(result.selectedQuestion?.nodeId).toBe("n-value");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("keeps a follow-up question when a resolved child still has an eligible sibling", async () => {
|
||||||
|
const { updateCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
|
const initialGraph = makeCommercialUpdateGraph();
|
||||||
|
const firstPass = applyValidatedProposal({
|
||||||
|
situationGraph: initialGraph,
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-anchor",
|
||||||
|
label: "Update anchor",
|
||||||
|
description:
|
||||||
|
"Anchor state introduced by the answer because the update must contain a meaningful change.",
|
||||||
|
kind: "state",
|
||||||
|
status: "known",
|
||||||
|
confidence: "low",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
updatedNodes: [],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(firstPass.success).toBe(true);
|
||||||
|
|
||||||
|
const provider = {
|
||||||
|
generateReconstruction: vi.fn().mockResolvedValue({
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [
|
||||||
|
{
|
||||||
|
nodeId: firstPass.selectedQuestion.nodeId,
|
||||||
|
previousStatus: "unknown",
|
||||||
|
newStatus: "resolved",
|
||||||
|
previousValue: null,
|
||||||
|
newValue:
|
||||||
|
"I experience it myself when deciding whether a project or investment is justified.",
|
||||||
|
reason: "The answer resolves the first child.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [firstPass.selectedQuestion.nodeId],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = await updateCase(
|
||||||
|
{
|
||||||
|
situationGraph: firstPass.updatedSituationGraph,
|
||||||
|
previousQuestion: firstPass.selectedQuestion.question,
|
||||||
|
answer:
|
||||||
|
"I experience it myself when deciding whether a project or investment is justified.",
|
||||||
|
promptVersion: "v0.4",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
provider,
|
||||||
|
config: MOCK_CONFIG,
|
||||||
|
applyProposal: true,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.selectedQuestion).toBeTruthy();
|
||||||
|
expect(result.newActiveUnknownNodeId).toBe(result.selectedQuestion?.nodeId);
|
||||||
|
expect(result.diagnostics.noQuestionReason).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
it("defaults to proposal-only mode", async () => {
|
it("defaults to proposal-only mode", async () => {
|
||||||
const { updateCase } = await import("@/lib/graph/orchestrator.js");
|
const { updateCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
const applyValidatedProposal = vi.fn();
|
const applyValidatedProposal = vi.fn();
|
||||||
@@ -1093,16 +1339,113 @@ describe("lib/graph/orchestrator startCase", () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
it("startCase behaviour remains unchanged", async () => {
|
it("reselects the other-people sibling after the first commercial child is answered", async () => {
|
||||||
|
const { updateCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
|
|
||||||
|
const seededGraph = applyValidatedProposal({
|
||||||
|
situationGraph: makeCommercialUpdateGraph(),
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-anchor",
|
||||||
|
label: "Update anchor",
|
||||||
|
description:
|
||||||
|
"Anchor state introduced by the answer because the update must contain a meaningful change.",
|
||||||
|
kind: "state",
|
||||||
|
status: "known",
|
||||||
|
confidence: "low",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
updatedNodes: [],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(seededGraph.success).toBe(true);
|
||||||
|
const resolvedFirstChildNodeId = seededGraph.selectedQuestion?.nodeId;
|
||||||
|
const resolvedFirstQuestion = seededGraph.selectedQuestion?.question;
|
||||||
|
|
||||||
|
const initial = await updateCase(
|
||||||
|
{
|
||||||
|
situationGraph: seededGraph.updatedSituationGraph,
|
||||||
|
previousQuestion: resolvedFirstQuestion,
|
||||||
|
answer:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
promptVersion: "v0.4",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
applyProposal: true,
|
||||||
|
config: MOCK_CONFIG,
|
||||||
|
provider: {
|
||||||
|
generateReconstruction: vi.fn().mockResolvedValue({
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [
|
||||||
|
{
|
||||||
|
nodeId: resolvedFirstChildNodeId,
|
||||||
|
previousStatus: "unknown",
|
||||||
|
newStatus: "resolved",
|
||||||
|
previousValue: null,
|
||||||
|
newValue:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
reason: "The answer confirms a self-observed instance.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [resolvedFirstChildNodeId],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(initial.success).toBe(true);
|
||||||
|
expect(initial.selectedQuestion?.nodeId).toBe(
|
||||||
|
initial.newActiveUnknownNodeId,
|
||||||
|
);
|
||||||
|
expect(initial.selectedQuestion?.question).toBe(
|
||||||
|
"What makes you think other people experience this problem too?",
|
||||||
|
);
|
||||||
|
expect(initial.selectedQuestion?.reasoningPattern).toBe("decision");
|
||||||
|
expect(initial.diagnostics.unresolvedCandidateCount).toBeGreaterThan(0);
|
||||||
|
expect(initial.diagnostics.eligibleCandidateCount).toBeGreaterThan(0);
|
||||||
|
expect(initial.diagnostics.candidateNodeIds).toContain(
|
||||||
|
initial.selectedQuestion?.nodeId,
|
||||||
|
);
|
||||||
|
expect(initial.diagnostics.resolvedCurrentTurnNodeIds).toContain(
|
||||||
|
resolvedFirstChildNodeId,
|
||||||
|
);
|
||||||
|
expect(initial.diagnostics.noQuestionReason).toBeNull();
|
||||||
|
expect(initial.diagnostics.reasoningPatternValidation).toMatchObject({
|
||||||
|
activePattern: "decision",
|
||||||
|
valid: true,
|
||||||
|
});
|
||||||
|
expect(initial.diagnostics.graphReasoningIntegrity).toBe("valid");
|
||||||
|
expect(initial.diagnostics.incompatibleNodeIds).toEqual([]);
|
||||||
|
expect(initial.selectedQuestion?.question.toLowerCase()).not.toMatch(
|
||||||
|
/price|budget|market size|pilot metrics|benchmark|technical differentiation/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("startCase no longer copies analysis nextQuestion directly when a graph-backed question exists", async () => {
|
||||||
mockAnalyseScenario.mockResolvedValue(makeAnalysisResult());
|
mockAnalyseScenario.mockResolvedValue(makeAnalysisResult());
|
||||||
const { startCase } = await import("@/lib/graph/orchestrator.js");
|
const { startCase } = await import("@/lib/graph/orchestrator.js");
|
||||||
|
|
||||||
const result = await startCase({ scenario: "Scenario text" });
|
const result = await startCase({ scenario: "Scenario text" });
|
||||||
|
|
||||||
expect(result.success).toBe(true);
|
expect(result.success).toBe(true);
|
||||||
expect(result.selectedQuestion).toEqual({
|
expect(result.selectedQuestion).toMatchObject({
|
||||||
id: "q-1",
|
nodeId: result.situationGraph.activeUnknownNodeId,
|
||||||
question: "What denominator is being used for the complaint rate?",
|
question:
|
||||||
|
"What would clarify need the denominator for complaint rate in this situation?",
|
||||||
});
|
});
|
||||||
|
expect(result.diagnostics.reconstructionQuestion).toBe(
|
||||||
|
"What denominator is being used for the complaint rate?",
|
||||||
|
);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import {
|
|||||||
assessUnknownAtomicity,
|
assessUnknownAtomicity,
|
||||||
formulateQuestion,
|
formulateQuestion,
|
||||||
formulateTieResolutionQuestion,
|
formulateTieResolutionQuestion,
|
||||||
|
selectReasoningPattern,
|
||||||
selectInvestigationStrategy,
|
selectInvestigationStrategy,
|
||||||
} from "@/lib/graph/question-formulator.js";
|
} from "@/lib/graph/question-formulator.js";
|
||||||
import { makeEdge, makeGraph, makeNode } from "@/lib/graph/schema.js";
|
import { makeEdge, makeGraph, makeNode } from "@/lib/graph/schema.js";
|
||||||
@@ -105,6 +106,8 @@ describe("formulateQuestion", () => {
|
|||||||
const result = formulateQuestion({ node: unknown, graph });
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
expect(result.strategy).toBe("decision_threshold");
|
expect(result.strategy).toBe("decision_threshold");
|
||||||
|
expect(result.reasoningPattern).toBe("decision");
|
||||||
|
expect(result.questionFamily).toBe("decision_threshold");
|
||||||
expect(result.question).toContain("What outcome");
|
expect(result.question).toContain("What outcome");
|
||||||
expect(result.question.toLowerCase()).toContain("justify");
|
expect(result.question.toLowerCase()).toContain("justify");
|
||||||
});
|
});
|
||||||
@@ -144,6 +147,8 @@ describe("formulateQuestion", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
expect(result.strategy).toBe("definition");
|
expect(result.strategy).toBe("definition");
|
||||||
|
expect(result.reasoningPattern).toBe("definition");
|
||||||
|
expect(result.questionFamily).toBe("definition");
|
||||||
expect(result.question).toMatch(/^What does /);
|
expect(result.question).toMatch(/^What does /);
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -213,9 +218,68 @@ describe("formulateQuestion", () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
expect(result.strategy).toBe("contradiction_resolution");
|
expect(result.strategy).toBe("contradiction_resolution");
|
||||||
|
expect(result.reasoningPattern).toBe("contradiction");
|
||||||
expect(result.question).toContain("resolve the contradiction");
|
expect(result.question).toContain("resolve the contradiction");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("reasoning pattern selection marks commercial validation as decision rather than explanation", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-commercial-pattern",
|
||||||
|
label:
|
||||||
|
"Whether the method addresses a genuine, high-priority problem for a specific audience",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a real problem for a clear audience before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "high",
|
||||||
|
});
|
||||||
|
const graph = makeGraphFor(unknown, {
|
||||||
|
centralStatement:
|
||||||
|
"Before investing more, we need to know whether continuing development is commercially justified.",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = selectReasoningPattern({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.pattern).toBe("decision");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("comparison scenario selects the comparison pattern", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-comparison-pattern",
|
||||||
|
label: "How the two observations were measured",
|
||||||
|
description:
|
||||||
|
"Need evidence about the measure used for each observation, because that could help explain the difference.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
const graph = makeGraphFor(unknown, {
|
||||||
|
centralStatement: "Traffic increased, but sales stayed flat.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-traffic-observation",
|
||||||
|
label: "Traffic increased.",
|
||||||
|
description: "Traffic increased.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-sales-observation",
|
||||||
|
label: "Sales stayed flat.",
|
||||||
|
description: "Sales stayed flat.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = selectReasoningPattern({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.pattern).toBe("comparison");
|
||||||
|
});
|
||||||
|
|
||||||
it("constraint unknown uses evidence-gathering within the fixed strategy set", () => {
|
it("constraint unknown uses evidence-gathering within the fixed strategy set", () => {
|
||||||
const unknown = makeNode({
|
const unknown = makeNode({
|
||||||
id: "n-constraint",
|
id: "n-constraint",
|
||||||
@@ -445,6 +509,7 @@ describe("formulateQuestion", () => {
|
|||||||
node: unknown,
|
node: unknown,
|
||||||
graph: makeGraphFor(unknown),
|
graph: makeGraphFor(unknown),
|
||||||
});
|
});
|
||||||
|
expect(result.reasoningPattern).toBe("diagnosis");
|
||||||
expect(result.strategy).toBeNull();
|
expect(result.strategy).toBeNull();
|
||||||
expect(result.question).toBe(
|
expect(result.question).toBe(
|
||||||
"What would clarify possible causes of the divergence in this situation?",
|
"What would clarify possible causes of the divergence in this situation?",
|
||||||
|
|||||||
@@ -0,0 +1,208 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
assessQuestionComplexity,
|
||||||
|
assessUnknownAtomicity,
|
||||||
|
formulateQuestion,
|
||||||
|
} from "@/lib/graph/question-formulator.js";
|
||||||
|
import { applyValidatedProposal } from "@/lib/graph/apply-proposal.js";
|
||||||
|
import { makeGraph, makeNode } from "@/lib/graph/schema.js";
|
||||||
|
|
||||||
|
const SCENARIO =
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.";
|
||||||
|
|
||||||
|
function makeCommercialValidationGraph() {
|
||||||
|
const parent = makeNode({
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
return makeGraph({
|
||||||
|
centralStatement: SCENARIO,
|
||||||
|
nodes: [parent],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: parent.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Commercial validation question simplicity fixture",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeMeaningfulNoOpProposal() {
|
||||||
|
return {
|
||||||
|
addedNodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-anchor",
|
||||||
|
label: "Update anchor",
|
||||||
|
description:
|
||||||
|
"Anchor state introduced by the answer because the update must contain a meaningful change.",
|
||||||
|
kind: "state",
|
||||||
|
status: "known",
|
||||||
|
confidence: "low",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
updatedNodes: [],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("question simplicity", () => {
|
||||||
|
it("rejects the original long compound financial-cost plus budget question", () => {
|
||||||
|
const graph = makeCommercialValidationGraph();
|
||||||
|
const unknown = graph.nodes.find(
|
||||||
|
(node) => node.id === "n-commercial-parent",
|
||||||
|
);
|
||||||
|
const question =
|
||||||
|
"Have you measured the current financial or operational cost to users who lack justified confidence, and what baseline budget do they currently allocate for comparable decision-support methods?";
|
||||||
|
|
||||||
|
const result = assessQuestionComplexity({
|
||||||
|
question,
|
||||||
|
selectedUnknown: unknown,
|
||||||
|
graph,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.acceptable).toBe(false);
|
||||||
|
expect(result.primaryConceptCount).toBeGreaterThan(1);
|
||||||
|
expect(result.cognitiveLoad).toBe("high");
|
||||||
|
expect(result.reasons).toContain("multiple_requested_answers");
|
||||||
|
expect(result.reasons).toContain("cost_and_budget_combined");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("accepts one simple concept question", () => {
|
||||||
|
const graph = makeCommercialValidationGraph();
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-who",
|
||||||
|
label: "Who experiences this problem",
|
||||||
|
description:
|
||||||
|
"Need to know who experiences this problem, because that must be clear before deciding whether it is commercially justified.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
parentId: "n-commercial-parent",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.question).toBe("Who experiences this problem?");
|
||||||
|
expect(result.questionComplexity.acceptable).toBe(true);
|
||||||
|
expect(result.questionComplexity.primaryConceptCount).toBe(1);
|
||||||
|
expect(result.question.match(/\?/g) || []).toHaveLength(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("classifies broad commercial-validation unknowns as composite", () => {
|
||||||
|
const graph = makeCommercialValidationGraph();
|
||||||
|
const unknown = graph.nodes.find(
|
||||||
|
(node) => node.id === "n-commercial-parent",
|
||||||
|
);
|
||||||
|
|
||||||
|
const result = assessUnknownAtomicity({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.atomicity).toBe("composite");
|
||||||
|
expect(result.decompositionKind).toBe("commercial_validation");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("decomposes a broad commercial-validation unknown instead of merely rewording it", () => {
|
||||||
|
const graph = makeCommercialValidationGraph();
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeMeaningfulNoOpProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.decompositionPerformed).toBe(true);
|
||||||
|
expect(result.selectedUnknownBefore).toBe("n-commercial-parent");
|
||||||
|
expect(result.selectedUnknownAfter).not.toBe("n-commercial-parent");
|
||||||
|
expect(result.childNodeIds.length).toBeGreaterThanOrEqual(2);
|
||||||
|
expect(result.selectedQuestion.nodeId).toBe(result.selectedUnknownAfter);
|
||||||
|
expect(result.selectedQuestion.question).toBe(
|
||||||
|
"Who experiences this problem?",
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion.reasoningPattern).toBe("decision");
|
||||||
|
expect(result.selectedQuestion.questionFamily).toBe("decision_foundation");
|
||||||
|
expect(result.selectedQuestion.selectedQuestionTemplate).toBe(
|
||||||
|
"decision_foundation_direct_child",
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion.question.match(/\?/g) || []).toHaveLength(1);
|
||||||
|
expect(result.questionComplexityAccepted).toBe(true);
|
||||||
|
expect(result.primaryConceptCount).toBe(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("selects a foundational child rather than price or budget", () => {
|
||||||
|
const graph = makeCommercialValidationGraph();
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeMeaningfulNoOpProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
const selectedNode = result.updatedSituationGraph.nodes.find(
|
||||||
|
(node) => node.id === result.selectedUnknownAfter,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(selectedNode.label).toBe("Who experiences this problem");
|
||||||
|
expect(selectedNode.label.toLowerCase()).not.toMatch(/pay|price|budget/);
|
||||||
|
expect(result.selectedQuestion.question.toLowerCase()).not.toMatch(
|
||||||
|
/pay|price|budget/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("plain-language replacements simplify formal phrasing when safe", () => {
|
||||||
|
const graph = makeCommercialValidationGraph();
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-actor-formal",
|
||||||
|
label: "Relevant customer or user",
|
||||||
|
description:
|
||||||
|
"Need to identify the relevant customer, user, or value recipient because the decision depends on it.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
|
const complexity = assessQuestionComplexity({
|
||||||
|
question:
|
||||||
|
"What evidence would clarify the relevant customer, user, or value recipient?",
|
||||||
|
selectedUnknown: unknown,
|
||||||
|
graph,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(complexity.acceptable).toBe(false);
|
||||||
|
expect(complexity.reasons).toContain("list_like_question");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("does not remove scenario-relevant jargon blindly", () => {
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement:
|
||||||
|
"The team is deciding whether to continue a decision-support product.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-jargon",
|
||||||
|
label: "Decision support methods",
|
||||||
|
description:
|
||||||
|
"Need evidence about decision support methods because the comparison depends on it.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: "n-jargon",
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Jargon retention fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateQuestion({ node: graph.nodes[0], graph });
|
||||||
|
|
||||||
|
expect(result.question.toLowerCase()).toContain("decision support methods");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,201 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
formulateQuestion,
|
||||||
|
formulateTieResolutionQuestion,
|
||||||
|
selectReasoningPattern,
|
||||||
|
} from "@/lib/graph/question-formulator.js";
|
||||||
|
import { makeGraph, makeNode } from "@/lib/graph/schema.js";
|
||||||
|
|
||||||
|
function makeGraphFor(node, extra = {}) {
|
||||||
|
return makeGraph({
|
||||||
|
centralStatement: extra.centralStatement || "Decision context",
|
||||||
|
nodes: [node, ...(extra.nodes || [])],
|
||||||
|
edges: extra.edges || [],
|
||||||
|
activeUnknownNodeId: node.id,
|
||||||
|
resolvedNodeIds: extra.resolvedNodeIds || [],
|
||||||
|
currentSummary: "Test summary",
|
||||||
|
reasoningState: extra.reasoningState,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("reasoning pattern selection", () => {
|
||||||
|
it("commercial-method scenario selects the decision pattern and rejects explanation family", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-commercial-method",
|
||||||
|
label:
|
||||||
|
"Whether the method addresses a genuine, high-priority problem for a specific audience",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a real problem for a clear audience before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "high",
|
||||||
|
});
|
||||||
|
const graph = makeGraphFor(unknown, {
|
||||||
|
centralStatement:
|
||||||
|
"Before investing significant time and money, we need to know whether continuing development is commercially justified.",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.reasoningPattern).toBe("decision");
|
||||||
|
expect(result.questionFamily).not.toBe("explanation");
|
||||||
|
expect(result.allowedQuestionFamilies).toContain("decision_foundation");
|
||||||
|
expect(result.rejectedQuestionFamilies).toContain("explanation");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("revenue and cash relationship scenario allows the explanation family", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-revenue-cash-explanation",
|
||||||
|
label:
|
||||||
|
"Explanation for why revenue increased by 18%, but cash in the bank fell over the same period",
|
||||||
|
description:
|
||||||
|
"Need to understand what change or event could explain why these observations differ.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
const graph = makeGraphFor(unknown, {
|
||||||
|
centralStatement:
|
||||||
|
"Revenue increased by 18%, but cash in the bank fell over the same period.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-revenue-observation",
|
||||||
|
label: "Revenue increased by 18%.",
|
||||||
|
description: "Revenue increased by 18%.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-cash-observation",
|
||||||
|
label: "Cash in the bank fell over the same period.",
|
||||||
|
description: "Cash in the bank fell over the same period.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.reasoningPattern).toBe("explanation");
|
||||||
|
expect(result.allowedQuestionFamilies).toContain("explanation");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("duplicate observations reject explanation family during tie resolution", () => {
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement: "The same figure was repeated twice.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-obs-1",
|
||||||
|
label: "Revenue increased by 10%.",
|
||||||
|
description: "Revenue increased by 10%.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-obs-2",
|
||||||
|
label: "Revenue increased by 10%.",
|
||||||
|
description: "Revenue increased by 10%.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: null,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Duplicate observation fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateTieResolutionQuestion({ graph });
|
||||||
|
|
||||||
|
expect(result.questionFamily).not.toBe("explanation");
|
||||||
|
expect(result.rejectedQuestionFamilies).toContain("explanation");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("definition scenario selects the definition pattern", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-definition-pattern",
|
||||||
|
label: "Definition of justified confidence",
|
||||||
|
description: "The term needs clearer boundaries.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = selectReasoningPattern({
|
||||||
|
node: unknown,
|
||||||
|
graph: makeGraphFor(unknown),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.pattern).toBe("definition");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("comparison scenario selects the comparison pattern", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-comparison-pattern-2",
|
||||||
|
label: "How the two observations were measured",
|
||||||
|
description:
|
||||||
|
"Need evidence about the measure used for each observation before comparing them.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
const graph = makeGraphFor(unknown, {
|
||||||
|
centralStatement: "Traffic increased, but sales stayed flat.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-traffic-2",
|
||||||
|
label: "Traffic increased.",
|
||||||
|
description: "Traffic increased.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-sales-2",
|
||||||
|
label: "Sales stayed flat.",
|
||||||
|
description: "Sales stayed flat.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
reasoningState: {
|
||||||
|
comparabilityStatus: "uncertain",
|
||||||
|
relationshipStatus: "insufficient_information",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = selectReasoningPattern({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.pattern).toBe("comparison");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("question family stays compatible with the reasoning pattern", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-decision-family-compatibility",
|
||||||
|
label: "Who experiences this problem",
|
||||||
|
description:
|
||||||
|
"Need to know who experiences this problem before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
const graph = makeGraphFor(unknown, {
|
||||||
|
centralStatement:
|
||||||
|
"We need to know whether continuing development is commercially justified.",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.reasoningPattern).toBe("decision");
|
||||||
|
expect(result.allowedQuestionFamilies).toContain(result.questionFamily);
|
||||||
|
expect(result.rejectedQuestionFamilies).not.toContain(
|
||||||
|
result.questionFamily,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,282 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import { applyValidatedProposal } from "@/lib/graph/apply-proposal.js";
|
||||||
|
import {
|
||||||
|
classifyObservationRelationship,
|
||||||
|
formulateQuestion,
|
||||||
|
selectReasoningPattern,
|
||||||
|
} from "@/lib/graph/question-formulator.js";
|
||||||
|
import { makeGraph, makeNode } from "@/lib/graph/schema.js";
|
||||||
|
|
||||||
|
const COMMERCIAL_SCENARIO =
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.";
|
||||||
|
|
||||||
|
function makeCommercialGraph() {
|
||||||
|
const parent = makeNode({
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
return makeGraph({
|
||||||
|
centralStatement: COMMERCIAL_SCENARIO,
|
||||||
|
nodes: [parent],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: parent.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Commercial pattern fixture",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeSeedProposal() {
|
||||||
|
return {
|
||||||
|
addedNodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-anchor",
|
||||||
|
label: "Update anchor",
|
||||||
|
description:
|
||||||
|
"Anchor state introduced by the answer because the update must contain a meaningful change.",
|
||||||
|
kind: "state",
|
||||||
|
status: "known",
|
||||||
|
confidence: "low",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
updatedNodes: [],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("reasoning-pattern validation", () => {
|
||||||
|
it("keeps the commercial-method scenario in decision mode without comparability-style unknowns", () => {
|
||||||
|
const seeded = applyValidatedProposal({
|
||||||
|
situationGraph: makeCommercialGraph(),
|
||||||
|
proposal: makeSeedProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(seeded.success).toBe(true);
|
||||||
|
expect(seeded.selectedQuestion?.reasoningPattern).toBe("decision");
|
||||||
|
expect(
|
||||||
|
seeded.updatedSituationGraph.nodes.some((node) =>
|
||||||
|
/two observations|measured|different timing/i.test(node.label),
|
||||||
|
),
|
||||||
|
).toBe(false);
|
||||||
|
|
||||||
|
const followUp = applyValidatedProposal({
|
||||||
|
situationGraph: seeded.updatedSituationGraph,
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [
|
||||||
|
{
|
||||||
|
nodeId: seeded.selectedQuestion.nodeId,
|
||||||
|
previousStatus: "unknown",
|
||||||
|
newStatus: "resolved",
|
||||||
|
previousValue: null,
|
||||||
|
newValue:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
reason: "Answered by the user.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [seeded.selectedQuestion.nodeId],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
previousQuestion: seeded.selectedQuestion.question,
|
||||||
|
answer:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(followUp.success).toBe(true);
|
||||||
|
expect(followUp.selectedQuestion?.reasoningPattern).toBe("decision");
|
||||||
|
expect(followUp.reasoningPatternValidation).toMatchObject({
|
||||||
|
activePattern: "decision",
|
||||||
|
valid: true,
|
||||||
|
});
|
||||||
|
expect(followUp.graphReasoningIntegrity).toBe("valid");
|
||||||
|
expect(followUp.incompatibleNodeIds).toEqual([]);
|
||||||
|
expect(followUp.selectedQuestion?.question.toLowerCase()).not.toMatch(
|
||||||
|
/two observations|measured|same basis|same scale|different timing/,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("allows comparability-style reasoning in an explanation scenario", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-explanation",
|
||||||
|
label:
|
||||||
|
"Explanation for why revenue increased by 18%, but cash in the bank fell over the same period",
|
||||||
|
description:
|
||||||
|
"Need to understand what change or event could explain why these observations differ.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement:
|
||||||
|
"Revenue increased by 18%, but cash in the bank fell over the same period.",
|
||||||
|
nodes: [
|
||||||
|
unknown,
|
||||||
|
makeNode({
|
||||||
|
id: "n-revenue",
|
||||||
|
label: "Revenue increased by 18%.",
|
||||||
|
description: "Revenue increased by 18%.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-cash",
|
||||||
|
label: "Cash in the bank fell over the same period.",
|
||||||
|
description: "Cash in the bank fell over the same period.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: unknown.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Explanation fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const pattern = selectReasoningPattern({ node: unknown, graph });
|
||||||
|
const question = formulateQuestion({ node: unknown, graph });
|
||||||
|
const relationship = classifyObservationRelationship(graph);
|
||||||
|
|
||||||
|
expect(pattern.pattern).toBe("explanation");
|
||||||
|
expect(question.reasoningPattern).toBe("explanation");
|
||||||
|
expect(relationship.questionRequired).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects explanation-family tie resolution for duplicate observations", () => {
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement: "Sales doubled. Sales doubled.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-sales-1",
|
||||||
|
label: "Sales doubled.",
|
||||||
|
description: "Sales doubled.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-sales-2",
|
||||||
|
label: "Sales doubled.",
|
||||||
|
description: "Sales doubled.",
|
||||||
|
kind: "observation",
|
||||||
|
status: "supported",
|
||||||
|
confidence: "high",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: null,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Duplicate observation fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const relationship = classifyObservationRelationship(graph);
|
||||||
|
|
||||||
|
expect(relationship.relationshipStatus).toBe("duplicate");
|
||||||
|
expect(relationship.questionRequired).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps definition scenarios inside compatible node families", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-definition",
|
||||||
|
label: "Definition of justified confidence",
|
||||||
|
description: "The term needs clearer boundaries.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement: "The team uses justified confidence inconsistently.",
|
||||||
|
nodes: [unknown],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: unknown.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Definition fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const pattern = selectReasoningPattern({ node: unknown, graph });
|
||||||
|
const result = formulateQuestion({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(pattern.pattern).toBe("definition");
|
||||||
|
expect(result.reasoningPattern).toBe("definition");
|
||||||
|
expect(result.allowedQuestionFamilies).toEqual(["definition"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("replaces an incompatible decision-mode active unknown with a pattern-compatible candidate", () => {
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement:
|
||||||
|
"Before investing more, we need to know whether continuing development is commercially justified.",
|
||||||
|
nodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
}),
|
||||||
|
makeNode({
|
||||||
|
id: "n-incompatible-child",
|
||||||
|
label: "How the two observations were measured",
|
||||||
|
description:
|
||||||
|
"Need evidence about the measure used for each observation before comparing them.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
parentId: "n-commercial-parent",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: "n-incompatible-child",
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Incompatible active unknown fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeSeedProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.reasoningPatternValidation).toMatchObject({
|
||||||
|
activePattern: "decision",
|
||||||
|
valid: true,
|
||||||
|
});
|
||||||
|
expect(result.graphReasoningIntegrity).toBe("valid");
|
||||||
|
expect(result.incompatibleNodeIds).toContain("n-incompatible-child");
|
||||||
|
expect(result.compatibilityFailures).toEqual(
|
||||||
|
expect.arrayContaining([
|
||||||
|
expect.objectContaining({
|
||||||
|
nodeId: "n-incompatible-child",
|
||||||
|
activePattern: "decision",
|
||||||
|
nodePattern: "comparison",
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
expect(result.replacementActions).toEqual(
|
||||||
|
expect.arrayContaining([
|
||||||
|
expect.objectContaining({
|
||||||
|
rejectedNodeId: "n-incompatible-child",
|
||||||
|
replacementNodeId: result.selectedQuestion?.nodeId,
|
||||||
|
}),
|
||||||
|
]),
|
||||||
|
);
|
||||||
|
expect(result.selectedQuestion?.nodeId).not.toBe("n-incompatible-child");
|
||||||
|
expect(result.selectedQuestion?.reasoningPattern).toBe("decision");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
import { describe, expect, it } from "vitest";
|
||||||
|
import {
|
||||||
|
assessUnknownAnswerability,
|
||||||
|
assessUnknownAtomicity,
|
||||||
|
} from "@/lib/graph/question-formulator.js";
|
||||||
|
import { applyValidatedProposal } from "@/lib/graph/apply-proposal.js";
|
||||||
|
import { makeGraph, makeNode } from "@/lib/graph/schema.js";
|
||||||
|
|
||||||
|
const COMMERCIAL_SCENARIO =
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.";
|
||||||
|
|
||||||
|
function makeMeaningfulNoOpProposal() {
|
||||||
|
return {
|
||||||
|
addedNodes: [
|
||||||
|
makeNode({
|
||||||
|
id: "n-anchor",
|
||||||
|
label: "Update anchor",
|
||||||
|
description:
|
||||||
|
"Anchor state introduced by the answer because the update must contain a meaningful change.",
|
||||||
|
kind: "state",
|
||||||
|
status: "known",
|
||||||
|
confidence: "low",
|
||||||
|
}),
|
||||||
|
],
|
||||||
|
updatedNodes: [],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: [],
|
||||||
|
affectedNodeIds: [],
|
||||||
|
selectedQuestion: null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeCommercialContainerGraph() {
|
||||||
|
const parent = makeNode({
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
});
|
||||||
|
|
||||||
|
return makeGraph({
|
||||||
|
centralStatement: COMMERCIAL_SCENARIO,
|
||||||
|
nodes: [parent],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: parent.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Commercial answerability fixture",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
describe("assessUnknownAnswerability", () => {
|
||||||
|
it("flags commercial-validation container unknowns as non-answerable", () => {
|
||||||
|
const graph = makeCommercialContainerGraph();
|
||||||
|
const unknown = graph.nodes[0];
|
||||||
|
|
||||||
|
const atomicity = assessUnknownAtomicity({ node: unknown, graph });
|
||||||
|
const answerability = assessUnknownAnswerability({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(atomicity.atomicity).toBe("composite");
|
||||||
|
expect(answerability.independentlyAnswerable).toBe(false);
|
||||||
|
expect(answerability.decompositionRequired).toBe(true);
|
||||||
|
expect(answerability.prerequisiteConceptCount).toBeGreaterThan(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps one-concept denominator unknowns independently answerable", () => {
|
||||||
|
const unknown = makeNode({
|
||||||
|
id: "n-denominator",
|
||||||
|
label: "Complaint rate denominator",
|
||||||
|
description:
|
||||||
|
"Need the denominator because it directly determines the complaint rate.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "high",
|
||||||
|
});
|
||||||
|
const graph = makeGraph({
|
||||||
|
centralStatement: "Production increased while complaints increased.",
|
||||||
|
nodes: [unknown],
|
||||||
|
edges: [],
|
||||||
|
activeUnknownNodeId: unknown.id,
|
||||||
|
resolvedNodeIds: [],
|
||||||
|
currentSummary: "Denominator answerability fixture",
|
||||||
|
});
|
||||||
|
|
||||||
|
const result = assessUnknownAnswerability({ node: unknown, graph });
|
||||||
|
|
||||||
|
expect(result.independentlyAnswerable).toBe(true);
|
||||||
|
expect(result.decompositionRequired).toBe(false);
|
||||||
|
expect(result.prerequisiteConceptCount).toBeLessThanOrEqual(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("answerability-triggered decomposition", () => {
|
||||||
|
it("decomposes a non-answerable parent into independently answerable child investigations", () => {
|
||||||
|
const graph = makeCommercialContainerGraph();
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeMeaningfulNoOpProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.decompositionPerformed).toBe(true);
|
||||||
|
expect(result.decompositionTriggeredByAnswerability).toBe(true);
|
||||||
|
expect(result.selectedContainerUnknown).toBe("n-commercial-parent");
|
||||||
|
expect(result.selectedChildUnknown).toBe(result.selectedUnknownAfter);
|
||||||
|
expect(result.independentlyAnswerable).toBe(false);
|
||||||
|
expect(result.prerequisiteConceptCount).toBeGreaterThan(1);
|
||||||
|
expect(result.selectedUnknownAfter).not.toBe("n-commercial-parent");
|
||||||
|
expect(result.selectedQuestion.question).toBe(
|
||||||
|
"Who experiences this problem?",
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps the parent unresolved while selecting a child unknown", () => {
|
||||||
|
const graph = makeCommercialContainerGraph();
|
||||||
|
|
||||||
|
const result = applyValidatedProposal({
|
||||||
|
situationGraph: graph,
|
||||||
|
proposal: makeMeaningfulNoOpProposal(),
|
||||||
|
});
|
||||||
|
|
||||||
|
const parentNode = result.updatedSituationGraph.nodes.find(
|
||||||
|
(node) => node.id === "n-commercial-parent",
|
||||||
|
);
|
||||||
|
const selectedChild = result.updatedSituationGraph.nodes.find(
|
||||||
|
(node) => node.id === result.selectedUnknownAfter,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(parentNode.status).toBe("unknown");
|
||||||
|
expect(selectedChild.parentId).toBe(parentNode.id);
|
||||||
|
expect(selectedChild.status).toBe("unknown");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -71,6 +71,33 @@ describe("normaliseAnalysisResponse", () => {
|
|||||||
expect(result.changesApplied).toHaveLength(1);
|
expect(result.changesApplied).toHaveLength(1);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("normalises reported_claim evidenceType to reported_statement", () => {
|
||||||
|
const input = {
|
||||||
|
evidence: [
|
||||||
|
{
|
||||||
|
id: "ev1",
|
||||||
|
description: "x",
|
||||||
|
evidenceType: "reported_claim",
|
||||||
|
confidence: "medium",
|
||||||
|
importance: "important",
|
||||||
|
source: "report",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
const result = normaliseAnalysisResponse(input);
|
||||||
|
|
||||||
|
expect(result.normalised.evidence[0].evidenceType).toBe(
|
||||||
|
"reported_statement",
|
||||||
|
);
|
||||||
|
expect(result.changesApplied).toEqual([
|
||||||
|
{
|
||||||
|
path: ["evidence", 0, "evidenceType"],
|
||||||
|
change: "Converted reported_claim to reported_statement",
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
it("does not invent a next question", () => {
|
it("does not invent a next question", () => {
|
||||||
const input = { evidence: [] };
|
const input = { evidence: [] };
|
||||||
const result = normaliseAnalysisResponse(input);
|
const result = normaliseAnalysisResponse(input);
|
||||||
@@ -165,6 +192,62 @@ describe("analyseScenario compatibility", () => {
|
|||||||
expect(result.nextQuestion).toBeUndefined();
|
expect(result.nextQuestion).toBeUndefined();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("succeeds when reported_claim is the only evidenceType mismatch", async () => {
|
||||||
|
mockGenerateReconstruction.mockResolvedValue({
|
||||||
|
inputClassification: {
|
||||||
|
primaryType: "unexplained_change",
|
||||||
|
secondaryTypes: [],
|
||||||
|
reasoningModes: ["validate_measurement"],
|
||||||
|
classificationReason: "reason",
|
||||||
|
confidence: "medium",
|
||||||
|
},
|
||||||
|
reconstruction: {
|
||||||
|
summary: "summary",
|
||||||
|
actors: [],
|
||||||
|
systemsOrObjects: [],
|
||||||
|
expectedStates: [],
|
||||||
|
observedStates: [],
|
||||||
|
differences: [],
|
||||||
|
knownTransitions: [],
|
||||||
|
unexplainedTransitions: [],
|
||||||
|
contradictions: [],
|
||||||
|
importantUnknowns: [],
|
||||||
|
plausibleInterpretations: [],
|
||||||
|
},
|
||||||
|
evidence: [
|
||||||
|
{
|
||||||
|
id: "ev1",
|
||||||
|
description: "desc",
|
||||||
|
evidenceType: "reported_claim",
|
||||||
|
attribution: null,
|
||||||
|
confidence: "medium",
|
||||||
|
importance: "important",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
nextQuestion: {
|
||||||
|
id: "q1",
|
||||||
|
question: "What denominator?",
|
||||||
|
targets: ["observedStates"],
|
||||||
|
reason: "reason",
|
||||||
|
expectedInformationValue: "high",
|
||||||
|
reasoningMode: "validate_measurement",
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const { analyseScenario } = await import("@/lib/analysis.js");
|
||||||
|
const result = await analyseScenario("Scenario text", {
|
||||||
|
promptVersion: "v0.3",
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(result.success).toBe(true);
|
||||||
|
expect(result.compatibilityApplied).toBe(true);
|
||||||
|
expect(result.compatibilityChanges).toContainEqual({
|
||||||
|
path: ["evidence", 0, "evidenceType"],
|
||||||
|
change: "Converted reported_claim to reported_statement",
|
||||||
|
});
|
||||||
|
expect(result.evidence[0].evidenceType).toBe("reported_statement");
|
||||||
|
});
|
||||||
|
|
||||||
it("malformed JSON still fails", async () => {
|
it("malformed JSON still fails", async () => {
|
||||||
mockGenerateReconstruction.mockResolvedValue("{not valid json");
|
mockGenerateReconstruction.mockResolvedValue("{not valid json");
|
||||||
|
|
||||||
|
|||||||
@@ -275,6 +275,94 @@ function makeUpdateSuccess(overrides = {}) {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function makeCommercialUpdateSuccess(overrides = {}) {
|
||||||
|
return {
|
||||||
|
success: true,
|
||||||
|
stage: "update_applied",
|
||||||
|
updatedSituationGraph: {
|
||||||
|
centralStatement:
|
||||||
|
"I have developed a new reasoning method that aims to help people determine whether they have enough justified confidence to make a decision. I believe it could become a commercial product, but I do not yet know whether it solves a genuine problem, whether people would value it enough to pay for it, or whether it is fundamentally different from existing AI tools. Before investing significant time and money into building it further, I want to determine whether continuing development is commercially justified.",
|
||||||
|
currentSummary: "Commercial update summary",
|
||||||
|
activeUnknownNodeId: "n-other-people",
|
||||||
|
resolvedNodeIds: ["n-who"],
|
||||||
|
nodes: [
|
||||||
|
{
|
||||||
|
id: "n-commercial-parent",
|
||||||
|
label:
|
||||||
|
"Commercial justification for whether continuing development is commercially justified",
|
||||||
|
description:
|
||||||
|
"Need to know whether this solves a genuine problem, whether people would value it enough to pay for it, and whether it is commercially justified before continuing development.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "provisional",
|
||||||
|
confidence: "medium",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "n-who",
|
||||||
|
label: "Who experiences this problem",
|
||||||
|
description:
|
||||||
|
"Need to know who experiences this problem, because that must be clear before deciding whether it is commercially justified.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "resolved",
|
||||||
|
confidence: "medium",
|
||||||
|
value:
|
||||||
|
"I experience it myself when I am trying to decide whether a project, idea or investment is justified, but I do not yet know how common that problem is for other people.",
|
||||||
|
parentId: "n-commercial-parent",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "n-other-people",
|
||||||
|
label: "Whether other people experience this problem",
|
||||||
|
description:
|
||||||
|
"Need to know whether other people experience this problem, because that must be established before deciding whether the problem is broadly important.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
parentId: "n-commercial-parent",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
edges: [
|
||||||
|
{
|
||||||
|
id: "e-other-parent",
|
||||||
|
fromNodeId: "n-other-people",
|
||||||
|
toNodeId: "n-commercial-parent",
|
||||||
|
relationship: "depends_on",
|
||||||
|
confidence: "medium",
|
||||||
|
description:
|
||||||
|
"This child unknown must be investigated before the broader parent explanation can be resolved.",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
proposal: {
|
||||||
|
addedNodes: [],
|
||||||
|
updatedNodes: [{ nodeId: "n-who", newStatus: "resolved", reason: "answered" }],
|
||||||
|
addedEdges: [],
|
||||||
|
removedEdgeIds: [],
|
||||||
|
resolvedUnknownNodeIds: ["n-who"],
|
||||||
|
affectedNodeIds: ["n-commercial-parent"],
|
||||||
|
selectedQuestion: null,
|
||||||
|
},
|
||||||
|
selectedQuestion: {
|
||||||
|
nodeId: "n-other-people",
|
||||||
|
question: "What makes you think other people experience this problem too?",
|
||||||
|
reason:
|
||||||
|
"Formulated as a direct foundational question because this child unknown should be answered one step at a time.",
|
||||||
|
reasoningPattern: "decision",
|
||||||
|
questionFamily: "decision_foundation",
|
||||||
|
},
|
||||||
|
previousActiveUnknownNodeId: "n-who",
|
||||||
|
newActiveUnknownNodeId: "n-other-people",
|
||||||
|
affectedNodeIds: ["n-commercial-parent"],
|
||||||
|
resolvedUnknownNodeIds: ["n-who"],
|
||||||
|
diagnostics: {
|
||||||
|
unresolvedCandidateCount: 2,
|
||||||
|
eligibleCandidateCount: 1,
|
||||||
|
candidateNodeIds: ["n-other-people"],
|
||||||
|
resolvedCurrentTurnNodeIds: ["n-who"],
|
||||||
|
noQuestionReason: null,
|
||||||
|
},
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
describe("scenario-form UI helpers", () => {
|
describe("scenario-form UI helpers", () => {
|
||||||
it("submits to /api/cases/start", async () => {
|
it("submits to /api/cases/start", async () => {
|
||||||
const fetchImpl = vi.fn().mockResolvedValue({ ok: true });
|
const fetchImpl = vi.fn().mockResolvedValue({ ok: true });
|
||||||
@@ -384,6 +472,44 @@ describe("graph-backed UI rendering", () => {
|
|||||||
expect(html).not.toContain("Update situation");
|
expect(html).not.toContain("Update situation");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("renders the initial graph-backed commercial question when start-case reselection succeeds", () => {
|
||||||
|
const html = renderToStaticMarkup(
|
||||||
|
<ScenarioResultPanels
|
||||||
|
result={makeGraphResult({
|
||||||
|
situationGraph: {
|
||||||
|
...makeGraphResult().situationGraph,
|
||||||
|
activeUnknownNodeId: "n-who",
|
||||||
|
nodes: [
|
||||||
|
...makeGraphResult().situationGraph.nodes,
|
||||||
|
{
|
||||||
|
id: "n-who",
|
||||||
|
label: "Who experiences this problem",
|
||||||
|
description:
|
||||||
|
"Need to know who experiences this problem, because that must be clear before deciding whether it is commercially justified.",
|
||||||
|
kind: "unknown",
|
||||||
|
status: "unknown",
|
||||||
|
confidence: "medium",
|
||||||
|
parentId: "n-parent",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
},
|
||||||
|
selectedQuestion: {
|
||||||
|
nodeId: "n-who",
|
||||||
|
question: "Who experiences this problem?",
|
||||||
|
reason: "Graph-backed commercial child selection.",
|
||||||
|
},
|
||||||
|
diagnostics: {
|
||||||
|
...makeGraphResult().diagnostics,
|
||||||
|
noQuestionReason: null,
|
||||||
|
},
|
||||||
|
})}
|
||||||
|
/>,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(html).toContain("Who experiences this problem?");
|
||||||
|
expect(html).toContain("Selected Question");
|
||||||
|
});
|
||||||
|
|
||||||
it("renders diagnostics", () => {
|
it("renders diagnostics", () => {
|
||||||
const html = renderToStaticMarkup(
|
const html = renderToStaticMarkup(
|
||||||
<DiagnosticsView result={makeGraphResult()} />,
|
<DiagnosticsView result={makeGraphResult()} />,
|
||||||
@@ -700,4 +826,16 @@ describe("graph-backed UI rendering", () => {
|
|||||||
|
|
||||||
expect(html).toContain("Proposal details");
|
expect(html).toContain("Proposal details");
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("does not show the no-question fallback when a commercial follow-up sibling exists", () => {
|
||||||
|
const html = renderToStaticMarkup(
|
||||||
|
<GraphUpdateView updateResult={makeCommercialUpdateSuccess()} />,
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(html).toContain(
|
||||||
|
"What makes you think other people experience this problem too?",
|
||||||
|
);
|
||||||
|
expect(html).toContain("New active unknown");
|
||||||
|
expect(html).not.toContain("No next question selected yet.");
|
||||||
|
});
|
||||||
});
|
});
|
||||||
Reference in New Issue
Block a user