feat(confidence-engine): checkpoint focused deconstruction reasoning
This commit is contained in:
@@ -97,10 +97,10 @@ Required top-level fields:
|
||||
Field rules (semantic contract):
|
||||
- targetNodeId must be included as a string identifying this investigation node
|
||||
- observations: only meaning directly supported by what the user's answer states. Do not strengthen implications into observations.
|
||||
- uncertainties: only things the answer explicitly leaves unknown or unclear. Preserve uncertainty at the narrowest scope justified by the answer: when the answer establishes one factor but provides no evidence about what else may matter, keep the remaining uncertainty broad rather than inventing specific additional factors, deficits, causes, requirements, or interventions.
|
||||
- uncertainties: must be a JSON array containing exactly one string — the single nearest unresolved relationship exposed by this answer. This is the specific gap between what the answer established and what remains unknown right here. Do NOT widen the frontier: identify only the one thing that must be understood before you can know what to ask after it. Do NOT include everything else that might matter, future constraints, broader capability questions, other branches of the investigation, or possible remedies. The uncertainty must be narrow enough that one user answer could materially clarify it. Use ordinary language that a capable person with no specialist vocabulary can understand immediately. If the uncertainty needs abstract phrases, management jargon, specialist terminology, or several concepts joined together to express it, break the reasoning down again before returning it. Simple wording of an over-composed idea is still a failure: first ask "what is the smallest thing we actually do not know yet?" then express that one thing simply.
|
||||
- assumptions: what unstated proposition does the user's answer itself rely upon for it to make sense? Include only when such a proposition is genuinely attributable to the user's reasoning. The boundary is narrow: attribute only propositions that the user's answer would cease to make sense if they were false. Do NOT import plausible interpretations from the wider investigation context, scenario framing, domain relevance, strategic implications, or model-generated analysis into this field — those belong in uncertainties, relationships (where permitted), or possibleFollowUpQuestions. Do NOT connect a factual statement the user makes to a broader capability or constraint concept unless the user explicitly links them. Example: answering "I only have bank account access" to a question about delegation constraints does NOT assume that "delegation feasibility is contingent upon banking access" — it only states a fact about access, and connecting that fact to delegation feasibility is your own scenario-level inference, not a user-held assumption. If the user's answer does not contain or rely upon an identifiable assumption, return assumptions: []. Do NOT require verbatim copying from the user's answer; paraphrasing is allowed only when the reasoning genuinely relies on it.
|
||||
- relationships: only connections that the user's answer directly establishes between items. Co-mentioned facts do not by themselves create causal, constraint, or dependency relationships. If a relationship is only plausible, omit it rather than assert it.
|
||||
- possibleFollowUpQuestions: questions that investigate genuinely unresolved areas exposed by this answer. Before formulating each follow-up, check whether the question tests a proposition (e.g., "there is a deficit", "X is required", "intervention Y should happen") against the current epistemic state or assumes it as already established. If an explanation, deficit, dependency, cause, intervention, recommendation, or solution has not been established by prior evidence, phrase the question so it tests whether that proposition is true rather than assuming it — verify the unresolved fact before seeking remedy. Prefer questions that identify what remains unknown, distinguish competing explanations, test whether a suspected factor actually matters, clarify scope, or identify what evidence would change the investigation. Do not jump to implementation details unless the answer has already established that intervention as the relevant next issue.
|
||||
- possibleFollowUpQuestions: must be a JSON array containing exactly one string — your single best follow-up question. Example shape: ["one question"]. This question must directly investigate the single uncertainty returned in uncertainties (uncertainties[0] → possibleFollowUpQuestions[0]): one unresolved proposition mapped to one question designed to clarify it. The question must not introduce a second unresolved issue, must not broaden beyond the uncertainty it is meant to resolve, and must not contain more than one investigative step. Do not provide alternatives, a roadmap, or questions that belong after this one has been answered. A later question must be generated only after the current question has been answered and deconstructed. Do not ask about consequences, expansion, requirements, interventions, or other branches until the immediate unresolved relationship has been clarified. Those may become later questions after new evidence is obtained. Ask only what the Engine has earned the right to ask now. Each epistemic step waits its turn — do not combine steps that should happen in sequence across multiple turns: one question that investigates one thing only, never a bundle of future reasoning joined together. Before formulating, check whether the question tests a proposition against the current epistemic state: if an explanation, deficit, dependency, cause, intervention, recommendation, or solution has not been established by prior evidence, phrase the question so it tests whether that proposition is true rather than assuming it — verify the unresolved fact before seeking remedy. Prefer questions that identify what remains unknown, distinguish competing explanations, test whether a suspected factor actually matters, clarify scope, or identify what evidence would change the investigation. Do not jump to implementation details unless the answer has already established that intervention as the relevant next issue. Use ordinary language that a capable person with no specialist vocabulary can understand immediately. If the question needs abstract phrases, management jargon, specialist terminology, or several concepts joined together to express it, break the reasoning down again before returning it. Simple wording of an over-composed idea is still a failure: first ask "what is the smallest thing we actually do not know yet?" then express that one thing simply.
|
||||
- cross-field ownership: preserve who or what owns each proposition. When a statement expresses the user's comfort, willingness, threshold, belief, uncertainty, preference, or judgement, keep it attached to that stance — do not elevate it into an objective requirement, capability fact, or situational constraint.
|
||||
|
||||
Focused case context:
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
/**
|
||||
* Durable narrow frontier prompt builders (EXP42).
|
||||
*
|
||||
* Two prompt-building functions that share one frontier-rule source,
|
||||
* one context formatter, and one JSON-only anchoring instruction.
|
||||
*
|
||||
* This file is experimental apparatus only. It is NOT production code.
|
||||
*/
|
||||
|
||||
// ── shared constants ───────────────────────────────────────────────────────
|
||||
|
||||
const FRONTIER_RULES = `Identify the single nearest thing that is still unknown because of this answer.
|
||||
|
||||
Find the smallest thing we actually do not know yet.
|
||||
|
||||
Return exactly one uncertainty.
|
||||
|
||||
Return exactly one follow-up question.
|
||||
|
||||
The question must directly investigate that uncertainty.
|
||||
|
||||
Do not jump to what might matter later.
|
||||
|
||||
Do not move to another branch before this one is understood.
|
||||
|
||||
Do not assume an explanation, deficit, cause, requirement, intervention, or solution that has not been established.
|
||||
|
||||
If a proposition has not been established, test whether it is true rather than assuming it.
|
||||
|
||||
One question must investigate one thing only.
|
||||
|
||||
Use ordinary language a capable non-expert can understand immediately.`;
|
||||
|
||||
const JSON_ONLY_INSTRUCTION = `Return exactly one JSON object.
|
||||
Return JSON only.
|
||||
Do not include prose, markdown, headings, commentary, or explanation outside the JSON object.`;
|
||||
|
||||
function formatContext({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
"Central situation:",
|
||||
centralStatement,
|
||||
"",
|
||||
"Focused target:",
|
||||
targetLabel,
|
||||
"",
|
||||
"Target description:",
|
||||
targetDescription,
|
||||
"",
|
||||
"Question asked:",
|
||||
question,
|
||||
"",
|
||||
"User answer:",
|
||||
answer,
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
// ── builders ───────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Build the minimal (two-field) frontier prompt.
|
||||
* Semantic workload: one uncertainty + one follow-up question.
|
||||
*/
|
||||
export function buildMinimalFrontierPrompt({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
JSON_ONLY_INSTRUCTION,
|
||||
"",
|
||||
"Required top-level fields:",
|
||||
"- uncertainties",
|
||||
"- possibleFollowUpQuestions",
|
||||
"",
|
||||
"Return this exact shape:",
|
||||
'{"uncertainties": ["one uncertainty"], "possibleFollowUpQuestions": ["one follow-up question"]}',
|
||||
"",
|
||||
FRONTIER_RULES,
|
||||
"",
|
||||
"Focused case context:",
|
||||
formatContext({ centralStatement, targetLabel, targetDescription, question, answer }),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the observation-grounded frontier prompt.
|
||||
* Semantic workload: observations + one uncertainty + one follow-up question.
|
||||
*/
|
||||
export function buildObservationFrontierPrompt({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
JSON_ONLY_INSTRUCTION,
|
||||
"",
|
||||
"Required top-level fields:",
|
||||
"- observations",
|
||||
"- uncertainties",
|
||||
"- possibleFollowUpQuestions",
|
||||
"",
|
||||
"Return this exact shape:",
|
||||
'{"observations": ["one or more directly supported observations"], "uncertainties": ["one uncertainty"], "possibleFollowUpQuestions": ["one follow-up question"]}',
|
||||
"",
|
||||
`observations: only meaning directly supported by what the user's answer states. Do not strengthen implications into observations.`,
|
||||
"",
|
||||
FRONTIER_RULES,
|
||||
"",
|
||||
"Focused case context:",
|
||||
formatContext({ centralStatement, targetLabel, targetDescription, question, answer }),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the relationship-frontier prompt.
|
||||
* Semantic workload: relationships + one uncertainty + one follow-up question.
|
||||
*/
|
||||
export function buildRelationshipFrontierPrompt({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
JSON_ONLY_INSTRUCTION,
|
||||
"",
|
||||
"Required top-level fields:",
|
||||
"- relationships",
|
||||
"- uncertainties",
|
||||
"- possibleFollowUpQuestions",
|
||||
"",
|
||||
"Return this exact shape:",
|
||||
'{"relationships": ["one or more directly established relationships"], "uncertainties": ["one uncertainty"], "possibleFollowUpQuestions": ["one follow-up question"]}',
|
||||
"",
|
||||
`relationships: only connections that the user's answer directly establishes between items. Co-mentioned facts do not by themselves create causal, constraint, or dependency relationships. If a relationship is only plausible, omit it rather than assert it.`,
|
||||
"",
|
||||
FRONTIER_RULES,
|
||||
"",
|
||||
"Focused case context:",
|
||||
formatContext({ centralStatement, targetLabel, targetDescription, question, answer }),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the assumption-frontier prompt.
|
||||
* Semantic workload: assumptions + one uncertainty + one follow-up question.
|
||||
*/
|
||||
export function buildAssumptionFrontierPrompt({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
JSON_ONLY_INSTRUCTION,
|
||||
"",
|
||||
"Required top-level fields:",
|
||||
"- assumptions",
|
||||
"- uncertainties",
|
||||
"- possibleFollowUpQuestions",
|
||||
"",
|
||||
"Return this exact shape:",
|
||||
'{"assumptions": ["zero or more genuinely user-held assumptions"], "uncertainties": ["one uncertainty"], "possibleFollowUpQuestions": ["one follow-up question"]}',
|
||||
"",
|
||||
`assumptions: what unstated proposition does the user's answer itself rely upon for it to make sense? Attribute only when such a proposition is genuinely attributable to the user's reasoning. The boundary is narrow: attribute only propositions that the user's answer would cease to make sense if they were false. Do NOT import plausible interpretations from the wider investigation context, scenario framing, domain relevance, strategic implications, or model-generated analysis into this field — those belong in uncertainties, other structured fields where permitted, or possibleFollowUpQuestions. Do NOT connect a factual statement the user makes to a broader capability or constraint concept unless the user explicitly links them. Example: answering "I only have bank account access" to a question about delegation constraints does NOT assume that "delegation feasibility is contingent upon banking access" — it only states a fact about access, and connecting that fact to delegation feasibility is your own scenario-level inference, not a user-held assumption. If the user's answer does not contain or rely upon an identifiable assumption, return assumptions: []. Do NOT require verbatim copying from the user's answer; paraphrasing is allowed only when the reasoning genuinely relies on it.`,
|
||||
"",
|
||||
FRONTIER_RULES,
|
||||
"",
|
||||
"Focused case context:",
|
||||
formatContext({ centralStatement, targetLabel, targetDescription, question, answer }),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the observation+assumption-frontier prompt.
|
||||
* Semantic workload: observations + assumptions + one uncertainty + one follow-up question.
|
||||
*/
|
||||
export function buildObservationAssumptionFrontierPrompt({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
JSON_ONLY_INSTRUCTION,
|
||||
"",
|
||||
"Required top-level fields:",
|
||||
"- observations",
|
||||
"- assumptions",
|
||||
"- uncertainties",
|
||||
"- possibleFollowUpQuestions",
|
||||
"",
|
||||
"Return this exact shape:",
|
||||
'{"observations": ["one or more directly supported observations"], "assumptions": ["zero or more genuinely user-held assumptions"], "uncertainties": ["one uncertainty"], "possibleFollowUpQuestions": ["one follow-up question"]}',
|
||||
"",
|
||||
`observations: only meaning directly supported by what the user's answer states. Do not strengthen implications into observations.`,
|
||||
"",
|
||||
`assumptions: what unstated proposition does the user's answer itself rely upon for it to make sense? Attribute only when such a proposition is genuinely attributable to the user's reasoning. The boundary is narrow: attribute only propositions that the user's answer would cease to make sense if they were false. Do NOT import plausible interpretations from the wider investigation context, scenario framing, domain relevance, strategic implications, or model-generated analysis into this field — those belong in uncertainties, other structured fields where permitted, or possibleFollowUpQuestions. Do NOT connect a factual statement the user makes to a broader capability or constraint concept unless the user explicitly links them. Example: answering "I only have bank account access" to a question about delegation constraints does NOT assume that "delegation feasibility is contingent upon banking access" — it only states a fact about access, and connecting that fact to delegation feasibility is your own scenario-level inference, not a user-held assumption. If the user's answer does not contain or rely upon an identifiable assumption, return assumptions: []. Do NOT require verbatim copying from the user's answer; paraphrasing is allowed only when the reasoning genuinely relies on it.`,
|
||||
"",
|
||||
FRONTIER_RULES,
|
||||
"",
|
||||
"Focused case context:",
|
||||
formatContext({ centralStatement, targetLabel, targetDescription, question, answer }),
|
||||
].join("\n");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the observation+strict-assumption-frontier prompt (EXP49).
|
||||
* Semantic workload: observations + assumptions + one uncertainty + one follow-up question.
|
||||
* Differs from buildObservationAssumptionFrontierPrompt only in the assumption rule,
|
||||
* which adds an explicit attribution boundary for co-mentioned / contrasted / juxtaposed facts.
|
||||
*/
|
||||
export function buildObservationStrictAssumptionFrontierPrompt({ centralStatement, targetLabel, targetDescription, question, answer }) {
|
||||
return [
|
||||
JSON_ONLY_INSTRUCTION,
|
||||
"",
|
||||
"Required top-level fields:",
|
||||
"- observations",
|
||||
"- assumptions",
|
||||
"- uncertainties",
|
||||
"- possibleFollowUpQuestions",
|
||||
"",
|
||||
"Return this exact shape:",
|
||||
'{"observations": ["one or more directly supported observations"], "assumptions": ["zero or more genuinely user-held assumptions"], "uncertainties": ["one uncertainty"], "possibleFollowUpQuestions": ["one follow-up question"]}',
|
||||
"",
|
||||
`observations: only meaning directly supported by what the user's answer states. Do not strengthen implications into observations.`,
|
||||
"",
|
||||
`assumptions: what unstated proposition does the user's answer itself rely upon for it to make sense? Attribute only when such a proposition is genuinely attributable to the user's reasoning. The boundary is narrow: attribute only propositions that the user's answer would cease to make sense if they were false. Do NOT import plausible interpretations from the wider investigation context, scenario framing, domain relevance, strategic implications, or model-generated analysis into this field — those belong in uncertainties, other structured fields where permitted, or possibleFollowUpQuestions. Co-mentioned, contrasted, or juxtaposed facts do not by themselves establish a user-held assumption. Do not invent a proposition merely to explain why two facts can both be true. A proposition belongs in assumptions only if the user's answer itself depends on that proposition for its meaning and would cease to make sense, or materially lose its intended reasoning, if the proposition were false. Otherwise return assumptions: []. If the user's answer does not contain or rely upon an identifiable assumption, return assumptions: []. Do NOT require verbatim copying from the user's answer; paraphrasing is allowed only when the reasoning genuinely relies on it.`,
|
||||
"",
|
||||
FRONTIER_RULES,
|
||||
"",
|
||||
"Focused case context:",
|
||||
formatContext({ centralStatement, targetLabel, targetDescription, question, answer }),
|
||||
].join("\n");
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -10,31 +10,22 @@
|
||||
* Relies on environment variables OLLAMA_BASE_URL and OLLAMA_MODEL being set.
|
||||
*/
|
||||
|
||||
import dotenv from "dotenv";
|
||||
import { buildFocusedDeconstructPrompt, validateFocusedDeconstructSchema } from "@/lib/graph/focused-investigation.js";
|
||||
import { getProvider } from "@/lib/llm/provider.js";
|
||||
|
||||
// Load configured OLLAMA_MODEL / OLLAMA_BASE_URL (same mechanism as all known-good live tests)
|
||||
dotenv.config({ path: ".env.local" });
|
||||
|
||||
// ── shared execution seam (extracted from existing body) ────────────────
|
||||
|
||||
/**
|
||||
* Run one live focused-deconstruction experiment.
|
||||
* Execute a focused-deconstruct prompt through the canonical live machinery.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.targetNodeId - the target node ID under investigation
|
||||
* @param {string} params.targetLabel - label of the target node
|
||||
* @param {string} params.targetDescription - description of the target node
|
||||
* @param {string} params.centralStatement - the case's central statement
|
||||
* @param {string} params.question - the exact real production question
|
||||
* @param {string} params.answer - the exact real production answer
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
* Caller supplies: prompt, targetNodeId, optional injected provider.
|
||||
* This function owns: provider resolution, model invocation, timing, validation, result construction.
|
||||
*/
|
||||
export async function runLiveFocusedDeconstructExperiment(params) {
|
||||
const { targetNodeId, targetLabel, targetDescription, centralStatement, question, answer, provider } = params;
|
||||
|
||||
// Production path: real prompt builder (never copies prompt logic)
|
||||
const prompt = buildFocusedDeconstructPrompt({
|
||||
targetLabel, targetDescription, centralStatement, question, answer,
|
||||
});
|
||||
|
||||
// Provider: use injected (test) or real (production)
|
||||
async function _executeFocusedPrompt({ prompt, targetNodeId, provider }) {
|
||||
const actualProvider = provider ?? getProvider();
|
||||
|
||||
const ollamaModel = process.env.OLLAMA_MODEL;
|
||||
@@ -44,7 +35,6 @@ export async function runLiveFocusedDeconstructExperiment(params) {
|
||||
const raw = await actualProvider.generateReconstruction(prompt, ollamaModel);
|
||||
const elapsedMs = Date.now() - startedAt;
|
||||
|
||||
// Production path: real schema validator (never copies schema logic)
|
||||
const validationErrors = validateFocusedDeconstructSchema(raw);
|
||||
if (validationErrors.length > 0) {
|
||||
throw new Error(
|
||||
@@ -65,4 +55,480 @@ export async function runLiveFocusedDeconstructExperiment(params) {
|
||||
};
|
||||
}
|
||||
|
||||
// ── existing API (preserved — unchanged behaviour) ──────────────────────
|
||||
|
||||
/**
|
||||
* Run one live focused-deconstruction experiment.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.targetNodeId - the target node ID under investigation
|
||||
* @param {string} params.targetLabel - label of the target node
|
||||
* @param {string} params.targetDescription - description of the target node
|
||||
* @param {string} params.centralStatement - the case's central statement
|
||||
* @param {string} params.question - the exact real production question
|
||||
* @param {string} params.answer - the exact real production answer
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedDeconstructExperiment(params) {
|
||||
const { targetNodeId, targetLabel, targetDescription, centralStatement, question, answer, provider } = params;
|
||||
|
||||
const prompt = buildFocusedDeconstructPrompt({
|
||||
targetLabel, targetDescription, centralStatement, question, answer,
|
||||
});
|
||||
|
||||
return _executeFocusedPrompt({ prompt, targetNodeId, provider });
|
||||
}
|
||||
|
||||
// ── new supplied-prompt API ──────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Execute an already-built focused-deconstruct prompt through the canonical
|
||||
* live machinery.
|
||||
*
|
||||
* The caller supplies a pre-constructed prompt string; this function owns:
|
||||
* - configured OLLAMA_MODEL reading
|
||||
* - provider creation/invocation
|
||||
* - elapsed timing
|
||||
* - focused-deconstruct schema validation
|
||||
* - structured result return
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.prompt - the already-built prompt to send
|
||||
* @param {string} [params.targetNodeId] - original graph node ID (preserved in result)
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedPromptExperiment({ prompt, targetNodeId, provider }) {
|
||||
return _executeFocusedPrompt({ prompt, targetNodeId, provider });
|
||||
}
|
||||
|
||||
// ── minimal frontier result contract (EXP37) ────────────────────────────
|
||||
|
||||
/**
|
||||
* Validate that a focused-deconstruction result contains exactly one
|
||||
* uncertainty and one follow-up question — the minimal frontier output.
|
||||
* Returns an array of errors (empty = valid).
|
||||
*/
|
||||
function validateMinimalFrontierResult(raw) {
|
||||
const errors = [];
|
||||
|
||||
if (!Array.isArray(raw.uncertainties)) {
|
||||
errors.push("uncertainties must be an array");
|
||||
} else if (raw.uncertainties.length !== 1) {
|
||||
errors.push("uncertainties must contain exactly one element, got " + raw.uncertainties.length);
|
||||
} else if (typeof raw.uncertainties[0] !== "string" || raw.uncertainties[0].trim() === "") {
|
||||
errors.push("uncertainties[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
if (!Array.isArray(raw.possibleFollowUpQuestions)) {
|
||||
errors.push("possibleFollowUpQuestions must be an array");
|
||||
} else if (raw.possibleFollowUpQuestions.length !== 1) {
|
||||
errors.push("possibleFollowUpQuestions must contain exactly one element, got " + raw.possibleFollowUpQuestions.length);
|
||||
} else if (typeof raw.possibleFollowUpQuestions[0] !== "string" || raw.possibleFollowUpQuestions[0].trim() === "") {
|
||||
errors.push("possibleFollowUpQuestions[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
// ── new frontier API (EXP37 minimal runner) ──────────────────────────────
|
||||
|
||||
/**
|
||||
* Execute an already-built prompt and validate the returned payload
|
||||
* against the minimal two-field frontier contract.
|
||||
*
|
||||
* Returns only: uncertainties, possibleFollowUpQuestions, elapsedMs, success.
|
||||
* Does NOT require or reconstruct observations, assumptions, relationships,
|
||||
* or targetNodeId.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.prompt - the already-built prompt to send
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedFrontierPromptExperiment({ prompt, provider }) {
|
||||
const actualProvider = provider ?? getProvider();
|
||||
|
||||
const ollamaModel = process.env.OLLAMA_MODEL;
|
||||
if (!ollamaModel) throw new Error("OLLAMA_MODEL is not set in environment");
|
||||
|
||||
const startedAt = Date.now();
|
||||
const raw = await actualProvider.generateReconstruction(prompt, ollamaModel);
|
||||
const elapsedMs = Date.now() - startedAt;
|
||||
|
||||
const validationErrors = validateMinimalFrontierResult(raw);
|
||||
if (validationErrors.length > 0) {
|
||||
throw new Error(
|
||||
"Focused frontier result did not match expected minimal contract:\n" +
|
||||
validationErrors.map((e) => " - " + e).join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
uncertainties: raw.uncertainties,
|
||||
possibleFollowUpQuestions: raw.possibleFollowUpQuestions,
|
||||
elapsedMs,
|
||||
};
|
||||
}
|
||||
|
||||
// ── three-field observation-frontier result contract (EXP38) ────────────
|
||||
|
||||
/**
|
||||
* Validate that a focused-deconstruction result contains observations + one
|
||||
* uncertainty + one follow-up question — the minimal observation-grounded
|
||||
* frontier output for EXP38.
|
||||
* Returns an array of errors (empty = valid).
|
||||
*/
|
||||
function validateObservationFrontierResult(raw) {
|
||||
const errors = [];
|
||||
|
||||
// observations: required, non-empty array of non-empty strings
|
||||
if (!Array.isArray(raw.observations)) {
|
||||
errors.push("observations must be an array");
|
||||
} else if (raw.observations.length === 0) {
|
||||
errors.push("observations must contain at least one element");
|
||||
} else {
|
||||
for (let i = 0; i < raw.observations.length; i++) {
|
||||
if (typeof raw.observations[i] !== "string" || raw.observations[i].trim() === "") {
|
||||
errors.push(`observations[${i}] must be a non-empty string`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// uncertainties: exactly one
|
||||
if (!Array.isArray(raw.uncertainties)) {
|
||||
errors.push("uncertainties must be an array");
|
||||
} else if (raw.uncertainties.length !== 1) {
|
||||
errors.push("uncertainties must contain exactly one element, got " + raw.uncertainties.length);
|
||||
} else if (typeof raw.uncertainties[0] !== "string" || raw.uncertainties[0].trim() === "") {
|
||||
errors.push("uncertainties[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
// possibleFollowUpQuestions: exactly one
|
||||
if (!Array.isArray(raw.possibleFollowUpQuestions)) {
|
||||
errors.push("possibleFollowUpQuestions must be an array");
|
||||
} else if (raw.possibleFollowUpQuestions.length !== 1) {
|
||||
errors.push("possibleFollowUpQuestions must contain exactly one element, got " + raw.possibleFollowUpQuestions.length);
|
||||
} else if (typeof raw.possibleFollowUpQuestions[0] !== "string" || raw.possibleFollowUpQuestions[0].trim() === "") {
|
||||
errors.push("possibleFollowUpQuestions[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
// ── observation-frontier runner (EXP38) ──────────────────────────────────
|
||||
|
||||
/**
|
||||
* Execute an already-built prompt and validate the returned payload
|
||||
* against the three-field observation-frontier contract.
|
||||
*
|
||||
* Requires: observations (array of non-empty strings), uncertainties (exactly 1),
|
||||
* possibleFollowUpQuestions (exactly 1).
|
||||
* Does NOT require assumptions, relationships, or targetNodeId.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.prompt - the already-built prompt to send
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedObservationFrontierPromptExperiment({ prompt, provider }) {
|
||||
const actualProvider = provider ?? getProvider();
|
||||
|
||||
const ollamaModel = process.env.OLLAMA_MODEL;
|
||||
if (!ollamaModel) throw new Error("OLLAMA_MODEL is not set in environment");
|
||||
|
||||
const startedAt = Date.now();
|
||||
const raw = await actualProvider.generateReconstruction(prompt, ollamaModel);
|
||||
const elapsedMs = Date.now() - startedAt;
|
||||
|
||||
const validationErrors = validateObservationFrontierResult(raw);
|
||||
if (validationErrors.length > 0) {
|
||||
throw new Error(
|
||||
"Focused observation-frontier result did not match expected three-field contract:\n" +
|
||||
validationErrors.map((e) => " - " + e).join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
observations: raw.observations,
|
||||
uncertainties: raw.uncertainties,
|
||||
possibleFollowUpQuestions: raw.possibleFollowUpQuestions,
|
||||
elapsedMs,
|
||||
};
|
||||
}
|
||||
|
||||
// ── observation+assumption-frontier result contract (EXP46) ──────────────
|
||||
|
||||
/**
|
||||
* Validate that a focused-deconstruction result contains observations (at least one)
|
||||
* + assumptions (zero or more) + one uncertainty + one follow-up question.
|
||||
* Returns an array of errors (empty = valid).
|
||||
*/
|
||||
function validateObservationAssumptionFrontierResult(raw) {
|
||||
const errors = [];
|
||||
|
||||
// observations: required, at least one non-empty string
|
||||
if (!Array.isArray(raw.observations)) {
|
||||
errors.push("observations must be an array");
|
||||
} else if (raw.observations.length === 0) {
|
||||
errors.push("observations must contain at least one element");
|
||||
} else {
|
||||
for (let i = 0; i < raw.observations.length; i++) {
|
||||
if (typeof raw.observations[i] !== "string" || raw.observations[i].trim() === "") {
|
||||
errors.push(`observations[${i}] must be a non-empty string`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// assumptions: zero or more, each a non-empty string if present
|
||||
if (!Array.isArray(raw.assumptions)) {
|
||||
errors.push("assumptions must be an array");
|
||||
} else {
|
||||
for (let i = 0; i < raw.assumptions.length; i++) {
|
||||
if (typeof raw.assumptions[i] !== "string" || raw.assumptions[i].trim() === "") {
|
||||
errors.push(`assumptions[${i}] must be a non-empty string`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// uncertainties: exactly one
|
||||
if (!Array.isArray(raw.uncertainties)) {
|
||||
errors.push("uncertainties must be an array");
|
||||
} else if (raw.uncertainties.length !== 1) {
|
||||
errors.push("uncertainties must contain exactly one element, got " + raw.uncertainties.length);
|
||||
} else if (typeof raw.uncertainties[0] !== "string" || raw.uncertainties[0].trim() === "") {
|
||||
errors.push("uncertainties[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
// possibleFollowUpQuestions: exactly one
|
||||
if (!Array.isArray(raw.possibleFollowUpQuestions)) {
|
||||
errors.push("possibleFollowUpQuestions must be an array");
|
||||
} else if (raw.possibleFollowUpQuestions.length !== 1) {
|
||||
errors.push("possibleFollowUpQuestions must contain exactly one element, got " + raw.possibleFollowUpQuestions.length);
|
||||
} else if (typeof raw.possibleFollowUpQuestions[0] !== "string" || raw.possibleFollowUpQuestions[0].trim() === "") {
|
||||
errors.push("possibleFollowUpQuestions[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
// ── observation+assumption-frontier runner (EXP46) ──────────────────────
|
||||
|
||||
/**
|
||||
* Execute an already-built prompt and validate the returned payload
|
||||
* against the observation+assumption-frontier contract.
|
||||
*
|
||||
* Requires: observations (array of non-empty strings with at least one),
|
||||
* assumptions (zero or more non-empty strings, [] is valid),
|
||||
* uncertainties (exactly 1), possibleFollowUpQuestions (exactly 1).
|
||||
* Does NOT require relationships, targetNodeId, or any other fields.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.prompt - the already-built prompt to send
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedObservationAssumptionFrontierPromptExperiment({ prompt, provider }) {
|
||||
const actualProvider = provider ?? getProvider();
|
||||
|
||||
const ollamaModel = process.env.OLLAMA_MODEL;
|
||||
if (!ollamaModel) throw new Error("OLLAMA_MODEL is not set in environment");
|
||||
|
||||
const startedAt = Date.now();
|
||||
const raw = await actualProvider.generateReconstruction(prompt, ollamaModel);
|
||||
const elapsedMs = Date.now() - startedAt;
|
||||
|
||||
const validationErrors = validateObservationAssumptionFrontierResult(raw);
|
||||
if (validationErrors.length > 0) {
|
||||
throw new Error(
|
||||
"Focused observation+assumption-frontier result did not match expected contract:\n" +
|
||||
validationErrors.map((e) => " - " + e).join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
observations: raw.observations,
|
||||
assumptions: raw.assumptions,
|
||||
uncertainties: raw.uncertainties,
|
||||
possibleFollowUpQuestions: raw.possibleFollowUpQuestions,
|
||||
elapsedMs,
|
||||
};
|
||||
}
|
||||
|
||||
// ── re-exports for convenience ──────────────────────────────────────────
|
||||
|
||||
export { buildFocusedDeconstructPrompt, validateFocusedDeconstructSchema };
|
||||
|
||||
// ── relationship-frontier result contract (EXP44) ──────────────────────
|
||||
|
||||
/**
|
||||
* Validate that a focused-deconstruction result contains relationships + one
|
||||
* uncertainty + one follow-up question — the relationship-frontier output.
|
||||
* Returns an array of errors (empty = valid).
|
||||
*/
|
||||
function validateRelationshipFrontierResult(raw) {
|
||||
const errors = [];
|
||||
|
||||
// relationships: required, at least one non-empty string
|
||||
if (!Array.isArray(raw.relationships)) {
|
||||
errors.push("relationships must be an array");
|
||||
} else if (raw.relationships.length === 0) {
|
||||
errors.push("relationships must contain at least one element");
|
||||
} else {
|
||||
for (let i = 0; i < raw.relationships.length; i++) {
|
||||
if (typeof raw.relationships[i] !== "string" || raw.relationships[i].trim() === "") {
|
||||
errors.push(`relationships[${i}] must be a non-empty string`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// uncertainties: exactly one
|
||||
if (!Array.isArray(raw.uncertainties)) {
|
||||
errors.push("uncertainties must be an array");
|
||||
} else if (raw.uncertainties.length !== 1) {
|
||||
errors.push("uncertainties must contain exactly one element, got " + raw.uncertainties.length);
|
||||
} else if (typeof raw.uncertainties[0] !== "string" || raw.uncertainties[0].trim() === "") {
|
||||
errors.push("uncertainties[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
// possibleFollowUpQuestions: exactly one
|
||||
if (!Array.isArray(raw.possibleFollowUpQuestions)) {
|
||||
errors.push("possibleFollowUpQuestions must be an array");
|
||||
} else if (raw.possibleFollowUpQuestions.length !== 1) {
|
||||
errors.push("possibleFollowUpQuestions must contain exactly one element, got " + raw.possibleFollowUpQuestions.length);
|
||||
} else if (typeof raw.possibleFollowUpQuestions[0] !== "string" || raw.possibleFollowUpQuestions[0].trim() === "") {
|
||||
errors.push("possibleFollowUpQuestions[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
// ── relationship-frontier runner (EXP44) ───────────────────────────────
|
||||
|
||||
/**
|
||||
* Execute an already-built prompt and validate the returned payload
|
||||
* against the relationship-frontier contract.
|
||||
*
|
||||
* Requires: relationships (array of non-empty strings with at least one),
|
||||
* uncertainties (exactly 1), possibleFollowUpQuestions (exactly 1).
|
||||
* Does NOT require observations, assumptions, or targetNodeId.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.prompt - the already-built prompt to send
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedRelationshipFrontierPromptExperiment({ prompt, provider }) {
|
||||
const actualProvider = provider ?? getProvider();
|
||||
|
||||
const ollamaModel = process.env.OLLAMA_MODEL;
|
||||
if (!ollamaModel) throw new Error("OLLAMA_MODEL is not set in environment");
|
||||
|
||||
const startedAt = Date.now();
|
||||
const raw = await actualProvider.generateReconstruction(prompt, ollamaModel);
|
||||
const elapsedMs = Date.now() - startedAt;
|
||||
|
||||
const validationErrors = validateRelationshipFrontierResult(raw);
|
||||
if (validationErrors.length > 0) {
|
||||
throw new Error(
|
||||
"Focused relationship-frontier result did not match expected contract:\n" +
|
||||
validationErrors.map((e) => " - " + e).join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
relationships: raw.relationships,
|
||||
uncertainties: raw.uncertainties,
|
||||
possibleFollowUpQuestions: raw.possibleFollowUpQuestions,
|
||||
elapsedMs,
|
||||
};
|
||||
}
|
||||
|
||||
// ── assumption-frontier result contract (EXP45) ────────────────────────
|
||||
|
||||
/**
|
||||
* Validate that a focused-deconstruction result contains assumptions (zero or more)
|
||||
* + one uncertainty + one follow-up question — the assumption-frontier output.
|
||||
* Returns an array of errors (empty = valid).
|
||||
*/
|
||||
function validateAssumptionFrontierResult(raw) {
|
||||
const errors = [];
|
||||
|
||||
// assumptions: zero or more, each a non-empty string if present
|
||||
if (!Array.isArray(raw.assumptions)) {
|
||||
errors.push("assumptions must be an array");
|
||||
} else {
|
||||
for (let i = 0; i < raw.assumptions.length; i++) {
|
||||
if (typeof raw.assumptions[i] !== "string" || raw.assumptions[i].trim() === "") {
|
||||
errors.push(`assumptions[${i}] must be a non-empty string`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// uncertainties: exactly one
|
||||
if (!Array.isArray(raw.uncertainties)) {
|
||||
errors.push("uncertainties must be an array");
|
||||
} else if (raw.uncertainties.length !== 1) {
|
||||
errors.push("uncertainties must contain exactly one element, got " + raw.uncertainties.length);
|
||||
} else if (typeof raw.uncertainties[0] !== "string" || raw.uncertainties[0].trim() === "") {
|
||||
errors.push("uncertainties[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
// possibleFollowUpQuestions: exactly one
|
||||
if (!Array.isArray(raw.possibleFollowUpQuestions)) {
|
||||
errors.push("possibleFollowUpQuestions must be an array");
|
||||
} else if (raw.possibleFollowUpQuestions.length !== 1) {
|
||||
errors.push("possibleFollowUpQuestions must contain exactly one element, got " + raw.possibleFollowUpQuestions.length);
|
||||
} else if (typeof raw.possibleFollowUpQuestions[0] !== "string" || raw.possibleFollowUpQuestions[0].trim() === "") {
|
||||
errors.push("possibleFollowUpQuestions[0] must be a non-empty string");
|
||||
}
|
||||
|
||||
return errors;
|
||||
}
|
||||
|
||||
// ── assumption-frontier runner (EXP45) ────────────────────────────────
|
||||
|
||||
/**
|
||||
* Execute an already-built prompt and validate the returned payload
|
||||
* against the assumption-frontier contract.
|
||||
*
|
||||
* Requires: assumptions (zero or more non-empty strings, [] is valid),
|
||||
* uncertainties (exactly 1), possibleFollowUpQuestions (exactly 1).
|
||||
* Does NOT require observations, relationships, or targetNodeId.
|
||||
*
|
||||
* @param {object} params
|
||||
* @param {string} params.prompt - the already-built prompt to send
|
||||
* @param {object} [params.provider] - optional injected provider (for test isolation)
|
||||
* @returns {object} validated result + timing details
|
||||
*/
|
||||
export async function runLiveFocusedAssumptionFrontierPromptExperiment({ prompt, provider }) {
|
||||
const actualProvider = provider ?? getProvider();
|
||||
|
||||
const ollamaModel = process.env.OLLAMA_MODEL;
|
||||
if (!ollamaModel) throw new Error("OLLAMA_MODEL is not set in environment");
|
||||
|
||||
const startedAt = Date.now();
|
||||
const raw = await actualProvider.generateReconstruction(prompt, ollamaModel);
|
||||
const elapsedMs = Date.now() - startedAt;
|
||||
|
||||
const validationErrors = validateAssumptionFrontierResult(raw);
|
||||
if (validationErrors.length > 0) {
|
||||
throw new Error(
|
||||
"Focused assumption-frontier result did not match expected contract:\n" +
|
||||
validationErrors.map((e) => " - " + e).join("\n")
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
assumptions: raw.assumptions,
|
||||
uncertainties: raw.uncertainties,
|
||||
possibleFollowUpQuestions: raw.possibleFollowUpQuestions,
|
||||
elapsedMs,
|
||||
};
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user