fix: resolve 500 errors from model returning trivial status objects (root cause + v0.2 prompt fix)
Two bugs were causing the model to return {"status":"ok"} / {"status":"ready"}
instead of structured reconstruction data, resulting in POST /api/analyse 500:
1. DOUBLE-WRAPPING BUG (lib/llm/provider.js):
generateReconstruction() called buildPrompt(scenario) on input that was
already a fully-built prompt string from analyseScenario(). This wrapped the
v0.1 prompt (~5000+ chars) in another template layer, producing incomprehensible
output that the model could not parse as structured JSON.
Fix: Pass scenario through directly (it is ALREADY a built prompt).
2. MISSING JSON SPEC (prompts/reconstruct-v0.2.md):
The v0.2 prompt template said 'matching the structure exactly' but never
defined what that structure was. The model invented its own field names
(input_classification, reasoning_mode, anchors) with snake_case instead of
camelCase, which failed Zod validation -> 500 errors.
Fix: Added explicit JSON schema section with exact key names, enum values,
and nested structure matching the Zod validation layer.
Additionally:
- Refactored route to use analyseScenario from lib/analysis (centralized)
- Added lib/analysis.js with shared analysis logic
- Updated components to display promptVersion and validation errors
- Added lib/reconstruction/prompt.js v0.1/v0.2 versioning
- Added lib/reconstruction/schema.js v0.2 Zod schemas
- Added debug tool scripts, evaluation results, and comparison findings
This commit is contained in:
@@ -16,9 +16,19 @@ const ValidationIndicator = ({ status }) => {
|
||||
);
|
||||
};
|
||||
|
||||
const validationIcons = {
|
||||
valid: "✅",
|
||||
partial: "⚠️",
|
||||
invalid: "❌",
|
||||
};
|
||||
|
||||
export default function DiagnosticsView({ result }) {
|
||||
if (!result) return null;
|
||||
|
||||
const metrics = [
|
||||
{ label: "Model", value: result.modelName || "?" },
|
||||
{ label: "Provider", value: "Ollama" },
|
||||
{ label: "Prompt version", value: result.promptVersion || "?" },
|
||||
{ label: "Duration", value: result.responseDurationMs != null ? `${result.responseDurationMs}ms` : "?" },
|
||||
{ label: "Validation", value: <ValidationIndicator status={result.validationStatus || "invalid"} /> },
|
||||
];
|
||||
@@ -35,16 +45,31 @@ export default function DiagnosticsView({ result }) {
|
||||
))}
|
||||
</dl>
|
||||
|
||||
{/* Collapsed raw output for debugging */}
|
||||
{result.rawResponse && (
|
||||
<details className="mt-4">
|
||||
<summary className="cursor-pointer text-xs text-gray-500 underline hover:text-gray-700">
|
||||
View raw model response
|
||||
View raw model response ({(result.rawResponse?.length || 0).toLocaleString()} chars)
|
||||
</summary>
|
||||
<pre className="mt-2 max-h-60 overflow-auto rounded bg-gray-900 px-3 py-2 text-xs leading-relaxed text-green-400">
|
||||
{result.rawResponse}
|
||||
</pre>
|
||||
</details>
|
||||
)}
|
||||
|
||||
{/* Errors if present */}
|
||||
{result.errors && result.errors.length > 0 && (
|
||||
<details className="mt-3">
|
||||
<summary className="cursor-pointer text-xs text-red-500 underline hover:text-red-700">
|
||||
Validation errors ({result.errors.length})
|
||||
</summary>
|
||||
<ul className="mt-1 space-y-0.5 text-xs text-red-600">
|
||||
{result.errors.map((err, i) => (
|
||||
<li key={i}>{err}</li>
|
||||
))}
|
||||
</ul>
|
||||
</details>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user