confidence-engine/docs/experiment-57j31.md

# Experiment 57J.31 — Rejected Proposal Diagnostics Integration

## Objective

Provide diagnostic visibility into the parsed proposal that fails at `proposal_compatibility` (Experiment 57J.30's blocking gap). When an update is rejected, the API currently returns only `{ success: false, stage, errors }` — no pre-validation proposal fields are visible. This experiment adds a compact `rejectedProposalSnapshot` to the diagnostics object in the failure path.

## Pre-written expectation recorded: YES

> Adding a snapshot of key proposal fields (answerMeaning, addedNodes, addedEdges, updatedNodes, resolvedUnknownNodeIds) to the rejection diagnostics will allow developers to determine whether the rejection was caused by stronger answerMeaning category language or a different structural element — without needing to modify production code that controls which proposals are rejected. The snapshot should not include raw model responses, prompts, or chain-of-thought content (privacy/performance constraint). It should only be present for `proposal_compatibility` failures, not other failure stages.

## Configured apparatus

- **Branch:** `feature/rejected-proposal-diagnostics-v0.16`
- **HEAD at experiment start:** `7937767` — experiment: capture proposal-boundary live variance
- **Host/model:** qwen-claude:latest at http://192.168.1.111:11434
- **Ollama calls:** 0 (diagnostic instrumentation does not invoke the model)

## Implementation

### orchestrator.js change (1 location, lines ~690–725)

In the `!applicationResult.success` return path of `updateCaseWithDependencies`:

```javascript
// Compact rejected-proposal snapshot for proposal_compatibility diagnostics.
const rejectedProposalSnapshot =
  applicationResult.stage === "proposal_compatibility" && parsedProposal.proposal
    ? {
        answerMeaning: parsedProposal.proposal.answerMeaning
          ? {
              userSupportedMeaning: ...,
              possibleInference: ...,
            }
          : null,
        updatedNodes: (parsedProposal.proposal.updatedNodes ?? []).map(n => ({ nodeId, newValue })),
        resolvedUnknownNodeIds: ...,
        addedNodes: (parsedProposal.proposal.addedNodes ?? []).map(n => ({ id, kind, label, description, parentId, dependsOn, affects, childIds })),
        addedEdges: (parsedProposal.proposal.addedEdges ?? []).map(e => ({ fromNodeId, toNodeId, relationship })),
      }
    : null;

// Then in the return diagnostics object:
...(rejectedProposalSnapshot && { rejectedProposalSnapshot }),
```

### Key design constraints

1. **Stage-gated:** Only populated when `stage === "proposal_compatibility"` and `parsedProposal.proposal` is truthy. Other failure stages (graph_validation, proposal_validation, application, result_validation) get no snapshot.
2. **Diagnostic-only:** The snapshot does not alter validation logic, mutation behavior, or error messages. It is purely evidence for developers.
3. **Compact field set:** Only answerMeaning fields, node/edge structural references are included. No raw model response, no prompt, no chain_of_thought.
4. **No production code changed outside orchestrator.js:** The route layer already forwards `diagnostics` to the API response, so this change flows through automatically.

## Validation approach

### Automated tests (8 new + 2 existing-verification tests)

1. **tests/graph/rejected-proposal-snapshot.test.js** (7 tests, all pass):
   - "includes rejectedProposalSnapshot when stage is proposal_compatibility" — confirms snapshot presence for the target failure stage.
   - "exposes answerMeaning.userSupportedMeaning and possibleInference in the snapshot" — confirms semantic content visibility.
   - "exposes added unknown label, description and structural references" — confirms addedNode field completeness (id, kind, label, description, parentId, dependsOn, affects, childIds).
   - "exposes added edges with fromNodeId, toNodeId and relationship" — confirms edge visibility.
   - "retains existing rejection stage and errors unchanged" — confirms the snapshot does not modify error strings or stage values.
   - "does not include raw model response or prompt in the snapshot" — confirms field-set constraint (no keys containing "raw", "prompt", "chain_of_thought", "provider_metadata").
   - "does not include rejectedProposalSnapshot for non-proposal_compatibility failures" — confirms stage-gating.

2. **tests/graph/apply-proposal.test.js** (2 new verification tests):
   - "identical rejected fixture still rejects" — confirms the rejection path in applyValidatedProposal is unchanged (same errors, no mutations).
   - "successful proposal behaviour unchanged" — confirms successful proposals still work as expected with the same pass result.

3. **tests/app/api/cases-update-route.test.js** (existing tests — 13 tests pass) — the route layer already forwards diagnostics correctly; this is a regression guard.

### Test results

```
✓ tests/graph/rejected-proposal-snapshot.test.js (7 tests) 9ms
✓ tests/graph/apply-proposal.test.js (64 tests) 208ms  [includes 2 new]
✓ tests/app/api/cases-update-route.test.js (13 tests) 113ms
Total: 84 passed, 0 failed

Pre-existing failure confirmed independent of this change:
✗ tests/graph/orchestrator.test.js (32 tests) — 1 pre-existing failure:
  "childUnknownCount" expects 5 but gets 2 (comparability decomposition test)
  This is not caused by the rejected-proposal-diagnostics change.
```

## Blocked observations

**No live model calls were made in this experiment.** The diagnostic snapshot is deterministic — it captures parsed proposal data that already exists at the point of rejection. No Ollama inference is needed.

What remains unproven:
- **Live rejection analysis:** Whether the actual rejected trial from Experiment 57J.30 (Trial 2) contained stronger `userSupportedMeaning` category language vs. a different structural element — this requires re-running Experiment 57J.30 with the new diagnostics field now available in the API response.
- **Route layer diagnostic forwarding:** The route layer already forwards `diagnostics` from the orchestrator result, but whether `rejectedProposalSnapshot` appears correctly in the actual HTTP response body (422 status) should be verified via a live call once Ollama is reachable.

## What this establishes

1. **The blocking visibility gap identified in Experiment 57J.30 is now closed at the code level.** Developers can inspect `diagnostics.rejectedProposalSnapshot` when receiving a 422 from proposal_compatibility to see: what answerMeaning was extracted, what nodes/edges were proposed, and which anchors were targeted — all before validation rejected them.
2. **No validation or mutation behavior changed.** The rejection itself (errors, stage, HTTP status code) is identical. Only the diagnostic surface is expanded.
3. **Stage gating ensures no snapshot leakage for other failure types.** Graph validation failures, provider errors, and application failures get no snapshot — the instrumentation is narrowly scoped to where Experiment 57J.30 identified the gap: proposal_compatibility.

## Production code changed

- `lib/graph/orchestrator.js` — added rejectedProposalSnapshot computation and inclusion in diagnostics (lines ~690–725).
- No changes to schema, route layer, validation logic, or mutation paths.

## Prompt changed: NO
## Schema changed: NO
## Temporary instrumentation removed: YES (no instrumentation added)
## Ollama calls beyond budget: 0

## Documentation updated: YES