test(confidence-engine): establish longitudinal synthesis checkpoints
Co-Authored-By: Claude Code <noreply@anthropic.com>
This commit is contained in:
@@ -104,3 +104,62 @@ Contents:
|
|||||||
- `findings`: 9 findings total; all 7 previously verified IDs present (`hciikc`, `ybfsm3`, `k45vyt`, `vhftyr`, `9bgkmw`, `23192q`, `kkh39k`)
|
- `findings`: 9 findings total; all 7 previously verified IDs present (`hciikc`, `ybfsm3`, `k45vyt`, `vhftyr`, `9bgkmw`, `23192q`, `kkh39k`)
|
||||||
|
|
||||||
Verified: helper parses without invoking live synthesis; fixture immutability confirmed via deep-clone roundtrip.
|
Verified: helper parses without invoking live synthesis; fixture immutability confirmed via deep-clone roundtrip.
|
||||||
|
|
||||||
|
## Iteration 6 — Longitudinal Checkpoints (Stage 2)
|
||||||
|
|
||||||
|
### New files
|
||||||
|
|
||||||
|
| File | Role |
|
||||||
|
|------|------|
|
||||||
|
| `tests/fixtures/current-understanding-longitudinal-checkpoints.json` | Compact checkpoint manifest referencing original fixture; lists included contribution IDs and finding IDs per checkpoint |
|
||||||
|
| `tests/graph/current-understanding-experiment-helper.test.js` (extended) | 15 new longitudinal assertions in `Longitudinal checkpoints — manifest integrity` describe block |
|
||||||
|
|
||||||
|
### Checkpoint manifest structure
|
||||||
|
|
||||||
|
The manifest contains:
|
||||||
|
- `manifestVersion`, `description`, `sourceFixture`, `contaminationAssessment`
|
||||||
|
- `contributions[]` — ordered list with timestamps and ordering integers
|
||||||
|
- `findingsById` — verbatim copy of each original finding (by ID)
|
||||||
|
- `checkpoints[]` — four cumulative states: `{ id, includedContributions[], includedFindings[] }`
|
||||||
|
|
||||||
|
### Checkpoint state summary
|
||||||
|
|
||||||
|
| Checkpoint | Description | Contribs | Findings | Count |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| checkpoint-0 | Baseline (no findings) | none | none | 0 |
|
||||||
|
| checkpoint-1 | contrib-0001 | contrib-0001 | 3bw5f0, i5zdcq | 2 |
|
||||||
|
| checkpoint-2 | contrib-0001 + contrib-0002 | contrib-0001, contrib-0002 | 7 findings | 7 |
|
||||||
|
| checkpoint-3 | All contributions | all three | all 9 original findings | 9 |
|
||||||
|
|
||||||
|
### Graph-state contamination analysis
|
||||||
|
|
||||||
|
`synthesizeCurrentUnderstanding()` and `buildGraphEvidenceProjection()` project **only**:
|
||||||
|
- Nodes with `status: "known"` or `"supported"` (filtered)
|
||||||
|
- `centralStatement` (framing only)
|
||||||
|
- Findings passed as separate parameter
|
||||||
|
|
||||||
|
**Explicitly excluded from prompt:** `resolvedNodeIds`, `activeUnknownNodeId`, `reasoningState`, edges, `currentSummary`, nodes outside known/supported statuses.
|
||||||
|
|
||||||
|
**Conclusion: NONE — final-state graph fields do not leak into synthesis input.**
|
||||||
|
|
||||||
|
### Compatibility gap
|
||||||
|
|
||||||
|
The existing helper consumes a single `{ situationGraph, findings }` fixture via `--file`. The checkpoint manifest is **NOT directly consumable by the existing helper** without bounded extension: the manifest uses `findingsById` lookup keyed by finding ID rather than a flat findings array alongside a pre-filtered graph. For longitudinal comparison experiments, the consuming code must iterate checkpoints and project `{ situationGraph (unchanged), findings[] }` per checkpoint before invoking the helper or synthesis function directly.
|
||||||
|
|
||||||
|
### Test assertions added (15 new tests)
|
||||||
|
|
||||||
|
1. Manifest has exactly 4 checkpoints
|
||||||
|
2. Checkpoint counts are 0, 2, 7, 9
|
||||||
|
3. checkpoint-0 has no included findings
|
||||||
|
4. checkpoint-1 includes exactly contrib-0001 findings
|
||||||
|
5. checkpoint-2 is cumulative of checkpoint-1 plus contrib-0002 findings
|
||||||
|
6. checkpoint-3 includes all 9 original findings
|
||||||
|
7. All checkpoint findings are verbatim from the original fixture
|
||||||
|
8. Contribution ordering matches recorded timestamps
|
||||||
|
9. Manifest ordering matches fixture finding order
|
||||||
|
10. checkpoint-3 is cumulative of all checkpoints before it
|
||||||
|
11. No findings appear before their contribution boundary
|
||||||
|
12. Original fixture unchanged (spot check)
|
||||||
|
13. Production synthesis contract accepts all checkpoint inputs
|
||||||
|
14. Manifest references the source fixture path
|
||||||
|
15. Manifest does not fabricate historical graph state
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
{
|
||||||
|
"manifestVersion": "1.0.0",
|
||||||
|
"description": "Longitudinal cumulative evidence checkpoints for Current Understanding synthesis comparison experiments.",
|
||||||
|
"sourceFixture": "tests/fixtures/current-understanding-original-investigation.json",
|
||||||
|
"contaminationAssessment": "NONE",
|
||||||
|
"contributions": [
|
||||||
|
{
|
||||||
|
"id": "contrib-0001",
|
||||||
|
"ordering": 1,
|
||||||
|
"timestamp": "2026-10-08T06:50:06.355Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "contrib-0002",
|
||||||
|
"ordering": 2,
|
||||||
|
"timestamp": "2026-10-08T06:55:45.665Z"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "contrib-0003",
|
||||||
|
"ordering": 3,
|
||||||
|
"timestamp": "2026-10-08T07:01:23.359Z"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"checkpoints": [
|
||||||
|
{
|
||||||
|
"id": "checkpoint-0",
|
||||||
|
"includedContributions": [],
|
||||||
|
"includedFindings": []
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "checkpoint-1",
|
||||||
|
"includedContributions": [
|
||||||
|
"contrib-0001"
|
||||||
|
],
|
||||||
|
"includedFindings": [
|
||||||
|
"finding-3bw5f0",
|
||||||
|
"finding-i5zdcq"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "checkpoint-2",
|
||||||
|
"includedContributions": [
|
||||||
|
"contrib-0001",
|
||||||
|
"contrib-0002"
|
||||||
|
],
|
||||||
|
"includedFindings": [
|
||||||
|
"finding-3bw5f0",
|
||||||
|
"finding-i5zdcq",
|
||||||
|
"finding-hciikc",
|
||||||
|
"finding-ybfsm3",
|
||||||
|
"finding-k45vyt",
|
||||||
|
"finding-vhftyr",
|
||||||
|
"finding-9bgkmw"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": "checkpoint-3",
|
||||||
|
"includedContributions": [
|
||||||
|
"contrib-0001",
|
||||||
|
"contrib-0002",
|
||||||
|
"contrib-0003"
|
||||||
|
],
|
||||||
|
"includedFindings": [
|
||||||
|
"finding-3bw5f0",
|
||||||
|
"finding-i5zdcq",
|
||||||
|
"finding-hciikc",
|
||||||
|
"finding-ybfsm3",
|
||||||
|
"finding-k45vyt",
|
||||||
|
"finding-vhftyr",
|
||||||
|
"finding-9bgkmw",
|
||||||
|
"finding-23192q",
|
||||||
|
"finding-kkh39k"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"findingsById": {
|
||||||
|
"finding-3bw5f0": {
|
||||||
|
"id": "finding-3bw5f0",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:50:06.355Z",
|
||||||
|
"proposition": "One enterprise customer has signed an LOI but no binding contract yet.",
|
||||||
|
"contributionId": "contrib-0001",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "One enterprise customer has signed an LOI but no binding contract yet.",
|
||||||
|
"originatingTargetNodeId": "nw0dwkz"
|
||||||
|
},
|
||||||
|
"finding-i5zdcq": {
|
||||||
|
"id": "finding-i5zdcq",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:50:06.355Z",
|
||||||
|
"proposition": "That customer's final contract execution depends on internal pricing approval expected within thirty days.",
|
||||||
|
"contributionId": "contrib-0001",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "That customer's final contract execution depends on internal pricing approval expected within thirty days.",
|
||||||
|
"originatingTargetNodeId": "nw0dwkz"
|
||||||
|
},
|
||||||
|
"finding-hciikc": {
|
||||||
|
"id": "finding-hciikc",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:55:45.665Z",
|
||||||
|
"proposition": "Absence of reliable evidence linking a twelve-month delay to changes in customer retention",
|
||||||
|
"contributionId": "contrib-0002",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "Absence of reliable evidence linking a twelve-month delay to changes in customer retention",
|
||||||
|
"originatingTargetNodeId": "nfl7ee3"
|
||||||
|
},
|
||||||
|
"finding-ybfsm3": {
|
||||||
|
"id": "finding-ybfsm3",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:55:45.665Z",
|
||||||
|
"proposition": "Failure to identify which prospective customers would remain interested after a delay",
|
||||||
|
"contributionId": "contrib-0002",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "Failure to identify which prospective customers would remain interested after a delay",
|
||||||
|
"originatingTargetNodeId": "nfl7ee3"
|
||||||
|
},
|
||||||
|
"finding-k45vyt": {
|
||||||
|
"id": "finding-k45vyt",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:55:45.665Z",
|
||||||
|
"proposition": "Failure to assess whether customer purchasing decisions would shift due to the delay",
|
||||||
|
"contributionId": "contrib-0002",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "Failure to assess whether customer purchasing decisions would shift due to the delay",
|
||||||
|
"originatingTargetNodeId": "nfl7ee3"
|
||||||
|
},
|
||||||
|
"finding-vhftyr": {
|
||||||
|
"id": "finding-vhftyr",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:55:45.665Z",
|
||||||
|
"proposition": "Unestablished impact of competitor activity on customer willingness to wait",
|
||||||
|
"contributionId": "contrib-0002",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "Unestablished impact of competitor activity on customer willingness to wait",
|
||||||
|
"originatingTargetNodeId": "nfl7ee3"
|
||||||
|
},
|
||||||
|
"finding-9bgkmw": {
|
||||||
|
"id": "finding-9bgkmw",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T06:55:45.665Z",
|
||||||
|
"proposition": "Unknown financial consequences tied to potential customer loss during the delay",
|
||||||
|
"contributionId": "contrib-0002",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "Unknown financial consequences tied to potential customer loss during the delay",
|
||||||
|
"originatingTargetNodeId": "nfl7ee3"
|
||||||
|
},
|
||||||
|
"finding-23192q": {
|
||||||
|
"id": "finding-23192q",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T07:01:23.359Z",
|
||||||
|
"proposition": "You lack reliable evidence regarding competitors' plans for the next twelve months.",
|
||||||
|
"contributionId": "contrib-0003",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "You lack reliable evidence regarding competitors' plans for the next twelve months.",
|
||||||
|
"originatingTargetNodeId": "nhgwmv7"
|
||||||
|
},
|
||||||
|
"finding-kkh39k": {
|
||||||
|
"id": "finding-kkh39k",
|
||||||
|
"status": "provisional",
|
||||||
|
"createdAt": "2026-10-08T07:01:23.359Z",
|
||||||
|
"proposition": "You have not established whether competitors will launch similar products, adjust prices, or shift customer expectations.",
|
||||||
|
"contributionId": "contrib-0003",
|
||||||
|
"userDisposition": null,
|
||||||
|
"sourceObservation": "You have not established whether competitors will launch similar products, adjust prices, or shift customer expectations.",
|
||||||
|
"originatingTargetNodeId": "nhgwmv7"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -612,3 +612,132 @@ describe("CU synthesis experiment helper — --include-output flag", function ()
|
|||||||
expect(JSON.stringify(findings)).toBe(JSON.stringify(fixBefore));
|
expect(JSON.stringify(findings)).toBe(JSON.stringify(fixBefore));
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// ── Longitudinal checkpoint deterministic verification ──
|
||||||
|
|
||||||
|
import longitudinalManifest from "@/tests/fixtures/current-understanding-longitudinal-checkpoints.json" assert { type: "json" };
|
||||||
|
import originalFixture from "@/tests/fixtures/current-understanding-original-investigation.json" assert { type: "json" };
|
||||||
|
|
||||||
|
describe("Longitudinal checkpoints — manifest integrity", function () {
|
||||||
|
|
||||||
|
it("manifest has exactly 4 checkpoints", async function () {
|
||||||
|
expect(longitudinalManifest.checkpoints.length).toBe(4);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("checkpoint counts are 0, 2, 7, 9", async function () {
|
||||||
|
const expectedCounts = [0, 2, 7, 9];
|
||||||
|
longitudinalManifest.checkpoints.forEach((cp, i) => {
|
||||||
|
expect(cp.includedFindings.length).toBe(expectedCounts[i]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("checkpoint-0 has no included findings", async function () {
|
||||||
|
expect(longitudinalManifest.checkpoints[0].includedFindings).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("checkpoint-1 includes exactly contrib-0001 findings", async function () {
|
||||||
|
const expected = ["finding-3bw5f0", "finding-i5zdcq"];
|
||||||
|
expect(longitudinalManifest.checkpoints[1].includedFindings).toEqual(expected);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("checkpoint-2 is cumulative of checkpoint-1 plus contrib-0002 findings", async function () {
|
||||||
|
const cp2 = new Set(longitudinalManifest.checkpoints[2].includedFindings);
|
||||||
|
const cp1 = longitudinalManifest.checkpoints[1].includedFindings;
|
||||||
|
cp1.forEach(f => expect(cp2.has(f)).toBe(true)); // all checkpoint-1 findings present in cp2
|
||||||
|
expect(cp2.size).toBe(7);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("checkpoint-3 includes all 9 original findings", async function () {
|
||||||
|
const expected = [
|
||||||
|
"finding-3bw5f0", "finding-i5zdcq",
|
||||||
|
"finding-hciikc", "finding-ybfsm3", "finding-k45vyt",
|
||||||
|
"finding-vhftyr", "finding-9bgkmw",
|
||||||
|
"finding-23192q", "finding-kkh39k"
|
||||||
|
];
|
||||||
|
expect(longitudinalManifest.checkpoints[3].includedFindings).toEqual(expected);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("all checkpoint findings are verbatim from the original fixture", async function () {
|
||||||
|
const originalMap = {};
|
||||||
|
originalFixture.findings.forEach(f => { originalMap[f.id] = f; });
|
||||||
|
longitudinalManifest.checkpoints.forEach(cp => {
|
||||||
|
cp.includedFindings.forEach(fid => {
|
||||||
|
expect(originalMap[fid]).toBeDefined();
|
||||||
|
const manifestFinding = longitudinalManifest.findingsById[fid];
|
||||||
|
expect(manifestFinding).toEqual(originalMap[fid]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("contribution ordering matches recorded timestamps", async function () {
|
||||||
|
const contribs = longitudinalManifest.contributions;
|
||||||
|
for (let i = 1; i < contribs.length; i++) {
|
||||||
|
expect(new Date(contribs[i].timestamp).getTime()).toBeGreaterThan(
|
||||||
|
new Date(contribs[i - 1].timestamp).getTime()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("manifest ordering matches fixture finding order", async function () {
|
||||||
|
const originalIds = originalFixture.findings.map(f => f.id);
|
||||||
|
const manifestIds = Object.keys(longitudinalManifest.findingsById);
|
||||||
|
expect(manifestIds).toEqual(originalIds);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("checkpoint-3 is cumulative of all checkpoints before it", async function () {
|
||||||
|
for (let i = 0; i < 3; i++) {
|
||||||
|
const cpSet = new Set(longitudinalManifest.checkpoints[i].includedFindings);
|
||||||
|
const cp3 = longitudinalManifest.checkpoints[3].includedFindings;
|
||||||
|
cpSet.forEach(f => expect(cp3.includes(f)).toBe(true));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("no findings appear before their contribution boundary", async function () {
|
||||||
|
const contribOrder = {};
|
||||||
|
longitudinalManifest.contributions.forEach(c => { contribOrder[c.id] = c.ordering; });
|
||||||
|
Object.entries(longitudinalManifest.findingsById).forEach(([fid, f]) => {
|
||||||
|
const cpIndex = longitudinalManifest.checkpoints.findIndex(cp =>
|
||||||
|
cp.includedFindings.includes(fid)
|
||||||
|
);
|
||||||
|
// Find which checkpoint first includes this finding
|
||||||
|
expect(cpIndex).toBeGreaterThanOrEqual(contribOrder[f.contributionId] - 1);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("original fixture is unchanged (spot check)", async function () {
|
||||||
|
expect(originalFixture.findings.length).toBe(9);
|
||||||
|
expect(longitudinalManifest.checkpoints[3].includedFindings.length).toBe(9);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("production synthesis contract accepts checkpoint inputs", async function () {
|
||||||
|
// Verify that the filtered findings from each checkpoint can reach synthesizeCurrentUnderstanding
|
||||||
|
const fake = makeFakeProvider();
|
||||||
|
|
||||||
|
for (const cp of longitudinalManifest.checkpoints) {
|
||||||
|
const graphBefore = deepClone(situationGraph);
|
||||||
|
const cpFindings = cp.includedFindings.map(fid => originalFixture.findings.find(f => f.id === fid));
|
||||||
|
|
||||||
|
await expect(
|
||||||
|
synthesizeCurrentUnderstanding(
|
||||||
|
{ situationGraph: situationGraph, findings: cpFindings },
|
||||||
|
{ provider: fake }
|
||||||
|
)
|
||||||
|
).resolves.toBeDefined();
|
||||||
|
|
||||||
|
// Immutability check per checkpoint
|
||||||
|
expect(JSON.stringify(situationGraph)).toBe(JSON.stringify(graphBefore));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("manifest references the source fixture path", async function () {
|
||||||
|
expect(longitudinalManifest.sourceFixture).toBe(
|
||||||
|
"tests/fixtures/current-understanding-original-investigation.json"
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("manifest does not fabricate historical graph state", async function () {
|
||||||
|
// Manifest contains findings data only, not resolvedNodeIds, activeUnknownNodeId, or reasoningState
|
||||||
|
expect(longitudinalManifest.checkpoints).not.toHaveProperty([0, "resolvedNodeIds"]);
|
||||||
|
expect(longitudinalManifest.contributions).not.toHaveProperty([0, "reasoningState"]);
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user