diff --git a/docs/current-handoff.md b/docs/current-handoff.md index 13441e1..c4ac412 100644 --- a/docs/current-handoff.md +++ b/docs/current-handoff.md @@ -791,3 +791,40 @@ Returns JSON-serializable record with: five deconstruction arrays (`observations **Targeted test result:** `node --check` on helper — PASS. Full deterministic test suite — 0 failures across all sections including 6 new save-evidence assertions (positive: file write, structured content, input fidelity, identity fidelity, return values; negative: missing path rejection, missing contribution rejection, ambiguous contribution rejection). All OS temp files cleaned in `finally` blocks. **Operational note:** Future live experiments must explicitly call `saveFocusedInvestigationEvidence()` after a completed checkpoint to persist the evidence record. The function is not invoked automatically. Experiment 4's raw output remains unavailable — it was never persisted by the apparatus at that time. + +### Iterated Nine — Explicit Dependency Evidence and Focused Investigation (Experiment 9, 2026-10-09) + +**Type:** Semantic observation experiment (Iteration 9 of bounded iteration loop). One live LLM call. + +**Objective:** Answer whether explicitly raising cross-area dependencies in the synthetic answer leads focused deconstruction to represent those relationships while retaining the selected recruitment uncertainty. + +**Fixed inputs:** +- Scenario: HS-03C — launching a new service within nine months, requiring two specialist hires, software integration and sufficient customer commitments. +- Target node: `n2yzwlv` — "Timeline and feasibility of recruiting two specialists within the nine-month window" +- `previousQuestion`: "What is the current hiring pipeline status for the two specialist roles, and can onboarding be completed within the nine-month window?" (supplied explicitly as controlled input) +- Synthetic answer (FICTIONAL TEST DATA): "We have identified several possible candidates for both specialist roles, but neither position has been filled. We think recruitment could take between three and six months, although candidate availability and notice periods remain unconfirmed. Both specialists would need onboarding before contributing fully. The software integration team also needs specialist input before its work can be completed, and customer commitments may depend on demonstrating that the integrated service works. If recruitment slips, integration and customer commitments could slip as well. We don't yet know whether these dependencies leave enough time to launch within nine months." +- Graph: constructed from HS-03C reconstruction fixture (`.evidence-temp/headless-semantic/HS-03C-result.json`) +- Model: `qwen-claude:latest` via Ollama (`http://192.168.1.111:11434`) +- Apparatus: `composeDeepInvestigationCheckpoint()` from canonical helper, calling production `focusedDeconstruct()`, then `saveFocusedInvestigationEvidence()` to disk +- Live calls: exactly 1 +- Elapsed time: 73,064 ms end-to-end + +**Hypothesis:** Explicit dependency information in the answer may lead focused deconstruction to represent relationships between recruitment, software integration and customer commitments, while still preserving the selected recruitment uncertainty. + +**Result: PARTIAL** semantic fidelity. + +- **Recruitment timing uncertainty — PRESERVED:** The model captured "estimated to take three to six months" as an observation (not a confirmed commitment), retained "candidate availability and notice periods remain unconfirmed," and correctly distinguished estimates from commitments throughout. +- **Recruitment feasibility — PRESERVED:** The model captured that both specialists require onboarding before contributing fully, and the overarching uncertainty explicitly includes whether "the unconfirmed candidate notice periods and required onboarding duration leave enough time." +- **Software integration dependency — NEWLY PRESENT (key observation):** Unlike Experiment 4, the model represented two cross-area relationships: (1) specialist recruitment completion is a prerequisite for software integration progress, and (2) completed software integration demonstration is a dependency for securing customer commitments. This directly addresses what was missing in Experiment 4's partial transformation pattern. +- **Customer commitment dependency — PRESERVED:** The model captured that customer commitments depend on demonstrating the integrated service works, maintaining this as an explicit relationship rather than losing it to hiring-specific framing. +- **Cascading-delay risk — PARTIALLY PRESERVED:** The overarching uncertainty captures the combined timeline constraint ("leave enough time for software integration and customer commitments to finish within the nine-month deadline"), which implicitly represents cascading delay risk, though it does not use the term "cascading" or explicitly map the three-way dependency chain from recruitment through integration to customer commitments. +- **Evidence integrity — FAITHFUL:** Correctly distinguished identified candidates from secured hires, estimated duration from confirmed timing, required onboarding from completed onboarding, and plausible dependencies from established facts. +- **Observation versus assumption — CLEAN:** The model produced zero assumptions (assumptions array empty), treating all inferences as observations or uncertainties. No unsupported causal claims introduced. +- **Follow-up question classification — LEGITIMATE SPECIFICITY:** "What are the actual notice periods for the identified candidates, and does their combined hiring plus onboarding timeline leave enough time to complete integration and secure commitments before the nine-month mark?" This narrows to a specific investigation path (notice periods) but retains explicit reference to the broader cross-area constraint (integration and commitments). It is not premature narrowing because it frames the follow-up as an investigation question rather than closing the broader uncertainty. +- **`canonicalTargetDescription` gap:** The apparatus returned an empty string for `canonicalTargetDescription` in the saved evidence — a known limitation in how the constructed graph was passed through the execution route. This is a procedural artifact, not a semantic loss (the description content appears correctly within the deconstruction observations). + +**Strongest counterevidence to FAITHFUL classification:** The overarching uncertainty combines multiple domains (notice periods + onboarding + integration + customer commitments) into a single compound question, which may obscure the original recruitment-specific framing of the canonical target. The follow-up question narrows to notice periods with broad timeline context rather than independently investigating each dependency layer — this represents a narrowing trade-off, not a complete loss. + +**What remains uncertain:** Whether explicit dependency information in other scenario shapes (non-recruitment) similarly improves cross-area relationship representation. Single-run limitation applies. + +**Evidence file:** `.evidence-temp/headless-semantic/HS-03C-focused-E9.json` diff --git a/docs/uncertainty-led-investigation-hypothesis.md b/docs/uncertainty-led-investigation-hypothesis.md index eb7269c..55bd714 100644 --- a/docs/uncertainty-led-investigation-hypothesis.md +++ b/docs/uncertainty-led-investigation-hypothesis.md @@ -105,3 +105,5 @@ Preserve existing graph relationships, stable identities, provenance, persistenc - **Experiment 7 — Durable Focused-Investigation Disk Persistence (2026-10-09, Iteration 7 of 10):** Verified that `serializeFocusedInvestigationEvidence()` output survives write-to-disk and read-back without loss. Extended deterministic test in `tests/graph/live-update-experiment-helper-checkpoint.test.cjs` writes serialized evidence to OS temp directory via `os.tmpdir()`, reads it back, parses recovered JSON, and asserts deep structural equality against the original. Explicitly verified: five structured arrays with correct lengths; exact synthetic inputs (`canonicalTargetDescription`, `actualQuestion`, `exactAnswer`) unchanged; identity fields preserved; execution metadata null-fields intact. Temp file removed in `finally` block. Zero live model calls. Zero browser calls. No production or canonical-helper modifications. Total test result: all assertions passed (0 failures). This test proves the serializer's JSON-serializable output survives disk persistence at the apparatus level — it does not establish a production/live-run command for writing evidence to files, which remains an untested operational boundary. Experiment 4's original structured output remains unavailable; never persisted by the apparatus at that time. - **Experiment 8 — Explicit Evidence Persistence Operation (2026-10-09, Iteration 8 of 10):** Added `saveFocusedInvestigationEvidence(checkpoint, identity, metadata, outputPath)` to the canonical helper (`tests/graph/live-update-experiment-helper.cjs`). This is a small wrapper around `serializeFocusedInvestigationEvidence()` that performs explicit persistence via an explicitly supplied output path. Validates: (1) non-empty output path required; (2) exactly one valid deconstruction contribution in checkpoint (rejects ambiguous multi-candidate); (3) all four identity fields present (no nulls); (4) focused-investigation input values present. Writes formatted JSON, reads back, verifies deep structural equality. Returns `{ output_path, verifiedEvidence }`. Zero live model calls. Zero browser calls. No production code modified. Total deterministic test result: 0 failures across all assertions including positive tests (file write, structured evidence completeness, input fidelity, identity fidelity, return values) and negative tests (missing path rejection, missing contribution rejection, ambiguous contribution rejection). All OS temp files cleaned in `finally` blocks. Future live experiments must explicitly call this save function after a completed checkpoint. Experiment 4's original structured output remains unavailable; never persisted by the apparatus at that time. + +- **Experiment 9 — Explicit Dependency Evidence and Focused Investigation (2026-10-09, Iteration 9 of 10):** Single-live-run semantic observation. Controlled scenario: synthetic answer explicitly raises three cross-area dependencies (recruitment → software integration dependency, integration → customer commitments dependency, cascading-delay risk). Model response: **PARTIAL** fidelity. Key finding — the model represented two explicit cross-area relationships and retained the overarching feasibility uncertainty ("Whether unconfirmed candidate notice periods and required onboarding duration leave enough time for software integration and customer commitments within nine months"), which was notably absent in Experiment 4's PARTIAL result. The follow-up question classifies as LEGITIMATE SPECIFICITY (narrows to notice periods but retains broader timeline context). Strongest counterevidence: overarching uncertainty combines multiple domains into one compound question, potentially obscuring the original recruitment-specific framing. Follow-up focuses on hiring logistics with cross-area context rather than independently investigating each dependency layer. Single-run limitation. Evidence file: `.evidence-temp/headless-semantic/HS-03C-focused-E9.json`.