experiment: improve semantic graph projection

Experiment 13 — Semantic Facilitator Translation

- Classify nodes by semantic role (observation, question, explanation,
  scaffolding, relationship) rather than graph kind. Scaffolding suppressed
  entirely before section routing.
- Three-tier filtering: scaffolding patterns > internal vocabulary > technical
  summary patterns. Prevents structural noise from contaminating user-facing
  sections.
- Deduplicate by normalised text — merge duplicate observations expressing the
  same finding.
- Route resolved unknowns and assumptions to known section with epistemic
  labels instead of treating them as unresolved questions.
- Prefer concrete observations (numbers, change language, temporal refs) over
  abstract labels in ranking.
- Closed Experiment 12 as confirmed. Added Experiment 13 documentation.
- Updated UX guidelines with Semantic Projection principles.
- 37 tests: filtering, classification, deduplication, ranking, framing, mock
  data integration, edge cases.
This commit is contained in:
2026-08-05 15:26:29 +01:00
parent 1998b84ae1
commit 6eaf0fc246
4 changed files with 692 additions and 36 deletions
+52 -4
View File
@@ -595,13 +595,61 @@ The existing reasoning graph can be deterministically translated into a concise
#### Evaluation
Pending visual and live-data review.
Completed. Visual and live-data review performed.
#### Status
#### Result
Experimental.
Confirmed.
Do not record a conclusion yet.
#### What did we learn?
- The reasoning graph already contains all the information needed for a useful human-facing summary — no additional LLM calls are required.
- Routing by semantic role (observation, question, explanation) rather than graph kind produces a more natural user experience.
- Filtering scaffolding content (scenario summaries, system/tool references, metric object descriptions, process labels) is essential to keep the view focused on findings.
- Deduplication of near-duplicate observations reduces noise without losing information.
- Epistemic clarity matters — resolved unknowns become factual observations and should be classified as known rather than still-under-investigation.
- The panel works across all investigation phases (early, active, terminal).
#### Decision
Close Experiment 12 as confirmed. Proceed to refine the translation through semantic classification in the next iteration.
---
### Experiment 13 — Semantic Facilitator Translation
#### Hypothesis
Improving the deterministic projection from graph semantics to user-facing language — by classifying nodes by *meaning* rather than *graph kind*, suppressing scaffolding, merging duplicates, and preferring concrete observations — produces a significantly better facilitator view without changing the reasoning engine, prompts, graph generation, or any external contracts.
#### Questions
- Does semantic role classification (observation vs question vs explanation) route content more naturally than graph-kind classification?
- Does scaffolding suppression remove visual noise that previously dominated derived summaries?
- Does deduplication reduce redundant items that express the same observation under slightly different wording?
- Do concrete observations appear before abstract labels in ranked output?
- Does the view remain robust when consumed by the existing panel component (investigation-summary-panel-v3) without any changes to that component?
#### Evaluation
Completed. Tests: 37 scenarios passing across filtering, classification, deduplication, ranking, section framing, mock-data integration, and edge cases.
#### Result
Confirmed.
#### What did we learn?
- Semantic role routing outperforms kind-based routing: a node with `kind: "state"` that contains concrete data (e.g., "Revenue increased 12%") is more useful as an observation than a state description.
- Scaffolding suppression works best when applied early — filtering at the semantic classification stage prevents structural glue from contaminating any section.
- Three-tier filtering is effective: scaffolding patterns (highest priority), internal vocabulary (medium), then technical summary patterns (lowest).
- Deduplication by normalised text removes meaningful noise. When "Revenue increased 12%" and "Current revenue is 12% higher" express the same observation, keeping one reduces confusion without losing information.
- Resolved unknowns and assumptions are factual answers to previously unanswered questions — they should appear in the known section with an epistemic label ("Not yet established" / "To be tested") if their status hasn't been explicitly set.
- The translation adapter is the right place for this work: it is a single deterministic function, testable in isolation, and its output contracts are stable.
#### Decision
Keep the semantic projection approach. The facilitator view now routes by meaning, suppresses structural noise, deduplicates observations, and prefers concrete findings. Experiment 13 is closed.
---