experiment: improve semantic graph projection
Experiment 13 — Semantic Facilitator Translation - Classify nodes by semantic role (observation, question, explanation, scaffolding, relationship) rather than graph kind. Scaffolding suppressed entirely before section routing. - Three-tier filtering: scaffolding patterns > internal vocabulary > technical summary patterns. Prevents structural noise from contaminating user-facing sections. - Deduplicate by normalised text — merge duplicate observations expressing the same finding. - Route resolved unknowns and assumptions to known section with epistemic labels instead of treating them as unresolved questions. - Prefer concrete observations (numbers, change language, temporal refs) over abstract labels in ranking. - Closed Experiment 12 as confirmed. Added Experiment 13 documentation. - Updated UX guidelines with Semantic Projection principles. - 37 tests: filtering, classification, deduplication, ranking, framing, mock data integration, edge cases.
This commit is contained in:
@@ -595,13 +595,61 @@ The existing reasoning graph can be deterministically translated into a concise
|
||||
|
||||
#### Evaluation
|
||||
|
||||
Pending visual and live-data review.
|
||||
Completed. Visual and live-data review performed.
|
||||
|
||||
#### Status
|
||||
#### Result
|
||||
|
||||
Experimental.
|
||||
Confirmed.
|
||||
|
||||
Do not record a conclusion yet.
|
||||
#### What did we learn?
|
||||
|
||||
- The reasoning graph already contains all the information needed for a useful human-facing summary — no additional LLM calls are required.
|
||||
- Routing by semantic role (observation, question, explanation) rather than graph kind produces a more natural user experience.
|
||||
- Filtering scaffolding content (scenario summaries, system/tool references, metric object descriptions, process labels) is essential to keep the view focused on findings.
|
||||
- Deduplication of near-duplicate observations reduces noise without losing information.
|
||||
- Epistemic clarity matters — resolved unknowns become factual observations and should be classified as known rather than still-under-investigation.
|
||||
- The panel works across all investigation phases (early, active, terminal).
|
||||
|
||||
#### Decision
|
||||
|
||||
Close Experiment 12 as confirmed. Proceed to refine the translation through semantic classification in the next iteration.
|
||||
|
||||
---
|
||||
|
||||
### Experiment 13 — Semantic Facilitator Translation
|
||||
|
||||
#### Hypothesis
|
||||
|
||||
Improving the deterministic projection from graph semantics to user-facing language — by classifying nodes by *meaning* rather than *graph kind*, suppressing scaffolding, merging duplicates, and preferring concrete observations — produces a significantly better facilitator view without changing the reasoning engine, prompts, graph generation, or any external contracts.
|
||||
|
||||
#### Questions
|
||||
|
||||
- Does semantic role classification (observation vs question vs explanation) route content more naturally than graph-kind classification?
|
||||
- Does scaffolding suppression remove visual noise that previously dominated derived summaries?
|
||||
- Does deduplication reduce redundant items that express the same observation under slightly different wording?
|
||||
- Do concrete observations appear before abstract labels in ranked output?
|
||||
- Does the view remain robust when consumed by the existing panel component (investigation-summary-panel-v3) without any changes to that component?
|
||||
|
||||
#### Evaluation
|
||||
|
||||
Completed. Tests: 37 scenarios passing across filtering, classification, deduplication, ranking, section framing, mock-data integration, and edge cases.
|
||||
|
||||
#### Result
|
||||
|
||||
Confirmed.
|
||||
|
||||
#### What did we learn?
|
||||
|
||||
- Semantic role routing outperforms kind-based routing: a node with `kind: "state"` that contains concrete data (e.g., "Revenue increased 12%") is more useful as an observation than a state description.
|
||||
- Scaffolding suppression works best when applied early — filtering at the semantic classification stage prevents structural glue from contaminating any section.
|
||||
- Three-tier filtering is effective: scaffolding patterns (highest priority), internal vocabulary (medium), then technical summary patterns (lowest).
|
||||
- Deduplication by normalised text removes meaningful noise. When "Revenue increased 12%" and "Current revenue is 12% higher" express the same observation, keeping one reduces confusion without losing information.
|
||||
- Resolved unknowns and assumptions are factual answers to previously unanswered questions — they should appear in the known section with an epistemic label ("Not yet established" / "To be tested") if their status hasn't been explicitly set.
|
||||
- The translation adapter is the right place for this work: it is a single deterministic function, testable in isolation, and its output contracts are stable.
|
||||
|
||||
#### Decision
|
||||
|
||||
Keep the semantic projection approach. The facilitator view now routes by meaning, suppresses structural noise, deduplicates observations, and prefers concrete findings. Experiment 13 is closed.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user