robbond/rdb-discovery
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(task-023): improve generated context quality

Browse Source
This commit is contained in:
robbond
2026-06-04 09:42:05 +01:00
parent c72ad6995e
commit 152096d433
9 changed files with 191 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.rdb/session-log.jsonl
+20
View File
@@ -112,3 +112,23 @@
{"timestamp": "2026-06-04T06:38:19.019668+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T06:38:19.108828+00:00", "event_type": "command", "target": "rdb generate", "details": {}}
{"timestamp": "2026-06-04T06:38:19.717117+00:00", "event_type": "command", "target": "rdb status", "details": {}}
{"timestamp": "2026-06-04T06:42:57.711062+00:00", "event_type": "command", "target": "rdb next", "details": {}}
{"timestamp": "2026-06-04T06:43:06.047568+00:00", "event_type": "command", "target": "rdb prompt", "details": {}}
{"timestamp": "2026-06-04T06:44:15.981811+00:00", "event_type": "command", "target": "rdb status", "details": {}}
{"timestamp": "2026-06-04T06:46:32.334206+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T06:46:32.354546+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T06:46:32.384086+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T06:46:32.469898+00:00", "event_type": "command", "target": "rdb generate", "details": {}}
{"timestamp": "2026-06-04T07:55:17.697713+00:00", "event_type": "command", "target": "rdb generate", "details": {}}
{"timestamp": "2026-06-04T08:04:05.989316+00:00", "event_type": "command", "target": "rdb generate", "details": {}}
{"timestamp": "2026-06-04T08:05:02.054038+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T08:05:02.078565+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T08:05:02.105459+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T08:05:02.195107+00:00", "event_type": "command", "target": "rdb generate", "details": {}}
{"timestamp": "2026-06-04T08:07:55.244668+00:00", "event_type": "command", "target": "rdb status", "details": {}}
{"timestamp": "2026-06-04T08:12:44.332202+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T08:12:44.356488+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T08:12:44.386225+00:00", "event_type": "command", "target": "rdb context-status", "details": {}}
{"timestamp": "2026-06-04T08:12:44.484757+00:00", "event_type": "command", "target": "rdb generate", "details": {}}
{"timestamp": "2026-06-04T08:24:16.640627+00:00", "event_type": "command", "target": "rdb next", "details": {}}
{"timestamp": "2026-06-04T08:41:45.929837+00:00", "event_type": "command", "target": "rdb complete", "details": {"task_id": "TASK-023"}}
AGENT_HANDOFF.md
+1 -1
View File
@@ -6,7 +6,7 @@ TASKS_READY
## Current Task
TASK-021 (Done)
TASK-023
## Instructions For Agent
PROJECT_STATE.md
+2 -2
View File
@@ -4,7 +4,7 @@ Current Stage: BUILDING
Previous Stage: BOOTSTRAP_READY
Next Stage: REVIEW_READY
Current Task: TASK-021
Current Task: TASK-023
Active Branch: main
Last Updated: 2026-06-03
Last Updated: 2026-06-04
RUN_LOG.md
+1
View File
@@ -13,3 +13,4 @@ TASK-002 marked Done — agent prompt generation complete.
| 2026-06-02T12:51:20 | Task completed | TASK-007 | enhanced prompts |
| 2026-06-02T13:45:00 | Task completed | TASK-012 | telemetry foundation — writer, JSONL storage, event reader, 12 tests passed |
| 2026-06-02T17:05:41 | Task completed | TASK-015 | updated README.md |
| 2026-06-04T09:41:51 | Task completed | TASK-023 | updated |
TASKS.md
+1 -1
View File
@@ -553,7 +553,7 @@ Files modified:
## TASK-023 — Improve generated context quality
Status: Todo
Status: Done
Role: Implementation Agent
context/discovery-context-mapping.md
+5 -3
View File
@@ -129,7 +129,7 @@ Maps to:
| Context File | Section | Guidance |
|---|---|---|
| `context/development-context.md` | `## Build & Test` | Document test framework, test execution commands, and local setup steps. |
| `TEST_PLAN.md` | Manual/Automated sections | Convert into the test plan with specific test cases. Each discovery answer becomes one or more test entries. |
| `context/agent-guidelines.md` | `## Testing Preferences` (append) | Add a new section describing how tests should be written, run, and verified. Do **not** map testing answers into Timeline / Milestones sections in other files. |
### Q-010 — How will it be deployed?
@@ -147,6 +147,7 @@ Maps to:
| Context File | Section | Guidance |
|---|---|---|
| `context/company-context.md` | `## Mission` | Record the organisation's core mission, vision, or strategic goals that relate to this project. If no company-level mission exists, record the team's purpose. |
| `context/company-context.md` | `## Products & Services` (append) | Add a new section describing the company's products and services if the answer includes them. |
| `context/project-brief.md` | `## Target Audience` | Restate in project-specific terms — who will use or benefit from the final product? |
### Q-012 — What is the technology stack, repository structure, and architecture approach?
@@ -157,6 +158,7 @@ Maps to:
|---|---|---|
| `context/development-context.md` | `## Tech Stack` | Extract languages, frameworks, libraries, tooling, and build/test tools. Record each as a distinct item. |
| `context/development-context.md` | `## Coding Standards` | Note any style conventions, naming patterns, linters, formatters, or agreed-upon patterns. If none exist, note "TBD". |
| `context/repository-context.md` | `## Dependencies` (append) | Add a new section describing project dependencies if not yet documented. |
| `context/architecture.md` | `## Overview` | Record high-level design principles, architectural patterns (e.g., monolith, microservices), and key design decisions. |
| `context/architecture.md` | `## Core Components` | List the main modules, services, or packages and their responsibilities. Use bullet points for each component. |
@@ -196,10 +198,10 @@ This matrix provides a quick-reference view of all mappings. "Primary" means the
| Q-006 | Systems to connect to? | architecture.md, infrastructure-context.md | decisions.md |
| Q-007 | What are the risks? | risks.md, open-questions.md | — |
| Q-008 | What must not happen? | assumptions.md, decisions.md | open-questions.md |
| Q-009 | How will we test it? | development-context.md, TEST_PLAN.md | — |
| Q-009 | How will we test it? | development-context.md, agent-guidelines.md | — |
| Q-010 | How will it be deployed? | architecture.md, infrastructure-context.md | — |
| Q-011 | Who owns and directs the project? What are its organisational goals? | company-context.md, project-brief.md | — |
| Q-012 | What is the technology stack, repository structure, architecture approach? | development-context.md, architecture.md | — |
| Q-012 | What is the technology stack, repository structure, architecture approach? | development-context.md, repository-context.md, architecture.md | — |
| Q-013 | Where is it hosted, which environments exist, and what are the security requirements? | infrastructure-context.md | — |
| Q-014 | What agent and developer workflow tools, constraints, and guidelines apply? | agent-guidelines.md, repository-context.md | — |
src/rdb_discovery/generate_context.py
+9 -5
View File
@@ -27,6 +27,8 @@ CONTEXT_MAP = {
],
"Q-002": [ # Who is the user?
("context/product-brief.md", "## Users", "body"),
("context/company-context.md", "## Customers / Users", "body"),
("context/company-context.md", "## Key Stakeholders", "body"),
("context/development-context.md", "## IDEs and Editors", "body"),
],
"Q-003": [ # What does success look like?
@@ -53,7 +55,7 @@ CONTEXT_MAP = {
],
"Q-009": [ # How will we test it?
("context/development-context.md", "## Build & Test", "body"),
("context/project-brief.md", "## Timeline & Milestones", "body"),
("context/agent-guidelines.md", "## Testing Preferences", "append"),
],
"Q-010": [ # How will it be deployed?
("context/architecture.md", "## Deployment Architecture", "body"),
@@ -61,11 +63,13 @@ CONTEXT_MAP = {
],
"Q-011": [ # Who owns and directs the project, and what are its organisational goals?
("context/company-context.md", "## Mission", "body"),
("context/company-context.md", "## Products & Services", "body"),
("context/project-brief.md", "## Target Audience", "body"),
],
"Q-012": [ # What is the technology stack, repository structure, and architecture approach?
("context/development-context.md", "## Tech Stack", "body"),
("context/development-context.md", "## Coding Standards", "body"),
("context/repository-context.md", "## Dependencies", "body"),
("context/architecture.md", "## Overview", "body"),
("context/architecture.md", "## Core Components", "body"),
],
@@ -140,7 +144,7 @@ def _write_section(root, file_path, section_name, format_type, answer, rules):
if format_type == "table":
_write_table_row(target, section_name, answer)
elif format_type == "append":
_append_new_section(target, answer, rules)
_append_new_section(target, answer, rules, file_path)
else:
# body text type
existing = target.read_text(encoding="utf-8") if target.exists() else ""
@@ -245,13 +249,13 @@ def _write_table_row(target, section_header, answer):
target.write_text(existing + "\n" + row, encoding="utf-8")
def _append_new_section(target, answer, rules):
def _append_new_section(target, answer, rules, file_path):
"""Append a new section to the end of an existing file."""
existing = target.read_text(encoding="utf-8") if target.exists() else ""
display_name = None
for _, sec, _ in rules:
if sec and sec.startswith("## "):
for fpath, sec, _ in rules:
if fpath == file_path and sec and sec.startswith("## "):
display_name = sec
break
if not display_name:
src/rdb_discovery/templates.py
+9 -5
View File
@@ -156,7 +156,7 @@ TBD — How to build, test, and run the codebase locally.
## Dependencies
TDB — External services, databases, and third-party APIs required.
TBD — External services, databases, and third-party APIs required.
""",
"context/infrastructure-context.md": """# Infrastructure Context
@@ -178,7 +178,7 @@ TBD — Tools used for monitoring, logging, and alerting.
## Security
TDB — Authentication, data protection, and access control measures.
TBD — Authentication, data protection, and access control measures.
""",
"context/agent-guidelines.md": """# Agent Guidelines
@@ -188,7 +188,11 @@ TBD — What should agents know about working with this project?
## Preferred Tools
TDB — Recommended editors, debuggers, testing frameworks, and CLI tools.
TBD — Recommended editors, debuggers, testing frameworks, and CLI tools.
## Testing Preferences
TBD — How the team prefers tests to be written, run, and verified.
## Common Tasks
@@ -206,7 +210,7 @@ TBD — What problem does this project solve?
## Target Audience
TDB — Who is the intended audience?
TBD — Who is the intended audience?
## Key Features (MVP)
@@ -232,7 +236,7 @@ TBD — Key modules, services, and their responsibilities.
## Data Flow
TDB — How data moves through the system (APIs, queues, databases).
TBD — How data moves through the system (APIs, queues, databases).
## External Integrations
tests/test_generate_context.py
+143
View File
@@ -292,6 +292,7 @@ class TestIntegration:
assert "context/product-brief.md" in result["generated"]
assert "context/project-brief.md" in result["generated"]
assert "context/risks.md" in result["generated"]
assert "context/company-context.md" in result["generated"] # Q-002 → company-context
assert "context/assumptions.md" not in result["generated"] # Q-008 is Low confidence
# Check that product-brief has filled sections
@@ -299,6 +300,11 @@ class TestIntegration:
assert "CLI tool for discovery workflow" in content
assert "Internal developers and small dev teams" in content
# Check company-context got users data from Q-002
company = (tmp_project / "context/company-context.md").read_text()
assert "## Customers / Users" in company
assert "Internal developers and small dev teams" in company
# Check that risks.md got a table row
risks = (tmp_project / "context/risks.md").read_text()
assert "| RISK-007 |" in risks
@@ -314,3 +320,140 @@ class TestIntegration:
result = (tmp_project / "context/infrastructure-context.md").read_text()
assert "Cloud provider details" in result # Original content preserved
class TestContextQuality:
"""Tests for TASK-023: improved context generation quality."""
def test_company_context_populated_from_users(self):
"""company-context.md should be populated when discovery answers include users/stakeholders."""
from pathlib import Path
import tempfile
import shutil
tmp_path = Path(tempfile.mkdtemp())
root = tmp_path / "testproject"
root.mkdir()
(root / "context").mkdir()
# Only Q-002 (users) - no other discovery data
(root / "context" / "discovery-log.md").write_text(
"# Discovery Log\n\n"
"| ID | Question | Answer | Confidence | Follow-up needed | Linked decision | Linked task | Date |\n"
"|---|---|---|---|---|---|---|---|\n"
'| Q-002 | Who is the user? | Internal dev teams and external customers | High | No | | | 2026-06-03 |\n',
encoding="utf-8",
)
from rdb_discovery.generate_context import generate_context_files
result = generate_context_files(root)
assert "context/company-context.md" in result["generated"]
assert "context/product-brief.md" in result["generated"]
company = (root / "context/company-context.md").read_text()
assert "## Customers / Users" in company
assert "Internal dev teams and external customers" in company
shutil.rmtree(tmp_path)
def test_agent_guidelines_gets_testing_preferences(self):
"""agent-guidelines.md should get Testing Preferences from testing discovery answers."""
from pathlib import Path
import tempfile
import shutil
tmp_path = Path(tempfile.mkdtemp())
root = tmp_path / "testproject"
root.mkdir()
(root / "context").mkdir()
# Only Q-009 (testing) - no agent tooling questions
(root / "context" / "discovery-log.md").write_text(
"# Discovery Log\n\n"
"| ID | Question | Answer | Confidence | Follow-up needed | Linked decision | Linked task | Date |\n"
"|---|---|---|---|---|---|---|---|\n"
'| Q-009 | How will we test it? | pytest with coverage, integration tests via docker-compose | Medium | No | | | 2026-06-03 |\n',
encoding="utf-8",
)
from rdb_discovery.generate_context import generate_context_files
result = generate_context_files(root)
assert "context/agent-guidelines.md" in result["generated"]
guidelines = (root / "context/agent-guidelines.md").read_text()
assert "## Testing Preferences" in guidelines
assert "pytest with coverage" in guidelines
shutil.rmtree(tmp_path)
def test_no_testing_answers_in_timeline(self):
"""Testing answers must not be mapped to Timeline & Milestones sections."""
from pathlib import Path
import tempfile
import shutil
tmp_path = Path(tempfile.mkdtemp())
root = tmp_path / "testproject"
root.mkdir()
(root / "context").mkdir()
# Only Q-009 (testing) - verify no timeline data appears
(root / "context" / "discovery-log.md").write_text(
"# Discovery Log\n\n"
"| ID | Question | Answer | Confidence | Follow-up needed | Linked decision | Linked task | Date |\n"
"|---|---|---|---|---|---|---|\n"
'| Q-009 | How will we test it? | pytest with coverage | High | No | | | 2026-06-03 |\n',
encoding="utf-8",
)
# Create project-brief.md that has a Timeline section already
(root / "context" / "project-brief.md").write_text(
"# Project Brief\n\n## Timeline & Milestones\n\nTBD\n",
encoding="utf-8",
)
from rdb_discovery.generate_context import generate_context_files
result = generate_context_files(root)
assert "context/project-brief.md" not in result["generated"]
timeline_content = (root / "context/project-brief.md").read_text()
# The Timeline section should remain as TBD (not filled with testing data)
assert "pytest with coverage" not in timeline_content
# Original TBD placeholder preserved
assert "TBD" in timeline_content
shutil.rmtree(tmp_path)
def test_repository_context_populated_from_dependencies(self):
"""repository-context.md should be populated when answers include dependencies."""
from pathlib import Path
import tempfile
import shutil
tmp_path = Path(tempfile.mkdtemp())
root = tmp_path / "testproject"
root.mkdir()
(root / "context").mkdir()
# Q-012 includes technology stack info that maps to repository dependencies
(root / "context" / "discovery-log.md").write_text(
"# Discovery Log\n\n"
"| ID | Question | Answer | Confidence | Follow-up needed | Linked decision | Linked task | Date |\n"
"|---|---|---|---|---|---|---|---|\n"
'| Q-012 | What is the technology stack, repository structure, and architecture approach? | Python 3.12, Typer, Rich; Docker; PostgreSQL | High | No | | | 2026-06-03 |\n',
encoding="utf-8",
)
from rdb_discovery.generate_context import generate_context_files
result = generate_context_files(root)
assert "context/repository-context.md" in result["generated"]
repo_ctx = (root / "context/repository-context.md").read_text()
assert "## Dependencies" in repo_ctx
assert "Python 3.12, Typer, Rich" in repo_ctx
shutil.rmtree(tmp_path)