From ad0d2bf285fc5a174d7e841653fe015ebd07d2db Mon Sep 17 00:00:00 2001 From: robbond Date: Tue, 24 Mar 2026 12:26:32 +0000 Subject: [PATCH 1/2] docs(governance): track context docs and add relay rollout controls --- .gitignore | 1 - context/architecture.md | 77 ++++++++++++++ context/coding-conventions.md | 52 +++++++++ context/domain-flows.md | 195 ++++++++++++++++++++++++++++++++++ context/integration-map.md | 116 ++++++++++++++++++++ context/onboarding-guide.md | 112 +++++++++++++++++++ context/project-overview.md | 88 +++++++++++++++ context/runbook.md | 118 ++++++++++++++++++++ memory-bank/change-log.md | 33 ++++++ memory-bank/decisions.md | 32 ++++++ memory-bank/patterns.md | 24 +++++ 11 files changed, 847 insertions(+), 1 deletion(-) create mode 100644 context/architecture.md create mode 100644 context/coding-conventions.md create mode 100644 context/domain-flows.md create mode 100644 context/integration-map.md create mode 100644 context/onboarding-guide.md create mode 100644 context/project-overview.md create mode 100644 context/runbook.md diff --git a/.gitignore b/.gitignore index 8a8ed18a..28014204 100644 --- a/.gitignore +++ b/.gitignore @@ -57,6 +57,5 @@ pages/holding.html CONTRIBUTING_AI.md GUARDRAILS.md ai-prompts/ -context/ workflows/ AI_CONTEXT.md diff --git a/context/architecture.md b/context/architecture.md new file mode 100644 index 00000000..8fc7a59c --- /dev/null +++ b/context/architecture.md @@ -0,0 +1,77 @@ +# Architecture Reference + +## Runtime Topology + +1. Next.js runtime serves UI routes and API routes (legacy custom server files are present but not active). +2. Next.js `pages/` router handles UI routes and API routes under `pages/api/**`. +3. Middleware (`middleware.js`) injects CSP nonce and security headers on requests/responses. +4. State management initialized in `pages/_app.js` with Redux wrapper and persistence. +5. Authentication handled by `next-auth` in `pages/api/auth/[...nextauth].js` with Prisma adapter. + +## Key Architectural Modules + +- **Presentation layer:** `pages/`, `components/`, `styles/` +- **State layer:** `store/store.js` + slice reducers +- **Domain/service helpers:** `actions/`, `lib/` +- **Auth persistence:** Prisma client + `prisma/schema.prisma` (next-auth tables in SQL Server) +- **Portal business data access:** API routes query Dynamics 365 CRM through Azure Service Bus Relay using REST + OData patterns +- **Relay integrity check:** forwarded CRM-bound API requests include a path-based hash, validated by relay with shared key before forwarding +- **Infrastructure glue:** `middleware.js`, `next.config.js`, `i18n.js` (plus legacy server files not in active runtime) + +## High-Risk/Guarded Paths + +1. `pages/api/auth/[...nextauth].js` + - Email sign-in flow, callback/redirect logic, session setup. +2. `middleware.js` and security headers in `next.config.js` + - CSP and browser hardening policies. +3. `store/store.js` + - HYDRATE, persistence, logout storage clearing. +4. `prisma/schema.prisma` + - Source of truth for auth/account persistence tables. +5. File and notification APIs in `pages/api/file/**`, `pages/api/email/**` + - Upload/document/email side effects and sensitive data handling. + +## i18n and Routing Model + +- Locales: `en`, `cy` (configured in `i18n.js`). +- Locale detection disabled; domain and route rewrites drive behavior. +- Welsh route aliases maintained in `next.config.js` rewrites. +- Any new user-facing route should consider: + - translation resources in `locales/en` and `locales/cy` + - rewrite parity where a Welsh alias is expected + - auth pages and callback URLs for locale correctness + +## External Integration Touchpoints + +- Dynamics 365 CRM (portal business data): reached via frontend API routes that send REST/OData queries through Azure Service Bus Relay. +- Azure Service Bus Relay request integrity: relay endpoint is configured via `API_ROOT`; request path (excluding domain) is hashed client-side and validated relay-side with shared hash key. + +- Azure Storage/Queue: `actions/azurestorage.js`, selected API handlers. +- GOV.UK Notify email: `actions/index.js`, `pages/api/email/**`, next-auth email provider. +- Application Insights: `components/azureappinsights.js` and related environment configuration. +- Mapping embeds and map libs: `components/mapping/**`, DNS/search components. +- PDF generation/rendering: `pages/api/file/generate*.js`, `components/pdftemplates/**`. + +## Current Endpoint Contract Hardening Status (2026-03) + +Recent bounded slices in the endpoint contract-consistency stream have standardized selected high-traffic handlers from raw relay error passthrough to structured response contracts (`respondSuccess` / `respondError`) with explicit required-input guards and phase21 contract coverage. + +Completed clusters include: + +- Search document retrieval cluster (`getsearchdocumenthistory*`, `getsearchdocumentdetails*`, `getsearchdocumentTypes_api`) +- My portal retrieval cluster (`getmycases_api`, `getmyrepresentations_api`, `getwatchedcases_api`, `getawaitingsubmission_api`) +- Basic search family cluster (`getbasicsearchdetails_api`, `getbasicsearchdetailspaged_api`, `getbasicsearchpaged_api`, `getbasicsearch_by_lparref_api`) + +Guardrail note: success payload contracts are intentionally preserved to avoid frontend regressions, while negative-path behavior is being normalized endpoint-by-endpoint with corresponding phase21 tests. + +## Legacy / Inactive Components + +The following files exist in the repository but are not part of the current active runtime model: + +- `server.js` +- `server/server.js` + +Guidance: + +- Do not treat these files as active runtime architecture unless explicitly reactivated. +- If reactivation is proposed, document rationale and rollout/rollback in `memory-bank/change-log.md` and `context/runbook.md`. diff --git a/context/coding-conventions.md b/context/coding-conventions.md new file mode 100644 index 00000000..bb239c0d --- /dev/null +++ b/context/coding-conventions.md @@ -0,0 +1,52 @@ +# Coding Conventions (Repository-Specific) + +## Language and Style + +- Default to JavaScript for new work unless area is already TypeScript. +- Use existing formatting: + - 4 spaces + - no trailing commas + - preserve existing quote style in touched file +- Avoid broad stylistic rewrites in functional changes. + +## File Placement and Boundaries + +- Keep `pages/` focused on route wiring, data fetch orchestration, and composition. +- Move reusable domain logic into `lib/` or `actions/`. +- Keep reusable UI in `components/` (feature subfolders preferred). +- Keep Redux behavior changes localized to relevant slice and `store/store.js` integration points. + +## API Route Conventions + +- Follow existing API naming where present (`*_api.js` in `pages/api/endpoint`, `pages/api/file`, `pages/api/admin`). +- Maintain explicit error handling and return predictable response shapes. +- For sensitive endpoints, include authorization and invalid-input paths. + +## Auth/Session Conventions + +- `pages/api/auth/[...nextauth].js` is authoritative for auth flow behavior. +- Do not bypass session checks in protected routes/components. +- Preserve secure redirect/callback behavior and cookie safety settings. + +## i18n Conventions + +- Add user-facing strings to locale files (`locales/en`, `locales/cy`) where feasible. +- Keep EN/CY route behavior aligned when adding/changing pages. +- If a route alias changes in Welsh, verify rewrite mapping in `next.config.js` and page namespaces in `i18n.js`. + +## Accessibility Conventions + +- Use semantic HTML and maintain heading hierarchy. +- Ensure keyboard access and visible focus states. +- Provide meaningful labels and error messages. + +## Logging and Error Handling + +- Never log secrets, tokens, personal details, or raw credential payloads. +- Keep operational logs concise and safe for production telemetry. +- Prefer structured, actionable errors over generic catch-and-ignore patterns. + +## Legacy and Mixed-Pattern Guidance + +- Where conventions conflict between folders, follow the dominant local pattern and record exceptions in `memory-bank/decisions.md`. +- Refactor incrementally: separate behavior changes from structural cleanup. diff --git a/context/domain-flows.md b/context/domain-flows.md new file mode 100644 index 00000000..a1eebc00 --- /dev/null +++ b/context/domain-flows.md @@ -0,0 +1,195 @@ +# Domain Flows Reference + +## 1) Public Search and Case Discovery + +**Representative routes** + +- `/search`, `/searchresults`, `/advancedsearch`, `/advancedsearchresults`, `/addresssearch`, `/addresssearchresults` +- `/case/[ticketnumber]`, `/dnsapplications`, `/dnsdetails` + +**Expected behavior** + +- Users can discover case information without authentication for public data. +- Search and result pagination should remain performant and stable. +- Route aliases in Welsh should mirror English flow intent. +- Search/case data queries route through frontend APIs to Dynamics 365 CRM via Azure Service Bus Relay. +- Relay-bound requests include path-based hash validation (computed client-side and verified relay-side with shared key). +- Case reference selection from results routes the user to a case summary page where representations may be initiated. + +### 1a) Basic search results page (UI behaviour) + +- Route is served from the public search flow and presents a dedicated **Search results** page for a simple keyword/reference query. +- Results heading shows: + - a result count (`@odata.count` from CRM response) + - the searched term in context (for basic search) + - a linked-case title variant when `lk=1` is present. +- Results are displayed as a tabular summary list with columns for: + - Case reference + - Site address + - Applicant + - Authority (including LPA reference where available) + - Case type + - Status +- Case reference links route to case detail (`/case/[ticketnumber]` or locale/my-portal equivalents), carrying current query context. + +### 1b) Search results interaction model + +- Supports column sorting (reference, applicant, authority, case type, status), with ascending/descending state. +- Supports page size selection (5/10/20/30/50) and paginated retrieval. +- While data is loading, page shows a spinner/loading message; CRM error paths render the CRM error component. +- If no matches are returned, page shows a clear no-records state. +- A follow-on CTA links users to advanced search. + +### 1c) Search matching and data presentation + +- Basic search can match on case/application references and surfaced metadata (for example authority refs shown in results rows). +- Matching query terms are highlighted in key visible fields where applicable. +- User-facing formatted values (case type/status/authority labels) are locale-aware, with EN/CY translation mapping applied. + +**Change risks** + +- Breaking query/filters or pagination contracts. +- Locale rewrite divergence causing EN/CY mismatch. +- Relay or CRM query regressions causing empty/slow/inconsistent results. +- Hash mismatch regressions causing relay rejection of otherwise valid requests. + +## 2) Account and Authentication + +**Representative routes/APIs** + +- `/auth/signin`, `/auth/verify-request`, `/account/register` +- `pages/api/auth/[...nextauth].js` + +**Expected behavior** + +- Email sign-in and callback flow should preserve secure redirect behavior. +- Session and account persistence must align with Prisma next-auth models in SQL Server. +- Locale-aware sign-in experience should remain coherent. +- Newly registered/authenticated users are assigned **Interested Party** involvement by default. + +**Change risks** + +- Callback URL mishandling, cross-domain issues, insecure redirects. +- Session expiry/update or cookie behavior regressions. + +## 3) My Portal / Casework Actions + +**Representative routes** + +- `/myportal`, `/myportal/viewall`, `/myportal/case/[ticketnumber]` +- new appeal, watched-cases, submissions, and representation workflows + +**Expected behavior** + +- Authenticated users manage personal case interactions reliably. +- Redux persistence + hydration should keep client/server state coherent. +- Portal data retrieval should remain stable through relay-backed CRM API queries. +- Dashboard capabilities include: + - **Make a new appeal** (guided appeal-type selection) + - **Search for a case** by case/application/project reference + - panel-based worklists for in-progress and submitted user activity + +### 3a) Portal dashboard panel architecture + +- The dashboard is a panel grid with two panel types: + - **Action panels:** start key actions (new appeal, case search, DNS/project listing). + - **Worklist panels:** list records by user relationship and workflow state. +- Worklist panel groups shown in UI include: + - Appeals awaiting submission + - My cases + - Watched cases + - Representations awaiting submission + - Submitted representations + +### 3b) Case lifecycle and statuses (portal-visible) + +- From dashboard behavior, case/representation records appear in practical states: + - **Awaiting submission** (draft/incomplete, resumable) + - **Pending** (submitted and active in processing) + - **Submission processing** (recently submitted representation still being processed) +- The UI separates these states into distinct panels rather than one combined status list. + +### 3c) Domain entities used in dashboard flows + +- **Case:** core planning case record identified by case reference (for example `CAS-...`). +- **Appeal:** a casework submission type associated with an appeal type and LPA. +- **Representation:** a statement/consultation response linked to a case. +- **LPA:** Local Planning Authority associated with the case/appeal and shown as context metadata. + +### 3d) User-to-case relationships + +- **Owner / My case:** user is the primary submitting party (“My cases” panel). +- **Watcher:** user follows case updates without being the owning submitter (“Watched cases” panel). +- **Representative / On behalf of:** user acts for another party in case/appeal activity (shown as “On behalf of” in case cards). +- Related role constraints remain: + - users become **Appellants** when they raise a new appeal + - CRM contact constraints allow only one role type, so Appellant remains their role once assigned + - **Agents** can submit appeals on behalf of multiple Appellants + - **LPA** users receive a different dashboard view focused on appeals in their local authority + - LPA users do not have the raise-appeal action in dashboard UI + - LPA users can raise/submit representations on appeals + +**Change risks** + +- HYDRATE/persist regressions causing stale or missing state. +- Unauthorized access gaps in protected paths. + +## 4) New Appeal and Representation Submission + +**Representative areas** + +- `pages/newappeal/**`, `components/newappeal/**` +- representation flows in `components/case/representation/**` + +**Expected behavior** + +- Form progression, save/continue, and submission paths are resilient. +- Generated PDFs/documents and notifications are consistent with submitted data. +- “Make representation” availability on case summary is criteria/date dependent. +- Users can begin representation flow from summary, but must be authenticated to submit. + +### 4a) Representation submission workflow (dashboard view) + +1. User finds/selects a case (search or existing case list). +2. User starts a representation (for example statement/consultation response). +3. Incomplete work appears in **Representations awaiting submission**. +4. On submit, status may briefly show **Submission processing**. +5. Completed items move to **Submitted representations**. + +### 4b) Case search behaviour (portal dashboard) + +- Search accepts multiple reference types from one input: + - case reference + - infrastructure project reference + - LPA application reference +- UI guidance shows accepted patterns similar to: + - `APP-A12345-A-21-1234567` + - `CAS-12345-A1B2C3` +- Search panel provides direct links to **Address search** and **Advanced Search** for broader discovery routes. + +**Change risks** + +- Data mapping/validation drift; malformed uploads or document payloads. +- Missing notifications or incorrect template/locale usage. + +## 5) Document, Upload, and Notification Processing + +**Representative APIs** + +- `pages/api/file/**` +- `pages/api/email/**` + +**Expected behavior** + +- File handling respects allowed types/size and error paths. +- Email/notification side effects are reliable and observable. + +**Change risks** + +- Unsafe file handling, leakage of sensitive details in logs, or queue failures. + +## Cross-Flow Non-Functional Expectations + +- Accessibility: keyboard/focus/labels/semantic headings for UI changes. +- Bilingual parity: EN/CY text/routes maintained together. +- Public-service reliability: avoid user-visible regressions in key journeys. diff --git a/context/integration-map.md b/context/integration-map.md new file mode 100644 index 00000000..9f4384af --- /dev/null +++ b/context/integration-map.md @@ -0,0 +1,116 @@ +# Integration Map + +## Overview + +This map identifies external or boundary integrations, where they are used, and what to validate when touched. + +## 1) Authentication + Account Persistence + +- **Tech:** `next-auth`, Prisma adapter, SQL Server via Prisma +- **Primary files:** + - `pages/api/auth/[...nextauth].js` + - `prisma/schema.prisma` +- **Data sensitivity:** High (session/account identity) +- **Validation focus:** + - Sign-in, verify-request, error, and callback behavior + - Safe redirect handling + - Session expiry and cookie behavior + +## 1a) Portal Data Access (CRM via Relay) + +- **System of record:** Microsoft Dynamics 365 CRM (portal business/case data) +- **Query style:** REST API calls and OData query patterns +- **Transport path:** Frontend API routes -> Azure Service Bus Relay -> Dynamics 365 CRM +- **Relay endpoint config:** `API_ROOT` environment variable +- **Request integrity:** Request path (excluding domain) is hashed before forwarding; relay recomputes using shared hash key and validates match +- **Primary files:** + - `pages/api/endpoint/*_api.js` + - `actions/index.js` (query construction/helpers, hash generation) +- **Data sensitivity:** High (case and user-associated business data) +- **Validation focus:** + - Query correctness and filter safety + - Relay failure/timeout handling + - Hash validation compatibility between frontend and relay + - API contract consistency between endpoint handlers + - No sensitive payload leakage in logs + +## 2) GOV.UK Notify (Email) + +- **Tech:** `notifications-node-client` +- **Primary files:** + - `actions/index.js` + - `pages/api/email/notify.js` + - `pages/api/email/getall.js` + - next-auth email provider in `pages/api/auth/[...nextauth].js` +- **Data sensitivity:** Medium-High (contact details + transactional messaging) +- **Validation focus:** + - Correct template selection (EN/CY) + - Error handling and retry/fallback behavior + - No sensitive data leakage in logs + +## 3) Azure Storage + Queues + +- **Tech:** `@azure/storage-blob`, `@azure/storage-queue`, managed identity patterns +- **Primary files:** + - `actions/azurestorage.js` + - `pages/api/file/**` + - selected admin utils under `components/admin/utils/**` +- **Data sensitivity:** High (documents/uploads) +- **Validation focus:** + - File type/size/path validation + - Upload/download/delete authorization boundaries + - Queue message integrity and failure handling + +## 4) Application Insights + +- **Tech:** `applicationinsights`, React app insights integration +- **Primary files:** + - `components/azureappinsights.js` +- **Data sensitivity:** Medium (telemetry may include operational metadata) +- **Validation focus:** + - Telemetry starts only when configured + - No personal/sensitive payloads in custom logs/events + +## 5) Mapping + +- **Tech:** `leaflet`, `react-leaflet`, `google-map-react`, embedded map URLs +- **Primary files:** + - `components/mapping/**` + - `components/search/dnssearchresults.js` + - CSP/middleware references to map frame sources +- **Validation focus:** + - CSP/frame-src compatibility + - Graceful handling when external map resources are unavailable + +## 6) PDF and Document Generation + +- **Tech:** `@react-pdf/renderer`, template components, upload APIs +- **Primary files:** + - `pages/api/file/generatepdf.js` + - `pages/api/file/generateappealpdf.js` + - `components/pdftemplates/**` +- **Validation focus:** + - Correct template selection by appeal/document type + - Encoding/formatting robustness for submitted rich text + - Safe download headers and document naming + +## Integration Change Checklist (apply whenever any integration is modified) + +1. Confirm required env vars are documented (without exposing values). +2. Verify timeout/error/negative path behavior. +3. Validate EN/CY output where user-facing content is integration-driven. +4. Record risk and validation evidence in PR and `memory-bank/change-log.md`. + +### Relay policy change minimum evidence (operational) + +For changes to shared relay timeout/retry/logging policy, include: + +1. Non-prod smoke evidence for: + - deterministic non-retry classes (`400`, `401`, `403`, `404`) + - transient retry classes (`429`, `503`, timeout transport failures) +2. Structured redacted logging verification and duplicate-log suppression confirmation. +3. Rollback/migration controls: + - quick config mitigation (`RELAY_RETRY_MAX=0`) + - full commit revert path. + +Reference: `context/runbook.md` -> **Relay Hardening Rollout Playbook (TASK22239)**. diff --git a/context/onboarding-guide.md b/context/onboarding-guide.md new file mode 100644 index 00000000..f13aecda --- /dev/null +++ b/context/onboarding-guide.md @@ -0,0 +1,112 @@ +# Engineer Onboarding Guide — PEDW FrontEnd + +## 1) Project Overview + +PEDW FrontEnd is the Planning and Environment Decisions Wales (PEDW) portal for discovering planning appeals and accessing personalised casework journeys. It supports public search/browse and authenticated dashboard workflows, with strong accessibility and bilingual (English/Welsh) requirements. + +## 2) Technology Stack + +- **Frontend:** Next.js 14 (Pages Router), React 18 +- **Language:** Primarily JavaScript (some TypeScript tooling present) +- **State:** Redux, `next-redux-wrapper`, `redux-persist`, `redux-thunk`, `redux-form` +- **Auth:** `next-auth` + Prisma adapter (email magic-link flow) +- **Auth data layer:** Prisma + SQL Server (`prisma/schema.prisma`) for next-auth identity/session tables +- **Portal business data layer:** Dynamics 365 CRM queried via REST + OData through frontend API routes +- **i18n:** `next-translate` + `i18n.js` + Welsh rewrites in `next.config.js` +- **Integrations:** Azure Service Bus Relay (CRM transport), Azure Blob/Queue, GOV.UK Notify, Application Insights, mapping (Leaflet/google-map-react), PDF generation + +## 3) Architecture Overview + +- Active runtime is **Next.js runtime** (legacy custom server files exist but are inactive). +- UI routes and APIs live in `pages/` and `pages/api/**`. +- Security controls are applied through `middleware.js` + security headers in `next.config.js`. +- Auth/session logic is centralized in `pages/api/auth/[...nextauth].js`. +- Shared integration/service logic sits mostly in `actions/` (notably large `actions/index.js`). +- Portal data calls are sent from `pages/api/endpoint/**` to Dynamics 365 CRM through Azure Service Bus Relay. +- For relay-bound calls, a hash is generated from request path (excluding domain) and appended; relay validates using the same shared hash key. + +## 4) Repository Structure + +- `pages/` — routes + API handlers +- `components/` — UI/features (case, DNS, account, admin, mapping, PDF templates) +- `actions/` — API client and side-effect helpers +- `lib/` — reusable form/domain helpers +- `store/` — Redux reducers/store setup/hydration/persistence +- `prisma/` — schema + migrations +- `locales/` — EN/CY translations +- `data/` — lookup and form metadata files +- `tests/` — test area (appears limited) +- `server/`, `server.js` — legacy/inactive runtime artifacts + +## 5) Core System Components + +- **Public search/case flows:** basic search (`/search`), advanced search (`/advancedsearch`), and address search (`/addresssearch`) with result pages and case detail/document UIs +- **Account/auth:** next-auth email verification and Prisma-backed sessions +- Portal routes (`/myportal/**`): user-specific case/representation/watchlist workflows +- **File/doc pipeline:** upload/download/blob flows and PDF generation under `pages/api/file/**` +- **Notifications:** GOV.UK Notify integrations under `pages/api/email/**` + auth email provider + +## 5a) User Involvement / Role Model + +- Newly registered/authenticated users default to **Interested Party** involvement. +- Users who raise a new appeal become **Appellants**. +- CRM contact constraints allow only one role type; once Appellant is assigned, it remains their role. +- **Agents** can submit appeals on behalf of multiple Appellants. +- **LPA (Local Planning Authority)** users are a separate persona with a distinct dashboard view. +- LPA dashboards focus on appeals within that authority. +- LPA users cannot raise appeals (option is hidden), but they can submit representations. + +## 6) Data Model + +Prisma schema is focused on next-auth persistence (SQL Server): + +- `User` +- `Account` +- `Session` +- `VerificationToken` + +Datasource is SQL Server (`DATABASE_URL`). This model underpins authentication/session behavior and should be treated as sensitive core infrastructure. Portal business/case data is sourced separately from Dynamics 365 CRM via relay-backed API calls. + +## 7) Key Workflows + +- **Sign-in:** user requests magic link -> email via Notify -> callback/session via next-auth. +- **Search journey:** frontend query -> `pages/api/endpoint/**` proxy endpoint(s) -> path hash appended -> Azure Service Bus Relay (validates hash with shared key) -> Dynamics 365 CRM (REST/OData) -> normalized UI rendering. +- **Case summary to representation:** user selects case reference from results -> lands on case summary -> “make representation” shown only when criteria/date rules allow -> user can start flow but must be authenticated to submit. +- **Dashboard journey (`/myportal`):** signed-in users can raise new appeals, search appeals, view watched cases, view partially completed appeals, view submitted appeals, and manage submitted/partially submitted representations. +- **Role behavior:** default involvement starts as Interested Party; new appeal creation sets Appellant role (persistent due to CRM single-role contact model); Agent users may act for multiple Appellants. +- **LPA dashboard behavior:** LPA users see an authority-scoped dashboard, do not see raise-appeal options, and can raise representations on existing appeals. +- **Document workflow:** user upload/submit -> blob storage + metadata updates -> PDF/document retrieval. +- **Bilingual routing:** Welsh aliases rewired in `next.config.js`, locale resources in `locales/en|cy`, page namespace mapping in `i18n.js`. + +## 8) Development Workflow + +From scripts and runbook: + +- `npm run dev` — local dev +- `npm run build` + `npm start` — production build/start +- `npm run lint` — baseline validation + +Expected change protocol emphasizes: + +- scoped changes, +- EN/CY parity checks, +- accessibility smoke checks, +- negative-path checks on sensitive flows, +- memory-bank/log updates for non-trivial changes. + +## 9) Observed Conventions + +- 4-space indentation, no trailing commas (per local conventions) +- Keep page-level logic thin where possible; place reusable logic in `lib/`/`components/`/`actions/` +- API naming pattern commonly uses `*_api.js` +- Heavy use of proxy-style API handlers +- Existing AI governance artifacts define guardrails (`.clinerules`, `GUARDRAILS.md`, `context/`, `memory-bank/`) + +## 10) Potential Risks / Weak Areas + +1. **Large `actions/index.js` coupling** (many responsibilities in one module) +2. **API contract inconsistency** (error/response handling varies across endpoints) +3. **Extensive `console.log` footprint** (signal/noise and potential sensitive logging concerns) +4. **Duplication across API proxy handlers** (token/header/hash/relay logic repeated) +5. **i18n complexity drift risk** (rewrite + locale namespace parity maintenance) +6. **Limited automated tests for high-risk flows** (manual validation burden) diff --git a/context/project-overview.md b/context/project-overview.md new file mode 100644 index 00000000..dab2a4f9 --- /dev/null +++ b/context/project-overview.md @@ -0,0 +1,88 @@ +# PEDW FrontEnd Project Overview + +## Purpose + +PEDW FrontEnd is a public-service web platform for planning casework and DNS (Developments of National Significance) journeys, with public search and citizen self-service plus authenticated portal/admin capabilities. + +## Current Stack + +- **Frontend:** Next.js 14 (`pages` router), React 18 +- **Server runtime:** Next.js runtime (legacy server files remain in-repo but are not used) +- **State:** Redux + `next-redux-wrapper` + `redux-persist` +- **Auth:** `next-auth` with Prisma adapter and email sign-in flow +- **Auth persistence:** Prisma + SQL Server schema (`prisma/schema.prisma`) for next-auth tables +- **Portal case data source:** Microsoft Dynamics 365 CRM (queried via REST + OData) +- **Portal API transport path:** Frontend API routes -> Azure Service Bus Relay -> Dynamics 365 CRM +- **Relay request integrity:** Forwarded API calls include a hash derived from request path (excluding domain). Relay recomputes hash with shared key and rejects mismatches. +- **i18n:** `next-translate`, locales `en` and `cy`, Welsh rewrites in `next.config.js` +- **Integrations:** Azure storage/queues, GOV.UK Notify, Application Insights, mapping, PDF generation + +## Repository Shape (Intent) + +- `pages/`: routes and API handlers (`pages/api/**`) +- `components/`: UI and feature components (case, account, admin, mapping, PDF templates) +- `actions/`: API client + shared side-effect logic +- `lib/`: reusable domain helpers and form-related logic +- `store/`: Redux reducers/store hydration/persistence +- `prisma/`: schema and migrations +- `locales/`: translation resources (EN/CY) +- `server/` + root `server.js`: legacy runtime files retained in repository + +## Core User/Business Areas + +1. Public appeal discovery via: + - basic search (`/search`) + - advanced search (`/advancedsearch`) + - address search (`/addresssearch`) +2. Search results and case summary journey (case reference selection -> summary page) +3. Representation flow from case summary (criteria/date dependent “make representation” action) +4. Passwordless authentication via next-auth magic links for authenticated submissions +5. Authenticated dashboard (`/myportal/**`) for personalised casework +6. New appeal and representation lifecycle management (partial + submitted states) +7. Admin/document workflows + +## Planning Casework Portal Dashboard (My Portal) + +The Planning Casework dashboard is the authenticated Appellant/Interested party user’s operational home page for: + +- starting new appeal submissions +- Viewing the users submitted appeals +- finding existing cases quickly +- resuming in-progress submissions +- monitoring submitted cases/representations +- tracking watched cases +- Information to view infrastructure project activity + +The Planning Casework dashboard is the authenticated LPA user’s operational home page for: + +- finding existing cases quickly +- resuming in-progress submissions +- monitoring submitted cases/representations +- tracking watched cases +- Information to view infrastructure project activity + +The dashboard is organized into task-focused panels that separate **action entry points** (for example, “Make a new appeal”, “Search for a case”) from **state-based worklists** (for example, “Appeals awaiting submission”, “My cases”, and representation lists). + +## Public Basic Search Results (Search Journey) + +In the public case-discovery journey, the basic search results page provides a structured list of matched cases and key metadata. The page supports sorting and pagination, links each case reference through to case detail, and presents a clear no-results or loading/error state when appropriate. + +Results content is locale-aware (EN/CY labels and formatted values) and includes highlighted query matches in key visible fields where supported by returned data. + +## User Involvement Model (Portal) + +- Newly registered/authenticated portal users default to **Interested Party** involvement. +- Users who raise a new appeal become **Appellants**. +- CRM contact constraints allow only one role type, so once set to Appellant this remains their role. +- **Agents** can submit appeals on behalf of multiple Appellants. +- **LPA (Local Planning Authority)** users have a distinct persona/dashboard view scoped to appeals raised within their authority. +- LPA users do **not** see the “raise appeal” option. +- LPA users can still make/submit representations on appeals. + +## Operating Constraints + +- Public-sector reliability expectations: avoid regressions on live service paths. +- Bilingual parity is mandatory for user-facing route/content changes. +- Accessibility expectations are high (keyboard and semantic behavior). +- Security-sensitive areas include auth/session, uploads/documents, notifications, and account data. +- Distinguish data paths: next-auth identity/session data is persisted in SQL Server; portal business data is sourced from Dynamics 365 CRM via relay-backed API calls with request-path hash validation. diff --git a/context/runbook.md b/context/runbook.md new file mode 100644 index 00000000..81c522a8 --- /dev/null +++ b/context/runbook.md @@ -0,0 +1,118 @@ +# Engineering Runbook (PEDW FrontEnd) + +## Purpose + +Provide a repeatable operational playbook for safe delivery and incident-aware change management. + +> Pre-flight: run `GUARDRAILS.md` checklist before significant implementation or merge. + +## Standard Change Workflow + +1. Clarify scope, constraints, and affected flows (public search/case, account/auth, portal/admin). +2. Identify risk level: + - **High:** auth/session, Prisma schema, middleware/CSP, uploads/documents, notifications + - **Medium:** routing/i18n rewrites, Redux hydration/persistence, PDF generation + - **Low:** isolated UI or content-only updates +3. Implement smallest viable change with clear rollback path. +4. Validate (minimum): + - `npm run lint` + - targeted manual checks for touched routes/APIs + - EN/CY parity checks + - accessibility smoke (keyboard, focus, labels, headings) + - negative-path checks for sensitive logic +5. Record outcomes in `memory-bank/change-log.md`. + +## Pre-Release Checklist + +- [ ] Scope and non-goals documented +- [ ] Risk notes cover auth/data/i18n/a11y impacts +- [ ] Validation evidence captured (command output + manual matrix) +- [ ] Rollback steps prepared +- [ ] Memory-bank updated for decisions/pitfalls/open questions + +## Incident Triage (Quick) + +1. Classify impact: service unavailable, degraded flow, data/security concern. +2. Identify blast radius: which routes/APIs/locales/users are affected. +3. Check telemetry and logs (server + app insights) for errors around deploy window. +4. Apply mitigation: + - rollback/revert high-risk change, + - or ship minimal hotfix with guarded scope. +5. Validate recovery on EN/CY critical journeys. +6. Capture post-incident notes in `memory-bank/change-log.md` and `memory-bank/pitfalls.md`. + +## High-Risk Flow Verification Matrix + +- **Auth:** sign-in, verify-request, callback redirect safety, session continuity +- **Portal/account:** protected route access and profile/account updates +- **Uploads/docs:** invalid file type/size handling, unauthorized access protection +- **Notifications:** template/language selection and failure handling +- **Routing/i18n:** rewritten Welsh routes land on expected handlers + +## Relay Hardening Rollout Playbook (TASK22239) + +Use this when changing shared relay forwarding policy (timeouts/retries/logging) or deploying relay policy updates. + +### Pre-Merge Governance Gate + +- [ ] PR scope states what changed and what did not (endpoint contracts vs relay internals) +- [ ] Risk notes include auth/data/i18n/a11y impact and relay behavior impact +- [ ] Validation evidence attached: + - relay hardening targeted tests + - endpoint contract regression tests + - lint status +- [ ] Rollback steps documented (config rollback + commit revert) +- [ ] Memory-bank updated (`change-log`, and where relevant `decisions`/`patterns`) + +### Non-Prod Smoke Matrix (Required) + +Run against a representative non-production environment: + +1. **Deterministic auth/client failures** + - force/verify `401` and `403` + - expected: no retries, immediate handled failure +2. **Deterministic validation failures** + - force/verify `400` or `404` + - expected: no retries +3. **Transient upstream failures** + - force/verify `503` / `429` + - expected: bounded retries + bounded backoff +4. **Timeout behavior** + - force latency above timeout threshold + - expected: bounded failure path and no retry storm +5. **Operational logging behavior** + - expected: structured redacted retry/failure events + - expected: no duplicate endpoint-layer error spam for already-logged relay failures + +### Progressive Runtime Rollout + +1. Deploy with conservative retry settings. +2. Verify service stability and log volume for first release window. +3. Tune only one variable at a time (`timeout`, then `retry count`, then delays). + +Recommended starting posture: + +- `RELAY_RETRY_MAX` in low range (e.g. `1` or `2`) +- bounded delay values aligned with user-facing latency tolerance +- avoid simultaneous increases of timeout and retries unless incident evidence requires it + +### Monitoring Checks (Day 1 / Day 3) + +- Relay retry rate trend +- Timeout/error ratio and top status buckets +- Upstream latency impact on citizen-facing journeys +- Log volume increase/decrease and duplicate-error noise + +### Fast Rollback / Mitigation + +1. Immediate mitigation: set `RELAY_RETRY_MAX=0` (disables retries without code rollback). +2. If needed, reduce timeout and delay knobs to baseline-safe values. +3. Full rollback path: revert relay hardening commit set and redeploy. +4. Record incident + mitigation outcome in `memory-bank/change-log.md` and `memory-bank/pitfalls.md`. + +## Definition of Ready for AI-Assisted Tasks + +- Clear acceptance criteria +- Named affected files/flows +- Risk classification assigned +- Validation plan agreed (including manual checks) diff --git a/memory-bank/change-log.md b/memory-bank/change-log.md index bab663e9..097f10da 100644 --- a/memory-bank/change-log.md +++ b/memory-bank/change-log.md @@ -1189,3 +1189,36 @@ Validation: Follow-ups: - Batch 2B: update memory/context docs with canonical relay hardening policy, env knobs, and rollback/tuning guidance. + +--- + +### CL-032: TASK22239 governance + operational rollout playbook bundle (relay hardening) + +date: 2026-03-24 +author: Cline +scope: `context/runbook.md`, `context/integration-map.md`, `memory-bank/{change-log,decisions,patterns}.md` +type: change +rationale: Deliver a combined governance and operational rollout slice after P2-S3 code hardening so relay policy changes have explicit merge gates, smoke matrix coverage, and rollback/monitoring discipline. +impact: Improves release safety and auditability for relay policy updates without changing runtime endpoint behavior. +status: completed + +Summary: + +- Added a dedicated **Relay Hardening Rollout Playbook** section to `context/runbook.md` covering: + - pre-merge governance gate checklist + - non-prod smoke matrix for deterministic and transient failure classes + - progressive rollout guidance + - Day 1/Day 3 monitoring checks + - fast mitigation/rollback sequence +- Added relay policy governance references in architecture/integration context docs to make rollout checks discoverable for future contributors. +- Recorded governance baseline updates in memory-bank: + - decision entry for relay policy-change governance gate + - reusable pattern entry for required rollout/smoke/rollback workflow + +Validation: + +- Documentation consistency review across `context/runbook.md`, `context/integration-map.md`, and memory-bank entries. + +Follow-ups: + +- Optional next iteration: add a PR template block in Azure DevOps mirroring the runbook governance gate checklist. diff --git a/memory-bank/decisions.md b/memory-bank/decisions.md index e7a676e2..f7af6d71 100644 --- a/memory-bank/decisions.md +++ b/memory-bank/decisions.md @@ -80,3 +80,35 @@ Related: - `memory-bank/change-log.md` (CL-030, CL-031) - `tests/phase21/relay-forwarding-hardening.test.cjs` + +--- + +### D-003: Relay policy changes require governance gate + rollout evidence + +date: 2026-03-24 +author: Cline +scope: relay policy changes (`pages/api/middleware/relayForwarding.js`) and related rollout docs +type: decision +rationale: Prevent regression risk from reliability-policy changes by requiring explicit pre-merge controls and operational smoke evidence. +impact: Improves production safety and auditability for relay behavior changes without altering endpoint API contracts. +status: accepted + +Decision: + +- Any relay policy change (timeout/retry/logging classification) must include: + - explicit PR governance checklist completion + - non-prod smoke matrix evidence for deterministic + transient classes + - rollback path and monitoring checks +- Policy and rollout guidance source-of-truth is maintained in: + - `context/runbook.md` Relay Hardening Rollout Playbook + - memory-bank change/decsion/pattern entries for traceability + +Consequences: + +- Relay hardening changes are treated as operationally sensitive even when endpoint contracts do not change. +- Future contributors have a repeatable gate for shipping policy changes safely. + +Related: + +- `context/runbook.md` +- `memory-bank/change-log.md` (CL-032) diff --git a/memory-bank/patterns.md b/memory-bank/patterns.md index a9a1e288..655f6fa8 100644 --- a/memory-bank/patterns.md +++ b/memory-bank/patterns.md @@ -89,3 +89,27 @@ Example paths: - `pages/api/endpoint/getadvancedsearch_api.js` - `pages/api/endpoint/getdnscoords_api.js` - `tests/phase21/relay-forwarding-hardening.test.cjs` + +### P-004: Relay Policy Change Bundle (Governance + Smoke + Rollback) + +date: 2026-03-24 +author: Cline +scope: relay policy changes + rollout documentation +type: pattern +rationale: Relay reliability changes can alter runtime behavior without changing endpoint contracts; bundle process controls and operational checks together to reduce regression risk. +impact: Improves release confidence and standardizes operational readiness for relay policy updates. +status: accepted + +Pattern: +For relay timeout/retry/logging policy updates, ship one documentation bundle that includes governance checklist, non-prod smoke matrix, monitoring checks, and rollback instructions. + +When to use: + +- Any change to retry classification, timeout defaults/bounds, or relay logging behavior +- Any change to relay environment tuning guidance + +Example paths: + +- `context/runbook.md` +- `memory-bank/change-log.md` +- `memory-bank/decisions.md` From 790394422388725b5fa33276ebd9cdd9e645ece2 Mon Sep 17 00:00:00 2001 From: robbond Date: Tue, 24 Mar 2026 12:41:30 +0000 Subject: [PATCH 2/2] docs(runbook): add relay PR evidence template and summary draft --- context/runbook.md | 25 ++++++++++++++++++++ memory-bank/TASK22239-pr-summary.md | 36 +++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) create mode 100644 memory-bank/TASK22239-pr-summary.md diff --git a/context/runbook.md b/context/runbook.md index 81c522a8..47d262f8 100644 --- a/context/runbook.md +++ b/context/runbook.md @@ -110,6 +110,31 @@ Recommended starting posture: 3. Full rollback path: revert relay hardening commit set and redeploy. 4. Record incident + mitigation outcome in `memory-bank/change-log.md` and `memory-bank/pitfalls.md`. +### PR Evidence Snapshot Template (copy into PR description) + +Use this template when shipping relay policy updates: + +``` +Relay policy smoke evidence (non-prod) + +- [ ] 401/403 deterministic auth failures -> no retries observed +- [ ] 400/404 deterministic validation failures -> no retries observed +- [ ] 429/503 transient failures -> bounded retries/backoff observed +- [ ] timeout path -> bounded failure path, no retry storm +- [ ] structured redacted logs emitted (`relay_request_retrying` / `relay_request_failed`) +- [ ] duplicate endpoint-layer error logging suppressed for already-logged relay failures + +Runtime knobs used during validation: +- RELAY_TIMEOUT_MS= +- RELAY_RETRY_MAX= +- RELAY_RETRY_BASE_DELAY_MS= +- RELAY_RETRY_MAX_DELAY_MS= + +Rollback readiness: +- [ ] quick mitigation validated: RELAY_RETRY_MAX=0 +- [ ] commit-revert path identified and documented +``` + ## Definition of Ready for AI-Assisted Tasks - Clear acceptance criteria diff --git a/memory-bank/TASK22239-pr-summary.md b/memory-bank/TASK22239-pr-summary.md new file mode 100644 index 00000000..d1db5681 --- /dev/null +++ b/memory-bank/TASK22239-pr-summary.md @@ -0,0 +1,36 @@ +# TASK22239 PR Summary Draft + +## Scope summary + +- Added relay-policy governance and rollout controls so timeout/retry/logging changes are treated as operationally sensitive changes. +- Added a copy/paste PR evidence template for non-prod smoke outcomes, runtime knobs used, and rollback readiness. + +## Files changed + +- `context/runbook.md` +- `context/integration-map.md` +- `memory-bank/change-log.md` (CL-032) +- `memory-bank/decisions.md` (D-003) +- `memory-bank/patterns.md` (P-004) +- `.gitignore` (context tracking enabled) + +## Risk notes + +- **Auth/Data:** no runtime auth or data-path behavior changes in this slice. +- **i18n/a11y:** no user-facing route/content behavior changes. +- **Operational:** process-only hardening; intended to reduce risk in future relay policy updates. + +## Validation evidence + +- Documentation consistency reviewed across: + - `context/runbook.md` + - `context/integration-map.md` + - `memory-bank/change-log.md` + - `memory-bank/decisions.md` + - `memory-bank/patterns.md` + +## Rollback plan + +1. Revert commit(s) from `TASK22239-relay-governance-rollout`. +2. If needed, restore previous `.gitignore` behavior for `context/` tracking. +3. Confirm branch clean state and re-run PR checks.