diff --git a/docs/superpowers/specs/2026-04-19-phase-3-e2e-tests-design.md b/docs/superpowers/specs/2026-04-19-phase-3-e2e-tests-design.md new file mode 100644 index 00000000..0d6dbda8 --- /dev/null +++ b/docs/superpowers/specs/2026-04-19-phase-3-e2e-tests-design.md @@ -0,0 +1,425 @@ +# Phase 3 Design: Playwright E2E Tests (Modular Workflows) + +**Date:** 2026-04-19 +**Status:** Design Approved +**Target Runtime:** <30 minutes (parallel execution) +**Test Scope:** 5 critical user workflows + error handling + +--- + +## 1. Executive Summary + +Phase 3 extends Phase 2 (284 unit tests) with end-to-end browser automation tests using Playwright. Each of 5 critical workflows runs in its own isolated Docker environment with a dedicated database, LDAP server, and app instance. Tests run in parallel, completing within <30 minutes. + +**Workflows:** +1. Login (LDAP + local authentication) +2. Scan → Adjust Stock (barcode matching) +3. AI Extraction (new item onboarding) +4. Admin Settings (configuration & user management) +5. Offline Sync (queue → sync idempotency) + +--- + +## 2. Test Architecture + +### 2.1 Directory Structure + +``` +frontend/e2e/ +├── workflows/ +│ ├── 1-login.spec.ts (LDAP + local auth flows) +│ ├── 2-scan-adjust.spec.ts (barcode scan, stock adjustment) +│ ├── 3-ai-extraction.spec.ts (photo → AI → validation) +│ ├── 4-admin-settings.spec.ts (admin dashboard, config changes) +│ └── 5-offline-sync.spec.ts (offline ops → sync) +├── fixtures/ +│ ├── db.ts (SQLite seeding, cleanup, per-workflow) +│ ├── ldap.ts (OpenLDAP container setup, test users) +│ ├── auth.ts (login helpers, session management) +│ └── test-data.ts (seed definitions, factories) +├── utils/ +│ ├── assertions.ts (custom Playwright matchers) +│ ├── docker.ts (container orchestration, lifecycle) +│ └── helpers.ts (navigation, wait conditions) +├── docker-compose.e2e.yml (shared services template) +├── playwright.config.ts (Playwright configuration) +└── README.md (setup & execution guide) +``` + +### 2.2 Per-Workflow Isolation + +Each workflow runs independently: + +| Workflow | Backend Port | Frontend Port | Database | LDAP | AI Mock | +|----------|--------------|---------------|----------|------|---------| +| 1-login | 8906 | 8907 | Fresh | Yes | N/A | +| 2-scan-adjust | 8916 | 8917 | Seeded (10 items) | No | N/A | +| 3-ai-extraction | 8926 | 8927 | Fresh | No | Gemini/Claude mocked | +| 4-admin-settings | 8936 | 8937 | Seeded (users, categories) | Yes | Mocked | +| 5-offline-sync | 8946 | 8947 | Fresh | No | N/A | + +**Benefits:** +- No port conflicts (workflows run in parallel) +- Failed workflow doesn't affect others +- Per-workflow database cleanup (no state leakage) +- Independent LDAP setup for auth workflows + +--- + +## 3. Workflow Test Scenarios + +### 3.1 Workflow 1: Login (LDAP + Local) + +**Setup:** LDAP container with test users + app + empty database + +**Scenarios:** +1. ✅ LDAP user login (valid credentials → dashboard) +2. ✅ Local user login (password → dashboard) +3. ✅ Invalid LDAP credentials → error message +4. ✅ Invalid local password → error message +5. ✅ Missing username/password → validation error +6. ✅ Session expiry (token timeout) → redirect to login +7. ✅ Logout (clear session) → login screen +8. ✅ Concurrent login attempts → proper queueing + +**Error Cases:** +- LDAP server down → fallback to local auth +- Network timeout → retry with backoff +- Invalid token format → re-authenticate + +--- + +### 3.2 Workflow 2: Scan → Adjust Stock + +**Setup:** App + seeded database (10 items with barcodes) + +**Scenarios:** +1. ✅ Scan valid barcode → match existing item → open adjustment UI +2. ✅ Adjust quantity (+5) → confirm → audit log updated +3. ✅ Scan unknown barcode → create new item flow +4. ✅ Multiple consecutive scans (5+) → batch operations queue +5. ✅ Scan while offline → queue operation → sync on reconnect +6. ✅ Barcode not found → OCR fallback search +7. ✅ Box label scan → multi-item selection UI +8. ✅ Concurrent scans → no race conditions + +**Error Cases:** +- Barcode decode failure → retry +- Network timeout during save → offline queue +- Inventory constraint violation (negative qty) → validation error +- Concurrent quantity updates → last-write-wins with audit + +--- + +### 3.3 Workflow 3: AI Extraction (New Item Onboarding) + +**Setup:** App + empty database + mocked Gemini/Claude APIs + +**Scenarios:** +1. ✅ Capture photo → send to AI → receive extraction +2. ✅ AI response (name, part number, category) → validation UI +3. ✅ Confirm extracted data → save item +4. ✅ Reject extraction → manual entry form +5. ✅ AI extraction with multiple items in photo +6. ✅ Box discovery mode (AI focuses on container labels) +7. ✅ AI timeout → retry with exponential backoff +8. ✅ Network failure during extraction → offline queue + +**Error Cases:** +- Image validation (blur, size, format) → error message +- Invalid EXIF data → degrade gracefully +- AI service timeout (>10s) → user can retry or enter manually +- Malformed AI response → fallback to manual entry +- Concurrent extraction requests → queue + process sequentially + +--- + +### 3.4 Workflow 4: Admin Settings + +**Setup:** App + seeded database (5 test users, 8 categories) + LDAP + +**Scenarios:** +1. ✅ Navigate to Admin Dashboard +2. ✅ Identity Manager: list users (LDAP + local) +3. ✅ Create new local user → email validation +4. ✅ Delete user → confirmation modal → audit log +5. ✅ AI Manager: switch provider (Gemini → Claude) +6. ✅ Update API key → test connection → success/failure +7. ✅ LDAP Manager: update server settings → test connection +8. ✅ Database Manager: view backup status → trigger backup +9. ✅ Category Manager: add/delete categories +10. ✅ Configuration saved → persists across sessions + +**Error Cases:** +- Invalid API key → error toast, no save +- LDAP connection timeout → error state, keep previous config +- Concurrent config updates → optimistic UI + server validation +- Missing required fields → inline validation +- Database backup failure → error state, rollback + +--- + +### 3.5 Workflow 5: Offline Sync + +**Setup:** App + empty database + simulated offline mode + +**Scenarios:** +1. ✅ Perform operation (scan, create item) → offline detection +2. ✅ Queue 5+ operations while offline +3. ✅ Go online → automatic sync batch to server +4. ✅ UUID idempotency: sync same batch twice → no duplicates +5. ✅ Partial sync failure → retry remaining items +6. ✅ Sync with network timeout → exponential backoff +7. ✅ Concurrent updates (offline + online) → conflict resolution +8. ✅ Local state persists (IndexedDB) → reload page → continues sync + +**Error Cases:** +- Sync failure mid-batch → remaining items queued +- Server rejects UUID → log error, mark item as failed +- IndexedDB quota exceeded → error toast +- Corrupted queue entry → skip + continue +- Server version mismatch (audit schema) → graceful degradation + +--- + +## 4. Error Handling & Resilience + +### 4.1 Network Failures + +**Timeout Handling:** +- API call timeout > 10s → retry 2x with exponential backoff (1s, 2s) +- Container startup timeout > 30s → fail fast, report health check failure +- Page load > 15s → timeout assertion + +**Connection Loss:** +- Offline detection: monitor navigator.onLine + failed API call +- Offline queue: IndexedDB stores operations with UUID + timestamp +- Sync on reconnect: automatic batch send, retry failed items + +### 4.2 Concurrent Operations + +**Race Condition Prevention:** +- Scanning: queue concurrent scans, process sequentially +- Stock adjustment: last-write-wins with server validation +- Config updates: optimistic UI, server validation, rollback on fail +- AI extraction: single extraction per session (prevent duplicate calls) + +### 4.3 Invalid Input Handling + +- Image validation (size, format, blur) → inline error +- Missing required fields → form validation error +- Invalid barcode → OCR fallback + manual entry +- Malformed AI response → user can retry or enter manually + +--- + +## 5. Docker & Infrastructure + +### 5.1 Docker Compose Setup + +**Base Configuration (`docker-compose.e2e.yml`):** +```yaml +services: + # App backend + backend: + image: ainventory-backend:test + ports: + - "${BACKEND_PORT}:8906" + environment: + DATABASE_URL: sqlite:///test-${WORKFLOW_ID}.db + LDAP_ENABLED: "${LDAP_ENABLED}" + AI_PROVIDER: "${AI_PROVIDER}" + GEMINI_API_KEY: "test-key" + CLAUDE_API_KEY: "test-key" + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:8906/health"] + interval: 2s + timeout: 5s + retries: 10 + + # Frontend dev server + frontend: + image: node:20 + working_dir: /app + ports: + - "${FRONTEND_PORT}:8907" + environment: + NEXT_PUBLIC_API_URL: "http://localhost:${BACKEND_PORT}" + healthcheck: + test: ["CMD", "curl", "-f", "http://localhost:3000"] + interval: 2s + retries: 10 + + # OpenLDAP (for auth workflows) + ldap: + image: osixia/openldap:latest + ports: + - "${LDAP_PORT}:389" + environment: + LDAP_ORGANISATION: "aInventory" + LDAP_BASE_DN: "dc=ainventory,dc=local" +``` + +### 5.2 Container Lifecycle + +**Per-Workflow:** +1. **Setup Phase (~15-20s)** + - Start Docker Compose for workflow + - Wait for health checks (backend, frontend, LDAP if needed) + - Seed database (SQL migrations) + - Pre-populate LDAP users (if needed) + +2. **Test Phase (~3-5 min)** + - Playwright runs test scenarios + - Browser automation against live app + - Real API calls to backend + +3. **Teardown Phase (~5-10s)** + - Stop all containers + - Clean database volume + - Collect logs for debugging + +--- + +## 6. Test Configuration + +### 6.1 Playwright Config + +```typescript +// playwright.config.ts +export default defineConfig({ + testDir: './e2e/workflows', + fullyParallel: true, + workers: 5, // Run 5 workflows in parallel + timeout: 30000, // 30s per test + expect: { timeout: 5000 }, + webServer: [], // No webServer (Docker manages this) + use: { + baseURL: 'http://localhost', // Dynamic per workflow + trace: 'on-first-retry', + screenshot: 'only-on-failure', + }, +}); +``` + +### 6.2 Environment Setup + +**Env Variables per Workflow:** +```bash +# .env.e2e.workflow-1 +BACKEND_PORT=8906 +FRONTEND_PORT=8907 +LDAP_ENABLED=true +LDAP_PORT=3389 +AI_PROVIDER=gemini + +# .env.e2e.workflow-2 +BACKEND_PORT=8916 +FRONTEND_PORT=8917 +LDAP_ENABLED=false +AI_PROVIDER=gemini +``` + +--- + +## 7. Test Execution & CI/CD + +### 7.1 Local Execution + +```bash +# Run all workflows in parallel +npm run e2e + +# Run specific workflow +npm run e2e -- workflows/1-login.spec.ts + +# Debug mode (headed browser) +npm run e2e:debug +``` + +### 7.2 Expected Runtime + +- **Per Workflow:** 3-5 minutes +- **Sequential Total:** 15-25 minutes +- **Parallel Total:** 8-10 minutes (5 workers) +- **Target:** <30 minutes ✅ + +### 7.3 CI/CD Integration + +```bash +# GitHub Actions / Local CI +npm run build +npm run e2e -- --reporter=html +# Report: playwright-report/index.html +``` + +--- + +## 8. Success Criteria + +✅ All 5 workflows tested +✅ 40+ test cases across workflows +✅ Error scenarios included +✅ Parallel execution <30 min +✅ Zero flaky tests (3x runs stable) +✅ Comprehensive error handling +✅ Docker isolation working +✅ Database cleanup per workflow +✅ HTML report generated + +--- + +## 9. Scope & Constraints + +**In Scope:** +- Happy path workflows +- Critical error scenarios (network, auth, validation) +- Concurrent operation handling +- Offline → online sync +- Docker-based isolation + +**Out of Scope:** +- Performance benchmarking +- Load testing +- Mobile-specific gestures (covered by Vitest unit tests) +- Visual regression testing +- Accessibility audits (covered by Phase 2) + +--- + +## 10. Dependencies & Prerequisites + +**Required:** +- Docker & Docker Compose +- Node.js 20+ +- Playwright (`@playwright/test`) +- Python 3.12+ (backend venv) + +**Optional:** +- `docker-compose` plugin +- `curl` (for health checks) + +--- + +## 11. Risk Mitigation + +| Risk | Mitigation | +|------|-----------| +| Docker startup slow | Health checks + parallel workers | +| Flaky network tests | Retry logic + exponential backoff | +| Port conflicts | Offset ports per workflow (8906, 8916, 8926, etc.) | +| Database state leakage | Fresh DB per workflow, cleanup after | +| LDAP timeout | Fallback to local auth, skip LDAP tests if unavailable | +| Concurrent AI calls | Queue extraction requests, single-at-a-time processing | + +--- + +## 12. Next Steps + +1. ✅ Design approved +2. → Create implementation plan (writing-plans skill) +3. → Install Playwright, set up docker-compose.e2e.yml +4. → Build test fixtures (db, ldap, auth) +5. → Implement 5 workflow test files +6. → Verify parallel execution <30 min +7. → Commit & tag `phase-3-complete`