--- phase: 07-config-consolidation plan: 01 type: execute wave: 1 depends_on: [] files_modified: - config/backend.yaml - config/backend.yaml.example - config/frontend.yaml - config/frontend.yaml.example - config/network.yaml - config/network.yaml.example - config/docker.yaml - config/docker.yaml.example - config/secrets.yaml.example - config/README.md - .gitignore autonomous: true requirements: - PHASE-7-CONFIG-STRUCT - PHASE-7-YAML-FORMAT - PHASE-7-SECRETS-MGMT - PHASE-7-DOCUMENTATION user_setup: [] must_haves: truths: - "config/ folder exists with all YAML files and examples" - "YAML structure matches backend, frontend, network, docker, and secrets domains" - "All *.yaml.example files committed to git showing complete schema" - "config/secrets.yaml is git-ignored, with example provided" - ".gitignore correctly tracks examples, ignores actual secrets" - "config/README.md documents every YAML file with required variables and setup instructions" artifacts: - path: "config/backend.yaml.example" provides: "Backend configuration schema (database, AI keys, auth, logging)" min_lines: 40 - path: "config/frontend.yaml.example" provides: "Frontend configuration schema (API endpoints, feature flags, PWA)" min_lines: 25 - path: "config/network.yaml.example" provides: "Network configuration schema (ports, SSL, CORS, server IPs)" min_lines: 20 - path: "config/docker.yaml.example" provides: "Docker-specific overrides (container resources, mount paths)" min_lines: 20 - path: "config/secrets.yaml.example" provides: "Secrets template (JWT, API keys, passwords)" min_lines: 15 - path: "config/README.md" provides: "Comprehensive documentation of all config files and setup" min_lines: 100 key_links: - from: "config/backend.yaml" to: "backend/config_loader.py" via: "YAML parsing in config loader" pattern: "load.*backend\\.yaml" - from: "config/secrets.yaml.example" to: ".gitignore" via: "git ignore rule" pattern: "config/\\*\\.yaml.*!.*\\.example" - from: "config/" to: "docker-compose.yml" via: "volume mount in service definition" pattern: "\\./config:/app/config" --- Create the config/ folder structure with YAML files for backend, frontend, network, docker, and secrets configurations. Establish the foundation for centralized configuration management with clear schema documentation. Purpose: Consolidate scattered configuration (currently in inventory.env) into a structured, domain-specific YAML format (per D-01, D-02, D-03). Output: config/ folder with 4 YAML config files + examples, secrets template, comprehensive README, and .gitignore updates. @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md @.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/07-config-consolidation/07-CONTEXT.md @PROJECT_ARCHITECTURE.md @DEPLOYMENT.md @inventory.env @inventory.env.example @docker-compose.yml Task 1: Create config/ folder structure and YAML example files config/backend.yaml.example config/frontend.yaml.example config/network.yaml.example config/docker.yaml.example config/secrets.yaml.example - inventory.env (current config source) - inventory.env.example (existing schema) - PROJECT_ARCHITECTURE.md (tech stack, components) - DEPLOYMENT.md (current config categories) - docker-compose.yml (container environment vars) Create config/ folder in project root with 5 YAML example files (per D-01, D-03): **config/backend.yaml.example** — Backend configuration template covering: - Database: sqlite_path, log_retention_days, wal_mode (from current code patterns) - AI: primary_ai_provider (gemini|claude), gemini_api_key, claude_api_key, fallback_provider - Auth: jwt_secret_key, ldap_server (optional), ldap_base_dn, password_cache_path - Logging: log_level (DEBUG|INFO|WARNING|ERROR), log_rotation_size_mb, log_rotation_count - Application: data_dir, logs_dir, cors_origins - Feature flags: ai_extraction_enabled, offline_sync_enabled, audit_logging_enabled **config/frontend.yaml.example** — Frontend configuration template covering: - API: backend_url (e.g., http://localhost:8916), timeout_ms - Feature flags: service_worker_enabled, offline_enabled, ai_extraction_ui_enabled - PWA: app_name, short_name, start_url, display_mode - Feature toggles: enable_qr_scanner, enable_barcode_scanner, enable_batch_import **config/network.yaml.example** — Network/deployment configuration template covering: - Ports: backend_port (8916), frontend_port (8917), backend_ssl_port (8918), frontend_ssl_port (8919) - SSL: ssl_enabled (true|false), certificate_path, key_path (or auto-generated by Caddy) - Proxy: caddy_log_level, proxy_read_timeout_s, max_request_size_mb - CORS: allowed_origins (comma-separated), allowed_methods, allowed_headers **config/docker.yaml.example** — Docker-specific overrides covering: - Images: backend_image, frontend_image, proxy_image (with tags) - Resources: backend_cpu_limit, backend_memory_limit, frontend_cpu_limit, frontend_memory_limit - Volumes: data_volume_driver, logs_volume_driver, use_named_volumes (true|false) - Networks: network_name, network_driver (bridge|overlay) **config/secrets.yaml.example** — Secrets template (git-ignored) covering: - JWT_SECRET_KEY: "CHANGE_ME_IN_PRODUCTION" (minimum 32 chars) - GEMINI_API_KEY: "your-api-key" - CLAUDE_API_KEY: "your-api-key" - DATABASE_PASSWORD: "db-password" (if using external DB) - LDAP_PASSWORD: "ldap-password" (if using LDAP) - CORS_ORIGIN_PASSWORD: (if CORS origins require auth) All examples should have comments explaining: - What each variable controls - Default values - Valid value ranges - Where to obtain secrets (e.g., "Generate with: openssl rand -hex 32") - Whether the variable is required or optional - `test -d config` (folder exists) - `test -f config/backend.yaml.example && grep -q "primary_ai_provider" config/backend.yaml.example` (backend schema present) - `test -f config/frontend.yaml.example && grep -q "backend_url" config/frontend.yaml.example` (frontend schema present) - `test -f config/network.yaml.example && grep -q "backend_port" config/network.yaml.example` (network schema present) - `test -f config/docker.yaml.example && grep -q "backend_cpu_limit" config/docker.yaml.example` (docker schema present) - `test -f config/secrets.yaml.example && grep -q "JWT_SECRET_KEY" config/secrets.yaml.example` (secrets template present) - All 5 files are readable and valid YAML syntax: `python3 -c "import yaml; [yaml.safe_load(open(f)) for f in ['config/backend.yaml.example', 'config/frontend.yaml.example', 'config/network.yaml.example', 'config/docker.yaml.example', 'config/secrets.yaml.example']]"` config/ folder created with 5 YAML example files, all valid YAML syntax, comprehensive comments documenting schema, required/optional status, and secret generation instructions. Task 2: Create actual config files from examples and establish .gitignore rules config/backend.yaml config/frontend.yaml config/network.yaml config/docker.yaml .gitignore - config/backend.yaml.example (just created) - config/frontend.yaml.example (just created) - config/network.yaml.example (just created) - config/docker.yaml.example (just created) - inventory.env (current actual values to migrate) - .gitignore (current ignore rules) Create actual (non-example) YAML config files by copying examples and filling in values from inventory.env (per D-03 pattern): **config/backend.yaml** — Copy from backend.yaml.example and fill in values from inventory.env: - primary_ai_provider: (from PRIMARY_AI_PROVIDER in inventory.env, default: gemini) - gemini_api_key: (from GEMINI_API_KEY if present) - claude_api_key: (from CLAUDE_API_KEY if present) - ldap_server: (from LDAP_SERVER if present, or empty) - log_level: (from LOG_LEVEL if present, default: INFO) - data_dir: ./data - logs_dir: ./logs - cors_origins: (from CORS_ORIGINS if present, or default: http://localhost:8917) **config/frontend.yaml** — Create from frontend.yaml.example: - backend_url: (from BACKEND_URL or derived from BACKEND_PORT, default: http://localhost:8916) - timeout_ms: 30000 - service_worker_enabled: true - offline_enabled: true - ai_extraction_ui_enabled: true **config/network.yaml** — Create from network.yaml.example: - backend_port: (from BACKEND_PORT in inventory.env, default: 8916) - frontend_port: (from FRONTEND_PORT in inventory.env, default: 8917) - backend_ssl_port: (from BACKEND_SSL_PORT if present, default: 8918) - frontend_ssl_port: (from FRONTEND_SSL_PORT if present, default: 8919) - ssl_enabled: true - allowed_origins: (from CORS_ORIGINS if present) **config/docker.yaml** — Create from docker.yaml.example: - backend_cpu_limit: "1.0" - backend_memory_limit: "1G" - frontend_cpu_limit: "0.5" - frontend_memory_limit: "512M" - use_named_volumes: true **Update .gitignore** (per D-08): Add rules to IGNORE actual config files (secrets protection) but TRACK examples: ``` # Config files — ignore all .yaml except examples config/*.yaml !config/*.yaml.example config/secrets.yaml !config/secrets.yaml.example ``` Do NOT delete old inventory.env yet (backward compatibility until Phase 7 complete, per D-04 deprecation timeline). - `test -f config/backend.yaml && grep -q "primary_ai_provider" config/backend.yaml` (backend.yaml exists and has content) - `test -f config/frontend.yaml && grep -q "backend_url" config/frontend.yaml` (frontend.yaml exists) - `test -f config/network.yaml && grep -q "backend_port" config/network.yaml` (network.yaml exists) - `test -f config/docker.yaml && grep -q "backend_cpu_limit" config/docker.yaml` (docker.yaml exists) - All 4 files are valid YAML: `python3 -c "import yaml; [yaml.safe_load(open(f)) for f in ['config/backend.yaml', 'config/frontend.yaml', 'config/network.yaml', 'config/docker.yaml']]"` - .gitignore contains rules: `grep -q "config/\*\.yaml" .gitignore && grep -q "!config/\*\.yaml\.example" .gitignore` - config/secrets.yaml does NOT exist (will be created manually by developers from example) Actual config files created with production values migrated from inventory.env, .gitignore updated to track examples and ignore actual config files (including secrets). Task 3: Create comprehensive config/README.md documentation config/README.md - config/backend.yaml.example - config/frontend.yaml.example - config/network.yaml.example - config/docker.yaml.example - config/secrets.yaml.example - DEPLOYMENT.md (current deployment docs) Create config/README.md with comprehensive documentation (per D-08) covering: **Section 1: Overview** - Explain that config/ is the single source of truth for application configuration - Mention load order: system environment variables > config/*.yaml > defaults in code - Note that secrets are separate (git-ignored) **Section 2: Quick Start** - Copy all *.example files to remove .example suffix - Fill in required values (especially JWT_SECRET_KEY, API keys) - Create secrets.yaml from secrets.yaml.example with actual values **Section 3: backend.yaml** - Explain each variable: purpose, valid values, defaults, required/optional - List how to obtain each value (e.g., "Generate JWT key with: openssl rand -hex 32") - Show example values - List environment variable override names (e.g., BACKEND_PRIMARY_AI_PROVIDER) **Section 4: frontend.yaml** - Explain each variable: purpose, valid values, defaults - List required vs optional flags - Show how feature flags affect UI behavior - List environment variable override names **Section 5: network.yaml** - Explain port assignments and SSL settings - Show how CORS origins work - List default values and adjustment guidance - List environment variable override names **Section 6: docker.yaml** - Explain container resource limits (CPU, memory) - Show how to adjust for different hardware - Explain volume management - List environment variable override names **Section 7: secrets.yaml** - Explain git-ignore protection - List all required secrets with generation/obtainment instructions - Warn about security implications - Show correct file permissions (600) **Section 8: Environment Variable Override** - Explain how system environment variables take precedence over YAML - Show naming convention (e.g., BACKEND_PRIMARY_AI_PROVIDER -> backend.yaml:primary_ai_provider) - Useful for Docker deployments where secrets come from docker run -e **Section 9: Troubleshooting** - Common issues: missing secrets, invalid YAML syntax, missing required values - How to validate YAML syntax - How to debug which config source is being used Include helpful tables showing all variables, their purposes, defaults, and how to override them. - `test -f config/README.md && wc -l config/README.md | awk '{print $1}' | awk '$1 >= 100 {print "pass"}'` (README has at least 100 lines) - `grep -q "Quick Start" config/README.md && grep -q "backend.yaml" config/README.md && grep -q "secrets.yaml" config/README.md` (README covers all files) - `grep -q "environment variable" config/README.md` (override mechanism documented) - `grep -q "JWT_SECRET_KEY" config/README.md && grep -q "openssl rand" config/README.md` (secret generation instructions present) config/README.md created with comprehensive documentation of all YAML files, required variables, generation instructions, environment variable overrides, and troubleshooting guide. ## Trust Boundaries | Boundary | Description | |----------|-------------| | Filesystem → Application | Config files loaded from disk must not be tampered with | | Environment → Application | System environment variables override YAML (untrusted if exposed) | | Git repository → Deployment | .gitignore must prevent accidental commit of secrets | ## STRIDE Threat Register | Threat ID | Category | Component | Disposition | Mitigation Plan | |-----------|----------|-----------|-------------|-----------------| | T-07-01 | Tampering | config/*.yaml | mitigate | File permissions enforced via git (0644 for examples, 0600 for secrets). Read-only volume mounts in Docker per docker-compose.yml line 19 `:ro` flag. | | T-07-02 | Information Disclosure | config/secrets.yaml | mitigate | .gitignore rule prevents accidental commit. File permissions enforced (chmod 600). config/README.md warns developers about security. Example file included to guide setup. | | T-07-03 | Denial of Service | config/*.yaml parsing | mitigate | PyYAML used with safe_load() only (no arbitrary code execution). Backend config_loader.py validates syntax before loading. Invalid YAML causes logged error + default fallback per D-06 load order. | | T-07-04 | Elevation of Privilege | JWT_SECRET_KEY exposure | accept | JWT secret hardcoded in docker-compose.yml example (line 25) warns with comment. Developers must provide production value. Risk low for development environments. | **Phase 7, Plan 1 Verification Checklist:** 1. **Config Folder Structure** - [ ] `config/` folder exists in project root - [ ] 5 example files present: backend.yaml.example, frontend.yaml.example, network.yaml.example, docker.yaml.example, secrets.yaml.example - [ ] All examples contain valid YAML syntax (parseable by python3 -m yaml) - [ ] All examples have comprehensive comments explaining each variable 2. **Actual Config Files** - [ ] 4 actual config files exist: backend.yaml, frontend.yaml, network.yaml, docker.yaml - [ ] All actual files contain valid YAML syntax - [ ] Values migrated from inventory.env are present and reasonable - [ ] secrets.yaml does NOT exist (will be created manually) 3. **Git Integration** - [ ] .gitignore updated with rules: `config/*.yaml`, `!config/*.yaml.example`, `config/secrets.yaml`, `!config/secrets.yaml.example` - [ ] Examples are tracked: `git status config/*.example` shows "new file" or no changes - [ ] Actual configs are ignored: `git check-ignore config/backend.yaml` returns success - [ ] Secrets example is tracked but secrets themselves are ignored 4. **Documentation** - [ ] config/README.md exists with 100+ lines - [ ] README covers all 5 YAML files (backend, frontend, network, docker, secrets) - [ ] README documents environment variable override mechanism - [ ] README includes secret generation/obtainment instructions - [ ] README has troubleshooting section 5. **Backward Compatibility (D-04)** - [ ] inventory.env still exists (will be fully deprecated after backend refactor in Plan 2) - [ ] No code changes yet (Config loading still uses inventory.env, Phase 7 Plan 2 updates backend) - config/ folder created with 5 YAML example files defining complete schema - 4 actual YAML config files created with values migrated from inventory.env - config/secrets.yaml.example provides template (actual secrets.yaml created manually by developers) - .gitignore updated to track examples, ignore actual config files and secrets - config/README.md provides comprehensive documentation and setup instructions - All YAML files are syntactically valid and parseable - No backend code changes yet (backward compatibility maintained per D-04) - Foundation ready for backend refactoring in Plan 2 After completion, create `.planning/phases/07-config-consolidation/07-01-SUMMARY.md`