---
phase: 07-config-consolidation
plan: 03
type: execute
wave: 2
depends_on:
  - 07-01
files_modified:
  - scripts/deploy.py
  - scripts/run_standalone.py
  - scripts/install_service.py
  - scripts/export_prod.py
autonomous: true
requirements:
  - PHASE-7-PYTHON-SCRIPTS
  - PHASE-7-YAML-PARSING
  - PHASE-7-DEPLOYMENT
user_setup: []

must_haves:
  truths:
    - "Python deployment scripts (deploy.py, run_standalone.py, install_service.py, export_prod.py) exist and parse YAML config"
    - "All scripts parse config/*.yaml files using PyYAML (per D-05)"
    - "deploy.py handles Docker deployment with health checks"
    - "run_standalone.py launches backend and frontend without Docker"
    - "install_service.py installs systemd service with new config paths"
    - "export_prod.py exports production data/config for backups"
    - "All scripts are executable and tested"
  artifacts:
    - path: "scripts/deploy.py"
      provides: "Docker deployment with YAML config parsing, pre-flight checks, health validation"
      exports: ["main()"]
      min_lines: 150
    - path: "scripts/run_standalone.py"
      provides: "Standalone launcher for backend (FastAPI) and frontend (Next.js) with YAML config"
      exports: ["main()"]
      min_lines: 120
    - path: "scripts/install_service.py"
      provides: "Systemd service installation with config paths"
      exports: ["main()"]
      min_lines: 100
    - path: "scripts/export_prod.py"
      provides: "Production export/backup script with YAML config support"
      exports: ["main()"]
      min_lines: 100
  key_links:
    - from: "scripts/deploy.py"
      to: "config/docker.yaml"
      via: "PyYAML parsing for container config"
      pattern: "yaml\\.safe_load.*docker\\.yaml"
    - from: "scripts/run_standalone.py"
      to: "config/backend.yaml"
      via: "Read port and path config"
      pattern: "yaml\\.safe_load.*backend\\.yaml"
    - from: "scripts/install_service.py"
      to: "inventory.service.template"
      via: "Service file generation"
      pattern: "template|service"
---

<objective>
Convert bash deployment scripts (deploy.sh, run_standalone.sh, install_service.sh, export_prod.sh) to Python with YAML config parsing. Provide consistent, maintainable deployment tooling that understands the new config structure.

Purpose: Implement D-05 (Python scripts with YAML parsing) for modern deployment infrastructure.

Output: 4 Python scripts in scripts/ folder with full deployment functionality and YAML config support.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/phases/07-config-consolidation/07-CONTEXT.md
@PROJECT_ARCHITECTURE.md
@DEPLOYMENT.md
@deploy.sh
@run_standalone.sh
@install_service.sh
@export_prod.sh
@config/backend.yaml.example
@config/docker.yaml.example
@docker-compose.yml
</context>

<tasks>

<task type="auto">
  <name>Task 1: Create scripts/deploy.py (Docker deployment with YAML config)</name>
  <files>
    scripts/deploy.py
  </files>
  <read_first>
    - deploy.sh (current bash implementation to port)
    - docker-compose.yml (structure and environment variables)
    - config/docker.yaml.example (schema)
    - config/backend.yaml.example (config structure)
    - DEPLOYMENT.md (deployment procedure documentation)
  </read_first>
  <action>
    Create scripts/deploy.py to replace deploy.sh with Python implementation (per D-05).

    **Key features:**

    1. **Script signature and usage:**
       ```bash
       python3 scripts/deploy.py [production|staging|development] [--rebuild]
       ```

    2. **Core functionality:**
       - Pre-flight checks: Docker, Docker Compose, docker-compose.yml, config files
       - Load config from config/docker.yaml and config/network.yaml
       - Port availability checks (from network.yaml: backend_port, frontend_port, etc.)
       - Environment file validation (config/backend.yaml exists and has required values)
       - Docker Compose up with appropriate flags (rebuild if --rebuild)
       - Health check polling (curl to /health endpoints)
       - Deployment report (services running, ports, access URLs)

    3. **Config file parsing:**
       - Use PyYAML to load config/docker.yaml (for container resource limits, image names)
       - Use PyYAML to load config/network.yaml (for port numbers and SSL settings)
       - Use PyYAML to load config/backend.yaml (to validate required values)
       - Fallback to sensible defaults if config files missing (but log warnings)

    4. **Pre-flight checks (Step 1-5):**
       - [ ] docker command available
       - [ ] docker-compose command available
       - [ ] docker-compose.yml exists
       - [ ] config/backend.yaml exists (with helpful error if missing)
       - [ ] config/network.yaml exists (with helpful error if missing)

    5. **Port availability check (Step 6):**
       - Read backend_port, frontend_port, backend_ssl_port, frontend_ssl_port from network.yaml
       - Use netstat or ss to check if ports are in use
       - Error if ports occupied, suggest alternatives

    6. **Environment validation (Step 7):**
       - Check config/backend.yaml for required values: JWT_SECRET_KEY, primary_ai_provider
       - Warn if API keys are placeholders
       - Proceed with warning (not error) for optional values

    7. **Docker Compose deployment (Step 8-9):**
       - Run `docker-compose up -d` (or with --build if --rebuild flag)
       - Capture and display output with color codes
       - Catch errors and provide helpful debugging steps

    8. **Health checks (Step 10-11):**
       - Poll backend health: `curl http://localhost:{backend_port}/health` (retry logic)
       - Poll frontend health: `curl http://localhost:{frontend_port}/` (retry logic)
       - Wait up to 2 minutes for services to become healthy
       - Display health status to user

    9. **Deployment report (Step 12):**
       - Display service status: `docker-compose ps`
       - Display access URLs:
         - Frontend: http://localhost:{frontend_port}
         - Backend API: http://localhost:{backend_port}/docs
         - HTTPS: https://localhost:{frontend_ssl_port} (if SSL enabled in network.yaml)
       - Display next steps (logs, troubleshooting, etc.)

    10. **Error handling:**
        - Descriptive error messages with suggested fixes
        - Log all actions and results to stdout/stderr
        - Use color output (GREEN for success, RED for errors, YELLOW for warnings, BLUE for info)
        - Exit codes: 0 for success, 1 for fatal error

    11. **Logging:**
        - Use Python logging module (not print)
        - Log level: INFO by default, DEBUG if --verbose flag
        - Each step logged: "Step N/M: Description..."
        - Results logged at end: "Deployment complete, services healthy"

    12. **Required libraries:**
        - sys, os, subprocess, time, socket (built-in)
        - yaml (PyYAML)
        - argparse (for CLI args)
        - logging (for logging)
        - No external deployment libraries (keep it simple)

    13. **Make executable:** `chmod +x scripts/deploy.py` and include shebang: `#!/usr/bin/env python3`
  </action>
  <verify>
    - `test -f scripts/deploy.py && head -1 scripts/deploy.py | grep -q "python3"` (shebang present)
    - `test -x scripts/deploy.py` (executable)
    - `python3 -m py_compile scripts/deploy.py` (valid Python syntax)
    - `python3 scripts/deploy.py --help | grep -q "deployment"` (help works)
    - `grep -q "import yaml" scripts/deploy.py` (PyYAML imported)
    - `grep -q "config/docker.yaml\|config/network.yaml" scripts/deploy.py` (loads config files)
    - `grep -q "docker-compose" scripts/deploy.py` (calls docker-compose)
    - `grep -q "curl.*health" scripts/deploy.py` (health checks present)
  </verify>
  <done>
    scripts/deploy.py created with Docker deployment, YAML config parsing, health checks, and error handling.
  </done>
</task>

<task type="auto">
  <name>Task 2: Create scripts/run_standalone.py (Standalone launcher with YAML config)</name>
  <files>
    scripts/run_standalone.py
  </files>
  <read_first>
    - run_standalone.sh (current bash implementation to port)
    - config/backend.yaml.example (schema)
    - config/frontend.yaml.example (schema)
    - backend/main.py (backend entry point)
    - frontend package.json or next.config.js (frontend startup)
  </read_first>
  <action>
    Create scripts/run_standalone.py to replace run_standalone.sh with Python implementation (per D-05).

    **Key features:**

    1. **Script signature:**
       ```bash
       python3 scripts/run_standalone.py [--backend-only|--frontend-only]
       ```

    2. **Core functionality:**
       - Load config from config/backend.yaml and config/frontend.yaml
       - Start FastAPI backend (uvicorn)
       - Start Next.js frontend (npm run dev or node server.js)
       - Display console output from both processes
       - Handle shutdown gracefully (SIGTERM/SIGINT kills both services)
       - Display health status and access URLs

    3. **Config file parsing:**
       - Load config/backend.yaml to get: backend_port, data_dir, logs_dir, log_level
       - Load config/frontend.yaml to get: frontend_port, backend_url
       - Use defaults if config files missing (with warnings)

    4. **Backend startup (--backend-only or default):**
       - Command: `uvicorn backend.main:app --host 0.0.0.0 --port {backend_port} --reload`
       - Set environment: DATA_DIR, LOGS_DIR, LOG_LEVEL (from config)
       - Capture output and display with [BACKEND] prefix
       - Wait for backend to log "Uvicorn running on..." or similar
       - Verify backend is listening on backend_port

    5. **Frontend startup (--frontend-only or default):**
       - Command: `npm run dev` (if in development) or `node server.js` (if built)
       - Set environment: NEXT_PUBLIC_API_URL (from config:frontend:backend_url)
       - Capture output and display with [FRONTEND] prefix
       - Wait for frontend to log "ready - started server on..." or similar
       - Verify frontend is listening on frontend_port

    6. **Process management:**
       - Use subprocess.Popen with shell=False (for security)
       - Manage both processes in list/tuple
       - Handle SIGTERM/SIGINT (Ctrl+C) to kill both processes
       - Display "Shutting down..." and wait for clean shutdown
       - Exit with code 0 if both shut down cleanly

    7. **Health monitoring:**
       - Periodically check if processes are alive (poll returncode)
       - If one process dies, log error and optionally shutdown other (per config flag)
       - Display uptime and status every 30 seconds

    8. **Logging and output:**
       - Use Python logging module
       - Log each process with [BACKEND] / [FRONTEND] prefix
       - Merge stdout/stderr from both processes to terminal
       - Show final status: "Backend running on http://localhost:{backend_port}, Frontend on http://localhost:{frontend_port}"

    9. **Error handling:**
       - If uvicorn not installed, error and suggest: `pip install uvicorn`
       - If npm not found, error and suggest: install Node.js
       - If ports already in use, error with port number
       - If config files missing, log warnings but use defaults

    10. **Required libraries:**
        - sys, os, subprocess, signal, time (built-in)
        - yaml (PyYAML)
        - argparse (for CLI args --backend-only, --frontend-only)
        - logging (for logging)

    11. **Make executable:** `chmod +x scripts/run_standalone.py` with shebang: `#!/usr/bin/env python3`
  </action>
  <verify>
    - `test -f scripts/run_standalone.py && head -1 scripts/run_standalone.py | grep -q "python3"` (shebang present)
    - `test -x scripts/run_standalone.py` (executable)
    - `python3 -m py_compile scripts/run_standalone.py` (valid Python syntax)
    - `grep -q "import yaml" scripts/run_standalone.py` (PyYAML imported)
    - `grep -q "config/backend.yaml\|config/frontend.yaml" scripts/run_standalone.py` (loads config)
    - `grep -q "uvicorn\|subprocess.Popen" scripts/run_standalone.py` (backend startup present)
    - `grep -q "npm\|node server" scripts/run_standalone.py` (frontend startup present)
    - `grep -q "signal.signal\|SIGTERM" scripts/run_standalone.py` (signal handling present)
  </verify>
  <done>
    scripts/run_standalone.py created with backend/frontend startup, YAML config parsing, process management, and graceful shutdown.
  </done>
</task>

<task type="auto">
  <name>Task 3: Create scripts/install_service.py (Systemd service installation)</name>
  <files>
    scripts/install_service.py
  </files>
  <read_first>
    - install_service.sh (current bash implementation to port)
    - inventory.service.template (systemd service template)
    - config/backend.yaml.example (to understand config structure)
    - config/network.yaml.example (for port information)
  </read_first>
  <action>
    Create scripts/install_service.py to replace install_service.sh with Python implementation (per D-05).

    **Key features:**

    1. **Script signature:**
       ```bash
       sudo python3 scripts/install_service.py [--user=service_user] [--port=port]
       ```

    2. **Core functionality:**
       - Read inventory.service.template (or create template inline)
       - Load config from config/backend.yaml (for paths, ports)
       - Generate systemd service file with correct paths and user/group
       - Install service file to /etc/systemd/system/ainventory.service
       - Enable service (systemctl enable)
       - Display installation summary and next steps

    3. **Config file parsing:**
       - Load config/backend.yaml to get: data_dir, logs_dir
       - Load config/network.yaml to get: backend_port (for documentation)
       - Use defaults if missing

    4. **Service file generation:**
       - Read inventory.service.template
       - Replace placeholders:
         - {PROJECT_DIR}: current working directory (project root)
         - {SERVICE_USER}: service user (default: www-data, configurable via --user)
         - {BACKEND_PORT}: from config/network.yaml
         - {DATA_DIR}: from config/backend.yaml
         - {LOGS_DIR}: from config/backend.yaml
       - Template should:
         - Type=simple
         - ExecStart=/usr/bin/python3 {PROJECT_DIR}/scripts/run_standalone.py --backend-only
         - WorkingDirectory={PROJECT_DIR}
         - User={SERVICE_USER}
         - Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
         - Restart=on-failure
         - RestartSec=10

    5. **Permission checks:**
       - Require sudo/root: `if os.getuid() != 0: error("Must run with sudo")`
       - Check project directory is readable
       - Check data_dir and logs_dir exist or can be created

    6. **Service file installation:**
       - Write service file to /etc/systemd/system/ainventory.service
       - Set permissions: 644 (readable, not writable by non-root)
       - Run `systemctl daemon-reload`
       - Run `systemctl enable ainventory.service`
       - Optionally start service: `systemctl start ainventory.service`

    7. **Status display:**
       - Show service file location
       - Show service user and group
       - Show project directory
       - Show next steps: `systemctl start ainventory`, `systemctl status ainventory`
       - Show logs: `journalctl -u ainventory -f`

    8. **Error handling:**
       - Check if service already installed (offer --force to overwrite)
       - Check if user exists (suggest: `useradd -r -s /bin/false {user}`)
       - Check if directories are writable
       - Descriptive errors with suggested fixes

    9. **Required libraries:**
       - sys, os, subprocess, pwd, grp (built-in)
       - yaml (PyYAML)
        - argparse (for CLI args)
        - logging (for logging)

    10. **Make executable:** `chmod +x scripts/install_service.py` with shebang: `#!/usr/bin/env python3`
  </action>
  <verify>
    - `test -f scripts/install_service.py && head -1 scripts/install_service.py | grep -q "python3"` (shebang present)
    - `test -x scripts/install_service.py` (executable)
    - `python3 -m py_compile scripts/install_service.py` (valid Python syntax)
    - `grep -q "import yaml" scripts/install_service.py` (PyYAML imported)
    - `grep -q "config/backend.yaml\|config/network.yaml" scripts/install_service.py` (loads config)
    - `grep -q "/etc/systemd/system\|systemctl" scripts/install_service.py` (systemd integration present)
    - `grep -q "os.getuid\|sudo" scripts/install_service.py` (permission check present)
  </verify>
  <done>
    scripts/install_service.py created with systemd service generation, config parsing, and installation logic.
  </done>
</task>

<task type="auto">
  <name>Task 4: Create scripts/export_prod.py (Production export/backup)</name>
  <files>
    scripts/export_prod.py
  </files>
  <read_first>
    - export_prod.sh (current bash implementation to port)
    - config/backend.yaml.example (for data_dir)
    - DEPLOYMENT.md (backup procedures)
  </read_first>
  <action>
    Create scripts/export_prod.py to replace export_prod.sh with Python implementation (per D-05).

    **Key features:**

    1. **Script signature:**
       ```bash
       python3 scripts/export_prod.py [--output=/path/to/backup.tar.gz] [--include-logs]
       ```

    2. **Core functionality:**
       - Load config from config/backend.yaml (to find data_dir, logs_dir)
       - Create tar.gz archive of production data
       - Include database file(s), config files (no secrets), and optionally logs
       - Output to specified location or default: backups/{timestamp}.tar.gz
       - Display archive size and location

    3. **Config file parsing:**
       - Load config/backend.yaml to get: data_dir, logs_dir
       - Use defaults if missing: data_dir=./data, logs_dir=./logs

    4. **Archive creation:**
       - Include: {data_dir}/* (all application data, database, etc.)
       - Include: config/*.yaml.example (config templates)
       - Include: config/backend.yaml, config/frontend.yaml, config/network.yaml (actual configs, no secrets)
       - Include: config/secrets.yaml.example (secrets template only, NOT actual secrets.yaml)
       - Include: logs/* (optional, if --include-logs flag)
       - Exclude: config/secrets.yaml (never backup actual secrets)
       - Exclude: node_modules/, __pycache__/, .git/, .venv/
       - Exclude: temporary files, cache

    5. **Archive naming:**
       - Default: backups/ainventory_{timestamp}.tar.gz
       - Timestamp format: YYYY-MM-DD_HH-MM-SS
       - Custom path via --output flag

    6. **Backup directory:**
       - Create backups/ directory if not exists
       - Set directory permissions: 750 (rwxr-x---)

    7. **Verification:**
       - Verify tar.gz was created successfully
       - Display archive size: X.XX MB
       - Display archive contents summary: "Includes database, data, and config (secrets excluded)"

    8. **Error handling:**
       - If data_dir doesn't exist, error and suggest creating it
       - If no write permission to backups/, error and suggest location
       - If tar command fails, show error and suggest troubleshooting

    9. **Output example:**
       ```
       [INFO] Loading config from config/backend.yaml
       [INFO] Data directory: ./data
       [INFO] Creating backup...
       [INFO] Archive created: backups/ainventory_2026-04-23_14-30-45.tar.gz
       [INFO] Archive size: 125.43 MB
       [INFO] Contents: database, data, config (secrets excluded)
       [INFO] Backup complete!
       ```

    10. **Required libraries:**
        - sys, os, subprocess, datetime, tarfile (built-in)
        - yaml (PyYAML)
        - argparse (for CLI args)
        - logging (for logging)

    11. **Make executable:** `chmod +x scripts/export_prod.py` with shebang: `#!/usr/bin/env python3`
  </action>
  <verify>
    - `test -f scripts/export_prod.py && head -1 scripts/export_prod.py | grep -q "python3"` (shebang present)
    - `test -x scripts/export_prod.py` (executable)
    - `python3 -m py_compile scripts/export_prod.py` (valid Python syntax)
    - `grep -q "import yaml" scripts/export_prod.py` (PyYAML imported)
    - `grep -q "config/backend.yaml" scripts/export_prod.py` (loads config)
    - `grep -q "tarfile\|tar.gz" scripts/export_prod.py` (tar archive creation present)
    - `grep -q "secrets.yaml" scripts/export_prod.py | grep -q "exclude"` (secrets excluded from backup)
  </verify>
  <done>
    scripts/export_prod.py created with production data export, YAML config parsing, archive creation, and secrets exclusion.
  </done>
</task>

</tasks>

<threat_model>
## Trust Boundaries

| Boundary | Description |
|----------|-------------|
| User input → Script | Script arguments and config files must be validated |
| Script → System | Scripts execute system commands (subprocess) — must escape/quote properly |
| Script → Network | Health checks make HTTP requests (must handle timeouts) |
| Script → Filesystem | Scripts read/write files (must respect permissions) |

## STRIDE Threat Register

| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-07-09 | Injection | deploy.py subprocess | mitigate | Use subprocess with shell=False and list args (not f-strings). Example: `subprocess.run(["docker-compose", "up", "-d"], ...)` not `subprocess.run(f"docker-compose up -d", shell=True)`. |
| T-07-10 | Elevation of Privilege | install_service.py sudo | mitigate | Check `os.getuid() != 0` at start. Require sudo for systemd operations only. Log all systemctl calls. |
| T-07-11 | Information Disclosure | export_prod.py backup | mitigate | Exclude config/secrets.yaml explicitly in tarfile. Log what is excluded. Verify file permissions (backups/ dir 750). |
| T-07-12 | Denial of Service | Health checks timeout | mitigate | Set socket timeout to 10 seconds. Limit retry attempts to 12 (2 minutes total). Log timeout errors. |

</threat_model>

<verification>
**Phase 7, Plan 3 Verification Checklist:**

1. **scripts/deploy.py**
   - [ ] File exists and is executable
   - [ ] Shebang present: `#!/usr/bin/env python3`
   - [ ] Loads config/docker.yaml and config/network.yaml
   - [ ] Pre-flight checks for Docker, Docker Compose, config files
   - [ ] Port availability checks implemented
   - [ ] Health checks poll backend and frontend endpoints
   - [ ] Color output for info/warning/success/error
   - [ ] Displays deployment summary and access URLs
   - [ ] Valid Python syntax

2. **scripts/run_standalone.py**
   - [ ] File exists and is executable
   - [ ] Shebang present: `#!/usr/bin/env python3`
   - [ ] Loads config/backend.yaml and config/frontend.yaml
   - [ ] Launches uvicorn for backend with correct port and settings
   - [ ] Launches frontend (npm dev or node server.js) with correct port
   - [ ] Signal handling (SIGTERM/SIGINT) for clean shutdown
   - [ ] Process monitoring and output display with prefixes
   - [ ] Valid Python syntax

3. **scripts/install_service.py**
   - [ ] File exists and is executable
   - [ ] Shebang present: `#!/usr/bin/env python3`
   - [ ] Checks for sudo/root permission
   - [ ] Loads config/backend.yaml and config/network.yaml
   - [ ] Generates systemd service file from template
   - [ ] Replaces placeholders: {PROJECT_DIR}, {SERVICE_USER}, {BACKEND_PORT}, etc.
   - [ ] Installs to /etc/systemd/system/ with correct permissions
   - [ ] Runs systemctl daemon-reload and enable
   - [ ] Valid Python syntax

4. **scripts/export_prod.py**
   - [ ] File exists and is executable
   - [ ] Shebang present: `#!/usr/bin/env python3`
   - [ ] Loads config/backend.yaml to find data_dir
   - [ ] Creates tar.gz archive with data and config files
   - [ ] Excludes config/secrets.yaml (actual secrets, not example)
   - [ ] Includes config/*.yaml.example files
   - [ ] Output to backups/{timestamp}.tar.gz or custom path
   - [ ] Displays archive size and summary
   - [ ] Valid Python syntax

5. **Security (subprocess, permissions, file ops)**
   - [ ] All subprocess calls use shell=False with list args
   - [ ] No f-strings in shell commands
   - [ ] File operations respect umask/permissions
   - [ ] No hardcoded credentials in scripts

6. **Integration**
   - [ ] Each script loads YAML config files correctly
   - [ ] Scripts reference new config/ structure (not inventory.env)
   - [ ] Error messages are helpful and actionable
</verification>

<success_criteria>
- 4 Python scripts created (deploy.py, run_standalone.py, install_service.py, export_prod.py)
- All scripts use PyYAML to parse config files
- All scripts are executable with proper shebangs
- deploy.py handles Docker deployment with health checks
- run_standalone.py launches backend and frontend without Docker
- install_service.py creates systemd service with new config paths
- export_prod.py exports production data excluding secrets
- All subprocess calls use shell=False (secure)
- Error handling and logging present in all scripts
- Scripts integrate with new config/ structure (D-05, D-06, D-07)
</success_criteria>

<output>
After completion, create `.planning/phases/07-config-consolidation/07-03-SUMMARY.md`
</output>