27 KiB
27 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, user_setup, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | user_setup | must_haves | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 07-config-consolidation | 04 | execute | 3 |
|
|
true |
|
|
Purpose: Complete D-07 (Docker & Compose update), D-08 (documentation), and D-04 (deprecation) for cohesive deployment experience.
Output: Updated docker-compose.yml, Dockerfile, entrypoint.sh, DEPLOYMENT.md, README.md, and .gitignore.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/phases/07-config-consolidation/07-CONTEXT.md @PROJECT_ARCHITECTURE.md @docker-compose.yml @backend/Dockerfile @backend/entrypoint.sh @DEPLOYMENT.md @README.md @.gitignore @config/README.md Task 1: Update docker-compose.yml to reference config/ and remove inventory.env env_file docker-compose.yml - docker-compose.yml (current file) - backend/entrypoint.sh (to understand how config is used in containers) - config/backend.yaml.example (to understand what config is needed) - config/docker.yaml.example (Docker-specific config) Update docker-compose.yml to integrate new config/ structure (per D-07):1. **Remove inventory.env env_file references:**
- Delete or comment out `env_file: - inventory.env` from backend service (currently line 14)
- Delete or comment out `env_file: - inventory.env` from frontend service (currently line 51)
- Keep proxy service as is (may not need env_file)
2. **Add config/ volume mount to backend service:**
- Keep existing volume mounts
- Add: `- ./config:/app/config:ro` (read-only, config should not be modified in container)
- Update volumes section to reflect new mount
3. **Add config/ volume mount to frontend service (if frontend needs config):**
- Add: `- ./config:/app/config:ro` if frontend needs to read config files
- Or skip if frontend doesn't read YAML config directly
4. **Update environment variables:**
- Keep all existing environment variables (Docker overrides are still valid per D-06)
- Ensure JWT_SECRET_KEY is still set with warning: `# CHANGE THIS IN PRODUCTION!`
- Add comment: "Environment variables override config/backend.yaml per D-06 load order"
- Ensure DATA_DIR and LOGS_DIR are set to persist volume locations
5. **Add proxy service config/ mount (if proxy reads config):**
- Check if Caddyfile uses any dynamic config
- If not, no change needed
- If yes, add: `- ./config:/app/config:ro`
6. **Verify volume definitions:**
- Named volumes (backend_data, backend_logs, frontend_logs, caddy_data, caddy_config) remain unchanged
- Config mount is bind mount (./config), not named volume
7. **Add comments explaining the change:**
- Add section comment before volumes: "# [D-07] New config/ structure — YAML config mounted read-only"
- Add comment on env_file removal: "# [D-04] inventory.env deprecated — config now in config/ folder"
- Reference Phase 7 decisions
8. **Maintain backward compatibility during transition:**
- Don't delete old env_file line yet (can exist but be ignored by modern docker-compose)
- Or clearly comment it out with deprecation notice
Example section after update:
```yaml
backend:
...
# [D-04] inventory.env deprecated — see config/ folder instead
# env_file: - inventory.env
volumes:
- backend_data:/app/data
- backend_logs:/app/logs
# [D-07] New config/ structure mounted read-only
- ./config:/app/config:ro
- ./scripts:/app/scripts:ro
...
```
1. **Add comments explaining config/ structure:**
- Add comment at top: "# [D-07] Backend container - config/ folder mounted at /app/config (read-only)"
- Add comment before WORKDIR: "# Config is expected in /app/config (mounted from host)"
2. **Ensure volume mount points exist:**
- Config is mounted at runtime by docker-compose, not created in Dockerfile
- No changes needed to RUN commands for config directory
- (Already handled by docker-compose volume mount)
3. **Update any hardcoded paths referencing inventory.env:**
- Search for "inventory.env" in Dockerfile
- Replace with reference to config/ or remove if no longer needed
- Example: If old CMD references inventory.env, update to reference config/
4. **Update ENTRYPOINT or CMD if needed:**
- Ensure entrypoint.sh (or equivalent) references config/ paths
- Add environment documentation: "# Config sources: /app/config/backend.yaml, /app/config/secrets.yaml, environment variables"
5. **Add healthcheck if not present:**
- Verify backend has healthcheck (curl to /health endpoint)
- Should already be in docker-compose.yml, but double-check Dockerfile
6. **Document environment variables:**
- Add comment: "# Environment variables override YAML config per D-06"
- List: DATA_DIR, LOGS_DIR, LOG_LEVEL (these come from env and/or config)
Example after update:
```dockerfile
# [D-07] Backend container - config/ folder mounted at /app/config (read-only)
# Config sources: /app/config/backend.yaml, /app/config/secrets.yaml, environment variables
# Environment variables override YAML config per D-06 load order
FROM python:3.12-slim
WORKDIR /app
# Copy code, requirements, and startup scripts
COPY backend/ ./backend/
COPY scripts/ ./scripts/
COPY requirements.txt .
# Install dependencies
RUN pip install --no-cache-dir -r requirements.txt
# Config is mounted at /app/config by docker-compose
# No need to COPY config/ here (it's mounted read-only)
ENTRYPOINT ["python", "backend/main.py"]
```
1. **Remove inventory.env sourcing:**
- Delete any lines that source or check for inventory.env
- Delete any EXPORT statements that copy inventory.env values to environment
2. **Add config/ path documentation:**
- Add comment at top: "# [D-07] Backend entrypoint - loads config from /app/config/ (YAML format)"
- Add comment: "# Config sources: /app/config/backend.yaml, /app/config/secrets.yaml, environment variables"
3. **Add environment variable override documentation:**
- Add comment: "# [D-06] Environment variables override YAML config — set below takes precedence"
- List typical overrides: JWT_SECRET_KEY, PRIMARY_AI_PROVIDER, LOG_LEVEL, etc.
4. **Ensure config validation:**
- Add check: if [ ! -f "/app/config/backend.yaml" ]; then log error and instructions
- Add comment: "# Config validation handled by Python config_loader.py"
5. **Set working directory:**
- Ensure WORKDIR is set to /app (should be done in Dockerfile, but double-check)
6. **Exec main process:**
- Ensure entrypoint uses `exec` to replace shell: `exec python backend/main.py`
- This ensures signals (SIGTERM) are properly handled by Python process
Example after update:
```bash
#!/bin/bash
# [D-07] Backend entrypoint - loads config from /app/config/ (YAML format)
# Config sources: /app/config/backend.yaml, /app/config/secrets.yaml, environment variables
# [D-06] Environment variables override YAML config (below takes precedence)
set -euo pipefail
cd /app
# Verify config is accessible
if [ ! -f "/app/config/backend.yaml" ]; then
echo "[ERROR] /app/config/backend.yaml not found!"
echo "[ERROR] Config must be mounted from host at /app/config/"
echo "[ERROR] See config/README.md for setup instructions"
exit 1
fi
# Environment variables below override YAML config
# (docker run -e JWT_SECRET_KEY="..." or docker-compose environment)
# Start backend (signals properly handled with exec)
exec python -m uvicorn backend.main:app --host 0.0.0.0 --port 8000
```
7. **Keep it minimal:**
- Entrypoint should be simple (most logic in Python config_loader.py)
- Just verify config exists and start the app
1. **Add config/ rules:**
```
# [D-08] Config folder — track examples, ignore actual configs and secrets
config/*.yaml
!config/*.yaml.example
config/secrets.yaml
!config/secrets.yaml.example
```
2. **Rationale:**
- `config/*.yaml` — Ignore all YAML files (actual configs with real secrets)
- `!config/*.yaml.example` — Except examples (these are tracked for schema/documentation)
- `config/secrets.yaml` — Explicitly ignore secrets file (redundant but clear)
- `!config/secrets.yaml.example` — Except example (for developer setup guidance)
3. **Clean up old rules:**
- Remove any existing `inventory.env` entries from .gitignore (deprecated)
- Or update to comment them as deprecated: `# inventory.env # [D-04] Deprecated - use config/backend.yaml`
4. **Add comment at top of config section:**
- Add comment: "# [D-04] inventory.env deprecated — see config/ folder instead"
- Add comment: "# [D-08] Config structure: examples tracked, actual configs ignored"
5. **Example .gitignore config section:**
```
# [D-04] inventory.env deprecated — see config/ folder instead
# [D-08] Config structure: examples tracked (schema), actual configs ignored (secrets)
config/*.yaml
!config/*.yaml.example
config/secrets.yaml
!config/secrets.yaml.example
```
6. **Verify git status:**
- After update, git status should show:
- config/*.yaml.example as "new file" or tracked
- config/*.yaml as ignored
- config/secrets.yaml as ignored
1. **Add section: Configuration (Before Quick Start)**
- Explain config/ as single source of truth
- List config files: backend.yaml, frontend.yaml, network.yaml, docker.yaml, secrets.yaml
- Reference config/README.md for detailed setup
- Explain examples and how to create actual config from examples
2. **Update Quick Start section:**
- Step 1: Clone and cd
- Step 2: Copy config examples to actual files: `cp config/*.yaml.example {without .example}`
- Step 3: Edit config files (backend.yaml, network.yaml, secrets.yaml) with your values
- Step 4: Choose deployment mode (Docker or Standalone)
3. **Update Docker Deployment section:**
- Replace old deploy.sh instructions with new deploy.py
- Usage: `python3 scripts/deploy.py [production|staging|development] [--rebuild]`
- Script handles: pre-flight checks, port validation, config loading, health checks
- Output: service status, access URLs
4. **Update Standalone Deployment section:**
- Replace old run_standalone.sh with new run_standalone.py
- Usage: `python3 scripts/run_standalone.py [--backend-only|--frontend-only]`
- Script handles: config loading, backend startup, frontend startup, signal handling
5. **Update Configuration Reference section:**
- Refer to config/README.md for complete reference
- List key files: config/backend.yaml, config/frontend.yaml, config/network.yaml, config/docker.yaml, config/secrets.yaml
- Explain environment variable overrides (D-06)
6. **Add Systemd Service Installation section:**
- Usage: `sudo python3 scripts/install_service.py [--user=www-data]`
- Explain what service does: runs standalone backend + frontend
- Show how to manage: `systemctl start|stop|status ainventory`
- Show logs: `journalctl -u ainventory -f`
7. **Add Backup & Export section:**
- Usage: `python3 scripts/export_prod.py [--output=/path/to/backup.tar.gz] [--include-logs]`
- Explains what is included: data, config, config templates
- Explains what is excluded: actual secrets (security), logs (optional)
8. **Add Security section:**
- Mention secrets.yaml is git-ignored
- Explain how to set up secrets (copy from example, fill in values)
- Warn about JWT_SECRET_KEY in docker-compose.yml (must change for production)
9. **Add Troubleshooting section:**
- Common issues: missing config files, invalid YAML syntax, port conflicts
- Debug steps: check config syntax, verify file permissions, run health checks
- Reference config/README.md for setup help
10. **Add Migration section (from old inventory.env):**
- Explain Phase 7 transition from inventory.env to config/ structure
- Provide migration script or manual steps
- Clear instructions: which old values map to which config files
Structure should be roughly:
- 1. Overview (unchanged)
- 2. Prerequisites (unchanged)
- **3. Configuration** (NEW)
- 4. Quick Start (UPDATED)
- 5. Deployment Modes (Docker, Standalone) (UPDATED to use Python scripts)
- 6. Systemd Service (UPDATED with Python script)
- 7. Backup & Export (UPDATED with Python script)
- 8. Operations & Health Monitoring (UPDATED with new paths)
- 9. Security (NEW or UPDATED)
- 10. Troubleshooting (UPDATED)
- 11. Migration from inventory.env (NEW)
1. **Add Configuration section (after Quick Start or before Deployment):**
- Brief explanation: config/ folder is single source of truth
- Quick steps: cp config/*.example to remove .example, edit with your values
- Reference config/README.md for detailed setup
- Reference DEPLOYMENT.md for deployment options
2. **Update Quick Start section (if exists):**
- Add configuration step before deployment
- Example:
```
# 1. Clone and setup
git clone ... && cd tfm-inventory
# 2. Configure application
cp config/*.yaml.example config/$(basename {} .example) # or similar
nano config/backend.yaml # Edit with your values
cp config/secrets.yaml.example config/secrets.yaml
nano config/secrets.yaml # Fill in API keys and secrets
# 3. Deploy (choose one)
python3 scripts/deploy.py production # Docker
# OR
python3 scripts/run_standalone.py # Standalone
```
3. **Add note about .gitignore:**
- Config examples are tracked (for schema)
- Actual configs are git-ignored (protect secrets)
- secrets.yaml is git-ignored (never commit)
4. **Cross-reference documentation:**
- Add links/references to:
- config/README.md (configuration reference)
- DEPLOYMENT.md (detailed deployment guide)
- dev_docs/ (for development setup)
5. **Add "Getting Help" section (if not exists):**
- Point to DEPLOYMENT.md troubleshooting
- Point to config/README.md for config questions
- Reference AI_RULES.md for project conventions
6. **Update any hardcoded inventory.env references:**
- Replace with config/ references
- Update any env-related docs/examples
7. **Maintain existing structure:**
- Don't delete or significantly reorder existing sections
- Just add/update config-related content and update cross-references
Keep README concise but informative. Detailed docs go in DEPLOYMENT.md and config/README.md.
<threat_model>
Trust Boundaries
| Boundary | Description |
|---|---|
| Git repository → Deployment | .gitignore must prevent secrets from being committed |
| Documentation → Users | Documentation must clearly explain security requirements |
| Environment → Container | Docker environment variables can expose secrets if logged |
STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|---|---|---|---|---|
| T-07-13 | Tampering | docker-compose volume mount | mitigate | Config volume mounted read-only :ro. Backend cannot modify config at runtime. Changes require host-level edits. |
| T-07-14 | Information Disclosure | DEPLOYMENT.md instructions | mitigate | Documentation warns to generate JWT_SECRET_KEY, don't use placeholder. Warns about secrets.yaml setup. |
| T-07-15 | Information Disclosure | .gitignore config rules | mitigate | Clear rules prevent accidental secret commits. !config/*.example exception ensures schema is tracked. |
| T-07-16 | Elevation of Privilege | Docker container permissions | mitigate | No RUN as root in Dockerfile. Container runs as unprivileged user (if specified in docker-compose). |
</threat_model>
**Phase 7, Plan 4 Verification Checklist:**-
docker-compose.yml
- inventory.env env_file removed or commented (per D-04)
- ./config:/app/config:ro volume mount added to backend service
- Syntax valid:
docker-compose configsucceeds - Comments reference D-07, D-04 decisions
- Environment variables preserved (JWT_SECRET_KEY etc. with production warning)
-
backend/Dockerfile
- No references to inventory.env
- Comments reference config/ structure and D-07
- ENTRYPOINT or CMD properly set
- Syntax valid:
docker build --dry-runor manual parse
-
backend/entrypoint.sh
- No sourcing of inventory.env
- References /app/config/ paths
- Checks if config/backend.yaml exists
- Documents environment variable override behavior
- Bash syntax valid:
bash -n backend/entrypoint.sh
-
.gitignore
- Rules added: config/.yaml, !config/.yaml.example, config/secrets.yaml, !config/secrets.yaml.example
- Old inventory.env references removed or marked deprecated
- Git test:
git check-ignore config/backend.yamlreturns success (ignored) - Git test:
git status config/*.exampleshows untracked (not ignored)
-
DEPLOYMENT.md
- Configuration section added before or after Quick Start
- References config/ files (backend.yaml, frontend.yaml, network.yaml, docker.yaml, secrets.yaml)
- Docker deployment updated to use scripts/deploy.py
- Standalone deployment updated to use scripts/run_standalone.py
- Systemd service section with scripts/install_service.py
- Backup section with scripts/export_prod.py
- Troubleshooting section
- Migration section (from inventory.env to config/)
- Length 150+ lines
-
README.md
- Configuration section added
- Quick start updated with config setup steps
- Cross-references to config/README.md and DEPLOYMENT.md
- Mentions secrets.yaml and .gitignore
- No hardcoded inventory.env references
-
Cross-document consistency
- README.md, DEPLOYMENT.md, config/README.md use consistent terminology
- All three documents reference each other appropriately
- No conflicting instructions across documents
<success_criteria>
- docker-compose.yml updated with config/ volume mount, inventory.env env_file removed (D-07)
- backend/Dockerfile and entrypoint.sh updated for new config paths
- .gitignore configured to track examples, ignore actual configs and secrets (D-08)
- DEPLOYMENT.md updated with YAML structure, Python scripts, systemd setup, troubleshooting (D-08)
- README.md updated with config setup and documentation cross-references (D-08)
- All documentation references config/ as single source of truth
- Docker deployment works with new config structure
- Clear migration path from inventory.env to new config/ structure documented </success_criteria>