# Phase 7: Config Consolidation - Context

**Gathered:** 2026-04-23
**Status:** Ready for planning
**Source:** User Requirements

---

## Phase Boundary

Consolidate all application configuration files into a centralized `config/` folder in the project root. This includes backend configurations, frontend configurations, deployment settings, and network configurations. Update all deployment scripts, application startup procedures, and backend/frontend code to load configurations from this centralized location.

**Scope:**
- Create and establish `config/` folder as the single source of truth for all application configuration
- Migrate existing configuration files (inventory.env and variants) to config folder with meaningful names
- Update all scripts (deploy.sh, run_standalone.sh, etc.) to reference the new config location
- Refactor backend config_loader.py and config_manager.py to read from config folder
- Update frontend environment loading if applicable
- Verify and clean up root directory scripts that are no longer needed
- Ensure Docker deployment, standalone deployment, and development all work correctly with new structure

**Deliverables:**
- `config/` folder with structured configuration files
- Updated backend configuration loading mechanism
- Updated deployment scripts
- Updated startup procedures (Docker and standalone)
- Documentation of configuration structure in README/DEPLOYMENT.md

---

## Implementation Decisions

### D-01: Configuration File Format
- **Standardize on YAML format** for all config files (backend.yaml, frontend.yaml, network.yaml, docker.yaml)
- All config files in `config/` folder will be YAML format
- Backend code updated to use PyYAML parser
- Rationale: YAML provides better structure for complex configs, easier validation, clearer schema

### D-02: Secrets Management (Separate File)
- Create dedicated `config/secrets.yaml` file for sensitive values (API keys, JWT secrets, database passwords)
- Add `config/secrets.yaml` to `.gitignore` with strict exclusion
- Commit `config/secrets.yaml.example` with placeholder values and clear format requirements
- Include strong documentation in config/README.md explaining each secret, where to obtain it, format requirements
- Rationale: Clear separation of concerns between configuration and secrets, guides developers on required values

### D-03: Config File Examples
- Commit `.example` files for ALL config files: `backend.yaml.example`, `frontend.yaml.example`, `network.yaml.example`, `docker.yaml.example`
- Developers copy examples to non-example versions locally and fill in values
- Example files show structure, defaults, and all available options
- Rationale: Clear onboarding path, version control of config schema, consistency guarantees

### D-04: Backward Compatibility - Immediate Deprecation
- **NO fallback to `inventory.env`** - immediate deprecation after Phase 7 completes
- All deployments must migrate to new `config/` structure during this phase
- Remove all code paths that read from root-level `inventory.env`
- Rationale: Clean break avoids ongoing dual-path support complexity

### D-05: Deployment Scripts - Convert Bash to Python
- Convert all necessary bash deployment scripts to Python with identical functionality
- Before conversion: **Audit all scripts to identify redundant/mergeable ones**
- Critical scripts to convert: `deploy.sh`, `run_standalone.sh`, `install_service.sh`, `export_prod.sh`
- Evaluate `__push_ALL_to_remote.sh` for necessity/consolidation
- All Python scripts will parse YAML config files
- Rationale: Consistent tooling across infrastructure, easier YAML parsing, reduced bash complexity

### D-06: Backend Config Loading
- Update `backend/config_loader.py` to parse YAML files
- Load order: System environment variables > `config/backend.yaml` > defaults in code
- Remove any fallback to `inventory.env` (Phase 7 end → fully deprecated)
- Log which config source is being used for debugging

### D-07: Docker & Docker Compose
- Docker Compose updated to reference `config/docker.yaml`
- Backend Dockerfile and frontend Dockerfile updated to source from new config structure
- Environment variable injection mechanism preserved (takes precedence over YAML files)

### D-08: Documentation & Git Structure
- Update DEPLOYMENT.md with new YAML config structure, required secrets, and format specifications
- Update README.md with configuration setup and onboarding instructions
- Add comprehensive `config/README.md` explaining all YAML files, required variables, examples, secrets setup
- Update .gitignore: ignore `config/*.yaml` (except examples), track `config/*.yaml.example`

---

## Specific Ideas

1. **YAML Config Files to Create:**
   - `config/backend.yaml` — Backend-specific variables (database, AI keys, auth settings, logging)
   - `config/backend.yaml.example` — Template showing all available options
   - `config/frontend.yaml` — Frontend-specific variables (API endpoints, feature flags, service worker settings)
   - `config/frontend.yaml.example` — Frontend config template
   - `config/network.yaml` — Network/deployment variables (ports, SSL, server IPs, CORS settings)
   - `config/network.yaml.example` — Network config template
   - `config/docker.yaml` — Docker-specific overrides (for docker-compose.yml)
   - `config/docker.yaml.example` — Docker config template
   - `config/secrets.yaml` — Sensitive values (git-ignored)
   - `config/secrets.yaml.example` — Secrets template with placeholders
   - `config/README.md` — Comprehensive documentation of all YAML files, structure, required values

2. **Python Scripts to Create (replacing bash):**
   - `scripts/deploy.py` — Docker deployment with YAML config parsing (replaces deploy.sh)
   - `scripts/run_standalone.py` — Standalone mode launcher (replaces run_standalone.sh)
   - `scripts/export_prod.py` — Production export/backup functionality
   - `scripts/install_service.py` — Systemd service installation (replaces install_service.sh)
   - Audit `__push_ALL_to_remote.sh` - determine if needed or consolidate into another script
   - All scripts will use PyYAML for config file parsing

3. **Backend Changes:**
   - `backend/config_loader.py` — Update to parse YAML files (backend.yaml + secrets.yaml)
   - Load order: System env vars > config/backend.yaml > config/secrets.yaml > defaults in code
   - Remove all code paths for reading inventory.env
   - Implement environment variable override mechanism (system env vars take precedence)
   - `backend/config_manager.py` — Update to read/write YAML (if config updates are needed at runtime)
   - `backend/entrypoint.sh` — Reference new config paths in container

4. **Testing Requirements:**
   - Docker deployment with YAML config structure
   - Standalone Python launcher with YAML config parsing
   - Environment variable override behavior with YAML configs
   - Secrets file permissions and git-ignore verification
   - All deployment paths tested end-to-end with new Python scripts
   - Confirm old inventory.env paths are NOT accessible (no fallback)

---

## Claude's Discretion

- Structure and organization of Python scripts in `scripts/` folder
- YAML validation schema and enforcement approach (basic vs. strict validation)
- Logging verbosity and format in Python deployment scripts

---

## Canonical References

**Downstream agents MUST read these before planning or implementing.**

### Configuration & Deployment
- `DEPLOYMENT.md` — Current deployment procedures (to be updated with YAML structure)
- `README.md` — Project setup instructions (to be updated with config onboarding)
- `PROJECT_ARCHITECTURE.md` — Technical stack and component overview

### Backend Configuration Loading
- `backend/config_loader.py` — Current config loading implementation (will be refactored for YAML)
- `backend/config_manager.py` — Current config management (will be updated)

### Deployment Scripts (to be rewritten in Python)
- `deploy.sh` — Docker deployment script
- `run_standalone.sh` — Standalone mode launcher
- `export_prod.sh` — Production export
- `install_service.sh` — Systemd service setup
- `__push_ALL_to_remote.sh` — Remote push utility (to be audited for necessity)

### Infrastructure Files
- `docker-compose.yml` — Docker composition (to be updated to reference config/docker.yaml)
- `backend/Dockerfile` — Backend container image (to be updated for YAML config paths)
- `frontend/Dockerfile` — Frontend container image (if applicable)
- `inventory.env` — Current config location (will be deprecated after Phase 7)

---

## Deferred Ideas

- Dynamic config hot-reload without restart (future optimization — Phase N)
- Config validation framework with JSON Schema (future enhancement — Phase N)
- Encrypted sensitive values in config files via KMS or Vault (future security enhancement — Phase N+1)
- Web UI for configuration management (future feature — Phase N+2)
- Config versioning and rollback mechanism (future ops enhancement)

---

*Phase: 7-config-consolidation*  
*Context gathered: 2026-04-23 via structured discussion*  
*Status: Ready for detailed planning with locked decisions*