bede/tfm_ainventory
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs(phase-7): capture consolidated context from structured discussion

Browse Source 
Discussion locked in 4 key decisions:

1. YAML Format: All configs standardized on YAML (backend.yaml, frontend.yaml,
   network.yaml, docker.yaml) for better structure and validation

2. Secrets Management: Separate config/secrets.yaml file (git-ignored) with
   config/secrets.yaml.example template + strong documentation

3. Examples: Commit all .yaml.example files for developer onboarding

4. Scripts: Audit bash scripts for redundancy, then convert all needed ones
   to Python for native YAML parsing and cleaner architecture

Immediate deprecation of inventory.env (no fallback after Phase 7).

Discussion log preserved in 07-DISCUSSION-LOG.md

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
This commit is contained in:
Daniel Bedeleanu
2026-04-23 12:20:53 +03:00
parent 2b7a2b3a89
commit dacf7d2921
2 changed files with 277 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

166
.planning/phases/07-config-consolidation/07-CONTEXT.md
View File

@@ -30,73 +30,145 @@ Consolidate all application configuration files into a centralized `config/` fol
## Implementation Decisions
### Configuration Structure
- Create `config/` folder in project root as the central location for all configurations
- Use meaningful filenames: `backend.env`, `frontend.env`, `network.env`, `docker.env` (for docker-compose)
- Keep backward compatibility: Support both old (`inventory.env`) and new (`config/backend.env`) paths during transition
- Environment variable precedence: System env vars > Docker build args > config/backend.env > defaults in code
### D-01: Configuration File Format
- **Standardize on YAML format** for all config files (backend.yaml, frontend.yaml, network.yaml, docker.yaml)
- All config files in `config/` folder will be YAML format
- Backend code updated to use PyYAML parser
- Rationale: YAML provides better structure for complex configs, easier validation, clearer schema
### Backend Loading
- Update `backend/config_loader.py` to look for configs in `config/backend.env` first
- If not found, fall back to `inventory.env` (backward compatibility)
- If neither exists, use system environment variables
### D-02: Secrets Management (Separate File)
- Create dedicated `config/secrets.yaml` file for sensitive values (API keys, JWT secrets, database passwords)
- Add `config/secrets.yaml` to `.gitignore` with strict exclusion
- Commit `config/secrets.yaml.example` with placeholder values and clear format requirements
- Include strong documentation in config/README.md explaining each secret, where to obtain it, format requirements
- Rationale: Clear separation of concerns between configuration and secrets, guides developers on required values
### D-03: Config File Examples
- Commit `.example` files for ALL config files: `backend.yaml.example`, `frontend.yaml.example`, `network.yaml.example`, `docker.yaml.example`
- Developers copy examples to non-example versions locally and fill in values
- Example files show structure, defaults, and all available options
- Rationale: Clear onboarding path, version control of config schema, consistency guarantees
### D-04: Backward Compatibility - Immediate Deprecation
- **NO fallback to `inventory.env`** - immediate deprecation after Phase 7 completes
- All deployments must migrate to new `config/` structure during this phase
- Remove all code paths that read from root-level `inventory.env`
- Rationale: Clean break avoids ongoing dual-path support complexity
### D-05: Deployment Scripts - Convert Bash to Python
- Convert all necessary bash deployment scripts to Python with identical functionality
- Before conversion: **Audit all scripts to identify redundant/mergeable ones**
- Critical scripts to convert: `deploy.sh`, `run_standalone.sh`, `install_service.sh`, `export_prod.sh`
- Evaluate `__push_ALL_to_remote.sh` for necessity/consolidation
- All Python scripts will parse YAML config files
- Rationale: Consistent tooling across infrastructure, easier YAML parsing, reduced bash complexity
### D-06: Backend Config Loading
- Update `backend/config_loader.py` to parse YAML files
- Load order: System environment variables > `config/backend.yaml` > defaults in code
- Remove any fallback to `inventory.env` (Phase 7 end → fully deprecated)
- Log which config source is being used for debugging
### Deployment Scripts
- `deploy.sh`: Source environment from `config/network.env` or `config/docker.env` instead of `inventory.env`
- `run_standalone.sh`: Source configuration from `config/backend.env` and `config/network.env`
- Docker Compose: Updated to reference `config/docker.env`
- Scripts should create symlinks or validate that config folder exists before running
### D-07: Docker & Docker Compose
- Docker Compose updated to reference `config/docker.yaml`
- Backend Dockerfile and frontend Dockerfile updated to source from new config structure
- Environment variable injection mechanism preserved (takes precedence over YAML files)
### Cleanup Actions
- Remove or archive `inventory.env` templates after migration
- Audit root directory scripts for redundancy
- Update .gitignore to ensure config/ folder structure is tracked (if needed)
### Documentation Updates
- Update DEPLOYMENT.md with new config folder structure
- Update README.md with configuration setup instructions
- Add config/README.md explaining each config file's purpose
### D-08: Documentation & Git Structure
- Update DEPLOYMENT.md with new YAML config structure, required secrets, and format specifications
- Update README.md with configuration setup and onboarding instructions
- Add comprehensive `config/README.md` explaining all YAML files, required variables, examples, secrets setup
- Update .gitignore: ignore `config/*.yaml` (except examples), track `config/*.yaml.example`
---
## Specific Ideas
1. **Config Files to Create:**
- `config/backend.env` — Backend-specific variables (database, AI keys, auth settings)
- `config/frontend.env` — Frontend-specific variables (API endpoints, feature flags)
- `config/network.env` — Network/deployment variables (ports, SSL, server IPs)
- `config/docker.env` — Docker-specific overrides (for docker-compose.yml)
- `config/README.md` — Documentation of all configuration files
1. **YAML Config Files to Create:**
- `config/backend.yaml` — Backend-specific variables (database, AI keys, auth settings, logging)
- `config/backend.yaml.example` — Template showing all available options
- `config/frontend.yaml` — Frontend-specific variables (API endpoints, feature flags, service worker settings)
- `config/frontend.yaml.example` — Frontend config template
- `config/network.yaml` — Network/deployment variables (ports, SSL, server IPs, CORS settings)
- `config/network.yaml.example` — Network config template
- `config/docker.yaml` — Docker-specific overrides (for docker-compose.yml)
- `config/docker.yaml.example` — Docker config template
- `config/secrets.yaml` — Sensitive values (git-ignored)
- `config/secrets.yaml.example` — Secrets template with placeholders
- `config/README.md` — Comprehensive documentation of all YAML files, structure, required values
2. **Scripts to Update:**
- `deploy.sh` — Update to source from config/ folder
- `run_standalone.sh` — Update network config loading
- `export_prod.sh` — Ensure it uses new config structure
- `install_service.sh` — Update systemd service to reference new config paths
- `__push_ALL_to_remote.sh` — Review for config-related updates
2. **Python Scripts to Create (replacing bash):**
- `scripts/deploy.py` — Docker deployment with YAML config parsing (replaces deploy.sh)
- `scripts/run_standalone.py` — Standalone mode launcher (replaces run_standalone.sh)
- `scripts/export_prod.py` — Production export/backup functionality
- `scripts/install_service.py` — Systemd service installation (replaces install_service.sh)
- Audit `__push_ALL_to_remote.sh` - determine if needed or consolidate into another script
- All scripts will use PyYAML for config file parsing
3. **Backend Changes:**
- `backend/config_loader.py` — Update load order: config/backend.env > inventory.env > env vars
- `backend/config_manager.py` — Update to read/write from config/backend.env
- `backend/entrypoint.sh` — Source from config/ instead of root level
- `backend/config_loader.py` — Update to parse YAML files (backend.yaml + secrets.yaml)
- Load order: System env vars > config/backend.yaml > config/secrets.yaml > defaults in code
- Remove all code paths for reading inventory.env
- Implement environment variable override mechanism (system env vars take precedence)
- `backend/config_manager.py` — Update to read/write YAML (if config updates are needed at runtime)
- `backend/entrypoint.sh` — Reference new config paths in container
4. **Testing Requirements:**
- Docker deployment with new config structure
- Standalone deployment with new config structure
- Environment variable override behavior still works
- Backward compatibility: old inventory.env still loads if config/ doesn't exist
- Docker deployment with YAML config structure
- Standalone Python launcher with YAML config parsing
- Environment variable override behavior with YAML configs
- Secrets file permissions and git-ignore verification
- All deployment paths tested end-to-end with new Python scripts
- Confirm old inventory.env paths are NOT accessible (no fallback)
---
## Claude's Discretion
- Structure and organization of Python scripts in `scripts/` folder
- YAML validation schema and enforcement approach (basic vs. strict validation)
- Logging verbosity and format in Python deployment scripts
---
## Canonical References
**Downstream agents MUST read these before planning or implementing.**
### Configuration & Deployment
- `DEPLOYMENT.md` — Current deployment procedures (to be updated with YAML structure)
- `README.md` — Project setup instructions (to be updated with config onboarding)
- `PROJECT_ARCHITECTURE.md` — Technical stack and component overview
### Backend Configuration Loading
- `backend/config_loader.py` — Current config loading implementation (will be refactored for YAML)
- `backend/config_manager.py` — Current config management (will be updated)
### Deployment Scripts (to be rewritten in Python)
- `deploy.sh` — Docker deployment script
- `run_standalone.sh` — Standalone mode launcher
- `export_prod.sh` — Production export
- `install_service.sh` — Systemd service setup
- `__push_ALL_to_remote.sh` — Remote push utility (to be audited for necessity)
### Infrastructure Files
- `docker-compose.yml` — Docker composition (to be updated to reference config/docker.yaml)
- `backend/Dockerfile` — Backend container image (to be updated for YAML config paths)
- `frontend/Dockerfile` — Frontend container image (if applicable)
- `inventory.env` — Current config location (will be deprecated after Phase 7)
---
## Deferred Ideas
- Dynamic config hot-reload without restart (future optimization)
- Config validation framework (future enhancement)
- Encrypted sensitive values in config files (future security enhancement)
- Web UI for configuration management (future feature)
- Dynamic config hot-reload without restart (future optimization — Phase N)
- Config validation framework with JSON Schema (future enhancement — Phase N)
- Encrypted sensitive values in config files via KMS or Vault (future security enhancement — Phase N+1)
- Web UI for configuration management (future feature — Phase N+2)
- Config versioning and rollback mechanism (future ops enhancement)
---
*Phase: 7-config-consolidation*
*Context gathered: 2026-04-23 from user requirements*
*Context gathered: 2026-04-23 via structured discussion*
*Status: Ready for detailed planning with locked decisions*