bede/tfm_ainventory
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Files
f2fb81024e7736cf032a959ae5d1d7773d08c6b1
tfm_ainventory/.planning/phases/07-config-consolidation/07-CONTEXT.md
Daniel Bedeleanu dacf7d2921 docs(phase-7): capture consolidated context from structured discussion 
Discussion locked in 4 key decisions:

1. YAML Format: All configs standardized on YAML (backend.yaml, frontend.yaml,
   network.yaml, docker.yaml) for better structure and validation

2. Secrets Management: Separate config/secrets.yaml file (git-ignored) with
   config/secrets.yaml.example template + strong documentation

3. Examples: Commit all .yaml.example files for developer onboarding

4. Scripts: Audit bash scripts for redundancy, then convert all needed ones
   to Python for native YAML parsing and cleaner architecture

Immediate deprecation of inventory.env (no fallback after Phase 7).

Discussion log preserved in 07-DISCUSSION-LOG.md

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-04-23 12:20:53 +03:00

175 lines
9.0 KiB
Markdown
Raw Blame History

# Phase 7: Config Consolidation - Context
**Gathered:** 2026-04-23
**Status:** Ready for planning
**Source:** User Requirements
---
## Phase Boundary
Consolidate all application configuration files into a centralized `config/` folder in the project root. This includes backend configurations, frontend configurations, deployment settings, and network configurations. Update all deployment scripts, application startup procedures, and backend/frontend code to load configurations from this centralized location.
**Scope:**
- Create and establish `config/` folder as the single source of truth for all application configuration
- Migrate existing configuration files (inventory.env and variants) to config folder with meaningful names
- Update all scripts (deploy.sh, run_standalone.sh, etc.) to reference the new config location
- Refactor backend config_loader.py and config_manager.py to read from config folder
- Update frontend environment loading if applicable
- Verify and clean up root directory scripts that are no longer needed
- Ensure Docker deployment, standalone deployment, and development all work correctly with new structure
**Deliverables:**
- `config/` folder with structured configuration files
- Updated backend configuration loading mechanism
- Updated deployment scripts
- Updated startup procedures (Docker and standalone)
- Documentation of configuration structure in README/DEPLOYMENT.md
---
## Implementation Decisions
### D-01: Configuration File Format
- **Standardize on YAML format** for all config files (backend.yaml, frontend.yaml, network.yaml, docker.yaml)
- All config files in `config/` folder will be YAML format
- Backend code updated to use PyYAML parser
- Rationale: YAML provides better structure for complex configs, easier validation, clearer schema
### D-02: Secrets Management (Separate File)
- Create dedicated `config/secrets.yaml` file for sensitive values (API keys, JWT secrets, database passwords)
- Add `config/secrets.yaml` to `.gitignore` with strict exclusion
- Commit `config/secrets.yaml.example` with placeholder values and clear format requirements
- Include strong documentation in config/README.md explaining each secret, where to obtain it, format requirements
- Rationale: Clear separation of concerns between configuration and secrets, guides developers on required values
### D-03: Config File Examples
- Commit `.example` files for ALL config files: `backend.yaml.example`, `frontend.yaml.example`, `network.yaml.example`, `docker.yaml.example`
- Developers copy examples to non-example versions locally and fill in values
- Example files show structure, defaults, and all available options
- Rationale: Clear onboarding path, version control of config schema, consistency guarantees
### D-04: Backward Compatibility - Immediate Deprecation
- **NO fallback to `inventory.env`** - immediate deprecation after Phase 7 completes
- All deployments must migrate to new `config/` structure during this phase
- Remove all code paths that read from root-level `inventory.env`
- Rationale: Clean break avoids ongoing dual-path support complexity
### D-05: Deployment Scripts - Convert Bash to Python
- Convert all necessary bash deployment scripts to Python with identical functionality
- Before conversion: **Audit all scripts to identify redundant/mergeable ones**
- Critical scripts to convert: `deploy.sh`, `run_standalone.sh`, `install_service.sh`, `export_prod.sh`
- Evaluate `__push_ALL_to_remote.sh` for necessity/consolidation
- All Python scripts will parse YAML config files
- Rationale: Consistent tooling across infrastructure, easier YAML parsing, reduced bash complexity
### D-06: Backend Config Loading
- Update `backend/config_loader.py` to parse YAML files
- Load order: System environment variables > `config/backend.yaml` > defaults in code
- Remove any fallback to `inventory.env` (Phase 7 end → fully deprecated)
- Log which config source is being used for debugging
### D-07: Docker & Docker Compose
- Docker Compose updated to reference `config/docker.yaml`
- Backend Dockerfile and frontend Dockerfile updated to source from new config structure
- Environment variable injection mechanism preserved (takes precedence over YAML files)
### D-08: Documentation & Git Structure
- Update DEPLOYMENT.md with new YAML config structure, required secrets, and format specifications
- Update README.md with configuration setup and onboarding instructions
- Add comprehensive `config/README.md` explaining all YAML files, required variables, examples, secrets setup
- Update .gitignore: ignore `config/*.yaml` (except examples), track `config/*.yaml.example`
---
## Specific Ideas
1. **YAML Config Files to Create:**
- `config/backend.yaml` — Backend-specific variables (database, AI keys, auth settings, logging)
- `config/backend.yaml.example` — Template showing all available options
- `config/frontend.yaml` — Frontend-specific variables (API endpoints, feature flags, service worker settings)
- `config/frontend.yaml.example` — Frontend config template
- `config/network.yaml` — Network/deployment variables (ports, SSL, server IPs, CORS settings)
- `config/network.yaml.example` — Network config template
- `config/docker.yaml` — Docker-specific overrides (for docker-compose.yml)
- `config/docker.yaml.example` — Docker config template
- `config/secrets.yaml` — Sensitive values (git-ignored)
- `config/secrets.yaml.example` — Secrets template with placeholders
- `config/README.md` — Comprehensive documentation of all YAML files, structure, required values
2. **Python Scripts to Create (replacing bash):**
- `scripts/deploy.py` — Docker deployment with YAML config parsing (replaces deploy.sh)
- `scripts/run_standalone.py` — Standalone mode launcher (replaces run_standalone.sh)
- `scripts/export_prod.py` — Production export/backup functionality
- `scripts/install_service.py` — Systemd service installation (replaces install_service.sh)
- Audit `__push_ALL_to_remote.sh` - determine if needed or consolidate into another script
- All scripts will use PyYAML for config file parsing
3. **Backend Changes:**
- `backend/config_loader.py` — Update to parse YAML files (backend.yaml + secrets.yaml)
- Load order: System env vars > config/backend.yaml > config/secrets.yaml > defaults in code
- Remove all code paths for reading inventory.env
- Implement environment variable override mechanism (system env vars take precedence)
- `backend/config_manager.py` — Update to read/write YAML (if config updates are needed at runtime)
- `backend/entrypoint.sh` — Reference new config paths in container
4. **Testing Requirements:**
- Docker deployment with YAML config structure
- Standalone Python launcher with YAML config parsing
- Environment variable override behavior with YAML configs
- Secrets file permissions and git-ignore verification
- All deployment paths tested end-to-end with new Python scripts
- Confirm old inventory.env paths are NOT accessible (no fallback)
---
## Claude's Discretion
- Structure and organization of Python scripts in `scripts/` folder
- YAML validation schema and enforcement approach (basic vs. strict validation)
- Logging verbosity and format in Python deployment scripts
---
## Canonical References
**Downstream agents MUST read these before planning or implementing.**
### Configuration & Deployment
- `DEPLOYMENT.md` — Current deployment procedures (to be updated with YAML structure)
- `README.md` — Project setup instructions (to be updated with config onboarding)
- `PROJECT_ARCHITECTURE.md` — Technical stack and component overview
### Backend Configuration Loading
- `backend/config_loader.py` — Current config loading implementation (will be refactored for YAML)
- `backend/config_manager.py` — Current config management (will be updated)
### Deployment Scripts (to be rewritten in Python)
- `deploy.sh` — Docker deployment script
- `run_standalone.sh` — Standalone mode launcher
- `export_prod.sh` — Production export
- `install_service.sh` — Systemd service setup
- `__push_ALL_to_remote.sh` — Remote push utility (to be audited for necessity)
### Infrastructure Files
- `docker-compose.yml` — Docker composition (to be updated to reference config/docker.yaml)
- `backend/Dockerfile` — Backend container image (to be updated for YAML config paths)
- `frontend/Dockerfile` — Frontend container image (if applicable)
- `inventory.env` — Current config location (will be deprecated after Phase 7)
---
## Deferred Ideas
- Dynamic config hot-reload without restart (future optimization — Phase N)
- Config validation framework with JSON Schema (future enhancement — Phase N)
- Encrypted sensitive values in config files via KMS or Vault (future security enhancement — Phase N+1)
- Web UI for configuration management (future feature — Phase N+2)
- Config versioning and rollback mechanism (future ops enhancement)
---
*Phase: 7-config-consolidation*
*Context gathered: 2026-04-23 via structured discussion*
*Status: Ready for detailed planning with locked decisions*
Reference in New Issue View Git Blame Copy Permalink