Daniel Bedeleanu
dacf7d2921
Discussion locked in 4 key decisions: 1. YAML Format: All configs standardized on YAML (backend.yaml, frontend.yaml, network.yaml, docker.yaml) for better structure and validation 2. Secrets Management: Separate config/secrets.yaml file (git-ignored) with config/secrets.yaml.example template + strong documentation 3. Examples: Commit all .yaml.example files for developer onboarding 4. Scripts: Audit bash scripts for redundancy, then convert all needed ones to Python for native YAML parsing and cleaner architecture Immediate deprecation of inventory.env (no fallback after Phase 7). Discussion log preserved in 07-DISCUSSION-LOG.md Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
9.3 KiB
Phase 7: Config Consolidation - Discussion Log
Audit trail only. Do not use as input to planning, research, or execution agents. Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
Date: 2026-04-23 Phase: 7-config-consolidation Areas discussed: Secrets Management, Config File Examples, Backward Compatibility Timeline, Config File Format & Scripts
1. Secrets & Sensitive Values
Question: How should sensitive values (API keys, JWT secrets, database passwords) be managed in the config structure?
| Option | Description | Selected |
|---|---|---|
| Store in config/backend.env (git-ignored) | Developers create their own local config/backend.env with secrets. File is in .gitignore so secrets never reach git. Example files committed separately for reference. | |
| Environment variables only | No sensitive values in config files. Deploy processes inject secrets via OS environment variables (Docker secrets, systemd, k8s secrets). Config files contain only non-sensitive settings. | |
| Separate secrets file | Create config/secrets.env (separate from backend.env) with stricter .gitignore rules. Non-sensitive backend config still in backend.env. Clearer separation of concerns. | ✓ |
| External secrets manager reference | Config files contain references/URLs to external secret storage (HashiCorp Vault, AWS Secrets Manager, etc.). Application loads secrets at runtime from external source. |
User's choice: Separate secrets file
Notes: Clear separation of concerns. Dedicated file for sensitive values with strict handling.
1b. Secrets Template Approach
Question: Should config/secrets.env.example be committed to git to guide developers on required secrets?
| Option | Description | Selected |
|---|---|---|
| Yes, commit example with placeholder values | config/secrets.env.example in git (not git-ignored) shows all required secrets and example formats. Developers copy it to config/secrets.env and fill in real values locally. | |
| No, document in README only | No example file in git. Required secrets documented in config/README.md and DEPLOYMENT.md. Reduces git noise, relies on good documentation. | |
| Both — example + strong documentation | Commit config/secrets.env.example AND provide detailed docs explaining each secret, where to get it, format requirements. Best for onboarding but more maintenance. | ✓ |
User's choice: Both — example + strong documentation
Notes: Provides clear onboarding path for developers while maintaining comprehensive documentation.
2. Config File Format & Examples
Question: How should non-sensitive config file examples be managed?
| Option | Description | Selected |
|---|---|---|
| Commit all .example files | All config files have .example variants in git. Developers copy them locally. Easy onboarding, version control of all config schemas. | ✓ |
| Only document in config/README.md | No .example files. config/README.md lists all variables, defaults, and descriptions. Cleaner git repo, single source of truth is docs. | |
| Template files (versioned, not copyrighted) | Commit minimal template files with just variable names (no values). Like .example but more concise. Developers fill in values. Balance between clarity and repo cleanliness. | |
| Docker compose example in separate file | Only docker.env.example committed (since it's tied to docker-compose.yml). Backend/frontend examples in docs only. Minimal git footprint for typical deployments. |
User's choice: Commit all .example files
Notes: Developer-friendly approach. Clear templates to copy and fill in.
3. Backward Compatibility Timeline
Question: How long should the system support loading from old inventory.env as fallback?
| Option | Description | Selected |
|---|---|---|
| This phase only (immediate deprecation) | After Phase 7 completes, inventory.env support is removed entirely. Requires all deployments to migrate immediately. Cleanest going forward, but higher migration burden. | ✓ |
| 2-3 releases (gradual sunset) | Support old paths for next 2-3 releases. Deprecated in logging. Then removed. Gives production deployments time to migrate gracefully. | |
| Keep indefinitely for backward compatibility | Never remove old paths. Scripts support both config/ and inventory.env forever. Highest compatibility, but code complexity persists. |
User's choice: This phase only (immediate deprecation)
Notes: Clean break. All deployments must migrate during Phase 7. No ongoing dual-path complexity.
4. Config File Format & Deployment Scripts
Question: What file format should the consolidated config files use?
| Option | Description | Selected |
|---|---|---|
| Keep .env format (minimal change) | Stick with KEY=VALUE .env files already used by python-dotenv. Least disruptive. Backend and deployment scripts already know how to read this. | |
| Standardize on .yaml (structured but verbose setup) | Migrate to YAML format. More structured, easier to validate, better for complex configs. Requires adding pyyaml dependency, updating all loaders. | ✓ |
| Use .json (most rigid, easiest parsing) | JSON format for all configs. Very structured, standard parsing libraries, but less friendly for manual editing and comments. | |
| Multi-format support (backend handles both) | Support both .env AND .yaml/.json. Loader auto-detects format. Flexibility, but adds parsing complexity and documentation burden. |
User's choice: Standardize on .yaml
Notes: More structured format. Better for complex configurations and validation.
4b. YAML Configuration in Deployment Scripts
Question: For YAML-based config, how should deployment scripts (bash) access configuration?
| Option | Description | Selected |
|---|---|---|
| Scripts read .env files, backend reads YAML — dual formats | Keep scripts using .env (familiar bash tooling). Backend uses YAML. Translation/sync between the two. More maintenance, but each tool uses its native format. | |
| Export YAML to env vars, scripts use vars | Config loading converts YAML to environment variables before scripts run. Scripts source env vars. Cleaner separation, requires loader to run first. | |
| Convert to .yaml everywhere, write yaml parser for scripts | Everything uses YAML. Write or use lightweight YAML parser in bash (yq). Consistent format end-to-end, adds bash dependency. | |
| Keep .env for network/docker, YAML for backend only | config/network.env and config/docker.env stay as .env (for scripts). Only config/backend.yaml uses YAML. Hybrid approach based on where each is used. |
User's choice (custom): Convert all config to YAML files, and convert all bash deployment scripts to Python scripts
Notes: Comprehensive modernization. Replace bash scripts with Python equivalents that natively parse YAML. More significant scope, but cleaner end-to-end architecture.
4c. Script Conversion Scope
Question: Is script conversion (bash → Python) part of Phase 7 scope, or a follow-up phase?
| Option | Description | Selected |
|---|---|---|
| Include in Phase 7 | Rewrite all deployment/utility scripts in Python as part of this phase. Phase 7 scope expands significantly but delivers complete Python-based stack. | ✓ |
| Phase 7: YAML config only, follow-up phase for scripts | Phase 7 focuses on YAML config consolidation. Bash scripts adapted to read YAML (via yq or similar). Script conversion deferred to a future phase. | |
| Hybrid approach this phase | Phase 7: Convert the critical deployment scripts (deploy.sh, run_standalone.sh) to Python. Defer non-critical scripts (export_prod.sh, __push_ALL_to_remote.sh) to future phase. |
User's choice: Include in Phase 7
Notes: Comprehensive Phase 7 scope. All necessary Python scripts completed in this phase.
4d. Script Priority & Consolidation
Question: Which scripts are critical for Phase 7 conversion? Which can be deferred?
User's choice (custom): Audit all bash scripts to identify redundant/mergeable ones. Convert all needed scripts to Python.
Notes: Consolidation-focused approach. Before converting, identify opportunities to merge redundant functionality and eliminate unnecessary scripts. Then convert only what's essential.
Summary of Locked Decisions
- D-01: YAML format for all config files (backend.yaml, frontend.yaml, network.yaml, docker.yaml)
- D-02: Separate secrets.yaml file (git-ignored) + secrets.yaml.example (committed)
- D-03: Commit all .example files for config schema reference
- D-04: Immediate deprecation of inventory.env (no fallback after Phase 7)
- D-05: Convert all necessary bash deployment scripts to Python
- D-06: Audit scripts first to consolidate redundancy before conversion
Claude's Discretion
Areas where the user deferred to Claude's judgment:
- Structure and organization of Python scripts in
scripts/folder - YAML validation schema and enforcement approach
- Logging verbosity and format in Python deployment scripts
Deferred Ideas
(None mentioned during discussion)
Discussion conducted: 2026-04-23
Format: Structured Q&A with alternatives considered
Outcome: All gray areas resolved; ready for detailed planning