Your path to DevOps mastery

25 minutes

1 / 8

304 — Config Management Patterns

Intermediate

Build a layered config loader that merges settings from four sources — hardcoded defaults, a YAML or JSON config file, a .env secrets file, and runtime environment variable overrides — with explicit precedence at every layer: env vars win over .env, .env wins over the config file, and the config file wins over defaults. This pattern is used in virtually every production DevOps automation tool.

Learning Objectives

Explain why production scripts load config from three separate sources: structured files for non-secret settings, .env for secrets, and env vars for runtime overrides

Define a DEFAULTS dict as the base layer that keeps the script runnable with no external config present

Load a YAML config file as the second layer and merge it over defaults using a dict update

Call load_dotenv() as the third layer so secrets in .env are available via os.getenv() without appearing in any config file

Apply an ENV_OVERRIDES mapping as the fourth and highest-priority layer using a type-safe _coerce() helper

Recognize that bool must be checked before int in _coerce() because bool is a subclass of int in Python

Build load_config() as a single function that assembles all four layers and returns a typed NamedTuple Config

Support both YAML and JSON config files by dispatching on the file extension inside load_config()

Build layered_config.py: a complete config loader that annotates each resolved value with its source layer

Step 1

Set up the lesson-304 workspace and create the three config source files

Create the lesson-304 directory, activate the virtual environment, and verify that PyYAML and python-dotenv are installed. Then create config.yaml (non-secret structured config), .env (secrets), and .gitignore. These three files together represent the three external config sources that the layered loader will merge.

Commands to Run

mkdir -p ~/devops-python/lesson-304

cd ~/devops-python/lesson-304

source ~/devops-python/lesson-101/devops-env/bin/activate

pip show pyyaml python-dotenv | grep -E 'Name|Version'

Files for This Step

config.yamlPaste this into the file

# Non-secret deployment config — safe to commit to source control
environment: staging
api_base_url: https://api.staging.example.internal
api_timeout: 15
debug: false
max_retries: 5
deploy_host: deploy.staging.example.internal
deploy_port: 8080

.envPaste this into the file

# Secrets — git-ignored, never commit this file
API_KEY=secret-key-abc123

.gitignorePaste this into the file

.env
.env.local
__pycache__/
*.pyc

After Saving

echo '=== config.yaml ===' && cat config.yaml && echo && echo '=== .env ===' && cat .env

What This Does

The three config sources have different trust levels and different use cases. config.yaml holds structured, non-secret settings that vary by environment — it is safe to commit and review in a pull request.

.env holds secrets (API keys, tokens, passwords) that must never reach a git repository.

Environment variables are injected at runtime by the platform (CI/CD, Docker, Kubernetes, systemd) and carry the highest precedence because they represent the live operating environment.

Separating these three sources is not just a convention: it is a security boundary.

Merging them in a single load_config() function gives the caller a clean Config object with no awareness of which source each value came from.

Expected Outcome

pip show prints Name: PyYAML and Name: python-dotenv with their versions. cat config.yaml shows the 7 non-secret config values. cat .env shows API_KEY=secret-key-abc123.

The lesson-304 directory contains config.yaml, .env, and .gitignore.

Pro Tips

1
The split between config.yaml and .env is intentional: config.yaml documents the shape of your configuration (what keys exist, what values are reasonable) while .env provides the secret values. Teammates can see config.yaml in code review; they cannot see .env because it is git-ignored.
2
In production, config.yaml would typically be committed to the repo with production-safe defaults or staging values. Secrets come from a secrets manager (AWS Secrets Manager, HashiCorp Vault, GitHub Actions secrets) that injects them as environment variables — meaning the .env file is only used in local development.

Common Mistakes to Avoid

⚠️Putting API keys or passwords in config.yaml because it is easier — once a secret appears in a committed file, it is in the git history permanently even if removed in a later commit. Treat config.yaml as public; treat .env as private.
⚠️Creating .gitignore after .env is already staged or committed — git tracks files it has seen. Run `git rm --cached .env` to untrack it, then add .gitignore and commit the removal.

Was this step helpful?