From first script to DevOps automation tool
Learn Python the way DevOps engineers actually use it — writing CLI tools with argparse, automating files and processes with pathlib and subprocess, handling JSON/YAML configs, calling REST APIs, and building a complete automation tool that queries GitHub and generates reports.
Before You Start
5 modules • 1 open
Install Python 3.12 on macOS or Linux, create an isolated virtual environment, and confirm pip is ready — the foundation every DevOps automation script depends on.
Write a Python script from scratch, add a shebang line so it runs directly from the terminal, and accept command-line arguments with sys.argv — the three foundational patterns every DevOps Python tool is built on.
Replace manual sys.argv handling with Python's built-in argparse module to define named arguments, optional flags, restricted choices, and auto-generated --help output — the pattern behind every professional Python CLI tool.
Replace bare print() calls with Python's stdlib logging module to emit structured, level-based output, then use sys.exit() with meaningful non-zero codes so CI/CD pipelines can detect failures automatically.
Use Python's stdlib pathlib module to read, write, append, rename, delete, and search files on disk — then combine those primitives into a DevOps script that reads a deploy config file and writes a timestamped run log.
Use Python's stdlib `os` module to read and write environment variables, inspect the running OS, and build portable file paths — then extend `deploy_check.py` to read DEPLOY_ENV, APP_VERSION, and a configurable base path from the environment so behaviour changes without touching argparse.
Use Python's stdlib `subprocess.run()` to shell out to system commands from your scripts — capturing stdout and stderr, checking exit codes, raising errors on failure with `check=True`, and passing custom environments. Then extend `env_deploy_check.py` with a live system health check that shells out to `git` or `df`.
Combine `pathlib`, `os`, and `subprocess` to build a log-rotation script that globs for old log files, compresses them into a dated archive with `tar`, and deletes the originals. You will add a `--dry-run` flag, read config from environment variables, and wire everything together with `argparse` — the same pattern used in real DevOps tooling.
Use Python's built-in `json` module to parse config files, serialize Python objects to JSON, handle malformed input safely, and build a reusable config loader — the same pattern used by deployment scripts that read environment config from JSON files.
Install PyYAML and use yaml.safe_load() and yaml.dump() to read, write, and update YAML config files — the format used by Docker Compose, Kubernetes manifests, and GitHub Actions workflows. Learn why yaml.safe_load() is always required and why bare yaml.load() is a security risk you must avoid.
Install python-dotenv and use load_dotenv() to load .env files into the process environment. Understand env var precedence, handle missing .env files safely, and learn why .env files must never be committed to source control — with a .gitignore pattern and a .env.example template to enforce that boundary.
Build a layered config loader that merges settings from four sources — hardcoded defaults, a YAML or JSON config file, a .env secrets file, and runtime environment variable overrides — with explicit precedence at every layer: env vars win over .env, .env wins over the config file, and the config file wins over defaults. This pattern is used in virtually every production DevOps automation tool.
Install the requests library and use it to send GET and POST calls to real public APIs, inspect response objects, handle HTTP errors with raise_for_status(), and catch network failures with try/except. Every request in this lesson includes an explicit timeout — the single most important habit for production HTTP scripts. The lesson closes by importing load_config() from lesson 304's layered_config.py to show how api_base_url flows from the config layer directly into requests calls.
Replace the placeholder token from lesson 401 with a real GitHub Personal Access Token stored as GITHUB_TOKEN and learn every authentication pattern used in production REST API work: Bearer token headers, API key headers, and query-string tokens. Credentials are loaded from .env via load_dotenv() and os.environ.get() — never hardcoded. Operational config (base URL, timeout) stays in config.yaml via load_config(). The lesson closes with a reusable requests.Session for authenticated sessions and explicit 401/403 error handling.
Go beyond single API calls: parse JSON responses into Python data structures, navigate nested and optional fields safely, paginate through multi-page GitHub API results using the Link header, and implement explicit exponential backoff for rate-limited endpoints. The lesson closes with a reusable json_api_client.py that imports check_credentials() from lesson 402 and composes all these patterns into a single, production-ready function.
Close the API loop by receiving inbound HTTP payloads from external services — without a web framework. Build a webhook server using Python's stdlib http.server, read POST bodies from the raw socket stream, validate GitHub-style HMAC-SHA256 signatures with hmac.compare_digest, and respond with correct HTTP status codes (200, 400, 401, 411). The lesson closes with webhook_receiver.py: a self-contained server that loads the shared secret from .env, routes events by type, and emits structured log output — all patterns used daily in DevOps event pipelines.