204 — Practical File Automation

Intermediate

Combine `pathlib`, `os`, and `subprocess` to build a log-rotation script that globs for old log files, compresses them into a dated archive with `tar`, and deletes the originals. You will add a `--dry-run` flag, read config from environment variables, and wire everything together with `argparse` — the same pattern used in real DevOps tooling.

Learning Objectives

Create sample log files and set their modification timestamps with os.utime() to simulate aged files

Glob for files matching a pattern and read their modification time using pathlib stat().st_mtime

Filter files by age using datetime.datetime.fromtimestamp() and datetime.timedelta

Archive a batch of files into a dated .tar.gz using subprocess.run() with tar

Verify archive integrity with tar -tzf and delete the original files with Path.unlink()

Add a --dry-run flag that reports what would happen without making any changes

Read LOG_ROTATE_DIR from an environment variable and let CLI flags override it

Build a complete log_rotate.py script that integrates all Module 2 patterns

Step 1

Set up the lesson 204 workspace and create sample log files

Create the lesson-204 directory, activate the virtual environment, and generate a set of synthetic log files with different ages. You will use `os.utime()` to backdate some files so the rotation logic has real candidates to archive.

Commands to Run

mkdir -p ~/devops-python/lesson-204/logs ~/devops-python/lesson-204/archives

cd ~/devops-python/lesson-204

source ~/devops-python/lesson-101/devops-env/bin/activate

Files for This Step

make_sample_logs.pyPaste this into the file

import os
import time
from pathlib import Path

logs_dir = Path('logs')

# Create 6 log files: 3 recent (0–2 days old) and 3 old (8–15 days old)
files = [
    ('app-today.log',     0),
    ('app-yesterday.log', 1),
    ('app-2days.log',     2),
    ('app-8days.log',     8),
    ('app-10days.log',   10),
    ('app-15days.log',   15),
]

now = time.time()

for name, days_old in files:
    path = logs_dir / name
    path.write_text(f'Log content for {name}. Generated for lesson 204.\n' * 5)
    # Shift mtime back by days_old days
    mtime = now - days_old * 86400
    os.utime(path, (mtime, mtime))
    print(f'Created {path}  ({days_old} days old)')

print(f'\n{len(files)} log files ready in {logs_dir}/')

After Saving

python3 make_sample_logs.py

ls -lh logs/

What This Does

os.utime(path, (atime, mtime)) sets the access time and modification time of a file.

Both values are POSIX timestamps (seconds since the Unix epoch).

Subtracting days_old * 86400 from the current time shifts the mtime backward — 86400 is the number of seconds in a day.

The ls -lh output confirms each file has the expected date.

This technique is useful in tests and automation scripts that need to create realistic file-system state without waiting for real time to pass.

Expected Outcome

make_sample_logs.py prints six lines showing each file name and its age. ls -lh logs/ shows six files with dates spread across the past 15 days — the three oldest files show dates 8, 10, and 15 days in the past.

Pro Tips

1
The `logs/` and `archives/` subdirectories are created in `~/devops-python/lesson-204/`, which is your working directory for this lesson. All subsequent commands assume you are in that directory.
2
`os.utime()` requires a tuple of exactly two floats: `(access_time, modification_time)`. Passing a single value is a `TypeError`. To set both to the same value, repeat it: `(ts, ts)`.

Was this step helpful?