File Handling: Reading and Writing Text, CSV and JSON Files (pathlib)

Automation scripts rarely keep data only in RAM. They read configuration files in JSON, parse daily exports in CSV, scan server logs line by line and write summary reports to disk. Python makes file input/output (I/O) safe and cross-platform through the with context manager, the built-in csv and json modules and modern object-oriented paths via pathlib.

Why You Should Always Use with open(...)

Opening a file claims an operating-system file handle. If your script crashes before calling file.close(), the file can stay locked or unwritten data can be lost. Using a with block (a context manager) guarantees Python closes the file automatically the moment the block exits—even if an exception occurs inside:

# Writing text to a file (UTF-8 encoding explicitly set)
with open("deployment.log", mode="w", encoding="utf-8") as log_file:
    log_file.write("2026-10-21 10:00:00 - Deployment started\n")
    log_file.write("2026-10-21 10:01:15 - Health check passed\n")

# Reading a file line by line (memory-efficient for huge files)
with open("deployment.log", mode="r", encoding="utf-8") as log_file:
    for line in log_file:
        print(line.strip())

File Open Modes Explained

Mode Meaning Behaviour If File Exists / Does Not Exist
"r" Read text (default) Reads file; raises FileNotFoundError if missing
"w" Write text Overwrites existing file completely, or creates a new file
"a" Append text Adds new lines to the end of the file without erasing existing content
"rb" / "wb" Read / Write binary Used for non-text files such as images, PDFs or zip archives

Cross-Platform File Paths With pathlib

Windows uses backslashes (\) while Linux and macOS use forward slashes (/). Instead of gluing path strings together manually, use Python’s standard pathlib.Path object, which overloads the / operator to build paths cleanly on every operating system:

from pathlib import Path

base_dir = Path("reports") / "2026" / "october"
base_dir.mkdir(parents=True, exist_ok=True)  # Creates folders if missing

report_file = base_dir / "summary.txt"
report_file.write_text("All systems operational.", encoding="utf-8")

if report_file.exists():
    print(f"Saved to {report_file.resolve()}")
    print("Content:", report_file.read_text(encoding="utf-8"))

Working With CSV and JSON Files

Never parse CSV files with line.split(",") (which breaks whenever a field contains a quoted comma). Use the built-in csv and json modules:

import csv
import json

# 1. Writing and reading structured JSON configuration
config_data = {"service": "auth-api", "port": 8443, " allowed_ips": ["10.0.0.1", "10.0.0.2"]}

with open("config.json", "w", encoding="utf-8") as f:
    json.dump(config_data, f, indent=2)

with open("config.json", "r", encoding="utf-8") as f:
    loaded_config = json.load(f)

# 2. Reading a CSV file into dictionaries by column header
with open("servers.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["hostname", "region", "status"])
    writer.writeheader()
    writer.writerow({"hostname": "srv-01", "region": "ap-south-1", "status": "online"})

with open("servers.csv", "r", encoding="utf-8") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(f"{row['hostname']} in {row['region']} is {row['status']}")

Common Mistakes

  • Omitting encoding="utf-8": On Windows, Python’s default file encoding can vary by system locale, causing UnicodeDecodeError when reading special characters or emojis. Always pass encoding="utf-8" explicitly.
  • Using file.read() on multi-gigabyte log files: Calling .read() or .readlines() loads the entire file into RAM at once. Iterate directly over the file object (for line in file:) to stream one line at a time.
  • Forgetting newline="" when writing CSVs on Windows: Without newline="" inside open(), Python’s csv.writer inserts blank lines between every row on Windows.
Key takeaways
  • Always open files inside a with open(..., encoding="utf-8") as f: block so file handles close automatically.
  • Use pathlib.Path instead of string concatenation for reliable, cross-platform file and directory operations.
  • Use csv.DictReader / csv.DictWriter for tabular files and json.load / json.dump for JSON data.

Related Lessons