Text files, JSON, and other formats

JSON in two lines

JSON is the format APIs and lots of modern tools use. Python's json module reads it straight into a dict.

import json

# Read
with open("config.json") as f:
    config = json.load(f)
print(config["api_key"])

# Write
data = {"name": "Alice", "scores": [10, 20, 30]}
with open("out.json", "w") as f:
    json.dump(data, f, indent=2)

JSON ↔ DataFrame

import pandas as pd
df = pd.read_json("orders.json")                  # array of objects
df.to_json("clean.json", orient="records", indent=2)

Parsing a log file

lines = Path("server.log").read_text().splitlines()
errors = [l for l in lines if "ERROR" in l]
print(len(errors), "errors")

XML — yes, you'll occasionally see it

For old enterprise systems and government data, you'll get XML. The fastest way: pandas can read simple XML tables.

df = pd.read_xml("data.xml")

For gnarlier XML, the standard library's xml.etree.ElementTree works, and the third-party lxml is faster.