Chapter 9 · Exception Handling and Best Practices

Chapter goals: understand the exception hierarchies of requests and httpx, and write robust error-handling code.

9.1 The requests Exception Hierarchy

requests has a clear inheritance tree:

requests.exceptions.RequestException
├── ConnectionError      # network connection failed
│   ├── ProxyError       # proxy connection error
│   └── SSLError         # SSL certificate error
├── HTTPError            # HTTP status error (4xx/5xx)
├── Timeout              # request timed out
│   ├── ConnectTimeout
│   └── ReadTimeout
├── TooManyRedirects     # too many redirects
└── MissingSchema        # URL missing a scheme

import requests

try:
    resp = requests.get('https://api.example.com/data', timeout=5)
    resp.raise_for_status()  # check for HTTP errors
except requests.exceptions.HTTPError as e:
    print(f'HTTP error: {e.response.status_code}')
except requests.exceptions.ConnectionError as e:
    print(f'Connection failed: {e}')
except requests.exceptions.Timeout as e:
    print(f'Timeout: {e}')
except requests.exceptions.RequestException as e:
    print(f'Other error: {e}')

9.2 The httpx Exception System

httpx exception names are more intuitive:

import httpx

try:
    with httpx.Client() as client:
        resp = client.get('https://api.example.com/data', timeout=5)
        resp.raise_for_status()
except httpx.ConnectTimeout:
    print('Connect timeout')
except httpx.ReadTimeout:
    print('Read timeout')
except httpx.HTTPStatusError as e:
    print(f'HTTP {e.response.status_code}')
except httpx.RequestError as e:
    print(f'Request error: {e}')

9.3 A Unified Error-Handling Helper

For production code, wrap a shared handler:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

def create_session(retries=3, backoff=1) -> requests.Session:
    """Create a session with a retry strategy"""
    session = requests.Session()
    retry = Retry(
        total=retries,
        backoff_factor=backoff,
        status_forcelist=[429, 500, 502, 503, 504],
        allowed_methods=["GET", "POST"]
    )
    adapter = HTTPAdapter(max_retries=retry)
    session.mount('http://', adapter)
    session.mount('https://', adapter)
    return session

def safe_get(url: str, **kwargs) -> requests.Response:
    """GET with centralized error handling"""
    session = create_session()
    try:
        resp = session.get(url, timeout=10, **kwargs)
        resp.raise_for_status()
        return resp
    except requests.exceptions.Timeout:
        print(f'Timed out: {url}')
    except requests.exceptions.HTTPError as e:
        print(f'HTTP error {e.response.status_code}: {url}')
    except requests.exceptions.ConnectionError:
        print(f'Connection failed: {url}')
    finally:
        session.close()

9.4 Context Managers and Resource Release

Important: always use with statements or close responses/clients manually:

import requests

# Option 1: with statement (recommended)
with requests.Session() as session:
    resp = session.get('https://api.example.com')
    data = resp.json()

# Option 2: manual cleanup
session = requests.Session()
try:
    resp = session.get('https://api.example.com')
    data = resp.json()
finally:
    session.close()

# httpx also needs closing
import httpx
with httpx.Client() as client:
    resp = client.get('https://api.example.com')

Failing to close connections leaks TCP connections until file descriptors run out.

9.5 Best Practices Checklist

1. Always set timeouts: timeout=(connect, read);
2. Use a Session: reuse connections for better performance;
3. Enable retries: automatic retries for idempotent calls (GET);
4. Catch specific exceptions: avoid bare except Exception;
5. Check status codes: call raise_for_status();
6. Release resources: use with or try/finally.

9.6 Chapter Summary

• requests exceptions inherit from RequestException, categorized precisely;
• httpx names are more intuitive, e.g. ConnectTimeout, HTTPStatusError;
• Always use context managers so connections release correctly;
• Wrap unified error handling in helper functions for robustness.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. How do you check for HTTP status errors in requests?

2. Which of these can cause connection leakage?

3. Which httpx exception class represents a connect timeout?

4. When enabling retries for idempotent requests (like GET), how should 5xx errors be treated?

🛠️ Hands-On Exercises

1. Write a unified handler that catches five different exception types and prints distinct messages.
2. Compare resource leakage with vs without with requests.Session() (watch file descriptors via lsof).
3. Wrap a retrying safe_get function and verify automatic retries on a 500 error.

After finishing the exercises, move on to the next chapter: Introducing httpx.