Chapter 16 · Performance Optimization and Resource Management

Chapter goals: Master connection pooling, caching, compression and other optimization techniques to improve HTTP client performance.

16.1 Connection Pool Configuration

import requests
from requests.adapters import HTTPAdapter

session = requests.Session()

# Fine-grained connection pool control
adapter = HTTPAdapter(
    pool_connections=20,      # Connection pool size
    pool_maxsize=100,         # Maximum connections
    max_retries=3             # Failure retries
)
session.mount('https://', adapter)
session.mount('http://', adapter)

16.2 Response Caching

import requests
from requests_cache import CachedSession

# Enable disk cache
session = CachedSession('cache.sqlite', expire_after=3600)

# Same requests automatically return cached response
resp1 = session.get('https://api.example.com/data')  # Real request
resp2 = session.get('https://api.example.com/data')  # Cache hit

16.3 Compression Transfer

import requests

# Automatically handle gzip/deflate compression
resp = requests.get('https://api.example.com/large-data')
print(resp.headers['content-encoding'])  # gzip

# Specify accepted encoding in request
resp = requests.get(
    'https://api.example.com/data',
    headers={'Accept-Encoding': 'gzip, deflate'}
)

16.4 httpx Performance Configuration

import httpx

# Fine-grained Limits configuration
limits = httpx.Limits(
    max_connections=100,           # Maximum connections
    max_keepalive_connections=20,  # Keep-alive connections
    keepalive_expiry=30.0          # Keep-alive timeout (seconds)
)

with httpx.Client(limits=limits, timeout=10.0) as client:
    resp = client.get('https://httpbin.org/get')

16.5 Chapter Summary

• Connection pool reuse significantly improves multiple-request performance;
• Response caching avoids duplicate network requests;
• Compression reduces bandwidth consumption;
• httpx.Limits finely controls connection resources.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What are the core configuration parameters for requests connection pool?

2. What is the main purpose of requests-cache?

3. What does keepalive_expiry in httpx.Limits do?

4. Which method cannot reduce HTTP request latency?

🛠️ Hands-on Practice

1. Configure connection pool size, compare different pool_maxsize impacts on concurrent performance.
2. Use requests-cache to cache API responses, verify second call speed for identical requests.
3. Enable gzip compression, measure response body size changes.

After completing the exercises, proceed to Next chapter: API Client Encapsulation Patterns.