Chapter 18 · Property-Based Testing with Hypothesis

Chapter goals: grasp the essential difference between property-based and example-based testing, master Hypothesis strategies, settings, and counterexample shrinking, and learn to combine it with parametrize.

18.1 From "Examples" to "Properties"

Every test so far was example-based: a human picks a few inputs and writes the expected outputs. The blind spot is obvious — you only test inputs you thought of. sorted looks fine against [], [1], [2,1]… but who thinks to pass a list containing float('nan')?

Property-based testing asks a different question: not "what's the output for this input" but "which properties must hold for every valid input". For a sorting function, properties include:

• the output is ordered;
• the output is a permutation of the input (same length, same multiset of elements);
• idempotence: sorting again changes nothing.

Hypothesis generates hundreds or thousands of inputs from strategies to attack these properties; the moment it finds a counterexample it reports and pins it.

pip install hypothesis

18.2 @given and strategies 101

@given is the single core entry point, taking one or more strategies:

# test_hypo_basic.py
from hypothesis import given, strategies as st


@given(st.integers())
def test_abs_non_negative(n):
    assert abs(n) >= 0            # Property: abs is non-negative for every integer


@given(st.text())
def test_encode_decode_roundtrip(s):
    # Property: decode(encode(s)) == s (roundtrip properties are the most common pattern)
    assert s.encode("utf-8").decode("utf-8") == s


@given(st.lists(st.integers()))
def test_sorted_properties(xs):
    result = sorted(xs)
    assert len(result) == len(xs)                 # Length preserved
    assert all(result[i] <= result[i + 1] for i in range(len(result) - 1))
    assert sorted(result) == result               # Idempotent

Quick reference for common built-in strategies:

Strategy	Generates
st.integers(min_value, max_value)	Integers
st.floats(allow_nan=False)	Floats
st.text(alphabet=..., max_size=...)	Unicode text
st.lists(elements, min_size, max_size, unique=True)	Lists / deduped lists
st.dictionaries(keys, values)	Dictionaries
st.one_of(a, b)	Any of several strategies
st.sampled_from([...])	Sampled from given values

Assemble dependent inputs with @st.composite, and filter with .filter() / assume():

from hypothesis import assume, given, strategies as st


@st.composite
def ordered_pair(draw):
    """The draw function is injected by Hypothesis, letting you build dependent values sequentially."""
    lo = draw(st.integers())
    hi = draw(st.integers(min_value=lo))   # Guarantees hi >= lo
    return (lo, hi)


@given(st.integers(), st.integers())
def test_div(n1, n2):
    assume(n2 != 0)                        # Discard divide-by-zero cases
    q = n1 / n2
    assert q * n2 == n1 or True            # Illustrative: floating-point error needs separate care

18.3 @settings and Deterministic Reproduction

import pytest
from hypothesis import given, settings, strategies as st


@settings(max_examples=50, deadline=None)   # Lower counts speed up CI; deadline disables per-case timing assertions
@given(st.lists(st.integers(), min_size=1))
def test_min_is_member(xs):
    assert min(xs) in xs


# Relax the deadline explicitly for slow DB/network tests:
@settings(deadline=500)                     # A case fails only past 500ms
@given(st.text(max_size=100))
def test_indexing(s):
    assert s.find("") == 0                  # find("") always returns 0

The key mechanism — shrinking: on failure, Hypothesis doesn't hurl a giant random blob at you. It automatically tries smaller, simpler inputs until it finds a minimal counterexample. Failure output looks like:

Falsifying example: test_parse(
    s='0',
)

It also records the counterexample in the .hypothesis/examples database and replays historical failures first on subsequent runs — so regressions are deterministic: green only when actually fixed.

18.4 Combining with pytest.mark.parametrize

Hypothesis officially supports stacking with parametrization: enumerate "operations" outside, generate data inside:

import pytest
from hypothesis import given, strategies as st


@pytest.mark.parametrize("operation", [reversed, sorted])
@given(st.lists(st.integers()))
def test_operation_preserves_length(operation, lst):
    assert len(list(operation(lst))) == len(lst)

Rule: @given must be the outermost decorator (hugging the function), with parametrize arguments passed through as-is. The reverse order (given on top, parametrize below) is not supported.

18.5 When Is Hypothesis Worth It

• ✅ Parsers/serializers: roundtrip properties (encode→decode, dump→load) are near-zero cost, high reward;
• ✅ Math/financial code: algebraic properties like commutativity, associativity, monotonicity;
• ✅ Stateful systems: RuleBasedStateMachine generates random operation sequences for model-based testing (advanced topic);
• ❌ Pure IO orchestration (glue layers calling third-party APIs): no verifiable pure-function properties; mocking strips most of the value.

Cost note

Each property test runs 100 examples by default — big suites slow down noticeably. A common setup: full exploration locally, a --hypothesis-profile=ci profile lowering max_examples in CI, and a nightly job at high exploration settings.

Chapter Summary

• Property-based tests verify invariants over arbitrary valid inputs, attacking the imagination gaps of example-based testing;
• @given(strategy) plus built-in strategies cover most types; composite/filter/assume handle complex constraints;
• @settings(max_examples, deadline) tunes strength vs runtime; failing examples shrink automatically and persist for replay;
• @given must be outermost and stacks with parametrize into an "operation × data" matrix;
• Best for parsing, algorithms, numeric logic; limited payoff for IO glue.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. What is the core advantage of property-based testing over example-based testing?

2. In Hypothesis, what does assume(cond) do?

3. Which statement about shrinking is correct?

4. When combining @given with @pytest.mark.parametrize, what is the correct decorator order?

🛠️ Hands-on Practice

1. Write a roundtrip property test for a JSON serializer: json.loads(json.dumps(x)) == x, generating nested structures with st.recursive.
2. Bug-hunt a hand-rolled parse_duration("3h30m"): state two properties — "seconds roundtrip consistently" and "invalid input must raise" — let Hypothesis attack it, and record the first bug it catches.
3. Convert a Chapter 15 parametrized test into a parametrize + given combo, controlling runtime with @settings(max_examples=20).

Next chapter zooms out to integration testing strategy for whole web services.