Chapter 2 · Test Discovery Rules and Run Control

Chapter goals: understand where pytest looks for tests, how to run exactly the subset you want, and how to read the exit code after each run.

2.1 The Full Test Discovery Rules

Chapter 1 boiled it down to two naming conventions, but pytest's collection process actually follows explicit recursive rules, straight from the official test discovery documentation:

1. With no arguments, collection starts from the testpaths setting (if configured) or from the current directory;
2. It recurses into subdirectories unless the directory name matches norecursedirs (defaults include *.egg, .*, _darcs, build, CVS, dist, node_modules, venv, etc.);
3. Within a directory, only files named test_*.py or *_test.py are imported as test modules;
4. From those files pytest collects:
   • test_-prefixed functions outside classes;
   • test_-prefixed methods inside classes that start with Test and have no __init__ method.

A single snippet makes collect vs. don't-collect explicit:

# tests/test_discovery_rules.py

def test_addition():            # ✔ collected: test_ prefix function
    ...


def helper_add():               # ✘ not collected: missing test_ prefix
    ...


def add_test():                 # ✘ not collected: must be a prefix, not a suffix
    ...


class TestCart:                 # ✔ test_ methods inside are collected
    def test_add_item(self):    # ✔
        ...

    def _setup_items(self):     # ✘ not collected: no test_ prefix
        ...


class CartTests:                # ✘ entire class skipped: doesn't start with Test
    def test_something(self):   # (ignored along with its class)
        ...

project/
├── src/                # source dir — never collected (doesn't match test filename patterns)
├── venv/               # skipped by default norecursedirs
├── tests/
│   ├── conftest.py     # not a test_*.py file — not collected as tests (but auto-loaded)
│   ├── test_app.py     # ✔ collected
│   └── helpers.py      # ✘ filename doesn't match the pattern
└── pyproject.toml

Common Pitfall: Helpers Collected as Tests

If you drop a utils_test.py full of helper functions under tests/, pytest will try to import and collect it. Conversely, when a test you wrote isn't discovered, first check the filename and function-name prefixes.

2.2 Selecting Tests Precisely with Node IDs

pytest assigns every test a node id in the form file path::ClassName::function_name; parametrized cases get [parameters] appended. Node ids let you pinpoint any test:

# Run all tests in one file
pytest tests/test_mod.py

# Run a single function
pytest tests/test_mod.py::test_func

# Run an entire test class
pytest tests/test_mod.py::TestClass

# Run one method of a class
pytest tests/test_mod.py::TestClass::test_method

# Run one specific parametrized case
pytest tests/test_mod.py::test_func[x1,y2]

Node IDs can be copied straight from pytest --collect-only -q output:

$ pytest --collect-only -q
tests/test_mod.py::test_func
tests/test_mod.py::TestClass::test_method

2 tests collected in 0.02s

2.3 Keyword Expressions -k and Marker Expressions -m

-k matches names by substring (case-insensitive) across filenames, class names, and function names. A demo set:

# tests/test_search_demo.py
class TestLogin:
    def test_valid_user(self):
        ...        # node id: test_search_demo.py::TestLogin::test_valid_user

    def test_invalid_password(self):
        ...


def test_login_rate_limit():
    ...

-k expressions support and / or / not combinations:

pytest -k "valid and not invalid"   # only TestLogin::test_valid_user
pytest -k "TestLogin"               # class name match — runs both methods
pytest -k "rate_limit"              # hits module-level test_login_rate_limit

Marker-based filtering uses -m (markers are covered in Chapter 4):

pytest -m slow              # only @pytest.mark.slow tests
pytest -m "not slow"        # exclude slow tests — an everyday dev habit

New in pytest 9

Modern versions also support marker expressions with arguments, e.g. pytest -m "slow(phase=1)", filtering by marker parameter values.

2.4 Common Run-Control Options

Option	Effect
-x / --maxfail=1	Stop at the first failure
--maxfail=N	Stop after N failures
-v	Show full node ID and result for every test
-q / -qq	Quiet mode, minimal output
--tb=short/no/line	Control traceback verbosity
-s	Disable output capturing so print goes straight through
--lf	Re-run only last-failed tests
--ff	Run last-failed tests first, then everything else

# A common CI combo: stop on first failure + short traceback
pytest -x --tb=short

# Local debugging: see what failed last time
pytest --lf -v

2.5 Exit Codes: Letting Scripts Know What Happened

Use a guaranteed-to-fail test to compare traceback levels:

# tests/test_tb_demo.py
def divide(a, b):
    return a / b


def test_divide():
    assert divide(6, 3) == 3   # deliberately wrong: correct value is 2

pytest tests/test_tb_demo.py --tb=short   # compact traceback, key lines only
pytest tests/test_tb_demo.py --tb=line    # one line per failure
pytest tests/test_tb_demo.py --tb=no      # suppress tracebacks entirely

In CI logs --tb=short balances readability against noise; drop to the default long mode when digging in locally.

When pytest finishes it returns an exit code, which CI pipelines and shell scripts use to judge success:

Exit Code	Meaning
0	All collected tests passed
1	At least one test failed or errored
2	Execution interrupted by the user (e.g. Ctrl-C)
3	Internal pytest error
4	Command-line usage error
5	No tests were collected at all

Exit code 5 is the most overlooked: if a typo'd path means nothing ran, pytest did NOT "succeed" — CI should treat it as a failure.

When invoking pytest from Python code, the exit code comes back as a return value instead of raising SystemExit:

# run_tests.py
import sys
import pytest

if __name__ == "__main__":
    # pass args explicitly; returns the exit code without exiting the process
    retcode = pytest.main(["-x", "-v", "tests"])
    if retcode == 5:
        print("Warning: no tests collected!")
    sys.exit(retcode)

Note: calling pytest.main() multiple times within one process is affected by Python's import cache — later calls may not reflect file changes, so the official docs discourage it.

2.6 Chapter Summary

• Collection = start point (testpaths or cwd) → prune via norecursedirs → match test_*.py / *_test.py → collect test_ functions and Test classes without __init__;
• The node id file::Class::func[param] is the universal key for pinpointing tests; --collect-only lists them all;
• -k filters by name expressions, -m by markers; -x/--maxfail fail fast;
• Of the six exit codes, 5 (no tests collected) is most often mistaken for success.

🧪 随堂测验

点击你认为正确的选项。答错时会展示正确答案与原因解析。

1. Which class will have its test_ methods collected by pytest?

2. pytest exits with exit code 5. What does it mean?

3. You only want to run the test_login method of class TestUser in tests/test_api.py. Which command is correct?

4. Regarding norecursedirs, which statement is correct?

🛠️ Hands-On Exercises

1. Run pytest --collect-only -q in your project, count how many cases get collected, then execute one via its node ID and another via -k; compare both workflows.
2. Create an "empty collection": run pytest some-nonexistent-dir, confirm exit code 5 (echo $?), then write a one-line shell check that detects this condition.
3. Run your suite once with pytest -x --tb=line, deliberately insert a doomed assertion somewhere, and observe the "stop at first failure" effect before restoring the code.

Once you can control which tests run confidently, the next chapter digs into the expressive power of assertions themselves.