Skillversitas

Test Structure and Discovery

The discovery is the rule that decides which files and functions are tests: the test_*.py or *_test.py files, the test_* functions, and the Test* classes (without the __init__) - the match is the name, and the rootdir plus the testpaths bound where the search runs. The import is the other half: the conftest.py locations set the rootdir, and the importmode (the prepend, the importlib) decides how the test module is imported (the sys.path insertion the package-under-test resolves). This chapter covers the discovery rules (the file, the function, the class), the rootdir and the testpaths, the conftest.py role, and the import modes - the structure that makes the bare pytest find exactly the right tests.

The Discovery Rules

# the FILE: the test_*.py (the prefix) or the *_test.py
# (the suffix) - the default is the prefix, the ini's
# python_files sets the pattern (the "*_test.py" the
# project that prefers the suffix)
tests/
  test_word.py          # the DISCOVERED (the prefix)
  word_test.py          # the NOT (the no pattern match)
  helpers.py            # the NOT (the no test_ prefix,
                        #   the import, not a test file)

# the FUNCTION: the test_* (the prefix) inside a
# discovered file - the plain function, the no return
def test_word_count(): ...      # the DISCOVERED
def check_word_count(): ...     # the NOT (the no prefix)

# the CLASS: the Test* (the prefix, the capital T) -
# the methods the test_* (the no __init__, the no
# __new__) - the the unittest.TestCase is the special
# case (the next chapters)
class TestPalindrome:
    def test_pal(self): ...     # the DISCOVERED
    def setup_method(self): ... # the NOT (the hook, not a test)

The discovery is the name match on three levels. The file is the test_*.py (the default prefix) or the *_test.py (the suffix, the python_files ini option when the project prefers it), and the helpers/utils modules that the tests import are not discovered (the name is the test). The function is the test_* inside a discovered file - the plain function, no return value (the return is ignored, the assert is the check), and the class is the Test* with the capital T, its test_* methods collected, and the __init__/__new__ forbidden (the instance is per-method, the state is the fixture or the setup_method). The --collect-only is the verify: it prints the tree of the collected items (the file, the class, the method) without running, and the count is the sanity check that the discovery found the suite the developer expects - the test that is silently not collected is the bug the flag catches.

The rootdir and the testpaths

# the rootdir: the the anchor the the config resolves
# from - the the pytest finds it by the the walking up
# from the the argument, looking for the the pyproject.toml
# (the [tool.pytest.ini_options]), the pytest.ini, the
# tox.ini, the setup.cfg, or the the conftest.py
# the the rootdir is the the base of the the relative
# paths (the the testpaths, the the cache)

# the the testpaths: the the DEFAULT search (the the bare
# "pytest" with no the argument searches these, not the
# whole the tree)
# the the pyproject.toml:
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]

# the the effect: the the bare "pytest" runs the tests/
# only (the the no stray test_*.py in the the src tree
# the collected), and the the explicit "pytest src/"
# overrides the the testpaths (the the argument wins)

The rootdir is the base the configuration resolves from: the pytest walks up from the argument (or the cwd) looking for the pyproject.toml with the [tool.pytest.ini_options], the pytest.ini, the tox.ini, or the setup.cfg, and the conftest.py at the top is the fallback anchor - the rootdir is where the relative paths (the testpaths, the .pytest_cache) resolve. The testpaths is the default search: the bare pytest (no argument) runs only those directories, which keeps the stray test_*.py in the src tree or the vendored package from being collected, and the explicit argument (the pytest src/) overrides it. The four python_* options are the discovery patterns (the file, the class, the function) the project tunes, and the conftest.py - the next section - is both the rootdir anchor and the shared-fixture module, the two roles that make it the file every pytest project has.

The conftest.py and the Import Mode

# the conftest.py: the DISCOVERY anchor + the SHARED
# fixture module - the pytest imports every conftest.py
# on the path from the rootdir to the test file (the
# the tests/conftest.py, the tests/sub/conftest.py)
# the the fixture defined in a conftest is available to
# every test in that directory and below

# the the tests/conftest.py (the the shared sample data):
import pytest

@pytest.fixture
def sample_text():
    return "the quick brown fox jumps over the lazy dog"

# the the importmode: the the HOW the the test module is
# imported (the the src/ not in the the sys.path, the
# the package the tested needs the the path)
#   prepend (the default): the the inserts the the
#     first directory WITHOUT an __init__.py (the the
#     tests/) into the the sys.path[0]
#   importlib: the the no sys.path manipulation, the
#     the module is imported by the the path (the the
#     namespace the the unique)
# the importlib is the the choice when the the
# the test file name COLLIDES (the the test_word.py in
# two the dirs, the the prepend the the shadow)

The conftest.py is the file the project always has, with the two roles. The first is the rootdir anchor: the pytest uses the conftest location (the absence of a higher conftest or the ini file) to fix the rootdir, and the relative configuration resolves from it. The second is the shared-fixture module: the pytest imports every conftest.py on the path from the rootdir down to the test file, and a fixture defined in the tests/conftest.py is available to every test in that directory and below (the sample_text the sample data, the no import in the test file - the fixture is requested by the name). The import mode is the other half of the structure: the prepend (the default) inserts the first directory without an __init__.py (the tests) into the sys.path so the test module imports, and the importlib mode skips the sys.path manipulation and imports by the path - the choice when two test files share the name (the test_word.py in two directories, the prepend would shadow one). The structure is the rules plus the anchor: the name match decides the test, the rootdir and the testpaths bound the search, and the conftest is the shared fixture - the next chapter moves from the discovery to the assertion, the check the collected test actually runs.

The Working Rules

  • The name is the test. The test_*.py file, the test_* function, the Test* class (the no __init__) - the python_files/python_classes/python_functions the tunable patterns, and the --collect-only the verify the suite is the one the developer expects.
  • The rootdir is the config's base, the testpaths the default search. The rootdir resolves from the ini/conftest the walk-up finds, the testpaths bounds the bare pytest (the no stray collection), and the explicit argument overrides the testpaths.
  • The conftest.py is the anchor plus the shared fixture. The pytest imports every conftest from the rootdir down, the fixture in the tests/conftest.py reaches every test below, and the conftest is the rootdir fallback when the ini is absent.
  • The importmode is the collision's switch. The prepend (the default) the sys.path insertion, the importlib the path-based import - the importlib when the test file name collides across the directories.

Summary

The discovery is the name match: the test_*.py file (the *_test.py the python_files option), the test_* function (the plain, the no return), and the Test* class (the no __init__, the test_* methods), with the --collect-only the tree and the count the sanity check. The rootdir is the base the configuration resolves from (the walk-up to the pyproject.toml/pytest.ini/tox.ini or the conftest.py), and the testpaths is the default search the bare pytest runs (the explicit argument the override). The conftest.py is the file every project has: the rootdir anchor and the shared-fixture module (the pytest imports every conftest from the rootdir down, the fixture reaches every test below), and the importmode (the prepend the sys.path insertion, the importlib the path-based import the collision's switch) is the import half of the structure. The next chapter is the assertion - the check the collected test runs, and the rewriting that makes the failure readable.