Code Coverage with pytest-cov

The pytest-cov plugin wraps the coverage.py engine: the --cov flag runs the suite under the tracer (the pytest --cov=texttools), and the report shows which lines the tests executed. The --cov-report chooses the format (the term, the html, the xml, the json), the --cov-fail-under sets the threshold (the exit 1 when the total is below), the --cov-branch adds the branch metric, and the [tool.coverage.run] section of the pyproject.toml versions the settings. This chapter covers the --cov measurement, the report formats, the --cov-fail-under gate, and the branch coverage the line-only report hides.

The --cov: The Measurement

# the --cov: the MEASURE (the coverage.py tracer)
#   the pip install pytest-cov first
pytest --cov=texttools
#   Name                   Stmts   Miss  Cover
#   ------                 -----   ----  -----
#   texttools/__init__.py     12      2    83%
#   texttools/cipher.py       18      6    67%
#   ------                  30     8    73%
#   the Stmts = the statements, the Miss = the
#   not-executed, the Cover = the percent

# the the package (the the measured) vs the the no
#   the arg (the the cwd):
pytest --cov=texttools --cov=texttools.cipher
pytest --cov

The --cov is the measurement: the pytest --cov=texttools runs the suite under the coverage.py tracer, and the report is the per-file (the Stmts the statements, the Miss the not-executed, the Cover the percent) plus the total (the 30 the statements, the 8 the misses, the 73% the cover). The --cov=texttools names the package measured, the multiple --cov args measure the several packages, and the no-arg form measures the cwd - the report is the where, the --cov-report (the next section) the format, and the rule is the one: the --cov the measure, the --cov-report the format, the --cov-fail-under the threshold.

The Reports and the Threshold

# the --cov-report: the FORMAT (the term, the html,
#   the xml, the json - the default is the term)
pytest --cov=texttools --cov-report=term-missing
# the terminal PLUS the missing line numbers (the
#   dev's the gap the fix targets)

pytest --cov=texttools --cov-report=html
# the htmlcov/index.html (the the file the browser
#   the line-by-line, the red the uncovered)

pytest --cov=texttools --cov-report=xml
# the coverage.xml (the CI's, the diff-coverage's
#   the input, the SonarQube's)

pytest --cov=texttools --cov-report=json
# the coverage.json (the tool's, the dashboard's)

# the --cov-fail-under: the THRESHOLD (the exit 1
#   when the total is below - the CI's the gate)
pytest --cov=texttools --cov-fail-under=80
# the the total 73% -> the FAILED (the below 80)

The --cov-report chooses the format: the term-missing is the terminal plus the uncovered line numbers (the dev's gap list), the html writes the htmlcov/index.html (the file the browser opens, the line-by-line red), the xml writes the coverage.xml (the CI's and the diff-coverage's input), and the json writes the coverage.json (the tool's and the dashboard's). The multiple reports (the --cov-report=term-missing --cov-report=html) coexist in one run. The --cov-fail-under=80 is the threshold: the total below the number exits with the 1 (the FAILED run, the suite's green but the coverage's red), which is the gate the CI enforces (the no regression below the floor, the 80 the common starting point) - the report is the where, the threshold is the whether.

The Branch Coverage and the Config

# the --cov-branch: the BRANCH metric (the if/else
#   the per-branch, the line-only the hides)
pytest --cov=texttools --cov-branch
# the report: the PARTIAL (the the line covered, the
#   the branch 50% - the if yes, the else never)

# the the CONFIG (the the pyproject.toml, the the
#   the the versioned, the the bare --cov picks up):
# [tool.coverage.run]
# branch = true
# source = ["texttools"]
# omit = ["tests/*"]

The --cov-branch adds the branch measurement: the line covered but the branch not (the if only one side) is the partial (the 50% the branch), and the report shows the branch percent beside the line percent (the gap the line-only hides - the if executed but the else never). The config lives in the [tool.coverage.run] section of the pyproject.toml (the branch = true, the source = ["texttools"], the omit = ["tests/*"]) so the bare pytest --cov carries the project's settings (the no repeated flag, the versioned). The rule is the one: the --cov the measure, the --cov-report the format, the --cov-fail-under the CI's gate, the --cov-branch the deeper metric, and the config the versioned default - the coverage is the metric the suite reports, not the goal the test chases (the next chapters' reporting and plugins extend the same run to the output and the ecosystem).

The Working Rules

The --cov is the measure, the pytest-cov the plugin. The pytest --cov=texttools runs the suite under the coverage.py tracer, the report the per-file Stmts/Miss/Cover plus the total - the pip install pytest-cov the one install.
The --cov-report is the format, the multi-report the coexist. The term-missing the dev's gap list, the html the browser's, the xml the CI's, the json the tool's - the --cov-report=term-missing --cov-report=html the two in one run.
The --cov-fail-under is the CI's gate, the 80 the common floor. The total below exits the 1 (the suite green, the coverage red), the no regression below the floor, the threshold the versioned in the addopts or the CI's command.
The --cov-branch is the deeper metric, the config the versioned default. The branch percent beside the line (the if one-side the 50% the hidden), and the [tool.coverage.run] (the branch, the source, the omit) the bare pytest --cov picks up.

Summary

The pytest-cov plugin wraps the coverage.py, and the --cov flag runs the suite with the measurement (the pytest --cov=texttools the per-file Stmts/Miss/Cover plus the total). The --cov-report chooses the format (the term-missing the dev's gap list, the html the browser's, the xml the CI's, the json the tool's, the multiple the coexist), the --cov-fail-under=80 is the threshold (the total below exits the 1, the CI's gate), and the --cov-branch adds the branch metric (the line covered but the branch not the partial). The config lives in the [tool.coverage.run] (the branch, the source, the omit) so the bare pytest --cov carries the project's settings. The rule is the one: the coverage is the metric the suite reports, not the goal the test chases, and the next chapter's reporting is the output's surface (the -v, the --tb, the junitxml, the pytest-html) the same run produces.