Skillversitas

Reporting and Test Output

The report is what the run prints: the verbosity (-v, the -q), the traceback form (--tb=auto/line/no/short), the locals (-l), the short summary (-ra), and the machine formats (the --junitxml, the pytest-html report). The same run produces several at once, and the CI's log is the -v plus the -ra the team reads. This chapter covers the verbosity levels, the traceback forms, the -l locals, the -ra summary, and the junitxml/html the machine reports.

The Verbosity and the Traceback

# the -v: the per-item line (the name + the RESULT)
pytest -v
# tests/test_word.py::test_empty PASSED
# tests/test_word.py::test_single FAILED

# the -q: the one-line summary (the dots + the count)
pytest -q
# ....F.  6 passed, 1 failed in 0.03s

# the --tb: the TRACEBACK form (the failure's body)
pytest --tb=auto     # the default (the full, the
                     #   the long + the short)
pytest --tb=short    # the SHORT (the no locals, the
                     #   the compact - the CI's)
pytest --tb=line     # the ONE line (the file:line:
                     #   the message - the triage)
pytest --tb=no       # the NONE (the the count only)

# the -l: the LOCALS (the values at the frame -
#   the debug's, the the state the assert shows)
pytest -v -l --tb=long

The verbosity is the two flags: the -v adds the per-item line (the test_file.py::test_name PASSED - the form the CI log reads, the failure found by scroll) and the -q compresses to the one-line summary (the 6 passed, 1 failed in 0.03s). The --tb chooses the traceback form: the auto (the default, the full with the assertion diff), the short (the compact, the CI's default choice), the line (the one line per failure, the triage of the many), and the no (the count only). The -l prints the local variables at each frame (the text = 'a b' the value the test held), which is the debug's view when the assertion diff alone is not enough - the flags compose (the pytest -v -l --tb=short), and the next section's -ra plus the machine formats are the report's two halves (the human's, the machine's).

The -ra Summary and the Machine Reports

# the -ra: the SHORT SUMMARY (the per-outcome line
#   at the end of the run)
pytest -ra
# the SKIPPED [1] tests/test_x.py:12: the reason
# the XFAIL [2] tests/test_x.py:30: bug #42
# the FAILED tests/test_word.py::test_single

# the -rf: the FAILED only
# the -rA: the ALL outcomes

# the --junitxml: the XML (the CI's input)
pytest --junitxml=report/junit.xml
#   the <testsuite> the suite, the <testcase> the
#   per-item, the <failure> the traceback - the
#   pipeline parses it (the GitHub Actions, the
#   the Jenkins, the badge)

# the pytest-html: the HTML (the browser's)
#   the pip install pytest-html first
pytest --html=report/report.html --self-contained-html
#   the single file, the no assets, the share

The -ra is the short summary: the per-outcome line (the SKIPPED [1] file:line: the reason, the XFAIL [2] ... bug #42, the FAILED ...::test_single) at the end of the run, so the skip and the xfail are visible without the -v scroll. The -rf is the failed-only, the -rA the all. The machine formats are the two: the --junitxml=report/junit.xml is the XML (the <testsuite> the suite, the <testcase> the item, the <failure> the traceback) that the CI reads (the GitHub Actions, the Jenkins, the badge), and the pytest-html plugin's --html=report/report.html --self-contained-html is the single-file HTML (the browser's, the share) - the pytest.ini can carry the addopts = -ra --junitxml=report/junit.xml so the bare pytest already emits both, and the next chapter's plugins are the pytest-html and friends the ecosystem adds.

The Working Rules

  • The -v is the per-item line, the -q the one-line summary. The test_file.py::test_name PASSED the CI log reads, the dots the quick check - the -v the commit gate, the -q the dev's loop.
  • The --tb is the failure's body, the short the CI's. The auto the full (the assertion diff), the short the compact, the line the triage's one-liner, the no the count only - and the -l the locals at the frame the debug opens.
  • The -ra is the per-outcome summary, the skip/xfail's visibility. The SKIPPED [1] file:line: the reason the tracked, the -rf the failed-only, the -rA the all - the addopts = -ra the project default.
  • The --junitxml the CI's, the pytest-html the browser's. The XML the <testcase>/<failure> the pipeline parses, the self-contained HTML the share - the addopts the both, the bare pytest the emit.

Summary

The report is the run's output: the -v the per-item line (the test_file.py::test_name PASSED the CI log), the -q the one-line summary, the --tb the traceback form (the auto the full, the short the CI's, the line the triage, the no the count) and the -l the locals at the frame. The -ra is the per-outcome short summary (the SKIPPED [1] file:line: the reason the visible, the -rf/-rA the variants), and the machine formats are the --junitxml (the <testcase>/<failure> the CI parses) plus the pytest-html (the self-contained HTML the browser). The addopts in the pyproject.toml carries the repeated flags so the bare pytest emits the right report, and the next chapter's plugins are the ecosystem the pytest-html belongs to - the pip install pytest-* the extend the same run.