Why Pytest

test
develop

Testing software is a requirement for ensuring it works as intended and because we use a test-driven development workflow. This post explains why we choose to use pytest over the built-in unittest framework for testing Python code.

Published

July 17, 2024

Context and problem statement

Testing software is a basic requirement for assessing its functionality, accuracy, and reliability. There are many categories of software testing, including integration testing, unit testing, or system testing. This post is focused on unit testing.

Since one of our development principles is to use a test-driven development workflow, we need a testing framework when building our applications. Since we are using Python and Django, we need a testing framework that works for both. The question then is:

Which testing frameworks are available and which one should we use?

Decision drivers

  • Knowing that software works as intended requires some form of testing.
  • Developing software is easier when we can compare the expected output with the actual output in a formal testing approach.
  • Identifying bugs is easier when there are tests that check the expected functionality of our code.
  • Identifying breaking changes is also easier when there are tests.

Considered options

Django’s default unit testing framework is unittest, which is also Python’s built-in testing framework. Aside from the default unittest, there is really only one other alternative for unit testing:

unittest

unittest is the built-in framework in Python. It was inspired by JUnit (the unit testing framework in Java) and has a similar structure to it.

Benefits

  • Comes by default with Django and Python.
  • Documentation on the official Django website uses unittest.

Drawbacks

  • Requires more extra code to set up and structure tests. For instance, all tests need to be a class object and some testing needs setUp() and tearDown() methods.
  • The test output is not very readable and beginner friendly.
  • Creating tests takes more effort and time compared to alternatives.
  • Writing tests follows more of an object-oriented programming style, which can be harder to reason about and design tests for (for multiple, highly technical reasons outside the scope of this post)

pytest

pytest, along with the Django plugin, is a newer testing framework that was built to improve on the limitations of the default unittest framework.

Benefits

  • Integrates with unittest style tests, so you don’t need to refactor all existing tests.
  • Has easier to read output of the test results.
  • Uses a simpler syntax for writing tests, with a simple assert statement, as compared to unittest’s assert statements, e.g., assertEqual and assertTrue.
  • Created and developed more recently and from scratch, so is better designed than unittest for Python (since unittest was modeled off of Java’s framework).
  • While it isn’t the default unit testing framework in Python, it is widely used.

Drawbacks

  • Requires installing two additional packages: pytest and pytest-django.
  • Integrating pytest with other types of testing frameworks can be a bit difficult and may require some configuration.

Decision outcome

We decided on using pytest because it is more developer friendly, has easier to read output, and requires less code to write tests (e.g., tests don’t need to be within a class object and the shorter assert syntax) still run unittest style tests, we don’t need to refactor all existing tests right away.

Consequences

  • We will have two additional package dependencies: pytest and pytest-django.
  • We will need to learn the pytest syntax and structure for writing tests.

Resources used for this post