Monday, July 11, 2011

Parameterized tests

There was a reddit discussion on assert vs. UnitTest-style assert_equal etc.

I really have no preference between assert x == y and assert_equal(x, y), given pytest's helpful tracebacks.

My problem with pytest is the awkward support for parameterized tests.  nose handles parameterized tests much better. (pytest handles the nose-style too, but that's hard to find in the docs.) This is an open issue for unittest2.
In a nutshell, when I test "purely functional" code (i.e. free of side-effects, referentially transparent) I want this:
  • To list inputs and correct outputs.
  • To apply each input to a specific function.
  • To consider each a separate testcase, so that they will all run even when one fails.
  • To learn which inputs failed, along with expected and actual results.
(ToDo: Write an example.)
GoogleTest (C++, not Python) has very good support for parameterized tests via TEST_P. However, GoogleTest does not allow TEST_P to be combined with TEST_F (parameterized within a fixture). That is something pytest allows, with a bit of work. 

No comments:

Post a Comment