1 :mod:`unittest` --- Unit testing framework
2 ==========================================
5 :synopsis: Unit testing framework for Python.
6 .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
7 .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
8 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
9 .. sectionauthor:: Raymond Hettinger <python@rcn.com>
14 .. versionchanged:: 2.7
15 Added test :ref:`skipping and expected failures <unittest-skipping>`.
17 The Python unit testing framework, sometimes referred to as "PyUnit," is a
18 Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
19 turn, a Java version of Kent's Smalltalk testing framework. Each is the de
20 facto standard unit testing framework for its respective language.
22 :mod:`unittest` supports test automation, sharing of setup and shutdown code for
23 tests, aggregation of tests into collections, and independence of the tests from
24 the reporting framework. The :mod:`unittest` module provides classes that make
25 it easy to support these qualities for a set of tests.
27 To achieve this, :mod:`unittest` supports some important concepts:
30 A :dfn:`test fixture` represents the preparation needed to perform one or more
31 tests, and any associate cleanup actions. This may involve, for example,
32 creating temporary or proxy databases, directories, or starting a server
36 A :dfn:`test case` is the smallest unit of testing. It checks for a specific
37 response to a particular set of inputs. :mod:`unittest` provides a base class,
38 :class:`TestCase`, which may be used to create new test cases.
41 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
42 used to aggregate tests that should be executed together.
45 A :dfn:`test runner` is a component which orchestrates the execution of tests
46 and provides the outcome to the user. The runner may use a graphical interface,
47 a textual interface, or return a special value to indicate the results of
50 The test case and test fixture concepts are supported through the
51 :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
52 used when creating new tests, and the latter can be used when integrating
53 existing test code with a :mod:`unittest`\ -driven framework. When building test
54 fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
55 :meth:`~TestCase.tearDown` methods can be overridden to provide initialization
56 and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions
57 can be passed to the constructor for these purposes. When the test is run, the
58 fixture initialization is run first; if it succeeds, the cleanup method is run
59 after the test has been executed, regardless of the outcome of the test. Each
60 instance of the :class:`TestCase` will only be used to run a single test method,
61 so a new fixture is created for each test.
63 Test suites are implemented by the :class:`TestSuite` class. This class allows
64 individual tests and test suites to be aggregated; when the suite is executed,
65 all tests added directly to the suite and in "child" test suites are run. A
66 :class:`ClassTestSuite` contains the test cases of a class.
68 A test runner is an object that provides a single method,
69 :meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
70 object as a parameter, and returns a result object. The class
71 :class:`TestResult` is provided for use as the result object. :mod:`unittest`
72 provides the :class:`TextTestRunner` as an example test runner which reports
73 test results on the standard error stream by default. Alternate runners can be
74 implemented for other environments (such as graphical environments) without any
75 need to derive from a specific class.
81 Another test-support module with a very different flavor.
83 `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
84 Kent Beck's original paper on testing frameworks using the pattern shared by
87 `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_
88 Third-party unittest frameworks with a lighter-weight syntax
89 for writing tests. For example, ``assert func(10) == 42``.
91 `python-mock <http://python-mock.sourceforge.net/>`_ and `minimock <http://blog.ianbicking.org/minimock.html>`_
92 Tools for creating mock test objects (objects simulating external resources).
94 .. _unittest-minimal-example:
99 The :mod:`unittest` module provides a rich set of tools for constructing and
100 running tests. This section demonstrates that a small subset of the tools
101 suffice to meet the needs of most users.
103 Here is a short script to test three functions from the :mod:`random` module::
108 class TestSequenceFunctions(unittest.TestCase):
113 def test_shuffle(self):
114 # make sure the shuffled sequence does not lose any elements
115 random.shuffle(self.seq)
117 self.assertEqual(self.seq, range(10))
119 def test_choice(self):
120 element = random.choice(self.seq)
121 self.assert_(element in self.seq)
123 def test_sample(self):
124 self.assertRaises(ValueError, random.sample, self.seq, 20)
125 for element in random.sample(self.seq, 5):
126 self.assert_(element in self.seq)
128 if __name__ == '__main__':
131 A testcase is created by subclassing :class:`unittest.TestCase`. The three
132 individual tests are defined with methods whose names start with the letters
133 ``test``. This naming convention informs the test runner about which methods
136 The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
137 expected result; :meth:`~TestCase.assert_` to verify a condition; or
138 :meth:`~TestCase.assertRaises` to verify that an expected exception gets raised.
139 These methods are used instead of the :keyword:`assert` statement so the test
140 runner can accumulate all test results and produce a report.
142 When a :meth:`~TestCase.setUp` method is defined, the test runner will run that
143 method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is
144 defined, the test runner will invoke that method after each test. In the
145 example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each
148 The final block shows a simple way to run the tests. :func:`unittest.main`
149 provides a command line interface to the test script. When run from the command
150 line, the above script produces an output that looks like this::
153 ----------------------------------------------------------------------
154 Ran 3 tests in 0.000s
158 Instead of :func:`unittest.main`, there are other ways to run the tests with a
159 finer level of control, less terse output, and no requirement to be run from the
160 command line. For example, the last two lines may be replaced with::
162 suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
163 unittest.TextTestRunner(verbosity=2).run(suite)
165 Running the revised script from the interpreter or another script produces the
168 testchoice (__main__.TestSequenceFunctions) ... ok
169 testsample (__main__.TestSequenceFunctions) ... ok
170 testshuffle (__main__.TestSequenceFunctions) ... ok
172 ----------------------------------------------------------------------
173 Ran 3 tests in 0.110s
177 The above examples show the most commonly used :mod:`unittest` features which
178 are sufficient to meet many everyday testing needs. The remainder of the
179 documentation explores the full feature set from first principles.
182 .. _organizing-tests:
187 The basic building blocks of unit testing are :dfn:`test cases` --- single
188 scenarios that must be set up and checked for correctness. In :mod:`unittest`,
189 test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
190 class. To make your own test cases you must write subclasses of
191 :class:`TestCase`, or use :class:`FunctionTestCase`.
193 An instance of a :class:`TestCase`\ -derived class is an object that can
194 completely run a single test method, together with optional set-up and tidy-up
197 The testing code of a :class:`TestCase` instance should be entirely self
198 contained, such that it can be run either in isolation or in arbitrary
199 combination with any number of other test cases.
201 The simplest :class:`TestCase` subclass will simply override the
202 :meth:`~TestCase.runTest` method in order to perform specific testing code::
206 class DefaultWidgetSizeTestCase(unittest.TestCase):
208 widget = Widget('The widget')
209 self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
211 Note that in order to test something, we use the one of the :meth:`assert\*`
212 methods provided by the :class:`TestCase` base class. If the
213 test fails, an exception will be raised, and :mod:`unittest` will identify the
214 test case as a :dfn:`failure`. Any other exceptions will be treated as
215 :dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are
216 caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are
217 caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect
220 The way to run a test case will be described later. For now, note that to
221 construct an instance of such a test case, we call its constructor without
224 testCase = DefaultWidgetSizeTestCase()
226 Now, such test cases can be numerous, and their set-up can be repetitive. In
227 the above case, constructing a :class:`Widget` in each of 100 Widget test case
228 subclasses would mean unsightly duplication.
230 Luckily, we can factor out such set-up code by implementing a method called
231 :meth:`~TestCase.setUp`, which the testing framework will automatically call for
232 us when we run the test::
236 class SimpleWidgetTestCase(unittest.TestCase):
238 self.widget = Widget('The widget')
240 class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
242 self.assertTrue(self.widget.size() == (50,50),
243 'incorrect default size')
245 class WidgetResizeTestCase(SimpleWidgetTestCase):
247 self.widget.resize(100,150)
248 self.assertTrue(self.widget.size() == (100,150),
249 'wrong size after resize')
251 If the :meth:`~TestCase.setUp` method raises an exception while the test is
252 running, the framework will consider the test to have suffered an error, and the
253 :meth:`~TestCase.runTest` method will not be executed.
255 Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
256 after the :meth:`~TestCase.runTest` method has been run::
260 class SimpleWidgetTestCase(unittest.TestCase):
262 self.widget = Widget('The widget')
265 self.widget.dispose()
268 If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
269 be run whether :meth:`~TestCase.runTest` succeeded or not.
271 Such a working environment for the testing code is called a :dfn:`fixture`.
273 Often, many small test cases will use the same fixture. In this case, we would
274 end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
275 classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
276 discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
281 class WidgetTestCase(unittest.TestCase):
283 self.widget = Widget('The widget')
286 self.widget.dispose()
289 def testDefaultSize(self):
290 self.assertTrue(self.widget.size() == (50,50),
291 'incorrect default size')
293 def testResize(self):
294 self.widget.resize(100,150)
295 self.assertTrue(self.widget.size() == (100,150),
296 'wrong size after resize')
298 Here we have not provided a :meth:`~TestCase.runTest` method, but have instead
299 provided two different test methods. Class instances will now each run one of
300 the :meth:`test\*` methods, with ``self.widget`` created and destroyed
301 separately for each instance. When creating an instance we must specify the
302 test method it is to run. We do this by passing the method name in the
305 defaultSizeTestCase = WidgetTestCase('testDefaultSize')
306 resizeTestCase = WidgetTestCase('testResize')
308 Test case instances are grouped together according to the features they test.
309 :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
310 represented by :mod:`unittest`'s :class:`TestSuite` class::
312 widgetTestSuite = unittest.TestSuite()
313 widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))
314 widgetTestSuite.addTest(WidgetTestCase('testResize'))
316 For the ease of running tests, as we will see later, it is a good idea to
317 provide in each test module a callable object that returns a pre-built test
321 suite = unittest.TestSuite()
322 suite.addTest(WidgetTestCase('testDefaultSize'))
323 suite.addTest(WidgetTestCase('testResize'))
329 tests = ['testDefaultSize', 'testResize']
331 return unittest.TestSuite(map(WidgetTestCase, tests))
333 Since it is a common pattern to create a :class:`TestCase` subclass with many
334 similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
335 class that can be used to automate the process of creating a test suite and
336 populating it with individual tests. For example, ::
338 suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
340 will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and
341 ``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method
342 name prefix to identify test methods automatically.
344 Note that the order in which the various test cases will be run is determined by
345 sorting the test function names with the built-in :func:`cmp` function.
347 Often it is desirable to group suites of test cases together, so as to run tests
348 for the whole system at once. This is easy, since :class:`TestSuite` instances
349 can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
350 added to a :class:`TestSuite`::
352 suite1 = module1.TheTestSuite()
353 suite2 = module2.TheTestSuite()
354 alltests = unittest.TestSuite([suite1, suite2])
356 You can place the definitions of test cases and test suites in the same modules
357 as the code they are to test (such as :file:`widget.py`), but there are several
358 advantages to placing the test code in a separate module, such as
359 :file:`test_widget.py`:
361 * The test module can be run standalone from the command line.
363 * The test code can more easily be separated from shipped code.
365 * There is less temptation to change test code to fit the code it tests without
368 * Test code should be modified much less frequently than the code it tests.
370 * Tested code can be refactored more easily.
372 * Tests for modules written in C must be in separate modules anyway, so why not
375 * If the testing strategy changes, there is no need to change the source code.
378 .. _legacy-unit-tests:
380 Re-using old test code
381 ----------------------
383 Some users will find that they have existing test code that they would like to
384 run from :mod:`unittest`, without converting every old test function to a
385 :class:`TestCase` subclass.
387 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
388 This subclass of :class:`TestCase` can be used to wrap an existing test
389 function. Set-up and tear-down functions can also be provided.
391 Given the following test function::
394 something = makeSomething()
395 assert something.name is not None
398 one can create an equivalent test case instance as follows::
400 testcase = unittest.FunctionTestCase(testSomething)
402 If there are additional set-up and tear-down methods that should be called as
403 part of the test case's operation, they can also be provided like so::
405 testcase = unittest.FunctionTestCase(testSomething,
406 setUp=makeSomethingDB,
407 tearDown=deleteSomethingDB)
409 To make migrating existing test suites easier, :mod:`unittest` supports tests
410 raising :exc:`AssertionError` to indicate test failure. However, it is
411 recommended that you use the explicit :meth:`TestCase.fail\*` and
412 :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
413 may treat :exc:`AssertionError` differently.
417 Even though :class:`FunctionTestCase` can be used to quickly convert an existing
418 test base over to a :mod:`unittest`\ -based system, this approach is not
419 recommended. Taking the time to set up proper :class:`TestCase` subclasses will
420 make future test refactorings infinitely easier.
422 In some cases, the existing tests may have been written using the :mod:`doctest`
423 module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
424 automatically build :class:`unittest.TestSuite` instances from the existing
425 :mod:`doctest`\ -based tests.
428 .. _unittest-skipping:
430 Skipping tests and expected failures
431 ------------------------------------
433 Unittest supports skipping individual test methods and even whole classes of
434 tests. In addition, it supports marking a test as a "expected failure," a test
435 that is broken and will fail, but shouldn't be counted as a failure on a
438 Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
439 or one of its conditional variants.
441 Basic skipping looks like this: ::
443 class MyTestCase(unittest.TestCase):
445 @unittest.skip("demonstrating skipping")
446 def test_nothing(self):
447 self.fail("shouldn't happen")
449 @unittest.skipIf(mylib.__version__ < (1, 3), "not supported in this library version")
450 def test_format(self):
451 # Tests that work for only a certain version of the library.
454 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
455 def test_windows_support(self):
456 # windows specific testing code
459 This is the output of running the example above in verbose mode: ::
461 test_format (__main__.MyTestCase) ... skipped 'not supported in this library version'
462 test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
463 test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'
465 ----------------------------------------------------------------------
466 Ran 3 tests in 0.005s
470 Classes can be skipped just like methods: ::
472 @skip("showing class skipping")
473 class MySkippedTestCase(unittest.TestCase):
474 def test_not_run(self):
477 :meth:`TestCase.setUp` can also skip the test. This is useful when a resource
478 that needs to be set up is not available.
480 Expected failures use the :func:`expectedFailure` decorator. ::
482 class ExpectedFailureTestCase(unittest.TestCase):
483 @unittest.expectedFailure
485 self.assertEqual(1, 0, "broken")
487 It's easy to roll your own skipping decorators by making a decorator that calls
488 :func:`skip` on the test when it wants it to be skipped. This decorator skips
489 the test unless the passed object has a certain attribute: ::
491 def skipUnlessHasattr(obj, attr):
492 if hasattr(obj, attr):
493 return lambda func: func
494 return unittest.skip("{0!r} doesn't have {1!r}".format(obj, attr))
496 The following decorators implement test skipping and expected failures:
498 .. function:: skip(reason)
500 Unconditionally skip the decorated test. *reason* should describe why the
501 test is being skipped.
503 .. function:: skipIf(condition, reason)
505 Skip the decorated test if *condition* is true.
507 .. function:: skipUnless(condition, reason)
509 Skip the decoratored test unless *condition* is true.
511 .. function:: expectedFailure
513 Mark the test as an expected failure. If the test fails when run, the test
514 is not counted as a failure.
517 .. _unittest-contents:
519 Classes and functions
520 ---------------------
522 This section describes in depth the API of :mod:`unittest`.
525 .. _testcase-objects:
530 .. class:: TestCase([methodName])
532 Instances of the :class:`TestCase` class represent the smallest testable units
533 in the :mod:`unittest` universe. This class is intended to be used as a base
534 class, with specific tests being implemented by concrete subclasses. This class
535 implements the interface needed by the test runner to allow it to drive the
536 test, and methods that the test code can use to check for and report various
539 Each instance of :class:`TestCase` will run a single test method: the method
540 named *methodName*. If you remember, we had an earlier example that went
541 something like this::
544 suite = unittest.TestSuite()
545 suite.addTest(WidgetTestCase('testDefaultSize'))
546 suite.addTest(WidgetTestCase('testResize'))
549 Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
552 *methodName* defaults to :meth:`runTest`.
554 :class:`TestCase` instances provide three groups of methods: one group used
555 to run the test, another used by the test implementation to check conditions
556 and report failures, and some inquiry methods allowing information about the
557 test itself to be gathered.
559 Methods in the first group (running the test) are:
564 Method called to prepare the test fixture. This is called immediately
565 before calling the test method; any exception raised by this method will
566 be considered an error rather than a test failure. The default
567 implementation does nothing.
570 .. method:: tearDown()
572 Method called immediately after the test method has been called and the
573 result recorded. This is called even if the test method raised an
574 exception, so the implementation in subclasses may need to be particularly
575 careful about checking internal state. Any exception raised by this
576 method will be considered an error rather than a test failure. This
577 method will only be called if the :meth:`setUp` succeeds, regardless of
578 the outcome of the test method. The default implementation does nothing.
581 .. method:: run([result])
583 Run the test, collecting the result into the test result object passed as
584 *result*. If *result* is omitted or :const:`None`, a temporary result
585 object is created (by calling the :meth:`defaultTestCase` method) and
586 used; this result object is not returned to :meth:`run`'s caller.
588 The same effect may be had by simply calling the :class:`TestCase`
592 .. method:: skipTest(reason)
594 Calling this during the a test method or :meth:`setUp` skips the current
595 test. See :ref:`unittest-skipping` for more information.
600 Run the test without collecting the result. This allows exceptions raised
601 by the test to be propagated to the caller, and can be used to support
602 running tests under a debugger.
604 The test code can use any of the following methods to check for and report
608 .. method:: assertTrue(expr[, msg])
610 failUnless(expr[, msg])
612 Signal a test failure if *expr* is false; the explanation for the error
613 will be *msg* if given, otherwise it will be :const:`None`.
619 .. method:: assertEqual(first, second[, msg])
620 failUnlessEqual(first, second[, msg])
622 Test that *first* and *second* are equal. If the values do not compare
623 equal, the test will fail with the explanation given by *msg*, or
624 :const:`None`. Note that using :meth:`assertEqual` improves upon
625 doing the comparison as the first parameter to :meth:`assertTrue`: the
626 default value for *msg* include representations of both *first* and
629 In addition, if *first* and *second* are the exact same type and one of
630 list, tuple, dict, set, or frozenset or any type that a subclass
631 registers :meth:`addTypeEqualityFunc` the type specific equality function
632 will be called in order to generate a more useful default error message.
634 .. versionchanged:: 2.7
635 Added the automatic calling of type specific equality function.
638 :meth:`failUnlessEqual`.
641 .. method:: assertNotEqual(first, second[, msg])
642 failIfEqual(first, second[, msg])
644 Test that *first* and *second* are not equal. If the values do compare
645 equal, the test will fail with the explanation given by *msg*, or
646 :const:`None`. Note that using :meth:`assertNotEqual` improves upon doing
647 the comparison as the first parameter to :meth:`assertTrue` is that the
648 default value for *msg* can be computed to include representations of both
649 *first* and *second*.
655 .. method:: assertAlmostEqual(first, second[, places[, msg]])
656 failUnlessAlmostEqual(first, second[, places[, msg]])
658 Test that *first* and *second* are approximately equal by computing the
659 difference, rounding to the given number of decimal *places* (default 7),
660 and comparing to zero.
662 Note that comparing a given number of decimal places is not the same as
663 comparing a given number of significant digits. If the values do not
664 compare equal, the test will fail with the explanation given by *msg*, or
668 :meth:`failUnlessAlmostEqual`.
671 .. method:: assertNotAlmostEqual(first, second[, places[, msg]])
672 failIfAlmostEqual(first, second[, places[, msg]])
674 Test that *first* and *second* are not approximately equal by computing
675 the difference, rounding to the given number of decimal *places* (default
676 7), and comparing to zero.
678 Note that comparing a given number of decimal places is not the same as
679 comparing a given number of significant digits. If the values do not
680 compare equal, the test will fail with the explanation given by *msg*, or
684 :meth:`failIfAlmostEqual`.
687 .. method:: assertGreater(first, second, msg=None)
688 assertGreaterEqual(first, second, msg=None)
689 assertLess(first, second, msg=None)
690 assertLessEqual(first, second, msg=None)
692 Test that *first* is respectively >, >=, < or <= than *second* depending
693 on the method name. If not, the test will fail with an explanation
694 or with the explanation given by *msg*::
696 >>> self.assertGreaterEqual(3, 4)
697 AssertionError: "3" unexpectedly not greater than or equal to "4"
699 .. versionadded:: 2.7
702 .. method:: assertMultiLineEqual(self, first, second, msg=None)
704 Test that the multiline string *first* is equal to the string *second*.
705 When not equal a diff of the two strings highlighting the differences
706 will be included in the error message.
708 If specified *msg* will be used as the error message on failure.
710 .. versionadded:: 2.7
713 .. method:: assertRegexpMatches(text, regexp[, msg=None]):
715 Verifies that a *regexp* search matches *text*. Fails with an error
716 message including the pattern and the *text*. *regexp* may be
717 a regular expression object or a string containing a regular expression
718 suitable for use by :func:`re.search`.
720 .. versionadded:: 2.7
723 .. method:: assertIn(first, second, msg=None)
724 assertNotIn(first, second, msg=None)
726 Tests that *first* is or is not in *second* with an explanatory error
727 message as appropriate.
729 If specified *msg* will be used as the error message on failure.
731 .. versionadded:: 2.7
734 .. method:: assertSameElements(expected, actual, msg=None)
736 Test that sequence *expected* contains the same elements as *actual*.
737 When they don't an error message listing the differences between the
738 sequences will be generated.
740 If specified *msg* will be used as the error message on failure.
742 .. versionadded:: 2.7
745 .. method:: assertSetEqual(set1, set2, msg=None)
747 Tests that two sets are equal. If not, an error message is constructed
748 that lists the differences between the sets.
750 Fails if either of *set1* or *set2* does not have a :meth:`set.difference`
753 If specified *msg* will be used as the error message on failure.
755 .. versionadded:: 2.7
758 .. method:: assertDictEqual(expected, actual, msg=None)
760 Test that two dictionaries are equal. If not, an error message is
761 constructed that shows the differences in the dictionaries.
763 If specified *msg* will be used as the error message on failure.
765 .. versionadded:: 2.7
768 .. method:: assertDictContainsSubset(expected, actual, msg=None)
770 Tests whether the key/value pairs in dictionary *actual* are a
771 superset of those in *expected*. If not, an error message listing
772 the missing keys and mismatched values is generated.
774 If specified *msg* will be used as the error message on failure.
776 .. versionadded:: 2.7
779 .. method:: assertListEqual(list1, list2, msg=None)
780 assertTupleEqual(tuple1, tuple2, msg=None)
782 Tests that two lists or tuples are equal. If not an error message is
783 constructed that shows only the differences between the two. An error
784 is also raised if either of the parameters are of the wrong type.
786 If specified *msg* will be used as the error message on failure.
788 .. versionadded:: 2.7
791 .. method:: assertSequenceEqual(seq1, seq2, msg=None, seq_type=None)
793 Tests that two sequences are equal. If a *seq_type* is supplied, both
794 *seq1* and *seq2* must be instances of *seq_type* or a failure will
795 be raised. If the sequences are different an error message is
796 constructed that shows the difference between the two.
798 If specified *msg* will be used as the error message on failure.
800 This method is used to implement :meth:`assertListEqual` and
801 :meth:`assertTupleEqual`.
803 .. versionadded:: 2.7
806 .. method:: assertRaises(exception[, callable, ...])
807 failUnlessRaises(exception[, callable, ...])
809 Test that an exception is raised when *callable* is called with any
810 positional or keyword arguments that are also passed to
811 :meth:`assertRaises`. The test passes if *exception* is raised, is an
812 error if another exception is raised, or fails if no exception is raised.
813 To catch any of a group of exceptions, a tuple containing the exception
814 classes may be passed as *exception*.
816 If *callable* is omitted or None, returns a context manager so that the
817 code under test can be written inline rather than as a function::
819 with self.failUnlessRaises(some_error_class):
822 .. versionchanged:: 2.7
823 Added the ability to use :meth:`assertRaises` as a context manager.
826 :meth:`failUnlessRaises`.
829 .. method:: assertRaisesRegexp(exception, regexp[, callable, ...])
831 Like :meth:`assertRaises` but also tests that *regexp* matches
832 on the string representation of the raised exception. *regexp* may be
833 a regular expression object or a string containing a regular expression
834 suitable for use by :func:`re.search`. Examples::
836 self.assertRaisesRegexp(ValueError, 'invalid literal for.*XYZ$',
841 with self.assertRaisesRegexp(ValueError, 'literal'):
844 .. versionadded:: 2.7
847 .. method:: assertIsNone(expr[, msg])
849 This signals a test failure if *expr* is not None.
851 .. versionadded:: 2.7
854 .. method:: assertIsNotNone(expr[, msg])
856 The inverse of the :meth:`assertIsNone` method.
857 This signals a test failure if *expr* is None.
859 .. versionadded:: 2.7
862 .. method:: assertIs(expr1, expr2[, msg])
864 This signals a test failure if *expr1* and *expr2* don't evaluate to the same
867 .. versionadded:: 2.7
870 .. method:: assertIsNot(expr1, expr2[, msg])
872 The inverse of the :meth:`assertIs` method.
873 This signals a test failure if *expr1* and *expr2* evaluate to the same
876 .. versionadded:: 2.7
879 .. method:: assertFalse(expr[, msg])
882 The inverse of the :meth:`assertTrue` method is the :meth:`assertFalse` method.
883 This signals a test failure if *expr* is true, with *msg* or :const:`None`
884 for the error message.
890 .. method:: fail([msg])
892 Signals a test failure unconditionally, with *msg* or :const:`None` for
896 .. attribute:: failureException
898 This class attribute gives the exception raised by the test method. If a
899 test framework needs to use a specialized exception, possibly to carry
900 additional information, it must subclass this exception in order to "play
901 fair" with the framework. The initial value of this attribute is
902 :exc:`AssertionError`.
905 .. attribute:: longMessage
907 If set to True then any explicit failure message you pass in to the
908 assert methods will be appended to the end of the normal failure message.
909 The normal messages contain useful information about the objects involved,
910 for example the message from assertEqual shows you the repr of the two
911 unequal objects. Setting this attribute to True allows you to have a
912 custom error message in addition to the normal one.
914 This attribute defaults to False, meaning that a custom message passed
915 to an assert method will silence the normal message.
917 The class setting can be overridden in individual tests by assigning an
918 instance attribute to True or False before calling the assert methods.
920 .. versionadded:: 2.7
923 Testing frameworks can use the following methods to collect information on
927 .. method:: countTestCases()
929 Return the number of tests represented by this test object. For
930 :class:`TestCase` instances, this will always be ``1``.
933 .. method:: defaultTestResult()
935 Return an instance of the test result class that should be used for this
936 test case class (if no other result instance is provided to the
939 For :class:`TestCase` instances, this will always be an instance of
940 :class:`TestResult`; subclasses of :class:`TestCase` should override this
946 Return a string identifying the specific test case. This is usually the
947 full name of the test method, including the module and class name.
950 .. method:: shortDescription()
952 Returns a description of the test, or :const:`None` if no description
953 has been provided. The default implementation of this method
954 returns the first line of the test method's docstring, if available,
955 along with the method name.
957 .. versionchanged:: 2.7
959 In earlier versions this only returned the first line of the test
960 method's docstring, if available or the :const:`None`. That led to
961 undesirable behavior of not printing the test name when someone was
962 thoughtful enough to write a docstring.
965 .. method:: addTypeEqualityFunc(typeobj, function)
967 Registers a type specific :meth:`assertEqual` equality checking
968 function to be called by :meth:`assertEqual` when both objects it has
969 been asked to compare are exactly *typeobj* (not subclasses).
970 *function* must take two positional arguments and a third msg=None
971 keyword argument just as :meth:`assertEqual` does. It must raise
972 ``self.failureException`` when inequality between the first two
973 parameters is detected.
975 One good use of custom equality checking functions for a type
976 is to raise ``self.failureException`` with an error message useful
977 for debugging the problem by explaining the inequalities in detail.
979 .. versionadded:: 2.7
982 .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
984 This class implements the portion of the :class:`TestCase` interface which
985 allows the test runner to drive the test, but does not provide the methods which
986 test code can use to check and report errors. This is used to create test cases
987 using legacy test code, allowing it to be integrated into a :mod:`unittest`\
988 -based test framework.
991 .. _testsuite-objects:
996 .. class:: TestSuite([tests])
998 This class represents an aggregation of individual tests cases and test suites.
999 The class presents the interface needed by the test runner to allow it to be run
1000 as any other test case. Running a :class:`TestSuite` instance is the same as
1001 iterating over the suite, running each test individually.
1003 If *tests* is given, it must be an iterable of individual test cases or other
1004 test suites that will be used to build the suite initially. Additional methods
1005 are provided to add test cases and suites to the collection later on.
1007 :class:`TestSuite` (including :class:`ClassTestSuite`) objects behave much
1008 like :class:`TestCase` objects, except they do not actually implement a test.
1009 Instead, they are used to aggregate tests into groups of tests that should be
1010 run together. Some additional methods are available to add tests to
1011 :class:`TestSuite` instances:
1014 .. method:: TestSuite.addTest(test)
1016 Add a :class:`TestCase` or :class:`TestSuite` to the suite.
1019 .. method:: TestSuite.addTests(tests)
1021 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
1022 instances to this test suite.
1024 This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
1027 :class:`TestSuite` shares the following methods with :class:`TestCase`:
1030 .. method:: run(result)
1032 Run the tests associated with this suite, collecting the result into the
1033 test result object passed as *result*. Note that unlike
1034 :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
1040 Run the tests associated with this suite without collecting the
1041 result. This allows exceptions raised by the test to be propagated to the
1042 caller and can be used to support running tests under a debugger.
1045 .. method:: countTestCases()
1047 Return the number of tests represented by this test object, including all
1048 individual tests and sub-suites.
1050 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
1051 is invoked by a :class:`TestRunner` rather than by the end-user test harness.
1054 .. class:: ClassTestSuite(tests, collected_from)
1056 This subclass of :class:`TestSuite` repesents an aggregation of individuals
1057 tests from one :class:`TestCase` class. *tests* is an iterable of
1058 :class:`TestCase` instances created from the class. *collected_from* is the
1059 class they came from.
1062 Loading and running tests
1063 ~~~~~~~~~~~~~~~~~~~~~~~~~
1065 .. class:: TestLoader()
1067 The :class:`TestLoader` class is used to create test suites from classes and
1068 modules. Normally, there is no need to create an instance of this class; the
1069 :mod:`unittest` module provides an instance that can be shared as
1070 ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
1071 customization of some configurable properties.
1073 :class:`TestLoader` objects have the following methods:
1076 .. method:: loadTestsFromTestCase(testCaseClass)
1078 Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
1079 :class:`testCaseClass`.
1082 .. method:: loadTestsFromModule(module)
1084 Return a suite of all tests cases contained in the given module. This
1085 method searches *module* for classes derived from :class:`TestCase` and
1086 creates an instance of the class for each test method defined for the
1091 While using a hierarchy of :class:`TestCase`\ -derived classes can be
1092 convenient in sharing fixtures and helper functions, defining test
1093 methods on base classes that are not intended to be instantiated
1094 directly does not play well with this method. Doing so, however, can
1095 be useful when the fixtures are different and defined in subclasses.
1098 .. method:: loadTestsFromName(name[, module])
1100 Return a suite of all tests cases given a string specifier.
1102 The specifier *name* is a "dotted name" that may resolve either to a
1103 module, a test case class, a test method within a test case class, a
1104 :class:`TestSuite` instance, or a callable object which returns a
1105 :class:`TestCase` or :class:`TestSuite` instance. These checks are
1106 applied in the order listed here; that is, a method on a possible test
1107 case class will be picked up as "a test method within a test case class",
1108 rather than "a callable object".
1110 For example, if you have a module :mod:`SampleTests` containing a
1111 :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
1112 methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
1113 specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
1114 suite which will run all three test methods. Using the specifier
1115 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
1116 which will run only the :meth:`test_two` test method. The specifier can refer
1117 to modules and packages which have not been imported; they will be imported as a
1120 The method optionally resolves *name* relative to the given *module*.
1123 .. method:: loadTestsFromNames(names[, module])
1125 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
1126 than a single name. The return value is a test suite which supports all
1127 the tests defined for each name.
1130 .. method:: getTestCaseNames(testCaseClass)
1132 Return a sorted sequence of method names found within *testCaseClass*;
1133 this should be a subclass of :class:`TestCase`.
1135 The following attributes of a :class:`TestLoader` can be configured either by
1136 subclassing or assignment on an instance:
1139 .. attribute:: testMethodPrefix
1141 String giving the prefix of method names which will be interpreted as test
1142 methods. The default value is ``'test'``.
1144 This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
1148 .. attribute:: sortTestMethodsUsing
1150 Function to be used to compare method names when sorting them in
1151 :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
1152 default value is the built-in :func:`cmp` function; the attribute can also
1153 be set to :const:`None` to disable the sort.
1156 .. attribute:: suiteClass
1158 Callable object that constructs a test suite from a list of tests. No
1159 methods on the resulting object are needed. The default value is the
1160 :class:`TestSuite` class.
1162 This affects all the :meth:`loadTestsFrom\*` methods.
1165 .. attribute:: classSuiteClass
1167 Callable object that constructs a test suite for the tests cases from one
1168 class. The default value is :class:`ClassTestSuite`.
1171 .. class:: TestResult
1173 This class is used to compile information about which tests have succeeded
1174 and which have failed.
1176 A :class:`TestResult` object stores the results of a set of tests. The
1177 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
1178 properly recorded; test authors do not need to worry about recording the
1181 Testing frameworks built on top of :mod:`unittest` may want access to the
1182 :class:`TestResult` object generated by running a set of tests for reporting
1183 purposes; a :class:`TestResult` instance is returned by the
1184 :meth:`TestRunner.run` method for this purpose.
1186 :class:`TestResult` instances have the following attributes that will be of
1187 interest when inspecting the results of running a set of tests:
1190 .. attribute:: errors
1192 A list containing 2-tuples of :class:`TestCase` instances and strings
1193 holding formatted tracebacks. Each tuple represents a test which raised an
1194 unexpected exception.
1196 .. versionchanged:: 2.2
1198 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1201 .. attribute:: failures
1203 A list containing 2-tuples of :class:`TestCase` instances and strings
1204 holding formatted tracebacks. Each tuple represents a test where a failure
1205 was explicitly signalled using the :meth:`TestCase.fail\*` or
1206 :meth:`TestCase.assert\*` methods.
1208 .. versionchanged:: 2.2
1210 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1212 .. attribute:: skipped
1214 A list containing 2-tuples of :class:`TestCase` instances and strings
1215 holding the reason for skipping the test.
1217 .. versionadded:: 2.7
1219 .. attribute:: expectedFailures
1221 A list contaning 2-tuples of :class:`TestCase` instances and strings
1222 holding formatted tracebacks. Each tuple represents a expected failures
1225 .. attribute:: unexpectedSuccesses
1227 A list containing :class:`TestCase` instances that were marked as expected
1228 failures, but succeeded.
1230 .. attribute:: shouldStop
1232 Set to ``True`` when the execution of tests should stop by :meth:`stop`.
1235 .. attribute:: testsRun
1237 The total number of tests run so far.
1240 .. method:: wasSuccessful()
1242 Return :const:`True` if all tests run so far have passed, otherwise returns
1248 This method can be called to signal that the set of tests being run should
1249 be aborted by setting the :attr:`shouldStop` attribute to :const:`True`.
1250 :class:`TestRunner` objects should respect this flag and return without
1251 running any additional tests.
1253 For example, this feature is used by the :class:`TextTestRunner` class to
1254 stop the test framework when the user signals an interrupt from the
1255 keyboard. Interactive tools which provide :class:`TestRunner`
1256 implementations can use this in a similar manner.
1258 The following methods of the :class:`TestResult` class are used to maintain
1259 the internal data structures, and may be extended in subclasses to support
1260 additional reporting requirements. This is particularly useful in building
1261 tools which support interactive reporting while tests are being run.
1264 .. method:: startTest(test)
1266 Called when the test case *test* is about to be run.
1268 The default implementation simply increments the instance's :attr:`testsRun`
1272 .. method:: stopTest(test)
1274 Called after the test case *test* has been executed, regardless of the
1277 The default implementation does nothing.
1280 .. method:: addError(test, err)
1282 Called when the test case *test* raises an unexpected exception *err* is a
1283 tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
1286 The default implementation appends a tuple ``(test, formatted_err)`` to
1287 the instance's :attr:`errors` attribute, where *formatted_err* is a
1288 formatted traceback derived from *err*.
1291 .. method:: addFailure(test, err)
1293 Called when the test case *test* signals a failure. *err* is a tuple of the form
1294 returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
1296 The default implementation appends a tuple ``(test, formatted_err)`` to
1297 the instance's :attr:`failures` attribute, where *formatted_err* is a
1298 formatted traceback derived from *err*.
1301 .. method:: addSuccess(test)
1303 Called when the test case *test* succeeds.
1305 The default implementation does nothing.
1308 .. method:: addSkip(test, reason)
1310 Called when the test case *test* is skipped. *reason* is the reason the
1311 test gave for skipping.
1313 The default implementation appends a tuple ``(test, reason)`` to the
1314 instance's :attr:`skipped` attribute.
1317 .. method:: addExpectedFailure(test, err)
1319 Called when the test case *test* fails, but was marked with the
1320 :func:`expectedFailure` decorator.
1322 The default implementation appends a tuple ``(test, formatted_err)`` to
1323 the instance's :attr:`expectedFailures` attribute, where *formatted_err*
1324 is a formatted traceback derived from *err*.
1327 .. method:: addUnexpectedSuccess(test)
1329 Called when the test case *test* was marked with the
1330 :func:`expectedFailure` decorator, but succeeded.
1332 The default implementation appends the test to the instance's
1333 :attr:`unexpectedSuccesses` attribute.
1336 .. data:: defaultTestLoader
1338 Instance of the :class:`TestLoader` class intended to be shared. If no
1339 customization of the :class:`TestLoader` is needed, this instance can be used
1340 instead of repeatedly creating new instances.
1343 .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
1345 A basic test runner implementation which prints results on standard error. It
1346 has a few configurable parameters, but is essentially very simple. Graphical
1347 applications which run test suites should provide alternate implementations.
1350 .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
1352 A command-line program that runs a set of tests; this is primarily for making
1353 test modules conveniently executable. The simplest use for this function is to
1354 include the following line at the end of a test script::
1356 if __name__ == '__main__':
1359 The *testRunner* argument can either be a test runner class or an already
1360 created instance of it.