2 :mod:`unittest` --- Unit testing framework
3 ==========================================
6 :synopsis: Unit testing framework for Python.
7 .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
8 .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
9 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
10 .. sectionauthor:: Raymond Hettinger <python@rcn.com>
15 .. versionchanged:: 2.7
16 Added test :ref:`skipping and expected failures <unittest-skipping>`.
18 The Python unit testing framework, sometimes referred to as "PyUnit," is a
19 Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
20 turn, a Java version of Kent's Smalltalk testing framework. Each is the de
21 facto standard unit testing framework for its respective language.
23 :mod:`unittest` supports test automation, sharing of setup and shutdown code for
24 tests, aggregation of tests into collections, and independence of the tests from
25 the reporting framework. The :mod:`unittest` module provides classes that make
26 it easy to support these qualities for a set of tests.
28 To achieve this, :mod:`unittest` supports some important concepts:
31 A :dfn:`test fixture` represents the preparation needed to perform one or more
32 tests, and any associate cleanup actions. This may involve, for example,
33 creating temporary or proxy databases, directories, or starting a server
37 A :dfn:`test case` is the smallest unit of testing. It checks for a specific
38 response to a particular set of inputs. :mod:`unittest` provides a base class,
39 :class:`TestCase`, which may be used to create new test cases.
42 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
43 used to aggregate tests that should be executed together.
46 A :dfn:`test runner` is a component which orchestrates the execution of tests
47 and provides the outcome to the user. The runner may use a graphical interface,
48 a textual interface, or return a special value to indicate the results of
51 The test case and test fixture concepts are supported through the
52 :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
53 used when creating new tests, and the latter can be used when integrating
54 existing test code with a :mod:`unittest`\ -driven framework. When building test
55 fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
56 :meth:`~TestCase.tearDown` methods can be overridden to provide initialization
57 and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions
58 can be passed to the constructor for these purposes. When the test is run, the
59 fixture initialization is run first; if it succeeds, the cleanup method is run
60 after the test has been executed, regardless of the outcome of the test. Each
61 instance of the :class:`TestCase` will only be used to run a single test method,
62 so a new fixture is created for each test.
64 Test suites are implemented by the :class:`TestSuite` class. This class allows
65 individual tests and test suites to be aggregated; when the suite is executed,
66 all tests added directly to the suite and in "child" test suites are run. A
67 :class:`ClassTestSuite` contains the test cases of a class.
69 A test runner is an object that provides a single method,
70 :meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
71 object as a parameter, and returns a result object. The class
72 :class:`TestResult` is provided for use as the result object. :mod:`unittest`
73 provides the :class:`TextTestRunner` as an example test runner which reports
74 test results on the standard error stream by default. Alternate runners can be
75 implemented for other environments (such as graphical environments) without any
76 need to derive from a specific class.
82 Another test-support module with a very different flavor.
84 `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
85 Kent Beck's original paper on testing frameworks using the pattern shared by
88 `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_
89 Third-party unittest frameworks with a lighter-weight syntax
90 for writing tests. For example, ``assert func(10) == 42``.
92 `python-mock <http://python-mock.sourceforge.net/>`_ and `minimock <http://blog.ianbicking.org/minimock.html>`_
93 Tools for creating mock test objects (objects simulating external resources).
95 .. _unittest-minimal-example:
100 The :mod:`unittest` module provides a rich set of tools for constructing and
101 running tests. This section demonstrates that a small subset of the tools
102 suffice to meet the needs of most users.
104 Here is a short script to test three functions from the :mod:`random` module::
109 class TestSequenceFunctions(unittest.TestCase):
114 def test_shuffle(self):
115 # make sure the shuffled sequence does not lose any elements
116 random.shuffle(self.seq)
118 self.assertEqual(self.seq, range(10))
120 def test_choice(self):
121 element = random.choice(self.seq)
122 self.assert_(element in self.seq)
124 def test_sample(self):
125 self.assertRaises(ValueError, random.sample, self.seq, 20)
126 for element in random.sample(self.seq, 5):
127 self.assert_(element in self.seq)
129 if __name__ == '__main__':
132 A testcase is created by subclassing :class:`unittest.TestCase`. The three
133 individual tests are defined with methods whose names start with the letters
134 ``test``. This naming convention informs the test runner about which methods
137 The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
138 expected result; :meth:`~TestCase.assert_` to verify a condition; or
139 :meth:`~TestCase.assertRaises` to verify that an expected exception gets raised.
140 These methods are used instead of the :keyword:`assert` statement so the test
141 runner can accumulate all test results and produce a report.
143 When a :meth:`~TestCase.setUp` method is defined, the test runner will run that
144 method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is
145 defined, the test runner will invoke that method after each test. In the
146 example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each
149 The final block shows a simple way to run the tests. :func:`unittest.main`
150 provides a command line interface to the test script. When run from the command
151 line, the above script produces an output that looks like this::
154 ----------------------------------------------------------------------
155 Ran 3 tests in 0.000s
159 Instead of :func:`unittest.main`, there are other ways to run the tests with a
160 finer level of control, less terse output, and no requirement to be run from the
161 command line. For example, the last two lines may be replaced with::
163 suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
164 unittest.TextTestRunner(verbosity=2).run(suite)
166 Running the revised script from the interpreter or another script produces the
169 testchoice (__main__.TestSequenceFunctions) ... ok
170 testsample (__main__.TestSequenceFunctions) ... ok
171 testshuffle (__main__.TestSequenceFunctions) ... ok
173 ----------------------------------------------------------------------
174 Ran 3 tests in 0.110s
178 The above examples show the most commonly used :mod:`unittest` features which
179 are sufficient to meet many everyday testing needs. The remainder of the
180 documentation explores the full feature set from first principles.
183 .. _organizing-tests:
188 The basic building blocks of unit testing are :dfn:`test cases` --- single
189 scenarios that must be set up and checked for correctness. In :mod:`unittest`,
190 test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
191 class. To make your own test cases you must write subclasses of
192 :class:`TestCase`, or use :class:`FunctionTestCase`.
194 An instance of a :class:`TestCase`\ -derived class is an object that can
195 completely run a single test method, together with optional set-up and tidy-up
198 The testing code of a :class:`TestCase` instance should be entirely self
199 contained, such that it can be run either in isolation or in arbitrary
200 combination with any number of other test cases.
202 The simplest :class:`TestCase` subclass will simply override the
203 :meth:`~TestCase.runTest` method in order to perform specific testing code::
207 class DefaultWidgetSizeTestCase(unittest.TestCase):
209 widget = Widget('The widget')
210 self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
212 Note that in order to test something, we use the one of the :meth:`assert\*` or
213 :meth:`fail\*` methods provided by the :class:`TestCase` base class. If the
214 test fails, an exception will be raised, and :mod:`unittest` will identify the
215 test case as a :dfn:`failure`. Any other exceptions will be treated as
216 :dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are
217 caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are
218 caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect
221 The way to run a test case will be described later. For now, note that to
222 construct an instance of such a test case, we call its constructor without
225 testCase = DefaultWidgetSizeTestCase()
227 Now, such test cases can be numerous, and their set-up can be repetitive. In
228 the above case, constructing a :class:`Widget` in each of 100 Widget test case
229 subclasses would mean unsightly duplication.
231 Luckily, we can factor out such set-up code by implementing a method called
232 :meth:`~TestCase.setUp`, which the testing framework will automatically call for
233 us when we run the test::
237 class SimpleWidgetTestCase(unittest.TestCase):
239 self.widget = Widget('The widget')
241 class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
243 self.failUnless(self.widget.size() == (50,50),
244 'incorrect default size')
246 class WidgetResizeTestCase(SimpleWidgetTestCase):
248 self.widget.resize(100,150)
249 self.failUnless(self.widget.size() == (100,150),
250 'wrong size after resize')
252 If the :meth:`~TestCase.setUp` method raises an exception while the test is
253 running, the framework will consider the test to have suffered an error, and the
254 :meth:`~TestCase.runTest` method will not be executed.
256 Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
257 after the :meth:`~TestCase.runTest` method has been run::
261 class SimpleWidgetTestCase(unittest.TestCase):
263 self.widget = Widget('The widget')
266 self.widget.dispose()
269 If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
270 be run whether :meth:`~TestCase.runTest` succeeded or not.
272 Such a working environment for the testing code is called a :dfn:`fixture`.
274 Often, many small test cases will use the same fixture. In this case, we would
275 end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
276 classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
277 discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
282 class WidgetTestCase(unittest.TestCase):
284 self.widget = Widget('The widget')
287 self.widget.dispose()
290 def testDefaultSize(self):
291 self.failUnless(self.widget.size() == (50,50),
292 'incorrect default size')
294 def testResize(self):
295 self.widget.resize(100,150)
296 self.failUnless(self.widget.size() == (100,150),
297 'wrong size after resize')
299 Here we have not provided a :meth:`~TestCase.runTest` method, but have instead
300 provided two different test methods. Class instances will now each run one of
301 the :meth:`test\*` methods, with ``self.widget`` created and destroyed
302 separately for each instance. When creating an instance we must specify the
303 test method it is to run. We do this by passing the method name in the
306 defaultSizeTestCase = WidgetTestCase('testDefaultSize')
307 resizeTestCase = WidgetTestCase('testResize')
309 Test case instances are grouped together according to the features they test.
310 :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
311 represented by :mod:`unittest`'s :class:`TestSuite` class::
313 widgetTestSuite = unittest.TestSuite()
314 widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))
315 widgetTestSuite.addTest(WidgetTestCase('testResize'))
317 For the ease of running tests, as we will see later, it is a good idea to
318 provide in each test module a callable object that returns a pre-built test
322 suite = unittest.TestSuite()
323 suite.addTest(WidgetTestCase('testDefaultSize'))
324 suite.addTest(WidgetTestCase('testResize'))
330 tests = ['testDefaultSize', 'testResize']
332 return unittest.TestSuite(map(WidgetTestCase, tests))
334 Since it is a common pattern to create a :class:`TestCase` subclass with many
335 similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
336 class that can be used to automate the process of creating a test suite and
337 populating it with individual tests. For example, ::
339 suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
341 will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and
342 ``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method
343 name prefix to identify test methods automatically.
345 Note that the order in which the various test cases will be run is determined by
346 sorting the test function names with the built-in :func:`cmp` function.
348 Often it is desirable to group suites of test cases together, so as to run tests
349 for the whole system at once. This is easy, since :class:`TestSuite` instances
350 can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
351 added to a :class:`TestSuite`::
353 suite1 = module1.TheTestSuite()
354 suite2 = module2.TheTestSuite()
355 alltests = unittest.TestSuite([suite1, suite2])
357 You can place the definitions of test cases and test suites in the same modules
358 as the code they are to test (such as :file:`widget.py`), but there are several
359 advantages to placing the test code in a separate module, such as
360 :file:`test_widget.py`:
362 * The test module can be run standalone from the command line.
364 * The test code can more easily be separated from shipped code.
366 * There is less temptation to change test code to fit the code it tests without
369 * Test code should be modified much less frequently than the code it tests.
371 * Tested code can be refactored more easily.
373 * Tests for modules written in C must be in separate modules anyway, so why not
376 * If the testing strategy changes, there is no need to change the source code.
379 .. _legacy-unit-tests:
381 Re-using old test code
382 ----------------------
384 Some users will find that they have existing test code that they would like to
385 run from :mod:`unittest`, without converting every old test function to a
386 :class:`TestCase` subclass.
388 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
389 This subclass of :class:`TestCase` can be used to wrap an existing test
390 function. Set-up and tear-down functions can also be provided.
392 Given the following test function::
395 something = makeSomething()
396 assert something.name is not None
399 one can create an equivalent test case instance as follows::
401 testcase = unittest.FunctionTestCase(testSomething)
403 If there are additional set-up and tear-down methods that should be called as
404 part of the test case's operation, they can also be provided like so::
406 testcase = unittest.FunctionTestCase(testSomething,
407 setUp=makeSomethingDB,
408 tearDown=deleteSomethingDB)
410 To make migrating existing test suites easier, :mod:`unittest` supports tests
411 raising :exc:`AssertionError` to indicate test failure. However, it is
412 recommended that you use the explicit :meth:`TestCase.fail\*` and
413 :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
414 may treat :exc:`AssertionError` differently.
418 Even though :class:`FunctionTestCase` can be used to quickly convert an existing
419 test base over to a :mod:`unittest`\ -based system, this approach is not
420 recommended. Taking the time to set up proper :class:`TestCase` subclasses will
421 make future test refactorings infinitely easier.
423 In some cases, the existing tests may have been written using the :mod:`doctest`
424 module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
425 automatically build :class:`unittest.TestSuite` instances from the existing
426 :mod:`doctest`\ -based tests.
429 .. _unittest-skipping:
431 Skipping tests and expected failures
432 ------------------------------------
434 Unittest supports skipping individual test methods and even whole classes of
435 tests. In addition, it supports marking a test as a "expected failure," a test
436 that is broken and will fail, but shouldn't be counted as a failure on a
439 Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
440 or one of its conditional variants.
442 Basic skipping looks like this: ::
444 class MyTestCase(unittest.TestCase):
446 @unittest.skip("demonstrating skipping")
447 def test_nothing(self):
448 self.fail("shouldn't happen")
450 @unittest.skipIf(mylib.__version__ < (1, 3), "not supported in this library version")
451 def test_format(self):
452 # 2.6+ only code here.
455 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
456 def test_windows_support(self):
457 # windows specific testing code
460 This is the output of running the example above in verbose mode: ::
462 test_format (__main__.MyTestCase) ... skipped 'not supported in this Python version'
463 test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
464 test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'
466 ----------------------------------------------------------------------
467 Ran 3 tests in 0.005s
471 Classes can be skipped just like methods: ::
473 @skip("showing class skipping")
474 class MySkippedTestCase(unittest.TestCase):
475 def test_not_run(self):
478 :meth:`TestCase.setUp` can also skip the test. This is useful when a resource
479 that needs to be set up is not available.
481 Expected failures use the :func:`expectedFailure` decorator. ::
483 class ExpectedFailureTestCase(unittest.TestCase):
484 @unittest.expectedFailure
486 self.assertEqual(1, 0, "broken")
488 It's easy to roll your own skipping decorators by making a decorator that calls
489 :func:`skip` on the test when it wants it to be skipped. This decorator skips
490 the test unless the passed object has a certain attribute: ::
492 def skipUnlessHasattr(obj, attr):
493 if hasattr(obj, attr):
494 return lambda func: func
495 return unittest.skip("{0!r} doesn't have {1!r}".format(obj, attr))
497 The following decorators implement test skipping and expected failures:
499 .. function:: skip(reason)
501 Unconditionally skip the decorated test. *reason* should describe why the
502 test is being skipped.
504 .. function:: skipIf(condition, reason)
506 Skip the decorated test if *condition* is true.
508 .. function:: skipUnless(condition, reason)
510 Skip the decoratored test unless *condition* is true.
512 .. function:: expectedFailure
514 Mark the test as an expected failure. If the test fails when run, the test
515 is not counted as a failure.
518 .. _unittest-contents:
520 Classes and functions
521 ---------------------
523 This section describes in depth the API of :mod:`unittest`.
526 .. _testcase-objects:
531 .. class:: TestCase([methodName])
533 Instances of the :class:`TestCase` class represent the smallest testable units
534 in the :mod:`unittest` universe. This class is intended to be used as a base
535 class, with specific tests being implemented by concrete subclasses. This class
536 implements the interface needed by the test runner to allow it to drive the
537 test, and methods that the test code can use to check for and report various
540 Each instance of :class:`TestCase` will run a single test method: the method
541 named *methodName*. If you remember, we had an earlier example that went
542 something like this::
545 suite = unittest.TestSuite()
546 suite.addTest(WidgetTestCase('testDefaultSize'))
547 suite.addTest(WidgetTestCase('testResize'))
550 Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
553 *methodName* defaults to :meth:`runTest`.
555 :class:`TestCase` instances provide three groups of methods: one group used
556 to run the test, another used by the test implementation to check conditions
557 and report failures, and some inquiry methods allowing information about the
558 test itself to be gathered.
560 Methods in the first group (running the test) are:
565 Method called to prepare the test fixture. This is called immediately
566 before calling the test method; any exception raised by this method will
567 be considered an error rather than a test failure. The default
568 implementation does nothing.
571 .. method:: tearDown()
573 Method called immediately after the test method has been called and the
574 result recorded. This is called even if the test method raised an
575 exception, so the implementation in subclasses may need to be particularly
576 careful about checking internal state. Any exception raised by this
577 method will be considered an error rather than a test failure. This
578 method will only be called if the :meth:`setUp` succeeds, regardless of
579 the outcome of the test method. The default implementation does nothing.
582 .. method:: run([result])
584 Run the test, collecting the result into the test result object passed as
585 *result*. If *result* is omitted or :const:`None`, a temporary result
586 object is created (by calling the :meth:`defaultTestCase` method) and
587 used; this result object is not returned to :meth:`run`'s caller.
589 The same effect may be had by simply calling the :class:`TestCase`
593 .. method:: skipTest(reason)
595 Calling this during the a test method or :meth:`setUp` skips the current
596 test. See :ref:`unittest-skipping` for more information.
601 Run the test without collecting the result. This allows exceptions raised
602 by the test to be propagated to the caller, and can be used to support
603 running tests under a debugger.
605 The test code can use any of the following methods to check for and report
609 .. method:: assert_(expr[, msg])
610 failUnless(expr[, msg])
611 assertTrue(expr[, msg])
613 Signal a test failure if *expr* is false; the explanation for the error
614 will be *msg* if given, otherwise it will be :const:`None`.
617 .. method:: assertEqual(first, second[, msg])
618 failUnlessEqual(first, second[, msg])
620 Test that *first* and *second* are equal. If the values do not compare
621 equal, the test will fail with the explanation given by *msg*, or
622 :const:`None`. Note that using :meth:`failUnlessEqual` improves upon
623 doing the comparison as the first parameter to :meth:`failUnless`: the
624 default value for *msg* can be computed to include representations of both
625 *first* and *second*.
628 .. method:: assertNotEqual(first, second[, msg])
629 failIfEqual(first, second[, msg])
631 Test that *first* and *second* are not equal. If the values do compare
632 equal, the test will fail with the explanation given by *msg*, or
633 :const:`None`. Note that using :meth:`failIfEqual` improves upon doing
634 the comparison as the first parameter to :meth:`failUnless` is that the
635 default value for *msg* can be computed to include representations of both
636 *first* and *second*.
639 .. method:: assertAlmostEqual(first, second[, places[, msg]])
640 failUnlessAlmostEqual(first, second[, places[, msg]])
642 Test that *first* and *second* are approximately equal by computing the
643 difference, rounding to the given number of decimal *places* (default 7),
644 and comparing to zero.
646 Note that comparing a given number of decimal places is not the same as
647 comparing a given number of significant digits. If the values do not
648 compare equal, the test will fail with the explanation given by *msg*, or
652 .. method:: assertNotAlmostEqual(first, second[, places[, msg]])
653 failIfAlmostEqual(first, second[, places[, msg]])
655 Test that *first* and *second* are not approximately equal by computing
656 the difference, rounding to the given number of decimal *places* (default
657 7), and comparing to zero.
659 Note that comparing a given number of decimal places is not the same as
660 comparing a given number of significant digits. If the values do not
661 compare equal, the test will fail with the explanation given by *msg*, or
665 .. method:: assertRaises(exception[, callable, ...])
666 failUnlessRaises(exception[, callable, ...])
668 Test that an exception is raised when *callable* is called with any
669 positional or keyword arguments that are also passed to
670 :meth:`assertRaises`. The test passes if *exception* is raised, is an
671 error if another exception is raised, or fails if no exception is raised.
672 To catch any of a group of exceptions, a tuple containing the exception
673 classes may be passed as *exception*.
675 If *callable* is omitted or None, returns a context manager so that the
676 code under test can be written inline rather than as a function::
678 with self.failUnlessRaises(some_error_class):
681 .. versionchanged:: 2.7
682 Added the ability to use :meth:`assertRaises` as a context manager.
685 .. method:: failIf(expr[, msg])
686 assertFalse(expr[, msg])
688 The inverse of the :meth:`failUnless` method is the :meth:`failIf` method.
689 This signals a test failure if *expr* is true, with *msg* or :const:`None`
690 for the error message.
693 .. method:: fail([msg])
695 Signals a test failure unconditionally, with *msg* or :const:`None` for
699 .. attribute:: failureException
701 This class attribute gives the exception raised by the test method. If a
702 test framework needs to use a specialized exception, possibly to carry
703 additional information, it must subclass this exception in order to "play
704 fair" with the framework. The initial value of this attribute is
705 :exc:`AssertionError`.
707 Testing frameworks can use the following methods to collect information on
711 .. method:: countTestCases()
713 Return the number of tests represented by this test object. For
714 :class:`TestCase` instances, this will always be ``1``.
717 .. method:: defaultTestResult()
719 Return an instance of the test result class that should be used for this
720 test case class (if no other result instance is provided to the
723 For :class:`TestCase` instances, this will always be an instance of
724 :class:`TestResult`; subclasses of :class:`TestCase` should override this
730 Return a string identifying the specific test case. This is usually the
731 full name of the test method, including the module and class name.
734 .. method:: shortDescription()
736 Returns a one-line description of the test, or :const:`None` if no
737 description has been provided. The default implementation of this method
738 returns the first line of the test method's docstring, if available, or
742 .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
744 This class implements the portion of the :class:`TestCase` interface which
745 allows the test runner to drive the test, but does not provide the methods which
746 test code can use to check and report errors. This is used to create test cases
747 using legacy test code, allowing it to be integrated into a :mod:`unittest`\
748 -based test framework.
751 .. _testsuite-objects:
756 .. class:: TestSuite([tests])
758 This class represents an aggregation of individual tests cases and test suites.
759 The class presents the interface needed by the test runner to allow it to be run
760 as any other test case. Running a :class:`TestSuite` instance is the same as
761 iterating over the suite, running each test individually.
763 If *tests* is given, it must be an iterable of individual test cases or other
764 test suites that will be used to build the suite initially. Additional methods
765 are provided to add test cases and suites to the collection later on.
767 :class:`TestSuite` (including :class:`ClassTestSuite`) objects behave much
768 like :class:`TestCase` objects, except they do not actually implement a test.
769 Instead, they are used to aggregate tests into groups of tests that should be
770 run together. Some additional methods are available to add tests to
771 :class:`TestSuite` instances:
774 .. method:: TestSuite.addTest(test)
776 Add a :class:`TestCase` or :class:`TestSuite` to the suite.
779 .. method:: TestSuite.addTests(tests)
781 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
782 instances to this test suite.
784 This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
787 :class:`TestSuite` shares the following methods with :class:`TestCase`:
790 .. method:: run(result)
792 Run the tests associated with this suite, collecting the result into the
793 test result object passed as *result*. Note that unlike
794 :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
800 Run the tests associated with this suite without collecting the
801 result. This allows exceptions raised by the test to be propagated to the
802 caller and can be used to support running tests under a debugger.
805 .. method:: countTestCases()
807 Return the number of tests represented by this test object, including all
808 individual tests and sub-suites.
810 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
811 is invoked by a :class:`TestRunner` rather than by the end-user test harness.
814 .. class:: ClassTestSuite(tests, collected_from)
816 This subclass of :class:`TestSuite` repesents an aggregation of individuals
817 tests from one :class:`TestCase` class. *tests* is an iterable of
818 :class:`TestCase` instances created from the class. *collected_from* is the
819 class they came from.
822 Loading and running tests
823 ~~~~~~~~~~~~~~~~~~~~~~~~~
825 .. class:: TestLoader()
827 The :class:`TestLoader` class is used to create test suites from classes and
828 modules. Normally, there is no need to create an instance of this class; the
829 :mod:`unittest` module provides an instance that can be shared as
830 ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
831 customization of some configurable properties.
833 :class:`TestLoader` objects have the following methods:
836 .. method:: loadTestsFromTestCase(testCaseClass)
838 Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
839 :class:`testCaseClass`.
842 .. method:: loadTestsFromModule(module)
844 Return a suite of all tests cases contained in the given module. This
845 method searches *module* for classes derived from :class:`TestCase` and
846 creates an instance of the class for each test method defined for the
851 While using a hierarchy of :class:`TestCase`\ -derived classes can be
852 convenient in sharing fixtures and helper functions, defining test
853 methods on base classes that are not intended to be instantiated
854 directly does not play well with this method. Doing so, however, can
855 be useful when the fixtures are different and defined in subclasses.
858 .. method:: loadTestsFromName(name[, module])
860 Return a suite of all tests cases given a string specifier.
862 The specifier *name* is a "dotted name" that may resolve either to a
863 module, a test case class, a test method within a test case class, a
864 :class:`TestSuite` instance, or a callable object which returns a
865 :class:`TestCase` or :class:`TestSuite` instance. These checks are
866 applied in the order listed here; that is, a method on a possible test
867 case class will be picked up as "a test method within a test case class",
868 rather than "a callable object".
870 For example, if you have a module :mod:`SampleTests` containing a
871 :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
872 methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
873 specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
874 suite which will run all three test methods. Using the specifier
875 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
876 which will run only the :meth:`test_two` test method. The specifier can refer
877 to modules and packages which have not been imported; they will be imported as a
880 The method optionally resolves *name* relative to the given *module*.
883 .. method:: loadTestsFromNames(names[, module])
885 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
886 than a single name. The return value is a test suite which supports all
887 the tests defined for each name.
890 .. method:: getTestCaseNames(testCaseClass)
892 Return a sorted sequence of method names found within *testCaseClass*;
893 this should be a subclass of :class:`TestCase`.
895 The following attributes of a :class:`TestLoader` can be configured either by
896 subclassing or assignment on an instance:
899 .. attribute:: testMethodPrefix
901 String giving the prefix of method names which will be interpreted as test
902 methods. The default value is ``'test'``.
904 This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
908 .. attribute:: sortTestMethodsUsing
910 Function to be used to compare method names when sorting them in
911 :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
912 default value is the built-in :func:`cmp` function; the attribute can also
913 be set to :const:`None` to disable the sort.
916 .. attribute:: suiteClass
918 Callable object that constructs a test suite from a list of tests. No
919 methods on the resulting object are needed. The default value is the
920 :class:`TestSuite` class.
922 This affects all the :meth:`loadTestsFrom\*` methods.
925 .. attribute:: classSuiteClass
927 Callable object that constructs a test suite for the tests cases from one
928 class. The default value is :class:`ClassTestSuite`.
931 .. class:: TestResult
933 This class is used to compile information about which tests have succeeded
934 and which have failed.
936 A :class:`TestResult` object stores the results of a set of tests. The
937 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
938 properly recorded; test authors do not need to worry about recording the
941 Testing frameworks built on top of :mod:`unittest` may want access to the
942 :class:`TestResult` object generated by running a set of tests for reporting
943 purposes; a :class:`TestResult` instance is returned by the
944 :meth:`TestRunner.run` method for this purpose.
946 :class:`TestResult` instances have the following attributes that will be of
947 interest when inspecting the results of running a set of tests:
950 .. attribute:: errors
952 A list containing 2-tuples of :class:`TestCase` instances and strings
953 holding formatted tracebacks. Each tuple represents a test which raised an
954 unexpected exception.
956 .. versionchanged:: 2.2
958 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
961 .. attribute:: failures
963 A list containing 2-tuples of :class:`TestCase` instances and strings
964 holding formatted tracebacks. Each tuple represents a test where a failure
965 was explicitly signalled using the :meth:`TestCase.fail\*` or
966 :meth:`TestCase.assert\*` methods.
968 .. versionchanged:: 2.2
970 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
972 .. attribute:: skipped
974 A list containing 2-tuples of :class:`TestCase` instances and strings
975 holding the reason for skipping the test.
977 .. versionadded:: 2.7
979 .. attribute:: expectedFailures
981 A list contaning 2-tuples of :class:`TestCase` instances and strings
982 holding formatted tracebacks. Each tuple represents a expected failures
985 .. attribute:: unexpectedSuccesses
987 A list containing :class:`TestCase` instances that were marked as expected
988 failures, but succeeded.
990 .. attribute:: shouldStop
992 Set to ``True`` when the execution of tests should stop by :meth:`stop`.
995 .. attribute:: testsRun
997 The total number of tests run so far.
1000 .. method:: wasSuccessful()
1002 Return :const:`True` if all tests run so far have passed, otherwise returns
1008 This method can be called to signal that the set of tests being run should
1009 be aborted by setting the :attr:`shouldStop` attribute to :const:`True`.
1010 :class:`TestRunner` objects should respect this flag and return without
1011 running any additional tests.
1013 For example, this feature is used by the :class:`TextTestRunner` class to
1014 stop the test framework when the user signals an interrupt from the
1015 keyboard. Interactive tools which provide :class:`TestRunner`
1016 implementations can use this in a similar manner.
1018 The following methods of the :class:`TestResult` class are used to maintain
1019 the internal data structures, and may be extended in subclasses to support
1020 additional reporting requirements. This is particularly useful in building
1021 tools which support interactive reporting while tests are being run.
1024 .. method:: startTest(test)
1026 Called when the test case *test* is about to be run.
1028 The default implementation simply increments the instance's :attr:`testsRun`
1032 .. method:: stopTest(test)
1034 Called after the test case *test* has been executed, regardless of the
1037 The default implementation does nothing.
1040 .. method:: addError(test, err)
1042 Called when the test case *test* raises an unexpected exception *err* is a
1043 tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
1046 The default implementation appends a tuple ``(test, formatted_err)`` to
1047 the instance's :attr:`errors` attribute, where *formatted_err* is a
1048 formatted traceback derived from *err*.
1051 .. method:: addFailure(test, err)
1053 Called when the test case *test* signals a failure. *err* is a tuple of the form
1054 returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
1056 The default implementation appends a tuple ``(test, formatted_err)`` to
1057 the instance's :attr:`failures` attribute, where *formatted_err* is a
1058 formatted traceback derived from *err*.
1061 .. method:: addSuccess(test)
1063 Called when the test case *test* succeeds.
1065 The default implementation does nothing.
1068 .. method:: addSkip(test, reason)
1070 Called when the test case *test* is skipped. *reason* is the reason the
1071 test gave for skipping.
1073 The default implementation appends a tuple ``(test, reason)`` to the
1074 instance's :attr:`skipped` attribute.
1077 .. method:: addExpectedFailure(test, err)
1079 Called when the test case *test* fails, but was marked with the
1080 :func:`expectedFailure` decorator.
1082 The default implementation appends a tuple ``(test, formatted_err)`` to
1083 the instance's :attr:`expectedFailures` attribute, where *formatted_err*
1084 is a formatted traceback derived from *err*.
1087 .. method:: addUnexpectedSuccess(test)
1089 Called when the test case *test* was marked with the
1090 :func:`expectedFailure` decorator, but succeeded.
1092 The default implementation appends the test to the instance's
1093 :attr:`unexpectedSuccesses` attribute.
1096 .. data:: defaultTestLoader
1098 Instance of the :class:`TestLoader` class intended to be shared. If no
1099 customization of the :class:`TestLoader` is needed, this instance can be used
1100 instead of repeatedly creating new instances.
1103 .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
1105 A basic test runner implementation which prints results on standard error. It
1106 has a few configurable parameters, but is essentially very simple. Graphical
1107 applications which run test suites should provide alternate implementations.
1110 .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
1112 A command-line program that runs a set of tests; this is primarily for making
1113 test modules conveniently executable. The simplest use for this function is to
1114 include the following line at the end of a test script::
1116 if __name__ == '__main__':
1119 The *testRunner* argument can either be a test runner class or an already
1120 created instance of it.