2 :mod:`unittest` --- Unit testing framework
3 ==========================================
6 :synopsis: Unit testing framework for Python.
7 .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
8 .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
9 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
10 .. sectionauthor:: Raymond Hettinger <python@rcn.com>
15 The Python unit testing framework, sometimes referred to as "PyUnit," is a
16 Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
17 turn, a Java version of Kent's Smalltalk testing framework. Each is the de
18 facto standard unit testing framework for its respective language.
20 :mod:`unittest` supports test automation, sharing of setup and shutdown code for
21 tests, aggregation of tests into collections, and independence of the tests from
22 the reporting framework. The :mod:`unittest` module provides classes that make
23 it easy to support these qualities for a set of tests.
25 To achieve this, :mod:`unittest` supports some important concepts:
28 A :dfn:`test fixture` represents the preparation needed to perform one or more
29 tests, and any associate cleanup actions. This may involve, for example,
30 creating temporary or proxy databases, directories, or starting a server
34 A :dfn:`test case` is the smallest unit of testing. It checks for a specific
35 response to a particular set of inputs. :mod:`unittest` provides a base class,
36 :class:`TestCase`, which may be used to create new test cases.
39 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
40 used to aggregate tests that should be executed together.
43 A :dfn:`test runner` is a component which orchestrates the execution of tests
44 and provides the outcome to the user. The runner may use a graphical interface,
45 a textual interface, or return a special value to indicate the results of
48 The test case and test fixture concepts are supported through the
49 :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
50 used when creating new tests, and the latter can be used when integrating
51 existing test code with a :mod:`unittest`\ -driven framework. When building test
52 fixtures using :class:`TestCase`, the :meth:`setUp` and :meth:`tearDown` methods
53 can be overridden to provide initialization and cleanup for the fixture. With
54 :class:`FunctionTestCase`, existing functions can be passed to the constructor
55 for these purposes. When the test is run, the fixture initialization is run
56 first; if it succeeds, the cleanup method is run after the test has been
57 executed, regardless of the outcome of the test. Each instance of the
58 :class:`TestCase` will only be used to run a single test method, so a new
59 fixture is created for each test.
61 Test suites are implemented by the :class:`TestSuite` class. This class allows
62 individual tests and test suites to be aggregated; when the suite is executed,
63 all tests added directly to the suite and in "child" test suites are run.
65 A test runner is an object that provides a single method, :meth:`run`, which
66 accepts a :class:`TestCase` or :class:`TestSuite` object as a parameter, and
67 returns a result object. The class :class:`TestResult` is provided for use as
68 the result object. :mod:`unittest` provides the :class:`TextTestRunner` as an
69 example test runner which reports test results on the standard error stream by
70 default. Alternate runners can be implemented for other environments (such as
71 graphical environments) without any need to derive from a specific class.
77 Another test-support module with a very different flavor.
79 `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
80 Kent Beck's original paper on testing frameworks using the pattern shared by
84 .. _unittest-minimal-example:
89 The :mod:`unittest` module provides a rich set of tools for constructing and
90 running tests. This section demonstrates that a small subset of the tools
91 suffice to meet the needs of most users.
93 Here is a short script to test three functions from the :mod:`random` module::
98 class TestSequenceFunctions(unittest.TestCase):
103 def testshuffle(self):
104 # make sure the shuffled sequence does not lose any elements
105 random.shuffle(self.seq)
107 self.assertEqual(self.seq, range(10))
109 def testchoice(self):
110 element = random.choice(self.seq)
111 self.assert_(element in self.seq)
113 def testsample(self):
114 self.assertRaises(ValueError, random.sample, self.seq, 20)
115 for element in random.sample(self.seq, 5):
116 self.assert_(element in self.seq)
118 if __name__ == '__main__':
121 A testcase is created by subclassing :class:`unittest.TestCase`. The three
122 individual tests are defined with methods whose names start with the letters
123 ``test``. This naming convention informs the test runner about which methods
126 The crux of each test is a call to :meth:`assertEqual` to check for an expected
127 result; :meth:`assert_` to verify a condition; or :meth:`assertRaises` to verify
128 that an expected exception gets raised. These methods are used instead of the
129 :keyword:`assert` statement so the test runner can accumulate all test results
130 and produce a report.
132 When a :meth:`setUp` method is defined, the test runner will run that method
133 prior to each test. Likewise, if a :meth:`tearDown` method is defined, the test
134 runner will invoke that method after each test. In the example, :meth:`setUp`
135 was used to create a fresh sequence for each test.
137 The final block shows a simple way to run the tests. :func:`unittest.main`
138 provides a command line interface to the test script. When run from the command
139 line, the above script produces an output that looks like this::
142 ----------------------------------------------------------------------
143 Ran 3 tests in 0.000s
147 Instead of :func:`unittest.main`, there are other ways to run the tests with a
148 finer level of control, less terse output, and no requirement to be run from the
149 command line. For example, the last two lines may be replaced with::
151 suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
152 unittest.TextTestRunner(verbosity=2).run(suite)
154 Running the revised script from the interpreter or another script produces the
157 testchoice (__main__.TestSequenceFunctions) ... ok
158 testsample (__main__.TestSequenceFunctions) ... ok
159 testshuffle (__main__.TestSequenceFunctions) ... ok
161 ----------------------------------------------------------------------
162 Ran 3 tests in 0.110s
166 The above examples show the most commonly used :mod:`unittest` features which
167 are sufficient to meet many everyday testing needs. The remainder of the
168 documentation explores the full feature set from first principles.
171 .. _organizing-tests:
176 The basic building blocks of unit testing are :dfn:`test cases` --- single
177 scenarios that must be set up and checked for correctness. In :mod:`unittest`,
178 test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
179 class. To make your own test cases you must write subclasses of
180 :class:`TestCase`, or use :class:`FunctionTestCase`.
182 An instance of a :class:`TestCase`\ -derived class is an object that can
183 completely run a single test method, together with optional set-up and tidy-up
186 The testing code of a :class:`TestCase` instance should be entirely self
187 contained, such that it can be run either in isolation or in arbitrary
188 combination with any number of other test cases.
190 The simplest :class:`TestCase` subclass will simply override the :meth:`runTest`
191 method in order to perform specific testing code::
195 class DefaultWidgetSizeTestCase(unittest.TestCase):
197 widget = Widget('The widget')
198 self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
200 Note that in order to test something, we use the one of the :meth:`assert\*` or
201 :meth:`fail\*` methods provided by the :class:`TestCase` base class. If the
202 test fails, an exception will be raised, and :mod:`unittest` will identify the
203 test case as a :dfn:`failure`. Any other exceptions will be treated as
204 :dfn:`errors`. This helps you identify where the problem is: :dfn:`failures` are
205 caused by incorrect results - a 5 where you expected a 6. :dfn:`Errors` are
206 caused by incorrect code - e.g., a :exc:`TypeError` caused by an incorrect
209 The way to run a test case will be described later. For now, note that to
210 construct an instance of such a test case, we call its constructor without
213 testCase = DefaultWidgetSizeTestCase()
215 Now, such test cases can be numerous, and their set-up can be repetitive. In
216 the above case, constructing a :class:`Widget` in each of 100 Widget test case
217 subclasses would mean unsightly duplication.
219 Luckily, we can factor out such set-up code by implementing a method called
220 :meth:`setUp`, which the testing framework will automatically call for us when
225 class SimpleWidgetTestCase(unittest.TestCase):
227 self.widget = Widget('The widget')
229 class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
231 self.failUnless(self.widget.size() == (50,50),
232 'incorrect default size')
234 class WidgetResizeTestCase(SimpleWidgetTestCase):
236 self.widget.resize(100,150)
237 self.failUnless(self.widget.size() == (100,150),
238 'wrong size after resize')
240 If the :meth:`setUp` method raises an exception while the test is running, the
241 framework will consider the test to have suffered an error, and the
242 :meth:`runTest` method will not be executed.
244 Similarly, we can provide a :meth:`tearDown` method that tidies up after the
245 :meth:`runTest` method has been run::
249 class SimpleWidgetTestCase(unittest.TestCase):
251 self.widget = Widget('The widget')
254 self.widget.dispose()
257 If :meth:`setUp` succeeded, the :meth:`tearDown` method will be run whether
258 :meth:`runTest` succeeded or not.
260 Such a working environment for the testing code is called a :dfn:`fixture`.
262 Often, many small test cases will use the same fixture. In this case, we would
263 end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
264 classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
265 discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
270 class WidgetTestCase(unittest.TestCase):
272 self.widget = Widget('The widget')
275 self.widget.dispose()
278 def testDefaultSize(self):
279 self.failUnless(self.widget.size() == (50,50),
280 'incorrect default size')
282 def testResize(self):
283 self.widget.resize(100,150)
284 self.failUnless(self.widget.size() == (100,150),
285 'wrong size after resize')
287 Here we have not provided a :meth:`runTest` method, but have instead provided
288 two different test methods. Class instances will now each run one of the
289 :meth:`test\*` methods, with ``self.widget`` created and destroyed separately
290 for each instance. When creating an instance we must specify the test method it
291 is to run. We do this by passing the method name in the constructor::
293 defaultSizeTestCase = WidgetTestCase('testDefaultSize')
294 resizeTestCase = WidgetTestCase('testResize')
296 Test case instances are grouped together according to the features they test.
297 :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
298 represented by :mod:`unittest`'s :class:`TestSuite` class::
300 widgetTestSuite = unittest.TestSuite()
301 widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))
302 widgetTestSuite.addTest(WidgetTestCase('testResize'))
304 For the ease of running tests, as we will see later, it is a good idea to
305 provide in each test module a callable object that returns a pre-built test
309 suite = unittest.TestSuite()
310 suite.addTest(WidgetTestCase('testDefaultSize'))
311 suite.addTest(WidgetTestCase('testResize'))
317 tests = ['testDefaultSize', 'testResize']
319 return unittest.TestSuite(map(WidgetTestCase, tests))
321 Since it is a common pattern to create a :class:`TestCase` subclass with many
322 similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
323 class that can be used to automate the process of creating a test suite and
324 populating it with individual tests. For example, ::
326 suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
328 will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and
329 ``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method
330 name prefix to identify test methods automatically.
332 Note that the order in which the various test cases will be run is determined by
333 sorting the test function names with the built-in :func:`cmp` function.
335 Often it is desirable to group suites of test cases together, so as to run tests
336 for the whole system at once. This is easy, since :class:`TestSuite` instances
337 can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
338 added to a :class:`TestSuite`::
340 suite1 = module1.TheTestSuite()
341 suite2 = module2.TheTestSuite()
342 alltests = unittest.TestSuite([suite1, suite2])
344 You can place the definitions of test cases and test suites in the same modules
345 as the code they are to test (such as :file:`widget.py`), but there are several
346 advantages to placing the test code in a separate module, such as
347 :file:`test_widget.py`:
349 * The test module can be run standalone from the command line.
351 * The test code can more easily be separated from shipped code.
353 * There is less temptation to change test code to fit the code it tests without
356 * Test code should be modified much less frequently than the code it tests.
358 * Tested code can be refactored more easily.
360 * Tests for modules written in C must be in separate modules anyway, so why not
363 * If the testing strategy changes, there is no need to change the source code.
366 .. _legacy-unit-tests:
368 Re-using old test code
369 ----------------------
371 Some users will find that they have existing test code that they would like to
372 run from :mod:`unittest`, without converting every old test function to a
373 :class:`TestCase` subclass.
375 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
376 This subclass of :class:`TestCase` can be used to wrap an existing test
377 function. Set-up and tear-down functions can also be provided.
379 Given the following test function::
382 something = makeSomething()
383 assert something.name is not None
386 one can create an equivalent test case instance as follows::
388 testcase = unittest.FunctionTestCase(testSomething)
390 If there are additional set-up and tear-down methods that should be called as
391 part of the test case's operation, they can also be provided like so::
393 testcase = unittest.FunctionTestCase(testSomething,
394 setUp=makeSomethingDB,
395 tearDown=deleteSomethingDB)
397 To make migrating existing test suites easier, :mod:`unittest` supports tests
398 raising :exc:`AssertionError` to indicate test failure. However, it is
399 recommended that you use the explicit :meth:`TestCase.fail\*` and
400 :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
401 may treat :exc:`AssertionError` differently.
405 Even though :class:`FunctionTestCase` can be used to quickly convert an existing
406 test base over to a :mod:`unittest`\ -based system, this approach is not
407 recommended. Taking the time to set up proper :class:`TestCase` subclasses will
408 make future test refactorings infinitely easier.
411 .. _unittest-contents:
413 Classes and functions
414 ---------------------
417 .. class:: TestCase([methodName])
419 Instances of the :class:`TestCase` class represent the smallest testable units
420 in the :mod:`unittest` universe. This class is intended to be used as a base
421 class, with specific tests being implemented by concrete subclasses. This class
422 implements the interface needed by the test runner to allow it to drive the
423 test, and methods that the test code can use to check for and report various
426 Each instance of :class:`TestCase` will run a single test method: the method
427 named *methodName*. If you remember, we had an earlier example that went
428 something like this::
431 suite = unittest.TestSuite()
432 suite.addTest(WidgetTestCase('testDefaultSize'))
433 suite.addTest(WidgetTestCase('testResize'))
436 Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
439 *methodName* defaults to ``'runTest'``.
442 .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
444 This class implements the portion of the :class:`TestCase` interface which
445 allows the test runner to drive the test, but does not provide the methods which
446 test code can use to check and report errors. This is used to create test cases
447 using legacy test code, allowing it to be integrated into a :mod:`unittest`\
448 -based test framework.
451 .. class:: TestSuite([tests])
453 This class represents an aggregation of individual tests cases and test suites.
454 The class presents the interface needed by the test runner to allow it to be run
455 as any other test case. Running a :class:`TestSuite` instance is the same as
456 iterating over the suite, running each test individually.
458 If *tests* is given, it must be an iterable of individual test cases or other
459 test suites that will be used to build the suite initially. Additional methods
460 are provided to add test cases and suites to the collection later on.
463 .. class:: TestLoader()
465 This class is responsible for loading tests according to various criteria and
466 returning them wrapped in a :class:`TestSuite`. It can load all tests within a
467 given module or :class:`TestCase` subclass.
470 .. class:: TestResult()
472 This class is used to compile information about which tests have succeeded and
476 .. data:: defaultTestLoader
478 Instance of the :class:`TestLoader` class intended to be shared. If no
479 customization of the :class:`TestLoader` is needed, this instance can be used
480 instead of repeatedly creating new instances.
483 .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
485 A basic test runner implementation which prints results on standard error. It
486 has a few configurable parameters, but is essentially very simple. Graphical
487 applications which run test suites should provide alternate implementations.
490 .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader]]]]])
492 A command-line program that runs a set of tests; this is primarily for making
493 test modules conveniently executable. The simplest use for this function is to
494 include the following line at the end of a test script::
496 if __name__ == '__main__':
499 The *testRunner* argument can either be a test runner class or an already
500 created instance of it.
502 In some cases, the existing tests may have been written using the :mod:`doctest`
503 module. If so, that module provides a :class:`DocTestSuite` class that can
504 automatically build :class:`unittest.TestSuite` instances from the existing
505 :mod:`doctest`\ -based tests.
507 .. versionadded:: 2.3
510 .. _testcase-objects:
515 Each :class:`TestCase` instance represents a single test, but each concrete
516 subclass may be used to define multiple tests --- the concrete class represents
517 a single test fixture. The fixture is created and cleaned up for each test
520 :class:`TestCase` instances provide three groups of methods: one group used to
521 run the test, another used by the test implementation to check conditions and
522 report failures, and some inquiry methods allowing information about the test
523 itself to be gathered.
525 Methods in the first group (running the test) are:
528 .. method:: TestCase.setUp()
530 Method called to prepare the test fixture. This is called immediately before
531 calling the test method; any exception raised by this method will be considered
532 an error rather than a test failure. The default implementation does nothing.
535 .. method:: TestCase.tearDown()
537 Method called immediately after the test method has been called and the result
538 recorded. This is called even if the test method raised an exception, so the
539 implementation in subclasses may need to be particularly careful about checking
540 internal state. Any exception raised by this method will be considered an error
541 rather than a test failure. This method will only be called if the
542 :meth:`setUp` succeeds, regardless of the outcome of the test method. The
543 default implementation does nothing.
546 .. method:: TestCase.run([result])
548 Run the test, collecting the result into the test result object passed as
549 *result*. If *result* is omitted or :const:`None`, a temporary result object is
550 created (by calling the :meth:`defaultTestCase` method) and used; this result
551 object is not returned to :meth:`run`'s caller.
553 The same effect may be had by simply calling the :class:`TestCase` instance.
556 .. method:: TestCase.debug()
558 Run the test without collecting the result. This allows exceptions raised by
559 the test to be propagated to the caller, and can be used to support running
560 tests under a debugger.
562 The test code can use any of the following methods to check for and report
566 .. method:: TestCase.assert_(expr[, msg])
567 TestCase.failUnless(expr[, msg])
568 TestCase.assertTrue(expr[, msg])
570 Signal a test failure if *expr* is false; the explanation for the error will be
571 *msg* if given, otherwise it will be :const:`None`.
574 .. method:: TestCase.assertEqual(first, second[, msg])
575 TestCase.failUnlessEqual(first, second[, msg])
577 Test that *first* and *second* are equal. If the values do not compare equal,
578 the test will fail with the explanation given by *msg*, or :const:`None`. Note
579 that using :meth:`failUnlessEqual` improves upon doing the comparison as the
580 first parameter to :meth:`failUnless`: the default value for *msg* can be
581 computed to include representations of both *first* and *second*.
584 .. method:: TestCase.assertNotEqual(first, second[, msg])
585 TestCase.failIfEqual(first, second[, msg])
587 Test that *first* and *second* are not equal. If the values do compare equal,
588 the test will fail with the explanation given by *msg*, or :const:`None`. Note
589 that using :meth:`failIfEqual` improves upon doing the comparison as the first
590 parameter to :meth:`failUnless` is that the default value for *msg* can be
591 computed to include representations of both *first* and *second*.
594 .. method:: TestCase.assertAlmostEqual(first, second[, places[, msg]])
595 TestCase.failUnlessAlmostEqual(first, second[, places[, msg]])
597 Test that *first* and *second* are approximately equal by computing the
598 difference, rounding to the given number of decimal *places* (default 7),
599 and comparing to zero.
600 Note that comparing a given number of decimal places is not the same as
601 comparing a given number of significant digits. If the values do not compare
602 equal, the test will fail with the explanation given by *msg*, or :const:`None`.
605 .. method:: TestCase.assertNotAlmostEqual(first, second[, places[, msg]])
606 TestCase.failIfAlmostEqual(first, second[, places[, msg]])
608 Test that *first* and *second* are not approximately equal by computing the
609 difference, rounding to the given number of decimal *places* (default 7),
610 and comparing to zero.
611 Note that comparing a given number of decimal places is not the same as
612 comparing a given number of significant digits. If the values do not compare
613 equal, the test will fail with the explanation given by *msg*, or :const:`None`.
616 .. method:: TestCase.assertRaises(exception[, callable, ...])
617 TestCase.failUnlessRaises(exception[, callable, ...])
619 Test that an exception is raised when *callable* is called with any positional
620 or keyword arguments that are also passed to :meth:`assertRaises`. The test
621 passes if *exception* is raised, is an error if another exception is raised, or
622 fails if no exception is raised. To catch any of a group of exceptions, a tuple
623 containing the exception classes may be passed as *exception*.
625 .. versionchanged:: 2.7
627 If *callable* is omitted or None, returns a context manager so that the code
628 under test can be written inline rather than as a function::
630 with self.failUnlessRaises(some_error_class):
633 .. method:: TestCase.failIf(expr[, msg])
634 TestCase.assertFalse(expr[, msg])
636 The inverse of the :meth:`failUnless` method is the :meth:`failIf` method. This
637 signals a test failure if *expr* is true, with *msg* or :const:`None` for the
641 .. method:: TestCase.fail([msg])
643 Signals a test failure unconditionally, with *msg* or :const:`None` for the
647 .. attribute:: TestCase.failureException
649 This class attribute gives the exception raised by the :meth:`test` method. If
650 a test framework needs to use a specialized exception, possibly to carry
651 additional information, it must subclass this exception in order to "play fair"
652 with the framework. The initial value of this attribute is
653 :exc:`AssertionError`.
655 Testing frameworks can use the following methods to collect information on the
659 .. method:: TestCase.countTestCases()
661 Return the number of tests represented by this test object. For
662 :class:`TestCase` instances, this will always be ``1``.
665 .. method:: TestCase.defaultTestResult()
667 Return an instance of the test result class that should be used for this test
668 case class (if no other result instance is provided to the :meth:`run` method).
670 For :class:`TestCase` instances, this will always be an instance of
671 :class:`TestResult`; subclasses of :class:`TestCase` should override this as
675 .. method:: TestCase.id()
677 Return a string identifying the specific test case. This is usually the full
678 name of the test method, including the module and class name.
681 .. method:: TestCase.shortDescription()
683 Returns a one-line description of the test, or :const:`None` if no description
684 has been provided. The default implementation of this method returns the first
685 line of the test method's docstring, if available, or :const:`None`.
688 .. _testsuite-objects:
693 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
694 they do not actually implement a test. Instead, they are used to aggregate
695 tests into groups of tests that should be run together. Some additional methods
696 are available to add tests to :class:`TestSuite` instances:
699 .. method:: TestSuite.addTest(test)
701 Add a :class:`TestCase` or :class:`TestSuite` to the suite.
704 .. method:: TestSuite.addTests(tests)
706 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
707 instances to this test suite.
709 This is equivalent to iterating over *tests*, calling :meth:`addTest` for each
712 :class:`TestSuite` shares the following methods with :class:`TestCase`:
715 .. method:: TestSuite.run(result)
717 Run the tests associated with this suite, collecting the result into the test
718 result object passed as *result*. Note that unlike :meth:`TestCase.run`,
719 :meth:`TestSuite.run` requires the result object to be passed in.
722 .. method:: TestSuite.debug()
724 Run the tests associated with this suite without collecting the result. This
725 allows exceptions raised by the test to be propagated to the caller and can be
726 used to support running tests under a debugger.
729 .. method:: TestSuite.countTestCases()
731 Return the number of tests represented by this test object, including all
732 individual tests and sub-suites.
734 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method is
735 invoked by a :class:`TestRunner` rather than by the end-user test harness.
738 .. _testresult-objects:
743 A :class:`TestResult` object stores the results of a set of tests. The
744 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
745 properly recorded; test authors do not need to worry about recording the outcome
748 Testing frameworks built on top of :mod:`unittest` may want access to the
749 :class:`TestResult` object generated by running a set of tests for reporting
750 purposes; a :class:`TestResult` instance is returned by the
751 :meth:`TestRunner.run` method for this purpose.
753 :class:`TestResult` instances have the following attributes that will be of
754 interest when inspecting the results of running a set of tests:
757 .. attribute:: TestResult.errors
759 A list containing 2-tuples of :class:`TestCase` instances and strings holding
760 formatted tracebacks. Each tuple represents a test which raised an unexpected
763 .. versionchanged:: 2.2
764 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
767 .. attribute:: TestResult.failures
769 A list containing 2-tuples of :class:`TestCase` instances and strings holding
770 formatted tracebacks. Each tuple represents a test where a failure was
771 explicitly signalled using the :meth:`TestCase.fail\*` or
772 :meth:`TestCase.assert\*` methods.
774 .. versionchanged:: 2.2
775 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
778 .. attribute:: TestResult.testsRun
780 The total number of tests run so far.
783 .. method:: TestResult.wasSuccessful()
785 Returns :const:`True` if all tests run so far have passed, otherwise returns
789 .. method:: TestResult.stop()
791 This method can be called to signal that the set of tests being run should be
792 aborted by setting the :class:`TestResult`'s ``shouldStop`` attribute to
793 :const:`True`. :class:`TestRunner` objects should respect this flag and return
794 without running any additional tests.
796 For example, this feature is used by the :class:`TextTestRunner` class to stop
797 the test framework when the user signals an interrupt from the keyboard.
798 Interactive tools which provide :class:`TestRunner` implementations can use this
801 The following methods of the :class:`TestResult` class are used to maintain the
802 internal data structures, and may be extended in subclasses to support
803 additional reporting requirements. This is particularly useful in building
804 tools which support interactive reporting while tests are being run.
807 .. method:: TestResult.startTest(test)
809 Called when the test case *test* is about to be run.
811 The default implementation simply increments the instance's ``testsRun``
815 .. method:: TestResult.stopTest(test)
817 Called after the test case *test* has been executed, regardless of the outcome.
819 The default implementation does nothing.
822 .. method:: TestResult.addError(test, err)
824 Called when the test case *test* raises an unexpected exception *err* is a tuple
825 of the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
827 The default implementation appends a tuple ``(test, formatted_err)`` to the
828 instance's ``errors`` attribute, where *formatted_err* is a formatted
829 traceback derived from *err*.
832 .. method:: TestResult.addFailure(test, err)
834 Called when the test case *test* signals a failure. *err* is a tuple of the form
835 returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
837 The default implementation appends a tuple ``(test, formatted_err)`` to the
838 instance's ``failures`` attribute, where *formatted_err* is a formatted
839 traceback derived from *err*.
842 .. method:: TestResult.addSuccess(test)
844 Called when the test case *test* succeeds.
846 The default implementation does nothing.
849 .. _testloader-objects:
854 The :class:`TestLoader` class is used to create test suites from classes and
855 modules. Normally, there is no need to create an instance of this class; the
856 :mod:`unittest` module provides an instance that can be shared as
857 ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
858 customization of some configurable properties.
860 :class:`TestLoader` objects have the following methods:
863 .. method:: TestLoader.loadTestsFromTestCase(testCaseClass)
865 Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
866 :class:`testCaseClass`.
869 .. method:: TestLoader.loadTestsFromModule(module)
871 Return a suite of all tests cases contained in the given module. This method
872 searches *module* for classes derived from :class:`TestCase` and creates an
873 instance of the class for each test method defined for the class.
877 While using a hierarchy of :class:`TestCase`\ -derived classes can be convenient
878 in sharing fixtures and helper functions, defining test methods on base classes
879 that are not intended to be instantiated directly does not play well with this
880 method. Doing so, however, can be useful when the fixtures are different and
881 defined in subclasses.
884 .. method:: TestLoader.loadTestsFromName(name[, module])
886 Return a suite of all tests cases given a string specifier.
888 The specifier *name* is a "dotted name" that may resolve either to a module, a
889 test case class, a test method within a test case class, a :class:`TestSuite`
890 instance, or a callable object which returns a :class:`TestCase` or
891 :class:`TestSuite` instance. These checks are applied in the order listed here;
892 that is, a method on a possible test case class will be picked up as "a test
893 method within a test case class", rather than "a callable object".
895 For example, if you have a module :mod:`SampleTests` containing a
896 :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
897 methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
898 specifier ``'SampleTests.SampleTestCase'`` would cause this method to return a
899 suite which will run all three test methods. Using the specifier
900 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test suite
901 which will run only the :meth:`test_two` test method. The specifier can refer
902 to modules and packages which have not been imported; they will be imported as a
905 The method optionally resolves *name* relative to the given *module*.
908 .. method:: TestLoader.loadTestsFromNames(names[, module])
910 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather than
911 a single name. The return value is a test suite which supports all the tests
912 defined for each name.
915 .. method:: TestLoader.getTestCaseNames(testCaseClass)
917 Return a sorted sequence of method names found within *testCaseClass*; this
918 should be a subclass of :class:`TestCase`.
920 The following attributes of a :class:`TestLoader` can be configured either by
921 subclassing or assignment on an instance:
924 .. attribute:: TestLoader.testMethodPrefix
926 String giving the prefix of method names which will be interpreted as test
927 methods. The default value is ``'test'``.
929 This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
933 .. attribute:: TestLoader.sortTestMethodsUsing
935 Function to be used to compare method names when sorting them in
936 :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
937 default value is the built-in :func:`cmp` function; the attribute can also be
938 set to :const:`None` to disable the sort.
941 .. attribute:: TestLoader.suiteClass
943 Callable object that constructs a test suite from a list of tests. No methods on
944 the resulting object are needed. The default value is the :class:`TestSuite`
947 This affects all the :meth:`loadTestsFrom\*` methods.