1 :mod:`unittest` --- Unit testing framework
2 ==========================================
5 :synopsis: Unit testing framework for Python.
6 .. moduleauthor:: Steve Purcell <stephen_purcell@yahoo.com>
7 .. sectionauthor:: Steve Purcell <stephen_purcell@yahoo.com>
8 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
9 .. sectionauthor:: Raymond Hettinger <python@rcn.com>
14 .. versionchanged:: 2.7
15 Added test :ref:`skipping and expected failures <unittest-skipping>`.
17 The Python unit testing framework, sometimes referred to as "PyUnit," is a
18 Python language version of JUnit, by Kent Beck and Erich Gamma. JUnit is, in
19 turn, a Java version of Kent's Smalltalk testing framework. Each is the de
20 facto standard unit testing framework for its respective language.
22 :mod:`unittest` supports test automation, sharing of setup and shutdown code for
23 tests, aggregation of tests into collections, and independence of the tests from
24 the reporting framework. The :mod:`unittest` module provides classes that make
25 it easy to support these qualities for a set of tests.
27 To achieve this, :mod:`unittest` supports some important concepts:
30 A :dfn:`test fixture` represents the preparation needed to perform one or more
31 tests, and any associate cleanup actions. This may involve, for example,
32 creating temporary or proxy databases, directories, or starting a server
36 A :dfn:`test case` is the smallest unit of testing. It checks for a specific
37 response to a particular set of inputs. :mod:`unittest` provides a base class,
38 :class:`TestCase`, which may be used to create new test cases.
41 A :dfn:`test suite` is a collection of test cases, test suites, or both. It is
42 used to aggregate tests that should be executed together.
45 A :dfn:`test runner` is a component which orchestrates the execution of tests
46 and provides the outcome to the user. The runner may use a graphical interface,
47 a textual interface, or return a special value to indicate the results of
50 The test case and test fixture concepts are supported through the
51 :class:`TestCase` and :class:`FunctionTestCase` classes; the former should be
52 used when creating new tests, and the latter can be used when integrating
53 existing test code with a :mod:`unittest`\ -driven framework. When building test
54 fixtures using :class:`TestCase`, the :meth:`~TestCase.setUp` and
55 :meth:`~TestCase.tearDown` methods can be overridden to provide initialization
56 and cleanup for the fixture. With :class:`FunctionTestCase`, existing functions
57 can be passed to the constructor for these purposes. When the test is run, the
58 fixture initialization is run first; if it succeeds, the cleanup method is run
59 after the test has been executed, regardless of the outcome of the test. Each
60 instance of the :class:`TestCase` will only be used to run a single test method,
61 so a new fixture is created for each test.
63 Test suites are implemented by the :class:`TestSuite` class. This class allows
64 individual tests and test suites to be aggregated; when the suite is executed,
65 all tests added directly to the suite and in "child" test suites are run.
67 A test runner is an object that provides a single method,
68 :meth:`~TestRunner.run`, which accepts a :class:`TestCase` or :class:`TestSuite`
69 object as a parameter, and returns a result object. The class
70 :class:`TestResult` is provided for use as the result object. :mod:`unittest`
71 provides the :class:`TextTestRunner` as an example test runner which reports
72 test results on the standard error stream by default. Alternate runners can be
73 implemented for other environments (such as graphical environments) without any
74 need to derive from a specific class.
80 Another test-support module with a very different flavor.
82 `Simple Smalltalk Testing: With Patterns <http://www.XProgramming.com/testfram.htm>`_
83 Kent Beck's original paper on testing frameworks using the pattern shared
86 `Nose <http://code.google.com/p/python-nose/>`_ and `py.test <http://pytest.org>`_
87 Third-party unittest frameworks with a lighter-weight syntax for writing
88 tests. For example, ``assert func(10) == 42``.
90 `python-mock <http://python-mock.sourceforge.net/>`_ and `minimock <http://blog.ianbicking.org/minimock.html>`_
91 Tools for creating mock test objects (objects simulating external
97 .. _unittest-minimal-example:
102 The :mod:`unittest` module provides a rich set of tools for constructing and
103 running tests. This section demonstrates that a small subset of the tools
104 suffice to meet the needs of most users.
106 Here is a short script to test three functions from the :mod:`random` module::
111 class TestSequenceFunctions(unittest.TestCase):
116 def test_shuffle(self):
117 # make sure the shuffled sequence does not lose any elements
118 random.shuffle(self.seq)
120 self.assertEqual(self.seq, range(10))
122 def test_choice(self):
123 element = random.choice(self.seq)
124 self.assert_(element in self.seq)
126 def test_sample(self):
127 self.assertRaises(ValueError, random.sample, self.seq, 20)
128 for element in random.sample(self.seq, 5):
129 self.assert_(element in self.seq)
131 if __name__ == '__main__':
134 A testcase is created by subclassing :class:`unittest.TestCase`. The three
135 individual tests are defined with methods whose names start with the letters
136 ``test``. This naming convention informs the test runner about which methods
139 The crux of each test is a call to :meth:`~TestCase.assertEqual` to check for an
140 expected result; :meth:`~TestCase.assert_` to verify a condition; or
141 :meth:`~TestCase.assertRaises` to verify that an expected exception gets raised.
142 These methods are used instead of the :keyword:`assert` statement so the test
143 runner can accumulate all test results and produce a report.
145 When a :meth:`~TestCase.setUp` method is defined, the test runner will run that
146 method prior to each test. Likewise, if a :meth:`~TestCase.tearDown` method is
147 defined, the test runner will invoke that method after each test. In the
148 example, :meth:`~TestCase.setUp` was used to create a fresh sequence for each
151 The final block shows a simple way to run the tests. :func:`unittest.main`
152 provides a command line interface to the test script. When run from the command
153 line, the above script produces an output that looks like this::
156 ----------------------------------------------------------------------
157 Ran 3 tests in 0.000s
161 Instead of :func:`unittest.main`, there are other ways to run the tests with a
162 finer level of control, less terse output, and no requirement to be run from the
163 command line. For example, the last two lines may be replaced with::
165 suite = unittest.TestLoader().loadTestsFromTestCase(TestSequenceFunctions)
166 unittest.TextTestRunner(verbosity=2).run(suite)
168 Running the revised script from the interpreter or another script produces the
171 testchoice (__main__.TestSequenceFunctions) ... ok
172 testsample (__main__.TestSequenceFunctions) ... ok
173 testshuffle (__main__.TestSequenceFunctions) ... ok
175 ----------------------------------------------------------------------
176 Ran 3 tests in 0.110s
180 The above examples show the most commonly used :mod:`unittest` features which
181 are sufficient to meet many everyday testing needs. The remainder of the
182 documentation explores the full feature set from first principles.
185 .. _unittest-command-line-interface:
187 Command Line Interface
188 ----------------------
190 The unittest module can be used from the command line to run tests from
191 modules, classes or even individual test methods::
193 python -m unittest test_module1 test_module2
194 python -m unittest test_module.TestClass
195 python -m unittest test_module.TestClass.test_method
197 You can pass in a list with any combination of module names, and fully
198 qualified class or method names.
200 You can run tests with more detail (higher verbosity) by passing in the -v flag::
202 python-m unittest -v test_module
204 For a list of all the command line options::
206 python -m unittest -h
208 .. versionchanged:: 2.7
209 In earlier versions it was only possible to run individual test methods and
210 not modules or classes.
212 The command line can also be used for test discovery, for running all of the
213 tests in a project or just a subset.
216 .. _unittest-test-discovery:
221 .. versionadded:: 2.7
223 Unittest supports simple test discovery. For a project's tests to be
224 compatible with test discovery they must all be importable from the top level
225 directory of the project (in other words, they must all be in Python packages).
227 Test discovery is implemented in :meth:`TestLoader.discover`, but can also be
228 used from the command line. The basic command line usage is::
231 python -m unittest discover
233 The ``discover`` sub-command has the following options:
235 -v, --verbose Verbose output
236 -s directory Directory to start discovery ('.' default)
237 -p pattern Pattern to match test files ('test*.py' default)
238 -t directory Top level directory of project (default to
241 The -s, -p, & -t options can be passsed in as positional arguments. The
242 following two command lines are equivalent::
244 python -m unittest -s project_directory -p '*_test.py'
245 python -m unittest project_directory '*_test.py'
247 Test modules and packages can customize test loading and discovery by through
248 the `load_tests protocol`_.
251 .. _organizing-tests:
256 The basic building blocks of unit testing are :dfn:`test cases` --- single
257 scenarios that must be set up and checked for correctness. In :mod:`unittest`,
258 test cases are represented by instances of :mod:`unittest`'s :class:`TestCase`
259 class. To make your own test cases you must write subclasses of
260 :class:`TestCase`, or use :class:`FunctionTestCase`.
262 An instance of a :class:`TestCase`\ -derived class is an object that can
263 completely run a single test method, together with optional set-up and tidy-up
266 The testing code of a :class:`TestCase` instance should be entirely self
267 contained, such that it can be run either in isolation or in arbitrary
268 combination with any number of other test cases.
270 The simplest :class:`TestCase` subclass will simply override the
271 :meth:`~TestCase.runTest` method in order to perform specific testing code::
275 class DefaultWidgetSizeTestCase(unittest.TestCase):
277 widget = Widget('The widget')
278 self.assertEqual(widget.size(), (50, 50), 'incorrect default size')
280 Note that in order to test something, we use the one of the :meth:`assert\*`
281 methods provided by the :class:`TestCase` base class. If the test fails, an
282 exception will be raised, and :mod:`unittest` will identify the test case as a
283 :dfn:`failure`. Any other exceptions will be treated as :dfn:`errors`. This
284 helps you identify where the problem is: :dfn:`failures` are caused by incorrect
285 results - a 5 where you expected a 6. :dfn:`Errors` are caused by incorrect
286 code - e.g., a :exc:`TypeError` caused by an incorrect function call.
288 The way to run a test case will be described later. For now, note that to
289 construct an instance of such a test case, we call its constructor without
292 testCase = DefaultWidgetSizeTestCase()
294 Now, such test cases can be numerous, and their set-up can be repetitive. In
295 the above case, constructing a :class:`Widget` in each of 100 Widget test case
296 subclasses would mean unsightly duplication.
298 Luckily, we can factor out such set-up code by implementing a method called
299 :meth:`~TestCase.setUp`, which the testing framework will automatically call for
300 us when we run the test::
304 class SimpleWidgetTestCase(unittest.TestCase):
306 self.widget = Widget('The widget')
308 class DefaultWidgetSizeTestCase(SimpleWidgetTestCase):
310 self.assertTrue(self.widget.size() == (50,50),
311 'incorrect default size')
313 class WidgetResizeTestCase(SimpleWidgetTestCase):
315 self.widget.resize(100,150)
316 self.assertTrue(self.widget.size() == (100,150),
317 'wrong size after resize')
319 If the :meth:`~TestCase.setUp` method raises an exception while the test is
320 running, the framework will consider the test to have suffered an error, and the
321 :meth:`~TestCase.runTest` method will not be executed.
323 Similarly, we can provide a :meth:`~TestCase.tearDown` method that tidies up
324 after the :meth:`~TestCase.runTest` method has been run::
328 class SimpleWidgetTestCase(unittest.TestCase):
330 self.widget = Widget('The widget')
333 self.widget.dispose()
336 If :meth:`~TestCase.setUp` succeeded, the :meth:`~TestCase.tearDown` method will
337 be run whether :meth:`~TestCase.runTest` succeeded or not.
339 Such a working environment for the testing code is called a :dfn:`fixture`.
341 Often, many small test cases will use the same fixture. In this case, we would
342 end up subclassing :class:`SimpleWidgetTestCase` into many small one-method
343 classes such as :class:`DefaultWidgetSizeTestCase`. This is time-consuming and
344 discouraging, so in the same vein as JUnit, :mod:`unittest` provides a simpler
349 class WidgetTestCase(unittest.TestCase):
351 self.widget = Widget('The widget')
354 self.widget.dispose()
357 def testDefaultSize(self):
358 self.assertTrue(self.widget.size() == (50,50),
359 'incorrect default size')
361 def testResize(self):
362 self.widget.resize(100,150)
363 self.assertTrue(self.widget.size() == (100,150),
364 'wrong size after resize')
366 Here we have not provided a :meth:`~TestCase.runTest` method, but have instead
367 provided two different test methods. Class instances will now each run one of
368 the :meth:`test\*` methods, with ``self.widget`` created and destroyed
369 separately for each instance. When creating an instance we must specify the
370 test method it is to run. We do this by passing the method name in the
373 defaultSizeTestCase = WidgetTestCase('testDefaultSize')
374 resizeTestCase = WidgetTestCase('testResize')
376 Test case instances are grouped together according to the features they test.
377 :mod:`unittest` provides a mechanism for this: the :dfn:`test suite`,
378 represented by :mod:`unittest`'s :class:`TestSuite` class::
380 widgetTestSuite = unittest.TestSuite()
381 widgetTestSuite.addTest(WidgetTestCase('testDefaultSize'))
382 widgetTestSuite.addTest(WidgetTestCase('testResize'))
384 For the ease of running tests, as we will see later, it is a good idea to
385 provide in each test module a callable object that returns a pre-built test
389 suite = unittest.TestSuite()
390 suite.addTest(WidgetTestCase('testDefaultSize'))
391 suite.addTest(WidgetTestCase('testResize'))
397 tests = ['testDefaultSize', 'testResize']
399 return unittest.TestSuite(map(WidgetTestCase, tests))
401 Since it is a common pattern to create a :class:`TestCase` subclass with many
402 similarly named test functions, :mod:`unittest` provides a :class:`TestLoader`
403 class that can be used to automate the process of creating a test suite and
404 populating it with individual tests. For example, ::
406 suite = unittest.TestLoader().loadTestsFromTestCase(WidgetTestCase)
408 will create a test suite that will run ``WidgetTestCase.testDefaultSize()`` and
409 ``WidgetTestCase.testResize``. :class:`TestLoader` uses the ``'test'`` method
410 name prefix to identify test methods automatically.
412 Note that the order in which the various test cases will be run is determined by
413 sorting the test function names with the built-in :func:`cmp` function.
415 Often it is desirable to group suites of test cases together, so as to run tests
416 for the whole system at once. This is easy, since :class:`TestSuite` instances
417 can be added to a :class:`TestSuite` just as :class:`TestCase` instances can be
418 added to a :class:`TestSuite`::
420 suite1 = module1.TheTestSuite()
421 suite2 = module2.TheTestSuite()
422 alltests = unittest.TestSuite([suite1, suite2])
424 You can place the definitions of test cases and test suites in the same modules
425 as the code they are to test (such as :file:`widget.py`), but there are several
426 advantages to placing the test code in a separate module, such as
427 :file:`test_widget.py`:
429 * The test module can be run standalone from the command line.
431 * The test code can more easily be separated from shipped code.
433 * There is less temptation to change test code to fit the code it tests without
436 * Test code should be modified much less frequently than the code it tests.
438 * Tested code can be refactored more easily.
440 * Tests for modules written in C must be in separate modules anyway, so why not
443 * If the testing strategy changes, there is no need to change the source code.
446 .. _legacy-unit-tests:
448 Re-using old test code
449 ----------------------
451 Some users will find that they have existing test code that they would like to
452 run from :mod:`unittest`, without converting every old test function to a
453 :class:`TestCase` subclass.
455 For this reason, :mod:`unittest` provides a :class:`FunctionTestCase` class.
456 This subclass of :class:`TestCase` can be used to wrap an existing test
457 function. Set-up and tear-down functions can also be provided.
459 Given the following test function::
462 something = makeSomething()
463 assert something.name is not None
466 one can create an equivalent test case instance as follows::
468 testcase = unittest.FunctionTestCase(testSomething)
470 If there are additional set-up and tear-down methods that should be called as
471 part of the test case's operation, they can also be provided like so::
473 testcase = unittest.FunctionTestCase(testSomething,
474 setUp=makeSomethingDB,
475 tearDown=deleteSomethingDB)
477 To make migrating existing test suites easier, :mod:`unittest` supports tests
478 raising :exc:`AssertionError` to indicate test failure. However, it is
479 recommended that you use the explicit :meth:`TestCase.fail\*` and
480 :meth:`TestCase.assert\*` methods instead, as future versions of :mod:`unittest`
481 may treat :exc:`AssertionError` differently.
485 Even though :class:`FunctionTestCase` can be used to quickly convert an
486 existing test base over to a :mod:`unittest`\ -based system, this approach is
487 not recommended. Taking the time to set up proper :class:`TestCase`
488 subclasses will make future test refactorings infinitely easier.
490 In some cases, the existing tests may have been written using the :mod:`doctest`
491 module. If so, :mod:`doctest` provides a :class:`DocTestSuite` class that can
492 automatically build :class:`unittest.TestSuite` instances from the existing
493 :mod:`doctest`\ -based tests.
496 .. _unittest-skipping:
498 Skipping tests and expected failures
499 ------------------------------------
501 Unittest supports skipping individual test methods and even whole classes of
502 tests. In addition, it supports marking a test as a "expected failure," a test
503 that is broken and will fail, but shouldn't be counted as a failure on a
506 Skipping a test is simply a matter of using the :func:`skip` :term:`decorator`
507 or one of its conditional variants.
509 Basic skipping looks like this: ::
511 class MyTestCase(unittest.TestCase):
513 @unittest.skip("demonstrating skipping")
514 def test_nothing(self):
515 self.fail("shouldn't happen")
517 @unittest.skipIf(mylib.__version__ < (1, 3),
518 "not supported in this library version")
519 def test_format(self):
520 # Tests that work for only a certain version of the library.
523 @unittest.skipUnless(sys.platform.startswith("win"), "requires Windows")
524 def test_windows_support(self):
525 # windows specific testing code
528 This is the output of running the example above in verbose mode: ::
530 test_format (__main__.MyTestCase) ... skipped 'not supported in this library version'
531 test_nothing (__main__.MyTestCase) ... skipped 'demonstrating skipping'
532 test_windows_support (__main__.MyTestCase) ... skipped 'requires Windows'
534 ----------------------------------------------------------------------
535 Ran 3 tests in 0.005s
539 Classes can be skipped just like methods: ::
541 @skip("showing class skipping")
542 class MySkippedTestCase(unittest.TestCase):
543 def test_not_run(self):
546 :meth:`TestCase.setUp` can also skip the test. This is useful when a resource
547 that needs to be set up is not available.
549 Expected failures use the :func:`expectedFailure` decorator. ::
551 class ExpectedFailureTestCase(unittest.TestCase):
552 @unittest.expectedFailure
554 self.assertEqual(1, 0, "broken")
556 It's easy to roll your own skipping decorators by making a decorator that calls
557 :func:`skip` on the test when it wants it to be skipped. This decorator skips
558 the test unless the passed object has a certain attribute: ::
560 def skipUnlessHasattr(obj, attr):
561 if hasattr(obj, attr):
562 return lambda func: func
563 return unittest.skip("{0!r} doesn't have {1!r}".format(obj, attr))
565 The following decorators implement test skipping and expected failures:
567 .. function:: skip(reason)
569 Unconditionally skip the decorated test. *reason* should describe why the
570 test is being skipped.
572 .. function:: skipIf(condition, reason)
574 Skip the decorated test if *condition* is true.
576 .. function:: skipUnless(condition, reason)
578 Skip the decoratored test unless *condition* is true.
580 .. function:: expectedFailure
582 Mark the test as an expected failure. If the test fails when run, the test
583 is not counted as a failure.
586 .. _unittest-contents:
588 Classes and functions
589 ---------------------
591 This section describes in depth the API of :mod:`unittest`.
594 .. _testcase-objects:
599 .. class:: TestCase([methodName])
601 Instances of the :class:`TestCase` class represent the smallest testable units
602 in the :mod:`unittest` universe. This class is intended to be used as a base
603 class, with specific tests being implemented by concrete subclasses. This class
604 implements the interface needed by the test runner to allow it to drive the
605 test, and methods that the test code can use to check for and report various
608 Each instance of :class:`TestCase` will run a single test method: the method
609 named *methodName*. If you remember, we had an earlier example that went
610 something like this::
613 suite = unittest.TestSuite()
614 suite.addTest(WidgetTestCase('testDefaultSize'))
615 suite.addTest(WidgetTestCase('testResize'))
618 Here, we create two instances of :class:`WidgetTestCase`, each of which runs a
621 *methodName* defaults to :meth:`runTest`.
623 :class:`TestCase` instances provide three groups of methods: one group used
624 to run the test, another used by the test implementation to check conditions
625 and report failures, and some inquiry methods allowing information about the
626 test itself to be gathered.
628 Methods in the first group (running the test) are:
633 Method called to prepare the test fixture. This is called immediately
634 before calling the test method; any exception raised by this method will
635 be considered an error rather than a test failure. The default
636 implementation does nothing.
639 .. method:: tearDown()
641 Method called immediately after the test method has been called and the
642 result recorded. This is called even if the test method raised an
643 exception, so the implementation in subclasses may need to be particularly
644 careful about checking internal state. Any exception raised by this
645 method will be considered an error rather than a test failure. This
646 method will only be called if the :meth:`setUp` succeeds, regardless of
647 the outcome of the test method. The default implementation does nothing.
650 .. method:: run([result])
652 Run the test, collecting the result into the test result object passed as
653 *result*. If *result* is omitted or :const:`None`, a temporary result
654 object is created (by calling the :meth:`defaultTestCase` method) and
655 used; this result object is not returned to :meth:`run`'s caller.
657 The same effect may be had by simply calling the :class:`TestCase`
661 .. method:: skipTest(reason)
663 Calling this during the a test method or :meth:`setUp` skips the current
664 test. See :ref:`unittest-skipping` for more information.
669 Run the test without collecting the result. This allows exceptions raised
670 by the test to be propagated to the caller, and can be used to support
671 running tests under a debugger.
673 The test code can use any of the following methods to check for and report
677 .. method:: assertTrue(expr[, msg])
679 failUnless(expr[, msg])
681 Signal a test failure if *expr* is false; the explanation for the failure
682 will be *msg* if given, otherwise it will be :const:`None`.
688 .. method:: assertEqual(first, second[, msg])
689 failUnlessEqual(first, second[, msg])
691 Test that *first* and *second* are equal. If the values do not compare
692 equal, the test will fail with the explanation given by *msg*, or
693 :const:`None`. Note that using :meth:`assertEqual` improves upon
694 doing the comparison as the first parameter to :meth:`assertTrue`: the
695 default value for *msg* include representations of both *first* and
698 In addition, if *first* and *second* are the exact same type and one of
699 list, tuple, dict, set, or frozenset or any type that a subclass
700 registers :meth:`addTypeEqualityFunc` the type specific equality function
701 will be called in order to generate a more useful default error message.
703 .. versionchanged:: 2.7
704 Added the automatic calling of type specific equality function.
707 :meth:`failUnlessEqual`.
710 .. method:: assertNotEqual(first, second[, msg])
711 failIfEqual(first, second[, msg])
713 Test that *first* and *second* are not equal. If the values do compare
714 equal, the test will fail with the explanation given by *msg*, or
715 :const:`None`. Note that using :meth:`assertNotEqual` improves upon doing
716 the comparison as the first parameter to :meth:`assertTrue` is that the
717 default value for *msg* can be computed to include representations of both
718 *first* and *second*.
724 .. method:: assertAlmostEqual(first, second[, places[, msg]])
725 failUnlessAlmostEqual(first, second[, places[, msg]])
727 Test that *first* and *second* are approximately equal by computing the
728 difference, rounding to the given number of decimal *places* (default 7),
729 and comparing to zero.
731 Note that comparing a given number of decimal places is not the same as
732 comparing a given number of significant digits. If the values do not
733 compare equal, the test will fail with the explanation given by *msg*, or
737 :meth:`failUnlessAlmostEqual`.
740 .. method:: assertNotAlmostEqual(first, second[, places[, msg]])
741 failIfAlmostEqual(first, second[, places[, msg]])
743 Test that *first* and *second* are not approximately equal by computing
744 the difference, rounding to the given number of decimal *places* (default
745 7), and comparing to zero.
747 Note that comparing a given number of decimal places is not the same as
748 comparing a given number of significant digits. If the values do not
749 compare equal, the test will fail with the explanation given by *msg*, or
753 :meth:`failIfAlmostEqual`.
756 .. method:: assertGreater(first, second, msg=None)
757 assertGreaterEqual(first, second, msg=None)
758 assertLess(first, second, msg=None)
759 assertLessEqual(first, second, msg=None)
761 Test that *first* is respectively >, >=, < or <= than *second* depending
762 on the method name. If not, the test will fail with an explanation
763 or with the explanation given by *msg*::
765 >>> self.assertGreaterEqual(3, 4)
766 AssertionError: "3" unexpectedly not greater than or equal to "4"
768 .. versionadded:: 2.7
771 .. method:: assertMultiLineEqual(self, first, second, msg=None)
773 Test that the multiline string *first* is equal to the string *second*.
774 When not equal a diff of the two strings highlighting the differences
775 will be included in the error message.
777 If specified *msg* will be used as the error message on failure.
779 .. versionadded:: 2.7
782 .. method:: assertRegexpMatches(text, regexp[, msg=None]):
784 Verifies that a *regexp* search matches *text*. Fails with an error
785 message including the pattern and the *text*. *regexp* may be
786 a regular expression object or a string containing a regular expression
787 suitable for use by :func:`re.search`.
789 .. versionadded:: 2.7
792 .. method:: assertIn(first, second, msg=None)
793 assertNotIn(first, second, msg=None)
795 Tests that *first* is or is not in *second* with an explanatory error
796 message as appropriate.
798 If specified *msg* will be used as the error message on failure.
800 .. versionadded:: 2.7
803 .. method:: assertSameElements(expected, actual, msg=None)
805 Test that sequence *expected* contains the same elements as *actual*.
806 When they don't an error message listing the differences between the
807 sequences will be generated.
809 If specified *msg* will be used as the error message on failure.
811 .. versionadded:: 2.7
814 .. method:: assertSetEqual(set1, set2, msg=None)
816 Tests that two sets are equal. If not, an error message is constructed
817 that lists the differences between the sets.
819 Fails if either of *set1* or *set2* does not have a :meth:`set.difference`
822 If specified *msg* will be used as the error message on failure.
824 .. versionadded:: 2.7
827 .. method:: assertDictEqual(expected, actual, msg=None)
829 Test that two dictionaries are equal. If not, an error message is
830 constructed that shows the differences in the dictionaries.
832 If specified *msg* will be used as the error message on failure.
834 .. versionadded:: 2.7
837 .. method:: assertDictContainsSubset(expected, actual, msg=None)
839 Tests whether the key/value pairs in dictionary *actual* are a
840 superset of those in *expected*. If not, an error message listing
841 the missing keys and mismatched values is generated.
843 If specified *msg* will be used as the error message on failure.
845 .. versionadded:: 2.7
848 .. method:: assertListEqual(list1, list2, msg=None)
849 assertTupleEqual(tuple1, tuple2, msg=None)
851 Tests that two lists or tuples are equal. If not an error message is
852 constructed that shows only the differences between the two. An error
853 is also raised if either of the parameters are of the wrong type.
855 If specified *msg* will be used as the error message on failure.
857 .. versionadded:: 2.7
860 .. method:: assertSequenceEqual(seq1, seq2, msg=None, seq_type=None)
862 Tests that two sequences are equal. If a *seq_type* is supplied, both
863 *seq1* and *seq2* must be instances of *seq_type* or a failure will
864 be raised. If the sequences are different an error message is
865 constructed that shows the difference between the two.
867 If specified *msg* will be used as the error message on failure.
869 This method is used to implement :meth:`assertListEqual` and
870 :meth:`assertTupleEqual`.
872 .. versionadded:: 2.7
875 .. method:: assertRaises(exception[, callable, ...])
876 failUnlessRaises(exception[, callable, ...])
878 Test that an exception is raised when *callable* is called with any
879 positional or keyword arguments that are also passed to
880 :meth:`assertRaises`. The test passes if *exception* is raised, is an
881 error if another exception is raised, or fails if no exception is raised.
882 To catch any of a group of exceptions, a tuple containing the exception
883 classes may be passed as *exception*.
885 If *callable* is omitted or None, returns a context manager so that the
886 code under test can be written inline rather than as a function::
888 with self.failUnlessRaises(some_error_class):
891 .. versionchanged:: 2.7
892 Added the ability to use :meth:`assertRaises` as a context manager.
895 :meth:`failUnlessRaises`.
898 .. method:: assertRaisesRegexp(exception, regexp[, callable, ...])
900 Like :meth:`assertRaises` but also tests that *regexp* matches
901 on the string representation of the raised exception. *regexp* may be
902 a regular expression object or a string containing a regular expression
903 suitable for use by :func:`re.search`. Examples::
905 self.assertRaisesRegexp(ValueError, 'invalid literal for.*XYZ$',
910 with self.assertRaisesRegexp(ValueError, 'literal'):
913 .. versionadded:: 2.7
916 .. method:: assertIsNone(expr[, msg])
918 This signals a test failure if *expr* is not None.
920 .. versionadded:: 2.7
923 .. method:: assertIsNotNone(expr[, msg])
925 The inverse of the :meth:`assertIsNone` method.
926 This signals a test failure if *expr* is None.
928 .. versionadded:: 2.7
931 .. method:: assertIs(expr1, expr2[, msg])
933 This signals a test failure if *expr1* and *expr2* don't evaluate to the same
936 .. versionadded:: 2.7
939 .. method:: assertIsNot(expr1, expr2[, msg])
941 The inverse of the :meth:`assertIs` method.
942 This signals a test failure if *expr1* and *expr2* evaluate to the same
945 .. versionadded:: 2.7
948 .. method:: assertFalse(expr[, msg])
951 The inverse of the :meth:`assertTrue` method is the :meth:`assertFalse` method.
952 This signals a test failure if *expr* is true, with *msg* or :const:`None`
953 for the error message.
959 .. method:: fail([msg])
961 Signals a test failure unconditionally, with *msg* or :const:`None` for
965 .. attribute:: failureException
967 This class attribute gives the exception raised by the test method. If a
968 test framework needs to use a specialized exception, possibly to carry
969 additional information, it must subclass this exception in order to "play
970 fair" with the framework. The initial value of this attribute is
971 :exc:`AssertionError`.
974 .. attribute:: longMessage
976 If set to True then any explicit failure message you pass in to the
977 assert methods will be appended to the end of the normal failure message.
978 The normal messages contain useful information about the objects involved,
979 for example the message from assertEqual shows you the repr of the two
980 unequal objects. Setting this attribute to True allows you to have a
981 custom error message in addition to the normal one.
983 This attribute defaults to False, meaning that a custom message passed
984 to an assert method will silence the normal message.
986 The class setting can be overridden in individual tests by assigning an
987 instance attribute to True or False before calling the assert methods.
989 .. versionadded:: 2.7
992 Testing frameworks can use the following methods to collect information on
996 .. method:: countTestCases()
998 Return the number of tests represented by this test object. For
999 :class:`TestCase` instances, this will always be ``1``.
1002 .. method:: defaultTestResult()
1004 Return an instance of the test result class that should be used for this
1005 test case class (if no other result instance is provided to the
1006 :meth:`run` method).
1008 For :class:`TestCase` instances, this will always be an instance of
1009 :class:`TestResult`; subclasses of :class:`TestCase` should override this
1015 Return a string identifying the specific test case. This is usually the
1016 full name of the test method, including the module and class name.
1019 .. method:: shortDescription()
1021 Returns a description of the test, or :const:`None` if no description
1022 has been provided. The default implementation of this method
1023 returns the first line of the test method's docstring, if available,
1024 along with the method name.
1026 .. versionchanged:: 2.7
1027 In earlier versions this only returned the first line of the test
1028 method's docstring, if available or the :const:`None`. That led to
1029 undesirable behavior of not printing the test name when someone was
1030 thoughtful enough to write a docstring.
1033 .. method:: addTypeEqualityFunc(typeobj, function)
1035 Registers a type specific :meth:`assertEqual` equality checking
1036 function to be called by :meth:`assertEqual` when both objects it has
1037 been asked to compare are exactly *typeobj* (not subclasses).
1038 *function* must take two positional arguments and a third msg=None
1039 keyword argument just as :meth:`assertEqual` does. It must raise
1040 ``self.failureException`` when inequality between the first two
1041 parameters is detected.
1043 One good use of custom equality checking functions for a type
1044 is to raise ``self.failureException`` with an error message useful
1045 for debugging the problem by explaining the inequalities in detail.
1047 .. versionadded:: 2.7
1050 .. method:: addCleanup(function[, *args[, **kwargs]])
1052 Add a function to be called after :meth:`tearDown` to cleanup resources
1053 used during the test. Functions will be called in reverse order to the
1054 order they are added (LIFO). They are called with any arguments and
1055 keyword arguments passed into :meth:`addCleanup` when they are
1058 If :meth:`setUp` fails, meaning that :meth:`tearDown` is not called,
1059 then any cleanup functions added will still be called.
1061 .. versionadded:: 2.7
1064 .. method:: doCleanups()
1066 This method is called uncoditionally after :meth:`tearDown`, or
1067 after :meth:`setUp` if :meth:`setUp` raises an exception.
1069 It is responsible for calling all the cleanup functions added by
1070 :meth:`addCleanup`. If you need cleanup functions to be called
1071 *prior* to :meth:`tearDown` then you can call :meth:`doCleanups`
1074 :meth:`doCleanups` pops methods off the stack of cleanup
1075 functions one at a time, so it can be called at any time.
1077 .. versionadded:: 2.7
1080 .. class:: FunctionTestCase(testFunc[, setUp[, tearDown[, description]]])
1082 This class implements the portion of the :class:`TestCase` interface which
1083 allows the test runner to drive the test, but does not provide the methods
1084 which test code can use to check and report errors. This is used to create
1085 test cases using legacy test code, allowing it to be integrated into a
1086 :mod:`unittest`-based test framework.
1089 .. _testsuite-objects:
1094 .. class:: TestSuite([tests])
1096 This class represents an aggregation of individual tests cases and test suites.
1097 The class presents the interface needed by the test runner to allow it to be run
1098 as any other test case. Running a :class:`TestSuite` instance is the same as
1099 iterating over the suite, running each test individually.
1101 If *tests* is given, it must be an iterable of individual test cases or other
1102 test suites that will be used to build the suite initially. Additional methods
1103 are provided to add test cases and suites to the collection later on.
1105 :class:`TestSuite` objects behave much like :class:`TestCase` objects, except
1106 they do not actually implement a test. Instead, they are used to aggregate
1107 tests into groups of tests that should be run together. Some additional
1108 methods are available to add tests to :class:`TestSuite` instances:
1111 .. method:: TestSuite.addTest(test)
1113 Add a :class:`TestCase` or :class:`TestSuite` to the suite.
1116 .. method:: TestSuite.addTests(tests)
1118 Add all the tests from an iterable of :class:`TestCase` and :class:`TestSuite`
1119 instances to this test suite.
1121 This is equivalent to iterating over *tests*, calling :meth:`addTest` for
1124 :class:`TestSuite` shares the following methods with :class:`TestCase`:
1127 .. method:: run(result)
1129 Run the tests associated with this suite, collecting the result into the
1130 test result object passed as *result*. Note that unlike
1131 :meth:`TestCase.run`, :meth:`TestSuite.run` requires the result object to
1137 Run the tests associated with this suite without collecting the
1138 result. This allows exceptions raised by the test to be propagated to the
1139 caller and can be used to support running tests under a debugger.
1142 .. method:: countTestCases()
1144 Return the number of tests represented by this test object, including all
1145 individual tests and sub-suites.
1148 .. method:: __iter__()
1150 Tests grouped by a :class:`TestSuite` are always accessed by iteration.
1151 Subclasses can lazily provide tests by overriding :meth:`__iter__`. Note
1152 that this method maybe called several times on a single suite
1153 (for example when counting tests or comparing for equality)
1154 so the tests returned must be the same for repeated iterations.
1156 .. versionchanged:: 2.7
1157 In earlier versions the :class:`TestSuite` accessed tests directly rather
1158 than through iteration, so overriding :meth:`__iter__` wasn't sufficient
1159 for providing tests.
1161 In the typical usage of a :class:`TestSuite` object, the :meth:`run` method
1162 is invoked by a :class:`TestRunner` rather than by the end-user test harness.
1165 Loading and running tests
1166 ~~~~~~~~~~~~~~~~~~~~~~~~~
1168 .. class:: TestLoader()
1170 The :class:`TestLoader` class is used to create test suites from classes and
1171 modules. Normally, there is no need to create an instance of this class; the
1172 :mod:`unittest` module provides an instance that can be shared as
1173 ``unittest.defaultTestLoader``. Using a subclass or instance, however, allows
1174 customization of some configurable properties.
1176 :class:`TestLoader` objects have the following methods:
1179 .. method:: loadTestsFromTestCase(testCaseClass)
1181 Return a suite of all tests cases contained in the :class:`TestCase`\ -derived
1182 :class:`testCaseClass`.
1185 .. method:: loadTestsFromModule(module)
1187 Return a suite of all tests cases contained in the given module. This
1188 method searches *module* for classes derived from :class:`TestCase` and
1189 creates an instance of the class for each test method defined for the
1194 While using a hierarchy of :class:`TestCase`\ -derived classes can be
1195 convenient in sharing fixtures and helper functions, defining test
1196 methods on base classes that are not intended to be instantiated
1197 directly does not play well with this method. Doing so, however, can
1198 be useful when the fixtures are different and defined in subclasses.
1200 If a module provides a ``load_tests`` function it will be called to
1201 load the tests. This allows modules to customize test loading.
1202 This is the `load_tests protocol`_.
1204 .. versionchanged:: 2.7
1205 Support for ``load_tests`` added.
1208 .. method:: loadTestsFromName(name[, module])
1210 Return a suite of all tests cases given a string specifier.
1212 The specifier *name* is a "dotted name" that may resolve either to a
1213 module, a test case class, a test method within a test case class, a
1214 :class:`TestSuite` instance, or a callable object which returns a
1215 :class:`TestCase` or :class:`TestSuite` instance. These checks are
1216 applied in the order listed here; that is, a method on a possible test
1217 case class will be picked up as "a test method within a test case class",
1218 rather than "a callable object".
1220 For example, if you have a module :mod:`SampleTests` containing a
1221 :class:`TestCase`\ -derived class :class:`SampleTestCase` with three test
1222 methods (:meth:`test_one`, :meth:`test_two`, and :meth:`test_three`), the
1223 specifier ``'SampleTests.SampleTestCase'`` would cause this method to
1224 return a suite which will run all three test methods. Using the specifier
1225 ``'SampleTests.SampleTestCase.test_two'`` would cause it to return a test
1226 suite which will run only the :meth:`test_two` test method. The specifier
1227 can refer to modules and packages which have not been imported; they will
1228 be imported as a side-effect.
1230 The method optionally resolves *name* relative to the given *module*.
1233 .. method:: loadTestsFromNames(names[, module])
1235 Similar to :meth:`loadTestsFromName`, but takes a sequence of names rather
1236 than a single name. The return value is a test suite which supports all
1237 the tests defined for each name.
1240 .. method:: getTestCaseNames(testCaseClass)
1242 Return a sorted sequence of method names found within *testCaseClass*;
1243 this should be a subclass of :class:`TestCase`.
1246 .. method:: discover(start_dir, pattern='test*.py', top_level_dir=None)
1248 Find and return all test modules from the specified start directory,
1249 recursing into subdirectories to find them. Only test files that match
1250 *pattern* will be loaded. (Using shell style pattern matching.)
1252 All test modules must be importable from the top level of the project. If
1253 the start directory is not the top level directory then the top level
1254 directory must be specified separately.
1256 If a test package name (directory with :file:`__init__.py`) matches the
1257 pattern then the package will be checked for a ``load_tests``
1258 function. If this exists then it will be called with *loader*, *tests*,
1261 If load_tests exists then discovery does *not* recurse into the package,
1262 ``load_tests`` is responsible for loading all tests in the package.
1264 The pattern is deliberately not stored as a loader attribute so that
1265 packages can continue discovery themselves. *top_level_dir* is stored so
1266 ``load_tests`` does not need to pass this argument in to
1267 ``loader.discover()``.
1270 The following attributes of a :class:`TestLoader` can be configured either by
1271 subclassing or assignment on an instance:
1274 .. attribute:: testMethodPrefix
1276 String giving the prefix of method names which will be interpreted as test
1277 methods. The default value is ``'test'``.
1279 This affects :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*`
1283 .. attribute:: sortTestMethodsUsing
1285 Function to be used to compare method names when sorting them in
1286 :meth:`getTestCaseNames` and all the :meth:`loadTestsFrom\*` methods. The
1287 default value is the built-in :func:`cmp` function; the attribute can also
1288 be set to :const:`None` to disable the sort.
1291 .. attribute:: suiteClass
1293 Callable object that constructs a test suite from a list of tests. No
1294 methods on the resulting object are needed. The default value is the
1295 :class:`TestSuite` class.
1297 This affects all the :meth:`loadTestsFrom\*` methods.
1300 .. class:: TestResult
1302 This class is used to compile information about which tests have succeeded
1303 and which have failed.
1305 A :class:`TestResult` object stores the results of a set of tests. The
1306 :class:`TestCase` and :class:`TestSuite` classes ensure that results are
1307 properly recorded; test authors do not need to worry about recording the
1310 Testing frameworks built on top of :mod:`unittest` may want access to the
1311 :class:`TestResult` object generated by running a set of tests for reporting
1312 purposes; a :class:`TestResult` instance is returned by the
1313 :meth:`TestRunner.run` method for this purpose.
1315 :class:`TestResult` instances have the following attributes that will be of
1316 interest when inspecting the results of running a set of tests:
1319 .. attribute:: errors
1321 A list containing 2-tuples of :class:`TestCase` instances and strings
1322 holding formatted tracebacks. Each tuple represents a test which raised an
1323 unexpected exception.
1325 .. versionchanged:: 2.2
1326 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1329 .. attribute:: failures
1331 A list containing 2-tuples of :class:`TestCase` instances and strings
1332 holding formatted tracebacks. Each tuple represents a test where a failure
1333 was explicitly signalled using the :meth:`TestCase.fail\*` or
1334 :meth:`TestCase.assert\*` methods.
1336 .. versionchanged:: 2.2
1337 Contains formatted tracebacks instead of :func:`sys.exc_info` results.
1339 .. attribute:: skipped
1341 A list containing 2-tuples of :class:`TestCase` instances and strings
1342 holding the reason for skipping the test.
1344 .. versionadded:: 2.7
1346 .. attribute:: expectedFailures
1348 A list contaning 2-tuples of :class:`TestCase` instances and strings
1349 holding formatted tracebacks. Each tuple represents a expected failures
1352 .. attribute:: unexpectedSuccesses
1354 A list containing :class:`TestCase` instances that were marked as expected
1355 failures, but succeeded.
1357 .. attribute:: shouldStop
1359 Set to ``True`` when the execution of tests should stop by :meth:`stop`.
1362 .. attribute:: testsRun
1364 The total number of tests run so far.
1367 .. method:: wasSuccessful()
1369 Return :const:`True` if all tests run so far have passed, otherwise returns
1375 This method can be called to signal that the set of tests being run should
1376 be aborted by setting the :attr:`shouldStop` attribute to :const:`True`.
1377 :class:`TestRunner` objects should respect this flag and return without
1378 running any additional tests.
1380 For example, this feature is used by the :class:`TextTestRunner` class to
1381 stop the test framework when the user signals an interrupt from the
1382 keyboard. Interactive tools which provide :class:`TestRunner`
1383 implementations can use this in a similar manner.
1385 The following methods of the :class:`TestResult` class are used to maintain
1386 the internal data structures, and may be extended in subclasses to support
1387 additional reporting requirements. This is particularly useful in building
1388 tools which support interactive reporting while tests are being run.
1391 .. method:: startTest(test)
1393 Called when the test case *test* is about to be run.
1395 The default implementation simply increments the instance's :attr:`testsRun`
1399 .. method:: stopTest(test)
1401 Called after the test case *test* has been executed, regardless of the
1404 The default implementation does nothing.
1407 .. method:: startTestRun(test)
1409 Called once before any tests are executed.
1411 .. versionadded:: 2.7
1414 .. method:: stopTestRun(test)
1416 Called once before any tests are executed.
1418 .. versionadded:: 2.7
1421 .. method:: addError(test, err)
1423 Called when the test case *test* raises an unexpected exception *err* is a
1424 tuple of the form returned by :func:`sys.exc_info`: ``(type, value,
1427 The default implementation appends a tuple ``(test, formatted_err)`` to
1428 the instance's :attr:`errors` attribute, where *formatted_err* is a
1429 formatted traceback derived from *err*.
1432 .. method:: addFailure(test, err)
1434 Called when the test case *test* signals a failure. *err* is a tuple of
1435 the form returned by :func:`sys.exc_info`: ``(type, value, traceback)``.
1437 The default implementation appends a tuple ``(test, formatted_err)`` to
1438 the instance's :attr:`failures` attribute, where *formatted_err* is a
1439 formatted traceback derived from *err*.
1442 .. method:: addSuccess(test)
1444 Called when the test case *test* succeeds.
1446 The default implementation does nothing.
1449 .. method:: addSkip(test, reason)
1451 Called when the test case *test* is skipped. *reason* is the reason the
1452 test gave for skipping.
1454 The default implementation appends a tuple ``(test, reason)`` to the
1455 instance's :attr:`skipped` attribute.
1458 .. method:: addExpectedFailure(test, err)
1460 Called when the test case *test* fails, but was marked with the
1461 :func:`expectedFailure` decorator.
1463 The default implementation appends a tuple ``(test, formatted_err)`` to
1464 the instance's :attr:`expectedFailures` attribute, where *formatted_err*
1465 is a formatted traceback derived from *err*.
1468 .. method:: addUnexpectedSuccess(test)
1470 Called when the test case *test* was marked with the
1471 :func:`expectedFailure` decorator, but succeeded.
1473 The default implementation appends the test to the instance's
1474 :attr:`unexpectedSuccesses` attribute.
1477 .. data:: defaultTestLoader
1479 Instance of the :class:`TestLoader` class intended to be shared. If no
1480 customization of the :class:`TestLoader` is needed, this instance can be used
1481 instead of repeatedly creating new instances.
1484 .. class:: TextTestRunner([stream[, descriptions[, verbosity]]])
1486 A basic test runner implementation which prints results on standard error. It
1487 has a few configurable parameters, but is essentially very simple. Graphical
1488 applications which run test suites should provide alternate implementations.
1490 .. method:: _makeResult()
1492 This method returns the instance of ``TestResult`` used by :meth:`run`.
1493 It is not intended to be called directly, but can be overridden in
1494 subclasses to provide a custom ``TestResult``.
1497 .. function:: main([module[, defaultTest[, argv[, testRunner[, testLoader[, exit, [verbosity]]]]]]])
1499 A command-line program that runs a set of tests; this is primarily for making
1500 test modules conveniently executable. The simplest use for this function is to
1501 include the following line at the end of a test script::
1503 if __name__ == '__main__':
1506 You can run tests with more detailed information by passing in the verbosity
1509 if __name__ == '__main__':
1510 unittest.main(verbosity=2)
1512 The *testRunner* argument can either be a test runner class or an already
1513 created instance of it. By default ``main`` calls :func:`sys.exit` with
1514 an exit code indicating success or failure of the tests run.
1516 ``main`` supports being used from the interactive interpreter by passing in the
1517 argument ``exit=False``. This displays the result on standard output without
1518 calling :func:`sys.exit`::
1520 >>> from unittest import main
1521 >>> main(module='test_module', exit=False)
1523 Calling ``main`` actually returns an instance of the ``TestProgram`` class.
1524 This stores the result of the tests run as the ``result`` attribute.
1526 .. versionchanged:: 2.7
1527 The ``exit`` and ``verbosity`` parameters were added.
1533 Modules or packages can customize how tests are loaded from them during normal
1534 test runs or test discovery by implementing a function called ``load_tests``.
1536 If a test module defines ``load_tests`` it will be called by
1537 :meth:`TestLoader.loadTestsFromModule` with the following arguments::
1539 load_tests(loader, standard_tests, None)
1541 It should return a :class:`TestSuite`.
1543 *loader* is the instance of :class:`TestLoader` doing the loading.
1544 *standard_tests* are the tests that would be loaded by default from the
1545 module. It is common for test modules to only want to add or remove tests
1546 from the standard set of tests.
1547 The third argument is used when loading packages as part of test discovery.
1549 A typical ``load_tests`` function that loads tests from a specific set of
1550 :class:`TestCase` classes may look like::
1552 test_cases = (TestCase1, TestCase2, TestCase3)
1554 def load_tests(loader, tests, pattern):
1556 for test_class in test_cases:
1557 tests = loader.loadTestsFromTestCase(test_class)
1558 suite.addTests(tests)
1561 If discovery is started, either from the command line or by calling
1562 :meth:`TestLoader.discover`, with a pattern that matches a package
1563 name then the package :file:`__init__.py` will be checked for ``load_tests``.
1567 The default pattern is 'test*.py'. This matches all python files
1568 that start with 'test' but *won't* match any test directories.
1570 A pattern like 'test*' will match test packages as well as
1573 If the package :file:`__init__.py` defines ``load_tests`` then it will be
1574 called and discovery not continued into the package. ``load_tests``
1575 is called with the following arguments::
1577 load_tests(loader, standard_tests, pattern)
1579 This should return a :class:`TestSuite` representing all the tests
1580 from the package. (``standard_tests`` will only contain tests
1581 collected from :file:`__init__.py`.)
1583 Because the pattern is passed into ``load_tests`` the package is free to
1584 continue (and potentially modify) test discovery. A 'do nothing'
1585 ``load_tests`` function for a test package would look like::
1587 def load_tests(loader, standard_tests, pattern):
1588 # top level directory cached on loader instance
1589 this_dir = os.path.dirname(__file__)
1590 package_tests = loader.discover(start_dir=this_dir, pattern=pattern)
1591 standard_tests.addTests(package_tests)
1592 return standard_tests