1 ****************************
2 What's New in Python 2.4
3 ****************************
7 .. |release| replace:: 1.02
9 .. $Id: whatsnew24.tex 54632 2007-03-31 11:59:54Z georg.brandl $
10 .. Don't write extensive text for new sections; I'll do that.
11 .. Feel free to add commented-out reminders of things that need
12 .. to be covered. --amk
14 This article explains the new features in Python 2.4.1, released on March 30,
17 Python 2.4 is a medium-sized release. It doesn't introduce as many changes as
18 the radical Python 2.2, but introduces more features than the conservative 2.3
19 release. The most significant new language features are function decorators and
20 generator expressions; most other changes are to the standard library.
22 According to the CVS change logs, there were 481 patches applied and 502 bugs
23 fixed between Python 2.3 and 2.4. Both figures are likely to be underestimates.
25 This article doesn't attempt to provide a complete specification of every single
26 new feature, but instead provides a brief introduction to each feature. For
27 full details, you should refer to the documentation for Python 2.4, such as the
28 Python Library Reference and the Python Reference Manual. Often you will be
29 referred to the PEP for a particular new feature for explanations of the
30 implementation and design rationale.
32 .. ======================================================================
35 PEP 218: Built-In Set Objects
36 =============================
38 Python 2.3 introduced the :mod:`sets` module. C implementations of set data
39 types have now been added to the Python core as two new built-in types,
40 :func:`set(iterable)` and :func:`frozenset(iterable)`. They provide high speed
41 operations for membership testing, for eliminating duplicates from sequences,
42 and for mathematical operations like unions, intersections, differences, and
43 symmetric differences. ::
45 >>> a = set('abracadabra') # form a set from a string
46 >>> 'z' in a # fast membership testing
48 >>> a # unique letters in a
49 set(['a', 'r', 'b', 'c', 'd'])
50 >>> ''.join(a) # convert back into a string
53 >>> b = set('alacazam') # form a second set
54 >>> a - b # letters in a but not in b
56 >>> a | b # letters in either a or b
57 set(['a', 'c', 'r', 'd', 'b', 'm', 'z', 'l'])
58 >>> a & b # letters in both a and b
60 >>> a ^ b # letters in a or b but not both
61 set(['r', 'd', 'b', 'm', 'z', 'l'])
63 >>> a.add('z') # add a new element
64 >>> a.update('wxy') # add multiple new elements
66 set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'x', 'z'])
67 >>> a.remove('x') # take one element out
69 set(['a', 'c', 'b', 'd', 'r', 'w', 'y', 'z'])
71 The :func:`frozenset` type is an immutable version of :func:`set`. Since it is
72 immutable and hashable, it may be used as a dictionary key or as a member of
75 The :mod:`sets` module remains in the standard library, and may be useful if you
76 wish to subclass the :class:`Set` or :class:`ImmutableSet` classes. There are
77 currently no plans to deprecate the module.
82 :pep:`218` - Adding a Built-In Set Object Type
83 Originally proposed by Greg Wilson and ultimately implemented by Raymond
86 .. ======================================================================
89 PEP 237: Unifying Long Integers and Integers
90 ============================================
92 The lengthy transition process for this PEP, begun in Python 2.2, takes another
93 step forward in Python 2.4. In 2.3, certain integer operations that would
94 behave differently after int/long unification triggered :exc:`FutureWarning`
95 warnings and returned values limited to 32 or 64 bits (depending on your
96 platform). In 2.4, these expressions no longer produce a warning and instead
97 produce a different result that's usually a long integer.
99 The problematic expressions are primarily left shifts and lengthy hexadecimal
100 and octal constants. For example, ``2 << 32`` results in a warning in 2.3,
101 evaluating to 0 on 32-bit platforms. In Python 2.4, this expression now returns
102 the correct answer, 8589934592.
107 :pep:`237` - Unifying Long Integers and Integers
108 Original PEP written by Moshe Zadka and GvR. The changes for 2.4 were
109 implemented by Kalle Svensson.
111 .. ======================================================================
114 PEP 289: Generator Expressions
115 ==============================
117 The iterator feature introduced in Python 2.2 and the :mod:`itertools` module
118 make it easier to write programs that loop through large data sets without
119 having the entire data set in memory at one time. List comprehensions don't fit
120 into this picture very well because they produce a Python list object containing
121 all of the items. This unavoidably pulls all of the objects into memory, which
122 can be a problem if your data set is very large. When trying to write a
123 functionally-styled program, it would be natural to write something like::
125 links = [link for link in get_all_links() if not link.followed]
131 for link in get_all_links():
136 The first form is more concise and perhaps more readable, but if you're dealing
137 with a large number of link objects you'd have to write the second form to avoid
138 having all link objects in memory at the same time.
140 Generator expressions work similarly to list comprehensions but don't
141 materialize the entire list; instead they create a generator that will return
142 elements one by one. The above example could be written as::
144 links = (link for link in get_all_links() if not link.followed)
148 Generator expressions always have to be written inside parentheses, as in the
149 above example. The parentheses signalling a function call also count, so if you
150 want to create an iterator that will be immediately passed to a function you
153 print sum(obj.count for obj in list_all_objects())
155 Generator expressions differ from list comprehensions in various small ways.
156 Most notably, the loop variable (*obj* in the above example) is not accessible
157 outside of the generator expression. List comprehensions leave the variable
158 assigned to its last value; future versions of Python will change this, making
159 list comprehensions match generator expressions in this respect.
164 :pep:`289` - Generator Expressions
165 Proposed by Raymond Hettinger and implemented by Jiwon Seo with early efforts
166 steered by Hye-Shik Chang.
168 .. ======================================================================
171 PEP 292: Simpler String Substitutions
172 =====================================
174 Some new classes in the standard library provide an alternative mechanism for
175 substituting variables into strings; this style of substitution may be better
176 for applications where untrained users need to edit templates.
178 The usual way of substituting variables by name is the ``%`` operator::
180 >>> '%(page)i: %(title)s' % {'page':2, 'title': 'The Best of Times'}
181 '2: The Best of Times'
183 When writing the template string, it can be easy to forget the ``i`` or ``s``
184 after the closing parenthesis. This isn't a big problem if the template is in a
185 Python module, because you run the code, get an "Unsupported format character"
186 :exc:`ValueError`, and fix the problem. However, consider an application such
187 as Mailman where template strings or translations are being edited by users who
188 aren't aware of the Python language. The format string's syntax is complicated
189 to explain to such users, and if they make a mistake, it's difficult to provide
190 helpful feedback to them.
192 PEP 292 adds a :class:`Template` class to the :mod:`string` module that uses
193 ``$`` to indicate a substitution::
196 >>> t = string.Template('$page: $title')
197 >>> t.substitute({'page':2, 'title': 'The Best of Times'})
198 '2: The Best of Times'
200 If a key is missing from the dictionary, the :meth:`substitute` method will
201 raise a :exc:`KeyError`. There's also a :meth:`safe_substitute` method that
202 ignores missing keys::
204 >>> t = string.Template('$page: $title')
205 >>> t.safe_substitute({'page':3})
211 :pep:`292` - Simpler String Substitutions
212 Written and implemented by Barry Warsaw.
214 .. ======================================================================
217 PEP 318: Decorators for Functions and Methods
218 =============================================
220 Python 2.2 extended Python's object model by adding static methods and class
221 methods, but it didn't extend Python's syntax to provide any new way of defining
222 static or class methods. Instead, you had to write a :keyword:`def` statement
223 in the usual way, and pass the resulting method to a :func:`staticmethod` or
224 :func:`classmethod` function that would wrap up the function as a method of the
225 new type. Your code would look like this::
231 meth = classmethod(meth) # Rebind name to wrapped-up class method
233 If the method was very long, it would be easy to miss or forget the
234 :func:`classmethod` invocation after the function body.
236 The intention was always to add some syntax to make such definitions more
237 readable, but at the time of 2.2's release a good syntax was not obvious. Today
238 a good syntax *still* isn't obvious but users are asking for easier access to
239 the feature; a new syntactic feature has been added to meet this need.
241 The new feature is called "function decorators". The name comes from the idea
242 that :func:`classmethod`, :func:`staticmethod`, and friends are storing
243 additional information on a function object; they're *decorating* functions with
246 The notation borrows from Java and uses the ``'@'`` character as an indicator.
247 Using the new syntax, the example above would be written::
256 The ``@classmethod`` is shorthand for the ``meth=classmethod(meth)`` assignment.
257 More generally, if you have the following::
265 It's equivalent to the following pre-decorator code::
270 Decorators must come on the line before a function definition, one decorator per
271 line, and can't be on the same line as the def statement, meaning that ``@A def
272 f(): ...`` is illegal. You can only decorate function definitions, either at
273 the module level or inside a class; you can't decorate class definitions.
275 A decorator is just a function that takes the function to be decorated as an
276 argument and returns either the same function or some new object. The return
277 value of the decorator need not be callable (though it typically is), unless
278 further decorators will be applied to the result. It's easy to write your own
279 decorators. The following simple example just sets an attribute on the function
283 ... func.attr = 'decorated'
290 <function f at 0x402ef0d4>
295 As a slightly more realistic example, the following decorator checks that the
296 supplied argument is an integer::
298 def require_int (func):
300 assert isinstance(arg, int)
313 An example in :pep:`318` contains a fancier version of this idea that lets you
314 both specify the required type and check the returned type.
316 Decorator functions can take arguments. If arguments are supplied, your
317 decorator function is called with only those arguments and must return a new
318 decorator function; this function must take a single function and return a
319 function, as previously described. In other words, ``@A @B @C(args)`` becomes::
325 Getting this right can be slightly brain-bending, but it's not too difficult.
327 A small related change makes the :attr:`func_name` attribute of functions
328 writable. This attribute is used to display function names in tracebacks, so
329 decorators should change the name of any new function that's constructed and
335 :pep:`318` - Decorators for Functions, Methods and Classes
336 Written by Kevin D. Smith, Jim Jewett, and Skip Montanaro. Several people
337 wrote patches implementing function decorators, but the one that was actually
338 checked in was patch #979728, written by Mark Russell.
340 http://www.python.org/moin/PythonDecoratorLibrary
341 This Wiki page contains several examples of decorators.
343 .. ======================================================================
346 PEP 322: Reverse Iteration
347 ==========================
349 A new built-in function, :func:`reversed(seq)`, takes a sequence and returns an
350 iterator that loops over the elements of the sequence in reverse order. ::
352 >>> for i in reversed(xrange(1,4)):
359 Compared to extended slicing, such as ``range(1,4)[::-1]``, :func:`reversed` is
360 easier to read, runs faster, and uses substantially less memory.
362 Note that :func:`reversed` only accepts sequences, not arbitrary iterators. If
363 you want to reverse an iterator, first convert it to a list with :func:`list`.
366 >>> input = open('/etc/passwd', 'r')
367 >>> for line in reversed(list(input)):
370 root:*:0:0:System Administrator:/var/root:/bin/tcsh
376 :pep:`322` - Reverse Iteration
377 Written and implemented by Raymond Hettinger.
379 .. ======================================================================
382 PEP 324: New subprocess Module
383 ==============================
385 The standard library provides a number of ways to execute a subprocess, offering
386 different features and different levels of complexity.
387 :func:`os.system(command)` is easy to use, but slow (it runs a shell process
388 which executes the command) and dangerous (you have to be careful about escaping
389 the shell's metacharacters). The :mod:`popen2` module offers classes that can
390 capture standard output and standard error from the subprocess, but the naming
391 is confusing. The :mod:`subprocess` module cleans this up, providing a unified
392 interface that offers all the features you might need.
394 Instead of :mod:`popen2`'s collection of classes, :mod:`subprocess` contains a
395 single class called :class:`Popen` whose constructor supports a number of
396 different keyword arguments. ::
398 class Popen(args, bufsize=0, executable=None,
399 stdin=None, stdout=None, stderr=None,
400 preexec_fn=None, close_fds=False, shell=False,
401 cwd=None, env=None, universal_newlines=False,
402 startupinfo=None, creationflags=0):
404 *args* is commonly a sequence of strings that will be the arguments to the
405 program executed as the subprocess. (If the *shell* argument is true, *args*
406 can be a string which will then be passed on to the shell for interpretation,
407 just as :func:`os.system` does.)
409 *stdin*, *stdout*, and *stderr* specify what the subprocess's input, output, and
410 error streams will be. You can provide a file object or a file descriptor, or
411 you can use the constant ``subprocess.PIPE`` to create a pipe between the
412 subprocess and the parent.
414 The constructor has a number of handy options:
416 * *close_fds* requests that all file descriptors be closed before running the
419 * *cwd* specifies the working directory in which the subprocess will be executed
420 (defaulting to whatever the parent's working directory is).
422 * *env* is a dictionary specifying environment variables.
424 * *preexec_fn* is a function that gets called before the child is started.
426 * *universal_newlines* opens the child's input and output using Python's
427 universal newline feature.
429 Once you've created the :class:`Popen` instance, you can call its :meth:`wait`
430 method to pause until the subprocess has exited, :meth:`poll` to check if it's
431 exited without pausing, or :meth:`communicate(data)` to send the string *data*
432 to the subprocess's standard input. :meth:`communicate(data)` then reads any
433 data that the subprocess has sent to its standard output or standard error,
434 returning a tuple ``(stdout_data, stderr_data)``.
436 :func:`call` is a shortcut that passes its arguments along to the :class:`Popen`
437 constructor, waits for the command to complete, and returns the status code of
438 the subprocess. It can serve as a safer analog to :func:`os.system`::
440 sts = subprocess.call(['dpkg', '-i', '/tmp/new-package.deb'])
445 # dpkg returned an error
448 The command is invoked without use of the shell. If you really do want to use
449 the shell, you can add ``shell=True`` as a keyword argument and provide a string
450 instead of a sequence::
452 sts = subprocess.call('dpkg -i /tmp/new-package.deb', shell=True)
454 The PEP takes various examples of shell and Python code and shows how they'd be
455 translated into Python code that uses :mod:`subprocess`. Reading this section
456 of the PEP is highly recommended.
461 :pep:`324` - subprocess - New process module
462 Written and implemented by Peter Ă…strand, with assistance from Fredrik Lundh and
465 .. ======================================================================
468 PEP 327: Decimal Data Type
469 ==========================
471 Python has always supported floating-point (FP) numbers, based on the underlying
472 C :ctype:`double` type, as a data type. However, while most programming
473 languages provide a floating-point type, many people (even programmers) are
474 unaware that floating-point numbers don't represent certain decimal fractions
475 accurately. The new :class:`Decimal` type can represent these fractions
476 accurately, up to a user-specified precision limit.
479 Why is Decimal needed?
480 ----------------------
482 The limitations arise from the representation used for floating-point numbers.
483 FP numbers are made up of three components:
485 * The sign, which is positive or negative.
487 * The mantissa, which is a single-digit binary number followed by a fractional
488 part. For example, ``1.01`` in base-2 notation is ``1 + 0/2 + 1/4``, or 1.25 in
491 * The exponent, which tells where the decimal point is located in the number
494 For example, the number 1.25 has positive sign, a mantissa value of 1.01 (in
495 binary), and an exponent of 0 (the decimal point doesn't need to be shifted).
496 The number 5 has the same sign and mantissa, but the exponent is 2 because the
497 mantissa is multiplied by 4 (2 to the power of the exponent 2); 1.25 \* 4 equals
500 Modern systems usually provide floating-point support that conforms to a
501 standard called IEEE 754. C's :ctype:`double` type is usually implemented as a
502 64-bit IEEE 754 number, which uses 52 bits of space for the mantissa. This
503 means that numbers can only be specified to 52 bits of precision. If you're
504 trying to represent numbers whose expansion repeats endlessly, the expansion is
505 cut off after 52 bits. Unfortunately, most software needs to produce output in
506 base 10, and common fractions in base 10 are often repeating decimals in binary.
507 For example, 1.1 decimal is binary ``1.0001100110011 ...``; .1 = 1/16 + 1/32 +
508 1/256 plus an infinite number of additional terms. IEEE 754 has to chop off
509 that infinitely repeated decimal after 52 digits, so the representation is
512 Sometimes you can see this inaccuracy when the number is printed::
517 The inaccuracy isn't always visible when you print the number because the FP-to-
518 decimal-string conversion is provided by the C library, and most C libraries try
519 to produce sensible output. Even if it's not displayed, however, the inaccuracy
520 is still there and subsequent operations can magnify the error.
522 For many applications this doesn't matter. If I'm plotting points and
523 displaying them on my monitor, the difference between 1.1 and 1.1000000000000001
524 is too small to be visible. Reports often limit output to a certain number of
525 decimal places, and if you round the number to two or three or even eight
526 decimal places, the error is never apparent. However, for applications where it
527 does matter, it's a lot of work to implement your own custom arithmetic
530 Hence, the :class:`Decimal` type was created.
533 The :class:`Decimal` type
534 -------------------------
536 A new module, :mod:`decimal`, was added to Python's standard library. It
537 contains two classes, :class:`Decimal` and :class:`Context`. :class:`Decimal`
538 instances represent numbers, and :class:`Context` instances are used to wrap up
539 various settings such as the precision and default rounding mode.
541 :class:`Decimal` instances are immutable, like regular Python integers and FP
542 numbers; once it's been created, you can't change the value an instance
543 represents. :class:`Decimal` instances can be created from integers or
547 >>> decimal.Decimal(1972)
549 >>> decimal.Decimal("1.1")
552 You can also provide tuples containing the sign, the mantissa represented as a
553 tuple of decimal digits, and the exponent::
555 >>> decimal.Decimal((1, (1, 4, 7, 5), -2))
558 Cautionary note: the sign bit is a Boolean value, so 0 is positive and 1 is
561 Converting from floating-point numbers poses a bit of a problem: should the FP
562 number representing 1.1 turn into the decimal number for exactly 1.1, or for 1.1
563 plus whatever inaccuracies are introduced? The decision was to dodge the issue
564 and leave such a conversion out of the API. Instead, you should convert the
565 floating-point number into a string using the desired precision and pass the
566 string to the :class:`Decimal` constructor::
569 >>> decimal.Decimal(str(f))
571 >>> decimal.Decimal('%.12f' % f)
572 Decimal("1.100000000000")
574 Once you have :class:`Decimal` instances, you can perform the usual mathematical
575 operations on them. One limitation: exponentiation requires an integer
578 >>> a = decimal.Decimal('35.72')
579 >>> b = decimal.Decimal('1.73')
587 Decimal("20.64739884393063583815028902")
591 Traceback (most recent call last):
593 decimal.InvalidOperation: x ** (non-integer)
595 You can combine :class:`Decimal` instances with integers, but not with floating-
601 Traceback (most recent call last):
603 TypeError: You can interact Decimal only with int, long or Decimal data types.
606 :class:`Decimal` numbers can be used with the :mod:`math` and :mod:`cmath`
607 modules, but note that they'll be immediately converted to floating-point
608 numbers before the operation is performed, resulting in a possible loss of
609 precision and accuracy. You'll also get back a regular floating-point number
610 and not a :class:`Decimal`. ::
612 >>> import math, cmath
613 >>> d = decimal.Decimal('123456789012.345')
619 :class:`Decimal` instances have a :meth:`sqrt` method that returns a
620 :class:`Decimal`, but if you need other things such as trigonometric functions
621 you'll have to implement them. ::
624 Decimal("351364.1828820134592177245001")
627 The :class:`Context` type
628 -------------------------
630 Instances of the :class:`Context` class encapsulate several settings for
633 * :attr:`prec` is the precision, the number of decimal places.
635 * :attr:`rounding` specifies the rounding mode. The :mod:`decimal` module has
636 constants for the various possibilities: :const:`ROUND_DOWN`,
637 :const:`ROUND_CEILING`, :const:`ROUND_HALF_EVEN`, and various others.
639 * :attr:`traps` is a dictionary specifying what happens on encountering certain
640 error conditions: either an exception is raised or a value is returned. Some
641 examples of error conditions are division by zero, loss of precision, and
644 There's a thread-local default context available by calling :func:`getcontext`;
645 you can change the properties of this context to alter the default precision,
646 rounding, or trap handling. The following example shows the effect of changing
647 the precision of the default context::
649 >>> decimal.getcontext().prec
651 >>> decimal.Decimal(1) / decimal.Decimal(7)
652 Decimal("0.1428571428571428571428571429")
653 >>> decimal.getcontext().prec = 9
654 >>> decimal.Decimal(1) / decimal.Decimal(7)
655 Decimal("0.142857143")
657 The default action for error conditions is selectable; the module can either
658 return a special value such as infinity or not-a-number, or exceptions can be
661 >>> decimal.Decimal(1) / decimal.Decimal(0)
662 Traceback (most recent call last):
664 decimal.DivisionByZero: x / 0
665 >>> decimal.getcontext().traps[decimal.DivisionByZero] = False
666 >>> decimal.Decimal(1) / decimal.Decimal(0)
670 The :class:`Context` instance also has various methods for formatting numbers
671 such as :meth:`to_eng_string` and :meth:`to_sci_string`.
673 For more information, see the documentation for the :mod:`decimal` module, which
674 includes a quick-start tutorial and a reference.
679 :pep:`327` - Decimal Data Type
680 Written by Facundo Batista and implemented by Facundo Batista, Eric Price,
681 Raymond Hettinger, Aahz, and Tim Peters.
683 http://www.lahey.com/float.htm
684 The article uses Fortran code to illustrate many of the problems that floating-
685 point inaccuracy can cause.
687 http://www2.hursley.ibm.com/decimal/
688 A description of a decimal-based representation. This representation is being
689 proposed as a standard, and underlies the new Python decimal type. Much of this
690 material was written by Mike Cowlishaw, designer of the Rexx language.
692 .. ======================================================================
695 PEP 328: Multi-line Imports
696 ===========================
698 One language change is a small syntactic tweak aimed at making it easier to
699 import many names from a module. In a ``from module import names`` statement,
700 *names* is a sequence of names separated by commas. If the sequence is very
701 long, you can either write multiple imports from the same module, or you can use
702 backslashes to escape the line endings like this::
704 from SimpleXMLRPCServer import SimpleXMLRPCServer,\
705 SimpleXMLRPCRequestHandler,\
706 CGIXMLRPCRequestHandler,\
707 resolve_dotted_attribute
709 The syntactic change in Python 2.4 simply allows putting the names within
710 parentheses. Python ignores newlines within a parenthesized expression, so the
711 backslashes are no longer needed::
713 from SimpleXMLRPCServer import (SimpleXMLRPCServer,
714 SimpleXMLRPCRequestHandler,
715 CGIXMLRPCRequestHandler,
716 resolve_dotted_attribute)
718 The PEP also proposes that all :keyword:`import` statements be absolute imports,
719 with a leading ``.`` character to indicate a relative import. This part of the
720 PEP was not implemented for Python 2.4, but was completed for Python 2.5.
725 :pep:`328` - Imports: Multi-Line and Absolute/Relative
726 Written by Aahz. Multi-line imports were implemented by Dima Dorfman.
728 .. ======================================================================
731 PEP 331: Locale-Independent Float/String Conversions
732 ====================================================
734 The :mod:`locale` modules lets Python software select various conversions and
735 display conventions that are localized to a particular country or language.
736 However, the module was careful to not change the numeric locale because various
737 functions in Python's implementation required that the numeric locale remain set
738 to the ``'C'`` locale. Often this was because the code was using the C
739 library's :cfunc:`atof` function.
741 Not setting the numeric locale caused trouble for extensions that used third-
742 party C libraries, however, because they wouldn't have the correct locale set.
743 The motivating example was GTK+, whose user interface widgets weren't displaying
744 numbers in the current locale.
746 The solution described in the PEP is to add three new functions to the Python
747 API that perform ASCII-only conversions, ignoring the locale setting:
749 * :cfunc:`PyOS_ascii_strtod(str, ptr)` and :cfunc:`PyOS_ascii_atof(str, ptr)`
750 both convert a string to a C :ctype:`double`.
752 * :cfunc:`PyOS_ascii_formatd(buffer, buf_len, format, d)` converts a
753 :ctype:`double` to an ASCII string.
755 The code for these functions came from the GLib library
756 (http://library.gnome.org/devel/glib/stable/), whose developers kindly
757 relicensed the relevant functions and donated them to the Python Software
758 Foundation. The :mod:`locale` module can now change the numeric locale,
759 letting extensions such as GTK+ produce the correct results.
764 :pep:`331` - Locale-Independent Float/String Conversions
765 Written by Christian R. Reis, and implemented by Gustavo Carneiro.
767 .. ======================================================================
770 Other Language Changes
771 ======================
773 Here are all of the changes that Python 2.4 makes to the core Python language.
775 * Decorators for functions and methods were added (:pep:`318`).
777 * Built-in :func:`set` and :func:`frozenset` types were added (:pep:`218`).
778 Other new built-ins include the :func:`reversed(seq)` function (:pep:`322`).
780 * Generator expressions were added (:pep:`289`).
782 * Certain numeric expressions no longer return values restricted to 32 or 64
785 * You can now put parentheses around the list of names in a ``from module import
786 names`` statement (:pep:`328`).
788 * The :meth:`dict.update` method now accepts the same argument forms as the
789 :class:`dict` constructor. This includes any mapping, any iterable of key/value
790 pairs, and keyword arguments. (Contributed by Raymond Hettinger.)
792 * The string methods :meth:`ljust`, :meth:`rjust`, and :meth:`center` now take
793 an optional argument for specifying a fill character other than a space.
794 (Contributed by Raymond Hettinger.)
796 * Strings also gained an :meth:`rsplit` method that works like the :meth:`split`
797 method but splits from the end of the string. (Contributed by Sean
800 >>> 'www.python.org'.split('.', 1)
801 ['www', 'python.org']
802 'www.python.org'.rsplit('.', 1)
803 ['www.python', 'org']
805 * Three keyword parameters, *cmp*, *key*, and *reverse*, were added to the
806 :meth:`sort` method of lists. These parameters make some common usages of
807 :meth:`sort` simpler. All of these parameters are optional.
809 For the *cmp* parameter, the value should be a comparison function that takes
810 two parameters and returns -1, 0, or +1 depending on how the parameters compare.
811 This function will then be used to sort the list. Previously this was the only
812 parameter that could be provided to :meth:`sort`.
814 *key* should be a single-parameter function that takes a list element and
815 returns a comparison key for the element. The list is then sorted using the
816 comparison keys. The following example sorts a list case-insensitively::
818 >>> L = ['A', 'b', 'c', 'D']
819 >>> L.sort() # Case-sensitive sort
822 >>> # Using 'key' parameter to sort list
823 >>> L.sort(key=lambda x: x.lower())
826 >>> # Old-fashioned way
827 >>> L.sort(cmp=lambda x,y: cmp(x.lower(), y.lower()))
831 The last example, which uses the *cmp* parameter, is the old way to perform a
832 case-insensitive sort. It works but is slower than using a *key* parameter.
833 Using *key* calls :meth:`lower` method once for each element in the list while
834 using *cmp* will call it twice for each comparison, so using *key* saves on
835 invocations of the :meth:`lower` method.
837 For simple key functions and comparison functions, it is often possible to avoid
838 a :keyword:`lambda` expression by using an unbound method instead. For example,
839 the above case-insensitive sort is best written as::
841 >>> L.sort(key=str.lower)
845 Finally, the *reverse* parameter takes a Boolean value. If the value is true,
846 the list will be sorted into reverse order. Instead of ``L.sort() ;
847 L.reverse()``, you can now write ``L.sort(reverse=True)``.
849 The results of sorting are now guaranteed to be stable. This means that two
850 entries with equal keys will be returned in the same order as they were input.
851 For example, you can sort a list of people by name, and then sort the list by
852 age, resulting in a list sorted by age where people with the same age are in
855 (All changes to :meth:`sort` contributed by Raymond Hettinger.)
857 * There is a new built-in function :func:`sorted(iterable)` that works like the
858 in-place :meth:`list.sort` method but can be used in expressions. The
861 * the input may be any iterable;
863 * a newly formed copy is sorted, leaving the original intact; and
865 * the expression returns the new sorted copy
869 >>> L = [9,7,8,3,2,4,1,6,5]
870 >>> [10+i for i in sorted(L)] # usable in a list comprehension
871 [11, 12, 13, 14, 15, 16, 17, 18, 19]
872 >>> L # original is left unchanged
874 >>> sorted('Monty Python') # any iterable may be an input
875 [' ', 'M', 'P', 'h', 'n', 'n', 'o', 'o', 't', 't', 'y', 'y']
877 >>> # List the contents of a dict sorted by key values
878 >>> colormap = dict(red=1, blue=2, green=3, black=4, yellow=5)
879 >>> for k, v in sorted(colormap.iteritems()):
888 (Contributed by Raymond Hettinger.)
890 * Integer operations will no longer trigger an :exc:`OverflowWarning`. The
891 :exc:`OverflowWarning` warning will disappear in Python 2.5.
893 * The interpreter gained a new switch, :option:`-m`, that takes a name, searches
894 for the corresponding module on ``sys.path``, and runs the module as a script.
895 For example, you can now run the Python profiler with ``python -m profile``.
896 (Contributed by Nick Coghlan.)
898 * The :func:`eval(expr, globals, locals)` and :func:`execfile(filename, globals,
899 locals)` functions and the :keyword:`exec` statement now accept any mapping type
900 for the *locals* parameter. Previously this had to be a regular Python
901 dictionary. (Contributed by Raymond Hettinger.)
903 * The :func:`zip` built-in function and :func:`itertools.izip` now return an
904 empty list if called with no arguments. Previously they raised a
905 :exc:`TypeError` exception. This makes them more suitable for use with variable
906 length argument lists::
908 >>> def transpose(array):
909 ... return zip(*array)
911 >>> transpose([(1,2,3), (4,5,6)])
912 [(1, 4), (2, 5), (3, 6)]
916 (Contributed by Raymond Hettinger.)
918 * Encountering a failure while importing a module no longer leaves a partially-
919 initialized module object in ``sys.modules``. The incomplete module object left
920 behind would fool further imports of the same module into succeeding, leading to
921 confusing errors. (Fixed by Tim Peters.)
923 * :const:`None` is now a constant; code that binds a new value to the name
924 ``None`` is now a syntax error. (Contributed by Raymond Hettinger.)
926 .. ======================================================================
932 * The inner loops for list and tuple slicing were optimized and now run about
933 one-third faster. The inner loops for dictionaries were also optimized,
934 resulting in performance boosts for :meth:`keys`, :meth:`values`, :meth:`items`,
935 :meth:`iterkeys`, :meth:`itervalues`, and :meth:`iteritems`. (Contributed by
938 * The machinery for growing and shrinking lists was optimized for speed and for
939 space efficiency. Appending and popping from lists now runs faster due to more
940 efficient code paths and less frequent use of the underlying system
941 :cfunc:`realloc`. List comprehensions also benefit. :meth:`list.extend` was
942 also optimized and no longer converts its argument into a temporary list before
943 extending the base list. (Contributed by Raymond Hettinger.)
945 * :func:`list`, :func:`tuple`, :func:`map`, :func:`filter`, and :func:`zip` now
946 run several times faster with non-sequence arguments that supply a
947 :meth:`__len__` method. (Contributed by Raymond Hettinger.)
949 * The methods :meth:`list.__getitem__`, :meth:`dict.__getitem__`, and
950 :meth:`dict.__contains__` are are now implemented as :class:`method_descriptor`
951 objects rather than :class:`wrapper_descriptor` objects. This form of access
952 doubles their performance and makes them more suitable for use as arguments to
953 functionals: ``map(mydict.__getitem__, keylist)``. (Contributed by Raymond
956 * Added a new opcode, ``LIST_APPEND``, that simplifies the generated bytecode
957 for list comprehensions and speeds them up by about a third. (Contributed by
960 * The peephole bytecode optimizer has been improved to produce shorter, faster
961 bytecode; remarkably, the resulting bytecode is more readable. (Enhanced by
964 * String concatenations in statements of the form ``s = s + "abc"`` and ``s +=
965 "abc"`` are now performed more efficiently in certain circumstances. This
966 optimization won't be present in other Python implementations such as Jython, so
967 you shouldn't rely on it; using the :meth:`join` method of strings is still
968 recommended when you want to efficiently glue a large number of strings
969 together. (Contributed by Armin Rigo.)
971 The net result of the 2.4 optimizations is that Python 2.4 runs the pystone
972 benchmark around 5% faster than Python 2.3 and 35% faster than Python 2.2.
973 (pystone is not a particularly good benchmark, but it's the most commonly used
974 measurement of Python's performance. Your own applications may show greater or
975 smaller benefits from Python 2.4.)
977 .. pystone is almost useless for comparing different versions of Python;
978 instead, it excels at predicting relative Python performance on different
979 machines. So, this section would be more informative if it used other tools
980 such as pybench and parrotbench. For a more application oriented benchmark,
981 try comparing the timings of test_decimal.py under 2.3 and 2.4.
983 .. ======================================================================
986 New, Improved, and Deprecated Modules
987 =====================================
989 As usual, Python's standard library received a number of enhancements and bug
990 fixes. Here's a partial list of the most notable changes, sorted alphabetically
991 by module name. Consult the :file:`Misc/NEWS` file in the source tree for a more
992 complete list of changes, or look through the CVS logs for all the details.
994 * The :mod:`asyncore` module's :func:`loop` function now has a *count* parameter
995 that lets you perform a limited number of passes through the polling loop. The
996 default is still to loop forever.
998 * The :mod:`base64` module now has more complete RFC 3548 support for Base64,
999 Base32, and Base16 encoding and decoding, including optional case folding and
1000 optional alternative alphabets. (Contributed by Barry Warsaw.)
1002 * The :mod:`bisect` module now has an underlying C implementation for improved
1003 performance. (Contributed by Dmitry Vasiliev.)
1005 * The CJKCodecs collections of East Asian codecs, maintained by Hye-Shik Chang,
1006 was integrated into 2.4. The new encodings are:
1008 * Chinese (PRC): gb2312, gbk, gb18030, big5hkscs, hz
1010 * Chinese (ROC): big5, cp950
1012 * Japanese: cp932, euc-jis-2004, euc-jp, euc-jisx0213, iso-2022-jp,
1013 iso-2022-jp-1, iso-2022-jp-2, iso-2022-jp-3, iso-2022-jp-ext, iso-2022-jp-2004,
1014 shift-jis, shift-jisx0213, shift-jis-2004
1016 * Korean: cp949, euc-kr, johab, iso-2022-kr
1018 * Some other new encodings were added: HP Roman8, ISO_8859-11, ISO_8859-16,
1019 PCTP-154, and TIS-620.
1021 * The UTF-8 and UTF-16 codecs now cope better with receiving partial input.
1022 Previously the :class:`StreamReader` class would try to read more data, making
1023 it impossible to resume decoding from the stream. The :meth:`read` method will
1024 now return as much data as it can and future calls will resume decoding where
1025 previous ones left off. (Implemented by Walter Dörwald.)
1027 * There is a new :mod:`collections` module for various specialized collection
1028 datatypes. Currently it contains just one type, :class:`deque`, a double-
1029 ended queue that supports efficiently adding and removing elements from either
1032 >>> from collections import deque
1033 >>> d = deque('ghi') # make a new deque with three items
1034 >>> d.append('j') # add a new entry to the right side
1035 >>> d.appendleft('f') # add a new entry to the left side
1036 >>> d # show the representation of the deque
1037 deque(['f', 'g', 'h', 'i', 'j'])
1038 >>> d.pop() # return and remove the rightmost item
1040 >>> d.popleft() # return and remove the leftmost item
1042 >>> list(d) # list the contents of the deque
1044 >>> 'h' in d # search the deque
1047 Several modules, such as the :mod:`Queue` and :mod:`threading` modules, now take
1048 advantage of :class:`collections.deque` for improved performance. (Contributed
1049 by Raymond Hettinger.)
1051 * The :mod:`ConfigParser` classes have been enhanced slightly. The :meth:`read`
1052 method now returns a list of the files that were successfully parsed, and the
1053 :meth:`set` method raises :exc:`TypeError` if passed a *value* argument that
1054 isn't a string. (Contributed by John Belmonte and David Goodger.)
1056 * The :mod:`curses` module now supports the ncurses extension
1057 :func:`use_default_colors`. On platforms where the terminal supports
1058 transparency, this makes it possible to use a transparent background.
1059 (Contributed by Jörg Lehmann.)
1061 * The :mod:`difflib` module now includes an :class:`HtmlDiff` class that creates
1062 an HTML table showing a side by side comparison of two versions of a text.
1063 (Contributed by Dan Gass.)
1065 * The :mod:`email` package was updated to version 3.0, which dropped various
1066 deprecated APIs and removes support for Python versions earlier than 2.3. The
1067 3.0 version of the package uses a new incremental parser for MIME messages,
1068 available in the :mod:`email.FeedParser` module. The new parser doesn't require
1069 reading the entire message into memory, and doesn't throw exceptions if a
1070 message is malformed; instead it records any problems in the :attr:`defect`
1071 attribute of the message. (Developed by Anthony Baxter, Barry Warsaw, Thomas
1072 Wouters, and others.)
1074 * The :mod:`heapq` module has been converted to C. The resulting tenfold
1075 improvement in speed makes the module suitable for handling high volumes of
1076 data. In addition, the module has two new functions :func:`nlargest` and
1077 :func:`nsmallest` that use heaps to find the N largest or smallest values in a
1078 dataset without the expense of a full sort. (Contributed by Raymond Hettinger.)
1080 * The :mod:`httplib` module now contains constants for HTTP status codes defined
1081 in various HTTP-related RFC documents. Constants have names such as
1082 :const:`OK`, :const:`CREATED`, :const:`CONTINUE`, and
1083 :const:`MOVED_PERMANENTLY`; use pydoc to get a full list. (Contributed by
1086 * The :mod:`imaplib` module now supports IMAP's THREAD command (contributed by
1087 Yves Dionne) and new :meth:`deleteacl` and :meth:`myrights` methods (contributed
1090 * The :mod:`itertools` module gained a :func:`groupby(iterable[, *func*])`
1091 function. *iterable* is something that can be iterated over to return a stream
1092 of elements, and the optional *func* parameter is a function that takes an
1093 element and returns a key value; if omitted, the key is simply the element
1094 itself. :func:`groupby` then groups the elements into subsequences which have
1095 matching values of the key, and returns a series of 2-tuples containing the key
1096 value and an iterator over the subsequence.
1098 Here's an example to make this clearer. The *key* function simply returns
1099 whether a number is even or odd, so the result of :func:`groupby` is to return
1100 consecutive runs of odd or even numbers. ::
1102 >>> import itertools
1103 >>> L = [2, 4, 6, 7, 8, 9, 11, 12, 14]
1104 >>> for key_val, it in itertools.groupby(L, lambda x: x % 2):
1105 ... print key_val, list(it)
1114 :func:`groupby` is typically used with sorted input. The logic for
1115 :func:`groupby` is similar to the Unix ``uniq`` filter which makes it handy for
1116 eliminating, counting, or identifying duplicate elements::
1118 >>> word = 'abracadabra'
1119 >>> letters = sorted(word) # Turn string into a sorted list of letters
1121 ['a', 'a', 'a', 'a', 'a', 'b', 'b', 'c', 'd', 'r', 'r']
1122 >>> for k, g in itertools.groupby(letters):
1123 ... print k, list(g)
1125 a ['a', 'a', 'a', 'a', 'a']
1130 >>> # List unique letters
1131 >>> [k for k, g in groupby(letters)]
1132 ['a', 'b', 'c', 'd', 'r']
1133 >>> # Count letter occurrences
1134 >>> [(k, len(list(g))) for k, g in groupby(letters)]
1135 [('a', 5), ('b', 2), ('c', 1), ('d', 1), ('r', 2)]
1137 (Contributed by Hye-Shik Chang.)
1139 * :mod:`itertools` also gained a function named :func:`tee(iterator, N)` that
1140 returns *N* independent iterators that replicate *iterator*. If *N* is omitted,
1141 the default is 2. ::
1144 >>> i1, i2 = itertools.tee(L)
1146 (<itertools.tee object at 0x402c2080>, <itertools.tee object at 0x402c2090>)
1147 >>> list(i1) # Run the first iterator to exhaustion
1149 >>> list(i2) # Run the second iterator to exhaustion
1152 Note that :func:`tee` has to keep copies of the values returned by the
1153 iterator; in the worst case, it may need to keep all of them. This should
1154 therefore be used carefully if the leading iterator can run far ahead of the
1155 trailing iterator in a long stream of inputs. If the separation is large, then
1156 you might as well use :func:`list` instead. When the iterators track closely
1157 with one another, :func:`tee` is ideal. Possible applications include
1158 bookmarking, windowing, or lookahead iterators. (Contributed by Raymond
1161 * A number of functions were added to the :mod:`locale` module, such as
1162 :func:`bind_textdomain_codeset` to specify a particular encoding and a family of
1163 :func:`l\*gettext` functions that return messages in the chosen encoding.
1164 (Contributed by Gustavo Niemeyer.)
1166 * Some keyword arguments were added to the :mod:`logging` package's
1167 :func:`basicConfig` function to simplify log configuration. The default
1168 behavior is to log messages to standard error, but various keyword arguments can
1169 be specified to log to a particular file, change the logging format, or set the
1170 logging level. For example::
1173 logging.basicConfig(filename='/var/log/application.log',
1174 level=0, # Log all messages
1175 format='%(levelname):%(process):%(thread):%(message)')
1177 Other additions to the :mod:`logging` package include a :meth:`log(level, msg)`
1178 convenience method, as well as a :class:`TimedRotatingFileHandler` class that
1179 rotates its log files at a timed interval. The module already had
1180 :class:`RotatingFileHandler`, which rotated logs once the file exceeded a
1181 certain size. Both classes derive from a new :class:`BaseRotatingHandler` class
1182 that can be used to implement other rotating handlers.
1184 (Changes implemented by Vinay Sajip.)
1186 * The :mod:`marshal` module now shares interned strings on unpacking a data
1187 structure. This may shrink the size of certain pickle strings, but the primary
1188 effect is to make :file:`.pyc` files significantly smaller. (Contributed by
1191 * The :mod:`nntplib` module's :class:`NNTP` class gained :meth:`description` and
1192 :meth:`descriptions` methods to retrieve newsgroup descriptions for a single
1193 group or for a range of groups. (Contributed by JĂĽrgen A. Erhard.)
1195 * Two new functions were added to the :mod:`operator` module,
1196 :func:`attrgetter(attr)` and :func:`itemgetter(index)`. Both functions return
1197 callables that take a single argument and return the corresponding attribute or
1198 item; these callables make excellent data extractors when used with :func:`map`
1199 or :func:`sorted`. For example::
1201 >>> L = [('c', 2), ('d', 1), ('a', 4), ('b', 3)]
1202 >>> map(operator.itemgetter(0), L)
1203 ['c', 'd', 'a', 'b']
1204 >>> map(operator.itemgetter(1), L)
1206 >>> sorted(L, key=operator.itemgetter(1)) # Sort list by second tuple item
1207 [('d', 1), ('c', 2), ('b', 3), ('a', 4)]
1209 (Contributed by Raymond Hettinger.)
1211 * The :mod:`optparse` module was updated in various ways. The module now passes
1212 its messages through :func:`gettext.gettext`, making it possible to
1213 internationalize Optik's help and error messages. Help messages for options can
1214 now include the string ``'%default'``, which will be replaced by the option's
1215 default value. (Contributed by Greg Ward.)
1217 * The long-term plan is to deprecate the :mod:`rfc822` module in some future
1218 Python release in favor of the :mod:`email` package. To this end, the
1219 :func:`email.Utils.formatdate` function has been changed to make it usable as a
1220 replacement for :func:`rfc822.formatdate`. You may want to write new e-mail
1221 processing code with this in mind. (Change implemented by Anthony Baxter.)
1223 * A new :func:`urandom(n)` function was added to the :mod:`os` module, returning
1224 a string containing *n* bytes of random data. This function provides access to
1225 platform-specific sources of randomness such as :file:`/dev/urandom` on Linux or
1226 the Windows CryptoAPI. (Contributed by Trevor Perrin.)
1228 * Another new function: :func:`os.path.lexists(path)` returns true if the file
1229 specified by *path* exists, whether or not it's a symbolic link. This differs
1230 from the existing :func:`os.path.exists(path)` function, which returns false if
1231 *path* is a symlink that points to a destination that doesn't exist.
1232 (Contributed by Beni Cherniavsky.)
1234 * A new :func:`getsid` function was added to the :mod:`posix` module that
1235 underlies the :mod:`os` module. (Contributed by J. Raynor.)
1237 * The :mod:`poplib` module now supports POP over SSL. (Contributed by Hector
1240 * The :mod:`profile` module can now profile C extension functions. (Contributed
1243 * The :mod:`random` module has a new method called :meth:`getrandbits(N)` that
1244 returns a long integer *N* bits in length. The existing :meth:`randrange`
1245 method now uses :meth:`getrandbits` where appropriate, making generation of
1246 arbitrarily large random numbers more efficient. (Contributed by Raymond
1249 * The regular expression language accepted by the :mod:`re` module was extended
1250 with simple conditional expressions, written as ``(?(group)A|B)``. *group* is
1251 either a numeric group ID or a group name defined with ``(?P<group>...)``
1252 earlier in the expression. If the specified group matched, the regular
1253 expression pattern *A* will be tested against the string; if the group didn't
1254 match, the pattern *B* will be used instead. (Contributed by Gustavo Niemeyer.)
1256 * The :mod:`re` module is also no longer recursive, thanks to a massive amount
1257 of work by Gustavo Niemeyer. In a recursive regular expression engine, certain
1258 patterns result in a large amount of C stack space being consumed, and it was
1259 possible to overflow the stack. For example, if you matched a 30000-byte string
1260 of ``a`` characters against the expression ``(a|b)+``, one stack frame was
1261 consumed per character. Python 2.3 tried to check for stack overflow and raise
1262 a :exc:`RuntimeError` exception, but certain patterns could sidestep the
1263 checking and if you were unlucky Python could segfault. Python 2.4's regular
1264 expression engine can match this pattern without problems.
1266 * The :mod:`signal` module now performs tighter error-checking on the parameters
1267 to the :func:`signal.signal` function. For example, you can't set a handler on
1268 the :const:`SIGKILL` signal; previous versions of Python would quietly accept
1269 this, but 2.4 will raise a :exc:`RuntimeError` exception.
1271 * Two new functions were added to the :mod:`socket` module. :func:`socketpair`
1272 returns a pair of connected sockets and :func:`getservbyport(port)` looks up the
1273 service name for a given port number. (Contributed by Dave Cole and Barry
1276 * The :func:`sys.exitfunc` function has been deprecated. Code should be using
1277 the existing :mod:`atexit` module, which correctly handles calling multiple exit
1278 functions. Eventually :func:`sys.exitfunc` will become a purely internal
1279 interface, accessed only by :mod:`atexit`.
1281 * The :mod:`tarfile` module now generates GNU-format tar files by default.
1282 (Contributed by Lars Gustaebel.)
1284 * The :mod:`threading` module now has an elegantly simple way to support
1285 thread-local data. The module contains a :class:`local` class whose attribute
1286 values are local to different threads. ::
1290 data = threading.local()
1292 data.url = ('www.python.org', 80)
1294 Other threads can assign and retrieve their own values for the :attr:`number`
1295 and :attr:`url` attributes. You can subclass :class:`local` to initialize
1296 attributes or to add methods. (Contributed by Jim Fulton.)
1298 * The :mod:`timeit` module now automatically disables periodic garbage
1299 collection during the timing loop. This change makes consecutive timings more
1300 comparable. (Contributed by Raymond Hettinger.)
1302 * The :mod:`weakref` module now supports a wider variety of objects including
1303 Python functions, class instances, sets, frozensets, deques, arrays, files,
1304 sockets, and regular expression pattern objects. (Contributed by Raymond
1307 * The :mod:`xmlrpclib` module now supports a multi-call extension for
1308 transmitting multiple XML-RPC calls in a single HTTP operation. (Contributed by
1311 * The :mod:`mpz`, :mod:`rotor`, and :mod:`xreadlines` modules have been
1314 .. ======================================================================
1315 .. whole new modules get described in subsections here
1316 .. =====================
1322 The :mod:`cookielib` library supports client-side handling for HTTP cookies,
1323 mirroring the :mod:`Cookie` module's server-side cookie support. Cookies are
1324 stored in cookie jars; the library transparently stores cookies offered by the
1325 web server in the cookie jar, and fetches the cookie from the jar when
1326 connecting to the server. As in web browsers, policy objects control whether
1327 cookies are accepted or not.
1329 In order to store cookies across sessions, two implementations of cookie jars
1330 are provided: one that stores cookies in the Netscape format so applications can
1331 use the Mozilla or Lynx cookie files, and one that stores cookies in the same
1332 format as the Perl libwww library.
1334 :mod:`urllib2` has been changed to interact with :mod:`cookielib`:
1335 :class:`HTTPCookieProcessor` manages a cookie jar that is used when accessing
1338 This module was contributed by John J. Lee.
1340 .. ==================
1346 The :mod:`doctest` module underwent considerable refactoring thanks to Edward
1347 Loper and Tim Peters. Testing can still be as simple as running
1348 :func:`doctest.testmod`, but the refactorings allow customizing the module's
1349 operation in various ways
1351 The new :class:`DocTestFinder` class extracts the tests from a given object's
1362 finder = doctest.DocTestFinder()
1364 # Get list of DocTest instances
1365 tests = finder.find(f)
1367 The new :class:`DocTestRunner` class then runs individual tests and can produce
1368 a summary of the results::
1370 runner = doctest.DocTestRunner()
1372 tried, failed = runner.run(t)
1374 runner.summarize(verbose=1)
1376 The above example produces the following output::
1378 1 items passed all tests:
1381 2 passed and 0 failed.
1384 :class:`DocTestRunner` uses an instance of the :class:`OutputChecker` class to
1385 compare the expected output with the actual output. This class takes a number
1386 of different flags that customize its behaviour; ambitious users can also write
1387 a completely new subclass of :class:`OutputChecker`.
1389 The default output checker provides a number of handy features. For example,
1390 with the :const:`doctest.ELLIPSIS` option flag, an ellipsis (``...``) in the
1391 expected output matches any substring, making it easier to accommodate outputs
1392 that vary in minor ways::
1396 <__main__.C instance at 0x...>
1400 Another special string, ``<BLANKLINE>``, matches a blank line::
1408 Another new capability is producing a diff-style display of the output by
1409 specifying the :const:`doctest.REPORT_UDIFF` (unified diffs),
1410 :const:`doctest.REPORT_CDIFF` (context diffs), or :const:`doctest.REPORT_NDIFF`
1411 (delta-style) option flags. For example::
1420 L = 'here is a rather lengthy list of words'.split()
1424 Running the above function's tests with :const:`doctest.REPORT_UDIFF` specified,
1425 you get the following output::
1427 **********************************************************************
1428 File "t.py", line 15, in g
1431 Differences (unified diff with -expected +actual):
1437 **********************************************************************
1439 .. ======================================================================
1442 Build and C API Changes
1443 =======================
1445 Some of the changes to Python's build process and to the C API are:
1447 * Three new convenience macros were added for common return values from
1448 extension functions: :cmacro:`Py_RETURN_NONE`, :cmacro:`Py_RETURN_TRUE`, and
1449 :cmacro:`Py_RETURN_FALSE`. (Contributed by Brett Cannon.)
1451 * Another new macro, :cmacro:`Py_CLEAR(obj)`, decreases the reference count of
1452 *obj* and sets *obj* to the null pointer. (Contributed by Jim Fulton.)
1454 * A new function, :cfunc:`PyTuple_Pack(N, obj1, obj2, ..., objN)`, constructs
1455 tuples from a variable length argument list of Python objects. (Contributed by
1458 * A new function, :cfunc:`PyDict_Contains(d, k)`, implements fast dictionary
1459 lookups without masking exceptions raised during the look-up process.
1460 (Contributed by Raymond Hettinger.)
1462 * The :cmacro:`Py_IS_NAN(X)` macro returns 1 if its float or double argument
1463 *X* is a NaN. (Contributed by Tim Peters.)
1465 * C code can avoid unnecessary locking by using the new
1466 :cfunc:`PyEval_ThreadsInitialized` function to tell if any thread operations
1467 have been performed. If this function returns false, no lock operations are
1468 needed. (Contributed by Nick Coghlan.)
1470 * A new function, :cfunc:`PyArg_VaParseTupleAndKeywords`, is the same as
1471 :cfunc:`PyArg_ParseTupleAndKeywords` but takes a :ctype:`va_list` instead of a
1472 number of arguments. (Contributed by Greg Chapman.)
1474 * A new method flag, :const:`METH_COEXISTS`, allows a function defined in slots
1475 to co-exist with a :ctype:`PyCFunction` having the same name. This can halve
1476 the access time for a method such as :meth:`set.__contains__`. (Contributed by
1479 * Python can now be built with additional profiling for the interpreter itself,
1480 intended as an aid to people developing the Python core. Providing
1481 :option:`----enable-profiling` to the :program:`configure` script will let you
1482 profile the interpreter with :program:`gprof`, and providing the
1483 :option:`----with-tsc` switch enables profiling using the Pentium's Time-Stamp-
1484 Counter register. Note that the :option:`----with-tsc` switch is slightly
1485 misnamed, because the profiling feature also works on the PowerPC platform,
1486 though that processor architecture doesn't call that register "the TSC
1487 register". (Contributed by Jeremy Hylton.)
1489 * The :ctype:`tracebackobject` type has been renamed to
1490 :ctype:`PyTracebackObject`.
1492 .. ======================================================================
1495 Port-Specific Changes
1496 ---------------------
1498 * The Windows port now builds under MSVC++ 7.1 as well as version 6.
1499 (Contributed by Martin von Löwis.)
1501 .. ======================================================================
1504 Porting to Python 2.4
1505 =====================
1507 This section lists previously described changes that may require changes to your
1510 * Left shifts and hexadecimal/octal constants that are too large no longer
1511 trigger a :exc:`FutureWarning` and return a value limited to 32 or 64 bits;
1512 instead they return a long integer.
1514 * Integer operations will no longer trigger an :exc:`OverflowWarning`. The
1515 :exc:`OverflowWarning` warning will disappear in Python 2.5.
1517 * The :func:`zip` built-in function and :func:`itertools.izip` now return an
1518 empty list instead of raising a :exc:`TypeError` exception if called with no
1521 * You can no longer compare the :class:`date` and :class:`datetime` instances
1522 provided by the :mod:`datetime` module. Two instances of different classes
1523 will now always be unequal, and relative comparisons (``<``, ``>``) will raise
1526 * :func:`dircache.listdir` now passes exceptions to the caller instead of
1527 returning empty lists.
1529 * :func:`LexicalHandler.startDTD` used to receive the public and system IDs in
1530 the wrong order. This has been corrected; applications relying on the wrong
1531 order need to be fixed.
1533 * :func:`fcntl.ioctl` now warns if the *mutate* argument is omitted and
1536 * The :mod:`tarfile` module now generates GNU-format tar files by default.
1538 * Encountering a failure while importing a module no longer leaves a partially-
1539 initialized module object in ``sys.modules``.
1541 * :const:`None` is now a constant; code that binds a new value to the name
1542 ``None`` is now a syntax error.
1544 * The :func:`signals.signal` function now raises a :exc:`RuntimeError` exception
1545 for certain illegal values; previously these errors would pass silently. For
1546 example, you can no longer set a handler on the :const:`SIGKILL` signal.
1548 .. ======================================================================
1556 The author would like to thank the following people for offering suggestions,
1557 corrections and assistance with various drafts of this article: Koray Can, Hye-
1558 Shik Chang, Michael Dyck, Raymond Hettinger, Brian Hurt, Hamish Lawson, Fredrik
1559 Lundh, Sean Reifschneider, Sadruddin Rejeb.