2 :mod:`re` --- Regular expression operations
3 ===========================================
6 :synopsis: Regular expression operations.
7 .. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
8 .. sectionauthor:: Andrew M. Kuchling <amk@amk.ca>
13 This module provides regular expression matching operations similar to
14 those found in Perl. Both patterns and strings to be searched can be
15 Unicode strings as well as 8-bit strings. The :mod:`re` module is
18 Regular expressions use the backslash character (``'\'``) to indicate
19 special forms or to allow special characters to be used without invoking
20 their special meaning. This collides with Python's usage of the same
21 character for the same purpose in string literals; for example, to match
22 a literal backslash, one might have to write ``'\\\\'`` as the pattern
23 string, because the regular expression must be ``\\``, and each
24 backslash must be expressed as ``\\`` inside a regular Python string
27 The solution is to use Python's raw string notation for regular expression
28 patterns; backslashes are not handled in any special way in a string literal
29 prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
30 ``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
31 newline. Usually patterns will be expressed in Python code using this raw
34 It is important to note that most regular expression operations are available as
35 module-level functions and :class:`RegexObject` methods. The functions are
36 shortcuts that don't require you to compile a regex object first, but miss some
37 fine-tuning parameters.
41 Mastering Regular Expressions
42 Book on regular expressions by Jeffrey Friedl, published by O'Reilly. The
43 second edition of the book no longer covers Python at all, but the first
44 edition covered writing good regular expression patterns in great detail.
49 Regular Expression Syntax
50 -------------------------
52 A regular expression (or RE) specifies a set of strings that matches it; the
53 functions in this module let you check if a particular string matches a given
54 regular expression (or if a given regular expression matches a particular
55 string, which comes down to the same thing).
57 Regular expressions can be concatenated to form new regular expressions; if *A*
58 and *B* are both regular expressions, then *AB* is also a regular expression.
59 In general, if a string *p* matches *A* and another string *q* matches *B*, the
60 string *pq* will match AB. This holds unless *A* or *B* contain low precedence
61 operations; boundary conditions between *A* and *B*; or have numbered group
62 references. Thus, complex expressions can easily be constructed from simpler
63 primitive expressions like the ones described here. For details of the theory
64 and implementation of regular expressions, consult the Friedl book referenced
65 above, or almost any textbook about compiler construction.
67 A brief explanation of the format of regular expressions follows. For further
68 information and a gentler presentation, consult the Regular Expression HOWTO,
69 accessible from http://www.python.org/doc/howto/.
71 Regular expressions can contain both special and ordinary characters. Most
72 ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
73 expressions; they simply match themselves. You can concatenate ordinary
74 characters, so ``last`` matches the string ``'last'``. (In the rest of this
75 section, we'll write RE's in ``this special style``, usually without quotes, and
76 strings to be matched ``'in single quotes'``.)
78 Some characters, like ``'|'`` or ``'('``, are special. Special
79 characters either stand for classes of ordinary characters, or affect
80 how the regular expressions around them are interpreted. Regular
81 expression pattern strings may not contain null bytes, but can specify
82 the null byte using the ``\number`` notation, e.g., ``'\x00'``.
85 The special characters are:
88 (Dot.) In the default mode, this matches any character except a newline. If
89 the :const:`DOTALL` flag has been specified, this matches any character
93 (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
94 matches immediately after each newline.
97 Matches the end of the string or just before the newline at the end of the
98 string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
99 matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
100 only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
101 matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
102 a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
103 the newline, and one at the end of the string.
106 Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
107 many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
108 by any number of 'b's.
111 Causes the resulting RE to match 1 or more repetitions of the preceding RE.
112 ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
116 Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
117 ``ab?`` will match either 'a' or 'ab'.
119 ``*?``, ``+?``, ``??``
120 The ``'*'``, ``'+'``, and ``'?'`` qualifiers are all :dfn:`greedy`; they match
121 as much text as possible. Sometimes this behaviour isn't desired; if the RE
122 ``<.*>`` is matched against ``'<H1>title</H1>'``, it will match the entire
123 string, and not just ``'<H1>'``. Adding ``'?'`` after the qualifier makes it
124 perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
125 characters as possible will be matched. Using ``.*?`` in the previous
126 expression will match only ``'<H1>'``.
129 Specifies that exactly *m* copies of the previous RE should be matched; fewer
130 matches cause the entire RE not to match. For example, ``a{6}`` will match
131 exactly six ``'a'`` characters, but not five.
134 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
135 RE, attempting to match as many repetitions as possible. For example,
136 ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
137 lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
138 example, ``a{4,}b`` will match ``aaaab`` or a thousand ``'a'`` characters
139 followed by a ``b``, but not ``aaab``. The comma may not be omitted or the
140 modifier would be confused with the previously described form.
143 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
144 RE, attempting to match as *few* repetitions as possible. This is the
145 non-greedy version of the previous qualifier. For example, on the
146 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
147 while ``a{3,5}?`` will only match 3 characters.
150 Either escapes special characters (permitting you to match characters like
151 ``'*'``, ``'?'``, and so forth), or signals a special sequence; special
152 sequences are discussed below.
154 If you're not using a raw string to express the pattern, remember that Python
155 also uses the backslash as an escape sequence in string literals; if the escape
156 sequence isn't recognized by Python's parser, the backslash and subsequent
157 character are included in the resulting string. However, if Python would
158 recognize the resulting sequence, the backslash should be repeated twice. This
159 is complicated and hard to understand, so it's highly recommended that you use
160 raw strings for all but the simplest expressions.
163 Used to indicate a set of characters. Characters can be listed individually, or
164 a range of characters can be indicated by giving two characters and separating
165 them by a ``'-'``. Special characters are not active inside sets. For example,
166 ``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
167 ``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
168 ``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
169 as ``\w`` or ``\S`` (defined below) are also acceptable inside a
170 range, although the characters they match depends on whether :const:`LOCALE`
171 or :const:`UNICODE` mode is in force. If you want to include a
172 ``']'`` or a ``'-'`` inside a set, precede it with a backslash, or
173 place it as the first character. The pattern ``[]]`` will match
174 ``']'``, for example.
176 You can match the characters not within a range by :dfn:`complementing` the set.
177 This is indicated by including a ``'^'`` as the first character of the set;
178 ``'^'`` elsewhere will simply match the ``'^'`` character. For example,
179 ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
180 character except ``'^'``.
183 ``A|B``, where A and B can be arbitrary REs, creates a regular expression that
184 will match either A or B. An arbitrary number of REs can be separated by the
185 ``'|'`` in this way. This can be used inside groups (see below) as well. As
186 the target string is scanned, REs separated by ``'|'`` are tried from left to
187 right. When one pattern completely matches, that branch is accepted. This means
188 that once ``A`` matches, ``B`` will not be tested further, even if it would
189 produce a longer overall match. In other words, the ``'|'`` operator is never
190 greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
191 character class, as in ``[|]``.
194 Matches whatever regular expression is inside the parentheses, and indicates the
195 start and end of a group; the contents of a group can be retrieved after a match
196 has been performed, and can be matched later in the string with the ``\number``
197 special sequence, described below. To match the literals ``'('`` or ``')'``,
198 use ``\(`` or ``\)``, or enclose them inside a character class: ``[(] [)]``.
201 This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
202 otherwise). The first character after the ``'?'`` determines what the meaning
203 and further syntax of the construct is. Extensions usually do not create a new
204 group; ``(?P<name>...)`` is the only exception to this rule. Following are the
205 currently supported extensions.
208 (One or more letters from the set ``'i'``, ``'L'``, ``'m'``, ``'s'``,
209 ``'u'``, ``'x'``.) The group matches the empty string; the letters
210 set the corresponding flags: :const:`re.I` (ignore case),
211 :const:`re.L` (locale dependent), :const:`re.M` (multi-line),
212 :const:`re.S` (dot matches all), :const:`re.U` (Unicode dependent),
213 and :const:`re.X` (verbose), for the entire regular expression. (The
214 flags are described in :ref:`contents-of-module-re`.) This
215 is useful if you wish to include the flags as part of the regular
216 expression, instead of passing a *flag* argument to the
217 :func:`compile` function.
219 Note that the ``(?x)`` flag changes how the expression is parsed. It should be
220 used first in the expression string, or after one or more whitespace characters.
221 If there are non-whitespace characters before the flag, the results are
225 A non-grouping version of regular parentheses. Matches whatever regular
226 expression is inside the parentheses, but the substring matched by the group
227 *cannot* be retrieved after performing a match or referenced later in the
231 Similar to regular parentheses, but the substring matched by the group is
232 accessible via the symbolic group name *name*. Group names must be valid Python
233 identifiers, and each group name must be defined only once within a regular
234 expression. A symbolic group is also a numbered group, just as if the group
235 were not named. So the group named 'id' in the example below can also be
236 referenced as the numbered group 1.
238 For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be
239 referenced by its name in arguments to methods of match objects, such as
240 ``m.group('id')`` or ``m.end('id')``, and also by name in pattern text (for
241 example, ``(?P=id)``) and replacement text (such as ``\g<id>``).
244 Matches whatever text was matched by the earlier group named *name*.
247 A comment; the contents of the parentheses are simply ignored.
250 Matches if ``...`` matches next, but doesn't consume any of the string. This is
251 called a lookahead assertion. For example, ``Isaac (?=Asimov)`` will match
252 ``'Isaac '`` only if it's followed by ``'Asimov'``.
255 Matches if ``...`` doesn't match next. This is a negative lookahead assertion.
256 For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
257 followed by ``'Asimov'``.
260 Matches if the current position in the string is preceded by a match for ``...``
261 that ends at the current position. This is called a :dfn:`positive lookbehind
262 assertion`. ``(?<=abc)def`` will find a match in ``abcdef``, since the
263 lookbehind will back up 3 characters and check if the contained pattern matches.
264 The contained pattern must only match strings of some fixed length, meaning that
265 ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
266 patterns which start with positive lookbehind assertions will never match at the
267 beginning of the string being searched; you will most likely want to use the
268 :func:`search` function rather than the :func:`match` function::
271 >>> m = re.search('(?<=abc)def', 'abcdef')
275 This example looks for a word following a hyphen::
277 >>> m = re.search('(?<=-)\w+', 'spam-egg')
282 Matches if the current position in the string is not preceded by a match for
283 ``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
284 positive lookbehind assertions, the contained pattern must only match strings of
285 some fixed length. Patterns which start with negative lookbehind assertions may
286 match at the beginning of the string being searched.
288 ``(?(id/name)yes-pattern|no-pattern)``
289 Will try to match with ``yes-pattern`` if the group with given *id* or *name*
290 exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is optional and
291 can be omitted. For example, ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)`` is a poor email
292 matching pattern, which will match with ``'<user@host.com>'`` as well as
293 ``'user@host.com'``, but not with ``'<user@host.com'``.
295 .. versionadded:: 2.4
297 The special sequences consist of ``'\'`` and a character from the list below.
298 If the ordinary character is not on the list, then the resulting RE will match
299 the second character. For example, ``\$`` matches the character ``'$'``.
302 Matches the contents of the group of the same number. Groups are numbered
303 starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
304 but not ``'the end'`` (note the space after the group). This special sequence
305 can only be used to match one of the first 99 groups. If the first digit of
306 *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
307 a group match, but as the character with octal value *number*. Inside the
308 ``'['`` and ``']'`` of a character class, all numeric escapes are treated as
312 Matches only at the start of the string.
315 Matches the empty string, but only at the beginning or end of a word. A word is
316 defined as a sequence of alphanumeric or underscore characters, so the end of a
317 word is indicated by whitespace or a non-alphanumeric, non-underscore character.
318 Note that ``\b`` is defined as the boundary between ``\w`` and ``\ W``, so the
319 precise set of characters deemed to be alphanumeric depends on the values of the
320 ``UNICODE`` and ``LOCALE`` flags. Inside a character range, ``\b`` represents
321 the backspace character, for compatibility with Python's string literals.
324 Matches the empty string, but only when it is *not* at the beginning or end of a
325 word. This is just the opposite of ``\b``, so is also subject to the settings
326 of ``LOCALE`` and ``UNICODE``.
329 When the :const:`UNICODE` flag is not specified, matches any decimal digit; this
330 is equivalent to the set ``[0-9]``. With :const:`UNICODE`, it will match
331 whatever is classified as a digit in the Unicode character properties database.
334 When the :const:`UNICODE` flag is not specified, matches any non-digit
335 character; this is equivalent to the set ``[^0-9]``. With :const:`UNICODE`, it
336 will match anything other than character marked as digits in the Unicode
337 character properties database.
340 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
341 any whitespace character; this is equivalent to the set ``[ \t\n\r\f\v]``. With
342 :const:`LOCALE`, it will match this set plus whatever characters are defined as
343 space for the current locale. If :const:`UNICODE` is set, this will match the
344 characters ``[ \t\n\r\f\v]`` plus whatever is classified as space in the Unicode
345 character properties database.
348 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
349 any non-whitespace character; this is equivalent to the set ``[^ \t\n\r\f\v]``
350 With :const:`LOCALE`, it will match any character not in this set, and not
351 defined as space in the current locale. If :const:`UNICODE` is set, this will
352 match anything other than ``[ \t\n\r\f\v]`` and characters marked as space in
353 the Unicode character properties database.
356 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
357 any alphanumeric character and the underscore; this is equivalent to the set
358 ``[a-zA-Z0-9_]``. With :const:`LOCALE`, it will match the set ``[0-9_]`` plus
359 whatever characters are defined as alphanumeric for the current locale. If
360 :const:`UNICODE` is set, this will match the characters ``[0-9_]`` plus whatever
361 is classified as alphanumeric in the Unicode character properties database.
364 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
365 any non-alphanumeric character; this is equivalent to the set ``[^a-zA-Z0-9_]``.
366 With :const:`LOCALE`, it will match any character not in the set ``[0-9_]``, and
367 not defined as alphanumeric for the current locale. If :const:`UNICODE` is set,
368 this will match anything other than ``[0-9_]`` and characters marked as
369 alphanumeric in the Unicode character properties database.
372 Matches only at the end of the string.
374 Most of the standard escapes supported by Python string literals are also
375 accepted by the regular expression parser::
381 Octal escapes are included in a limited form: If the first digit is a 0, or if
382 there are three octal digits, it is considered an octal escape. Otherwise, it is
383 a group reference. As for string literals, octal escapes are always at most
384 three digits in length.
387 .. _matching-searching:
389 Matching vs Searching
390 ---------------------
392 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
395 Python offers two different primitive operations based on regular expressions:
396 **match** checks for a match only at the beginning of the string, while
397 **search** checks for a match anywhere in the string (this is what Perl does
400 Note that match may differ from search even when using a regular expression
401 beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in
402 :const:`MULTILINE` mode also immediately following a newline. The "match"
403 operation succeeds only if the pattern matches at the start of the string
404 regardless of mode, or at the starting position given by the optional *pos*
405 argument regardless of whether a newline precedes it. ::
407 >>> re.match("c", "abcdef") # No match
408 >>> re.search("c", "abcdef")
409 <_sre.SRE_Match object at 0x827e9c0> # Match
412 .. _contents-of-module-re:
417 The module defines several functions, constants, and an exception. Some of the
418 functions are simplified versions of the full featured methods for compiled
419 regular expressions. Most non-trivial applications always use the compiled
423 .. function:: compile(pattern[, flags])
425 Compile a regular expression pattern into a regular expression object, which
426 can be used for matching using its :func:`match` and :func:`search` methods,
429 The expression's behaviour can be modified by specifying a *flags* value.
430 Values can be any of the following variables, combined using bitwise OR (the
435 prog = re.compile(pat)
436 result = prog.match(str)
440 result = re.match(pat, str)
442 but the version using :func:`compile` is more efficient when the expression
443 will be used several times in a single program.
445 .. (The compiled version of the last pattern passed to :func:`re.match` or
446 :func:`re.search` is cached, so programs that use only a single regular
447 expression at a time needn't worry about compiling regular expressions.)
453 Perform case-insensitive matching; expressions like ``[A-Z]`` will match
454 lowercase letters, too. This is not affected by the current locale.
460 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` dependent on the
467 When specified, the pattern character ``'^'`` matches at the beginning of the
468 string and at the beginning of each line (immediately following each newline);
469 and the pattern character ``'$'`` matches at the end of the string and at the
470 end of each line (immediately preceding each newline). By default, ``'^'``
471 matches only at the beginning of the string, and ``'$'`` only at the end of the
472 string and immediately before the newline (if any) at the end of the string.
478 Make the ``'.'`` special character match any character at all, including a
479 newline; without this flag, ``'.'`` will match anything *except* a newline.
485 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S`` dependent
486 on the Unicode character properties database.
488 .. versionadded:: 2.0
494 This flag allows you to write regular expressions that look nicer. Whitespace
495 within the pattern is ignored, except when in a character class or preceded by
496 an unescaped backslash, and, when a line contains a ``'#'`` neither in a
497 character class or preceded by an unescaped backslash, all characters from the
498 leftmost such ``'#'`` through the end of the line are ignored.
500 That means that the two following regular expression objects that match a
501 decimal number are functionally equal::
503 a = re.compile(r"""\d + # the integral part
504 \. # the decimal point
505 \d * # some fractional digits""", re.X)
506 b = re.compile(r"\d+\.\d*")
509 .. function:: search(pattern, string[, flags])
511 Scan through *string* looking for a location where the regular expression
512 *pattern* produces a match, and return a corresponding :class:`MatchObject`
513 instance. Return ``None`` if no position in the string matches the pattern; note
514 that this is different from finding a zero-length match at some point in the
518 .. function:: match(pattern, string[, flags])
520 If zero or more characters at the beginning of *string* match the regular
521 expression *pattern*, return a corresponding :class:`MatchObject` instance.
522 Return ``None`` if the string does not match the pattern; note that this is
523 different from a zero-length match.
527 If you want to locate a match anywhere in *string*, use :meth:`search`
531 .. function:: split(pattern, string[, maxsplit=0])
533 Split *string* by the occurrences of *pattern*. If capturing parentheses are
534 used in *pattern*, then the text of all groups in the pattern are also returned
535 as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
536 splits occur, and the remainder of the string is returned as the final element
537 of the list. (Incompatibility note: in the original Python 1.5 release,
538 *maxsplit* was ignored. This has been fixed in later releases.) ::
540 >>> re.split('\W+', 'Words, words, words.')
541 ['Words', 'words', 'words', '']
542 >>> re.split('(\W+)', 'Words, words, words.')
543 ['Words', ', ', 'words', ', ', 'words', '.', '']
544 >>> re.split('\W+', 'Words, words, words.', 1)
545 ['Words', 'words, words.']
547 Note that *split* will never split a string on an empty pattern match.
550 >>> re.split('x*', 'foo')
552 >>> re.split("(?m)^$", "foo\n\nbar\n")
555 .. function:: findall(pattern, string[, flags])
557 Return all non-overlapping matches of *pattern* in *string*, as a list of
558 strings. If one or more groups are present in the pattern, return a list of
559 groups; this will be a list of tuples if the pattern has more than one group.
560 Empty matches are included in the result unless they touch the beginning of
563 .. versionadded:: 1.5.2
565 .. versionchanged:: 2.4
566 Added the optional flags argument.
569 .. function:: finditer(pattern, string[, flags])
571 Return an :term:`iterator` yielding :class:`MatchObject` instances over all
572 non-overlapping matches for the RE *pattern* in *string*. Empty matches are
573 included in the result unless they touch the beginning of another match.
575 .. versionadded:: 2.2
577 .. versionchanged:: 2.4
578 Added the optional flags argument.
581 .. function:: sub(pattern, repl, string[, count])
583 Return the string obtained by replacing the leftmost non-overlapping occurrences
584 of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
585 *string* is returned unchanged. *repl* can be a string or a function; if it is
586 a string, any backslash escapes in it are processed. That is, ``\n`` is
587 converted to a single newline character, ``\r`` is converted to a linefeed, and
588 so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
589 as ``\6``, are replaced with the substring matched by group 6 in the pattern.
592 >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
593 ... r'static PyObject*\npy_\1(void)\n{',
595 'static PyObject*\npy_myfunc(void)\n{'
597 If *repl* is a function, it is called for every non-overlapping occurrence of
598 *pattern*. The function takes a single match object argument, and returns the
599 replacement string. For example::
601 >>> def dashrepl(matchobj):
602 ... if matchobj.group(0) == '-': return ' '
604 >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
607 The pattern may be a string or an RE object; if you need to specify regular
608 expression flags, you must use a RE object, or use embedded modifiers in a
609 pattern; for example, ``sub("(?i)b+", "x", "bbbb BBBB")`` returns ``'x x'``.
611 The optional argument *count* is the maximum number of pattern occurrences to be
612 replaced; *count* must be a non-negative integer. If omitted or zero, all
613 occurrences will be replaced. Empty matches for the pattern are replaced only
614 when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
617 In addition to character escapes and backreferences as described above,
618 ``\g<name>`` will use the substring matched by the group named ``name``, as
619 defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
620 group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
621 in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
622 reference to group 20, not a reference to group 2 followed by the literal
623 character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
624 substring matched by the RE.
627 .. function:: subn(pattern, repl, string[, count])
629 Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
630 number_of_subs_made)``.
633 .. function:: escape(string)
635 Return *string* with all non-alphanumerics backslashed; this is useful if you
636 want to match an arbitrary literal string that may have regular expression
637 metacharacters in it.
642 Exception raised when a string passed to one of the functions here is not a
643 valid regular expression (for example, it might contain unmatched parentheses)
644 or when some other error occurs during compilation or matching. It is never an
645 error if a string contains no match for a pattern.
650 Regular Expression Objects
651 --------------------------
653 Compiled regular expression objects support the following methods and
657 .. method:: RegexObject.match(string[, pos[, endpos]])
659 If zero or more characters at the beginning of *string* match this regular
660 expression, return a corresponding :class:`MatchObject` instance. Return
661 ``None`` if the string does not match the pattern; note that this is different
662 from a zero-length match.
666 If you want to locate a match anywhere in *string*, use :meth:`search`
669 The optional second parameter *pos* gives an index in the string where the
670 search is to start; it defaults to ``0``. This is not completely equivalent to
671 slicing the string; the ``'^'`` pattern character matches at the real beginning
672 of the string and at positions just after a newline, but not necessarily at the
673 index where the search is to start.
675 The optional parameter *endpos* limits how far the string will be searched; it
676 will be as if the string is *endpos* characters long, so only the characters
677 from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
678 than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
679 expression object, ``rx.match(string, 0, 50)`` is equivalent to
680 ``rx.match(string[:50], 0)``. ::
682 >>> pattern = re.compile("o")
683 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
684 >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
685 <_sre.SRE_Match object at 0x827eb10>
688 .. method:: RegexObject.search(string[, pos[, endpos]])
690 Scan through *string* looking for a location where this regular expression
691 produces a match, and return a corresponding :class:`MatchObject` instance.
692 Return ``None`` if no position in the string matches the pattern; note that this
693 is different from finding a zero-length match at some point in the string.
695 The optional *pos* and *endpos* parameters have the same meaning as for the
696 :meth:`match` method.
699 .. method:: RegexObject.split(string[, maxsplit=0])
701 Identical to the :func:`split` function, using the compiled pattern.
704 .. method:: RegexObject.findall(string[, pos[, endpos]])
706 Identical to the :func:`findall` function, using the compiled pattern.
709 .. method:: RegexObject.finditer(string[, pos[, endpos]])
711 Identical to the :func:`finditer` function, using the compiled pattern.
714 .. method:: RegexObject.sub(repl, string[, count=0])
716 Identical to the :func:`sub` function, using the compiled pattern.
719 .. method:: RegexObject.subn(repl, string[, count=0])
721 Identical to the :func:`subn` function, using the compiled pattern.
724 .. attribute:: RegexObject.flags
726 The flags argument used when the RE object was compiled, or ``0`` if no flags
730 .. attribute:: RegexObject.groupindex
732 A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
733 numbers. The dictionary is empty if no symbolic groups were used in the
737 .. attribute:: RegexObject.pattern
739 The pattern string from which the RE object was compiled.
747 Match objects always have a boolean value of :const:`True`, so that you can test
748 whether e.g. :func:`match` resulted in a match with a simple if statement. They
749 support the following methods and attributes:
752 .. method:: MatchObject.expand(template)
754 Return the string obtained by doing backslash substitution on the template
755 string *template*, as done by the :meth:`sub` method. Escapes such as ``\n`` are
756 converted to the appropriate characters, and numeric backreferences (``\1``,
757 ``\2``) and named backreferences (``\g<1>``, ``\g<name>``) are replaced by the
758 contents of the corresponding group.
761 .. method:: MatchObject.group([group1, ...])
763 Returns one or more subgroups of the match. If there is a single argument, the
764 result is a single string; if there are multiple arguments, the result is a
765 tuple with one item per argument. Without arguments, *group1* defaults to zero
766 (the whole match is returned). If a *groupN* argument is zero, the corresponding
767 return value is the entire matching string; if it is in the inclusive range
768 [1..99], it is the string matching the corresponding parenthesized group. If a
769 group number is negative or larger than the number of groups defined in the
770 pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
771 part of the pattern that did not match, the corresponding result is ``None``.
772 If a group is contained in a part of the pattern that matched multiple times,
773 the last match is returned. ::
775 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
777 'Isaac Newton' # The entire match
779 'Isaac' # The first parenthesized subgroup.
781 'Newton' # The second parenthesized subgroup.
783 ('Isaac', 'Newton') # Multiple arguments give us a tuple.
785 If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
786 arguments may also be strings identifying groups by their group name. If a
787 string argument is not used as a group name in the pattern, an :exc:`IndexError`
790 A moderately complicated example::
792 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
793 >>> m.group('first_name')
795 >>> m.group('last_name')
798 Named groups can also be referred to by their index::
805 If a group matches multiple times, only the last match is accessible::
806 >>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
807 >>> m.group(1) # Returns only the last match.
811 .. method:: MatchObject.groups([default])
813 Return a tuple containing all the subgroups of the match, from 1 up to however
814 many groups are in the pattern. The *default* argument is used for groups that
815 did not participate in the match; it defaults to ``None``. (Incompatibility
816 note: in the original Python 1.5 release, if the tuple was one element long, a
817 string would be returned instead. In later versions (from 1.5.1 on), a
818 singleton tuple is returned in such cases.)
822 >>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
826 If we make the decimal place and everything after it optional, not all groups
827 might participate in the match. These groups will default to ``None`` unless
828 the *default* argument is given::
830 >>> m = re.match(r"(\d+)\.?(\d+)?", "24")
832 ('24', None) # Second group defaults to None.
834 ('24', '0') # Now, the second group defaults to '0'.
837 .. method:: MatchObject.groupdict([default])
839 Return a dictionary containing all the *named* subgroups of the match, keyed by
840 the subgroup name. The *default* argument is used for groups that did not
841 participate in the match; it defaults to ``None``. For example::
843 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
845 {'first_name': 'Malcom', 'last_name': 'Reynolds'}
848 .. method:: MatchObject.start([group])
849 MatchObject.end([group])
851 Return the indices of the start and end of the substring matched by *group*;
852 *group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
853 *group* exists but did not contribute to the match. For a match object *m*, and
854 a group *g* that did contribute to the match, the substring matched by group *g*
855 (equivalent to ``m.group(g)``) is ::
857 m.string[m.start(g):m.end(g)]
859 Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
860 null string. For example, after ``m = re.search('b(c?)', 'cba')``,
861 ``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
862 2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
864 An example that will remove *remove_this* from email addresses::
866 >>> email = "tony@tiremove_thisger.net"
867 >>> m = re.search("remove_this", email)
868 >>> email[:m.start()] + email[m.end():]
872 .. method:: MatchObject.span([group])
874 For :class:`MatchObject` *m*, return the 2-tuple ``(m.start(group),
875 m.end(group))``. Note that if *group* did not contribute to the match, this is
876 ``(-1, -1)``. *group* defaults to zero, the entire match.
879 .. attribute:: MatchObject.pos
881 The value of *pos* which was passed to the :func:`search` or :func:`match`
882 method of the :class:`RegexObject`. This is the index into the string at which
883 the RE engine started looking for a match.
886 .. attribute:: MatchObject.endpos
888 The value of *endpos* which was passed to the :func:`search` or :func:`match`
889 method of the :class:`RegexObject`. This is the index into the string beyond
890 which the RE engine will not go.
893 .. attribute:: MatchObject.lastindex
895 The integer index of the last matched capturing group, or ``None`` if no group
896 was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
897 ``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
898 the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
902 .. attribute:: MatchObject.lastgroup
904 The name of the last matched capturing group, or ``None`` if the group didn't
905 have a name, or if no group was matched at all.
908 .. attribute:: MatchObject.re
910 The regular expression object whose :meth:`match` or :meth:`search` method
911 produced this :class:`MatchObject` instance.
914 .. attribute:: MatchObject.string
916 The string passed to :func:`match` or :func:`search`.
926 In this example, we'll use the following helper function to display match
927 objects a little more gracefully::
929 def displaymatch(match):
932 return '<Match: %r, groups=%r>' % (match.group(), match.groups())
934 Suppose you are writing a poker program where a player's hand is represented as
935 a 5-character string with each character representing a card, "a" for ace, "k"
936 for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
937 representing the card with that value.
939 To see if a given string is a valid hand, one could do the following::
941 >>> valid = re.compile(r"[0-9akqj]{5}$"
942 >>> displaymatch(valid.match("ak05q")) # Valid.
943 <Match: 'ak05q', groups=()>
944 >>> displaymatch(valid.match("ak05e")) # Invalid.
945 >>> displaymatch(valid.match("ak0")) # Invalid.
946 >>> displaymatch(valid.match("727ak")) # Valid.
947 <Match: '727ak', groups=()>
949 That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
950 To match this with a regular expression, one could use backreferences as such::
952 >>> pair = re.compile(r".*(.).*\1")
953 >>> displaymatch(pair.match("717ak")) # Pair of 7s.
954 <Match: '717', groups=('7',)>
955 >>> displaymatch(pair.match("718ak")) # No pairs.
956 >>> displaymatch(pair.match("354aa")) # Pair of aces.
957 <Match: '345aa', groups=('a',)>
959 To find out what card the pair consists of, one could use the :func:`group`
960 method of :class:`MatchObject` in the following manner::
962 >>> pair.match("717ak").group(1)
965 # Error because re.match() returns None, which doesn't have a group() method:
966 >>> pair.match("718ak").group(1)
967 Traceback (most recent call last):
968 File "<pyshell#23>", line 1, in <module>
969 re.match(r".*(.).*\1", "718ak").group(1)
970 AttributeError: 'NoneType' object has no attribute 'group'
972 >>> pair.match("354aa").group(1)
979 .. index:: single: scanf()
981 Python does not currently have an equivalent to :cfunc:`scanf`. Regular
982 expressions are generally more powerful, though also more verbose, than
983 :cfunc:`scanf` format strings. The table below offers some more-or-less
984 equivalent mappings between :cfunc:`scanf` format tokens and regular
987 +--------------------------------+---------------------------------------------+
988 | :cfunc:`scanf` Token | Regular Expression |
989 +================================+=============================================+
991 +--------------------------------+---------------------------------------------+
992 | ``%5c`` | ``.{5}`` |
993 +--------------------------------+---------------------------------------------+
994 | ``%d`` | ``[-+]?\d+`` |
995 +--------------------------------+---------------------------------------------+
996 | ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
997 +--------------------------------+---------------------------------------------+
998 | ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
999 +--------------------------------+---------------------------------------------+
1000 | ``%o`` | ``0[0-7]*`` |
1001 +--------------------------------+---------------------------------------------+
1002 | ``%s`` | ``\S+`` |
1003 +--------------------------------+---------------------------------------------+
1004 | ``%u`` | ``\d+`` |
1005 +--------------------------------+---------------------------------------------+
1006 | ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` |
1007 +--------------------------------+---------------------------------------------+
1009 To extract the filename and numbers from a string like ::
1011 /usr/sbin/sendmail - 0 errors, 4 warnings
1013 you would use a :cfunc:`scanf` format like ::
1015 %s - %d errors, %d warnings
1017 The equivalent regular expression would be ::
1019 (\S+) - (\d+) errors, (\d+) warnings
1025 If you create regular expressions that require the engine to perform a lot of
1026 recursion, you may encounter a :exc:`RuntimeError` exception with the message
1027 ``maximum recursion limit`` exceeded. For example, ::
1030 >>> s = 'Begin ' + 1000*'a very long string ' + 'end'
1031 >>> re.match('Begin (\w| )*? end', s).end()
1032 Traceback (most recent call last):
1033 File "<stdin>", line 1, in ?
1034 File "/usr/local/lib/python2.5/re.py", line 132, in match
1035 return _compile(pattern, flags).match(string)
1036 RuntimeError: maximum recursion limit exceeded
1038 You can often restructure your regular expression to avoid recursion.
1040 Starting with Python 2.3, simple uses of the ``*?`` pattern are special-cased to
1041 avoid recursion. Thus, the above regular expression can avoid recursion by
1042 being recast as ``Begin [a-zA-Z0-9_ ]*?end``. As a further benefit, such
1043 regular expressions will run faster than their recursive equivalents.
1046 search() vs. match()
1047 ^^^^^^^^^^^^^^^^^^^^
1049 In a nutshell, :func:`match` only attempts to match a pattern at the beginning
1050 of a string where :func:`search` will match a pattern anywhere in a string.
1053 >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
1054 >>> re.search("o", "dog") # Match as search() looks everywhere in the string.
1055 <_sre.SRE_Match object at 0x827e9f8>
1059 The following applies only to regular expression objects like those created
1060 with ``re.compile("pattern")``, not the primitives
1061 ``re.match(pattern, string)`` or ``re.search(pattern, string)``.
1063 :func:`match` has an optional second parameter that gives an index in the string
1064 where the search is to start::
1066 >>> pattern = re.compile("o")
1067 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
1068 # Equivalent to the above expression as 0 is the default starting index:
1069 >>> pattern.match("dog", 0)
1070 # Match as "o" is the 2nd character of "dog" (index 0 is the first):
1071 >>> pattern.match("dog", 1)
1072 <_sre.SRE_Match object at 0x827eb10>
1073 >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
1079 :func:`split` splits a string into a list delimited by the passed pattern. The
1080 method is invaluable for converting textual data into data structures that can be
1081 easily read and modified by Python as demonstrated in the following example that
1082 creates a phonebook.
1084 First, here is the input. Normally it may come from a file, here we are using
1085 triple-quoted string syntax::
1087 >>> input = """Ross McFluff: 834.345.1254 155 Elm Street
1089 Ronald Heathmore: 892.345.3428 436 Finley Avenue
1090 Frank Burger: 925.541.7625 662 South Dogwood Way
1093 Heather Albrecht: 548.326.4584 919 Park Place"""
1095 The entries are separated by one or more newlines. Now we convert the string
1096 into a list with each nonempty line having its own entry::
1098 >>> entries = re.split("\n+", input)
1100 ['Ross McFluff 834.345.1254 155 Elm Street',
1101 'Ronald Heathmore 892.345.3428 436 Finley Avenue',
1102 'Frank Burger 925.541.7625 662 South Dogwood Way',
1103 'Heather Albrecht 548.326.4584 919 Park Place']
1105 Finally, split each entry into a list with first name, last name, telephone
1106 number, and address. We use the ``maxsplit`` paramater of :func:`split`
1107 because the address has spaces, our splitting pattern, in it::
1109 >>> [re.split(":? ", entry, 3) for entry in entries]
1110 [['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1111 ['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1112 ['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1113 ['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1115 The ``:?`` pattern matches the colon after the last name, so that it does not
1116 occur in the result list. With a ``maxsplit`` of ``4``, we could seperate the
1117 house number from the street name::
1119 >>> [re.split(":? ", entry, 4) for entry in entries]
1120 [['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1121 ['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1122 ['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1123 ['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1129 :func:`sub` replaces every occurrence of a pattern with a string or the
1130 result of a function. This example demonstrates using :func:`sub` with
1131 a function to "munge" text, or randomize the order of all the characters
1132 in each word of a sentence except for the first and last characters::
1135 ... inner_word = list(m.group(2))
1136 ... random.shuffle(inner_word)
1137 ... return m.group(1) + "".join(inner_word) + m.group(3)
1138 >>> text = "Professor Abdolmalek, please report your absences promptly."
1139 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1140 'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1141 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1142 'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1148 :func:`findall` matches *all* occurences of a pattern, not just the first
1149 one as :func:`search` does. For example, if one was a writer and wanted to
1150 find all of the adverbs in some text, he or she might use :func:`findall` in
1151 the following manner::
1153 >>> text = "He was carefully disguised but captured quickly by police."
1154 >>> re.findall(r"\w+ly", text)
1155 ['carefully', 'quickly']
1158 Finding all Adverbs and their Positions
1159 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1161 If one wants more information about all matches of a pattern than the matched
1162 text, :func:`finditer` is useful as it provides instances of
1163 :class:`MatchObject` instead of strings. Continuing with the previous example,
1164 if one was a writer who wanted to find all of the adverbs *and their positions*
1165 in some text, he or she would use :func:`finditer` in the following manner::
1167 >>> text = "He was carefully disguised but captured quickly by police."
1168 >>> for m in re.finditer(r"\w+ly", text):
1169 print '%02d-%02d: %s' % (m.start(), m.end(), m.group(0))
1177 Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1178 every backslash (``'\'``) in a regular expression would have to be prefixed with
1179 another one to escape it. For example, the two following lines of code are
1180 functionally identical::
1182 >>> re.match(r"\W(.)\1\W", " ff ")
1183 <_sre.SRE_Match object at 0x8262760>
1184 >>> re.match("\\W(.)\\1\\W", " ff ")
1185 <_sre.SRE_Match object at 0x82627a0>
1187 When one wants to match a literal backslash, it must be escaped in the regular
1188 expression. With raw string notation, this means ``r"\\"``. Without raw string
1189 notation, one must use ``"\\\\"``, making the following lines of code
1190 functionally identical::
1192 >>> re.match(r"\\", r"\\")
1193 <_sre.SRE_Match object at 0x827eb48>
1194 >>> re.match("\\\\", r"\\")
1195 <_sre.SRE_Match object at 0x827ec60>