2 :mod:`re` --- Regular expression operations
3 ===========================================
6 :synopsis: Regular expression operations.
7 .. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
8 .. sectionauthor:: Andrew M. Kuchling <amk@amk.ca>
13 This module provides regular expression matching operations similar to
14 those found in Perl. Both patterns and strings to be searched can be
15 Unicode strings as well as 8-bit strings. The :mod:`re` module is
18 Regular expressions use the backslash character (``'\'``) to indicate
19 special forms or to allow special characters to be used without invoking
20 their special meaning. This collides with Python's usage of the same
21 character for the same purpose in string literals; for example, to match
22 a literal backslash, one might have to write ``'\\\\'`` as the pattern
23 string, because the regular expression must be ``\\``, and each
24 backslash must be expressed as ``\\`` inside a regular Python string
27 The solution is to use Python's raw string notation for regular expression
28 patterns; backslashes are not handled in any special way in a string literal
29 prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
30 ``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
31 newline. Usually patterns will be expressed in Python code using this raw
34 It is important to note that most regular expression operations are available as
35 module-level functions and :class:`RegexObject` methods. The functions are
36 shortcuts that don't require you to compile a regex object first, but miss some
37 fine-tuning parameters.
41 Mastering Regular Expressions
42 Book on regular expressions by Jeffrey Friedl, published by O'Reilly. The
43 second edition of the book no longer covers Python at all, but the first
44 edition covered writing good regular expression patterns in great detail.
46 `Kodos <http://kodos.sf.net/>`_
47 is a graphical regular expression debugger written in Python.
52 Regular Expression Syntax
53 -------------------------
55 A regular expression (or RE) specifies a set of strings that matches it; the
56 functions in this module let you check if a particular string matches a given
57 regular expression (or if a given regular expression matches a particular
58 string, which comes down to the same thing).
60 Regular expressions can be concatenated to form new regular expressions; if *A*
61 and *B* are both regular expressions, then *AB* is also a regular expression.
62 In general, if a string *p* matches *A* and another string *q* matches *B*, the
63 string *pq* will match AB. This holds unless *A* or *B* contain low precedence
64 operations; boundary conditions between *A* and *B*; or have numbered group
65 references. Thus, complex expressions can easily be constructed from simpler
66 primitive expressions like the ones described here. For details of the theory
67 and implementation of regular expressions, consult the Friedl book referenced
68 above, or almost any textbook about compiler construction.
70 A brief explanation of the format of regular expressions follows. For further
71 information and a gentler presentation, consult the :ref:`regex-howto`.
73 Regular expressions can contain both special and ordinary characters. Most
74 ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
75 expressions; they simply match themselves. You can concatenate ordinary
76 characters, so ``last`` matches the string ``'last'``. (In the rest of this
77 section, we'll write RE's in ``this special style``, usually without quotes, and
78 strings to be matched ``'in single quotes'``.)
80 Some characters, like ``'|'`` or ``'('``, are special. Special
81 characters either stand for classes of ordinary characters, or affect
82 how the regular expressions around them are interpreted. Regular
83 expression pattern strings may not contain null bytes, but can specify
84 the null byte using the ``\number`` notation, e.g., ``'\x00'``.
87 The special characters are:
90 (Dot.) In the default mode, this matches any character except a newline. If
91 the :const:`DOTALL` flag has been specified, this matches any character
95 (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
96 matches immediately after each newline.
99 Matches the end of the string or just before the newline at the end of the
100 string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
101 matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
102 only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
103 matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
104 a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
105 the newline, and one at the end of the string.
108 Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
109 many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
110 by any number of 'b's.
113 Causes the resulting RE to match 1 or more repetitions of the preceding RE.
114 ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
118 Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
119 ``ab?`` will match either 'a' or 'ab'.
121 ``*?``, ``+?``, ``??``
122 The ``'*'``, ``'+'``, and ``'?'`` qualifiers are all :dfn:`greedy`; they match
123 as much text as possible. Sometimes this behaviour isn't desired; if the RE
124 ``<.*>`` is matched against ``'<H1>title</H1>'``, it will match the entire
125 string, and not just ``'<H1>'``. Adding ``'?'`` after the qualifier makes it
126 perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
127 characters as possible will be matched. Using ``.*?`` in the previous
128 expression will match only ``'<H1>'``.
131 Specifies that exactly *m* copies of the previous RE should be matched; fewer
132 matches cause the entire RE not to match. For example, ``a{6}`` will match
133 exactly six ``'a'`` characters, but not five.
136 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
137 RE, attempting to match as many repetitions as possible. For example,
138 ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
139 lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
140 example, ``a{4,}b`` will match ``aaaab`` or a thousand ``'a'`` characters
141 followed by a ``b``, but not ``aaab``. The comma may not be omitted or the
142 modifier would be confused with the previously described form.
145 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
146 RE, attempting to match as *few* repetitions as possible. This is the
147 non-greedy version of the previous qualifier. For example, on the
148 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
149 while ``a{3,5}?`` will only match 3 characters.
152 Either escapes special characters (permitting you to match characters like
153 ``'*'``, ``'?'``, and so forth), or signals a special sequence; special
154 sequences are discussed below.
156 If you're not using a raw string to express the pattern, remember that Python
157 also uses the backslash as an escape sequence in string literals; if the escape
158 sequence isn't recognized by Python's parser, the backslash and subsequent
159 character are included in the resulting string. However, if Python would
160 recognize the resulting sequence, the backslash should be repeated twice. This
161 is complicated and hard to understand, so it's highly recommended that you use
162 raw strings for all but the simplest expressions.
165 Used to indicate a set of characters. Characters can be listed individually, or
166 a range of characters can be indicated by giving two characters and separating
167 them by a ``'-'``. Special characters are not active inside sets. For example,
168 ``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
169 ``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
170 ``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
171 as ``\w`` or ``\S`` (defined below) are also acceptable inside a
172 range, although the characters they match depends on whether :const:`LOCALE`
173 or :const:`UNICODE` mode is in force. If you want to include a
174 ``']'`` or a ``'-'`` inside a set, precede it with a backslash, or
175 place it as the first character. The pattern ``[]]`` will match
176 ``']'``, for example.
178 You can match the characters not within a range by :dfn:`complementing` the set.
179 This is indicated by including a ``'^'`` as the first character of the set;
180 ``'^'`` elsewhere will simply match the ``'^'`` character. For example,
181 ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
182 character except ``'^'``.
185 ``A|B``, where A and B can be arbitrary REs, creates a regular expression that
186 will match either A or B. An arbitrary number of REs can be separated by the
187 ``'|'`` in this way. This can be used inside groups (see below) as well. As
188 the target string is scanned, REs separated by ``'|'`` are tried from left to
189 right. When one pattern completely matches, that branch is accepted. This means
190 that once ``A`` matches, ``B`` will not be tested further, even if it would
191 produce a longer overall match. In other words, the ``'|'`` operator is never
192 greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
193 character class, as in ``[|]``.
196 Matches whatever regular expression is inside the parentheses, and indicates the
197 start and end of a group; the contents of a group can be retrieved after a match
198 has been performed, and can be matched later in the string with the ``\number``
199 special sequence, described below. To match the literals ``'('`` or ``')'``,
200 use ``\(`` or ``\)``, or enclose them inside a character class: ``[(] [)]``.
203 This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
204 otherwise). The first character after the ``'?'`` determines what the meaning
205 and further syntax of the construct is. Extensions usually do not create a new
206 group; ``(?P<name>...)`` is the only exception to this rule. Following are the
207 currently supported extensions.
210 (One or more letters from the set ``'i'``, ``'L'``, ``'m'``, ``'s'``,
211 ``'u'``, ``'x'``.) The group matches the empty string; the letters
212 set the corresponding flags: :const:`re.I` (ignore case),
213 :const:`re.L` (locale dependent), :const:`re.M` (multi-line),
214 :const:`re.S` (dot matches all), :const:`re.U` (Unicode dependent),
215 and :const:`re.X` (verbose), for the entire regular expression. (The
216 flags are described in :ref:`contents-of-module-re`.) This
217 is useful if you wish to include the flags as part of the regular
218 expression, instead of passing a *flag* argument to the
219 :func:`compile` function.
221 Note that the ``(?x)`` flag changes how the expression is parsed. It should be
222 used first in the expression string, or after one or more whitespace characters.
223 If there are non-whitespace characters before the flag, the results are
227 A non-grouping version of regular parentheses. Matches whatever regular
228 expression is inside the parentheses, but the substring matched by the group
229 *cannot* be retrieved after performing a match or referenced later in the
233 Similar to regular parentheses, but the substring matched by the group is
234 accessible via the symbolic group name *name*. Group names must be valid Python
235 identifiers, and each group name must be defined only once within a regular
236 expression. A symbolic group is also a numbered group, just as if the group
237 were not named. So the group named 'id' in the example below can also be
238 referenced as the numbered group 1.
240 For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be
241 referenced by its name in arguments to methods of match objects, such as
242 ``m.group('id')`` or ``m.end('id')``, and also by name in pattern text (for
243 example, ``(?P=id)``) and replacement text (such as ``\g<id>``).
246 Matches whatever text was matched by the earlier group named *name*.
249 A comment; the contents of the parentheses are simply ignored.
252 Matches if ``...`` matches next, but doesn't consume any of the string. This is
253 called a lookahead assertion. For example, ``Isaac (?=Asimov)`` will match
254 ``'Isaac '`` only if it's followed by ``'Asimov'``.
257 Matches if ``...`` doesn't match next. This is a negative lookahead assertion.
258 For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
259 followed by ``'Asimov'``.
262 Matches if the current position in the string is preceded by a match for ``...``
263 that ends at the current position. This is called a :dfn:`positive lookbehind
264 assertion`. ``(?<=abc)def`` will find a match in ``abcdef``, since the
265 lookbehind will back up 3 characters and check if the contained pattern matches.
266 The contained pattern must only match strings of some fixed length, meaning that
267 ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
268 patterns which start with positive lookbehind assertions will never match at the
269 beginning of the string being searched; you will most likely want to use the
270 :func:`search` function rather than the :func:`match` function:
273 >>> m = re.search('(?<=abc)def', 'abcdef')
277 This example looks for a word following a hyphen:
279 >>> m = re.search('(?<=-)\w+', 'spam-egg')
284 Matches if the current position in the string is not preceded by a match for
285 ``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
286 positive lookbehind assertions, the contained pattern must only match strings of
287 some fixed length. Patterns which start with negative lookbehind assertions may
288 match at the beginning of the string being searched.
290 ``(?(id/name)yes-pattern|no-pattern)``
291 Will try to match with ``yes-pattern`` if the group with given *id* or *name*
292 exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is optional and
293 can be omitted. For example, ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)`` is a poor email
294 matching pattern, which will match with ``'<user@host.com>'`` as well as
295 ``'user@host.com'``, but not with ``'<user@host.com'``.
297 .. versionadded:: 2.4
299 The special sequences consist of ``'\'`` and a character from the list below.
300 If the ordinary character is not on the list, then the resulting RE will match
301 the second character. For example, ``\$`` matches the character ``'$'``.
304 Matches the contents of the group of the same number. Groups are numbered
305 starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
306 but not ``'the end'`` (note the space after the group). This special sequence
307 can only be used to match one of the first 99 groups. If the first digit of
308 *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
309 a group match, but as the character with octal value *number*. Inside the
310 ``'['`` and ``']'`` of a character class, all numeric escapes are treated as
314 Matches only at the start of the string.
317 Matches the empty string, but only at the beginning or end of a word. A word is
318 defined as a sequence of alphanumeric or underscore characters, so the end of a
319 word is indicated by whitespace or a non-alphanumeric, non-underscore character.
320 Note that ``\b`` is defined as the boundary between ``\w`` and ``\ W``, so the
321 precise set of characters deemed to be alphanumeric depends on the values of the
322 ``UNICODE`` and ``LOCALE`` flags. Inside a character range, ``\b`` represents
323 the backspace character, for compatibility with Python's string literals.
326 Matches the empty string, but only when it is *not* at the beginning or end of a
327 word. This is just the opposite of ``\b``, so is also subject to the settings
328 of ``LOCALE`` and ``UNICODE``.
331 When the :const:`UNICODE` flag is not specified, matches any decimal digit; this
332 is equivalent to the set ``[0-9]``. With :const:`UNICODE`, it will match
333 whatever is classified as a digit in the Unicode character properties database.
336 When the :const:`UNICODE` flag is not specified, matches any non-digit
337 character; this is equivalent to the set ``[^0-9]``. With :const:`UNICODE`, it
338 will match anything other than character marked as digits in the Unicode
339 character properties database.
342 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
343 any whitespace character; this is equivalent to the set ``[ \t\n\r\f\v]``. With
344 :const:`LOCALE`, it will match this set plus whatever characters are defined as
345 space for the current locale. If :const:`UNICODE` is set, this will match the
346 characters ``[ \t\n\r\f\v]`` plus whatever is classified as space in the Unicode
347 character properties database.
350 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
351 any non-whitespace character; this is equivalent to the set ``[^ \t\n\r\f\v]``
352 With :const:`LOCALE`, it will match any character not in this set, and not
353 defined as space in the current locale. If :const:`UNICODE` is set, this will
354 match anything other than ``[ \t\n\r\f\v]`` and characters marked as space in
355 the Unicode character properties database.
358 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
359 any alphanumeric character and the underscore; this is equivalent to the set
360 ``[a-zA-Z0-9_]``. With :const:`LOCALE`, it will match the set ``[0-9_]`` plus
361 whatever characters are defined as alphanumeric for the current locale. If
362 :const:`UNICODE` is set, this will match the characters ``[0-9_]`` plus whatever
363 is classified as alphanumeric in the Unicode character properties database.
366 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
367 any non-alphanumeric character; this is equivalent to the set ``[^a-zA-Z0-9_]``.
368 With :const:`LOCALE`, it will match any character not in the set ``[0-9_]``, and
369 not defined as alphanumeric for the current locale. If :const:`UNICODE` is set,
370 this will match anything other than ``[0-9_]`` and characters marked as
371 alphanumeric in the Unicode character properties database.
374 Matches only at the end of the string.
376 Most of the standard escapes supported by Python string literals are also
377 accepted by the regular expression parser::
383 Octal escapes are included in a limited form: If the first digit is a 0, or if
384 there are three octal digits, it is considered an octal escape. Otherwise, it is
385 a group reference. As for string literals, octal escapes are always at most
386 three digits in length.
389 .. _matching-searching:
391 Matching vs Searching
392 ---------------------
394 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
397 Python offers two different primitive operations based on regular expressions:
398 **match** checks for a match only at the beginning of the string, while
399 **search** checks for a match anywhere in the string (this is what Perl does
402 Note that match may differ from search even when using a regular expression
403 beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in
404 :const:`MULTILINE` mode also immediately following a newline. The "match"
405 operation succeeds only if the pattern matches at the start of the string
406 regardless of mode, or at the starting position given by the optional *pos*
407 argument regardless of whether a newline precedes it.
409 >>> re.match("c", "abcdef") # No match
410 >>> re.search("c", "abcdef") # Match
411 <_sre.SRE_Match object at ...>
414 .. _contents-of-module-re:
419 The module defines several functions, constants, and an exception. Some of the
420 functions are simplified versions of the full featured methods for compiled
421 regular expressions. Most non-trivial applications always use the compiled
425 .. function:: compile(pattern[, flags])
427 Compile a regular expression pattern into a regular expression object, which
428 can be used for matching using its :func:`match` and :func:`search` methods,
431 The expression's behaviour can be modified by specifying a *flags* value.
432 Values can be any of the following variables, combined using bitwise OR (the
437 prog = re.compile(pat)
438 result = prog.match(str)
442 result = re.match(pat, str)
444 but the version using :func:`compile` is more efficient when the expression
445 will be used several times in a single program.
447 .. (The compiled version of the last pattern passed to :func:`re.match` or
448 :func:`re.search` is cached, so programs that use only a single regular
449 expression at a time needn't worry about compiling regular expressions.)
455 Perform case-insensitive matching; expressions like ``[A-Z]`` will match
456 lowercase letters, too. This is not affected by the current locale.
462 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` dependent on the
469 When specified, the pattern character ``'^'`` matches at the beginning of the
470 string and at the beginning of each line (immediately following each newline);
471 and the pattern character ``'$'`` matches at the end of the string and at the
472 end of each line (immediately preceding each newline). By default, ``'^'``
473 matches only at the beginning of the string, and ``'$'`` only at the end of the
474 string and immediately before the newline (if any) at the end of the string.
480 Make the ``'.'`` special character match any character at all, including a
481 newline; without this flag, ``'.'`` will match anything *except* a newline.
487 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S`` dependent
488 on the Unicode character properties database.
490 .. versionadded:: 2.0
496 This flag allows you to write regular expressions that look nicer. Whitespace
497 within the pattern is ignored, except when in a character class or preceded by
498 an unescaped backslash, and, when a line contains a ``'#'`` neither in a
499 character class or preceded by an unescaped backslash, all characters from the
500 leftmost such ``'#'`` through the end of the line are ignored.
502 That means that the two following regular expression objects that match a
503 decimal number are functionally equal::
505 a = re.compile(r"""\d + # the integral part
506 \. # the decimal point
507 \d * # some fractional digits""", re.X)
508 b = re.compile(r"\d+\.\d*")
511 .. function:: search(pattern, string[, flags])
513 Scan through *string* looking for a location where the regular expression
514 *pattern* produces a match, and return a corresponding :class:`MatchObject`
515 instance. Return ``None`` if no position in the string matches the pattern; note
516 that this is different from finding a zero-length match at some point in the
520 .. function:: match(pattern, string[, flags])
522 If zero or more characters at the beginning of *string* match the regular
523 expression *pattern*, return a corresponding :class:`MatchObject` instance.
524 Return ``None`` if the string does not match the pattern; note that this is
525 different from a zero-length match.
529 If you want to locate a match anywhere in *string*, use :meth:`search`
533 .. function:: split(pattern, string[, maxsplit=0])
535 Split *string* by the occurrences of *pattern*. If capturing parentheses are
536 used in *pattern*, then the text of all groups in the pattern are also returned
537 as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
538 splits occur, and the remainder of the string is returned as the final element
539 of the list. (Incompatibility note: in the original Python 1.5 release,
540 *maxsplit* was ignored. This has been fixed in later releases.)
542 >>> re.split('\W+', 'Words, words, words.')
543 ['Words', 'words', 'words', '']
544 >>> re.split('(\W+)', 'Words, words, words.')
545 ['Words', ', ', 'words', ', ', 'words', '.', '']
546 >>> re.split('\W+', 'Words, words, words.', 1)
547 ['Words', 'words, words.']
549 If there are capturing groups in the separator and it matches at the start of
550 the string, the result will start with an empty string. The same holds for
551 the end of the string:
553 >>> re.split('(\W+)', '...words, words...')
554 ['', '...', 'words', ', ', 'words', '...', '']
556 That way, separator components are always found at the same relative
557 indices within the result list (e.g., if there's one capturing group
558 in the separator, the 0th, the 2nd and so forth).
560 Note that *split* will never split a string on an empty pattern match.
563 >>> re.split('x*', 'foo')
565 >>> re.split("(?m)^$", "foo\n\nbar\n")
569 .. function:: findall(pattern, string[, flags])
571 Return all non-overlapping matches of *pattern* in *string*, as a list of
572 strings. If one or more groups are present in the pattern, return a list of
573 groups; this will be a list of tuples if the pattern has more than one group.
574 Empty matches are included in the result unless they touch the beginning of
577 .. versionadded:: 1.5.2
579 .. versionchanged:: 2.4
580 Added the optional flags argument.
583 .. function:: finditer(pattern, string[, flags])
585 Return an :term:`iterator` yielding :class:`MatchObject` instances over all
586 non-overlapping matches for the RE *pattern* in *string*. Empty matches are
587 included in the result unless they touch the beginning of another match.
589 .. versionadded:: 2.2
591 .. versionchanged:: 2.4
592 Added the optional flags argument.
595 .. function:: sub(pattern, repl, string[, count])
597 Return the string obtained by replacing the leftmost non-overlapping occurrences
598 of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
599 *string* is returned unchanged. *repl* can be a string or a function; if it is
600 a string, any backslash escapes in it are processed. That is, ``\n`` is
601 converted to a single newline character, ``\r`` is converted to a linefeed, and
602 so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
603 as ``\6``, are replaced with the substring matched by group 6 in the pattern.
606 >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
607 ... r'static PyObject*\npy_\1(void)\n{',
609 'static PyObject*\npy_myfunc(void)\n{'
611 If *repl* is a function, it is called for every non-overlapping occurrence of
612 *pattern*. The function takes a single match object argument, and returns the
613 replacement string. For example:
615 >>> def dashrepl(matchobj):
616 ... if matchobj.group(0) == '-': return ' '
618 >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
621 The pattern may be a string or an RE object; if you need to specify regular
622 expression flags, you must use a RE object, or use embedded modifiers in a
623 pattern; for example, ``sub("(?i)b+", "x", "bbbb BBBB")`` returns ``'x x'``.
625 The optional argument *count* is the maximum number of pattern occurrences to be
626 replaced; *count* must be a non-negative integer. If omitted or zero, all
627 occurrences will be replaced. Empty matches for the pattern are replaced only
628 when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
631 In addition to character escapes and backreferences as described above,
632 ``\g<name>`` will use the substring matched by the group named ``name``, as
633 defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
634 group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
635 in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
636 reference to group 20, not a reference to group 2 followed by the literal
637 character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
638 substring matched by the RE.
641 .. function:: subn(pattern, repl, string[, count])
643 Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
644 number_of_subs_made)``.
647 .. function:: escape(string)
649 Return *string* with all non-alphanumerics backslashed; this is useful if you
650 want to match an arbitrary literal string that may have regular expression
651 metacharacters in it.
656 Exception raised when a string passed to one of the functions here is not a
657 valid regular expression (for example, it might contain unmatched parentheses)
658 or when some other error occurs during compilation or matching. It is never an
659 error if a string contains no match for a pattern.
664 Regular Expression Objects
665 --------------------------
667 Compiled regular expression objects support the following methods and
671 .. method:: RegexObject.match(string[, pos[, endpos]])
673 If zero or more characters at the beginning of *string* match this regular
674 expression, return a corresponding :class:`MatchObject` instance. Return
675 ``None`` if the string does not match the pattern; note that this is different
676 from a zero-length match.
680 If you want to locate a match anywhere in *string*, use :meth:`search`
683 The optional second parameter *pos* gives an index in the string where the
684 search is to start; it defaults to ``0``. This is not completely equivalent to
685 slicing the string; the ``'^'`` pattern character matches at the real beginning
686 of the string and at positions just after a newline, but not necessarily at the
687 index where the search is to start.
689 The optional parameter *endpos* limits how far the string will be searched; it
690 will be as if the string is *endpos* characters long, so only the characters
691 from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
692 than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
693 expression object, ``rx.match(string, 0, 50)`` is equivalent to
694 ``rx.match(string[:50], 0)``.
696 >>> pattern = re.compile("o")
697 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
698 >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
699 <_sre.SRE_Match object at ...>
702 .. method:: RegexObject.search(string[, pos[, endpos]])
704 Scan through *string* looking for a location where this regular expression
705 produces a match, and return a corresponding :class:`MatchObject` instance.
706 Return ``None`` if no position in the string matches the pattern; note that this
707 is different from finding a zero-length match at some point in the string.
709 The optional *pos* and *endpos* parameters have the same meaning as for the
710 :meth:`match` method.
713 .. method:: RegexObject.split(string[, maxsplit=0])
715 Identical to the :func:`split` function, using the compiled pattern.
718 .. method:: RegexObject.findall(string[, pos[, endpos]])
720 Identical to the :func:`findall` function, using the compiled pattern.
723 .. method:: RegexObject.finditer(string[, pos[, endpos]])
725 Identical to the :func:`finditer` function, using the compiled pattern.
728 .. method:: RegexObject.sub(repl, string[, count=0])
730 Identical to the :func:`sub` function, using the compiled pattern.
733 .. method:: RegexObject.subn(repl, string[, count=0])
735 Identical to the :func:`subn` function, using the compiled pattern.
738 .. attribute:: RegexObject.flags
740 The flags argument used when the RE object was compiled, or ``0`` if no flags
744 .. attribute:: RegexObject.groupindex
746 A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
747 numbers. The dictionary is empty if no symbolic groups were used in the
751 .. attribute:: RegexObject.pattern
753 The pattern string from which the RE object was compiled.
761 Match objects always have a boolean value of :const:`True`, so that you can test
762 whether e.g. :func:`match` resulted in a match with a simple if statement. They
763 support the following methods and attributes:
766 .. method:: MatchObject.expand(template)
768 Return the string obtained by doing backslash substitution on the template
769 string *template*, as done by the :meth:`sub` method. Escapes such as ``\n`` are
770 converted to the appropriate characters, and numeric backreferences (``\1``,
771 ``\2``) and named backreferences (``\g<1>``, ``\g<name>``) are replaced by the
772 contents of the corresponding group.
775 .. method:: MatchObject.group([group1, ...])
777 Returns one or more subgroups of the match. If there is a single argument, the
778 result is a single string; if there are multiple arguments, the result is a
779 tuple with one item per argument. Without arguments, *group1* defaults to zero
780 (the whole match is returned). If a *groupN* argument is zero, the corresponding
781 return value is the entire matching string; if it is in the inclusive range
782 [1..99], it is the string matching the corresponding parenthesized group. If a
783 group number is negative or larger than the number of groups defined in the
784 pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
785 part of the pattern that did not match, the corresponding result is ``None``.
786 If a group is contained in a part of the pattern that matched multiple times,
787 the last match is returned.
789 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
790 >>> m.group(0) # The entire match
792 >>> m.group(1) # The first parenthesized subgroup.
794 >>> m.group(2) # The second parenthesized subgroup.
796 >>> m.group(1, 2) # Multiple arguments give us a tuple.
799 If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
800 arguments may also be strings identifying groups by their group name. If a
801 string argument is not used as a group name in the pattern, an :exc:`IndexError`
804 A moderately complicated example:
806 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
807 >>> m.group('first_name')
809 >>> m.group('last_name')
812 Named groups can also be referred to by their index:
819 If a group matches multiple times, only the last match is accessible:
821 >>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
822 >>> m.group(1) # Returns only the last match.
826 .. method:: MatchObject.groups([default])
828 Return a tuple containing all the subgroups of the match, from 1 up to however
829 many groups are in the pattern. The *default* argument is used for groups that
830 did not participate in the match; it defaults to ``None``. (Incompatibility
831 note: in the original Python 1.5 release, if the tuple was one element long, a
832 string would be returned instead. In later versions (from 1.5.1 on), a
833 singleton tuple is returned in such cases.)
837 >>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
841 If we make the decimal place and everything after it optional, not all groups
842 might participate in the match. These groups will default to ``None`` unless
843 the *default* argument is given:
845 >>> m = re.match(r"(\d+)\.?(\d+)?", "24")
846 >>> m.groups() # Second group defaults to None.
848 >>> m.groups('0') # Now, the second group defaults to '0'.
852 .. method:: MatchObject.groupdict([default])
854 Return a dictionary containing all the *named* subgroups of the match, keyed by
855 the subgroup name. The *default* argument is used for groups that did not
856 participate in the match; it defaults to ``None``. For example:
858 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
860 {'first_name': 'Malcom', 'last_name': 'Reynolds'}
863 .. method:: MatchObject.start([group])
864 MatchObject.end([group])
866 Return the indices of the start and end of the substring matched by *group*;
867 *group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
868 *group* exists but did not contribute to the match. For a match object *m*, and
869 a group *g* that did contribute to the match, the substring matched by group *g*
870 (equivalent to ``m.group(g)``) is ::
872 m.string[m.start(g):m.end(g)]
874 Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
875 null string. For example, after ``m = re.search('b(c?)', 'cba')``,
876 ``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
877 2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
879 An example that will remove *remove_this* from email addresses:
881 >>> email = "tony@tiremove_thisger.net"
882 >>> m = re.search("remove_this", email)
883 >>> email[:m.start()] + email[m.end():]
887 .. method:: MatchObject.span([group])
889 For :class:`MatchObject` *m*, return the 2-tuple ``(m.start(group),
890 m.end(group))``. Note that if *group* did not contribute to the match, this is
891 ``(-1, -1)``. *group* defaults to zero, the entire match.
894 .. attribute:: MatchObject.pos
896 The value of *pos* which was passed to the :func:`search` or :func:`match`
897 method of the :class:`RegexObject`. This is the index into the string at which
898 the RE engine started looking for a match.
901 .. attribute:: MatchObject.endpos
903 The value of *endpos* which was passed to the :func:`search` or :func:`match`
904 method of the :class:`RegexObject`. This is the index into the string beyond
905 which the RE engine will not go.
908 .. attribute:: MatchObject.lastindex
910 The integer index of the last matched capturing group, or ``None`` if no group
911 was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
912 ``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
913 the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
917 .. attribute:: MatchObject.lastgroup
919 The name of the last matched capturing group, or ``None`` if the group didn't
920 have a name, or if no group was matched at all.
923 .. attribute:: MatchObject.re
925 The regular expression object whose :meth:`match` or :meth:`search` method
926 produced this :class:`MatchObject` instance.
929 .. attribute:: MatchObject.string
931 The string passed to :func:`match` or :func:`search`.
941 In this example, we'll use the following helper function to display match
942 objects a little more gracefully:
946 def displaymatch(match):
949 return '<Match: %r, groups=%r>' % (match.group(), match.groups())
951 Suppose you are writing a poker program where a player's hand is represented as
952 a 5-character string with each character representing a card, "a" for ace, "k"
953 for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
954 representing the card with that value.
956 To see if a given string is a valid hand, one could do the following:
958 >>> valid = re.compile(r"[0-9akqj]{5}$")
959 >>> displaymatch(valid.match("ak05q")) # Valid.
960 "<Match: 'ak05q', groups=()>"
961 >>> displaymatch(valid.match("ak05e")) # Invalid.
962 >>> displaymatch(valid.match("ak0")) # Invalid.
963 >>> displaymatch(valid.match("727ak")) # Valid.
964 "<Match: '727ak', groups=()>"
966 That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
967 To match this with a regular expression, one could use backreferences as such:
969 >>> pair = re.compile(r".*(.).*\1")
970 >>> displaymatch(pair.match("717ak")) # Pair of 7s.
971 "<Match: '717', groups=('7',)>"
972 >>> displaymatch(pair.match("718ak")) # No pairs.
973 >>> displaymatch(pair.match("354aa")) # Pair of aces.
974 "<Match: '354aa', groups=('a',)>"
976 To find out what card the pair consists of, one could use the :func:`group`
977 method of :class:`MatchObject` in the following manner:
981 >>> pair.match("717ak").group(1)
984 # Error because re.match() returns None, which doesn't have a group() method:
985 >>> pair.match("718ak").group(1)
986 Traceback (most recent call last):
987 File "<pyshell#23>", line 1, in <module>
988 re.match(r".*(.).*\1", "718ak").group(1)
989 AttributeError: 'NoneType' object has no attribute 'group'
991 >>> pair.match("354aa").group(1)
998 .. index:: single: scanf()
1000 Python does not currently have an equivalent to :cfunc:`scanf`. Regular
1001 expressions are generally more powerful, though also more verbose, than
1002 :cfunc:`scanf` format strings. The table below offers some more-or-less
1003 equivalent mappings between :cfunc:`scanf` format tokens and regular
1006 +--------------------------------+---------------------------------------------+
1007 | :cfunc:`scanf` Token | Regular Expression |
1008 +================================+=============================================+
1010 +--------------------------------+---------------------------------------------+
1011 | ``%5c`` | ``.{5}`` |
1012 +--------------------------------+---------------------------------------------+
1013 | ``%d`` | ``[-+]?\d+`` |
1014 +--------------------------------+---------------------------------------------+
1015 | ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
1016 +--------------------------------+---------------------------------------------+
1017 | ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
1018 +--------------------------------+---------------------------------------------+
1019 | ``%o`` | ``0[0-7]*`` |
1020 +--------------------------------+---------------------------------------------+
1021 | ``%s`` | ``\S+`` |
1022 +--------------------------------+---------------------------------------------+
1023 | ``%u`` | ``\d+`` |
1024 +--------------------------------+---------------------------------------------+
1025 | ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` |
1026 +--------------------------------+---------------------------------------------+
1028 To extract the filename and numbers from a string like ::
1030 /usr/sbin/sendmail - 0 errors, 4 warnings
1032 you would use a :cfunc:`scanf` format like ::
1034 %s - %d errors, %d warnings
1036 The equivalent regular expression would be ::
1038 (\S+) - (\d+) errors, (\d+) warnings
1044 If you create regular expressions that require the engine to perform a lot of
1045 recursion, you may encounter a :exc:`RuntimeError` exception with the message
1046 ``maximum recursion limit`` exceeded. For example, ::
1048 >>> s = 'Begin ' + 1000*'a very long string ' + 'end'
1049 >>> re.match('Begin (\w| )*? end', s).end()
1050 Traceback (most recent call last):
1051 File "<stdin>", line 1, in ?
1052 File "/usr/local/lib/python2.5/re.py", line 132, in match
1053 return _compile(pattern, flags).match(string)
1054 RuntimeError: maximum recursion limit exceeded
1056 You can often restructure your regular expression to avoid recursion.
1058 Starting with Python 2.3, simple uses of the ``*?`` pattern are special-cased to
1059 avoid recursion. Thus, the above regular expression can avoid recursion by
1060 being recast as ``Begin [a-zA-Z0-9_ ]*?end``. As a further benefit, such
1061 regular expressions will run faster than their recursive equivalents.
1064 search() vs. match()
1065 ^^^^^^^^^^^^^^^^^^^^
1067 In a nutshell, :func:`match` only attempts to match a pattern at the beginning
1068 of a string where :func:`search` will match a pattern anywhere in a string.
1071 >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
1072 >>> re.search("o", "dog") # Match as search() looks everywhere in the string.
1073 <_sre.SRE_Match object at ...>
1077 The following applies only to regular expression objects like those created
1078 with ``re.compile("pattern")``, not the primitives ``re.match(pattern,
1079 string)`` or ``re.search(pattern, string)``.
1081 :func:`match` has an optional second parameter that gives an index in the string
1082 where the search is to start:
1084 >>> pattern = re.compile("o")
1085 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
1087 # Equivalent to the above expression as 0 is the default starting index:
1088 >>> pattern.match("dog", 0)
1090 # Match as "o" is the 2nd character of "dog" (index 0 is the first):
1091 >>> pattern.match("dog", 1)
1092 <_sre.SRE_Match object at ...>
1093 >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
1099 :func:`split` splits a string into a list delimited by the passed pattern. The
1100 method is invaluable for converting textual data into data structures that can be
1101 easily read and modified by Python as demonstrated in the following example that
1102 creates a phonebook.
1104 First, here is the input. Normally it may come from a file, here we are using
1105 triple-quoted string syntax:
1107 >>> input = """Ross McFluff: 834.345.1254 155 Elm Street
1109 ... Ronald Heathmore: 892.345.3428 436 Finley Avenue
1110 ... Frank Burger: 925.541.7625 662 South Dogwood Way
1113 ... Heather Albrecht: 548.326.4584 919 Park Place"""
1115 The entries are separated by one or more newlines. Now we convert the string
1116 into a list with each nonempty line having its own entry:
1119 :options: +NORMALIZE_WHITESPACE
1121 >>> entries = re.split("\n+", input)
1123 ['Ross McFluff: 834.345.1254 155 Elm Street',
1124 'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
1125 'Frank Burger: 925.541.7625 662 South Dogwood Way',
1126 'Heather Albrecht: 548.326.4584 919 Park Place']
1128 Finally, split each entry into a list with first name, last name, telephone
1129 number, and address. We use the ``maxsplit`` parameter of :func:`split`
1130 because the address has spaces, our splitting pattern, in it:
1133 :options: +NORMALIZE_WHITESPACE
1135 >>> [re.split(":? ", entry, 3) for entry in entries]
1136 [['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1137 ['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1138 ['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1139 ['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1141 The ``:?`` pattern matches the colon after the last name, so that it does not
1142 occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
1143 house number from the street name:
1146 :options: +NORMALIZE_WHITESPACE
1148 >>> [re.split(":? ", entry, 4) for entry in entries]
1149 [['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1150 ['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1151 ['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1152 ['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1158 :func:`sub` replaces every occurrence of a pattern with a string or the
1159 result of a function. This example demonstrates using :func:`sub` with
1160 a function to "munge" text, or randomize the order of all the characters
1161 in each word of a sentence except for the first and last characters::
1164 ... inner_word = list(m.group(2))
1165 ... random.shuffle(inner_word)
1166 ... return m.group(1) + "".join(inner_word) + m.group(3)
1167 >>> text = "Professor Abdolmalek, please report your absences promptly."
1168 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1169 'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1170 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1171 'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1177 :func:`findall` matches *all* occurrences of a pattern, not just the first
1178 one as :func:`search` does. For example, if one was a writer and wanted to
1179 find all of the adverbs in some text, he or she might use :func:`findall` in
1180 the following manner:
1182 >>> text = "He was carefully disguised but captured quickly by police."
1183 >>> re.findall(r"\w+ly", text)
1184 ['carefully', 'quickly']
1187 Finding all Adverbs and their Positions
1188 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1190 If one wants more information about all matches of a pattern than the matched
1191 text, :func:`finditer` is useful as it provides instances of
1192 :class:`MatchObject` instead of strings. Continuing with the previous example,
1193 if one was a writer who wanted to find all of the adverbs *and their positions*
1194 in some text, he or she would use :func:`finditer` in the following manner:
1196 >>> text = "He was carefully disguised but captured quickly by police."
1197 >>> for m in re.finditer(r"\w+ly", text):
1198 ... print '%02d-%02d: %s' % (m.start(), m.end(), m.group(0))
1206 Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1207 every backslash (``'\'``) in a regular expression would have to be prefixed with
1208 another one to escape it. For example, the two following lines of code are
1209 functionally identical:
1211 >>> re.match(r"\W(.)\1\W", " ff ")
1212 <_sre.SRE_Match object at ...>
1213 >>> re.match("\\W(.)\\1\\W", " ff ")
1214 <_sre.SRE_Match object at ...>
1216 When one wants to match a literal backslash, it must be escaped in the regular
1217 expression. With raw string notation, this means ``r"\\"``. Without raw string
1218 notation, one must use ``"\\\\"``, making the following lines of code
1219 functionally identical:
1221 >>> re.match(r"\\", r"\\")
1222 <_sre.SRE_Match object at ...>
1223 >>> re.match("\\\\", r"\\")
1224 <_sre.SRE_Match object at ...>