2 :mod:`re` --- Regular expression operations
3 ===========================================
6 :synopsis: Regular expression operations.
7 .. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
8 .. sectionauthor:: Andrew M. Kuchling <amk@amk.ca>
13 This module provides regular expression matching operations similar to
14 those found in Perl. Both patterns and strings to be searched can be
15 Unicode strings as well as 8-bit strings. The :mod:`re` module is
18 Regular expressions use the backslash character (``'\'``) to indicate
19 special forms or to allow special characters to be used without invoking
20 their special meaning. This collides with Python's usage of the same
21 character for the same purpose in string literals; for example, to match
22 a literal backslash, one might have to write ``'\\\\'`` as the pattern
23 string, because the regular expression must be ``\\``, and each
24 backslash must be expressed as ``\\`` inside a regular Python string
27 The solution is to use Python's raw string notation for regular expression
28 patterns; backslashes are not handled in any special way in a string literal
29 prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
30 ``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
31 newline. Usually patterns will be expressed in Python code using this raw
34 It is important to note that most regular expression operations are available as
35 module-level functions and :class:`RegexObject` methods. The functions are
36 shortcuts that don't require you to compile a regex object first, but miss some
37 fine-tuning parameters.
41 Mastering Regular Expressions
42 Book on regular expressions by Jeffrey Friedl, published by O'Reilly. The
43 second edition of the book no longer covers Python at all, but the first
44 edition covered writing good regular expression patterns in great detail.
46 `Kodos <http://kodos.sf.net/>`_
47 is a graphical regular expression debugger written in Python.
52 Regular Expression Syntax
53 -------------------------
55 A regular expression (or RE) specifies a set of strings that matches it; the
56 functions in this module let you check if a particular string matches a given
57 regular expression (or if a given regular expression matches a particular
58 string, which comes down to the same thing).
60 Regular expressions can be concatenated to form new regular expressions; if *A*
61 and *B* are both regular expressions, then *AB* is also a regular expression.
62 In general, if a string *p* matches *A* and another string *q* matches *B*, the
63 string *pq* will match AB. This holds unless *A* or *B* contain low precedence
64 operations; boundary conditions between *A* and *B*; or have numbered group
65 references. Thus, complex expressions can easily be constructed from simpler
66 primitive expressions like the ones described here. For details of the theory
67 and implementation of regular expressions, consult the Friedl book referenced
68 above, or almost any textbook about compiler construction.
70 A brief explanation of the format of regular expressions follows. For further
71 information and a gentler presentation, consult the :ref:`regex-howto`.
73 Regular expressions can contain both special and ordinary characters. Most
74 ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
75 expressions; they simply match themselves. You can concatenate ordinary
76 characters, so ``last`` matches the string ``'last'``. (In the rest of this
77 section, we'll write RE's in ``this special style``, usually without quotes, and
78 strings to be matched ``'in single quotes'``.)
80 Some characters, like ``'|'`` or ``'('``, are special. Special
81 characters either stand for classes of ordinary characters, or affect
82 how the regular expressions around them are interpreted. Regular
83 expression pattern strings may not contain null bytes, but can specify
84 the null byte using the ``\number`` notation, e.g., ``'\x00'``.
87 The special characters are:
90 (Dot.) In the default mode, this matches any character except a newline. If
91 the :const:`DOTALL` flag has been specified, this matches any character
95 (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
96 matches immediately after each newline.
99 Matches the end of the string or just before the newline at the end of the
100 string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
101 matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
102 only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
103 matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
104 a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
105 the newline, and one at the end of the string.
108 Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
109 many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
110 by any number of 'b's.
113 Causes the resulting RE to match 1 or more repetitions of the preceding RE.
114 ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
118 Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
119 ``ab?`` will match either 'a' or 'ab'.
121 ``*?``, ``+?``, ``??``
122 The ``'*'``, ``'+'``, and ``'?'`` qualifiers are all :dfn:`greedy`; they match
123 as much text as possible. Sometimes this behaviour isn't desired; if the RE
124 ``<.*>`` is matched against ``'<H1>title</H1>'``, it will match the entire
125 string, and not just ``'<H1>'``. Adding ``'?'`` after the qualifier makes it
126 perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
127 characters as possible will be matched. Using ``.*?`` in the previous
128 expression will match only ``'<H1>'``.
131 Specifies that exactly *m* copies of the previous RE should be matched; fewer
132 matches cause the entire RE not to match. For example, ``a{6}`` will match
133 exactly six ``'a'`` characters, but not five.
136 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
137 RE, attempting to match as many repetitions as possible. For example,
138 ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
139 lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
140 example, ``a{4,}b`` will match ``aaaab`` or a thousand ``'a'`` characters
141 followed by a ``b``, but not ``aaab``. The comma may not be omitted or the
142 modifier would be confused with the previously described form.
145 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
146 RE, attempting to match as *few* repetitions as possible. This is the
147 non-greedy version of the previous qualifier. For example, on the
148 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
149 while ``a{3,5}?`` will only match 3 characters.
152 Either escapes special characters (permitting you to match characters like
153 ``'*'``, ``'?'``, and so forth), or signals a special sequence; special
154 sequences are discussed below.
156 If you're not using a raw string to express the pattern, remember that Python
157 also uses the backslash as an escape sequence in string literals; if the escape
158 sequence isn't recognized by Python's parser, the backslash and subsequent
159 character are included in the resulting string. However, if Python would
160 recognize the resulting sequence, the backslash should be repeated twice. This
161 is complicated and hard to understand, so it's highly recommended that you use
162 raw strings for all but the simplest expressions.
165 Used to indicate a set of characters. Characters can be listed individually, or
166 a range of characters can be indicated by giving two characters and separating
167 them by a ``'-'``. Special characters are not active inside sets. For example,
168 ``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
169 ``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
170 ``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
171 as ``\w`` or ``\S`` (defined below) are also acceptable inside a
172 range, although the characters they match depends on whether :const:`LOCALE`
173 or :const:`UNICODE` mode is in force. If you want to include a
174 ``']'`` or a ``'-'`` inside a set, precede it with a backslash, or
175 place it as the first character. The pattern ``[]]`` will match
176 ``']'``, for example.
178 You can match the characters not within a range by :dfn:`complementing` the set.
179 This is indicated by including a ``'^'`` as the first character of the set;
180 ``'^'`` elsewhere will simply match the ``'^'`` character. For example,
181 ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
182 character except ``'^'``.
184 Note that inside ``[]`` the special forms and special characters lose
185 their meanings and only the syntaxes described here are valid. For
186 example, ``+``, ``*``, ``(``, ``)``, and so on are treated as
187 literals inside ``[]``, and backreferences cannot be used inside
191 ``A|B``, where A and B can be arbitrary REs, creates a regular expression that
192 will match either A or B. An arbitrary number of REs can be separated by the
193 ``'|'`` in this way. This can be used inside groups (see below) as well. As
194 the target string is scanned, REs separated by ``'|'`` are tried from left to
195 right. When one pattern completely matches, that branch is accepted. This means
196 that once ``A`` matches, ``B`` will not be tested further, even if it would
197 produce a longer overall match. In other words, the ``'|'`` operator is never
198 greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
199 character class, as in ``[|]``.
202 Matches whatever regular expression is inside the parentheses, and indicates the
203 start and end of a group; the contents of a group can be retrieved after a match
204 has been performed, and can be matched later in the string with the ``\number``
205 special sequence, described below. To match the literals ``'('`` or ``')'``,
206 use ``\(`` or ``\)``, or enclose them inside a character class: ``[(] [)]``.
209 This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
210 otherwise). The first character after the ``'?'`` determines what the meaning
211 and further syntax of the construct is. Extensions usually do not create a new
212 group; ``(?P<name>...)`` is the only exception to this rule. Following are the
213 currently supported extensions.
216 (One or more letters from the set ``'i'``, ``'L'``, ``'m'``, ``'s'``,
217 ``'u'``, ``'x'``.) The group matches the empty string; the letters
218 set the corresponding flags: :const:`re.I` (ignore case),
219 :const:`re.L` (locale dependent), :const:`re.M` (multi-line),
220 :const:`re.S` (dot matches all), :const:`re.U` (Unicode dependent),
221 and :const:`re.X` (verbose), for the entire regular expression. (The
222 flags are described in :ref:`contents-of-module-re`.) This
223 is useful if you wish to include the flags as part of the regular
224 expression, instead of passing a *flag* argument to the
225 :func:`compile` function.
227 Note that the ``(?x)`` flag changes how the expression is parsed. It should be
228 used first in the expression string, or after one or more whitespace characters.
229 If there are non-whitespace characters before the flag, the results are
233 A non-grouping version of regular parentheses. Matches whatever regular
234 expression is inside the parentheses, but the substring matched by the group
235 *cannot* be retrieved after performing a match or referenced later in the
239 Similar to regular parentheses, but the substring matched by the group is
240 accessible via the symbolic group name *name*. Group names must be valid Python
241 identifiers, and each group name must be defined only once within a regular
242 expression. A symbolic group is also a numbered group, just as if the group
243 were not named. So the group named 'id' in the example below can also be
244 referenced as the numbered group 1.
246 For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be
247 referenced by its name in arguments to methods of match objects, such as
248 ``m.group('id')`` or ``m.end('id')``, and also by name in pattern text (for
249 example, ``(?P=id)``) and replacement text (such as ``\g<id>``).
252 Matches whatever text was matched by the earlier group named *name*.
255 A comment; the contents of the parentheses are simply ignored.
258 Matches if ``...`` matches next, but doesn't consume any of the string. This is
259 called a lookahead assertion. For example, ``Isaac (?=Asimov)`` will match
260 ``'Isaac '`` only if it's followed by ``'Asimov'``.
263 Matches if ``...`` doesn't match next. This is a negative lookahead assertion.
264 For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
265 followed by ``'Asimov'``.
268 Matches if the current position in the string is preceded by a match for ``...``
269 that ends at the current position. This is called a :dfn:`positive lookbehind
270 assertion`. ``(?<=abc)def`` will find a match in ``abcdef``, since the
271 lookbehind will back up 3 characters and check if the contained pattern matches.
272 The contained pattern must only match strings of some fixed length, meaning that
273 ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
274 patterns which start with positive lookbehind assertions will never match at the
275 beginning of the string being searched; you will most likely want to use the
276 :func:`search` function rather than the :func:`match` function:
279 >>> m = re.search('(?<=abc)def', 'abcdef')
283 This example looks for a word following a hyphen:
285 >>> m = re.search('(?<=-)\w+', 'spam-egg')
290 Matches if the current position in the string is not preceded by a match for
291 ``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
292 positive lookbehind assertions, the contained pattern must only match strings of
293 some fixed length. Patterns which start with negative lookbehind assertions may
294 match at the beginning of the string being searched.
296 ``(?(id/name)yes-pattern|no-pattern)``
297 Will try to match with ``yes-pattern`` if the group with given *id* or *name*
298 exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is optional and
299 can be omitted. For example, ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)`` is a poor email
300 matching pattern, which will match with ``'<user@host.com>'`` as well as
301 ``'user@host.com'``, but not with ``'<user@host.com'``.
303 .. versionadded:: 2.4
305 The special sequences consist of ``'\'`` and a character from the list below.
306 If the ordinary character is not on the list, then the resulting RE will match
307 the second character. For example, ``\$`` matches the character ``'$'``.
310 Matches the contents of the group of the same number. Groups are numbered
311 starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
312 but not ``'the end'`` (note the space after the group). This special sequence
313 can only be used to match one of the first 99 groups. If the first digit of
314 *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
315 a group match, but as the character with octal value *number*. Inside the
316 ``'['`` and ``']'`` of a character class, all numeric escapes are treated as
320 Matches only at the start of the string.
323 Matches the empty string, but only at the beginning or end of a word. A word is
324 defined as a sequence of alphanumeric or underscore characters, so the end of a
325 word is indicated by whitespace or a non-alphanumeric, non-underscore character.
326 Note that ``\b`` is defined as the boundary between ``\w`` and ``\ W``, so the
327 precise set of characters deemed to be alphanumeric depends on the values of the
328 ``UNICODE`` and ``LOCALE`` flags. Inside a character range, ``\b`` represents
329 the backspace character, for compatibility with Python's string literals.
332 Matches the empty string, but only when it is *not* at the beginning or end of a
333 word. This is just the opposite of ``\b``, so is also subject to the settings
334 of ``LOCALE`` and ``UNICODE``.
337 When the :const:`UNICODE` flag is not specified, matches any decimal digit; this
338 is equivalent to the set ``[0-9]``. With :const:`UNICODE`, it will match
339 whatever is classified as a digit in the Unicode character properties database.
342 When the :const:`UNICODE` flag is not specified, matches any non-digit
343 character; this is equivalent to the set ``[^0-9]``. With :const:`UNICODE`, it
344 will match anything other than character marked as digits in the Unicode
345 character properties database.
348 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
349 any whitespace character; this is equivalent to the set ``[ \t\n\r\f\v]``. With
350 :const:`LOCALE`, it will match this set plus whatever characters are defined as
351 space for the current locale. If :const:`UNICODE` is set, this will match the
352 characters ``[ \t\n\r\f\v]`` plus whatever is classified as space in the Unicode
353 character properties database.
356 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
357 any non-whitespace character; this is equivalent to the set ``[^ \t\n\r\f\v]``
358 With :const:`LOCALE`, it will match any character not in this set, and not
359 defined as space in the current locale. If :const:`UNICODE` is set, this will
360 match anything other than ``[ \t\n\r\f\v]`` and characters marked as space in
361 the Unicode character properties database.
364 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
365 any alphanumeric character and the underscore; this is equivalent to the set
366 ``[a-zA-Z0-9_]``. With :const:`LOCALE`, it will match the set ``[0-9_]`` plus
367 whatever characters are defined as alphanumeric for the current locale. If
368 :const:`UNICODE` is set, this will match the characters ``[0-9_]`` plus whatever
369 is classified as alphanumeric in the Unicode character properties database.
372 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
373 any non-alphanumeric character; this is equivalent to the set ``[^a-zA-Z0-9_]``.
374 With :const:`LOCALE`, it will match any character not in the set ``[0-9_]``, and
375 not defined as alphanumeric for the current locale. If :const:`UNICODE` is set,
376 this will match anything other than ``[0-9_]`` and characters marked as
377 alphanumeric in the Unicode character properties database.
380 Matches only at the end of the string.
382 Most of the standard escapes supported by Python string literals are also
383 accepted by the regular expression parser::
389 Octal escapes are included in a limited form: If the first digit is a 0, or if
390 there are three octal digits, it is considered an octal escape. Otherwise, it is
391 a group reference. As for string literals, octal escapes are always at most
392 three digits in length.
395 .. _matching-searching:
397 Matching vs Searching
398 ---------------------
400 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
403 Python offers two different primitive operations based on regular expressions:
404 **match** checks for a match only at the beginning of the string, while
405 **search** checks for a match anywhere in the string (this is what Perl does
408 Note that match may differ from search even when using a regular expression
409 beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in
410 :const:`MULTILINE` mode also immediately following a newline. The "match"
411 operation succeeds only if the pattern matches at the start of the string
412 regardless of mode, or at the starting position given by the optional *pos*
413 argument regardless of whether a newline precedes it.
415 >>> re.match("c", "abcdef") # No match
416 >>> re.search("c", "abcdef") # Match
417 <_sre.SRE_Match object at ...>
420 .. _contents-of-module-re:
425 The module defines several functions, constants, and an exception. Some of the
426 functions are simplified versions of the full featured methods for compiled
427 regular expressions. Most non-trivial applications always use the compiled
431 .. function:: compile(pattern[, flags])
433 Compile a regular expression pattern into a regular expression object, which
434 can be used for matching using its :func:`match` and :func:`search` methods,
437 The expression's behaviour can be modified by specifying a *flags* value.
438 Values can be any of the following variables, combined using bitwise OR (the
443 prog = re.compile(pat)
444 result = prog.match(str)
448 result = re.match(pat, str)
450 but the version using :func:`compile` is more efficient when the expression
451 will be used several times in a single program.
453 .. (The compiled version of the last pattern passed to :func:`re.match` or
454 :func:`re.search` is cached, so programs that use only a single regular
455 expression at a time needn't worry about compiling regular expressions.)
461 Perform case-insensitive matching; expressions like ``[A-Z]`` will match
462 lowercase letters, too. This is not affected by the current locale.
468 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` dependent on the
475 When specified, the pattern character ``'^'`` matches at the beginning of the
476 string and at the beginning of each line (immediately following each newline);
477 and the pattern character ``'$'`` matches at the end of the string and at the
478 end of each line (immediately preceding each newline). By default, ``'^'``
479 matches only at the beginning of the string, and ``'$'`` only at the end of the
480 string and immediately before the newline (if any) at the end of the string.
486 Make the ``'.'`` special character match any character at all, including a
487 newline; without this flag, ``'.'`` will match anything *except* a newline.
493 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S`` dependent
494 on the Unicode character properties database.
496 .. versionadded:: 2.0
502 This flag allows you to write regular expressions that look nicer. Whitespace
503 within the pattern is ignored, except when in a character class or preceded by
504 an unescaped backslash, and, when a line contains a ``'#'`` neither in a
505 character class or preceded by an unescaped backslash, all characters from the
506 leftmost such ``'#'`` through the end of the line are ignored.
508 That means that the two following regular expression objects that match a
509 decimal number are functionally equal::
511 a = re.compile(r"""\d + # the integral part
512 \. # the decimal point
513 \d * # some fractional digits""", re.X)
514 b = re.compile(r"\d+\.\d*")
517 .. function:: search(pattern, string[, flags])
519 Scan through *string* looking for a location where the regular expression
520 *pattern* produces a match, and return a corresponding :class:`MatchObject`
521 instance. Return ``None`` if no position in the string matches the pattern; note
522 that this is different from finding a zero-length match at some point in the
526 .. function:: match(pattern, string[, flags])
528 If zero or more characters at the beginning of *string* match the regular
529 expression *pattern*, return a corresponding :class:`MatchObject` instance.
530 Return ``None`` if the string does not match the pattern; note that this is
531 different from a zero-length match.
535 If you want to locate a match anywhere in *string*, use :meth:`search`
539 .. function:: split(pattern, string[, maxsplit=0])
541 Split *string* by the occurrences of *pattern*. If capturing parentheses are
542 used in *pattern*, then the text of all groups in the pattern are also returned
543 as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
544 splits occur, and the remainder of the string is returned as the final element
545 of the list. (Incompatibility note: in the original Python 1.5 release,
546 *maxsplit* was ignored. This has been fixed in later releases.)
548 >>> re.split('\W+', 'Words, words, words.')
549 ['Words', 'words', 'words', '']
550 >>> re.split('(\W+)', 'Words, words, words.')
551 ['Words', ', ', 'words', ', ', 'words', '.', '']
552 >>> re.split('\W+', 'Words, words, words.', 1)
553 ['Words', 'words, words.']
555 If there are capturing groups in the separator and it matches at the start of
556 the string, the result will start with an empty string. The same holds for
557 the end of the string:
559 >>> re.split('(\W+)', '...words, words...')
560 ['', '...', 'words', ', ', 'words', '...', '']
562 That way, separator components are always found at the same relative
563 indices within the result list (e.g., if there's one capturing group
564 in the separator, the 0th, the 2nd and so forth).
566 Note that *split* will never split a string on an empty pattern match.
569 >>> re.split('x*', 'foo')
571 >>> re.split("(?m)^$", "foo\n\nbar\n")
575 .. function:: findall(pattern, string[, flags])
577 Return all non-overlapping matches of *pattern* in *string*, as a list of
578 strings. The *string* is scanned left-to-right, and matches are returned in
579 the order found. If one or more groups are present in the pattern, return a
580 list of groups; this will be a list of tuples if the pattern has more than
581 one group. Empty matches are included in the result unless they touch the
582 beginning of another match.
584 .. versionadded:: 1.5.2
586 .. versionchanged:: 2.4
587 Added the optional flags argument.
590 .. function:: finditer(pattern, string[, flags])
592 Return an :term:`iterator` yielding :class:`MatchObject` instances over all
593 non-overlapping matches for the RE *pattern* in *string*. The *string* is
594 scanned left-to-right, and matches are returned in the order found. Empty
595 matches are included in the result unless they touch the beginning of another
598 .. versionadded:: 2.2
600 .. versionchanged:: 2.4
601 Added the optional flags argument.
604 .. function:: sub(pattern, repl, string[, count])
606 Return the string obtained by replacing the leftmost non-overlapping occurrences
607 of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
608 *string* is returned unchanged. *repl* can be a string or a function; if it is
609 a string, any backslash escapes in it are processed. That is, ``\n`` is
610 converted to a single newline character, ``\r`` is converted to a linefeed, and
611 so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
612 as ``\6``, are replaced with the substring matched by group 6 in the pattern.
615 >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
616 ... r'static PyObject*\npy_\1(void)\n{',
618 'static PyObject*\npy_myfunc(void)\n{'
620 If *repl* is a function, it is called for every non-overlapping occurrence of
621 *pattern*. The function takes a single match object argument, and returns the
622 replacement string. For example:
624 >>> def dashrepl(matchobj):
625 ... if matchobj.group(0) == '-': return ' '
627 >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
630 The pattern may be a string or an RE object; if you need to specify regular
631 expression flags, you must use a RE object, or use embedded modifiers in a
632 pattern; for example, ``sub("(?i)b+", "x", "bbbb BBBB")`` returns ``'x x'``.
634 The optional argument *count* is the maximum number of pattern occurrences to be
635 replaced; *count* must be a non-negative integer. If omitted or zero, all
636 occurrences will be replaced. Empty matches for the pattern are replaced only
637 when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
640 In addition to character escapes and backreferences as described above,
641 ``\g<name>`` will use the substring matched by the group named ``name``, as
642 defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
643 group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
644 in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
645 reference to group 20, not a reference to group 2 followed by the literal
646 character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
647 substring matched by the RE.
650 .. function:: subn(pattern, repl, string[, count])
652 Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
653 number_of_subs_made)``.
656 .. function:: escape(string)
658 Return *string* with all non-alphanumerics backslashed; this is useful if you
659 want to match an arbitrary literal string that may have regular expression
660 metacharacters in it.
665 Exception raised when a string passed to one of the functions here is not a
666 valid regular expression (for example, it might contain unmatched parentheses)
667 or when some other error occurs during compilation or matching. It is never an
668 error if a string contains no match for a pattern.
673 Regular Expression Objects
674 --------------------------
676 Compiled regular expression objects support the following methods and
680 .. method:: RegexObject.match(string[, pos[, endpos]])
682 If zero or more characters at the beginning of *string* match this regular
683 expression, return a corresponding :class:`MatchObject` instance. Return
684 ``None`` if the string does not match the pattern; note that this is different
685 from a zero-length match.
689 If you want to locate a match anywhere in *string*, use :meth:`search`
692 The optional second parameter *pos* gives an index in the string where the
693 search is to start; it defaults to ``0``. This is not completely equivalent to
694 slicing the string; the ``'^'`` pattern character matches at the real beginning
695 of the string and at positions just after a newline, but not necessarily at the
696 index where the search is to start.
698 The optional parameter *endpos* limits how far the string will be searched; it
699 will be as if the string is *endpos* characters long, so only the characters
700 from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
701 than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
702 expression object, ``rx.match(string, 0, 50)`` is equivalent to
703 ``rx.match(string[:50], 0)``.
705 >>> pattern = re.compile("o")
706 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
707 >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
708 <_sre.SRE_Match object at ...>
711 .. method:: RegexObject.search(string[, pos[, endpos]])
713 Scan through *string* looking for a location where this regular expression
714 produces a match, and return a corresponding :class:`MatchObject` instance.
715 Return ``None`` if no position in the string matches the pattern; note that this
716 is different from finding a zero-length match at some point in the string.
718 The optional *pos* and *endpos* parameters have the same meaning as for the
719 :meth:`match` method.
722 .. method:: RegexObject.split(string[, maxsplit=0])
724 Identical to the :func:`split` function, using the compiled pattern.
727 .. method:: RegexObject.findall(string[, pos[, endpos]])
729 Identical to the :func:`findall` function, using the compiled pattern.
732 .. method:: RegexObject.finditer(string[, pos[, endpos]])
734 Identical to the :func:`finditer` function, using the compiled pattern.
737 .. method:: RegexObject.sub(repl, string[, count=0])
739 Identical to the :func:`sub` function, using the compiled pattern.
742 .. method:: RegexObject.subn(repl, string[, count=0])
744 Identical to the :func:`subn` function, using the compiled pattern.
747 .. attribute:: RegexObject.flags
749 The flags argument used when the RE object was compiled, or ``0`` if no flags
753 .. attribute:: RegexObject.groups
755 The number of capturing groups in the pattern.
758 .. attribute:: RegexObject.groupindex
760 A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
761 numbers. The dictionary is empty if no symbolic groups were used in the
765 .. attribute:: RegexObject.pattern
767 The pattern string from which the RE object was compiled.
775 Match objects always have a boolean value of :const:`True`, so that you can test
776 whether e.g. :func:`match` resulted in a match with a simple if statement. They
777 support the following methods and attributes:
780 .. method:: MatchObject.expand(template)
782 Return the string obtained by doing backslash substitution on the template
783 string *template*, as done by the :meth:`sub` method. Escapes such as ``\n`` are
784 converted to the appropriate characters, and numeric backreferences (``\1``,
785 ``\2``) and named backreferences (``\g<1>``, ``\g<name>``) are replaced by the
786 contents of the corresponding group.
789 .. method:: MatchObject.group([group1, ...])
791 Returns one or more subgroups of the match. If there is a single argument, the
792 result is a single string; if there are multiple arguments, the result is a
793 tuple with one item per argument. Without arguments, *group1* defaults to zero
794 (the whole match is returned). If a *groupN* argument is zero, the corresponding
795 return value is the entire matching string; if it is in the inclusive range
796 [1..99], it is the string matching the corresponding parenthesized group. If a
797 group number is negative or larger than the number of groups defined in the
798 pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
799 part of the pattern that did not match, the corresponding result is ``None``.
800 If a group is contained in a part of the pattern that matched multiple times,
801 the last match is returned.
803 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
804 >>> m.group(0) # The entire match
806 >>> m.group(1) # The first parenthesized subgroup.
808 >>> m.group(2) # The second parenthesized subgroup.
810 >>> m.group(1, 2) # Multiple arguments give us a tuple.
813 If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
814 arguments may also be strings identifying groups by their group name. If a
815 string argument is not used as a group name in the pattern, an :exc:`IndexError`
818 A moderately complicated example:
820 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
821 >>> m.group('first_name')
823 >>> m.group('last_name')
826 Named groups can also be referred to by their index:
833 If a group matches multiple times, only the last match is accessible:
835 >>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
836 >>> m.group(1) # Returns only the last match.
840 .. method:: MatchObject.groups([default])
842 Return a tuple containing all the subgroups of the match, from 1 up to however
843 many groups are in the pattern. The *default* argument is used for groups that
844 did not participate in the match; it defaults to ``None``. (Incompatibility
845 note: in the original Python 1.5 release, if the tuple was one element long, a
846 string would be returned instead. In later versions (from 1.5.1 on), a
847 singleton tuple is returned in such cases.)
851 >>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
855 If we make the decimal place and everything after it optional, not all groups
856 might participate in the match. These groups will default to ``None`` unless
857 the *default* argument is given:
859 >>> m = re.match(r"(\d+)\.?(\d+)?", "24")
860 >>> m.groups() # Second group defaults to None.
862 >>> m.groups('0') # Now, the second group defaults to '0'.
866 .. method:: MatchObject.groupdict([default])
868 Return a dictionary containing all the *named* subgroups of the match, keyed by
869 the subgroup name. The *default* argument is used for groups that did not
870 participate in the match; it defaults to ``None``. For example:
872 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
874 {'first_name': 'Malcom', 'last_name': 'Reynolds'}
877 .. method:: MatchObject.start([group])
878 MatchObject.end([group])
880 Return the indices of the start and end of the substring matched by *group*;
881 *group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
882 *group* exists but did not contribute to the match. For a match object *m*, and
883 a group *g* that did contribute to the match, the substring matched by group *g*
884 (equivalent to ``m.group(g)``) is ::
886 m.string[m.start(g):m.end(g)]
888 Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
889 null string. For example, after ``m = re.search('b(c?)', 'cba')``,
890 ``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
891 2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
893 An example that will remove *remove_this* from email addresses:
895 >>> email = "tony@tiremove_thisger.net"
896 >>> m = re.search("remove_this", email)
897 >>> email[:m.start()] + email[m.end():]
901 .. method:: MatchObject.span([group])
903 For :class:`MatchObject` *m*, return the 2-tuple ``(m.start(group),
904 m.end(group))``. Note that if *group* did not contribute to the match, this is
905 ``(-1, -1)``. *group* defaults to zero, the entire match.
908 .. attribute:: MatchObject.pos
910 The value of *pos* which was passed to the :func:`search` or :func:`match`
911 method of the :class:`RegexObject`. This is the index into the string at which
912 the RE engine started looking for a match.
915 .. attribute:: MatchObject.endpos
917 The value of *endpos* which was passed to the :func:`search` or :func:`match`
918 method of the :class:`RegexObject`. This is the index into the string beyond
919 which the RE engine will not go.
922 .. attribute:: MatchObject.lastindex
924 The integer index of the last matched capturing group, or ``None`` if no group
925 was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
926 ``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
927 the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
931 .. attribute:: MatchObject.lastgroup
933 The name of the last matched capturing group, or ``None`` if the group didn't
934 have a name, or if no group was matched at all.
937 .. attribute:: MatchObject.re
939 The regular expression object whose :meth:`match` or :meth:`search` method
940 produced this :class:`MatchObject` instance.
943 .. attribute:: MatchObject.string
945 The string passed to :func:`match` or :func:`search`.
955 In this example, we'll use the following helper function to display match
956 objects a little more gracefully:
960 def displaymatch(match):
963 return '<Match: %r, groups=%r>' % (match.group(), match.groups())
965 Suppose you are writing a poker program where a player's hand is represented as
966 a 5-character string with each character representing a card, "a" for ace, "k"
967 for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
968 representing the card with that value.
970 To see if a given string is a valid hand, one could do the following:
972 >>> valid = re.compile(r"[0-9akqj]{5}$")
973 >>> displaymatch(valid.match("ak05q")) # Valid.
974 "<Match: 'ak05q', groups=()>"
975 >>> displaymatch(valid.match("ak05e")) # Invalid.
976 >>> displaymatch(valid.match("ak0")) # Invalid.
977 >>> displaymatch(valid.match("727ak")) # Valid.
978 "<Match: '727ak', groups=()>"
980 That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
981 To match this with a regular expression, one could use backreferences as such:
983 >>> pair = re.compile(r".*(.).*\1")
984 >>> displaymatch(pair.match("717ak")) # Pair of 7s.
985 "<Match: '717', groups=('7',)>"
986 >>> displaymatch(pair.match("718ak")) # No pairs.
987 >>> displaymatch(pair.match("354aa")) # Pair of aces.
988 "<Match: '354aa', groups=('a',)>"
990 To find out what card the pair consists of, one could use the :func:`group`
991 method of :class:`MatchObject` in the following manner:
995 >>> pair.match("717ak").group(1)
998 # Error because re.match() returns None, which doesn't have a group() method:
999 >>> pair.match("718ak").group(1)
1000 Traceback (most recent call last):
1001 File "<pyshell#23>", line 1, in <module>
1002 re.match(r".*(.).*\1", "718ak").group(1)
1003 AttributeError: 'NoneType' object has no attribute 'group'
1005 >>> pair.match("354aa").group(1)
1012 .. index:: single: scanf()
1014 Python does not currently have an equivalent to :cfunc:`scanf`. Regular
1015 expressions are generally more powerful, though also more verbose, than
1016 :cfunc:`scanf` format strings. The table below offers some more-or-less
1017 equivalent mappings between :cfunc:`scanf` format tokens and regular
1020 +--------------------------------+---------------------------------------------+
1021 | :cfunc:`scanf` Token | Regular Expression |
1022 +================================+=============================================+
1024 +--------------------------------+---------------------------------------------+
1025 | ``%5c`` | ``.{5}`` |
1026 +--------------------------------+---------------------------------------------+
1027 | ``%d`` | ``[-+]?\d+`` |
1028 +--------------------------------+---------------------------------------------+
1029 | ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
1030 +--------------------------------+---------------------------------------------+
1031 | ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
1032 +--------------------------------+---------------------------------------------+
1033 | ``%o`` | ``0[0-7]*`` |
1034 +--------------------------------+---------------------------------------------+
1035 | ``%s`` | ``\S+`` |
1036 +--------------------------------+---------------------------------------------+
1037 | ``%u`` | ``\d+`` |
1038 +--------------------------------+---------------------------------------------+
1039 | ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` |
1040 +--------------------------------+---------------------------------------------+
1042 To extract the filename and numbers from a string like ::
1044 /usr/sbin/sendmail - 0 errors, 4 warnings
1046 you would use a :cfunc:`scanf` format like ::
1048 %s - %d errors, %d warnings
1050 The equivalent regular expression would be ::
1052 (\S+) - (\d+) errors, (\d+) warnings
1058 If you create regular expressions that require the engine to perform a lot of
1059 recursion, you may encounter a :exc:`RuntimeError` exception with the message
1060 ``maximum recursion limit`` exceeded. For example, ::
1062 >>> s = 'Begin ' + 1000*'a very long string ' + 'end'
1063 >>> re.match('Begin (\w| )*? end', s).end()
1064 Traceback (most recent call last):
1065 File "<stdin>", line 1, in ?
1066 File "/usr/local/lib/python2.5/re.py", line 132, in match
1067 return _compile(pattern, flags).match(string)
1068 RuntimeError: maximum recursion limit exceeded
1070 You can often restructure your regular expression to avoid recursion.
1072 Starting with Python 2.3, simple uses of the ``*?`` pattern are special-cased to
1073 avoid recursion. Thus, the above regular expression can avoid recursion by
1074 being recast as ``Begin [a-zA-Z0-9_ ]*?end``. As a further benefit, such
1075 regular expressions will run faster than their recursive equivalents.
1078 search() vs. match()
1079 ^^^^^^^^^^^^^^^^^^^^
1081 In a nutshell, :func:`match` only attempts to match a pattern at the beginning
1082 of a string where :func:`search` will match a pattern anywhere in a string.
1085 >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
1086 >>> re.search("o", "dog") # Match as search() looks everywhere in the string.
1087 <_sre.SRE_Match object at ...>
1091 The following applies only to regular expression objects like those created
1092 with ``re.compile("pattern")``, not the primitives ``re.match(pattern,
1093 string)`` or ``re.search(pattern, string)``.
1095 :func:`match` has an optional second parameter that gives an index in the string
1096 where the search is to start:
1098 >>> pattern = re.compile("o")
1099 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
1101 # Equivalent to the above expression as 0 is the default starting index:
1102 >>> pattern.match("dog", 0)
1104 # Match as "o" is the 2nd character of "dog" (index 0 is the first):
1105 >>> pattern.match("dog", 1)
1106 <_sre.SRE_Match object at ...>
1107 >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
1113 :func:`split` splits a string into a list delimited by the passed pattern. The
1114 method is invaluable for converting textual data into data structures that can be
1115 easily read and modified by Python as demonstrated in the following example that
1116 creates a phonebook.
1118 First, here is the input. Normally it may come from a file, here we are using
1119 triple-quoted string syntax:
1121 >>> input = """Ross McFluff: 834.345.1254 155 Elm Street
1123 ... Ronald Heathmore: 892.345.3428 436 Finley Avenue
1124 ... Frank Burger: 925.541.7625 662 South Dogwood Way
1127 ... Heather Albrecht: 548.326.4584 919 Park Place"""
1129 The entries are separated by one or more newlines. Now we convert the string
1130 into a list with each nonempty line having its own entry:
1133 :options: +NORMALIZE_WHITESPACE
1135 >>> entries = re.split("\n+", input)
1137 ['Ross McFluff: 834.345.1254 155 Elm Street',
1138 'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
1139 'Frank Burger: 925.541.7625 662 South Dogwood Way',
1140 'Heather Albrecht: 548.326.4584 919 Park Place']
1142 Finally, split each entry into a list with first name, last name, telephone
1143 number, and address. We use the ``maxsplit`` parameter of :func:`split`
1144 because the address has spaces, our splitting pattern, in it:
1147 :options: +NORMALIZE_WHITESPACE
1149 >>> [re.split(":? ", entry, 3) for entry in entries]
1150 [['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1151 ['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1152 ['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1153 ['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1155 The ``:?`` pattern matches the colon after the last name, so that it does not
1156 occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
1157 house number from the street name:
1160 :options: +NORMALIZE_WHITESPACE
1162 >>> [re.split(":? ", entry, 4) for entry in entries]
1163 [['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1164 ['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1165 ['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1166 ['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1172 :func:`sub` replaces every occurrence of a pattern with a string or the
1173 result of a function. This example demonstrates using :func:`sub` with
1174 a function to "munge" text, or randomize the order of all the characters
1175 in each word of a sentence except for the first and last characters::
1178 ... inner_word = list(m.group(2))
1179 ... random.shuffle(inner_word)
1180 ... return m.group(1) + "".join(inner_word) + m.group(3)
1181 >>> text = "Professor Abdolmalek, please report your absences promptly."
1182 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1183 'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1184 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1185 'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1191 :func:`findall` matches *all* occurrences of a pattern, not just the first
1192 one as :func:`search` does. For example, if one was a writer and wanted to
1193 find all of the adverbs in some text, he or she might use :func:`findall` in
1194 the following manner:
1196 >>> text = "He was carefully disguised but captured quickly by police."
1197 >>> re.findall(r"\w+ly", text)
1198 ['carefully', 'quickly']
1201 Finding all Adverbs and their Positions
1202 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1204 If one wants more information about all matches of a pattern than the matched
1205 text, :func:`finditer` is useful as it provides instances of
1206 :class:`MatchObject` instead of strings. Continuing with the previous example,
1207 if one was a writer who wanted to find all of the adverbs *and their positions*
1208 in some text, he or she would use :func:`finditer` in the following manner:
1210 >>> text = "He was carefully disguised but captured quickly by police."
1211 >>> for m in re.finditer(r"\w+ly", text):
1212 ... print '%02d-%02d: %s' % (m.start(), m.end(), m.group(0))
1220 Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1221 every backslash (``'\'``) in a regular expression would have to be prefixed with
1222 another one to escape it. For example, the two following lines of code are
1223 functionally identical:
1225 >>> re.match(r"\W(.)\1\W", " ff ")
1226 <_sre.SRE_Match object at ...>
1227 >>> re.match("\\W(.)\\1\\W", " ff ")
1228 <_sre.SRE_Match object at ...>
1230 When one wants to match a literal backslash, it must be escaped in the regular
1231 expression. With raw string notation, this means ``r"\\"``. Without raw string
1232 notation, one must use ``"\\\\"``, making the following lines of code
1233 functionally identical:
1235 >>> re.match(r"\\", r"\\")
1236 <_sre.SRE_Match object at ...>
1237 >>> re.match("\\\\", r"\\")
1238 <_sre.SRE_Match object at ...>