2 :mod:`re` --- Regular expression operations
3 ===========================================
6 :synopsis: Regular expression operations.
7 .. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
8 .. sectionauthor:: Andrew M. Kuchling <amk@amk.ca>
11 This module provides regular expression matching operations similar to
12 those found in Perl. Both patterns and strings to be searched can be
13 Unicode strings as well as 8-bit strings.
15 Regular expressions use the backslash character (``'\'``) to indicate
16 special forms or to allow special characters to be used without invoking
17 their special meaning. This collides with Python's usage of the same
18 character for the same purpose in string literals; for example, to match
19 a literal backslash, one might have to write ``'\\\\'`` as the pattern
20 string, because the regular expression must be ``\\``, and each
21 backslash must be expressed as ``\\`` inside a regular Python string
24 The solution is to use Python's raw string notation for regular expression
25 patterns; backslashes are not handled in any special way in a string literal
26 prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
27 ``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
28 newline. Usually patterns will be expressed in Python code using this raw
31 It is important to note that most regular expression operations are available as
32 module-level functions and :class:`RegexObject` methods. The functions are
33 shortcuts that don't require you to compile a regex object first, but miss some
34 fine-tuning parameters.
38 Mastering Regular Expressions
39 Book on regular expressions by Jeffrey Friedl, published by O'Reilly. The
40 second edition of the book no longer covers Python at all, but the first
41 edition covered writing good regular expression patterns in great detail.
46 Regular Expression Syntax
47 -------------------------
49 A regular expression (or RE) specifies a set of strings that matches it; the
50 functions in this module let you check if a particular string matches a given
51 regular expression (or if a given regular expression matches a particular
52 string, which comes down to the same thing).
54 Regular expressions can be concatenated to form new regular expressions; if *A*
55 and *B* are both regular expressions, then *AB* is also a regular expression.
56 In general, if a string *p* matches *A* and another string *q* matches *B*, the
57 string *pq* will match AB. This holds unless *A* or *B* contain low precedence
58 operations; boundary conditions between *A* and *B*; or have numbered group
59 references. Thus, complex expressions can easily be constructed from simpler
60 primitive expressions like the ones described here. For details of the theory
61 and implementation of regular expressions, consult the Friedl book referenced
62 above, or almost any textbook about compiler construction.
64 A brief explanation of the format of regular expressions follows. For further
65 information and a gentler presentation, consult the :ref:`regex-howto`.
67 Regular expressions can contain both special and ordinary characters. Most
68 ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
69 expressions; they simply match themselves. You can concatenate ordinary
70 characters, so ``last`` matches the string ``'last'``. (In the rest of this
71 section, we'll write RE's in ``this special style``, usually without quotes, and
72 strings to be matched ``'in single quotes'``.)
74 Some characters, like ``'|'`` or ``'('``, are special. Special
75 characters either stand for classes of ordinary characters, or affect
76 how the regular expressions around them are interpreted. Regular
77 expression pattern strings may not contain null bytes, but can specify
78 the null byte using the ``\number`` notation, e.g., ``'\x00'``.
81 The special characters are:
84 (Dot.) In the default mode, this matches any character except a newline. If
85 the :const:`DOTALL` flag has been specified, this matches any character
89 (Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
90 matches immediately after each newline.
93 Matches the end of the string or just before the newline at the end of the
94 string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
95 matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
96 only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
97 matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
98 a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
99 the newline, and one at the end of the string.
102 Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
103 many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
104 by any number of 'b's.
107 Causes the resulting RE to match 1 or more repetitions of the preceding RE.
108 ``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
112 Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
113 ``ab?`` will match either 'a' or 'ab'.
115 ``*?``, ``+?``, ``??``
116 The ``'*'``, ``'+'``, and ``'?'`` qualifiers are all :dfn:`greedy`; they match
117 as much text as possible. Sometimes this behaviour isn't desired; if the RE
118 ``<.*>`` is matched against ``'<H1>title</H1>'``, it will match the entire
119 string, and not just ``'<H1>'``. Adding ``'?'`` after the qualifier makes it
120 perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
121 characters as possible will be matched. Using ``.*?`` in the previous
122 expression will match only ``'<H1>'``.
125 Specifies that exactly *m* copies of the previous RE should be matched; fewer
126 matches cause the entire RE not to match. For example, ``a{6}`` will match
127 exactly six ``'a'`` characters, but not five.
130 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
131 RE, attempting to match as many repetitions as possible. For example,
132 ``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
133 lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
134 example, ``a{4,}b`` will match ``aaaab`` or a thousand ``'a'`` characters
135 followed by a ``b``, but not ``aaab``. The comma may not be omitted or the
136 modifier would be confused with the previously described form.
139 Causes the resulting RE to match from *m* to *n* repetitions of the preceding
140 RE, attempting to match as *few* repetitions as possible. This is the
141 non-greedy version of the previous qualifier. For example, on the
142 6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
143 while ``a{3,5}?`` will only match 3 characters.
146 Either escapes special characters (permitting you to match characters like
147 ``'*'``, ``'?'``, and so forth), or signals a special sequence; special
148 sequences are discussed below.
150 If you're not using a raw string to express the pattern, remember that Python
151 also uses the backslash as an escape sequence in string literals; if the escape
152 sequence isn't recognized by Python's parser, the backslash and subsequent
153 character are included in the resulting string. However, if Python would
154 recognize the resulting sequence, the backslash should be repeated twice. This
155 is complicated and hard to understand, so it's highly recommended that you use
156 raw strings for all but the simplest expressions.
159 Used to indicate a set of characters. Characters can be listed individually, or
160 a range of characters can be indicated by giving two characters and separating
161 them by a ``'-'``. Special characters are not active inside sets. For example,
162 ``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
163 ``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
164 ``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
165 as ``\w`` or ``\S`` (defined below) are also acceptable inside a
166 range, although the characters they match depends on whether :const:`LOCALE`
167 or :const:`UNICODE` mode is in force. If you want to include a
168 ``']'`` or a ``'-'`` inside a set, precede it with a backslash, or
169 place it as the first character. The pattern ``[]]`` will match
170 ``']'``, for example.
172 You can match the characters not within a range by :dfn:`complementing` the set.
173 This is indicated by including a ``'^'`` as the first character of the set;
174 ``'^'`` elsewhere will simply match the ``'^'`` character. For example,
175 ``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
176 character except ``'^'``.
178 Note that inside ``[]`` the special forms and special characters lose
179 their meanings and only the syntaxes described here are valid. For
180 example, ``+``, ``*``, ``(``, ``)``, and so on are treated as
181 literals inside ``[]``, and backreferences cannot be used inside
185 ``A|B``, where A and B can be arbitrary REs, creates a regular expression that
186 will match either A or B. An arbitrary number of REs can be separated by the
187 ``'|'`` in this way. This can be used inside groups (see below) as well. As
188 the target string is scanned, REs separated by ``'|'`` are tried from left to
189 right. When one pattern completely matches, that branch is accepted. This means
190 that once ``A`` matches, ``B`` will not be tested further, even if it would
191 produce a longer overall match. In other words, the ``'|'`` operator is never
192 greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
193 character class, as in ``[|]``.
196 Matches whatever regular expression is inside the parentheses, and indicates the
197 start and end of a group; the contents of a group can be retrieved after a match
198 has been performed, and can be matched later in the string with the ``\number``
199 special sequence, described below. To match the literals ``'('`` or ``')'``,
200 use ``\(`` or ``\)``, or enclose them inside a character class: ``[(] [)]``.
203 This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
204 otherwise). The first character after the ``'?'`` determines what the meaning
205 and further syntax of the construct is. Extensions usually do not create a new
206 group; ``(?P<name>...)`` is the only exception to this rule. Following are the
207 currently supported extensions.
210 (One or more letters from the set ``'i'``, ``'L'``, ``'m'``, ``'s'``,
211 ``'u'``, ``'x'``.) The group matches the empty string; the letters
212 set the corresponding flags: :const:`re.I` (ignore case),
213 :const:`re.L` (locale dependent), :const:`re.M` (multi-line),
214 :const:`re.S` (dot matches all), :const:`re.U` (Unicode dependent),
215 and :const:`re.X` (verbose), for the entire regular expression. (The
216 flags are described in :ref:`contents-of-module-re`.) This
217 is useful if you wish to include the flags as part of the regular
218 expression, instead of passing a *flag* argument to the
219 :func:`compile` function.
221 Note that the ``(?x)`` flag changes how the expression is parsed. It should be
222 used first in the expression string, or after one or more whitespace characters.
223 If there are non-whitespace characters before the flag, the results are
227 A non-grouping version of regular parentheses. Matches whatever regular
228 expression is inside the parentheses, but the substring matched by the group
229 *cannot* be retrieved after performing a match or referenced later in the
233 Similar to regular parentheses, but the substring matched by the group is
234 accessible within the rest of the regular expression via the symbolic group
235 name *name*. Group names must be valid Python identifiers, and each group
236 name must be defined only once within a regular expression. A symbolic group
237 is also a numbered group, just as if the group were not named. So the group
238 named ``id`` in the example below can also be referenced as the numbered group
241 For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be
242 referenced by its name in arguments to methods of match objects, such as
243 ``m.group('id')`` or ``m.end('id')``, and also by name in the regular
244 expression itself (using ``(?P=id)``) and replacement text given to
245 ``.sub()`` (using ``\g<id>``).
248 Matches whatever text was matched by the earlier group named *name*.
251 A comment; the contents of the parentheses are simply ignored.
254 Matches if ``...`` matches next, but doesn't consume any of the string. This is
255 called a lookahead assertion. For example, ``Isaac (?=Asimov)`` will match
256 ``'Isaac '`` only if it's followed by ``'Asimov'``.
259 Matches if ``...`` doesn't match next. This is a negative lookahead assertion.
260 For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
261 followed by ``'Asimov'``.
264 Matches if the current position in the string is preceded by a match for ``...``
265 that ends at the current position. This is called a :dfn:`positive lookbehind
266 assertion`. ``(?<=abc)def`` will find a match in ``abcdef``, since the
267 lookbehind will back up 3 characters and check if the contained pattern matches.
268 The contained pattern must only match strings of some fixed length, meaning that
269 ``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
270 patterns which start with positive lookbehind assertions will never match at the
271 beginning of the string being searched; you will most likely want to use the
272 :func:`search` function rather than the :func:`match` function:
275 >>> m = re.search('(?<=abc)def', 'abcdef')
279 This example looks for a word following a hyphen:
281 >>> m = re.search('(?<=-)\w+', 'spam-egg')
286 Matches if the current position in the string is not preceded by a match for
287 ``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
288 positive lookbehind assertions, the contained pattern must only match strings of
289 some fixed length. Patterns which start with negative lookbehind assertions may
290 match at the beginning of the string being searched.
292 ``(?(id/name)yes-pattern|no-pattern)``
293 Will try to match with ``yes-pattern`` if the group with given *id* or *name*
294 exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is optional and
295 can be omitted. For example, ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)`` is a poor email
296 matching pattern, which will match with ``'<user@host.com>'`` as well as
297 ``'user@host.com'``, but not with ``'<user@host.com'``.
299 .. versionadded:: 2.4
301 The special sequences consist of ``'\'`` and a character from the list below.
302 If the ordinary character is not on the list, then the resulting RE will match
303 the second character. For example, ``\$`` matches the character ``'$'``.
306 Matches the contents of the group of the same number. Groups are numbered
307 starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
308 but not ``'the end'`` (note the space after the group). This special sequence
309 can only be used to match one of the first 99 groups. If the first digit of
310 *number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
311 a group match, but as the character with octal value *number*. Inside the
312 ``'['`` and ``']'`` of a character class, all numeric escapes are treated as
316 Matches only at the start of the string.
319 Matches the empty string, but only at the beginning or end of a word. A word is
320 defined as a sequence of alphanumeric or underscore characters, so the end of a
321 word is indicated by whitespace or a non-alphanumeric, non-underscore character.
322 Note that ``\b`` is defined as the boundary between ``\w`` and ``\ W``, so the
323 precise set of characters deemed to be alphanumeric depends on the values of the
324 ``UNICODE`` and ``LOCALE`` flags. Inside a character range, ``\b`` represents
325 the backspace character, for compatibility with Python's string literals.
328 Matches the empty string, but only when it is *not* at the beginning or end of a
329 word. This is just the opposite of ``\b``, so is also subject to the settings
330 of ``LOCALE`` and ``UNICODE``.
333 When the :const:`UNICODE` flag is not specified, matches any decimal digit; this
334 is equivalent to the set ``[0-9]``. With :const:`UNICODE`, it will match
335 whatever is classified as a digit in the Unicode character properties database.
338 When the :const:`UNICODE` flag is not specified, matches any non-digit
339 character; this is equivalent to the set ``[^0-9]``. With :const:`UNICODE`, it
340 will match anything other than character marked as digits in the Unicode
341 character properties database.
344 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
345 any whitespace character; this is equivalent to the set ``[ \t\n\r\f\v]``. With
346 :const:`LOCALE`, it will match this set plus whatever characters are defined as
347 space for the current locale. If :const:`UNICODE` is set, this will match the
348 characters ``[ \t\n\r\f\v]`` plus whatever is classified as space in the Unicode
349 character properties database.
352 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
353 any non-whitespace character; this is equivalent to the set ``[^ \t\n\r\f\v]``
354 With :const:`LOCALE`, it will match any character not in this set, and not
355 defined as space in the current locale. If :const:`UNICODE` is set, this will
356 match anything other than ``[ \t\n\r\f\v]`` and characters marked as space in
357 the Unicode character properties database.
360 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
361 any alphanumeric character and the underscore; this is equivalent to the set
362 ``[a-zA-Z0-9_]``. With :const:`LOCALE`, it will match the set ``[0-9_]`` plus
363 whatever characters are defined as alphanumeric for the current locale. If
364 :const:`UNICODE` is set, this will match the characters ``[0-9_]`` plus whatever
365 is classified as alphanumeric in the Unicode character properties database.
368 When the :const:`LOCALE` and :const:`UNICODE` flags are not specified, matches
369 any non-alphanumeric character; this is equivalent to the set ``[^a-zA-Z0-9_]``.
370 With :const:`LOCALE`, it will match any character not in the set ``[0-9_]``, and
371 not defined as alphanumeric for the current locale. If :const:`UNICODE` is set,
372 this will match anything other than ``[0-9_]`` and characters marked as
373 alphanumeric in the Unicode character properties database.
376 Matches only at the end of the string.
378 Most of the standard escapes supported by Python string literals are also
379 accepted by the regular expression parser::
385 Octal escapes are included in a limited form: If the first digit is a 0, or if
386 there are three octal digits, it is considered an octal escape. Otherwise, it is
387 a group reference. As for string literals, octal escapes are always at most
388 three digits in length.
391 .. _matching-searching:
393 Matching vs Searching
394 ---------------------
396 .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
399 Python offers two different primitive operations based on regular expressions:
400 **match** checks for a match only at the beginning of the string, while
401 **search** checks for a match anywhere in the string (this is what Perl does
404 Note that match may differ from search even when using a regular expression
405 beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in
406 :const:`MULTILINE` mode also immediately following a newline. The "match"
407 operation succeeds only if the pattern matches at the start of the string
408 regardless of mode, or at the starting position given by the optional *pos*
409 argument regardless of whether a newline precedes it.
411 >>> re.match("c", "abcdef") # No match
412 >>> re.search("c", "abcdef") # Match
413 <_sre.SRE_Match object at ...>
416 .. _contents-of-module-re:
421 The module defines several functions, constants, and an exception. Some of the
422 functions are simplified versions of the full featured methods for compiled
423 regular expressions. Most non-trivial applications always use the compiled
427 .. function:: compile(pattern[, flags])
429 Compile a regular expression pattern into a regular expression object, which
430 can be used for matching using its :func:`match` and :func:`search` methods,
433 The expression's behaviour can be modified by specifying a *flags* value.
434 Values can be any of the following variables, combined using bitwise OR (the
439 prog = re.compile(pattern)
440 result = prog.match(string)
444 result = re.match(pattern, string)
446 but using :func:`compile` and saving the resulting regular expression object
447 for reuse is more efficient when the expression will be used several times
452 The compiled versions of the most recent patterns passed to
453 :func:`re.match`, :func:`re.search` or :func:`re.compile` are cached, so
454 programs that use only a few regular expressions at a time needn't worry
455 about compiling regular expressions.
461 Perform case-insensitive matching; expressions like ``[A-Z]`` will match
462 lowercase letters, too. This is not affected by the current locale.
468 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` dependent on the
475 When specified, the pattern character ``'^'`` matches at the beginning of the
476 string and at the beginning of each line (immediately following each newline);
477 and the pattern character ``'$'`` matches at the end of the string and at the
478 end of each line (immediately preceding each newline). By default, ``'^'``
479 matches only at the beginning of the string, and ``'$'`` only at the end of the
480 string and immediately before the newline (if any) at the end of the string.
486 Make the ``'.'`` special character match any character at all, including a
487 newline; without this flag, ``'.'`` will match anything *except* a newline.
493 Make ``\w``, ``\W``, ``\b``, ``\B``, ``\d``, ``\D``, ``\s`` and ``\S`` dependent
494 on the Unicode character properties database.
496 .. versionadded:: 2.0
502 This flag allows you to write regular expressions that look nicer. Whitespace
503 within the pattern is ignored, except when in a character class or preceded by
504 an unescaped backslash, and, when a line contains a ``'#'`` neither in a
505 character class or preceded by an unescaped backslash, all characters from the
506 leftmost such ``'#'`` through the end of the line are ignored.
508 That means that the two following regular expression objects that match a
509 decimal number are functionally equal::
511 a = re.compile(r"""\d + # the integral part
512 \. # the decimal point
513 \d * # some fractional digits""", re.X)
514 b = re.compile(r"\d+\.\d*")
517 .. function:: search(pattern, string[, flags])
519 Scan through *string* looking for a location where the regular expression
520 *pattern* produces a match, and return a corresponding :class:`MatchObject`
521 instance. Return ``None`` if no position in the string matches the pattern; note
522 that this is different from finding a zero-length match at some point in the
526 .. function:: match(pattern, string[, flags])
528 If zero or more characters at the beginning of *string* match the regular
529 expression *pattern*, return a corresponding :class:`MatchObject` instance.
530 Return ``None`` if the string does not match the pattern; note that this is
531 different from a zero-length match.
535 If you want to locate a match anywhere in *string*, use :meth:`search`
539 .. function:: split(pattern, string[, maxsplit=0, flags=0])
541 Split *string* by the occurrences of *pattern*. If capturing parentheses are
542 used in *pattern*, then the text of all groups in the pattern are also returned
543 as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
544 splits occur, and the remainder of the string is returned as the final element
545 of the list. (Incompatibility note: in the original Python 1.5 release,
546 *maxsplit* was ignored. This has been fixed in later releases.)
548 >>> re.split('\W+', 'Words, words, words.')
549 ['Words', 'words', 'words', '']
550 >>> re.split('(\W+)', 'Words, words, words.')
551 ['Words', ', ', 'words', ', ', 'words', '.', '']
552 >>> re.split('\W+', 'Words, words, words.', 1)
553 ['Words', 'words, words.']
554 >>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
557 If there are capturing groups in the separator and it matches at the start of
558 the string, the result will start with an empty string. The same holds for
559 the end of the string:
561 >>> re.split('(\W+)', '...words, words...')
562 ['', '...', 'words', ', ', 'words', '...', '']
564 That way, separator components are always found at the same relative
565 indices within the result list (e.g., if there's one capturing group
566 in the separator, the 0th, the 2nd and so forth).
568 Note that *split* will never split a string on an empty pattern match.
571 >>> re.split('x*', 'foo')
573 >>> re.split("(?m)^$", "foo\n\nbar\n")
576 .. versionchanged:: 2.7,3.1
577 Added the optional flags argument.
580 .. function:: findall(pattern, string[, flags])
582 Return all non-overlapping matches of *pattern* in *string*, as a list of
583 strings. The *string* is scanned left-to-right, and matches are returned in
584 the order found. If one or more groups are present in the pattern, return a
585 list of groups; this will be a list of tuples if the pattern has more than
586 one group. Empty matches are included in the result unless they touch the
587 beginning of another match.
589 .. versionadded:: 1.5.2
591 .. versionchanged:: 2.4
592 Added the optional flags argument.
595 .. function:: finditer(pattern, string[, flags])
597 Return an :term:`iterator` yielding :class:`MatchObject` instances over all
598 non-overlapping matches for the RE *pattern* in *string*. The *string* is
599 scanned left-to-right, and matches are returned in the order found. Empty
600 matches are included in the result unless they touch the beginning of another
603 .. versionadded:: 2.2
605 .. versionchanged:: 2.4
606 Added the optional flags argument.
609 .. function:: sub(pattern, repl, string[, count, flags])
611 Return the string obtained by replacing the leftmost non-overlapping occurrences
612 of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
613 *string* is returned unchanged. *repl* can be a string or a function; if it is
614 a string, any backslash escapes in it are processed. That is, ``\n`` is
615 converted to a single newline character, ``\r`` is converted to a linefeed, and
616 so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
617 as ``\6``, are replaced with the substring matched by group 6 in the pattern.
620 >>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
621 ... r'static PyObject*\npy_\1(void)\n{',
623 'static PyObject*\npy_myfunc(void)\n{'
625 If *repl* is a function, it is called for every non-overlapping occurrence of
626 *pattern*. The function takes a single match object argument, and returns the
627 replacement string. For example:
629 >>> def dashrepl(matchobj):
630 ... if matchobj.group(0) == '-': return ' '
632 >>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
634 >>> re.sub(r'\sAND\s', ' & ', 'Baked Beans And Spam', flags=re.IGNORECASE)
637 The pattern may be a string or an RE object; if you need to specify regular
638 expression flags, you must use a RE object, or use embedded modifiers in a
639 pattern; for example, ``sub("(?i)b+", "x", "bbbb BBBB")`` returns ``'x x'``.
641 The optional argument *count* is the maximum number of pattern occurrences to be
642 replaced; *count* must be a non-negative integer. If omitted or zero, all
643 occurrences will be replaced. Empty matches for the pattern are replaced only
644 when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
647 In addition to character escapes and backreferences as described above,
648 ``\g<name>`` will use the substring matched by the group named ``name``, as
649 defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
650 group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
651 in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
652 reference to group 20, not a reference to group 2 followed by the literal
653 character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
654 substring matched by the RE.
656 .. versionchanged:: 2.7,3.1
657 Added the optional flags argument.
660 .. function:: subn(pattern, repl, string[, count, flags])
662 Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
663 number_of_subs_made)``.
665 .. versionchanged:: 2.7,3.1
666 Added the optional flags argument.
669 .. function:: escape(string)
671 Return *string* with all non-alphanumerics backslashed; this is useful if you
672 want to match an arbitrary literal string that may have regular expression
673 metacharacters in it.
678 Exception raised when a string passed to one of the functions here is not a
679 valid regular expression (for example, it might contain unmatched parentheses)
680 or when some other error occurs during compilation or matching. It is never an
681 error if a string contains no match for a pattern.
686 Regular Expression Objects
687 --------------------------
689 Compiled regular expression objects support the following methods and
693 .. method:: RegexObject.match(string[, pos[, endpos]])
695 If zero or more characters at the beginning of *string* match this regular
696 expression, return a corresponding :class:`MatchObject` instance. Return
697 ``None`` if the string does not match the pattern; note that this is different
698 from a zero-length match.
702 If you want to locate a match anywhere in *string*, use :meth:`search`
705 The optional second parameter *pos* gives an index in the string where the
706 search is to start; it defaults to ``0``. This is not completely equivalent to
707 slicing the string; the ``'^'`` pattern character matches at the real beginning
708 of the string and at positions just after a newline, but not necessarily at the
709 index where the search is to start.
711 The optional parameter *endpos* limits how far the string will be searched; it
712 will be as if the string is *endpos* characters long, so only the characters
713 from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
714 than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
715 expression object, ``rx.match(string, 0, 50)`` is equivalent to
716 ``rx.match(string[:50], 0)``.
718 >>> pattern = re.compile("o")
719 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
720 >>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
721 <_sre.SRE_Match object at ...>
724 .. method:: RegexObject.search(string[, pos[, endpos]])
726 Scan through *string* looking for a location where this regular expression
727 produces a match, and return a corresponding :class:`MatchObject` instance.
728 Return ``None`` if no position in the string matches the pattern; note that this
729 is different from finding a zero-length match at some point in the string.
731 The optional *pos* and *endpos* parameters have the same meaning as for the
732 :meth:`match` method.
735 .. method:: RegexObject.split(string[, maxsplit=0])
737 Identical to the :func:`split` function, using the compiled pattern.
740 .. method:: RegexObject.findall(string[, pos[, endpos]])
742 Identical to the :func:`findall` function, using the compiled pattern.
745 .. method:: RegexObject.finditer(string[, pos[, endpos]])
747 Identical to the :func:`finditer` function, using the compiled pattern.
750 .. method:: RegexObject.sub(repl, string[, count=0])
752 Identical to the :func:`sub` function, using the compiled pattern.
755 .. method:: RegexObject.subn(repl, string[, count=0])
757 Identical to the :func:`subn` function, using the compiled pattern.
760 .. attribute:: RegexObject.flags
762 The flags argument used when the RE object was compiled, or ``0`` if no flags
766 .. attribute:: RegexObject.groups
768 The number of capturing groups in the pattern.
771 .. attribute:: RegexObject.groupindex
773 A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
774 numbers. The dictionary is empty if no symbolic groups were used in the
778 .. attribute:: RegexObject.pattern
780 The pattern string from which the RE object was compiled.
788 Match objects always have a boolean value of :const:`True`, so that you can test
789 whether e.g. :func:`match` resulted in a match with a simple if statement. They
790 support the following methods and attributes:
793 .. method:: MatchObject.expand(template)
795 Return the string obtained by doing backslash substitution on the template
796 string *template*, as done by the :meth:`sub` method. Escapes such as ``\n`` are
797 converted to the appropriate characters, and numeric backreferences (``\1``,
798 ``\2``) and named backreferences (``\g<1>``, ``\g<name>``) are replaced by the
799 contents of the corresponding group.
802 .. method:: MatchObject.group([group1, ...])
804 Returns one or more subgroups of the match. If there is a single argument, the
805 result is a single string; if there are multiple arguments, the result is a
806 tuple with one item per argument. Without arguments, *group1* defaults to zero
807 (the whole match is returned). If a *groupN* argument is zero, the corresponding
808 return value is the entire matching string; if it is in the inclusive range
809 [1..99], it is the string matching the corresponding parenthesized group. If a
810 group number is negative or larger than the number of groups defined in the
811 pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
812 part of the pattern that did not match, the corresponding result is ``None``.
813 If a group is contained in a part of the pattern that matched multiple times,
814 the last match is returned.
816 >>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
817 >>> m.group(0) # The entire match
819 >>> m.group(1) # The first parenthesized subgroup.
821 >>> m.group(2) # The second parenthesized subgroup.
823 >>> m.group(1, 2) # Multiple arguments give us a tuple.
826 If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
827 arguments may also be strings identifying groups by their group name. If a
828 string argument is not used as a group name in the pattern, an :exc:`IndexError`
831 A moderately complicated example:
833 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
834 >>> m.group('first_name')
836 >>> m.group('last_name')
839 Named groups can also be referred to by their index:
846 If a group matches multiple times, only the last match is accessible:
848 >>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
849 >>> m.group(1) # Returns only the last match.
853 .. method:: MatchObject.groups([default])
855 Return a tuple containing all the subgroups of the match, from 1 up to however
856 many groups are in the pattern. The *default* argument is used for groups that
857 did not participate in the match; it defaults to ``None``. (Incompatibility
858 note: in the original Python 1.5 release, if the tuple was one element long, a
859 string would be returned instead. In later versions (from 1.5.1 on), a
860 singleton tuple is returned in such cases.)
864 >>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
868 If we make the decimal place and everything after it optional, not all groups
869 might participate in the match. These groups will default to ``None`` unless
870 the *default* argument is given:
872 >>> m = re.match(r"(\d+)\.?(\d+)?", "24")
873 >>> m.groups() # Second group defaults to None.
875 >>> m.groups('0') # Now, the second group defaults to '0'.
879 .. method:: MatchObject.groupdict([default])
881 Return a dictionary containing all the *named* subgroups of the match, keyed by
882 the subgroup name. The *default* argument is used for groups that did not
883 participate in the match; it defaults to ``None``. For example:
885 >>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
887 {'first_name': 'Malcom', 'last_name': 'Reynolds'}
890 .. method:: MatchObject.start([group])
891 MatchObject.end([group])
893 Return the indices of the start and end of the substring matched by *group*;
894 *group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
895 *group* exists but did not contribute to the match. For a match object *m*, and
896 a group *g* that did contribute to the match, the substring matched by group *g*
897 (equivalent to ``m.group(g)``) is ::
899 m.string[m.start(g):m.end(g)]
901 Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
902 null string. For example, after ``m = re.search('b(c?)', 'cba')``,
903 ``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
904 2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
906 An example that will remove *remove_this* from email addresses:
908 >>> email = "tony@tiremove_thisger.net"
909 >>> m = re.search("remove_this", email)
910 >>> email[:m.start()] + email[m.end():]
914 .. method:: MatchObject.span([group])
916 For :class:`MatchObject` *m*, return the 2-tuple ``(m.start(group),
917 m.end(group))``. Note that if *group* did not contribute to the match, this is
918 ``(-1, -1)``. *group* defaults to zero, the entire match.
921 .. attribute:: MatchObject.pos
923 The value of *pos* which was passed to the :func:`search` or :func:`match`
924 method of the :class:`RegexObject`. This is the index into the string at which
925 the RE engine started looking for a match.
928 .. attribute:: MatchObject.endpos
930 The value of *endpos* which was passed to the :func:`search` or :func:`match`
931 method of the :class:`RegexObject`. This is the index into the string beyond
932 which the RE engine will not go.
935 .. attribute:: MatchObject.lastindex
937 The integer index of the last matched capturing group, or ``None`` if no group
938 was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
939 ``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
940 the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
944 .. attribute:: MatchObject.lastgroup
946 The name of the last matched capturing group, or ``None`` if the group didn't
947 have a name, or if no group was matched at all.
950 .. attribute:: MatchObject.re
952 The regular expression object whose :meth:`match` or :meth:`search` method
953 produced this :class:`MatchObject` instance.
956 .. attribute:: MatchObject.string
958 The string passed to :func:`match` or :func:`search`.
968 In this example, we'll use the following helper function to display match
969 objects a little more gracefully:
973 def displaymatch(match):
976 return '<Match: %r, groups=%r>' % (match.group(), match.groups())
978 Suppose you are writing a poker program where a player's hand is represented as
979 a 5-character string with each character representing a card, "a" for ace, "k"
980 for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
981 representing the card with that value.
983 To see if a given string is a valid hand, one could do the following:
985 >>> valid = re.compile(r"[0-9akqj]{5}$")
986 >>> displaymatch(valid.match("ak05q")) # Valid.
987 "<Match: 'ak05q', groups=()>"
988 >>> displaymatch(valid.match("ak05e")) # Invalid.
989 >>> displaymatch(valid.match("ak0")) # Invalid.
990 >>> displaymatch(valid.match("727ak")) # Valid.
991 "<Match: '727ak', groups=()>"
993 That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
994 To match this with a regular expression, one could use backreferences as such:
996 >>> pair = re.compile(r".*(.).*\1")
997 >>> displaymatch(pair.match("717ak")) # Pair of 7s.
998 "<Match: '717', groups=('7',)>"
999 >>> displaymatch(pair.match("718ak")) # No pairs.
1000 >>> displaymatch(pair.match("354aa")) # Pair of aces.
1001 "<Match: '354aa', groups=('a',)>"
1003 To find out what card the pair consists of, one could use the :func:`group`
1004 method of :class:`MatchObject` in the following manner:
1008 >>> pair.match("717ak").group(1)
1011 # Error because re.match() returns None, which doesn't have a group() method:
1012 >>> pair.match("718ak").group(1)
1013 Traceback (most recent call last):
1014 File "<pyshell#23>", line 1, in <module>
1015 re.match(r".*(.).*\1", "718ak").group(1)
1016 AttributeError: 'NoneType' object has no attribute 'group'
1018 >>> pair.match("354aa").group(1)
1025 .. index:: single: scanf()
1027 Python does not currently have an equivalent to :cfunc:`scanf`. Regular
1028 expressions are generally more powerful, though also more verbose, than
1029 :cfunc:`scanf` format strings. The table below offers some more-or-less
1030 equivalent mappings between :cfunc:`scanf` format tokens and regular
1033 +--------------------------------+---------------------------------------------+
1034 | :cfunc:`scanf` Token | Regular Expression |
1035 +================================+=============================================+
1037 +--------------------------------+---------------------------------------------+
1038 | ``%5c`` | ``.{5}`` |
1039 +--------------------------------+---------------------------------------------+
1040 | ``%d`` | ``[-+]?\d+`` |
1041 +--------------------------------+---------------------------------------------+
1042 | ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
1043 +--------------------------------+---------------------------------------------+
1044 | ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
1045 +--------------------------------+---------------------------------------------+
1046 | ``%o`` | ``0[0-7]*`` |
1047 +--------------------------------+---------------------------------------------+
1048 | ``%s`` | ``\S+`` |
1049 +--------------------------------+---------------------------------------------+
1050 | ``%u`` | ``\d+`` |
1051 +--------------------------------+---------------------------------------------+
1052 | ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` |
1053 +--------------------------------+---------------------------------------------+
1055 To extract the filename and numbers from a string like ::
1057 /usr/sbin/sendmail - 0 errors, 4 warnings
1059 you would use a :cfunc:`scanf` format like ::
1061 %s - %d errors, %d warnings
1063 The equivalent regular expression would be ::
1065 (\S+) - (\d+) errors, (\d+) warnings
1071 If you create regular expressions that require the engine to perform a lot of
1072 recursion, you may encounter a :exc:`RuntimeError` exception with the message
1073 ``maximum recursion limit`` exceeded. For example, ::
1075 >>> s = 'Begin ' + 1000*'a very long string ' + 'end'
1076 >>> re.match('Begin (\w| )*? end', s).end()
1077 Traceback (most recent call last):
1078 File "<stdin>", line 1, in ?
1079 File "/usr/local/lib/python2.5/re.py", line 132, in match
1080 return _compile(pattern, flags).match(string)
1081 RuntimeError: maximum recursion limit exceeded
1083 You can often restructure your regular expression to avoid recursion.
1085 Starting with Python 2.3, simple uses of the ``*?`` pattern are special-cased to
1086 avoid recursion. Thus, the above regular expression can avoid recursion by
1087 being recast as ``Begin [a-zA-Z0-9_ ]*?end``. As a further benefit, such
1088 regular expressions will run faster than their recursive equivalents.
1091 search() vs. match()
1092 ^^^^^^^^^^^^^^^^^^^^
1094 In a nutshell, :func:`match` only attempts to match a pattern at the beginning
1095 of a string where :func:`search` will match a pattern anywhere in a string.
1098 >>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
1099 >>> re.search("o", "dog") # Match as search() looks everywhere in the string.
1100 <_sre.SRE_Match object at ...>
1104 The following applies only to regular expression objects like those created
1105 with ``re.compile("pattern")``, not the primitives ``re.match(pattern,
1106 string)`` or ``re.search(pattern, string)``.
1108 :func:`match` has an optional second parameter that gives an index in the string
1109 where the search is to start::
1111 >>> pattern = re.compile("o")
1112 >>> pattern.match("dog") # No match as "o" is not at the start of "dog."
1114 # Equivalent to the above expression as 0 is the default starting index:
1115 >>> pattern.match("dog", 0)
1117 # Match as "o" is the 2nd character of "dog" (index 0 is the first):
1118 >>> pattern.match("dog", 1)
1119 <_sre.SRE_Match object at ...>
1120 >>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
1126 :func:`split` splits a string into a list delimited by the passed pattern. The
1127 method is invaluable for converting textual data into data structures that can be
1128 easily read and modified by Python as demonstrated in the following example that
1129 creates a phonebook.
1131 First, here is the input. Normally it may come from a file, here we are using
1132 triple-quoted string syntax:
1134 >>> input = """Ross McFluff: 834.345.1254 155 Elm Street
1136 ... Ronald Heathmore: 892.345.3428 436 Finley Avenue
1137 ... Frank Burger: 925.541.7625 662 South Dogwood Way
1140 ... Heather Albrecht: 548.326.4584 919 Park Place"""
1142 The entries are separated by one or more newlines. Now we convert the string
1143 into a list with each nonempty line having its own entry:
1146 :options: +NORMALIZE_WHITESPACE
1148 >>> entries = re.split("\n+", input)
1150 ['Ross McFluff: 834.345.1254 155 Elm Street',
1151 'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
1152 'Frank Burger: 925.541.7625 662 South Dogwood Way',
1153 'Heather Albrecht: 548.326.4584 919 Park Place']
1155 Finally, split each entry into a list with first name, last name, telephone
1156 number, and address. We use the ``maxsplit`` parameter of :func:`split`
1157 because the address has spaces, our splitting pattern, in it:
1160 :options: +NORMALIZE_WHITESPACE
1162 >>> [re.split(":? ", entry, 3) for entry in entries]
1163 [['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1164 ['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1165 ['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1166 ['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1168 The ``:?`` pattern matches the colon after the last name, so that it does not
1169 occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
1170 house number from the street name:
1173 :options: +NORMALIZE_WHITESPACE
1175 >>> [re.split(":? ", entry, 4) for entry in entries]
1176 [['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1177 ['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1178 ['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1179 ['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1185 :func:`sub` replaces every occurrence of a pattern with a string or the
1186 result of a function. This example demonstrates using :func:`sub` with
1187 a function to "munge" text, or randomize the order of all the characters
1188 in each word of a sentence except for the first and last characters::
1191 ... inner_word = list(m.group(2))
1192 ... random.shuffle(inner_word)
1193 ... return m.group(1) + "".join(inner_word) + m.group(3)
1194 >>> text = "Professor Abdolmalek, please report your absences promptly."
1195 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1196 'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1197 >>> re.sub("(\w)(\w+)(\w)", repl, text)
1198 'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1204 :func:`findall` matches *all* occurrences of a pattern, not just the first
1205 one as :func:`search` does. For example, if one was a writer and wanted to
1206 find all of the adverbs in some text, he or she might use :func:`findall` in
1207 the following manner:
1209 >>> text = "He was carefully disguised but captured quickly by police."
1210 >>> re.findall(r"\w+ly", text)
1211 ['carefully', 'quickly']
1214 Finding all Adverbs and their Positions
1215 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1217 If one wants more information about all matches of a pattern than the matched
1218 text, :func:`finditer` is useful as it provides instances of
1219 :class:`MatchObject` instead of strings. Continuing with the previous example,
1220 if one was a writer who wanted to find all of the adverbs *and their positions*
1221 in some text, he or she would use :func:`finditer` in the following manner:
1223 >>> text = "He was carefully disguised but captured quickly by police."
1224 >>> for m in re.finditer(r"\w+ly", text):
1225 ... print '%02d-%02d: %s' % (m.start(), m.end(), m.group(0))
1233 Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1234 every backslash (``'\'``) in a regular expression would have to be prefixed with
1235 another one to escape it. For example, the two following lines of code are
1236 functionally identical:
1238 >>> re.match(r"\W(.)\1\W", " ff ")
1239 <_sre.SRE_Match object at ...>
1240 >>> re.match("\\W(.)\\1\\W", " ff ")
1241 <_sre.SRE_Match object at ...>
1243 When one wants to match a literal backslash, it must be escaped in the regular
1244 expression. With raw string notation, this means ``r"\\"``. Without raw string
1245 notation, one must use ``"\\\\"``, making the following lines of code
1246 functionally identical:
1248 >>> re.match(r"\\", r"\\")
1249 <_sre.SRE_Match object at ...>
1250 >>> re.match("\\\\", r"\\")
1251 <_sre.SRE_Match object at ...>