1 @node Pattern Matching, I/O Overview, Searching and Sorting, Top
2 @c %MENU% Matching shell ``globs'' and regular expressions
3 @chapter Pattern Matching
5 @Theglibc{} provides pattern matching facilities for two kinds of
6 patterns: regular expressions and file-name wildcards. The library also
7 provides a facility for expanding variable and command references and
8 parsing text into words in the way the shell does.
11 * Wildcard Matching:: Matching a wildcard pattern against a single string.
12 * Globbing:: Finding the files that match a wildcard pattern.
13 * Regular Expressions:: Matching regular expressions against strings.
14 * Word Expansion:: Expanding shell variables, nested commands,
15 arithmetic, and wildcards.
16 This is what the shell does with shell commands.
19 @node Wildcard Matching
20 @section Wildcard Matching
23 This section describes how to match a wildcard pattern against a
24 particular string. The result is a yes or no answer: does the
25 string fit the pattern or not. The symbols described here are all
26 declared in @file{fnmatch.h}.
30 @deftypefun int fnmatch (const char *@var{pattern}, const char *@var{string}, int @var{flags})
31 This function tests whether the string @var{string} matches the pattern
32 @var{pattern}. It returns @code{0} if they do match; otherwise, it
33 returns the nonzero value @code{FNM_NOMATCH}. The arguments
34 @var{pattern} and @var{string} are both strings.
36 The argument @var{flags} is a combination of flag bits that alter the
37 details of matching. See below for a list of the defined flags.
39 In @theglibc{}, @code{fnmatch} cannot experience an ``error''---it
40 always returns an answer for whether the match succeeds. However, other
41 implementations of @code{fnmatch} might sometimes report ``errors''.
42 They would do so by returning nonzero values that are not equal to
46 These are the available flags for the @var{flags} argument:
52 Treat the @samp{/} character specially, for matching file names. If
53 this flag is set, wildcard constructs in @var{pattern} cannot match
54 @samp{/} in @var{string}. Thus, the only way to match @samp{/} is with
55 an explicit @samp{/} in @var{pattern}.
60 This is an alias for @code{FNM_FILE_NAME}; it comes from POSIX.2. We
61 don't recommend this name because we don't use the term ``pathname'' for
67 Treat the @samp{.} character specially if it appears at the beginning of
68 @var{string}. If this flag is set, wildcard constructs in @var{pattern}
69 cannot match @samp{.} as the first character of @var{string}.
71 If you set both @code{FNM_PERIOD} and @code{FNM_FILE_NAME}, then the
72 special treatment applies to @samp{.} following @samp{/} as well as to
73 @samp{.} at the beginning of @var{string}. (The shell uses the
74 @code{FNM_PERIOD} and @code{FNM_FILE_NAME} flags together for matching
80 Don't treat the @samp{\} character specially in patterns. Normally,
81 @samp{\} quotes the following character, turning off its special meaning
82 (if any) so that it matches only itself. When quoting is enabled, the
83 pattern @samp{\?} matches only the string @samp{?}, because the question
84 mark in the pattern acts like an ordinary character.
86 If you use @code{FNM_NOESCAPE}, then @samp{\} is an ordinary character.
91 Ignore a trailing sequence of characters starting with a @samp{/} in
92 @var{string}; that is to say, test whether @var{string} starts with a
93 directory name that @var{pattern} matches.
95 If this flag is set, either @samp{foo*} or @samp{foobar} as a pattern
96 would match the string @samp{foobar/frobozz}.
101 Ignore case in comparing @var{string} to @var{pattern}.
108 Recognize beside the normal patterns also the extended patterns
109 introduced in @file{ksh}. The patterns are written in the form
110 explained in the following table where @var{pattern-list} is a @code{|}
111 separated list of patterns.
114 @item ?(@var{pattern-list})
115 The pattern matches if zero or one occurrences of any of the patterns
116 in the @var{pattern-list} allow matching the input string.
118 @item *(@var{pattern-list})
119 The pattern matches if zero or more occurrences of any of the patterns
120 in the @var{pattern-list} allow matching the input string.
122 @item +(@var{pattern-list})
123 The pattern matches if one or more occurrences of any of the patterns
124 in the @var{pattern-list} allow matching the input string.
126 @item @@(@var{pattern-list})
127 The pattern matches if exactly one occurrence of any of the patterns in
128 the @var{pattern-list} allows matching the input string.
130 @item !(@var{pattern-list})
131 The pattern matches if the input string cannot be matched with any of
132 the patterns in the @var{pattern-list}.
140 The archetypal use of wildcards is for matching against the files in a
141 directory, and making a list of all the matches. This is called
144 You could do this using @code{fnmatch}, by reading the directory entries
145 one by one and testing each one with @code{fnmatch}. But that would be
146 slow (and complex, since you would have to handle subdirectories by
149 The library provides a function @code{glob} to make this particular use
150 of wildcards convenient. @code{glob} and the other symbols in this
151 section are declared in @file{glob.h}.
154 * Calling Glob:: Basic use of @code{glob}.
155 * Flags for Globbing:: Flags that enable various options in @code{glob}.
156 * More Flags for Globbing:: GNU specific extensions to @code{glob}.
160 @subsection Calling @code{glob}
162 The result of globbing is a vector of file names (strings). To return
163 this vector, @code{glob} uses a special data type, @code{glob_t}, which
164 is a structure. You pass @code{glob} the address of the structure, and
165 it fills in the structure's fields to tell you about the results.
169 @deftp {Data Type} glob_t
170 This data type holds a pointer to a word vector. More precisely, it
171 records both the address of the word vector and its size. The GNU
172 implementation contains some more fields which are non-standard
177 The number of elements in the vector, excluding the initial null entries
178 if the GLOB_DOOFFS flag is used (see gl_offs below).
181 The address of the vector. This field has type @w{@code{char **}}.
184 The offset of the first real element of the vector, from its nominal
185 address in the @code{gl_pathv} field. Unlike the other fields, this
186 is always an input to @code{glob}, rather than an output from it.
188 If you use a nonzero offset, then that many elements at the beginning of
189 the vector are left empty. (The @code{glob} function fills them with
192 The @code{gl_offs} field is meaningful only if you use the
193 @code{GLOB_DOOFFS} flag. Otherwise, the offset is always zero
194 regardless of what is in this field, and the first real element comes at
195 the beginning of the vector.
198 The address of an alternative implementation of the @code{closedir}
199 function. It is used if the @code{GLOB_ALTDIRFUNC} bit is set in
200 the flag parameter. The type of this field is
201 @w{@code{void (*) (void *)}}.
203 This is a GNU extension.
206 The address of an alternative implementation of the @code{readdir}
207 function used to read the contents of a directory. It is used if the
208 @code{GLOB_ALTDIRFUNC} bit is set in the flag parameter. The type of
209 this field is @w{@code{struct dirent *(*) (void *)}}.
211 This is a GNU extension.
214 The address of an alternative implementation of the @code{opendir}
215 function. It is used if the @code{GLOB_ALTDIRFUNC} bit is set in
216 the flag parameter. The type of this field is
217 @w{@code{void *(*) (const char *)}}.
219 This is a GNU extension.
222 The address of an alternative implementation of the @code{stat} function
223 to get information about an object in the filesystem. It is used if the
224 @code{GLOB_ALTDIRFUNC} bit is set in the flag parameter. The type of
225 this field is @w{@code{int (*) (const char *, struct stat *)}}.
227 This is a GNU extension.
230 The address of an alternative implementation of the @code{lstat}
231 function to get information about an object in the filesystems, not
232 following symbolic links. It is used if the @code{GLOB_ALTDIRFUNC} bit
233 is set in the flag parameter. The type of this field is @code{@w{int
234 (*) (const char *,} @w{struct stat *)}}.
236 This is a GNU extension.
239 The flags used when @code{glob} was called. In addition, @code{GLOB_MAGCHAR}
240 might be set. See @ref{Flags for Globbing} for more details.
242 This is a GNU extension.
246 For use in the @code{glob64} function @file{glob.h} contains another
247 definition for a very similar type. @code{glob64_t} differs from
248 @code{glob_t} only in the types of the members @code{gl_readdir},
249 @code{gl_stat}, and @code{gl_lstat}.
253 @deftp {Data Type} glob64_t
254 This data type holds a pointer to a word vector. More precisely, it
255 records both the address of the word vector and its size. The GNU
256 implementation contains some more fields which are non-standard
261 The number of elements in the vector, excluding the initial null entries
262 if the GLOB_DOOFFS flag is used (see gl_offs below).
265 The address of the vector. This field has type @w{@code{char **}}.
268 The offset of the first real element of the vector, from its nominal
269 address in the @code{gl_pathv} field. Unlike the other fields, this
270 is always an input to @code{glob}, rather than an output from it.
272 If you use a nonzero offset, then that many elements at the beginning of
273 the vector are left empty. (The @code{glob} function fills them with
276 The @code{gl_offs} field is meaningful only if you use the
277 @code{GLOB_DOOFFS} flag. Otherwise, the offset is always zero
278 regardless of what is in this field, and the first real element comes at
279 the beginning of the vector.
282 The address of an alternative implementation of the @code{closedir}
283 function. It is used if the @code{GLOB_ALTDIRFUNC} bit is set in
284 the flag parameter. The type of this field is
285 @w{@code{void (*) (void *)}}.
287 This is a GNU extension.
290 The address of an alternative implementation of the @code{readdir64}
291 function used to read the contents of a directory. It is used if the
292 @code{GLOB_ALTDIRFUNC} bit is set in the flag parameter. The type of
293 this field is @w{@code{struct dirent64 *(*) (void *)}}.
295 This is a GNU extension.
298 The address of an alternative implementation of the @code{opendir}
299 function. It is used if the @code{GLOB_ALTDIRFUNC} bit is set in
300 the flag parameter. The type of this field is
301 @w{@code{void *(*) (const char *)}}.
303 This is a GNU extension.
306 The address of an alternative implementation of the @code{stat64} function
307 to get information about an object in the filesystem. It is used if the
308 @code{GLOB_ALTDIRFUNC} bit is set in the flag parameter. The type of
309 this field is @w{@code{int (*) (const char *, struct stat64 *)}}.
311 This is a GNU extension.
314 The address of an alternative implementation of the @code{lstat64}
315 function to get information about an object in the filesystems, not
316 following symbolic links. It is used if the @code{GLOB_ALTDIRFUNC} bit
317 is set in the flag parameter. The type of this field is @code{@w{int
318 (*) (const char *,} @w{struct stat64 *)}}.
320 This is a GNU extension.
323 The flags used when @code{glob} was called. In addition, @code{GLOB_MAGCHAR}
324 might be set. See @ref{Flags for Globbing} for more details.
326 This is a GNU extension.
332 @deftypefun int glob (const char *@var{pattern}, int @var{flags}, int (*@var{errfunc}) (const char *@var{filename}, int @var{error-code}), glob_t *@var{vector-ptr})
333 The function @code{glob} does globbing using the pattern @var{pattern}
334 in the current directory. It puts the result in a newly allocated
335 vector, and stores the size and address of this vector into
336 @code{*@var{vector-ptr}}. The argument @var{flags} is a combination of
337 bit flags; see @ref{Flags for Globbing}, for details of the flags.
339 The result of globbing is a sequence of file names. The function
340 @code{glob} allocates a string for each resulting word, then
341 allocates a vector of type @code{char **} to store the addresses of
342 these strings. The last element of the vector is a null pointer.
343 This vector is called the @dfn{word vector}.
345 To return this vector, @code{glob} stores both its address and its
346 length (number of elements, not counting the terminating null pointer)
347 into @code{*@var{vector-ptr}}.
349 Normally, @code{glob} sorts the file names alphabetically before
350 returning them. You can turn this off with the flag @code{GLOB_NOSORT}
351 if you want to get the information as fast as possible. Usually it's
352 a good idea to let @code{glob} sort them---if you process the files in
353 alphabetical order, the users will have a feel for the rate of progress
354 that your application is making.
356 If @code{glob} succeeds, it returns 0. Otherwise, it returns one
357 of these error codes:
363 There was an error opening a directory, and you used the flag
364 @code{GLOB_ERR} or your specified @var{errfunc} returned a nonzero
370 @xref{Flags for Globbing},
372 for an explanation of the @code{GLOB_ERR} flag and @var{errfunc}.
377 The pattern didn't match any existing files. If you use the
378 @code{GLOB_NOCHECK} flag, then you never get this error code, because
379 that flag tells @code{glob} to @emph{pretend} that the pattern matched
385 It was impossible to allocate memory to hold the result.
388 In the event of an error, @code{glob} stores information in
389 @code{*@var{vector-ptr}} about all the matches it has found so far.
391 It is important to notice that the @code{glob} function will not fail if
392 it encounters directories or files which cannot be handled without the
393 LFS interfaces. The implementation of @code{glob} is supposed to use
394 these functions internally. This at least is the assumptions made by
395 the Unix standard. The GNU extension of allowing the user to provide
396 own directory handling and @code{stat} functions complicates things a
397 bit. If these callback functions are used and a large file or directory
398 is encountered @code{glob} @emph{can} fail.
403 @deftypefun int glob64 (const char *@var{pattern}, int @var{flags}, int (*@var{errfunc}) (const char *@var{filename}, int @var{error-code}), glob64_t *@var{vector-ptr})
404 The @code{glob64} function was added as part of the Large File Summit
405 extensions but is not part of the original LFS proposal. The reason for
406 this is simple: it is not necessary. The necessity for a @code{glob64}
407 function is added by the extensions of the GNU @code{glob}
408 implementation which allows the user to provide own directory handling
409 and @code{stat} functions. The @code{readdir} and @code{stat} functions
410 do depend on the choice of @code{_FILE_OFFSET_BITS} since the definition
411 of the types @code{struct dirent} and @code{struct stat} will change
412 depending on the choice.
414 Beside this difference the @code{glob64} works just like @code{glob} in
417 This function is a GNU extension.
420 @node Flags for Globbing
421 @subsection Flags for Globbing
423 This section describes the standard flags that you can specify in the
424 @var{flags} argument to @code{glob}. Choose the flags you want,
425 and combine them with the C bitwise OR operator @code{|}.
427 Note that there are @ref{More Flags for Globbing} available as GNU extensions.
433 Append the words from this expansion to the vector of words produced by
434 previous calls to @code{glob}. This way you can effectively expand
435 several words as if they were concatenated with spaces between them.
437 In order for appending to work, you must not modify the contents of the
438 word vector structure between calls to @code{glob}. And, if you set
439 @code{GLOB_DOOFFS} in the first call to @code{glob}, you must also
440 set it when you append to the results.
442 Note that the pointer stored in @code{gl_pathv} may no longer be valid
443 after you call @code{glob} the second time, because @code{glob} might
444 have relocated the vector. So always fetch @code{gl_pathv} from the
445 @code{glob_t} structure after each @code{glob} call; @strong{never} save
446 the pointer across calls.
451 Leave blank slots at the beginning of the vector of words.
452 The @code{gl_offs} field says how many slots to leave.
453 The blank slots contain null pointers.
458 Give up right away and report an error if there is any difficulty
459 reading the directories that must be read in order to expand @var{pattern}
460 fully. Such difficulties might include a directory in which you don't
461 have the requisite access. Normally, @code{glob} tries its best to keep
462 on going despite any errors, reading whatever directories it can.
464 You can exercise even more control than this by specifying an
465 error-handler function @var{errfunc} when you call @code{glob}. If
466 @var{errfunc} is not a null pointer, then @code{glob} doesn't give up
467 right away when it can't read a directory; instead, it calls
468 @var{errfunc} with two arguments, like this:
471 (*@var{errfunc}) (@var{filename}, @var{error-code})
475 The argument @var{filename} is the name of the directory that
476 @code{glob} couldn't open or couldn't read, and @var{error-code} is the
477 @code{errno} value that was reported to @code{glob}.
479 If the error handler function returns nonzero, then @code{glob} gives up
480 right away. Otherwise, it continues.
485 If the pattern matches the name of a directory, append @samp{/} to the
486 directory's name when returning it.
491 If the pattern doesn't match any file names, return the pattern itself
492 as if it were a file name that had been matched. (Normally, when the
493 pattern doesn't match anything, @code{glob} returns that there were no
499 Don't treat the @samp{\} character specially in patterns. Normally,
500 @samp{\} quotes the following character, turning off its special meaning
501 (if any) so that it matches only itself. When quoting is enabled, the
502 pattern @samp{\?} matches only the string @samp{?}, because the question
503 mark in the pattern acts like an ordinary character.
505 If you use @code{GLOB_NOESCAPE}, then @samp{\} is an ordinary character.
507 @code{glob} does its work by calling the function @code{fnmatch}
508 repeatedly. It handles the flag @code{GLOB_NOESCAPE} by turning on the
509 @code{FNM_NOESCAPE} flag in calls to @code{fnmatch}.
514 Don't sort the file names; return them in no particular order.
515 (In practice, the order will depend on the order of the entries in
516 the directory.) The only reason @emph{not} to sort is to save time.
519 @node More Flags for Globbing
520 @subsection More Flags for Globbing
522 Beside the flags described in the last section, the GNU implementation of
523 @code{glob} allows a few more flags which are also defined in the
524 @file{glob.h} file. Some of the extensions implement functionality
525 which is available in modern shell implementations.
531 The @code{.} character (period) is treated special. It cannot be
532 matched by wildcards. @xref{Wildcard Matching}, @code{FNM_PERIOD}.
537 The @code{GLOB_MAGCHAR} value is not to be given to @code{glob} in the
538 @var{flags} parameter. Instead, @code{glob} sets this bit in the
539 @var{gl_flags} element of the @var{glob_t} structure provided as the
540 result if the pattern used for matching contains any wildcard character.
544 @item GLOB_ALTDIRFUNC
545 Instead of the using the using the normal functions for accessing the
546 filesystem the @code{glob} implementation uses the user-supplied
547 functions specified in the structure pointed to by @var{pglob}
548 parameter. For more information about the functions refer to the
549 sections about directory handling see @ref{Accessing Directories}, and
550 @ref{Reading Attributes}.
555 If this flag is given the handling of braces in the pattern is changed.
556 It is now required that braces appear correctly grouped. I.e., for each
557 opening brace there must be a closing one. Braces can be used
558 recursively. So it is possible to define one brace expression in
559 another one. It is important to note that the range of each brace
560 expression is completely contained in the outer brace expression (if
563 The string between the matching braces is separated into single
564 expressions by splitting at @code{,} (comma) characters. The commas
565 themselves are discarded. Please note what we said above about recursive
566 brace expressions. The commas used to separate the subexpressions must
567 be at the same level. Commas in brace subexpressions are not matched.
568 They are used during expansion of the brace expression of the deeper
569 level. The example below shows this
572 glob ("@{foo/@{,bar,biz@},baz@}", GLOB_BRACE, NULL, &result)
576 is equivalent to the sequence
579 glob ("foo/", GLOB_BRACE, NULL, &result)
580 glob ("foo/bar", GLOB_BRACE|GLOB_APPEND, NULL, &result)
581 glob ("foo/biz", GLOB_BRACE|GLOB_APPEND, NULL, &result)
582 glob ("baz", GLOB_BRACE|GLOB_APPEND, NULL, &result)
586 if we leave aside error handling.
591 If the pattern contains no wildcard constructs (it is a literal file name),
592 return it as the sole ``matching'' word, even if no file exists by that name.
597 If this flag is used the character @code{~} (tilde) is handled special
598 if it appears at the beginning of the pattern. Instead of being taken
599 verbatim it is used to represent the home directory of a known user.
601 If @code{~} is the only character in pattern or it is followed by a
602 @code{/} (slash), the home directory of the process owner is
603 substituted. Using @code{getlogin} and @code{getpwnam} the information
604 is read from the system databases. As an example take user @code{bart}
605 with his home directory at @file{/home/bart}. For him a call like
608 glob ("~/bin/*", GLOB_TILDE, NULL, &result)
612 would return the contents of the directory @file{/home/bart/bin}.
613 Instead of referring to the own home directory it is also possible to
614 name the home directory of other users. To do so one has to append the
615 user name after the tilde character. So the contents of user
616 @code{homer}'s @file{bin} directory can be retrieved by
619 glob ("~homer/bin/*", GLOB_TILDE, NULL, &result)
622 If the user name is not valid or the home directory cannot be determined
623 for some reason the pattern is left untouched and itself used as the
624 result. I.e., if in the last example @code{home} is not available the
625 tilde expansion yields to @code{"~homer/bin/*"} and @code{glob} is not
626 looking for a directory named @code{~homer}.
628 This functionality is equivalent to what is available in C-shells if the
629 @code{nonomatch} flag is set.
633 @item GLOB_TILDE_CHECK
634 If this flag is used @code{glob} behaves like as if @code{GLOB_TILDE} is
635 given. The only difference is that if the user name is not available or
636 the home directory cannot be determined for other reasons this leads to
637 an error. @code{glob} will return @code{GLOB_NOMATCH} instead of using
638 the pattern itself as the name.
640 This functionality is equivalent to what is available in C-shells if
641 @code{nonomatch} flag is not set.
646 If this flag is used the globbing function takes this as a
647 @strong{hint} that the caller is only interested in directories
648 matching the pattern. If the information about the type of the file
649 is easily available non-directories will be rejected but no extra
650 work will be done to determine the information for each file. I.e.,
651 the caller must still be able to filter directories out.
653 This functionality is only available with the GNU @code{glob}
654 implementation. It is mainly used internally to increase the
655 performance but might be useful for a user as well and therefore is
659 Calling @code{glob} will in most cases allocate resources which are used
660 to represent the result of the function call. If the same object of
661 type @code{glob_t} is used in multiple call to @code{glob} the resources
662 are freed or reused so that no leaks appear. But this does not include
663 the time when all @code{glob} calls are done.
667 @deftypefun void globfree (glob_t *@var{pglob})
668 The @code{globfree} function frees all resources allocated by previous
669 calls to @code{glob} associated with the object pointed to by
670 @var{pglob}. This function should be called whenever the currently used
671 @code{glob_t} typed object isn't used anymore.
676 @deftypefun void globfree64 (glob64_t *@var{pglob})
677 This function is equivalent to @code{globfree} but it frees records of
678 type @code{glob64_t} which were allocated by @code{glob64}.
682 @node Regular Expressions
683 @section Regular Expression Matching
685 @Theglibc{} supports two interfaces for matching regular
686 expressions. One is the standard POSIX.2 interface, and the other is
687 what @theglibc{} has had for many years.
689 Both interfaces are declared in the header file @file{regex.h}.
690 If you define @w{@code{_POSIX_C_SOURCE}}, then only the POSIX.2
691 functions, structures, and constants are declared.
692 @c !!! we only document the POSIX.2 interface here!!
695 * POSIX Regexp Compilation:: Using @code{regcomp} to prepare to match.
696 * Flags for POSIX Regexps:: Syntax variations for @code{regcomp}.
697 * Matching POSIX Regexps:: Using @code{regexec} to match the compiled
698 pattern that you get from @code{regcomp}.
699 * Regexp Subexpressions:: Finding which parts of the string were matched.
700 * Subexpression Complications:: Find points of which parts were matched.
701 * Regexp Cleanup:: Freeing storage; reporting errors.
704 @node POSIX Regexp Compilation
705 @subsection POSIX Regular Expression Compilation
707 Before you can actually match a regular expression, you must
708 @dfn{compile} it. This is not true compilation---it produces a special
709 data structure, not machine instructions. But it is like ordinary
710 compilation in that its purpose is to enable you to ``execute'' the
711 pattern fast. (@xref{Matching POSIX Regexps}, for how to use the
712 compiled regular expression for matching.)
714 There is a special data type for compiled regular expressions:
718 @deftp {Data Type} regex_t
719 This type of object holds a compiled regular expression.
720 It is actually a structure. It has just one field that your programs
725 This field holds the number of parenthetical subexpressions in the
726 regular expression that was compiled.
729 There are several other fields, but we don't describe them here, because
730 only the functions in the library should use them.
733 After you create a @code{regex_t} object, you can compile a regular
734 expression into it by calling @code{regcomp}.
738 @deftypefun int regcomp (regex_t *restrict @var{compiled}, const char *restrict @var{pattern}, int @var{cflags})
739 The function @code{regcomp} ``compiles'' a regular expression into a
740 data structure that you can use with @code{regexec} to match against a
741 string. The compiled regular expression format is designed for
742 efficient matching. @code{regcomp} stores it into @code{*@var{compiled}}.
744 It's up to you to allocate an object of type @code{regex_t} and pass its
745 address to @code{regcomp}.
747 The argument @var{cflags} lets you specify various options that control
748 the syntax and semantics of regular expressions. @xref{Flags for POSIX
751 If you use the flag @code{REG_NOSUB}, then @code{regcomp} omits from
752 the compiled regular expression the information necessary to record
753 how subexpressions actually match. In this case, you might as well
754 pass @code{0} for the @var{matchptr} and @var{nmatch} arguments when
755 you call @code{regexec}.
757 If you don't use @code{REG_NOSUB}, then the compiled regular expression
758 does have the capacity to record how subexpressions match. Also,
759 @code{regcomp} tells you how many subexpressions @var{pattern} has, by
760 storing the number in @code{@var{compiled}->re_nsub}. You can use that
761 value to decide how long an array to allocate to hold information about
762 subexpression matches.
764 @code{regcomp} returns @code{0} if it succeeds in compiling the regular
765 expression; otherwise, it returns a nonzero error code (see the table
766 below). You can use @code{regerror} to produce an error message string
767 describing the reason for a nonzero value; see @ref{Regexp Cleanup}.
771 Here are the possible nonzero values that @code{regcomp} can return:
777 There was an invalid @samp{\@{@dots{}\@}} construct in the regular
778 expression. A valid @samp{\@{@dots{}\@}} construct must contain either
779 a single number, or two numbers in increasing order separated by a
785 There was a syntax error in the regular expression.
790 A repetition operator such as @samp{?} or @samp{*} appeared in a bad
791 position (with no preceding subexpression to act on).
796 The regular expression referred to an invalid collating element (one not
797 defined in the current locale for string collation). @xref{Locale
803 The regular expression referred to an invalid character class name.
808 The regular expression ended with @samp{\}.
813 There was an invalid number in the @samp{\@var{digit}} construct.
818 There were unbalanced square brackets in the regular expression.
823 An extended regular expression had unbalanced parentheses,
824 or a basic regular expression had unbalanced @samp{\(} and @samp{\)}.
829 The regular expression had unbalanced @samp{\@{} and @samp{\@}}.
834 One of the endpoints in a range expression was invalid.
839 @code{regcomp} ran out of memory.
842 @node Flags for POSIX Regexps
843 @subsection Flags for POSIX Regular Expressions
845 These are the bit flags that you can use in the @var{cflags} operand when
846 compiling a regular expression with @code{regcomp}.
852 Treat the pattern as an extended regular expression, rather than as a
853 basic regular expression.
858 Ignore case when matching letters.
863 Don't bother storing the contents of the @var{matches-ptr} array.
868 Treat a newline in @var{string} as dividing @var{string} into multiple
869 lines, so that @samp{$} can match before the newline and @samp{^} can
870 match after. Also, don't permit @samp{.} to match a newline, and don't
871 permit @samp{[^@dots{}]} to match a newline.
873 Otherwise, newline acts like any other ordinary character.
876 @node Matching POSIX Regexps
877 @subsection Matching a Compiled POSIX Regular Expression
879 Once you have compiled a regular expression, as described in @ref{POSIX
880 Regexp Compilation}, you can match it against strings using
881 @code{regexec}. A match anywhere inside the string counts as success,
882 unless the regular expression contains anchor characters (@samp{^} or
887 @deftypefun int regexec (const regex_t *restrict @var{compiled}, const char *restrict @var{string}, size_t @var{nmatch}, regmatch_t @var{matchptr}[restrict], int @var{eflags})
888 This function tries to match the compiled regular expression
889 @code{*@var{compiled}} against @var{string}.
891 @code{regexec} returns @code{0} if the regular expression matches;
892 otherwise, it returns a nonzero value. See the table below for
893 what nonzero values mean. You can use @code{regerror} to produce an
894 error message string describing the reason for a nonzero value;
895 see @ref{Regexp Cleanup}.
897 The argument @var{eflags} is a word of bit flags that enable various
900 If you want to get information about what part of @var{string} actually
901 matched the regular expression or its subexpressions, use the arguments
902 @var{matchptr} and @var{nmatch}. Otherwise, pass @code{0} for
903 @var{nmatch}, and @code{NULL} for @var{matchptr}. @xref{Regexp
907 You must match the regular expression with the same set of current
908 locales that were in effect when you compiled the regular expression.
910 The function @code{regexec} accepts the following flags in the
911 @var{eflags} argument:
917 Do not regard the beginning of the specified string as the beginning of
918 a line; more generally, don't make any assumptions about what text might
924 Do not regard the end of the specified string as the end of a line; more
925 generally, don't make any assumptions about what text might follow it.
928 Here are the possible nonzero values that @code{regexec} can return:
934 The pattern didn't match the string. This isn't really an error.
939 @code{regexec} ran out of memory.
942 @node Regexp Subexpressions
943 @subsection Match Results with Subexpressions
945 When @code{regexec} matches parenthetical subexpressions of
946 @var{pattern}, it records which parts of @var{string} they match. It
947 returns that information by storing the offsets into an array whose
948 elements are structures of type @code{regmatch_t}. The first element of
949 the array (index @code{0}) records the part of the string that matched
950 the entire regular expression. Each other element of the array records
951 the beginning and end of the part that matched a single parenthetical
956 @deftp {Data Type} regmatch_t
957 This is the data type of the @var{matcharray} array that you pass to
958 @code{regexec}. It contains two structure fields, as follows:
962 The offset in @var{string} of the beginning of a substring. Add this
963 value to @var{string} to get the address of that part.
966 The offset in @var{string} of the end of the substring.
972 @deftp {Data Type} regoff_t
973 @code{regoff_t} is an alias for another signed integer type.
974 The fields of @code{regmatch_t} have type @code{regoff_t}.
977 The @code{regmatch_t} elements correspond to subexpressions
978 positionally; the first element (index @code{1}) records where the first
979 subexpression matched, the second element records the second
980 subexpression, and so on. The order of the subexpressions is the order
983 When you call @code{regexec}, you specify how long the @var{matchptr}
984 array is, with the @var{nmatch} argument. This tells @code{regexec} how
985 many elements to store. If the actual regular expression has more than
986 @var{nmatch} subexpressions, then you won't get offset information about
987 the rest of them. But this doesn't alter whether the pattern matches a
988 particular string or not.
990 If you don't want @code{regexec} to return any information about where
991 the subexpressions matched, you can either supply @code{0} for
992 @var{nmatch}, or use the flag @code{REG_NOSUB} when you compile the
993 pattern with @code{regcomp}.
995 @node Subexpression Complications
996 @subsection Complications in Subexpression Matching
998 Sometimes a subexpression matches a substring of no characters. This
999 happens when @samp{f\(o*\)} matches the string @samp{fum}. (It really
1000 matches just the @samp{f}.) In this case, both of the offsets identify
1001 the point in the string where the null substring was found. In this
1002 example, the offsets are both @code{1}.
1004 Sometimes the entire regular expression can match without using some of
1005 its subexpressions at all---for example, when @samp{ba\(na\)*} matches the
1006 string @samp{ba}, the parenthetical subexpression is not used. When
1007 this happens, @code{regexec} stores @code{-1} in both fields of the
1008 element for that subexpression.
1010 Sometimes matching the entire regular expression can match a particular
1011 subexpression more than once---for example, when @samp{ba\(na\)*}
1012 matches the string @samp{bananana}, the parenthetical subexpression
1013 matches three times. When this happens, @code{regexec} usually stores
1014 the offsets of the last part of the string that matched the
1015 subexpression. In the case of @samp{bananana}, these offsets are
1016 @code{6} and @code{8}.
1018 But the last match is not always the one that is chosen. It's more
1019 accurate to say that the last @emph{opportunity} to match is the one
1020 that takes precedence. What this means is that when one subexpression
1021 appears within another, then the results reported for the inner
1022 subexpression reflect whatever happened on the last match of the outer
1023 subexpression. For an example, consider @samp{\(ba\(na\)*s \)*} matching
1024 the string @samp{bananas bas }. The last time the inner expression
1025 actually matches is near the end of the first word. But it is
1026 @emph{considered} again in the second word, and fails to match there.
1027 @code{regexec} reports nonuse of the ``na'' subexpression.
1029 Another place where this rule applies is when the regular expression
1031 \(ba\(na\)*s \|nefer\(ti\)* \)*
1034 matches @samp{bananas nefertiti}. The ``na'' subexpression does match
1035 in the first word, but it doesn't match in the second word because the
1036 other alternative is used there. Once again, the second repetition of
1037 the outer subexpression overrides the first, and within that second
1038 repetition, the ``na'' subexpression is not used. So @code{regexec}
1039 reports nonuse of the ``na'' subexpression.
1041 @node Regexp Cleanup
1042 @subsection POSIX Regexp Matching Cleanup
1044 When you are finished using a compiled regular expression, you can
1045 free the storage it uses by calling @code{regfree}.
1049 @deftypefun void regfree (regex_t *@var{compiled})
1050 Calling @code{regfree} frees all the storage that @code{*@var{compiled}}
1051 points to. This includes various internal fields of the @code{regex_t}
1052 structure that aren't documented in this manual.
1054 @code{regfree} does not free the object @code{*@var{compiled}} itself.
1057 You should always free the space in a @code{regex_t} structure with
1058 @code{regfree} before using the structure to compile another regular
1061 When @code{regcomp} or @code{regexec} reports an error, you can use
1062 the function @code{regerror} to turn it into an error message string.
1066 @deftypefun size_t regerror (int @var{errcode}, const regex_t *restrict @var{compiled}, char *restrict @var{buffer}, size_t @var{length})
1067 This function produces an error message string for the error code
1068 @var{errcode}, and stores the string in @var{length} bytes of memory
1069 starting at @var{buffer}. For the @var{compiled} argument, supply the
1070 same compiled regular expression structure that @code{regcomp} or
1071 @code{regexec} was working with when it got the error. Alternatively,
1072 you can supply @code{NULL} for @var{compiled}; you will still get a
1073 meaningful error message, but it might not be as detailed.
1075 If the error message can't fit in @var{length} bytes (including a
1076 terminating null character), then @code{regerror} truncates it.
1077 The string that @code{regerror} stores is always null-terminated
1078 even if it has been truncated.
1080 The return value of @code{regerror} is the minimum length needed to
1081 store the entire error message. If this is less than @var{length}, then
1082 the error message was not truncated, and you can use it. Otherwise, you
1083 should call @code{regerror} again with a larger buffer.
1085 Here is a function which uses @code{regerror}, but always dynamically
1086 allocates a buffer for the error message:
1089 char *get_regerror (int errcode, regex_t *compiled)
1091 size_t length = regerror (errcode, compiled, NULL, 0);
1092 char *buffer = xmalloc (length);
1093 (void) regerror (errcode, compiled, buffer, length);
1099 @node Word Expansion
1100 @section Shell-Style Word Expansion
1101 @cindex word expansion
1102 @cindex expansion of shell words
1104 @dfn{Word expansion} means the process of splitting a string into
1105 @dfn{words} and substituting for variables, commands, and wildcards
1106 just as the shell does.
1108 For example, when you write @samp{ls -l foo.c}, this string is split
1109 into three separate words---@samp{ls}, @samp{-l} and @samp{foo.c}.
1110 This is the most basic function of word expansion.
1112 When you write @samp{ls *.c}, this can become many words, because
1113 the word @samp{*.c} can be replaced with any number of file names.
1114 This is called @dfn{wildcard expansion}, and it is also a part of
1117 When you use @samp{echo $PATH} to print your path, you are taking
1118 advantage of @dfn{variable substitution}, which is also part of word
1121 Ordinary programs can perform word expansion just like the shell by
1122 calling the library function @code{wordexp}.
1125 * Expansion Stages:: What word expansion does to a string.
1126 * Calling Wordexp:: How to call @code{wordexp}.
1127 * Flags for Wordexp:: Options you can enable in @code{wordexp}.
1128 * Wordexp Example:: A sample program that does word expansion.
1129 * Tilde Expansion:: Details of how tilde expansion works.
1130 * Variable Substitution:: Different types of variable substitution.
1133 @node Expansion Stages
1134 @subsection The Stages of Word Expansion
1136 When word expansion is applied to a sequence of words, it performs the
1137 following transformations in the order shown here:
1141 @cindex tilde expansion
1142 @dfn{Tilde expansion}: Replacement of @samp{~foo} with the name of
1143 the home directory of @samp{foo}.
1146 Next, three different transformations are applied in the same step,
1151 @cindex variable substitution
1152 @cindex substitution of variables and commands
1153 @dfn{Variable substitution}: Environment variables are substituted for
1154 references such as @samp{$foo}.
1157 @cindex command substitution
1158 @dfn{Command substitution}: Constructs such as @w{@samp{`cat foo`}} and
1159 the equivalent @w{@samp{$(cat foo)}} are replaced with the output from
1163 @cindex arithmetic expansion
1164 @dfn{Arithmetic expansion}: Constructs such as @samp{$(($x-1))} are
1165 replaced with the result of the arithmetic computation.
1169 @cindex field splitting
1170 @dfn{Field splitting}: subdivision of the text into @dfn{words}.
1173 @cindex wildcard expansion
1174 @dfn{Wildcard expansion}: The replacement of a construct such as @samp{*.c}
1175 with a list of @samp{.c} file names. Wildcard expansion applies to an
1176 entire word at a time, and replaces that word with 0 or more file names
1177 that are themselves words.
1180 @cindex quote removal
1181 @cindex removal of quotes
1182 @dfn{Quote removal}: The deletion of string-quotes, now that they have
1183 done their job by inhibiting the above transformations when appropriate.
1186 For the details of these transformations, and how to write the constructs
1187 that use them, see @w{@cite{The BASH Manual}} (to appear).
1189 @node Calling Wordexp
1190 @subsection Calling @code{wordexp}
1192 All the functions, constants and data types for word expansion are
1193 declared in the header file @file{wordexp.h}.
1195 Word expansion produces a vector of words (strings). To return this
1196 vector, @code{wordexp} uses a special data type, @code{wordexp_t}, which
1197 is a structure. You pass @code{wordexp} the address of the structure,
1198 and it fills in the structure's fields to tell you about the results.
1202 @deftp {Data Type} {wordexp_t}
1203 This data type holds a pointer to a word vector. More precisely, it
1204 records both the address of the word vector and its size.
1208 The number of elements in the vector.
1211 The address of the vector. This field has type @w{@code{char **}}.
1214 The offset of the first real element of the vector, from its nominal
1215 address in the @code{we_wordv} field. Unlike the other fields, this
1216 is always an input to @code{wordexp}, rather than an output from it.
1218 If you use a nonzero offset, then that many elements at the beginning of
1219 the vector are left empty. (The @code{wordexp} function fills them with
1222 The @code{we_offs} field is meaningful only if you use the
1223 @code{WRDE_DOOFFS} flag. Otherwise, the offset is always zero
1224 regardless of what is in this field, and the first real element comes at
1225 the beginning of the vector.
1231 @deftypefun int wordexp (const char *@var{words}, wordexp_t *@var{word-vector-ptr}, int @var{flags})
1232 Perform word expansion on the string @var{words}, putting the result in
1233 a newly allocated vector, and store the size and address of this vector
1234 into @code{*@var{word-vector-ptr}}. The argument @var{flags} is a
1235 combination of bit flags; see @ref{Flags for Wordexp}, for details of
1238 You shouldn't use any of the characters @samp{|&;<>} in the string
1239 @var{words} unless they are quoted; likewise for newline. If you use
1240 these characters unquoted, you will get the @code{WRDE_BADCHAR} error
1241 code. Don't use parentheses or braces unless they are quoted or part of
1242 a word expansion construct. If you use quotation characters @samp{'"`},
1243 they should come in pairs that balance.
1245 The results of word expansion are a sequence of words. The function
1246 @code{wordexp} allocates a string for each resulting word, then
1247 allocates a vector of type @code{char **} to store the addresses of
1248 these strings. The last element of the vector is a null pointer.
1249 This vector is called the @dfn{word vector}.
1251 To return this vector, @code{wordexp} stores both its address and its
1252 length (number of elements, not counting the terminating null pointer)
1253 into @code{*@var{word-vector-ptr}}.
1255 If @code{wordexp} succeeds, it returns 0. Otherwise, it returns one
1256 of these error codes:
1262 The input string @var{words} contains an unquoted invalid character such
1268 The input string refers to an undefined shell variable, and you used the flag
1269 @code{WRDE_UNDEF} to forbid such references.
1274 The input string uses command substitution, and you used the flag
1275 @code{WRDE_NOCMD} to forbid command substitution.
1280 It was impossible to allocate memory to hold the result. In this case,
1281 @code{wordexp} can store part of the results---as much as it could
1287 There was a syntax error in the input string. For example, an unmatched
1288 quoting character is a syntax error.
1294 @deftypefun void wordfree (wordexp_t *@var{word-vector-ptr})
1295 Free the storage used for the word-strings and vector that
1296 @code{*@var{word-vector-ptr}} points to. This does not free the
1297 structure @code{*@var{word-vector-ptr}} itself---only the other
1301 @node Flags for Wordexp
1302 @subsection Flags for Word Expansion
1304 This section describes the flags that you can specify in the
1305 @var{flags} argument to @code{wordexp}. Choose the flags you want,
1306 and combine them with the C operator @code{|}.
1312 Append the words from this expansion to the vector of words produced by
1313 previous calls to @code{wordexp}. This way you can effectively expand
1314 several words as if they were concatenated with spaces between them.
1316 In order for appending to work, you must not modify the contents of the
1317 word vector structure between calls to @code{wordexp}. And, if you set
1318 @code{WRDE_DOOFFS} in the first call to @code{wordexp}, you must also
1319 set it when you append to the results.
1324 Leave blank slots at the beginning of the vector of words.
1325 The @code{we_offs} field says how many slots to leave.
1326 The blank slots contain null pointers.
1331 Don't do command substitution; if the input requests command substitution,
1337 Reuse a word vector made by a previous call to @code{wordexp}.
1338 Instead of allocating a new vector of words, this call to @code{wordexp}
1339 will use the vector that already exists (making it larger if necessary).
1341 Note that the vector may move, so it is not safe to save an old pointer
1342 and use it again after calling @code{wordexp}. You must fetch
1343 @code{we_pathv} anew after each call.
1348 Do show any error messages printed by commands run by command substitution.
1349 More precisely, allow these commands to inherit the standard error output
1350 stream of the current process. By default, @code{wordexp} gives these
1351 commands a standard error stream that discards all output.
1356 If the input refers to a shell variable that is not defined, report an
1360 @node Wordexp Example
1361 @subsection @code{wordexp} Example
1363 Here is an example of using @code{wordexp} to expand several strings
1364 and use the results to run a shell command. It also shows the use of
1365 @code{WRDE_APPEND} to concatenate the expansions and of @code{wordfree}
1366 to free the space allocated by @code{wordexp}.
1370 expand_and_execute (const char *program, const char **options)
1376 /* @r{Expand the string for the program to run.} */
1377 switch (wordexp (program, &result, 0))
1379 case 0: /* @r{Successful}. */
1382 /* @r{If the error was @code{WRDE_NOSPACE},}
1383 @r{then perhaps part of the result was allocated.} */
1385 default: /* @r{Some other error.} */
1389 /* @r{Expand the strings specified for the arguments.} */
1390 for (i = 0; options[i] != NULL; i++)
1392 if (wordexp (options[i], &result, WRDE_APPEND))
1402 /* @r{This is the child process. Execute the command.} */
1403 execv (result.we_wordv[0], result.we_wordv);
1404 exit (EXIT_FAILURE);
1407 /* @r{The fork failed. Report failure.} */
1410 /* @r{This is the parent process. Wait for the child to complete.} */
1411 if (waitpid (pid, &status, 0) != pid)
1419 @node Tilde Expansion
1420 @subsection Details of Tilde Expansion
1422 It's a standard part of shell syntax that you can use @samp{~} at the
1423 beginning of a file name to stand for your own home directory. You
1424 can use @samp{~@var{user}} to stand for @var{user}'s home directory.
1426 @dfn{Tilde expansion} is the process of converting these abbreviations
1427 to the directory names that they stand for.
1429 Tilde expansion applies to the @samp{~} plus all following characters up
1430 to whitespace or a slash. It takes place only at the beginning of a
1431 word, and only if none of the characters to be transformed is quoted in
1434 Plain @samp{~} uses the value of the environment variable @code{HOME}
1435 as the proper home directory name. @samp{~} followed by a user name
1436 uses @code{getpwname} to look up that user in the user database, and
1437 uses whatever directory is recorded there. Thus, @samp{~} followed
1438 by your own name can give different results from plain @samp{~}, if
1439 the value of @code{HOME} is not really your home directory.
1441 @node Variable Substitution
1442 @subsection Details of Variable Substitution
1444 Part of ordinary shell syntax is the use of @samp{$@var{variable}} to
1445 substitute the value of a shell variable into a command. This is called
1446 @dfn{variable substitution}, and it is one part of doing word expansion.
1448 There are two basic ways you can write a variable reference for
1452 @item $@{@var{variable}@}
1453 If you write braces around the variable name, then it is completely
1454 unambiguous where the variable name ends. You can concatenate
1455 additional letters onto the end of the variable value by writing them
1456 immediately after the close brace. For example, @samp{$@{foo@}s}
1457 expands into @samp{tractors}.
1459 @item $@var{variable}
1460 If you do not put braces around the variable name, then the variable
1461 name consists of all the alphanumeric characters and underscores that
1462 follow the @samp{$}. The next punctuation character ends the variable
1463 name. Thus, @samp{$foo-bar} refers to the variable @code{foo} and expands
1464 into @samp{tractor-bar}.
1467 When you use braces, you can also use various constructs to modify the
1468 value that is substituted, or test it in various ways.
1471 @item $@{@var{variable}:-@var{default}@}
1472 Substitute the value of @var{variable}, but if that is empty or
1473 undefined, use @var{default} instead.
1475 @item $@{@var{variable}:=@var{default}@}
1476 Substitute the value of @var{variable}, but if that is empty or
1477 undefined, use @var{default} instead and set the variable to
1480 @item $@{@var{variable}:?@var{message}@}
1481 If @var{variable} is defined and not empty, substitute its value.
1483 Otherwise, print @var{message} as an error message on the standard error
1484 stream, and consider word expansion a failure.
1486 @c ??? How does wordexp report such an error?
1487 @c WRDE_BADVAL is returned.
1489 @item $@{@var{variable}:+@var{replacement}@}
1490 Substitute @var{replacement}, but only if @var{variable} is defined and
1491 nonempty. Otherwise, substitute nothing for this construct.
1495 @item $@{#@var{variable}@}
1496 Substitute a numeral which expresses in base ten the number of
1497 characters in the value of @var{variable}. @samp{$@{#foo@}} stands for
1498 @samp{7}, because @samp{tractor} is seven characters.
1501 These variants of variable substitution let you remove part of the
1502 variable's value before substituting it. The @var{prefix} and
1503 @var{suffix} are not mere strings; they are wildcard patterns, just
1504 like the patterns that you use to match multiple file names. But
1505 in this context, they match against parts of the variable value
1506 rather than against file names.
1509 @item $@{@var{variable}%%@var{suffix}@}
1510 Substitute the value of @var{variable}, but first discard from that
1511 variable any portion at the end that matches the pattern @var{suffix}.
1513 If there is more than one alternative for how to match against
1514 @var{suffix}, this construct uses the longest possible match.
1516 Thus, @samp{$@{foo%%r*@}} substitutes @samp{t}, because the largest
1517 match for @samp{r*} at the end of @samp{tractor} is @samp{ractor}.
1519 @item $@{@var{variable}%@var{suffix}@}
1520 Substitute the value of @var{variable}, but first discard from that
1521 variable any portion at the end that matches the pattern @var{suffix}.
1523 If there is more than one alternative for how to match against
1524 @var{suffix}, this construct uses the shortest possible alternative.
1526 Thus, @samp{$@{foo%r*@}} substitutes @samp{tracto}, because the shortest
1527 match for @samp{r*} at the end of @samp{tractor} is just @samp{r}.
1529 @item $@{@var{variable}##@var{prefix}@}
1530 Substitute the value of @var{variable}, but first discard from that
1531 variable any portion at the beginning that matches the pattern @var{prefix}.
1533 If there is more than one alternative for how to match against
1534 @var{prefix}, this construct uses the longest possible match.
1536 Thus, @samp{$@{foo##*t@}} substitutes @samp{or}, because the largest
1537 match for @samp{*t} at the beginning of @samp{tractor} is @samp{tract}.
1539 @item $@{@var{variable}#@var{prefix}@}
1540 Substitute the value of @var{variable}, but first discard from that
1541 variable any portion at the beginning that matches the pattern @var{prefix}.
1543 If there is more than one alternative for how to match against
1544 @var{prefix}, this construct uses the shortest possible alternative.
1546 Thus, @samp{$@{foo#*t@}} substitutes @samp{ractor}, because the shortest
1547 match for @samp{*t} at the beginning of @samp{tractor} is just @samp{t}.