1 @node Pattern Matching, I/O Overview, Searching and Sorting, Top
2 @c %MENU% Matching shell ``globs'' and regular expressions
3 @chapter Pattern Matching
5 The GNU C Library provides pattern matching facilities for two kinds of
6 patterns: regular expressions and file-name wildcards. The library also
7 provides a facility for expanding variable and command references and
8 parsing text into words in the way the shell does.
11 * Wildcard Matching:: Matching a wildcard pattern against a single string.
12 * Globbing:: Finding the files that match a wildcard pattern.
13 * Regular Expressions:: Matching regular expressions against strings.
14 * Word Expansion:: Expanding shell variables, nested commands,
15 arithmetic, and wildcards.
16 This is what the shell does with shell commands.
19 @node Wildcard Matching
20 @section Wildcard Matching
23 This section describes how to match a wildcard pattern against a
24 particular string. The result is a yes or no answer: does the
25 string fit the pattern or not. The symbols described here are all
26 declared in @file{fnmatch.h}.
30 @deftypefun int fnmatch (const char *@var{pattern}, const char *@var{string}, int @var{flags})
31 This function tests whether the string @var{string} matches the pattern
32 @var{pattern}. It returns @code{0} if they do match; otherwise, it
33 returns the nonzero value @code{FNM_NOMATCH}. The arguments
34 @var{pattern} and @var{string} are both strings.
36 The argument @var{flags} is a combination of flag bits that alter the
37 details of matching. See below for a list of the defined flags.
39 In the GNU C Library, @code{fnmatch} cannot experience an ``error''---it
40 always returns an answer for whether the match succeeds. However, other
41 implementations of @code{fnmatch} might sometimes report ``errors''.
42 They would do so by returning nonzero values that are not equal to
46 These are the available flags for the @var{flags} argument:
52 Treat the @samp{/} character specially, for matching file names. If
53 this flag is set, wildcard constructs in @var{pattern} cannot match
54 @samp{/} in @var{string}. Thus, the only way to match @samp{/} is with
55 an explicit @samp{/} in @var{pattern}.
60 This is an alias for @code{FNM_FILE_NAME}; it comes from POSIX.2. We
61 don't recommend this name because we don't use the term ``pathname'' for
67 Treat the @samp{.} character specially if it appears at the beginning of
68 @var{string}. If this flag is set, wildcard constructs in @var{pattern}
69 cannot match @samp{.} as the first character of @var{string}.
71 If you set both @code{FNM_PERIOD} and @code{FNM_FILE_NAME}, then the
72 special treatment applies to @samp{.} following @samp{/} as well as to
73 @samp{.} at the beginning of @var{string}. (The shell uses the
74 @code{FNM_PERIOD} and @code{FNM_FILE_NAME} flags together for matching
80 Don't treat the @samp{\} character specially in patterns. Normally,
81 @samp{\} quotes the following character, turning off its special meaning
82 (if any) so that it matches only itself. When quoting is enabled, the
83 pattern @samp{\?} matches only the string @samp{?}, because the question
84 mark in the pattern acts like an ordinary character.
86 If you use @code{FNM_NOESCAPE}, then @samp{\} is an ordinary character.
91 Ignore a trailing sequence of characters starting with a @samp{/} in
92 @var{string}; that is to say, test whether @var{string} starts with a
93 directory name that @var{pattern} matches.
95 If this flag is set, either @samp{foo*} or @samp{foobar} as a pattern
96 would match the string @samp{foobar/frobozz}.
101 Ignore case in comparing @var{string} to @var{pattern}.
108 The archetypal use of wildcards is for matching against the files in a
109 directory, and making a list of all the matches. This is called
112 You could do this using @code{fnmatch}, by reading the directory entries
113 one by one and testing each one with @code{fnmatch}. But that would be
114 slow (and complex, since you would have to handle subdirectories by
117 The library provides a function @code{glob} to make this particular use
118 of wildcards convenient. @code{glob} and the other symbols in this
119 section are declared in @file{glob.h}.
122 * Calling Glob:: Basic use of @code{glob}.
123 * Flags for Globbing:: Flags that enable various options in @code{glob}.
124 * More Flags for Globbing:: GNU specific extensions to @code{glob}.
128 @subsection Calling @code{glob}
130 The result of globbing is a vector of file names (strings). To return
131 this vector, @code{glob} uses a special data type, @code{glob_t}, which
132 is a structure. You pass @code{glob} the address of the structure, and
133 it fills in the structure's fields to tell you about the results.
137 @deftp {Data Type} glob_t
138 This data type holds a pointer to a word vector. More precisely, it
139 records both the address of the word vector and its size. The GNU
140 implementation contains some more fields which are non-standard
145 The number of elements in the vector.
148 The address of the vector. This field has type @w{@code{char **}}.
151 The offset of the first real element of the vector, from its nominal
152 address in the @code{gl_pathv} field. Unlike the other fields, this
153 is always an input to @code{glob}, rather than an output from it.
155 If you use a nonzero offset, then that many elements at the beginning of
156 the vector are left empty. (The @code{glob} function fills them with
159 The @code{gl_offs} field is meaningful only if you use the
160 @code{GLOB_DOOFFS} flag. Otherwise, the offset is always zero
161 regardless of what is in this field, and the first real element comes at
162 the beginning of the vector.
165 The address of an alternative implementation of the @code{closedir}
166 function. It is used if the @code{GLOB_ALTDIRFUNC} bit is set in
167 the flag parameter. The type of this field is
168 @w{@code{void (*) (void *)}}.
170 This is a GNU extension.
173 The address of an alternative implementation of the @code{readdir}
174 function used to read the contents of a directory. It is used if the
175 @code{GLOB_ALTDIRFUNC} bit is set in the flag parameter. The type of
176 this field is @w{@code{struct dirent *(*) (void *)}}.
178 This is a GNU extension.
181 The address of an alternative implementation of the @code{opendir}
182 function. It is used if the @code{GLOB_ALTDIRFUNC} bit is set in
183 the flag parameter. The type of this field is
184 @w{@code{void *(*) (const char *)}}.
186 This is a GNU extension.
189 The address of an alternative implementation of the @code{stat} function
190 to get information about an object in the filesystem. It is used if the
191 @code{GLOB_ALTDIRFUNC} bit is set in the flag parameter. The type of
192 this field is @w{@code{int (*) (const char *, struct stat *)}}.
194 This is a GNU extension.
197 The address of an alternative implementation of the @code{lstat}
198 function to get information about an object in the filesystems, not
199 following symbolic links. It is used if the @code{GLOB_ALTDIRFUNC} bit
200 is set in the flag parameter. The type of this field is @code{@w{int
201 (*) (const char *,} @w{struct stat *)}}.
203 This is a GNU extension.
209 @deftypefun int glob (const char *@var{pattern}, int @var{flags}, int (*@var{errfunc}) (const char *@var{filename}, int @var{error-code}), glob_t *@var{vector-ptr})
210 The function @code{glob} does globbing using the pattern @var{pattern}
211 in the current directory. It puts the result in a newly allocated
212 vector, and stores the size and address of this vector into
213 @code{*@var{vector-ptr}}. The argument @var{flags} is a combination of
214 bit flags; see @ref{Flags for Globbing}, for details of the flags.
216 The result of globbing is a sequence of file names. The function
217 @code{glob} allocates a string for each resulting word, then
218 allocates a vector of type @code{char **} to store the addresses of
219 these strings. The last element of the vector is a null pointer.
220 This vector is called the @dfn{word vector}.
222 To return this vector, @code{glob} stores both its address and its
223 length (number of elements, not counting the terminating null pointer)
224 into @code{*@var{vector-ptr}}.
226 Normally, @code{glob} sorts the file names alphabetically before
227 returning them. You can turn this off with the flag @code{GLOB_NOSORT}
228 if you want to get the information as fast as possible. Usually it's
229 a good idea to let @code{glob} sort them---if you process the files in
230 alphabetical order, the users will have a feel for the rate of progress
231 that your application is making.
233 If @code{glob} succeeds, it returns 0. Otherwise, it returns one
234 of these error codes:
240 There was an error opening a directory, and you used the flag
241 @code{GLOB_ERR} or your specified @var{errfunc} returned a nonzero
247 @xref{Flags for Globbing},
249 for an explanation of the @code{GLOB_ERR} flag and @var{errfunc}.
254 The pattern didn't match any existing files. If you use the
255 @code{GLOB_NOCHECK} flag, then you never get this error code, because
256 that flag tells @code{glob} to @emph{pretend} that the pattern matched
262 It was impossible to allocate memory to hold the result.
265 In the event of an error, @code{glob} stores information in
266 @code{*@var{vector-ptr}} about all the matches it has found so far.
269 @node Flags for Globbing
270 @subsection Flags for Globbing
272 This section describes the flags that you can specify in the
273 @var{flags} argument to @code{glob}. Choose the flags you want,
274 and combine them with the C bitwise OR operator @code{|}.
280 Append the words from this expansion to the vector of words produced by
281 previous calls to @code{glob}. This way you can effectively expand
282 several words as if they were concatenated with spaces between them.
284 In order for appending to work, you must not modify the contents of the
285 word vector structure between calls to @code{glob}. And, if you set
286 @code{GLOB_DOOFFS} in the first call to @code{glob}, you must also
287 set it when you append to the results.
289 Note that the pointer stored in @code{gl_pathv} may no longer be valid
290 after you call @code{glob} the second time, because @code{glob} might
291 have relocated the vector. So always fetch @code{gl_pathv} from the
292 @code{glob_t} structure after each @code{glob} call; @strong{never} save
293 the pointer across calls.
298 Leave blank slots at the beginning of the vector of words.
299 The @code{gl_offs} field says how many slots to leave.
300 The blank slots contain null pointers.
305 Give up right away and report an error if there is any difficulty
306 reading the directories that must be read in order to expand @var{pattern}
307 fully. Such difficulties might include a directory in which you don't
308 have the requisite access. Normally, @code{glob} tries its best to keep
309 on going despite any errors, reading whatever directories it can.
311 You can exercise even more control than this by specifying an
312 error-handler function @var{errfunc} when you call @code{glob}. If
313 @var{errfunc} is not a null pointer, then @code{glob} doesn't give up
314 right away when it can't read a directory; instead, it calls
315 @var{errfunc} with two arguments, like this:
318 (*@var{errfunc}) (@var{filename}, @var{error-code})
322 The argument @var{filename} is the name of the directory that
323 @code{glob} couldn't open or couldn't read, and @var{error-code} is the
324 @code{errno} value that was reported to @code{glob}.
326 If the error handler function returns nonzero, then @code{glob} gives up
327 right away. Otherwise, it continues.
332 If the pattern matches the name of a directory, append @samp{/} to the
333 directory's name when returning it.
338 If the pattern doesn't match any file names, return the pattern itself
339 as if it were a file name that had been matched. (Normally, when the
340 pattern doesn't match anything, @code{glob} returns that there were no
346 Don't sort the file names; return them in no particular order.
347 (In practice, the order will depend on the order of the entries in
348 the directory.) The only reason @emph{not} to sort is to save time.
353 Don't treat the @samp{\} character specially in patterns. Normally,
354 @samp{\} quotes the following character, turning off its special meaning
355 (if any) so that it matches only itself. When quoting is enabled, the
356 pattern @samp{\?} matches only the string @samp{?}, because the question
357 mark in the pattern acts like an ordinary character.
359 If you use @code{GLOB_NOESCAPE}, then @samp{\} is an ordinary character.
361 @code{glob} does its work by calling the function @code{fnmatch}
362 repeatedly. It handles the flag @code{GLOB_NOESCAPE} by turning on the
363 @code{FNM_NOESCAPE} flag in calls to @code{fnmatch}.
366 @node More Flags for Globbing
367 @subsection More Flags for Globbing
369 Beside the flags described in the last section, the GNU implementation of
370 @code{glob} allows a few more flags which are also defined in the
371 @file{glob.h} file. Some of the extensions implement functionality
372 which is available in modern shell implementations.
378 The @code{.} character (period) is treated special. It cannot be
379 matched by wildcards. @xref{Wildcard Matching}, @code{FNM_PERIOD}.
384 The @code{GLOB_MAGCHAR} value is not to be given to @code{glob} in the
385 @var{flags} parameter. Instead, @code{glob} sets this bit in the
386 @var{gl_flags} element of the @var{glob_t} structure provided as the
387 result if the pattern used for matching contains any wildcard character.
391 @item GLOB_ALTDIRFUNC
392 Instead of the using the using the normal functions for accessing the
393 filesystem the @code{glob} implementation uses the user-supplied
394 functions specified in the structure pointed to by @var{pglob}
395 parameter. For more information about the functions refer to the
396 sections about directory handling @ref{Accessing Directories} and
397 @ref{Reading Attributes}.
402 If this flag is given the handling of braces in the pattern is changed.
403 It is now required that braces appear correctly grouped. I.e., for each
404 opening brace there must be a closing one. Braces can be used
405 recursively. So it is possible to define one brace expression in
406 another one. It is important to note that the range of each brace
407 expression is completely contained in the outer brace expression (if
410 The string between the matching braces is separated into single
411 expressions by splitting at @code{,} (comma) characters. The commas
412 themself are discarded. Please note what we said above about recursive
413 brace expressions. The commas used to separate the subexpressions must
414 be at the same level. Commas in brace subexpressions are not matched.
415 They are used during expansion of the brace expression of the deeper
416 level. The example below shows this
419 glob ("@{foo/@{,bar,biz@},baz@}", GLOB_BRACE, NULL, &result)
423 is equivalent to the sequence
426 glob ("foo/", GLOB_BRACE, NULL, &result)
427 glob ("foo/bar", GLOB_BRACE|GLOB_APPEND, NULL, &result)
428 glob ("foo/biz", GLOB_BRACE|GLOB_APPEND, NULL, &result)
429 glob ("baz", GLOB_BRACE|GLOB_APPEND, NULL, &result)
433 if we leave aside error handling.
438 If the pattern contains no wildcard constructs (it is a literal file name),
439 return it as the sole ``matching'' word, even if no file exists by that name.
444 If this flag is used the character @code{~} (tilde) is handled special
445 if it appears at the beginning of the pattern. Instead of being taken
446 verbatim it is used to represent the home directory of a known user.
448 If @code{~} is the only character in pattern or it is followed by a
449 @code{/} (slash), the home directory of the process owner is
450 substituted. Using @code{getlogin} and @code{getpwnam} the information
451 is read from the system databases. As an example take user @code{bart}
452 with his home directory at @file{/home/bart}. For him a call like
455 glob ("~/bin/*", GLOB_TILDE, NULL, &result)
459 would return the contents of the directory @file{/home/bart/bin}.
460 Instead of referring to the own home directory it is also possible to
461 name the home directory of other users. To do so one has to append the
462 user name after the tilde character. So the contents of user
463 @code{homer}'s @file{bin} directory can be retrieved by
466 glob ("~homer/bin/*", GLOB_TILDE, NULL, &result)
469 If the user name is not valid or the home directory cannot be determined
470 for some reason the pattern is left untouched and itself used as the
471 result. I.e., if in the last example @code{home} is not available the
472 tilde expansion yields to @code{"~homer/bin/*"} and @code{glob} is not
473 looking for a directory named @code{~homer}.
475 This functionality is equivalent to what is available in C-shells if the
476 @code{nonomatch} flag is set.
480 @item GLOB_TILDE_CHECK
481 If this flag is used @code{glob} behaves like as if @code{GLOB_TILDE} is
482 given. The only difference is that if the user name is not available or
483 the home directory cannot be determined for other reasons this leads to
484 an error. @code{glob} will return @code{GLOB_NOMATCH} instead of using
485 the pattern itself as the name.
487 This functionality is equivalent to what is available in C-shells if
488 @code{nonomatch} flag is not set.
493 If this flag is used the globbing function takes this as a
494 @strong{hint} that the caller is only interested in directories
495 matching the pattern. If the information about the type of the file
496 is easily available non-directories will be rejected but no extra
497 work will be done to determine the information for each file. I.e.,
498 the caller must still be able to filter directories out.
500 This functionality is only available with the GNU @code{glob}
501 implementation. It is mainly used internally to increase the
502 performance but might be useful for a user as well and therefore is
506 Calling @code{glob} will in most cases allocate resources which are used
507 to represent the result of the function call. If the same object of
508 type @code{glob_t} is used in multiple call to @code{glob} the resources
509 are freed or reused so that no leaks appear. But this does not include
510 the time when all @code{glob} calls are done.
514 @deftypefun void globfree (glob_t *@var{pglob})
515 The @code{globfree} function frees all resources allocated by previous
516 calls to @code{glob} associated with the object pointed to by
517 @var{pglob}. This function should be called whenever the currently used
518 @code{glob_t} typed object isn't used anymore.
522 @node Regular Expressions
523 @section Regular Expression Matching
525 The GNU C library supports two interfaces for matching regular
526 expressions. One is the standard POSIX.2 interface, and the other is
527 what the GNU system has had for many years.
529 Both interfaces are declared in the header file @file{regex.h}.
530 If you define @w{@code{_POSIX_C_SOURCE}}, then only the POSIX.2
531 functions, structures, and constants are declared.
532 @c !!! we only document the POSIX.2 interface here!!
535 * POSIX Regexp Compilation:: Using @code{regcomp} to prepare to match.
536 * Flags for POSIX Regexps:: Syntax variations for @code{regcomp}.
537 * Matching POSIX Regexps:: Using @code{regexec} to match the compiled
538 pattern that you get from @code{regcomp}.
539 * Regexp Subexpressions:: Finding which parts of the string were matched.
540 * Subexpression Complications:: Find points of which parts were matched.
541 * Regexp Cleanup:: Freeing storage; reporting errors.
544 @node POSIX Regexp Compilation
545 @subsection POSIX Regular Expression Compilation
547 Before you can actually match a regular expression, you must
548 @dfn{compile} it. This is not true compilation---it produces a special
549 data structure, not machine instructions. But it is like ordinary
550 compilation in that its purpose is to enable you to ``execute'' the
551 pattern fast. (@xref{Matching POSIX Regexps}, for how to use the
552 compiled regular expression for matching.)
554 There is a special data type for compiled regular expressions:
558 @deftp {Data Type} regex_t
559 This type of object holds a compiled regular expression.
560 It is actually a structure. It has just one field that your programs
565 This field holds the number of parenthetical subexpressions in the
566 regular expression that was compiled.
569 There are several other fields, but we don't describe them here, because
570 only the functions in the library should use them.
573 After you create a @code{regex_t} object, you can compile a regular
574 expression into it by calling @code{regcomp}.
578 @deftypefun int regcomp (regex_t *@var{compiled}, const char *@var{pattern}, int @var{cflags})
579 The function @code{regcomp} ``compiles'' a regular expression into a
580 data structure that you can use with @code{regexec} to match against a
581 string. The compiled regular expression format is designed for
582 efficient matching. @code{regcomp} stores it into @code{*@var{compiled}}.
584 It's up to you to allocate an object of type @code{regex_t} and pass its
585 address to @code{regcomp}.
587 The argument @var{cflags} lets you specify various options that control
588 the syntax and semantics of regular expressions. @xref{Flags for POSIX
591 If you use the flag @code{REG_NOSUB}, then @code{regcomp} omits from
592 the compiled regular expression the information necessary to record
593 how subexpressions actually match. In this case, you might as well
594 pass @code{0} for the @var{matchptr} and @var{nmatch} arguments when
595 you call @code{regexec}.
597 If you don't use @code{REG_NOSUB}, then the compiled regular expression
598 does have the capacity to record how subexpressions match. Also,
599 @code{regcomp} tells you how many subexpressions @var{pattern} has, by
600 storing the number in @code{@var{compiled}->re_nsub}. You can use that
601 value to decide how long an array to allocate to hold information about
602 subexpression matches.
604 @code{regcomp} returns @code{0} if it succeeds in compiling the regular
605 expression; otherwise, it returns a nonzero error code (see the table
606 below). You can use @code{regerror} to produce an error message string
607 describing the reason for a nonzero value; see @ref{Regexp Cleanup}.
611 Here are the possible nonzero values that @code{regcomp} can return:
617 There was an invalid @samp{\@{@dots{}\@}} construct in the regular
618 expression. A valid @samp{\@{@dots{}\@}} construct must contain either
619 a single number, or two numbers in increasing order separated by a
625 There was a syntax error in the regular expression.
630 A repetition operator such as @samp{?} or @samp{*} appeared in a bad
631 position (with no preceding subexpression to act on).
636 The regular expression referred to an invalid collating element (one not
637 defined in the current locale for string collation). @xref{Locale
643 The regular expression referred to an invalid character class name.
648 The regular expression ended with @samp{\}.
653 There was an invalid number in the @samp{\@var{digit}} construct.
658 There were unbalanced square brackets in the regular expression.
663 An extended regular expression had unbalanced parentheses,
664 or a basic regular expression had unbalanced @samp{\(} and @samp{\)}.
669 The regular expression had unbalanced @samp{\@{} and @samp{\@}}.
674 One of the endpoints in a range expression was invalid.
679 @code{regcomp} ran out of memory.
682 @node Flags for POSIX Regexps
683 @subsection Flags for POSIX Regular Expressions
685 These are the bit flags that you can use in the @var{cflags} operand when
686 compiling a regular expression with @code{regcomp}.
692 Treat the pattern as an extended regular expression, rather than as a
693 basic regular expression.
698 Ignore case when matching letters.
703 Don't bother storing the contents of the @var{matches-ptr} array.
708 Treat a newline in @var{string} as dividing @var{string} into multiple
709 lines, so that @samp{$} can match before the newline and @samp{^} can
710 match after. Also, don't permit @samp{.} to match a newline, and don't
711 permit @samp{[^@dots{}]} to match a newline.
713 Otherwise, newline acts like any other ordinary character.
716 @node Matching POSIX Regexps
717 @subsection Matching a Compiled POSIX Regular Expression
719 Once you have compiled a regular expression, as described in @ref{POSIX
720 Regexp Compilation}, you can match it against strings using
721 @code{regexec}. A match anywhere inside the string counts as success,
722 unless the regular expression contains anchor characters (@samp{^} or
727 @deftypefun int regexec (regex_t *@var{compiled}, char *@var{string}, size_t @var{nmatch}, regmatch_t @var{matchptr} @t{[]}, int @var{eflags})
728 This function tries to match the compiled regular expression
729 @code{*@var{compiled}} against @var{string}.
731 @code{regexec} returns @code{0} if the regular expression matches;
732 otherwise, it returns a nonzero value. See the table below for
733 what nonzero values mean. You can use @code{regerror} to produce an
734 error message string describing the reason for a nonzero value;
735 see @ref{Regexp Cleanup}.
737 The argument @var{eflags} is a word of bit flags that enable various
740 If you want to get information about what part of @var{string} actually
741 matched the regular expression or its subexpressions, use the arguments
742 @var{matchptr} and @var{nmatch}. Otherwise, pass @code{0} for
743 @var{nmatch}, and @code{NULL} for @var{matchptr}. @xref{Regexp
747 You must match the regular expression with the same set of current
748 locales that were in effect when you compiled the regular expression.
750 The function @code{regexec} accepts the following flags in the
751 @var{eflags} argument:
757 Do not regard the beginning of the specified string as the beginning of
758 a line; more generally, don't make any assumptions about what text might
764 Do not regard the end of the specified string as the end of a line; more
765 generally, don't make any assumptions about what text might follow it.
768 Here are the possible nonzero values that @code{regexec} can return:
774 The pattern didn't match the string. This isn't really an error.
779 @code{regexec} ran out of memory.
782 @node Regexp Subexpressions
783 @subsection Match Results with Subexpressions
785 When @code{regexec} matches parenthetical subexpressions of
786 @var{pattern}, it records which parts of @var{string} they match. It
787 returns that information by storing the offsets into an array whose
788 elements are structures of type @code{regmatch_t}. The first element of
789 the array (index @code{0}) records the part of the string that matched
790 the entire regular expression. Each other element of the array records
791 the beginning and end of the part that matched a single parenthetical
796 @deftp {Data Type} regmatch_t
797 This is the data type of the @var{matcharray} array that you pass to
798 @code{regexec}. It contains two structure fields, as follows:
802 The offset in @var{string} of the beginning of a substring. Add this
803 value to @var{string} to get the address of that part.
806 The offset in @var{string} of the end of the substring.
812 @deftp {Data Type} regoff_t
813 @code{regoff_t} is an alias for another signed integer type.
814 The fields of @code{regmatch_t} have type @code{regoff_t}.
817 The @code{regmatch_t} elements correspond to subexpressions
818 positionally; the first element (index @code{1}) records where the first
819 subexpression matched, the second element records the second
820 subexpression, and so on. The order of the subexpressions is the order
823 When you call @code{regexec}, you specify how long the @var{matchptr}
824 array is, with the @var{nmatch} argument. This tells @code{regexec} how
825 many elements to store. If the actual regular expression has more than
826 @var{nmatch} subexpressions, then you won't get offset information about
827 the rest of them. But this doesn't alter whether the pattern matches a
828 particular string or not.
830 If you don't want @code{regexec} to return any information about where
831 the subexpressions matched, you can either supply @code{0} for
832 @var{nmatch}, or use the flag @code{REG_NOSUB} when you compile the
833 pattern with @code{regcomp}.
835 @node Subexpression Complications
836 @subsection Complications in Subexpression Matching
838 Sometimes a subexpression matches a substring of no characters. This
839 happens when @samp{f\(o*\)} matches the string @samp{fum}. (It really
840 matches just the @samp{f}.) In this case, both of the offsets identify
841 the point in the string where the null substring was found. In this
842 example, the offsets are both @code{1}.
844 Sometimes the entire regular expression can match without using some of
845 its subexpressions at all---for example, when @samp{ba\(na\)*} matches the
846 string @samp{ba}, the parenthetical subexpression is not used. When
847 this happens, @code{regexec} stores @code{-1} in both fields of the
848 element for that subexpression.
850 Sometimes matching the entire regular expression can match a particular
851 subexpression more than once---for example, when @samp{ba\(na\)*}
852 matches the string @samp{bananana}, the parenthetical subexpression
853 matches three times. When this happens, @code{regexec} usually stores
854 the offsets of the last part of the string that matched the
855 subexpression. In the case of @samp{bananana}, these offsets are
856 @code{6} and @code{8}.
858 But the last match is not always the one that is chosen. It's more
859 accurate to say that the last @emph{opportunity} to match is the one
860 that takes precedence. What this means is that when one subexpression
861 appears within another, then the results reported for the inner
862 subexpression reflect whatever happened on the last match of the outer
863 subexpression. For an example, consider @samp{\(ba\(na\)*s \)*} matching
864 the string @samp{bananas bas }. The last time the inner expression
865 actually matches is near the end of the first word. But it is
866 @emph{considered} again in the second word, and fails to match there.
867 @code{regexec} reports nonuse of the ``na'' subexpression.
869 Another place where this rule applies is when the regular expression
871 \(ba\(na\)*s \|nefer\(ti\)* \)*
874 matches @samp{bananas nefertiti}. The ``na'' subexpression does match
875 in the first word, but it doesn't match in the second word because the
876 other alternative is used there. Once again, the second repetition of
877 the outer subexpression overrides the first, and within that second
878 repetition, the ``na'' subexpression is not used. So @code{regexec}
879 reports nonuse of the ``na'' subexpression.
882 @subsection POSIX Regexp Matching Cleanup
884 When you are finished using a compiled regular expression, you can
885 free the storage it uses by calling @code{regfree}.
889 @deftypefun void regfree (regex_t *@var{compiled})
890 Calling @code{regfree} frees all the storage that @code{*@var{compiled}}
891 points to. This includes various internal fields of the @code{regex_t}
892 structure that aren't documented in this manual.
894 @code{regfree} does not free the object @code{*@var{compiled}} itself.
897 You should always free the space in a @code{regex_t} structure with
898 @code{regfree} before using the structure to compile another regular
901 When @code{regcomp} or @code{regexec} reports an error, you can use
902 the function @code{regerror} to turn it into an error message string.
906 @deftypefun size_t regerror (int @var{errcode}, regex_t *@var{compiled}, char *@var{buffer}, size_t @var{length})
907 This function produces an error message string for the error code
908 @var{errcode}, and stores the string in @var{length} bytes of memory
909 starting at @var{buffer}. For the @var{compiled} argument, supply the
910 same compiled regular expression structure that @code{regcomp} or
911 @code{regexec} was working with when it got the error. Alternatively,
912 you can supply @code{NULL} for @var{compiled}; you will still get a
913 meaningful error message, but it might not be as detailed.
915 If the error message can't fit in @var{length} bytes (including a
916 terminating null character), then @code{regerror} truncates it.
917 The string that @code{regerror} stores is always null-terminated
918 even if it has been truncated.
920 The return value of @code{regerror} is the minimum length needed to
921 store the entire error message. If this is less than @var{length}, then
922 the error message was not truncated, and you can use it. Otherwise, you
923 should call @code{regerror} again with a larger buffer.
925 Here is a function which uses @code{regerror}, but always dynamically
926 allocates a buffer for the error message:
929 char *get_regerror (int errcode, regex_t *compiled)
931 size_t length = regerror (errcode, compiled, NULL, 0);
932 char *buffer = xmalloc (length);
933 (void) regerror (errcode, compiled, buffer, length);
939 @c !!!! this is not actually in the library....
941 @section Shell-Style Word Expansion
942 @cindex word expansion
943 @cindex expansion of shell words
945 @dfn{Word expansion} means the process of splitting a string into
946 @dfn{words} and substituting for variables, commands, and wildcards
947 just as the shell does.
949 For example, when you write @samp{ls -l foo.c}, this string is split
950 into three separate words---@samp{ls}, @samp{-l} and @samp{foo.c}.
951 This is the most basic function of word expansion.
953 When you write @samp{ls *.c}, this can become many words, because
954 the word @samp{*.c} can be replaced with any number of file names.
955 This is called @dfn{wildcard expansion}, and it is also a part of
958 When you use @samp{echo $PATH} to print your path, you are taking
959 advantage of @dfn{variable substitution}, which is also part of word
962 Ordinary programs can perform word expansion just like the shell by
963 calling the library function @code{wordexp}.
966 * Expansion Stages:: What word expansion does to a string.
967 * Calling Wordexp:: How to call @code{wordexp}.
968 * Flags for Wordexp:: Options you can enable in @code{wordexp}.
969 * Wordexp Example:: A sample program that does word expansion.
972 @node Expansion Stages
973 @subsection The Stages of Word Expansion
975 When word expansion is applied to a sequence of words, it performs the
976 following transformations in the order shown here:
980 @cindex tilde expansion
981 @dfn{Tilde expansion}: Replacement of @samp{~foo} with the name of
982 the home directory of @samp{foo}.
985 Next, three different transformations are applied in the same step,
990 @cindex variable substitution
991 @cindex substitution of variables and commands
992 @dfn{Variable substitution}: Environment variables are substituted for
993 references such as @samp{$foo}.
996 @cindex command substitution
997 @dfn{Command substitution}: Constructs such as @w{@samp{`cat foo`}} and
998 the equivalent @w{@samp{$(cat foo)}} are replaced with the output from
1002 @cindex arithmetic expansion
1003 @dfn{Arithmetic expansion}: Constructs such as @samp{$(($x-1))} are
1004 replaced with the result of the arithmetic computation.
1008 @cindex field splitting
1009 @dfn{Field splitting}: subdivision of the text into @dfn{words}.
1012 @cindex wildcard expansion
1013 @dfn{Wildcard expansion}: The replacement of a construct such as @samp{*.c}
1014 with a list of @samp{.c} file names. Wildcard expansion applies to an
1015 entire word at a time, and replaces that word with 0 or more file names
1016 that are themselves words.
1019 @cindex quote removal
1020 @cindex removal of quotes
1021 @dfn{Quote removal}: The deletion of string-quotes, now that they have
1022 done their job by inhibiting the above transformations when appropriate.
1025 For the details of these transformations, and how to write the constructs
1026 that use them, see @w{@cite{The BASH Manual}} (to appear).
1028 @node Calling Wordexp
1029 @subsection Calling @code{wordexp}
1031 All the functions, constants and data types for word expansion are
1032 declared in the header file @file{wordexp.h}.
1034 Word expansion produces a vector of words (strings). To return this
1035 vector, @code{wordexp} uses a special data type, @code{wordexp_t}, which
1036 is a structure. You pass @code{wordexp} the address of the structure,
1037 and it fills in the structure's fields to tell you about the results.
1041 @deftp {Data Type} {wordexp_t}
1042 This data type holds a pointer to a word vector. More precisely, it
1043 records both the address of the word vector and its size.
1047 The number of elements in the vector.
1050 The address of the vector. This field has type @w{@code{char **}}.
1053 The offset of the first real element of the vector, from its nominal
1054 address in the @code{we_wordv} field. Unlike the other fields, this
1055 is always an input to @code{wordexp}, rather than an output from it.
1057 If you use a nonzero offset, then that many elements at the beginning of
1058 the vector are left empty. (The @code{wordexp} function fills them with
1061 The @code{we_offs} field is meaningful only if you use the
1062 @code{WRDE_DOOFFS} flag. Otherwise, the offset is always zero
1063 regardless of what is in this field, and the first real element comes at
1064 the beginning of the vector.
1070 @deftypefun int wordexp (const char *@var{words}, wordexp_t *@var{word-vector-ptr}, int @var{flags})
1071 Perform word expansion on the string @var{words}, putting the result in
1072 a newly allocated vector, and store the size and address of this vector
1073 into @code{*@var{word-vector-ptr}}. The argument @var{flags} is a
1074 combination of bit flags; see @ref{Flags for Wordexp}, for details of
1077 You shouldn't use any of the characters @samp{|&;<>} in the string
1078 @var{words} unless they are quoted; likewise for newline. If you use
1079 these characters unquoted, you will get the @code{WRDE_BADCHAR} error
1080 code. Don't use parentheses or braces unless they are quoted or part of
1081 a word expansion construct. If you use quotation characters @samp{'"`},
1082 they should come in pairs that balance.
1084 The results of word expansion are a sequence of words. The function
1085 @code{wordexp} allocates a string for each resulting word, then
1086 allocates a vector of type @code{char **} to store the addresses of
1087 these strings. The last element of the vector is a null pointer.
1088 This vector is called the @dfn{word vector}.
1090 To return this vector, @code{wordexp} stores both its address and its
1091 length (number of elements, not counting the terminating null pointer)
1092 into @code{*@var{word-vector-ptr}}.
1094 If @code{wordexp} succeeds, it returns 0. Otherwise, it returns one
1095 of these error codes:
1101 The input string @var{words} contains an unquoted invalid character such
1107 The input string refers to an undefined shell variable, and you used the flag
1108 @code{WRDE_UNDEF} to forbid such references.
1113 The input string uses command substitution, and you used the flag
1114 @code{WRDE_NOCMD} to forbid command substitution.
1119 It was impossible to allocate memory to hold the result. In this case,
1120 @code{wordexp} can store part of the results---as much as it could
1126 There was a syntax error in the input string. For example, an unmatched
1127 quoting character is a syntax error.
1133 @deftypefun void wordfree (wordexp_t *@var{word-vector-ptr})
1134 Free the storage used for the word-strings and vector that
1135 @code{*@var{word-vector-ptr}} points to. This does not free the
1136 structure @code{*@var{word-vector-ptr}} itself---only the other
1140 @node Flags for Wordexp
1141 @subsection Flags for Word Expansion
1143 This section describes the flags that you can specify in the
1144 @var{flags} argument to @code{wordexp}. Choose the flags you want,
1145 and combine them with the C operator @code{|}.
1151 Append the words from this expansion to the vector of words produced by
1152 previous calls to @code{wordexp}. This way you can effectively expand
1153 several words as if they were concatenated with spaces between them.
1155 In order for appending to work, you must not modify the contents of the
1156 word vector structure between calls to @code{wordexp}. And, if you set
1157 @code{WRDE_DOOFFS} in the first call to @code{wordexp}, you must also
1158 set it when you append to the results.
1163 Leave blank slots at the beginning of the vector of words.
1164 The @code{we_offs} field says how many slots to leave.
1165 The blank slots contain null pointers.
1170 Don't do command substitution; if the input requests command substitution,
1176 Reuse a word vector made by a previous call to @code{wordexp}.
1177 Instead of allocating a new vector of words, this call to @code{wordexp}
1178 will use the vector that already exists (making it larger if necessary).
1180 Note that the vector may move, so it is not safe to save an old pointer
1181 and use it again after calling @code{wordexp}. You must fetch
1182 @code{we_pathv} anew after each call.
1187 Do show any error messages printed by commands run by command substitution.
1188 More precisely, allow these commands to inherit the standard error output
1189 stream of the current process. By default, @code{wordexp} gives these
1190 commands a standard error stream that discards all output.
1195 If the input refers to a shell variable that is not defined, report an
1199 @node Wordexp Example
1200 @subsection @code{wordexp} Example
1202 Here is an example of using @code{wordexp} to expand several strings
1203 and use the results to run a shell command. It also shows the use of
1204 @code{WRDE_APPEND} to concatenate the expansions and of @code{wordfree}
1205 to free the space allocated by @code{wordexp}.
1209 expand_and_execute (const char *program, const char *options)
1215 /* @r{Expand the string for the program to run.} */
1216 switch (wordexp (program, &result, 0))
1218 case 0: /* @r{Successful}. */
1221 /* @r{If the error was @code{WRDE_NOSPACE},}
1222 @r{then perhaps part of the result was allocated.} */
1224 default: /* @r{Some other error.} */
1228 /* @r{Expand the strings specified for the arguments.} */
1229 for (i = 0; args[i]; i++)
1231 if (wordexp (options, &result, WRDE_APPEND))
1241 /* @r{This is the child process. Execute the command.} */
1242 execv (result.we_wordv[0], result.we_wordv);
1243 exit (EXIT_FAILURE);
1246 /* @r{The fork failed. Report failure.} */
1249 /* @r{This is the parent process. Wait for the child to complete.} */
1250 if (waitpid (pid, &status, 0) != pid)
1259 @c No sense finishing this for here.
1261 @node Tilde Expansion
1262 @subsection Details of Tilde Expansion
1264 It's a standard part of shell syntax that you can use @samp{~} at the
1265 beginning of a file name to stand for your own home directory. You
1266 can use @samp{~@var{user}} to stand for @var{user}'s home directory.
1268 @dfn{Tilde expansion} is the process of converting these abbreviations
1269 to the directory names that they stand for.
1271 Tilde expansion applies to the @samp{~} plus all following characters up
1272 to whitespace or a slash. It takes place only at the beginning of a
1273 word, and only if none of the characters to be transformed is quoted in
1276 Plain @samp{~} uses the value of the environment variable @code{HOME}
1277 as the proper home directory name. @samp{~} followed by a user name
1278 uses @code{getpwname} to look up that user in the user database, and
1279 uses whatever directory is recorded there. Thus, @samp{~} followed
1280 by your own name can give different results from plain @samp{~}, if
1281 the value of @code{HOME} is not really your home directory.
1283 @node Variable Substitution
1284 @subsection Details of Variable Substitution
1286 Part of ordinary shell syntax is the use of @samp{$@var{variable}} to
1287 substitute the value of a shell variable into a command. This is called
1288 @dfn{variable substitution}, and it is one part of doing word expansion.
1290 There are two basic ways you can write a variable reference for
1294 @item $@{@var{variable}@}
1295 If you write braces around the variable name, then it is completely
1296 unambiguous where the variable name ends. You can concatenate
1297 additional letters onto the end of the variable value by writing them
1298 immediately after the close brace. For example, @samp{$@{foo@}s}
1299 expands into @samp{tractors}.
1301 @item $@var{variable}
1302 If you do not put braces around the variable name, then the variable
1303 name consists of all the alphanumeric characters and underscores that
1304 follow the @samp{$}. The next punctuation character ends the variable
1305 name. Thus, @samp{$foo-bar} refers to the variable @code{foo} and expands
1306 into @samp{tractor-bar}.
1309 When you use braces, you can also use various constructs to modify the
1310 value that is substituted, or test it in various ways.
1313 @item $@{@var{variable}:-@var{default}@}
1314 Substitute the value of @var{variable}, but if that is empty or
1315 undefined, use @var{default} instead.
1317 @item $@{@var{variable}:=@var{default}@}
1318 Substitute the value of @var{variable}, but if that is empty or
1319 undefined, use @var{default} instead and set the variable to
1322 @item $@{@var{variable}:?@var{message}@}
1323 If @var{variable} is defined and not empty, substitute its value.
1325 Otherwise, print @var{message} as an error message on the standard error
1326 stream, and consider word expansion a failure.
1328 @c ??? How does wordexp report such an error?
1330 @item $@{@var{variable}:+@var{replacement}@}
1331 Substitute @var{replacement}, but only if @var{variable} is defined and
1332 nonempty. Otherwise, substitute nothing for this construct.
1336 @item $@{#@var{variable}@}
1337 Substitute a numeral which expresses in base ten the number of
1338 characters in the value of @var{variable}. @samp{$@{#foo@}} stands for
1339 @samp{7}, because @samp{tractor} is seven characters.
1342 These variants of variable substitution let you remove part of the
1343 variable's value before substituting it. The @var{prefix} and
1344 @var{suffix} are not mere strings; they are wildcard patterns, just
1345 like the patterns that you use to match multiple file names. But
1346 in this context, they match against parts of the variable value
1347 rather than against file names.
1350 @item $@{@var{variable}%%@var{suffix}@}
1351 Substitute the value of @var{variable}, but first discard from that
1352 variable any portion at the end that matches the pattern @var{suffix}.
1354 If there is more than one alternative for how to match against
1355 @var{suffix}, this construct uses the longest possible match.
1357 Thus, @samp{$@{foo%%r*@}} substitutes @samp{t}, because the largest
1358 match for @samp{r*} at the end of @samp{tractor} is @samp{ractor}.
1360 @item $@{@var{variable}%@var{suffix}@}
1361 Substitute the value of @var{variable}, but first discard from that
1362 variable any portion at the end that matches the pattern @var{suffix}.
1364 If there is more than one alternative for how to match against
1365 @var{suffix}, this construct uses the shortest possible alternative.
1367 Thus, @samp{$@{foo%%r*@}} substitutes @samp{tracto}, because the shortest
1368 match for @samp{r*} at the end of @samp{tractor} is just @samp{r}.
1370 @item $@{@var{variable}##@var{prefix}@}
1371 Substitute the value of @var{variable}, but first discard from that
1372 variable any portion at the beginning that matches the pattern @var{prefix}.
1374 If there is more than one alternative for how to match against
1375 @var{prefix}, this construct uses the longest possible match.
1377 Thus, @samp{$@{foo%%r*@}} substitutes @samp{t}, because the largest
1378 match for @samp{r*} at the end of @samp{tractor} is @samp{ractor}.
1380 @item $@{@var{variable}#@var{prefix}@}
1381 Substitute the value of @var{variable}, but first discard from that
1382 variable any portion at the beginning that matches the pattern @var{prefix}.
1384 If there is more than one alternative for how to match against
1385 @var{prefix}, this construct uses the shortest possible alternative.
1387 Thus, @samp{$@{foo%%r*@}} substitutes @samp{tracto}, because the shortest
1388 match for @samp{r*} at the end of @samp{tractor} is just @samp{r}.