2 @c This is part of the GNU Emacs Lisp Reference Manual.
3 @c Copyright (C) 1990, 1991, 1992, 1993, 1994, 1995, 1998, 1999, 2001,
4 @c 2002, 2003, 2004, 2005, 2006, 2007 Free Software Foundation, Inc.
5 @c See the file elisp.texi for copying conditions.
6 @setfilename ../info/syntax
7 @node Syntax Tables, Abbrevs, Searching and Matching, Top
9 @cindex parsing buffer text
13 A @dfn{syntax table} specifies the syntactic textual function of each
14 character. This information is used by the @dfn{parsing functions}, the
15 complex movement commands, and others to determine where words, symbols,
16 and other syntactic constructs begin and end. The current syntax table
17 controls the meaning of the word motion functions (@pxref{Word Motion})
18 and the list motion functions (@pxref{List Motion}), as well as the
19 functions in this chapter.
22 * Basics: Syntax Basics. Basic concepts of syntax tables.
23 * Desc: Syntax Descriptors. How characters are classified.
24 * Syntax Table Functions:: How to create, examine and alter syntax tables.
25 * Syntax Properties:: Overriding syntax with text properties.
26 * Motion and Syntax:: Moving over characters with certain syntaxes.
27 * Parsing Expressions:: Parsing balanced expressions
28 using the syntax table.
29 * Standard Syntax Tables:: Syntax tables used by various major modes.
30 * Syntax Table Internals:: How syntax table information is stored.
31 * Categories:: Another way of classifying character syntax.
35 @section Syntax Table Concepts
38 A @dfn{syntax table} provides Emacs with the information that
39 determines the syntactic use of each character in a buffer. This
40 information is used by the parsing commands, the complex movement
41 commands, and others to determine where words, symbols, and other
42 syntactic constructs begin and end. The current syntax table controls
43 the meaning of the word motion functions (@pxref{Word Motion}) and the
44 list motion functions (@pxref{List Motion}) as well as the functions in
48 A syntax table is a char-table (@pxref{Char-Tables}). The element at
49 index @var{c} describes the character with code @var{c}. The element's
50 value should be a list that encodes the syntax of the character in
53 Syntax tables are used only for moving across text, not for the Emacs
54 Lisp reader. Emacs Lisp uses built-in syntactic rules when reading Lisp
55 expressions, and these rules cannot be changed. (Some Lisp systems
56 provide ways to redefine the read syntax, but we decided to leave this
57 feature out of Emacs Lisp for simplicity.)
59 Each buffer has its own major mode, and each major mode has its own
60 idea of the syntactic class of various characters. For example, in Lisp
61 mode, the character @samp{;} begins a comment, but in C mode, it
62 terminates a statement. To support these variations, Emacs makes the
63 choice of syntax table local to each buffer. Typically, each major
64 mode has its own syntax table and installs that table in each buffer
65 that uses that mode. Changing this table alters the syntax in all
66 those buffers as well as in any buffers subsequently put in that mode.
67 Occasionally several similar modes share one syntax table.
68 @xref{Example Major Modes}, for an example of how to set up a syntax
71 A syntax table can inherit the data for some characters from the
72 standard syntax table, while specifying other characters itself. The
73 ``inherit'' syntax class means ``inherit this character's syntax from
74 the standard syntax table.'' Just changing the standard syntax for a
75 character affects all syntax tables that inherit from it.
77 @defun syntax-table-p object
78 This function returns @code{t} if @var{object} is a syntax table.
81 @node Syntax Descriptors
82 @section Syntax Descriptors
85 This section describes the syntax classes and flags that denote the
86 syntax of a character, and how they are represented as a @dfn{syntax
87 descriptor}, which is a Lisp string that you pass to
88 @code{modify-syntax-entry} to specify the syntax you want.
90 The syntax table specifies a syntax class for each character. There
91 is no necessary relationship between the class of a character in one
92 syntax table and its class in any other table.
94 Each class is designated by a mnemonic character, which serves as the
95 name of the class when you need to specify a class. Usually the
96 designator character is one that is often assigned that class; however,
97 its meaning as a designator is unvarying and independent of what syntax
98 that character currently has. Thus, @samp{\} as a designator character
99 always gives ``escape character'' syntax, regardless of what syntax
100 @samp{\} currently has.
102 @cindex syntax descriptor
103 A syntax descriptor is a Lisp string that specifies a syntax class, a
104 matching character (used only for the parenthesis classes) and flags.
105 The first character is the designator for a syntax class. The second
106 character is the character to match; if it is unused, put a space there.
107 Then come the characters for any desired flags. If no matching
108 character or flags are needed, one character is sufficient.
110 For example, the syntax descriptor for the character @samp{*} in C
111 mode is @samp{@w{. 23}} (i.e., punctuation, matching character slot
112 unused, second character of a comment-starter, first character of a
113 comment-ender), and the entry for @samp{/} is @samp{@w{. 14}} (i.e.,
114 punctuation, matching character slot unused, first character of a
115 comment-starter, second character of a comment-ender).
118 * Syntax Class Table:: Table of syntax classes.
119 * Syntax Flags:: Additional flags each character can have.
122 @node Syntax Class Table
123 @subsection Table of Syntax Classes
125 Here is a table of syntax classes, the characters that stand for them,
126 their meanings, and examples of their use.
128 @deffn {Syntax class} @w{whitespace character}
129 @dfn{Whitespace characters} (designated by @w{@samp{@ }} or @samp{-})
130 separate symbols and words from each other. Typically, whitespace
131 characters have no other syntactic significance, and multiple whitespace
132 characters are syntactically equivalent to a single one. Space, tab,
133 newline and formfeed are classified as whitespace in almost all major
137 @deffn {Syntax class} @w{word constituent}
138 @dfn{Word constituents} (designated by @samp{w}) are parts of words in
139 human languages, and are typically used in variable and command names
140 in programs. All upper- and lower-case letters, and the digits, are
141 typically word constituents.
144 @deffn {Syntax class} @w{symbol constituent}
145 @dfn{Symbol constituents} (designated by @samp{_}) are the extra
146 characters that are used in variable and command names along with word
147 constituents. For example, the symbol constituents class is used in
148 Lisp mode to indicate that certain characters may be part of symbol
149 names even though they are not part of English words. These characters
150 are @samp{$&*+-_<>}. In standard C, the only non-word-constituent
151 character that is valid in symbols is underscore (@samp{_}).
154 @deffn {Syntax class} @w{punctuation character}
155 @dfn{Punctuation characters} (designated by @samp{.}) are those
156 characters that are used as punctuation in English, or are used in some
157 way in a programming language to separate symbols from one another.
158 Some programming language modes, such as Emacs Lisp mode, have no
159 characters in this class since the few characters that are not symbol or
160 word constituents all have other uses. Other programming language modes,
161 such as C mode, use punctuation syntax for operators.
164 @deffn {Syntax class} @w{open parenthesis character}
165 @deffnx {Syntax class} @w{close parenthesis character}
166 @cindex parenthesis syntax
167 Open and close @dfn{parenthesis characters} are characters used in
168 dissimilar pairs to surround sentences or expressions. Such a grouping
169 is begun with an open parenthesis character and terminated with a close.
170 Each open parenthesis character matches a particular close parenthesis
171 character, and vice versa. Normally, Emacs indicates momentarily the
172 matching open parenthesis when you insert a close parenthesis.
175 The class of open parentheses is designated by @samp{(}, and that of
176 close parentheses by @samp{)}.
178 In English text, and in C code, the parenthesis pairs are @samp{()},
179 @samp{[]}, and @samp{@{@}}. In Emacs Lisp, the delimiters for lists and
180 vectors (@samp{()} and @samp{[]}) are classified as parenthesis
184 @deffn {Syntax class} @w{string quote}
185 @dfn{String quote characters} (designated by @samp{"}) are used in
186 many languages, including Lisp and C, to delimit string constants. The
187 same string quote character appears at the beginning and the end of a
188 string. Such quoted strings do not nest.
190 The parsing facilities of Emacs consider a string as a single token.
191 The usual syntactic meanings of the characters in the string are
194 The Lisp modes have two string quote characters: double-quote (@samp{"})
195 and vertical bar (@samp{|}). @samp{|} is not used in Emacs Lisp, but it
196 is used in Common Lisp. C also has two string quote characters:
197 double-quote for strings, and single-quote (@samp{'}) for character
200 English text has no string quote characters because English is not a
201 programming language. Although quotation marks are used in English,
202 we do not want them to turn off the usual syntactic properties of
203 other characters in the quotation.
206 @deffn {Syntax class} @w{escape-syntax character}
207 An @dfn{escape character} (designated by @samp{\}) starts an escape
208 sequence such as is used in C string and character constants. The
209 character @samp{\} belongs to this class in both C and Lisp. (In C, it
210 is used thus only inside strings, but it turns out to cause no trouble
211 to treat it this way throughout C code.)
213 Characters in this class count as part of words if
214 @code{words-include-escapes} is non-@code{nil}. @xref{Word Motion}.
217 @deffn {Syntax class} @w{character quote}
218 A @dfn{character quote character} (designated by @samp{/}) quotes the
219 following character so that it loses its normal syntactic meaning. This
220 differs from an escape character in that only the character immediately
221 following is ever affected.
223 Characters in this class count as part of words if
224 @code{words-include-escapes} is non-@code{nil}. @xref{Word Motion}.
226 This class is used for backslash in @TeX{} mode.
229 @deffn {Syntax class} @w{paired delimiter}
230 @dfn{Paired delimiter characters} (designated by @samp{$}) are like
231 string quote characters except that the syntactic properties of the
232 characters between the delimiters are not suppressed. Only @TeX{} mode
233 uses a paired delimiter presently---the @samp{$} that both enters and
237 @deffn {Syntax class} @w{expression prefix}
238 An @dfn{expression prefix operator} (designated by @samp{'}) is used for
239 syntactic operators that are considered as part of an expression if they
240 appear next to one. In Lisp modes, these characters include the
241 apostrophe, @samp{'} (used for quoting), the comma, @samp{,} (used in
242 macros), and @samp{#} (used in the read syntax for certain data types).
245 @deffn {Syntax class} @w{comment starter}
246 @deffnx {Syntax class} @w{comment ender}
247 @cindex comment syntax
248 The @dfn{comment starter} and @dfn{comment ender} characters are used in
249 various languages to delimit comments. These classes are designated
250 by @samp{<} and @samp{>}, respectively.
252 English text has no comment characters. In Lisp, the semicolon
253 (@samp{;}) starts a comment and a newline or formfeed ends one.
256 @deffn {Syntax class} @w{inherit standard syntax}
257 This syntax class does not specify a particular syntax. It says to look
258 in the standard syntax table to find the syntax of this character. The
259 designator for this syntax class is @samp{@@}.
262 @deffn {Syntax class} @w{generic comment delimiter}
263 A @dfn{generic comment delimiter} (designated by @samp{!}) starts
264 or ends a special kind of comment. @emph{Any} generic comment delimiter
265 matches @emph{any} generic comment delimiter, but they cannot match
266 a comment starter or comment ender; generic comment delimiters can only
269 This syntax class is primarily meant for use with the
270 @code{syntax-table} text property (@pxref{Syntax Properties}). You can
271 mark any range of characters as forming a comment, by giving the first
272 and last characters of the range @code{syntax-table} properties
273 identifying them as generic comment delimiters.
276 @deffn {Syntax class} @w{generic string delimiter}
277 A @dfn{generic string delimiter} (designated by @samp{|}) starts or ends
278 a string. This class differs from the string quote class in that @emph{any}
279 generic string delimiter can match any other generic string delimiter; but
280 they do not match ordinary string quote characters.
282 This syntax class is primarily meant for use with the
283 @code{syntax-table} text property (@pxref{Syntax Properties}). You can
284 mark any range of characters as forming a string constant, by giving the
285 first and last characters of the range @code{syntax-table} properties
286 identifying them as generic string delimiters.
290 @subsection Syntax Flags
293 In addition to the classes, entries for characters in a syntax table
294 can specify flags. There are seven possible flags, represented by the
295 characters @samp{1}, @samp{2}, @samp{3}, @samp{4}, @samp{b}, @samp{n},
298 All the flags except @samp{n} and @samp{p} are used to describe
299 multi-character comment delimiters. The digit flags indicate that a
300 character can @emph{also} be part of a comment sequence, in addition to
301 the syntactic properties associated with its character class. The flags
302 are independent of the class and each other for the sake of characters
303 such as @samp{*} in C mode, which is a punctuation character, @emph{and}
304 the second character of a start-of-comment sequence (@samp{/*}),
305 @emph{and} the first character of an end-of-comment sequence
308 Here is a table of the possible flags for a character @var{c},
313 @samp{1} means @var{c} is the start of a two-character comment-start
317 @samp{2} means @var{c} is the second character of such a sequence.
320 @samp{3} means @var{c} is the start of a two-character comment-end
324 @samp{4} means @var{c} is the second character of such a sequence.
328 @samp{b} means that @var{c} as a comment delimiter belongs to the
329 alternative ``b'' comment style.
331 Emacs supports two comment styles simultaneously in any one syntax
332 table. This is for the sake of C++. Each style of comment syntax has
333 its own comment-start sequence and its own comment-end sequence. Each
334 comment must stick to one style or the other; thus, if it starts with
335 the comment-start sequence of style ``b,'' it must also end with the
336 comment-end sequence of style ``b.''
338 The two comment-start sequences must begin with the same character; only
339 the second character may differ. Mark the second character of the
340 ``b''-style comment-start sequence with the @samp{b} flag.
342 A comment-end sequence (one or two characters) applies to the ``b''
343 style if its first character has the @samp{b} flag set; otherwise, it
344 applies to the ``a'' style.
346 The appropriate comment syntax settings for C++ are as follows:
357 This defines four comment-delimiting sequences:
361 This is a comment-start sequence for ``a'' style because the
362 second character, @samp{*}, does not have the @samp{b} flag.
365 This is a comment-start sequence for ``b'' style because the second
366 character, @samp{/}, does have the @samp{b} flag.
369 This is a comment-end sequence for ``a'' style because the first
370 character, @samp{*}, does not have the @samp{b} flag.
373 This is a comment-end sequence for ``b'' style, because the newline
374 character has the @samp{b} flag.
378 @samp{n} on a comment delimiter character specifies
379 that this kind of comment can be nested. For a two-character
380 comment delimiter, @samp{n} on either character makes it
385 @samp{p} identifies an additional ``prefix character'' for Lisp syntax.
386 These characters are treated as whitespace when they appear between
387 expressions. When they appear within an expression, they are handled
388 according to their usual syntax classes.
390 The function @code{backward-prefix-chars} moves back over these
391 characters, as well as over characters whose primary syntax class is
392 prefix (@samp{'}). @xref{Motion and Syntax}.
395 @node Syntax Table Functions
396 @section Syntax Table Functions
398 In this section we describe functions for creating, accessing and
399 altering syntax tables.
401 @defun make-syntax-table &optional table
402 This function creates a new syntax table, with all values initialized
403 to @code{nil}. If @var{table} is non-@code{nil}, it becomes the
404 parent of the new syntax table, otherwise the standard syntax table is
405 the parent. Like all char-tables, a syntax table inherits from its
406 parent. Thus the original syntax of all characters in the returned
407 syntax table is determined by the parent. @xref{Char-Tables}.
409 Most major mode syntax tables are created in this way.
412 @defun copy-syntax-table &optional table
413 This function constructs a copy of @var{table} and returns it. If
414 @var{table} is not supplied (or is @code{nil}), it returns a copy of the
415 standard syntax table. Otherwise, an error is signaled if @var{table} is
419 @deffn Command modify-syntax-entry char syntax-descriptor &optional table
420 This function sets the syntax entry for @var{char} according to
421 @var{syntax-descriptor}. The syntax is changed only for @var{table},
422 which defaults to the current buffer's syntax table, and not in any
423 other syntax table. The argument @var{syntax-descriptor} specifies the
424 desired syntax; this is a string beginning with a class designator
425 character, and optionally containing a matching character and flags as
426 well. @xref{Syntax Descriptors}.
428 This function always returns @code{nil}. The old syntax information in
429 the table for this character is discarded.
431 An error is signaled if the first character of the syntax descriptor is not
432 one of the seventeen syntax class designator characters. An error is also
433 signaled if @var{char} is not a character.
437 @exdent @r{Examples:}
439 ;; @r{Put the space character in class whitespace.}
440 (modify-syntax-entry ?\s " ")
445 ;; @r{Make @samp{$} an open parenthesis character,}
446 ;; @r{with @samp{^} as its matching close.}
447 (modify-syntax-entry ?$ "(^")
452 ;; @r{Make @samp{^} a close parenthesis character,}
453 ;; @r{with @samp{$} as its matching open.}
454 (modify-syntax-entry ?^ ")$")
459 ;; @r{Make @samp{/} a punctuation character,}
460 ;; @r{the first character of a start-comment sequence,}
461 ;; @r{and the second character of an end-comment sequence.}
462 ;; @r{This is used in C mode.}
463 (modify-syntax-entry ?/ ". 14")
469 @defun char-syntax character
470 This function returns the syntax class of @var{character}, represented
471 by its mnemonic designator character. This returns @emph{only} the
472 class, not any matching parenthesis or flags.
474 An error is signaled if @var{char} is not a character.
476 The following examples apply to C mode. The first example shows that
477 the syntax class of space is whitespace (represented by a space). The
478 second example shows that the syntax of @samp{/} is punctuation. This
479 does not show the fact that it is also part of comment-start and -end
480 sequences. The third example shows that open parenthesis is in the class
481 of open parentheses. This does not show the fact that it has a matching
486 (string (char-syntax ?\s))
491 (string (char-syntax ?/))
496 (string (char-syntax ?\())
501 We use @code{string} to make it easier to see the character returned by
505 @defun set-syntax-table table
506 This function makes @var{table} the syntax table for the current buffer.
507 It returns @var{table}.
511 This function returns the current syntax table, which is the table for
515 @defmac with-syntax-table @var{table} @var{body}@dots{}
516 This macro executes @var{body} using @var{table} as the current syntax
517 table. It returns the value of the last form in @var{body}, after
518 restoring the old current syntax table.
520 Since each buffer has its own current syntax table, we should make that
521 more precise: @code{with-syntax-table} temporarily alters the current
522 syntax table of whichever buffer is current at the time the macro
523 execution starts. Other buffers are not affected.
526 @node Syntax Properties
527 @section Syntax Properties
528 @kindex syntax-table @r{(text property)}
530 When the syntax table is not flexible enough to specify the syntax of
531 a language, you can use @code{syntax-table} text properties to
532 override the syntax table for specific character occurrences in the
533 buffer. @xref{Text Properties}. You can use Font Lock mode to set
534 @code{syntax-table} text properties. @xref{Setting Syntax
537 The valid values of @code{syntax-table} text property are:
540 @item @var{syntax-table}
541 If the property value is a syntax table, that table is used instead of
542 the current buffer's syntax table to determine the syntax for this
543 occurrence of the character.
545 @item @code{(@var{syntax-code} . @var{matching-char})}
546 A cons cell of this format specifies the syntax for this
547 occurrence of the character. (@pxref{Syntax Table Internals})
550 If the property is @code{nil}, the character's syntax is determined from
551 the current syntax table in the usual way.
554 @defvar parse-sexp-lookup-properties
555 If this is non-@code{nil}, the syntax scanning functions pay attention
556 to syntax text properties. Otherwise they use only the current syntax
560 @node Motion and Syntax
561 @section Motion and Syntax
563 This section describes functions for moving across characters that
564 have certain syntax classes.
566 @defun skip-syntax-forward syntaxes &optional limit
567 This function moves point forward across characters having syntax
568 classes mentioned in @var{syntaxes} (a string of syntax class
569 characters). It stops when it encounters the end of the buffer, or
570 position @var{limit} (if specified), or a character it is not supposed
573 If @var{syntaxes} starts with @samp{^}, then the function skips
574 characters whose syntax is @emph{not} in @var{syntaxes}.
576 The return value is the distance traveled, which is a nonnegative
580 @defun skip-syntax-backward syntaxes &optional limit
581 This function moves point backward across characters whose syntax
582 classes are mentioned in @var{syntaxes}. It stops when it encounters
583 the beginning of the buffer, or position @var{limit} (if specified), or
584 a character it is not supposed to skip.
586 If @var{syntaxes} starts with @samp{^}, then the function skips
587 characters whose syntax is @emph{not} in @var{syntaxes}.
589 The return value indicates the distance traveled. It is an integer that
593 @defun backward-prefix-chars
594 This function moves point backward over any number of characters with
595 expression prefix syntax. This includes both characters in the
596 expression prefix syntax class, and characters with the @samp{p} flag.
599 @node Parsing Expressions
600 @section Parsing Expressions
602 This section describes functions for parsing and scanning balanced
603 expressions, also known as @dfn{sexps}. Basically, a sexp is either a
604 balanced parenthetical grouping, a string, or a symbol name (a
605 sequence of characters whose syntax is either word constituent or
606 symbol constituent). However, characters whose syntax is expression
607 prefix are treated as part of the sexp if they appear next to it.
609 The syntax table controls the interpretation of characters, so these
610 functions can be used for Lisp expressions when in Lisp mode and for C
611 expressions when in C mode. @xref{List Motion}, for convenient
612 higher-level functions for moving over balanced expressions.
614 A character's syntax controls how it changes the state of the
615 parser, rather than describing the state itself. For example, a
616 string delimiter character toggles the parser state between
617 ``in-string'' and ``in-code,'' but the syntax of characters does not
618 directly say whether they are inside a string. For example (note that
619 15 is the syntax code for generic string delimiters),
622 (put-text-property 1 9 'syntax-table '(15 . nil))
626 does not tell Emacs that the first eight chars of the current buffer
627 are a string, but rather that they are all string delimiters. As a
628 result, Emacs treats them as four consecutive empty string constants.
631 * Motion via Parsing:: Motion functions that work by parsing.
632 * Position Parse:: Determining the syntactic state of a position.
633 * Parser State:: How Emacs represents a syntactic state.
634 * Low-Level Parsing:: Parsing across a specified region.
635 * Control Parsing:: Parameters that affect parsing.
638 @node Motion via Parsing
639 @subsection Motion Commands Based on Parsing
641 This section describes simple point-motion functions that operate
642 based on parsing expressions.
644 @defun scan-lists from count depth
645 This function scans forward @var{count} balanced parenthetical groupings
646 from position @var{from}. It returns the position where the scan stops.
647 If @var{count} is negative, the scan moves backwards.
649 If @var{depth} is nonzero, parenthesis depth counting begins from that
650 value. The only candidates for stopping are places where the depth in
651 parentheses becomes zero; @code{scan-lists} counts @var{count} such
652 places and then stops. Thus, a positive value for @var{depth} means go
653 out @var{depth} levels of parenthesis.
655 Scanning ignores comments if @code{parse-sexp-ignore-comments} is
658 If the scan reaches the beginning or end of the buffer (or its
659 accessible portion), and the depth is not zero, an error is signaled.
660 If the depth is zero but the count is not used up, @code{nil} is
664 @defun scan-sexps from count
665 This function scans forward @var{count} sexps from position @var{from}.
666 It returns the position where the scan stops. If @var{count} is
667 negative, the scan moves backwards.
669 Scanning ignores comments if @code{parse-sexp-ignore-comments} is
672 If the scan reaches the beginning or end of (the accessible part of) the
673 buffer while in the middle of a parenthetical grouping, an error is
674 signaled. If it reaches the beginning or end between groupings but
675 before count is used up, @code{nil} is returned.
678 @defun forward-comment count
679 This function moves point forward across @var{count} complete comments
680 (that is, including the starting delimiter and the terminating
681 delimiter if any), plus any whitespace encountered on the way. It
682 moves backward if @var{count} is negative. If it encounters anything
683 other than a comment or whitespace, it stops, leaving point at the
684 place where it stopped. This includes (for instance) finding the end
685 of a comment when moving forward and expecting the beginning of one.
686 The function also stops immediately after moving over the specified
687 number of complete comments. If @var{count} comments are found as
688 expected, with nothing except whitespace between them, it returns
689 @code{t}; otherwise it returns @code{nil}.
691 This function cannot tell whether the ``comments'' it traverses are
692 embedded within a string. If they look like comments, it treats them
696 To move forward over all comments and whitespace following point, use
697 @code{(forward-comment (buffer-size))}. @code{(buffer-size)} is a good
698 argument to use, because the number of comments in the buffer cannot
702 @subsection Finding the Parse State for a Position
704 For syntactic analysis, such as in indentation, often the useful
705 thing is to compute the syntactic state corresponding to a given buffer
706 position. This function does that conveniently.
708 @defun syntax-ppss &optional pos
709 This function returns the parser state (see next section) that the
710 parser would reach at position @var{pos} starting from the beginning
711 of the buffer. This is equivalent to @code{(parse-partial-sexp
712 (point-min) @var{pos})}, except that @code{syntax-ppss} uses a cache
713 to speed up the computation. Due to this optimization, the 2nd value
714 (previous complete subexpression) and 6th value (minimum parenthesis
715 depth) of the returned parser state are not meaningful.
718 @code{syntax-ppss} automatically hooks itself to
719 @code{before-change-functions} to keep its cache consistent. But
720 updating can fail if @code{syntax-ppss} is called while
721 @code{before-change-functions} is temporarily let-bound, or if the
722 buffer is modified without obeying the hook, such as when using
723 @code{inhibit-modification-hooks}. For this reason, it is sometimes
724 necessary to flush the cache manually.
726 @defun syntax-ppss-flush-cache beg
727 This function flushes the cache used by @code{syntax-ppss}, starting at
731 Major modes can make @code{syntax-ppss} run faster by specifying
732 where it needs to start parsing.
734 @defvar syntax-begin-function
735 If this is non-@code{nil}, it should be a function that moves to an
736 earlier buffer position where the parser state is equivalent to
737 @code{nil}---in other words, a position outside of any comment,
738 string, or parenthesis. @code{syntax-ppss} uses it to further
739 optimize its computations, when the cache gives no help.
743 @subsection Parser State
746 A @dfn{parser state} is a list of ten elements describing the final
747 state of parsing text syntactically as part of an expression. The
748 parsing functions in the following sections return a parser state as
749 the value, and in some cases accept one as an argument also, so that
750 you can resume parsing after it stops. Here are the meanings of the
751 elements of the parser state:
755 The depth in parentheses, counting from 0. @strong{Warning:} this can
756 be negative if there are more close parens than open parens between
757 the start of the defun and point.
760 @cindex innermost containing parentheses
761 The character position of the start of the innermost parenthetical
762 grouping containing the stopping point; @code{nil} if none.
765 @cindex previous complete subexpression
766 The character position of the start of the last complete subexpression
767 terminated; @code{nil} if none.
770 @cindex inside string
771 Non-@code{nil} if inside a string. More precisely, this is the
772 character that will terminate the string, or @code{t} if a generic
773 string delimiter character should terminate it.
776 @cindex inside comment
777 @code{t} if inside a comment (of either style),
778 or the comment nesting level if inside a kind of comment
782 @cindex quote character
783 @code{t} if point is just after a quote character.
786 The minimum parenthesis depth encountered during this scan.
789 What kind of comment is active: @code{nil} for a comment of style
790 ``a'' or when not inside a comment, @code{t} for a comment of style
791 ``b,'' and @code{syntax-table} for a comment that should be ended by a
792 generic comment delimiter character.
795 The string or comment start position. While inside a comment, this is
796 the position where the comment began; while inside a string, this is the
797 position where the string began. When outside of strings and comments,
798 this element is @code{nil}.
801 Internal data for continuing the parsing. The meaning of this
802 data is subject to change; it is used if you pass this list
803 as the @var{state} argument to another call.
806 Elements 1, 2, and 6 are ignored in a state which you pass as an
807 argument to continue parsing, and elements 8 and 9 are used only in
808 trivial cases. Those elements serve primarily to convey information
809 to the Lisp program which does the parsing.
811 One additional piece of useful information is available from a
812 parser state using this function:
814 @defun syntax-ppss-toplevel-pos state
815 This function extracts, from parser state @var{state}, the last
816 position scanned in the parse which was at top level in grammatical
817 structure. ``At top level'' means outside of any parentheses,
818 comments, or strings.
820 The value is @code{nil} if @var{state} represents a parse which has
821 arrived at a top level position.
824 We have provided this access function rather than document how the
825 data is represented in the state, because we plan to change the
826 representation in the future.
828 @node Low-Level Parsing
829 @subsection Low-Level Parsing
831 The most basic way to use the expression parser is to tell it
832 to start at a given position with a certain state, and parse up to
833 a specified end position.
835 @defun parse-partial-sexp start limit &optional target-depth stop-before state stop-comment
836 This function parses a sexp in the current buffer starting at
837 @var{start}, not scanning past @var{limit}. It stops at position
838 @var{limit} or when certain criteria described below are met, and sets
839 point to the location where parsing stops. It returns a parser state
840 describing the status of the parse at the point where it stops.
842 @cindex parenthesis depth
843 If the third argument @var{target-depth} is non-@code{nil}, parsing
844 stops if the depth in parentheses becomes equal to @var{target-depth}.
845 The depth starts at 0, or at whatever is given in @var{state}.
847 If the fourth argument @var{stop-before} is non-@code{nil}, parsing
848 stops when it comes to any character that starts a sexp. If
849 @var{stop-comment} is non-@code{nil}, parsing stops when it comes to the
850 start of a comment. If @var{stop-comment} is the symbol
851 @code{syntax-table}, parsing stops after the start of a comment or a
852 string, or the end of a comment or a string, whichever comes first.
854 If @var{state} is @code{nil}, @var{start} is assumed to be at the top
855 level of parenthesis structure, such as the beginning of a function
856 definition. Alternatively, you might wish to resume parsing in the
857 middle of the structure. To do this, you must provide a @var{state}
858 argument that describes the initial status of parsing. The value
859 returned by a previous call to @code{parse-partial-sexp} will do
863 @node Control Parsing
864 @subsection Parameters to Control Parsing
866 @defvar multibyte-syntax-as-symbol
867 If this variable is non-@code{nil}, @code{scan-sexps} treats all
868 non-@acronym{ASCII} characters as symbol constituents regardless
869 of what the syntax table says about them. (However, text properties
870 can still override the syntax.)
873 @defopt parse-sexp-ignore-comments
874 @cindex skipping comments
875 If the value is non-@code{nil}, then comments are treated as
876 whitespace by the functions in this section and by @code{forward-sexp},
877 @code{scan-lists} and @code{scan-sexps}.
880 @vindex parse-sexp-lookup-properties
881 The behavior of @code{parse-partial-sexp} is also affected by
882 @code{parse-sexp-lookup-properties} (@pxref{Syntax Properties}).
884 You can use @code{forward-comment} to move forward or backward over
885 one comment or several comments.
887 @node Standard Syntax Tables
888 @section Some Standard Syntax Tables
890 Most of the major modes in Emacs have their own syntax tables. Here
893 @defun standard-syntax-table
894 This function returns the standard syntax table, which is the syntax
895 table used in Fundamental mode.
898 @defvar text-mode-syntax-table
899 The value of this variable is the syntax table used in Text mode.
902 @defvar c-mode-syntax-table
903 The value of this variable is the syntax table for C-mode buffers.
906 @defvar emacs-lisp-mode-syntax-table
907 The value of this variable is the syntax table used in Emacs Lisp mode
908 by editing commands. (It has no effect on the Lisp @code{read}
912 @node Syntax Table Internals
913 @section Syntax Table Internals
914 @cindex syntax table internals
916 Lisp programs don't usually work with the elements directly; the
917 Lisp-level syntax table functions usually work with syntax descriptors
918 (@pxref{Syntax Descriptors}). Nonetheless, here we document the
919 internal format. This format is used mostly when manipulating
922 Each element of a syntax table is a cons cell of the form
923 @code{(@var{syntax-code} . @var{matching-char})}. The @sc{car},
924 @var{syntax-code}, is an integer that encodes the syntax class, and any
925 flags. The @sc{cdr}, @var{matching-char}, is non-@code{nil} if
926 a character to match was specified.
928 This table gives the value of @var{syntax-code} which corresponds
929 to each syntactic type.
931 @multitable @columnfractions .05 .3 .3 .31
934 @i{Integer} @i{Class}
936 @i{Integer} @i{Class}
938 @i{Integer} @i{Class}
943 5 @ @ close parenthesis
945 10 @ @ character quote
950 6 @ @ expression prefix
964 8 @ @ paired delimiter
969 4 @ @ open parenthesis
973 14 @ @ generic comment
979 For example, the usual syntax value for @samp{(} is @code{(4 . 41)}.
980 (41 is the character code for @samp{)}.)
982 The flags are encoded in higher order bits, starting 16 bits from the
983 least significant bit. This table gives the power of two which
984 corresponds to each syntax flag.
986 @multitable @columnfractions .05 .3 .3 .3
996 @samp{1} @ @ @code{(lsh 1 16)}
998 @samp{4} @ @ @code{(lsh 1 19)}
1000 @samp{b} @ @ @code{(lsh 1 21)}
1003 @samp{2} @ @ @code{(lsh 1 17)}
1005 @samp{p} @ @ @code{(lsh 1 20)}
1007 @samp{n} @ @ @code{(lsh 1 22)}
1010 @samp{3} @ @ @code{(lsh 1 18)}
1013 @defun string-to-syntax @var{desc}
1014 This function returns the internal form @code{(@var{syntax-code} .
1015 @var{matching-char})} corresponding to the syntax descriptor @var{desc}.
1018 @defun syntax-after pos
1019 This function returns the syntax code of the character in the buffer
1020 after position @var{pos}, taking account of syntax properties as well
1021 as the syntax table. If @var{pos} is outside the buffer's accessible
1022 portion (@pxref{Narrowing, accessible portion}), this function returns
1026 @defun syntax-class syntax
1027 This function returns the syntax class of the syntax code
1028 @var{syntax}. (It masks off the high 16 bits that hold the flags
1029 encoded in the syntax descriptor.) If @var{syntax} is @code{nil}, it
1030 returns @code{nil}; this is so evaluating the expression
1033 (syntax-class (syntax-after pos))
1037 where @code{pos} is outside the buffer's accessible portion, will
1038 yield @code{nil} without throwing errors or producing wrong syntax
1044 @cindex categories of characters
1045 @cindex character categories
1047 @dfn{Categories} provide an alternate way of classifying characters
1048 syntactically. You can define several categories as needed, then
1049 independently assign each character to one or more categories. Unlike
1050 syntax classes, categories are not mutually exclusive; it is normal for
1051 one character to belong to several categories.
1053 @cindex category table
1054 Each buffer has a @dfn{category table} which records which categories
1055 are defined and also which characters belong to each category. Each
1056 category table defines its own categories, but normally these are
1057 initialized by copying from the standard categories table, so that the
1058 standard categories are available in all modes.
1060 Each category has a name, which is an @acronym{ASCII} printing character in
1061 the range @w{@samp{ }} to @samp{~}. You specify the name of a category
1062 when you define it with @code{define-category}.
1064 The category table is actually a char-table (@pxref{Char-Tables}).
1065 The element of the category table at index @var{c} is a @dfn{category
1066 set}---a bool-vector---that indicates which categories character @var{c}
1067 belongs to. In this category set, if the element at index @var{cat} is
1068 @code{t}, that means category @var{cat} is a member of the set, and that
1069 character @var{c} belongs to category @var{cat}.
1071 For the next three functions, the optional argument @var{table}
1072 defaults to the current buffer's category table.
1074 @defun define-category char docstring &optional table
1075 This function defines a new category, with name @var{char} and
1076 documentation @var{docstring}, for the category table @var{table}.
1079 @defun category-docstring category &optional table
1080 This function returns the documentation string of category @var{category}
1081 in category table @var{table}.
1084 (category-docstring ?a)
1086 (category-docstring ?l)
1091 @defun get-unused-category &optional table
1092 This function returns a category name (a character) which is not
1093 currently defined in @var{table}. If all possible categories are in use
1094 in @var{table}, it returns @code{nil}.
1097 @defun category-table
1098 This function returns the current buffer's category table.
1101 @defun category-table-p object
1102 This function returns @code{t} if @var{object} is a category table,
1103 otherwise @code{nil}.
1106 @defun standard-category-table
1107 This function returns the standard category table.
1110 @defun copy-category-table &optional table
1111 This function constructs a copy of @var{table} and returns it. If
1112 @var{table} is not supplied (or is @code{nil}), it returns a copy of the
1113 standard category table. Otherwise, an error is signaled if @var{table}
1114 is not a category table.
1117 @defun set-category-table table
1118 This function makes @var{table} the category table for the current
1119 buffer. It returns @var{table}.
1122 @defun make-category-table
1123 This creates and returns an empty category table. In an empty category
1124 table, no categories have been allocated, and no characters belong to
1128 @defun make-category-set categories
1129 This function returns a new category set---a bool-vector---whose initial
1130 contents are the categories listed in the string @var{categories}. The
1131 elements of @var{categories} should be category names; the new category
1132 set has @code{t} for each of those categories, and @code{nil} for all
1136 (make-category-set "al")
1137 @result{} #&128"\0\0\0\0\0\0\0\0\0\0\0\0\2\20\0\0"
1141 @defun char-category-set char
1142 This function returns the category set for character @var{char} in the
1143 current buffer's category table. This is the bool-vector which
1144 records which categories the character @var{char} belongs to. The
1145 function @code{char-category-set} does not allocate storage, because
1146 it returns the same bool-vector that exists in the category table.
1149 (char-category-set ?a)
1150 @result{} #&128"\0\0\0\0\0\0\0\0\0\0\0\0\2\20\0\0"
1154 @defun category-set-mnemonics category-set
1155 This function converts the category set @var{category-set} into a string
1156 containing the characters that designate the categories that are members
1160 (category-set-mnemonics (char-category-set ?a))
1165 @defun modify-category-entry character category &optional table reset
1166 This function modifies the category set of @var{character} in category
1167 table @var{table} (which defaults to the current buffer's category
1170 Normally, it modifies the category set by adding @var{category} to it.
1171 But if @var{reset} is non-@code{nil}, then it deletes @var{category}
1175 @deffn Command describe-categories &optional buffer-or-name
1176 This function describes the category specifications in the current
1177 category table. It inserts the descriptions in a buffer, and then
1178 displays that buffer. If @var{buffer-or-name} is non-@code{nil}, it
1179 describes the category table of that buffer instead.
1183 arch-tag: 4d914e96-0283-445c-9233-75d33662908c