1 \input texinfo @c -*-texinfo-*-
2 @comment %**start of header
3 @setfilename bison.info
4 @documentencoding UTF-8
6 @settitle Bison @value{VERSION}
7 @xrefautomaticsectiontitle on
9 @c cite a reference in text. Could not find a means to have a single
10 @c definition that looks nice in all the output formats.
22 @c cite a reference in parentheses.
25 (@pxref{\ref\,,\ref\})
35 @c ## ---------------------- ##
36 @c ## Diagnostics in color. ##
37 @c ## ---------------------- ##
40 \gdef\rgbGreen{0 .80 0}
44 \gdef\rgbYellow{1 .5 0}
46 \setcolor{\rgbYellow}%
56 \gdef\rgbPurple{0.50 0 0.50}
58 \setcolor{\rgbPurple}%
61 \setcolor{\maincolor}%
64 \gdef\rgbError{0.80 0 0}
68 \gdef\rgbNotice{0 0 0.80}
70 \setcolor{\rgbNotice}%
72 \gdef\rgbWarning{0.50 0 0.50}
74 \setcolor{\rgbWarning}%
77 \setcolor{\maincolor}%
83 @inlineraw{html, <span style="color:green">}
86 @inlineraw{html, <span style="color:#ff8000">}
89 @inlineraw{html, <span style="color:red">}
92 @inlineraw{html, <span style="color:blue">}
95 @inlineraw{html, <span style="color:darkviolet">}
98 @inlineraw{html, </span>}
102 @inlineraw{html, <b style="color:red">}
105 @inlineraw{html, <b style="color:darkcyan">}
108 @inlineraw{html, <b style="color:darkviolet">}
111 @inlineraw{html, </b>}
116 @colorGreen{}\text\@colorOff{}
120 @colorYellow{}\text\@colorOff{}
124 @colorRed{}\text\@colorOff{}
128 @colorBlue{}\text\@colorOff{}
132 @colorPurple{}\text\@colorOff{}
135 @macro dwarning{text}
136 @diagWarning{}\text\@diagOff{}
140 @diagError{}\text\@diagOff{}
144 @diagNotice{}\text\@diagOff{}
149 @c SMALL BOOK version
150 @c This edition has been formatted so that you can format and print it in
151 @c the smallbook format.
153 @c @setchapternewpage odd
155 @c Set following if you want to document %default-prec and %no-default-prec.
156 @c This feature is experimental and may change in future Bison versions.
169 @comment %**end of header
173 This manual (@value{UPDATED}) is for GNU Bison (version @value{VERSION}),
174 the GNU parser generator.
176 Copyright @copyright{} 1988--1993, 1995, 1998--2015, 2018--2021 Free
177 Software Foundation, Inc.
180 Permission is granted to copy, distribute and/or modify this document under
181 the terms of the GNU Free Documentation License, Version 1.3 or any later
182 version published by the Free Software Foundation; with no Invariant
183 Sections, with the Front-Cover texts being ``A GNU Manual,'' and with the
184 Back-Cover Texts as in (a) below. A copy of the license is included in the
185 section entitled ``GNU Free Documentation License.''
187 (a) The FSF's Back-Cover Text is: ``You have the freedom to copy and modify
188 this GNU manual. Buying copies from the FSF supports it in developing GNU
189 and promoting software freedom.''
193 @dircategory Software development
195 * bison: (bison). GNU parser generator (Yacc replacement).
200 @subtitle The Yacc-compatible Parser Generator
201 @subtitle @value{UPDATED}, Bison Version @value{VERSION}
203 @author by Charles Donnelly and Richard Stallman
206 @vskip 0pt plus 1filll
209 Published by the Free Software Foundation @*
210 51 Franklin Street, Fifth Floor @*
211 Boston, MA 02110-1301 USA @*
212 Printed copies are available from the Free Software Foundation.@*
215 Cover art by Etienne Suvasa.
227 * Introduction:: What GNU Bison is.
228 * Conditions:: Conditions for using Bison and its output.
229 * Copying:: The GNU General Public License says
230 how you can copy and share Bison.
233 * Concepts:: Basic concepts for understanding Bison.
234 * Examples:: Three simple explained examples of using Bison.
237 * Grammar File:: Writing Bison declarations and rules.
238 * Interface:: C-language interface to the parser function @code{yyparse}.
239 * Algorithm:: How the Bison parser works at run-time.
240 * Error Recovery:: Writing rules for error recovery.
241 * Context Dependency:: What to do if your language syntax is too
242 messy for Bison to handle straightforwardly.
243 * Debugging:: Understanding or debugging Bison parsers.
244 * Invocation:: How to run Bison (to produce the parser implementation).
245 * Other Languages:: Creating C++, D and Java parsers.
246 * History:: How Bison came to be
247 * Versioning:: Dealing with Bison versioning
248 * FAQ:: Frequently Asked Questions
249 * Table of Symbols:: All the keywords of the Bison language are explained.
250 * Glossary:: Basic concepts are explained.
251 * GNU Free Documentation License:: Copying and sharing this manual
252 * Bibliography:: Publications cited in this manual.
253 * Index of Terms:: Cross-references to the text.
256 --- The Detailed Node Listing ---
258 The Concepts of Bison
260 * Language and Grammar:: Languages and context-free grammars,
261 as mathematical ideas.
262 * Grammar in Bison:: How we represent grammars for Bison's sake.
263 * Semantic Values:: Each token or syntactic grouping can have
264 a semantic value (the value of an integer,
265 the name of an identifier, etc.).
266 * Semantic Actions:: Each rule can have an action containing C code.
267 * GLR Parsers:: Writing parsers for general context-free languages.
268 * Locations:: Overview of location tracking.
269 * Bison Parser:: What are Bison's input and output,
270 how is the output used?
271 * Stages:: Stages in writing and running Bison grammars.
272 * Grammar Layout:: Overall structure of a Bison grammar file.
276 * Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
277 * Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
278 * GLR Semantic Actions:: Considerations for semantic values and deferred actions.
279 * Semantic Predicates:: Controlling a parse with arbitrary computations.
283 * RPN Calc:: Reverse Polish Notation Calculator;
284 a first example with no operator precedence.
285 * Infix Calc:: Infix (algebraic) notation calculator.
286 Operator precedence is introduced.
287 * Simple Error Recovery:: Continuing after syntax errors.
288 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
289 * Multi-function Calc:: Calculator with memory and trig functions.
290 It uses multiple data-types for semantic values.
291 * Exercises:: Ideas for improving the multi-function calculator.
293 Reverse Polish Notation Calculator
295 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
296 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
297 * Rpcalc Lexer:: The lexical analyzer.
298 * Rpcalc Main:: The controlling function.
299 * Rpcalc Error:: The error reporting function.
300 * Rpcalc Generate:: Running Bison on the grammar file.
301 * Rpcalc Compile:: Run the C compiler on the output code.
303 Grammar Rules for @code{rpcalc}
305 * Rpcalc Input:: Explanation of the @code{input} nonterminal
306 * Rpcalc Line:: Explanation of the @code{line} nonterminal
307 * Rpcalc Exp:: Explanation of the @code{exp} nonterminal
309 Location Tracking Calculator: @code{ltcalc}
311 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
312 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
313 * Ltcalc Lexer:: The lexical analyzer.
315 Multi-Function Calculator: @code{mfcalc}
317 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
318 * Mfcalc Rules:: Grammar rules for the calculator.
319 * Mfcalc Symbol Table:: Symbol table management subroutines.
320 * Mfcalc Lexer:: The lexical analyzer.
321 * Mfcalc Main:: The controlling function.
325 * Grammar Outline:: Overall layout of the grammar file.
326 * Symbols:: Terminal and nonterminal symbols.
327 * Rules:: How to write grammar rules.
328 * Semantics:: Semantic values and actions.
329 * Tracking Locations:: Locations and actions.
330 * Named References:: Using named references in actions.
331 * Declarations:: All kinds of Bison declarations are described here.
332 * Multiple Parsers:: Putting more than one Bison parser in one program.
334 Outline of a Bison Grammar
336 * Prologue:: Syntax and usage of the prologue.
337 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
338 * Bison Declarations:: Syntax and usage of the Bison declarations section.
339 * Grammar Rules:: Syntax and usage of the grammar rules section.
340 * Epilogue:: Syntax and usage of the epilogue.
344 * Rules Syntax:: Syntax of the rules.
345 * Empty Rules:: Symbols that can match the empty string.
346 * Recursion:: Writing recursive rules.
349 Defining Language Semantics
351 * Value Type:: Specifying one data type for all semantic values.
352 * Multiple Types:: Specifying several alternative data types.
353 * Type Generation:: Generating the semantic value type.
354 * Union Decl:: Declaring the set of all semantic value types.
355 * Structured Value Type:: Providing a structured semantic value type.
356 * Actions:: An action is the semantic definition of a grammar rule.
357 * Action Types:: Specifying data types for actions to operate on.
358 * Midrule Actions:: Most actions go at the end of a rule.
359 This says when, why and how to use the exceptional
360 action in the middle of a rule.
364 * Using Midrule Actions:: Putting an action in the middle of a rule.
365 * Typed Midrule Actions:: Specifying the semantic type of their values.
366 * Midrule Action Translation:: How midrule actions are actually processed.
367 * Midrule Conflicts:: Midrule actions can cause conflicts.
371 * Location Type:: Specifying a data type for locations.
372 * Actions and Locations:: Using locations in actions.
373 * Location Default Action:: Defining a general way to compute locations.
377 * Require Decl:: Requiring a Bison version.
378 * Token Decl:: Declaring terminal symbols.
379 * Precedence Decl:: Declaring terminals with precedence and associativity.
380 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
381 * Symbol Decls:: Summary of the Syntax of Symbol Declarations.
382 * Initial Action Decl:: Code run before parsing starts.
383 * Destructor Decl:: Declaring how symbols are freed.
384 * Printer Decl:: Declaring how symbol values are displayed.
385 * Expect Decl:: Suppressing warnings about parsing conflicts.
386 * Start Decl:: Specifying the start symbol.
387 * Pure Decl:: Requesting a reentrant parser.
388 * Push Decl:: Requesting a push parser.
389 * Decl Summary:: Table of all Bison declarations.
390 * %define Summary:: Defining variables to adjust Bison's behavior.
391 * %code Summary:: Inserting code into the parser source.
393 Parser C-Language Interface
395 * Parser Function:: How to call @code{yyparse} and what it returns.
396 * Push Parser Interface:: How to create, use, and destroy push parsers.
397 * Lexical:: You must supply a function @code{yylex}
399 * Error Reporting:: Passing error messages to the user.
400 * Action Features:: Special features for use in actions.
401 * Internationalization:: How to let the parser speak in the user's
404 The Lexical Analyzer Function @code{yylex}
406 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
407 * Special Tokens:: Signaling end-of-file and errors to the parser.
408 * Tokens from Literals:: Finding token kinds from string aliases.
409 * Token Values:: How @code{yylex} must return the semantic value
410 of the token it has read.
411 * Token Locations:: How @code{yylex} must return the text location
412 (line number, etc.) of the token, if the
414 * Pure Calling:: How the calling convention differs in a pure parser
419 * Error Reporting Function:: You must supply a @code{yyerror} function.
420 * Syntax Error Reporting Function:: You can supply a @code{yyreport_syntax_error} function.
422 Parser Internationalization
424 * Enabling I18n:: Preparing your project to support internationalization.
425 * Token I18n:: Preparing tokens for internationalization in error messages.
427 The Bison Parser Algorithm
429 * Lookahead:: Parser looks one token ahead when deciding what to do.
430 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
431 * Precedence:: Operator precedence works by resolving conflicts.
432 * Contextual Precedence:: When an operator's precedence depends on context.
433 * Parser States:: The parser is a finite-state-machine with stack.
434 * Reduce/Reduce:: When two rules are applicable in the same situation.
435 * Mysterious Conflicts:: Conflicts that look unjustified.
436 * Tuning LR:: How to tune fundamental aspects of LR-based parsing.
437 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
438 * Memory Management:: What happens when memory is exhausted. How to avoid it.
442 * Why Precedence:: An example showing why precedence is needed.
443 * Using Precedence:: How to specify precedence and associativity.
444 * Precedence Only:: How to specify precedence only.
445 * Precedence Examples:: How these features are used in the previous example.
446 * How Precedence:: How they work.
447 * Non Operators:: Using precedence for general conflicts.
451 * LR Table Construction:: Choose a different construction algorithm.
452 * Default Reductions:: Disable default reductions.
453 * LAC:: Correct lookahead sets in the parser states.
454 * Unreachable States:: Keep unreachable parser states for debugging.
456 Handling Context Dependencies
458 * Semantic Tokens:: Token parsing can depend on the semantic context.
459 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
460 * Tie-in Recovery:: Lexical tie-ins have implications for how
461 error recovery rules must be written.
463 Debugging Your Parser
465 * Counterexamples:: Understanding conflicts.
466 * Understanding:: Understanding the structure of your parser.
467 * Graphviz:: Getting a visual representation of the parser.
468 * Xml:: Getting a markup representation of the parser.
469 * Tracing:: Tracing the execution of your parser.
473 * Enabling Traces:: Activating run-time trace support
474 * Mfcalc Traces:: Extending @code{mfcalc} to support traces
475 * The YYPRINT Macro:: Obsolete interface for semantic value reports
479 * Bison Options:: All the options described in detail,
480 in alphabetical order by short options.
481 * Option Cross Key:: Alphabetical list of long options.
482 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
486 * Operation Modes:: Options controlling the global behavior of @command{bison}
487 * Diagnostics:: Options controlling the diagnostics
488 * Tuning the Parser:: Options changing the generated parsers
489 * Output Files:: Options controlling the output
491 Parsers Written In Other Languages
493 * C++ Parsers:: The interface to generate C++ parser classes
494 * D Parsers:: The interface to generate D parser classes
495 * Java Parsers:: The interface to generate Java parser classes
499 * A Simple C++ Example:: A short introduction to C++ parsers
500 * C++ Bison Interface:: Asking for C++ parser generation
501 * C++ Parser Interface:: Instantiating and running the parser
502 * C++ Semantic Values:: %union vs. C++
503 * C++ Location Values:: The position and location classes
504 * C++ Parser Context:: You can supply a @code{report_syntax_error} function.
505 * C++ Scanner Interface:: Exchanges between yylex and parse
506 * A Complete C++ Example:: Demonstrating their use
510 * C++ position:: One point in the source file
511 * C++ location:: Two points in the source file
512 * Exposing the Location Classes:: Using the Bison location class in your
514 * User Defined Location Type:: Required interface for locations
516 A Complete C++ Example
518 * Calc++ --- C++ Calculator:: The specifications
519 * Calc++ Parsing Driver:: An active parsing context
520 * Calc++ Parser:: A parser class
521 * Calc++ Scanner:: A pure C++ Flex scanner
522 * Calc++ Top Level:: Conducting the band
526 * D Bison Interface:: Asking for D parser generation
527 * D Semantic Values:: %token and %nterm vs. D
528 * D Location Values:: The position and location classes
529 * D Parser Interface:: Instantiating and running the parser
530 * D Parser Context Interface:: Circumstances of a syntax error
531 * D Scanner Interface:: Specifying the scanner for the parser
532 * D Action Features:: Special features for use in actions
536 * Java Bison Interface:: Asking for Java parser generation
537 * Java Semantic Values:: %token and %nterm vs. Java
538 * Java Location Values:: The position and location classes
539 * Java Parser Interface:: Instantiating and running the parser
540 * Java Parser Context Interface:: Circumstances of a syntax error
541 * Java Scanner Interface:: Specifying the scanner for the parser
542 * Java Action Features:: Special features for use in actions
543 * Java Push Parser Interface:: Instantiating and running the a push parser
544 * Java Differences:: Differences between C/C++ and Java Grammars
545 * Java Declarations Summary:: List of Bison declarations used with Java
547 A Brief History of the Greater Ungulates
549 * Yacc:: The original Yacc
550 * yacchack:: An obscure early implementation of reentrancy
551 * Byacc:: Berkeley Yacc
552 * Bison:: This program
553 * Other Ungulates:: Similar programs
555 Bison Version Compatibility
557 * Versioning:: Dealing with Bison versioning
559 Frequently Asked Questions
561 * Memory Exhausted:: Breaking the Stack Limits
562 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
563 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
564 * Implementing Gotos/Loops:: Control Flow in the Calculator
565 * Multiple start-symbols:: Factoring closely related grammars
566 * Enabling Relocatability:: Moving Bison/using it through network shares
567 * Secure? Conform?:: Is Bison POSIX safe?
568 * I can't build Bison:: Troubleshooting
569 * Where can I find help?:: Troubleshouting
570 * Bug Reports:: Troublereporting
571 * More Languages:: Parsers in C++, Java, and so on
572 * Beta Testing:: Experimenting development versions
573 * Mailing Lists:: Meeting other Bison users
577 * GNU Free Documentation License:: Copying and sharing this manual
583 @unnumbered Introduction
586 @dfn{Bison} is a general-purpose parser generator that converts an annotated
587 context-free grammar into a deterministic LR or generalized LR (GLR) parser
588 employing LALR(1), IELR(1) or canonical LR(1) parser tables. Once you are
589 proficient with Bison, you can use it to develop a wide range of language
590 parsers, from those used in simple desk calculators to complex programming
593 Bison is upward compatible with Yacc: all properly-written Yacc grammars
594 ought to work with Bison with no change. Anyone familiar with Yacc should
595 be able to use Bison with little trouble. You need to be fluent in C, C++,
596 D or Java programming in order to use Bison or to understand this manual.
598 We begin with tutorial chapters that explain the basic concepts of
599 using Bison and show three explained examples, each building on the
600 last. If you don't know Bison or Yacc, start by reading these
601 chapters. Reference chapters follow, which describe specific aspects
604 Bison was written originally by Robert Corbett. Richard Stallman made
605 it Yacc-compatible. Wilfred Hansen of Carnegie Mellon University
606 added multi-character string literals and other features. Since then,
607 Bison has grown more robust and evolved many other new features thanks
608 to the hard work of a long list of volunteers. For details, see the
609 @file{THANKS} and @file{ChangeLog} files included in the Bison
612 This edition corresponds to version @value{VERSION} of Bison.
615 @unnumbered Conditions for Using Bison
617 The distribution terms for Bison-generated parsers permit using the parsers
618 in nonfree programs. Before Bison version 2.2, these extra permissions
619 applied only when Bison was generating LALR(1) parsers in C@. And before
620 Bison version 1.24, Bison-generated parsers could be used only in programs
621 that were free software.
623 The other GNU programming tools, such as the GNU C compiler, have never had
624 such a requirement. They could always be used for nonfree software. The
625 reason Bison was different was not due to a special policy decision; it
626 resulted from applying the usual General Public License to all of the Bison
629 The main output of the Bison utility---the Bison parser implementation
630 file---contains a verbatim copy of a sizable piece of Bison, which is the
631 code for the parser's implementation. (The actions from your grammar are
632 inserted into this implementation at one point, but most of the rest of the
633 implementation is not changed.) When we applied the GPL terms to the
634 skeleton code for the parser's implementation, the effect was to restrict
635 the use of Bison output to free software.
637 We didn't change the terms because of sympathy for people who want to make
638 software proprietary. @strong{Software should be free.} But we concluded
639 that limiting Bison's use to free software was doing little to encourage
640 people to make other software free. So we decided to make the practical
641 conditions for using Bison match the practical conditions for using the
644 This exception applies when Bison is generating code for a parser. You can
645 tell whether the exception applies to a Bison output file by inspecting the
646 file for text beginning with ``As a special exception@dots{}''. The text
647 spells out the exact terms of the exception.
650 @unnumbered GNU GENERAL PUBLIC LICENSE
651 @include gpl-3.0.texi
654 @chapter The Concepts of Bison
656 This chapter introduces many of the basic concepts without which the details
657 of Bison will not make sense. If you do not already know how to use Bison
658 or Yacc, we suggest you start by reading this chapter carefully.
661 * Language and Grammar:: Languages and context-free grammars,
662 as mathematical ideas.
663 * Grammar in Bison:: How we represent grammars for Bison's sake.
664 * Semantic Values:: Each token or syntactic grouping can have
665 a semantic value (the value of an integer,
666 the name of an identifier, etc.).
667 * Semantic Actions:: Each rule can have an action containing C code.
668 * GLR Parsers:: Writing parsers for general context-free languages.
669 * Locations:: Overview of location tracking.
670 * Bison Parser:: What are Bison's input and output,
671 how is the output used?
672 * Stages:: Stages in writing and running Bison grammars.
673 * Grammar Layout:: Overall structure of a Bison grammar file.
676 @node Language and Grammar
677 @section Languages and Context-Free Grammars
679 @cindex context-free grammar
680 @cindex grammar, context-free
681 In order for Bison to parse a language, it must be described by a
682 @dfn{context-free grammar}. This means that you specify one or more
683 @dfn{syntactic groupings} and give rules for constructing them from their
684 parts. For example, in the C language, one kind of grouping is called an
685 `expression'. One rule for making an expression might be, ``An expression
686 can be made of a minus sign and another expression''. Another would be,
687 ``An expression can be an integer''. As you can see, rules are often
688 recursive, but there must be at least one rule which leads out of the
692 @cindex Backus-Naur form
693 The most common formal system for presenting such rules for humans to read
694 is @dfn{Backus-Naur Form} or ``BNF'', which was developed in
695 order to specify the language Algol 60. Any grammar expressed in
696 BNF is a context-free grammar. The input to Bison is
697 essentially machine-readable BNF.
699 @cindex LALR grammars
700 @cindex IELR grammars
702 There are various important subclasses of context-free grammars. Although
703 it can handle almost all context-free grammars, Bison is optimized for what
704 are called LR(1) grammars. In brief, in these grammars, it must be possible
705 to tell how to parse any portion of an input string with just a single token
706 of lookahead. For historical reasons, Bison by default is limited by the
707 additional restrictions of LALR(1), which is hard to explain simply.
708 @xref{Mysterious Conflicts}, for more information on this. You can escape
709 these additional restrictions by requesting IELR(1) or canonical LR(1)
710 parser tables. @xref{LR Table Construction}, to learn how.
713 @cindex generalized LR (GLR) parsing
714 @cindex ambiguous grammars
715 @cindex nondeterministic parsing
717 Parsers for LR(1) grammars are @dfn{deterministic}, meaning
718 roughly that the next grammar rule to apply at any point in the input is
719 uniquely determined by the preceding input and a fixed, finite portion
720 (called a @dfn{lookahead}) of the remaining input. A context-free
721 grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
722 apply the grammar rules to get the same inputs. Even unambiguous
723 grammars can be @dfn{nondeterministic}, meaning that no fixed
724 lookahead always suffices to determine the next grammar rule to apply.
725 With the proper declarations, Bison is also able to parse these more
726 general context-free grammars, using a technique known as GLR
727 parsing (for Generalized LR). Bison's GLR parsers
728 are able to handle any context-free grammar for which the number of
729 possible parses of any given string is finite.
731 @cindex symbols (abstract)
733 @cindex syntactic grouping
734 @cindex grouping, syntactic
735 In the formal grammatical rules for a language, each kind of syntactic unit
736 or grouping is named by a @dfn{symbol}. Those which are built by grouping
737 smaller constructs according to grammatical rules are called
738 @dfn{nonterminal symbols}; those which can't be subdivided are called
739 @dfn{terminal symbols} or @dfn{token kinds}. We call a piece of input
740 corresponding to a single terminal symbol a @dfn{token}, and a piece
741 corresponding to a single nonterminal symbol a @dfn{grouping}.
743 We can use the C language as an example of what symbols, terminal and
744 nonterminal, mean. The tokens of C are identifiers, constants (numeric
745 and string), and the various keywords, arithmetic operators and
746 punctuation marks. So the terminal symbols of a grammar for C include
747 `identifier', `number', `string', plus one symbol for each keyword,
748 operator or punctuation mark: `if', `return', `const', `static', `int',
749 `char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
750 (These tokens can be subdivided into characters, but that is a matter of
751 lexicography, not grammar.)
753 Here is a simple C function subdivided into tokens:
756 int /* @r{keyword `int'} */
757 square (int x) /* @r{identifier, open-paren, keyword `int',}
758 @r{identifier, close-paren} */
759 @{ /* @r{open-brace} */
760 return x * x; /* @r{keyword `return', identifier, asterisk,}
761 @r{identifier, semicolon} */
762 @} /* @r{close-brace} */
765 The syntactic groupings of C include the expression, the statement, the
766 declaration, and the function definition. These are represented in the
767 grammar of C by nonterminal symbols `expression', `statement',
768 `declaration' and `function definition'. The full grammar uses dozens of
769 additional language constructs, each with its own nonterminal symbol, in
770 order to express the meanings of these four. The example above is a
771 function definition; it contains one declaration, and one statement. In
772 the statement, each @samp{x} is an expression and so is @samp{x * x}.
774 Each nonterminal symbol must have grammatical rules showing how it is made
775 out of simpler constructs. For example, one kind of C statement is the
776 @code{return} statement; this would be described with a grammar rule which
777 reads informally as follows:
780 A `statement' can be made of a `return' keyword, an `expression' and a
785 There would be many other rules for `statement', one for each kind of
789 One nonterminal symbol must be distinguished as the special one which
790 defines a complete utterance in the language. It is called the @dfn{start
791 symbol}. In a compiler, this means a complete input program. In the C
792 language, the nonterminal symbol `sequence of definitions and declarations'
795 For example, @samp{1 + 2} is a valid C expression---a valid part of a C
796 program---but it is not valid as an @emph{entire} C program. In the
797 context-free grammar of C, this follows from the fact that `expression' is
798 not the start symbol.
800 The Bison parser reads a sequence of tokens as its input, and groups the
801 tokens using the grammar rules. If the input is valid, the end result is
802 that the entire token sequence reduces to a single grouping whose symbol is
803 the grammar's start symbol. If we use a grammar for C, the entire input
804 must be a `sequence of definitions and declarations'. If not, the parser
805 reports a syntax error.
807 @node Grammar in Bison
808 @section From Formal Rules to Bison Input
809 @cindex Bison grammar
810 @cindex grammar, Bison
811 @cindex formal grammar
813 A formal grammar is a mathematical construct. To define the language
814 for Bison, you must write a file expressing the grammar in Bison syntax:
815 a @dfn{Bison grammar} file. @xref{Grammar File}.
817 A nonterminal symbol in the formal grammar is represented in Bison input
818 as an identifier, like an identifier in C@. By convention, it should be
819 in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
821 The Bison representation for a terminal symbol is also called a @dfn{token
822 kind}. Token kinds as well can be represented as C-like identifiers. By
823 convention, these identifiers should be upper case to distinguish them from
824 nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
825 @code{RETURN}. A terminal symbol that stands for a particular keyword in
826 the language should be named after that keyword converted to upper case.
827 The terminal symbol @code{error} is reserved for error recovery.
830 A terminal symbol can also be represented as a character literal, just like
831 a C character constant. You should do this whenever a token is just a
832 single character (parenthesis, plus-sign, etc.): use that same character in
833 a literal as the terminal symbol for that token.
835 A third way to represent a terminal symbol is with a C string constant
836 containing several characters. @xref{Symbols}, for more information.
838 The grammar rules also have an expression in Bison syntax. For example,
839 here is the Bison rule for a C @code{return} statement. The semicolon in
840 quotes is a literal character token, representing part of the C syntax for
841 the statement; the naked semicolon, and the colon, are Bison punctuation
845 stmt: RETURN expr ';' ;
851 @node Semantic Values
852 @section Semantic Values
853 @cindex semantic value
854 @cindex value, semantic
856 A formal grammar selects tokens only by their classifications: for example,
857 if a rule mentions the terminal symbol `integer constant', it means that
858 @emph{any} integer constant is grammatically valid in that position. The
859 precise value of the constant is irrelevant to how to parse the input: if
860 @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
863 But the precise value is very important for what the input means once it is
864 parsed. A compiler is useless if it fails to distinguish between 4, 1 and
865 3989 as constants in the program! Therefore, each token in a Bison grammar
866 has both a token kind and a @dfn{semantic value}. @xref{Semantics}, for
869 The token kind is a terminal symbol defined in the grammar, such as
870 @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything you
871 need to know to decide where the token may validly appear and how to group
872 it with other tokens. The grammar rules know nothing about tokens except
875 The semantic value has all the rest of the information about the
876 meaning of the token, such as the value of an integer, or the name of an
877 identifier. (A token such as @code{','} which is just punctuation doesn't
878 need to have any semantic value.)
880 For example, an input token might be classified as token kind @code{INTEGER}
881 and have the semantic value 4. Another input token might have the same
882 token kind @code{INTEGER} but value 3989. When a grammar rule says that
883 @code{INTEGER} is allowed, either of these tokens is acceptable because each
884 is an @code{INTEGER}. When the parser accepts the token, it keeps track of
885 the token's semantic value.
887 Each grouping can also have a semantic value as well as its nonterminal
888 symbol. For example, in a calculator, an expression typically has a
889 semantic value that is a number. In a compiler for a programming
890 language, an expression typically has a semantic value that is a tree
891 structure describing the meaning of the expression.
893 @node Semantic Actions
894 @section Semantic Actions
895 @cindex semantic actions
896 @cindex actions, semantic
898 In order to be useful, a program must do more than parse input; it must
899 also produce some output based on the input. In a Bison grammar, a grammar
900 rule can have an @dfn{action} made up of C statements. Each time the
901 parser recognizes a match for that rule, the action is executed.
904 Most of the time, the purpose of an action is to compute the semantic value
905 of the whole construct from the semantic values of its parts. For example,
906 suppose we have a rule which says an expression can be the sum of two
907 expressions. When the parser recognizes such a sum, each of the
908 subexpressions has a semantic value which describes how it was built up.
909 The action for this rule should create a similar sort of value for the
910 newly recognized larger expression.
912 For example, here is a rule that says an expression can be the sum of
916 expr: expr '+' expr @{ $$ = $1 + $3; @} ;
920 The action says how to produce the semantic value of the sum expression
921 from the values of the two subexpressions.
924 @section Writing GLR Parsers
926 @cindex generalized LR (GLR) parsing
929 @cindex shift/reduce conflicts
930 @cindex reduce/reduce conflicts
932 In some grammars, Bison's deterministic
933 LR(1) parsing algorithm cannot decide whether to apply a
934 certain grammar rule at a given point. That is, it may not be able to
935 decide (on the basis of the input read so far) which of two possible
936 reductions (applications of a grammar rule) applies, or whether to apply
937 a reduction or read more of the input and apply a reduction later in the
938 input. These are known respectively as @dfn{reduce/reduce} conflicts
939 (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
940 (@pxref{Shift/Reduce}).
942 To use a grammar that is not easily modified to be LR(1), a more general
943 parsing algorithm is sometimes necessary. If you include @code{%glr-parser}
944 among the Bison declarations in your file (@pxref{Grammar Outline}), the
945 result is a Generalized LR (GLR) parser. These parsers handle Bison
946 grammars that contain no unresolved conflicts (i.e., after applying
947 precedence declarations) identically to deterministic parsers. However,
948 when faced with unresolved shift/reduce and reduce/reduce conflicts, GLR
949 parsers use the simple expedient of doing both, effectively cloning the
950 parser to follow both possibilities. Each of the resulting parsers can
951 again split, so that at any given time, there can be any number of possible
952 parses being explored. The parsers proceed in lockstep; that is, all of
953 them consume (shift) a given input symbol before any of them proceed to the
954 next. Each of the cloned parsers eventually meets one of two possible
955 fates: either it runs into a parsing error, in which case it simply
956 vanishes, or it merges with another parser, because the two of them have
957 reduced the input to an identical set of symbols.
959 During the time that there are multiple parsers, semantic actions are
960 recorded, but not performed. When a parser disappears, its recorded
961 semantic actions disappear as well, and are never performed. When a
962 reduction makes two parsers identical, causing them to merge, Bison records
963 both sets of semantic actions. Whenever the last two parsers merge,
964 reverting to the single-parser case, Bison resolves all the outstanding
965 actions either by precedences given to the grammar rules involved, or by
966 performing both actions, and then calling a designated user-defined function
967 on the resulting values to produce an arbitrary merged result.
970 * Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
971 * Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
972 * GLR Semantic Actions:: Considerations for semantic values and deferred actions.
973 * Semantic Predicates:: Controlling a parse with arbitrary computations.
976 @node Simple GLR Parsers
977 @subsection Using GLR on Unambiguous Grammars
978 @cindex GLR parsing, unambiguous grammars
979 @cindex generalized LR (GLR) parsing, unambiguous grammars
983 @cindex reduce/reduce conflicts
984 @cindex shift/reduce conflicts
986 In the simplest cases, you can use the GLR algorithm
987 to parse grammars that are unambiguous but fail to be LR(1).
988 Such grammars typically require more than one symbol of lookahead.
990 Consider a problem that
991 arises in the declaration of enumerated and subrange types in the
992 programming language Pascal. Here are some examples:
995 type subrange = lo .. hi;
996 type enum = (a, b, c);
1000 The original language standard allows only numeric literals and constant
1001 identifiers for the subrange bounds (@samp{lo} and @samp{hi}), but Extended
1002 Pascal (ISO/IEC 10206) and many other Pascal implementations allow arbitrary
1003 expressions there. This gives rise to the following situation, containing a
1004 superfluous pair of parentheses:
1007 type subrange = (a) .. b;
1011 Compare this to the following declaration of an enumerated
1012 type with only one value:
1019 (These declarations are contrived, but they are syntactically valid, and
1020 more-complicated cases can come up in practical programs.)
1022 These two declarations look identical until the @samp{..} token. With
1023 normal LR(1) one-token lookahead it is not possible to decide between the
1024 two forms when the identifier @samp{a} is parsed. It is, however, desirable
1025 for a parser to decide this, since in the latter case @samp{a} must become a
1026 new identifier to represent the enumeration value, while in the former case
1027 @samp{a} must be evaluated with its current meaning, which may be a constant
1028 or even a function call.
1030 You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
1031 to be resolved later, but this typically requires substantial contortions in
1032 both semantic actions and large parts of the grammar, where the parentheses
1033 are nested in the recursive rules for expressions.
1035 You might think of using the lexer to distinguish between the two forms by
1036 returning different tokens for currently defined and undefined identifiers.
1037 But if these declarations occur in a local scope, and @samp{a} is defined in
1038 an outer scope, then both forms are possible---either locally redefining
1039 @samp{a}, or using the value of @samp{a} from the outer scope. So this
1040 approach cannot work.
1042 A simple solution to this problem is to declare the parser to use the GLR
1043 algorithm. When the GLR parser reaches the critical state, it merely splits
1044 into two branches and pursues both syntax rules simultaneously. Sooner or
1045 later, one of them runs into a parsing error. If there is a @samp{..} token
1046 before the next @samp{;}, the rule for enumerated types fails since it
1047 cannot accept @samp{..} anywhere; otherwise, the subrange type rule fails
1048 since it requires a @samp{..} token. So one of the branches fails silently,
1049 and the other one continues normally, performing all the intermediate
1050 actions that were postponed during the split.
1052 If the input is syntactically incorrect, both branches fail and the parser
1053 reports a syntax error as usual.
1055 The effect of all this is that the parser seems to ``guess'' the correct
1056 branch to take, or in other words, it seems to use more lookahead than the
1057 underlying LR(1) algorithm actually allows for. In this example, LR(2)
1058 would suffice, but also some cases that are not LR(@math{k}) for any
1059 @math{k} can be handled this way.
1061 In general, a GLR parser can take quadratic or cubic worst-case time, and
1062 the current Bison parser even takes exponential time and space for some
1063 grammars. In practice, this rarely happens, and for many grammars it is
1064 possible to prove that it cannot happen. The present example contains only
1065 one conflict between two rules, and the type-declaration context containing
1066 the conflict cannot be nested. So the number of branches that can exist at
1067 any time is limited by the constant 2, and the parsing time is still linear.
1069 Here is a Bison grammar corresponding to the example above. It
1070 parses a vastly simplified form of Pascal type declarations.
1073 %token TYPE DOTDOT ID
1081 type_decl: TYPE ID '=' type ';' ;
1109 When used as a normal LR(1) grammar, Bison correctly complains
1110 about one reduce/reduce conflict. In the conflicting situation the
1111 parser chooses one of the alternatives, arbitrarily the one
1112 declared first. Therefore the following correct input is not
1119 The parser can be turned into a GLR parser, while also telling Bison
1120 to be silent about the one known reduce/reduce conflict, by adding
1121 these two declarations to the Bison grammar file (before the first
1130 No change in the grammar itself is required. Now the parser recognizes all
1131 valid declarations, according to the limited syntax above, transparently.
1132 In fact, the user does not even notice when the parser splits.
1134 So here we have a case where we can use the benefits of GLR, almost without
1135 disadvantages. Even in simple cases like this, however, there are at least
1136 two potential problems to beware. First, always analyze the conflicts
1137 reported by Bison to make sure that GLR splitting is only done where it is
1138 intended. A GLR parser splitting inadvertently may cause problems less
1139 obvious than an LR parser statically choosing the wrong alternative in a
1140 conflict. Second, consider interactions with the lexer (@pxref{Semantic
1141 Tokens}) with great care. Since a split parser consumes tokens without
1142 performing any actions during the split, the lexer cannot obtain information
1143 via parser actions. Some cases of lexer interactions can be eliminated by
1144 using GLR to shift the complications from the lexer to the parser. You must
1145 check the remaining cases for correctness.
1147 In our example, it would be safe for the lexer to return tokens based on
1148 their current meanings in some symbol table, because no new symbols are
1149 defined in the middle of a type declaration. Though it is possible for a
1150 parser to define the enumeration constants as they are parsed, before the
1151 type declaration is completed, it actually makes no difference since they
1152 cannot be used within the same enumerated type declaration.
1154 @node Merging GLR Parses
1155 @subsection Using GLR to Resolve Ambiguities
1156 @cindex GLR parsing, ambiguous grammars
1157 @cindex generalized LR (GLR) parsing, ambiguous grammars
1161 @cindex reduce/reduce conflicts
1163 Let's consider an example, vastly simplified from a C++ grammar.
1168 #define YYSTYPE char const *
1170 void yyerror (char const *);
1184 | prog stmt @{ printf ("\n"); @}
1193 ID @{ printf ("%s ", $$); @}
1194 | TYPENAME '(' expr ')'
1195 @{ printf ("%s <cast> ", $1); @}
1196 | expr '+' expr @{ printf ("+ "); @}
1197 | expr '=' expr @{ printf ("= "); @}
1201 TYPENAME declarator ';'
1202 @{ printf ("%s <declare> ", $1); @}
1203 | TYPENAME declarator '=' expr ';'
1204 @{ printf ("%s <init-declare> ", $1); @}
1208 ID @{ printf ("\"%s\" ", $1); @}
1209 | '(' declarator ')'
1214 This models a problematic part of the C++ grammar---the ambiguity between
1215 certain declarations and statements. For example,
1222 parses as either an @code{expr} or a @code{stmt}
1223 (assuming that @samp{T} is recognized as a @code{TYPENAME} and
1224 @samp{x} as an @code{ID}).
1225 Bison detects this as a reduce/reduce conflict between the rules
1226 @code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1227 time it encounters @code{x} in the example above. Since this is a
1228 GLR parser, it therefore splits the problem into two parses, one for
1229 each choice of resolving the reduce/reduce conflict.
1230 Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1231 however, neither of these parses ``dies,'' because the grammar as it stands is
1232 ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1233 the other reduces @code{stmt : decl}, after which both parsers are in an
1234 identical state: they've seen @samp{prog stmt} and have the same unprocessed
1235 input remaining. We say that these parses have @dfn{merged.}
1237 At this point, the GLR parser requires a specification in the
1238 grammar of how to choose between the competing parses.
1239 In the example above, the two @code{%dprec}
1240 declarations specify that Bison is to give precedence
1241 to the parse that interprets the example as a
1242 @code{decl}, which implies that @code{x} is a declarator.
1243 The parser therefore prints
1246 "x" y z + T <init-declare>
1249 The @code{%dprec} declarations only come into play when more than one
1250 parse survives. Consider a different input string for this parser:
1257 This is another example of using GLR to parse an unambiguous
1258 construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1259 Here, there is no ambiguity (this cannot be parsed as a declaration).
1260 However, at the time the Bison parser encounters @code{x}, it does not
1261 have enough information to resolve the reduce/reduce conflict (again,
1262 between @code{x} as an @code{expr} or a @code{declarator}). In this
1263 case, no precedence declaration is used. Again, the parser splits
1264 into two, one assuming that @code{x} is an @code{expr}, and the other
1265 assuming @code{x} is a @code{declarator}. The second of these parsers
1266 then vanishes when it sees @code{+}, and the parser prints
1272 Suppose that instead of resolving the ambiguity, you wanted to see all
1273 the possibilities. For this purpose, you must merge the semantic
1274 actions of the two possible parsers, rather than choosing one over the
1275 other. To do so, you could change the declaration of @code{stmt} as
1280 expr ';' %merge <stmtMerge>
1281 | decl %merge <stmtMerge>
1286 and define the @code{stmtMerge} function as:
1290 stmtMerge (YYSTYPE x0, YYSTYPE x1)
1298 with an accompanying forward declaration
1299 in the C declarations at the beginning of the file:
1303 #define YYSTYPE char const *
1304 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1309 With these declarations, the resulting parser parses the first example
1310 as both an @code{expr} and a @code{decl}, and prints
1313 "x" y z + T <init-declare> x T <cast> y z + = <OR>
1316 Bison requires that all of the
1317 productions that participate in any particular merge have identical
1318 @samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1319 and the parser will report an error during any parse that results in
1320 the offending merge.
1322 @node GLR Semantic Actions
1323 @subsection GLR Semantic Actions
1325 The nature of GLR parsing and the structure of the generated
1326 parsers give rise to certain restrictions on semantic values and actions.
1328 @subsubsection Deferred semantic actions
1329 @cindex deferred semantic actions
1330 By definition, a deferred semantic action is not performed at the same time as
1331 the associated reduction.
1332 This raises caveats for several Bison features you might use in a semantic
1333 action in a GLR parser.
1336 @cindex GLR parsers and @code{yychar}
1338 @cindex GLR parsers and @code{yylval}
1340 @cindex GLR parsers and @code{yylloc}
1341 In any semantic action, you can examine @code{yychar} to determine the kind
1342 of the lookahead token present at the time of the associated reduction.
1343 After checking that @code{yychar} is not set to @code{YYEMPTY} or
1344 @code{YYEOF}, you can then examine @code{yylval} and @code{yylloc} to
1345 determine the lookahead token's semantic value and location, if any. In a
1346 nondeferred semantic action, you can also modify any of these variables to
1347 influence syntax analysis. @xref{Lookahead}.
1350 @cindex GLR parsers and @code{yyclearin}
1351 In a deferred semantic action, it's too late to influence syntax analysis.
1352 In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1353 shallow copies of the values they had at the time of the associated reduction.
1354 For this reason alone, modifying them is dangerous.
1355 Moreover, the result of modifying them is undefined and subject to change with
1356 future versions of Bison.
1357 For example, if a semantic action might be deferred, you should never write it
1358 to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1359 memory referenced by @code{yylval}.
1361 @subsubsection YYERROR
1363 @cindex GLR parsers and @code{YYERROR}
1364 Another Bison feature requiring special consideration is @code{YYERROR}
1365 (@pxref{Action Features}), which you can invoke in a semantic action to
1366 initiate error recovery.
1367 During deterministic GLR operation, the effect of @code{YYERROR} is
1368 the same as its effect in a deterministic parser.
1369 The effect in a deferred action is similar, but the precise point of the
1370 error is undefined; instead, the parser reverts to deterministic operation,
1371 selecting an unspecified stack on which to continue with a syntax error.
1372 In a semantic predicate (see @ref{Semantic Predicates}) during nondeterministic
1373 parsing, @code{YYERROR} silently prunes
1374 the parse that invoked the test.
1376 @subsubsection Restrictions on semantic values and locations
1377 GLR parsers require that you use POD (Plain Old Data) types for
1378 semantic values and location types when using the generated parsers as
1381 @node Semantic Predicates
1382 @subsection Controlling a Parse with Arbitrary Predicates
1384 @cindex Semantic predicates in GLR parsers
1386 In addition to the @code{%dprec} and @code{%merge} directives,
1388 allow you to reject parses on the basis of arbitrary computations executed
1389 in user code, without having Bison treat this rejection as an error
1390 if there are alternative parses. For example,
1394 %?@{ new_syntax @} "widget" id new_args @{ $$ = f($3, $4); @}
1395 | %?@{ !new_syntax @} "widget" id old_args @{ $$ = f($3, $4); @}
1400 is one way to allow the same parser to handle two different syntaxes for
1401 widgets. The clause preceded by @code{%?} is treated like an ordinary
1402 midrule action, except that its text is handled as an expression and is always
1403 evaluated immediately (even when in nondeterministic mode). If the
1404 expression yields 0 (false), the clause is treated as a syntax error,
1405 which, in a nondeterministic parser, causes the stack in which it is reduced
1406 to die. In a deterministic parser, it acts like @code{YYERROR}.
1408 As the example shows, predicates otherwise look like semantic actions, and
1409 therefore you must take them into account when determining the numbers
1410 to use for denoting the semantic values of right-hand side symbols.
1411 Predicate actions, however, have no defined value, and may not be given
1414 There is a subtle difference between semantic predicates and ordinary
1415 actions in nondeterministic mode, since the latter are deferred.
1416 For example, we could try to rewrite the previous example as
1420 @{ if (!new_syntax) YYERROR; @}
1421 "widget" id new_args @{ $$ = f($3, $4); @}
1422 | @{ if (new_syntax) YYERROR; @}
1423 "widget" id old_args @{ $$ = f($3, $4); @}
1428 (reversing the sense of the predicate tests to cause an error when they are
1429 false). However, this
1430 does @emph{not} have the same effect if @code{new_args} and @code{old_args}
1431 have overlapping syntax.
1432 Since the midrule actions testing @code{new_syntax} are deferred,
1433 a GLR parser first encounters the unresolved ambiguous reduction
1434 for cases where @code{new_args} and @code{old_args} recognize the same string
1435 @emph{before} performing the tests of @code{new_syntax}. It therefore
1438 Finally, be careful in writing predicates: deferred actions have not been
1439 evaluated, so that using them in a predicate will have undefined effects.
1444 @cindex textual location
1445 @cindex location, textual
1447 Many applications, like interpreters or compilers, have to produce verbose
1448 and useful error messages. To achieve this, one must be able to keep track of
1449 the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1450 Bison provides a mechanism for handling these locations.
1452 Each token has a semantic value. In a similar fashion, each token has an
1453 associated location, but the type of locations is the same for all tokens
1454 and groupings. Moreover, the output parser is equipped with a default data
1455 structure for storing locations (@pxref{Tracking Locations}, for more
1458 Like semantic values, locations can be reached in actions using a dedicated
1459 set of constructs. In the example above, the location of the whole grouping
1460 is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1463 When a rule is matched, a default action is used to compute the semantic value
1464 of its left hand side (@pxref{Actions}). In the same way, another default
1465 action is used for locations. However, the action for locations is general
1466 enough for most cases, meaning there is usually no need to describe for each
1467 rule how @code{@@$} should be formed. When building a new location for a given
1468 grouping, the default behavior of the output parser is to take the beginning
1469 of the first symbol, and the end of the last symbol.
1472 @section Bison Output: the Parser Implementation File
1473 @cindex Bison parser
1474 @cindex Bison utility
1475 @cindex lexical analyzer, purpose
1478 When you run Bison, you give it a Bison grammar file as input. The
1479 most important output is a C source file that implements a parser for
1480 the language described by the grammar. This parser is called a
1481 @dfn{Bison parser}, and this file is called a @dfn{Bison parser
1482 implementation file}. Keep in mind that the Bison utility and the
1483 Bison parser are two distinct programs: the Bison utility is a program
1484 whose output is the Bison parser implementation file that becomes part
1487 The job of the Bison parser is to group tokens into groupings according to
1488 the grammar rules---for example, to build identifiers and operators into
1489 expressions. As it does this, it runs the actions for the grammar rules it
1492 The tokens come from a function called the @dfn{lexical analyzer} that
1493 you must supply in some fashion (such as by writing it in C). The Bison
1494 parser calls the lexical analyzer each time it wants a new token. It
1495 doesn't know what is ``inside'' the tokens (though their semantic values
1496 may reflect this). Typically the lexical analyzer makes the tokens by
1497 parsing characters of text, but Bison does not depend on this.
1500 The Bison parser implementation file is C code which defines a
1501 function named @code{yyparse} which implements that grammar. This
1502 function does not make a complete C program: you must supply some
1503 additional functions. One is the lexical analyzer. Another is an
1504 error-reporting function which the parser calls to report an error.
1505 In addition, a complete C program must start with a function called
1506 @code{main}; you have to provide this, and arrange for it to call
1507 @code{yyparse} or the parser will never run. @xref{Interface}.
1509 Aside from the token kind names and the symbols in the actions you
1510 write, all symbols defined in the Bison parser implementation file
1511 itself begin with @samp{yy} or @samp{YY}. This includes interface
1512 functions such as the lexical analyzer function @code{yylex}, the
1513 error reporting function @code{yyerror} and the parser function
1514 @code{yyparse} itself. This also includes numerous identifiers used
1515 for internal purposes. Therefore, you should avoid using C
1516 identifiers starting with @samp{yy} or @samp{YY} in the Bison grammar
1517 file except for the ones defined in this manual. Also, you should
1518 avoid using the C identifiers @samp{malloc} and @samp{free} for
1519 anything other than their usual meanings.
1521 In some cases the Bison parser implementation file includes system
1522 headers, and in those cases your code should respect the identifiers
1523 reserved by those headers. On some non-GNU hosts, @code{<limits.h>},
1524 @code{<stddef.h>}, @code{<stdint.h>} (if available), and @code{<stdlib.h>}
1525 are included to declare memory allocators and integer types and constants.
1526 @code{<libintl.h>} is included if message translation is in use
1527 (@pxref{Internationalization}). Other system headers may be included
1528 if you define @code{YYDEBUG} (@pxref{Tracing}) or
1529 @code{YYSTACK_USE_ALLOCA} (@pxref{Table of Symbols}) to a nonzero value.
1532 @section Stages in Using Bison
1533 @cindex stages in using Bison
1536 The actual language-design process using Bison, from grammar specification
1537 to a working compiler or interpreter, has these parts:
1541 Formally specify the grammar in a form recognized by Bison
1542 (@pxref{Grammar File}). For each grammatical rule
1543 in the language, describe the action that is to be taken when an
1544 instance of that rule is recognized. The action is described by a
1545 sequence of C statements.
1548 Write a lexical analyzer to process input and pass tokens to the parser.
1549 The lexical analyzer may be written by hand in C (@pxref{Lexical}). It
1550 could also be produced using Lex, but the use of Lex is not discussed in
1554 Write a controlling function that calls the Bison-produced parser.
1557 Write error-reporting routines.
1560 To turn this source code as written into a runnable program, you
1561 must follow these steps:
1565 Run Bison on the grammar to produce the parser.
1568 Compile the code output by Bison, as well as any other source files.
1571 Link the object files to produce the finished product.
1574 @node Grammar Layout
1575 @section The Overall Layout of a Bison Grammar
1576 @cindex grammar file
1578 @cindex format of grammar file
1579 @cindex layout of Bison grammar
1581 The input file for the Bison utility is a @dfn{Bison grammar file}. The
1582 general form of a Bison grammar file is as follows:
1589 @var{Bison declarations}
1598 The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1599 in every Bison grammar file to separate the sections.
1601 The prologue may define types and variables used in the actions. You can
1602 also use preprocessor commands to define macros used there, and use
1603 @code{#include} to include header files that do any of these things.
1604 You need to declare the lexical analyzer @code{yylex} and the error
1605 printer @code{yyerror} here, along with any other global identifiers
1606 used by the actions in the grammar rules.
1608 The Bison declarations declare the names of the terminal and nonterminal
1609 symbols, and may also describe operator precedence and the data types of
1610 semantic values of various symbols.
1612 The grammar rules define how to construct each nonterminal symbol from its
1615 The epilogue can contain any code you want to use. Often the
1616 definitions of functions declared in the prologue go here. In a
1617 simple program, all the rest of the program can go here.
1621 @cindex simple examples
1622 @cindex examples, simple
1624 Now we show and explain several sample programs written using Bison: a
1625 Reverse Polish Notation calculator, an algebraic (infix) notation
1626 calculator --- later extended to track ``locations'' ---
1627 and a multi-function calculator. All
1628 produce usable, though limited, interactive desk-top calculators.
1630 These examples are simple, but Bison grammars for real programming
1631 languages are written the same way. You can copy these examples into a
1632 source file to try them.
1635 * RPN Calc:: Reverse Polish Notation Calculator;
1636 a first example with no operator precedence.
1637 * Infix Calc:: Infix (algebraic) notation calculator.
1638 Operator precedence is introduced.
1639 * Simple Error Recovery:: Continuing after syntax errors.
1640 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1641 * Multi-function Calc:: Calculator with memory and trig functions.
1642 It uses multiple data-types for semantic values.
1643 * Exercises:: Ideas for improving the multi-function calculator.
1647 @section Reverse Polish Notation Calculator
1648 @cindex Reverse Polish Notation
1649 @cindex @code{rpcalc}
1650 @cindex calculator, simple
1652 The first example is that of a simple double-precision @dfn{Reverse Polish
1653 Notation} calculator (a calculator using postfix operators). This example
1654 provides a good starting point, since operator precedence is not an issue.
1655 The second example will illustrate how operator precedence is handled.
1657 The source code for this calculator is named @file{rpcalc.y}. The
1658 @samp{.y} extension is a convention used for Bison grammar files.
1661 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1662 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1663 * Rpcalc Lexer:: The lexical analyzer.
1664 * Rpcalc Main:: The controlling function.
1665 * Rpcalc Error:: The error reporting function.
1666 * Rpcalc Generate:: Running Bison on the grammar file.
1667 * Rpcalc Compile:: Run the C compiler on the output code.
1670 @node Rpcalc Declarations
1671 @subsection Declarations for @code{rpcalc}
1673 Here are the C and Bison declarations for the Reverse Polish Notation
1674 calculator. As in C, comments are placed between @samp{/*@dots{}*/} or
1678 @comment file: c/rpcalc/rpcalc.y
1680 /* Parser for rpcalc. -*- C -*-
1682 Copyright (C) 1988-1993, 1995, 1998-2015, 2018-2021 Free Software
1685 This file is part of Bison, the GNU Compiler Compiler.
1687 This program is free software: you can redistribute it and/or modify
1688 it under the terms of the GNU General Public License as published by
1689 the Free Software Foundation, either version 3 of the License, or
1690 (at your option) any later version.
1692 This program is distributed in the hope that it will be useful,
1693 but WITHOUT ANY WARRANTY; without even the implied warranty of
1694 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1695 GNU General Public License for more details.
1697 You should have received a copy of the GNU General Public License
1698 along with this program. If not, see <http://www.gnu.org/licenses/>. */
1702 @comment file: c/rpcalc/rpcalc.y
1704 /* Reverse Polish Notation calculator. */
1711 void yyerror (char const *);
1715 %define api.value.type @{double@}
1718 %% /* Grammar rules and actions follow. */
1721 The declarations section (@pxref{Prologue}) contains two
1722 preprocessor directives and two forward declarations.
1724 The @code{#include} directive is used to declare the exponentiation
1725 function @code{pow}.
1727 The forward declarations for @code{yylex} and @code{yyerror} are
1728 needed because the C language requires that functions be declared
1729 before they are used. These functions will be defined in the
1730 epilogue, but the parser calls them so they must be declared in the
1733 The second section, Bison declarations, provides information to Bison about
1734 the tokens and their types (@pxref{Bison Declarations}).
1736 The @code{%define} directive defines the variable @code{api.value.type},
1737 thus specifying the C data type for semantic values of both tokens and
1738 groupings (@pxref{Value Type}). The Bison
1739 parser will use whatever type @code{api.value.type} is defined as; if you
1740 don't define it, @code{int} is the default. Because we specify
1741 @samp{@{double@}}, each token and each expression has an associated value,
1742 which is a floating point number. C code can use @code{YYSTYPE} to refer to
1743 the value @code{api.value.type}.
1745 Each terminal symbol that is not a single-character literal must be
1746 declared. (Single-character literals normally don't need to be declared.)
1747 In this example, all the arithmetic operators are designated by
1748 single-character literals, so the only terminal symbol that needs to be
1749 declared is @code{NUM}, the token kind for numeric constants.
1752 @subsection Grammar Rules for @code{rpcalc}
1754 Here are the grammar rules for the Reverse Polish Notation calculator.
1756 @comment file: c/rpcalc/rpcalc.y
1768 | exp '\n' @{ printf ("%.10g\n", $1); @}
1775 | exp exp '+' @{ $$ = $1 + $2; @}
1776 | exp exp '-' @{ $$ = $1 - $2; @}
1777 | exp exp '*' @{ $$ = $1 * $2; @}
1778 | exp exp '/' @{ $$ = $1 / $2; @}
1779 | exp exp '^' @{ $$ = pow ($1, $2); @} /* Exponentiation */
1780 | exp 'n' @{ $$ = -$1; @} /* Unary minus */
1786 The groupings of the rpcalc ``language'' defined here are the expression
1787 (given the name @code{exp}), the line of input (@code{line}), and the
1788 complete input transcript (@code{input}). Each of these nonterminal
1789 symbols has several alternate rules, joined by the vertical bar @samp{|}
1790 which is read as ``or''. The following sections explain what these rules
1793 The semantics of the language is determined by the actions taken when a
1794 grouping is recognized. The actions are the C code that appears inside
1795 braces. @xref{Actions}.
1797 You must specify these actions in C, but Bison provides the means for
1798 passing semantic values between the rules. In each action, the
1799 pseudo-variable @code{$$} stands for the semantic value for the grouping
1800 that the rule is going to construct. Assigning a value to @code{$$} is the
1801 main job of most actions. The semantic values of the components of the
1802 rule are referred to as @code{$1}, @code{$2}, and so on.
1805 * Rpcalc Input:: Explanation of the @code{input} nonterminal
1806 * Rpcalc Line:: Explanation of the @code{line} nonterminal
1807 * Rpcalc Exp:: Explanation of the @code{exp} nonterminal
1811 @subsubsection Explanation of @code{input}
1813 Consider the definition of @code{input}:
1822 This definition reads as follows: ``A complete input is either an empty
1823 string, or a complete input followed by an input line''. Notice that
1824 ``complete input'' is defined in terms of itself. This definition is said
1825 to be @dfn{left recursive} since @code{input} appears always as the
1826 leftmost symbol in the sequence. @xref{Recursion}.
1828 The first alternative is empty because there are no symbols between the
1829 colon and the first @samp{|}; this means that @code{input} can match an
1830 empty string of input (no tokens). We write the rules this way because it
1831 is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1832 It's conventional to put an empty alternative first and to use the
1833 (optional) @code{%empty} directive, or to write the comment @samp{/* empty
1834 */} in it (@pxref{Empty Rules}).
1836 The second alternate rule (@code{input line}) handles all nontrivial input.
1837 It means, ``After reading any number of lines, read one more line if
1838 possible.'' The left recursion makes this rule into a loop. Since the
1839 first alternative matches empty input, the loop can be executed zero or
1842 The parser function @code{yyparse} continues to process input until a
1843 grammatical error is seen or the lexical analyzer says there are no more
1844 input tokens; we will arrange for the latter to happen at end-of-input.
1847 @subsubsection Explanation of @code{line}
1849 Now consider the definition of @code{line}:
1854 | exp '\n' @{ printf ("%.10g\n", $1); @}
1858 The first alternative is a token which is a newline character; this means
1859 that rpcalc accepts a blank line (and ignores it, since there is no
1860 action). The second alternative is an expression followed by a newline.
1861 This is the alternative that makes rpcalc useful. The semantic value of
1862 the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1863 question is the first symbol in the alternative. The action prints this
1864 value, which is the result of the computation the user asked for.
1866 This action is unusual because it does not assign a value to @code{$$}. As
1867 a consequence, the semantic value associated with the @code{line} is
1868 uninitialized (its value will be unpredictable). This would be a bug if
1869 that value were ever used, but we don't use it: once rpcalc has printed the
1870 value of the user's input line, that value is no longer needed.
1873 @subsubsection Explanation of @code{exp}
1875 The @code{exp} grouping has several rules, one for each kind of expression.
1876 The first rule handles the simplest expressions: those that are just
1877 numbers. The second handles an addition-expression, which looks like two
1878 expressions followed by a plus-sign. The third handles subtraction, and so
1884 | exp exp '+' @{ $$ = $1 + $2; @}
1885 | exp exp '-' @{ $$ = $1 - $2; @}
1890 We have used @samp{|} to join all the rules for @code{exp}, but we could
1891 equally well have written them separately:
1895 exp: exp exp '+' @{ $$ = $1 + $2; @};
1896 exp: exp exp '-' @{ $$ = $1 - $2; @};
1900 Most of the rules have actions that compute the value of the expression in
1901 terms of the value of its parts. For example, in the rule for addition,
1902 @code{$1} refers to the first component @code{exp} and @code{$2} refers to
1903 the second one. The third component, @code{'+'}, has no meaningful
1904 associated semantic value, but if it had one you could refer to it as
1905 @code{$3}. The first rule relies on the implicit default action: @samp{@{
1909 When @code{yyparse} recognizes a sum expression using this rule, the sum of
1910 the two subexpressions' values is produced as the value of the entire
1911 expression. @xref{Actions}.
1913 You don't have to give an action for every rule. When a rule has no action,
1914 Bison by default copies the value of @code{$1} into @code{$$}. This is what
1915 happens in the first rule (the one that uses @code{NUM}).
1917 The formatting shown here is the recommended convention, but Bison does not
1918 require it. You can add or change white space as much as you wish. For
1922 exp: NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1926 means the same thing as this:
1931 | exp exp '+' @{ $$ = $1 + $2; @}
1937 The latter, however, is much more readable.
1940 @subsection The @code{rpcalc} Lexical Analyzer
1941 @cindex writing a lexical analyzer
1942 @cindex lexical analyzer, writing
1944 The lexical analyzer's job is low-level parsing: converting characters
1945 or sequences of characters into tokens. The Bison parser gets its
1946 tokens by calling the lexical analyzer. @xref{Lexical}.
1948 Only a simple lexical analyzer is needed for the RPN
1950 lexical analyzer skips blanks and tabs, then reads in numbers as
1951 @code{double} and returns them as @code{NUM} tokens. Any other character
1952 that isn't part of a number is a separate token. Note that the token-code
1953 for such a single-character token is the character itself.
1955 The return value of the lexical analyzer function is a numeric code which
1956 represents a token kind. The same text used in Bison rules to stand for
1957 this token kind is also a C expression for the numeric code of the kind.
1958 This works in two ways. If the token kind is a character literal, then its
1959 numeric code is that of the character; you can use the same character
1960 literal in the lexical analyzer to express the number. If the token kind is
1961 an identifier, that identifier is defined by Bison as a C enum whose
1962 definition is the appropriate code. In this example, therefore, @code{NUM}
1963 becomes an enum for @code{yylex} to use.
1965 The semantic value of the token (if it has one) is stored into the global
1966 variable @code{yylval}, which is where the Bison parser will look for it.
1967 (The C data type of @code{yylval} is @code{YYSTYPE}, whose value was defined
1968 at the beginning of the grammar via @samp{%define api.value.type
1969 @{double@}}; @pxref{Rpcalc Declarations}.)
1971 A token kind code of zero is returned if the end-of-input is encountered.
1972 (Bison recognizes any nonpositive value as indicating end-of-input.)
1974 Here is the code for the lexical analyzer:
1976 @comment file: c/rpcalc/rpcalc.y
1979 /* The lexical analyzer returns a double floating point
1980 number on the stack and the token NUM, or the numeric code
1981 of the character read if not a number. It skips all blanks
1982 and tabs, and returns 0 for end-of-input. */
1993 /* Skip white space. */
1994 while (c == ' ' || c == '\t')
1998 /* Process numbers. */
1999 if (c == '.' || isdigit (c))
2002 if (scanf ("%lf", &yylval) != 1)
2008 /* Return end-of-input. */
2011 /* Return a single char. */
2019 @subsection The Controlling Function
2020 @cindex controlling function
2021 @cindex main function in simple example
2023 In keeping with the spirit of this example, the controlling function is
2024 kept to the bare minimum. The only requirement is that it call
2025 @code{yyparse} to start the process of parsing.
2027 @comment file: c/rpcalc/rpcalc.y
2039 @subsection The Error Reporting Routine
2040 @cindex error reporting routine
2042 When @code{yyparse} detects a syntax error, it calls the error reporting
2043 function @code{yyerror} to print an error message (usually but not
2044 always @code{"syntax error"}). It is up to the programmer to supply
2045 @code{yyerror} (@pxref{Interface}), so
2046 here is the definition we will use:
2048 @comment file: c/rpcalc/rpcalc.y
2053 /* Called by yyparse on error. */
2055 yyerror (char const *s)
2057 fprintf (stderr, "%s\n", s);
2062 After @code{yyerror} returns, the Bison parser may recover from the error
2063 and continue parsing if the grammar contains a suitable error rule
2064 (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
2065 have not written any error rules in this example, so any invalid input will
2066 cause the calculator program to exit. This is not clean behavior for a
2067 real calculator, but it is adequate for the first example.
2069 @node Rpcalc Generate
2070 @subsection Running Bison to Make the Parser
2071 @cindex running Bison (introduction)
2073 Before running Bison to produce a parser, we need to decide how to
2074 arrange all the source code in one or more source files. For such a
2075 simple example, the easiest thing is to put everything in one file,
2076 the grammar file. The definitions of @code{yylex}, @code{yyerror} and
2077 @code{main} go at the end, in the epilogue of the grammar file
2078 (@pxref{Grammar Layout}).
2080 For a large project, you would probably have several source files, and use
2081 @code{make} to arrange to recompile them.
2083 With all the source in the grammar file, you use the following command
2084 to convert it into a parser implementation file:
2087 $ @kbd{bison @var{file}.y}
2091 In this example, the grammar file is called @file{rpcalc.y} (for
2092 ``Reverse Polish @sc{calc}ulator''). Bison produces a parser
2093 implementation file named @file{@var{file}.tab.c}, removing the
2094 @samp{.y} from the grammar file name. The parser implementation file
2095 contains the source code for @code{yyparse}. The additional functions
2096 in the grammar file (@code{yylex}, @code{yyerror} and @code{main}) are
2097 copied verbatim to the parser implementation file.
2099 @node Rpcalc Compile
2100 @subsection Compiling the Parser Implementation File
2101 @cindex compiling the parser
2103 Here is how to compile and run the parser implementation file:
2107 # @r{List files in current directory.}
2109 rpcalc.tab.c rpcalc.y
2113 # @r{Compile the Bison parser.}
2114 # @r{@option{-lm} tells compiler to search math library for @code{pow}.}
2115 $ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
2119 # @r{List files again.}
2121 rpcalc rpcalc.tab.c rpcalc.y
2125 The file @file{rpcalc} now contains the executable code. Here is an
2126 example session using @code{rpcalc}.
2132 @kbd{3 7 + 3 4 5 *+-}
2134 @kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
2137 @result{} -3.166666667
2138 @kbd{3 4 ^} @r{Exponentiation}
2140 @kbd{^D} @r{End-of-file indicator}
2145 @section Infix Notation Calculator: @code{calc}
2146 @cindex infix notation calculator
2148 @cindex calculator, infix notation
2150 We now modify rpcalc to handle infix operators instead of postfix. Infix
2151 notation involves the concept of operator precedence and the need for
2152 parentheses nested to arbitrary depth. Here is the Bison code for
2153 @file{calc.y}, an infix desk-top calculator.
2156 /* Infix notation calculator. */
2163 void yyerror (char const *);
2168 /* Bison declarations. */
2169 %define api.value.type @{double@}
2173 %precedence NEG /* negation--unary minus */
2174 %right '^' /* exponentiation */
2177 %% /* The grammar follows. */
2188 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2195 | exp '+' exp @{ $$ = $1 + $3; @}
2196 | exp '-' exp @{ $$ = $1 - $3; @}
2197 | exp '*' exp @{ $$ = $1 * $3; @}
2198 | exp '/' exp @{ $$ = $1 / $3; @}
2199 | '-' exp %prec NEG @{ $$ = -$2; @}
2200 | exp '^' exp @{ $$ = pow ($1, $3); @}
2201 | '(' exp ')' @{ $$ = $2; @}
2208 The functions @code{yylex}, @code{yyerror} and @code{main} can be the
2211 There are two important new features shown in this code.
2213 In the second section (Bison declarations), @code{%left} declares token
2214 kinds and says they are left-associative operators. The declarations
2215 @code{%left} and @code{%right} (right associativity) take the place of
2216 @code{%token} which is used to declare a token kind name without
2217 associativity/precedence. (These tokens are single-character literals,
2218 which ordinarily don't need to be declared. We declare them here to specify
2219 the associativity/precedence.)
2221 Operator precedence is determined by the line ordering of the
2222 declarations; the higher the line number of the declaration (lower on
2223 the page or screen), the higher the precedence. Hence, exponentiation
2224 has the highest precedence, unary minus (@code{NEG}) is next, followed
2225 by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
2226 only precedence matters (@code{%precedence}. @xref{Precedence}.
2228 The other important new feature is the @code{%prec} in the grammar
2229 section for the unary minus operator. The @code{%prec} simply instructs
2230 Bison that the rule @samp{| '-' exp} has the same precedence as
2231 @code{NEG}---in this case the next-to-highest. @xref{Contextual
2234 Here is a sample run of @file{calc.y}:
2239 @kbd{4 + 4.5 - (34/(8*3+-3))}
2247 @node Simple Error Recovery
2248 @section Simple Error Recovery
2249 @cindex error recovery, simple
2251 Up to this point, this manual has not addressed the issue of @dfn{error
2252 recovery}---how to continue parsing after the parser detects a syntax
2253 error. All we have handled is error reporting with @code{yyerror}.
2254 Recall that by default @code{yyparse} returns after calling
2255 @code{yyerror}. This means that an erroneous input line causes the
2256 calculator program to exit. Now we show how to rectify this deficiency.
2258 The Bison language itself includes the reserved word @code{error}, which
2259 may be included in the grammar rules. In the example below it has
2260 been added to one of the alternatives for @code{line}:
2266 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2267 | error '\n' @{ yyerrok; @}
2272 This addition to the grammar allows for simple error recovery in the
2273 event of a syntax error. If an expression that cannot be evaluated is
2274 read, the error will be recognized by the third rule for @code{line},
2275 and parsing will continue. (The @code{yyerror} function is still called
2276 upon to print its message as well.) The action executes the statement
2277 @code{yyerrok}, a macro defined automatically by Bison; its meaning is
2278 that error recovery is complete (@pxref{Error Recovery}). Note the
2279 difference between @code{yyerrok} and @code{yyerror}; neither one is a
2282 This form of error recovery deals with syntax errors. There are other
2283 kinds of errors; for example, division by zero, which raises an exception
2284 signal that is normally fatal. A real calculator program must handle this
2285 signal and use @code{longjmp} to return to @code{main} and resume parsing
2286 input lines; it would also have to discard the rest of the current line of
2287 input. We won't discuss this issue further because it is not specific to
2290 @node Location Tracking Calc
2291 @section Location Tracking Calculator: @code{ltcalc}
2292 @cindex location tracking calculator
2293 @cindex @code{ltcalc}
2294 @cindex calculator, location tracking
2296 This example extends the infix notation calculator with location
2297 tracking. This feature will be used to improve the error messages. For
2298 the sake of clarity, this example is a simple integer calculator, since
2299 most of the work needed to use locations will be done in the lexical
2303 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
2304 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
2305 * Ltcalc Lexer:: The lexical analyzer.
2308 @node Ltcalc Declarations
2309 @subsection Declarations for @code{ltcalc}
2311 The C and Bison declarations for the location tracking calculator are
2312 the same as the declarations for the infix notation calculator.
2315 /* Location tracking calculator. */
2320 void yyerror (char const *);
2323 /* Bison declarations. */
2324 %define api.value.type @{int@}
2332 %% /* The grammar follows. */
2336 Note there are no declarations specific to locations. Defining a data type
2337 for storing locations is not needed: we will use the type provided by
2338 default (@pxref{Location Type}), which is a four member structure with the
2339 following integer fields: @code{first_line}, @code{first_column},
2340 @code{last_line} and @code{last_column}. By conventions, and in accordance
2341 with the GNU Coding Standards and common practice, the line and column count
2345 @subsection Grammar Rules for @code{ltcalc}
2347 Whether handling locations or not has no effect on the syntax of your
2348 language. Therefore, grammar rules for this example will be very close
2349 to those of the previous example: we will only modify them to benefit
2350 from the new information.
2352 Here, we will use locations to report divisions by zero, and locate the
2353 wrong expressions or subexpressions.
2366 | exp '\n' @{ printf ("%d\n", $1); @}
2373 | exp '+' exp @{ $$ = $1 + $3; @}
2374 | exp '-' exp @{ $$ = $1 - $3; @}
2375 | exp '*' exp @{ $$ = $1 * $3; @}
2385 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2386 @@3.first_line, @@3.first_column,
2387 @@3.last_line, @@3.last_column);
2392 | '-' exp %prec NEG @{ $$ = -$2; @}
2393 | exp '^' exp @{ $$ = pow ($1, $3); @}
2394 | '(' exp ')' @{ $$ = $2; @}
2398 This code shows how to reach locations inside of semantic actions, by
2399 using the pseudo-variables @code{@@@var{n}} for rule components, and the
2400 pseudo-variable @code{@@$} for groupings.
2402 We don't need to assign a value to @code{@@$}: the output parser does it
2403 automatically. By default, before executing the C code of each action,
2404 @code{@@$} is set to range from the beginning of @code{@@1} to the end of
2405 @code{@@@var{n}}, for a rule with @var{n} components. This behavior can be
2406 redefined (@pxref{Location Default Action}), and for very specific rules,
2407 @code{@@$} can be computed by hand.
2410 @subsection The @code{ltcalc} Lexical Analyzer.
2412 Until now, we relied on Bison's defaults to enable location
2413 tracking. The next step is to rewrite the lexical analyzer, and make it
2414 able to feed the parser with the token locations, as it already does for
2417 To this end, we must take into account every single character of the
2418 input text, to avoid the computed locations of being fuzzy or wrong:
2429 /* Skip white space. */
2430 while ((c = getchar ()) == ' ' || c == '\t')
2431 ++yylloc.last_column;
2436 yylloc.first_line = yylloc.last_line;
2437 yylloc.first_column = yylloc.last_column;
2441 /* Process numbers. */
2445 ++yylloc.last_column;
2446 while (isdigit (c = getchar ()))
2448 ++yylloc.last_column;
2449 yylval = yylval * 10 + c - '0';
2456 /* Return end-of-input. */
2461 /* Return a single char, and update location. */
2465 yylloc.last_column = 0;
2468 ++yylloc.last_column;
2474 Basically, the lexical analyzer performs the same processing as before: it
2475 skips blanks and tabs, and reads numbers or single-character tokens. In
2476 addition, it updates @code{yylloc}, the global variable (of type
2477 @code{YYLTYPE}) containing the token's location.
2479 Now, each time this function returns a token, the parser has its kind as
2480 well as its semantic value, and its location in the text. The last needed
2481 change is to initialize @code{yylloc}, for example in the controlling
2489 yylloc.first_line = yylloc.last_line = 1;
2490 yylloc.first_column = yylloc.last_column = 0;
2496 Remember that computing locations is not a matter of syntax. Every
2497 character must be associated to a location update, whether it is in
2498 valid input, in comments, in literal strings, and so on.
2500 @node Multi-function Calc
2501 @section Multi-Function Calculator: @code{mfcalc}
2502 @cindex multi-function calculator
2503 @cindex @code{mfcalc}
2504 @cindex calculator, multi-function
2506 Now that the basics of Bison have been discussed, it is time to move on to
2507 a more advanced problem. The above calculators provided only five
2508 functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2509 be nice to have a calculator that provides other mathematical functions such
2510 as @code{sin}, @code{cos}, etc.
2512 It is easy to add new operators to the infix calculator as long as they are
2513 only single-character literals. The lexical analyzer @code{yylex} passes
2514 back all nonnumeric characters as tokens, so new grammar rules suffice for
2515 adding a new operator. But we want something more flexible: built-in
2516 functions whose syntax has this form:
2519 @var{function_name} (@var{argument})
2523 At the same time, we will add memory to the calculator, by allowing you
2524 to create named variables, store values in them, and use them later.
2525 Here is a sample session with the multi-function calculator:
2530 @kbd{pi = 3.141592653589}
2531 @result{} 3.1415926536
2535 @result{} 0.0000000000
2537 @kbd{alpha = beta1 = 2.3}
2538 @result{} 2.3000000000
2540 @result{} 2.3000000000
2542 @result{} 0.8329091229
2543 @kbd{exp(ln(beta1))}
2544 @result{} 2.3000000000
2548 Note that multiple assignment and nested function calls are permitted.
2551 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
2552 * Mfcalc Rules:: Grammar rules for the calculator.
2553 * Mfcalc Symbol Table:: Symbol table management subroutines.
2554 * Mfcalc Lexer:: The lexical analyzer.
2555 * Mfcalc Main:: The controlling function.
2558 @node Mfcalc Declarations
2559 @subsection Declarations for @code{mfcalc}
2561 Here are the C and Bison declarations for the multi-function calculator.
2564 @comment file: c/mfcalc/mfcalc.y
2566 /* Parser for mfcalc. -*- C -*-
2568 Copyright (C) 1988-1993, 1995, 1998-2015, 2018-2021 Free Software
2571 This file is part of Bison, the GNU Compiler Compiler.
2573 This program is free software: you can redistribute it and/or modify
2574 it under the terms of the GNU General Public License as published by
2575 the Free Software Foundation, either version 3 of the License, or
2576 (at your option) any later version.
2578 This program is distributed in the hope that it will be useful,
2579 but WITHOUT ANY WARRANTY; without even the implied warranty of
2580 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2581 GNU General Public License for more details.
2583 You should have received a copy of the GNU General Public License
2584 along with this program. If not, see <http://www.gnu.org/licenses/>. */
2588 @comment file: c/mfcalc/mfcalc.y: 1
2592 #include <stdio.h> /* For printf, etc. */
2593 #include <math.h> /* For pow, used in the grammar. */
2594 #include "calc.h" /* Contains definition of 'symrec'. */
2596 void yyerror (char const *);
2600 %define api.value.type union /* Generate YYSTYPE from these types: */
2601 %token <double> NUM /* Double precision number. */
2602 %token <symrec*> VAR FUN /* Symbol table pointer: variable/function. */
2609 %precedence NEG /* negation--unary minus */
2610 %right '^' /* exponentiation */
2614 The above grammar introduces only two new features of the Bison language.
2615 These features allow semantic values to have various data types
2616 (@pxref{Multiple Types}).
2618 The special @code{union} value assigned to the @code{%define} variable
2619 @code{api.value.type} specifies that the symbols are defined with their data
2620 types. Bison will generate an appropriate definition of @code{YYSTYPE} to
2623 Since values can now have various types, it is necessary to associate a type
2624 with each grammar symbol whose semantic value is used. These symbols are
2625 @code{NUM}, @code{VAR}, @code{FUN}, and @code{exp}. Their declarations are
2626 augmented with their data type (placed between angle brackets). For
2627 instance, values of @code{NUM} are stored in @code{double}.
2629 The Bison construct @code{%nterm} is used for declaring nonterminal symbols,
2630 just as @code{%token} is used for declaring token kinds. Previously we did
2631 not use @code{%nterm} before because nonterminal symbols are normally
2632 declared implicitly by the rules that define them. But @code{exp} must be
2633 declared explicitly so we can specify its value type. @xref{Type Decl}.
2636 @subsection Grammar Rules for @code{mfcalc}
2638 Here are the grammar rules for the multi-function calculator.
2639 Most of them are copied directly from @code{calc}; three rules,
2640 those which mention @code{VAR} or @code{FUN}, are new.
2642 @comment file: c/mfcalc/mfcalc.y: 3
2644 %% /* The grammar follows. */
2655 | exp '\n' @{ printf ("%.10g\n", $1); @}
2656 | error '\n' @{ yyerrok; @}
2663 | VAR @{ $$ = $1->value.var; @}
2664 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2665 | FUN '(' exp ')' @{ $$ = $1->value.fun ($3); @}
2666 | exp '+' exp @{ $$ = $1 + $3; @}
2667 | exp '-' exp @{ $$ = $1 - $3; @}
2668 | exp '*' exp @{ $$ = $1 * $3; @}
2669 | exp '/' exp @{ $$ = $1 / $3; @}
2670 | '-' exp %prec NEG @{ $$ = -$2; @}
2671 | exp '^' exp @{ $$ = pow ($1, $3); @}
2672 | '(' exp ')' @{ $$ = $2; @}
2675 /* End of grammar. */
2679 @node Mfcalc Symbol Table
2680 @subsection The @code{mfcalc} Symbol Table
2681 @cindex symbol table example
2683 The multi-function calculator requires a symbol table to keep track of the
2684 names and meanings of variables and functions. This doesn't affect the
2685 grammar rules (except for the actions) or the Bison declarations, but it
2686 requires some additional C functions for support.
2688 The symbol table itself consists of a linked list of records. Its
2689 definition, which is kept in the header @file{calc.h}, is as follows. It
2690 provides for either functions or variables to be placed in the table.
2693 @comment file: c/mfcalc/calc.h
2695 /* Functions for mfcalc. -*- C -*-
2697 Copyright (C) 1988-1993, 1995, 1998-2015, 2018-2021 Free Software
2700 This file is part of Bison, the GNU Compiler Compiler.
2702 This program is free software: you can redistribute it and/or modify
2703 it under the terms of the GNU General Public License as published by
2704 the Free Software Foundation, either version 3 of the License, or
2705 (at your option) any later version.
2707 This program is distributed in the hope that it will be useful,
2708 but WITHOUT ANY WARRANTY; without even the implied warranty of
2709 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2710 GNU General Public License for more details.
2712 You should have received a copy of the GNU General Public License
2713 along with this program. If not, see <http://www.gnu.org/licenses/>. */
2717 @comment file: c/mfcalc/calc.h
2720 /* Function type. */
2721 typedef double (func_t) (double);
2725 /* Data type for links in the chain of symbols. */
2728 char *name; /* name of symbol */
2729 int type; /* type of symbol: either VAR or FUN */
2732 double var; /* value of a VAR */
2733 func_t *fun; /* value of a FUN */
2735 struct symrec *next; /* link field */
2740 typedef struct symrec symrec;
2742 /* The symbol table: a chain of 'struct symrec'. */
2743 extern symrec *sym_table;
2745 symrec *putsym (char const *name, int sym_type);
2746 symrec *getsym (char const *name);
2750 The new version of @code{main} will call @code{init_table} to initialize
2753 @comment file: c/mfcalc/mfcalc.y: 3
2764 struct init const funs[] =
2777 /* The symbol table: a chain of 'struct symrec'. */
2782 /* Put functions in table. */
2788 for (int i = 0; funs[i].name; i++)
2790 symrec *ptr = putsym (funs[i].name, FUN);
2791 ptr->value.fun = funs[i].fun;
2797 By simply editing the initialization list and adding the necessary include
2798 files, you can add additional functions to the calculator.
2800 Two important functions allow look-up and installation of symbols in the
2801 symbol table. The function @code{putsym} is passed a name and the kind
2802 (@code{VAR} or @code{FUN}) of the object to be installed. The object is
2803 linked to the front of the list, and a pointer to the object is returned.
2804 The function @code{getsym} is passed the name of the symbol to look up. If
2805 found, a pointer to that symbol is returned; otherwise zero is returned.
2807 @comment file: c/mfcalc/mfcalc.y: 3
2810 /* The mfcalc code assumes that malloc and realloc
2811 always succeed, and that integer calculations
2812 never overflow. Production-quality code should
2813 not make these assumptions. */
2815 #include <stdlib.h> /* malloc, realloc. */
2816 #include <string.h> /* strlen. */
2821 putsym (char const *name, int sym_type)
2823 symrec *res = (symrec *) malloc (sizeof (symrec));
2824 res->name = strdup (name);
2825 res->type = sym_type;
2826 res->value.var = 0; /* Set value to 0 even if fun. */
2827 res->next = sym_table;
2835 getsym (char const *name)
2837 for (symrec *p = sym_table; p; p = p->next)
2838 if (strcmp (p->name, name) == 0)
2846 @subsection The @code{mfcalc} Lexer
2848 The function @code{yylex} must now recognize variables, numeric values, and
2849 the single-character arithmetic operators. Strings of alphanumeric
2850 characters with a leading letter are recognized as either variables or
2851 functions depending on what the symbol table says about them.
2853 The string is passed to @code{getsym} for look up in the symbol table. If
2854 the name appears in the table, a pointer to its location and its type
2855 (@code{VAR} or @code{FUN}) is returned to @code{yyparse}. If it is not
2856 already in the table, then it is installed as a @code{VAR} using
2857 @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2858 returned to @code{yyparse}.
2860 No change is needed in the handling of numeric values and arithmetic
2861 operators in @code{yylex}.
2863 @comment file: c/mfcalc/mfcalc.y: 3
2874 /* Ignore white space, get first nonwhite character. */
2875 while (c == ' ' || c == '\t')
2883 /* Char starts a number => parse the number. */
2884 if (c == '.' || isdigit (c))
2887 if (scanf ("%lf", &yylval.NUM) != 1)
2895 Bison generated a definition of @code{YYSTYPE} with a member named
2896 @code{NUM} to store value of @code{NUM} symbols.
2898 @comment file: c/mfcalc/mfcalc.y: 3
2901 /* Char starts an identifier => read the name. */
2904 static ptrdiff_t bufsize = 0;
2905 static char *symbuf = 0;
2911 /* If buffer is full, make it bigger. */
2914 bufsize = 2 * bufsize + 40;
2915 symbuf = realloc (symbuf, (size_t) bufsize);
2917 /* Add this character to the buffer. */
2918 symbuf[i++] = (char) c;
2919 /* Get another character. */
2924 while (isalnum (c));
2931 symrec *s = getsym (symbuf);
2933 s = putsym (symbuf, VAR);
2934 yylval.VAR = s; /* or yylval.FUN = s. */
2938 /* Any other character is a token by itself. */
2945 @subsection The @code{mfcalc} Main
2947 The error reporting function is unchanged, and the new version of
2948 @code{main} includes a call to @code{init_table} and sets the @code{yydebug}
2949 on user demand (@xref{Tracing}, for details):
2951 @comment file: c/mfcalc/mfcalc.y: 3
2954 /* Called by yyparse on error. */
2955 void yyerror (char const *s)
2957 fprintf (stderr, "%s\n", s);
2962 int main (int argc, char const* argv[])
2966 /* Enable parse traces on option -p. */
2967 if (argc == 2 && strcmp(argv[1], "-p") == 0)
2977 This program is both powerful and flexible. You may easily add new
2978 functions, and it is a simple job to modify this code to install
2979 predefined variables such as @code{pi} or @code{e} as well.
2987 Add some new functions from @file{math.h} to the initialization list.
2990 Add another array that contains constants and their values. Then modify
2991 @code{init_table} to add these constants to the symbol table. It will be
2992 easiest to give the constants type @code{VAR}.
2995 Make the program report an error if the user refers to an uninitialized
2996 variable in any way except to store a value in it.
3000 @chapter Bison Grammar Files
3002 Bison takes as input a context-free grammar specification and produces a
3003 C-language function that recognizes correct instances of the grammar.
3005 The Bison grammar file conventionally has a name ending in @samp{.y}.
3009 * Grammar Outline:: Overall layout of the grammar file.
3010 * Symbols:: Terminal and nonterminal symbols.
3011 * Rules:: How to write grammar rules.
3012 * Semantics:: Semantic values and actions.
3013 * Tracking Locations:: Locations and actions.
3014 * Named References:: Using named references in actions.
3015 * Declarations:: All kinds of Bison declarations are described here.
3016 * Multiple Parsers:: Putting more than one Bison parser in one program.
3019 @node Grammar Outline
3020 @section Outline of a Bison Grammar
3023 @findex /* @dots{} */
3025 A Bison grammar file has four main sections, shown here with the
3026 appropriate delimiters:
3033 @var{Bison declarations}
3042 Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
3043 As a GNU extension, @samp{//} introduces a comment that continues until end
3047 * Prologue:: Syntax and usage of the prologue.
3048 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
3049 * Bison Declarations:: Syntax and usage of the Bison declarations section.
3050 * Grammar Rules:: Syntax and usage of the grammar rules section.
3051 * Epilogue:: Syntax and usage of the epilogue.
3055 @subsection The prologue
3056 @cindex declarations section
3058 @cindex declarations
3060 The @var{Prologue} section contains macro definitions and declarations of
3061 functions and variables that are used in the actions in the grammar rules.
3062 These are copied to the beginning of the parser implementation file so that
3063 they precede the definition of @code{yyparse}. You can use @samp{#include}
3064 to get the declarations from a header file. If you don't need any C
3065 declarations, you may omit the @samp{%@{} and @samp{%@}} delimiters that
3066 bracket this section.
3068 The @var{Prologue} section is terminated by the first occurrence of
3069 @samp{%@}} that is outside a comment, a string literal, or a character
3072 You may have more than one @var{Prologue} section, intermixed with the
3073 @var{Bison declarations}. This allows you to have C and Bison declarations
3074 that refer to each other. For example, the @code{%union} declaration may
3075 use types defined in a header file, and you may wish to prototype functions
3076 that take arguments of type @code{YYSTYPE}. This can be done with two
3077 @var{Prologue} blocks, one before and one after the @code{%union}
3092 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3098 static void print_token (yytoken_kind_t token, YYSTYPE val);
3105 When in doubt, it is usually safer to put prologue code before all Bison
3106 declarations, rather than after. For example, any definitions of feature
3107 test macros like @code{_GNU_SOURCE} or @code{_POSIX_C_SOURCE} should appear
3108 before all Bison declarations, as feature test macros can affect the
3109 behavior of Bison-generated @code{#include} directives.
3111 @node Prologue Alternatives
3112 @subsection Prologue Alternatives
3113 @cindex Prologue Alternatives
3116 @findex %code requires
3117 @findex %code provides
3120 The functionality of @var{Prologue} sections can often be subtle and
3121 inflexible. As an alternative, Bison provides a @code{%code} directive with
3122 an explicit qualifier field, which identifies the purpose of the code and
3123 thus the location(s) where Bison should generate it. For C/C++, the
3124 qualifier can be omitted for the default location, or it can be one of
3125 @code{requires}, @code{provides}, @code{top}. @xref{%code Summary}.
3127 Look again at the example of the previous section:
3141 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3147 static void print_token (yytoken_kind_t token, YYSTYPE val);
3155 Notice that there are two @var{Prologue} sections here, but there's a subtle
3156 distinction between their functionality. For example, if you decide to
3157 override Bison's default definition for @code{YYLTYPE}, in which
3158 @var{Prologue} section should you write your new definition? You should
3159 write it in the first since Bison will insert that code into the parser
3160 implementation file @emph{before} the default @code{YYLTYPE} definition. In
3161 which @var{Prologue} section should you prototype an internal function,
3162 @code{trace_token}, that accepts @code{YYLTYPE} and @code{yytoken_kind_t} as
3163 arguments? You should prototype it in the second since Bison will insert
3164 that code @emph{after} the @code{YYLTYPE} and @code{yytoken_kind_t}
3167 This distinction in functionality between the two @var{Prologue} sections is
3168 established by the appearance of the @code{%union} between them. This
3169 behavior raises a few questions. First, why should the position of a
3170 @code{%union} affect definitions related to @code{YYLTYPE} and
3171 @code{yytoken_kind_t}? Second, what if there is no @code{%union}? In that
3172 case, the second kind of @var{Prologue} section is not available. This
3173 behavior is not intuitive.
3175 To avoid this subtle @code{%union} dependency, rewrite the example using a
3176 @code{%code top} and an unqualified @code{%code}. Let's go ahead and add
3177 the new @code{YYLTYPE} definition and the @code{trace_token} prototype at
3185 /* WARNING: The following code really belongs
3186 * in a '%code requires'; see below. */
3189 #define YYLTYPE YYLTYPE
3190 typedef struct YYLTYPE
3203 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3209 static void print_token (yytoken_kind_t token, YYSTYPE val);
3210 static void trace_token (yytoken_kind_t token, YYLTYPE loc);
3218 In this way, @code{%code top} and the unqualified @code{%code} achieve the
3219 same functionality as the two kinds of @var{Prologue} sections, but it's
3220 always explicit which kind you intend. Moreover, both kinds are always
3221 available even in the absence of @code{%union}.
3223 The @code{%code top} block above logically contains two parts. The first
3224 two lines before the warning need to appear near the top of the parser
3225 implementation file. The first line after the warning is required by
3226 @code{YYSTYPE} and thus also needs to appear in the parser implementation
3227 file. However, if you've instructed Bison to generate a parser header file
3228 (@pxref{Decl Summary}), you probably want that line to appear
3229 before the @code{YYSTYPE} definition in that header file as well. The
3230 @code{YYLTYPE} definition should also appear in the parser header file to
3231 override the default @code{YYLTYPE} definition there.
3233 In other words, in the @code{%code top} block above, all but the first two
3234 lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
3236 Thus, they belong in one or more @code{%code requires}:
3254 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3260 #define YYLTYPE YYLTYPE
3261 typedef struct YYLTYPE
3274 static void print_token (yytoken_kind_t token, YYSTYPE val);
3275 static void trace_token (yytoken_kind_t token, YYLTYPE loc);
3283 Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
3284 definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
3285 definitions in both the parser implementation file and the parser header
3286 file. (By the same reasoning, @code{%code requires} would also be the
3287 appropriate place to write your own definition for @code{YYSTYPE}.)
3289 When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE},
3290 you should prefer @code{%code requires} over @code{%code top} regardless of
3291 whether you instruct Bison to generate a parser header file. When you are
3292 writing code that you need Bison to insert only into the parser
3293 implementation file and that has no special need to appear at the top of
3294 that file, you should prefer the unqualified @code{%code} over @code{%code
3295 top}. These practices will make the purpose of each block of your code
3296 explicit to Bison and to other developers reading your grammar file.
3297 Following these practices, we expect the unqualified @code{%code} and
3298 @code{%code requires} to be the most important of the four @var{Prologue}
3301 At some point while developing your parser, you might decide to provide
3302 @code{trace_token} to modules that are external to your parser. Thus, you
3303 might wish for Bison to insert the prototype into both the parser header
3304 file and the parser implementation file. Since this function is not a
3305 dependency required by @code{YYSTYPE} or @code{YYLTYPE}, it doesn't make
3306 sense to move its prototype to a @code{%code requires}. More importantly,
3307 since it depends upon @code{YYLTYPE} and @code{yytoken_kind_t}, @code{%code
3308 requires} is not sufficient. Instead, move its prototype from the
3309 unqualified @code{%code} to a @code{%code provides}:
3327 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3333 #define YYLTYPE YYLTYPE
3334 typedef struct YYLTYPE
3347 void trace_token (yytoken_kind_t token, YYLTYPE loc);
3353 static void print_token (FILE *file, int token, YYSTYPE val);
3361 Bison will insert the @code{trace_token} prototype into both the parser
3362 header file and the parser implementation file after the definitions for
3363 @code{yytoken_kind_t}, @code{YYLTYPE}, and @code{YYSTYPE}.
3365 The above examples are careful to write directives in an order that reflects
3366 the layout of the generated parser implementation and header files:
3367 @code{%code top}, @code{%code requires}, @code{%code provides}, and then
3368 @code{%code}. While your grammar files may generally be easier to read if
3369 you also follow this order, Bison does not require it. Instead, Bison lets
3370 you choose an organization that makes sense to you.
3372 You may declare any of these directives multiple times in the grammar file.
3373 In that case, Bison concatenates the contained code in declaration order.
3374 This is the only way in which the position of one of these directives within
3375 the grammar file affects its functionality.
3377 The result of the previous two properties is greater flexibility in how you may
3378 organize your grammar file.
3379 For example, you may organize semantic-type-related directives by semantic
3384 %code requires @{ #include "type1.h" @}
3385 %union @{ type1 field1; @}
3386 %destructor @{ type1_free ($$); @} <field1>
3387 %printer @{ type1_print (yyo, $$); @} <field1>
3391 %code requires @{ #include "type2.h" @}
3392 %union @{ type2 field2; @}
3393 %destructor @{ type2_free ($$); @} <field2>
3394 %printer @{ type2_print (yyo, $$); @} <field2>
3399 You could even place each of the above directive groups in the rules section of
3400 the grammar file next to the set of rules that uses the associated semantic
3402 (In the rules section, you must terminate each of those directives with a
3404 And you don't have to worry that some directive (like a @code{%union}) in the
3405 definitions section is going to adversely affect their functionality in some
3406 counter-intuitive manner just because it comes first.
3407 Such an organization is not possible using @var{Prologue} sections.
3409 This section has been concerned with explaining the advantages of the four
3410 @var{Prologue} alternatives over the original Yacc @var{Prologue}.
3411 However, in most cases when using these directives, you shouldn't need to
3412 think about all the low-level ordering issues discussed here.
3413 Instead, you should simply use these directives to label each block of your
3414 code according to its purpose and let Bison handle the ordering.
3415 @code{%code} is the most generic label.
3416 Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
3419 @node Bison Declarations
3420 @subsection The Bison Declarations Section
3421 @cindex Bison declarations (introduction)
3422 @cindex declarations, Bison (introduction)
3424 The @var{Bison declarations} section contains declarations that define
3425 terminal and nonterminal symbols, specify precedence, and so on.
3426 In some simple grammars you may not need any declarations.
3427 @xref{Declarations}.
3430 @subsection The Grammar Rules Section
3431 @cindex grammar rules section
3432 @cindex rules section for grammar
3434 The @dfn{grammar rules} section contains one or more Bison grammar
3435 rules, and nothing else. @xref{Rules}.
3437 There must always be at least one grammar rule, and the first
3438 @samp{%%} (which precedes the grammar rules) may never be omitted even
3439 if it is the first thing in the file.
3442 @subsection The epilogue
3443 @cindex additional C code section
3445 @cindex C code, section for additional
3447 The @var{Epilogue} is copied verbatim to the end of the parser
3448 implementation file, just as the @var{Prologue} is copied to the
3449 beginning. This is the most convenient place to put anything that you
3450 want to have in the parser implementation file but which need not come
3451 before the definition of @code{yyparse}. For example, the definitions
3452 of @code{yylex} and @code{yyerror} often go here. Because C requires
3453 functions to be declared before being used, you often need to declare
3454 functions like @code{yylex} and @code{yyerror} in the Prologue, even
3455 if you define them in the Epilogue. @xref{Interface}.
3457 If the last section is empty, you may omit the @samp{%%} that separates it
3458 from the grammar rules.
3460 The Bison parser itself contains many macros and identifiers whose names
3461 start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3462 any such names (except those documented in this manual) in the epilogue
3463 of the grammar file.
3466 @section Symbols, Terminal and Nonterminal
3467 @cindex nonterminal symbol
3468 @cindex terminal symbol
3472 @dfn{Symbols} in Bison grammars represent the grammatical classifications
3475 A @dfn{terminal symbol} (also known as a @dfn{token kind}) represents a
3476 class of syntactically equivalent tokens. You use the symbol in grammar
3477 rules to mean that a token in that class is allowed. The symbol is
3478 represented in the Bison parser by a numeric code, and the @code{yylex}
3479 function returns a token kind code to indicate what kind of token has been
3480 read. You don't need to know what the code value is; you can use the symbol
3483 A @dfn{nonterminal symbol} stands for a class of syntactically
3484 equivalent groupings. The symbol name is used in writing grammar rules.
3485 By convention, it should be all lower case.
3487 Symbol names can contain letters, underscores, periods, and non-initial
3488 digits and dashes. Dashes in symbol names are a GNU extension, incompatible
3489 with POSIX Yacc. Periods and dashes make symbol names less convenient to
3490 use with named references, which require brackets around such names
3491 (@pxref{Named References}). Terminal symbols that contain periods or dashes
3492 make little sense: since they are not valid symbols (in most programming
3493 languages) they are not exported as token names.
3495 There are three ways of writing terminal symbols in the grammar:
3499 A @dfn{named token kind} is written with an identifier, like an identifier
3500 in C@. By convention, it should be all upper case. Each such name must be
3501 defined with a Bison declaration such as @code{%token}. @xref{Token Decl}.
3504 @cindex character token
3505 @cindex literal token
3506 @cindex single-character literal
3507 A @dfn{character token kind} (or @dfn{literal character token}) is written
3508 in the grammar using the same syntax used in C for character constants; for
3509 example, @code{'+'} is a character token kind. A character token kind
3510 doesn't need to be declared unless you need to specify its semantic value
3511 data type (@pxref{Value Type}), associativity, or precedence
3512 (@pxref{Precedence}).
3514 By convention, a character token kind is used only to represent a token that
3515 consists of that particular character. Thus, the token kind @code{'+'} is
3516 used to represent the character @samp{+} as a token. Nothing enforces this
3517 convention, but if you depart from it, your program will confuse other
3520 All the usual escape sequences used in character literals in C can be used
3521 in Bison as well, but you must not use the null character as a character
3522 literal because its numeric code, zero, signifies end-of-input
3523 (@pxref{Calling Convention}). Also, unlike standard C, trigraphs have no
3524 special meaning in Bison character literals, nor is backslash-newline
3528 @cindex string token
3529 @cindex literal string token
3530 @cindex multicharacter literal
3531 A @dfn{literal string token} is written like a C string constant; for
3532 example, @code{"<="} is a literal string token. A literal string token
3533 doesn't need to be declared unless you need to specify its semantic
3534 value data type (@pxref{Value Type}), associativity, or precedence
3535 (@pxref{Precedence}).
3537 You can associate the literal string token with a symbolic name as an alias,
3538 using the @code{%token} declaration (@pxref{Token Decl}). If you don't do
3539 that, the lexical analyzer has to retrieve the token code for the literal
3540 string token from the @code{yytname} table (@pxref{Calling Convention}).
3542 @strong{Warning}: literal string tokens do not work in Yacc.
3544 By convention, a literal string token is used only to represent a token
3545 that consists of that particular string. Thus, you should use the token
3546 kind @code{"<="} to represent the string @samp{<=} as a token. Bison
3547 does not enforce this convention, but if you depart from it, people who
3548 read your program will be confused.
3550 All the escape sequences used in string literals in C can be used in
3551 Bison as well, except that you must not use a null character within a
3552 string literal. Also, unlike Standard C, trigraphs have no special
3553 meaning in Bison string literals, nor is backslash-newline allowed. A
3554 literal string token must contain two or more characters; for a token
3555 containing just one character, use a character token (see above).
3558 How you choose to write a terminal symbol has no effect on its
3559 grammatical meaning. That depends only on where it appears in rules and
3560 on when the parser function returns that symbol.
3562 The value returned by @code{yylex} is always one of the terminal
3563 symbols, except that a zero or negative value signifies end-of-input.
3564 Whichever way you write the token kind in the grammar rules, you write
3565 it the same way in the definition of @code{yylex}. The numeric code
3566 for a character token kind is simply the positive numeric code of the
3567 character, so @code{yylex} can use the identical value to generate the
3568 requisite code, though you may need to convert it to @code{unsigned
3569 char} to avoid sign-extension on hosts where @code{char} is signed.
3570 Each named token kind becomes a C macro in the parser implementation
3571 file, so @code{yylex} can use the name to stand for the code. (This
3572 is why periods don't make sense in terminal symbols.) @xref{Calling
3575 If @code{yylex} is defined in a separate file, you need to arrange for the
3576 token-kind definitions to be available there. Use the @option{-d} option
3577 when you run Bison, so that it will write these definitions into a separate
3578 header file @file{@var{name}.tab.h} which you can include in the other
3579 source files that need it. @xref{Invocation}.
3581 If you want to write a grammar that is portable to any Standard C
3582 host, you must use only nonnull character tokens taken from the basic
3583 execution character set of Standard C@. This set consists of the ten
3584 digits, the 52 lower- and upper-case English letters, and the
3585 characters in the following C-language string:
3588 "\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3591 The @code{yylex} function and Bison must use a consistent character set
3592 and encoding for character tokens. For example, if you run Bison in an
3593 ASCII environment, but then compile and run the resulting
3594 program in an environment that uses an incompatible character set like
3595 EBCDIC, the resulting program may not work because the tables
3596 generated by Bison will assume ASCII numeric values for
3597 character tokens. It is standard practice for software distributions to
3598 contain C source files that were generated by Bison in an
3599 ASCII environment, so installers on platforms that are
3600 incompatible with ASCII must rebuild those files before
3603 The symbol @code{error} is a terminal symbol reserved for error recovery
3604 (@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3605 In particular, @code{yylex} should never return this value. The default
3606 value of the error token is 256, unless you explicitly assigned 256 to
3607 one of your tokens with a @code{%token} declaration.
3610 @section Grammar Rules
3612 A Bison grammar is a list of rules.
3615 * Rules Syntax:: Syntax of the rules.
3616 * Empty Rules:: Symbols that can match the empty string.
3617 * Recursion:: Writing recursive rules.
3621 @subsection Syntax of Grammar Rules
3623 @cindex grammar rule syntax
3624 @cindex syntax of grammar rules
3626 A Bison grammar rule has the following general form:
3629 @var{result}: @var{components}@dots{};
3633 where @var{result} is the nonterminal symbol that this rule describes,
3634 and @var{components} are various terminal and nonterminal symbols that
3635 are put together by this rule (@pxref{Symbols}).
3644 says that two groupings of type @code{exp}, with a @samp{+} token in between,
3645 can be combined into a larger grouping of type @code{exp}.
3647 White space in rules is significant only to separate symbols. You can add
3648 extra white space as you wish.
3650 Scattered among the components can be @var{actions} that determine
3651 the semantics of the rule. An action looks like this:
3654 @{@var{C statements}@}
3659 This is an example of @dfn{braced code}, that is, C code surrounded by
3660 braces, much like a compound statement in C@. Braced code can contain
3661 any sequence of C tokens, so long as its braces are balanced. Bison
3662 does not check the braced code for correctness directly; it merely
3663 copies the code to the parser implementation file, where the C
3664 compiler can check it.
3666 Within braced code, the balanced-brace count is not affected by braces
3667 within comments, string literals, or character constants, but it is
3668 affected by the C digraphs @samp{<%} and @samp{%>} that represent
3669 braces. At the top level braced code must be terminated by @samp{@}}
3670 and not by a digraph. Bison does not look for trigraphs, so if braced
3671 code uses trigraphs you should ensure that they do not affect the
3672 nesting of braces or the boundaries of comments, string literals, or
3673 character constants.
3675 Usually there is only one action and it follows the components.
3679 Multiple rules for the same @var{result} can be written separately or can
3680 be joined with the vertical-bar character @samp{|} as follows:
3685 @var{rule1-components}@dots{}
3686 | @var{rule2-components}@dots{}
3693 They are still considered distinct rules even when joined in this way.
3696 @subsection Empty Rules
3701 A rule is said to be @dfn{empty} if its right-hand side (@var{components})
3702 is empty. It means that @var{result} in the previous example can match the
3703 empty string. As another example, here is how to define an optional
3707 semicolon.opt: | ";";
3711 It is easy not to see an empty rule, especially when @code{|} is used. The
3712 @code{%empty} directive allows to make explicit that a rule is empty on
3724 Flagging a non-empty rule with @code{%empty} is an error. If run with
3725 @option{-Wempty-rule}, @command{bison} will report empty rules without
3726 @code{%empty}. Using @code{%empty} enables this warning, unless
3727 @option{-Wno-empty-rule} was specified.
3729 The @code{%empty} directive is a Bison extension, it does not work with
3730 Yacc. To remain compatible with POSIX Yacc, it is customary to write a
3731 comment @samp{/* empty */} in each rule with no components:
3744 @subsection Recursive Rules
3745 @cindex recursive rule
3746 @cindex rule, recursive
3748 A rule is called @dfn{recursive} when its @var{result} nonterminal
3749 appears also on its right hand side. Nearly all Bison grammars need to
3750 use recursion, because that is the only way to define a sequence of any
3751 number of a particular thing. Consider this recursive definition of a
3752 comma-separated sequence of one or more expressions:
3763 @cindex left recursion
3764 @cindex right recursion
3766 Since the recursive use of @code{expseq1} is the leftmost symbol in the
3767 right hand side, we call this @dfn{left recursion}. By contrast, here
3768 the same construct is defined using @dfn{right recursion}:
3780 Any kind of sequence can be defined using either left recursion or right
3781 recursion, but you should always use left recursion, because it can
3782 parse a sequence of any number of elements with bounded stack space.
3783 Right recursion uses up space on the Bison stack in proportion to the
3784 number of elements in the sequence, because all the elements must be
3785 shifted onto the stack before the rule can be applied even once.
3786 @xref{Algorithm}, for further explanation
3789 @cindex mutual recursion
3790 @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3791 rule does not appear directly on its right hand side, but does appear
3792 in rules for other nonterminals which do appear on its right hand
3801 | primary '+' primary
3814 defines two mutually-recursive nonterminals, since each refers to the
3818 @section Defining Language Semantics
3819 @cindex defining language semantics
3820 @cindex language semantics, defining
3822 The grammar rules for a language determine only the syntax. The semantics
3823 are determined by the semantic values associated with various tokens and
3824 groupings, and by the actions taken when various groupings are recognized.
3826 For example, the calculator calculates properly because the value
3827 associated with each expression is the proper number; it adds properly
3828 because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3829 the numbers associated with @var{x} and @var{y}.
3832 * Value Type:: Specifying one data type for all semantic values.
3833 * Multiple Types:: Specifying several alternative data types.
3834 * Type Generation:: Generating the semantic value type.
3835 * Union Decl:: Declaring the set of all semantic value types.
3836 * Structured Value Type:: Providing a structured semantic value type.
3837 * Actions:: An action is the semantic definition of a grammar rule.
3838 * Action Types:: Specifying data types for actions to operate on.
3839 * Midrule Actions:: Most actions go at the end of a rule.
3840 This says when, why and how to use the exceptional
3841 action in the middle of a rule.
3845 @subsection Data Types of Semantic Values
3846 @cindex semantic value type
3847 @cindex value type, semantic
3848 @cindex data types of semantic values
3849 @cindex default data type
3851 In a simple program it may be sufficient to use the same data type for
3852 the semantic values of all language constructs. This was true in the
3853 RPN and infix calculator examples (@pxref{RPN Calc}).
3855 Bison normally uses the type @code{int} for semantic values if your program
3856 uses the same data type for all language constructs. To specify some other
3857 type, define the @code{%define} variable @code{api.value.type} like this:
3860 %define api.value.type @{double@}
3867 %define api.value.type @{struct semantic_value_type@}
3870 The value of @code{api.value.type} should be a type name that does not
3871 contain parentheses or square brackets.
3873 Alternatively, instead of relying of Bison's @code{%define} support, you may
3874 rely on the C/C++ preprocessor and define @code{YYSTYPE} as a macro, like
3878 #define YYSTYPE double
3882 This macro definition must go in the prologue of the grammar file
3883 (@pxref{Grammar Outline}). If compatibility with POSIX Yacc matters to you,
3884 use this. Note however that Bison cannot know @code{YYSTYPE}'s value, not
3885 even whether it is defined, so there are services it cannot provide.
3886 Besides this works only for languages that have a preprocessor.
3888 @node Multiple Types
3889 @subsection More Than One Value Type
3891 In most programs, you will need different data types for different kinds
3892 of tokens and groupings. For example, a numeric constant may need type
3893 @code{int} or @code{long}, while a string constant needs type
3894 @code{char *}, and an identifier might need a pointer to an entry in the
3897 To use more than one data type for semantic values in one parser, Bison
3898 requires you to do two things:
3902 Specify the entire collection of possible data types. There are several
3906 let Bison compute the union type from the tags you assign to symbols;
3909 use the @code{%union} Bison declaration (@pxref{Union Decl});
3912 define the @code{%define} variable @code{api.value.type} to be a union type
3913 whose members are the type tags (@pxref{Structured Value Type});
3916 use a @code{typedef} or a @code{#define} to define @code{YYSTYPE} to be a
3917 union type whose member names are the type tags.
3921 Choose one of those types for each symbol (terminal or nonterminal) for
3922 which semantic values are used. This is done for tokens with the
3923 @code{%token} Bison declaration (@pxref{Token Decl}) and
3924 for groupings with the @code{%nterm}/@code{%type} Bison declarations
3925 (@pxref{Type Decl}).
3928 @node Type Generation
3929 @subsection Generating the Semantic Value Type
3930 @cindex declaring value types
3931 @cindex value types, declaring
3932 @findex %define api.value.type union
3934 The special value @code{union} of the @code{%define} variable
3935 @code{api.value.type} instructs Bison that the type tags (used with the
3936 @code{%token}, @code{%nterm} and @code{%type} directives) are genuine types,
3937 not names of members of @code{YYSTYPE}.
3942 %define api.value.type union
3943 %token <int> INT "integer"
3946 %token <char const *> ID "identifier"
3950 generates an appropriate value of @code{YYSTYPE} to support each symbol
3951 type. The name of the member of @code{YYSTYPE} for tokens than have a
3952 declared identifier @var{id} (such as @code{INT} and @code{ID} above, but
3953 not @code{'n'}) is @code{@var{id}}. The other symbols have unspecified
3954 names on which you should not depend; instead, relying on C casts to access
3955 the semantic value with the appropriate type:
3958 /* For an "integer". */
3962 /* For an 'n', also declared as int. */
3963 *((int*)&yylval) = 42;
3966 /* For an "identifier". */
3971 If the @code{%define} variable @code{api.token.prefix} is defined
3972 (@pxref{%define Summary}), then it is also used to prefix
3973 the union member names. For instance, with @samp{%define api.token.prefix
3977 /* For an "integer". */
3978 yylval.TOK_INT = 42;
3982 This Bison extension cannot work if @code{%yacc} (or
3983 @option{-y}/@option{--yacc}) is enabled, as POSIX mandates that Yacc
3984 generate tokens as macros (e.g., @samp{#define INT 258}, or @samp{#define
3987 A similar feature is provided for C++ that in addition overcomes C++
3988 limitations (that forbid non-trivial objects to be part of a @code{union}):
3989 @samp{%define api.value.type variant}, see @ref{C++ Variants}.
3992 @subsection The Union Declaration
3993 @cindex declaring value types
3994 @cindex value types, declaring
3997 The @code{%union} declaration specifies the entire collection of possible
3998 data types for semantic values. The keyword @code{%union} is followed by
3999 braced code containing the same thing that goes inside a @code{union} in C@.
4013 This says that the two alternative types are @code{double} and @code{symrec
4014 *}. They are given names @code{val} and @code{tptr}; these names are used
4015 in the @code{%token}, @code{%nterm} and @code{%type} declarations to pick
4016 one of the types for a terminal or nonterminal symbol (@pxref{Type Decl}).
4018 As an extension to POSIX, a tag is allowed after the @code{%union}. For
4031 specifies the union tag @code{value}, so the corresponding C type is
4032 @code{union value}. If you do not specify a tag, it defaults to
4033 @code{YYSTYPE} (@pxref{%define Summary}).
4035 As another extension to POSIX, you may specify multiple @code{%union}
4036 declarations; their contents are concatenated. However, only the first
4037 @code{%union} declaration can specify a tag.
4039 Note that, unlike making a @code{union} declaration in C, you need not write
4040 a semicolon after the closing brace.
4042 @node Structured Value Type
4043 @subsection Providing a Structured Semantic Value Type
4044 @cindex declaring value types
4045 @cindex value types, declaring
4048 Instead of @code{%union}, you can define and use your own union type
4049 @code{YYSTYPE} if your grammar contains at least one @samp{<@var{type}>}
4050 tag. For example, you can put the following into a header file
4063 and then your grammar can use the following instead of @code{%union}:
4070 %define api.value.type @{union YYSTYPE@}
4076 Actually, you may also provide a @code{struct} rather that a @code{union},
4077 which may be handy if you want to track information for every symbol (such
4078 as preceding comments).
4080 The type you provide may even be structured and include pointers, in which
4081 case the type tags you provide may be composite, with @samp{.} and @samp{->}
4090 @vindex $[@var{name}]
4092 An action accompanies a syntactic rule and contains C code to be executed
4093 each time an instance of that rule is recognized. The task of most actions
4094 is to compute a semantic value for the grouping built by the rule from the
4095 semantic values associated with tokens or smaller groupings.
4097 An action consists of braced code containing C statements, and can be
4098 placed at any position in the rule;
4099 it is executed at that position. Most rules have just one action at the
4100 end of the rule, following all the components. Actions in the middle of
4101 a rule are tricky and used only for special purposes (@pxref{Midrule
4104 The C code in an action can refer to the semantic values of the
4105 components matched by the rule with the construct @code{$@var{n}},
4106 which stands for the value of the @var{n}th component. The semantic
4107 value for the grouping being constructed is @code{$$}. In addition,
4108 the semantic values of symbols can be accessed with the named
4109 references construct @code{$@var{name}} or @code{$[@var{name}]}.
4110 Bison translates both of these constructs into expressions of the
4111 appropriate type when it copies the actions into the parser
4112 implementation file. @code{$$} (or @code{$@var{name}}, when it stands
4113 for the current grouping) is translated to a modifiable lvalue, so it
4116 Here is a typical example:
4122 | exp '+' exp @{ $$ = $1 + $3; @}
4126 Or, in terms of named references:
4132 | exp[left] '+' exp[right] @{ $result = $left + $right; @}
4137 This rule constructs an @code{exp} from two smaller @code{exp} groupings
4138 connected by a plus-sign token. In the action, @code{$1} and @code{$3}
4139 (@code{$left} and @code{$right})
4140 refer to the semantic values of the two component @code{exp} groupings,
4141 which are the first and third symbols on the right hand side of the rule.
4142 The sum is stored into @code{$$} (@code{$result}) so that it becomes the
4144 the addition-expression just recognized by the rule. If there were a
4145 useful semantic value associated with the @samp{+} token, it could be
4146 referred to as @code{$2}.
4148 @xref{Named References}, for more information about using the named
4149 references construct.
4151 Note that the vertical-bar character @samp{|} is really a rule
4152 separator, and actions are attached to a single rule. This is a
4153 difference with tools like Flex, for which @samp{|} stands for either
4154 ``or'', or ``the same action as that of the next rule''. In the
4155 following example, the action is triggered only when @samp{b} is found:
4158 a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
4161 @cindex default action
4162 If you don't specify an action for a rule, Bison supplies a default:
4163 @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
4164 becomes the value of the whole rule. Of course, the default action is
4165 valid only if the two data types match. There is no meaningful default
4166 action for an empty rule; every empty rule must have an explicit action
4167 unless the rule's value does not matter.
4169 @code{$@var{n}} with @var{n} zero or negative is allowed for reference
4170 to tokens and groupings on the stack @emph{before} those that match the
4171 current rule. This is a very risky practice, and to use it reliably
4172 you must be certain of the context in which the rule is applied. Here
4173 is a case in which you can use this reliably:
4178 expr bar '+' expr @{ @dots{} @}
4179 | expr bar '-' expr @{ @dots{} @}
4185 %empty @{ previous_expr = $0; @}
4190 As long as @code{bar} is used only in the fashion shown here, @code{$0}
4191 always refers to the @code{expr} which precedes @code{bar} in the
4192 definition of @code{foo}.
4195 It is also possible to access the semantic value of the lookahead token, if
4196 any, from a semantic action.
4197 This semantic value is stored in @code{yylval}.
4198 @xref{Action Features}.
4201 @subsection Data Types of Values in Actions
4202 @cindex action data types
4203 @cindex data types in actions
4205 If you have chosen a single data type for semantic values, the @code{$$}
4206 and @code{$@var{n}} constructs always have that data type.
4208 If you have used @code{%union} to specify a variety of data types, then you
4209 must declare a choice among these types for each terminal or nonterminal
4210 symbol that can have a semantic value. Then each time you use @code{$$} or
4211 @code{$@var{n}}, its data type is determined by which symbol it refers to
4212 in the rule. In this example,
4218 | exp '+' exp @{ $$ = $1 + $3; @}
4223 @code{$1} and @code{$3} refer to instances of @code{exp}, so they all
4224 have the data type declared for the nonterminal symbol @code{exp}. If
4225 @code{$2} were used, it would have the data type declared for the
4226 terminal symbol @code{'+'}, whatever that might be.
4228 Alternatively, you can specify the data type when you refer to the value,
4229 by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
4230 reference. For example, if you have defined types as shown here:
4242 then you can write @code{$<itype>1} to refer to the first subunit of the
4243 rule as an integer, or @code{$<dtype>1} to refer to it as a double.
4245 @node Midrule Actions
4246 @subsection Actions in Midrule
4247 @cindex actions in midrule
4248 @cindex midrule actions
4250 Occasionally it is useful to put an action in the middle of a rule.
4251 These actions are written just like usual end-of-rule actions, but they
4252 are executed before the parser even recognizes the following components.
4255 * Using Midrule Actions:: Putting an action in the middle of a rule.
4256 * Typed Midrule Actions:: Specifying the semantic type of their values.
4257 * Midrule Action Translation:: How midrule actions are actually processed.
4258 * Midrule Conflicts:: Midrule actions can cause conflicts.
4261 @node Using Midrule Actions
4262 @subsubsection Using Midrule Actions
4264 A midrule action may refer to the components preceding it using
4265 @code{$@var{n}}, but it may not refer to subsequent components because
4266 it is run before they are parsed.
4268 The midrule action itself counts as one of the components of the rule.
4269 This makes a difference when there is another action later in the same rule
4270 (and usually there is another at the end): you have to count the actions
4271 along with the symbols when working out which number @var{n} to use in
4274 The midrule action can also have a semantic value. The action can set
4275 its value with an assignment to @code{$$}, and actions later in the rule
4276 can refer to the value using @code{$@var{n}}. Since there is no symbol
4277 to name the action, there is no way to declare a data type for the value
4278 in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
4279 specify a data type each time you refer to this value.
4281 There is no way to set the value of the entire rule with a midrule
4282 action, because assignments to @code{$$} do not have that effect. The
4283 only way to set the value for the entire rule is with an ordinary action
4284 at the end of the rule.
4286 Here is an example from a hypothetical compiler, handling a @code{let}
4287 statement that looks like @samp{let (@var{variable}) @var{statement}} and
4288 serves to create a variable named @var{variable} temporarily for the
4289 duration of @var{statement}. To parse this construct, we must put
4290 @var{variable} into the symbol table while @var{statement} is parsed, then
4291 remove it afterward. Here is how it is done:
4298 $<context>$ = push_context ();
4299 declare_variable ($3);
4304 pop_context ($<context>5);
4310 As soon as @samp{let (@var{variable})} has been recognized, the first
4311 action is run. It saves a copy of the current semantic context (the
4312 list of accessible variables) as its semantic value, using alternative
4313 @code{context} in the data-type union. Then it calls
4314 @code{declare_variable} to add the new variable to that list. Once the
4315 first action is finished, the embedded statement @code{stmt} can be
4318 Note that the midrule action is component number 5, so the @samp{stmt} is
4319 component number 6. Named references can be used to improve the readability
4320 and maintainability (@pxref{Named References}):
4327 $<context>let = push_context ();
4328 declare_variable ($3);
4333 pop_context ($<context>let);
4338 After the embedded statement is parsed, its semantic value becomes the
4339 value of the entire @code{let}-statement. Then the semantic value from the
4340 earlier action is used to restore the prior list of variables. This
4341 removes the temporary @code{let}-variable from the list so that it won't
4342 appear to exist while the rest of the program is parsed.
4344 Because the types of the semantic values of midrule actions are unknown to
4345 Bison, type-based features (e.g., @samp{%printer}, @samp{%destructor}) do
4346 not work, which could result in memory leaks. They also forbid the use of
4347 the @code{variant} implementation of the @code{api.value.type} in C++
4348 (@pxref{C++ Variants}).
4350 @xref{Typed Midrule Actions}, for one way to address this issue, and
4351 @ref{Midrule Action Translation}, for another: turning mid-action actions
4352 into regular actions.
4355 @node Typed Midrule Actions
4356 @subsubsection Typed Midrule Actions
4359 @cindex discarded symbols, midrule actions
4360 @cindex error recovery, midrule actions
4361 In the above example, if the parser initiates error recovery (@pxref{Error
4362 Recovery}) while parsing the tokens in the embedded statement @code{stmt},
4363 it might discard the previous semantic context @code{$<context>5} without
4364 restoring it. Thus, @code{$<context>5} needs a destructor
4365 (@pxref{Destructor Decl}), and Bison needs the
4366 type of the semantic value (@code{context}) to select the right destructor.
4368 As an extension to Yacc's midrule actions, Bison offers a means to type
4369 their semantic value: specify its type tag (@samp{<...>} before the midrule
4372 Consider the previous example, with an untyped midrule action:
4379 $<context>$ = push_context (); // ***
4380 declare_variable ($3);
4385 pop_context ($<context>5); // ***
4391 If instead you write:
4398 $$ = push_context (); // ***
4399 declare_variable ($3);
4404 pop_context ($5); // ***
4410 then @code{%printer} and @code{%destructor} work properly (no more leaks!),
4411 C++ @code{variant}s can be used, and redundancy is reduced (@code{<context>}
4415 @node Midrule Action Translation
4416 @subsubsection Midrule Action Translation
4420 Midrule actions are actually transformed into regular rules and actions.
4421 The various reports generated by Bison (textual, graphical, etc., see
4422 @ref{Understanding}) reveal this translation,
4423 best explained by means of an example. The following rule:
4426 exp: @{ a(); @} "b" @{ c(); @} @{ d(); @} "e" @{ f(); @};
4433 $@@1: %empty @{ a(); @};
4434 $@@2: %empty @{ c(); @};
4435 $@@3: %empty @{ d(); @};
4436 exp: $@@1 "b" $@@2 $@@3 "e" @{ f(); @};
4440 with new nonterminal symbols @code{$@@@var{n}}, where @var{n} is a number.
4442 A midrule action is expected to generate a value if it uses @code{$$}, or
4443 the (final) action uses @code{$@var{n}} where @var{n} denote the midrule
4444 action. In that case its nonterminal is rather named @code{@@@var{n}}:
4447 exp: @{ a(); @} "b" @{ $$ = c(); @} @{ d(); @} "e" @{ f = $1; @};
4454 @@1: %empty @{ a(); @};
4455 @@2: %empty @{ $$ = c(); @};
4456 $@@3: %empty @{ d(); @};
4457 exp: @@1 "b" @@2 $@@3 "e" @{ f = $1; @}
4460 There are probably two errors in the above example: the first midrule action
4461 does not generate a value (it does not use @code{$$} although the final
4462 action uses it), and the value of the second one is not used (the final
4463 action does not use @code{$3}). Bison reports these errors when the
4464 @code{midrule-value} warnings are enabled (@pxref{Invocation}):
4467 $ @kbd{bison -Wmidrule-value mid.y}
4469 mid.y:2.6-13: @dwarning{warning}: unset value: $$
4470 2 | exp: @dwarning{@{ a(); @}} "b" @{ $$ = c(); @} @{ d(); @} "e" @{ f = $1; @};
4471 | @dwarning{^~~~~~~~}
4474 mid.y:2.19-31: @dwarning{warning}: unused value: $3
4475 2 | exp: @{ a(); @} "b" @dwarning{@{ $$ = c(); @}} @{ d(); @} "e" @{ f = $1; @};
4476 | @dwarning{^~~~~~~~~~~~~}
4482 It is sometimes useful to turn midrule actions into regular actions, e.g.,
4483 to factor them, or to escape from their limitations. For instance, as an
4484 alternative to @emph{typed} midrule action, you may bury the midrule action
4485 inside a nonterminal symbol and to declare a printer and a destructor for
4490 %nterm <context> let
4491 %destructor @{ pop_context ($$); @} let
4492 %printer @{ print_context (yyo, $$); @} let
4510 $let = push_context ();
4511 declare_variable ($var);
4520 @node Midrule Conflicts
4521 @subsubsection Conflicts due to Midrule Actions
4522 Taking action before a rule is completely recognized often leads to
4523 conflicts since the parser must commit to a parse in order to execute the
4524 action. For example, the following two rules, without midrule actions,
4525 can coexist in a working parser because the parser can shift the open-brace
4526 token and look at what follows before deciding whether there is a
4532 '@{' declarations statements '@}'
4533 | '@{' statements '@}'
4539 But when we add a midrule action as follows, the rules become nonfunctional:
4544 @{ prepare_for_local_variables (); @}
4545 '@{' declarations statements '@}'
4548 | '@{' statements '@}'
4554 Now the parser is forced to decide whether to run the midrule action
4555 when it has read no farther than the open-brace. In other words, it
4556 must commit to using one rule or the other, without sufficient
4557 information to do it correctly. (The open-brace token is what is called
4558 the @dfn{lookahead} token at this time, since the parser is still
4559 deciding what to do about it. @xref{Lookahead}.)
4561 You might think that you could correct the problem by putting identical
4562 actions into the two rules, like this:
4567 @{ prepare_for_local_variables (); @}
4568 '@{' declarations statements '@}'
4569 | @{ prepare_for_local_variables (); @}
4570 '@{' statements '@}'
4576 But this does not help, because Bison does not realize that the two actions
4577 are identical. (Bison never tries to understand the C code in an action.)
4579 If the grammar is such that a declaration can be distinguished from a
4580 statement by the first token (which is true in C), then one solution which
4581 does work is to put the action after the open-brace, like this:
4586 '@{' @{ prepare_for_local_variables (); @}
4587 declarations statements '@}'
4588 | '@{' statements '@}'
4594 Now the first token of the following declaration or statement,
4595 which would in any case tell Bison which rule to use, can still do so.
4597 Another solution is to bury the action inside a nonterminal symbol which
4598 serves as a subroutine:
4603 %empty @{ prepare_for_local_variables (); @}
4609 subroutine '@{' declarations statements '@}'
4610 | subroutine '@{' statements '@}'
4616 Now Bison can execute the action in the rule for @code{subroutine} without
4617 deciding which rule for @code{compound} it will eventually use.
4620 @node Tracking Locations
4621 @section Tracking Locations
4623 @cindex textual location
4624 @cindex location, textual
4626 Though grammar rules and semantic actions are enough to write a fully
4627 functional parser, it can be useful to process some additional information,
4628 especially symbol locations.
4630 The way locations are handled is defined by providing a data type, and
4631 actions to take when rules are matched.
4634 * Location Type:: Specifying a data type for locations.
4635 * Actions and Locations:: Using locations in actions.
4636 * Location Default Action:: Defining a general way to compute locations.
4640 @subsection Data Type of Locations
4641 @cindex data type of locations
4642 @cindex default location type
4644 Defining a data type for locations is much simpler than for semantic values,
4645 since all tokens and groupings always use the same type.
4647 You can specify the type of locations by defining a macro called
4648 @code{YYLTYPE}, just as you can specify the semantic value type by defining
4649 a @code{YYSTYPE} macro (@pxref{Value Type}). When @code{YYLTYPE} is not
4650 defined, Bison uses a default structure type with four members:
4653 typedef struct YYLTYPE
4662 While default locations represent a range in the source file(s), this is not
4663 a requirement. It could be a single point or just a line number, or even
4664 more complex structures.
4666 When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
4667 initializes all these fields to 1 for @code{yylloc}. To initialize
4668 @code{yylloc} with a custom location type (or to chose a different
4669 initialization), use the @code{%initial-action} directive. @xref{Initial
4673 @node Actions and Locations
4674 @subsection Actions and Locations
4675 @cindex location actions
4676 @cindex actions, location
4679 @vindex @@@var{name}
4680 @vindex @@[@var{name}]
4682 Actions are not only useful for defining language semantics, but also for
4683 describing the behavior of the output parser with locations.
4685 The most obvious way for building locations of syntactic groupings is very
4686 similar to the way semantic values are computed. In a given rule, several
4687 constructs can be used to access the locations of the elements being matched.
4688 The location of the @var{n}th component of the right hand side is
4689 @code{@@@var{n}}, while the location of the left hand side grouping is
4692 In addition, the named references construct @code{@@@var{name}} and
4693 @code{@@[@var{name}]} may also be used to address the symbol locations.
4694 @xref{Named References}, for more information about using the named
4695 references construct.
4697 Here is a basic example using the default data type for locations:
4705 @@$.first_column = @@1.first_column;
4706 @@$.first_line = @@1.first_line;
4707 @@$.last_column = @@3.last_column;
4708 @@$.last_line = @@3.last_line;
4714 fprintf (stderr, "%d.%d-%d.%d: division by zero",
4715 @@3.first_line, @@3.first_column,
4716 @@3.last_line, @@3.last_column);
4722 As for semantic values, there is a default action for locations that is
4723 run each time a rule is matched. It sets the beginning of @code{@@$} to the
4724 beginning of the first symbol, and the end of @code{@@$} to the end of the
4727 With this default action, the location tracking can be fully automatic. The
4728 example above simply rewrites this way:
4741 fprintf (stderr, "%d.%d-%d.%d: division by zero",
4742 @@3.first_line, @@3.first_column,
4743 @@3.last_line, @@3.last_column);
4750 It is also possible to access the location of the lookahead token, if any,
4751 from a semantic action.
4752 This location is stored in @code{yylloc}.
4753 @xref{Action Features}.
4755 @node Location Default Action
4756 @subsection Default Action for Locations
4757 @vindex YYLLOC_DEFAULT
4758 @cindex GLR parsers and @code{YYLLOC_DEFAULT}
4760 Actually, actions are not the best place to compute locations. Since
4761 locations are much more general than semantic values, there is room in
4762 the output parser to redefine the default action to take for each
4763 rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
4764 matched, before the associated action is run. It is also invoked
4765 while processing a syntax error, to compute the error's location.
4766 Before reporting an unresolvable syntactic ambiguity, a GLR
4767 parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4770 Most of the time, this macro is general enough to suppress location
4771 dedicated code from semantic actions.
4773 The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
4774 the location of the grouping (the result of the computation). When a
4775 rule is matched, the second parameter identifies locations of
4776 all right hand side elements of the rule being matched, and the third
4777 parameter is the size of the rule's right hand side.
4778 When a GLR parser reports an ambiguity, which of multiple candidate
4779 right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4780 When processing a syntax error, the second parameter identifies locations
4781 of the symbols that were discarded during error processing, and the third
4782 parameter is the number of discarded symbols.
4784 By default, @code{YYLLOC_DEFAULT} is defined this way:
4788 # define YYLLOC_DEFAULT(Cur, Rhs, N) \
4792 (Cur).first_line = YYRHSLOC(Rhs, 1).first_line; \
4793 (Cur).first_column = YYRHSLOC(Rhs, 1).first_column; \
4794 (Cur).last_line = YYRHSLOC(Rhs, N).last_line; \
4795 (Cur).last_column = YYRHSLOC(Rhs, N).last_column; \
4799 (Cur).first_line = (Cur).last_line = \
4800 YYRHSLOC(Rhs, 0).last_line; \
4801 (Cur).first_column = (Cur).last_column = \
4802 YYRHSLOC(Rhs, 0).last_column; \
4809 where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4810 in @var{rhs} when @var{k} is positive, and the location of the symbol
4811 just before the reduction when @var{k} and @var{n} are both zero.
4813 When defining @code{YYLLOC_DEFAULT}, you should consider that:
4817 All arguments are free of side-effects. However, only the first one (the
4818 result) should be modified by @code{YYLLOC_DEFAULT}.
4821 For consistency with semantic actions, valid indexes within the
4822 right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4823 valid index, and it refers to the symbol just before the reduction.
4824 During error processing @var{n} is always positive.
4827 Your macro should parenthesize its arguments, if need be, since the
4828 actual arguments may not be surrounded by parentheses. Also, your
4829 macro should expand to something that can be used as a single
4830 statement when it is followed by a semicolon.
4833 @node Named References
4834 @section Named References
4835 @cindex named references
4837 As described in the preceding sections, the traditional way to refer to any
4838 semantic value or location is a @dfn{positional reference}, which takes the
4839 form @code{$@var{n}}, @code{$$}, @code{@@@var{n}}, and @code{@@$}. However,
4840 such a reference is not very descriptive. Moreover, if you later decide to
4841 insert or remove symbols in the right-hand side of a grammar rule, the need
4842 to renumber such references can be tedious and error-prone.
4844 To avoid these issues, you can also refer to a semantic value or location
4845 using a @dfn{named reference}. First of all, original symbol names may be
4846 used as named references. For example:
4850 invocation: op '(' args ')'
4851 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
4856 Positional and named references can be mixed arbitrarily. For example:
4860 invocation: op '(' args ')'
4861 @{ $$ = new_invocation ($op, $args, @@$); @}
4866 However, sometimes regular symbol names are not sufficient due to
4872 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
4875 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
4878 @{ $$ = $1 / $3; @} // No error.
4883 When ambiguity occurs, explicitly declared names may be used for values and
4884 locations. Explicit names are declared as a bracketed name after a symbol
4885 appearance in rule definitions. For example:
4888 exp[result]: exp[left] '/' exp[right]
4889 @{ $result = $left / $right; @}
4894 In order to access a semantic value generated by a midrule action, an
4895 explicit name may also be declared by putting a bracketed name after the
4896 closing brace of the midrule action code:
4899 exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
4900 @{ $res = $left + $right; @}
4906 In references, in order to specify names containing dots and dashes, an explicit
4907 bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
4910 if-stmt: "if" '(' expr ')' "then" then.stmt ';'
4911 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
4915 It often happens that named references are followed by a dot, dash or other
4916 C punctuation marks and operators. By default, Bison will read
4917 @samp{$name.suffix} as a reference to symbol value @code{$name} followed by
4918 @samp{.suffix}, i.e., an access to the @code{suffix} field of the semantic
4919 value. In order to force Bison to recognize @samp{name.suffix} in its
4920 entirety as the name of a semantic value, the bracketed syntax
4921 @samp{$[name.suffix]} must be used.
4924 @section Bison Declarations
4925 @cindex declarations, Bison
4926 @cindex Bison declarations
4928 The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4929 used in formulating the grammar and the data types of semantic values.
4932 All token kind names (but not single-character literal tokens such as
4933 @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4934 declared if you need to specify which data type to use for the semantic
4935 value (@pxref{Multiple Types}).
4937 The first rule in the grammar file also specifies the start symbol, by
4938 default. If you want some other symbol to be the start symbol, you
4939 must declare it explicitly (@pxref{Language and Grammar}).
4942 * Require Decl:: Requiring a Bison version.
4943 * Token Decl:: Declaring terminal symbols.
4944 * Precedence Decl:: Declaring terminals with precedence and associativity.
4945 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
4946 * Symbol Decls:: Summary of the Syntax of Symbol Declarations.
4947 * Initial Action Decl:: Code run before parsing starts.
4948 * Destructor Decl:: Declaring how symbols are freed.
4949 * Printer Decl:: Declaring how symbol values are displayed.
4950 * Expect Decl:: Suppressing warnings about parsing conflicts.
4951 * Start Decl:: Specifying the start symbol.
4952 * Pure Decl:: Requesting a reentrant parser.
4953 * Push Decl:: Requesting a push parser.
4954 * Decl Summary:: Table of all Bison declarations.
4955 * %define Summary:: Defining variables to adjust Bison's behavior.
4956 * %code Summary:: Inserting code into the parser source.
4960 @subsection Require a Version of Bison
4961 @cindex version requirement
4962 @cindex requiring a version of Bison
4965 You may require the minimum version of Bison to process the grammar. If
4966 the requirement is not met, @command{bison} exits with an error (exit
4970 %require "@var{version}"
4973 Some deprecated behaviors are disabled for some required @var{version}:
4975 @item @code{"3.2"} (or better)
4976 The C++ deprecated files @file{position.hh} and @file{stack.hh} are no
4979 @item @code{"3.4"} (or better)
4981 @uref{https://marc.info/?l=graphviz-devel&m=129418103126092, recommendations
4982 of the Graphviz team}, use the @code{.gv} extension instead of @code{.dot}
4983 for the name of the generated DOT file. @xref{Graphviz}.
4988 @subsection Token Kind Names
4989 @cindex declaring token kind names
4990 @cindex token kind names, declaring
4991 @cindex declaring literal string tokens
4994 The basic way to declare a token kind name (terminal symbol) is as follows:
5000 Bison will convert this into a definition in the parser, so that the
5001 function @code{yylex} (if it is in this file) can use the name @var{name} to
5002 stand for this token kind's code.
5004 Alternatively, you can use @code{%left}, @code{%right}, @code{%precedence},
5005 or @code{%nonassoc} instead of @code{%token}, if you wish to specify
5006 associativity and precedence. @xref{Precedence Decl}. However, for
5007 clarity, we recommend to use these directives only to declare associativity
5008 and precedence, and not to add string aliases, semantic types, etc.
5010 You can explicitly specify the numeric code for a token kind by appending a
5011 nonnegative decimal or hexadecimal integer value in the field immediately
5012 following the token name:
5016 %token XNUM 0x12d // a GNU extension
5020 It is generally best, however, to let Bison choose the numeric codes for all
5021 token kinds. Bison will automatically select codes that don't conflict with
5022 each other or with normal characters.
5024 In the event that the stack type is a union, you must augment the
5025 @code{%token} or other token declaration to include the data type
5026 alternative delimited by angle-brackets (@pxref{Multiple Types}).
5032 %union @{ /* define stack type */
5036 %token <val> NUM /* define token NUM and its type */
5040 You can associate a literal string token with a token kind name by writing
5041 the literal string at the end of a @code{%token} declaration which declares
5042 the name. For example:
5049 For example, a grammar for the C language might specify these names with
5050 equivalent literal string tokens:
5053 %token <operator> OR "||"
5054 %token <operator> LE 134 "<="
5059 Once you equate the literal string and the token kind name, you can use them
5060 interchangeably in further declarations or the grammar rules. The
5061 @code{yylex} function can use the token name or the literal string to obtain
5062 the token kind code (@pxref{Calling Convention}).
5064 String aliases allow for better error messages using the literal strings
5065 instead of the token names, such as @samp{syntax error, unexpected ||,
5066 expecting number or (} rather than @samp{syntax error, unexpected OR,
5067 expecting NUM or LPAREN}.
5069 String aliases may also be marked for internationalization (@pxref{Token
5077 '\n' _("end of line")
5083 would produce in French @samp{erreur de syntaxe, || inattendu, attendait
5084 nombre ou (} rather than @samp{erreur de syntaxe, || inattendu, attendait
5087 @node Precedence Decl
5088 @subsection Operator Precedence
5089 @cindex precedence declarations
5090 @cindex declaring operator precedence
5091 @cindex operator precedence, declaring
5093 Use the @code{%left}, @code{%right}, @code{%nonassoc}, or @code{%precedence}
5094 declaration to declare a token and specify its precedence and associativity,
5095 all at once. These are called @dfn{precedence declarations}.
5096 @xref{Precedence}, for general information on operator
5099 The syntax of a precedence declaration is nearly the same as that of
5100 @code{%token}: either
5103 %left @var{symbols}@dots{}
5110 %left <@var{type}> @var{symbols}@dots{}
5113 And indeed any of these declarations serves the purposes of @code{%token}.
5114 But in addition, they specify the associativity and relative precedence for
5115 all the @var{symbols}:
5119 The associativity of an operator @var{op} determines how repeated uses of
5120 the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op} @var{z}}
5121 is parsed by grouping @var{x} with @var{y} first or by grouping @var{y} with
5122 @var{z} first. @code{%left} specifies left-associativity (grouping @var{x}
5123 with @var{y} first) and @code{%right} specifies right-associativity
5124 (grouping @var{y} with @var{z} first). @code{%nonassoc} specifies no
5125 associativity, which means that @samp{@var{x} @var{op} @var{y} @var{op}
5126 @var{z}} is considered a syntax error.
5128 @code{%precedence} gives only precedence to the @var{symbols}, and defines
5129 no associativity at all. Use this to define precedence only, and leave any
5130 potential conflict due to associativity enabled.
5133 The precedence of an operator determines how it nests with other operators.
5134 All the tokens declared in a single precedence declaration have equal
5135 precedence and nest together according to their associativity. When two
5136 tokens declared in different precedence declarations associate, the one
5137 declared later has the higher precedence and is grouped first.
5140 For backward compatibility, there is a confusing difference between the
5141 argument lists of @code{%token} and precedence declarations. Only a
5142 @code{%token} can associate a literal string with a token kind name. A
5143 precedence declaration always interprets a literal string as a reference to
5144 a separate token. For example:
5147 %left OR "<=" // Does not declare an alias.
5148 %left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
5152 @subsection Nonterminal Symbols
5153 @cindex declaring value types, nonterminals
5154 @cindex value types, nonterminals, declaring
5159 When you use @code{%union} to specify multiple value types, you must
5160 declare the value type of each nonterminal symbol for which values are
5161 used. This is done with a @code{%type} declaration, like this:
5164 %type <@var{type}> @var{nonterminal}@dots{}
5168 Here @var{nonterminal} is the name of a nonterminal symbol, and @var{type}
5169 is the name given in the @code{%union} to the alternative that you want
5170 (@pxref{Union Decl}). You can give any number of nonterminal symbols in the
5171 same @code{%type} declaration, if they have the same value type. Use spaces
5172 to separate the symbol names.
5174 While POSIX Yacc allows @code{%type} only for nonterminals, Bison accepts
5175 that this directive be also applied to terminal symbols. To declare
5176 exclusively nonterminal symbols, use the safer @code{%nterm}:
5179 %nterm <@var{type}> @var{nonterminal}@dots{}
5184 @subsection Syntax of Symbol Declarations
5190 The syntax of the various directives to declare symbols is as follows.
5193 %token @var{tag}? ( @var{id} @var{number}? @var{string}? )+ ( @var{tag} ( @var{id} @var{number}? @var{string}? )+ )*
5194 %left @var{tag}? ( @var{id} @var{number}?)+ ( @var{tag} ( @var{id} @var{number}? )+ )*
5195 %type @var{tag}? ( @var{id} | @var{char} | @var{string} )+ ( @var{tag} ( @var{id} | @var{char} | @var{string} )+ )*
5196 %nterm @var{tag}? @var{id}+ ( @var{tag} @var{id}+ )*
5200 where @var{tag} denotes a type tag such as @samp{<ival>}, @var{id} denotes
5201 an identifier such as @samp{NUM}, @var{number} a decimal or hexadecimal
5202 integer such as @samp{300} or @samp{0x12d}, @var{char} a character literal
5203 such as @samp{'+'}, and @var{string} a string literal such as
5204 @samp{"number"}. The postfix quantifiers are @samp{?} (zero or one),
5205 @samp{*} (zero or more) and @samp{+} (one or more).
5207 The directives @code{%precedence}, @code{%right} and @code{%nonassoc} behave
5210 @node Initial Action Decl
5211 @subsection Performing Actions before Parsing
5212 @findex %initial-action
5214 Sometimes your parser needs to perform some initializations before parsing.
5215 The @code{%initial-action} directive allows for such arbitrary code.
5217 @deffn {Directive} %initial-action @{ @var{code} @}
5218 @findex %initial-action
5219 Declare that the braced @var{code} must be invoked before parsing each time
5220 @code{yyparse} is called. The @var{code} may use @code{$$} (or
5221 @code{$<@var{tag}>$}) and @code{@@$} --- initial value and location of the
5222 lookahead --- and the @code{%parse-param}.
5225 For instance, if your locations use a file name, you may use
5228 %parse-param @{ char const *file_name @};
5231 @@$.initialize (file_name);
5236 @node Destructor Decl
5237 @subsection Freeing Discarded Symbols
5238 @cindex freeing discarded symbols
5242 During error recovery (@pxref{Error Recovery}), symbols already pushed on
5243 the stack and tokens coming from the rest of the file are discarded until
5244 the parser falls on its feet. If the parser runs out of memory, or if it
5245 returns via @code{YYABORT}, @code{YYACCEPT} or @code{YYNOMEM}, all the
5246 symbols on the stack must be discarded. Even if the parser succeeds, it
5247 must discard the start symbol.
5249 When discarded symbols convey heap based information, this memory is
5250 lost. While this behavior can be tolerable for batch parsers, such as
5251 in traditional compilers, it is unacceptable for programs like shells or
5252 protocol implementations that may parse and execute indefinitely.
5254 The @code{%destructor} directive defines code that is called when a
5255 symbol is automatically discarded.
5257 @deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
5259 Invoke the braced @var{code} whenever the parser discards one of the
5260 @var{symbols}. Within @var{code}, @code{$$} (or @code{$<@var{tag}>$})
5261 designates the semantic value associated with the discarded symbol, and
5262 @code{@@$} designates its location. The additional parser parameters are
5263 also available (@pxref{Parser Function}).
5265 When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
5266 per-symbol @code{%destructor}.
5267 You may also define a per-type @code{%destructor} by listing a semantic type
5268 tag among @var{symbols}.
5269 In that case, the parser will invoke this @var{code} whenever it discards any
5270 grammar symbol that has that semantic type tag unless that symbol has its own
5271 per-symbol @code{%destructor}.
5273 Finally, you can define two different kinds of default @code{%destructor}s.
5274 You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
5275 exactly one @code{%destructor} declaration in your grammar file.
5276 The parser will invoke the @var{code} associated with one of these whenever it
5277 discards any user-defined grammar symbol that has no per-symbol and no per-type
5279 The parser uses the @var{code} for @code{<*>} in the case of such a grammar
5280 symbol for which you have formally declared a semantic type tag (@code{%token},
5281 @code{%nterm}, and @code{%type}
5282 count as such a declaration, but @code{$<tag>$} does not).
5283 The parser uses the @var{code} for @code{<>} in the case of such a grammar
5284 symbol that has no declared semantic type tag.
5291 %union @{ char *string; @}
5292 %token <string> STRING1 STRING2
5293 %nterm <string> string1 string2
5294 %union @{ char character; @}
5295 %token <character> CHR
5296 %nterm <character> chr
5299 %destructor @{ @} <character>
5300 %destructor @{ free ($$); @} <*>
5301 %destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
5302 %destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
5306 guarantees that, when the parser discards any user-defined symbol that has a
5307 semantic type tag other than @code{<character>}, it passes its semantic value
5308 to @code{free} by default.
5309 However, when the parser discards a @code{STRING1} or a @code{string1},
5310 it uses the third @code{%destructor}, which frees it and
5311 prints its line number to @code{stdout} (@code{free} is invoked only once).
5312 Finally, the parser merely prints a message whenever it discards any symbol,
5313 such as @code{TAGLESS}, that has no semantic type tag.
5315 A Bison-generated parser invokes the default @code{%destructor}s only for
5316 user-defined as opposed to Bison-defined symbols.
5317 For example, the parser will not invoke either kind of default
5318 @code{%destructor} for the special Bison-defined symbols @code{$accept},
5319 @code{$undefined}, or @code{$end} (@pxref{Table of Symbols}),
5320 none of which you can reference in your grammar.
5321 It also will not invoke either for the @code{error} token (@pxref{Table of
5322 Symbols}), which is always defined by Bison regardless of whether you
5323 reference it in your grammar.
5324 However, it may invoke one of them for the end token (token 0) if you
5325 redefine it from @code{$end} to, for example, @code{END}:
5331 @cindex actions in midrule
5332 @cindex midrule actions
5333 Finally, Bison will never invoke a @code{%destructor} for an unreferenced
5334 midrule semantic value (@pxref{Midrule Actions}).
5335 That is, Bison does not consider a midrule to have a semantic value if you
5336 do not reference @code{$$} in the midrule's action or @code{$@var{n}}
5337 (where @var{n} is the right-hand side symbol position of the midrule) in
5338 any later action in that rule. However, if you do reference either, the
5339 Bison-generated parser will invoke the @code{<>} @code{%destructor} whenever
5340 it discards the midrule symbol.
5344 In the future, it may be possible to redefine the @code{error} token as a
5345 nonterminal that captures the discarded symbols.
5346 In that case, the parser will invoke the default destructor for it as well.
5351 @cindex discarded symbols
5352 @dfn{Discarded symbols} are the following:
5356 stacked symbols popped during the first phase of error recovery,
5358 incoming terminals during the second phase of error recovery,
5360 the current lookahead and the entire stack (except the current
5361 right-hand side symbols) when the parser returns immediately, and
5363 the current lookahead and the entire stack (including the current right-hand
5364 side symbols) when the C++ parser (@file{lalr1.cc}) catches an exception in
5367 the start symbol, when the parser succeeds.
5370 The parser can @dfn{return immediately} because of an explicit call to
5371 @code{YYABORT}, @code{YYACCEPT} or @code{YYNOMEM}, or failed error recovery,
5372 or memory exhaustion.
5374 Right-hand side symbols of a rule that explicitly triggers a syntax
5375 error via @code{YYERROR} are not discarded automatically. As a rule
5376 of thumb, destructors are invoked only when user actions cannot manage
5380 @subsection Printing Semantic Values
5381 @cindex printing semantic values
5385 When run-time traces are enabled (@pxref{Tracing}),
5386 the parser reports its actions, such as reductions. When a symbol involved
5387 in an action is reported, only its kind is displayed, as the parser cannot
5388 know how semantic values should be formatted.
5390 The @code{%printer} directive defines code that is called when a symbol is
5391 reported. Its syntax is the same as @code{%destructor} (@pxref{Destructor
5394 @deffn {Directive} %printer @{ @var{code} @} @var{symbols}
5397 @c This is the same text as for %destructor.
5398 Invoke the braced @var{code} whenever the parser displays one of the
5399 @var{symbols}. Within @var{code}, @code{yyo} denotes the output stream (a
5400 @code{FILE*} in C, and an @code{std::ostream&} in C++), @code{$$} (or
5401 @code{$<@var{tag}>$}) designates the semantic value associated with the
5402 symbol, and @code{@@$} its location. The additional parser parameters are
5403 also available (@pxref{Parser Function}).
5405 The @var{symbols} are defined as for @code{%destructor} (@pxref{Destructor
5406 Decl}.): they can be per-type (e.g.,
5407 @samp{<ival>}), per-symbol (e.g., @samp{exp}, @samp{NUM}, @samp{"float"}),
5408 typed per-default (i.e., @samp{<*>}, or untyped per-default (i.e.,
5416 %union @{ char *string; @}
5417 %token <string> STRING1 STRING2
5418 %nterm <string> string1 string2
5419 %union @{ char character; @}
5420 %token <character> CHR
5421 %nterm <character> chr
5424 %printer @{ fprintf (yyo, "'%c'", $$); @} <character>
5425 %printer @{ fprintf (yyo, "&%p", $$); @} <*>
5426 %printer @{ fprintf (yyo, "\"%s\"", $$); @} STRING1 string1
5427 %printer @{ fprintf (yyo, "<>"); @} <>
5431 guarantees that, when the parser print any symbol that has a semantic type
5432 tag other than @code{<character>}, it display the address of the semantic
5433 value by default. However, when the parser displays a @code{STRING1} or a
5434 @code{string1}, it formats it as a string in double quotes. It performs
5435 only the second @code{%printer} in this case, so it prints only once.
5436 Finally, the parser print @samp{<>} for any symbol, such as @code{TAGLESS},
5437 that has no semantic type tag. @xref{Mfcalc Traces}, for a complete example.
5442 @subsection Suppressing Conflict Warnings
5443 @cindex suppressing conflict warnings
5444 @cindex preventing warnings about conflicts
5445 @cindex warnings, preventing
5446 @cindex conflicts, suppressing warnings of
5450 Bison normally warns if there are any conflicts in the grammar
5451 (@pxref{Shift/Reduce}), but most real grammars
5452 have harmless shift/reduce conflicts which are resolved in a predictable
5453 way and would be difficult to eliminate. It is desirable to suppress
5454 the warning about these conflicts unless the number of conflicts
5455 changes. You can do this with the @code{%expect} declaration.
5457 The declaration looks like this:
5463 Here @var{n} is a decimal integer. The declaration says there should
5464 be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
5465 Bison reports an error if the number of shift/reduce conflicts differs
5466 from @var{n}, or if there are any reduce/reduce conflicts.
5468 For deterministic parsers, reduce/reduce conflicts are more
5469 serious, and should be eliminated entirely. Bison will always report
5470 reduce/reduce conflicts for these parsers. With GLR
5471 parsers, however, both kinds of conflicts are routine; otherwise,
5472 there would be no need to use GLR parsing. Therefore, it is
5473 also possible to specify an expected number of reduce/reduce conflicts
5474 in GLR parsers, using the declaration:
5480 You may wish to be more specific in your
5481 specification of expected conflicts. To this end, you can also attach
5482 @code{%expect} and @code{%expect-rr} modifiers to individual rules.
5483 The interpretation of these modifiers differs from their use as
5484 declarations. When attached to rules, they indicate the number of states
5485 in which the rule is involved in a conflict. You will need to consult the
5486 output resulting from @option{-v} to determine appropriate numbers to use.
5487 For example, for the following grammar fragment, the first rule for
5488 @code{empty_dims} appears in two states in which the @samp{[} token is a
5489 lookahead. Having determined that, you can document this fact with an
5490 @code{%expect} modifier as follows:
5500 | empty_dims '[' ']'
5504 Mid-rule actions generate implicit rules that are also subject to conflicts
5505 (@pxref{Midrule Conflicts}). To attach
5506 an @code{%expect} or @code{%expect-rr} annotation to an implicit
5507 mid-rule action's rule, put it before the action. For example,
5516 "condition" %expect-rr 1 @{ value_mode(); @} '(' exprs ')'
5517 | "condition" %expect-rr 1 @{ class_mode(); @} '(' types ')'
5522 Here, the appropriate mid-rule action will not be determined until after
5523 the @samp{(} token is shifted. Thus,
5524 the two actions will clash with each other, and we should expect one
5525 reduce/reduce conflict for each.
5527 In general, using @code{%expect} involves these steps:
5531 Compile your grammar without @code{%expect}. Use the @option{-v} option
5532 to get a verbose list of where the conflicts occur. Bison will also
5533 print the number of conflicts.
5536 Check each of the conflicts to make sure that Bison's default
5537 resolution is what you really want. If not, rewrite the grammar and
5538 go back to the beginning.
5541 Add an @code{%expect} declaration, copying the number @var{n} from the
5542 number that Bison printed. With GLR parsers, add an
5543 @code{%expect-rr} declaration as well.
5546 Optionally, count up the number of states in which one or more
5547 conflicted reductions for particular rules appear and add these numbers
5548 to the affected rules as @code{%expect-rr} or @code{%expect} modifiers
5549 as appropriate. Rules that are in conflict appear in the output listing
5550 surrounded by square brackets or, in the case of reduce/reduce conflicts,
5551 as reductions having the same lookahead symbol as a square-bracketed
5552 reduction in the same state.
5555 Now Bison will report an error if you introduce an unexpected conflict,
5556 but will keep silent otherwise.
5559 @subsection The Start-Symbol
5560 @cindex declaring the start symbol
5561 @cindex start symbol, declaring
5562 @cindex default start symbol
5565 Bison assumes by default that the start symbol for the grammar is the first
5566 nonterminal specified in the grammar specification section. The programmer
5567 may override this restriction with the @code{%start} declaration as follows:
5574 @subsection A Pure (Reentrant) Parser
5575 @cindex reentrant parser
5577 @findex %define api.pure
5579 A @dfn{reentrant} program is one which does not alter in the course of
5580 execution; in other words, it consists entirely of @dfn{pure} (read-only)
5581 code. Reentrancy is important whenever asynchronous execution is possible;
5582 for example, a nonreentrant program may not be safe to call from a signal
5583 handler. In systems with multiple threads of control, a nonreentrant
5584 program must be called only within interlocks.
5586 Normally, Bison generates a parser which is not reentrant. This is
5587 suitable for most uses, and it permits compatibility with Yacc. (The
5588 standard Yacc interfaces are inherently nonreentrant, because they use
5589 statically allocated variables for communication with @code{yylex},
5590 including @code{yylval} and @code{yylloc}.)
5592 Alternatively, you can generate a pure, reentrant parser. The Bison
5593 declaration @samp{%define api.pure} says that you want the parser to be
5594 reentrant. It looks like this:
5597 %define api.pure full
5600 The result is that the communication variables @code{yylval} and
5601 @code{yylloc} become local variables in @code{yyparse}, and a different
5602 calling convention is used for the lexical analyzer function @code{yylex}.
5603 @xref{Pure Calling}, for the details of this. The variable @code{yynerrs}
5604 becomes local in @code{yyparse} in pull mode but it becomes a member of
5605 @code{yypstate} in push mode. (@pxref{Error Reporting Function}). The
5606 convention for calling @code{yyparse} itself is unchanged.
5608 Whether the parser is pure has nothing to do with the grammar rules.
5609 You can generate either a pure parser or a nonreentrant parser from any
5613 @subsection A Push Parser
5616 @findex %define api.push-pull
5618 A pull parser is called once and it takes control until all its input
5619 is completely parsed. A push parser, on the other hand, is called
5620 each time a new token is made available.
5622 A push parser is typically useful when the parser is part of a
5623 main event loop in the client's application. This is typically
5624 a requirement of a GUI, when the main event loop needs to be triggered
5625 within a certain time period.
5627 Normally, Bison generates a pull parser.
5628 The following Bison declaration says that you want the parser to be a push
5629 parser (@pxref{%define Summary}):
5632 %define api.push-pull push
5635 In almost all cases, you want to ensure that your push parser is also
5636 a pure parser (@pxref{Pure Decl}). The only
5637 time you should create an impure push parser is to have backwards
5638 compatibility with the impure Yacc pull mode interface. Unless you know
5639 what you are doing, your declarations should look like this:
5642 %define api.pure full
5643 %define api.push-pull push
5646 There is a major notable functional difference between the pure push parser
5647 and the impure push parser. It is acceptable for a pure push parser to have
5648 many parser instances, of the same type of parser, in memory at the same time.
5649 An impure push parser should only use one parser at a time.
5651 When a push parser is selected, Bison will generate some new symbols in
5652 the generated parser. @code{yypstate} is a structure that the generated
5653 parser uses to store the parser's state. @code{yypstate_new} is the
5654 function that will create a new parser instance. @code{yypstate_delete}
5655 will free the resources associated with the corresponding parser instance.
5656 Finally, @code{yypush_parse} is the function that should be called whenever a
5657 token is available to provide the parser. A trivial example
5658 of using a pure push parser would look like this:
5662 yypstate *ps = yypstate_new ();
5664 status = yypush_parse (ps, yylex (), NULL);
5665 @} while (status == YYPUSH_MORE);
5666 yypstate_delete (ps);
5669 If the user decided to use an impure push parser, a few things about the
5670 generated parser will change. The @code{yychar} variable becomes a global
5671 variable instead of a local one in the @code{yypush_parse} function. For
5672 this reason, the signature of the @code{yypush_parse} function is changed to
5673 remove the token as a parameter. A nonreentrant push parser example would
5674 thus look like this:
5679 yypstate *ps = yypstate_new ();
5682 status = yypush_parse (ps);
5683 @} while (status == YYPUSH_MORE);
5684 yypstate_delete (ps);
5687 That's it. Notice the next token is put into the global variable @code{yychar}
5688 for use by the next invocation of the @code{yypush_parse} function.
5690 Bison also supports both the push parser interface along with the pull parser
5691 interface in the same generated parser. In order to get this functionality,
5692 you should replace the @samp{%define api.push-pull push} declaration with the
5693 @samp{%define api.push-pull both} declaration. Doing this will create all of
5694 the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
5695 and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
5696 would be used. However, the user should note that it is implemented in the
5697 generated parser by calling @code{yypull_parse}.
5698 This makes the @code{yyparse} function that is generated with the
5699 @samp{%define api.push-pull both} declaration slower than the normal
5700 @code{yyparse} function. If the user
5701 calls the @code{yypull_parse} function it will parse the rest of the input
5702 stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
5703 and then @code{yypull_parse} the rest of the input stream. If you would like
5704 to switch back and forth between between parsing styles, you would have to
5705 write your own @code{yypull_parse} function that knows when to quit looking
5706 for input. An example of using the @code{yypull_parse} function would look
5710 yypstate *ps = yypstate_new ();
5711 yypull_parse (ps); /* Will call the lexer */
5712 yypstate_delete (ps);
5715 Adding the @samp{%define api.pure} declaration does exactly the same thing to
5716 the generated parser with @samp{%define api.push-pull both} as it did for
5717 @samp{%define api.push-pull push}.
5720 @subsection Bison Declaration Summary
5721 @cindex Bison declaration summary
5722 @cindex declaration summary
5723 @cindex summary, Bison declaration
5725 Here is a summary of the declarations used to define a grammar:
5727 @deffn {Directive} %union
5728 Declare the collection of data types that semantic values may have
5729 (@pxref{Union Decl}).
5732 @deffn {Directive} %token
5733 Declare a terminal symbol (token kind name) with no precedence
5734 or associativity specified (@pxref{Token Decl}).
5737 @deffn {Directive} %right
5738 Declare a terminal symbol (token kind name) that is right-associative
5739 (@pxref{Precedence Decl}).
5742 @deffn {Directive} %left
5743 Declare a terminal symbol (token kind name) that is left-associative
5744 (@pxref{Precedence Decl}).
5747 @deffn {Directive} %nonassoc
5748 Declare a terminal symbol (token kind name) that is nonassociative
5749 (@pxref{Precedence Decl}).
5750 Using it in a way that would be associative is a syntax error.
5754 @deffn {Directive} %default-prec
5755 Assign a precedence to rules lacking an explicit @code{%prec} modifier
5756 (@pxref{Contextual Precedence}).
5760 @deffn {Directive} %nterm
5761 Declare the type of semantic values for a nonterminal symbol (@pxref{Type
5765 @deffn {Directive} %type
5766 Declare the type of semantic values for a symbol (@pxref{Type Decl}).
5769 @deffn {Directive} %start
5770 Specify the grammar's start symbol (@pxref{Start Decl}).
5773 @deffn {Directive} %expect
5774 Declare the expected number of shift/reduce conflicts, either overall or
5776 (@pxref{Expect Decl}).
5779 @deffn {Directive} %expect-rr
5780 Declare the expected number of reduce/reduce conflicts, either overall or
5782 (@pxref{Expect Decl}).
5788 In order to change the behavior of @command{bison}, use the following
5791 @deffn {Directive} %code @{@var{code}@}
5792 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
5794 Insert @var{code} verbatim into the output parser source at the
5795 default location or at the location specified by @var{qualifier}.
5796 @xref{%code Summary}.
5799 @deffn {Directive} %debug
5800 Instrument the parser for traces. Obsoleted by @samp{%define
5805 @deffn {Directive} %define @var{variable}
5806 @deffnx {Directive} %define @var{variable} @var{value}
5807 @deffnx {Directive} %define @var{variable} @{@var{value}@}
5808 @deffnx {Directive} %define @var{variable} "@var{value}"
5809 Define a variable to adjust Bison's behavior. @xref{%define Summary}.
5812 @deffn {Directive} %defines
5813 @deffnx {Directive} %defines @var{defines-file}
5814 Historical name for @code{%header}. @xref{%header,,@code{%header}}.
5817 @deffn {Directive} %destructor
5818 Specify how the parser should reclaim the memory associated to
5819 discarded symbols. @xref{Destructor Decl}.
5822 @deffn {Directive} %file-prefix "@var{prefix}"
5823 Specify a prefix to use for all Bison output file names. The names
5824 are chosen as if the grammar file were named @file{@var{prefix}.y}.
5828 @deffn {Directive} %header
5829 Write a parser header file containing definitions for the token kind names
5830 defined in the grammar as well as a few other declarations. If the parser
5831 implementation file is named @file{@var{name}.c} then the parser header file
5832 is named @file{@var{name}.h}.
5834 For C parsers, the parser header file declares @code{YYSTYPE} unless
5835 @code{YYSTYPE} is already defined as a macro or you have used a
5836 @code{<@var{type}>} tag without using @code{%union}. Therefore, if you are
5837 using a @code{%union} (@pxref{Multiple Types}) with components that require
5838 other definitions, or if you have defined a @code{YYSTYPE} macro or type
5839 definition (@pxref{Value Type}), you need to arrange for these definitions
5840 to be propagated to all modules, e.g., by putting them in a prerequisite
5841 header that is included both by your parser and by any other module that
5842 needs @code{YYSTYPE}.
5844 Unless your parser is pure, the parser header file declares
5845 @code{yylval} as an external variable. @xref{Pure Decl}.
5847 If you have also used locations, the parser header file declares
5848 @code{YYLTYPE} and @code{yylloc} using a protocol similar to that of the
5849 @code{YYSTYPE} macro and @code{yylval}. @xref{Tracking Locations}.
5851 This parser header file is normally essential if you wish to put the
5852 definition of @code{yylex} in a separate source file, because
5853 @code{yylex} typically needs to be able to refer to the
5854 above-mentioned declarations and to the token kind codes. @xref{Token
5857 @findex %code requires
5858 @findex %code provides
5859 If you have declared @code{%code requires} or @code{%code provides}, the output
5860 header also contains their code.
5861 @xref{%code Summary}.
5863 @cindex Header guard
5864 The generated header is protected against multiple inclusions with a C
5865 preprocessor guard: @samp{YY_@var{PREFIX}_@var{FILE}_INCLUDED}, where
5866 @var{PREFIX} and @var{FILE} are the prefix (@pxref{Multiple Parsers}) and
5867 generated file name turned uppercase, with each series of non alphanumerical
5868 characters converted to a single underscore.
5870 For instance with @samp{%define api.prefix @{calc@}} and @samp{%header
5871 "lib/parse.h"}, the header will be guarded as follows.
5873 #ifndef YY_CALC_LIB_PARSE_H_INCLUDED
5874 # define YY_CALC_LIB_PARSE_H_INCLUDED
5876 #endif /* ! YY_CALC_LIB_PARSE_H_INCLUDED */
5879 Introduced in Bison 3.8.
5882 @deffn {Directive} %header @var{header-file}
5883 Same as above, but save in the file @file{@var{header-file}}.
5886 @deffn {Directive} %language "@var{language}"
5887 Specify the programming language for the generated parser. Currently
5888 supported languages include C, C++, D and Java. @var{language} is
5892 @deffn {Directive} %locations
5893 Generate the code processing the locations (@pxref{Action Features}). This
5894 mode is enabled as soon as the grammar uses the special @samp{@@@var{n}}
5895 tokens, but if your grammar does not use it, using @samp{%locations} allows
5896 for more accurate syntax error messages.
5899 @deffn {Directive} %name-prefix "@var{prefix}"
5900 Obsoleted by @samp{%define api.prefix @{@var{prefix}@}}. @xref{Multiple
5901 Parsers}. For C++ parsers, see the
5902 @samp{%define api.namespace} documentation in this section.
5904 Rename the external symbols used in the parser so that they start with
5905 @var{prefix} instead of @samp{yy}. The precise list of symbols renamed in C
5906 parsers is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5907 @code{yylval}, @code{yychar}, @code{yydebug}, and (if locations are used)
5908 @code{yylloc}. If you use a push parser, @code{yypush_parse},
5909 @code{yypull_parse}, @code{yypstate}, @code{yypstate_new} and
5910 @code{yypstate_delete} will also be renamed. For example, if you use
5911 @samp{%name-prefix "c_"}, the names become @code{c_parse}, @code{c_lex}, and
5914 Contrary to defining @code{api.prefix}, some symbols are @emph{not} renamed
5915 by @code{%name-prefix}, for instance @code{YYDEBUG}, @code{YYTOKENTYPE},
5916 @code{yytoken_kind_t}, @code{YYSTYPE}, @code{YYLTYPE}.
5920 @deffn {Directive} %no-default-prec
5921 Do not assign a precedence to rules lacking an explicit @code{%prec}
5922 modifier (@pxref{Contextual Precedence}).
5926 @deffn {Directive} %no-lines
5927 Don't generate any @code{#line} preprocessor commands in the parser
5928 implementation file. Ordinarily Bison writes these commands in the parser
5929 implementation file so that the C compiler and debuggers will associate
5930 errors and object code with your source file (the grammar file). This
5931 directive causes them to associate errors with the parser implementation
5932 file, treating it as an independent source file in its own right.
5935 @deffn {Directive} %output "@var{file}"
5936 Generate the parser implementation in @file{@var{file}}.
5939 @deffn {Directive} %pure-parser
5940 Deprecated version of @samp{%define api.pure} (@pxref{%define
5941 Summary}), for which Bison is more careful to warn about
5945 @deffn {Directive} %require "@var{version}"
5946 Require version @var{version} or higher of Bison. @xref{Require Decl}.
5949 @deffn {Directive} %skeleton "@var{file}"
5950 Specify the skeleton to use.
5952 @c You probably don't need this option unless you are developing Bison.
5953 @c You should use @code{%language} if you want to specify the skeleton for a
5954 @c different language, because it is clearer and because it will always choose the
5955 @c correct skeleton for non-deterministic or push parsers.
5957 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5958 file in the Bison installation directory.
5959 If it does, @var{file} is an absolute file name or a file name relative to the
5960 directory of the grammar file.
5961 This is similar to how most shells resolve commands.
5964 @deffn {Directive} %token-table
5965 This feature is obsolescent, avoid it in new projects.
5967 Generate an array of token names in the parser implementation file. The
5968 name of the array is @code{yytname}; @code{yytname[@var{i}]} is the name of
5969 the token whose internal Bison token code is @var{i}. The first three
5970 elements of @code{yytname} correspond to the predefined tokens
5971 @code{"$end"}, @code{"error"}, and @code{"$undefined"}; after these come the
5972 symbols defined in the grammar file.
5974 The name in the table includes all the characters needed to represent the
5975 token in Bison. For single-character literals and literal strings, this
5976 includes the surrounding quoting characters and any escape sequences. For
5977 example, the Bison single-character literal @code{'+'} corresponds to a
5978 three-character name, represented in C as @code{"'+'"}; and the Bison
5979 two-character literal string @code{"\\/"} corresponds to a five-character
5980 name, represented in C as @code{"\"\\\\/\""}.
5982 When you specify @code{%token-table}, Bison also generates macro definitions
5983 for macros @code{YYNTOKENS}, @code{YYNNTS}, and @code{YYNRULES}, and
5988 The number of terminal symbols, i.e., the highest token code, plus one.
5990 The number of nonterminal symbols.
5992 The number of grammar rules,
5994 The number of parser states (@pxref{Parser States}).
5997 Here's code for looking up a multicharacter token in @code{yytname},
5998 assuming that the characters of the token are stored in @code{token_buffer},
5999 and assuming that the token does not contain any characters like @samp{"}
6000 that require escaping.
6003 for (int i = 0; i < YYNTOKENS; i++)
6005 && yytname[i][0] == '"'
6006 && ! strncmp (yytname[i] + 1, token_buffer,
6007 strlen (token_buffer))
6008 && yytname[i][strlen (token_buffer) + 1] == '"'
6009 && yytname[i][strlen (token_buffer) + 2] == 0)
6013 This method is discouraged: the primary purpose of string aliases is forging
6014 good error messages, not describing the spelling of keywords. In addition,
6015 looking for the token kind at runtime incurs a (small but noticeable) cost.
6017 Finally, @code{%token-table} is incompatible with the @code{custom} and
6018 @code{detailed} values of the @code{parse.error} @code{%define} variable.
6021 @deffn {Directive} %verbose
6022 Write an extra output file containing verbose descriptions of the parser
6023 states and what is done for each type of lookahead token in that state.
6024 @xref{Understanding}, for more information.
6027 @deffn {Directive} %yacc
6028 Pretend the option @option{--yacc} was given, i.e., imitate Yacc, including
6029 its naming conventions. Only makes sense with the @file{yacc.c}
6030 skeleton. @xref{Tuning the Parser}, for more.
6032 Of course @code{%yacc} is a Bison extension@dots{}
6036 @node %define Summary
6037 @subsection %define Summary
6039 There are many features of Bison's behavior that can be controlled by
6040 assigning the feature a single value. For historical reasons, some such
6041 features are assigned values by dedicated directives, such as @code{%start},
6042 which assigns the start symbol. However, newer such features are associated
6043 with variables, which are assigned by the @code{%define} directive:
6045 @deffn {Directive} %define @var{variable}
6046 @deffnx {Directive} %define @var{variable} @var{value}
6047 @deffnx {Directive} %define @var{variable} @{@var{value}@}
6048 @deffnx {Directive} %define @var{variable} "@var{value}"
6049 Define @var{variable} to @var{value}.
6051 The type of the values depend on the syntax. Braces denote value in the
6052 target language (e.g., a namespace, a type, etc.). Keyword values (no
6053 delimiters) denote finite choice (e.g., a variation of a feature). String
6054 values denote remaining cases (e.g., a file name).
6056 It is an error if a @var{variable} is defined by @code{%define} multiple
6057 times, but see @ref{Tuning the Parser,,@option{-D @var{name}[=@var{value}]}}.
6060 The rest of this section summarizes variables and values that @code{%define}
6063 Some @var{variable}s take Boolean values. In this case, Bison will complain
6064 if the variable definition does not meet one of the following four
6068 @item @code{@var{value}} is @code{true}
6070 @item @code{@var{value}} is omitted (or @code{""} is specified).
6071 This is equivalent to @code{true}.
6073 @item @code{@var{value}} is @code{false}.
6075 @item @var{variable} is never defined.
6076 In this case, Bison selects a default value.
6079 What @var{variable}s are accepted, as well as their meanings and default
6080 values, depend on the selected target language and/or the parser skeleton
6081 (@pxref{Decl Summary}, @pxref{Decl Summary}).
6082 Unaccepted @var{variable}s produce an error. Some of the accepted
6083 @var{variable}s are described below.
6086 @c ================================================== api.filename.file
6087 @anchor{api-filename-type}
6088 @deffn {Directive} {%define api.filename.type} @{@var{type}@}
6091 @item Language(s): C++
6094 Define the type of file names in Bison's default location and position
6095 types. @xref{Exposing the Location Classes}.
6097 @item Accepted Values:
6098 Any type that is printable (via streams) and comparable (with @code{==} and
6101 @item Default Value: @code{const std::string}.
6104 Introduced in Bison 2.0 as @code{filename_type} (with @code{std::string} as
6105 default), renamed as @code{api.filename.type} in Bison 3.7 (with @code{const
6106 std::string} as default).
6111 @c ================================================== api.header.include
6112 @deffn Directive {%define api.header.include} @{"header.h"@}
6113 @deffnx Directive {%define api.header.include} @{<header.h>@}
6115 @item Languages(s): C (@file{yacc.c})
6117 @item Purpose: Specify how the generated parser should include the generated header.
6119 Historically, when option @option{-d} or @option{--header} was used,
6120 @command{bison} generated a header and pasted an exact copy of it into the
6121 generated parser implementation file. Since Bison 3.6, it is
6122 @code{#include}d as @samp{"@var{basename}.h"}, instead of duplicated, unless
6123 @var{file} is @samp{y.tab}, see below.
6125 The @code{api.header.include} variable allows to control how the generated
6126 parser @code{#include}s the generated header. For instance:
6129 %define api.header.include @{"parse.h"@}
6136 %define api.header.include @{<parser/parse.h>@}
6139 Using @code{api.header.include} does not change the name of the generated
6140 header, only how it is included.
6142 To work around limitations of Automake's @command{ylwrap} (which runs
6143 @command{bison} with @option{--yacc}), @code{api.header.include} is
6144 @emph{not} predefined when the output file is @file{y.tab.c}. Define it to
6145 avoid the duplication.
6147 @item Accepted Values:
6148 An argument for @code{#include}.
6150 @item Default Value:
6151 @samp{"@var{header-basename}"}, unless the header file is @file{y.tab.h},
6152 where @var{header-basename} is the name of the generated header, without
6153 directory part. For instance with @command{bison -d calc/parse.y},
6154 @code{api.header.include} defaults to @samp{"parse.h"}, not
6155 @samp{"calc/parse.h"}.
6158 Introduced in Bison 3.4. Defaults to @samp{"@var{basename}.h"} since Bison
6159 3.7, unless the header file is @file{y.tab.h}.
6162 @c api.header.include
6165 @c ================================================== api.location.file
6166 @deffn {Directive} {%define api.location.file} "@var{file}"
6167 @deffnx {Directive} {%define api.location.file} @code{none}
6170 @item Language(s): C++
6173 Define the name of the file in which Bison's default location and position
6174 types are generated. @xref{Exposing the Location Classes}.
6176 @item Accepted Values:
6179 If locations are enabled, generate the definition of the @code{position} and
6180 @code{location} classes in the header file if @code{%header}, otherwise in
6181 the parser implementation.
6184 Generate the definition of the @code{position} and @code{location} classes
6185 in @var{file}. This file name can be relative (to where the parser file is
6186 output) or absolute.
6189 @item Default Value:
6190 Not applicable if locations are not enabled, or if a user location type is
6191 specified (see @code{api.location.type}). Otherwise, Bison's
6192 @code{location} is generated in @file{location.hh} (@pxref{C++ location}).
6195 Introduced in Bison 3.2.
6200 @c ================================================== api.location.file
6201 @deffn {Directive} {%define api.location.include} @{"@var{file}"@}
6202 @deffnx {Directive} {%define api.location.include} @{<@var{file}>@}
6205 @item Language(s): C++
6208 Specify how the generated file that defines the @code{position} and
6209 @code{location} classes is included. This makes sense when the
6210 @code{location} class is exposed to the rest of your application/library in
6211 another directory. @xref{Exposing the Location Classes}.
6213 @item Accepted Values: Argument for @code{#include}.
6215 @item Default Value:
6216 @samp{"@var{dir}/location.hh"} where @var{dir} is the directory part of the
6217 output. For instance @file{src/parse} if
6218 @option{--output=src/parse/parser.cc} was given.
6221 Introduced in Bison 3.2.
6227 @c ================================================== api.location.type
6228 @deffn {Directive} {%define api.location.type} @{@var{type}@}
6231 @item Language(s): C, C++, Java
6233 @item Purpose: Define the location type.
6234 @xref{User Defined Location Type}.
6236 @item Accepted Values: String
6238 @item Default Value: none
6241 Introduced in Bison 2.7 for C++ and Java, in Bison 3.4 for C. Was
6242 originally named @code{location_type} in Bison 2.5 and 2.6.
6247 @c ================================================== api.namespace
6248 @deffn Directive {%define api.namespace} @{@var{namespace}@}
6250 @item Languages(s): C++
6252 @item Purpose: Specify the namespace for the parser class.
6253 For example, if you specify:
6256 %define api.namespace @{foo::bar@}
6259 Bison uses @code{foo::bar} verbatim in references such as:
6262 foo::bar::parser::value_type
6265 However, to open a namespace, Bison removes any leading @code{::} and then
6266 splits on any remaining occurrences:
6269 namespace foo @{ namespace bar @{
6275 @item Accepted Values:
6276 Any absolute or relative C++ namespace reference without a trailing
6277 @code{"::"}. For example, @code{"foo"} or @code{"::foo::bar"}.
6279 @item Default Value:
6280 @code{yy}, unless you used the obsolete @samp{%name-prefix "@var{prefix}"}
6287 @c ================================================== api.parser.class
6288 @deffn Directive {%define api.parser.class} @{@var{name}@}
6294 The name of the parser class.
6296 @item Accepted Values:
6297 Any valid identifier.
6299 @item Default Value:
6300 In C++, @code{parser}. In D and Java, @code{YYParser} or
6301 @code{@var{api.prefix}Parser} (@pxref{Java Bison Interface}).
6304 Introduced in Bison 3.3 to replace @code{parser_class_name}.
6310 @c ================================================== api.prefix
6311 @deffn {Directive} {%define api.prefix} @{@var{prefix}@}
6314 @item Language(s): All
6316 @item Purpose: Rename exported symbols.
6317 @xref{Multiple Parsers}.
6319 @item Accepted Values: String
6321 @item Default Value: @code{YY} for Java, @code{yy} otherwise.
6324 introduced in Bison 2.6, with its argument in double quotes. Uses braces
6325 since Bison 3.0 (double quotes are still supported for backward
6331 @c ================================================== api.pure
6332 @deffn Directive {%define api.pure} @var{purity}
6335 @item Language(s): C
6337 @item Purpose: Request a pure (reentrant) parser program.
6340 @item Accepted Values: @code{true}, @code{false}, @code{full}
6342 The value may be omitted: this is equivalent to specifying @code{true}, as is
6343 the case for Boolean values.
6345 When @code{%define api.pure full} is used, the parser is made reentrant. This
6346 changes the signature for @code{yylex} (@pxref{Pure Calling}), and also that of
6347 @code{yyerror} when the tracking of locations has been activated, as shown
6350 The @code{true} value is very similar to the @code{full} value, the only
6351 difference is in the signature of @code{yyerror} on Yacc parsers without
6352 @code{%parse-param}, for historical reasons.
6354 I.e., if @samp{%locations %define api.pure} is passed then the prototypes for
6358 void yyerror (char const *msg); // Yacc parsers.
6359 void yyerror (YYLTYPE *locp, char const *msg); // GLR parsers.
6362 But if @samp{%locations %define api.pure %parse-param @{int *nastiness@}} is
6363 used, then both parsers have the same signature:
6366 void yyerror (YYLTYPE *llocp, int *nastiness, char const *msg);
6369 (@pxref{Error Reporting Function})
6371 @item Default Value: @code{false}
6374 the @code{full} value was introduced in Bison 2.7
6381 @c ================================================== api.push-pull
6382 @deffn Directive {%define api.push-pull} @var{kind}
6385 @item Language(s): C (deterministic parsers only), Java
6387 @item Purpose: Request a pull parser, a push parser, or both.
6390 @item Accepted Values: @code{pull}, @code{push}, @code{both}
6392 @item Default Value: @code{pull}
6399 @c ================================================== api.symbol.prefix
6400 @deffn Directive {%define api.symbol.prefix} @{@var{prefix}@}
6403 @item Languages(s): all
6406 Add a prefix to the name of the symbol kinds. For instance
6409 %define api.symbol.prefix @{S_@}
6410 %token FILE for ERROR
6412 start: FILE for ERROR;
6416 generates this definition in C:
6420 enum yysymbol_kind_t
6422 S_YYEMPTY = -2, /* No symbol. */
6423 S_YYEOF = 0, /* $end */
6424 S_YYERROR = 1, /* error */
6425 S_YYUNDEF = 2, /* $undefined */
6426 S_FILE = 3, /* FILE */
6427 S_for = 4, /* for */
6428 S_ERROR = 5, /* ERROR */
6429 S_YYACCEPT = 6, /* $accept */
6430 S_start = 7 /* start */
6434 @item Accepted Values:
6435 Any non empty string. Must be a valid identifier in the target language
6436 (typically a non empty sequence of letters, underscores, and ---not at the
6437 beginning--- digits).
6439 The empty prefix is (generally) invalid:
6442 in C it would create collision with the @code{YYERROR} macro, and
6443 potentially token kind definitions and symbol kind definitions would
6446 unnamed symbols (such as @samp{'+'}) have a name which starts with a digit;
6448 even in languages with scoped enumerations such as Java, an empty prefix is
6449 dangerous: symbol names may collide with the target language keywords, or
6450 with other members of the @code{SymbolKind} class.
6454 @item Default Value:
6455 @code{YYSYMBOL_} in C. @code{S_} in C++ and Java. The default prefix is
6458 introduced in Bison 3.6.
6461 @c api.symbol.prefix
6464 @c ================================================== api.token.constructor
6465 @deffn Directive {%define api.token.constructor}
6472 When variant-based semantic values are enabled (@pxref{C++ Variants}),
6473 request that symbols be handled as a whole (type, value, and possibly
6474 location) in the scanner. @xref{Complete Symbols}, for details.
6476 @item Accepted Values:
6479 @item Default Value:
6482 introduced in Bison 3.0.
6485 @c api.token.constructor
6488 @c ================================================== api.token.prefix
6489 @anchor{api-token-prefix}
6490 @deffn Directive {%define api.token.prefix} @{@var{prefix}@}
6492 @item Languages(s): all
6495 Add a prefix to the token names when generating their definition in the
6496 target language. For instance
6499 %define api.token.prefix @{TOK_@}
6500 %token FILE for ERROR
6502 start: FILE for ERROR;
6506 generates the definition of the symbols @code{TOK_FILE}, @code{TOK_for}, and
6507 @code{TOK_ERROR} in the generated source files. In particular, the scanner
6508 must use these prefixed token names, while the grammar itself may still use
6509 the short names (as in the sample rule given above). The generated
6510 informational files (@file{*.output}, @file{*.xml}, @file{*.gv}) are not
6511 modified by this prefix.
6513 Bison also prefixes the generated member names of the semantic value union.
6514 @xref{Type Generation}, for more
6517 See @ref{Calc++ Parser} and @ref{Calc++ Scanner}, for a complete example.
6519 @item Accepted Values:
6520 Any string. Must be a valid identifier prefix in the target language
6521 (typically, a possibly empty sequence of letters, underscores, and ---not at
6522 the beginning--- digits).
6524 @item Default Value:
6527 introduced in Bison 3.0.
6533 @c ================================================== api.token.raw
6534 @deffn Directive {%define api.token.raw}
6541 The output files normally define the enumeration of the @emph{token kinds}
6542 with Yacc-compatible token codes: sequential numbers starting at 257 except
6543 for single character tokens which stand for themselves (e.g., in ASCII,
6544 @samp{'a'} is numbered 65). The parser however uses @emph{symbol kinds}
6545 which are assigned numbers sequentially starting at 0. Therefore each time
6546 the scanner returns an (external) token kind, it must be mapped to the
6547 (internal) symbol kind.
6549 When @code{api.token.raw} is set, the code of the token kinds are forced to
6550 coincide with the symbol kind. This saves one table lookup per token to map
6551 them from the token kind to the symbol kind, and also saves the generation
6552 of the mapping table. The gain is typically moderate, but in extreme cases
6553 (very simple user actions), a 10% improvement can be observed.
6555 When @code{api.token.raw} is set, the grammar cannot use character literals
6556 (such as @samp{'a'}).
6558 @item Accepted Values: Boolean.
6560 @item Default Value:
6561 @code{true} in D, @code{false} otherwise
6563 introduced in Bison 3.5. Was initially introduced in Bison 1.25 as
6564 @samp{%raw}, but never worked and was removed in Bison 1.29.
6570 @c ================================================== api.value.automove
6571 @deffn Directive {%define api.value.automove}
6578 Let occurrences of semantic values of the right-hand sides of a rule be
6579 implicitly turned in rvalues. When enabled, a grammar such as:
6583 "number" @{ $$ = make_number ($1); @}
6584 | exp "+" exp @{ $$ = make_binary (add, $1, $3); @}
6585 | "(" exp ")" @{ $$ = $2; @}
6589 is actually compiled as if you had written:
6593 "number" @{ $$ = make_number (std::move ($1)); @}
6594 | exp "+" exp @{ $$ = make_binary (add,
6597 | "(" exp ")" @{ $$ = std::move ($2); @}
6600 Using a value several times with automove enabled is typically an error.
6601 For instance, instead of:
6604 exp: "twice" exp @{ $$ = make_binary (add, $2, $2); @}
6611 exp: "twice" exp @{ auto v = $2; $$ = make_binary (add, v, v); @}
6615 It is tempting to use @code{std::move} on one of the @code{v}, but the
6616 argument evaluation order in C++ is unspecified.
6618 @item Accepted Values:
6621 @item Default Value:
6624 introduced in Bison 3.2
6627 @c api.value.automove
6630 @c ================================================== api.value.type
6631 @deffn Directive {%define api.value.type} @var{support}
6632 @deffnx Directive {%define api.value.type} @{@var{type}@}
6638 The type for semantic values.
6640 @item Accepted Values:
6643 This grammar has no semantic value at all. This is not properly supported
6645 @item @samp{union-directive} (C, C++, D)
6646 The type is defined thanks to the @code{%union} directive. You don't have
6647 to define @code{api.value.type} in that case, using @code{%union} suffices.
6651 %define api.value.type union-directive
6657 %token <ival> INT "integer"
6658 %token <sval> STR "string"
6661 @item @samp{union} (C, C++)
6662 The symbols are defined with type names, from which Bison will generate a
6663 @code{union}. For instance:
6665 %define api.value.type union
6666 %token <int> INT "integer"
6667 %token <char *> STR "string"
6669 Most C++ objects cannot be stored in a @code{union}, use @samp{variant}
6672 @item @samp{variant} (C++)
6673 This is similar to @code{union}, but special storage techniques are used to
6674 allow any kind of C++ object to be used. For instance:
6676 %define api.value.type variant
6677 %token <int> INT "integer"
6678 %token <std::string> STR "string"
6680 @xref{C++ Variants}.
6682 @item @samp{@{@var{type}@}}
6683 Use this @var{type} as semantic value.
6700 %define api.value.type @{struct my_value@}
6701 %token <u.ival> INT "integer"
6702 %token <u.sval> STR "string"
6706 @item Default Value:
6709 @code{union-directive} if @code{%union} is used, otherwise @dots{}
6711 @code{int} if type tags are used (i.e., @samp{%token <@var{type}>@dots{}} or
6712 @samp{%nterm <@var{type}>@dots{}} is used), otherwise @dots{}
6718 introduced in Bison 3.0. Was introduced for Java only in 2.3b as
6725 @c ================================================== api.value.union.name
6726 @deffn Directive {%define api.value.union.name} @var{name}
6732 The tag of the generated @code{union} (@emph{not} the name of the
6733 @code{typedef}). This variable is set to @code{@var{id}} when @samp{%union
6734 @var{id}} is used. There is no clear reason to give this union a name.
6736 @item Accepted Values:
6737 Any valid identifier.
6739 @item Default Value:
6743 Introduced in Bison 3.0.3.
6749 @c ================================================== lr.default-reduction
6751 @deffn Directive {%define lr.default-reduction} @var{when}
6754 @item Language(s): all
6756 @item Purpose: Specify the kind of states that are permitted to
6757 contain default reductions. @xref{Default Reductions}.
6759 @item Accepted Values: @code{most}, @code{consistent}, @code{accepting}
6760 @item Default Value:
6762 @item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
6763 @item @code{most} otherwise.
6766 introduced as @code{lr.default-reductions} in 2.5, renamed as
6767 @code{lr.default-reduction} in 3.0.
6772 @c ============================================ lr.keep-unreachable-state
6774 @deffn Directive {%define lr.keep-unreachable-state}
6777 @item Language(s): all
6778 @item Purpose: Request that Bison allow unreachable parser states to
6779 remain in the parser tables. @xref{Unreachable States}.
6780 @item Accepted Values: Boolean
6781 @item Default Value: @code{false}
6783 introduced as @code{lr.keep_unreachable_states} in 2.3b, renamed as
6784 @code{lr.keep-unreachable-states} in 2.5, and as
6785 @code{lr.keep-unreachable-state} in 3.0.
6788 @c lr.keep-unreachable-state
6791 @c ================================================== lr.type
6793 @deffn Directive {%define lr.type} @var{type}
6796 @item Language(s): all
6798 @item Purpose: Specify the type of parser tables within the
6799 LR(1) family. @xref{LR Table Construction}.
6801 @item Accepted Values: @code{lalr}, @code{ielr}, @code{canonical-lr}
6803 @item Default Value: @code{lalr}
6808 @c ================================================== namespace
6809 @deffn Directive %define namespace @{@var{namespace}@}
6810 Obsoleted by @code{api.namespace}
6815 @c ================================================== parse.assert
6816 @deffn Directive {%define parse.assert}
6819 @item Languages(s): C, C++
6821 @item Purpose: Issue runtime assertions to catch invalid uses.
6822 In C, some important invariants in the implementation of the parser are
6823 checked when this option is enabled.
6825 In C++, when variants are used (@pxref{C++ Variants}), symbols must be
6826 constructed and destroyed properly. This option checks these constraints
6827 using runtime type information (RTTI). Therefore the generated code cannot
6828 be compiled with RTTI disabled (via compiler options such as
6829 @option{-fno-rtti}).
6831 @item Accepted Values: Boolean
6833 @item Default Value: @code{false}
6839 @c ================================================== parse.error
6840 @deffn Directive {%define parse.error} @var{verbosity}
6845 Control the generation of syntax error messages. @xref{Error Reporting}.
6846 @item Accepted Values:
6849 Error messages passed to @code{yyerror} are simply @w{@code{"syntax
6852 @item @code{detailed}
6853 Error messages report the unexpected token, and possibly the expected ones.
6854 However, this report can often be incorrect when LAC is not enabled
6855 (@pxref{LAC}). Token name internationalization is supported.
6857 @item @code{verbose}
6858 Similar (but inferior) to @code{detailed}.
6860 Error messages report the unexpected token, and possibly the expected ones.
6861 However, this report can often be incorrect when LAC is not enabled
6864 Does not support token internationalization. Using non-ASCII characters in
6865 token aliases is not portable.
6868 The user is in charge of generating the syntax error message by defining the
6869 @code{yyreport_syntax_error} function. @xref{Syntax Error Reporting
6873 @item Default Value:
6877 introduced in 3.0 with support for @code{simple} and @code{verbose}. Values
6878 @code{custom} and @code{detailed} were introduced in 3.6.
6884 @c ================================================== parse.lac
6885 @deffn Directive {%define parse.lac} @var{when}
6888 @item Languages(s): C/C++ (deterministic parsers only), D and Java.
6890 @item Purpose: Enable LAC (lookahead correction) to improve
6891 syntax error handling. @xref{LAC}.
6892 @item Accepted Values: @code{none}, @code{full}
6893 @item Default Value: @code{none}
6899 @c ================================================== parse.trace
6900 @deffn Directive {%define parse.trace}
6903 @item Languages(s): C, C++, D, Java
6905 @item Purpose: Require parser instrumentation for tracing.
6908 In C/C++, define the macro @code{YYDEBUG} (or @code{@var{prefix}DEBUG} with
6909 @samp{%define api.prefix @{@var{prefix}@}}), see @ref{Multiple Parsers}) to
6910 1 in the parser implementation file if it is not already defined, so that
6911 the debugging facilities are compiled.
6913 @item Accepted Values: Boolean
6915 @item Default Value: @code{false}
6921 @c ================================================== parser_class_name
6922 @deffn Directive %define parser_class_name @{@var{name}@}
6923 Obsoleted by @code{api.parser.class}
6925 @c parser_class_name
6933 @subsection %code Summary
6937 The @code{%code} directive inserts code verbatim into the output
6938 parser source at any of a predefined set of locations. It thus serves
6939 as a flexible and user-friendly alternative to the traditional Yacc
6940 prologue, @code{%@{@var{code}%@}}. This section summarizes the
6941 functionality of @code{%code} for the various target languages
6942 supported by Bison. For a detailed discussion of how to use
6943 @code{%code} in place of @code{%@{@var{code}%@}} for C/C++ and why it
6944 is advantageous to do so, @pxref{Prologue Alternatives}.
6946 @deffn {Directive} %code @{@var{code}@}
6947 This is the unqualified form of the @code{%code} directive. It
6948 inserts @var{code} verbatim at a language-dependent default location
6949 in the parser implementation.
6951 For C/C++, the default location is the parser implementation file
6952 after the usual contents of the parser header file. Thus, the
6953 unqualified form replaces @code{%@{@var{code}%@}} for most purposes.
6955 For D and Java, the default location is inside the parser class.
6958 @deffn {Directive} %code @var{qualifier} @{@var{code}@}
6959 This is the qualified form of the @code{%code} directive.
6960 @var{qualifier} identifies the purpose of @var{code} and thus the
6961 location(s) where Bison should insert it. That is, if you need to
6962 specify location-sensitive @var{code} that does not belong at the
6963 default location selected by the unqualified @code{%code} form, use
6967 For any particular qualifier or for the unqualified form, if there are
6968 multiple occurrences of the @code{%code} directive, Bison concatenates
6969 the specified code in the order in which it appears in the grammar
6972 Not all qualifiers are accepted for all target languages. Unaccepted
6973 qualifiers produce an error. Some of the accepted qualifiers are:
6977 @findex %code requires
6980 @item Language(s): C, C++
6982 @item Purpose: This is the best place to write dependency code required for
6983 @code{YYSTYPE} and @code{YYLTYPE}. In other words, it's the best place to
6984 define types referenced in @code{%union} directives. If you use
6985 @code{#define} to override Bison's default @code{YYSTYPE} and @code{YYLTYPE}
6986 definitions, then it is also the best place. However you should rather
6987 @code{%define} @code{api.value.type} and @code{api.location.type}.
6989 @item Location(s): The parser header file and the parser implementation file
6990 before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
6995 @findex %code provides
6998 @item Language(s): C, C++
7000 @item Purpose: This is the best place to write additional definitions and
7001 declarations that should be provided to other modules.
7003 @item Location(s): The parser header file and the parser implementation
7004 file after the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and
7012 @item Language(s): C, C++
7014 @item Purpose: The unqualified @code{%code} or @code{%code requires}
7015 should usually be more appropriate than @code{%code top}. However,
7016 occasionally it is necessary to insert code much nearer the top of the
7017 parser implementation file. For example:
7026 @item Location(s): Near the top of the parser implementation file.
7030 @findex %code imports
7033 @item Language(s): D, Java
7035 @item Purpose: This is the best place to write Java import directives. D syntax
7036 allows for import statements all throughout the code.
7038 @item Location(s): The parser Java file after any Java package directive and
7039 before any class definitions. The parser D file before any class definitions.
7043 Though we say the insertion locations are language-dependent, they are
7044 technically skeleton-dependent. Writers of non-standard skeletons
7045 however should choose their locations consistently with the behavior
7046 of the standard Bison skeletons.
7049 @node Multiple Parsers
7050 @section Multiple Parsers in the Same Program
7052 Most programs that use Bison parse only one language and therefore contain
7053 only one Bison parser. But what if you want to parse more than one language
7054 with the same program? Then you need to avoid name conflicts between
7055 different definitions of functions and variables such as @code{yyparse},
7056 @code{yylval}. To use different parsers from the same compilation unit, you
7057 also need to avoid conflicts on types and macros (e.g., @code{YYSTYPE})
7058 exported in the generated header.
7060 The easy way to do this is to define the @code{%define} variable
7061 @code{api.prefix}. With different @code{api.prefix}s it is guaranteed that
7062 headers do not conflict when included together, and that compiled objects
7063 can be linked together too. Specifying @samp{%define api.prefix
7064 @{@var{prefix}@}} (or passing the option @option{-Dapi.prefix=@{@var{prefix}@}}, see
7065 @ref{Invocation}) renames the interface functions and
7066 variables of the Bison parser to start with @var{prefix} instead of
7067 @samp{yy}, and all the macros to start by @var{PREFIX} (i.e., @var{prefix}
7068 upper-cased) instead of @samp{YY}.
7070 The renamed symbols include @code{yyparse}, @code{yylex}, @code{yyerror},
7071 @code{yynerrs}, @code{yylval}, @code{yylloc}, @code{yychar} and
7072 @code{yydebug}. If you use a push parser, @code{yypush_parse},
7073 @code{yypull_parse}, @code{yypstate}, @code{yypstate_new} and
7074 @code{yypstate_delete} will also be renamed. The renamed macros include
7075 @code{YYSTYPE}, @code{YYLTYPE}, and @code{YYDEBUG}, which is treated
7076 specifically --- more about this below.
7078 For example, if you use @samp{%define api.prefix @{c@}}, the names become
7079 @code{cparse}, @code{clex}, @dots{}, @code{CSTYPE}, @code{CLTYPE}, and so
7082 Users of Flex must update the signature of the generated @code{yylex}
7083 function. Since the Flex scanner usually includes the generated header of
7084 the parser (to get the definitions of the tokens, etc.), the most convenient
7085 way is to insert the declaration of @code{yylex} in the @code{provides}
7089 %define api.prefix @{c@}
7090 // Emitted in the header file, after the definition of YYSTYPE.
7093 // Tell Flex the expected prototype of yylex.
7095 int clex (CSTYPE *yylval, CLTYPE *yylloc)
7097 // Declare the scanner.
7104 The @code{%define} variable @code{api.prefix} works in two different ways.
7105 In the implementation file, it works by adding macro definitions to the
7106 beginning of the parser implementation file, defining @code{yyparse} as
7107 @code{@var{prefix}parse}, and so on:
7110 #define YYSTYPE CTYPE
7111 #define yyparse cparse
7112 #define yylval clval
7118 This effectively substitutes one name for the other in the entire parser
7119 implementation file, thus the ``original'' names (@code{yylex},
7120 @code{YYSTYPE}, @dots{}) are also usable in the parser implementation file.
7122 However, in the parser header file, the symbols are defined renamed, for
7126 extern CSTYPE clval;
7130 The macro @code{YYDEBUG} is commonly used to enable the tracing support in
7131 parsers. To comply with this tradition, when @code{api.prefix} is used,
7132 @code{YYDEBUG} (not renamed) is used as a default value:
7137 # if defined YYDEBUG
7154 Prior to Bison 2.6, a feature similar to @code{api.prefix} was provided by
7155 the obsolete directive @code{%name-prefix} (@pxref{Table of Symbols}) and
7156 the option @option{--name-prefix} (@pxref{Output Files}).
7159 @chapter Parser C-Language Interface
7160 @cindex C-language interface
7163 The Bison parser is actually a C function named @code{yyparse}. Here we
7164 describe the interface conventions of @code{yyparse} and the other
7165 functions that it needs to use.
7167 Keep in mind that the parser uses many C identifiers starting with
7168 @samp{yy} and @samp{YY} for internal purposes. If you use such an
7169 identifier (aside from those in this manual) in an action or in epilogue
7170 in the grammar file, you are likely to run into trouble.
7173 * Parser Function:: How to call @code{yyparse} and what it returns.
7174 * Push Parser Interface:: How to create, use, and destroy push parsers.
7175 * Lexical:: You must supply a function @code{yylex}
7177 * Error Reporting:: Passing error messages to the user.
7178 * Action Features:: Special features for use in actions.
7179 * Internationalization:: How to let the parser speak in the user's
7183 @node Parser Function
7184 @section The Parser Function @code{yyparse}
7187 You call the function @code{yyparse} to cause parsing to occur. This
7188 function reads tokens, executes actions, and ultimately returns when it
7189 encounters end-of-input or an unrecoverable syntax error. You can also
7190 write an action which directs @code{yyparse} to return immediately
7191 without reading further.
7194 @deftypefun int yyparse (@code{void})
7195 The value returned by @code{yyparse} is 0 if parsing was successful (return
7196 is due to end-of-input).
7198 The value is 1 if parsing failed because of invalid input, i.e., input
7199 that contains a syntax error or that causes @code{YYABORT} to be
7202 The value is 2 if parsing failed due to memory exhaustion.
7205 In an action, you can cause immediate return from @code{yyparse} by using
7210 Return immediately with value 0 (to report success).
7215 Return immediately with value 1 (to report failure).
7220 Return immediately with value 2 (to report memory exhaustion).
7223 If you use a reentrant parser, you can optionally pass additional
7224 parameter information to it in a reentrant way. To do so, use the
7225 declaration @code{%parse-param}:
7227 @deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
7228 @findex %parse-param
7229 Declare that one or more
7230 @var{argument-declaration} are additional @code{yyparse} arguments.
7231 The @var{argument-declaration} is used when declaring
7232 functions or prototypes. The last identifier in
7233 @var{argument-declaration} must be the argument name.
7236 Here's an example. Write this in the parser:
7239 %parse-param @{int *nastiness@} @{int *randomness@}
7243 Then call the parser like this:
7247 int nastiness, randomness;
7248 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
7249 value = yyparse (&nastiness, &randomness);
7255 In the grammar actions, use expressions like this to refer to the data:
7258 exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
7262 Using the following:
7264 %parse-param @{int *randomness@}
7267 Results in these signatures:
7269 void yyerror (int *randomness, const char *msg);
7270 int yyparse (int *randomness);
7274 Or, if both @code{%define api.pure full} (or just @code{%define api.pure})
7275 and @code{%locations} are used:
7278 void yyerror (YYLTYPE *llocp, int *randomness, const char *msg);
7279 int yyparse (int *randomness);
7282 @node Push Parser Interface
7283 @section Push Parser Interface
7285 @findex yypstate_new
7286 You call the function @code{yypstate_new} to create a new parser instance.
7287 This function is available if either the @samp{%define api.push-pull push}
7288 or @samp{%define api.push-pull both} declaration is used. @xref{Push Decl}.
7290 @anchor{yypstate_new}
7291 @deftypefun {yypstate*} yypstate_new (@code{void})
7292 Return a valid parser instance if there is memory available, 0 otherwise.
7293 In impure mode, it will also return 0 if a parser instance is currently
7297 @findex yypstate_delete
7298 You call the function @code{yypstate_delete} to delete a parser instance.
7299 function is available if either the @samp{%define api.push-pull push} or
7300 @samp{%define api.push-pull both} declaration is used.
7303 @anchor{yypstate_delete}
7304 @deftypefun void yypstate_delete (@code{yypstate *}@var{yyps})
7305 Reclaim the memory associated with a parser instance. After this call, you
7306 should no longer attempt to use the parser instance.
7309 @findex yypush_parse
7310 You call the function @code{yypush_parse} to parse a single token. This
7311 function is available if either the @samp{%define api.push-pull push} or
7312 @samp{%define api.push-pull both} declaration is used. @xref{Push Decl}.
7314 @anchor{yypush_parse}
7315 @deftypefun int yypush_parse (@code{yypstate *}@var{yyps})
7316 The value returned by @code{yypush_parse} is the same as for @code{yyparse}
7317 with the following exception: it returns @code{YYPUSH_MORE} if more input is
7318 required to finish parsing the grammar.
7320 After @code{yypush_parse} returned, the instance may be consulted. For
7321 instance check @code{yynerrs} to see whether there were (possibly recovered)
7324 After @code{yypush_parse} returns a status other than @code{YYPUSH_MORE},
7325 the parser instance @code{yyps} may be reused for a new parse.
7328 The fact that the parser state is reusable even after an error simplifies
7329 reuse. For example, a calculator application which parses each input line
7330 as an expression can just keep reusing the same @code{yyps} even if an input
7333 You call the function @code{yypull_parse} to parse the rest of the input
7334 stream. This function is available if the @samp{%define api.push-pull both}
7335 declaration is used. @xref{Push Decl}.
7337 @anchor{yypull_parse}
7338 @deftypefun int yypull_parse (@code{yypstate *}@var{yyps})
7339 The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
7341 The parser instance @code{yyps} may be reused for new parses.
7344 @deftypefun int yypstate_expected_tokens (@code{const yypstate *}yyps, @code{yysymbol_kind_t} @var{argv}@code{[]}, @code{int} @var{argc})
7345 Fill @var{argv} with the expected tokens, which never includes
7346 @code{YYSYMBOL_YYEMPTY}, @code{YYSYMBOL_YYerror}, or
7347 @code{YYSYMBOL_YYUNDEF}.
7349 Never put more than @var{argc} elements into @var{argv}, and on success
7350 return the number of tokens stored in @var{argv}. If there are more
7351 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
7352 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
7353 to @code{YYSYMBOL_YYEMPTY}.
7355 When LAC is enabled, may return a negative number on errors,
7356 such as @code{YYENOMEM} on memory exhaustion.
7358 If @var{argv} is null, return the size needed to store all the possible
7359 values, which is always less than @code{YYNTOKENS}.
7364 @section The Lexical Analyzer Function @code{yylex}
7366 @cindex lexical analyzer
7368 The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
7369 the input stream and returns them to the parser. Bison does not create
7370 this function automatically; you must write it so that @code{yyparse} can
7371 call it. The function is sometimes referred to as a lexical scanner.
7373 In simple programs, @code{yylex} is often defined at the end of the Bison
7374 grammar file. If @code{yylex} is defined in a separate source file, you
7375 need to arrange for the token-kind definitions to be available there. To do
7376 this, use the @option{-d} option when you run Bison, so that it will write
7377 these definitions into the separate parser header file,
7378 @file{@var{name}.tab.h}, which you can include in the other source files
7379 that need it. @xref{Invocation}.
7382 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
7383 * Special Tokens:: Signaling end-of-file and errors to the parser.
7384 * Tokens from Literals:: Finding token kinds from string aliases.
7385 * Token Values:: How @code{yylex} must return the semantic value
7386 of the token it has read.
7387 * Token Locations:: How @code{yylex} must return the text location
7388 (line number, etc.) of the token, if the
7390 * Pure Calling:: How the calling convention differs in a pure parser
7391 (@pxref{Pure Decl}).
7394 @node Calling Convention
7395 @subsection Calling Convention for @code{yylex}
7397 The value that @code{yylex} returns must be the positive numeric code for
7398 the kind of token it has just found; a zero or negative value signifies
7401 When a token kind is referred to in the grammar rules by a name, that name
7402 in the parser implementation file becomes an enumerator of the enum
7403 @code{yytoken_kind_t} whose definition is the proper numeric code for that
7404 token kind. So @code{yylex} should use the name to indicate that type.
7407 When a token is referred to in the grammar rules by a character literal, the
7408 numeric code for that character is also the code for the token kind. So
7409 @code{yylex} can simply return that character code, possibly converted to
7410 @code{unsigned char} to avoid sign-extension. The null character must not
7411 be used this way, because its code is zero and that signifies end-of-input.
7413 Here is an example showing these things:
7420 if (c == EOF) /* Detect end-of-input. */
7423 else if (c == '+' || c == '-')
7424 return c; /* Assume token kind for '+' is '+'. */
7427 return INT; /* Return the kind of the token. */
7433 This interface has been designed so that the output from the @code{lex}
7434 utility can be used without change as the definition of @code{yylex}.
7437 @node Special Tokens
7438 @subsection Special Tokens
7440 In addition to the user defined tokens, Bison generates a few special tokens
7441 that @code{yylex} may return.
7443 The @code{YYEOF} token denotes the end of file, and signals to the parser
7444 that there is nothing left afterwards. @xref{Calling Convention}, for an
7447 Returning @code{YYUNDEF} tells the parser that some lexical error was found.
7448 It will emit an error message about an ``invalid token'', and enter
7449 error-recovery (@pxref{Error Recovery}). Returning an unknown token kind
7450 results in the exact same behavior.
7452 Returning @code{YYerror} requires the parser to enter error-recovery
7453 @emph{without} emitting an error message. This way the lexical analyzer can
7454 produce an accurate error messages about the invalid input (something the
7455 parser cannot do), and yet benefit from the error-recovery features of the
7466 case '0': case '1': case '2': case '3': case '4':
7467 case '5': case '6': case '7': case '8': case '9':
7474 yyerror ("syntax error: invalid character: %c", c);
7480 @node Tokens from Literals
7481 @subsection Finding Tokens by String Literals
7483 If the grammar uses literal string tokens, there are two ways that
7484 @code{yylex} can determine the token kind codes for them:
7488 If the grammar defines symbolic token names as aliases for the literal
7489 string tokens, @code{yylex} can use these symbolic names like all others.
7490 In this case, the use of the literal string tokens in the grammar file has
7491 no effect on @code{yylex}.
7493 This is the preferred approach.
7496 @code{yylex} can search for the multicharacter token in the @code{yytname}
7497 table. This method is discouraged: the primary purpose of string aliases is
7498 forging good error messages, not describing the spelling of keywords. In
7499 addition, looking for the token kind at runtime incurs a (small but
7502 The @code{yytname} table is generated only if you use the
7503 @code{%token-table} declaration. @xref{Decl Summary}.
7508 @subsection Semantic Values of Tokens
7511 In an ordinary (nonreentrant) parser, the semantic value of the token must
7512 be stored into the global variable @code{yylval}. When you are using just
7513 one data type for semantic values, @code{yylval} has that type. Thus, if
7514 the type is @code{int} (the default), you might write this in @code{yylex}:
7519 yylval = value; /* Put value onto Bison stack. */
7520 return INT; /* Return the kind of the token. */
7525 When you are using multiple data types, @code{yylval}'s type is a union made
7526 from the @code{%union} declaration (@pxref{Union Decl}). So when you store
7527 a token's value, you must use the proper member of the union. If the
7528 @code{%union} declaration looks like this:
7541 then the code in @code{yylex} might look like this:
7546 yylval.intval = value; /* Put value onto Bison stack. */
7547 return INT; /* Return the kind of the token. */
7552 @node Token Locations
7553 @subsection Textual Locations of Tokens
7556 If you are using the @samp{@@@var{n}}-feature (@pxref{Tracking Locations})
7557 in actions to keep track of the textual locations of tokens and groupings,
7558 then you must provide this information in @code{yylex}. The function
7559 @code{yyparse} expects to find the textual location of a token just parsed
7560 in the global variable @code{yylloc}. So @code{yylex} must store the proper
7561 data in that variable.
7563 By default, the value of @code{yylloc} is a structure and you need only
7564 initialize the members that are going to be used by the actions. The
7565 four members are called @code{first_line}, @code{first_column},
7566 @code{last_line} and @code{last_column}. Note that the use of this
7567 feature makes the parser noticeably slower.
7570 The data type of @code{yylloc} has the name @code{YYLTYPE}.
7573 @subsection Calling Conventions for Pure Parsers
7575 When you use the Bison declaration @code{%define api.pure full} to request a
7576 pure, reentrant parser, the global communication variables @code{yylval} and
7577 @code{yylloc} cannot be used. (@xref{Pure Decl}.) In such parsers the two
7578 global variables are replaced by pointers passed as arguments to
7579 @code{yylex}. You must declare them as shown here, and pass the information
7580 back by storing it through those pointers.
7584 yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
7587 *lvalp = value; /* Put value onto Bison stack. */
7588 return INT; /* Return the kind of the token. */
7593 If the grammar file does not use the @samp{@@} constructs to refer to
7594 textual locations, then the type @code{YYLTYPE} will not be defined. In
7595 this case, omit the second argument; @code{yylex} will be called with
7598 If you wish to pass additional arguments to @code{yylex}, use
7599 @code{%lex-param} just like @code{%parse-param} (@pxref{Parser
7600 Function}). To pass additional arguments to both @code{yylex} and
7601 @code{yyparse}, use @code{%param}.
7603 @deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
7605 Specify that @var{argument-declaration} are additional @code{yylex} argument
7606 declarations. You may pass one or more such declarations, which is
7607 equivalent to repeating @code{%lex-param}.
7610 @deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
7612 Specify that @var{argument-declaration} are additional
7613 @code{yylex}/@code{yyparse} argument declaration. This is equivalent to
7614 @samp{%lex-param @{@var{argument-declaration}@} @dots{} %parse-param
7615 @{@var{argument-declaration}@} @dots{}}. You may pass one or more
7616 declarations, which is equivalent to repeating @code{%param}.
7623 %lex-param @{scanner_mode *mode@}
7624 %parse-param @{parser_mode *mode@}
7625 %param @{environment_type *env@}
7629 results in the following signatures:
7632 int yylex (scanner_mode *mode, environment_type *env);
7633 int yyparse (parser_mode *mode, environment_type *env);
7636 If @samp{%define api.pure full} is added:
7639 int yylex (YYSTYPE *lvalp, scanner_mode *mode, environment_type *env);
7640 int yyparse (parser_mode *mode, environment_type *env);
7644 and finally, if both @samp{%define api.pure full} and @code{%locations} are
7648 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp,
7649 scanner_mode *mode, environment_type *env);
7650 int yyparse (parser_mode *mode, environment_type *env);
7654 @node Error Reporting
7655 @section Error Reporting
7657 During its execution the parser may have error messages to pass to the user,
7658 such as syntax error, or memory exhaustion. How this message is delivered
7659 to the user must be specified by the developer.
7662 * Error Reporting Function:: You must supply a @code{yyerror} function.
7663 * Syntax Error Reporting Function:: You can supply a @code{yyreport_syntax_error} function.
7666 @node Error Reporting Function
7667 @subsection The Error Reporting Function @code{yyerror}
7668 @cindex error reporting function
7671 @cindex syntax error
7673 The Bison parser detects a @dfn{syntax error} (or @dfn{parse error})
7674 whenever it reads a token which cannot satisfy any syntax rule. An
7675 action in the grammar can also explicitly proclaim an error, using the
7676 macro @code{YYERROR} (@pxref{Action Features}).
7678 The Bison parser expects to report the error by calling an error
7679 reporting function named @code{yyerror}, which you must supply. It is
7680 called by @code{yyparse} whenever a syntax error is found, and it
7681 receives one argument. For a syntax error, the string is normally
7682 @w{@code{"syntax error"}}.
7684 @findex %define parse.error detailed
7685 @findex %define parse.error verbose
7686 If you invoke @samp{%define parse.error detailed} (or @samp{custom}) in the
7687 Bison declarations section (@pxref{Bison Declarations}), then Bison provides
7688 a more verbose and specific error message string instead of just plain
7689 @w{@code{"syntax error"}}. However, that message sometimes contains
7690 incorrect information if LAC is not enabled (@pxref{LAC}).
7692 The parser can detect one other kind of error: memory exhaustion. This
7693 can happen when the input contains constructions that are very deeply
7694 nested. It isn't likely you will encounter this, since the Bison
7695 parser normally extends its stack automatically up to a very large limit. But
7696 if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
7697 fashion, except that the argument string is @w{@code{"memory exhausted"}}.
7699 In some cases diagnostics like @w{@code{"syntax error"}} are
7700 translated automatically from English to some other language before
7701 they are passed to @code{yyerror}. @xref{Internationalization}.
7703 The following definition suffices in simple programs:
7708 yyerror (char const *s)
7712 fprintf (stderr, "%s\n", s);
7717 After @code{yyerror} returns to @code{yyparse}, the latter will attempt
7718 error recovery if you have written suitable error recovery grammar rules
7719 (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
7720 immediately return 1.
7722 Obviously, in location tracking pure parsers, @code{yyerror} should have
7723 an access to the current location. With @code{%define api.pure}, this is
7724 indeed the case for the GLR parsers, but not for the Yacc parser, for
7725 historical reasons, and this is the why @code{%define api.pure full} should be
7726 preferred over @code{%define api.pure}.
7728 When @code{%locations %define api.pure full} is used, @code{yyerror} has the
7729 following signature:
7732 void yyerror (YYLTYPE *locp, char const *msg);
7736 The prototypes are only indications of how the code produced by Bison
7737 uses @code{yyerror}. Bison-generated code always ignores the returned
7738 value, so @code{yyerror} can return any type, including @code{void}.
7739 Also, @code{yyerror} can be a variadic function; that is why the
7740 message is always passed last.
7742 Traditionally @code{yyerror} returns an @code{int} that is always
7743 ignored, but this is purely for historical reasons, and @code{void} is
7744 preferable since it more accurately describes the return type for
7748 The variable @code{yynerrs} contains the number of syntax errors
7749 reported so far. Normally this variable is global; but if you
7750 request a pure parser (@pxref{Pure Decl})
7751 then it is a local variable which only the actions can access.
7754 @node Syntax Error Reporting Function
7755 @subsection The Syntax Error Reporting Function @code{yyreport_syntax_error}
7757 @findex %define parse.error custom
7758 If you invoke @samp{%define parse.error custom} (@pxref{Bison
7759 Declarations}), then the parser no longer passes syntax error messages to
7760 @code{yyerror}, rather it delegates that task to the user by calling the
7761 @code{yyreport_syntax_error} function.
7763 The following functions and types are ``@code{static}'': they are defined in
7764 the implementation file (@file{*.c}) and available only from there. They
7765 are meant to be used from the grammar's epilogue.
7767 @deftypefun {static int} yyreport_syntax_error (@code{const yypcontext_t *}@var{ctx})
7768 Report a syntax error to the user. Return 0 on success, @code{YYENOMEM} on
7769 memory exhaustion. Whether it uses @code{yyerror} is up to the user.
7772 Use the following types and functions to build the error message.
7774 @deffn {Type} yypcontext_t
7775 An opaque type that captures the circumstances of the syntax error.
7778 @deffn {Type} yysymbol_kind_t
7779 An enum of all the grammar symbols, tokens and nonterminals. Its
7780 enumerators are forged from the symbol names:
7783 enum yysymbol_kind_t
7785 YYSYMBOL_YYEMPTY = -2, /* No symbol. */
7786 YYSYMBOL_YYEOF = 0, /* "end of file" */
7787 YYSYMBOL_YYerror = 1, /* error */
7788 YYSYMBOL_YYUNDEF = 2, /* "invalid token" */
7789 YYSYMBOL_PLUS = 3, /* "+" */
7790 YYSYMBOL_MINUS = 4, /* "-" */
7792 YYSYMBOL_VAR = 14, /* "variable" */
7793 YYSYMBOL_NEG = 15, /* NEG */
7794 YYSYMBOL_YYACCEPT = 16, /* $accept */
7795 YYSYMBOL_exp = 17, /* exp */
7796 YYSYMBOL_input = 18 /* input */
7798 typedef enum yysymbol_kind_t yysymbol_kind_t;
7802 @deftypefun {static yysymbol_kind_t} yypcontext_token (@code{const yypcontext_t *}@var{ctx})
7803 The ``unexpected'' token: the symbol kind of the lookahead token that caused
7804 the syntax error. Returns @code{YYSYMBOL_YYEMPTY} if there is no lookahead.
7807 @deftypefun {static YYLTYPE *} yypcontext_location (@code{const yypcontext_t *}@var{ctx})
7808 The location of the syntax error (that of the unexpected token).
7811 @deftypefun {static int} yypcontext_expected_tokens (@code{const yypcontext_t *}ctx, @code{yysymbol_kind_t} @var{argv}@code{[]}, @code{int} @var{argc})
7812 Fill @var{argv} with the expected tokens, which never includes
7813 @code{YYSYMBOL_YYEMPTY}, @code{YYSYMBOL_YYerror}, or
7814 @code{YYSYMBOL_YYUNDEF}.
7816 Never put more than @var{argc} elements into @var{argv}, and on success
7817 return the number of tokens stored in @var{argv}. If there are more
7818 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
7819 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
7820 to @code{YYSYMBOL_YYEMPTY}.
7822 When LAC is enabled, may return a negative number on errors,
7823 such as @code{YYENOMEM} on memory exhaustion.
7825 If @var{argv} is null, return the size needed to store all the possible
7826 values, which is always less than @code{YYNTOKENS}.
7829 @deftypefun {static const char *} yysymbol_name (@code{symbol_kind_t} @var{symbol})
7830 The name of the symbol whose kind is @var{symbol}, possibly translated.
7833 A custom syntax error function looks as follows. This implementation is
7834 inappropriate for internationalization, see the @file{c/bistromathic}
7835 example for a better alternative.
7839 yyreport_syntax_error (const yypcontext_t *ctx)
7842 YY_LOCATION_PRINT (stderr, *yypcontext_location (ctx));
7843 fprintf (stderr, ": syntax error");
7844 // Report the tokens expected at this point.
7846 enum @{ TOKENMAX = 5 @};
7847 yysymbol_kind_t expected[TOKENMAX];
7848 int n = yypcontext_expected_tokens (ctx, expected, TOKENMAX);
7850 // Forward errors to yyparse.
7853 for (int i = 0; i < n; ++i)
7854 fprintf (stderr, "%s %s",
7855 i == 0 ? ": expected" : " or", yysymbol_name (expected[i]));
7857 // Report the unexpected token.
7859 yysymbol_kind_t lookahead = yypcontext_token (ctx);
7860 if (lookahead != YYSYMBOL_YYEMPTY)
7861 fprintf (stderr, " before %s", yysymbol_name (lookahead));
7863 fprintf (stderr, "\n");
7868 You still must provide a @code{yyerror} function, used for instance to
7869 report memory exhaustion.
7871 @node Action Features
7872 @section Special Features for Use in Actions
7873 @cindex summary, action features
7874 @cindex action features summary
7876 Here is a table of Bison constructs, variables and macros that are useful in
7879 @deffn {Variable} $$
7880 Acts like a variable that contains the semantic value for the
7881 grouping made by the current rule. @xref{Actions}.
7884 @deffn {Variable} $@var{n}
7885 Acts like a variable that contains the semantic value for the
7886 @var{n}th component of the current rule. @xref{Actions}.
7889 @deffn {Variable} $<@var{typealt}>$
7890 Like @code{$$} but specifies alternative @var{typealt} in the union
7891 specified by the @code{%union} declaration. @xref{Action Types}.
7894 @deffn {Variable} $<@var{typealt}>@var{n}
7895 Like @code{$@var{n}} but specifies alternative @var{typealt} in the
7896 union specified by the @code{%union} declaration.
7897 @xref{Action Types}.
7900 @deffn {Macro} YYABORT @code{;}
7901 Return immediately from @code{yyparse}, indicating failure.
7902 @xref{Parser Function}.
7905 @deffn {Macro} YYACCEPT @code{;}
7906 Return immediately from @code{yyparse}, indicating success.
7907 @xref{Parser Function}.
7910 @deffn {Macro} YYBACKUP (@var{token}, @var{value})@code{;}
7912 Unshift a token. This macro is allowed only for rules that reduce
7913 a single value, and only when there is no lookahead token.
7914 It is also disallowed in GLR parsers.
7915 It installs a lookahead token with token kind @var{token} and
7916 semantic value @var{value}; then it discards the value that was
7917 going to be reduced by this rule.
7919 If the macro is used when it is not valid, such as when there is
7920 a lookahead token already, then it reports a syntax error with
7921 a message @samp{cannot back up} and performs ordinary error
7924 In either case, the rest of the action is not executed.
7927 @deffn {Value} YYEMPTY
7928 Value stored in @code{yychar} when there is no lookahead token.
7931 @deffn {Value} YYEOF
7932 Value stored in @code{yychar} when the lookahead is the end of the input
7936 @deffn {Macro} YYERROR @code{;}
7937 Cause an immediate syntax error. This statement initiates error
7938 recovery just as if the parser itself had detected an error; however, it
7939 does not call @code{yyerror}, and does not print any message. If you
7940 want to print an error message, call @code{yyerror} explicitly before
7941 the @samp{YYERROR;} statement. @xref{Error Recovery}.
7944 @deffn {Macro} YYNOMEM @code{;}
7945 Return immediately from @code{yyparse}, indicating memory exhaustion.
7946 @xref{Parser Function}.
7949 @deffn {Macro} YYRECOVERING
7950 @findex YYRECOVERING
7951 The expression @code{YYRECOVERING ()} yields 1 when the parser
7952 is recovering from a syntax error, and 0 otherwise.
7953 @xref{Error Recovery}.
7956 @deffn {Variable} yychar
7957 Variable containing either the lookahead token, or @code{YYEOF} when the
7958 lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
7959 has been performed so the next token is not yet known.
7960 Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
7965 @deffn {Macro} yyclearin @code{;}
7966 Discard the current lookahead token. This is useful primarily in
7968 Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
7970 @xref{Error Recovery}.
7973 @deffn {Macro} yyerrok @code{;}
7974 Resume generating error messages immediately for subsequent syntax
7975 errors. This is useful primarily in error rules.
7976 @xref{Error Recovery}.
7979 @deffn {Variable} yylloc
7980 Variable containing the lookahead token location when @code{yychar} is not set
7981 to @code{YYEMPTY} or @code{YYEOF}.
7982 Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
7984 @xref{Actions and Locations}.
7987 @deffn {Variable} yylval
7988 Variable containing the lookahead token semantic value when @code{yychar} is
7989 not set to @code{YYEMPTY} or @code{YYEOF}.
7990 Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
7996 Acts like a structure variable containing information on the textual
7997 location of the grouping made by the current rule. @xref{Tracking
8000 @c Check if those paragraphs are still useful or not.
8004 @c int first_line, last_line;
8005 @c int first_column, last_column;
8009 @c Thus, to get the starting line number of the third component, you would
8010 @c use @samp{@@3.first_line}.
8012 @c In order for the members of this structure to contain valid information,
8013 @c you must make @code{yylex} supply this information about each token.
8014 @c If you need only certain members, then @code{yylex} need only fill in
8017 @c The use of this feature makes the parser noticeably slower.
8020 @deffn {Value} @@@var{n}
8022 Acts like a structure variable containing information on the textual
8023 location of the @var{n}th component of the current rule. @xref{Tracking
8027 @node Internationalization
8028 @section Parser Internationalization
8029 @cindex internationalization
8035 A Bison-generated parser can print diagnostics, including error and
8036 tracing messages. By default, they appear in English. However, Bison
8037 also supports outputting diagnostics in the user's native language. To
8038 make this work, the user should set the usual environment variables.
8039 @xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
8040 For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
8041 set the user's locale to French Canadian using the UTF-8
8042 encoding. The exact set of available locales depends on the user's
8046 * Enabling I18n:: Preparing your project to support internationalization.
8047 * Token I18n:: Preparing tokens for internationalization in error messages.
8051 @subsection Enabling Internationalization
8053 The maintainer of a package that uses a Bison-generated parser enables
8054 the internationalization of the parser's output through the following
8055 steps. Here we assume a package that uses GNU Autoconf and
8060 @cindex bison-i18n.m4
8061 Into the directory containing the GNU Autoconf macros used
8062 by the package ---often called @file{m4}--- copy the
8063 @file{bison-i18n.m4} file installed by Bison under
8064 @samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
8068 cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
8073 @vindex BISON_LOCALEDIR
8074 @vindex YYENABLE_NLS
8075 In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
8076 invocation, add an invocation of @code{BISON_I18N}. This macro is
8077 defined in the file @file{bison-i18n.m4} that you copied earlier. It
8078 causes @code{configure} to find the value of the
8079 @code{BISON_LOCALEDIR} variable, and it defines the source-language
8080 symbol @code{YYENABLE_NLS} to enable translations in the
8081 Bison-generated parser.
8084 In the @code{main} function of your program, designate the directory
8085 containing Bison's runtime message catalog, through a call to
8086 @samp{bindtextdomain} with domain name @samp{bison-runtime}.
8090 bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
8093 Typically this appears after any other call @code{bindtextdomain
8094 (PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
8095 @samp{BISON_LOCALEDIR} to be defined as a string through the
8099 In the @file{Makefile.am} that controls the compilation of the @code{main}
8100 function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
8101 either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
8104 DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
8110 AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
8114 Finally, invoke the command @command{autoreconf} to generate the build
8119 @subsection Token Internationalization
8121 When the @code{%define} variable @code{parse.error} is set to @code{custom}
8122 or @code{detailed}, token aliases can be internationalized:
8126 '\n' _("end of line")
8134 The remainder of the grammar may freely use either the token symbol
8135 (@code{FUN}) or its alias (@code{"function"}), but not with the
8136 internationalization marker (@code{_("function")}).
8138 If at least one token alias is internationalized, then the generated parser
8139 will use both @code{N_} and @code{_}, that must be defined
8140 (@pxref{Programmers, , The Programmer’s View, gettext, GNU @code{gettext}
8141 utilities}). They are used only on string aliases marked for translation.
8142 In other words, even if your catalog features a translation for
8143 ``function'', then with
8153 ``function'' will appear untranslated in debug traces and error messages.
8155 Unless defined by the user, the end-of-file token, @code{YYEOF}, is provided
8156 ``end of file'' as an alias. It is also internationalized if the user
8157 internationalized tokens. To map it to another string, use:
8160 %token END 0 _("end of input")
8165 @chapter The Bison Parser Algorithm
8166 @cindex Bison parser algorithm
8167 @cindex algorithm of parser
8170 @cindex parser stack
8171 @cindex stack, parser
8173 As Bison reads tokens, it pushes them onto a stack along with their
8174 semantic values. The stack is called the @dfn{parser stack}. Pushing a
8175 token is traditionally called @dfn{shifting}.
8177 For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
8178 @samp{3} to come. The stack will have four elements, one for each token
8181 But the stack does not always have an element for each token read. When
8182 the last @var{n} tokens and groupings shifted match the components of a
8183 grammar rule, they can be combined according to that rule. This is called
8184 @dfn{reduction}. Those tokens and groupings are replaced on the stack by a
8185 single grouping whose symbol is the result (left hand side) of that rule.
8186 Running the rule's action is part of the process of reduction, because this
8187 is what computes the semantic value of the resulting grouping.
8189 For example, if the infix calculator's parser stack contains this:
8196 and the next input token is a newline character, then the last three
8197 elements can be reduced to 15 via the rule:
8200 expr: expr '*' expr;
8204 Then the stack contains just these three elements:
8211 At this point, another reduction can be made, resulting in the single value
8212 16. Then the newline token can be shifted.
8214 The parser tries, by shifts and reductions, to reduce the entire input down
8215 to a single grouping whose symbol is the grammar's start-symbol
8216 (@pxref{Language and Grammar}).
8218 This kind of parser is known in the literature as a bottom-up parser.
8221 * Lookahead:: Parser looks one token ahead when deciding what to do.
8222 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
8223 * Precedence:: Operator precedence works by resolving conflicts.
8224 * Contextual Precedence:: When an operator's precedence depends on context.
8225 * Parser States:: The parser is a finite-state-machine with stack.
8226 * Reduce/Reduce:: When two rules are applicable in the same situation.
8227 * Mysterious Conflicts:: Conflicts that look unjustified.
8228 * Tuning LR:: How to tune fundamental aspects of LR-based parsing.
8229 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
8230 * Memory Management:: What happens when memory is exhausted. How to avoid it.
8234 @section Lookahead Tokens
8235 @cindex lookahead token
8237 The Bison parser does @emph{not} always reduce immediately as soon as the
8238 last @var{n} tokens and groupings match a rule. This is because such a
8239 simple strategy is inadequate to handle most languages. Instead, when a
8240 reduction is possible, the parser sometimes ``looks ahead'' at the next
8241 token in order to decide what to do.
8243 When a token is read, it is not immediately shifted; first it becomes the
8244 @dfn{lookahead token}, which is not on the stack. Now the parser can
8245 perform one or more reductions of tokens and groupings on the stack, while
8246 the lookahead token remains off to the side. When no more reductions
8247 should take place, the lookahead token is shifted onto the stack. This
8248 does not mean that all possible reductions have been done; depending on the
8249 token kind of the lookahead token, some rules may choose to delay their
8252 Here is a simple case where lookahead is needed. These three rules define
8253 expressions which contain binary addition operators and postfix unary
8254 factorial operators (@samp{!}), and allow parentheses for grouping.
8273 Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
8274 should be done? If the following token is @samp{)}, then the first three
8275 tokens must be reduced to form an @code{expr}. This is the only valid
8276 course, because shifting the @samp{)} would produce a sequence of symbols
8277 @w{@code{term ')'}}, and no rule allows this.
8279 If the following token is @samp{!}, then it must be shifted immediately so
8280 that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
8281 parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
8282 @code{expr}. It would then be impossible to shift the @samp{!} because
8283 doing so would produce on the stack the sequence of symbols @code{expr
8284 '!'}. No rule allows that sequence.
8289 The lookahead token is stored in the variable @code{yychar}. Its semantic
8290 value and location, if any, are stored in the variables @code{yylval} and
8291 @code{yylloc}. @xref{Action Features}.
8294 @section Shift/Reduce Conflicts
8296 @cindex shift/reduce conflicts
8297 @cindex dangling @code{else}
8298 @cindex @code{else}, dangling
8300 Suppose we are parsing a language which has if-then and if-then-else
8301 statements, with a pair of rules like this:
8306 "if" expr "then" stmt
8307 | "if" expr "then" stmt "else" stmt
8313 Here @code{"if"}, @code{"then"} and @code{"else"} are terminal symbols for
8314 specific keyword tokens.
8316 When the @code{"else"} token is read and becomes the lookahead token, the
8317 contents of the stack (assuming the input is valid) are just right for
8318 reduction by the first rule. But it is also legitimate to shift the
8319 @code{"else"}, because that would lead to eventual reduction by the second
8322 This situation, where either a shift or a reduction would be valid, is
8323 called a @dfn{shift/reduce conflict}. Bison is designed to resolve
8324 these conflicts by choosing to shift, unless otherwise directed by
8325 operator precedence declarations. To see the reason for this, let's
8326 contrast it with the other alternative.
8328 Since the parser prefers to shift the @code{"else"}, the result is to attach
8329 the else-clause to the innermost if-statement, making these two inputs
8333 if x then if y then win; else lose;
8335 if x then do; if y then win; else lose; end;
8338 But if the parser chose to reduce when possible rather than shift, the
8339 result would be to attach the else-clause to the outermost if-statement,
8340 making these two inputs equivalent:
8343 if x then if y then win; else lose;
8345 if x then do; if y then win; end; else lose;
8348 The conflict exists because the grammar as written is ambiguous: either
8349 parsing of the simple nested if-statement is legitimate. The established
8350 convention is that these ambiguities are resolved by attaching the
8351 else-clause to the innermost if-statement; this is what Bison accomplishes
8352 by choosing to shift rather than reduce. (It would ideally be cleaner to
8353 write an unambiguous grammar, but that is very hard to do in this case.)
8354 This particular ambiguity was first encountered in the specifications of
8355 Algol 60 and is called the ``dangling @code{else}'' ambiguity.
8357 To assist the grammar author in understanding the nature of each conflict,
8358 Bison can be asked to generate ``counterexamples''. In the present case it
8359 actually even proves that the grammar is ambiguous by exhibiting a string
8360 with two different parses:
8362 @macro danglingElseCex
8365 Example: @yellow{"if" expr "then"} @blue{"if" expr "then" stmt} @red{•} @blue{"else" stmt}
8368 @yellow{↳ 3: "if" expr "then"} @green{stmt}
8369 @green{↳ 2:} @blue{if_stmt}
8370 @blue{↳ 4: "if" expr "then" stmt} @red{•} @blue{"else" stmt}
8371 Example: @yellow{"if" expr "then"} @blue{"if" expr "then" stmt} @red{•} @yellow{"else" stmt}
8374 @yellow{↳ 4: "if" expr "then"} @green{stmt} @yellow{"else" stmt}
8375 @green{↳ 2:} @blue{if_stmt}
8376 @blue{↳ 3: "if" expr "then" stmt} @red{•}
8379 Example: @yellow{"if" expr "then"} @blue{"if" expr "then" stmt} @red{•} @blue{"else" stmt}
8382 @yellow{@arrow{} 3: "if" expr "then"} @green{stmt}
8383 @green{@arrow{} 2:} @blue{if_stmt}
8384 @blue{@arrow{} 4: "if" expr "then" stmt} @red{•} @blue{"else" stmt}
8385 Example: @yellow{"if" expr "then"} @blue{"if" expr "then" stmt} @red{•} @yellow{"else" stmt}
8388 @yellow{@arrow{} 4: "if" expr "then"} @green{stmt} @yellow{"else" stmt}
8389 @green{@arrow{} 2:} @blue{if_stmt}
8390 @blue{@arrow{} 3: "if" expr "then" stmt} @red{•}
8399 @xref{Counterexamples}, for more details.
8403 To avoid warnings from Bison about predictable, @emph{legitimate} shift/reduce
8404 conflicts, you can use the @code{%expect @var{n}} declaration.
8405 There will be no warning as long as the number of shift/reduce conflicts
8406 is exactly @var{n}, and Bison will report an error if there is a
8408 @xref{Expect Decl}. However, we don't
8409 recommend the use of @code{%expect} (except @samp{%expect 0}!), as an equal
8410 number of conflicts does not mean that they are the @emph{same}. When
8411 possible, you should rather use precedence directives to @emph{fix} the
8412 conflicts explicitly (@pxref{Non Operators}).
8414 The definition of @code{if_stmt} above is solely to blame for the
8415 conflict, but the conflict does not actually appear without additional
8416 rules. Here is a complete Bison grammar file that actually manifests
8430 "if" expr "then" stmt
8431 | "if" expr "then" stmt "else" stmt
8441 @section Operator Precedence
8442 @cindex operator precedence
8443 @cindex precedence of operators
8445 Another situation where shift/reduce conflicts appear is in arithmetic
8446 expressions. Here shifting is not always the preferred resolution; the
8447 Bison declarations for operator precedence allow you to specify when to
8448 shift and when to reduce.
8451 * Why Precedence:: An example showing why precedence is needed.
8452 * Using Precedence:: How to specify precedence and associativity.
8453 * Precedence Only:: How to specify precedence only.
8454 * Precedence Examples:: How these features are used in the previous example.
8455 * How Precedence:: How they work.
8456 * Non Operators:: Using precedence for general conflicts.
8459 @node Why Precedence
8460 @subsection When Precedence is Needed
8462 Consider the following ambiguous grammar fragment (ambiguous because the
8463 input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
8478 Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
8479 should it reduce them via the rule for the subtraction operator? It
8480 depends on the next token. Of course, if the next token is @samp{)}, we
8481 must reduce; shifting is invalid because no single rule can reduce the
8482 token sequence @w{@samp{- 2 )}} or anything starting with that. But if
8483 the next token is @samp{*} or @samp{<}, we have a choice: either
8484 shifting or reduction would allow the parse to complete, but with
8487 To decide which one Bison should do, we must consider the results. If
8488 the next operator token @var{op} is shifted, then it must be reduced
8489 first in order to permit another opportunity to reduce the difference.
8490 The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
8491 hand, if the subtraction is reduced before shifting @var{op}, the result
8492 is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
8493 reduce should depend on the relative precedence of the operators
8494 @samp{-} and @var{op}: @samp{*} should be shifted first, but not
8497 @cindex associativity
8498 What about input such as @w{@samp{1 - 2 - 5}}; should this be
8499 @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
8500 operators we prefer the former, which is called @dfn{left association}.
8501 The latter alternative, @dfn{right association}, is desirable for
8502 assignment operators. The choice of left or right association is a
8503 matter of whether the parser chooses to shift or reduce when the stack
8504 contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
8505 makes right-associativity.
8507 @node Using Precedence
8508 @subsection Specifying Operator Precedence
8514 Bison allows you to specify these choices with the operator precedence
8515 declarations @code{%left} and @code{%right}. Each such declaration
8516 contains a list of tokens, which are operators whose precedence and
8517 associativity is being declared. The @code{%left} declaration makes all
8518 those operators left-associative and the @code{%right} declaration makes
8519 them right-associative. A third alternative is @code{%nonassoc}, which
8520 declares that it is a syntax error to find the same operator twice ``in a
8522 The last alternative, @code{%precedence}, allows to define only
8523 precedence and no associativity at all. As a result, any
8524 associativity-related conflict that remains will be reported as an
8525 compile-time error. The directive @code{%nonassoc} creates run-time
8526 error: using the operator in a associative way is a syntax error. The
8527 directive @code{%precedence} creates compile-time errors: an operator
8528 @emph{can} be involved in an associativity-related conflict, contrary to
8529 what expected the grammar author.
8531 The relative precedence of different operators is controlled by the
8532 order in which they are declared. The first precedence/associativity
8533 declaration in the file declares the operators whose
8534 precedence is lowest, the next such declaration declares the operators
8535 whose precedence is a little higher, and so on.
8537 @node Precedence Only
8538 @subsection Specifying Precedence Only
8541 Since POSIX Yacc defines only @code{%left}, @code{%right}, and
8542 @code{%nonassoc}, which all defines precedence and associativity, little
8543 attention is paid to the fact that precedence cannot be defined without
8544 defining associativity. Yet, sometimes, when trying to solve a
8545 conflict, precedence suffices. In such a case, using @code{%left},
8546 @code{%right}, or @code{%nonassoc} might hide future (associativity
8547 related) conflicts that would remain hidden.
8549 The dangling @code{else} ambiguity (@pxref{Shift/Reduce}) can be solved
8550 explicitly. This shift/reduce conflicts occurs in the following situation,
8551 where the period denotes the current parsing state:
8554 if @var{e1} then if @var{e2} then @var{s1} • else @var{s2}
8557 The conflict involves the reduction of the rule @samp{IF expr THEN
8558 stmt}, which precedence is by default that of its last token
8559 (@code{THEN}), and the shifting of the token @code{ELSE}. The usual
8560 disambiguation (attach the @code{else} to the closest @code{if}),
8561 shifting must be preferred, i.e., the precedence of @code{ELSE} must be
8562 higher than that of @code{THEN}. But neither is expected to be involved
8563 in an associativity related conflict, which can be specified as follows.
8570 The unary-minus is another typical example where associativity is usually
8571 over-specified, see @ref{Infix Calc}. The @code{%left} directive is
8572 traditionally used to declare the precedence of @code{NEG}, which is more
8573 than needed since it also defines its associativity. While this is harmless
8574 in the traditional example, who knows how @code{NEG} might be used in future
8575 evolutions of the grammar@dots{}
8577 @node Precedence Examples
8578 @subsection Precedence Examples
8580 In our example, we would want the following declarations:
8588 In a more complete example, which supports other operators as well, we
8589 would declare them in groups of equal precedence. For example, @code{'+'} is
8590 declared with @code{'-'}:
8593 %left '<' '>' '=' "!=" "<=" ">="
8598 @node How Precedence
8599 @subsection How Precedence Works
8601 The first effect of the precedence declarations is to assign precedence
8602 levels to the terminal symbols declared. The second effect is to assign
8603 precedence levels to certain rules: each rule gets its precedence from
8604 the last terminal symbol mentioned in the components. (You can also
8605 specify explicitly the precedence of a rule. @xref{Contextual
8608 Finally, the resolution of conflicts works by comparing the precedence
8609 of the rule being considered with that of the lookahead token. If the
8610 token's precedence is higher, the choice is to shift. If the rule's
8611 precedence is higher, the choice is to reduce. If they have equal
8612 precedence, the choice is made based on the associativity of that
8613 precedence level. The verbose output file made by @option{-v}
8614 (@pxref{Invocation}) says how each conflict was
8617 Not all rules and not all tokens have precedence. If either the rule or
8618 the lookahead token has no precedence, then the default is to shift.
8621 @subsection Using Precedence For Non Operators
8623 Using properly precedence and associativity directives can help fixing
8624 shift/reduce conflicts that do not involve arithmetic-like operators. For
8625 instance, the ``dangling @code{else}'' problem (@pxref{Shift/Reduce}) can be
8626 solved elegantly in two different ways.
8628 In the present case, the conflict is between the token @code{"else"} willing
8629 to be shifted, and the rule @samp{if_stmt: "if" expr "then" stmt}, asking
8630 for reduction. By default, the precedence of a rule is that of its last
8631 token, here @code{"then"}, so the conflict will be solved appropriately
8632 by giving @code{"else"} a precedence higher than that of @code{"then"}, for
8633 instance as follows:
8642 Alternatively, you may give both tokens the same precedence, in which case
8643 associativity is used to solve the conflict. To preserve the shift action,
8644 use right associativity:
8647 %right "then" "else"
8650 Neither solution is perfect however. Since Bison does not provide, so far,
8651 ``scoped'' precedence, both force you to declare the precedence
8652 of these keywords with respect to the other operators your grammar.
8653 Therefore, instead of being warned about new conflicts you would be unaware
8654 of (e.g., a shift/reduce conflict due to @samp{if test then 1 else 2 + 3}
8655 being ambiguous: @samp{if test then 1 else (2 + 3)} or @samp{(if test then 1
8656 else 2) + 3}?), the conflict will be already ``fixed''.
8658 @node Contextual Precedence
8659 @section Context-Dependent Precedence
8660 @cindex context-dependent precedence
8661 @cindex unary operator precedence
8662 @cindex precedence, context-dependent
8663 @cindex precedence, unary operator
8666 Often the precedence of an operator depends on the context. This sounds
8667 outlandish at first, but it is really very common. For example, a minus
8668 sign typically has a very high precedence as a unary operator, and a
8669 somewhat lower precedence (lower than multiplication) as a binary operator.
8671 The Bison precedence declarations
8672 can only be used once for a given token; so a token has
8673 only one precedence declared in this way. For context-dependent
8674 precedence, you need to use an additional mechanism: the @code{%prec}
8677 The @code{%prec} modifier declares the precedence of a particular rule by
8678 specifying a terminal symbol whose precedence should be used for that rule.
8679 It's not necessary for that symbol to appear otherwise in the rule. The
8680 modifier's syntax is:
8683 %prec @var{terminal-symbol}
8687 and it is written after the components of the rule. Its effect is to
8688 assign the rule the precedence of @var{terminal-symbol}, overriding
8689 the precedence that would be deduced for it in the ordinary way. The
8690 altered rule precedence then affects how conflicts involving that rule
8691 are resolved (@pxref{Precedence}).
8693 Here is how @code{%prec} solves the problem of unary minus. First, declare
8694 a precedence for a fictitious terminal symbol named @code{UMINUS}. There
8695 are no tokens of this type, but the symbol serves to stand for its
8705 Now the precedence of @code{UMINUS} can be used in specific rules:
8713 | '-' exp %prec UMINUS
8718 If you forget to append @code{%prec UMINUS} to the rule for unary
8719 minus, Bison silently assumes that minus has its usual precedence.
8720 This kind of problem can be tricky to debug, since one typically
8721 discovers the mistake only by testing the code.
8723 The @code{%no-default-prec;} declaration makes it easier to discover
8724 this kind of problem systematically. It causes rules that lack a
8725 @code{%prec} modifier to have no precedence, even if the last terminal
8726 symbol mentioned in their components has a declared precedence.
8728 If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
8729 for all rules that participate in precedence conflict resolution.
8730 Then you will see any shift/reduce conflict until you tell Bison how
8731 to resolve it, either by changing your grammar or by adding an
8732 explicit precedence. This will probably add declarations to the
8733 grammar, but it helps to protect against incorrect rule precedences.
8735 The effect of @code{%no-default-prec;} can be reversed by giving
8736 @code{%default-prec;}, which is the default.
8740 @section Parser States
8741 @cindex finite-state machine
8742 @cindex parser state
8743 @cindex state (of parser)
8745 The function @code{yyparse} is implemented using a finite-state machine.
8746 The values pushed on the parser stack are not simply token kind codes; they
8747 represent the entire sequence of terminal and nonterminal symbols at or
8748 near the top of the stack. The current state collects all the information
8749 about previous input which is relevant to deciding what to do next.
8751 Each time a lookahead token is read, the current parser state together with
8752 the kind of lookahead token are looked up in a table. This table entry can
8753 say, ``Shift the lookahead token.'' In this case, it also specifies the new
8754 parser state, which is pushed onto the top of the parser stack. Or it can
8755 say, ``Reduce using rule number @var{n}.'' This means that a certain number
8756 of tokens or groupings are taken off the top of the stack, and replaced by
8757 one grouping. In other words, that number of states are popped from the
8758 stack, and one new state is pushed.
8760 There is one other alternative: the table can say that the lookahead token
8761 is erroneous in the current state. This causes error processing to begin
8762 (@pxref{Error Recovery}).
8765 @section Reduce/Reduce Conflicts
8766 @cindex reduce/reduce conflict
8767 @cindex conflicts, reduce/reduce
8769 A reduce/reduce conflict occurs if there are two or more rules that apply
8770 to the same sequence of input. This usually indicates a serious error
8773 For example, here is an erroneous attempt to define a sequence
8774 of zero or more @code{word} groupings.
8779 %empty @{ printf ("empty sequence\n"); @}
8781 | sequence word @{ printf ("added word %s\n", $2); @}
8787 %empty @{ printf ("empty maybeword\n"); @}
8788 | word @{ printf ("single word %s\n", $1); @}
8794 The error is an ambiguity: as counterexample generation would demonstrate
8795 (@pxref{Counterexamples}), there is more than one way to parse a single
8796 @code{word} into a @code{sequence}. It could be reduced to a
8797 @code{maybeword} and then into a @code{sequence} via the second rule.
8798 Alternatively, nothing-at-all could be reduced into a @code{sequence}
8799 via the first rule, and this could be combined with the @code{word}
8800 using the third rule for @code{sequence}.
8802 There is also more than one way to reduce nothing-at-all into a
8803 @code{sequence}. This can be done directly via the first rule,
8804 or indirectly via @code{maybeword} and then the second rule.
8806 You might think that this is a distinction without a difference, because it
8807 does not change whether any particular input is valid or not. But it does
8808 affect which actions are run. One parsing order runs the second rule's
8809 action; the other runs the first rule's action and the third rule's action.
8810 In this example, the output of the program changes.
8812 Bison resolves a reduce/reduce conflict by choosing to use the rule that
8813 appears first in the grammar, but it is very risky to rely on this. Every
8814 reduce/reduce conflict must be studied and usually eliminated. Here is the
8815 proper way to define @code{sequence}:
8820 %empty @{ printf ("empty sequence\n"); @}
8821 | sequence word @{ printf ("added word %s\n", $2); @}
8826 Here is another common error that yields a reduce/reduce conflict:
8833 | sequence redirects
8847 | redirects redirect
8853 The intention here is to define a sequence which can contain either
8854 @code{word} or @code{redirect} groupings. The individual definitions of
8855 @code{sequence}, @code{words} and @code{redirects} are error-free, but the
8856 three together make a subtle ambiguity: even an empty input can be parsed
8857 in infinitely many ways!
8859 Consider: nothing-at-all could be a @code{words}. Or it could be two
8860 @code{words} in a row, or three, or any number. It could equally well be a
8861 @code{redirects}, or two, or any number. Or it could be a @code{words}
8862 followed by three @code{redirects} and another @code{words}. And so on.
8864 Here are two ways to correct these rules. First, to make it a single level
8875 Second, to prevent either a @code{words} or a @code{redirects}
8883 | sequence redirects
8897 | redirects redirect
8902 Yet this proposal introduces another kind of ambiguity! The input
8903 @samp{word word} can be parsed as a single @code{words} composed of two
8904 @samp{word}s, or as two one-@code{word} @code{words} (and likewise for
8905 @code{redirect}/@code{redirects}). However this ambiguity is now a
8906 shift/reduce conflict, and therefore it can now be addressed with precedence
8909 To simplify the matter, we will proceed with @code{word} and @code{redirect}
8910 being tokens: @code{"word"} and @code{"redirect"}.
8912 To prefer the longest @code{words}, the conflict between the token
8913 @code{"word"} and the rule @samp{sequence: sequence words} must be resolved
8914 as a shift. To this end, we use the same techniques as exposed above, see
8915 @ref{Non Operators}. One solution
8916 relies on precedences: use @code{%prec} to give a lower precedence to the
8921 %precedence "sequence"
8926 | sequence word %prec "sequence"
8927 | sequence redirect %prec "sequence"
8939 Another solution relies on associativity: provide both the token and the
8940 rule with the same precedence, but make them right-associative:
8943 %right "word" "redirect"
8948 | sequence word %prec "word"
8949 | sequence redirect %prec "redirect"
8954 @node Mysterious Conflicts
8955 @section Mysterious Conflicts
8956 @cindex Mysterious Conflicts
8958 Sometimes reduce/reduce conflicts can occur that don't look warranted.
8964 def: param_spec return_spec ',';
8967 | name_list ':' type
8984 | name ',' name_list
8989 It would seem that this grammar can be parsed with only a single token of
8990 lookahead: when a @code{param_spec} is being read, an @code{"id"} is a
8991 @code{name} if a comma or colon follows, or a @code{type} if another
8992 @code{"id"} follows. In other words, this grammar is LR(1). Yet Bison
8993 finds one reduce/reduce conflict, for which counterexample generation
8994 (@pxref{Counterexamples}) would find a @emph{nonunifying} example.
8998 This is because Bison does not handle all LR(1) grammars @emph{by default},
8999 for historical reasons.
9000 In this grammar, two contexts, that after an @code{"id"} at the beginning
9001 of a @code{param_spec} and likewise at the beginning of a
9002 @code{return_spec}, are similar enough that Bison assumes they are the
9004 They appear similar because the same set of rules would be
9005 active---the rule for reducing to a @code{name} and that for reducing to
9006 a @code{type}. Bison is unable to determine at that stage of processing
9007 that the rules would require different lookahead tokens in the two
9008 contexts, so it makes a single parser state for them both. Combining
9009 the two contexts causes a conflict later. In parser terminology, this
9010 occurrence means that the grammar is not LALR(1).
9013 @cindex canonical LR
9014 For many practical grammars (specifically those that fall into the non-LR(1)
9015 class), the limitations of LALR(1) result in difficulties beyond just
9016 mysterious reduce/reduce conflicts. The best way to fix all these problems
9017 is to select a different parser table construction algorithm. Either
9018 IELR(1) or canonical LR(1) would suffice, but the former is more efficient
9019 and easier to debug during development. @xref{LR Table Construction}, for
9022 If you instead wish to work around LALR(1)'s limitations, you
9023 can often fix a mysterious conflict by identifying the two parser states
9024 that are being confused, and adding something to make them look
9025 distinct. In the above example, adding one rule to
9026 @code{return_spec} as follows makes the problem go away:
9034 | "id" "bogus" /* This rule is never used. */
9039 This corrects the problem because it introduces the possibility of an
9040 additional active rule in the context after the @code{"id"} at the beginning of
9041 @code{return_spec}. This rule is not active in the corresponding context
9042 in a @code{param_spec}, so the two contexts receive distinct parser states.
9043 As long as the token @code{"bogus"} is never generated by @code{yylex},
9044 the added rule cannot alter the way actual input is parsed.
9046 In this particular example, there is another way to solve the problem:
9047 rewrite the rule for @code{return_spec} to use @code{"id"} directly
9048 instead of via @code{name}. This also causes the two confusing
9049 contexts to have different sets of active rules, because the one for
9050 @code{return_spec} activates the altered rule for @code{return_spec}
9051 rather than the one for @code{name}.
9057 | name_list ':' type
9069 For a more detailed exposition of LALR(1) parsers and parser generators, see
9070 @tcite{DeRemer 1982}.
9075 The default behavior of Bison's LR-based parsers is chosen mostly for
9076 historical reasons, but that behavior is often not robust. For example, in
9077 the previous section, we discussed the mysterious conflicts that can be
9078 produced by LALR(1), Bison's default parser table construction algorithm.
9079 Another example is Bison's @code{%define parse.error verbose} directive,
9080 which instructs the generated parser to produce verbose syntax error
9081 messages, which can sometimes contain incorrect information.
9083 In this section, we explore several modern features of Bison that allow you
9084 to tune fundamental aspects of the generated LR-based parsers. Some of
9085 these features easily eliminate shortcomings like those mentioned above.
9086 Others can be helpful purely for understanding your parser.
9089 * LR Table Construction:: Choose a different construction algorithm.
9090 * Default Reductions:: Disable default reductions.
9091 * LAC:: Correct lookahead sets in the parser states.
9092 * Unreachable States:: Keep unreachable parser states for debugging.
9095 @node LR Table Construction
9096 @subsection LR Table Construction
9097 @cindex Mysterious Conflict
9100 @cindex canonical LR
9101 @findex %define lr.type
9103 For historical reasons, Bison constructs LALR(1) parser tables by default.
9104 However, LALR does not possess the full language-recognition power of LR.
9105 As a result, the behavior of parsers employing LALR parser tables is often
9106 mysterious. We presented a simple example of this effect in @ref{Mysterious
9109 As we also demonstrated in that example, the traditional approach to
9110 eliminating such mysterious behavior is to restructure the grammar.
9111 Unfortunately, doing so correctly is often difficult. Moreover, merely
9112 discovering that LALR causes mysterious behavior in your parser can be
9115 Fortunately, Bison provides an easy way to eliminate the possibility of such
9116 mysterious behavior altogether. You simply need to activate a more powerful
9117 parser table construction algorithm by using the @code{%define lr.type}
9120 @deffn {Directive} {%define lr.type} @var{type}
9121 Specify the type of parser tables within the LR(1) family. The accepted
9122 values for @var{type} are:
9125 @item @code{lalr} (default)
9127 @item @code{canonical-lr}
9131 For example, to activate IELR, you might add the following directive to you
9135 %define lr.type ielr
9138 @noindent For the example in @ref{Mysterious Conflicts}, the mysterious
9139 conflict is then eliminated, so there is no need to invest time in
9140 comprehending the conflict or restructuring the grammar to fix it. If,
9141 during future development, the grammar evolves such that all mysterious
9142 behavior would have disappeared using just LALR, you need not fear that
9143 continuing to use IELR will result in unnecessarily large parser tables.
9144 That is, IELR generates LALR tables when LALR (using a deterministic parsing
9145 algorithm) is sufficient to support the full language-recognition power of
9146 LR. Thus, by enabling IELR at the start of grammar development, you can
9147 safely and completely eliminate the need to consider LALR's shortcomings.
9149 While IELR is almost always preferable, there are circumstances where LALR
9150 or the canonical LR parser tables described by Knuth @pcite{Knuth 1965} can
9151 be useful. Here we summarize the relative advantages of each parser table
9152 construction algorithm within Bison:
9157 There are at least two scenarios where LALR can be worthwhile:
9160 @item GLR without static conflict resolution.
9162 @cindex GLR with LALR
9163 When employing GLR parsers (@pxref{GLR Parsers}), if you do not resolve any
9164 conflicts statically (for example, with @code{%left} or @code{%precedence}),
9166 the parser explores all potential parses of any given input. In this case,
9167 the choice of parser table construction algorithm is guaranteed not to alter
9168 the language accepted by the parser. LALR parser tables are the smallest
9169 parser tables Bison can currently construct, so they may then be preferable.
9170 Nevertheless, once you begin to resolve conflicts statically, GLR behaves
9171 more like a deterministic parser in the syntactic contexts where those
9172 conflicts appear, and so either IELR or canonical LR can then be helpful to
9173 avoid LALR's mysterious behavior.
9175 @item Malformed grammars.
9177 Occasionally during development, an especially malformed grammar with a
9178 major recurring flaw may severely impede the IELR or canonical LR parser
9179 table construction algorithm. LALR can be a quick way to construct parser
9180 tables in order to investigate such problems while ignoring the more subtle
9181 differences from IELR and canonical LR.
9186 IELR (Inadequacy Elimination LR) is a minimal LR algorithm. That is, given
9187 any grammar (LR or non-LR), parsers using IELR or canonical LR parser tables
9188 always accept exactly the same set of sentences. However, like LALR, IELR
9189 merges parser states during parser table construction so that the number of
9190 parser states is often an order of magnitude less than for canonical LR.
9191 More importantly, because canonical LR's extra parser states may contain
9192 duplicate conflicts in the case of non-LR grammars, the number of conflicts
9193 for IELR is often an order of magnitude less as well. This effect can
9194 significantly reduce the complexity of developing a grammar.
9198 @cindex delayed syntax error detection
9201 While inefficient, canonical LR parser tables can be an interesting means to
9202 explore a grammar because they possess a property that IELR and LALR tables
9203 do not. That is, if @code{%nonassoc} is not used and default reductions are
9204 left disabled (@pxref{Default Reductions}), then, for every left context of
9205 every canonical LR state, the set of tokens accepted by that state is
9206 guaranteed to be the exact set of tokens that is syntactically acceptable in
9207 that left context. It might then seem that an advantage of canonical LR
9208 parsers in production is that, under the above constraints, they are
9209 guaranteed to detect a syntax error as soon as possible without performing
9210 any unnecessary reductions. However, IELR parsers that use LAC are also
9211 able to achieve this behavior without sacrificing @code{%nonassoc} or
9212 default reductions. For details and a few caveats of LAC, @pxref{LAC}.
9215 For a more detailed exposition of the mysterious behavior in LALR parsers
9216 and the benefits of IELR, see @tcite{Denny 2008}, and @tcite{Denny 2010
9219 @node Default Reductions
9220 @subsection Default Reductions
9221 @cindex default reductions
9222 @findex %define lr.default-reduction
9225 After parser table construction, Bison identifies the reduction with the
9226 largest lookahead set in each parser state. To reduce the size of the
9227 parser state, traditional Bison behavior is to remove that lookahead set and
9228 to assign that reduction to be the default parser action. Such a reduction
9229 is known as a @dfn{default reduction}.
9231 Default reductions affect more than the size of the parser tables. They
9232 also affect the behavior of the parser:
9235 @item Delayed @code{yylex} invocations.
9237 @cindex delayed yylex invocations
9238 @cindex consistent states
9239 @cindex defaulted states
9240 A @dfn{consistent state} is a state that has only one possible parser
9241 action. If that action is a reduction and is encoded as a default
9242 reduction, then that consistent state is called a @dfn{defaulted state}.
9243 Upon reaching a defaulted state, a Bison-generated parser does not bother to
9244 invoke @code{yylex} to fetch the next token before performing the reduction.
9245 In other words, whether default reductions are enabled in consistent states
9246 determines how soon a Bison-generated parser invokes @code{yylex} for a
9247 token: immediately when it @emph{reaches} that token in the input or when it
9248 eventually @emph{needs} that token as a lookahead to determine the next
9249 parser action. Traditionally, default reductions are enabled, and so the
9250 parser exhibits the latter behavior.
9252 The presence of defaulted states is an important consideration when
9253 designing @code{yylex} and the grammar file. That is, if the behavior of
9254 @code{yylex} can influence or be influenced by the semantic actions
9255 associated with the reductions in defaulted states, then the delay of the
9256 next @code{yylex} invocation until after those reductions is significant.
9257 For example, the semantic actions might pop a scope stack that @code{yylex}
9258 uses to determine what token to return. Thus, the delay might be necessary
9259 to ensure that @code{yylex} does not look up the next token in a scope that
9260 should already be considered closed.
9262 @item Delayed syntax error detection.
9264 @cindex delayed syntax error detection
9265 When the parser fetches a new token by invoking @code{yylex}, it checks
9266 whether there is an action for that token in the current parser state. The
9267 parser detects a syntax error if and only if either (1) there is no action
9268 for that token or (2) the action for that token is the error action (due to
9269 the use of @code{%nonassoc}). However, if there is a default reduction in
9270 that state (which might or might not be a defaulted state), then it is
9271 impossible for condition 1 to exist. That is, all tokens have an action.
9272 Thus, the parser sometimes fails to detect the syntax error until it reaches
9276 @c If there's an infinite loop, default reductions can prevent an incorrect
9277 @c sentence from being rejected.
9278 While default reductions never cause the parser to accept syntactically
9279 incorrect sentences, the delay of syntax error detection can have unexpected
9280 effects on the behavior of the parser. However, the delay can be caused
9281 anyway by parser state merging and the use of @code{%nonassoc}, and it can
9282 be fixed by another Bison feature, LAC. We discuss the effects of delayed
9283 syntax error detection and LAC more in the next section (@pxref{LAC}).
9286 For canonical LR, the only default reduction that Bison enables by default
9287 is the accept action, which appears only in the accepting state, which has
9288 no other action and is thus a defaulted state. However, the default accept
9289 action does not delay any @code{yylex} invocation or syntax error detection
9290 because the accept action ends the parse.
9292 For LALR and IELR, Bison enables default reductions in nearly all states by
9293 default. There are only two exceptions. First, states that have a shift
9294 action on the @code{error} token do not have default reductions because
9295 delayed syntax error detection could then prevent the @code{error} token
9296 from ever being shifted in that state. However, parser state merging can
9297 cause the same effect anyway, and LAC fixes it in both cases, so future
9298 versions of Bison might drop this exception when LAC is activated. Second,
9299 GLR parsers do not record the default reduction as the action on a lookahead
9300 token for which there is a conflict. The correct action in this case is to
9301 split the parse instead.
9303 To adjust which states have default reductions enabled, use the
9304 @code{%define lr.default-reduction} directive.
9306 @deffn {Directive} {%define lr.default-reduction} @var{where}
9307 Specify the kind of states that are permitted to contain default reductions.
9308 The accepted values of @var{where} are:
9310 @item @code{most} (default for LALR and IELR)
9311 @item @code{consistent}
9312 @item @code{accepting} (default for canonical LR)
9318 @findex %define parse.lac
9320 @cindex lookahead correction
9322 Canonical LR, IELR, and LALR can suffer from a couple of problems upon
9323 encountering a syntax error. First, the parser might perform additional
9324 parser stack reductions before discovering the syntax error. Such
9325 reductions can perform user semantic actions that are unexpected because
9326 they are based on an invalid token, and they cause error recovery to begin
9327 in a different syntactic context than the one in which the invalid token was
9328 encountered. Second, when verbose error messages are enabled (@pxref{Error
9329 Reporting}), the expected token list in the syntax error message can both
9330 contain invalid tokens and omit valid tokens.
9332 The culprits for the above problems are @code{%nonassoc}, default reductions
9333 in inconsistent states (@pxref{Default Reductions}), and parser state
9334 merging. Because IELR and LALR merge parser states, they suffer the most.
9335 Canonical LR can suffer only if @code{%nonassoc} is used or if default
9336 reductions are enabled for inconsistent states.
9338 LAC (Lookahead Correction) is a new mechanism within the parsing algorithm
9339 that solves these problems for canonical LR, IELR, and LALR without
9340 sacrificing @code{%nonassoc}, default reductions, or state merging. You can
9341 enable LAC with the @code{%define parse.lac} directive.
9343 @deffn {Directive} {%define parse.lac} @var{value}
9344 Enable LAC to improve syntax error handling.
9346 @item @code{none} (default)
9349 This feature is currently only available for deterministic parsers in C and C++.
9352 Conceptually, the LAC mechanism is straight-forward. Whenever the parser
9353 fetches a new token from the scanner so that it can determine the next
9354 parser action, it immediately suspends normal parsing and performs an
9355 exploratory parse using a temporary copy of the normal parser state stack.
9356 During this exploratory parse, the parser does not perform user semantic
9357 actions. If the exploratory parse reaches a shift action, normal parsing
9358 then resumes on the normal parser stacks. If the exploratory parse reaches
9359 an error instead, the parser reports a syntax error. If verbose syntax
9360 error messages are enabled, the parser must then discover the list of
9361 expected tokens, so it performs a separate exploratory parse for each token
9364 There is one subtlety about the use of LAC. That is, when in a consistent
9365 parser state with a default reduction, the parser will not attempt to fetch
9366 a token from the scanner because no lookahead is needed to determine the
9367 next parser action. Thus, whether default reductions are enabled in
9368 consistent states (@pxref{Default Reductions}) affects how soon the parser
9369 detects a syntax error: immediately when it @emph{reaches} an erroneous
9370 token or when it eventually @emph{needs} that token as a lookahead to
9371 determine the next parser action. The latter behavior is probably more
9372 intuitive, so Bison currently provides no way to achieve the former behavior
9373 while default reductions are enabled in consistent states.
9375 Thus, when LAC is in use, for some fixed decision of whether to enable
9376 default reductions in consistent states, canonical LR and IELR behave almost
9377 exactly the same for both syntactically acceptable and syntactically
9378 unacceptable input. While LALR still does not support the full
9379 language-recognition power of canonical LR and IELR, LAC at least enables
9380 LALR's syntax error handling to correctly reflect LALR's
9381 language-recognition power.
9383 There are a few caveats to consider when using LAC:
9386 @item Infinite parsing loops.
9388 IELR plus LAC does have one shortcoming relative to canonical LR. Some
9389 parsers generated by Bison can loop infinitely. LAC does not fix infinite
9390 parsing loops that occur between encountering a syntax error and detecting
9391 it, but enabling canonical LR or disabling default reductions sometimes
9394 @item Verbose error message limitations.
9396 Because of internationalization considerations, Bison-generated parsers
9397 limit the size of the expected token list they are willing to report in a
9398 verbose syntax error message. If the number of expected tokens exceeds that
9399 limit, the list is simply dropped from the message. Enabling LAC can
9400 increase the size of the list and thus cause the parser to drop it. Of
9401 course, dropping the list is better than reporting an incorrect list.
9405 Because LAC requires many parse actions to be performed twice, it can have a
9406 performance penalty. However, not all parse actions must be performed
9407 twice. Specifically, during a series of default reductions in consistent
9408 states and shift actions, the parser never has to initiate an exploratory
9409 parse. Moreover, the most time-consuming tasks in a parse are often the
9410 file I/O, the lexical analysis performed by the scanner, and the user's
9411 semantic actions, but none of these are performed during the exploratory
9412 parse. Finally, the base of the temporary stack used during an exploratory
9413 parse is a pointer into the normal parser state stack so that the stack is
9414 never physically copied. In our experience, the performance penalty of LAC
9415 has proved insignificant for practical grammars.
9418 While the LAC algorithm shares techniques that have been recognized in the
9419 parser community for years, for the publication that introduces LAC, see
9420 @tcite{Denny 2010 May}.
9422 @node Unreachable States
9423 @subsection Unreachable States
9424 @findex %define lr.keep-unreachable-state
9425 @cindex unreachable states
9427 If there exists no sequence of transitions from the parser's start state to
9428 some state @var{s}, then Bison considers @var{s} to be an @dfn{unreachable
9429 state}. A state can become unreachable during conflict resolution if Bison
9430 disables a shift action leading to it from a predecessor state.
9432 By default, Bison removes unreachable states from the parser after conflict
9433 resolution because they are useless in the generated parser. However,
9434 keeping unreachable states is sometimes useful when trying to understand the
9435 relationship between the parser and the grammar.
9437 @deffn {Directive} {%define lr.keep-unreachable-state} @var{value}
9438 Request that Bison allow unreachable states to remain in the parser tables.
9439 @var{value} must be a Boolean. The default is @code{false}.
9442 There are a few caveats to consider:
9445 @item Missing or extraneous warnings.
9447 Unreachable states may contain conflicts and may use rules not used in any
9448 other state. Thus, keeping unreachable states may induce warnings that are
9449 irrelevant to your parser's behavior, and it may eliminate warnings that are
9450 relevant. Of course, the change in warnings may actually be relevant to a
9451 parser table analysis that wants to keep unreachable states, so this
9452 behavior will likely remain in future Bison releases.
9454 @item Other useless states.
9456 While Bison is able to remove unreachable states, it is not guaranteed to
9457 remove other kinds of useless states. Specifically, when Bison disables
9458 reduce actions during conflict resolution, some goto actions may become
9459 useless, and thus some additional states may become useless. If Bison were
9460 to compute which goto actions were useless and then disable those actions,
9461 it could identify such states as unreachable and then remove those states.
9462 However, Bison does not compute which goto actions are useless.
9465 @node Generalized LR Parsing
9466 @section Generalized LR (GLR) Parsing
9468 @cindex generalized LR (GLR) parsing
9469 @cindex ambiguous grammars
9470 @cindex nondeterministic parsing
9472 Bison produces @emph{deterministic} parsers that choose uniquely
9473 when to reduce and which reduction to apply
9474 based on a summary of the preceding input and on one extra token of lookahead.
9475 As a result, normal Bison handles a proper subset of the family of
9476 context-free languages.
9477 Ambiguous grammars, since they have strings with more than one possible
9478 sequence of reductions cannot have deterministic parsers in this sense.
9479 The same is true of languages that require more than one symbol of
9480 lookahead, since the parser lacks the information necessary to make a
9481 decision at the point it must be made in a shift/reduce parser.
9482 Finally, as previously mentioned (@pxref{Mysterious Conflicts}),
9483 there are languages where Bison's default choice of how to
9484 summarize the input seen so far loses necessary information.
9486 When you use the @samp{%glr-parser} declaration in your grammar file,
9487 Bison generates a parser that uses a different algorithm, called
9488 Generalized LR (or GLR). A Bison GLR
9489 parser uses the same basic
9490 algorithm for parsing as an ordinary Bison parser, but behaves
9491 differently in cases where there is a shift/reduce conflict that has not
9492 been resolved by precedence rules (@pxref{Precedence}) or a
9493 reduce/reduce conflict. When a GLR parser encounters such a
9495 effectively @emph{splits} into a several parsers, one for each possible
9496 shift or reduction. These parsers then proceed as usual, consuming
9497 tokens in lock-step. Some of the stacks may encounter other conflicts
9498 and split further, with the result that instead of a sequence of states,
9499 a Bison GLR parsing stack is what is in effect a tree of states.
9501 In effect, each stack represents a guess as to what the proper parse
9502 is. Additional input may indicate that a guess was wrong, in which case
9503 the appropriate stack silently disappears. Otherwise, the semantics
9504 actions generated in each stack are saved, rather than being executed
9505 immediately. When a stack disappears, its saved semantic actions never
9506 get executed. When a reduction causes two stacks to become equivalent,
9507 their sets of semantic actions are both saved with the state that
9508 results from the reduction. We say that two stacks are equivalent
9509 when they both represent the same sequence of states,
9510 and each pair of corresponding states represents a
9511 grammar symbol that produces the same segment of the input token
9514 Whenever the parser makes a transition from having multiple
9515 states to having one, it reverts to the normal deterministic parsing
9516 algorithm, after resolving and executing the saved-up actions.
9517 At this transition, some of the states on the stack will have semantic
9518 values that are sets (actually multisets) of possible actions. The
9519 parser tries to pick one of the actions by first finding one whose rule
9520 has the highest dynamic precedence, as set by the @samp{%dprec}
9521 declaration. Otherwise, if the alternative actions are not ordered by
9522 precedence, but there the same merging function is declared for both
9523 rules by the @samp{%merge} declaration,
9524 Bison resolves and evaluates both and then calls the merge function on
9525 the result. Otherwise, it reports an ambiguity.
9527 It is possible to use a data structure for the GLR parsing tree that
9528 permits the processing of any LR(1) grammar in linear time (in the
9529 size of the input), any unambiguous (not necessarily
9531 quadratic worst-case time, and any general (possibly ambiguous)
9532 context-free grammar in cubic worst-case time. However, Bison currently
9533 uses a simpler data structure that requires time proportional to the
9534 length of the input times the maximum number of stacks required for any
9535 prefix of the input. Thus, really ambiguous or nondeterministic
9536 grammars can require exponential time and space to process. Such badly
9537 behaving examples, however, are not generally of practical interest.
9538 Usually, nondeterminism in a grammar is local---the parser is ``in
9539 doubt'' only for a few tokens at a time. Therefore, the current data
9540 structure should generally be adequate. On LR(1) portions of a
9541 grammar, in particular, it is only slightly slower than with the
9542 deterministic LR(1) Bison parser.
9544 For a more detailed exposition of GLR parsers, see @tcite{Scott 2000}.
9546 @node Memory Management
9547 @section Memory Management, and How to Avoid Memory Exhaustion
9548 @cindex memory exhaustion
9549 @cindex memory management
9550 @cindex stack overflow
9551 @cindex parser stack overflow
9552 @cindex overflow of parser stack
9554 The Bison parser stack can run out of memory if too many tokens are shifted and
9555 not reduced. When this happens, the parser function @code{yyparse}
9556 calls @code{yyerror} and then returns 2.
9558 Because Bison parsers have growing stacks, hitting the upper limit
9559 usually results from using a right recursion instead of a left
9560 recursion, see @ref{Recursion}.
9563 By defining the macro @code{YYMAXDEPTH}, you can control how deep the
9564 parser stack can become before memory is exhausted. Define the
9565 macro with a value that is an integer. This value is the maximum number
9566 of tokens that can be shifted (and not reduced) before overflow.
9568 The stack space allowed is not necessarily allocated. If you specify a
9569 large value for @code{YYMAXDEPTH}, the parser normally allocates a small
9570 stack at first, and then makes it bigger by stages as needed. This
9571 increasing allocation happens automatically and silently. Therefore,
9572 you do not need to make @code{YYMAXDEPTH} painfully small merely to save
9573 space for ordinary inputs that do not need much stack.
9575 However, do not allow @code{YYMAXDEPTH} to be a value so large that
9576 arithmetic overflow could occur when calculating the size of the stack
9577 space. Also, do not allow @code{YYMAXDEPTH} to be less than
9580 @cindex default stack limit
9581 The default value of @code{YYMAXDEPTH}, if you do not define it, is
9585 You can control how much stack is allocated initially by defining the
9586 macro @code{YYINITDEPTH} to a positive integer. For the deterministic
9587 parser in C, this value must be a compile-time constant
9588 unless you are assuming C99 or some other target language or compiler
9589 that allows variable-length arrays. The default is 200.
9591 Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
9593 You can generate a deterministic parser containing C++ user code from the
9594 default (C) skeleton, as well as from the C++ skeleton (@pxref{C++
9595 Parsers}). However, if you do use the default skeleton and want to allow
9596 the parsing stack to grow, be careful not to use semantic types or location
9597 types that require non-trivial copy constructors. The C skeleton bypasses
9598 these constructors when copying data to new, larger stacks.
9600 @node Error Recovery
9601 @chapter Error Recovery
9602 @cindex error recovery
9603 @cindex recovery from errors
9605 It is not usually acceptable to have a program terminate on a syntax
9606 error. For example, a compiler should recover sufficiently to parse the
9607 rest of the input file and check it for errors; a calculator should accept
9610 In a simple interactive command parser where each input is one line, it may
9611 be sufficient to allow @code{yyparse} to return 1 on error and have the
9612 caller ignore the rest of the input line when that happens (and then call
9613 @code{yyparse} again). But this is inadequate for a compiler, because it
9614 forgets all the syntactic context leading up to the error. A syntax error
9615 deep within a function in the compiler input should not cause the compiler
9616 to treat the following line like the beginning of a source file.
9619 You can define how to recover from a syntax error by writing rules to
9620 recognize the special token @code{error}. This is a terminal symbol that
9621 is always defined (you need not declare it) and reserved for error
9622 handling. The Bison parser generates an @code{error} token whenever a
9623 syntax error happens; if you have provided a rule to recognize this token
9624 in the current context, the parse can continue.
9636 The fourth rule in this example says that an error followed by a newline
9637 makes a valid addition to any @code{stmts}.
9639 What happens if a syntax error occurs in the middle of an @code{exp}? The
9640 error recovery rule, interpreted strictly, applies to the precise sequence
9641 of a @code{stmts}, an @code{error} and a newline. If an error occurs in
9642 the middle of an @code{exp}, there will probably be some additional tokens
9643 and subexpressions on the stack after the last @code{stmts}, and there
9644 will be tokens to read before the next newline. So the rule is not
9645 applicable in the ordinary way.
9647 But Bison can force the situation to fit the rule, by discarding part of the
9648 semantic context and part of the input. First it discards states and
9649 objects from the stack until it gets back to a state in which the
9650 @code{error} token is acceptable. (This means that the subexpressions
9651 already parsed are discarded, back to the last complete @code{stmts}.) At
9652 this point the @code{error} token can be shifted. Then, if the old
9653 lookahead token is not acceptable to be shifted next, the parser reads
9654 tokens and discards them until it finds a token which is acceptable. In
9655 this example, Bison reads and discards input until the next newline so that
9656 the fourth rule can apply. Note that discarded symbols are possible sources
9657 of memory leaks, see @ref{Destructor Decl}, for a means to reclaim this
9660 The choice of error rules in the grammar is a choice of strategies for
9661 error recovery. A simple and useful strategy is simply to skip the rest of
9662 the current input line or current statement if an error is detected:
9665 stmt: error ';' /* On error, skip until ';' is read. */
9668 It is also useful to recover to the matching close-delimiter of an
9669 opening-delimiter that has already been parsed. Otherwise the
9670 close-delimiter will probably appear to be unmatched, and generate another,
9671 spurious error message:
9681 Error recovery strategies are necessarily guesses. When they guess wrong,
9682 one syntax error often leads to another. In the above example, the error
9683 recovery rule guesses that an error is due to bad input within one
9684 @code{stmt}. Suppose that instead a spurious semicolon is inserted in the
9685 middle of a valid @code{stmt}. After the error recovery rule recovers from
9686 the first error, another syntax error will be found straight away, since the
9687 text following the spurious semicolon is also an invalid @code{stmt}.
9689 To prevent an outpouring of error messages, the parser will output no error
9690 message for another syntax error that happens shortly after the first; only
9691 after three consecutive input tokens have been successfully shifted will
9692 error messages resume.
9694 Note that rules which accept the @code{error} token may have actions, just
9695 as any other rules can.
9698 You can make error messages resume immediately by using the macro
9699 @code{yyerrok} in an action. If you do this in the error rule's action, no
9700 error messages will be suppressed. This macro requires no arguments;
9701 @samp{yyerrok;} is a valid C statement.
9704 The previous lookahead token is reanalyzed immediately after an error. If
9705 this is unacceptable, then the macro @code{yyclearin} may be used to clear
9706 this token. Write the statement @samp{yyclearin;} in the error rule's
9708 @xref{Action Features}.
9710 For example, suppose that on a syntax error, an error handling routine is
9711 called that advances the input stream to some point where parsing should
9712 once again commence. The next symbol returned by the lexical scanner is
9713 probably correct. The previous lookahead token ought to be discarded
9714 with @samp{yyclearin;}.
9716 @vindex YYRECOVERING
9717 The expression @code{YYRECOVERING ()} yields 1 when the parser
9718 is recovering from a syntax error, and 0 otherwise.
9719 Syntax error diagnostics are suppressed while recovering from a syntax
9722 @node Context Dependency
9723 @chapter Handling Context Dependencies
9725 The Bison paradigm is to parse tokens first, then group them into larger
9726 syntactic units. In many languages, the meaning of a token is affected by
9727 its context. Although this violates the Bison paradigm, certain techniques
9728 (known as @dfn{kludges}) may enable you to write Bison parsers for such
9732 * Semantic Tokens:: Token parsing can depend on the semantic context.
9733 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
9734 * Tie-in Recovery:: Lexical tie-ins have implications for how
9735 error recovery rules must be written.
9738 (Actually, ``kludge'' means any technique that gets its job done but is
9739 neither clean nor robust.)
9741 @node Semantic Tokens
9742 @section Semantic Info in Token Kinds
9744 The C language has a context dependency: the way an identifier is used
9745 depends on what its current meaning is. For example, consider this:
9751 This looks like a function call statement, but if @code{foo} is a typedef
9752 name, then this is actually a declaration of @code{x}. How can a Bison
9753 parser for C decide how to parse this input?
9755 The method used in GNU C is to have two different token kinds,
9756 @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
9757 identifier, it looks up the current declaration of the identifier in order
9758 to decide which token kind to return: @code{TYPENAME} if the identifier is
9759 declared as a typedef, @code{IDENTIFIER} otherwise.
9761 The grammar rules can then express the context dependency by the choice of
9762 token kind to recognize. @code{IDENTIFIER} is accepted as an expression,
9763 but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
9764 @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
9765 is @emph{not} significant, such as in declarations that can shadow a
9766 typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
9767 accepted---there is one rule for each of the two token kinds.
9769 This technique is simple to use if the decision of which kinds of
9770 identifiers to allow is made at a place close to where the identifier is
9771 parsed. But in C this is not always so: C allows a declaration to
9772 redeclare a typedef name provided an explicit type has been specified
9776 typedef int foo, bar;
9780 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
9781 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
9787 Unfortunately, the name being declared is separated from the declaration
9788 construct itself by a complicated syntactic structure---the ``declarator''.
9790 As a result, part of the Bison parser for C needs to be duplicated, with
9791 all the nonterminal names changed: once for parsing a declaration in
9792 which a typedef name can be redefined, and once for parsing a
9793 declaration in which that can't be done. Here is a part of the
9794 duplication, with actions omitted for brevity:
9799 declarator maybeasm '=' init
9800 | declarator maybeasm
9806 notype_declarator maybeasm '=' init
9807 | notype_declarator maybeasm
9813 Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
9814 cannot. The distinction between @code{declarator} and
9815 @code{notype_declarator} is the same sort of thing.
9817 There is some similarity between this technique and a lexical tie-in
9818 (described next), in that information which alters the lexical analysis is
9819 changed during parsing by other parts of the program. The difference is
9820 here the information is global, and is used for other purposes in the
9821 program. A true lexical tie-in has a special-purpose flag controlled by
9822 the syntactic context.
9824 @node Lexical Tie-ins
9825 @section Lexical Tie-ins
9826 @cindex lexical tie-in
9828 One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
9829 which is set by Bison actions, whose purpose is to alter the way tokens are
9832 For example, suppose we have a language vaguely like C, but with a special
9833 construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
9834 an expression in parentheses in which all integers are hexadecimal. In
9835 particular, the token @samp{a1b} must be treated as an integer rather than
9836 as an identifier if it appears in that context. Here is how you can do it:
9843 void yyerror (char const *);
9852 | HEX '(' @{ hexflag = 1; @}
9853 expr ')' @{ hexflag = 0; $$ = $4; @}
9854 | expr '+' expr @{ $$ = make_sum ($1, $3); @}
9868 Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
9869 it is nonzero, all integers are parsed in hexadecimal, and tokens starting
9870 with letters are parsed as integers if possible.
9872 The declaration of @code{hexflag} shown in the prologue of the grammar file
9873 is needed to make it accessible to the actions (@pxref{Prologue}). You must
9874 also write the code in @code{yylex} to obey the flag.
9876 @node Tie-in Recovery
9877 @section Lexical Tie-ins and Error Recovery
9879 Lexical tie-ins make strict demands on any error recovery rules you have.
9880 @xref{Error Recovery}.
9882 The reason for this is that the purpose of an error recovery rule is to
9883 abort the parsing of one construct and resume in some larger construct.
9884 For example, in C-like languages, a typical error recovery rule is to skip
9885 tokens until the next semicolon, and then start a new statement, like this:
9890 | IF '(' expr ')' stmt @{ @dots{} @}
9892 | error ';' @{ hexflag = 0; @}
9896 If there is a syntax error in the middle of a @samp{hex (@var{expr})}
9897 construct, this error rule will apply, and then the action for the
9898 completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
9899 remain set for the entire rest of the input, or until the next @code{hex}
9900 keyword, causing identifiers to be misinterpreted as integers.
9902 To avoid this problem the error recovery rule itself clears @code{hexflag}.
9904 There may also be an error recovery rule that works within expressions.
9905 For example, there could be a rule which applies within parentheses
9906 and skips to the close-parenthesis:
9912 | '(' expr ')' @{ $$ = $2; @}
9918 If this rule acts within the @code{hex} construct, it is not going to abort
9919 that construct (since it applies to an inner level of parentheses within
9920 the construct). Therefore, it should not clear the flag: the rest of
9921 the @code{hex} construct should be parsed with the flag still in effect.
9923 What if there is an error recovery rule which might abort out of the
9924 @code{hex} construct or might not, depending on circumstances? There is no
9925 way you can write the action to determine whether a @code{hex} construct is
9926 being aborted or not. So if you are using a lexical tie-in, you had better
9927 make sure your error recovery rules are not of this kind. Each rule must
9928 be such that you can be sure that it always will, or always won't, have to
9931 @c ================================================== Debugging Your Parser
9934 @chapter Debugging Your Parser
9936 Developing a parser can be a challenge, especially if you don't understand
9937 the algorithm (@pxref{Algorithm}). This chapter explains how to understand
9940 The most frequent issue users face is solving their conflicts. To fix them,
9941 the first step is understanding how they arise in a given grammar. This is
9942 made much easier by automated generation of counterexamples, cover in the
9943 first section (@pxref{Counterexamples}).
9945 In most cases though, looking at the structure of the automaton is still
9946 needed. The following sections explain how to generate and read the
9947 detailed structural description of the automaton. There are several formats
9951 as text, see @ref{Understanding};
9954 as a graph, see @ref{Graphviz};
9957 or as a markup report that can be turned, for instance, into HTML, see
9961 The last section focuses on the dynamic part of the parser: how to enable
9962 and understand the parser run-time traces (@pxref{Tracing}).
9965 * Counterexamples:: Understanding conflicts.
9966 * Understanding:: Understanding the structure of your parser.
9967 * Graphviz:: Getting a visual representation of the parser.
9968 * Xml:: Getting a markup representation of the parser.
9969 * Tracing:: Tracing the execution of your parser.
9972 @node Counterexamples
9973 @section Generation of Counterexamples
9975 @cindex counterexamples
9976 @cindex conflict counterexamples
9978 Solving conflicts is probably the most delicate part of the design of an LR
9979 parser, as demonstrated by the number of sections devoted to them in this
9980 very documentation. To solve a conflict, one must understand it: when does
9981 it occur? Is it because of a flaw in the grammar? Is it rather because
9982 LR(1) cannot cope with this grammar?
9984 One difficulty is that conflicts occur in the @emph{automaton}, and it can
9985 be tricky to relate them to issues in the @emph{grammar} itself. With
9986 experience and patience, analysis of the detailed description of the
9987 automaton (@pxref{Understanding}) allows one to find example strings that
9988 reach these conflicts.
9990 That task is made much easier thanks to the generation of counterexamples,
9991 initially developed by Chinawat Isradisaikul and Andrew Myers
9992 @pcite{Isradisaikul 2015}.
9994 As a first example, see the grammar of @ref{Shift/Reduce}, which features
9995 one shift/reduce conflict:
9999 $ @kbd{bison else.y}
10000 else.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10001 else.y: @dnotice{note}: rerun with option '-Wcounterexamples' to generate conflict counterexamples
10005 Let's rerun @command{bison} with the option
10006 @option{-Wcex}/@option{-Wcounterexamples}@inlinefmt{info, (the following
10007 output is actually in color)}:
10010 else.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10011 else.y: @dwarning{warning}: shift/reduce conflict on token "else" [@dwarning{-Wcounterexamples}]
10015 This shows two different derivations for one single expression, which proves
10016 that the grammar is ambiguous.
10020 As a more delicate example, consider the example grammar of
10021 @ref{Reduce/Reduce}, which features a reduce/reduce conflict:
10037 Bison generates the following counterexamples:
10041 $ @kbd{bison -Wcex sequence.y}
10042 sequence.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10043 sequence.y: @dwarning{warning}: 2 reduce/reduce conflicts [@dwarning{-Wconflicts-rr}]
10047 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10048 Example: @red{•} @green{"word"}
10051 @yellow{↳ 2:} @green{maybeword}
10052 @green{↳ 5:} @red{•} @green{"word"}
10053 Example: @red{•} @yellow{"word"}
10056 @yellow{↳ 3:} @green{sequence} @yellow{"word"}
10057 @green{↳ 1:} @red{•}
10060 sequence.y: @dwarning{warning}: reduce/reduce conflict on tokens $end, "word" [@dwarning{-Wcounterexamples}]
10062 First reduce derivation
10064 @yellow{↳ 1:} @red{•}
10066 Second reduce derivation
10068 @yellow{↳ 2:} @green{maybeword}
10069 @green{↳ 4:} @red{•}
10072 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10073 Example: @red{•} @green{"word"}
10076 @yellow{↳ 2:} @green{maybeword}
10077 @green{↳ 5:} @red{•} @green{"word"}
10078 Example: @red{•} @yellow{"word"}
10081 @yellow{↳ 3:} @green{sequence} @yellow{"word"}
10082 @green{↳ 2:} @blue{maybeword}
10083 @blue{↳ 4:} @red{•}
10086 sequence.y:8.3-45: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10087 8 | @dwarning{%empty @{ printf ("empty maybeword\n"); @}}
10088 | @dwarning{^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
10093 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10094 Example: @red{•} @green{"word"}
10097 @yellow{@arrow{} 2:} @green{maybeword}
10098 @green{@arrow{} 5:} @red{•} @green{"word"}
10099 Example: @red{•} @yellow{"word"}
10102 @yellow{@arrow{} 3:} @green{sequence} @yellow{"word"}
10103 @green{@arrow{} 1:} @red{•}
10106 sequence.y: @dwarning{warning}: reduce/reduce conflict on tokens $end, "word" [@dwarning{-Wcounterexamples}]
10108 First reduce derivation
10110 @yellow{@arrow{} 1:} @red{•}
10112 Second reduce derivation
10114 @yellow{@arrow{} 2:} @green{maybeword}
10115 @green{@arrow{}: 4} @red{•}
10118 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10119 Example: @red{•} @green{"word"}
10122 @yellow{@arrow{} 2:} @green{maybeword}
10123 @green{@arrow{} 5:} @red{•} @green{"word"}
10124 Example: @red{•} @yellow{"word"}
10127 @yellow{@arrow{} 3:} @green{sequence} @yellow{"word"}
10128 @green{@arrow{} 2:} @blue{maybeword}
10129 @blue{@arrow{} 4:} @red{•}
10132 sequence.y:8.3-45: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10133 8 | @dwarning{%empty @{ printf ("empty maybeword\n"); @}}
10134 | @dwarning{^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
10139 Each of these three conflicts, again, prove that the grammar is ambiguous.
10140 For instance, the second conflict (the reduce/reduce one) shows that the
10141 grammar accepts the empty input in two different ways.
10145 Sometimes, the search will not find an example that can be derived in two
10146 ways. In these cases, counterexample generation will provide two examples
10147 that are the same up until the dot. Most notably, this will happen when
10148 your grammar requires a stronger parser (more lookahead, LR instead of
10149 LALR). The following example isn't LR(1):
10157 expr: %empty | expr ID ','
10160 @command{bison} reports:
10163 ids.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10164 ids.y: @dwarning{warning}: shift/reduce conflict on token ID [@dwarning{-Wcounterexamples}]
10167 First example: @purple{expr} @red{•} @purple{ID ','} @green{ID} @yellow{$end}
10170 @yellow{↳ 0:} @green{s} @yellow{$end}
10171 @green{↳ 1:} @blue{a} @green{ID}
10172 @blue{↳ 2:} @purple{expr}
10173 @purple{↳ 4: expr} @red{•} @purple{ID ','}
10174 Second example: @blue{expr} @red{•} @green{ID} @yellow{$end}
10177 @yellow{↳ 0:} @green{s} @yellow{$end}
10178 @green{↳ 1:} @blue{a} @green{ID}
10179 @blue{↳ 2: expr} @red{•}
10182 ids.y:4.4-7: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10189 First example: @purple{expr} @red{•} @purple{ID ','} @green{ID} @yellow{$end}
10192 @yellow{@arrow{} 0:} @green{s} @yellow{$end}
10193 @green{@arrow{} 1:} @blue{a} @green{ID}
10194 @blue{@arrow{} 2:} @purple{expr}
10195 @purple{@arrow{} 4: expr} @red{•} @purple{ID ','}
10196 Second example: @blue{expr} @red{•} @green{ID} @yellow{$end}
10199 @yellow{@arrow{} 0:} @green{s} @yellow{$end}
10200 @green{@arrow{} 1:} @blue{a} @green{ID}
10201 @blue{@arrow{} 2: expr} @red{•}
10204 ids.y:4.4-7: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10211 This conflict is caused by the parser not having enough information to know
10212 the difference between these two examples. The parser would need an
10213 additional lookahead token to know whether or not a comma follows the
10214 @code{ID} after @code{expr}. These types of conflicts tend to be more
10215 difficult to fix, and usually need a rework of the grammar. In this case,
10216 it can be fixed by changing around the recursion: @code{expr: ID | ',' expr
10219 Alternatively, you might also want to consider using a GLR parser
10220 (@pxref{GLR Parsers}).
10224 On occasions, it is useful to look at counterexamples @emph{in situ}: with
10225 the automaton report (@xref{Understanding}, in particular @ref{state-8,,
10228 @node Understanding
10229 @section Understanding Your Parser
10231 Bison parsers are @dfn{shift/reduce automata} (@pxref{Algorithm}). In some
10232 cases (much more frequent than one would hope), looking at this automaton is
10233 required to tune or simply fix a parser.
10235 The textual file is generated when the options @option{--report} or
10236 @option{--verbose} are specified, see @ref{Invocation}. Its name is made by
10237 removing @samp{.tab.c} or @samp{.c} from the parser implementation file
10238 name, and adding @samp{.output} instead. Therefore, if the grammar file is
10239 @file{foo.y}, then the parser implementation file is called @file{foo.tab.c}
10240 by default. As a consequence, the verbose output file is called
10243 The following grammar file, @file{calc.y}, will be used in the sequel:
10260 %nterm <sval> useless
10280 @command{bison} reports:
10283 calc.y: @dwarning{warning}: 1 nonterminal useless in grammar [@dwarning{-Wother}]
10284 calc.y: @dwarning{warning}: 1 rule useless in grammar [@dwarning{-Wother}]
10285 calc.y:19.1-7: @dwarning{warning}: nonterminal useless in grammar: useless [@dwarning{-Wother}]
10286 19 | @dwarning{useless: STR;}
10287 | @dwarning{^~~~~~~}
10288 calc.y: @dwarning{warning}: 7 shift/reduce conflicts [@dwarning{-Wconflicts-sr}]
10289 calc.y: @dnotice{note}: rerun with option '-Wcounterexamples' to generate conflict counterexamples
10292 Going back to the calc example, when given @option{--report=state},
10293 in addition to @file{calc.tab.c}, it creates a file @file{calc.output}
10294 with contents detailed below. The order of the output and the exact
10295 presentation might vary, but the interpretation is the same.
10298 @cindex token, useless
10299 @cindex useless token
10300 @cindex nonterminal, useless
10301 @cindex useless nonterminal
10302 @cindex rule, useless
10303 @cindex useless rule
10304 The first section reports useless tokens, nonterminals and rules. Useless
10305 nonterminals and rules are removed in order to produce a smaller parser, but
10306 useless tokens are preserved, since they might be used by the scanner (note
10307 the difference between ``useless'' and ``unused'' below):
10310 Nonterminals useless in grammar
10313 Terminals unused in grammar
10316 Rules useless in grammar
10321 The next section lists states that still have conflicts.
10324 State 8 conflicts: 1 shift/reduce
10325 State 9 conflicts: 1 shift/reduce
10326 State 10 conflicts: 1 shift/reduce
10327 State 11 conflicts: 4 shift/reduce
10331 Then Bison reproduces the exact grammar it used:
10336 0 $accept: exp $end
10346 and reports the uses of the symbols:
10350 Terminals, with rules where they appear
10363 Nonterminals, with rules where they appear
10369 on right: 0 1 2 3 4
10375 @cindex dotted rule
10376 @cindex rule, dotted
10377 Bison then proceeds onto the automaton itself, describing each state with
10378 its set of @dfn{items}, also known as @dfn{dotted rules}. Each item is a
10379 production rule together with a point (@samp{.}) marking the location of the
10385 0 $accept: • exp $end
10387 NUM shift, and go to state 1
10392 This reads as follows: ``state 0 corresponds to being at the very
10393 beginning of the parsing, in the initial rule, right before the start
10394 symbol (here, @code{exp}). When the parser returns to this state right
10395 after having reduced a rule that produced an @code{exp}, the control
10396 flow jumps to state 2. If there is no such transition on a nonterminal
10397 symbol, and the lookahead is a @code{NUM}, then this token is shifted onto
10398 the parse stack, and the control flow jumps to state 1. Any other
10399 lookahead triggers a syntax error.''
10401 @cindex core, item set
10402 @cindex item set core
10403 @cindex kernel, item set
10404 @cindex item set core
10405 Even though the only active rule in state 0 seems to be rule 0, the
10406 report lists @code{NUM} as a lookahead token because @code{NUM} can be
10407 at the beginning of any rule deriving an @code{exp}. By default Bison
10408 reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
10409 you want to see more detail you can invoke @command{bison} with
10410 @option{--report=itemset} to list the derived items as well:
10415 0 $accept: • exp $end
10416 1 exp: • exp '+' exp
10417 2 | • exp '-' exp
10418 3 | • exp '*' exp
10419 4 | • exp '/' exp
10422 NUM shift, and go to state 1
10428 In the state 1@dots{}
10435 $default reduce using rule 5 (exp)
10439 the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
10440 (@samp{$default}), the parser will reduce it. If it was coming from State
10441 0, then, after this reduction it will return to state 0, and will jump to
10442 state 2 (@samp{exp: go to state 2}).
10447 0 $accept: exp • $end
10448 1 exp: exp • '+' exp
10449 2 | exp • '-' exp
10450 3 | exp • '*' exp
10451 4 | exp • '/' exp
10453 $end shift, and go to state 3
10454 '+' shift, and go to state 4
10455 '-' shift, and go to state 5
10456 '*' shift, and go to state 6
10457 '/' shift, and go to state 7
10461 In state 2, the automaton can only shift a symbol. For instance, because of
10462 the item @samp{exp: exp • '+' exp}, if the lookahead is @samp{+} it is
10463 shifted onto the parse stack, and the automaton jumps to state 4,
10464 corresponding to the item @samp{exp: exp '+' • exp}. Since there is no
10465 default action, any lookahead not listed triggers a syntax error.
10467 @cindex accepting state
10468 The state 3 is named the @dfn{final state}, or the @dfn{accepting
10474 0 $accept: exp $end •
10480 the initial rule is completed (the start symbol and the end-of-input were
10481 read), the parsing exits successfully.
10483 The interpretation of states 4 to 7 is straightforward, and is left to
10489 1 exp: exp '+' • exp
10491 NUM shift, and go to state 1
10498 2 exp: exp '-' • exp
10500 NUM shift, and go to state 1
10507 3 exp: exp '*' • exp
10509 NUM shift, and go to state 1
10516 4 exp: exp '/' • exp
10518 NUM shift, and go to state 1
10524 As was announced in beginning of the report, @samp{State 8 conflicts:
10530 1 exp: exp • '+' exp
10531 1 | exp '+' exp •
10532 2 | exp • '-' exp
10533 3 | exp • '*' exp
10534 4 | exp • '/' exp
10536 '*' shift, and go to state 6
10537 '/' shift, and go to state 7
10539 '/' [reduce using rule 1 (exp)]
10540 $default reduce using rule 1 (exp)
10543 Indeed, there are two actions associated to the lookahead @samp{/}:
10544 either shifting (and going to state 7), or reducing rule 1. The
10545 conflict means that either the grammar is ambiguous, or the parser lacks
10546 information to make the right decision. Indeed the grammar is
10547 ambiguous, as, since we did not specify the precedence of @samp{/}, the
10548 sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
10549 NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
10550 NUM}, which corresponds to reducing rule 1.
10552 Because in deterministic parsing a single decision can be made, Bison
10553 arbitrarily chose to disable the reduction, see @ref{Shift/Reduce}.
10554 Discarded actions are reported between square brackets.
10556 Note that all the previous states had a single possible action: either
10557 shifting the next token and going to the corresponding state, or
10558 reducing a single rule. In the other cases, i.e., when shifting
10559 @emph{and} reducing is possible or when @emph{several} reductions are
10560 possible, the lookahead is required to select the action. State 8 is
10561 one such state: if the lookahead is @samp{*} or @samp{/} then the action
10562 is shifting, otherwise the action is reducing rule 1. In other words,
10563 the first two items, corresponding to rule 1, are not eligible when the
10564 lookahead token is @samp{*}, since we specified that @samp{*} has higher
10565 precedence than @samp{+}. More generally, some items are eligible only
10566 with some set of possible lookahead tokens. When run with
10567 @option{--report=lookahead}, Bison specifies these lookahead tokens:
10572 1 exp: exp • '+' exp
10573 1 | exp '+' exp • [$end, '+', '-', '/']
10574 2 | exp • '-' exp
10575 3 | exp • '*' exp
10576 4 | exp • '/' exp
10578 '*' shift, and go to state 6
10579 '/' shift, and go to state 7
10581 '/' [reduce using rule 1 (exp)]
10582 $default reduce using rule 1 (exp)
10585 Note however that while @samp{NUM + NUM / NUM} is ambiguous (which results in
10586 the conflicts on @samp{/}), @samp{NUM + NUM * NUM} is not: the conflict was
10587 solved thanks to associativity and precedence directives. If invoked with
10588 @option{--report=solved}, Bison includes information about the solved
10589 conflicts in the report:
10592 Conflict between rule 1 and token '+' resolved as reduce (%left '+').
10593 Conflict between rule 1 and token '-' resolved as reduce (%left '-').
10594 Conflict between rule 1 and token '*' resolved as shift ('+' < '*').
10597 When given @option{--report=counterexamples}, @command{bison} will generate
10598 counterexamples within the report, augmented with the corresponding items
10599 (@pxref{Counterexamples}).
10603 shift/reduce conflict on token '/':
10604 1 exp: exp '+' exp •
10605 4 exp: exp • '/' exp
10607 Example: exp '+' exp • '/' exp
10611 ↳ 4: exp • '/' exp
10612 Example: exp '+' exp • '/' exp
10616 ↳ 1: exp '+' exp •
10622 shift/reduce conflict on token '/':
10623 1 exp: exp '+' exp •
10624 4 exp: exp • '/' exp
10626 Example: exp '+' exp • '/' exp
10629 @arrow{} 1: exp '+' exp
10630 @arrow{} 4: exp • '/' exp
10631 Example: exp '+' exp • '/' exp
10634 @arrow{} 4: exp '/' exp
10635 @arrow{} 1: exp '+' exp •
10640 This shows two separate derivations in the grammar for the same @code{exp}:
10641 @samp{e1 + e2 / e3}. The derivations show how your rules would parse the
10642 given example. Here, the first derivation completes a reduction when seeing
10643 @samp{/}, causing @samp{e1 + e2} to be grouped as an @code{exp}. The second
10644 derivation shifts on @samp{/}, resulting in @samp{e2 / e3} being grouped as
10645 an @code{exp}. Therefore, it is easy to see that adding
10646 precedence/associativity directives would fix this conflict.
10648 The remaining states are similar:
10654 1 exp: exp • '+' exp
10655 2 | exp • '-' exp
10656 2 | exp '-' exp •
10657 3 | exp • '*' exp
10658 4 | exp • '/' exp
10660 '*' shift, and go to state 6
10661 '/' shift, and go to state 7
10663 '/' [reduce using rule 2 (exp)]
10664 $default reduce using rule 2 (exp)
10670 1 exp: exp • '+' exp
10671 2 | exp • '-' exp
10672 3 | exp • '*' exp
10673 3 | exp '*' exp •
10674 4 | exp • '/' exp
10676 '/' shift, and go to state 7
10678 '/' [reduce using rule 3 (exp)]
10679 $default reduce using rule 3 (exp)
10685 1 exp: exp • '+' exp
10686 2 | exp • '-' exp
10687 3 | exp • '*' exp
10688 4 | exp • '/' exp
10689 4 | exp '/' exp •
10691 '+' shift, and go to state 4
10692 '-' shift, and go to state 5
10693 '*' shift, and go to state 6
10694 '/' shift, and go to state 7
10696 '+' [reduce using rule 4 (exp)]
10697 '-' [reduce using rule 4 (exp)]
10698 '*' [reduce using rule 4 (exp)]
10699 '/' [reduce using rule 4 (exp)]
10700 $default reduce using rule 4 (exp)
10705 Observe that state 11 contains conflicts not only due to the lack of
10706 precedence of @samp{/} with respect to @samp{+}, @samp{-}, and @samp{*}, but
10707 also because the associativity of @samp{/} is not specified.
10709 Bison may also produce an HTML version of this output, via an XML file and
10710 XSLT processing (@pxref{Xml}).
10712 @c ================================================= Graphical Representation
10715 @section Visualizing Your Parser
10718 As another means to gain better understanding of the shift/reduce
10719 automaton corresponding to the Bison parser, a DOT file can be generated. Note
10720 that debugging a real grammar with this is tedious at best, and impractical
10721 most of the times, because the generated files are huge (the generation of
10722 a PDF or PNG file from it will take very long, and more often than not it will
10723 fail due to memory exhaustion). This option was rather designed for beginners,
10724 to help them understand LR parsers.
10726 This file is generated when the @option{--graph} option is specified
10727 (@pxref{Invocation}). Its name is made by removing
10728 @samp{.tab.c} or @samp{.c} from the parser implementation file name, and
10729 adding @samp{.gv} instead. If the grammar file is @file{foo.y}, the
10730 Graphviz output file is called @file{foo.gv}. A DOT file may also be
10731 produced via an XML file and XSLT processing (@pxref{Xml}).
10734 The following grammar file, @file{rr.y}, will be used in the sequel:
10739 exp: a ";" | b ".";
10745 The graphical output
10747 (see @ref{fig:graph})
10749 is very similar to the textual one, and as such it is easier understood by
10750 making direct comparisons between them. @xref{Debugging}, for a detailed
10751 analysis of the textual report.
10754 @float Figure,fig:graph
10755 @center @image{figs/example, 430pt,,,.svg}
10756 @caption{A graphical rendering of the parser.}
10760 @subheading Graphical Representation of States
10762 The items (dotted rules) for each state are grouped together in graph nodes.
10763 Their numbering is the same as in the verbose file. See the following
10764 points, about transitions, for examples
10766 When invoked with @option{--report=lookaheads}, the lookahead tokens, when
10767 needed, are shown next to the relevant rule between square brackets as a
10768 comma separated list. This is the case in the figure for the representation of
10773 The transitions are represented as directed edges between the current and
10776 @subheading Graphical Representation of Shifts
10778 Shifts are shown as solid arrows, labeled with the lookahead token for that
10779 shift. The following describes a reduction in the @file{rr.output} file:
10787 ";" shift, and go to state 6
10791 A Graphviz rendering of this portion of the graph could be:
10793 @center @image{figs/example-shift, 100pt,,,.svg}
10795 @subheading Graphical Representation of Reductions
10797 Reductions are shown as solid arrows, leading to a diamond-shaped node
10798 bearing the number of the reduction rule. The arrow is labeled with the
10799 appropriate comma separated lookahead tokens. If the reduction is the default
10800 action for the given state, there is no such label.
10802 This is how reductions are represented in the verbose file @file{rr.output}:
10809 "." reduce using rule 4 (b)
10810 $default reduce using rule 3 (a)
10813 A Graphviz rendering of this portion of the graph could be:
10815 @center @image{figs/example-reduce, 120pt,,,.svg}
10817 When unresolved conflicts are present, because in deterministic parsing
10818 a single decision can be made, Bison can arbitrarily choose to disable a
10819 reduction, see @ref{Shift/Reduce}. Discarded actions
10820 are distinguished by a red filling color on these nodes, just like how they are
10821 reported between square brackets in the verbose file.
10823 The reduction corresponding to the rule number 0 is the acceptation
10824 state. It is shown as a blue diamond, labeled ``Acc''.
10826 @subheading Graphical Representation of Gotos
10828 The @samp{go to} jump transitions are represented as dotted lines bearing
10829 the name of the rule being jumped to.
10831 @c ================================================= XML
10834 @section Visualizing your parser in multiple formats
10837 Bison supports two major report formats: textual output
10838 (@pxref{Understanding}) when invoked
10839 with option @option{--verbose}, and DOT
10840 (@pxref{Graphviz}) when invoked with
10841 option @option{--graph}. However,
10842 another alternative is to output an XML file that may then be, with
10843 @command{xsltproc}, rendered as either a raw text format equivalent to the
10844 verbose file, or as an HTML version of the same file, with clickable
10845 transitions, or even as a DOT. The @file{.output} and DOT files obtained via
10846 XSLT have no difference whatsoever with those obtained by invoking
10847 @command{bison} with options @option{--verbose} or @option{--graph}.
10849 The XML file is generated when the options @option{-x} or
10850 @option{--xml[=FILE]} are specified, see @ref{Invocation}.
10851 If not specified, its name is made by removing @samp{.tab.c} or @samp{.c}
10852 from the parser implementation file name, and adding @samp{.xml} instead.
10853 For instance, if the grammar file is @file{foo.y}, the default XML output
10854 file is @file{foo.xml}.
10856 Bison ships with a @file{data/xslt} directory, containing XSL Transformation
10857 files to apply to the XML file. Their names are non-ambiguous:
10861 Used to output a copy of the DOT visualization of the automaton.
10863 Used to output a copy of the @samp{.output} file.
10864 @item xml2xhtml.xsl
10865 Used to output an xhtml enhancement of the @samp{.output} file.
10868 Sample usage (requires @command{xsltproc}):
10870 $ @kbd{bison -x gr.y}
10872 $ @kbd{bison --print-datadir}
10873 /usr/local/share/bison
10875 $ @kbd{xsltproc /usr/local/share/bison/xslt/xml2xhtml.xsl gr.xml >gr.html}
10878 @c ================================================= Tracing
10881 @section Tracing Your Parser
10884 @cindex tracing the parser
10886 When a Bison grammar compiles properly but parses ``incorrectly'', the
10887 @code{yydebug} parser-trace feature helps figuring out why.
10890 * Enabling Traces:: Activating run-time trace support
10891 * Mfcalc Traces:: Extending @code{mfcalc} to support traces
10892 * The YYPRINT Macro:: Obsolete interface for semantic value reports
10895 @node Enabling Traces
10896 @subsection Enabling Traces
10897 There are several means to enable compilation of trace facilities, in
10898 decreasing order of preference:
10901 @item the variable @samp{parse.trace}
10902 @findex %define parse.trace
10903 Add the @samp{%define parse.trace} directive (@pxref{%define
10904 Summary}), or pass the @option{-Dparse.trace} option
10905 (@pxref{Tuning the Parser}). This is a Bison extension. Unless POSIX and
10906 Yacc portability matter to you, this is the preferred solution.
10908 @item the option @option{-t} (POSIX Yacc compliant)
10909 @itemx the option @option{--debug} (Bison extension)
10910 Use the @option{-t} option when you run Bison (@pxref{Invocation}). With
10911 @samp{%define api.prefix @{c@}}, it defines @code{CDEBUG} to 1, otherwise it
10912 defines @code{YYDEBUG} to 1.
10914 @item the directive @samp{%debug} (deprecated)
10916 Add the @code{%debug} directive (@pxref{Decl Summary}). This Bison
10917 extension is maintained for backward compatibility with previous versions of
10918 Bison; use @code{%define parse.trace} instead.
10920 @item the macro @code{YYDEBUG} (C/C++ only)
10922 Define the macro @code{YYDEBUG} to a nonzero value when you compile the
10923 parser. This is compliant with POSIX Yacc. You could use
10924 @option{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
10925 YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue}).
10927 If the @code{%define} variable @code{api.prefix} is used (@pxref{Multiple
10928 Parsers}), for instance @samp{%define
10929 api.prefix @{c@}}, then if @code{CDEBUG} is defined, its value controls the
10930 tracing feature (enabled if and only if nonzero); otherwise tracing is
10931 enabled if and only if @code{YYDEBUG} is nonzero.
10934 We suggest that you always enable the trace option so that debugging is
10938 In C the trace facility outputs messages with macro calls of the form
10939 @code{YYFPRINTF (stderr, @var{format}, @var{args})} where @var{format} and
10940 @var{args} are the usual @code{printf} format and variadic arguments. If
10941 you define @code{YYDEBUG} to a nonzero value but do not define
10942 @code{YYFPRINTF}, @code{<stdio.h>} is automatically included and
10943 @code{YYFPRINTF} is defined to @code{fprintf}.
10945 Once you have compiled the program with trace facilities, the way to request
10946 a trace is to store a nonzero value in the variable @code{yydebug}. You can
10947 do this by making the C code do it (in @code{main}, perhaps), or you can
10948 alter the value with a C debugger.
10950 Each step taken by the parser when @code{yydebug} is nonzero produces a line
10951 or two of trace information, written on @code{stderr}. The trace messages
10952 tell you these things:
10956 Each time the parser calls @code{yylex}, what kind of token was read.
10959 Each time a token is shifted, the depth and complete contents of the state
10960 stack (@pxref{Parser States}).
10963 Each time a rule is reduced, which rule it is, and the complete contents of
10964 the state stack afterward.
10967 To make sense of this information, it helps to refer to the automaton
10968 description file (@pxref{Understanding}). This
10969 file shows the meaning of each state in terms of positions in various rules,
10970 and also what each state will do with each possible input token. As you
10971 read the successive trace messages, you can see that the parser is
10972 functioning according to its specification in the listing file. Eventually
10973 you will arrive at the place where something undesirable happens, and you
10974 will see which parts of the grammar are to blame.
10976 The parser implementation file is a C/C++/D/Java program and you can use
10977 debuggers on it, but it's not easy to interpret what it is doing. The
10978 parser function is a finite-state machine interpreter, and aside from the
10979 actions it executes the same code over and over. Only the values of
10980 variables show where in the grammar it is working.
10982 @node Mfcalc Traces
10983 @subsection Enabling Debug Traces for @code{mfcalc}
10985 The debugging information normally gives the token kind of each token read,
10986 but not its semantic value. The @code{%printer} directive allows specify
10987 how semantic values are reported, see @ref{Printer Decl}.
10989 As a demonstration of @code{%printer}, consider the multi-function
10990 calculator, @code{mfcalc} (@pxref{Multi-function Calc}). To enable run-time
10991 traces, and semantic value reports, insert the following directives in its
10994 @comment file: c/mfcalc/mfcalc.y: 2
10996 /* Generate the parser description file. */
10998 /* Enable run-time traces (yydebug). */
10999 %define parse.trace
11001 /* Formatting semantic values. */
11002 %printer @{ fprintf (yyo, "%s", $$->name); @} VAR;
11003 %printer @{ fprintf (yyo, "%s()", $$->name); @} FUN;
11004 %printer @{ fprintf (yyo, "%g", $$); @} <double>;
11007 The @code{%define} directive instructs Bison to generate run-time trace
11008 support. Then, activation of these traces is controlled at run-time by the
11009 @code{yydebug} variable, which is disabled by default. Because these traces
11010 will refer to the ``states'' of the parser, it is helpful to ask for the
11011 creation of a description of that parser; this is the purpose of (admittedly
11012 ill-named) @code{%verbose} directive.
11014 The set of @code{%printer} directives demonstrates how to format the
11015 semantic value in the traces. Note that the specification can be done
11016 either on the symbol type (e.g., @code{VAR} or @code{FUN}), or on the type
11017 tag: since @code{<double>} is the type for both @code{NUM} and @code{exp},
11018 this printer will be used for them.
11020 Here is a sample of the information provided by run-time traces. The traces
11021 are sent onto standard error.
11024 $ @kbd{echo 'sin(1-1)' | ./mfcalc -p}
11027 Reducing stack by rule 1 (line 34):
11028 -> $$ = nterm input ()
11034 This first batch shows a specific feature of this grammar: the first rule
11035 (which is in line 34 of @file{mfcalc.y} can be reduced without even having
11036 to look for the first token. The resulting left-hand symbol (@code{$$}) is
11037 a valueless (@samp{()}) @code{input} nonterminal (@code{nterm}).
11039 Then the parser calls the scanner.
11042 Next token is token FUN (sin())
11043 Shifting token FUN (sin())
11048 That token (@code{token}) is a function (@code{FUN}) whose value is
11049 @samp{sin} as formatted per our @code{%printer} specification: @samp{sin()}.
11050 The parser stores (@code{Shifting}) that token, and others, until it can do
11051 something about it.
11055 Next token is token '(' ()
11056 Shifting token '(' ()
11059 Next token is token NUM (1.000000)
11060 Shifting token NUM (1.000000)
11062 Reducing stack by rule 6 (line 44):
11063 $1 = token NUM (1.000000)
11064 -> $$ = nterm exp (1.000000)
11070 The previous reduction demonstrates the @code{%printer} directive for
11071 @code{<double>}: both the token @code{NUM} and the resulting nonterminal
11072 @code{exp} have @samp{1} as value.
11076 Next token is token '-' ()
11077 Shifting token '-' ()
11080 Next token is token NUM (1.000000)
11081 Shifting token NUM (1.000000)
11083 Reducing stack by rule 6 (line 44):
11084 $1 = token NUM (1.000000)
11085 -> $$ = nterm exp (1.000000)
11086 Stack now 0 1 6 14 24 17
11089 Next token is token ')' ()
11090 Reducing stack by rule 11 (line 49):
11091 $1 = nterm exp (1.000000)
11093 $3 = nterm exp (1.000000)
11094 -> $$ = nterm exp (0.000000)
11100 The rule for the subtraction was just reduced. The parser is about to
11101 discover the end of the call to @code{sin}.
11104 Next token is token ')' ()
11105 Shifting token ')' ()
11107 Reducing stack by rule 9 (line 47):
11108 $1 = token FUN (sin())
11110 $3 = nterm exp (0.000000)
11112 -> $$ = nterm exp (0.000000)
11118 Finally, the end-of-line allow the parser to complete the computation, and
11119 display its result.
11123 Next token is token '\n' ()
11124 Shifting token '\n' ()
11126 Reducing stack by rule 4 (line 40):
11127 $1 = nterm exp (0.000000)
11130 -> $$ = nterm line ()
11133 Reducing stack by rule 2 (line 35):
11134 $1 = nterm input ()
11136 -> $$ = nterm input ()
11141 The parser has returned into state 1, in which it is waiting for the next
11142 expression to evaluate, or for the end-of-file token, which causes the
11143 completion of the parsing.
11147 Now at end of input.
11148 Shifting token $end ()
11151 Cleanup: popping token $end ()
11152 Cleanup: popping nterm input ()
11156 @node The YYPRINT Macro
11157 @subsection The @code{YYPRINT} Macro
11160 The @code{%printer} directive was introduced in Bison 1.50 (November 2002).
11161 Before then, @code{YYPRINT} provided a similar feature, but only for
11162 terminal symbols and only with the @file{yacc.c} skeleton.
11164 @deffn {Macro} YYPRINT (@var{stream}, @var{token}, @var{value});
11166 Deprecated, will be removed eventually.
11168 If you define @code{YYPRINT}, it should take three arguments. The parser
11169 will pass a standard I/O stream, the numeric code for the token kind, and
11170 the token value (from @code{yylval}).
11172 For @file{yacc.c} only. Obsoleted by @code{%printer}.
11175 Here is an example of @code{YYPRINT} suitable for the multi-function
11176 calculator (@pxref{Mfcalc Declarations}):
11180 static void print_token_value (FILE *file, int type, YYSTYPE value);
11181 #define YYPRINT(File, Type, Value) \
11182 print_token_value (File, Type, Value)
11185 @dots{} %% @dots{} %% @dots{}
11188 print_token_value (FILE *file, yytoken_kind_t kind, YYSTYPE value)
11191 fprintf (file, "%s", value.tptr->name);
11192 else if (kind == NUM)
11193 fprintf (file, "%d", value.val);
11197 @xref{Mfcalc Traces}, for the proper use of @code{%printer}.
11199 @c ================================================= Invoking Bison
11202 @chapter Invoking Bison
11203 @cindex invoking Bison
11204 @cindex Bison invocation
11205 @cindex options for invoking Bison
11207 The usual way to invoke Bison is as follows:
11210 $ @kbd{bison @var{file}}
11213 Here @var{file} is the grammar file name, which usually ends in @samp{.y}.
11214 The parser implementation file's name is made by replacing the @samp{.y}
11215 with @samp{.tab.c} and removing any leading directory. Thus, the
11216 @samp{bison foo.y} file name yields @file{foo.tab.c}, and the @samp{bison
11217 hack/foo.y} file name yields @file{foo.tab.c}. It's also possible, in case
11218 you are writing C++ code instead of C in your grammar file, to name it
11219 @file{foo.ypp} or @file{foo.y++}. Then, the output files will take an
11220 extension like the given one as input (respectively @file{foo.tab.cpp} and
11221 @file{foo.tab.c++}). This feature takes effect with all options that
11222 manipulate file names like @option{-o} or @option{-d}.
11227 $ @kbd{bison -d @var{file.yxx}}
11230 will produce @file{file.tab.cxx} and @file{file.tab.hxx}, and
11233 $ @kbd{bison -d -o @var{output.c++} @var{file.y}}
11236 will produce @file{output.c++} and @file{output.h++}.
11238 For compatibility with POSIX, the standard Bison distribution also contains
11239 a shell script called @command{yacc} that invokes Bison with the @option{-y}
11244 The exit status of @command{bison} is:
11247 when there were no errors. Warnings, which are diagnostics about dubious
11248 constructs, do not change the exit status, unless they are turned into
11249 errors (@pxref{Werror,,@option{-Werror}}).
11252 when there were errors. No file was generated (except the reports generated
11253 by @option{--verbose}, etc.). In particular, the output files that possibly
11254 existed were not changed.
11256 @item 63 (mismatch)
11257 when @command{bison} does not meet the version requirements of the grammar
11258 file. @xref{Require Decl}. No file was generated or changed.
11263 * Bison Options:: All the options described in detail,
11264 in alphabetical order by short options.
11265 * Option Cross Key:: Alphabetical list of long options.
11266 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
11269 @node Bison Options
11270 @section Bison Options
11272 Bison supports both traditional single-letter options and mnemonic long
11273 option names. Long option names are indicated with @option{--} instead of
11274 @option{-}. Abbreviations for option names are allowed as long as they
11275 are unique. When a long option takes an argument, like
11276 @option{--file-prefix}, connect the option name and the argument with
11279 Here is a list of options that can be used with Bison. It is followed by a
11280 cross key alphabetized by long option.
11283 * Operation Modes:: Options controlling the global behavior of @command{bison}
11284 * Diagnostics:: Options controlling the diagnostics
11285 * Tuning the Parser:: Options changing the generated parsers
11286 * Output Files:: Options controlling the output
11289 @node Operation Modes
11290 @subsection Operation Modes
11292 Options controlling the global behavior of @command{bison}.
11294 @c Please, keep this ordered as in 'bison --help'.
11298 Print a summary of the command-line options to Bison and exit.
11302 Print the version number of Bison and exit.
11304 @item --print-localedir
11305 Print the name of the directory containing locale-dependent data.
11307 @item --print-datadir
11308 Print the name of the directory containing skeletons, CSS and XSLT.
11312 Update the grammar file (remove duplicates, update deprecated directives,
11313 etc.) and exit (i.e., do not generate any of the output files). Leaves a
11314 backup of the original file with a @code{~} appended. For instance:
11320 %define parse.error verbose
11325 $ @kbd{bison -u foo.y}
11326 foo.y:1.1-14: @dwarning{warning}: deprecated directive, use '%define parse.error verbose' [@dwarning{-Wdeprecated}]
11327 1 | @dwarning{%error-verbose}
11328 | @dwarning{^~~~~~~~~~~~~~}
11329 foo.y:2.1-27: @dwarning{warning}: %define variable 'parse.error' redefined [@dwarning{-Wother}]
11330 2 | @dwarning{%define parse.error verbose}
11331 | @dwarning{^~~~~~~~~~~~~~~~~~~~~~~~~~~}
11332 foo.y:1.1-14: previous definition
11333 1 | @dnotice{%error-verbose}
11334 | @dnotice{^~~~~~~~~~~~~~}
11335 bison: file 'foo.y' was updated (backup: 'foo.y~')
11339 %define parse.error verbose
11345 See the documentation of @option{--feature=fixit} below for more details.
11347 @item -f [@var{feature}]
11348 @itemx --feature[=@var{feature}]
11349 Activate miscellaneous @var{feature}s. @var{Feature} can be one of:
11352 @itemx diagnostics-show-caret
11353 Show caret errors, in a manner similar to GCC's
11354 @option{-fdiagnostics-show-caret}, or Clang's
11355 @option{-fcaret-diagnostics}. The location provided with the message is used
11356 to quote the corresponding line of the source file, underlining the
11357 important part of it with carets (@samp{^}). Here is an example, using the
11358 following file @file{in.y}:
11363 exp: exp '+' exp @{ $exp = $1 + $2; @};
11366 When invoked with @option{-fcaret} (or nothing), Bison will report:
11370 in.y:3.20-23: @derror{error}: ambiguous reference: '$exp'
11371 3 | exp: exp '+' exp @{ @derror{$exp} = $1 + $2; @};
11375 in.y:3.1-3: refers to: $exp at $$
11376 3 | @dnotice{exp}: exp '+' exp @{ $exp = $1 + $2; @};
11380 in.y:3.6-8: refers to: $exp at $1
11381 3 | exp: @dnotice{exp} '+' exp @{ $exp = $1 + $2; @};
11385 in.y:3.14-16: refers to: $exp at $3
11386 3 | exp: exp '+' @dnotice{exp} @{ $exp = $1 + $2; @};
11390 in.y:3.32-33: @derror{error}: $2 of 'exp' has no declared type
11391 3 | exp: exp '+' exp @{ $exp = $1 + @derror{$2}; @};
11396 Whereas, when invoked with @option{-fno-caret}, Bison will only report:
11400 in.y:3.20-23: @derror{error}: ambiguous reference: '$exp'
11401 in.y:3.1-3: refers to: $exp at $$
11402 in.y:3.6-8: refers to: $exp at $1
11403 in.y:3.14-16: refers to: $exp at $3
11404 in.y:3.32-33: @derror{error}: $2 of 'exp' has no declared type
11408 This option is activated by default.
11411 @itemx diagnostics-parseable-fixits
11412 Show machine-readable fixes, in a manner similar to GCC's and Clang's
11413 @option{-fdiagnostics-parseable-fixits}.
11415 Fix-its are generated for duplicate directives:
11420 %define api.prefix @{foo@}
11421 %define api.prefix @{bar@}
11427 $ @kbd{bison -ffixit foo.y}
11428 foo.y:2.1-24: @derror{error}: %define variable 'api.prefix' redefined
11429 2 | @derror{%define api.prefix @{bar@}}
11430 | @derror{^~~~~~~~~~~~~~~~~~~~~~~~}
11431 foo.y:1.1-24: previous definition
11432 1 | @dnotice{%define api.prefix @{foo@}}
11433 | @dnotice{^~~~~~~~~~~~~~~~~~~~~~~~}
11434 fix-it:"foo.y":@{2:1-2:25@}:""
11435 foo.y: @dwarning{warning}: fix-its can be applied. Rerun with option '--update'. [@dwarning{-Wother}]
11439 They are also generated to update deprecated directives, unless
11440 @option{-Wno-deprecated} was given:
11444 $ @kbd{cat /tmp/foo.yy}
11451 $ @kbd{bison foo.y}
11452 foo.y:1.1-14: @dwarning{warning}: deprecated directive, use '%define parse.error verbose' [@dwarning{-Wdeprecated}]
11453 1 | @dwarning{%error-verbose}
11454 | @dwarning{^~~~~~~~~~~~~~}
11455 foo.y:2.1-18: @dwarning{warning}: deprecated directive, use '%define api.prefix @{foo@}' [@dwarning{-Wdeprecated}]
11456 2 | @dwarning{%name-prefix "foo"}
11457 | @dwarning{^~~~~~~~~~~~~~~~~~}
11458 foo.y: @dwarning{warning}: fix-its can be applied. Rerun with option '--update'. [@dwarning{-Wother}]
11462 The fix-its are applied by @command{bison} itself when given the option
11463 @option{-u}/@option{--update}. See its documentation above.
11466 Do not generate the output files. The name of this feature is somewhat
11467 misleading as more than just checking the syntax is done: every stage is run
11468 (including checking for conflicts for instance), except the generation of
11475 @subsection Diagnostics
11477 Options controlling the diagnostics.
11479 @c Please, keep this ordered as in 'bison --help'.
11481 @item -W [@var{category}]
11482 @itemx --warnings[=@var{category}]
11483 Output warnings falling in @var{category}. @var{category} can be one
11486 @item @anchor{Wconflicts-sr}conflicts-sr
11487 @itemx @anchor{Wconflicts-rr}conflicts-rr
11488 S/R and R/R conflicts. These warnings are enabled by default. However, if
11489 the @code{%expect} or @code{%expect-rr} directive is specified, an
11490 unexpected number of conflicts is an error, and an expected number of
11491 conflicts is not reported, so @option{-W} and @option{--warning} then have
11492 no effect on the conflict report.
11494 @item @anchor{Wcounterexamples}counterexamples
11496 Provide counterexamples for conflicts. @xref{Counterexamples}.
11497 Counterexamples take time to compute. The option @option{-Wcex} should be
11498 used by the developer when working on the grammar; it hardly makes sense to
11501 @item @anchor{Wdangling-alias}dangling-alias
11502 Report string literals that are not bound to a token symbol.
11504 String literals, which allow for better error messages, are (too) liberally
11505 accepted by Bison, which might result in silent errors. For instance
11508 %type <exVal> cond "condition"
11512 does not define ``condition'' as a string alias to @code{cond}---nonterminal
11513 symbols do not have string aliases. It is rather equivalent to
11516 %nterm <exVal> cond
11517 %token <exVal> "condition"
11521 i.e., it gives the @samp{"condition"} token the type @code{exVal}.
11523 Also, because string aliases do not need to be defined, typos such as
11524 @samp{"baz"} instead of @samp{"bar"} will be not reported.
11526 The option @option{-Wdangling-alias} catches these situations. On
11530 %type <ival> foo "foo"
11536 @command{bison -Wdangling-alias} reports
11539 @dwarning{warning}: string literal not attached to a symbol
11540 | %type <ival> foo @dwarning{"foo"}
11542 @dwarning{warning}: string literal not attached to a symbol
11543 | foo: @dwarning{"baz"} @{@}
11547 @item @anchor{Wdeprecated}deprecated
11548 Deprecated constructs whose support will be removed in future versions of
11551 @item @anchor{Wempty-rule}empty-rule
11552 Empty rules without @code{%empty}. @xref{Empty Rules}. Disabled by
11553 default, but enabled by uses of @code{%empty}, unless
11554 @option{-Wno-empty-rule} was specified.
11556 @item @anchor{Wmidrule-values}midrule-values
11557 Warn about midrule values that are set but not used within any of the actions
11558 of the parent rule.
11559 For example, warn about unused @code{$2} in:
11562 exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
11565 Also warn about midrule values that are used but not set.
11566 For example, warn about unset @code{$$} in the midrule action in:
11569 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
11572 These warnings are not enabled by default since they sometimes prove to
11573 be false alarms in existing grammars employing the Yacc constructs
11574 @code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
11576 @item @anchor{Wprecedence}precedence
11577 Useless precedence and associativity directives. Disabled by default.
11579 Consider for instance the following grammar:
11608 @c cannot leave the location and the [-Wprecedence] for lack of
11612 @dwarning{warning}: useless precedence and associativity for "="
11613 | %nonassoc @dwarning{"="}
11617 @dwarning{warning}: useless associativity for "*", use %precedence
11618 | %left @dwarning{"*"}
11622 @dwarning{warning}: useless precedence for "("
11623 | %precedence @dwarning{"("}
11628 One would get the exact same parser with the following directives instead:
11637 @item @anchor{Wyacc}yacc
11638 Incompatibilities with POSIX Yacc.
11640 @item @anchor{Wother}other
11641 All warnings not categorized above. These warnings are enabled by default.
11643 This category is provided merely for the sake of completeness. Future
11644 releases of Bison may move warnings from this category to new, more specific
11647 @item @anchor{Wall}all
11648 All the warnings except @code{counterexamples}, @code{dangling-alias} and
11651 @item @anchor{Wnone}none
11652 Turn off all the warnings.
11655 See @option{-Werror}, below.
11658 A category can be turned off by prefixing its name with @samp{no-}. For
11659 instance, @option{-Wno-yacc} will hide the warnings about
11660 POSIX Yacc incompatibilities.
11662 @item @anchor{Werror}-Werror
11663 Turn enabled warnings for every @var{category} into errors, unless they are
11664 explicitly disabled by @option{-Wno-error=@var{category}}.
11666 @item -Werror=@var{category}
11667 Enable warnings falling in @var{category}, and treat them as errors.
11669 @var{category} is the same as for @option{--warnings}, with the exception that
11670 it may not be prefixed with @samp{no-} (see above).
11672 Note that the precedence of the @samp{=} and @samp{,} operators is such that
11673 the following commands are @emph{not} equivalent, as the first will not treat
11674 S/R conflicts as errors.
11677 $ @kbd{bison -Werror=yacc,conflicts-sr input.y}
11678 $ @kbd{bison -Werror=yacc,error=conflicts-sr input.y}
11682 Do not turn enabled warnings for every @var{category} into errors, unless
11683 they are explicitly enabled by @option{-Werror=@var{category}}.
11685 @item -Wno-error=@var{category}
11686 Deactivate the error treatment for this @var{category}. However, the warning
11687 itself won't be disabled, or enabled, by this option.
11690 Equivalent to @option{--color=always}.
11692 @item --color=@var{when}
11693 Control whether diagnostics are colorized, depending on @var{when}:
11697 Enable colorized diagnostics.
11701 Disable colorized diagnostics.
11703 @item auto @r{(default)}
11705 Diagnostics will be colorized if the output device is a tty, i.e. when the
11706 output goes directly to a text screen or terminal emulator window.
11709 @item --style=@var{file}
11710 Specifies the CSS style @var{file} to use when colorizing. It has an effect
11711 only when the @option{--color} option is effective. The
11712 @file{bison-default.css} file provide a good example from which to define
11713 your own style file. See the documentation of libtextstyle for more
11717 @node Tuning the Parser
11718 @subsection Tuning the Parser
11720 Options changing the generated parsers.
11722 @c Please, keep this ordered as in 'bison --help'.
11726 In the parser implementation file, define the macro @code{YYDEBUG} to 1 if
11727 it is not already defined, so that the debugging facilities are compiled.
11730 @item -D @var{name}[=@var{value}]
11731 @itemx --define=@var{name}[=@var{value}]
11732 @itemx -F @var{name}[=@var{value}]
11733 @itemx --force-define=@var{name}[=@var{value}]
11734 Each of these is equivalent to @samp{%define @var{name} @var{value}}
11735 (@pxref{%define Summary}). Note that the delimiters are part of
11736 @var{value}: @option{-Dapi.value.type=union},
11737 @option{-Dapi.value.type=@{union@}} and @option{-Dapi.value.type="union"}
11738 correspond to @samp{%define api.value.type union}, @samp{%define
11739 api.value.type @{union@}} and @samp{%define api.value.type "union"}.
11741 Bison processes multiple definitions for the same @var{name} as follows:
11745 Bison quietly ignores all command-line definitions for @var{name} except
11748 If that command-line definition is specified by a @option{-D} or
11749 @option{--define}, Bison reports an error for any @code{%define} definition
11752 If that command-line definition is specified by a @option{-F} or
11753 @option{--force-define} instead, Bison quietly ignores all @code{%define}
11754 definitions for @var{name}.
11756 Otherwise, Bison reports an error if there are multiple @code{%define}
11757 definitions for @var{name}.
11760 You should avoid using @option{-F} and @option{--force-define} in your
11761 make files unless you are confident that it is safe to quietly ignore
11762 any conflicting @code{%define} that may be added to the grammar file.
11764 @item -L @var{language}
11765 @itemx --language=@var{language}
11766 Specify the programming language for the generated parser, as if
11767 @code{%language} was specified (@pxref{Decl Summary}). Currently supported
11768 languages include C, C++, D and Java. @var{language} is case-insensitive.
11771 Pretend that @code{%locations} was specified. @xref{Decl Summary}.
11773 @item -p @var{prefix}
11774 @itemx --name-prefix=@var{prefix}
11775 Pretend that @code{%name-prefix "@var{prefix}"} was specified (@pxref{Decl
11776 Summary}). Obsoleted by @option{-Dapi.prefix=@var{prefix}}. @xref{Multiple
11781 Don't put any @code{#line} preprocessor commands in the parser
11782 implementation file. Ordinarily Bison puts them in the parser
11783 implementation file so that the C compiler and debuggers will
11784 associate errors with your source file, the grammar file. This option
11785 causes them to associate errors with the parser implementation file,
11786 treating it as an independent source file in its own right.
11788 @item -S @var{file}
11789 @itemx --skeleton=@var{file}
11790 Specify the skeleton to use, similar to @code{%skeleton}
11791 (@pxref{Decl Summary}).
11793 @c You probably don't need this option unless you are developing Bison.
11794 @c You should use @option{--language} if you want to specify the skeleton for a
11795 @c different language, because it is clearer and because it will always
11796 @c choose the correct skeleton for non-deterministic or push parsers.
11798 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
11799 file in the Bison installation directory.
11800 If it does, @var{file} is an absolute file name or a file name relative to the
11801 current working directory.
11802 This is similar to how most shells resolve commands.
11805 @itemx --token-table
11806 Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
11810 Act more like the traditional @command{yacc} command. This can cause
11811 different diagnostics to be generated (it implies @option{-Wyacc}), and may
11812 change behavior in other minor ways. Most importantly, imitate Yacc's
11813 output file name conventions, so that the parser implementation file is
11814 called @file{y.tab.c}, and the other outputs are called @file{y.output} and
11815 @file{y.tab.h}. Also, generate @code{#define} statements in addition to an
11816 @code{enum} to associate token codes with token kind names. Thus, the
11817 following shell script can substitute for Yacc, and the Bison distribution
11818 contains such a script for compatibility with POSIX:
11825 The @option{-y}/@option{--yacc} option is intended for use with traditional
11826 Yacc grammars. This option only makes sense for the default C skeleton,
11827 @file{yacc.c}. If your grammar uses Bison extensions Bison cannot be
11828 Yacc-compatible, even if this option is specified.
11832 @subsection Output Files
11834 Options controlling the output.
11836 @c Please, keep this ordered as in 'bison --help'.
11838 @item -H [@var{file}]
11839 @itemx --header=[@var{file}]
11840 Pretend that @code{%header} was specified, i.e., write an extra output file
11841 containing definitions for the token kind names defined in the grammar, as
11842 well as a few other declarations. @xref{Decl Summary}.
11844 @item --defines[=@var{file}]
11845 Historical name for option @option{--header} before Bison 3.8.
11848 This is the same as @option{--header} except @option{-d} does not accept a
11849 @var{file} argument since POSIX Yacc requires that @option{-d} can be
11850 bundled with other short options.
11852 @item -b @var{file-prefix}
11853 @itemx --file-prefix=@var{prefix}
11854 Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
11855 for all Bison output file names. @xref{Decl Summary}.
11857 @item -r @var{things}
11858 @itemx --report=@var{things}
11859 Write an extra output file containing verbose description of the comma
11860 separated list of @var{things} among:
11864 Description of the grammar, conflicts (resolved and unresolved), and
11865 parser's automaton.
11868 Implies @code{state} and augments the description of the automaton with
11869 the full set of items for each state, instead of its core only.
11872 Implies @code{state} and augments the description of the automaton with
11873 each rule's lookahead set.
11876 Implies @code{state}. Explain how conflicts were solved thanks to
11877 precedence and associativity directives.
11879 @item counterexamples
11881 Look for counterexamples for the conflicts. @xref{Counterexamples}.
11882 Counterexamples take time to compute. The option @option{-rcex} should be
11883 used by the developer when working on the grammar; it hardly makes sense to
11887 Enable all the items.
11890 Do not generate the report.
11893 @item --report-file=@var{file}
11894 Specify the @var{file} for the verbose description.
11898 Pretend that @code{%verbose} was specified, i.e., write an extra output
11899 file containing verbose descriptions of the grammar and
11900 parser. @xref{Decl Summary}.
11902 @item -o @var{file}
11903 @itemx --output=@var{file}
11904 Specify the @var{file} for the parser implementation file.
11906 The names of the other output files are constructed from @var{file} as
11907 described under the @option{-v} and @option{-d} options.
11909 @item -g [@var{file}]
11910 @itemx --graph[=@var{file}]
11911 Output a graphical representation of the parser's automaton computed by
11912 Bison, in @uref{http://www.graphviz.org/, Graphviz}
11913 @uref{http://www.graphviz.org/doc/info/lang.html, DOT} format.
11914 @code{@var{file}} is optional. If omitted and the grammar file is
11915 @file{foo.y}, the output file will be @file{foo.gv} if the @code{%required}
11916 version is 3.4 or better, @file{foo.dot} otherwise.
11918 @item -x [@var{file}]
11919 @itemx --xml[=@var{file}]
11920 Output an XML report of the parser's automaton computed by Bison.
11921 @code{@var{file}} is optional.
11922 If omitted and the grammar file is @file{foo.y}, the output file will be
11925 @item -M @var{old}=@var{new}
11926 @itemx --file-prefix-map=@var{old}=@var{new}
11927 Replace prefix @var{old} with @var{new} when writing file paths in output files
11930 @node Option Cross Key
11931 @section Option Cross Key
11933 Here is a list of options, alphabetized by long option, to help you find
11934 the corresponding short option and directive.
11936 @multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
11937 @headitem Long Option @tab Short Option @tab Bison Directive
11938 @include cross-options.texi
11942 @section Yacc Library
11944 The Yacc library contains default implementations of the @code{yyerror} and
11945 @code{main} functions. These default implementations are normally not
11946 useful, but POSIX requires them. To use the Yacc library, link your program
11947 with the @option{-ly} option. Note that Bison's implementation of the Yacc
11948 library is distributed under the terms of the GNU General Public License
11951 If you use the Yacc library's @code{yyerror} function, you should declare
11952 @code{yyerror} as follows:
11955 int yyerror (char const *);
11959 The @code{int} value returned by this @code{yyerror} is ignored.
11961 The implementation of Yacc library's @code{main} function is:
11966 setlocale (LC_ALL, "");
11972 so if you use it, the internationalization support is enabled (e.g., error
11973 messages are translated), and your @code{yyparse} function should have the
11974 following type signature:
11977 int yyparse (void);
11980 @c ================================================= C++ Bison
11982 @node Other Languages
11983 @chapter Parsers Written In Other Languages
11985 In addition to C, Bison can generate parsers in C++, D and Java. This chapter
11986 is devoted to these languages. The reader is expected to understand how
11987 Bison works; read the introductory chapters first if you don't.
11990 * C++ Parsers:: The interface to generate C++ parser classes
11991 * D Parsers:: The interface to generate D parser classes
11992 * Java Parsers:: The interface to generate Java parser classes
11996 @section C++ Parsers
11998 The Bison parser in C++ is an object, an instance of the class
12002 * A Simple C++ Example:: A short introduction to C++ parsers
12003 * C++ Bison Interface:: Asking for C++ parser generation
12004 * C++ Parser Interface:: Instantiating and running the parser
12005 * C++ Semantic Values:: %union vs. C++
12006 * C++ Location Values:: The position and location classes
12007 * C++ Parser Context:: You can supply a @code{report_syntax_error} function.
12008 * C++ Scanner Interface:: Exchanges between yylex and parse
12009 * A Complete C++ Example:: Demonstrating their use
12012 @node A Simple C++ Example
12013 @subsection A Simple C++ Example
12015 This tutorial about C++ parsers is based on a simple, self contained
12016 example. The following sections are the reference manual for Bison with
12017 C++, the last one showing a fully blown example (@pxref{A Complete C++
12020 To look nicer, our example will be in C++14. It is not required: Bison
12021 supports the original C++98 standard.
12023 A Bison file has three parts. In the first part, the prologue, we start by
12024 making sure we run a version of Bison which is recent enough, and that we
12028 @comment file: c++/simple.yy: 1
12030 /* Simple variant-based parser. -*- C++ -*-
12032 Copyright (C) 2018-2021 Free Software Foundation, Inc.
12034 This file is part of Bison, the GNU Compiler Compiler.
12036 This program is free software: you can redistribute it and/or modify
12037 it under the terms of the GNU General Public License as published by
12038 the Free Software Foundation, either version 3 of the License, or
12039 (at your option) any later version.
12041 This program is distributed in the hope that it will be useful,
12042 but WITHOUT ANY WARRANTY; without even the implied warranty of
12043 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12044 GNU General Public License for more details.
12046 You should have received a copy of the GNU General Public License
12047 along with this program. If not, see <http://www.gnu.org/licenses/>. */
12051 @comment file: c++/simple.yy: 1
12057 Let's dive directly into the middle part: the grammar. Our input is a
12058 simple list of strings, that we display once the parsing is done.
12060 @comment file: c++/simple.yy: 2
12065 list @{ std::cout << $1 << '\n'; @}
12069 %nterm <std::vector<std::string>> list;
12072 %empty @{ /* Generates an empty string list */ @}
12073 | list item @{ $$ = $1; $$.push_back ($2); @}
12078 We used a vector of strings as a semantic value! To use genuine C++ objects
12079 as semantic values---not just PODs---we cannot rely on the union that Bison
12080 uses by default to store them, we need @emph{variants} (@pxref{C++
12083 @comment file: c++/simple.yy: 1
12085 %define api.value.type variant
12088 Obviously, the rule for @code{result} needs to print a vector of strings.
12089 In the prologue, we add:
12091 @comment file: c++/simple.yy: 1
12095 // Print a list of strings.
12097 operator<< (std::ostream& o, const std::vector<std::string>& ss)
12101 const char *sep = "";
12103 for (const auto& s: ss)
12115 You may want to move it into the @code{yy} namespace to avoid leaking it in
12116 your default namespace. We recommend that you keep the actions simple, and
12117 move details into auxiliary functions, as we did with @code{operator<<}.
12119 Our list of strings will be built from two types of items: numbers and
12122 @comment file: c++/simple.yy: 2
12124 %nterm <std::string> item;
12125 %token <std::string> TEXT;
12126 %token <int> NUMBER;
12130 | NUMBER @{ $$ = std::to_string ($1); @}
12135 In the case of @code{TEXT}, the implicit default action applies: @w{@code{$$
12140 Our scanner deserves some attention. The traditional interface of
12141 @code{yylex} is not type safe: since the token kind and the token value are
12142 not correlated, you may return a @code{NUMBER} with a string as semantic
12143 value. To avoid this, we use @emph{token constructors} (@pxref{Complete
12144 Symbols}). This directive:
12146 @comment file: c++/simple.yy: 1
12148 %define api.token.constructor
12152 requests that Bison generates the functions @code{make_TEXT} and
12153 @code{make_NUMBER}, but also @code{make_YYEOF}, for the end of input.
12155 Everything is in place for our scanner:
12157 @comment file: c++/simple.yy: 1
12163 // Return the next token.
12164 auto yylex () -> parser::symbol_type
12166 static int count = 0;
12167 switch (int stage = count++)
12171 return parser::make_TEXT ("I have three numbers for you.");
12174 case 1: case 2: case 3:
12175 return parser::make_NUMBER (stage);
12179 return parser::make_TEXT ("And that's all!");
12183 return parser::make_YYEOF ();
12191 In the epilogue, the third part of a Bison grammar file, we leave simple
12192 details: the error reporting function, and the main function.
12194 @comment file: c++/simple.yy: 3
12199 // Report an error to the user.
12200 auto parser::error (const std::string& msg) -> void
12202 std::cerr << msg << '\n';
12216 $ @kbd{bison simple.yy -o simple.cc}
12217 $ @kbd{g++ -std=c++14 simple.cc -o simple}
12220 @{I have three numbers for you., 1, 2, 3, And that's all!@}
12224 @node C++ Bison Interface
12225 @subsection C++ Bison Interface
12226 @c - %skeleton "lalr1.cc"
12228 @c - initial action
12230 The C++ deterministic parser is selected using the skeleton directive,
12231 @samp{%skeleton "lalr1.cc"}. @xref{Decl Summary}.
12233 When run, @command{bison} will create several entities in the @samp{yy}
12235 @findex %define api.namespace
12236 Use the @samp{%define api.namespace} directive to change the namespace name,
12237 see @ref{%define Summary}. The various classes are generated
12238 in the following files:
12241 @item @var{file}.hh
12242 (Assuming the extension of the grammar file was @samp{.yy}.) The
12243 declaration of the C++ parser class and auxiliary types. By default, this
12244 file is not generated (@pxref{Decl Summary}).
12246 @item @var{file}.cc
12247 The implementation of the C++ parser class. The basename and extension of
12248 these two files (@file{@var{file}.hh} and @file{@var{file}.cc}) follow the
12249 same rules as with regular C parsers (@pxref{Invocation}).
12252 Generated when both @code{%header} and @code{%locations} are enabled, this
12253 file contains the definition of the classes @code{position} and
12254 @code{location}, used for location tracking. It is not generated if
12255 @samp{%define api.location.file none} is specified, or if user defined
12256 locations are used. @xref{C++ Location Values}.
12260 Useless legacy files. To get rid of then, use @samp{%require "3.2"} or
12264 All these files are documented using Doxygen; run @command{doxygen} for a
12265 complete and accurate documentation.
12267 @node C++ Parser Interface
12268 @subsection C++ Parser Interface
12270 The output files @file{@var{file}.hh} and @file{@var{file}.cc} declare and
12271 define the parser class in the namespace @code{yy}. The class name defaults
12272 to @code{parser}, but may be changed using @samp{%define api.parser.class
12273 @{@var{name}@}}. The interface of this class is detailed below. It can be
12274 extended using the @code{%parse-param} feature: its semantics is slightly
12275 changed since it describes an additional member of the parser class, and an
12276 additional argument for its constructor.
12279 @defcv {Type} {parser} {token}
12280 A structure that contains (only) the @code{token_kind_type} enumeration,
12281 which defines the tokens. To refer to the token @code{FOO}, use
12282 @code{yy::parser::token::FOO}. The scanner can use @samp{typedef
12283 yy::parser::token token;} to ``import'' the token enumeration (@pxref{Calc++
12287 @defcv {Type} {parser} {token_kind_type}
12288 An enumeration of the token kinds. Its enumerators are forged from the
12289 token names, with a possible token prefix
12290 (@pxref{api-token-prefix,,@code{api.token.prefix}}):
12296 enum token_kind_type
12298 YYEMPTY = -2, // No token.
12299 YYEOF = 0, // "end of file"
12300 YYerror = 256, // error
12301 YYUNDEF = 257, // "invalid token"
12303 MINUS = 259, // "-"
12305 VAR = 271, // "variable"
12310 /// Token kind, as returned by yylex.
12311 typedef token::token_kind_type token_kind_type;
12315 @defcv {Type} {parser} {value_type}
12316 The types for semantic values. @xref{C++ Semantic Values}.
12319 @defcv {Type} {parser} {location_type}
12320 The type of locations, if location tracking is enabled. @xref{C++ Location
12324 @defcv {Type} {parser} {syntax_error}
12325 This class derives from @code{std::runtime_error}. Throw instances of it
12326 from the scanner or from the actions to raise parse errors. This is
12327 equivalent with first invoking @code{error} to report the location and
12328 message of the syntax error, and then to invoke @code{YYERROR} to enter the
12329 error-recovery mode. But contrary to @code{YYERROR} which can only be
12330 invoked from user actions (i.e., written in the action itself), the
12331 exception can be thrown from functions invoked from the user action.
12334 @deftypeop {Constructor} {parser} {} parser ()
12335 @deftypeopx {Constructor} {parser} {} parser (@var{type1} @var{arg1}, ...)
12336 Build a new parser object. There are no arguments, unless
12337 @samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
12340 @deftypeop {Constructor} {syntax_error} {} syntax_error (@code{const location_type&} @var{l}, @code{const std::string&} @var{m})
12341 @deftypeopx {Constructor} {syntax_error} {} syntax_error (@code{const std::string&} @var{m})
12342 Instantiate a syntax-error exception.
12345 @deftypemethod {parser} {int} operator() ()
12346 @deftypemethodx {parser} {int} parse ()
12347 Run the syntactic analysis, and return 0 on success, 1 otherwise. Both
12348 routines are equivalent, @code{operator()} being more C++ish.
12351 The whole function is wrapped in a @code{try}/@code{catch} block, so that
12352 when an exception is thrown, the @code{%destructor}s are called to release
12353 the lookahead symbol, and the symbols pushed on the stack.
12355 Exception related code in the generated parser is protected by CPP guards
12356 (@code{#if}) and disabled when exceptions are not supported (i.e., passing
12357 @option{-fno-exceptions} to the C++ compiler).
12360 @deftypemethod {parser} {std::ostream&} debug_stream ()
12361 @deftypemethodx {parser} {void} set_debug_stream (@code{std::ostream&} @var{o})
12362 Get or set the stream used for tracing the parsing. It defaults to
12366 @deftypemethod {parser} {debug_level_type} debug_level ()
12367 @deftypemethodx {parser} {void} set_debug_level (debug_level_type @var{l})
12368 Get or set the tracing level (an integral). Currently its value is either
12369 0, no trace, or nonzero, full tracing.
12372 @deftypemethod {parser} {void} error (@code{const location_type&} @var{l}, @code{const std::string&} @var{m})
12373 @deftypemethodx {parser} {void} error (@code{const std::string&} @var{m})
12374 The definition for this member function must be supplied by the user: the
12375 parser uses it to report a parser error occurring at @var{l}, described by
12376 @var{m}. If location tracking is not enabled, the second signature is used.
12380 @node C++ Semantic Values
12381 @subsection C++ Semantic Values
12382 @c - No objects in unions
12384 @c - Printer and destructor
12386 Bison supports two different means to handle semantic values in C++. One is
12387 alike the C interface, and relies on unions. As C++ practitioners know,
12388 unions are inconvenient in C++, therefore another approach is provided,
12392 * C++ Unions:: Semantic values cannot be objects
12393 * C++ Variants:: Using objects as semantic values
12397 @subsubsection C++ Unions
12399 The @code{%union} directive works as for C, see @ref{Union Decl}. In
12400 particular it produces a genuine @code{union}, which have a few specific
12404 The type @code{YYSTYPE} is defined but its use is discouraged: rather you
12405 should refer to the parser's encapsulated type
12406 @code{yy::parser::value_type}.
12408 Non POD (Plain Old Data) types cannot be used. C++98 forbids any instance
12409 of classes with constructors in unions: only @emph{pointers} to such objects
12410 are allowed. C++11 relaxed this constraints, but at the cost of safety.
12413 Because objects have to be stored via pointers, memory is not
12414 reclaimed automatically: using the @code{%destructor} directive is the
12415 only means to avoid leaks. @xref{Destructor Decl}.
12418 @subsubsection C++ Variants
12420 Bison provides a @emph{variant} based implementation of semantic values for
12421 C++. This alleviates all the limitations reported in the previous section,
12422 and in particular, object types can be used without pointers.
12424 To enable variant-based semantic values, set the @code{%define} variable
12425 @code{api.value.type} to @code{variant} (@pxref{%define Summary}). Then
12426 @code{%union} is ignored; instead of using the name of the fields of the
12427 @code{%union} to ``type'' the symbols, use genuine types.
12429 For instance, instead of:
12437 %token <ival> NUMBER;
12438 %token <sval> STRING;
12445 %token <int> NUMBER;
12446 %token <std::string> STRING;
12449 @code{STRING} is no longer a pointer, which should fairly simplify the user
12450 actions in the grammar and in the scanner (in particular the memory
12453 Since C++ features destructors, and since it is customary to specialize
12454 @code{operator<<} to support uniform printing of values, variants also
12455 typically simplify Bison printers and destructors.
12457 Variants are stricter than unions. When based on unions, you may play any
12458 dirty game with @code{yylval}, say storing an @code{int}, reading a
12459 @code{char*}, and then storing a @code{double} in it. This is no longer
12460 possible with variants: they must be initialized, then assigned to, and
12461 eventually, destroyed. As a matter of fact, Bison variants forbid the use
12462 of alternative types such as @samp{$<int>2} or @samp{$<std::string>$}, even
12463 in midrule actions. It is mandatory to use typed midrule actions
12464 (@pxref{Typed Midrule Actions}).
12466 @deftypemethod {value_type} {T&} {emplace<T>} ()
12467 @deftypemethodx {value_type} {T&} {emplace<T>} (@code{const T&} @var{t})
12468 Available in C++98/C++03 only. Default construct/copy-construct from
12469 @var{t}. Return a reference to where the actual value may be stored.
12470 Requires that the variant was not initialized yet.
12473 @deftypemethod {value_type} {T&} {emplace<T, U>} (@code{U&&...} @var{u})
12474 Available in C++11 and later only. Build a variant of type @code{T} from
12475 the variadic forwarding references @var{u...}.
12478 @strong{Warning}: We do not use Boost.Variant, for two reasons. First, it
12479 appeared unacceptable to require Boost on the user's machine (i.e., the
12480 machine on which the generated parser will be compiled, not the machine on
12481 which @command{bison} was run). Second, for each possible semantic value,
12482 Boost.Variant not only stores the value, but also a tag specifying its
12483 type. But the parser already ``knows'' the type of the semantic value, so
12484 that would be duplicating the information.
12486 We do not use C++17's @code{std::variant} either: we want to support all the
12487 C++ standards, and of course @code{std::variant} also stores a tag to record
12490 Therefore we developed light-weight variants whose type tag is external (so
12491 they are really like @code{unions} for C++ actually). There is a number of
12492 limitations in (the current implementation of) variants:
12495 Alignment must be enforced: values should be aligned in memory according to
12496 the most demanding type. Computing the smallest alignment possible requires
12497 meta-programming techniques that are not currently implemented in Bison, and
12498 therefore, since, as far as we know, @code{double} is the most demanding
12499 type on all platforms, alignments are enforced for @code{double} whatever
12500 types are actually used. This may waste space in some cases.
12503 There might be portability issues we are not aware of.
12506 As far as we know, these limitations @emph{can} be alleviated. All it takes
12507 is some time and/or some talented C++ hacker willing to contribute to Bison.
12509 @node C++ Location Values
12510 @subsection C++ Location Values
12512 When the directive @code{%locations} is used, the C++ parser supports
12513 location tracking, see @ref{Tracking Locations}.
12515 By default, two auxiliary classes define a @code{position}, a single point
12516 in a file, and a @code{location}, a range composed of a pair of
12517 @code{position}s (possibly spanning several files). If the @code{%define}
12518 variable @code{api.location.type} is defined, then these classes will not be
12519 generated, and the user defined type will be used.
12522 * C++ position:: One point in the source file
12523 * C++ location:: Two points in the source file
12524 * Exposing the Location Classes:: Using the Bison location class in your
12526 * User Defined Location Type:: Required interface for locations
12530 @subsubsection C++ @code{position}
12532 @defcv {Type} {position} {filename_type}
12533 The base type for file names. Defaults to @code{const std::string}.
12534 @xref{api-filename-type,,@code{api.filename.type}}, to change its definition.
12537 @defcv {Type} {position} {counter_type}
12538 The type used to store line and column numbers. Defined as @code{int}.
12541 @deftypeop {Constructor} {position} {} position (@code{filename_type*} @var{file} = nullptr, @code{counter_type} @var{line} = 1, @code{counter_type} @var{col} = 1)
12542 Create a @code{position} denoting a given point. Note that @code{file} is
12543 not reclaimed when the @code{position} is destroyed: memory managed must be
12547 @deftypemethod {position} {void} initialize (@code{filename_type*} @var{file} = nullptr, @code{counter_type} @var{line} = 1, @code{counter_type} @var{col} = 1)
12548 Reset the position to the given values.
12551 @deftypeivar {position} {filename_type*} file
12552 The name of the file. It will always be handled as a pointer, the parser
12553 will never duplicate nor deallocate it.
12556 @deftypeivar {position} {counter_type} line
12557 The line, starting at 1.
12560 @deftypemethod {position} {void} lines (@code{counter_type} @var{height} = 1)
12561 If @var{height} is not null, advance by @var{height} lines, resetting the
12562 column number. The resulting line number cannot be less than 1.
12565 @deftypeivar {position} {counter_type} column
12566 The column, starting at 1.
12569 @deftypemethod {position} {void} columns (@code{counter_type} @var{width} = 1)
12570 Advance by @var{width} columns, without changing the line number. The
12571 resulting column number cannot be less than 1.
12574 @deftypemethod {position} {position&} operator+= (@code{counter_type} @var{width})
12575 @deftypemethodx {position} {position} operator+ (@code{counter_type} @var{width})
12576 @deftypemethodx {position} {position&} operator-= (@code{counter_type} @var{width})
12577 @deftypemethodx {position} {position} operator- (@code{counter_type} @var{width})
12578 Various forms of syntactic sugar for @code{columns}.
12581 @deftypemethod {position} {bool} operator== (@code{const position&} @var{that})
12582 @deftypemethodx {position} {bool} operator!= (@code{const position&} @var{that})
12583 Whether @code{*this} and @code{that} denote equal/different positions.
12586 @deftypefun {std::ostream&} operator<< (@code{std::ostream&} @var{o}, @code{const position&} @var{p})
12587 Report @var{p} on @var{o} like this:
12588 @samp{@var{file}:@var{line}.@var{column}}, or
12589 @samp{@var{line}.@var{column}} if @var{file} is null.
12593 @subsubsection C++ @code{location}
12595 @deftypeop {Constructor} {location} {} location (@code{const position&} @var{begin}, @code{const position&} @var{end})
12596 Create a @code{Location} from the endpoints of the range.
12599 @deftypeop {Constructor} {location} {} location (@code{const position&} @var{pos} = position())
12600 @deftypeopx {Constructor} {location} {} location (@code{filename_type*} @var{file}, @code{counter_type} @var{line}, @code{counter_type} @var{col})
12601 Create a @code{Location} denoting an empty range located at a given point.
12604 @deftypemethod {location} {void} initialize (@code{filename_type*} @var{file} = nullptr, @code{counter_type} @var{line} = 1, @code{counter_type} @var{col} = 1)
12605 Reset the location to an empty range at the given values.
12608 @deftypeivar {location} {position} begin
12609 @deftypeivarx {location} {position} end
12610 The first, inclusive, position of the range, and the first beyond.
12613 @deftypemethod {location} {void} columns (@code{counter_type} @var{width} = 1)
12614 @deftypemethodx {location} {void} lines (@code{counter_type} @var{height} = 1)
12615 Forwarded to the @code{end} position.
12618 @deftypemethod {location} {location} operator+ (@code{counter_type} @var{width})
12619 @deftypemethodx {location} {location} operator+= (@code{counter_type} @var{width})
12620 @deftypemethodx {location} {location} operator- (@code{counter_type} @var{width})
12621 @deftypemethodx {location} {location} operator-= (@code{counter_type} @var{width})
12622 Various forms of syntactic sugar for @code{columns}.
12625 @deftypemethod {location} {location} operator+ (@code{const location&} @var{end})
12626 @deftypemethodx {location} {location} operator+= (@code{const location&} @var{end})
12627 Join two locations: starts at the position of the first one, and ends at the
12628 position of the second.
12631 @deftypemethod {location} {void} step ()
12632 Move @code{begin} onto @code{end}.
12635 @deftypemethod {location} {bool} operator== (@code{const location&} @var{that})
12636 @deftypemethodx {location} {bool} operator!= (@code{const location&} @var{that})
12637 Whether @code{*this} and @code{that} denote equal/different ranges of
12641 @deftypefun {std::ostream&} operator<< (@code{std::ostream&} @var{o}, @code{const location&} @var{p})
12642 Report @var{p} on @var{o}, taking care of special cases such as: no
12643 @code{filename} defined, or equal filename/line or column.
12646 @node Exposing the Location Classes
12647 @subsubsection Exposing the Location Classes
12649 When both @code{%header} and @code{%locations} are enabled, Bison generates
12650 an additional file: @file{location.hh}. If you don't use locations outside
12651 of the parser, you may avoid its creation with @samp{%define
12652 api.location.file none}.
12654 However this file is useful if, for instance, your parser builds an abstract
12655 syntax tree decorated with locations: you may use Bison's @code{location}
12656 type independently of Bison's parser. You may name the file differently,
12657 e.g., @samp{%define api.location.file "include/ast/location.hh"}: this name
12658 can have directory components, or even be absolute. The way the location
12659 file is included is controlled by @code{api.location.include}.
12661 This way it is possible to have several parsers share the same location
12664 For instance, in @file{src/foo/parser.yy}, generate the
12665 @file{include/ast/loc.hh} file:
12668 // src/foo/parser.yy
12670 %define api.namespace @{foo@}
12671 %define api.location.file "include/ast/loc.hh"
12672 %define api.location.include @{<ast/loc.hh>@}
12676 and use it in @file{src/bar/parser.yy}:
12679 // src/bar/parser.yy
12681 %define api.namespace @{bar@}
12682 %code requires @{#include <ast/loc.hh>@}
12683 %define api.location.type @{bar::location@}
12686 Absolute file names are supported; it is safe in your @file{Makefile} to
12688 @option{-Dapi.location.file='"$(top_srcdir)/include/ast/loc.hh"'} to
12689 @command{bison} for @file{src/foo/parser.yy}. The generated file will not
12690 have references to this absolute path, thanks to @samp{%define
12691 api.location.include @{<ast/loc.hh>@}}. Adding @samp{-I
12692 $(top_srcdir)/include} to your @code{CPPFLAGS} will suffice for the compiler
12693 to find @file{ast/loc.hh}.
12695 @node User Defined Location Type
12696 @subsubsection User Defined Location Type
12697 @findex %define api.location.type
12699 Instead of using the built-in types you may use the @code{%define} variable
12700 @code{api.location.type} to specify your own type:
12703 %define api.location.type @{@var{LocationType}@}
12706 The requirements over your @var{LocationType} are:
12709 it must be copyable;
12712 in order to compute the (default) value of @code{@@$} in a reduction, the
12713 parser basically runs
12715 @@$.begin = @@1.begin;
12716 @@$.end = @@@var{N}.end; // The location of last right-hand side symbol.
12719 so there must be copyable @code{begin} and @code{end} members;
12722 alternatively you may redefine the computation of the default location, in
12723 which case these members are not required (@pxref{Location Default Action});
12726 if traces are enabled, then there must exist an @samp{std::ostream&
12727 operator<< (std::ostream& o, const @var{LocationType}& s)} function.
12732 In programs with several C++ parsers, you may also use the @code{%define}
12733 variable @code{api.location.type} to share a common set of built-in
12734 definitions for @code{position} and @code{location}. For instance, one
12735 parser @file{master/parser.yy} might use:
12740 %define api.namespace @{master::@}
12744 to generate the @file{master/position.hh} and @file{master/location.hh}
12745 files, reused by other parsers as follows:
12748 %define api.location.type @{master::location@}
12749 %code requires @{ #include <master/location.hh> @}
12753 @node C++ Parser Context
12754 @subsection C++ Parser Context
12756 When @samp{%define parse.error custom} is used (@pxref{Syntax Error
12757 Reporting Function}), the user must define the following function.
12759 @deftypemethod {parser} {void} report_syntax_error (@code{const context_type&}@var{ctx}) @code{const}
12760 Report a syntax error to the user. Whether it uses @code{yyerror} is up to
12764 Use the following types and functions to build the error message.
12766 @defcv {Type} {parser} {context}
12767 A type that captures the circumstances of the syntax error.
12770 @defcv {Type} {parser} {symbol_kind_type}
12771 An enum of all the grammar symbols, tokens and nonterminals. Its
12772 enumerators are forged from the symbol names:
12777 enum symbol_kind_type
12779 S_YYEMPTY = -2, // No symbol.
12780 S_YYEOF = 0, // "end of file"
12781 S_YYERROR = 1, // error
12782 S_YYUNDEF = 2, // "invalid token"
12784 S_MINUS = 4, // "-"
12786 S_VAR = 14, // "variable"
12788 S_YYACCEPT = 16, // $accept
12790 S_input = 18 // input
12793 typedef symbol_kind::symbol_kind_t symbol_kind_type;
12797 @deftypemethod {context} {const symbol_type&} lookahead () @code{const}
12798 The ``unexpected'' token: the lookahead that caused the syntax error.
12801 @deftypemethod {context} {symbol_kind_type} token () @code{const}
12802 The symbol kind of the lookahead token that caused the syntax error. Returns
12803 @code{symbol_kind::S_YYEMPTY} if there is no lookahead.
12806 @deftypemethod {context} {const location&} location () @code{const}
12807 The location of the syntax error (that of the lookahead).
12810 @deftypemethod {context} int expected_tokens (@code{symbol_kind_type} @var{argv}@code{[]}, @code{int} @var{argc}) @code{const}
12811 Fill @var{argv} with the expected tokens, which never includes
12812 @code{symbol_kind::S_YYEMPTY}, @code{symbol_kind::S_YYERROR}, or
12813 @code{symbol_kind::S_YYUNDEF}.
12815 Never put more than @var{argc} elements into @var{argv}, and on success
12816 return the number of tokens stored in @var{argv}. If there are more
12817 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
12818 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
12819 to @code{symbol_kind::S_YYEMPTY}.
12821 If @var{argv} is null, return the size needed to store all the possible
12822 values, which is always less than @code{YYNTOKENS}.
12825 @deftypemethod {parser} {const char *} symbol_name (@code{symbol_kind_t} @var{symbol}) @code{const}
12826 The name of the symbol whose kind is @var{symbol}, possibly translated.
12828 Returns a @code{std::string} when @code{parse.error} is @code{verbose}.
12831 A custom syntax error function looks as follows. This implementation is
12832 inappropriate for internationalization, see the @file{c/bistromathic}
12833 example for a better alternative.
12837 yy::parser::report_syntax_error (const context& ctx)
12840 std::cerr << ctx.location () << ": syntax error";
12841 // Report the tokens expected at this point.
12843 enum @{ TOKENMAX = 5 @};
12844 symbol_kind_type expected[TOKENMAX];
12845 int n = ctx.expected_tokens (ctx, expected, TOKENMAX);
12846 for (int i = 0; i < n; ++i)
12847 std::cerr << i == 0 ? ": expected " : " or "
12848 << symbol_name (expected[i]);
12850 // Report the unexpected token.
12852 symbol_kind_type lookahead = ctx.token ();
12853 if (lookahead != symbol_kind::S_YYEMPTY)
12854 std::cerr << " before " << symbol_name (lookahead));
12860 You still must provide a @code{yyerror} function, used for instance to
12861 report memory exhaustion.
12864 @node C++ Scanner Interface
12865 @subsection C++ Scanner Interface
12866 @c - prefix for yylex.
12867 @c - Pure interface to yylex
12870 The parser invokes the scanner by calling @code{yylex}. Contrary to C
12871 parsers, C++ parsers are always pure: there is no point in using the
12872 @samp{%define api.pure} directive. The actual interface with @code{yylex}
12873 depends whether you use unions, or variants.
12876 * Split Symbols:: Passing symbols as two/three components
12877 * Complete Symbols:: Making symbols a whole
12880 @node Split Symbols
12881 @subsubsection Split Symbols
12883 The generated parser expects @code{yylex} to have the following prototype.
12885 @deftypefun {int} yylex (@code{value_type*} @var{yylval}, @code{location_type*} @var{yylloc}, @var{type1} @var{arg1}, @dots{})
12886 @deftypefunx {int} yylex (@code{value_type*} @var{yylval}, @var{type1} @var{arg1}, @dots{})
12887 Return the next token. Its kind is the return value, its semantic value and
12888 location (if enabled) being @var{yylval} and @var{yylloc}. Invocations of
12889 @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
12892 Note that when using variants, the interface for @code{yylex} is the same,
12893 but @code{yylval} is handled differently.
12895 Regular union-based code in Lex scanner typically looks like:
12899 yylval->ival = text_to_int (yytext);
12900 return yy::parser::token::INTEGER;
12903 yylval->sval = new std::string (yytext);
12904 return yy::parser::token::IDENTIFIER;
12908 Using variants, @code{yylval} is already constructed, but it is not
12909 initialized. So the code would look like:
12913 yylval->emplace<int> () = text_to_int (yytext);
12914 return yy::parser::token::INTEGER;
12917 yylval->emplace<std::string> () = yytext;
12918 return yy::parser::token::IDENTIFIER;
12927 yylval->emplace (text_to_int (yytext));
12928 return yy::parser::token::INTEGER;
12931 yylval->emplace (yytext);
12932 return yy::parser::token::IDENTIFIER;
12937 @node Complete Symbols
12938 @subsubsection Complete Symbols
12940 With both @code{%define api.value.type variant} and @code{%define
12941 api.token.constructor}, the parser defines the type @code{symbol_type}, and
12942 expects @code{yylex} to have the following prototype.
12944 @deftypefun {parser::symbol_type} yylex ()
12945 @deftypefunx {parser::symbol_type} yylex (@var{type1} @var{arg1}, @dots{})
12946 Return a @emph{complete} symbol, aggregating its type (i.e., the traditional
12947 value returned by @code{yylex}), its semantic value, and possibly its
12948 location. Invocations of @samp{%lex-param @{@var{type1} @var{arg1}@}} yield
12949 additional arguments.
12952 @defcv {Type} {parser} {symbol_type}
12953 A ``complete symbol'', that binds together its kind, value and (when
12954 applicable) location.
12957 @deftypemethod {symbol_type} {symbol_kind_type} kind () @code{const}
12958 The kind of this symbol.
12961 @deftypemethod {symbol_type} {const char *} name () @code{const}
12962 The name of the kind of this symbol.
12964 Returns a @code{std::string} when @code{parse.error} is @code{verbose}.
12969 For each token kind, Bison generates named constructors as follows.
12971 @deftypeop {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token}, @code{const @var{value_type}&} @var{value}, @code{const location_type&} @var{location})
12972 @deftypeopx {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token}, @code{const location_type&} @var{location})
12973 @deftypeopx {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token}, @code{const @var{value_type}&} @var{value})
12974 @deftypeopx {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token})
12975 Build a complete terminal symbol for the token kind @var{token} (including
12976 the @code{api.token.prefix}), whose semantic value, if it has one, is
12977 @var{value} of adequate @var{value_type}. Pass the @var{location} iff
12978 location tracking is enabled.
12980 Consistency between @var{token} and @var{value_type} is checked via an
12984 For instance, given the following declarations:
12987 %define api.token.prefix @{TOK_@}
12988 %token <std::string> IDENTIFIER;
12989 %token <int> INTEGER;
12994 you may use these constructors:
12997 symbol_type (int token, const std::string&, const location_type&);
12998 symbol_type (int token, const int&, const location_type&);
12999 symbol_type (int token, const location_type&);
13002 Correct matching between token kinds and value types is checked via
13003 @code{assert}; for instance, @samp{symbol_type (ID, 42)} would abort. Named
13004 constructors are preferable (see below), as they offer better type safety
13005 (for instance @samp{make_ID (42)} would not even compile), but symbol_type
13006 constructors may help when token kinds are discovered at run-time, e.g.,
13011 if (auto i = lookup_keyword (yytext))
13012 return yy::parser::symbol_type (i, loc);
13014 return yy::parser::make_ID (yytext, loc);
13021 Note that it is possible to generate and compile type incorrect code
13022 (e.g. @samp{symbol_type (':', yytext, loc)}). It will fail at run time,
13023 provided the assertions are enabled (i.e., @option{-DNDEBUG} was not passed
13024 to the compiler). Bison supports an alternative that guarantees that type
13025 incorrect code will not even compile. Indeed, it generates @emph{named
13026 constructors} as follows.
13028 @deftypemethod {parser} {symbol_type} {make_@var{token}} (@code{const @var{value_type}&} @var{value}, @code{const location_type&} @var{location})
13029 @deftypemethodx {parser} {symbol_type} {make_@var{token}} (@code{const location_type&} @var{location})
13030 @deftypemethodx {parser} {symbol_type} {make_@var{token}} (@code{const @var{value_type}&} @var{value})
13031 @deftypemethodx {parser} {symbol_type} {make_@var{token}} ()
13032 Build a complete terminal symbol for the token kind @var{token} (not
13033 including the @code{api.token.prefix}), whose semantic value, if it has one,
13034 is @var{value} of adequate @var{value_type}. Pass the @var{location} iff
13035 location tracking is enabled.
13038 For instance, given the following declarations:
13041 %define api.token.prefix @{TOK_@}
13042 %token <std::string> IDENTIFIER;
13043 %token <int> INTEGER;
13052 symbol_type make_IDENTIFIER (const std::string&, const location_type&);
13053 symbol_type make_INTEGER (const int&, const location_type&);
13054 symbol_type make_COLON (const location_type&);
13055 symbol_type make_EOF (const location_type&);
13059 which should be used in a scanner as follows.
13062 [a-z]+ return yy::parser::make_IDENTIFIER (yytext, loc);
13063 [0-9]+ return yy::parser::make_INTEGER (text_to_int (yytext), loc);
13064 ":" return yy::parser::make_COLON (loc);
13065 <<EOF>> return yy::parser::make_EOF (loc);
13068 Tokens that do not have an identifier are not accessible: you cannot simply
13069 use characters such as @code{':'}, they must be declared with @code{%token},
13070 including the end-of-file token.
13073 @node A Complete C++ Example
13074 @subsection A Complete C++ Example
13076 This section demonstrates the use of a C++ parser with a simple but complete
13077 example. This example should be available on your system, ready to compile,
13078 in the directory @dfn{.../share/doc/bison/examples/calc++}. It focuses on
13079 the use of Bison, therefore the design of the various C++ classes is very
13080 naive: no accessors, no encapsulation of members etc. We will use a Lex
13081 scanner, and more precisely, a Flex scanner, to demonstrate the various
13082 interactions. A hand-written scanner is actually easier to interface with.
13085 * Calc++ --- C++ Calculator:: The specifications
13086 * Calc++ Parsing Driver:: An active parsing context
13087 * Calc++ Parser:: A parser class
13088 * Calc++ Scanner:: A pure C++ Flex scanner
13089 * Calc++ Top Level:: Conducting the band
13092 @node Calc++ --- C++ Calculator
13093 @subsubsection Calc++ --- C++ Calculator
13095 Of course the grammar is dedicated to arithmetic, a single expression,
13096 possibly preceded by variable assignments. An environment containing
13097 possibly predefined variables such as @code{one} and @code{two}, is
13098 exchanged with the parser. An example of valid input follows.
13102 seven := one + two * three
13106 @node Calc++ Parsing Driver
13107 @subsubsection Calc++ Parsing Driver
13109 @c - A place to store error messages
13110 @c - A place for the result
13112 To support a pure interface with the parser (and the scanner) the technique
13113 of the ``parsing context'' is convenient: a structure containing all the
13114 data to exchange. Since, in addition to simply launch the parsing, there
13115 are several auxiliary tasks to execute (open the file for scanning,
13116 instantiate the parser etc.), we recommend transforming the simple parsing
13117 context structure into a fully blown @dfn{parsing driver} class.
13119 The declaration of this driver class, in @file{driver.hh}, is as follows.
13120 The first part includes the CPP guard and imports the required standard
13121 library components, and the declaration of the parser class.
13124 @comment file: c++/calc++/driver.hh
13126 /* Driver for calc++. -*- C++ -*-
13128 Copyright (C) 2005-2015, 2018-2021 Free Software Foundation, Inc.
13130 This file is part of Bison, the GNU Compiler Compiler.
13132 This program is free software: you can redistribute it and/or modify
13133 it under the terms of the GNU General Public License as published by
13134 the Free Software Foundation, either version 3 of the License, or
13135 (at your option) any later version.
13137 This program is distributed in the hope that it will be useful,
13138 but WITHOUT ANY WARRANTY; without even the implied warranty of
13139 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13140 GNU General Public License for more details.
13142 You should have received a copy of the GNU General Public License
13143 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13147 @comment file: c++/calc++/driver.hh
13153 # include "parser.hh"
13158 Then comes the declaration of the scanning function. Flex expects the
13159 signature of @code{yylex} to be defined in the macro @code{YY_DECL}, and the
13160 C++ parser expects it to be declared. We can factor both as follows.
13162 @comment file: c++/calc++/driver.hh
13164 // Give Flex the prototype of yylex we want ...
13166 yy::parser::symbol_type yylex (driver& drv)
13167 // ... and declare it for the parser's sake.
13172 The @code{driver} class is then declared with its most obvious members.
13174 @comment file: c++/calc++/driver.hh
13176 // Conducting the whole scanning and parsing of Calc++.
13182 std::map<std::string, int> variables;
13188 The main routine is of course calling the parser.
13190 @comment file: c++/calc++/driver.hh
13192 // Run the parser on file F. Return 0 on success.
13193 int parse (const std::string& f);
13194 // The name of the file being parsed.
13196 // Whether to generate parser debug traces.
13197 bool trace_parsing;
13201 To encapsulate the coordination with the Flex scanner, it is useful to have
13202 member functions to open and close the scanning phase.
13204 @comment file: c++/calc++/driver.hh
13206 // Handling the scanner.
13207 void scan_begin ();
13209 // Whether to generate scanner debug traces.
13210 bool trace_scanning;
13211 // The token's location used by the scanner.
13212 yy::location location;
13214 #endif // ! DRIVER_HH
13217 The implementation of the driver (@file{driver.cc}) is straightforward.
13220 @comment file: c++/calc++/driver.cc
13222 /* Driver for calc++. -*- C++ -*-
13224 Copyright (C) 2005-2015, 2018-2021 Free Software Foundation, Inc.
13226 This file is part of Bison, the GNU Compiler Compiler.
13228 This program is free software: you can redistribute it and/or modify
13229 it under the terms of the GNU General Public License as published by
13230 the Free Software Foundation, either version 3 of the License, or
13231 (at your option) any later version.
13233 This program is distributed in the hope that it will be useful,
13234 but WITHOUT ANY WARRANTY; without even the implied warranty of
13235 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13236 GNU General Public License for more details.
13238 You should have received a copy of the GNU General Public License
13239 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13243 @comment file: c++/calc++/driver.cc
13245 #include "driver.hh"
13246 #include "parser.hh"
13250 : trace_parsing (false), trace_scanning (false)
13252 variables["one"] = 1;
13253 variables["two"] = 2;
13258 The @code{parse} member function deserves some attention.
13260 @comment file: c++/calc++/driver.cc
13264 driver::parse (const std::string &f)
13267 location.initialize (&file);
13269 yy::parser parse (*this);
13270 parse.set_debug_level (trace_parsing);
13271 int res = parse ();
13278 @node Calc++ Parser
13279 @subsubsection Calc++ Parser
13281 The grammar file @file{parser.yy} starts by asking for the C++ deterministic
13282 parser skeleton, the creation of the parser header file. Because the C++
13283 skeleton changed several times, it is safer to require the version you
13284 designed the grammar for.
13287 @comment file: c++/calc++/parser.yy
13289 /* Parser for calc++. -*- C++ -*-
13291 Copyright (C) 2005-2015, 2018-2021 Free Software Foundation, Inc.
13293 This file is part of Bison, the GNU Compiler Compiler.
13295 This program is free software: you can redistribute it and/or modify
13296 it under the terms of the GNU General Public License as published by
13297 the Free Software Foundation, either version 3 of the License, or
13298 (at your option) any later version.
13300 This program is distributed in the hope that it will be useful,
13301 but WITHOUT ANY WARRANTY; without even the implied warranty of
13302 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13303 GNU General Public License for more details.
13305 You should have received a copy of the GNU General Public License
13306 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13310 @comment file: c++/calc++/parser.yy
13312 %skeleton "lalr1.cc" // -*- C++ -*-
13313 %require "@value{VERSION}"
13318 @findex %define api.token.raw
13319 Because our scanner returns only genuine tokens and never simple characters
13320 (i.e., it returns @samp{PLUS}, not @samp{'+'}), we can avoid conversions.
13322 @comment file: c++/calc++/parser.yy
13324 %define api.token.raw
13328 @findex %define api.token.constructor
13329 @findex %define api.value.type variant
13330 This example uses genuine C++ objects as semantic values, therefore, we
13331 require the variant-based storage of semantic values. To make sure we
13332 properly use it, we enable assertions. To fully benefit from type-safety
13333 and more natural definition of ``symbol'', we enable
13334 @code{api.token.constructor}.
13336 @comment file: c++/calc++/parser.yy
13338 %define api.token.constructor
13339 %define api.value.type variant
13340 %define parse.assert
13344 @findex %code requires
13345 Then come the declarations/inclusions needed by the semantic values.
13346 Because the parser uses the parsing driver and reciprocally, both would like
13347 to include the header of the other, which is, of course, insane. This
13348 mutual dependency will be broken using forward declarations. Because the
13349 driver's header needs detailed knowledge about the parser class (in
13350 particular its inner types), it is the parser's header which will use a
13351 forward declaration of the driver. @xref{%code Summary}.
13353 @comment file: c++/calc++/parser.yy
13364 The driver is passed by reference to the parser and to the scanner.
13365 This provides a simple but effective pure interface, not relying on
13368 @comment file: c++/calc++/parser.yy
13370 // The parsing context.
13371 %param @{ driver& drv @}
13375 Then we request location tracking.
13377 @comment file: c++/calc++/parser.yy
13383 Use the following two directives to enable parser tracing and detailed error
13384 messages. However, detailed error messages can contain incorrect
13385 information if lookahead correction is not enabled (@pxref{LAC}).
13387 @comment file: c++/calc++/parser.yy
13389 %define parse.trace
13390 %define parse.error detailed
13391 %define parse.lac full
13396 The code between @samp{%code @{} and @samp{@}} is output in the @file{*.cc}
13397 file; it needs detailed knowledge about the driver.
13399 @comment file: c++/calc++/parser.yy
13403 # include "driver.hh"
13410 User friendly names are provided for each symbol. To avoid name clashes in
13411 the generated files (@pxref{Calc++ Scanner}), prefix tokens with @code{TOK_}
13412 (@pxref{%define Summary}).
13414 @comment file: c++/calc++/parser.yy
13416 %define api.token.prefix @{TOK_@}
13429 Since we use variant-based semantic values, @code{%union} is not used, and
13430 @code{%token}, @code{%nterm} and @code{%type} expect genuine types, not type
13433 @comment file: c++/calc++/parser.yy
13435 %token <std::string> IDENTIFIER "identifier"
13436 %token <int> NUMBER "number"
13441 No @code{%destructor} is needed to enable memory deallocation during error
13442 recovery; the memory, for strings for instance, will be reclaimed by the
13443 regular destructors. All the values are printed using their
13444 @code{operator<<} (@pxref{Printer Decl}).
13446 @comment file: c++/calc++/parser.yy
13448 %printer @{ yyo << $$; @} <*>;
13452 The grammar itself is straightforward (@pxref{Location Tracking Calc}).
13454 @comment file: c++/calc++/parser.yy
13458 unit: assignments exp @{ drv.result = $2; @};
13462 | assignments assignment @{@};
13465 "identifier" ":=" exp @{ drv.variables[$1] = $3; @};
13471 | "identifier" @{ $$ = drv.variables[$1]; @}
13472 | exp "+" exp @{ $$ = $1 + $3; @}
13473 | exp "-" exp @{ $$ = $1 - $3; @}
13474 | exp "*" exp @{ $$ = $1 * $3; @}
13475 | exp "/" exp @{ $$ = $1 / $3; @}
13476 | "(" exp ")" @{ $$ = $2; @}
13481 Finally the @code{error} member function reports the errors.
13483 @comment file: c++/calc++/parser.yy
13486 yy::parser::error (const location_type& l, const std::string& m)
13488 std::cerr << l << ": " << m << '\n';
13492 @node Calc++ Scanner
13493 @subsubsection Calc++ Scanner
13495 In addition to standard headers, the Flex scanner includes the driver's,
13496 then the parser's to get the set of defined tokens.
13499 @comment file: c++/calc++/scanner.ll
13501 /* Scanner for calc++. -*- C++ -*-
13503 Copyright (C) 2005-2015, 2018-2021 Free Software Foundation, Inc.
13505 This file is part of Bison, the GNU Compiler Compiler.
13507 This program is free software: you can redistribute it and/or modify
13508 it under the terms of the GNU General Public License as published by
13509 the Free Software Foundation, either version 3 of the License, or
13510 (at your option) any later version.
13512 This program is distributed in the hope that it will be useful,
13513 but WITHOUT ANY WARRANTY; without even the implied warranty of
13514 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13515 GNU General Public License for more details.
13517 You should have received a copy of the GNU General Public License
13518 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13522 @comment file: c++/calc++/scanner.ll
13524 %@{ /* -*- C++ -*- */
13526 # include <climits>
13527 # include <cstdlib>
13528 # include <cstring> // strerror
13530 # include "driver.hh"
13531 # include "parser.hh"
13536 @comment file: c++/calc++/scanner.ll
13539 #if defined __clang__
13540 # define CLANG_VERSION (__clang_major__ * 100 + __clang_minor__)
13543 // Clang and ICC like to pretend they are GCC.
13544 #if defined __GNUC__ && !defined __clang__ && !defined __ICC
13545 # define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
13548 // Pacify warnings in yy_init_buffer (observed with Flex 2.6.4)
13549 // and GCC 6.4.0, 7.3.0 with -O3.
13550 #if defined GCC_VERSION && 600 <= GCC_VERSION
13551 # pragma GCC diagnostic ignored "-Wnull-dereference"
13554 // This example uses Flex's C back end, yet compiles it as C++.
13555 // So expect warnings about C style casts and NULL.
13556 #if defined CLANG_VERSION && 500 <= CLANG_VERSION
13557 # pragma clang diagnostic ignored "-Wold-style-cast"
13558 # pragma clang diagnostic ignored "-Wzero-as-null-pointer-constant"
13559 #elif defined GCC_VERSION && 407 <= GCC_VERSION
13560 # pragma GCC diagnostic ignored "-Wold-style-cast"
13561 # pragma GCC diagnostic ignored "-Wzero-as-null-pointer-constant"
13564 #define FLEX_VERSION (YY_FLEX_MAJOR_VERSION * 100 + YY_FLEX_MINOR_VERSION)
13566 // Old versions of Flex (2.5.35) generate an incomplete documentation comment.
13568 // In file included from src/scan-code-c.c:3:
13569 // src/scan-code.c:2198:21: error: empty paragraph passed to '@param' command
13570 // [-Werror,-Wdocumentation]
13571 // * @param line_number
13572 // ~~~~~~~~~~~~~~~~~^
13573 // 1 error generated.
13574 #if FLEX_VERSION < 206 && defined CLANG_VERSION
13575 # pragma clang diagnostic ignored "-Wdocumentation"
13578 // Old versions of Flex (2.5.35) use 'register'. Warnings introduced in
13579 // GCC 7 and Clang 6.
13580 #if FLEX_VERSION < 206
13581 # if defined CLANG_VERSION && 600 <= CLANG_VERSION
13582 # pragma clang diagnostic ignored "-Wdeprecated-register"
13583 # elif defined GCC_VERSION && 700 <= GCC_VERSION
13584 # pragma GCC diagnostic ignored "-Wregister"
13588 #if FLEX_VERSION < 206
13589 # if defined CLANG_VERSION
13590 # pragma clang diagnostic ignored "-Wconversion"
13591 # pragma clang diagnostic ignored "-Wdocumentation"
13592 # pragma clang diagnostic ignored "-Wshorten-64-to-32"
13593 # pragma clang diagnostic ignored "-Wsign-conversion"
13594 # elif defined GCC_VERSION
13595 # pragma GCC diagnostic ignored "-Wconversion"
13596 # pragma GCC diagnostic ignored "-Wsign-conversion"
13600 // Flex 2.6.4, GCC 9
13601 // warning: useless cast to type 'int' [-Wuseless-cast]
13602 // 1361 | YY_CURRENT_BUFFER_LVALUE->yy_buf_size = (int) (new_size - 2);
13604 #if defined GCC_VERSION && 900 <= GCC_VERSION
13605 # pragma GCC diagnostic ignored "-Wuseless-cast"
13612 Since our calculator has no @code{#include}-like feature, we don't need
13613 @code{yywrap}. We don't need the @code{unput} and @code{input} functions
13614 either, and we parse an actual file, this is not an interactive session with
13615 the user. Finally, we enable scanner tracing.
13617 @comment file: c++/calc++/scanner.ll
13619 %option noyywrap nounput noinput batch debug
13623 The following function will be handy to convert a string denoting a number
13624 into a @code{NUMBER} token.
13626 @comment file: c++/calc++/scanner.ll
13629 // A number symbol corresponding to the value in S.
13630 yy::parser::symbol_type
13631 make_NUMBER (const std::string &s, const yy::parser::location_type& loc);
13636 Abbreviations allow for more readable rules.
13638 @comment file: c++/calc++/scanner.ll
13640 id [a-zA-Z][a-zA-Z_0-9]*
13646 The following paragraph suffices to track locations accurately. Each time
13647 @code{yylex} is invoked, the begin position is moved onto the end position.
13648 Then when a pattern is matched, its width is added to the end column. When
13649 matching ends of lines, the end cursor is adjusted, and each time blanks are
13650 matched, the begin cursor is moved onto the end cursor to effectively ignore
13651 the blanks preceding tokens. Comments would be treated equally.
13653 @comment file: c++/calc++/scanner.ll
13657 // Code run each time a pattern is matched.
13658 # define YY_USER_ACTION loc.columns (yyleng);
13664 // A handy shortcut to the location held by the driver.
13665 yy::location& loc = drv.location;
13666 // Code run each time yylex is called.
13670 @{blank@}+ loc.step ();
13671 \n+ loc.lines (yyleng); loc.step ();
13675 The rules are simple. The driver is used to report errors.
13677 @comment file: c++/calc++/scanner.ll
13679 "-" return yy::parser::make_MINUS (loc);
13680 "+" return yy::parser::make_PLUS (loc);
13681 "*" return yy::parser::make_STAR (loc);
13682 "/" return yy::parser::make_SLASH (loc);
13683 "(" return yy::parser::make_LPAREN (loc);
13684 ")" return yy::parser::make_RPAREN (loc);
13685 ":=" return yy::parser::make_ASSIGN (loc);
13687 @{int@} return make_NUMBER (yytext, loc);
13688 @{id@} return yy::parser::make_IDENTIFIER (yytext, loc);
13691 throw yy::parser::syntax_error
13692 (loc, "invalid character: " + std::string(yytext));
13695 <<EOF>> return yy::parser::make_YYEOF (loc);
13700 You should keep your rules simple, both in the parser and in the scanner.
13701 Throwing from the auxiliary functions is then very handy to report errors.
13703 @comment file: c++/calc++/scanner.ll
13706 yy::parser::symbol_type
13707 make_NUMBER (const std::string &s, const yy::parser::location_type& loc)
13710 long n = strtol (s.c_str(), NULL, 10);
13711 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
13712 throw yy::parser::syntax_error (loc, "integer is out of range: " + s);
13713 return yy::parser::make_NUMBER ((int) n, loc);
13719 Finally, because the scanner-related driver's member-functions depend
13720 on the scanner's data, it is simpler to implement them in this file.
13722 @comment file: c++/calc++/scanner.ll
13726 driver::scan_begin ()
13728 yy_flex_debug = trace_scanning;
13729 if (file.empty () || file == "-")
13731 else if (!(yyin = fopen (file.c_str (), "r")))
13733 std::cerr << "cannot open " << file << ": " << strerror (errno) << '\n';
13734 exit (EXIT_FAILURE);
13741 driver::scan_end ()
13748 @node Calc++ Top Level
13749 @subsubsection Calc++ Top Level
13751 The top level file, @file{calc++.cc}, poses no problem.
13754 @comment file: c++/calc++/calc++.cc
13756 /* Main for calc++. -*- C++ -*-
13758 Copyright (C) 2005-2015, 2018-2021 Free Software Foundation, Inc.
13760 This file is part of Bison, the GNU Compiler Compiler.
13762 This program is free software: you can redistribute it and/or modify
13763 it under the terms of the GNU General Public License as published by
13764 the Free Software Foundation, either version 3 of the License, or
13765 (at your option) any later version.
13767 This program is distributed in the hope that it will be useful,
13768 but WITHOUT ANY WARRANTY; without even the implied warranty of
13769 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13770 GNU General Public License for more details.
13772 You should have received a copy of the GNU General Public License
13773 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13777 @comment file: c++/calc++/calc++.cc
13779 #include <iostream>
13780 #include "driver.hh"
13784 main (int argc, char *argv[])
13788 for (int i = 1; i < argc; ++i)
13789 if (argv[i] == std::string ("-p"))
13790 drv.trace_parsing = true;
13791 else if (argv[i] == std::string ("-s"))
13792 drv.trace_scanning = true;
13793 else if (!drv.parse (argv[i]))
13794 std::cout << drv.result << '\n';
13806 * D Bison Interface:: Asking for D parser generation
13807 * D Semantic Values:: %token and %nterm vs. D
13808 * D Location Values:: The position and location classes
13809 * D Parser Interface:: Instantiating and running the parser
13810 * D Parser Context Interface:: Circumstances of a syntax error
13811 * D Scanner Interface:: Specifying the scanner for the parser
13812 * D Action Features:: Special features for use in actions
13815 @node D Bison Interface
13816 @subsection D Bison Interface
13819 The D parser skeletons are selected using the @code{%language "D"}
13820 directive or the @option{-L D}/@option{--language=D} option.
13822 @c FIXME: Documented bug.
13823 When generating a D parser, @samp{bison @var{basename}.y} will create a
13824 single D source file named @file{@var{basename}.d} containing the
13825 parser implementation. Using a grammar file without a @file{.y} suffix is
13826 currently broken. The basename of the parser implementation file can be
13827 changed by the @code{%file-prefix} directive or the
13828 @option{-b}/@option{--file-prefix} option. The entire parser implementation
13829 file name can be changed by the @code{%output} directive or the
13830 @option{-o}/@option{--output} option. The parser implementation file
13831 contains a single class for the parser.
13833 You can create documentation for generated parsers using Ddoc.
13835 GLR parsers are currently unsupported in D. Do not use the
13836 @code{glr-parser} directive.
13838 No header file can be generated for D parsers. Do not use the
13839 @code{%defines} directive or the @option{-d}/@option{--defines} options.
13841 @node D Semantic Values
13842 @subsection D Semantic Values
13845 @c - Printer and destructor
13847 Semantic types are handled by %union, same as for C/C++ parsers.
13849 D parsers do not support @code{%destructor}, since the language
13850 adopts garbage collection. The parser will try to hold references
13851 to semantic values for as little time as needed.
13853 D parsers do not support @code{%printer}, as @code{toString()}
13854 can be used to print the semantic values. This however may change
13855 (in a backwards-compatible way) in future versions of Bison.
13858 @node D Location Values
13859 @subsection D Location Values
13861 @c - class Position
13862 @c - class Location
13864 When the directive @code{%locations} is used, the D parser supports
13865 location tracking, see @ref{Tracking Locations}. The position and
13866 the location structures are provided.
13868 @deftypeivar {Location} {Position} begin
13869 @deftypeivarx {Location} {Position} end
13870 The first, inclusive, position of the range, and the first beyond.
13873 @deftypeop {Constructor} {Location} {} this(@code{Position} @var{loc})
13874 Create a @code{Location} denoting an empty range located at a given point.
13877 @deftypeop {Constructor} {Location} {} this(@code{Position} @var{begin}, @code{Position} @var{end})
13878 Create a @code{Location} from the endpoints of the range.
13881 @deftypemethod {Location} {string} toString()
13882 Prints the range represented by the location.
13886 @node D Parser Interface
13887 @subsection D Parser Interface
13889 The name of the generated parser class defaults to @code{YYParser}. The
13890 @code{YY} prefix may be changed using the @samp{%define api.prefix}.
13891 Alternatively, use @samp{%define api.parser.class @{@var{name}@}} to give a
13892 custom name to the class. The interface of this class is detailed below.
13894 By default, the parser class has public visibility. You can use @code{api.parser.public}, @code{api.parser.abstract} and
13895 @code{api.parser.final} and the @code{%define} declaration to add
13896 modifiers to the parser class.
13898 The superclass and the implemented
13899 interfaces of the parser class can be specified with the @code{%define
13900 api.parser.extends} and @samp{%define api.parser.implements} directives.
13902 The parser class defines a inner
13903 interface, @code{Lexer} (see @ref{D Scanner Interface}). Other than
13904 these inner class/interface, and the members described in the interface
13905 below, all the other members and fields are preceded with a @code{yy} or
13906 @code{YY} prefix to avoid clashes with user code.
13908 The parser class can be extended using the @code{%parse-param}
13909 directive. Each occurrence of the directive will add a by default public field to the parser class, and an argument to its constructor,
13910 which initialize them automatically.
13912 @deftypeop {Constructor} {YYParser} {} this(@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
13913 Build a new parser object with embedded @code{%code lexer}. There are
13914 no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
13915 @code{%lex-param}s are used.
13918 @deftypeop {Constructor} {YYParser} {} this(@code{Lexer} @var{lexer}, @var{parse_param}, @dots{})
13919 Build a new parser object using the specified scanner. There are no
13920 additional parameters unless @code{%param}s and/or @code{%parse-param}s are
13924 @deftypemethod {YYParser} {boolean} parse()
13925 Run the syntactic analysis, and return @code{true} on success,
13926 @code{false} otherwise.
13929 @deftypemethod {YYParser} {boolean} getErrorVerbose()
13930 @deftypemethodx {YYParser} {void} setErrorVerbose(boolean @var{verbose})
13931 Get or set the option to produce verbose error messages. These are only
13932 available with @samp{%define parse.error detailed},
13933 which also turns on verbose error messages.
13936 @deftypemethod {YYParser} {void} yyerror(@code{string} @var{msg})
13937 @deftypemethodx {YYParser} {void} yyerror(@code{Location} @var{loc}, @code{string} @var{msg})
13938 Print an error message using the @code{yyerror} method of the scanner
13939 instance in use. The @code{Location} and @code{Position} parameters are
13940 available only if location tracking is active.
13943 @deftypemethod {YYParser} {boolean} recovering()
13944 During the syntactic analysis, return @code{true} if recovering
13945 from a syntax error.
13946 @xref{Error Recovery}.
13949 @deftypemethod {YYParser} {File} getDebugStream()
13950 @deftypemethodx {YYParser} {void} setDebugStream(@code{File} @var{o})
13951 Get or set the stream used for tracing the parsing. It defaults to
13955 @deftypemethod {YYParser} {int} getDebugLevel()
13956 @deftypemethodx {YYParser} {void} setDebugLevel(@code{int} @var{l})
13957 Get or set the tracing level. Currently its value is either 0, no trace,
13958 or nonzero, full tracing.
13961 @deftypecv {Constant} {YYParser} {string} {bisonVersion}
13962 @deftypecvx {Constant} {YYParser} {string} {bisonSkeleton}
13963 Identify the Bison version and skeleton used to generate this parser.
13966 The internationalization in D is very similar to the one in C. The D
13967 parser uses @code{dgettext} for translating Bison messages.
13969 To enable internationalization, compile using
13970 @code{-version ENABLE_NLS -version YYENABLE_NLS} and import
13971 @code{bindtextdomain} and @code{textdomain} from C:
13974 extern(C) char* bindtextdomain(const char* domainname, const char* dirname);
13975 extern(C) char* textdomain(const char* domainname);
13978 The main function should load the translation catalogs, similarly to the
13979 @file{c/bistromathic} example:
13984 import core.stdc.locale;
13986 // Set up internationalization.
13987 setlocale(LC_ALL, "");
13988 // Use Bison's standard translation catalog for error messages
13989 // (the generated messages).
13990 bindtextdomain("bison-runtime", BISON_LOCALEDIR);
13991 // For the translation catalog of your own project, use the
13992 // name of your project.
13993 bindtextdomain("bison", LOCALEDIR);
13994 textdomain("bison");
13996 // usual main content
14001 For user messages translations, the user must implement the
14002 @code{string} _(@code{const char*} @var{msg}) function and it is recommended
14003 to use @code{gettext}:
14007 static if (!is(typeof(_)))
14009 version(ENABLE_NLS)
14011 extern(C) char* gettext(const char*);
14012 string _(const char* s)
14014 return to!string(gettext(s));
14018 static if (!is(typeof(_)))
14020 pragma(inline, true)
14021 string _(string msg) @{ return msg; @}
14026 @node D Parser Context Interface
14027 @subsection D Parser Context Interface
14028 The parser context provides information to build error reports when you
14029 invoke @samp{%define parse.error custom}.
14031 @defcv {Type} {YYParser} {SymbolKind}
14032 A struct containing an enum of all the grammar symbols, tokens and nonterminals. Its
14033 enumerators are forged from the symbol names. Use void toString(W)(W sink) to get
14037 @deftypemethod {YYParser.Context} {YYParser.SymbolKind} getToken()
14038 The kind of the lookahead. Return @code{null} iff there is no lookahead.
14041 @deftypemethod {YYParser.Context} {YYParser.Location} getLocation()
14042 The location of the lookahead.
14045 @deftypemethod {YYParser.Context} {int} getExpectedTokens(@code{YYParser.SymbolKind[]} @var{argv}, @code{int} @var{argc})
14046 Fill @var{argv} with the expected tokens, which never includes
14047 @code{SymbolKind.YYERROR}, or @code{SymbolKind.YYUNDEF}.
14049 Never put more than @var{argc} elements into @var{argv}, and on success
14050 return the number of tokens stored in @var{argv}. If there are more
14051 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
14052 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
14055 If @var{argv} is null, return the size needed to store all the possible
14056 values, which is always less than @code{YYNTOKENS}.
14060 @node D Scanner Interface
14061 @subsection D Scanner Interface
14064 @c - Lexer interface
14066 There are two possible ways to interface a Bison-generated D parser
14067 with a scanner: the scanner may be defined by @code{%code lexer}, or
14068 defined elsewhere. In either case, the scanner has to implement the
14069 @code{Lexer} inner interface of the parser class. This interface also
14070 contains constants for all user-defined token names and the predefined
14071 @code{YYEOF} token.
14073 In the first case, the body of the scanner class is placed in
14074 @code{%code lexer} blocks. If you want to pass parameters from the
14075 parser constructor to the scanner constructor, specify them with
14076 @code{%lex-param}; they are passed before @code{%parse-param}s to the
14079 In the second case, the scanner has to implement the @code{Lexer} interface,
14080 which is defined within the parser class (e.g., @code{YYParser.Lexer}).
14081 The constructor of the parser object will then accept an object
14082 implementing the interface; @code{%lex-param} is not used in this
14085 In both cases, the scanner has to implement the following methods.
14087 @deftypemethod {Lexer} {void} yyerror(@code{Location} @var{loc}, @code{string} @var{msg})
14088 This method is defined by the user to emit an error message. The first
14089 parameter is omitted if location tracking is not active.
14092 @deftypemethod {Lexer} {Symbol} yylex()
14093 Return the next token. The return value is of type @code{Symbol}, which
14094 binds together the kind, the semantic value and the location.
14097 @deftypemethod {Lexer} {void} reportSyntaxError(@code{YYParser.Context} @var{ctx})
14098 If you invoke @samp{%define parse.error custom} (@pxref{Bison
14099 Declarations}), then the parser no longer passes syntax error messages to
14100 @code{yyerror}, rather it delegates that task to the user by calling the
14101 @code{reportSyntaxError} function.
14103 Whether it uses @code{yyerror} is up to the user.
14105 Here is an example of a reporting function (@pxref{D Parser Context
14109 public void reportSyntaxError(YYParser.Context ctx)
14111 stderr.write(ctx.getLocation(), ": syntax error");
14112 // Report the expected tokens.
14114 immutable int TOKENMAX = 5;
14115 YYParser.SymbolKind[] arg = new YYParser.SymbolKind[TOKENMAX];
14116 int n = ctx.getExpectedTokens(arg, TOKENMAX);
14118 for (int i = 0; i < n; ++i)
14119 stderr.write((i == 0 ? ": expected " : " or "), arg[i]);
14121 // Report the unexpected token which triggered the error.
14123 YYParser.SymbolKind lookahead = ctx.getToken();
14124 stderr.writeln(" before ", lookahead);
14130 This implementation is inappropriate for internationalization, see
14131 the @file{c/bistromathic} example for a better alternative.
14134 @node D Action Features
14135 @subsection Special Features for Use in D Actions
14137 Here is a table of Bison constructs, variables and functions that are useful in
14140 @deffn {Variable} $$
14141 Acts like a variable that contains the semantic value for the
14142 grouping made by the current rule. @xref{Actions}.
14145 @deffn {Variable} $@var{n}
14146 Acts like a variable that contains the semantic value for the
14147 @var{n}th component of the current rule. @xref{Actions}.
14150 @deffn {Function} yyerrok
14151 Resume generating error messages immediately for subsequent syntax
14152 errors. This is useful primarily in error rules.
14153 @xref{Error Recovery}.
14157 @section Java Parsers
14160 * Java Bison Interface:: Asking for Java parser generation
14161 * Java Semantic Values:: %token and %nterm vs. Java
14162 * Java Location Values:: The position and location classes
14163 * Java Parser Interface:: Instantiating and running the parser
14164 * Java Parser Context Interface:: Circumstances of a syntax error
14165 * Java Scanner Interface:: Specifying the scanner for the parser
14166 * Java Action Features:: Special features for use in actions
14167 * Java Push Parser Interface:: Instantiating and running the a push parser
14168 * Java Differences:: Differences between C/C++ and Java Grammars
14169 * Java Declarations Summary:: List of Bison declarations used with Java
14172 @node Java Bison Interface
14173 @subsection Java Bison Interface
14174 @c - %language "Java"
14176 The Java parser skeletons are selected using the @code{%language "Java"}
14177 directive or the @option{-L java}/@option{--language=java} option.
14179 @c FIXME: Documented bug.
14180 When generating a Java parser, @samp{bison @var{basename}.y} will create a
14181 single Java source file named @file{@var{basename}.java} containing the
14182 parser implementation. Using a grammar file without a @file{.y} suffix is
14183 currently broken. The basename of the parser implementation file can be
14184 changed by the @code{%file-prefix} directive or the
14185 @option{-b}/@option{--file-prefix} option. The entire parser implementation
14186 file name can be changed by the @code{%output} directive or the
14187 @option{-o}/@option{--output} option. The parser implementation file
14188 contains a single class for the parser.
14190 You can create documentation for generated parsers using Javadoc.
14192 Contrary to C parsers, Java parsers do not use global variables; the state
14193 of the parser is always local to an instance of the parser class.
14194 Therefore, all Java parsers are ``pure'', and the @code{%define api.pure}
14195 directive does nothing when used in Java.
14197 GLR parsers are currently unsupported in Java. Do not use the
14198 @code{glr-parser} directive.
14200 No header file can be generated for Java parsers. Do not use the
14201 @code{%header} directive or the @option{-d}/@option{-H}/@option{--header}
14204 @c FIXME: Possible code change.
14205 Currently, support for tracing is always compiled in. Thus the
14206 @samp{%define parse.trace} and @samp{%token-table} directives and the
14207 @option{-t}/@option{--debug} and @option{-k}/@option{--token-table} options
14208 have no effect. This may change in the future to eliminate unused code in
14209 the generated parser, so use @samp{%define parse.trace} explicitly if
14210 needed. Also, in the future the @code{%token-table} directive might enable
14211 a public interface to access the token names and codes.
14213 Getting a ``code too large'' error from the Java compiler means the code hit
14214 the 64KB bytecode per method limitation of the Java class file. Try
14215 reducing the amount of code in actions and static initializers; otherwise,
14216 report a bug so that the parser skeleton will be improved.
14219 @node Java Semantic Values
14220 @subsection Java Semantic Values
14221 @c - No %union, specify type in %nterm/%token.
14223 @c - Printer and destructor
14225 There is no @code{%union} directive in Java parsers. Instead, the semantic
14226 values' types (class names) should be specified in the @code{%nterm} or
14227 @code{%token} directive:
14230 %nterm <Expression> expr assignment_expr term factor
14231 %nterm <Integer> number
14234 By default, the semantic stack is declared to have @code{Object} members,
14235 which means that the class types you specify can be of any class.
14236 To improve the type safety of the parser, you can declare the common
14237 superclass of all the semantic values using the @samp{%define api.value.type}
14238 directive. For example, after the following declaration:
14241 %define api.value.type @{ASTNode@}
14245 any @code{%token}, @code{%nterm} or @code{%type} specifying a semantic type
14246 which is not a subclass of @code{ASTNode}, will cause a compile-time error.
14248 @c FIXME: Documented bug.
14249 Types used in the directives may be qualified with a package name.
14250 Primitive data types are accepted for Java version 1.5 or later. Note
14251 that in this case the autoboxing feature of Java 1.5 will be used.
14252 Generic types may not be used; this is due to a limitation in the
14253 implementation of Bison, and may change in future releases.
14255 Java parsers do not support @code{%destructor}, since the language
14256 adopts garbage collection. The parser will try to hold references
14257 to semantic values for as little time as needed.
14259 Java parsers do not support @code{%printer}, as @code{toString()}
14260 can be used to print the semantic values. This however may change
14261 (in a backwards-compatible way) in future versions of Bison.
14264 @node Java Location Values
14265 @subsection Java Location Values
14267 @c - class Position
14268 @c - class Location
14270 When the directive @code{%locations} is used, the Java parser supports
14271 location tracking, see @ref{Tracking Locations}. An auxiliary user-defined
14272 class defines a @dfn{position}, a single point in a file; Bison itself
14273 defines a class representing a @dfn{location}, a range composed of a pair of
14274 positions (possibly spanning several files). The location class is an inner
14275 class of the parser; the name is @code{Location} by default, and may also be
14276 renamed using @code{%define api.location.type @{@var{class-name}@}}.
14278 The location class treats the position as a completely opaque value.
14279 By default, the class name is @code{Position}, but this can be changed
14280 with @code{%define api.position.type @{@var{class-name}@}}. This class must
14281 be supplied by the user.
14284 @deftypeivar {Location} {Position} begin
14285 @deftypeivarx {Location} {Position} end
14286 The first, inclusive, position of the range, and the first beyond.
14289 @deftypeop {Constructor} {Location} {} Location (@code{Position} @var{loc})
14290 Create a @code{Location} denoting an empty range located at a given point.
14293 @deftypeop {Constructor} {Location} {} Location (@code{Position} @var{begin}, @code{Position} @var{end})
14294 Create a @code{Location} from the endpoints of the range.
14297 @deftypemethod {Location} {String} toString ()
14298 Prints the range represented by the location. For this to work
14299 properly, the position class should override the @code{equals} and
14300 @code{toString} methods appropriately.
14304 @node Java Parser Interface
14305 @subsection Java Parser Interface
14307 The name of the generated parser class defaults to @code{YYParser}. The
14308 @code{YY} prefix may be changed using the @samp{%define api.prefix}.
14309 Alternatively, use @samp{%define api.parser.class @{@var{name}@}} to give a
14310 custom name to the class. The interface of this class is detailed below.
14312 By default, the parser class has package visibility. A declaration
14313 @samp{%define api.parser.public} will change to public visibility. Remember
14314 that, according to the Java language specification, the name of the
14315 @file{.java} file should match the name of the class in this case.
14316 Similarly, you can use @code{api.parser.abstract}, @code{api.parser.final}
14317 and @code{api.parser.strictfp} with the @code{%define} declaration to add
14318 other modifiers to the parser class. A single @samp{%define
14319 api.parser.annotations @{@var{annotations}@}} directive can be used to add
14320 any number of annotations to the parser class.
14322 The Java package name of the parser class can be specified using the
14323 @samp{%define package} directive. The superclass and the implemented
14324 interfaces of the parser class can be specified with the @code{%define
14325 api.parser.extends} and @samp{%define api.parser.implements} directives.
14327 The parser class defines an inner class, @code{Location}, that is used
14328 for location tracking (see @ref{Java Location Values}), and a inner
14329 interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
14330 these inner class/interface, and the members described in the interface
14331 below, all the other members and fields are preceded with a @code{yy} or
14332 @code{YY} prefix to avoid clashes with user code.
14334 The parser class can be extended using the @code{%parse-param}
14335 directive. Each occurrence of the directive will add a @code{protected
14336 final} field to the parser class, and an argument to its constructor,
14337 which initialize them automatically.
14339 @deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
14340 Build a new parser object with embedded @code{%code lexer}. There are
14341 no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
14342 @code{%lex-param}s are used.
14344 Use @code{%code init} for code added to the start of the constructor
14345 body. This is especially useful to initialize superclasses. Use
14346 @samp{%define init_throws} to specify any uncaught exceptions.
14349 @deftypeop {Constructor} {YYParser} {} YYParser (@code{Lexer} @var{lexer}, @var{parse_param}, @dots{})
14350 Build a new parser object using the specified scanner. There are no
14351 additional parameters unless @code{%param}s and/or @code{%parse-param}s are
14354 If the scanner is defined by @code{%code lexer}, this constructor is
14355 declared @code{protected} and is called automatically with a scanner
14356 created with the correct @code{%param}s and/or @code{%lex-param}s.
14358 Use @code{%code init} for code added to the start of the constructor
14359 body. This is especially useful to initialize superclasses. Use
14360 @samp{%define init_throws} to specify any uncaught exceptions.
14363 @deftypemethod {YYParser} {boolean} parse ()
14364 Run the syntactic analysis, and return @code{true} on success,
14365 @code{false} otherwise.
14368 @deftypemethod {YYParser} {boolean} getErrorVerbose ()
14369 @deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
14370 Get or set the option to produce verbose error messages. These are only
14371 available with @samp{%define parse.error detailed} (or @samp{verbose}),
14372 which also turns on verbose error messages.
14375 @deftypemethod {YYParser} {void} yyerror (@code{String} @var{msg})
14376 @deftypemethodx {YYParser} {void} yyerror (@code{Position} @var{pos}, @code{String} @var{msg})
14377 @deftypemethodx {YYParser} {void} yyerror (@code{Location} @var{loc}, @code{String} @var{msg})
14378 Print an error message using the @code{yyerror} method of the scanner
14379 instance in use. The @code{Location} and @code{Position} parameters are
14380 available only if location tracking is active.
14383 @deftypemethod {YYParser} {boolean} recovering ()
14384 During the syntactic analysis, return @code{true} if recovering
14385 from a syntax error.
14386 @xref{Error Recovery}.
14389 @deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
14390 @deftypemethodx {YYParser} {void} setDebugStream (@code{java.io.PrintStream} @var{o})
14391 Get or set the stream used for tracing the parsing. It defaults to
14395 @deftypemethod {YYParser} {int} getDebugLevel ()
14396 @deftypemethodx {YYParser} {void} setDebugLevel (@code{int} @var{l})
14397 Get or set the tracing level. Currently its value is either 0, no trace,
14398 or nonzero, full tracing.
14401 @deftypecv {Constant} {YYParser} {String} {bisonVersion}
14402 @deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
14403 Identify the Bison version and skeleton used to generate this parser.
14406 If you enabled token internationalization (@pxref{Token I18n}), you must
14407 provide the parser with the following function:
14409 @deftypecv {Static Method} {YYParser} {String} {i18n} (@code{string} @var{s})
14410 Return the translation of @var{s} in the user's language. As an example:
14414 static ResourceBundle myResources
14415 = ResourceBundle.getBundle("domain-name");
14416 static final String i18n(String s) @{
14417 return myResources.getString(s);
14423 @node Java Parser Context Interface
14424 @subsection Java Parser Context Interface
14426 The parser context provides information to build error reports when you
14427 invoke @samp{%define parse.error custom}.
14429 @defcv {Type} {YYParser} {SymbolKind}
14430 An enum of all the grammar symbols, tokens and nonterminals. Its
14431 enumerators are forged from the symbol names:
14434 public enum SymbolKind
14436 S_YYEOF(0), /* "end of file" */
14437 S_YYERROR(1), /* error */
14438 S_YYUNDEF(2), /* "invalid token" */
14439 S_BANG(3), /* "!" */
14440 S_PLUS(4), /* "+" */
14441 S_MINUS(5), /* "-" */
14443 S_NUM(13), /* "number" */
14444 S_NEG(14), /* NEG */
14445 S_YYACCEPT(15), /* $accept */
14446 S_input(16), /* input */
14447 S_line(17); /* line */
14452 @deftypemethod {YYParser.SymbolKind} {String} getName ()
14453 The name of this symbol, possibly translated.
14456 @deftypemethod {YYParser.Context} {YYParser.SymbolKind} getToken ()
14457 The kind of the lookahead. Return @code{null} iff there is no lookahead.
14460 @deftypemethod {YYParser.Context} {YYParser.Location} getLocation ()
14461 The location of the lookahead.
14464 @deftypemethod {YYParser.Context} {int} getExpectedTokens (@code{YYParser.SymbolKind[]} @var{argv}, @code{int} @var{argc})
14465 Fill @var{argv} with the expected tokens, which never includes
14466 @code{SymbolKind.S_YYERROR}, or @code{SymbolKind.S_YYUNDEF}.
14468 Never put more than @var{argc} elements into @var{argv}, and on success
14469 return the number of tokens stored in @var{argv}. If there are more
14470 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
14471 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
14474 If @var{argv} is null, return the size needed to store all the possible
14475 values, which is always less than @code{YYNTOKENS}.
14479 @node Java Scanner Interface
14480 @subsection Java Scanner Interface
14483 @c - Lexer interface
14485 There are two possible ways to interface a Bison-generated Java parser
14486 with a scanner: the scanner may be defined by @code{%code lexer}, or
14487 defined elsewhere. In either case, the scanner has to implement the
14488 @code{Lexer} inner interface of the parser class. This interface also
14489 contains constants for all user-defined token names and the predefined
14490 @code{YYEOF} token.
14492 In the first case, the body of the scanner class is placed in
14493 @code{%code lexer} blocks. If you want to pass parameters from the
14494 parser constructor to the scanner constructor, specify them with
14495 @code{%lex-param}; they are passed before @code{%parse-param}s to the
14498 In the second case, the scanner has to implement the @code{Lexer} interface,
14499 which is defined within the parser class (e.g., @code{YYParser.Lexer}).
14500 The constructor of the parser object will then accept an object
14501 implementing the interface; @code{%lex-param} is not used in this
14504 In both cases, the scanner has to implement the following methods.
14506 @deftypemethod {Lexer} {void} yyerror (@code{Location} @var{loc}, @code{String} @var{msg})
14507 This method is defined by the user to emit an error message. The first
14508 parameter is omitted if location tracking is not active. Its type can be
14509 changed using @code{%define api.location.type @{@var{class-name}@}}.
14512 @deftypemethod {Lexer} {int} yylex ()
14513 Return the next token. Its type is the return value, its semantic value and
14514 location are saved and returned by the their methods in the interface. Not
14515 needed for push-only parsers.
14517 Use @samp{%define lex_throws} to specify any uncaught exceptions.
14518 Default is @code{java.io.IOException}.
14521 @deftypemethod {Lexer} {Position} getStartPos ()
14522 @deftypemethodx {Lexer} {Position} getEndPos ()
14523 Return respectively the first position of the last token that @code{yylex}
14524 returned, and the first position beyond it. These methods are not needed
14525 unless location tracking and pull parsing are active.
14527 They should return new objects for each call, to avoid that all the symbol
14528 share the same Position boundaries.
14530 The return type can be changed using @code{%define api.position.type
14531 @{@var{class-name}@}}.
14534 @deftypemethod {Lexer} {Object} getLVal ()
14535 Return the semantic value of the last token that yylex returned. Not needed
14536 for push-only parsers.
14538 The return type can be changed using @samp{%define api.value.type
14539 @{@var{class-name}@}}.
14542 @deftypemethod {Lexer} {void} reportSyntaxError (@code{YYParser.Context} @var{ctx})
14543 If you invoke @samp{%define parse.error custom} (@pxref{Bison
14544 Declarations}), then the parser no longer passes syntax error messages to
14545 @code{yyerror}, rather it delegates that task to the user by calling the
14546 @code{reportSyntaxError} function.
14548 Whether it uses @code{yyerror} is up to the user.
14550 Here is an example of a reporting function (@pxref{Java Parser Context
14554 public void reportSyntaxError(YYParser.Context ctx) @{
14555 System.err.print(ctx.getLocation() + ": syntax error");
14556 // Report the expected tokens.
14558 final int TOKENMAX = 5;
14559 YYParser.SymbolKind[] arg = new YYParser.SymbolKind[TOKENMAX];
14560 int n = ctx.getExpectedTokens(arg, TOKENMAX);
14561 for (int i = 0; i < n; ++i)
14562 System.err.print((i == 0 ? ": expected " : " or ")
14563 + arg[i].getName());
14565 // Report the unexpected token which triggered the error.
14567 YYParser.SymbolKind lookahead = ctx.getToken();
14568 if (lookahead != null)
14569 System.err.print(" before " + lookahead.getName());
14571 System.err.println("");
14576 This implementation is inappropriate for internationalization, see the
14577 @file{c/bistromathic} example for a better alternative.
14580 @node Java Action Features
14581 @subsection Special Features for Use in Java Actions
14583 The following special constructs can be uses in Java actions.
14584 Other analogous C action features are currently unavailable for Java.
14586 Use @samp{%define throws} to specify any uncaught exceptions from parser
14587 actions, and initial actions specified by @code{%initial-action}.
14590 The semantic value for the @var{n}th component of the current rule.
14591 This may not be assigned to.
14592 @xref{Java Semantic Values}.
14595 @defvar $<@var{typealt}>@var{n}
14596 Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
14597 @xref{Java Semantic Values}.
14601 The semantic value for the grouping made by the current rule. As a
14602 value, this is in the base type (@code{Object} or as specified by
14603 @samp{%define api.value.type}) as in not cast to the declared subtype because
14604 casts are not allowed on the left-hand side of Java assignments.
14605 Use an explicit Java cast if the correct subtype is needed.
14606 @xref{Java Semantic Values}.
14609 @defvar $<@var{typealt}>$
14610 Same as @code{$$} since Java always allow assigning to the base type.
14611 Perhaps we should use this and @code{$<>$} for the value and @code{$$}
14612 for setting the value but there is currently no easy way to distinguish
14614 @xref{Java Semantic Values}.
14618 The location information of the @var{n}th component of the current rule.
14619 This may not be assigned to.
14620 @xref{Java Location Values}.
14624 The location information of the grouping made by the current rule.
14625 @xref{Java Location Values}.
14628 @deftypefn {Statement} return YYABORT @code{;}
14629 Return immediately from the parser, indicating failure.
14630 @xref{Java Parser Interface}.
14633 @deftypefn {Statement} return YYACCEPT @code{;}
14634 Return immediately from the parser, indicating success.
14635 @xref{Java Parser Interface}.
14638 @deftypefn {Statement} {return} YYERROR @code{;}
14639 Start error recovery (without printing an error message).
14640 @xref{Error Recovery}.
14643 @deftypefn {Function} {boolean} recovering ()
14644 Return whether error recovery is being done. In this state, the parser
14645 reads token until it reaches a known state, and then restarts normal
14647 @xref{Error Recovery}.
14650 @deftypefn {Function} {void} yyerror (@code{String} @var{msg})
14651 @deftypefnx {Function} {void} yyerror (@code{Position} @var{loc}, @code{String} @var{msg})
14652 @deftypefnx {Function} {void} yyerror (@code{Location} @var{loc}, @code{String} @var{msg})
14653 Print an error message using the @code{yyerror} method of the scanner
14654 instance in use. The @code{Location} and @code{Position} parameters are
14655 available only if location tracking is active.
14658 @node Java Push Parser Interface
14659 @subsection Java Push Parser Interface
14660 @c - define push_parse
14661 @findex %define api.push-pull
14663 Normally, Bison generates a pull parser for Java.
14664 The following Bison declaration says that you want the parser to be a push
14665 parser (@pxref{%define Summary}):
14668 %define api.push-pull push
14671 Most of the discussion about the Java pull Parser Interface, (@pxref{Java
14672 Parser Interface}) applies to the push parser interface as well.
14674 When generating a push parser, the method @code{push_parse} is created with
14675 the following signature (depending on if locations are enabled).
14677 @deftypemethod {YYParser} {void} push_parse (@code{int} @var{token}, @code{Object} @var{yylval})
14678 @deftypemethodx {YYParser} {void} push_parse (@code{int} @var{token}, @code{Object} @var{yylval}, @code{Location} @var{yyloc})
14679 @deftypemethodx {YYParser} {void} push_parse (@code{int} @var{token}, @code{Object} @var{yylval}, @code{Position} @var{yypos})
14682 The primary difference with respect to a pull parser is that the parser
14683 method @code{push_parse} is invoked repeatedly to parse each token. This
14684 function is available if either the "%define api.push-pull push" or "%define
14685 api.push-pull both" declaration is used (@pxref{%define
14686 Summary}). The @code{Location} and @code{Position}
14687 parameters are available only if location tracking is active.
14689 The value returned by the @code{push_parse} method is one of the following:
14690 0 (success), 1 (abort), 2 (memory exhaustion), or @code{YYPUSH_MORE}. This
14691 new value, @code{YYPUSH_MORE}, may be returned if more input is required to
14692 finish parsing the grammar.
14694 If api.push-pull is declared as @code{both}, then the generated parser class
14695 will also implement the @code{parse} method. This method's body is a loop
14696 that repeatedly invokes the scanner and then passes the values obtained from
14697 the scanner to the @code{push_parse} method.
14699 There is one additional complication. Technically, the push parser does not
14700 need to know about the scanner (i.e. an object implementing the
14701 @code{YYParser.Lexer} interface), but it does need access to the
14702 @code{yyerror} method. Currently, the @code{yyerror} method is defined in
14703 the @code{YYParser.Lexer} interface. Hence, an implementation of that
14704 interface is still required in order to provide an implementation of
14705 @code{yyerror}. The current approach (and subject to change) is to require
14706 the @code{YYParser} constructor to be given an object implementing the
14707 @code{YYParser.Lexer} interface. This object need only implement the
14708 @code{yyerror} method; the other methods can be stubbed since they will
14709 never be invoked. The simplest way to do this is to add a trivial scanner
14710 implementation to your grammar file using whatever implementation of
14711 @code{yyerror} is desired. The following code sample shows a simple way to
14717 public Object getLVal () @{return null;@}
14718 public int yylex () @{return 0;@}
14719 public void yyerror (String s) @{System.err.println(s);@}
14723 @node Java Differences
14724 @subsection Differences between C/C++ and Java Grammars
14726 The different structure of the Java language forces several differences
14727 between C/C++ grammars, and grammars designed for Java parsers. This
14728 section summarizes these differences.
14732 Java lacks a preprocessor, so obviously the @code{YYERROR}, @code{YYACCEPT},
14733 @code{YYABORT} symbols (@pxref{Table of Symbols}) cannot be macros.
14734 Instead, they should be preceded by @code{return} when they appear in an
14735 action. The actual definition of these symbols is opaque to the Bison
14736 grammar, and it might change in the future. The only meaningful operation
14737 that you can do, is to return them. @xref{Java Action Features}.
14739 Note that of these three symbols, only @code{YYACCEPT} and
14740 @code{YYABORT} will cause a return from the @code{yyparse}
14741 method@footnote{Java parsers include the actions in a separate
14742 method than @code{yyparse} in order to have an intuitive syntax that
14743 corresponds to these C macros.}.
14746 Java lacks unions, so @code{%union} has no effect. Instead, semantic
14747 values have a common base type: @code{Object} or as specified by
14748 @samp{%define api.value.type}. Angle brackets on @code{%token}, @code{type},
14749 @code{$@var{n}} and @code{$$} specify subtypes rather than fields of
14750 an union. The type of @code{$$}, even with angle brackets, is the base
14751 type since Java casts are not allow on the left-hand side of assignments.
14752 Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
14753 left-hand side of assignments. @xref{Java Semantic Values}, and
14754 @ref{Java Action Features}.
14757 The prologue declarations have a different meaning than in C/C++ code.
14759 @item @code{%code imports}
14760 blocks are placed at the beginning of the Java source code. They may
14761 include copyright notices. For a @code{package} declarations, use
14762 @samp{%define api.package} instead.
14764 @item unqualified @code{%code}
14765 blocks are placed inside the parser class.
14767 @item @code{%code lexer}
14768 blocks, if specified, should include the implementation of the
14769 scanner. If there is no such block, the scanner can be any class
14770 that implements the appropriate interface (@pxref{Java Scanner
14774 Other @code{%code} blocks are not supported in Java parsers.
14775 In particular, @code{%@{ @dots{} %@}} blocks should not be used
14776 and may give an error in future versions of Bison.
14778 The epilogue has the same meaning as in C/C++ code and it can
14779 be used to define other classes used by the parser @emph{outside}
14784 @node Java Declarations Summary
14785 @subsection Java Declarations Summary
14787 This summary only include declarations specific to Java or have special
14788 meaning when used in a Java parser.
14790 @deffn {Directive} {%language "Java"}
14791 Generate a Java class for the parser.
14794 @deffn {Directive} %lex-param @{@var{type} @var{name}@}
14795 A parameter for the lexer class defined by @code{%code lexer}
14796 @emph{only}, added as parameters to the lexer constructor and the parser
14797 constructor that @emph{creates} a lexer. Default is none.
14798 @xref{Java Scanner Interface}.
14801 @deffn {Directive} %parse-param @{@var{type} @var{name}@}
14802 A parameter for the parser class added as parameters to constructor(s)
14803 and as fields initialized by the constructor(s). Default is none.
14804 @xref{Java Parser Interface}.
14807 @deffn {Directive} %token <@var{type}> @var{token} @dots{}
14808 Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
14809 @xref{Java Semantic Values}.
14812 @deffn {Directive} %nterm <@var{type}> @var{nonterminal} @dots{}
14813 Declare the type of nonterminals. Note that the angle brackets enclose
14814 a Java @emph{type}.
14815 @xref{Java Semantic Values}.
14818 @deffn {Directive} %code @{ @var{code} @dots{} @}
14819 Code appended to the inside of the parser class.
14820 @xref{Java Differences}.
14823 @deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
14824 Code inserted just after the @code{package} declaration.
14825 @xref{Java Differences}.
14828 @deffn {Directive} {%code init} @{ @var{code} @dots{} @}
14829 Code inserted at the beginning of the parser constructor body.
14830 @xref{Java Parser Interface}.
14833 @deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
14834 Code added to the body of a inner lexer class within the parser class.
14835 @xref{Java Scanner Interface}.
14838 @deffn {Directive} %% @var{code} @dots{}
14839 Code (after the second @code{%%}) appended to the end of the file,
14840 @emph{outside} the parser class.
14841 @xref{Java Differences}.
14844 @deffn {Directive} %@{ @var{code} @dots{} %@}
14845 Not supported. Use @code{%code imports} instead.
14846 @xref{Java Differences}.
14849 @deffn {Directive} {%define api.prefix} @{@var{prefix}@}
14850 The prefix of the parser class name @code{@var{prefix}Parser} if
14851 @samp{%define api.parser.class} is not used. Default is @code{YY}.
14852 @xref{Java Bison Interface}.
14855 @deffn {Directive} {%define api.parser.abstract}
14856 Whether the parser class is declared @code{abstract}. Default is false.
14857 @xref{Java Bison Interface}.
14860 @deffn {Directive} {%define api.parser.annotations} @{@var{annotations}@}
14861 The Java annotations for the parser class. Default is none.
14862 @xref{Java Bison Interface}.
14865 @deffn {Directive} {%define api.parser.class} @{@var{name}@}
14866 The name of the parser class. Default is @code{YYParser} or
14867 @code{@var{api.prefix}Parser}. @xref{Java Bison Interface}.
14870 @deffn {Directive} {%define api.parser.extends} @{@var{superclass}@}
14871 The superclass of the parser class. Default is none.
14872 @xref{Java Bison Interface}.
14875 @deffn {Directive} {%define api.parser.final}
14876 Whether the parser class is declared @code{final}. Default is false.
14877 @xref{Java Bison Interface}.
14880 @deffn {Directive} {%define api.parser.implements} @{@var{interfaces}@}
14881 The implemented interfaces of the parser class, a comma-separated list.
14883 @xref{Java Bison Interface}.
14886 @deffn {Directive} {%define api.parser.public}
14887 Whether the parser class is declared @code{public}. Default is false.
14888 @xref{Java Bison Interface}.
14891 @deffn {Directive} {%define api.parser.strictfp}
14892 Whether the parser class is declared @code{strictfp}. Default is false.
14893 @xref{Java Bison Interface}.
14896 @deffn {Directive} {%define init_throws} @{@var{exceptions}@}
14897 The exceptions thrown by @code{%code init} from the parser class
14898 constructor. Default is none.
14899 @xref{Java Parser Interface}.
14902 @deffn {Directive} {%define lex_throws} @{@var{exceptions}@}
14903 The exceptions thrown by the @code{yylex} method of the lexer, a
14904 comma-separated list. Default is @code{java.io.IOException}.
14905 @xref{Java Scanner Interface}.
14908 @deffn {Directive} {%define api.location.type} @{@var{class}@}
14909 The name of the class used for locations (a range between two
14910 positions). This class is generated as an inner class of the parser
14911 class by @command{bison}. Default is @code{Location}.
14912 Formerly named @code{location_type}.
14913 @xref{Java Location Values}.
14916 @deffn {Directive} {%define api.package} @{@var{package}@}
14917 The package to put the parser class in. Default is none.
14918 @xref{Java Bison Interface}.
14919 Renamed from @code{package} in Bison 3.7.
14922 @deffn {Directive} {%define api.position.type} @{@var{class}@}
14923 The name of the class used for positions. This class must be supplied by
14924 the user. Default is @code{Position}.
14925 Formerly named @code{position_type}.
14926 @xref{Java Location Values}.
14929 @deffn {Directive} {%define api.value.type} @{@var{class}@}
14930 The base type of semantic values. Default is @code{Object}.
14931 @xref{Java Semantic Values}.
14934 @deffn {Directive} {%define throws} @{@var{exceptions}@}
14935 The exceptions thrown by user-supplied parser actions and
14936 @code{%initial-action}, a comma-separated list. Default is none.
14937 @xref{Java Parser Interface}.
14941 @c ================================================= History
14944 @chapter A Brief History of the Greater Ungulates
14949 * Yacc:: The original Yacc
14950 * yacchack:: An obscure early implementation of reentrancy
14951 * Byacc:: Berkeley Yacc
14952 * Bison:: This program
14953 * Other Ungulates:: Similar programs
14957 @section The ancestral Yacc
14959 Bison originated as a workalike of a program called Yacc --- Yet Another
14960 Compiler Compiler.@footnote{Because of the acronym, the name is sometimes
14961 given as ``YACC'', but Johnson used ``Yacc'' in the descriptive paper
14963 @url{https://s3.amazonaws.com/plan9-bell-labs/7thEdMan/v7vol2b.pdf, Version
14964 7 Unix Manual}.} Yacc was written at Bell Labs as part of the very early
14965 development of Unix; one of its first uses was to develop the original
14966 Portable C Compiler, pcc. The same person, Steven C. Johnson, wrote Yacc and
14969 According to the author
14970 @footnote{@url{https://lists.gnu.org/archive/html/bison-patches/2019-02/msg00061.html}},
14971 Yacc was first invented in 1971 and reached a form recognizably similar to
14972 the C version in 1973. Johnson published @cite{A Portable Compiler: Theory
14973 and Practice} @pcite{Johnson 1978}.
14975 Yacc was not itself originally written in C but in its predecessor language,
14976 B. This goes far to explain its odd interface, which exposes a large number
14977 of global variables rather than bundling them into a C struct. All other
14978 Yacc-like programs are descended from the C port of Yacc.
14980 Yacc, through both its deployment in pcc and as a standalone tool for
14981 generating other parsers, helped drive the early spread of Unix. Yacc
14982 itself, however, passed out of use after around 1990 when workalikes
14983 with less restrictive licenses and more features became available.
14985 Original Yacc became generally available when Caldera released the sources
14986 of old versions of Unix up to V7 and 32V in 2002. By that time it had been
14987 long superseded in practical use by Bison even on Yacc's native Unix
14994 One of the deficiencies of original Yacc was its inability to produce
14995 reentrant parsers. This was first remedied by a set of drop-in
14996 modifications called ``yacchack'', published by Eric S. Raymond on USENET
14997 around 1983. This code was quickly forgotten when zoo and Berkeley Yacc
14998 became available a few years later.
15001 @section Berkeley Yacc
15004 Berkeley Yacc was originated in 1985 by Robert Corbett @pcite{Corbett
15005 1984}. It was originally named ``zoo'', but by October 1989 it became
15006 known as Berkeley Yacc or byacc.
15008 Berkeley Yacc had three advantages over the ancestral Yacc: it generated
15009 faster parsers, it could generate reentrant parsers, and the source code was
15010 released to the public domain rather than being under an AT&T proprietary
15011 license. The better performance came from implementing techniques from
15012 DeRemer and Penello's seminal paper on LALR parsing @pcite{DeRemer 1982}.
15014 Use of byacc spread rapidly due to its public domain license. However, once
15015 Bison became available, byacc itself passed out of general use.
15021 Robert Corbett actually wrote two (closely related) LALR parsers in 1985,
15022 both using the DeRemer/Penello techniques. One was ``zoo'', the other was
15023 ``Byson''. In 1987 Richard Stallman began working on Byson; the name changed
15024 to Bison and the interface became Yacc-compatible.
15026 The main visible difference between Yacc and Byson/Bison at the time of
15027 Byson's first release is that Byson supported the @code{@@@var{n}} construct
15028 (giving access to the starting and ending line number and character number
15029 associated with any of the symbols in the current rule).
15031 There was also the command @samp{%expect @var{n}} which said not to mention the
15032 conflicts if there are @var{n} shift/reduce conflicts and no reduce/reduce
15033 conflicts. In more recent versions of Bison, @code{%expect} and its
15034 @code{%expect-rr} variant for reduce/reduce conflicts can be applied to
15037 Later versions of Bison added many more new features.
15039 Bison error reporting has been improved in various ways. Notably. ancestral
15040 Yacc and Byson did not have carets in error messages.
15042 Compared to Yacc Bison uses a faster but less space-efficient encoding for
15043 the parse tables @pcite{Corbett 1984}, and more modern techniques for
15044 generating the lookahead sets @pcite{DeRemer 1982}. This approach is the
15045 standard one since then.
15047 (It has also been plausibly alleged the differences in the algorithms stem
15048 mainly from the horrible kludges that Johnson had to perpetrate to make
15049 the original Yacc fit in a PDP-11.)
15051 Named references, semantic predicates, @code{%locations},
15052 @code{%glr-parser}, @code{%printer}, %destructor, dumps to DOT,
15053 @code{%parse-param}, @code{%lex-param}, and dumps to XSLT, LAC, and IELR(1)
15054 generation are new in Bison.
15056 Bison also has many features to support C++ that were not present in the
15057 ancestral Yacc or Byson.
15059 Bison obsolesced all previous Yacc variants and workalikes generating C by
15062 @node Other Ungulates
15063 @section Other Ungulates
15065 The Yacc concept has frequently been ported to other languages. Some of the
15066 early ports are extinct along with the languages that hosted them; others
15067 have been superseded by parser skeletons shipped with Bison.
15069 However, independent implementations persist. One of the best-known
15070 still in use is David Beazley's ``PLY'' (Python Lex-Yacc) for
15071 Python. Another is goyacc, supporting the Go language. An ``ocamlyacc''
15072 is shipped as part of the Ocaml compiler suite.
15074 @c ================================================= Version Compatibility
15077 @chapter Bison Version Compatibility: Best Practices
15079 @cindex compatibility
15081 Bison provides a Yacc compatibility mode in which it strives to conform with
15082 the POSIX standard. Grammar files which are written to the POSIX standard, and
15083 do not take advantage of any of the special capabilities of Bison, should
15084 work with many versions of Bison without modification.
15086 All other features of Bison are particular to Bison, and are changing. Bison
15087 is actively maintained and continuously evolving. It should come as no
15088 surprise that an older version of Bison will not accept Bison source code which
15089 uses newer features that do no not exist at all in the older Bison.
15090 Regrettably, in spite of reasonable effort to maintain compatibility, the
15091 reverse situation may also occur: it may happen that code developed using an
15092 older version of Bison does not build with a newer version of Bison without
15095 Because Bison is a code generation tool, it is possible to retain its output
15096 and distribute that to the users of the program. The users are then not
15097 required to have Bison installed at all, only an implementation of the
15098 programming language, such as C, which is required for processing the generated
15101 It is the output of Bison that is intended to be of the utmost portability.
15102 So, that is to say, whereas the Bison grammar source code may have a dependency
15103 on specific versions of Bison, the generated parser from any version of Bison
15104 should work with with a large number of implementations of C, or whatever
15105 language is applicable.
15107 The recommended best practice for using Bison (in the context of software that
15108 is distributed in source code form) is to ship the generated parser to the
15109 downstream users. Only those downstream users who engage in active development
15110 of the program who need to make changes to the grammar file need to have Bison
15111 installed at all, and those users can install the specific version of Bison
15114 Following this recommended practice also makes it possible to use a more recent
15115 Bison than what is available to users through operating system distributions,
15116 thereby taking advantage of the latest techniques that Bison allows.
15118 Some features of Bison have been, or are being adopted into other Yacc-like
15119 programs. Therefore it might seem that is a good idea to write grammar code
15120 which targets multiple implementations, similarly to the way C programs are
15121 often written to target multiple compilers and language versions. Other than
15122 the Yacc subset described by POSIX, the Bison language is not rigorously
15123 standardized. When a Bison feature is adopted by another parser generator, it
15124 may be initially compatible with that version of Bison on which it was based,
15125 but the compatibility may degrade going forward. Developers who strive to make
15126 their Bison code simultaneously compatible with other parser generators are
15127 encouraged to nevertheless use specific versions of all generators, and still
15128 follow the recommended practice of shipping generated output. For example,
15129 a project can internally maintain compatibility with multiple generators,
15130 and choose the output of a particular one to ship to the users. Or else,
15131 the project could ship all of the outputs, arranging for a way for the user
15132 to specify which one is used to build the program.
15134 @c ================================================= FAQ
15137 @chapter Frequently Asked Questions
15138 @cindex frequently asked questions
15141 Several questions about Bison come up occasionally. Here some of them
15145 * Memory Exhausted:: Breaking the Stack Limits
15146 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
15147 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
15148 * Implementing Gotos/Loops:: Control Flow in the Calculator
15149 * Multiple start-symbols:: Factoring closely related grammars
15150 * Secure? Conform?:: Is Bison POSIX safe?
15151 * Enabling Relocatability:: Moving Bison/using it through network shares
15152 * I can't build Bison:: Troubleshooting
15153 * Where can I find help?:: Troubleshouting
15154 * Bug Reports:: Troublereporting
15155 * More Languages:: Parsers in C++, Java, and so on
15156 * Beta Testing:: Experimenting development versions
15157 * Mailing Lists:: Meeting other Bison users
15160 @node Memory Exhausted
15161 @section Memory Exhausted
15164 My parser returns with error with a @samp{memory exhausted}
15165 message. What can I do?
15168 This question is already addressed elsewhere, see @ref{Recursion}.
15170 @node How Can I Reset the Parser
15171 @section How Can I Reset the Parser
15173 The following phenomenon has several symptoms, resulting in the
15174 following typical questions:
15177 I invoke @code{yyparse} several times, and on correct input it works
15178 properly; but when a parse error is found, all the other calls fail
15179 too. How can I reset the error flag of @code{yyparse}?
15186 My parser includes support for an @samp{#include}-like feature, in which
15187 case I run @code{yyparse} from @code{yyparse}. This fails although I did
15188 specify @samp{%define api.pure full}.
15191 These problems typically come not from Bison itself, but from
15192 Lex-generated scanners. Because these scanners use large buffers for
15193 speed, they might not notice a change of input file. As a
15194 demonstration, consider the following source file,
15195 @file{first-line.l}:
15201 #include <stdlib.h>
15205 .*\n ECHO; return 1;
15209 yyparse (char const *file)
15211 yyin = fopen (file, "r");
15215 exit (EXIT_FAILURE);
15219 /* One token only. */
15221 if (fclose (yyin) != 0)
15224 exit (EXIT_FAILURE);
15242 If the file @file{input} contains
15250 then instead of getting the first line twice, you get:
15253 $ @kbd{flex -ofirst-line.c first-line.l}
15254 $ @kbd{gcc -ofirst-line first-line.c -ll}
15255 $ @kbd{./first-line}
15260 Therefore, whenever you change @code{yyin}, you must tell the
15261 Lex-generated scanner to discard its current buffer and switch to the
15262 new one. This depends upon your implementation of Lex; see its
15263 documentation for more. For Flex, it suffices to call
15264 @samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
15265 Flex-generated scanner needs to read from several input streams to
15266 handle features like include files, you might consider using Flex
15267 functions like @samp{yy_switch_to_buffer} that manipulate multiple
15270 If your Flex-generated scanner uses start conditions (@pxref{Start
15271 conditions, , Start conditions, flex, The Flex Manual}), you might
15272 also want to reset the scanner's state, i.e., go back to the initial
15273 start condition, through a call to @samp{BEGIN (0)}.
15275 @node Strings are Destroyed
15276 @section Strings are Destroyed
15279 My parser seems to destroy old strings, or maybe it loses track of
15280 them. Instead of reporting @samp{"foo", "bar"}, it reports
15281 @samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
15284 This error is probably the single most frequent ``bug report'' sent to
15285 Bison lists, but is only concerned with a misunderstanding of the role
15286 of the scanner. Consider the following Lex code:
15292 char *yylval = NULL;
15297 .* yylval = yytext; return 1;
15305 /* Similar to using $1, $2 in a Bison action. */
15306 char *fst = (yylex (), yylval);
15307 char *snd = (yylex (), yylval);
15308 printf ("\"%s\", \"%s\"\n", fst, snd);
15314 If you compile and run this code, you get:
15317 $ @kbd{flex -osplit-lines.c split-lines.l}
15318 $ @kbd{gcc -osplit-lines split-lines.c -ll}
15319 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
15325 this is because @code{yytext} is a buffer provided for @emph{reading}
15326 in the action, but if you want to keep it, you have to duplicate it
15327 (e.g., using @code{strdup}). Note that the output may depend on how
15328 your implementation of Lex handles @code{yytext}. For instance, when
15329 given the Lex compatibility option @option{-l} (which triggers the
15330 option @samp{%array}) Flex generates a different behavior:
15333 $ @kbd{flex -l -osplit-lines.c split-lines.l}
15334 $ @kbd{gcc -osplit-lines split-lines.c -ll}
15335 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
15340 @node Implementing Gotos/Loops
15341 @section Implementing Gotos/Loops
15344 My simple calculator supports variables, assignments, and functions,
15345 but how can I implement gotos, or loops?
15348 Although very pedagogical, the examples included in the document blur
15349 the distinction to make between the parser---whose job is to recover
15350 the structure of a text and to transmit it to subsequent modules of
15351 the program---and the processing (such as the execution) of this
15352 structure. This works well with so called straight line programs,
15353 i.e., precisely those that have a straightforward execution model:
15354 execute simple instructions one after the others.
15356 @cindex abstract syntax tree
15358 If you want a richer model, you will probably need to use the parser
15359 to construct a tree that does represent the structure it has
15360 recovered; this tree is usually called the @dfn{abstract syntax tree},
15361 or @dfn{AST} for short. Then, walking through this tree,
15362 traversing it in various ways, will enable treatments such as its
15363 execution or its translation, which will result in an interpreter or a
15366 This topic is way beyond the scope of this manual, and the reader is
15367 invited to consult the dedicated literature.
15370 @node Multiple start-symbols
15371 @section Multiple start-symbols
15374 I have several closely related grammars, and I would like to share their
15375 implementations. In fact, I could use a single grammar but with
15376 multiple entry points.
15379 Bison does not support multiple start-symbols, but there is a very
15380 simple means to simulate them. If @code{foo} and @code{bar} are the two
15381 pseudo start-symbols, then introduce two new tokens, say
15382 @code{START_FOO} and @code{START_BAR}, and use them as switches from the
15386 %token START_FOO START_BAR;
15393 These tokens prevents the introduction of new conflicts. As far as the
15394 parser goes, that is all that is needed.
15396 Now the difficult part is ensuring that the scanner will send these tokens
15397 first. If your scanner is hand-written, that should be straightforward. If
15398 your scanner is generated by Lex, them there is simple means to do it:
15399 recall that anything between @samp{%@{ ... %@}} after the first @code{%%} is
15400 copied verbatim in the top of the generated @code{yylex} function. Make
15401 sure a variable @code{start_token} is available in the scanner (e.g., a
15402 global variable or using @code{%lex-param} etc.), and use the following:
15405 /* @r{Prologue.} */
15410 int t = start_token;
15415 /* @r{The rules.} */
15419 @node Secure? Conform?
15420 @section Secure? Conform?
15423 Is Bison secure? Does it conform to POSIX?
15426 If you're looking for a guarantee or certification, we don't provide it.
15427 However, Bison is intended to be a reliable program that conforms to the
15428 POSIX specification for Yacc. If you run into problems, please send us a
15431 @include relocatable.texi
15433 @node I can't build Bison
15434 @section I can't build Bison
15437 I can't build Bison because @command{make} complains that
15438 @code{msgfmt} is not found.
15442 Like most GNU packages with internationalization support, that feature
15443 is turned on by default. If you have problems building in the @file{po}
15444 subdirectory, it indicates that your system's internationalization
15445 support is lacking. You can re-configure Bison with
15446 @option{--disable-nls} to turn off this support, or you can install GNU
15447 gettext from @url{https://ftp.gnu.org/gnu/gettext/} and re-configure
15448 Bison. See the file @file{ABOUT-NLS} for more information.
15451 I can't build Bison because my C compiler is too old.
15454 Except for GLR parsers (which require C99), the C code that Bison generates
15455 requires only C89 or later. However, Bison itself requires common C99
15456 features such as declarations after statements. Bison's @code{configure}
15457 script attempts to enable C99 (or later) support on compilers that default
15458 to pre-C99. If your compiler lacks these C99 features entirely, GCC may
15459 well be a better choice; or you can try upgrading to your compiler's latest
15462 @node Where can I find help?
15463 @section Where can I find help?
15466 I'm having trouble using Bison. Where can I find help?
15469 First, read this fine manual. Beyond that, you can send mail to
15470 @email{help-bison@@gnu.org}. This mailing list is intended to be
15471 populated with people who are willing to answer questions about using
15472 and installing Bison. Please keep in mind that (most of) the people on
15473 the list have aspects of their lives which are not related to Bison (!),
15474 so you may not receive an answer to your question right away. This can
15475 be frustrating, but please try not to honk them off; remember that any
15476 help they provide is purely voluntary and out of the kindness of their
15480 @section Bug Reports
15483 I found a bug. What should I include in the bug report?
15486 Before sending a bug report, make sure you are using the latest
15487 version. Check @url{https://ftp.gnu.org/pub/gnu/bison/} or one of its
15488 mirrors. Be sure to include the version number in your bug report. If
15489 the bug is present in the latest version but not in a previous version,
15490 try to determine the most recent version which did not contain the bug.
15492 If the bug is parser-related, you should include the smallest grammar
15493 you can which demonstrates the bug. The grammar file should also be
15494 complete (i.e., I should be able to run it through Bison without having
15495 to edit or add anything). The smaller and simpler the grammar, the
15496 easier it will be to fix the bug.
15498 Include information about your compilation environment, including your
15499 operating system's name and version and your compiler's name and
15500 version. If you have trouble compiling, you should also include a
15501 transcript of the build session, starting with the invocation of
15502 @code{configure}. Depending on the nature of the bug, you may be asked to
15503 send additional files as well (such as @file{config.h} or @file{config.cache}).
15505 Patches are most welcome, but not required. That is, do not hesitate to
15506 send a bug report just because you cannot provide a fix.
15508 Send bug reports to @email{bug-bison@@gnu.org}.
15510 @node More Languages
15511 @section More Languages
15514 Will Bison ever have C++ and Java support? How about @var{insert your
15515 favorite language here}?
15518 C++, D and Java support is there now, and is documented. We'd love to add other
15519 languages; contributions are welcome.
15522 @section Beta Testing
15525 What is involved in being a beta tester?
15528 It's not terribly involved. Basically, you would download a test
15529 release, compile it, and use it to build and run a parser or two. After
15530 that, you would submit either a bug report or a message saying that
15531 everything is okay. It is important to report successes as well as
15532 failures because test releases eventually become mainstream releases,
15533 but only if they are adequately tested. If no one tests, development is
15534 essentially halted.
15536 Beta testers are particularly needed for operating systems to which the
15537 developers do not have easy access. They currently have easy access to
15538 recent GNU/Linux and Solaris versions. Reports about other operating
15539 systems are especially welcome.
15541 @node Mailing Lists
15542 @section Mailing Lists
15545 How do I join the help-bison and bug-bison mailing lists?
15548 See @url{http://lists.gnu.org/}.
15550 @c ================================================= Table of Symbols
15552 @node Table of Symbols
15553 @appendix Bison Symbols
15554 @cindex Bison symbols, table of
15555 @cindex symbols in Bison, table of
15557 @deffn {Variable} @@$
15558 In an action, the location of the left-hand side of the rule.
15559 @xref{Tracking Locations}.
15562 @deffn {Variable} @@@var{n}
15563 @deffnx {Symbol} @@@var{n}
15564 In an action, the location of the @var{n}-th symbol of the right-hand side
15565 of the rule. @xref{Tracking Locations}.
15567 In a grammar, the Bison-generated nonterminal symbol for a midrule action
15568 with a semantic value. @xref{Midrule Action Translation}.
15571 @deffn {Variable} @@@var{name}
15572 @deffnx {Variable} @@[@var{name}]
15573 In an action, the location of a symbol addressed by @var{name}.
15574 @xref{Tracking Locations}.
15577 @deffn {Symbol} $@@@var{n}
15578 In a grammar, the Bison-generated nonterminal symbol for a midrule action
15579 with no semantics value. @xref{Midrule Action Translation}.
15582 @deffn {Variable} $$
15583 In an action, the semantic value of the left-hand side of the rule.
15587 @deffn {Variable} $@var{n}
15588 In an action, the semantic value of the @var{n}-th symbol of the
15589 right-hand side of the rule. @xref{Actions}.
15592 @deffn {Variable} $@var{name}
15593 @deffnx {Variable} $[@var{name}]
15594 In an action, the semantic value of a symbol addressed by @var{name}.
15598 @deffn {Delimiter} %%
15599 Delimiter used to separate the grammar rule section from the
15600 Bison declarations section or the epilogue.
15601 @xref{Grammar Layout}.
15604 @c Don't insert spaces, or check the DVI output.
15605 @deffn {Delimiter} %@{@var{code}%@}
15606 All code listed between @samp{%@{} and @samp{%@}} is copied verbatim
15607 to the parser implementation file. Such code forms the prologue of
15608 the grammar file. @xref{Grammar Outline}.
15611 @deffn {Directive} %?@{@var{expression}@}
15612 Predicate actions. This is a type of action clause that may appear in
15613 rules. The expression is evaluated, and if false, causes a syntax error. In
15614 GLR parsers during nondeterministic operation,
15615 this silently causes an alternative parse to die. During deterministic
15616 operation, it is the same as the effect of YYERROR.
15617 @xref{Semantic Predicates}.
15620 @deffn {Construct} /* @dots{} */
15621 @deffnx {Construct} // @dots{}
15622 Comments, as in C/C++.
15625 @deffn {Delimiter} :
15626 Separates a rule's result from its components. @xref{Rules}.
15629 @deffn {Delimiter} ;
15630 Terminates a rule. @xref{Rules}.
15633 @deffn {Delimiter} |
15634 Separates alternate rules for the same result nonterminal.
15638 @deffn {Directive} <*>
15639 Used to define a default tagged @code{%destructor} or default tagged
15642 @xref{Destructor Decl}.
15645 @deffn {Directive} <>
15646 Used to define a default tagless @code{%destructor} or default tagless
15649 @xref{Destructor Decl}.
15652 @deffn {Symbol} $accept
15653 The predefined nonterminal whose only rule is @samp{$accept: @var{start}
15654 $end}, where @var{start} is the start symbol. @xref{Start Decl}. It cannot
15655 be used in the grammar.
15658 @deffn {Directive} %code @{@var{code}@}
15659 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
15660 Insert @var{code} verbatim into the output parser source at the
15661 default location or at the location specified by @var{qualifier}.
15662 @xref{%code Summary}.
15665 @deffn {Directive} %debug
15666 Equip the parser for debugging. @xref{Decl Summary}.
15670 @deffn {Directive} %default-prec
15671 Assign a precedence to rules that lack an explicit @samp{%prec}
15672 modifier. @xref{Contextual Precedence}.
15676 @deffn {Directive} %define @var{variable}
15677 @deffnx {Directive} %define @var{variable} @var{value}
15678 @deffnx {Directive} %define @var{variable} @{@var{value}@}
15679 @deffnx {Directive} %define @var{variable} "@var{value}"
15680 Define a variable to adjust Bison's behavior. @xref{%define Summary}.
15683 @deffn {Directive} %defines
15684 @deffnx {Directive} %defines @var{defines-file}
15685 Historical name for @code{%header}.
15686 @xref{Decl Summary}.
15689 @deffn {Directive} %destructor
15690 Specify how the parser should reclaim the memory associated to
15691 discarded symbols. @xref{Destructor Decl}.
15694 @deffn {Directive} %dprec
15695 Bison declaration to assign a precedence to a rule that is used at parse
15696 time to resolve reduce/reduce conflicts. @xref{GLR Parsers}.
15699 @deffn {Directive} %empty
15700 Bison declaration to declare make explicit that a rule has an empty
15701 right-hand side. @xref{Empty Rules}.
15704 @deffn {Symbol} $end
15705 The predefined token marking the end of the token stream. It cannot be
15706 used in the grammar.
15709 @deffn {Symbol} error
15710 A token name reserved for error recovery. This token may be used in
15711 grammar rules so as to allow the Bison parser to recognize an error in
15712 the grammar without halting the process. In effect, a sentence
15713 containing an error may be recognized as valid. On a syntax error, the
15714 token @code{error} becomes the current lookahead token. Actions
15715 corresponding to @code{error} are then executed, and the lookahead
15716 token is reset to the token that originally caused the violation.
15717 @xref{Error Recovery}.
15720 @deffn {Directive} %error-verbose
15721 An obsolete directive standing for @samp{%define parse.error verbose}.
15724 @deffn {Directive} %file-prefix "@var{prefix}"
15725 Bison declaration to set the prefix of the output files. @xref{Decl
15729 @deffn {Directive} %glr-parser
15730 Bison declaration to produce a GLR parser. @xref{GLR
15734 @deffn {Directive} %header
15735 Bison declaration to create a parser header file, which is usually
15736 meant for the scanner. @xref{Decl Summary}.
15739 @deffn {Directive} %header @var{header-file}
15740 Same as above, but save in the file @var{header-file}.
15741 @xref{Decl Summary}.
15744 @deffn {Directive} %initial-action
15745 Run user code before parsing. @xref{Initial Action Decl}.
15748 @deffn {Directive} %language
15749 Specify the programming language for the generated parser.
15750 @xref{Decl Summary}.
15753 @deffn {Directive} %left
15754 Bison declaration to assign precedence and left associativity to token(s).
15755 @xref{Precedence Decl}.
15758 @deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
15759 Bison declaration to specifying additional arguments that
15760 @code{yylex} should accept. @xref{Pure Calling}.
15763 @deffn {Directive} %merge
15764 Bison declaration to assign a merging function to a rule. If there is a
15765 reduce/reduce conflict with a rule having the same merging function, the
15766 function is applied to the two semantic values to get a single result.
15767 @xref{GLR Parsers}.
15770 @deffn {Directive} %name-prefix "@var{prefix}"
15771 Obsoleted by the @code{%define} variable @code{api.prefix} (@pxref{Multiple
15774 Rename the external symbols (variables and functions) used in the parser so
15775 that they start with @var{prefix} instead of @samp{yy}. Contrary to
15776 @code{api.prefix}, do no rename types and macros.
15778 The precise list of symbols renamed in C parsers is @code{yyparse},
15779 @code{yylex}, @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yychar},
15780 @code{yydebug}, and (if locations are used) @code{yylloc}. If you use a
15781 push parser, @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
15782 @code{yypstate_new} and @code{yypstate_delete} will also be renamed. For
15783 example, if you use @samp{%name-prefix "c_"}, the names become
15784 @code{c_parse}, @code{c_lex}, and so on. For C++ parsers, see the
15785 @code{%define api.namespace} documentation in this section.
15790 @deffn {Directive} %no-default-prec
15791 Do not assign a precedence to rules that lack an explicit @samp{%prec}
15792 modifier. @xref{Contextual Precedence}.
15796 @deffn {Directive} %no-lines
15797 Bison declaration to avoid generating @code{#line} directives in the
15798 parser implementation file. @xref{Decl Summary}.
15801 @deffn {Directive} %nonassoc
15802 Bison declaration to assign precedence and nonassociativity to token(s).
15803 @xref{Precedence Decl}.
15806 @deffn {Directive} %nterm
15807 Bison declaration to declare nonterminals. @xref{Type Decl}.
15810 @deffn {Directive} %output "@var{file}"
15811 Bison declaration to set the name of the parser implementation file.
15812 @xref{Decl Summary}.
15815 @deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
15816 Bison declaration to specify additional arguments that both
15817 @code{yylex} and @code{yyparse} should accept. @xref{Parser Function}.
15820 @deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
15821 Bison declaration to specify additional arguments that @code{yyparse}
15822 should accept. @xref{Parser Function}.
15825 @deffn {Directive} %prec
15826 Bison declaration to assign a precedence to a specific rule.
15827 @xref{Contextual Precedence}.
15830 @deffn {Directive} %precedence
15831 Bison declaration to assign precedence to token(s), but no associativity
15832 @xref{Precedence Decl}.
15835 @deffn {Directive} %pure-parser
15836 Deprecated version of @samp{%define api.pure} (@pxref{%define
15837 Summary}), for which Bison is more careful to warn about
15838 unreasonable usage.
15841 @deffn {Directive} %require "@var{version}"
15842 Require version @var{version} or higher of Bison. @xref{Require Decl}.
15845 @deffn {Directive} %right
15846 Bison declaration to assign precedence and right associativity to token(s).
15847 @xref{Precedence Decl}.
15850 @deffn {Directive} %skeleton
15851 Specify the skeleton to use; usually for development.
15852 @xref{Decl Summary}.
15855 @deffn {Directive} %start
15856 Bison declaration to specify the start symbol. @xref{Start Decl}.
15859 @deffn {Directive} %token
15860 Bison declaration to declare token(s) without specifying precedence.
15864 @deffn {Directive} %token-table
15865 Bison declaration to include a token name table in the parser implementation
15866 file. @xref{Decl Summary}.
15869 @deffn {Directive} %type
15870 Bison declaration to declare symbol value types. @xref{Type Decl}.
15873 @deffn {Symbol} $undefined
15874 The predefined token onto which all undefined values returned by
15875 @code{yylex} are mapped. It cannot be used in the grammar, rather, use
15879 @deffn {Directive} %union
15880 Bison declaration to specify several possible data types for semantic
15881 values. @xref{Union Decl}.
15884 @deffn {Macro} YYABORT
15885 Macro to pretend that an unrecoverable syntax error has occurred, by making
15886 @code{yyparse} return 1 immediately. The error reporting function
15887 @code{yyerror} is not called. @xref{Parser Function}.
15889 For Java parsers, this functionality is invoked using @code{return YYABORT;}
15893 @deffn {Macro} YYACCEPT
15894 Macro to pretend that a complete utterance of the language has been
15895 read, by making @code{yyparse} return 0 immediately.
15896 @xref{Parser Function}.
15898 For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
15902 @deffn {Macro} YYBACKUP
15903 Macro to discard a value from the parser stack and fake a lookahead
15904 token. @xref{Action Features}.
15907 @deffn {Macro} YYBISON
15908 The version of Bison as an integer, for instance 30704 for version 3.7.4.
15909 Defined in @file{yacc.c} only. Before version 3.7.4, @code{YYBISON} was
15913 @deffn {Variable} yychar
15914 External integer variable that contains the integer value of the
15915 lookahead token. (In a pure parser, it is a local variable within
15916 @code{yyparse}.) Error-recovery rule actions may examine this variable.
15917 @xref{Action Features}.
15920 @deffn {Variable} yyclearin
15921 Macro used in error-recovery rule actions. It clears the previous
15922 lookahead token. @xref{Error Recovery}.
15925 @deffn {Macro} YYDEBUG
15926 Macro to define to equip the parser with tracing code. @xref{Tracing}.
15929 @deffn {Variable} yydebug
15930 External integer variable set to zero by default. If @code{yydebug}
15931 is given a nonzero value, the parser will output information on input
15932 symbols and parser action. @xref{Tracing}.
15935 @deffn {Value} YYEMPTY
15936 The pseudo token kind when there is no lookahead token.
15939 @deffn {Value} YYEOF
15940 The token kind denoting is the end of the input stream.
15943 @deffn {Macro} yyerrok
15944 Macro to cause parser to recover immediately to its normal mode
15945 after a syntax error. @xref{Error Recovery}.
15948 @deffn {Macro} YYERROR
15949 Cause an immediate syntax error. This statement initiates error
15950 recovery just as if the parser itself had detected an error; however, it
15951 does not call @code{yyerror}, and does not print any message. If you
15952 want to print an error message, call @code{yyerror} explicitly before
15953 the @samp{YYERROR;} statement. @xref{Error Recovery}.
15955 For Java parsers, this functionality is invoked using @code{return YYERROR;}
15959 @deffn {Function} yyerror
15960 User-supplied function to be called by @code{yyparse} on error.
15961 @xref{Error Reporting Function}.
15964 @deffn {Macro} YYFPRINTF
15965 Macro used to output run-time traces in C.
15966 @xref{Enabling Traces}.
15969 @deffn {Macro} YYINITDEPTH
15970 Macro for specifying the initial size of the parser stack.
15971 @xref{Memory Management}.
15974 @deffn {Function} yylex
15975 User-supplied lexical analyzer function, called with no arguments to get
15976 the next token. @xref{Lexical}.
15979 @deffn {Variable} yylloc
15980 External variable in which @code{yylex} should place the line and column
15981 numbers associated with a token. (In a pure parser, it is a local
15982 variable within @code{yyparse}, and its address is passed to
15984 You can ignore this variable if you don't use the @samp{@@} feature in the
15986 @xref{Token Locations}.
15987 In semantic actions, it stores the location of the lookahead token.
15988 @xref{Actions and Locations}.
15991 @deffn {Type} YYLTYPE
15992 Data type of @code{yylloc}. By default in C, a structure with four members
15993 (start/end line/column). @xref{Location Type}.
15996 @deffn {Variable} yylval
15997 External variable in which @code{yylex} should place the semantic
15998 value associated with a token. (In a pure parser, it is a local
15999 variable within @code{yyparse}, and its address is passed to
16001 @xref{Token Values}.
16002 In semantic actions, it stores the semantic value of the lookahead token.
16006 @deffn {Macro} YYMAXDEPTH
16007 Macro for specifying the maximum size of the parser stack. @xref{Memory
16011 @deffn {Variable} yynerrs
16012 Global variable which Bison increments each time it reports a syntax error.
16013 (In a pure parser, it is a local variable within @code{yyparse}. In a
16014 pure push parser, it is a member of @code{yypstate}.)
16015 @xref{Error Reporting Function}.
16018 @deffn {Macro} YYNOMEM
16019 Macro to pretend that memory is exhausted, by making @code{yyparse} return 2
16020 immediately. The error reporting function @code{yyerror} is called.
16021 @xref{Parser Function}.
16024 @deffn {Function} yyparse
16025 The parser function produced by Bison; call this function to start
16026 parsing. @xref{Parser Function}.
16029 @deffn {Macro} YYPRINT
16030 Macro used to output token semantic values. For @file{yacc.c} only.
16031 Deprecated, use @code{%printer} instead (@pxref{Printer Decl}).
16032 @xref{The YYPRINT Macro}.
16035 @deffn {Function} yypstate_delete
16036 The function to delete a parser instance, produced by Bison in push mode;
16037 call this function to delete the memory associated with a parser.
16038 @xref{yypstate_delete,,@code{yypstate_delete}}. Does nothing when called
16039 with a null pointer.
16042 @deffn {Function} yypstate_new
16043 The function to create a parser instance, produced by Bison in push mode;
16044 call this function to create a new parser.
16045 @xref{yypstate_new,,@code{yypstate_new}}.
16048 @deffn {Function} yypull_parse
16049 The parser function produced by Bison in push mode; call this function to
16050 parse the rest of the input stream.
16051 @xref{yypull_parse,,@code{yypull_parse}}.
16054 @deffn {Function} yypush_parse
16055 The parser function produced by Bison in push mode; call this function to
16056 parse a single token.
16057 @xref{yypush_parse,,@code{yypush_parse}}.
16060 @deffn {Macro} YYRECOVERING
16061 The expression @code{YYRECOVERING ()} yields 1 when the parser
16062 is recovering from a syntax error, and 0 otherwise.
16063 @xref{Action Features}.
16066 @deffn {Macro} YYSTACK_USE_ALLOCA
16067 Macro used to control the use of @code{alloca} when the
16068 deterministic parser in C needs to extend its stacks. If defined to 0,
16069 the parser will use @code{malloc} to extend its stacks and memory exhaustion
16070 occurs if @code{malloc} fails (@pxref{Memory Management}). If defined to
16071 1, the parser will use @code{alloca}. Values other than 0 and 1 are
16072 reserved for future Bison extensions. If not defined,
16073 @code{YYSTACK_USE_ALLOCA} defaults to 0.
16075 In the all-too-common case where your code may run on a host with a
16076 limited stack and with unreliable stack-overflow checking, you should
16077 set @code{YYMAXDEPTH} to a value that cannot possibly result in
16078 unchecked stack overflow on any of your target hosts when
16079 @code{alloca} is called. You can inspect the code that Bison
16080 generates in order to determine the proper numeric values. This will
16081 require some expertise in low-level implementation details.
16084 @deffn {Type} YYSTYPE
16085 Deprecated in favor of the @code{%define} variable @code{api.value.type}.
16086 Data type of semantic values; @code{int} by default.
16090 @deffn {Type} yysymbol_kind_t
16091 An enum of all the symbols, tokens and nonterminals, of the grammar.
16092 @xref{Syntax Error Reporting Function}. The symbol kinds are used
16093 internally by the parser, and should not be confused with the token kinds:
16094 the symbol kind of a terminal symbol is not equal to its token kind! (Unless
16095 @samp{%define api.token.raw} was used.)
16098 @deffn {Type} yytoken_kind_t
16099 An enum of all the @dfn{token kinds} declared with @code{%token}
16100 (@pxref{Token Decl}). These are the return values for @code{yylex}. They
16101 should not be confused with the @emph{symbol kinds}, used internally by the
16105 @deffn {Value} YYUNDEF
16106 The token kind denoting an unknown token.
16115 @item Accepting state
16116 A state whose only action is the accept action.
16117 The accepting state is thus a consistent state.
16118 @xref{Understanding}.
16120 @item Backus-Naur Form (BNF; also called ``Backus Normal Form'')
16121 Formal method of specifying context-free grammars originally proposed
16122 by John Backus, and slightly improved by Peter Naur in his 1960-01-02
16123 committee document contributing to what became the Algol 60 report.
16124 @xref{Language and Grammar}.
16126 @item Consistent state
16127 A state containing only one possible action. @xref{Default Reductions}.
16129 @item Context-free grammars
16130 Grammars specified as rules that can be applied regardless of context.
16131 Thus, if there is a rule which says that an integer can be used as an
16132 expression, integers are allowed @emph{anywhere} an expression is
16133 permitted. @xref{Language and Grammar}.
16135 @item Counterexample
16136 A sequence of tokens and/or nonterminals, with one dot, that demonstrates a
16137 conflict. The dot marks the place where the conflict occurs.
16139 @cindex unifying counterexample
16140 @cindex counterexample, unifying
16141 @cindex nonunifying counterexample
16142 @cindex counterexample, nonunifying
16143 A @emph{unifying} counterexample is a single string that has two different
16144 parses; its existence proves that the grammar is ambiguous. When a unifying
16145 counterexample cannot be found in reasonable time, a @emph{nonunifying}
16146 counterexample is built: @emph{two} different string sharing the prefix up
16149 @xref{Counterexamples}
16151 @item Default reduction
16152 The reduction that a parser should perform if the current parser state
16153 contains no other action for the lookahead token. In permitted parser
16154 states, Bison declares the reduction with the largest lookahead set to be
16155 the default reduction and removes that lookahead set. @xref{Default
16158 @item Defaulted state
16159 A consistent state with a default reduction. @xref{Default Reductions}.
16161 @item Dynamic allocation
16162 Allocation of memory that occurs during execution, rather than at
16163 compile time or on entry to a function.
16166 Analogous to the empty set in set theory, the empty string is a
16167 character string of length zero.
16169 @item Finite-state stack machine
16170 A ``machine'' that has discrete states in which it is said to exist at
16171 each instant in time. As input to the machine is processed, the
16172 machine moves from state to state as specified by the logic of the
16173 machine. In the case of the parser, the input is the language being
16174 parsed, and the states correspond to various stages in the grammar
16175 rules. @xref{Algorithm}.
16177 @item Generalized LR (GLR)
16178 A parsing algorithm that can handle all context-free grammars, including those
16179 that are not LR(1). It resolves situations that Bison's
16180 deterministic parsing
16181 algorithm cannot by effectively splitting off multiple parsers, trying all
16182 possible parsers, and discarding those that fail in the light of additional
16183 right context. @xref{Generalized LR Parsing}.
16186 A language construct that is (in general) grammatically divisible;
16187 for example, `expression' or `declaration' in C@.
16188 @xref{Language and Grammar}.
16190 @item IELR(1) (Inadequacy Elimination LR(1))
16191 A minimal LR(1) parser table construction algorithm. That is, given any
16192 context-free grammar, IELR(1) generates parser tables with the full
16193 language-recognition power of canonical LR(1) but with nearly the same
16194 number of parser states as LALR(1). This reduction in parser states is
16195 often an order of magnitude. More importantly, because canonical LR(1)'s
16196 extra parser states may contain duplicate conflicts in the case of non-LR(1)
16197 grammars, the number of conflicts for IELR(1) is often an order of magnitude
16198 less as well. This can significantly reduce the complexity of developing a
16199 grammar. @xref{LR Table Construction}.
16201 @item Infix operator
16202 An arithmetic operator that is placed between the operands on which it
16203 performs some operation.
16206 A continuous flow of data between devices or programs.
16209 ``Token'' and ``symbol'' are each overloaded to mean either a grammar symbol
16210 (kind) or all parse info (kind, value, location) associated with occurrences
16211 of that grammar symbol from the input. To disambiguate,
16215 we use ``token kind'' and ``symbol kind'' to mean both grammar symbols and
16216 the values that represent them in a base programming language (C, C++,
16217 etc.). The names of the types of these values are typically
16218 @code{token_kind_t}, or @code{token_kind_type}, or @code{TokenKind},
16219 depending on the programming language.
16222 we use ``token'' and ``symbol'' without the word ``kind'' to mean parsed
16223 occurrences, and we append the word ``type'' to refer to the types that
16224 represent them in a base programming language.
16227 In summary: When you see ``kind'', interpret ``symbol'' or ``token'' to mean
16228 a @emph{grammar symbol}. When you don't see ``kind'' (including when you
16229 see ``type''), interpret ``symbol'' or ``token'' to mean a @emph{parsed
16232 @item LAC (Lookahead Correction)
16233 A parsing mechanism that fixes the problem of delayed syntax error
16234 detection, which is caused by LR state merging, default reductions, and the
16235 use of @code{%nonassoc}. Delayed syntax error detection results in
16236 unexpected semantic actions, initiation of error recovery in the wrong
16237 syntactic context, and an incorrect list of expected tokens in a verbose
16238 syntax error message. @xref{LAC}.
16240 @item Language construct
16241 One of the typical usage schemas of the language. For example, one of
16242 the constructs of the C language is the @code{if} statement.
16243 @xref{Language and Grammar}.
16245 @item Left associativity
16246 Operators having left associativity are analyzed from left to right:
16247 @samp{a+b+c} first computes @samp{a+b} and then combines with
16248 @samp{c}. @xref{Precedence}.
16250 @item Left recursion
16251 A rule whose result symbol is also its first component symbol; for
16252 example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion}.
16254 @item Left-to-right parsing
16255 Parsing a sentence of a language by analyzing it token by token from
16256 left to right. @xref{Algorithm}.
16258 @item Lexical analyzer (scanner)
16259 A function that reads an input stream and returns tokens one by one.
16262 @item Lexical tie-in
16263 A flag, set by actions in the grammar rules, which alters the way
16264 tokens are parsed. @xref{Lexical Tie-ins}.
16266 @item Literal string token
16267 A token which consists of two or more fixed characters. @xref{Symbols}.
16269 @item Lookahead token
16270 A token already read but not yet shifted. @xref{Lookahead}.
16273 The class of context-free grammars that Bison (like most other parser
16274 generators) can handle by default; a subset of LR(1).
16275 @xref{Mysterious Conflicts}.
16278 The class of context-free grammars in which at most one token of
16279 lookahead is needed to disambiguate the parsing of any piece of input.
16281 @item Nonterminal symbol
16282 A grammar symbol standing for a grammatical construct that can
16283 be expressed through rules in terms of smaller constructs; in other
16284 words, a construct that is not a token. @xref{Symbols}.
16287 A function that recognizes valid sentences of a language by analyzing
16288 the syntax structure of a set of tokens passed to it from a lexical
16291 @item Postfix operator
16292 An arithmetic operator that is placed after the operands upon which it
16293 performs some operation.
16296 Replacing a string of nonterminals and/or terminals with a single
16297 nonterminal, according to a grammar rule. @xref{Algorithm}.
16300 A reentrant subprogram is a subprogram which can be in invoked any
16301 number of times in parallel, without interference between the various
16302 invocations. @xref{Pure Decl}.
16304 @item Reverse Polish Notation
16305 A language in which all operators are postfix operators.
16307 @item Right recursion
16308 A rule whose result symbol is also its last component symbol; for
16309 example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion}.
16312 In computer languages, the semantics are specified by the actions
16313 taken for each instance of the language, i.e., the meaning of
16314 each statement. @xref{Semantics}.
16317 A parser is said to shift when it makes the choice of analyzing
16318 further input from the stream rather than reducing immediately some
16319 already-recognized rule. @xref{Algorithm}.
16321 @item Single-character literal
16322 A single character that is recognized and interpreted as is.
16323 @xref{Grammar in Bison}.
16326 The nonterminal symbol that stands for a complete valid utterance in
16327 the language being parsed. The start symbol is usually listed as the
16328 first nonterminal symbol in a language specification.
16332 A (finite) enumeration of the grammar symbols, as processed by the parser.
16336 A data structure where symbol names and associated data are stored during
16337 parsing to allow for recognition and use of existing information in repeated
16338 uses of a symbol. @xref{Multi-function Calc}.
16341 An error encountered during parsing of an input stream due to invalid
16342 syntax. @xref{Error Recovery}.
16344 @item Terminal symbol
16345 A grammar symbol that has no rules in the grammar and therefore is
16346 grammatically indivisible. The piece of text it represents is a token.
16347 @xref{Language and Grammar}.
16350 A basic, grammatically indivisible unit of a language. The symbol that
16351 describes a token in the grammar is a terminal symbol. The input of the
16352 Bison parser is a stream of tokens which comes from the lexical analyzer.
16356 A (finite) enumeration of the grammar terminals, as discriminated by the
16357 scanner. @xref{Symbols}.
16359 @item Unreachable state
16360 A parser state to which there does not exist a sequence of transitions from
16361 the parser's start state. A state can become unreachable during conflict
16362 resolution. @xref{Unreachable States}.
16365 @node GNU Free Documentation License
16366 @appendix GNU Free Documentation License
16371 @unnumbered Bibliography
16373 @c Please follow the following canvas to add more references.
16374 @c And keep sorted alphabetically.
16377 @anchor{Corbett 1984}
16378 @item [Corbett 1984]
16380 Robert Paul Corbett,
16382 Static Semantics in Compiler Error Recovery
16384 Ph.D. Dissertation, Report No. UCB/CSD 85/251,
16386 Department of Electrical Engineering and Computer Science, Compute Science
16387 Division, University of California, Berkeley, California
16391 @uref{http://xtf.lib.berkeley.edu/reports/TRWebData/accessPages/CSD-85-251.html}
16393 @anchor{Denny 2008}
16395 Joel E. Denny and Brian A. Malloy, IELR(1): Practical LR(1) Parser Tables
16396 for Non-LR(1) Grammars with Conflict Resolution, in @cite{Proceedings of the
16397 2008 ACM Symposium on Applied Computing} (SAC'08), ACM, New York, NY, USA,
16398 pp.@: 240--245. @uref{http://dx.doi.org/10.1145/1363686.1363747}
16400 @anchor{Denny 2010 May}
16401 @item [Denny 2010 May]
16402 Joel E. Denny, PSLR(1): Pseudo-Scannerless Minimal LR(1) for the
16403 Deterministic Parsing of Composite Languages, Ph.D. Dissertation, Clemson
16404 University, Clemson, SC, USA (May 2010).
16405 @uref{http://proquest.umi.com/pqdlink?did=2041473591&Fmt=7&clientId=79356&RQT=309&VName=PQD}
16407 @anchor{Denny 2010 November}
16408 @item [Denny 2010 November]
16409 Joel E. Denny and Brian A. Malloy, The IELR(1) Algorithm for Generating
16410 Minimal LR(1) Parser Tables for Non-LR(1) Grammars with Conflict Resolution,
16411 in @cite{Science of Computer Programming}, Vol.@: 75, Issue 11 (November
16412 2010), pp.@: 943--979. @uref{http://dx.doi.org/10.1016/j.scico.2009.08.001}
16414 @anchor{DeRemer 1982}
16415 @item [DeRemer 1982]
16416 Frank DeRemer and Thomas Pennello, Efficient Computation of LALR(1)
16417 Look-Ahead Sets, in @cite{ACM Transactions on Programming Languages and
16418 Systems}, Vol.@: 4, No.@: 4 (October 1982), pp.@:
16419 615--649. @uref{http://dx.doi.org/10.1145/69622.357187}
16421 @anchor{Isradisaikul 2015}
16422 @item [Isradisaikul 2015]
16423 Chinawat Isradisaikul, Andrew Myers,
16424 Finding Counterexamples from Parsing Conflicts,
16425 in @cite{Proceedings of the 36th ACM SIGPLAN Conference on
16426 Programming Language Design and Implementation} (PLDI '15),
16427 ACM, pp.@: 555--564.
16428 @uref{https://www.cs.cornell.edu/andru/papers/cupex/cupex.pdf}
16430 @anchor{Johnson 1978}
16431 @item [Johnson 1978]
16433 A portable compiler: theory and practice,
16434 in @cite{Proceedings of the 5th ACM SIGACT-SIGPLAN symposium on
16435 Principles of programming languages} (POPL '78),
16437 @uref{https://dx.doi.org/10.1145/512760.512771}.
16439 @anchor{Knuth 1965}
16441 Donald E. Knuth, On the Translation of Languages from Left to Right, in
16442 @cite{Information and Control}, Vol.@: 8, Issue 6 (December 1965), pp.@:
16443 607--639. @uref{http://dx.doi.org/10.1016/S0019-9958(65)90426-2}
16445 @anchor{Scott 2000}
16447 Elizabeth Scott, Adrian Johnstone, and Shamsa Sadaf Hussain,
16448 @cite{Tomita-Style Generalised LR Parsers}, Royal Holloway, University of
16449 London, Department of Computer Science, TR-00-12 (December 2000).
16450 @uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps}
16453 @node Index of Terms
16454 @unnumbered Index of Terms
16460 @c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
16461 @c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
16462 @c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
16463 @c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
16464 @c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
16465 @c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
16466 @c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
16467 @c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
16468 @c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
16469 @c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
16470 @c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
16471 @c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
16472 @c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
16473 @c LocalWords: symrec val tptr FUN func struct sym enum IEC syntaxes Byacc
16474 @c LocalWords: fun putsym getsym arith funs atan ptr malloc sizeof Lex pcc
16475 @c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
16476 @c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
16477 @c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
16478 @c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
16479 @c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
16480 @c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
16481 @c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
16482 @c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
16483 @c LocalWords: YYINITDEPTH stmts ref initdcl maybeasm notype Lookahead ctx
16484 @c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
16485 @c LocalWords: ypp yxx itemx tex leaderfill Troubleshouting sqrt Graphviz
16486 @c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
16487 @c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
16488 @c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
16489 @c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC
16490 @c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs sr
16491 @c LocalWords: yytokentype destructor multicharacter nonnull EBCDIC nterm LR's
16492 @c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
16493 @c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative Ph
16494 @c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
16495 @c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
16496 @c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
16497 @c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz ACM
16498 @c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
16499 @c LocalWords: multitable headitem hh basename Doxygen fno filename gdef de
16500 @c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
16501 @c LocalWords: Ctor defcv defcvx arg accessors CPP ifndef CALCXX YYerror
16502 @c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
16503 @c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
16504 @c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc PSLR
16505 @c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
16506 @c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
16507 @c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
16508 @c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
16509 @c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
16510 @c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt Corbett LALR's
16511 @c LocalWords: subdirectory Solaris nonassociativity perror schemas Malloy ints
16512 @c LocalWords: Scannerless ispell american ChangeLog smallexample CSTYPE CLTYPE
16513 @c LocalWords: clval CDEBUG cdebug deftypeopx yyterminate LocationType yyo
16514 @c LocalWords: parsers parser's documentencoding documentlanguage Wempty ss
16515 @c LocalWords: associativity subclasses precedences unresolvable runnable
16516 @c LocalWords: allocators subunit initializations unreferenced untyped dir
16517 @c LocalWords: errorVerbose subtype subtypes Wmidrule midrule's src rvalues
16518 @c LocalWords: automove evolutions Wother Wconflicts PNG lookaheads Acc sep
16519 @c LocalWords: xsltproc XSL xsl xhtml html num Wprecedence Werror fcaret gv
16520 @c LocalWords: fdiagnostics setlocale nullptr ast srcdir iff drv rgbWarning
16521 @c LocalWords: deftypefunx pragma Wnull dereference Wdocumentation elif ish
16522 @c LocalWords: Wdeprecated Wregister noinput yyloc yypos PODs sstream Wsign
16523 @c LocalWords: typename emplace Wconversion Wshorten yacchack reentrancy ou
16524 @c LocalWords: Relocatability exprs fixit Wyacc parseable fixits ffixit svg
16525 @c LocalWords: DNDEBUG cstring Wzero workalike POPL workalikes byacc UCB
16526 @c LocalWords: Penello's Penello Byson Byson's Corbett's CSD TOPLAS PDP cex
16527 @c LocalWords: Beazley's goyacc ocamlyacc SIGACT SIGPLAN colorWarning exVal
16528 @c LocalWords: setcolor rgbError colorError rgbNotice colorNotice derror
16529 @c LocalWords: colorOff maincolor inlineraw darkviolet darkcyan dwarning
16530 @c LocalWords: dnotice copyable stdint ptrdiff bufsize yyreport invariants
16531 @c LocalWords: xrefautomaticsectiontitle yysyntax yysymbol ARGMAX cond RTTI
16532 @c LocalWords: Wdangling yytoken erreur syntaxe inattendu attendait nombre
16533 @c LocalWords: YYUNDEF SymbolKind yypcontext YYENOMEM TOKENMAX getBundle
16534 @c LocalWords: ResourceBundle myResources getString getName getToken ylwrap
16535 @c LocalWords: getLocation getExpectedTokens reportSyntaxError bistromathic
16536 @c LocalWords: TokenKind Automake's rtti Wcounterexamples Chinawat PLDI buf
16537 @c LocalWords: Isradisaikul tcite pcite rgbGreen colorGreen rgbYellow Wcex
16538 @c LocalWords: colorYellow rgbRed colorRed rgbBlue colorBlue rgbPurple Ddoc
16539 @c LocalWords: colorPurple ifhtml ifnothtml situ rcex MERCHANTABILITY Wnone
16540 @c LocalWords: diagError diagNotice diagWarning diagOff danglingElseCex
16541 @c LocalWords: nonunifying YYNOMEM Wuseless dgettext textdomain domainname
16542 @c LocalWords: dirname typeof writeln YYBISON
16544 @c Local Variables:
16545 @c ispell-dictionary: "american"