1 \input texinfo @c -*-texinfo-*-
2 @comment %**start of header
3 @setfilename bison.info
4 @documentencoding UTF-8
6 @settitle Bison @value{VERSION}
7 @xrefautomaticsectiontitle on
9 @c cite a reference in text. Could not find a means to have a single
10 @c definition that looks nice in all the output formats.
22 @c cite a reference in parentheses.
25 (@pxref{\ref\,,\ref\})
35 @c ## ---------------------- ##
36 @c ## Diagnostics in color. ##
37 @c ## ---------------------- ##
40 \gdef\rgbGreen{0 .80 0}
44 \gdef\rgbYellow{1 .5 0}
46 \setcolor{\rgbYellow}%
56 \gdef\rgbPurple{0.50 0 0.50}
58 \setcolor{\rgbPurple}%
61 \gdef\rgbError{0.80 0 0}
65 \gdef\rgbNotice{0 0 0.80}
67 \setcolor{\rgbNotice}%
70 \setcolor{\maincolor}%
76 @inlineraw{html, <b style="color:green">}
79 @inlineraw{html, <b style="color:#ff8000">}
82 @inlineraw{html, <b style="color:red">}
85 @inlineraw{html, <b style="color:blue">}
88 @inlineraw{html, <b style="color:darkviolet">}
92 @inlineraw{html, <b style="color:red">}
95 @inlineraw{html, <b style="color:darkcyan">}
98 @inlineraw{html, </b>}
103 @colorGreen{}\text\@colorOff{}
107 @colorYellow{}\text\@colorOff{}
111 @colorRed{}\text\@colorOff{}
115 @colorBlue{}\text\@colorOff{}
119 @colorPurple{}\text\@colorOff{}
122 @macro dwarning{text}
127 @colorError{}\text\@colorOff{}
131 @colorNotice{}\text\@colorOff{}
136 @c SMALL BOOK version
137 @c This edition has been formatted so that you can format and print it in
138 @c the smallbook format.
140 @c @setchapternewpage odd
142 @c Set following if you want to document %default-prec and %no-default-prec.
143 @c This feature is experimental and may change in future Bison versions.
156 @comment %**end of header
160 This manual (@value{UPDATED}) is for GNU Bison (version @value{VERSION}),
161 the GNU parser generator.
163 Copyright @copyright{} 1988--1993, 1995, 1998--2015, 2018--2020 Free
164 Software Foundation, Inc.
167 Permission is granted to copy, distribute and/or modify this document under
168 the terms of the GNU Free Documentation License, Version 1.3 or any later
169 version published by the Free Software Foundation; with no Invariant
170 Sections, with the Front-Cover texts being ``A GNU Manual,'' and with the
171 Back-Cover Texts as in (a) below. A copy of the license is included in the
172 section entitled ``GNU Free Documentation License.''
174 (a) The FSF's Back-Cover Text is: ``You have the freedom to copy and modify
175 this GNU manual. Buying copies from the FSF supports it in developing GNU
176 and promoting software freedom.''
180 @dircategory Software development
182 * bison: (bison). GNU parser generator (Yacc replacement).
187 @subtitle The Yacc-compatible Parser Generator
188 @subtitle @value{UPDATED}, Bison Version @value{VERSION}
190 @author by Charles Donnelly and Richard Stallman
193 @vskip 0pt plus 1filll
196 Published by the Free Software Foundation @*
197 51 Franklin Street, Fifth Floor @*
198 Boston, MA 02110-1301 USA @*
199 Printed copies are available from the Free Software Foundation.@*
202 Cover art by Etienne Suvasa.
214 * Introduction:: What GNU Bison is.
215 * Conditions:: Conditions for using Bison and its output.
216 * Copying:: The GNU General Public License says
217 how you can copy and share Bison.
220 * Concepts:: Basic concepts for understanding Bison.
221 * Examples:: Three simple explained examples of using Bison.
224 * Grammar File:: Writing Bison declarations and rules.
225 * Interface:: C-language interface to the parser function @code{yyparse}.
226 * Algorithm:: How the Bison parser works at run-time.
227 * Error Recovery:: Writing rules for error recovery.
228 * Context Dependency:: What to do if your language syntax is too
229 messy for Bison to handle straightforwardly.
230 * Debugging:: Understanding or debugging Bison parsers.
231 * Invocation:: How to run Bison (to produce the parser implementation).
232 * Other Languages:: Creating C++ and Java parsers.
233 * History:: How Bison came to be
234 * FAQ:: Frequently Asked Questions
235 * Table of Symbols:: All the keywords of the Bison language are explained.
236 * Glossary:: Basic concepts are explained.
237 * GNU Free Documentation License:: Copying and sharing this manual
238 * Bibliography:: Publications cited in this manual.
239 * Index of Terms:: Cross-references to the text.
242 --- The Detailed Node Listing ---
244 The Concepts of Bison
246 * Language and Grammar:: Languages and context-free grammars,
247 as mathematical ideas.
248 * Grammar in Bison:: How we represent grammars for Bison's sake.
249 * Semantic Values:: Each token or syntactic grouping can have
250 a semantic value (the value of an integer,
251 the name of an identifier, etc.).
252 * Semantic Actions:: Each rule can have an action containing C code.
253 * GLR Parsers:: Writing parsers for general context-free languages.
254 * Locations:: Overview of location tracking.
255 * Bison Parser:: What are Bison's input and output,
256 how is the output used?
257 * Stages:: Stages in writing and running Bison grammars.
258 * Grammar Layout:: Overall structure of a Bison grammar file.
262 * Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
263 * Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
264 * GLR Semantic Actions:: Considerations for semantic values and deferred actions.
265 * Semantic Predicates:: Controlling a parse with arbitrary computations.
269 * RPN Calc:: Reverse Polish Notation Calculator;
270 a first example with no operator precedence.
271 * Infix Calc:: Infix (algebraic) notation calculator.
272 Operator precedence is introduced.
273 * Simple Error Recovery:: Continuing after syntax errors.
274 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
275 * Multi-function Calc:: Calculator with memory and trig functions.
276 It uses multiple data-types for semantic values.
277 * Exercises:: Ideas for improving the multi-function calculator.
279 Reverse Polish Notation Calculator
281 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
282 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
283 * Rpcalc Lexer:: The lexical analyzer.
284 * Rpcalc Main:: The controlling function.
285 * Rpcalc Error:: The error reporting function.
286 * Rpcalc Generate:: Running Bison on the grammar file.
287 * Rpcalc Compile:: Run the C compiler on the output code.
289 Grammar Rules for @code{rpcalc}
291 * Rpcalc Input:: Explanation of the @code{input} nonterminal
292 * Rpcalc Line:: Explanation of the @code{line} nonterminal
293 * Rpcalc Expr:: Explanation of the @code{expr} nonterminal
295 Location Tracking Calculator: @code{ltcalc}
297 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
298 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
299 * Ltcalc Lexer:: The lexical analyzer.
301 Multi-Function Calculator: @code{mfcalc}
303 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
304 * Mfcalc Rules:: Grammar rules for the calculator.
305 * Mfcalc Symbol Table:: Symbol table management subroutines.
306 * Mfcalc Lexer:: The lexical analyzer.
307 * Mfcalc Main:: The controlling function.
311 * Grammar Outline:: Overall layout of the grammar file.
312 * Symbols:: Terminal and nonterminal symbols.
313 * Rules:: How to write grammar rules.
314 * Semantics:: Semantic values and actions.
315 * Tracking Locations:: Locations and actions.
316 * Named References:: Using named references in actions.
317 * Declarations:: All kinds of Bison declarations are described here.
318 * Multiple Parsers:: Putting more than one Bison parser in one program.
320 Outline of a Bison Grammar
322 * Prologue:: Syntax and usage of the prologue.
323 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
324 * Bison Declarations:: Syntax and usage of the Bison declarations section.
325 * Grammar Rules:: Syntax and usage of the grammar rules section.
326 * Epilogue:: Syntax and usage of the epilogue.
330 * Rules Syntax:: Syntax of the rules.
331 * Empty Rules:: Symbols that can match the empty string.
332 * Recursion:: Writing recursive rules.
335 Defining Language Semantics
337 * Value Type:: Specifying one data type for all semantic values.
338 * Multiple Types:: Specifying several alternative data types.
339 * Type Generation:: Generating the semantic value type.
340 * Union Decl:: Declaring the set of all semantic value types.
341 * Structured Value Type:: Providing a structured semantic value type.
342 * Actions:: An action is the semantic definition of a grammar rule.
343 * Action Types:: Specifying data types for actions to operate on.
344 * Midrule Actions:: Most actions go at the end of a rule.
345 This says when, why and how to use the exceptional
346 action in the middle of a rule.
350 * Using Midrule Actions:: Putting an action in the middle of a rule.
351 * Typed Midrule Actions:: Specifying the semantic type of their values.
352 * Midrule Action Translation:: How midrule actions are actually processed.
353 * Midrule Conflicts:: Midrule actions can cause conflicts.
357 * Location Type:: Specifying a data type for locations.
358 * Actions and Locations:: Using locations in actions.
359 * Location Default Action:: Defining a general way to compute locations.
363 * Require Decl:: Requiring a Bison version.
364 * Token Decl:: Declaring terminal symbols.
365 * Precedence Decl:: Declaring terminals with precedence and associativity.
366 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
367 * Symbol Decls:: Summary of the Syntax of Symbol Declarations.
368 * Initial Action Decl:: Code run before parsing starts.
369 * Destructor Decl:: Declaring how symbols are freed.
370 * Printer Decl:: Declaring how symbol values are displayed.
371 * Expect Decl:: Suppressing warnings about parsing conflicts.
372 * Start Decl:: Specifying the start symbol.
373 * Pure Decl:: Requesting a reentrant parser.
374 * Push Decl:: Requesting a push parser.
375 * Decl Summary:: Table of all Bison declarations.
376 * %define Summary:: Defining variables to adjust Bison's behavior.
377 * %code Summary:: Inserting code into the parser source.
379 Parser C-Language Interface
381 * Parser Function:: How to call @code{yyparse} and what it returns.
382 * Push Parser Interface:: How to create, use, and destroy push parsers.
383 * Lexical:: You must supply a function @code{yylex}
385 * Error Reporting:: Passing error messages to the user.
386 * Action Features:: Special features for use in actions.
387 * Internationalization:: How to let the parser speak in the user's
390 The Lexical Analyzer Function @code{yylex}
392 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
393 * Special Tokens:: Signaling end-of-file and errors to the parser.
394 * Tokens from Literals:: Finding token kinds from string aliases.
395 * Token Values:: How @code{yylex} must return the semantic value
396 of the token it has read.
397 * Token Locations:: How @code{yylex} must return the text location
398 (line number, etc.) of the token, if the
400 * Pure Calling:: How the calling convention differs in a pure parser
405 * Error Reporting Function:: You must supply a @code{yyerror} function.
406 * Syntax Error Reporting Function:: You can supply a @code{yyreport_syntax_error} function.
408 Parser Internationalization
410 * Enabling I18n:: Preparing your project to support internationalization.
411 * Token I18n:: Preparing tokens for internationalization in error messages.
413 The Bison Parser Algorithm
415 * Lookahead:: Parser looks one token ahead when deciding what to do.
416 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
417 * Precedence:: Operator precedence works by resolving conflicts.
418 * Contextual Precedence:: When an operator's precedence depends on context.
419 * Parser States:: The parser is a finite-state-machine with stack.
420 * Reduce/Reduce:: When two rules are applicable in the same situation.
421 * Mysterious Conflicts:: Conflicts that look unjustified.
422 * Tuning LR:: How to tune fundamental aspects of LR-based parsing.
423 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
424 * Memory Management:: What happens when memory is exhausted. How to avoid it.
428 * Why Precedence:: An example showing why precedence is needed.
429 * Using Precedence:: How to specify precedence and associativity.
430 * Precedence Only:: How to specify precedence only.
431 * Precedence Examples:: How these features are used in the previous example.
432 * How Precedence:: How they work.
433 * Non Operators:: Using precedence for general conflicts.
437 * LR Table Construction:: Choose a different construction algorithm.
438 * Default Reductions:: Disable default reductions.
439 * LAC:: Correct lookahead sets in the parser states.
440 * Unreachable States:: Keep unreachable parser states for debugging.
442 Handling Context Dependencies
444 * Semantic Tokens:: Token parsing can depend on the semantic context.
445 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
446 * Tie-in Recovery:: Lexical tie-ins have implications for how
447 error recovery rules must be written.
449 Debugging Your Parser
451 * Counterexamples:: Understanding conflicts.
452 * Understanding:: Understanding the structure of your parser.
453 * Graphviz:: Getting a visual representation of the parser.
454 * Xml:: Getting a markup representation of the parser.
455 * Tracing:: Tracing the execution of your parser.
459 * Enabling Traces:: Activating run-time trace support
460 * Mfcalc Traces:: Extending @code{mfcalc} to support traces
461 * The YYPRINT Macro:: Obsolete interface for semantic value reports
465 * Bison Options:: All the options described in detail,
466 in alphabetical order by short options.
467 * Option Cross Key:: Alphabetical list of long options.
468 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
472 * Operation Modes:: Options controlling the global behavior of @command{bison}
473 * Diagnostics:: Options controlling the diagnostics
474 * Tuning the Parser:: Options changing the generated parsers
475 * Output Files:: Options controlling the output
477 Parsers Written In Other Languages
479 * C++ Parsers:: The interface to generate C++ parser classes
480 * Java Parsers:: The interface to generate Java parser classes
484 * A Simple C++ Example:: A short introduction to C++ parsers
485 * C++ Bison Interface:: Asking for C++ parser generation
486 * C++ Parser Interface:: Instantiating and running the parser
487 * C++ Semantic Values:: %union vs. C++
488 * C++ Location Values:: The position and location classes
489 * C++ Parser Context:: You can supply a @code{report_syntax_error} function.
490 * C++ Scanner Interface:: Exchanges between yylex and parse
491 * A Complete C++ Example:: Demonstrating their use
495 * C++ position:: One point in the source file
496 * C++ location:: Two points in the source file
497 * Exposing the Location Classes:: Using the Bison location class in your
499 * User Defined Location Type:: Required interface for locations
501 A Complete C++ Example
503 * Calc++ --- C++ Calculator:: The specifications
504 * Calc++ Parsing Driver:: An active parsing context
505 * Calc++ Parser:: A parser class
506 * Calc++ Scanner:: A pure C++ Flex scanner
507 * Calc++ Top Level:: Conducting the band
511 * Java Bison Interface:: Asking for Java parser generation
512 * Java Semantic Values:: %token and %nterm vs. Java
513 * Java Location Values:: The position and location classes
514 * Java Parser Interface:: Instantiating and running the parser
515 * Java Parser Context Interface:: Circumstances of a syntax error
516 * Java Scanner Interface:: Specifying the scanner for the parser
517 * Java Action Features:: Special features for use in actions
518 * Java Push Parser Interface:: Instantiating and running the a push parser
519 * Java Differences:: Differences between C/C++ and Java Grammars
520 * Java Declarations Summary:: List of Bison declarations used with Java
522 A Brief History of the Greater Ungulates
524 * Yacc:: The original Yacc
525 * yacchack:: An obscure early implementation of reentrancy
526 * Byacc:: Berkeley Yacc
527 * Bison:: This program
528 * Other Ungulates:: Similar programs
530 Frequently Asked Questions
532 * Memory Exhausted:: Breaking the Stack Limits
533 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
534 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
535 * Implementing Gotos/Loops:: Control Flow in the Calculator
536 * Multiple start-symbols:: Factoring closely related grammars
537 * Enabling Relocatability:: Moving Bison/using it through network shares
538 * Secure? Conform?:: Is Bison POSIX safe?
539 * I can't build Bison:: Troubleshooting
540 * Where can I find help?:: Troubleshouting
541 * Bug Reports:: Troublereporting
542 * More Languages:: Parsers in C++, Java, and so on
543 * Beta Testing:: Experimenting development versions
544 * Mailing Lists:: Meeting other Bison users
548 * GNU Free Documentation License:: Copying and sharing this manual
554 @unnumbered Introduction
557 @dfn{Bison} is a general-purpose parser generator that converts an annotated
558 context-free grammar into a deterministic LR or generalized LR (GLR) parser
559 employing LALR(1), IELR(1) or canonical LR(1) parser tables. Once you are
560 proficient with Bison, you can use it to develop a wide range of language
561 parsers, from those used in simple desk calculators to complex programming
564 Bison is upward compatible with Yacc: all properly-written Yacc grammars
565 ought to work with Bison with no change. Anyone familiar with Yacc should
566 be able to use Bison with little trouble. You need to be fluent in C, C++
567 or Java programming in order to use Bison or to understand this manual.
569 We begin with tutorial chapters that explain the basic concepts of
570 using Bison and show three explained examples, each building on the
571 last. If you don't know Bison or Yacc, start by reading these
572 chapters. Reference chapters follow, which describe specific aspects
575 Bison was written originally by Robert Corbett. Richard Stallman made
576 it Yacc-compatible. Wilfred Hansen of Carnegie Mellon University
577 added multi-character string literals and other features. Since then,
578 Bison has grown more robust and evolved many other new features thanks
579 to the hard work of a long list of volunteers. For details, see the
580 @file{THANKS} and @file{ChangeLog} files included in the Bison
583 This edition corresponds to version @value{VERSION} of Bison.
586 @unnumbered Conditions for Using Bison
588 The distribution terms for Bison-generated parsers permit using the parsers
589 in nonfree programs. Before Bison version 2.2, these extra permissions
590 applied only when Bison was generating LALR(1) parsers in C@. And before
591 Bison version 1.24, Bison-generated parsers could be used only in programs
592 that were free software.
594 The other GNU programming tools, such as the GNU C compiler, have never had
595 such a requirement. They could always be used for nonfree software. The
596 reason Bison was different was not due to a special policy decision; it
597 resulted from applying the usual General Public License to all of the Bison
600 The main output of the Bison utility---the Bison parser implementation
601 file---contains a verbatim copy of a sizable piece of Bison, which is the
602 code for the parser's implementation. (The actions from your grammar are
603 inserted into this implementation at one point, but most of the rest of the
604 implementation is not changed.) When we applied the GPL terms to the
605 skeleton code for the parser's implementation, the effect was to restrict
606 the use of Bison output to free software.
608 We didn't change the terms because of sympathy for people who want to make
609 software proprietary. @strong{Software should be free.} But we concluded
610 that limiting Bison's use to free software was doing little to encourage
611 people to make other software free. So we decided to make the practical
612 conditions for using Bison match the practical conditions for using the
615 This exception applies when Bison is generating code for a parser. You can
616 tell whether the exception applies to a Bison output file by inspecting the
617 file for text beginning with ``As a special exception@dots{}''. The text
618 spells out the exact terms of the exception.
621 @unnumbered GNU GENERAL PUBLIC LICENSE
622 @include gpl-3.0.texi
625 @chapter The Concepts of Bison
627 This chapter introduces many of the basic concepts without which the details
628 of Bison will not make sense. If you do not already know how to use Bison
629 or Yacc, we suggest you start by reading this chapter carefully.
632 * Language and Grammar:: Languages and context-free grammars,
633 as mathematical ideas.
634 * Grammar in Bison:: How we represent grammars for Bison's sake.
635 * Semantic Values:: Each token or syntactic grouping can have
636 a semantic value (the value of an integer,
637 the name of an identifier, etc.).
638 * Semantic Actions:: Each rule can have an action containing C code.
639 * GLR Parsers:: Writing parsers for general context-free languages.
640 * Locations:: Overview of location tracking.
641 * Bison Parser:: What are Bison's input and output,
642 how is the output used?
643 * Stages:: Stages in writing and running Bison grammars.
644 * Grammar Layout:: Overall structure of a Bison grammar file.
647 @node Language and Grammar
648 @section Languages and Context-Free Grammars
650 @cindex context-free grammar
651 @cindex grammar, context-free
652 In order for Bison to parse a language, it must be described by a
653 @dfn{context-free grammar}. This means that you specify one or more
654 @dfn{syntactic groupings} and give rules for constructing them from their
655 parts. For example, in the C language, one kind of grouping is called an
656 `expression'. One rule for making an expression might be, ``An expression
657 can be made of a minus sign and another expression''. Another would be,
658 ``An expression can be an integer''. As you can see, rules are often
659 recursive, but there must be at least one rule which leads out of the
663 @cindex Backus-Naur form
664 The most common formal system for presenting such rules for humans to read
665 is @dfn{Backus-Naur Form} or ``BNF'', which was developed in
666 order to specify the language Algol 60. Any grammar expressed in
667 BNF is a context-free grammar. The input to Bison is
668 essentially machine-readable BNF.
670 @cindex LALR grammars
671 @cindex IELR grammars
673 There are various important subclasses of context-free grammars. Although
674 it can handle almost all context-free grammars, Bison is optimized for what
675 are called LR(1) grammars. In brief, in these grammars, it must be possible
676 to tell how to parse any portion of an input string with just a single token
677 of lookahead. For historical reasons, Bison by default is limited by the
678 additional restrictions of LALR(1), which is hard to explain simply.
679 @xref{Mysterious Conflicts}, for more information on this. You can escape
680 these additional restrictions by requesting IELR(1) or canonical LR(1)
681 parser tables. @xref{LR Table Construction}, to learn how.
684 @cindex generalized LR (GLR) parsing
685 @cindex ambiguous grammars
686 @cindex nondeterministic parsing
688 Parsers for LR(1) grammars are @dfn{deterministic}, meaning
689 roughly that the next grammar rule to apply at any point in the input is
690 uniquely determined by the preceding input and a fixed, finite portion
691 (called a @dfn{lookahead}) of the remaining input. A context-free
692 grammar can be @dfn{ambiguous}, meaning that there are multiple ways to
693 apply the grammar rules to get the same inputs. Even unambiguous
694 grammars can be @dfn{nondeterministic}, meaning that no fixed
695 lookahead always suffices to determine the next grammar rule to apply.
696 With the proper declarations, Bison is also able to parse these more
697 general context-free grammars, using a technique known as GLR
698 parsing (for Generalized LR). Bison's GLR parsers
699 are able to handle any context-free grammar for which the number of
700 possible parses of any given string is finite.
702 @cindex symbols (abstract)
704 @cindex syntactic grouping
705 @cindex grouping, syntactic
706 In the formal grammatical rules for a language, each kind of syntactic unit
707 or grouping is named by a @dfn{symbol}. Those which are built by grouping
708 smaller constructs according to grammatical rules are called
709 @dfn{nonterminal symbols}; those which can't be subdivided are called
710 @dfn{terminal symbols} or @dfn{token kinds}. We call a piece of input
711 corresponding to a single terminal symbol a @dfn{token}, and a piece
712 corresponding to a single nonterminal symbol a @dfn{grouping}.
714 We can use the C language as an example of what symbols, terminal and
715 nonterminal, mean. The tokens of C are identifiers, constants (numeric
716 and string), and the various keywords, arithmetic operators and
717 punctuation marks. So the terminal symbols of a grammar for C include
718 `identifier', `number', `string', plus one symbol for each keyword,
719 operator or punctuation mark: `if', `return', `const', `static', `int',
720 `char', `plus-sign', `open-brace', `close-brace', `comma' and many more.
721 (These tokens can be subdivided into characters, but that is a matter of
722 lexicography, not grammar.)
724 Here is a simple C function subdivided into tokens:
727 int /* @r{keyword `int'} */
728 square (int x) /* @r{identifier, open-paren, keyword `int',}
729 @r{identifier, close-paren} */
730 @{ /* @r{open-brace} */
731 return x * x; /* @r{keyword `return', identifier, asterisk,}
732 @r{identifier, semicolon} */
733 @} /* @r{close-brace} */
736 The syntactic groupings of C include the expression, the statement, the
737 declaration, and the function definition. These are represented in the
738 grammar of C by nonterminal symbols `expression', `statement',
739 `declaration' and `function definition'. The full grammar uses dozens of
740 additional language constructs, each with its own nonterminal symbol, in
741 order to express the meanings of these four. The example above is a
742 function definition; it contains one declaration, and one statement. In
743 the statement, each @samp{x} is an expression and so is @samp{x * x}.
745 Each nonterminal symbol must have grammatical rules showing how it is made
746 out of simpler constructs. For example, one kind of C statement is the
747 @code{return} statement; this would be described with a grammar rule which
748 reads informally as follows:
751 A `statement' can be made of a `return' keyword, an `expression' and a
756 There would be many other rules for `statement', one for each kind of
760 One nonterminal symbol must be distinguished as the special one which
761 defines a complete utterance in the language. It is called the @dfn{start
762 symbol}. In a compiler, this means a complete input program. In the C
763 language, the nonterminal symbol `sequence of definitions and declarations'
766 For example, @samp{1 + 2} is a valid C expression---a valid part of a C
767 program---but it is not valid as an @emph{entire} C program. In the
768 context-free grammar of C, this follows from the fact that `expression' is
769 not the start symbol.
771 The Bison parser reads a sequence of tokens as its input, and groups the
772 tokens using the grammar rules. If the input is valid, the end result is
773 that the entire token sequence reduces to a single grouping whose symbol is
774 the grammar's start symbol. If we use a grammar for C, the entire input
775 must be a `sequence of definitions and declarations'. If not, the parser
776 reports a syntax error.
778 @node Grammar in Bison
779 @section From Formal Rules to Bison Input
780 @cindex Bison grammar
781 @cindex grammar, Bison
782 @cindex formal grammar
784 A formal grammar is a mathematical construct. To define the language
785 for Bison, you must write a file expressing the grammar in Bison syntax:
786 a @dfn{Bison grammar} file. @xref{Grammar File}.
788 A nonterminal symbol in the formal grammar is represented in Bison input
789 as an identifier, like an identifier in C@. By convention, it should be
790 in lower case, such as @code{expr}, @code{stmt} or @code{declaration}.
792 The Bison representation for a terminal symbol is also called a @dfn{token
793 kind}. Token kinds as well can be represented as C-like identifiers. By
794 convention, these identifiers should be upper case to distinguish them from
795 nonterminals: for example, @code{INTEGER}, @code{IDENTIFIER}, @code{IF} or
796 @code{RETURN}. A terminal symbol that stands for a particular keyword in
797 the language should be named after that keyword converted to upper case.
798 The terminal symbol @code{error} is reserved for error recovery.
801 A terminal symbol can also be represented as a character literal, just like
802 a C character constant. You should do this whenever a token is just a
803 single character (parenthesis, plus-sign, etc.): use that same character in
804 a literal as the terminal symbol for that token.
806 A third way to represent a terminal symbol is with a C string constant
807 containing several characters. @xref{Symbols}, for more information.
809 The grammar rules also have an expression in Bison syntax. For example,
810 here is the Bison rule for a C @code{return} statement. The semicolon in
811 quotes is a literal character token, representing part of the C syntax for
812 the statement; the naked semicolon, and the colon, are Bison punctuation
816 stmt: RETURN expr ';' ;
822 @node Semantic Values
823 @section Semantic Values
824 @cindex semantic value
825 @cindex value, semantic
827 A formal grammar selects tokens only by their classifications: for example,
828 if a rule mentions the terminal symbol `integer constant', it means that
829 @emph{any} integer constant is grammatically valid in that position. The
830 precise value of the constant is irrelevant to how to parse the input: if
831 @samp{x+4} is grammatical then @samp{x+1} or @samp{x+3989} is equally
834 But the precise value is very important for what the input means once it is
835 parsed. A compiler is useless if it fails to distinguish between 4, 1 and
836 3989 as constants in the program! Therefore, each token in a Bison grammar
837 has both a token kind and a @dfn{semantic value}. @xref{Semantics}, for
840 The token kind is a terminal symbol defined in the grammar, such as
841 @code{INTEGER}, @code{IDENTIFIER} or @code{','}. It tells everything you
842 need to know to decide where the token may validly appear and how to group
843 it with other tokens. The grammar rules know nothing about tokens except
846 The semantic value has all the rest of the information about the
847 meaning of the token, such as the value of an integer, or the name of an
848 identifier. (A token such as @code{','} which is just punctuation doesn't
849 need to have any semantic value.)
851 For example, an input token might be classified as token kind @code{INTEGER}
852 and have the semantic value 4. Another input token might have the same
853 token kind @code{INTEGER} but value 3989. When a grammar rule says that
854 @code{INTEGER} is allowed, either of these tokens is acceptable because each
855 is an @code{INTEGER}. When the parser accepts the token, it keeps track of
856 the token's semantic value.
858 Each grouping can also have a semantic value as well as its nonterminal
859 symbol. For example, in a calculator, an expression typically has a
860 semantic value that is a number. In a compiler for a programming
861 language, an expression typically has a semantic value that is a tree
862 structure describing the meaning of the expression.
864 @node Semantic Actions
865 @section Semantic Actions
866 @cindex semantic actions
867 @cindex actions, semantic
869 In order to be useful, a program must do more than parse input; it must
870 also produce some output based on the input. In a Bison grammar, a grammar
871 rule can have an @dfn{action} made up of C statements. Each time the
872 parser recognizes a match for that rule, the action is executed.
875 Most of the time, the purpose of an action is to compute the semantic value
876 of the whole construct from the semantic values of its parts. For example,
877 suppose we have a rule which says an expression can be the sum of two
878 expressions. When the parser recognizes such a sum, each of the
879 subexpressions has a semantic value which describes how it was built up.
880 The action for this rule should create a similar sort of value for the
881 newly recognized larger expression.
883 For example, here is a rule that says an expression can be the sum of
887 expr: expr '+' expr @{ $$ = $1 + $3; @} ;
891 The action says how to produce the semantic value of the sum expression
892 from the values of the two subexpressions.
895 @section Writing GLR Parsers
897 @cindex generalized LR (GLR) parsing
900 @cindex shift/reduce conflicts
901 @cindex reduce/reduce conflicts
903 In some grammars, Bison's deterministic
904 LR(1) parsing algorithm cannot decide whether to apply a
905 certain grammar rule at a given point. That is, it may not be able to
906 decide (on the basis of the input read so far) which of two possible
907 reductions (applications of a grammar rule) applies, or whether to apply
908 a reduction or read more of the input and apply a reduction later in the
909 input. These are known respectively as @dfn{reduce/reduce} conflicts
910 (@pxref{Reduce/Reduce}), and @dfn{shift/reduce} conflicts
911 (@pxref{Shift/Reduce}).
913 To use a grammar that is not easily modified to be LR(1), a more general
914 parsing algorithm is sometimes necessary. If you include @code{%glr-parser}
915 among the Bison declarations in your file (@pxref{Grammar Outline}), the
916 result is a Generalized LR (GLR) parser. These parsers handle Bison
917 grammars that contain no unresolved conflicts (i.e., after applying
918 precedence declarations) identically to deterministic parsers. However,
919 when faced with unresolved shift/reduce and reduce/reduce conflicts, GLR
920 parsers use the simple expedient of doing both, effectively cloning the
921 parser to follow both possibilities. Each of the resulting parsers can
922 again split, so that at any given time, there can be any number of possible
923 parses being explored. The parsers proceed in lockstep; that is, all of
924 them consume (shift) a given input symbol before any of them proceed to the
925 next. Each of the cloned parsers eventually meets one of two possible
926 fates: either it runs into a parsing error, in which case it simply
927 vanishes, or it merges with another parser, because the two of them have
928 reduced the input to an identical set of symbols.
930 During the time that there are multiple parsers, semantic actions are
931 recorded, but not performed. When a parser disappears, its recorded
932 semantic actions disappear as well, and are never performed. When a
933 reduction makes two parsers identical, causing them to merge, Bison records
934 both sets of semantic actions. Whenever the last two parsers merge,
935 reverting to the single-parser case, Bison resolves all the outstanding
936 actions either by precedences given to the grammar rules involved, or by
937 performing both actions, and then calling a designated user-defined function
938 on the resulting values to produce an arbitrary merged result.
941 * Simple GLR Parsers:: Using GLR parsers on unambiguous grammars.
942 * Merging GLR Parses:: Using GLR parsers to resolve ambiguities.
943 * GLR Semantic Actions:: Considerations for semantic values and deferred actions.
944 * Semantic Predicates:: Controlling a parse with arbitrary computations.
947 @node Simple GLR Parsers
948 @subsection Using GLR on Unambiguous Grammars
949 @cindex GLR parsing, unambiguous grammars
950 @cindex generalized LR (GLR) parsing, unambiguous grammars
954 @cindex reduce/reduce conflicts
955 @cindex shift/reduce conflicts
957 In the simplest cases, you can use the GLR algorithm
958 to parse grammars that are unambiguous but fail to be LR(1).
959 Such grammars typically require more than one symbol of lookahead.
961 Consider a problem that
962 arises in the declaration of enumerated and subrange types in the
963 programming language Pascal. Here are some examples:
966 type subrange = lo .. hi;
967 type enum = (a, b, c);
971 The original language standard allows only numeric literals and constant
972 identifiers for the subrange bounds (@samp{lo} and @samp{hi}), but Extended
973 Pascal (ISO/IEC 10206) and many other Pascal implementations allow arbitrary
974 expressions there. This gives rise to the following situation, containing a
975 superfluous pair of parentheses:
978 type subrange = (a) .. b;
982 Compare this to the following declaration of an enumerated
983 type with only one value:
990 (These declarations are contrived, but they are syntactically valid, and
991 more-complicated cases can come up in practical programs.)
993 These two declarations look identical until the @samp{..} token. With
994 normal LR(1) one-token lookahead it is not possible to decide between the
995 two forms when the identifier @samp{a} is parsed. It is, however, desirable
996 for a parser to decide this, since in the latter case @samp{a} must become a
997 new identifier to represent the enumeration value, while in the former case
998 @samp{a} must be evaluated with its current meaning, which may be a constant
999 or even a function call.
1001 You could parse @samp{(a)} as an ``unspecified identifier in parentheses'',
1002 to be resolved later, but this typically requires substantial contortions in
1003 both semantic actions and large parts of the grammar, where the parentheses
1004 are nested in the recursive rules for expressions.
1006 You might think of using the lexer to distinguish between the two forms by
1007 returning different tokens for currently defined and undefined identifiers.
1008 But if these declarations occur in a local scope, and @samp{a} is defined in
1009 an outer scope, then both forms are possible---either locally redefining
1010 @samp{a}, or using the value of @samp{a} from the outer scope. So this
1011 approach cannot work.
1013 A simple solution to this problem is to declare the parser to use the GLR
1014 algorithm. When the GLR parser reaches the critical state, it merely splits
1015 into two branches and pursues both syntax rules simultaneously. Sooner or
1016 later, one of them runs into a parsing error. If there is a @samp{..} token
1017 before the next @samp{;}, the rule for enumerated types fails since it
1018 cannot accept @samp{..} anywhere; otherwise, the subrange type rule fails
1019 since it requires a @samp{..} token. So one of the branches fails silently,
1020 and the other one continues normally, performing all the intermediate
1021 actions that were postponed during the split.
1023 If the input is syntactically incorrect, both branches fail and the parser
1024 reports a syntax error as usual.
1026 The effect of all this is that the parser seems to ``guess'' the correct
1027 branch to take, or in other words, it seems to use more lookahead than the
1028 underlying LR(1) algorithm actually allows for. In this example, LR(2)
1029 would suffice, but also some cases that are not LR(@math{k}) for any
1030 @math{k} can be handled this way.
1032 In general, a GLR parser can take quadratic or cubic worst-case time, and
1033 the current Bison parser even takes exponential time and space for some
1034 grammars. In practice, this rarely happens, and for many grammars it is
1035 possible to prove that it cannot happen. The present example contains only
1036 one conflict between two rules, and the type-declaration context containing
1037 the conflict cannot be nested. So the number of branches that can exist at
1038 any time is limited by the constant 2, and the parsing time is still linear.
1040 Here is a Bison grammar corresponding to the example above. It
1041 parses a vastly simplified form of Pascal type declarations.
1044 %token TYPE DOTDOT ID
1052 type_decl: TYPE ID '=' type ';' ;
1080 When used as a normal LR(1) grammar, Bison correctly complains
1081 about one reduce/reduce conflict. In the conflicting situation the
1082 parser chooses one of the alternatives, arbitrarily the one
1083 declared first. Therefore the following correct input is not
1090 The parser can be turned into a GLR parser, while also telling Bison
1091 to be silent about the one known reduce/reduce conflict, by adding
1092 these two declarations to the Bison grammar file (before the first
1101 No change in the grammar itself is required. Now the parser recognizes all
1102 valid declarations, according to the limited syntax above, transparently.
1103 In fact, the user does not even notice when the parser splits.
1105 So here we have a case where we can use the benefits of GLR, almost without
1106 disadvantages. Even in simple cases like this, however, there are at least
1107 two potential problems to beware. First, always analyze the conflicts
1108 reported by Bison to make sure that GLR splitting is only done where it is
1109 intended. A GLR parser splitting inadvertently may cause problems less
1110 obvious than an LR parser statically choosing the wrong alternative in a
1111 conflict. Second, consider interactions with the lexer (@pxref{Semantic
1112 Tokens}) with great care. Since a split parser consumes tokens without
1113 performing any actions during the split, the lexer cannot obtain information
1114 via parser actions. Some cases of lexer interactions can be eliminated by
1115 using GLR to shift the complications from the lexer to the parser. You must
1116 check the remaining cases for correctness.
1118 In our example, it would be safe for the lexer to return tokens based on
1119 their current meanings in some symbol table, because no new symbols are
1120 defined in the middle of a type declaration. Though it is possible for a
1121 parser to define the enumeration constants as they are parsed, before the
1122 type declaration is completed, it actually makes no difference since they
1123 cannot be used within the same enumerated type declaration.
1125 @node Merging GLR Parses
1126 @subsection Using GLR to Resolve Ambiguities
1127 @cindex GLR parsing, ambiguous grammars
1128 @cindex generalized LR (GLR) parsing, ambiguous grammars
1132 @cindex reduce/reduce conflicts
1134 Let's consider an example, vastly simplified from a C++ grammar.
1139 #define YYSTYPE char const *
1141 void yyerror (char const *);
1155 | prog stmt @{ printf ("\n"); @}
1164 ID @{ printf ("%s ", $$); @}
1165 | TYPENAME '(' expr ')'
1166 @{ printf ("%s <cast> ", $1); @}
1167 | expr '+' expr @{ printf ("+ "); @}
1168 | expr '=' expr @{ printf ("= "); @}
1172 TYPENAME declarator ';'
1173 @{ printf ("%s <declare> ", $1); @}
1174 | TYPENAME declarator '=' expr ';'
1175 @{ printf ("%s <init-declare> ", $1); @}
1179 ID @{ printf ("\"%s\" ", $1); @}
1180 | '(' declarator ')'
1185 This models a problematic part of the C++ grammar---the ambiguity between
1186 certain declarations and statements. For example,
1193 parses as either an @code{expr} or a @code{stmt}
1194 (assuming that @samp{T} is recognized as a @code{TYPENAME} and
1195 @samp{x} as an @code{ID}).
1196 Bison detects this as a reduce/reduce conflict between the rules
1197 @code{expr : ID} and @code{declarator : ID}, which it cannot resolve at the
1198 time it encounters @code{x} in the example above. Since this is a
1199 GLR parser, it therefore splits the problem into two parses, one for
1200 each choice of resolving the reduce/reduce conflict.
1201 Unlike the example from the previous section (@pxref{Simple GLR Parsers}),
1202 however, neither of these parses ``dies,'' because the grammar as it stands is
1203 ambiguous. One of the parsers eventually reduces @code{stmt : expr ';'} and
1204 the other reduces @code{stmt : decl}, after which both parsers are in an
1205 identical state: they've seen @samp{prog stmt} and have the same unprocessed
1206 input remaining. We say that these parses have @dfn{merged.}
1208 At this point, the GLR parser requires a specification in the
1209 grammar of how to choose between the competing parses.
1210 In the example above, the two @code{%dprec}
1211 declarations specify that Bison is to give precedence
1212 to the parse that interprets the example as a
1213 @code{decl}, which implies that @code{x} is a declarator.
1214 The parser therefore prints
1217 "x" y z + T <init-declare>
1220 The @code{%dprec} declarations only come into play when more than one
1221 parse survives. Consider a different input string for this parser:
1228 This is another example of using GLR to parse an unambiguous
1229 construct, as shown in the previous section (@pxref{Simple GLR Parsers}).
1230 Here, there is no ambiguity (this cannot be parsed as a declaration).
1231 However, at the time the Bison parser encounters @code{x}, it does not
1232 have enough information to resolve the reduce/reduce conflict (again,
1233 between @code{x} as an @code{expr} or a @code{declarator}). In this
1234 case, no precedence declaration is used. Again, the parser splits
1235 into two, one assuming that @code{x} is an @code{expr}, and the other
1236 assuming @code{x} is a @code{declarator}. The second of these parsers
1237 then vanishes when it sees @code{+}, and the parser prints
1243 Suppose that instead of resolving the ambiguity, you wanted to see all
1244 the possibilities. For this purpose, you must merge the semantic
1245 actions of the two possible parsers, rather than choosing one over the
1246 other. To do so, you could change the declaration of @code{stmt} as
1251 expr ';' %merge <stmtMerge>
1252 | decl %merge <stmtMerge>
1257 and define the @code{stmtMerge} function as:
1261 stmtMerge (YYSTYPE x0, YYSTYPE x1)
1269 with an accompanying forward declaration
1270 in the C declarations at the beginning of the file:
1274 #define YYSTYPE char const *
1275 static YYSTYPE stmtMerge (YYSTYPE x0, YYSTYPE x1);
1280 With these declarations, the resulting parser parses the first example
1281 as both an @code{expr} and a @code{decl}, and prints
1284 "x" y z + T <init-declare> x T <cast> y z + = <OR>
1287 Bison requires that all of the
1288 productions that participate in any particular merge have identical
1289 @samp{%merge} clauses. Otherwise, the ambiguity would be unresolvable,
1290 and the parser will report an error during any parse that results in
1291 the offending merge.
1293 @node GLR Semantic Actions
1294 @subsection GLR Semantic Actions
1296 The nature of GLR parsing and the structure of the generated
1297 parsers give rise to certain restrictions on semantic values and actions.
1299 @subsubsection Deferred semantic actions
1300 @cindex deferred semantic actions
1301 By definition, a deferred semantic action is not performed at the same time as
1302 the associated reduction.
1303 This raises caveats for several Bison features you might use in a semantic
1304 action in a GLR parser.
1307 @cindex GLR parsers and @code{yychar}
1309 @cindex GLR parsers and @code{yylval}
1311 @cindex GLR parsers and @code{yylloc}
1312 In any semantic action, you can examine @code{yychar} to determine the kind
1313 of the lookahead token present at the time of the associated reduction.
1314 After checking that @code{yychar} is not set to @code{YYEMPTY} or
1315 @code{YYEOF}, you can then examine @code{yylval} and @code{yylloc} to
1316 determine the lookahead token's semantic value and location, if any. In a
1317 nondeferred semantic action, you can also modify any of these variables to
1318 influence syntax analysis. @xref{Lookahead}.
1321 @cindex GLR parsers and @code{yyclearin}
1322 In a deferred semantic action, it's too late to influence syntax analysis.
1323 In this case, @code{yychar}, @code{yylval}, and @code{yylloc} are set to
1324 shallow copies of the values they had at the time of the associated reduction.
1325 For this reason alone, modifying them is dangerous.
1326 Moreover, the result of modifying them is undefined and subject to change with
1327 future versions of Bison.
1328 For example, if a semantic action might be deferred, you should never write it
1329 to invoke @code{yyclearin} (@pxref{Action Features}) or to attempt to free
1330 memory referenced by @code{yylval}.
1332 @subsubsection YYERROR
1334 @cindex GLR parsers and @code{YYERROR}
1335 Another Bison feature requiring special consideration is @code{YYERROR}
1336 (@pxref{Action Features}), which you can invoke in a semantic action to
1337 initiate error recovery.
1338 During deterministic GLR operation, the effect of @code{YYERROR} is
1339 the same as its effect in a deterministic parser.
1340 The effect in a deferred action is similar, but the precise point of the
1341 error is undefined; instead, the parser reverts to deterministic operation,
1342 selecting an unspecified stack on which to continue with a syntax error.
1343 In a semantic predicate (see @ref{Semantic Predicates}) during nondeterministic
1344 parsing, @code{YYERROR} silently prunes
1345 the parse that invoked the test.
1347 @subsubsection Restrictions on semantic values and locations
1348 GLR parsers require that you use POD (Plain Old Data) types for
1349 semantic values and location types when using the generated parsers as
1352 @node Semantic Predicates
1353 @subsection Controlling a Parse with Arbitrary Predicates
1355 @cindex Semantic predicates in GLR parsers
1357 In addition to the @code{%dprec} and @code{%merge} directives,
1359 allow you to reject parses on the basis of arbitrary computations executed
1360 in user code, without having Bison treat this rejection as an error
1361 if there are alternative parses. For example,
1365 %?@{ new_syntax @} "widget" id new_args @{ $$ = f($3, $4); @}
1366 | %?@{ !new_syntax @} "widget" id old_args @{ $$ = f($3, $4); @}
1371 is one way to allow the same parser to handle two different syntaxes for
1372 widgets. The clause preceded by @code{%?} is treated like an ordinary
1373 midrule action, except that its text is handled as an expression and is always
1374 evaluated immediately (even when in nondeterministic mode). If the
1375 expression yields 0 (false), the clause is treated as a syntax error,
1376 which, in a nondeterministic parser, causes the stack in which it is reduced
1377 to die. In a deterministic parser, it acts like @code{YYERROR}.
1379 As the example shows, predicates otherwise look like semantic actions, and
1380 therefore you must take them into account when determining the numbers
1381 to use for denoting the semantic values of right-hand side symbols.
1382 Predicate actions, however, have no defined value, and may not be given
1385 There is a subtle difference between semantic predicates and ordinary
1386 actions in nondeterministic mode, since the latter are deferred.
1387 For example, we could try to rewrite the previous example as
1391 @{ if (!new_syntax) YYERROR; @}
1392 "widget" id new_args @{ $$ = f($3, $4); @}
1393 | @{ if (new_syntax) YYERROR; @}
1394 "widget" id old_args @{ $$ = f($3, $4); @}
1399 (reversing the sense of the predicate tests to cause an error when they are
1400 false). However, this
1401 does @emph{not} have the same effect if @code{new_args} and @code{old_args}
1402 have overlapping syntax.
1403 Since the midrule actions testing @code{new_syntax} are deferred,
1404 a GLR parser first encounters the unresolved ambiguous reduction
1405 for cases where @code{new_args} and @code{old_args} recognize the same string
1406 @emph{before} performing the tests of @code{new_syntax}. It therefore
1409 Finally, be careful in writing predicates: deferred actions have not been
1410 evaluated, so that using them in a predicate will have undefined effects.
1415 @cindex textual location
1416 @cindex location, textual
1418 Many applications, like interpreters or compilers, have to produce verbose
1419 and useful error messages. To achieve this, one must be able to keep track of
1420 the @dfn{textual location}, or @dfn{location}, of each syntactic construct.
1421 Bison provides a mechanism for handling these locations.
1423 Each token has a semantic value. In a similar fashion, each token has an
1424 associated location, but the type of locations is the same for all tokens
1425 and groupings. Moreover, the output parser is equipped with a default data
1426 structure for storing locations (@pxref{Tracking Locations}, for more
1429 Like semantic values, locations can be reached in actions using a dedicated
1430 set of constructs. In the example above, the location of the whole grouping
1431 is @code{@@$}, while the locations of the subexpressions are @code{@@1} and
1434 When a rule is matched, a default action is used to compute the semantic value
1435 of its left hand side (@pxref{Actions}). In the same way, another default
1436 action is used for locations. However, the action for locations is general
1437 enough for most cases, meaning there is usually no need to describe for each
1438 rule how @code{@@$} should be formed. When building a new location for a given
1439 grouping, the default behavior of the output parser is to take the beginning
1440 of the first symbol, and the end of the last symbol.
1443 @section Bison Output: the Parser Implementation File
1444 @cindex Bison parser
1445 @cindex Bison utility
1446 @cindex lexical analyzer, purpose
1449 When you run Bison, you give it a Bison grammar file as input. The
1450 most important output is a C source file that implements a parser for
1451 the language described by the grammar. This parser is called a
1452 @dfn{Bison parser}, and this file is called a @dfn{Bison parser
1453 implementation file}. Keep in mind that the Bison utility and the
1454 Bison parser are two distinct programs: the Bison utility is a program
1455 whose output is the Bison parser implementation file that becomes part
1458 The job of the Bison parser is to group tokens into groupings according to
1459 the grammar rules---for example, to build identifiers and operators into
1460 expressions. As it does this, it runs the actions for the grammar rules it
1463 The tokens come from a function called the @dfn{lexical analyzer} that
1464 you must supply in some fashion (such as by writing it in C). The Bison
1465 parser calls the lexical analyzer each time it wants a new token. It
1466 doesn't know what is ``inside'' the tokens (though their semantic values
1467 may reflect this). Typically the lexical analyzer makes the tokens by
1468 parsing characters of text, but Bison does not depend on this.
1471 The Bison parser implementation file is C code which defines a
1472 function named @code{yyparse} which implements that grammar. This
1473 function does not make a complete C program: you must supply some
1474 additional functions. One is the lexical analyzer. Another is an
1475 error-reporting function which the parser calls to report an error.
1476 In addition, a complete C program must start with a function called
1477 @code{main}; you have to provide this, and arrange for it to call
1478 @code{yyparse} or the parser will never run. @xref{Interface}.
1480 Aside from the token kind names and the symbols in the actions you
1481 write, all symbols defined in the Bison parser implementation file
1482 itself begin with @samp{yy} or @samp{YY}. This includes interface
1483 functions such as the lexical analyzer function @code{yylex}, the
1484 error reporting function @code{yyerror} and the parser function
1485 @code{yyparse} itself. This also includes numerous identifiers used
1486 for internal purposes. Therefore, you should avoid using C
1487 identifiers starting with @samp{yy} or @samp{YY} in the Bison grammar
1488 file except for the ones defined in this manual. Also, you should
1489 avoid using the C identifiers @samp{malloc} and @samp{free} for
1490 anything other than their usual meanings.
1492 In some cases the Bison parser implementation file includes system
1493 headers, and in those cases your code should respect the identifiers
1494 reserved by those headers. On some non-GNU hosts, @code{<limits.h>},
1495 @code{<stddef.h>}, @code{<stdint.h>} (if available), and @code{<stdlib.h>}
1496 are included to declare memory allocators and integer types and constants.
1497 @code{<libintl.h>} is included if message translation is in use
1498 (@pxref{Internationalization}). Other system headers may be included
1499 if you define @code{YYDEBUG} (@pxref{Tracing}) or
1500 @code{YYSTACK_USE_ALLOCA} (@pxref{Table of Symbols}) to a nonzero value.
1503 @section Stages in Using Bison
1504 @cindex stages in using Bison
1507 The actual language-design process using Bison, from grammar specification
1508 to a working compiler or interpreter, has these parts:
1512 Formally specify the grammar in a form recognized by Bison
1513 (@pxref{Grammar File}). For each grammatical rule
1514 in the language, describe the action that is to be taken when an
1515 instance of that rule is recognized. The action is described by a
1516 sequence of C statements.
1519 Write a lexical analyzer to process input and pass tokens to the parser.
1520 The lexical analyzer may be written by hand in C (@pxref{Lexical}). It
1521 could also be produced using Lex, but the use of Lex is not discussed in
1525 Write a controlling function that calls the Bison-produced parser.
1528 Write error-reporting routines.
1531 To turn this source code as written into a runnable program, you
1532 must follow these steps:
1536 Run Bison on the grammar to produce the parser.
1539 Compile the code output by Bison, as well as any other source files.
1542 Link the object files to produce the finished product.
1545 @node Grammar Layout
1546 @section The Overall Layout of a Bison Grammar
1547 @cindex grammar file
1549 @cindex format of grammar file
1550 @cindex layout of Bison grammar
1552 The input file for the Bison utility is a @dfn{Bison grammar file}. The
1553 general form of a Bison grammar file is as follows:
1560 @var{Bison declarations}
1569 The @samp{%%}, @samp{%@{} and @samp{%@}} are punctuation that appears
1570 in every Bison grammar file to separate the sections.
1572 The prologue may define types and variables used in the actions. You can
1573 also use preprocessor commands to define macros used there, and use
1574 @code{#include} to include header files that do any of these things.
1575 You need to declare the lexical analyzer @code{yylex} and the error
1576 printer @code{yyerror} here, along with any other global identifiers
1577 used by the actions in the grammar rules.
1579 The Bison declarations declare the names of the terminal and nonterminal
1580 symbols, and may also describe operator precedence and the data types of
1581 semantic values of various symbols.
1583 The grammar rules define how to construct each nonterminal symbol from its
1586 The epilogue can contain any code you want to use. Often the
1587 definitions of functions declared in the prologue go here. In a
1588 simple program, all the rest of the program can go here.
1592 @cindex simple examples
1593 @cindex examples, simple
1595 Now we show and explain several sample programs written using Bison: a
1596 Reverse Polish Notation calculator, an algebraic (infix) notation
1597 calculator --- later extended to track ``locations'' ---
1598 and a multi-function calculator. All
1599 produce usable, though limited, interactive desk-top calculators.
1601 These examples are simple, but Bison grammars for real programming
1602 languages are written the same way. You can copy these examples into a
1603 source file to try them.
1606 * RPN Calc:: Reverse Polish Notation Calculator;
1607 a first example with no operator precedence.
1608 * Infix Calc:: Infix (algebraic) notation calculator.
1609 Operator precedence is introduced.
1610 * Simple Error Recovery:: Continuing after syntax errors.
1611 * Location Tracking Calc:: Demonstrating the use of @@@var{n} and @@$.
1612 * Multi-function Calc:: Calculator with memory and trig functions.
1613 It uses multiple data-types for semantic values.
1614 * Exercises:: Ideas for improving the multi-function calculator.
1618 @section Reverse Polish Notation Calculator
1619 @cindex Reverse Polish Notation
1620 @cindex @code{rpcalc}
1621 @cindex calculator, simple
1623 The first example is that of a simple double-precision @dfn{Reverse Polish
1624 Notation} calculator (a calculator using postfix operators). This example
1625 provides a good starting point, since operator precedence is not an issue.
1626 The second example will illustrate how operator precedence is handled.
1628 The source code for this calculator is named @file{rpcalc.y}. The
1629 @samp{.y} extension is a convention used for Bison grammar files.
1632 * Rpcalc Declarations:: Prologue (declarations) for rpcalc.
1633 * Rpcalc Rules:: Grammar Rules for rpcalc, with explanation.
1634 * Rpcalc Lexer:: The lexical analyzer.
1635 * Rpcalc Main:: The controlling function.
1636 * Rpcalc Error:: The error reporting function.
1637 * Rpcalc Generate:: Running Bison on the grammar file.
1638 * Rpcalc Compile:: Run the C compiler on the output code.
1641 @node Rpcalc Declarations
1642 @subsection Declarations for @code{rpcalc}
1644 Here are the C and Bison declarations for the Reverse Polish Notation
1645 calculator. As in C, comments are placed between @samp{/*@dots{}*/} or
1649 @comment file: rpcalc.y
1651 /* Parser for rpcalc. -*- C -*-
1653 Copyright (C) 1988-1993, 1995, 1998-2015, 2018-2020 Free Software
1656 This file is part of Bison, the GNU Compiler Compiler.
1658 This program is free software: you can redistribute it and/or modify
1659 it under the terms of the GNU General Public License as published by
1660 the Free Software Foundation, either version 3 of the License, or
1661 (at your option) any later version.
1663 This program is distributed in the hope that it will be useful,
1664 but WITHOUT ANY WARRANTY; without even the implied warranty of
1665 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
1666 GNU General Public License for more details.
1668 You should have received a copy of the GNU General Public License
1669 along with this program. If not, see <http://www.gnu.org/licenses/>. */
1673 @comment file: rpcalc.y
1675 /* Reverse Polish Notation calculator. */
1682 void yyerror (char const *);
1686 %define api.value.type @{double@}
1689 %% /* Grammar rules and actions follow. */
1692 The declarations section (@pxref{Prologue}) contains two
1693 preprocessor directives and two forward declarations.
1695 The @code{#include} directive is used to declare the exponentiation
1696 function @code{pow}.
1698 The forward declarations for @code{yylex} and @code{yyerror} are
1699 needed because the C language requires that functions be declared
1700 before they are used. These functions will be defined in the
1701 epilogue, but the parser calls them so they must be declared in the
1704 The second section, Bison declarations, provides information to Bison about
1705 the tokens and their types (@pxref{Bison Declarations}).
1707 The @code{%define} directive defines the variable @code{api.value.type},
1708 thus specifying the C data type for semantic values of both tokens and
1709 groupings (@pxref{Value Type}). The Bison
1710 parser will use whatever type @code{api.value.type} is defined as; if you
1711 don't define it, @code{int} is the default. Because we specify
1712 @samp{@{double@}}, each token and each expression has an associated value,
1713 which is a floating point number. C code can use @code{YYSTYPE} to refer to
1714 the value @code{api.value.type}.
1716 Each terminal symbol that is not a single-character literal must be
1717 declared. (Single-character literals normally don't need to be declared.)
1718 In this example, all the arithmetic operators are designated by
1719 single-character literals, so the only terminal symbol that needs to be
1720 declared is @code{NUM}, the token kind for numeric constants.
1723 @subsection Grammar Rules for @code{rpcalc}
1725 Here are the grammar rules for the Reverse Polish Notation calculator.
1727 @comment file: rpcalc.y
1739 | exp '\n' @{ printf ("%.10g\n", $1); @}
1746 | exp exp '+' @{ $$ = $1 + $2; @}
1747 | exp exp '-' @{ $$ = $1 - $2; @}
1748 | exp exp '*' @{ $$ = $1 * $2; @}
1749 | exp exp '/' @{ $$ = $1 / $2; @}
1750 | exp exp '^' @{ $$ = pow ($1, $2); @} /* Exponentiation */
1751 | exp 'n' @{ $$ = -$1; @} /* Unary minus */
1757 The groupings of the rpcalc ``language'' defined here are the expression
1758 (given the name @code{exp}), the line of input (@code{line}), and the
1759 complete input transcript (@code{input}). Each of these nonterminal
1760 symbols has several alternate rules, joined by the vertical bar @samp{|}
1761 which is read as ``or''. The following sections explain what these rules
1764 The semantics of the language is determined by the actions taken when a
1765 grouping is recognized. The actions are the C code that appears inside
1766 braces. @xref{Actions}.
1768 You must specify these actions in C, but Bison provides the means for
1769 passing semantic values between the rules. In each action, the
1770 pseudo-variable @code{$$} stands for the semantic value for the grouping
1771 that the rule is going to construct. Assigning a value to @code{$$} is the
1772 main job of most actions. The semantic values of the components of the
1773 rule are referred to as @code{$1}, @code{$2}, and so on.
1776 * Rpcalc Input:: Explanation of the @code{input} nonterminal
1777 * Rpcalc Line:: Explanation of the @code{line} nonterminal
1778 * Rpcalc Expr:: Explanation of the @code{expr} nonterminal
1782 @subsubsection Explanation of @code{input}
1784 Consider the definition of @code{input}:
1793 This definition reads as follows: ``A complete input is either an empty
1794 string, or a complete input followed by an input line''. Notice that
1795 ``complete input'' is defined in terms of itself. This definition is said
1796 to be @dfn{left recursive} since @code{input} appears always as the
1797 leftmost symbol in the sequence. @xref{Recursion}.
1799 The first alternative is empty because there are no symbols between the
1800 colon and the first @samp{|}; this means that @code{input} can match an
1801 empty string of input (no tokens). We write the rules this way because it
1802 is legitimate to type @kbd{Ctrl-d} right after you start the calculator.
1803 It's conventional to put an empty alternative first and to use the
1804 (optional) @code{%empty} directive, or to write the comment @samp{/* empty
1805 */} in it (@pxref{Empty Rules}).
1807 The second alternate rule (@code{input line}) handles all nontrivial input.
1808 It means, ``After reading any number of lines, read one more line if
1809 possible.'' The left recursion makes this rule into a loop. Since the
1810 first alternative matches empty input, the loop can be executed zero or
1813 The parser function @code{yyparse} continues to process input until a
1814 grammatical error is seen or the lexical analyzer says there are no more
1815 input tokens; we will arrange for the latter to happen at end-of-input.
1818 @subsubsection Explanation of @code{line}
1820 Now consider the definition of @code{line}:
1825 | exp '\n' @{ printf ("%.10g\n", $1); @}
1829 The first alternative is a token which is a newline character; this means
1830 that rpcalc accepts a blank line (and ignores it, since there is no
1831 action). The second alternative is an expression followed by a newline.
1832 This is the alternative that makes rpcalc useful. The semantic value of
1833 the @code{exp} grouping is the value of @code{$1} because the @code{exp} in
1834 question is the first symbol in the alternative. The action prints this
1835 value, which is the result of the computation the user asked for.
1837 This action is unusual because it does not assign a value to @code{$$}. As
1838 a consequence, the semantic value associated with the @code{line} is
1839 uninitialized (its value will be unpredictable). This would be a bug if
1840 that value were ever used, but we don't use it: once rpcalc has printed the
1841 value of the user's input line, that value is no longer needed.
1844 @subsubsection Explanation of @code{expr}
1846 The @code{exp} grouping has several rules, one for each kind of expression.
1847 The first rule handles the simplest expressions: those that are just
1848 numbers. The second handles an addition-expression, which looks like two
1849 expressions followed by a plus-sign. The third handles subtraction, and so
1855 | exp exp '+' @{ $$ = $1 + $2; @}
1856 | exp exp '-' @{ $$ = $1 - $2; @}
1861 We have used @samp{|} to join all the rules for @code{exp}, but we could
1862 equally well have written them separately:
1866 exp: exp exp '+' @{ $$ = $1 + $2; @};
1867 exp: exp exp '-' @{ $$ = $1 - $2; @};
1871 Most of the rules have actions that compute the value of the expression in
1872 terms of the value of its parts. For example, in the rule for addition,
1873 @code{$1} refers to the first component @code{exp} and @code{$2} refers to
1874 the second one. The third component, @code{'+'}, has no meaningful
1875 associated semantic value, but if it had one you could refer to it as
1876 @code{$3}. The first rule relies on the implicit default action: @samp{@{
1880 When @code{yyparse} recognizes a sum expression using this rule, the sum of
1881 the two subexpressions' values is produced as the value of the entire
1882 expression. @xref{Actions}.
1884 You don't have to give an action for every rule. When a rule has no action,
1885 Bison by default copies the value of @code{$1} into @code{$$}. This is what
1886 happens in the first rule (the one that uses @code{NUM}).
1888 The formatting shown here is the recommended convention, but Bison does not
1889 require it. You can add or change white space as much as you wish. For
1893 exp: NUM | exp exp '+' @{$$ = $1 + $2; @} | @dots{} ;
1897 means the same thing as this:
1902 | exp exp '+' @{ $$ = $1 + $2; @}
1908 The latter, however, is much more readable.
1911 @subsection The @code{rpcalc} Lexical Analyzer
1912 @cindex writing a lexical analyzer
1913 @cindex lexical analyzer, writing
1915 The lexical analyzer's job is low-level parsing: converting characters
1916 or sequences of characters into tokens. The Bison parser gets its
1917 tokens by calling the lexical analyzer. @xref{Lexical}.
1919 Only a simple lexical analyzer is needed for the RPN
1921 lexical analyzer skips blanks and tabs, then reads in numbers as
1922 @code{double} and returns them as @code{NUM} tokens. Any other character
1923 that isn't part of a number is a separate token. Note that the token-code
1924 for such a single-character token is the character itself.
1926 The return value of the lexical analyzer function is a numeric code which
1927 represents a token kind. The same text used in Bison rules to stand for
1928 this token kind is also a C expression for the numeric code of the kind.
1929 This works in two ways. If the token kind is a character literal, then its
1930 numeric code is that of the character; you can use the same character
1931 literal in the lexical analyzer to express the number. If the token kind is
1932 an identifier, that identifier is defined by Bison as a C enum whose
1933 definition is the appropriate code. In this example, therefore, @code{NUM}
1934 becomes an enum for @code{yylex} to use.
1936 The semantic value of the token (if it has one) is stored into the global
1937 variable @code{yylval}, which is where the Bison parser will look for it.
1938 (The C data type of @code{yylval} is @code{YYSTYPE}, whose value was defined
1939 at the beginning of the grammar via @samp{%define api.value.type
1940 @{double@}}; @pxref{Rpcalc Declarations}.)
1942 A token kind code of zero is returned if the end-of-input is encountered.
1943 (Bison recognizes any nonpositive value as indicating end-of-input.)
1945 Here is the code for the lexical analyzer:
1947 @comment file: rpcalc.y
1950 /* The lexical analyzer returns a double floating point
1951 number on the stack and the token NUM, or the numeric code
1952 of the character read if not a number. It skips all blanks
1953 and tabs, and returns 0 for end-of-input. */
1964 /* Skip white space. */
1965 while (c == ' ' || c == '\t')
1969 /* Process numbers. */
1970 if (c == '.' || isdigit (c))
1973 if (scanf ("%lf", &yylval) != 1)
1979 /* Return end-of-input. */
1982 /* Return a single char. */
1990 @subsection The Controlling Function
1991 @cindex controlling function
1992 @cindex main function in simple example
1994 In keeping with the spirit of this example, the controlling function is
1995 kept to the bare minimum. The only requirement is that it call
1996 @code{yyparse} to start the process of parsing.
1998 @comment file: rpcalc.y
2010 @subsection The Error Reporting Routine
2011 @cindex error reporting routine
2013 When @code{yyparse} detects a syntax error, it calls the error reporting
2014 function @code{yyerror} to print an error message (usually but not
2015 always @code{"syntax error"}). It is up to the programmer to supply
2016 @code{yyerror} (@pxref{Interface}), so
2017 here is the definition we will use:
2019 @comment file: rpcalc.y
2024 /* Called by yyparse on error. */
2026 yyerror (char const *s)
2028 fprintf (stderr, "%s\n", s);
2033 After @code{yyerror} returns, the Bison parser may recover from the error
2034 and continue parsing if the grammar contains a suitable error rule
2035 (@pxref{Error Recovery}). Otherwise, @code{yyparse} returns nonzero. We
2036 have not written any error rules in this example, so any invalid input will
2037 cause the calculator program to exit. This is not clean behavior for a
2038 real calculator, but it is adequate for the first example.
2040 @node Rpcalc Generate
2041 @subsection Running Bison to Make the Parser
2042 @cindex running Bison (introduction)
2044 Before running Bison to produce a parser, we need to decide how to
2045 arrange all the source code in one or more source files. For such a
2046 simple example, the easiest thing is to put everything in one file,
2047 the grammar file. The definitions of @code{yylex}, @code{yyerror} and
2048 @code{main} go at the end, in the epilogue of the grammar file
2049 (@pxref{Grammar Layout}).
2051 For a large project, you would probably have several source files, and use
2052 @code{make} to arrange to recompile them.
2054 With all the source in the grammar file, you use the following command
2055 to convert it into a parser implementation file:
2058 $ @kbd{bison @var{file}.y}
2062 In this example, the grammar file is called @file{rpcalc.y} (for
2063 ``Reverse Polish @sc{calc}ulator''). Bison produces a parser
2064 implementation file named @file{@var{file}.tab.c}, removing the
2065 @samp{.y} from the grammar file name. The parser implementation file
2066 contains the source code for @code{yyparse}. The additional functions
2067 in the grammar file (@code{yylex}, @code{yyerror} and @code{main}) are
2068 copied verbatim to the parser implementation file.
2070 @node Rpcalc Compile
2071 @subsection Compiling the Parser Implementation File
2072 @cindex compiling the parser
2074 Here is how to compile and run the parser implementation file:
2078 # @r{List files in current directory.}
2080 rpcalc.tab.c rpcalc.y
2084 # @r{Compile the Bison parser.}
2085 # @r{@option{-lm} tells compiler to search math library for @code{pow}.}
2086 $ @kbd{cc -lm -o rpcalc rpcalc.tab.c}
2090 # @r{List files again.}
2092 rpcalc rpcalc.tab.c rpcalc.y
2096 The file @file{rpcalc} now contains the executable code. Here is an
2097 example session using @code{rpcalc}.
2103 @kbd{3 7 + 3 4 5 *+-}
2105 @kbd{3 7 + 3 4 5 * + - n} @r{Note the unary minus, @samp{n}}
2108 @result{} -3.166666667
2109 @kbd{3 4 ^} @r{Exponentiation}
2111 @kbd{^D} @r{End-of-file indicator}
2116 @section Infix Notation Calculator: @code{calc}
2117 @cindex infix notation calculator
2119 @cindex calculator, infix notation
2121 We now modify rpcalc to handle infix operators instead of postfix. Infix
2122 notation involves the concept of operator precedence and the need for
2123 parentheses nested to arbitrary depth. Here is the Bison code for
2124 @file{calc.y}, an infix desk-top calculator.
2127 /* Infix notation calculator. */
2134 void yyerror (char const *);
2139 /* Bison declarations. */
2140 %define api.value.type @{double@}
2144 %precedence NEG /* negation--unary minus */
2145 %right '^' /* exponentiation */
2148 %% /* The grammar follows. */
2159 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2166 | exp '+' exp @{ $$ = $1 + $3; @}
2167 | exp '-' exp @{ $$ = $1 - $3; @}
2168 | exp '*' exp @{ $$ = $1 * $3; @}
2169 | exp '/' exp @{ $$ = $1 / $3; @}
2170 | '-' exp %prec NEG @{ $$ = -$2; @}
2171 | exp '^' exp @{ $$ = pow ($1, $3); @}
2172 | '(' exp ')' @{ $$ = $2; @}
2179 The functions @code{yylex}, @code{yyerror} and @code{main} can be the
2182 There are two important new features shown in this code.
2184 In the second section (Bison declarations), @code{%left} declares token
2185 kinds and says they are left-associative operators. The declarations
2186 @code{%left} and @code{%right} (right associativity) take the place of
2187 @code{%token} which is used to declare a token kind name without
2188 associativity/precedence. (These tokens are single-character literals,
2189 which ordinarily don't need to be declared. We declare them here to specify
2190 the associativity/precedence.)
2192 Operator precedence is determined by the line ordering of the
2193 declarations; the higher the line number of the declaration (lower on
2194 the page or screen), the higher the precedence. Hence, exponentiation
2195 has the highest precedence, unary minus (@code{NEG}) is next, followed
2196 by @samp{*} and @samp{/}, and so on. Unary minus is not associative,
2197 only precedence matters (@code{%precedence}. @xref{Precedence}.
2199 The other important new feature is the @code{%prec} in the grammar
2200 section for the unary minus operator. The @code{%prec} simply instructs
2201 Bison that the rule @samp{| '-' exp} has the same precedence as
2202 @code{NEG}---in this case the next-to-highest. @xref{Contextual
2205 Here is a sample run of @file{calc.y}:
2210 @kbd{4 + 4.5 - (34/(8*3+-3))}
2218 @node Simple Error Recovery
2219 @section Simple Error Recovery
2220 @cindex error recovery, simple
2222 Up to this point, this manual has not addressed the issue of @dfn{error
2223 recovery}---how to continue parsing after the parser detects a syntax
2224 error. All we have handled is error reporting with @code{yyerror}.
2225 Recall that by default @code{yyparse} returns after calling
2226 @code{yyerror}. This means that an erroneous input line causes the
2227 calculator program to exit. Now we show how to rectify this deficiency.
2229 The Bison language itself includes the reserved word @code{error}, which
2230 may be included in the grammar rules. In the example below it has
2231 been added to one of the alternatives for @code{line}:
2237 | exp '\n' @{ printf ("\t%.10g\n", $1); @}
2238 | error '\n' @{ yyerrok; @}
2243 This addition to the grammar allows for simple error recovery in the
2244 event of a syntax error. If an expression that cannot be evaluated is
2245 read, the error will be recognized by the third rule for @code{line},
2246 and parsing will continue. (The @code{yyerror} function is still called
2247 upon to print its message as well.) The action executes the statement
2248 @code{yyerrok}, a macro defined automatically by Bison; its meaning is
2249 that error recovery is complete (@pxref{Error Recovery}). Note the
2250 difference between @code{yyerrok} and @code{yyerror}; neither one is a
2253 This form of error recovery deals with syntax errors. There are other
2254 kinds of errors; for example, division by zero, which raises an exception
2255 signal that is normally fatal. A real calculator program must handle this
2256 signal and use @code{longjmp} to return to @code{main} and resume parsing
2257 input lines; it would also have to discard the rest of the current line of
2258 input. We won't discuss this issue further because it is not specific to
2261 @node Location Tracking Calc
2262 @section Location Tracking Calculator: @code{ltcalc}
2263 @cindex location tracking calculator
2264 @cindex @code{ltcalc}
2265 @cindex calculator, location tracking
2267 This example extends the infix notation calculator with location
2268 tracking. This feature will be used to improve the error messages. For
2269 the sake of clarity, this example is a simple integer calculator, since
2270 most of the work needed to use locations will be done in the lexical
2274 * Ltcalc Declarations:: Bison and C declarations for ltcalc.
2275 * Ltcalc Rules:: Grammar rules for ltcalc, with explanations.
2276 * Ltcalc Lexer:: The lexical analyzer.
2279 @node Ltcalc Declarations
2280 @subsection Declarations for @code{ltcalc}
2282 The C and Bison declarations for the location tracking calculator are
2283 the same as the declarations for the infix notation calculator.
2286 /* Location tracking calculator. */
2291 void yyerror (char const *);
2294 /* Bison declarations. */
2295 %define api.value.type @{int@}
2303 %% /* The grammar follows. */
2307 Note there are no declarations specific to locations. Defining a data type
2308 for storing locations is not needed: we will use the type provided by
2309 default (@pxref{Location Type}), which is a four member structure with the
2310 following integer fields: @code{first_line}, @code{first_column},
2311 @code{last_line} and @code{last_column}. By conventions, and in accordance
2312 with the GNU Coding Standards and common practice, the line and column count
2316 @subsection Grammar Rules for @code{ltcalc}
2318 Whether handling locations or not has no effect on the syntax of your
2319 language. Therefore, grammar rules for this example will be very close
2320 to those of the previous example: we will only modify them to benefit
2321 from the new information.
2323 Here, we will use locations to report divisions by zero, and locate the
2324 wrong expressions or subexpressions.
2337 | exp '\n' @{ printf ("%d\n", $1); @}
2344 | exp '+' exp @{ $$ = $1 + $3; @}
2345 | exp '-' exp @{ $$ = $1 - $3; @}
2346 | exp '*' exp @{ $$ = $1 * $3; @}
2356 fprintf (stderr, "%d.%d-%d.%d: division by zero",
2357 @@3.first_line, @@3.first_column,
2358 @@3.last_line, @@3.last_column);
2363 | '-' exp %prec NEG @{ $$ = -$2; @}
2364 | exp '^' exp @{ $$ = pow ($1, $3); @}
2365 | '(' exp ')' @{ $$ = $2; @}
2369 This code shows how to reach locations inside of semantic actions, by
2370 using the pseudo-variables @code{@@@var{n}} for rule components, and the
2371 pseudo-variable @code{@@$} for groupings.
2373 We don't need to assign a value to @code{@@$}: the output parser does it
2374 automatically. By default, before executing the C code of each action,
2375 @code{@@$} is set to range from the beginning of @code{@@1} to the end of
2376 @code{@@@var{n}}, for a rule with @var{n} components. This behavior can be
2377 redefined (@pxref{Location Default Action}), and for very specific rules,
2378 @code{@@$} can be computed by hand.
2381 @subsection The @code{ltcalc} Lexical Analyzer.
2383 Until now, we relied on Bison's defaults to enable location
2384 tracking. The next step is to rewrite the lexical analyzer, and make it
2385 able to feed the parser with the token locations, as it already does for
2388 To this end, we must take into account every single character of the
2389 input text, to avoid the computed locations of being fuzzy or wrong:
2400 /* Skip white space. */
2401 while ((c = getchar ()) == ' ' || c == '\t')
2402 ++yylloc.last_column;
2407 yylloc.first_line = yylloc.last_line;
2408 yylloc.first_column = yylloc.last_column;
2412 /* Process numbers. */
2416 ++yylloc.last_column;
2417 while (isdigit (c = getchar ()))
2419 ++yylloc.last_column;
2420 yylval = yylval * 10 + c - '0';
2427 /* Return end-of-input. */
2432 /* Return a single char, and update location. */
2436 yylloc.last_column = 0;
2439 ++yylloc.last_column;
2445 Basically, the lexical analyzer performs the same processing as before: it
2446 skips blanks and tabs, and reads numbers or single-character tokens. In
2447 addition, it updates @code{yylloc}, the global variable (of type
2448 @code{YYLTYPE}) containing the token's location.
2450 Now, each time this function returns a token, the parser has its kind as
2451 well as its semantic value, and its location in the text. The last needed
2452 change is to initialize @code{yylloc}, for example in the controlling
2460 yylloc.first_line = yylloc.last_line = 1;
2461 yylloc.first_column = yylloc.last_column = 0;
2467 Remember that computing locations is not a matter of syntax. Every
2468 character must be associated to a location update, whether it is in
2469 valid input, in comments, in literal strings, and so on.
2471 @node Multi-function Calc
2472 @section Multi-Function Calculator: @code{mfcalc}
2473 @cindex multi-function calculator
2474 @cindex @code{mfcalc}
2475 @cindex calculator, multi-function
2477 Now that the basics of Bison have been discussed, it is time to move on to
2478 a more advanced problem. The above calculators provided only five
2479 functions, @samp{+}, @samp{-}, @samp{*}, @samp{/} and @samp{^}. It would
2480 be nice to have a calculator that provides other mathematical functions such
2481 as @code{sin}, @code{cos}, etc.
2483 It is easy to add new operators to the infix calculator as long as they are
2484 only single-character literals. The lexical analyzer @code{yylex} passes
2485 back all nonnumeric characters as tokens, so new grammar rules suffice for
2486 adding a new operator. But we want something more flexible: built-in
2487 functions whose syntax has this form:
2490 @var{function_name} (@var{argument})
2494 At the same time, we will add memory to the calculator, by allowing you
2495 to create named variables, store values in them, and use them later.
2496 Here is a sample session with the multi-function calculator:
2501 @kbd{pi = 3.141592653589}
2502 @result{} 3.1415926536
2506 @result{} 0.0000000000
2508 @kbd{alpha = beta1 = 2.3}
2509 @result{} 2.3000000000
2511 @result{} 2.3000000000
2513 @result{} 0.8329091229
2514 @kbd{exp(ln(beta1))}
2515 @result{} 2.3000000000
2519 Note that multiple assignment and nested function calls are permitted.
2522 * Mfcalc Declarations:: Bison declarations for multi-function calculator.
2523 * Mfcalc Rules:: Grammar rules for the calculator.
2524 * Mfcalc Symbol Table:: Symbol table management subroutines.
2525 * Mfcalc Lexer:: The lexical analyzer.
2526 * Mfcalc Main:: The controlling function.
2529 @node Mfcalc Declarations
2530 @subsection Declarations for @code{mfcalc}
2532 Here are the C and Bison declarations for the multi-function calculator.
2535 @comment file: mfcalc.y
2537 /* Parser for mfcalc. -*- C -*-
2539 Copyright (C) 1988-1993, 1995, 1998-2015, 2018-2020 Free Software
2542 This file is part of Bison, the GNU Compiler Compiler.
2544 This program is free software: you can redistribute it and/or modify
2545 it under the terms of the GNU General Public License as published by
2546 the Free Software Foundation, either version 3 of the License, or
2547 (at your option) any later version.
2549 This program is distributed in the hope that it will be useful,
2550 but WITHOUT ANY WARRANTY; without even the implied warranty of
2551 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2552 GNU General Public License for more details.
2554 You should have received a copy of the GNU General Public License
2555 along with this program. If not, see <http://www.gnu.org/licenses/>. */
2559 @comment file: mfcalc.y: 1
2563 #include <stdio.h> /* For printf, etc. */
2564 #include <math.h> /* For pow, used in the grammar. */
2565 #include "calc.h" /* Contains definition of 'symrec'. */
2567 void yyerror (char const *);
2571 %define api.value.type union /* Generate YYSTYPE from these types: */
2572 %token <double> NUM /* Double precision number. */
2573 %token <symrec*> VAR FUN /* Symbol table pointer: variable/function. */
2580 %precedence NEG /* negation--unary minus */
2581 %right '^' /* exponentiation */
2585 The above grammar introduces only two new features of the Bison language.
2586 These features allow semantic values to have various data types
2587 (@pxref{Multiple Types}).
2589 The special @code{union} value assigned to the @code{%define} variable
2590 @code{api.value.type} specifies that the symbols are defined with their data
2591 types. Bison will generate an appropriate definition of @code{YYSTYPE} to
2594 Since values can now have various types, it is necessary to associate a type
2595 with each grammar symbol whose semantic value is used. These symbols are
2596 @code{NUM}, @code{VAR}, @code{FUN}, and @code{exp}. Their declarations are
2597 augmented with their data type (placed between angle brackets). For
2598 instance, values of @code{NUM} are stored in @code{double}.
2600 The Bison construct @code{%nterm} is used for declaring nonterminal symbols,
2601 just as @code{%token} is used for declaring token kinds. Previously we did
2602 not use @code{%nterm} before because nonterminal symbols are normally
2603 declared implicitly by the rules that define them. But @code{exp} must be
2604 declared explicitly so we can specify its value type. @xref{Type Decl}.
2607 @subsection Grammar Rules for @code{mfcalc}
2609 Here are the grammar rules for the multi-function calculator.
2610 Most of them are copied directly from @code{calc}; three rules,
2611 those which mention @code{VAR} or @code{FUN}, are new.
2613 @comment file: mfcalc.y: 3
2615 %% /* The grammar follows. */
2626 | exp '\n' @{ printf ("%.10g\n", $1); @}
2627 | error '\n' @{ yyerrok; @}
2634 | VAR @{ $$ = $1->value.var; @}
2635 | VAR '=' exp @{ $$ = $3; $1->value.var = $3; @}
2636 | FUN '(' exp ')' @{ $$ = $1->value.fun ($3); @}
2637 | exp '+' exp @{ $$ = $1 + $3; @}
2638 | exp '-' exp @{ $$ = $1 - $3; @}
2639 | exp '*' exp @{ $$ = $1 * $3; @}
2640 | exp '/' exp @{ $$ = $1 / $3; @}
2641 | '-' exp %prec NEG @{ $$ = -$2; @}
2642 | exp '^' exp @{ $$ = pow ($1, $3); @}
2643 | '(' exp ')' @{ $$ = $2; @}
2646 /* End of grammar. */
2650 @node Mfcalc Symbol Table
2651 @subsection The @code{mfcalc} Symbol Table
2652 @cindex symbol table example
2654 The multi-function calculator requires a symbol table to keep track of the
2655 names and meanings of variables and functions. This doesn't affect the
2656 grammar rules (except for the actions) or the Bison declarations, but it
2657 requires some additional C functions for support.
2659 The symbol table itself consists of a linked list of records. Its
2660 definition, which is kept in the header @file{calc.h}, is as follows. It
2661 provides for either functions or variables to be placed in the table.
2664 @comment file: calc.h
2666 /* Functions for mfcalc. -*- C -*-
2668 Copyright (C) 1988-1993, 1995, 1998-2015, 2018-2020 Free Software
2671 This file is part of Bison, the GNU Compiler Compiler.
2673 This program is free software: you can redistribute it and/or modify
2674 it under the terms of the GNU General Public License as published by
2675 the Free Software Foundation, either version 3 of the License, or
2676 (at your option) any later version.
2678 This program is distributed in the hope that it will be useful,
2679 but WITHOUT ANY WARRANTY; without even the implied warranty of
2680 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
2681 GNU General Public License for more details.
2683 You should have received a copy of the GNU General Public License
2684 along with this program. If not, see <http://www.gnu.org/licenses/>. */
2688 @comment file: calc.h
2691 /* Function type. */
2692 typedef double (func_t) (double);
2696 /* Data type for links in the chain of symbols. */
2699 char *name; /* name of symbol */
2700 int type; /* type of symbol: either VAR or FUN */
2703 double var; /* value of a VAR */
2704 func_t *fun; /* value of a FUN */
2706 struct symrec *next; /* link field */
2711 typedef struct symrec symrec;
2713 /* The symbol table: a chain of 'struct symrec'. */
2714 extern symrec *sym_table;
2716 symrec *putsym (char const *name, int sym_type);
2717 symrec *getsym (char const *name);
2721 The new version of @code{main} will call @code{init_table} to initialize
2724 @comment file: mfcalc.y: 3
2735 struct init const funs[] =
2748 /* The symbol table: a chain of 'struct symrec'. */
2753 /* Put functions in table. */
2759 for (int i = 0; funs[i].name; i++)
2761 symrec *ptr = putsym (funs[i].name, FUN);
2762 ptr->value.fun = funs[i].fun;
2768 By simply editing the initialization list and adding the necessary include
2769 files, you can add additional functions to the calculator.
2771 Two important functions allow look-up and installation of symbols in the
2772 symbol table. The function @code{putsym} is passed a name and the kind
2773 (@code{VAR} or @code{FUN}) of the object to be installed. The object is
2774 linked to the front of the list, and a pointer to the object is returned.
2775 The function @code{getsym} is passed the name of the symbol to look up. If
2776 found, a pointer to that symbol is returned; otherwise zero is returned.
2778 @comment file: mfcalc.y: 3
2781 /* The mfcalc code assumes that malloc and realloc
2782 always succeed, and that integer calculations
2783 never overflow. Production-quality code should
2784 not make these assumptions. */
2786 #include <stdlib.h> /* malloc, realloc. */
2787 #include <string.h> /* strlen. */
2792 putsym (char const *name, int sym_type)
2794 symrec *res = (symrec *) malloc (sizeof (symrec));
2795 res->name = strdup (name);
2796 res->type = sym_type;
2797 res->value.var = 0; /* Set value to 0 even if fun. */
2798 res->next = sym_table;
2806 getsym (char const *name)
2808 for (symrec *p = sym_table; p; p = p->next)
2809 if (strcmp (p->name, name) == 0)
2817 @subsection The @code{mfcalc} Lexer
2819 The function @code{yylex} must now recognize variables, numeric values, and
2820 the single-character arithmetic operators. Strings of alphanumeric
2821 characters with a leading letter are recognized as either variables or
2822 functions depending on what the symbol table says about them.
2824 The string is passed to @code{getsym} for look up in the symbol table. If
2825 the name appears in the table, a pointer to its location and its type
2826 (@code{VAR} or @code{FUN}) is returned to @code{yyparse}. If it is not
2827 already in the table, then it is installed as a @code{VAR} using
2828 @code{putsym}. Again, a pointer and its type (which must be @code{VAR}) is
2829 returned to @code{yyparse}.
2831 No change is needed in the handling of numeric values and arithmetic
2832 operators in @code{yylex}.
2834 @comment file: mfcalc.y: 3
2845 /* Ignore white space, get first nonwhite character. */
2846 while (c == ' ' || c == '\t')
2854 /* Char starts a number => parse the number. */
2855 if (c == '.' || isdigit (c))
2858 if (scanf ("%lf", &yylval.NUM) != 1)
2866 Bison generated a definition of @code{YYSTYPE} with a member named
2867 @code{NUM} to store value of @code{NUM} symbols.
2869 @comment file: mfcalc.y: 3
2872 /* Char starts an identifier => read the name. */
2875 static ptrdiff_t bufsize = 0;
2876 static char *symbuf = 0;
2882 /* If buffer is full, make it bigger. */
2885 bufsize = 2 * bufsize + 40;
2886 symbuf = realloc (symbuf, (size_t) bufsize);
2888 /* Add this character to the buffer. */
2889 symbuf[i++] = (char) c;
2890 /* Get another character. */
2895 while (isalnum (c));
2902 symrec *s = getsym (symbuf);
2904 s = putsym (symbuf, VAR);
2905 yylval.VAR = s; /* or yylval.FUN = s. */
2909 /* Any other character is a token by itself. */
2916 @subsection The @code{mfcalc} Main
2918 The error reporting function is unchanged, and the new version of
2919 @code{main} includes a call to @code{init_table} and sets the @code{yydebug}
2920 on user demand (@xref{Tracing}, for details):
2922 @comment file: mfcalc.y: 3
2925 /* Called by yyparse on error. */
2926 void yyerror (char const *s)
2928 fprintf (stderr, "%s\n", s);
2933 int main (int argc, char const* argv[])
2937 /* Enable parse traces on option -p. */
2938 if (argc == 2 && strcmp(argv[1], "-p") == 0)
2948 This program is both powerful and flexible. You may easily add new
2949 functions, and it is a simple job to modify this code to install
2950 predefined variables such as @code{pi} or @code{e} as well.
2958 Add some new functions from @file{math.h} to the initialization list.
2961 Add another array that contains constants and their values. Then modify
2962 @code{init_table} to add these constants to the symbol table. It will be
2963 easiest to give the constants type @code{VAR}.
2966 Make the program report an error if the user refers to an uninitialized
2967 variable in any way except to store a value in it.
2971 @chapter Bison Grammar Files
2973 Bison takes as input a context-free grammar specification and produces a
2974 C-language function that recognizes correct instances of the grammar.
2976 The Bison grammar file conventionally has a name ending in @samp{.y}.
2980 * Grammar Outline:: Overall layout of the grammar file.
2981 * Symbols:: Terminal and nonterminal symbols.
2982 * Rules:: How to write grammar rules.
2983 * Semantics:: Semantic values and actions.
2984 * Tracking Locations:: Locations and actions.
2985 * Named References:: Using named references in actions.
2986 * Declarations:: All kinds of Bison declarations are described here.
2987 * Multiple Parsers:: Putting more than one Bison parser in one program.
2990 @node Grammar Outline
2991 @section Outline of a Bison Grammar
2994 @findex /* @dots{} */
2996 A Bison grammar file has four main sections, shown here with the
2997 appropriate delimiters:
3004 @var{Bison declarations}
3013 Comments enclosed in @samp{/* @dots{} */} may appear in any of the sections.
3014 As a GNU extension, @samp{//} introduces a comment that continues until end
3018 * Prologue:: Syntax and usage of the prologue.
3019 * Prologue Alternatives:: Syntax and usage of alternatives to the prologue.
3020 * Bison Declarations:: Syntax and usage of the Bison declarations section.
3021 * Grammar Rules:: Syntax and usage of the grammar rules section.
3022 * Epilogue:: Syntax and usage of the epilogue.
3026 @subsection The prologue
3027 @cindex declarations section
3029 @cindex declarations
3031 The @var{Prologue} section contains macro definitions and declarations of
3032 functions and variables that are used in the actions in the grammar rules.
3033 These are copied to the beginning of the parser implementation file so that
3034 they precede the definition of @code{yyparse}. You can use @samp{#include}
3035 to get the declarations from a header file. If you don't need any C
3036 declarations, you may omit the @samp{%@{} and @samp{%@}} delimiters that
3037 bracket this section.
3039 The @var{Prologue} section is terminated by the first occurrence of
3040 @samp{%@}} that is outside a comment, a string literal, or a character
3043 You may have more than one @var{Prologue} section, intermixed with the
3044 @var{Bison declarations}. This allows you to have C and Bison declarations
3045 that refer to each other. For example, the @code{%union} declaration may
3046 use types defined in a header file, and you may wish to prototype functions
3047 that take arguments of type @code{YYSTYPE}. This can be done with two
3048 @var{Prologue} blocks, one before and one after the @code{%union}
3063 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3069 static void print_token (yytoken_kind_t token, YYSTYPE val);
3076 When in doubt, it is usually safer to put prologue code before all Bison
3077 declarations, rather than after. For example, any definitions of feature
3078 test macros like @code{_GNU_SOURCE} or @code{_POSIX_C_SOURCE} should appear
3079 before all Bison declarations, as feature test macros can affect the
3080 behavior of Bison-generated @code{#include} directives.
3082 @node Prologue Alternatives
3083 @subsection Prologue Alternatives
3084 @cindex Prologue Alternatives
3087 @findex %code requires
3088 @findex %code provides
3091 The functionality of @var{Prologue} sections can often be subtle and
3092 inflexible. As an alternative, Bison provides a @code{%code} directive with
3093 an explicit qualifier field, which identifies the purpose of the code and
3094 thus the location(s) where Bison should generate it. For C/C++, the
3095 qualifier can be omitted for the default location, or it can be one of
3096 @code{requires}, @code{provides}, @code{top}. @xref{%code Summary}.
3098 Look again at the example of the previous section:
3112 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3118 static void print_token (yytoken_kind_t token, YYSTYPE val);
3126 Notice that there are two @var{Prologue} sections here, but there's a subtle
3127 distinction between their functionality. For example, if you decide to
3128 override Bison's default definition for @code{YYLTYPE}, in which
3129 @var{Prologue} section should you write your new definition? You should
3130 write it in the first since Bison will insert that code into the parser
3131 implementation file @emph{before} the default @code{YYLTYPE} definition. In
3132 which @var{Prologue} section should you prototype an internal function,
3133 @code{trace_token}, that accepts @code{YYLTYPE} and @code{yytoken_kind_t} as
3134 arguments? You should prototype it in the second since Bison will insert
3135 that code @emph{after} the @code{YYLTYPE} and @code{yytoken_kind_t}
3138 This distinction in functionality between the two @var{Prologue} sections is
3139 established by the appearance of the @code{%union} between them. This
3140 behavior raises a few questions. First, why should the position of a
3141 @code{%union} affect definitions related to @code{YYLTYPE} and
3142 @code{yytoken_kind_t}? Second, what if there is no @code{%union}? In that
3143 case, the second kind of @var{Prologue} section is not available. This
3144 behavior is not intuitive.
3146 To avoid this subtle @code{%union} dependency, rewrite the example using a
3147 @code{%code top} and an unqualified @code{%code}. Let's go ahead and add
3148 the new @code{YYLTYPE} definition and the @code{trace_token} prototype at
3156 /* WARNING: The following code really belongs
3157 * in a '%code requires'; see below. */
3160 #define YYLTYPE YYLTYPE
3161 typedef struct YYLTYPE
3174 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3180 static void print_token (yytoken_kind_t token, YYSTYPE val);
3181 static void trace_token (yytoken_kind_t token, YYLTYPE loc);
3189 In this way, @code{%code top} and the unqualified @code{%code} achieve the
3190 same functionality as the two kinds of @var{Prologue} sections, but it's
3191 always explicit which kind you intend. Moreover, both kinds are always
3192 available even in the absence of @code{%union}.
3194 The @code{%code top} block above logically contains two parts. The first
3195 two lines before the warning need to appear near the top of the parser
3196 implementation file. The first line after the warning is required by
3197 @code{YYSTYPE} and thus also needs to appear in the parser implementation
3198 file. However, if you've instructed Bison to generate a parser header file
3199 (@pxref{Decl Summary}), you probably want that line to appear
3200 before the @code{YYSTYPE} definition in that header file as well. The
3201 @code{YYLTYPE} definition should also appear in the parser header file to
3202 override the default @code{YYLTYPE} definition there.
3204 In other words, in the @code{%code top} block above, all but the first two
3205 lines are dependency code required by the @code{YYSTYPE} and @code{YYLTYPE}
3207 Thus, they belong in one or more @code{%code requires}:
3225 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3231 #define YYLTYPE YYLTYPE
3232 typedef struct YYLTYPE
3245 static void print_token (yytoken_kind_t token, YYSTYPE val);
3246 static void trace_token (yytoken_kind_t token, YYLTYPE loc);
3254 Now Bison will insert @code{#include "ptypes.h"} and the new @code{YYLTYPE}
3255 definition before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
3256 definitions in both the parser implementation file and the parser header
3257 file. (By the same reasoning, @code{%code requires} would also be the
3258 appropriate place to write your own definition for @code{YYSTYPE}.)
3260 When you are writing dependency code for @code{YYSTYPE} and @code{YYLTYPE},
3261 you should prefer @code{%code requires} over @code{%code top} regardless of
3262 whether you instruct Bison to generate a parser header file. When you are
3263 writing code that you need Bison to insert only into the parser
3264 implementation file and that has no special need to appear at the top of
3265 that file, you should prefer the unqualified @code{%code} over @code{%code
3266 top}. These practices will make the purpose of each block of your code
3267 explicit to Bison and to other developers reading your grammar file.
3268 Following these practices, we expect the unqualified @code{%code} and
3269 @code{%code requires} to be the most important of the four @var{Prologue}
3272 At some point while developing your parser, you might decide to provide
3273 @code{trace_token} to modules that are external to your parser. Thus, you
3274 might wish for Bison to insert the prototype into both the parser header
3275 file and the parser implementation file. Since this function is not a
3276 dependency required by @code{YYSTYPE} or @code{YYLTYPE}, it doesn't make
3277 sense to move its prototype to a @code{%code requires}. More importantly,
3278 since it depends upon @code{YYLTYPE} and @code{yytoken_kind_t}, @code{%code
3279 requires} is not sufficient. Instead, move its prototype from the
3280 unqualified @code{%code} to a @code{%code provides}:
3298 tree t; /* @r{@code{tree} is defined in @file{ptypes.h}.} */
3304 #define YYLTYPE YYLTYPE
3305 typedef struct YYLTYPE
3318 void trace_token (yytoken_kind_t token, YYLTYPE loc);
3324 static void print_token (FILE *file, int token, YYSTYPE val);
3332 Bison will insert the @code{trace_token} prototype into both the parser
3333 header file and the parser implementation file after the definitions for
3334 @code{yytoken_kind_t}, @code{YYLTYPE}, and @code{YYSTYPE}.
3336 The above examples are careful to write directives in an order that reflects
3337 the layout of the generated parser implementation and header files:
3338 @code{%code top}, @code{%code requires}, @code{%code provides}, and then
3339 @code{%code}. While your grammar files may generally be easier to read if
3340 you also follow this order, Bison does not require it. Instead, Bison lets
3341 you choose an organization that makes sense to you.
3343 You may declare any of these directives multiple times in the grammar file.
3344 In that case, Bison concatenates the contained code in declaration order.
3345 This is the only way in which the position of one of these directives within
3346 the grammar file affects its functionality.
3348 The result of the previous two properties is greater flexibility in how you may
3349 organize your grammar file.
3350 For example, you may organize semantic-type-related directives by semantic
3355 %code requires @{ #include "type1.h" @}
3356 %union @{ type1 field1; @}
3357 %destructor @{ type1_free ($$); @} <field1>
3358 %printer @{ type1_print (yyo, $$); @} <field1>
3362 %code requires @{ #include "type2.h" @}
3363 %union @{ type2 field2; @}
3364 %destructor @{ type2_free ($$); @} <field2>
3365 %printer @{ type2_print (yyo, $$); @} <field2>
3370 You could even place each of the above directive groups in the rules section of
3371 the grammar file next to the set of rules that uses the associated semantic
3373 (In the rules section, you must terminate each of those directives with a
3375 And you don't have to worry that some directive (like a @code{%union}) in the
3376 definitions section is going to adversely affect their functionality in some
3377 counter-intuitive manner just because it comes first.
3378 Such an organization is not possible using @var{Prologue} sections.
3380 This section has been concerned with explaining the advantages of the four
3381 @var{Prologue} alternatives over the original Yacc @var{Prologue}.
3382 However, in most cases when using these directives, you shouldn't need to
3383 think about all the low-level ordering issues discussed here.
3384 Instead, you should simply use these directives to label each block of your
3385 code according to its purpose and let Bison handle the ordering.
3386 @code{%code} is the most generic label.
3387 Move code to @code{%code requires}, @code{%code provides}, or @code{%code top}
3390 @node Bison Declarations
3391 @subsection The Bison Declarations Section
3392 @cindex Bison declarations (introduction)
3393 @cindex declarations, Bison (introduction)
3395 The @var{Bison declarations} section contains declarations that define
3396 terminal and nonterminal symbols, specify precedence, and so on.
3397 In some simple grammars you may not need any declarations.
3398 @xref{Declarations}.
3401 @subsection The Grammar Rules Section
3402 @cindex grammar rules section
3403 @cindex rules section for grammar
3405 The @dfn{grammar rules} section contains one or more Bison grammar
3406 rules, and nothing else. @xref{Rules}.
3408 There must always be at least one grammar rule, and the first
3409 @samp{%%} (which precedes the grammar rules) may never be omitted even
3410 if it is the first thing in the file.
3413 @subsection The epilogue
3414 @cindex additional C code section
3416 @cindex C code, section for additional
3418 The @var{Epilogue} is copied verbatim to the end of the parser
3419 implementation file, just as the @var{Prologue} is copied to the
3420 beginning. This is the most convenient place to put anything that you
3421 want to have in the parser implementation file but which need not come
3422 before the definition of @code{yyparse}. For example, the definitions
3423 of @code{yylex} and @code{yyerror} often go here. Because C requires
3424 functions to be declared before being used, you often need to declare
3425 functions like @code{yylex} and @code{yyerror} in the Prologue, even
3426 if you define them in the Epilogue. @xref{Interface}.
3428 If the last section is empty, you may omit the @samp{%%} that separates it
3429 from the grammar rules.
3431 The Bison parser itself contains many macros and identifiers whose names
3432 start with @samp{yy} or @samp{YY}, so it is a good idea to avoid using
3433 any such names (except those documented in this manual) in the epilogue
3434 of the grammar file.
3437 @section Symbols, Terminal and Nonterminal
3438 @cindex nonterminal symbol
3439 @cindex terminal symbol
3443 @dfn{Symbols} in Bison grammars represent the grammatical classifications
3446 A @dfn{terminal symbol} (also known as a @dfn{token kind}) represents a
3447 class of syntactically equivalent tokens. You use the symbol in grammar
3448 rules to mean that a token in that class is allowed. The symbol is
3449 represented in the Bison parser by a numeric code, and the @code{yylex}
3450 function returns a token kind code to indicate what kind of token has been
3451 read. You don't need to know what the code value is; you can use the symbol
3454 A @dfn{nonterminal symbol} stands for a class of syntactically
3455 equivalent groupings. The symbol name is used in writing grammar rules.
3456 By convention, it should be all lower case.
3458 Symbol names can contain letters, underscores, periods, and non-initial
3459 digits and dashes. Dashes in symbol names are a GNU extension, incompatible
3460 with POSIX Yacc. Periods and dashes make symbol names less convenient to
3461 use with named references, which require brackets around such names
3462 (@pxref{Named References}). Terminal symbols that contain periods or dashes
3463 make little sense: since they are not valid symbols (in most programming
3464 languages) they are not exported as token names.
3466 There are three ways of writing terminal symbols in the grammar:
3470 A @dfn{named token kind} is written with an identifier, like an identifier
3471 in C@. By convention, it should be all upper case. Each such name must be
3472 defined with a Bison declaration such as @code{%token}. @xref{Token Decl}.
3475 @cindex character token
3476 @cindex literal token
3477 @cindex single-character literal
3478 A @dfn{character token kind} (or @dfn{literal character token}) is written
3479 in the grammar using the same syntax used in C for character constants; for
3480 example, @code{'+'} is a character token kind. A character token kind
3481 doesn't need to be declared unless you need to specify its semantic value
3482 data type (@pxref{Value Type}), associativity, or precedence
3483 (@pxref{Precedence}).
3485 By convention, a character token kind is used only to represent a token that
3486 consists of that particular character. Thus, the token kind @code{'+'} is
3487 used to represent the character @samp{+} as a token. Nothing enforces this
3488 convention, but if you depart from it, your program will confuse other
3491 All the usual escape sequences used in character literals in C can be used
3492 in Bison as well, but you must not use the null character as a character
3493 literal because its numeric code, zero, signifies end-of-input
3494 (@pxref{Calling Convention}). Also, unlike standard C, trigraphs have no
3495 special meaning in Bison character literals, nor is backslash-newline
3499 @cindex string token
3500 @cindex literal string token
3501 @cindex multicharacter literal
3502 A @dfn{literal string token} is written like a C string constant; for
3503 example, @code{"<="} is a literal string token. A literal string token
3504 doesn't need to be declared unless you need to specify its semantic
3505 value data type (@pxref{Value Type}), associativity, or precedence
3506 (@pxref{Precedence}).
3508 You can associate the literal string token with a symbolic name as an alias,
3509 using the @code{%token} declaration (@pxref{Token Decl}). If you don't do
3510 that, the lexical analyzer has to retrieve the token code for the literal
3511 string token from the @code{yytname} table (@pxref{Calling Convention}).
3513 @strong{Warning}: literal string tokens do not work in Yacc.
3515 By convention, a literal string token is used only to represent a token
3516 that consists of that particular string. Thus, you should use the token
3517 kind @code{"<="} to represent the string @samp{<=} as a token. Bison
3518 does not enforce this convention, but if you depart from it, people who
3519 read your program will be confused.
3521 All the escape sequences used in string literals in C can be used in
3522 Bison as well, except that you must not use a null character within a
3523 string literal. Also, unlike Standard C, trigraphs have no special
3524 meaning in Bison string literals, nor is backslash-newline allowed. A
3525 literal string token must contain two or more characters; for a token
3526 containing just one character, use a character token (see above).
3529 How you choose to write a terminal symbol has no effect on its
3530 grammatical meaning. That depends only on where it appears in rules and
3531 on when the parser function returns that symbol.
3533 The value returned by @code{yylex} is always one of the terminal
3534 symbols, except that a zero or negative value signifies end-of-input.
3535 Whichever way you write the token kind in the grammar rules, you write
3536 it the same way in the definition of @code{yylex}. The numeric code
3537 for a character token kind is simply the positive numeric code of the
3538 character, so @code{yylex} can use the identical value to generate the
3539 requisite code, though you may need to convert it to @code{unsigned
3540 char} to avoid sign-extension on hosts where @code{char} is signed.
3541 Each named token kind becomes a C macro in the parser implementation
3542 file, so @code{yylex} can use the name to stand for the code. (This
3543 is why periods don't make sense in terminal symbols.) @xref{Calling
3546 If @code{yylex} is defined in a separate file, you need to arrange for the
3547 token-kind definitions to be available there. Use the @option{-d} option
3548 when you run Bison, so that it will write these definitions into a separate
3549 header file @file{@var{name}.tab.h} which you can include in the other
3550 source files that need it. @xref{Invocation}.
3552 If you want to write a grammar that is portable to any Standard C
3553 host, you must use only nonnull character tokens taken from the basic
3554 execution character set of Standard C@. This set consists of the ten
3555 digits, the 52 lower- and upper-case English letters, and the
3556 characters in the following C-language string:
3559 "\a\b\t\n\v\f\r !\"#%&'()*+,-./:;<=>?[\\]^_@{|@}~"
3562 The @code{yylex} function and Bison must use a consistent character set
3563 and encoding for character tokens. For example, if you run Bison in an
3564 ASCII environment, but then compile and run the resulting
3565 program in an environment that uses an incompatible character set like
3566 EBCDIC, the resulting program may not work because the tables
3567 generated by Bison will assume ASCII numeric values for
3568 character tokens. It is standard practice for software distributions to
3569 contain C source files that were generated by Bison in an
3570 ASCII environment, so installers on platforms that are
3571 incompatible with ASCII must rebuild those files before
3574 The symbol @code{error} is a terminal symbol reserved for error recovery
3575 (@pxref{Error Recovery}); you shouldn't use it for any other purpose.
3576 In particular, @code{yylex} should never return this value. The default
3577 value of the error token is 256, unless you explicitly assigned 256 to
3578 one of your tokens with a @code{%token} declaration.
3581 @section Grammar Rules
3583 A Bison grammar is a list of rules.
3586 * Rules Syntax:: Syntax of the rules.
3587 * Empty Rules:: Symbols that can match the empty string.
3588 * Recursion:: Writing recursive rules.
3592 @subsection Syntax of Grammar Rules
3594 @cindex grammar rule syntax
3595 @cindex syntax of grammar rules
3597 A Bison grammar rule has the following general form:
3600 @var{result}: @var{components}@dots{};
3604 where @var{result} is the nonterminal symbol that this rule describes,
3605 and @var{components} are various terminal and nonterminal symbols that
3606 are put together by this rule (@pxref{Symbols}).
3615 says that two groupings of type @code{exp}, with a @samp{+} token in between,
3616 can be combined into a larger grouping of type @code{exp}.
3618 White space in rules is significant only to separate symbols. You can add
3619 extra white space as you wish.
3621 Scattered among the components can be @var{actions} that determine
3622 the semantics of the rule. An action looks like this:
3625 @{@var{C statements}@}
3630 This is an example of @dfn{braced code}, that is, C code surrounded by
3631 braces, much like a compound statement in C@. Braced code can contain
3632 any sequence of C tokens, so long as its braces are balanced. Bison
3633 does not check the braced code for correctness directly; it merely
3634 copies the code to the parser implementation file, where the C
3635 compiler can check it.
3637 Within braced code, the balanced-brace count is not affected by braces
3638 within comments, string literals, or character constants, but it is
3639 affected by the C digraphs @samp{<%} and @samp{%>} that represent
3640 braces. At the top level braced code must be terminated by @samp{@}}
3641 and not by a digraph. Bison does not look for trigraphs, so if braced
3642 code uses trigraphs you should ensure that they do not affect the
3643 nesting of braces or the boundaries of comments, string literals, or
3644 character constants.
3646 Usually there is only one action and it follows the components.
3650 Multiple rules for the same @var{result} can be written separately or can
3651 be joined with the vertical-bar character @samp{|} as follows:
3656 @var{rule1-components}@dots{}
3657 | @var{rule2-components}@dots{}
3664 They are still considered distinct rules even when joined in this way.
3667 @subsection Empty Rules
3672 A rule is said to be @dfn{empty} if its right-hand side (@var{components})
3673 is empty. It means that @var{result} in the previous example can match the
3674 empty string. As another example, here is how to define an optional
3678 semicolon.opt: | ";";
3682 It is easy not to see an empty rule, especially when @code{|} is used. The
3683 @code{%empty} directive allows to make explicit that a rule is empty on
3695 Flagging a non-empty rule with @code{%empty} is an error. If run with
3696 @option{-Wempty-rule}, @command{bison} will report empty rules without
3697 @code{%empty}. Using @code{%empty} enables this warning, unless
3698 @option{-Wno-empty-rule} was specified.
3700 The @code{%empty} directive is a Bison extension, it does not work with
3701 Yacc. To remain compatible with POSIX Yacc, it is customary to write a
3702 comment @samp{/* empty */} in each rule with no components:
3715 @subsection Recursive Rules
3716 @cindex recursive rule
3717 @cindex rule, recursive
3719 A rule is called @dfn{recursive} when its @var{result} nonterminal
3720 appears also on its right hand side. Nearly all Bison grammars need to
3721 use recursion, because that is the only way to define a sequence of any
3722 number of a particular thing. Consider this recursive definition of a
3723 comma-separated sequence of one or more expressions:
3734 @cindex left recursion
3735 @cindex right recursion
3737 Since the recursive use of @code{expseq1} is the leftmost symbol in the
3738 right hand side, we call this @dfn{left recursion}. By contrast, here
3739 the same construct is defined using @dfn{right recursion}:
3751 Any kind of sequence can be defined using either left recursion or right
3752 recursion, but you should always use left recursion, because it can
3753 parse a sequence of any number of elements with bounded stack space.
3754 Right recursion uses up space on the Bison stack in proportion to the
3755 number of elements in the sequence, because all the elements must be
3756 shifted onto the stack before the rule can be applied even once.
3757 @xref{Algorithm}, for further explanation
3760 @cindex mutual recursion
3761 @dfn{Indirect} or @dfn{mutual} recursion occurs when the result of the
3762 rule does not appear directly on its right hand side, but does appear
3763 in rules for other nonterminals which do appear on its right hand
3772 | primary '+' primary
3785 defines two mutually-recursive nonterminals, since each refers to the
3789 @section Defining Language Semantics
3790 @cindex defining language semantics
3791 @cindex language semantics, defining
3793 The grammar rules for a language determine only the syntax. The semantics
3794 are determined by the semantic values associated with various tokens and
3795 groupings, and by the actions taken when various groupings are recognized.
3797 For example, the calculator calculates properly because the value
3798 associated with each expression is the proper number; it adds properly
3799 because the action for the grouping @w{@samp{@var{x} + @var{y}}} is to add
3800 the numbers associated with @var{x} and @var{y}.
3803 * Value Type:: Specifying one data type for all semantic values.
3804 * Multiple Types:: Specifying several alternative data types.
3805 * Type Generation:: Generating the semantic value type.
3806 * Union Decl:: Declaring the set of all semantic value types.
3807 * Structured Value Type:: Providing a structured semantic value type.
3808 * Actions:: An action is the semantic definition of a grammar rule.
3809 * Action Types:: Specifying data types for actions to operate on.
3810 * Midrule Actions:: Most actions go at the end of a rule.
3811 This says when, why and how to use the exceptional
3812 action in the middle of a rule.
3816 @subsection Data Types of Semantic Values
3817 @cindex semantic value type
3818 @cindex value type, semantic
3819 @cindex data types of semantic values
3820 @cindex default data type
3822 In a simple program it may be sufficient to use the same data type for
3823 the semantic values of all language constructs. This was true in the
3824 RPN and infix calculator examples (@pxref{RPN Calc}).
3826 Bison normally uses the type @code{int} for semantic values if your program
3827 uses the same data type for all language constructs. To specify some other
3828 type, define the @code{%define} variable @code{api.value.type} like this:
3831 %define api.value.type @{double@}
3838 %define api.value.type @{struct semantic_type@}
3841 The value of @code{api.value.type} should be a type name that does not
3842 contain parentheses or square brackets.
3844 Alternatively, instead of relying of Bison's @code{%define} support, you may
3845 rely on the C/C++ preprocessor and define @code{YYSTYPE} as a macro, like
3849 #define YYSTYPE double
3853 This macro definition must go in the prologue of the grammar file
3854 (@pxref{Grammar Outline}). If compatibility with POSIX Yacc matters to you,
3855 use this. Note however that Bison cannot know @code{YYSTYPE}'s value, not
3856 even whether it is defined, so there are services it cannot provide.
3857 Besides this works only for languages that have a preprocessor.
3859 @node Multiple Types
3860 @subsection More Than One Value Type
3862 In most programs, you will need different data types for different kinds
3863 of tokens and groupings. For example, a numeric constant may need type
3864 @code{int} or @code{long}, while a string constant needs type
3865 @code{char *}, and an identifier might need a pointer to an entry in the
3868 To use more than one data type for semantic values in one parser, Bison
3869 requires you to do two things:
3873 Specify the entire collection of possible data types. There are several
3877 let Bison compute the union type from the tags you assign to symbols;
3880 use the @code{%union} Bison declaration (@pxref{Union Decl});
3883 define the @code{%define} variable @code{api.value.type} to be a union type
3884 whose members are the type tags (@pxref{Structured Value Type});
3887 use a @code{typedef} or a @code{#define} to define @code{YYSTYPE} to be a
3888 union type whose member names are the type tags.
3892 Choose one of those types for each symbol (terminal or nonterminal) for
3893 which semantic values are used. This is done for tokens with the
3894 @code{%token} Bison declaration (@pxref{Token Decl}) and
3895 for groupings with the @code{%nterm}/@code{%type} Bison declarations
3896 (@pxref{Type Decl}).
3899 @node Type Generation
3900 @subsection Generating the Semantic Value Type
3901 @cindex declaring value types
3902 @cindex value types, declaring
3903 @findex %define api.value.type union
3905 The special value @code{union} of the @code{%define} variable
3906 @code{api.value.type} instructs Bison that the type tags (used with the
3907 @code{%token}, @code{%nterm} and @code{%type} directives) are genuine types,
3908 not names of members of @code{YYSTYPE}.
3913 %define api.value.type union
3914 %token <int> INT "integer"
3917 %token <char const *> ID "identifier"
3921 generates an appropriate value of @code{YYSTYPE} to support each symbol
3922 type. The name of the member of @code{YYSTYPE} for tokens than have a
3923 declared identifier @var{id} (such as @code{INT} and @code{ID} above, but
3924 not @code{'n'}) is @code{@var{id}}. The other symbols have unspecified
3925 names on which you should not depend; instead, relying on C casts to access
3926 the semantic value with the appropriate type:
3929 /* For an "integer". */
3933 /* For an 'n', also declared as int. */
3934 *((int*)&yylval) = 42;
3937 /* For an "identifier". */
3942 If the @code{%define} variable @code{api.token.prefix} is defined
3943 (@pxref{%define Summary}), then it is also used to prefix
3944 the union member names. For instance, with @samp{%define api.token.prefix
3948 /* For an "integer". */
3949 yylval.TOK_INT = 42;
3953 This Bison extension cannot work if @code{%yacc} (or
3954 @option{-y}/@option{--yacc}) is enabled, as POSIX mandates that Yacc
3955 generate tokens as macros (e.g., @samp{#define INT 258}, or @samp{#define
3958 A similar feature is provided for C++ that in addition overcomes C++
3959 limitations (that forbid non-trivial objects to be part of a @code{union}):
3960 @samp{%define api.value.type variant}, see @ref{C++ Variants}.
3963 @subsection The Union Declaration
3964 @cindex declaring value types
3965 @cindex value types, declaring
3968 The @code{%union} declaration specifies the entire collection of possible
3969 data types for semantic values. The keyword @code{%union} is followed by
3970 braced code containing the same thing that goes inside a @code{union} in C@.
3984 This says that the two alternative types are @code{double} and @code{symrec
3985 *}. They are given names @code{val} and @code{tptr}; these names are used
3986 in the @code{%token}, @code{%nterm} and @code{%type} declarations to pick
3987 one of the types for a terminal or nonterminal symbol (@pxref{Type Decl}).
3989 As an extension to POSIX, a tag is allowed after the @code{%union}. For
4002 specifies the union tag @code{value}, so the corresponding C type is
4003 @code{union value}. If you do not specify a tag, it defaults to
4004 @code{YYSTYPE} (@pxref{%define Summary}).
4006 As another extension to POSIX, you may specify multiple @code{%union}
4007 declarations; their contents are concatenated. However, only the first
4008 @code{%union} declaration can specify a tag.
4010 Note that, unlike making a @code{union} declaration in C, you need not write
4011 a semicolon after the closing brace.
4013 @node Structured Value Type
4014 @subsection Providing a Structured Semantic Value Type
4015 @cindex declaring value types
4016 @cindex value types, declaring
4019 Instead of @code{%union}, you can define and use your own union type
4020 @code{YYSTYPE} if your grammar contains at least one @samp{<@var{type}>}
4021 tag. For example, you can put the following into a header file
4034 and then your grammar can use the following instead of @code{%union}:
4041 %define api.value.type @{union YYSTYPE@}
4047 Actually, you may also provide a @code{struct} rather that a @code{union},
4048 which may be handy if you want to track information for every symbol (such
4049 as preceding comments).
4051 The type you provide may even be structured and include pointers, in which
4052 case the type tags you provide may be composite, with @samp{.} and @samp{->}
4061 @vindex $[@var{name}]
4063 An action accompanies a syntactic rule and contains C code to be executed
4064 each time an instance of that rule is recognized. The task of most actions
4065 is to compute a semantic value for the grouping built by the rule from the
4066 semantic values associated with tokens or smaller groupings.
4068 An action consists of braced code containing C statements, and can be
4069 placed at any position in the rule;
4070 it is executed at that position. Most rules have just one action at the
4071 end of the rule, following all the components. Actions in the middle of
4072 a rule are tricky and used only for special purposes (@pxref{Midrule
4075 The C code in an action can refer to the semantic values of the
4076 components matched by the rule with the construct @code{$@var{n}},
4077 which stands for the value of the @var{n}th component. The semantic
4078 value for the grouping being constructed is @code{$$}. In addition,
4079 the semantic values of symbols can be accessed with the named
4080 references construct @code{$@var{name}} or @code{$[@var{name}]}.
4081 Bison translates both of these constructs into expressions of the
4082 appropriate type when it copies the actions into the parser
4083 implementation file. @code{$$} (or @code{$@var{name}}, when it stands
4084 for the current grouping) is translated to a modifiable lvalue, so it
4087 Here is a typical example:
4093 | exp '+' exp @{ $$ = $1 + $3; @}
4097 Or, in terms of named references:
4103 | exp[left] '+' exp[right] @{ $result = $left + $right; @}
4108 This rule constructs an @code{exp} from two smaller @code{exp} groupings
4109 connected by a plus-sign token. In the action, @code{$1} and @code{$3}
4110 (@code{$left} and @code{$right})
4111 refer to the semantic values of the two component @code{exp} groupings,
4112 which are the first and third symbols on the right hand side of the rule.
4113 The sum is stored into @code{$$} (@code{$result}) so that it becomes the
4115 the addition-expression just recognized by the rule. If there were a
4116 useful semantic value associated with the @samp{+} token, it could be
4117 referred to as @code{$2}.
4119 @xref{Named References}, for more information about using the named
4120 references construct.
4122 Note that the vertical-bar character @samp{|} is really a rule
4123 separator, and actions are attached to a single rule. This is a
4124 difference with tools like Flex, for which @samp{|} stands for either
4125 ``or'', or ``the same action as that of the next rule''. In the
4126 following example, the action is triggered only when @samp{b} is found:
4129 a-or-b: 'a'|'b' @{ a_or_b_found = 1; @};
4132 @cindex default action
4133 If you don't specify an action for a rule, Bison supplies a default:
4134 @w{@code{$$ = $1}.} Thus, the value of the first symbol in the rule
4135 becomes the value of the whole rule. Of course, the default action is
4136 valid only if the two data types match. There is no meaningful default
4137 action for an empty rule; every empty rule must have an explicit action
4138 unless the rule's value does not matter.
4140 @code{$@var{n}} with @var{n} zero or negative is allowed for reference
4141 to tokens and groupings on the stack @emph{before} those that match the
4142 current rule. This is a very risky practice, and to use it reliably
4143 you must be certain of the context in which the rule is applied. Here
4144 is a case in which you can use this reliably:
4149 expr bar '+' expr @{ @dots{} @}
4150 | expr bar '-' expr @{ @dots{} @}
4156 %empty @{ previous_expr = $0; @}
4161 As long as @code{bar} is used only in the fashion shown here, @code{$0}
4162 always refers to the @code{expr} which precedes @code{bar} in the
4163 definition of @code{foo}.
4166 It is also possible to access the semantic value of the lookahead token, if
4167 any, from a semantic action.
4168 This semantic value is stored in @code{yylval}.
4169 @xref{Action Features}.
4172 @subsection Data Types of Values in Actions
4173 @cindex action data types
4174 @cindex data types in actions
4176 If you have chosen a single data type for semantic values, the @code{$$}
4177 and @code{$@var{n}} constructs always have that data type.
4179 If you have used @code{%union} to specify a variety of data types, then you
4180 must declare a choice among these types for each terminal or nonterminal
4181 symbol that can have a semantic value. Then each time you use @code{$$} or
4182 @code{$@var{n}}, its data type is determined by which symbol it refers to
4183 in the rule. In this example,
4189 | exp '+' exp @{ $$ = $1 + $3; @}
4194 @code{$1} and @code{$3} refer to instances of @code{exp}, so they all
4195 have the data type declared for the nonterminal symbol @code{exp}. If
4196 @code{$2} were used, it would have the data type declared for the
4197 terminal symbol @code{'+'}, whatever that might be.
4199 Alternatively, you can specify the data type when you refer to the value,
4200 by inserting @samp{<@var{type}>} after the @samp{$} at the beginning of the
4201 reference. For example, if you have defined types as shown here:
4213 then you can write @code{$<itype>1} to refer to the first subunit of the
4214 rule as an integer, or @code{$<dtype>1} to refer to it as a double.
4216 @node Midrule Actions
4217 @subsection Actions in Midrule
4218 @cindex actions in midrule
4219 @cindex midrule actions
4221 Occasionally it is useful to put an action in the middle of a rule.
4222 These actions are written just like usual end-of-rule actions, but they
4223 are executed before the parser even recognizes the following components.
4226 * Using Midrule Actions:: Putting an action in the middle of a rule.
4227 * Typed Midrule Actions:: Specifying the semantic type of their values.
4228 * Midrule Action Translation:: How midrule actions are actually processed.
4229 * Midrule Conflicts:: Midrule actions can cause conflicts.
4232 @node Using Midrule Actions
4233 @subsubsection Using Midrule Actions
4235 A midrule action may refer to the components preceding it using
4236 @code{$@var{n}}, but it may not refer to subsequent components because
4237 it is run before they are parsed.
4239 The midrule action itself counts as one of the components of the rule.
4240 This makes a difference when there is another action later in the same rule
4241 (and usually there is another at the end): you have to count the actions
4242 along with the symbols when working out which number @var{n} to use in
4245 The midrule action can also have a semantic value. The action can set
4246 its value with an assignment to @code{$$}, and actions later in the rule
4247 can refer to the value using @code{$@var{n}}. Since there is no symbol
4248 to name the action, there is no way to declare a data type for the value
4249 in advance, so you must use the @samp{$<@dots{}>@var{n}} construct to
4250 specify a data type each time you refer to this value.
4252 There is no way to set the value of the entire rule with a midrule
4253 action, because assignments to @code{$$} do not have that effect. The
4254 only way to set the value for the entire rule is with an ordinary action
4255 at the end of the rule.
4257 Here is an example from a hypothetical compiler, handling a @code{let}
4258 statement that looks like @samp{let (@var{variable}) @var{statement}} and
4259 serves to create a variable named @var{variable} temporarily for the
4260 duration of @var{statement}. To parse this construct, we must put
4261 @var{variable} into the symbol table while @var{statement} is parsed, then
4262 remove it afterward. Here is how it is done:
4269 $<context>$ = push_context ();
4270 declare_variable ($3);
4275 pop_context ($<context>5);
4281 As soon as @samp{let (@var{variable})} has been recognized, the first
4282 action is run. It saves a copy of the current semantic context (the
4283 list of accessible variables) as its semantic value, using alternative
4284 @code{context} in the data-type union. Then it calls
4285 @code{declare_variable} to add the new variable to that list. Once the
4286 first action is finished, the embedded statement @code{stmt} can be
4289 Note that the midrule action is component number 5, so the @samp{stmt} is
4290 component number 6. Named references can be used to improve the readability
4291 and maintainability (@pxref{Named References}):
4298 $<context>let = push_context ();
4299 declare_variable ($3);
4304 pop_context ($<context>let);
4309 After the embedded statement is parsed, its semantic value becomes the
4310 value of the entire @code{let}-statement. Then the semantic value from the
4311 earlier action is used to restore the prior list of variables. This
4312 removes the temporary @code{let}-variable from the list so that it won't
4313 appear to exist while the rest of the program is parsed.
4315 Because the types of the semantic values of midrule actions are unknown to
4316 Bison, type-based features (e.g., @samp{%printer}, @samp{%destructor}) do
4317 not work, which could result in memory leaks. They also forbid the use of
4318 the @code{variant} implementation of the @code{api.value.type} in C++
4319 (@pxref{C++ Variants}).
4321 @xref{Typed Midrule Actions}, for one way to address this issue, and
4322 @ref{Midrule Action Translation}, for another: turning mid-action actions
4323 into regular actions.
4326 @node Typed Midrule Actions
4327 @subsubsection Typed Midrule Actions
4330 @cindex discarded symbols, midrule actions
4331 @cindex error recovery, midrule actions
4332 In the above example, if the parser initiates error recovery (@pxref{Error
4333 Recovery}) while parsing the tokens in the embedded statement @code{stmt},
4334 it might discard the previous semantic context @code{$<context>5} without
4335 restoring it. Thus, @code{$<context>5} needs a destructor
4336 (@pxref{Destructor Decl}), and Bison needs the
4337 type of the semantic value (@code{context}) to select the right destructor.
4339 As an extension to Yacc's midrule actions, Bison offers a means to type
4340 their semantic value: specify its type tag (@samp{<...>} before the midrule
4343 Consider the previous example, with an untyped midrule action:
4350 $<context>$ = push_context (); // ***
4351 declare_variable ($3);
4356 pop_context ($<context>5); // ***
4362 If instead you write:
4369 $$ = push_context (); // ***
4370 declare_variable ($3);
4375 pop_context ($5); // ***
4381 then @code{%printer} and @code{%destructor} work properly (no more leaks!),
4382 C++ @code{variant}s can be used, and redundancy is reduced (@code{<context>}
4386 @node Midrule Action Translation
4387 @subsubsection Midrule Action Translation
4391 Midrule actions are actually transformed into regular rules and actions.
4392 The various reports generated by Bison (textual, graphical, etc., see
4393 @ref{Understanding}) reveal this translation,
4394 best explained by means of an example. The following rule:
4397 exp: @{ a(); @} "b" @{ c(); @} @{ d(); @} "e" @{ f(); @};
4404 $@@1: %empty @{ a(); @};
4405 $@@2: %empty @{ c(); @};
4406 $@@3: %empty @{ d(); @};
4407 exp: $@@1 "b" $@@2 $@@3 "e" @{ f(); @};
4411 with new nonterminal symbols @code{$@@@var{n}}, where @var{n} is a number.
4413 A midrule action is expected to generate a value if it uses @code{$$}, or
4414 the (final) action uses @code{$@var{n}} where @var{n} denote the midrule
4415 action. In that case its nonterminal is rather named @code{@@@var{n}}:
4418 exp: @{ a(); @} "b" @{ $$ = c(); @} @{ d(); @} "e" @{ f = $1; @};
4425 @@1: %empty @{ a(); @};
4426 @@2: %empty @{ $$ = c(); @};
4427 $@@3: %empty @{ d(); @};
4428 exp: @@1 "b" @@2 $@@3 "e" @{ f = $1; @}
4431 There are probably two errors in the above example: the first midrule action
4432 does not generate a value (it does not use @code{$$} although the final
4433 action uses it), and the value of the second one is not used (the final
4434 action does not use @code{$3}). Bison reports these errors when the
4435 @code{midrule-value} warnings are enabled (@pxref{Invocation}):
4438 $ @kbd{bison -Wmidrule-value mid.y}
4440 mid.y:2.6-13: @dwarning{warning}: unset value: $$
4441 2 | exp: @dwarning{@{ a(); @}} "b" @{ $$ = c(); @} @{ d(); @} "e" @{ f = $1; @};
4442 | @dwarning{^~~~~~~~}
4445 mid.y:2.19-31: @dwarning{warning}: unused value: $3
4446 2 | exp: @{ a(); @} "b" @dwarning{@{ $$ = c(); @}} @{ d(); @} "e" @{ f = $1; @};
4447 | @dwarning{^~~~~~~~~~~~~}
4453 It is sometimes useful to turn midrule actions into regular actions, e.g.,
4454 to factor them, or to escape from their limitations. For instance, as an
4455 alternative to @emph{typed} midrule action, you may bury the midrule action
4456 inside a nonterminal symbol and to declare a printer and a destructor for
4461 %nterm <context> let
4462 %destructor @{ pop_context ($$); @} let
4463 %printer @{ print_context (yyo, $$); @} let
4481 $let = push_context ();
4482 declare_variable ($var);
4491 @node Midrule Conflicts
4492 @subsubsection Conflicts due to Midrule Actions
4493 Taking action before a rule is completely recognized often leads to
4494 conflicts since the parser must commit to a parse in order to execute the
4495 action. For example, the following two rules, without midrule actions,
4496 can coexist in a working parser because the parser can shift the open-brace
4497 token and look at what follows before deciding whether there is a
4503 '@{' declarations statements '@}'
4504 | '@{' statements '@}'
4510 But when we add a midrule action as follows, the rules become nonfunctional:
4515 @{ prepare_for_local_variables (); @}
4516 '@{' declarations statements '@}'
4519 | '@{' statements '@}'
4525 Now the parser is forced to decide whether to run the midrule action
4526 when it has read no farther than the open-brace. In other words, it
4527 must commit to using one rule or the other, without sufficient
4528 information to do it correctly. (The open-brace token is what is called
4529 the @dfn{lookahead} token at this time, since the parser is still
4530 deciding what to do about it. @xref{Lookahead}.)
4532 You might think that you could correct the problem by putting identical
4533 actions into the two rules, like this:
4538 @{ prepare_for_local_variables (); @}
4539 '@{' declarations statements '@}'
4540 | @{ prepare_for_local_variables (); @}
4541 '@{' statements '@}'
4547 But this does not help, because Bison does not realize that the two actions
4548 are identical. (Bison never tries to understand the C code in an action.)
4550 If the grammar is such that a declaration can be distinguished from a
4551 statement by the first token (which is true in C), then one solution which
4552 does work is to put the action after the open-brace, like this:
4557 '@{' @{ prepare_for_local_variables (); @}
4558 declarations statements '@}'
4559 | '@{' statements '@}'
4565 Now the first token of the following declaration or statement,
4566 which would in any case tell Bison which rule to use, can still do so.
4568 Another solution is to bury the action inside a nonterminal symbol which
4569 serves as a subroutine:
4574 %empty @{ prepare_for_local_variables (); @}
4580 subroutine '@{' declarations statements '@}'
4581 | subroutine '@{' statements '@}'
4587 Now Bison can execute the action in the rule for @code{subroutine} without
4588 deciding which rule for @code{compound} it will eventually use.
4591 @node Tracking Locations
4592 @section Tracking Locations
4594 @cindex textual location
4595 @cindex location, textual
4597 Though grammar rules and semantic actions are enough to write a fully
4598 functional parser, it can be useful to process some additional information,
4599 especially symbol locations.
4601 The way locations are handled is defined by providing a data type, and
4602 actions to take when rules are matched.
4605 * Location Type:: Specifying a data type for locations.
4606 * Actions and Locations:: Using locations in actions.
4607 * Location Default Action:: Defining a general way to compute locations.
4611 @subsection Data Type of Locations
4612 @cindex data type of locations
4613 @cindex default location type
4615 Defining a data type for locations is much simpler than for semantic values,
4616 since all tokens and groupings always use the same type.
4618 You can specify the type of locations by defining a macro called
4619 @code{YYLTYPE}, just as you can specify the semantic value type by defining
4620 a @code{YYSTYPE} macro (@pxref{Value Type}). When @code{YYLTYPE} is not
4621 defined, Bison uses a default structure type with four members:
4624 typedef struct YYLTYPE
4633 While default locations represent a range in the source file(s), this is not
4634 a requirement. It could be a single point or just a line number, or even
4635 more complex structures.
4637 When @code{YYLTYPE} is not defined, at the beginning of the parsing, Bison
4638 initializes all these fields to 1 for @code{yylloc}. To initialize
4639 @code{yylloc} with a custom location type (or to chose a different
4640 initialization), use the @code{%initial-action} directive. @xref{Initial
4644 @node Actions and Locations
4645 @subsection Actions and Locations
4646 @cindex location actions
4647 @cindex actions, location
4650 @vindex @@@var{name}
4651 @vindex @@[@var{name}]
4653 Actions are not only useful for defining language semantics, but also for
4654 describing the behavior of the output parser with locations.
4656 The most obvious way for building locations of syntactic groupings is very
4657 similar to the way semantic values are computed. In a given rule, several
4658 constructs can be used to access the locations of the elements being matched.
4659 The location of the @var{n}th component of the right hand side is
4660 @code{@@@var{n}}, while the location of the left hand side grouping is
4663 In addition, the named references construct @code{@@@var{name}} and
4664 @code{@@[@var{name}]} may also be used to address the symbol locations.
4665 @xref{Named References}, for more information about using the named
4666 references construct.
4668 Here is a basic example using the default data type for locations:
4676 @@$.first_column = @@1.first_column;
4677 @@$.first_line = @@1.first_line;
4678 @@$.last_column = @@3.last_column;
4679 @@$.last_line = @@3.last_line;
4685 fprintf (stderr, "%d.%d-%d.%d: division by zero",
4686 @@3.first_line, @@3.first_column,
4687 @@3.last_line, @@3.last_column);
4693 As for semantic values, there is a default action for locations that is
4694 run each time a rule is matched. It sets the beginning of @code{@@$} to the
4695 beginning of the first symbol, and the end of @code{@@$} to the end of the
4698 With this default action, the location tracking can be fully automatic. The
4699 example above simply rewrites this way:
4712 fprintf (stderr, "%d.%d-%d.%d: division by zero",
4713 @@3.first_line, @@3.first_column,
4714 @@3.last_line, @@3.last_column);
4721 It is also possible to access the location of the lookahead token, if any,
4722 from a semantic action.
4723 This location is stored in @code{yylloc}.
4724 @xref{Action Features}.
4726 @node Location Default Action
4727 @subsection Default Action for Locations
4728 @vindex YYLLOC_DEFAULT
4729 @cindex GLR parsers and @code{YYLLOC_DEFAULT}
4731 Actually, actions are not the best place to compute locations. Since
4732 locations are much more general than semantic values, there is room in
4733 the output parser to redefine the default action to take for each
4734 rule. The @code{YYLLOC_DEFAULT} macro is invoked each time a rule is
4735 matched, before the associated action is run. It is also invoked
4736 while processing a syntax error, to compute the error's location.
4737 Before reporting an unresolvable syntactic ambiguity, a GLR
4738 parser invokes @code{YYLLOC_DEFAULT} recursively to compute the location
4741 Most of the time, this macro is general enough to suppress location
4742 dedicated code from semantic actions.
4744 The @code{YYLLOC_DEFAULT} macro takes three parameters. The first one is
4745 the location of the grouping (the result of the computation). When a
4746 rule is matched, the second parameter identifies locations of
4747 all right hand side elements of the rule being matched, and the third
4748 parameter is the size of the rule's right hand side.
4749 When a GLR parser reports an ambiguity, which of multiple candidate
4750 right hand sides it passes to @code{YYLLOC_DEFAULT} is undefined.
4751 When processing a syntax error, the second parameter identifies locations
4752 of the symbols that were discarded during error processing, and the third
4753 parameter is the number of discarded symbols.
4755 By default, @code{YYLLOC_DEFAULT} is defined this way:
4759 # define YYLLOC_DEFAULT(Cur, Rhs, N) \
4763 (Cur).first_line = YYRHSLOC(Rhs, 1).first_line; \
4764 (Cur).first_column = YYRHSLOC(Rhs, 1).first_column; \
4765 (Cur).last_line = YYRHSLOC(Rhs, N).last_line; \
4766 (Cur).last_column = YYRHSLOC(Rhs, N).last_column; \
4770 (Cur).first_line = (Cur).last_line = \
4771 YYRHSLOC(Rhs, 0).last_line; \
4772 (Cur).first_column = (Cur).last_column = \
4773 YYRHSLOC(Rhs, 0).last_column; \
4780 where @code{YYRHSLOC (rhs, k)} is the location of the @var{k}th symbol
4781 in @var{rhs} when @var{k} is positive, and the location of the symbol
4782 just before the reduction when @var{k} and @var{n} are both zero.
4784 When defining @code{YYLLOC_DEFAULT}, you should consider that:
4788 All arguments are free of side-effects. However, only the first one (the
4789 result) should be modified by @code{YYLLOC_DEFAULT}.
4792 For consistency with semantic actions, valid indexes within the
4793 right hand side range from 1 to @var{n}. When @var{n} is zero, only 0 is a
4794 valid index, and it refers to the symbol just before the reduction.
4795 During error processing @var{n} is always positive.
4798 Your macro should parenthesize its arguments, if need be, since the
4799 actual arguments may not be surrounded by parentheses. Also, your
4800 macro should expand to something that can be used as a single
4801 statement when it is followed by a semicolon.
4804 @node Named References
4805 @section Named References
4806 @cindex named references
4808 As described in the preceding sections, the traditional way to refer to any
4809 semantic value or location is a @dfn{positional reference}, which takes the
4810 form @code{$@var{n}}, @code{$$}, @code{@@@var{n}}, and @code{@@$}. However,
4811 such a reference is not very descriptive. Moreover, if you later decide to
4812 insert or remove symbols in the right-hand side of a grammar rule, the need
4813 to renumber such references can be tedious and error-prone.
4815 To avoid these issues, you can also refer to a semantic value or location
4816 using a @dfn{named reference}. First of all, original symbol names may be
4817 used as named references. For example:
4821 invocation: op '(' args ')'
4822 @{ $invocation = new_invocation ($op, $args, @@invocation); @}
4827 Positional and named references can be mixed arbitrarily. For example:
4831 invocation: op '(' args ')'
4832 @{ $$ = new_invocation ($op, $args, @@$); @}
4837 However, sometimes regular symbol names are not sufficient due to
4843 @{ $exp = $exp / $exp; @} // $exp is ambiguous.
4846 @{ $$ = $1 / $exp; @} // One usage is ambiguous.
4849 @{ $$ = $1 / $3; @} // No error.
4854 When ambiguity occurs, explicitly declared names may be used for values and
4855 locations. Explicit names are declared as a bracketed name after a symbol
4856 appearance in rule definitions. For example:
4859 exp[result]: exp[left] '/' exp[right]
4860 @{ $result = $left / $right; @}
4865 In order to access a semantic value generated by a midrule action, an
4866 explicit name may also be declared by putting a bracketed name after the
4867 closing brace of the midrule action code:
4870 exp[res]: exp[x] '+' @{$left = $x;@}[left] exp[right]
4871 @{ $res = $left + $right; @}
4877 In references, in order to specify names containing dots and dashes, an explicit
4878 bracketed syntax @code{$[name]} and @code{@@[name]} must be used:
4881 if-stmt: "if" '(' expr ')' "then" then.stmt ';'
4882 @{ $[if-stmt] = new_if_stmt ($expr, $[then.stmt]); @}
4886 It often happens that named references are followed by a dot, dash or other
4887 C punctuation marks and operators. By default, Bison will read
4888 @samp{$name.suffix} as a reference to symbol value @code{$name} followed by
4889 @samp{.suffix}, i.e., an access to the @code{suffix} field of the semantic
4890 value. In order to force Bison to recognize @samp{name.suffix} in its
4891 entirety as the name of a semantic value, the bracketed syntax
4892 @samp{$[name.suffix]} must be used.
4895 @section Bison Declarations
4896 @cindex declarations, Bison
4897 @cindex Bison declarations
4899 The @dfn{Bison declarations} section of a Bison grammar defines the symbols
4900 used in formulating the grammar and the data types of semantic values.
4903 All token kind names (but not single-character literal tokens such as
4904 @code{'+'} and @code{'*'}) must be declared. Nonterminal symbols must be
4905 declared if you need to specify which data type to use for the semantic
4906 value (@pxref{Multiple Types}).
4908 The first rule in the grammar file also specifies the start symbol, by
4909 default. If you want some other symbol to be the start symbol, you
4910 must declare it explicitly (@pxref{Language and Grammar}).
4913 * Require Decl:: Requiring a Bison version.
4914 * Token Decl:: Declaring terminal symbols.
4915 * Precedence Decl:: Declaring terminals with precedence and associativity.
4916 * Type Decl:: Declaring the choice of type for a nonterminal symbol.
4917 * Symbol Decls:: Summary of the Syntax of Symbol Declarations.
4918 * Initial Action Decl:: Code run before parsing starts.
4919 * Destructor Decl:: Declaring how symbols are freed.
4920 * Printer Decl:: Declaring how symbol values are displayed.
4921 * Expect Decl:: Suppressing warnings about parsing conflicts.
4922 * Start Decl:: Specifying the start symbol.
4923 * Pure Decl:: Requesting a reentrant parser.
4924 * Push Decl:: Requesting a push parser.
4925 * Decl Summary:: Table of all Bison declarations.
4926 * %define Summary:: Defining variables to adjust Bison's behavior.
4927 * %code Summary:: Inserting code into the parser source.
4931 @subsection Require a Version of Bison
4932 @cindex version requirement
4933 @cindex requiring a version of Bison
4936 You may require the minimum version of Bison to process the grammar. If
4937 the requirement is not met, @command{bison} exits with an error (exit
4941 %require "@var{version}"
4944 Some deprecated behaviors are disabled for some required @var{version}:
4946 @item @code{"3.2"} (or better)
4947 The C++ deprecated files @file{position.hh} and @file{stack.hh} are no
4950 @item @code{"3.4"} (or better)
4952 @uref{https://marc.info/?l=graphviz-devel&m=129418103126092, recommendations
4953 of the Graphviz team}, use the @code{.gv} extension instead of @code{.dot}
4954 for the name of the generated DOT file. @xref{Graphviz}.
4959 @subsection Token Kind Names
4960 @cindex declaring token kind names
4961 @cindex token kind names, declaring
4962 @cindex declaring literal string tokens
4965 The basic way to declare a token kind name (terminal symbol) is as follows:
4971 Bison will convert this into a definition in the parser, so that the
4972 function @code{yylex} (if it is in this file) can use the name @var{name} to
4973 stand for this token kind's code.
4975 Alternatively, you can use @code{%left}, @code{%right}, @code{%precedence},
4976 or @code{%nonassoc} instead of @code{%token}, if you wish to specify
4977 associativity and precedence. @xref{Precedence Decl}. However, for
4978 clarity, we recommend to use these directives only to declare associativity
4979 and precedence, and not to add string aliases, semantic types, etc.
4981 You can explicitly specify the numeric code for a token kind by appending a
4982 nonnegative decimal or hexadecimal integer value in the field immediately
4983 following the token name:
4987 %token XNUM 0x12d // a GNU extension
4991 It is generally best, however, to let Bison choose the numeric codes for all
4992 token kinds. Bison will automatically select codes that don't conflict with
4993 each other or with normal characters.
4995 In the event that the stack type is a union, you must augment the
4996 @code{%token} or other token declaration to include the data type
4997 alternative delimited by angle-brackets (@pxref{Multiple Types}).
5003 %union @{ /* define stack type */
5007 %token <val> NUM /* define token NUM and its type */
5011 You can associate a literal string token with a token kind name by writing
5012 the literal string at the end of a @code{%token} declaration which declares
5013 the name. For example:
5020 For example, a grammar for the C language might specify these names with
5021 equivalent literal string tokens:
5024 %token <operator> OR "||"
5025 %token <operator> LE 134 "<="
5030 Once you equate the literal string and the token kind name, you can use them
5031 interchangeably in further declarations or the grammar rules. The
5032 @code{yylex} function can use the token name or the literal string to obtain
5033 the token kind code (@pxref{Calling Convention}).
5035 String aliases allow for better error messages using the literal strings
5036 instead of the token names, such as @samp{syntax error, unexpected ||,
5037 expecting number or (} rather than @samp{syntax error, unexpected OR,
5038 expecting NUM or LPAREN}.
5040 String aliases may also be marked for internationalization (@pxref{Token
5048 '\n' _("end of line")
5054 would produce in French @samp{erreur de syntaxe, || inattendu, attendait
5055 nombre ou (} rather than @samp{erreur de syntaxe, || inattendu, attendait
5058 @node Precedence Decl
5059 @subsection Operator Precedence
5060 @cindex precedence declarations
5061 @cindex declaring operator precedence
5062 @cindex operator precedence, declaring
5064 Use the @code{%left}, @code{%right}, @code{%nonassoc}, or @code{%precedence}
5065 declaration to declare a token and specify its precedence and associativity,
5066 all at once. These are called @dfn{precedence declarations}.
5067 @xref{Precedence}, for general information on operator
5070 The syntax of a precedence declaration is nearly the same as that of
5071 @code{%token}: either
5074 %left @var{symbols}@dots{}
5081 %left <@var{type}> @var{symbols}@dots{}
5084 And indeed any of these declarations serves the purposes of @code{%token}.
5085 But in addition, they specify the associativity and relative precedence for
5086 all the @var{symbols}:
5090 The associativity of an operator @var{op} determines how repeated uses of
5091 the operator nest: whether @samp{@var{x} @var{op} @var{y} @var{op} @var{z}}
5092 is parsed by grouping @var{x} with @var{y} first or by grouping @var{y} with
5093 @var{z} first. @code{%left} specifies left-associativity (grouping @var{x}
5094 with @var{y} first) and @code{%right} specifies right-associativity
5095 (grouping @var{y} with @var{z} first). @code{%nonassoc} specifies no
5096 associativity, which means that @samp{@var{x} @var{op} @var{y} @var{op}
5097 @var{z}} is considered a syntax error.
5099 @code{%precedence} gives only precedence to the @var{symbols}, and defines
5100 no associativity at all. Use this to define precedence only, and leave any
5101 potential conflict due to associativity enabled.
5104 The precedence of an operator determines how it nests with other operators.
5105 All the tokens declared in a single precedence declaration have equal
5106 precedence and nest together according to their associativity. When two
5107 tokens declared in different precedence declarations associate, the one
5108 declared later has the higher precedence and is grouped first.
5111 For backward compatibility, there is a confusing difference between the
5112 argument lists of @code{%token} and precedence declarations. Only a
5113 @code{%token} can associate a literal string with a token kind name. A
5114 precedence declaration always interprets a literal string as a reference to
5115 a separate token. For example:
5118 %left OR "<=" // Does not declare an alias.
5119 %left OR 134 "<=" 135 // Declares 134 for OR and 135 for "<=".
5123 @subsection Nonterminal Symbols
5124 @cindex declaring value types, nonterminals
5125 @cindex value types, nonterminals, declaring
5130 When you use @code{%union} to specify multiple value types, you must
5131 declare the value type of each nonterminal symbol for which values are
5132 used. This is done with a @code{%type} declaration, like this:
5135 %type <@var{type}> @var{nonterminal}@dots{}
5139 Here @var{nonterminal} is the name of a nonterminal symbol, and @var{type}
5140 is the name given in the @code{%union} to the alternative that you want
5141 (@pxref{Union Decl}). You can give any number of nonterminal symbols in the
5142 same @code{%type} declaration, if they have the same value type. Use spaces
5143 to separate the symbol names.
5145 While POSIX Yacc allows @code{%type} only for nonterminals, Bison accepts
5146 that this directive be also applied to terminal symbols. To declare
5147 exclusively nonterminal symbols, use the safer @code{%nterm}:
5150 %nterm <@var{type}> @var{nonterminal}@dots{}
5155 @subsection Syntax of Symbol Declarations
5161 The syntax of the various directives to declare symbols is as follows.
5164 %token @var{tag}? ( @var{id} @var{number}? @var{string}? )+ ( @var{tag} ( @var{id} @var{number}? @var{string}? )+ )*
5165 %left @var{tag}? ( @var{id} @var{number}?)+ ( @var{tag} ( @var{id} @var{number}? )+ )*
5166 %type @var{tag}? ( @var{id} | @var{char} | @var{string} )+ ( @var{tag} ( @var{id} | @var{char} | @var{string} )+ )*
5167 %nterm @var{tag}? @var{id}+ ( @var{tag} @var{id}+ )*
5171 where @var{tag} denotes a type tag such as @samp{<ival>}, @var{id} denotes
5172 an identifier such as @samp{NUM}, @var{number} a decimal or hexadecimal
5173 integer such as @samp{300} or @samp{0x12d}, @var{char} a character literal
5174 such as @samp{'+'}, and @var{string} a string literal such as
5175 @samp{"number"}. The postfix quantifiers are @samp{?} (zero or one),
5176 @samp{*} (zero or more) and @samp{+} (one or more).
5178 The directives @code{%precedence}, @code{%right} and @code{%nonassoc} behave
5181 @node Initial Action Decl
5182 @subsection Performing Actions before Parsing
5183 @findex %initial-action
5185 Sometimes your parser needs to perform some initializations before parsing.
5186 The @code{%initial-action} directive allows for such arbitrary code.
5188 @deffn {Directive} %initial-action @{ @var{code} @}
5189 @findex %initial-action
5190 Declare that the braced @var{code} must be invoked before parsing each time
5191 @code{yyparse} is called. The @var{code} may use @code{$$} (or
5192 @code{$<@var{tag}>$}) and @code{@@$} --- initial value and location of the
5193 lookahead --- and the @code{%parse-param}.
5196 For instance, if your locations use a file name, you may use
5199 %parse-param @{ char const *file_name @};
5202 @@$.initialize (file_name);
5207 @node Destructor Decl
5208 @subsection Freeing Discarded Symbols
5209 @cindex freeing discarded symbols
5213 During error recovery (@pxref{Error Recovery}), symbols already pushed
5214 on the stack and tokens coming from the rest of the file are discarded
5215 until the parser falls on its feet. If the parser runs out of memory,
5216 or if it returns via @code{YYABORT} or @code{YYACCEPT}, all the
5217 symbols on the stack must be discarded. Even if the parser succeeds, it
5218 must discard the start symbol.
5220 When discarded symbols convey heap based information, this memory is
5221 lost. While this behavior can be tolerable for batch parsers, such as
5222 in traditional compilers, it is unacceptable for programs like shells or
5223 protocol implementations that may parse and execute indefinitely.
5225 The @code{%destructor} directive defines code that is called when a
5226 symbol is automatically discarded.
5228 @deffn {Directive} %destructor @{ @var{code} @} @var{symbols}
5230 Invoke the braced @var{code} whenever the parser discards one of the
5231 @var{symbols}. Within @var{code}, @code{$$} (or @code{$<@var{tag}>$})
5232 designates the semantic value associated with the discarded symbol, and
5233 @code{@@$} designates its location. The additional parser parameters are
5234 also available (@pxref{Parser Function}).
5236 When a symbol is listed among @var{symbols}, its @code{%destructor} is called a
5237 per-symbol @code{%destructor}.
5238 You may also define a per-type @code{%destructor} by listing a semantic type
5239 tag among @var{symbols}.
5240 In that case, the parser will invoke this @var{code} whenever it discards any
5241 grammar symbol that has that semantic type tag unless that symbol has its own
5242 per-symbol @code{%destructor}.
5244 Finally, you can define two different kinds of default @code{%destructor}s.
5245 You can place each of @code{<*>} and @code{<>} in the @var{symbols} list of
5246 exactly one @code{%destructor} declaration in your grammar file.
5247 The parser will invoke the @var{code} associated with one of these whenever it
5248 discards any user-defined grammar symbol that has no per-symbol and no per-type
5250 The parser uses the @var{code} for @code{<*>} in the case of such a grammar
5251 symbol for which you have formally declared a semantic type tag (@code{%token},
5252 @code{%nterm}, and @code{%type}
5253 count as such a declaration, but @code{$<tag>$} does not).
5254 The parser uses the @var{code} for @code{<>} in the case of such a grammar
5255 symbol that has no declared semantic type tag.
5262 %union @{ char *string; @}
5263 %token <string> STRING1 STRING2
5264 %nterm <string> string1 string2
5265 %union @{ char character; @}
5266 %token <character> CHR
5267 %nterm <character> chr
5270 %destructor @{ @} <character>
5271 %destructor @{ free ($$); @} <*>
5272 %destructor @{ free ($$); printf ("%d", @@$.first_line); @} STRING1 string1
5273 %destructor @{ printf ("Discarding tagless symbol.\n"); @} <>
5277 guarantees that, when the parser discards any user-defined symbol that has a
5278 semantic type tag other than @code{<character>}, it passes its semantic value
5279 to @code{free} by default.
5280 However, when the parser discards a @code{STRING1} or a @code{string1},
5281 it uses the third @code{%destructor}, which frees it and
5282 prints its line number to @code{stdout} (@code{free} is invoked only once).
5283 Finally, the parser merely prints a message whenever it discards any symbol,
5284 such as @code{TAGLESS}, that has no semantic type tag.
5286 A Bison-generated parser invokes the default @code{%destructor}s only for
5287 user-defined as opposed to Bison-defined symbols.
5288 For example, the parser will not invoke either kind of default
5289 @code{%destructor} for the special Bison-defined symbols @code{$accept},
5290 @code{$undefined}, or @code{$end} (@pxref{Table of Symbols}),
5291 none of which you can reference in your grammar.
5292 It also will not invoke either for the @code{error} token (@pxref{Table of
5293 Symbols}), which is always defined by Bison regardless of whether you
5294 reference it in your grammar.
5295 However, it may invoke one of them for the end token (token 0) if you
5296 redefine it from @code{$end} to, for example, @code{END}:
5302 @cindex actions in midrule
5303 @cindex midrule actions
5304 Finally, Bison will never invoke a @code{%destructor} for an unreferenced
5305 midrule semantic value (@pxref{Midrule Actions}).
5306 That is, Bison does not consider a midrule to have a semantic value if you
5307 do not reference @code{$$} in the midrule's action or @code{$@var{n}}
5308 (where @var{n} is the right-hand side symbol position of the midrule) in
5309 any later action in that rule. However, if you do reference either, the
5310 Bison-generated parser will invoke the @code{<>} @code{%destructor} whenever
5311 it discards the midrule symbol.
5315 In the future, it may be possible to redefine the @code{error} token as a
5316 nonterminal that captures the discarded symbols.
5317 In that case, the parser will invoke the default destructor for it as well.
5322 @cindex discarded symbols
5323 @dfn{Discarded symbols} are the following:
5327 stacked symbols popped during the first phase of error recovery,
5329 incoming terminals during the second phase of error recovery,
5331 the current lookahead and the entire stack (except the current
5332 right-hand side symbols) when the parser returns immediately, and
5334 the current lookahead and the entire stack (including the current right-hand
5335 side symbols) when the C++ parser (@file{lalr1.cc}) catches an exception in
5338 the start symbol, when the parser succeeds.
5341 The parser can @dfn{return immediately} because of an explicit call to
5342 @code{YYABORT} or @code{YYACCEPT}, or failed error recovery, or memory
5345 Right-hand side symbols of a rule that explicitly triggers a syntax
5346 error via @code{YYERROR} are not discarded automatically. As a rule
5347 of thumb, destructors are invoked only when user actions cannot manage
5351 @subsection Printing Semantic Values
5352 @cindex printing semantic values
5356 When run-time traces are enabled (@pxref{Tracing}),
5357 the parser reports its actions, such as reductions. When a symbol involved
5358 in an action is reported, only its kind is displayed, as the parser cannot
5359 know how semantic values should be formatted.
5361 The @code{%printer} directive defines code that is called when a symbol is
5362 reported. Its syntax is the same as @code{%destructor} (@pxref{Destructor
5365 @deffn {Directive} %printer @{ @var{code} @} @var{symbols}
5368 @c This is the same text as for %destructor.
5369 Invoke the braced @var{code} whenever the parser displays one of the
5370 @var{symbols}. Within @var{code}, @code{yyo} denotes the output stream (a
5371 @code{FILE*} in C, and an @code{std::ostream&} in C++), @code{$$} (or
5372 @code{$<@var{tag}>$}) designates the semantic value associated with the
5373 symbol, and @code{@@$} its location. The additional parser parameters are
5374 also available (@pxref{Parser Function}).
5376 The @var{symbols} are defined as for @code{%destructor} (@pxref{Destructor
5377 Decl}.): they can be per-type (e.g.,
5378 @samp{<ival>}), per-symbol (e.g., @samp{exp}, @samp{NUM}, @samp{"float"}),
5379 typed per-default (i.e., @samp{<*>}, or untyped per-default (i.e.,
5387 %union @{ char *string; @}
5388 %token <string> STRING1 STRING2
5389 %nterm <string> string1 string2
5390 %union @{ char character; @}
5391 %token <character> CHR
5392 %nterm <character> chr
5395 %printer @{ fprintf (yyo, "'%c'", $$); @} <character>
5396 %printer @{ fprintf (yyo, "&%p", $$); @} <*>
5397 %printer @{ fprintf (yyo, "\"%s\"", $$); @} STRING1 string1
5398 %printer @{ fprintf (yyo, "<>"); @} <>
5402 guarantees that, when the parser print any symbol that has a semantic type
5403 tag other than @code{<character>}, it display the address of the semantic
5404 value by default. However, when the parser displays a @code{STRING1} or a
5405 @code{string1}, it formats it as a string in double quotes. It performs
5406 only the second @code{%printer} in this case, so it prints only once.
5407 Finally, the parser print @samp{<>} for any symbol, such as @code{TAGLESS},
5408 that has no semantic type tag. @xref{Mfcalc Traces}, for a complete example.
5413 @subsection Suppressing Conflict Warnings
5414 @cindex suppressing conflict warnings
5415 @cindex preventing warnings about conflicts
5416 @cindex warnings, preventing
5417 @cindex conflicts, suppressing warnings of
5421 Bison normally warns if there are any conflicts in the grammar
5422 (@pxref{Shift/Reduce}), but most real grammars
5423 have harmless shift/reduce conflicts which are resolved in a predictable
5424 way and would be difficult to eliminate. It is desirable to suppress
5425 the warning about these conflicts unless the number of conflicts
5426 changes. You can do this with the @code{%expect} declaration.
5428 The declaration looks like this:
5434 Here @var{n} is a decimal integer. The declaration says there should
5435 be @var{n} shift/reduce conflicts and no reduce/reduce conflicts.
5436 Bison reports an error if the number of shift/reduce conflicts differs
5437 from @var{n}, or if there are any reduce/reduce conflicts.
5439 For deterministic parsers, reduce/reduce conflicts are more
5440 serious, and should be eliminated entirely. Bison will always report
5441 reduce/reduce conflicts for these parsers. With GLR
5442 parsers, however, both kinds of conflicts are routine; otherwise,
5443 there would be no need to use GLR parsing. Therefore, it is
5444 also possible to specify an expected number of reduce/reduce conflicts
5445 in GLR parsers, using the declaration:
5451 You may wish to be more specific in your
5452 specification of expected conflicts. To this end, you can also attach
5453 @code{%expect} and @code{%expect-rr} modifiers to individual rules.
5454 The interpretation of these modifiers differs from their use as
5455 declarations. When attached to rules, they indicate the number of states
5456 in which the rule is involved in a conflict. You will need to consult the
5457 output resulting from @option{-v} to determine appropriate numbers to use.
5458 For example, for the following grammar fragment, the first rule for
5459 @code{empty_dims} appears in two states in which the @samp{[} token is a
5460 lookahead. Having determined that, you can document this fact with an
5461 @code{%expect} modifier as follows:
5471 | empty_dims '[' ']'
5475 Mid-rule actions generate implicit rules that are also subject to conflicts
5476 (@pxref{Midrule Conflicts}). To attach
5477 an @code{%expect} or @code{%expect-rr} annotation to an implicit
5478 mid-rule action's rule, put it before the action. For example,
5487 "condition" %expect-rr 1 @{ value_mode(); @} '(' exprs ')'
5488 | "condition" %expect-rr 1 @{ class_mode(); @} '(' types ')'
5493 Here, the appropriate mid-rule action will not be determined until after
5494 the @samp{(} token is shifted. Thus,
5495 the two actions will clash with each other, and we should expect one
5496 reduce/reduce conflict for each.
5498 In general, using @code{%expect} involves these steps:
5502 Compile your grammar without @code{%expect}. Use the @option{-v} option
5503 to get a verbose list of where the conflicts occur. Bison will also
5504 print the number of conflicts.
5507 Check each of the conflicts to make sure that Bison's default
5508 resolution is what you really want. If not, rewrite the grammar and
5509 go back to the beginning.
5512 Add an @code{%expect} declaration, copying the number @var{n} from the
5513 number that Bison printed. With GLR parsers, add an
5514 @code{%expect-rr} declaration as well.
5517 Optionally, count up the number of states in which one or more
5518 conflicted reductions for particular rules appear and add these numbers
5519 to the affected rules as @code{%expect-rr} or @code{%expect} modifiers
5520 as appropriate. Rules that are in conflict appear in the output listing
5521 surrounded by square brackets or, in the case of reduce/reduce conflicts,
5522 as reductions having the same lookahead symbol as a square-bracketed
5523 reduction in the same state.
5526 Now Bison will report an error if you introduce an unexpected conflict,
5527 but will keep silent otherwise.
5530 @subsection The Start-Symbol
5531 @cindex declaring the start symbol
5532 @cindex start symbol, declaring
5533 @cindex default start symbol
5536 Bison assumes by default that the start symbol for the grammar is the first
5537 nonterminal specified in the grammar specification section. The programmer
5538 may override this restriction with the @code{%start} declaration as follows:
5545 @subsection A Pure (Reentrant) Parser
5546 @cindex reentrant parser
5548 @findex %define api.pure
5550 A @dfn{reentrant} program is one which does not alter in the course of
5551 execution; in other words, it consists entirely of @dfn{pure} (read-only)
5552 code. Reentrancy is important whenever asynchronous execution is possible;
5553 for example, a nonreentrant program may not be safe to call from a signal
5554 handler. In systems with multiple threads of control, a nonreentrant
5555 program must be called only within interlocks.
5557 Normally, Bison generates a parser which is not reentrant. This is
5558 suitable for most uses, and it permits compatibility with Yacc. (The
5559 standard Yacc interfaces are inherently nonreentrant, because they use
5560 statically allocated variables for communication with @code{yylex},
5561 including @code{yylval} and @code{yylloc}.)
5563 Alternatively, you can generate a pure, reentrant parser. The Bison
5564 declaration @samp{%define api.pure} says that you want the parser to be
5565 reentrant. It looks like this:
5568 %define api.pure full
5571 The result is that the communication variables @code{yylval} and
5572 @code{yylloc} become local variables in @code{yyparse}, and a different
5573 calling convention is used for the lexical analyzer function @code{yylex}.
5574 @xref{Pure Calling}, for the details of this. The variable @code{yynerrs}
5575 becomes local in @code{yyparse} in pull mode but it becomes a member of
5576 @code{yypstate} in push mode. (@pxref{Error Reporting Function}). The
5577 convention for calling @code{yyparse} itself is unchanged.
5579 Whether the parser is pure has nothing to do with the grammar rules.
5580 You can generate either a pure parser or a nonreentrant parser from any
5584 @subsection A Push Parser
5587 @findex %define api.push-pull
5589 A pull parser is called once and it takes control until all its input
5590 is completely parsed. A push parser, on the other hand, is called
5591 each time a new token is made available.
5593 A push parser is typically useful when the parser is part of a
5594 main event loop in the client's application. This is typically
5595 a requirement of a GUI, when the main event loop needs to be triggered
5596 within a certain time period.
5598 Normally, Bison generates a pull parser.
5599 The following Bison declaration says that you want the parser to be a push
5600 parser (@pxref{%define Summary}):
5603 %define api.push-pull push
5606 In almost all cases, you want to ensure that your push parser is also
5607 a pure parser (@pxref{Pure Decl}). The only
5608 time you should create an impure push parser is to have backwards
5609 compatibility with the impure Yacc pull mode interface. Unless you know
5610 what you are doing, your declarations should look like this:
5613 %define api.pure full
5614 %define api.push-pull push
5617 There is a major notable functional difference between the pure push parser
5618 and the impure push parser. It is acceptable for a pure push parser to have
5619 many parser instances, of the same type of parser, in memory at the same time.
5620 An impure push parser should only use one parser at a time.
5622 When a push parser is selected, Bison will generate some new symbols in
5623 the generated parser. @code{yypstate} is a structure that the generated
5624 parser uses to store the parser's state. @code{yypstate_new} is the
5625 function that will create a new parser instance. @code{yypstate_delete}
5626 will free the resources associated with the corresponding parser instance.
5627 Finally, @code{yypush_parse} is the function that should be called whenever a
5628 token is available to provide the parser. A trivial example
5629 of using a pure push parser would look like this:
5633 yypstate *ps = yypstate_new ();
5635 status = yypush_parse (ps, yylex (), NULL);
5636 @} while (status == YYPUSH_MORE);
5637 yypstate_delete (ps);
5640 If the user decided to use an impure push parser, a few things about the
5641 generated parser will change. The @code{yychar} variable becomes a global
5642 variable instead of a local one in the @code{yypush_parse} function. For
5643 this reason, the signature of the @code{yypush_parse} function is changed to
5644 remove the token as a parameter. A nonreentrant push parser example would
5645 thus look like this:
5650 yypstate *ps = yypstate_new ();
5653 status = yypush_parse (ps);
5654 @} while (status == YYPUSH_MORE);
5655 yypstate_delete (ps);
5658 That's it. Notice the next token is put into the global variable @code{yychar}
5659 for use by the next invocation of the @code{yypush_parse} function.
5661 Bison also supports both the push parser interface along with the pull parser
5662 interface in the same generated parser. In order to get this functionality,
5663 you should replace the @samp{%define api.push-pull push} declaration with the
5664 @samp{%define api.push-pull both} declaration. Doing this will create all of
5665 the symbols mentioned earlier along with the two extra symbols, @code{yyparse}
5666 and @code{yypull_parse}. @code{yyparse} can be used exactly as it normally
5667 would be used. However, the user should note that it is implemented in the
5668 generated parser by calling @code{yypull_parse}.
5669 This makes the @code{yyparse} function that is generated with the
5670 @samp{%define api.push-pull both} declaration slower than the normal
5671 @code{yyparse} function. If the user
5672 calls the @code{yypull_parse} function it will parse the rest of the input
5673 stream. It is possible to @code{yypush_parse} tokens to select a subgrammar
5674 and then @code{yypull_parse} the rest of the input stream. If you would like
5675 to switch back and forth between between parsing styles, you would have to
5676 write your own @code{yypull_parse} function that knows when to quit looking
5677 for input. An example of using the @code{yypull_parse} function would look
5681 yypstate *ps = yypstate_new ();
5682 yypull_parse (ps); /* Will call the lexer */
5683 yypstate_delete (ps);
5686 Adding the @samp{%define api.pure} declaration does exactly the same thing to
5687 the generated parser with @samp{%define api.push-pull both} as it did for
5688 @samp{%define api.push-pull push}.
5691 @subsection Bison Declaration Summary
5692 @cindex Bison declaration summary
5693 @cindex declaration summary
5694 @cindex summary, Bison declaration
5696 Here is a summary of the declarations used to define a grammar:
5698 @deffn {Directive} %union
5699 Declare the collection of data types that semantic values may have
5700 (@pxref{Union Decl}).
5703 @deffn {Directive} %token
5704 Declare a terminal symbol (token kind name) with no precedence
5705 or associativity specified (@pxref{Token Decl}).
5708 @deffn {Directive} %right
5709 Declare a terminal symbol (token kind name) that is right-associative
5710 (@pxref{Precedence Decl}).
5713 @deffn {Directive} %left
5714 Declare a terminal symbol (token kind name) that is left-associative
5715 (@pxref{Precedence Decl}).
5718 @deffn {Directive} %nonassoc
5719 Declare a terminal symbol (token kind name) that is nonassociative
5720 (@pxref{Precedence Decl}).
5721 Using it in a way that would be associative is a syntax error.
5725 @deffn {Directive} %default-prec
5726 Assign a precedence to rules lacking an explicit @code{%prec} modifier
5727 (@pxref{Contextual Precedence}).
5731 @deffn {Directive} %nterm
5732 Declare the type of semantic values for a nonterminal symbol (@pxref{Type
5736 @deffn {Directive} %type
5737 Declare the type of semantic values for a symbol (@pxref{Type Decl}).
5740 @deffn {Directive} %start
5741 Specify the grammar's start symbol (@pxref{Start Decl}).
5744 @deffn {Directive} %expect
5745 Declare the expected number of shift/reduce conflicts, either overall or
5747 (@pxref{Expect Decl}).
5750 @deffn {Directive} %expect-rr
5751 Declare the expected number of reduce/reduce conflicts, either overall or
5753 (@pxref{Expect Decl}).
5759 In order to change the behavior of @command{bison}, use the following
5762 @deffn {Directive} %code @{@var{code}@}
5763 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
5765 Insert @var{code} verbatim into the output parser source at the
5766 default location or at the location specified by @var{qualifier}.
5767 @xref{%code Summary}.
5770 @deffn {Directive} %debug
5771 Instrument the parser for traces. Obsoleted by @samp{%define
5776 @deffn {Directive} %define @var{variable}
5777 @deffnx {Directive} %define @var{variable} @var{value}
5778 @deffnx {Directive} %define @var{variable} @{@var{value}@}
5779 @deffnx {Directive} %define @var{variable} "@var{value}"
5780 Define a variable to adjust Bison's behavior. @xref{%define Summary}.
5783 @deffn {Directive} %defines
5784 Write a parser header file containing definitions for the token kind names
5785 defined in the grammar as well as a few other declarations. If the parser
5786 implementation file is named @file{@var{name}.c} then the parser header file
5787 is named @file{@var{name}.h}.
5789 For C parsers, the parser header file declares @code{YYSTYPE} unless
5790 @code{YYSTYPE} is already defined as a macro or you have used a
5791 @code{<@var{type}>} tag without using @code{%union}. Therefore, if you are
5792 using a @code{%union} (@pxref{Multiple Types}) with components that require
5793 other definitions, or if you have defined a @code{YYSTYPE} macro or type
5794 definition (@pxref{Value Type}), you need to arrange for these definitions
5795 to be propagated to all modules, e.g., by putting them in a prerequisite
5796 header that is included both by your parser and by any other module that
5797 needs @code{YYSTYPE}.
5799 Unless your parser is pure, the parser header file declares
5800 @code{yylval} as an external variable. @xref{Pure Decl}.
5802 If you have also used locations, the parser header file declares
5803 @code{YYLTYPE} and @code{yylloc} using a protocol similar to that of the
5804 @code{YYSTYPE} macro and @code{yylval}. @xref{Tracking Locations}.
5806 This parser header file is normally essential if you wish to put the
5807 definition of @code{yylex} in a separate source file, because
5808 @code{yylex} typically needs to be able to refer to the
5809 above-mentioned declarations and to the token kind codes. @xref{Token
5812 @findex %code requires
5813 @findex %code provides
5814 If you have declared @code{%code requires} or @code{%code provides}, the output
5815 header also contains their code.
5816 @xref{%code Summary}.
5818 @cindex Header guard
5819 The generated header is protected against multiple inclusions with a C
5820 preprocessor guard: @samp{YY_@var{PREFIX}_@var{FILE}_INCLUDED}, where
5821 @var{PREFIX} and @var{FILE} are the prefix (@pxref{Multiple Parsers}) and
5822 generated file name turned uppercase, with each series of non alphanumerical
5823 characters converted to a single underscore.
5825 For instance with @samp{%define api.prefix @{calc@}} and @samp{%defines
5826 "lib/parse.h"}, the header will be guarded as follows.
5828 #ifndef YY_CALC_LIB_PARSE_H_INCLUDED
5829 # define YY_CALC_LIB_PARSE_H_INCLUDED
5831 #endif /* ! YY_CALC_LIB_PARSE_H_INCLUDED */
5835 @deffn {Directive} %defines @var{defines-file}
5836 Same as above, but save in the file @file{@var{defines-file}}.
5839 @deffn {Directive} %destructor
5840 Specify how the parser should reclaim the memory associated to
5841 discarded symbols. @xref{Destructor Decl}.
5844 @deffn {Directive} %file-prefix "@var{prefix}"
5845 Specify a prefix to use for all Bison output file names. The names
5846 are chosen as if the grammar file were named @file{@var{prefix}.y}.
5849 @deffn {Directive} %language "@var{language}"
5850 Specify the programming language for the generated parser. Currently
5851 supported languages include C, C++, and Java. @var{language} is
5855 @deffn {Directive} %locations
5856 Generate the code processing the locations (@pxref{Action Features}). This
5857 mode is enabled as soon as the grammar uses the special @samp{@@@var{n}}
5858 tokens, but if your grammar does not use it, using @samp{%locations} allows
5859 for more accurate syntax error messages.
5862 @deffn {Directive} %name-prefix "@var{prefix}"
5863 Obsoleted by @samp{%define api.prefix @{@var{prefix}@}}. @xref{Multiple
5864 Parsers}. For C++ parsers, see the
5865 @samp{%define api.namespace} documentation in this section.
5867 Rename the external symbols used in the parser so that they start with
5868 @var{prefix} instead of @samp{yy}. The precise list of symbols renamed in C
5869 parsers is @code{yyparse}, @code{yylex}, @code{yyerror}, @code{yynerrs},
5870 @code{yylval}, @code{yychar}, @code{yydebug}, and (if locations are used)
5871 @code{yylloc}. If you use a push parser, @code{yypush_parse},
5872 @code{yypull_parse}, @code{yypstate}, @code{yypstate_new} and
5873 @code{yypstate_delete} will also be renamed. For example, if you use
5874 @samp{%name-prefix "c_"}, the names become @code{c_parse}, @code{c_lex}, and
5877 Contrary to defining @code{api.prefix}, some symbols are @emph{not} renamed
5878 by @code{%name-prefix}, for instance @code{YYDEBUG}, @code{YYTOKENTYPE},
5879 @code{yytoken_kind_t}, @code{YYSTYPE}, @code{YYLTYPE}.
5883 @deffn {Directive} %no-default-prec
5884 Do not assign a precedence to rules lacking an explicit @code{%prec}
5885 modifier (@pxref{Contextual Precedence}).
5889 @deffn {Directive} %no-lines
5890 Don't generate any @code{#line} preprocessor commands in the parser
5891 implementation file. Ordinarily Bison writes these commands in the parser
5892 implementation file so that the C compiler and debuggers will associate
5893 errors and object code with your source file (the grammar file). This
5894 directive causes them to associate errors with the parser implementation
5895 file, treating it as an independent source file in its own right.
5898 @deffn {Directive} %output "@var{file}"
5899 Generate the parser implementation in @file{@var{file}}.
5902 @deffn {Directive} %pure-parser
5903 Deprecated version of @samp{%define api.pure} (@pxref{%define
5904 Summary}), for which Bison is more careful to warn about
5908 @deffn {Directive} %require "@var{version}"
5909 Require version @var{version} or higher of Bison. @xref{Require Decl}.
5912 @deffn {Directive} %skeleton "@var{file}"
5913 Specify the skeleton to use.
5915 @c You probably don't need this option unless you are developing Bison.
5916 @c You should use @code{%language} if you want to specify the skeleton for a
5917 @c different language, because it is clearer and because it will always choose the
5918 @c correct skeleton for non-deterministic or push parsers.
5920 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
5921 file in the Bison installation directory.
5922 If it does, @var{file} is an absolute file name or a file name relative to the
5923 directory of the grammar file.
5924 This is similar to how most shells resolve commands.
5927 @deffn {Directive} %token-table
5928 This feature is obsolescent, avoid it in new projects.
5930 Generate an array of token names in the parser implementation file. The
5931 name of the array is @code{yytname}; @code{yytname[@var{i}]} is the name of
5932 the token whose internal Bison token code is @var{i}. The first three
5933 elements of @code{yytname} correspond to the predefined tokens
5934 @code{"$end"}, @code{"error"}, and @code{"$undefined"}; after these come the
5935 symbols defined in the grammar file.
5937 The name in the table includes all the characters needed to represent the
5938 token in Bison. For single-character literals and literal strings, this
5939 includes the surrounding quoting characters and any escape sequences. For
5940 example, the Bison single-character literal @code{'+'} corresponds to a
5941 three-character name, represented in C as @code{"'+'"}; and the Bison
5942 two-character literal string @code{"\\/"} corresponds to a five-character
5943 name, represented in C as @code{"\"\\\\/\""}.
5945 When you specify @code{%token-table}, Bison also generates macro definitions
5946 for macros @code{YYNTOKENS}, @code{YYNNTS}, and @code{YYNRULES}, and
5951 The number of terminal symbols, i.e., the highest token code, plus one.
5953 The number of nonterminal symbols.
5955 The number of grammar rules,
5957 The number of parser states (@pxref{Parser States}).
5960 Here's code for looking up a multicharacter token in @code{yytname},
5961 assuming that the characters of the token are stored in @code{token_buffer},
5962 and assuming that the token does not contain any characters like @samp{"}
5963 that require escaping.
5966 for (int i = 0; i < YYNTOKENS; i++)
5968 && yytname[i][0] == '"'
5969 && ! strncmp (yytname[i] + 1, token_buffer,
5970 strlen (token_buffer))
5971 && yytname[i][strlen (token_buffer) + 1] == '"'
5972 && yytname[i][strlen (token_buffer) + 2] == 0)
5976 This method is discouraged: the primary purpose of string aliases is forging
5977 good error messages, not describing the spelling of keywords. In addition,
5978 looking for the token kind at runtime incurs a (small but noticeable) cost.
5980 Finally, @code{%token-table} is incompatible with the @code{custom} and
5981 @code{detailed} values of the @code{parse.error} @code{%define} variable.
5984 @deffn {Directive} %verbose
5985 Write an extra output file containing verbose descriptions of the parser
5986 states and what is done for each type of lookahead token in that state.
5987 @xref{Understanding}, for more information.
5990 @deffn {Directive} %yacc
5991 Pretend the option @option{--yacc} was given, i.e., imitate Yacc, including
5992 its naming conventions. Only makes sense with the @file{yacc.c}
5993 skeleton. @xref{Tuning the Parser}, for more.
5995 Of course @code{%yacc} is a Bison extension@dots{}
5999 @node %define Summary
6000 @subsection %define Summary
6002 There are many features of Bison's behavior that can be controlled by
6003 assigning the feature a single value. For historical reasons, some such
6004 features are assigned values by dedicated directives, such as @code{%start},
6005 which assigns the start symbol. However, newer such features are associated
6006 with variables, which are assigned by the @code{%define} directive:
6008 @deffn {Directive} %define @var{variable}
6009 @deffnx {Directive} %define @var{variable} @var{value}
6010 @deffnx {Directive} %define @var{variable} @{@var{value}@}
6011 @deffnx {Directive} %define @var{variable} "@var{value}"
6012 Define @var{variable} to @var{value}.
6014 The type of the values depend on the syntax. Braces denote value in the
6015 target language (e.g., a namespace, a type, etc.). Keyword values (no
6016 delimiters) denote finite choice (e.g., a variation of a feature). String
6017 values denote remaining cases (e.g., a file name).
6019 It is an error if a @var{variable} is defined by @code{%define} multiple
6020 times, but see @ref{Tuning the Parser,,@option{-D @var{name}[=@var{value}]}}.
6023 The rest of this section summarizes variables and values that @code{%define}
6026 Some @var{variable}s take Boolean values. In this case, Bison will complain
6027 if the variable definition does not meet one of the following four
6031 @item @code{@var{value}} is @code{true}
6033 @item @code{@var{value}} is omitted (or @code{""} is specified).
6034 This is equivalent to @code{true}.
6036 @item @code{@var{value}} is @code{false}.
6038 @item @var{variable} is never defined.
6039 In this case, Bison selects a default value.
6042 What @var{variable}s are accepted, as well as their meanings and default
6043 values, depend on the selected target language and/or the parser skeleton
6044 (@pxref{Decl Summary}, @pxref{Decl Summary}).
6045 Unaccepted @var{variable}s produce an error. Some of the accepted
6046 @var{variable}s are described below.
6049 @c ================================================== api.filename.file
6050 @anchor{api-filename-type}
6051 @deffn {Directive} {%define api.filename.type} @{@var{type}@}
6054 @item Language(s): C++
6057 Define the type of file names in Bison's default location and position
6058 types. @xref{Exposing the Location Classes}.
6060 @item Accepted Values:
6061 Any type that is printable (via streams) and comparable (with @code{==} and
6064 @item Default Value: @code{const std::string}.
6067 Introduced in Bison 2.0 as @code{filename_type} (with @code{std::string} as
6068 default), renamed as @code{api.filename.type} in Bison 3.7 (with @code{const
6069 std::string} as default).
6074 @c ================================================== api.header.include
6075 @deffn Directive {%define api.header.include} @{"header.h"@}
6076 @deffnx Directive {%define api.header.include} @{<header.h>@}
6078 @item Languages(s): C (@file{yacc.c})
6080 @item Purpose: Specify how the generated parser should include the generated header.
6082 Historically, when option @option{-D}/@option{--defines} was used,
6083 @command{bison} generated a header and pasted an exact copy of it into the
6084 generated parser implementation file. Since Bison 3.6, it is
6085 @code{#include}d as @samp{"@var{basename}.h"}, instead of duplicated, unless
6086 @var{file} is @samp{y.tab}, see below.
6088 The @code{api.header.include} variable allows to control how the generated
6089 parser @code{#include}s the generated header. For instance:
6092 %define api.header.include @{"parse.h"@}
6099 %define api.header.include @{<parser/parse.h>@}
6102 Using @code{api.header.include} does not change the name of the generated
6103 header, only how it is included.
6105 To work around limitations of Automake's @command{ylwrap} (which runs
6106 @command{bison} with @option{--yacc}), @code{api.header.include} is
6107 @emph{not} predefined when the output file is @file{y.tab.c}. Define it to
6108 avoid the duplication.
6110 @item Accepted Values:
6111 An argument for @code{#include}.
6113 @item Default Value:
6114 @samp{"@var{header-basename}"}, unless the header file is @file{y.tab.h},
6115 where @var{header-basename} is the name of the generated header, without
6116 directory part. For instance with @command{bison -d calc/parse.y},
6117 @code{api.header.include} defaults to @samp{"parse.h"}, not
6118 @samp{"calc/parse.h"}.
6121 Introduced in Bison 3.4. Defaults to @samp{"@var{basename}.h"} since Bison
6122 3.7, unless the header file is @file{y.tab.h}.
6125 @c api.header.include
6128 @c ================================================== api.location.file
6129 @deffn {Directive} {%define api.location.file} "@var{file}"
6130 @deffnx {Directive} {%define api.location.file} @code{none}
6133 @item Language(s): C++
6136 Define the name of the file in which Bison's default location and position
6137 types are generated. @xref{Exposing the Location Classes}.
6139 @item Accepted Values:
6142 If locations are enabled, generate the definition of the @code{position} and
6143 @code{location} classes in the header file if @code{%defines}, otherwise in
6144 the parser implementation.
6147 Generate the definition of the @code{position} and @code{location} classes
6148 in @var{file}. This file name can be relative (to where the parser file is
6149 output) or absolute.
6152 @item Default Value:
6153 Not applicable if locations are not enabled, or if a user location type is
6154 specified (see @code{api.location.type}). Otherwise, Bison's
6155 @code{location} is generated in @file{location.hh} (@pxref{C++ location}).
6158 Introduced in Bison 3.2.
6163 @c ================================================== api.location.file
6164 @deffn {Directive} {%define api.location.include} @{"@var{file}"@}
6165 @deffnx {Directive} {%define api.location.include} @{<@var{file}>@}
6168 @item Language(s): C++
6171 Specify how the generated file that defines the @code{position} and
6172 @code{location} classes is included. This makes sense when the
6173 @code{location} class is exposed to the rest of your application/library in
6174 another directory. @xref{Exposing the Location Classes}.
6176 @item Accepted Values: Argument for @code{#include}.
6178 @item Default Value:
6179 @samp{"@var{dir}/location.hh"} where @var{dir} is the directory part of the
6180 output. For instance @file{src/parse} if
6181 @option{--output=src/parse/parser.cc} was given.
6184 Introduced in Bison 3.2.
6190 @c ================================================== api.location.type
6191 @deffn {Directive} {%define api.location.type} @{@var{type}@}
6194 @item Language(s): C, C++, Java
6196 @item Purpose: Define the location type.
6197 @xref{User Defined Location Type}.
6199 @item Accepted Values: String
6201 @item Default Value: none
6204 Introduced in Bison 2.7 for C++ and Java, in Bison 3.4 for C. Was
6205 originally named @code{location_type} in Bison 2.5 and 2.6.
6210 @c ================================================== api.namespace
6211 @deffn Directive {%define api.namespace} @{@var{namespace}@}
6213 @item Languages(s): C++
6215 @item Purpose: Specify the namespace for the parser class.
6216 For example, if you specify:
6219 %define api.namespace @{foo::bar@}
6222 Bison uses @code{foo::bar} verbatim in references such as:
6225 foo::bar::parser::semantic_type
6228 However, to open a namespace, Bison removes any leading @code{::} and then
6229 splits on any remaining occurrences:
6232 namespace foo @{ namespace bar @{
6238 @item Accepted Values:
6239 Any absolute or relative C++ namespace reference without a trailing
6240 @code{"::"}. For example, @code{"foo"} or @code{"::foo::bar"}.
6242 @item Default Value:
6243 @code{yy}, unless you used the obsolete @samp{%name-prefix "@var{prefix}"}
6250 @c ================================================== api.parser.class
6251 @deffn Directive {%define api.parser.class} @{@var{name}@}
6257 The name of the parser class.
6259 @item Accepted Values:
6260 Any valid identifier.
6262 @item Default Value:
6263 In C++, @code{parser}. In Java, @code{YYParser} or
6264 @code{@var{api.prefix}Parser} (@pxref{Java Bison Interface}).
6267 Introduced in Bison 3.3 to replace @code{parser_class_name}.
6273 @c ================================================== api.prefix
6274 @deffn {Directive} {%define api.prefix} @{@var{prefix}@}
6277 @item Language(s): All
6279 @item Purpose: Rename exported symbols.
6280 @xref{Multiple Parsers}.
6282 @item Accepted Values: String
6284 @item Default Value: @code{YY} for Java, @code{yy} otherwise.
6286 @item History: introduced in Bison 2.6
6291 @c ================================================== api.pure
6292 @deffn Directive {%define api.pure} @var{purity}
6295 @item Language(s): C
6297 @item Purpose: Request a pure (reentrant) parser program.
6300 @item Accepted Values: @code{true}, @code{false}, @code{full}
6302 The value may be omitted: this is equivalent to specifying @code{true}, as is
6303 the case for Boolean values.
6305 When @code{%define api.pure full} is used, the parser is made reentrant. This
6306 changes the signature for @code{yylex} (@pxref{Pure Calling}), and also that of
6307 @code{yyerror} when the tracking of locations has been activated, as shown
6310 The @code{true} value is very similar to the @code{full} value, the only
6311 difference is in the signature of @code{yyerror} on Yacc parsers without
6312 @code{%parse-param}, for historical reasons.
6314 I.e., if @samp{%locations %define api.pure} is passed then the prototypes for
6318 void yyerror (char const *msg); // Yacc parsers.
6319 void yyerror (YYLTYPE *locp, char const *msg); // GLR parsers.
6322 But if @samp{%locations %define api.pure %parse-param @{int *nastiness@}} is
6323 used, then both parsers have the same signature:
6326 void yyerror (YYLTYPE *llocp, int *nastiness, char const *msg);
6329 (@pxref{Error Reporting Function})
6331 @item Default Value: @code{false}
6334 the @code{full} value was introduced in Bison 2.7
6341 @c ================================================== api.push-pull
6342 @deffn Directive {%define api.push-pull} @var{kind}
6345 @item Language(s): C (deterministic parsers only), Java
6347 @item Purpose: Request a pull parser, a push parser, or both.
6350 @item Accepted Values: @code{pull}, @code{push}, @code{both}
6352 @item Default Value: @code{pull}
6359 @c ================================================== api.symbol.prefix
6360 @deffn Directive {%define api.symbol.prefix} @{@var{prefix}@}
6363 @item Languages(s): all
6366 Add a prefix to the name of the symbol kinds. For instance
6369 %define api.symbol.prefix @{S_@}
6370 %token FILE for ERROR
6372 start: FILE for ERROR;
6376 generates this definition in C:
6380 enum yysymbol_kind_t
6382 S_YYEMPTY = -2, /* No symbol. */
6383 S_YYEOF = 0, /* $end */
6384 S_YYERROR = 1, /* error */
6385 S_YYUNDEF = 2, /* $undefined */
6386 S_FILE = 3, /* FILE */
6387 S_for = 4, /* for */
6388 S_ERROR = 5, /* ERROR */
6389 S_YYACCEPT = 6, /* $accept */
6390 S_start = 7 /* start */
6394 @item Accepted Values:
6395 Any non empty string. Must be a valid identifier in the target language
6396 (typically a non empty sequence of letters, underscores, and ---not at the
6397 beginning--- digits).
6399 The empty prefix is invalid:
6402 in C it would create collision with the @code{YYERROR} macro, and
6403 potentially token kind definitions and symbol kind definitions would
6406 unnamed symbols (such as @samp{'+'}) have a name which starts with a digit;
6408 even in languages with scoped enumerations such as Java, an empty prefix is
6409 dangerous: symbol names may collide with the target language keywords, or
6410 with other members of the @code{SymbolKind} class.
6414 @item Default Value:
6415 @code{YYSYMBOL_} in C. @code{S_} in C++, D and Java.
6417 introduced in Bison 3.6.
6420 @c api.symbol.prefix
6423 @c ================================================== api.token.constructor
6424 @deffn Directive {%define api.token.constructor}
6431 When variant-based semantic values are enabled (@pxref{C++ Variants}),
6432 request that symbols be handled as a whole (type, value, and possibly
6433 location) in the scanner. @xref{Complete Symbols}, for details.
6435 @item Accepted Values:
6438 @item Default Value:
6441 introduced in Bison 3.0.
6444 @c api.token.constructor
6447 @c ================================================== api.token.prefix
6448 @anchor{api-token-prefix}
6449 @deffn Directive {%define api.token.prefix} @{@var{prefix}@}
6451 @item Languages(s): all
6454 Add a prefix to the token names when generating their definition in the
6455 target language. For instance
6458 %define api.token.prefix @{TOK_@}
6459 %token FILE for ERROR
6461 start: FILE for ERROR;
6465 generates the definition of the symbols @code{TOK_FILE}, @code{TOK_for}, and
6466 @code{TOK_ERROR} in the generated source files. In particular, the scanner
6467 must use these prefixed token names, while the grammar itself may still use
6468 the short names (as in the sample rule given above). The generated
6469 informational files (@file{*.output}, @file{*.xml}, @file{*.gv}) are not
6470 modified by this prefix.
6472 Bison also prefixes the generated member names of the semantic value union.
6473 @xref{Type Generation}, for more
6476 See @ref{Calc++ Parser} and @ref{Calc++ Scanner}, for a complete example.
6478 @item Accepted Values:
6479 Any string. Must be a valid identifier prefix in the target language
6480 (typically, a possibly empty sequence of letters, underscores, and ---not at
6481 the beginning--- digits).
6483 @item Default Value:
6486 introduced in Bison 3.0.
6492 @c ================================================== api.token.raw
6493 @deffn Directive {%define api.token.raw}
6500 The output files normally define the enumeration of the @emph{token kinds}
6501 with Yacc-compatible token codes: sequential numbers starting at 257 except
6502 for single character tokens which stand for themselves (e.g., in ASCII,
6503 @samp{'a'} is numbered 65). The parser however uses @emph{symbol kinds}
6504 which are assigned numbers sequentially starting at 0. Therefore each time
6505 the scanner returns an (external) token kind, it must be mapped to the
6506 (internal) symbol kind.
6508 When @code{api.token.raw} is set, the code of the token kinds are forced to
6509 coincide with the symbol kind. This saves one table lookup per token to map
6510 them from the token kind to the symbol kind, and also saves the generation
6511 of the mapping table. The gain is typically moderate, but in extreme cases
6512 (very simple user actions), a 10% improvement can be observed.
6514 When @code{api.token.raw} is set, the grammar cannot use character literals
6515 (such as @samp{'a'}).
6517 @item Accepted Values: Boolean.
6519 @item Default Value:
6522 introduced in Bison 3.5. Was initially introduced in Bison 1.25 as
6523 @samp{%raw}, but never worked and was removed in Bison 1.29.
6529 @c ================================================== api.value.automove
6530 @deffn Directive {%define api.value.automove}
6537 Let occurrences of semantic values of the right-hand sides of a rule be
6538 implicitly turned in rvalues. When enabled, a grammar such as:
6542 "number" @{ $$ = make_number ($1); @}
6543 | exp "+" exp @{ $$ = make_binary (add, $1, $3); @}
6544 | "(" exp ")" @{ $$ = $2; @}
6548 is actually compiled as if you had written:
6552 "number" @{ $$ = make_number (std::move ($1)); @}
6553 | exp "+" exp @{ $$ = make_binary (add,
6556 | "(" exp ")" @{ $$ = std::move ($2); @}
6559 Using a value several times with automove enabled is typically an error.
6560 For instance, instead of:
6563 exp: "twice" exp @{ $$ = make_binary (add, $2, $2); @}
6570 exp: "twice" exp @{ auto v = $2; $$ = make_binary (add, v, v); @}
6574 It is tempting to use @code{std::move} on one of the @code{v}, but the
6575 argument evaluation order in C++ is unspecified.
6577 @item Accepted Values:
6580 @item Default Value:
6583 introduced in Bison 3.2
6586 @c api.value.automove
6589 @c ================================================== api.value.type
6590 @deffn Directive {%define api.value.type} @var{support}
6591 @deffnx Directive {%define api.value.type} @{@var{type}@}
6597 The type for semantic values.
6599 @item Accepted Values:
6602 This grammar has no semantic value at all. This is not properly supported
6604 @item @samp{union-directive} (C, C++)
6605 The type is defined thanks to the @code{%union} directive. You don't have
6606 to define @code{api.value.type} in that case, using @code{%union} suffices.
6610 %define api.value.type union-directive
6616 %token <ival> INT "integer"
6617 %token <sval> STR "string"
6620 @item @samp{union} (C, C++)
6621 The symbols are defined with type names, from which Bison will generate a
6622 @code{union}. For instance:
6624 %define api.value.type union
6625 %token <int> INT "integer"
6626 %token <char *> STR "string"
6628 Most C++ objects cannot be stored in a @code{union}, use @samp{variant}
6631 @item @samp{variant} (C++)
6632 This is similar to @code{union}, but special storage techniques are used to
6633 allow any kind of C++ object to be used. For instance:
6635 %define api.value.type variant
6636 %token <int> INT "integer"
6637 %token <std::string> STR "string"
6639 @xref{C++ Variants}.
6641 @item @samp{@{@var{type}@}}
6642 Use this @var{type} as semantic value.
6659 %define api.value.type @{struct my_value@}
6660 %token <u.ival> INT "integer"
6661 %token <u.sval> STR "string"
6665 @item Default Value:
6668 @code{union-directive} if @code{%union} is used, otherwise @dots{}
6670 @code{int} if type tags are used (i.e., @samp{%token <@var{type}>@dots{}} or
6671 @samp{%nterm <@var{type}>@dots{}} is used), otherwise @dots{}
6677 introduced in Bison 3.0. Was introduced for Java only in 2.3b as
6684 @c ================================================== api.value.union.name
6685 @deffn Directive {%define api.value.union.name} @var{name}
6691 The tag of the generated @code{union} (@emph{not} the name of the
6692 @code{typedef}). This variable is set to @code{@var{id}} when @samp{%union
6693 @var{id}} is used. There is no clear reason to give this union a name.
6695 @item Accepted Values:
6696 Any valid identifier.
6698 @item Default Value:
6702 Introduced in Bison 3.0.3.
6708 @c ================================================== lr.default-reduction
6710 @deffn Directive {%define lr.default-reduction} @var{when}
6713 @item Language(s): all
6715 @item Purpose: Specify the kind of states that are permitted to
6716 contain default reductions. @xref{Default Reductions}.
6718 @item Accepted Values: @code{most}, @code{consistent}, @code{accepting}
6719 @item Default Value:
6721 @item @code{accepting} if @code{lr.type} is @code{canonical-lr}.
6722 @item @code{most} otherwise.
6725 introduced as @code{lr.default-reductions} in 2.5, renamed as
6726 @code{lr.default-reduction} in 3.0.
6731 @c ============================================ lr.keep-unreachable-state
6733 @deffn Directive {%define lr.keep-unreachable-state}
6736 @item Language(s): all
6737 @item Purpose: Request that Bison allow unreachable parser states to
6738 remain in the parser tables. @xref{Unreachable States}.
6739 @item Accepted Values: Boolean
6740 @item Default Value: @code{false}
6742 introduced as @code{lr.keep_unreachable_states} in 2.3b, renamed as
6743 @code{lr.keep-unreachable-states} in 2.5, and as
6744 @code{lr.keep-unreachable-state} in 3.0.
6747 @c lr.keep-unreachable-state
6750 @c ================================================== lr.type
6752 @deffn Directive {%define lr.type} @var{type}
6755 @item Language(s): all
6757 @item Purpose: Specify the type of parser tables within the
6758 LR(1) family. @xref{LR Table Construction}.
6760 @item Accepted Values: @code{lalr}, @code{ielr}, @code{canonical-lr}
6762 @item Default Value: @code{lalr}
6767 @c ================================================== namespace
6768 @deffn Directive %define namespace @{@var{namespace}@}
6769 Obsoleted by @code{api.namespace}
6774 @c ================================================== parse.assert
6775 @deffn Directive {%define parse.assert}
6778 @item Languages(s): C, C++
6780 @item Purpose: Issue runtime assertions to catch invalid uses.
6781 In C, some important invariants in the implementation of the parser are
6782 checked when this option is enabled.
6784 In C++, when variants are used (@pxref{C++ Variants}), symbols must be
6785 constructed and destroyed properly. This option checks these constraints
6786 using runtime type information (RTTI). Therefore the generated code cannot
6787 be compiled with RTTI disabled (via compiler options such as
6788 @option{-fno-rtti}).
6790 @item Accepted Values: Boolean
6792 @item Default Value: @code{false}
6798 @c ================================================== parse.error
6799 @deffn Directive {%define parse.error} @var{verbosity}
6804 Control the generation of syntax error messages. @xref{Error Reporting}.
6805 @item Accepted Values:
6808 Error messages passed to @code{yyerror} are simply @w{@code{"syntax
6811 @item @code{detailed}
6812 Error messages report the unexpected token, and possibly the expected ones.
6813 However, this report can often be incorrect when LAC is not enabled
6814 (@pxref{LAC}). Token name internationalization is supported.
6816 @item @code{verbose}
6817 Similar (but inferior) to @code{detailed}.
6819 Error messages report the unexpected token, and possibly the expected ones.
6820 However, this report can often be incorrect when LAC is not enabled
6823 Does not support token internationalization. Using non-ASCII characters in
6824 token aliases is not portable.
6827 The user is in charge of generating the syntax error message by defining the
6828 @code{yyreport_syntax_error} function. @xref{Syntax Error Reporting
6832 @item Default Value:
6836 introduced in 3.0 with support for @code{simple} and @code{verbose}. Values
6837 @code{custom} and @code{detailed} were introduced in 3.6.
6843 @c ================================================== parse.lac
6844 @deffn Directive {%define parse.lac} @var{when}
6847 @item Languages(s): C (deterministic parsers only)
6849 @item Purpose: Enable LAC (lookahead correction) to improve
6850 syntax error handling. @xref{LAC}.
6851 @item Accepted Values: @code{none}, @code{full}
6852 @item Default Value: @code{none}
6858 @c ================================================== parse.trace
6859 @deffn Directive {%define parse.trace}
6862 @item Languages(s): C, C++, Java
6864 @item Purpose: Require parser instrumentation for tracing.
6867 In C/C++, define the macro @code{YYDEBUG} (or @code{@var{prefix}DEBUG} with
6868 @samp{%define api.prefix @{@var{prefix}@}}), see @ref{Multiple Parsers}) to
6869 1 in the parser implementation file if it is not already defined, so that
6870 the debugging facilities are compiled.
6872 @item Accepted Values: Boolean
6874 @item Default Value: @code{false}
6880 @c ================================================== parser_class_name
6881 @deffn Directive %define parser_class_name @{@var{name}@}
6882 Obsoleted by @code{api.parser.class}
6884 @c parser_class_name
6892 @subsection %code Summary
6896 The @code{%code} directive inserts code verbatim into the output
6897 parser source at any of a predefined set of locations. It thus serves
6898 as a flexible and user-friendly alternative to the traditional Yacc
6899 prologue, @code{%@{@var{code}%@}}. This section summarizes the
6900 functionality of @code{%code} for the various target languages
6901 supported by Bison. For a detailed discussion of how to use
6902 @code{%code} in place of @code{%@{@var{code}%@}} for C/C++ and why it
6903 is advantageous to do so, @pxref{Prologue Alternatives}.
6905 @deffn {Directive} %code @{@var{code}@}
6906 This is the unqualified form of the @code{%code} directive. It
6907 inserts @var{code} verbatim at a language-dependent default location
6908 in the parser implementation.
6910 For C/C++, the default location is the parser implementation file
6911 after the usual contents of the parser header file. Thus, the
6912 unqualified form replaces @code{%@{@var{code}%@}} for most purposes.
6914 For Java, the default location is inside the parser class.
6917 @deffn {Directive} %code @var{qualifier} @{@var{code}@}
6918 This is the qualified form of the @code{%code} directive.
6919 @var{qualifier} identifies the purpose of @var{code} and thus the
6920 location(s) where Bison should insert it. That is, if you need to
6921 specify location-sensitive @var{code} that does not belong at the
6922 default location selected by the unqualified @code{%code} form, use
6926 For any particular qualifier or for the unqualified form, if there are
6927 multiple occurrences of the @code{%code} directive, Bison concatenates
6928 the specified code in the order in which it appears in the grammar
6931 Not all qualifiers are accepted for all target languages. Unaccepted
6932 qualifiers produce an error. Some of the accepted qualifiers are:
6936 @findex %code requires
6939 @item Language(s): C, C++
6941 @item Purpose: This is the best place to write dependency code required for
6942 @code{YYSTYPE} and @code{YYLTYPE}. In other words, it's the best place to
6943 define types referenced in @code{%union} directives. If you use
6944 @code{#define} to override Bison's default @code{YYSTYPE} and @code{YYLTYPE}
6945 definitions, then it is also the best place. However you should rather
6946 @code{%define} @code{api.value.type} and @code{api.location.type}.
6948 @item Location(s): The parser header file and the parser implementation file
6949 before the Bison-generated @code{YYSTYPE} and @code{YYLTYPE}
6954 @findex %code provides
6957 @item Language(s): C, C++
6959 @item Purpose: This is the best place to write additional definitions and
6960 declarations that should be provided to other modules.
6962 @item Location(s): The parser header file and the parser implementation
6963 file after the Bison-generated @code{YYSTYPE}, @code{YYLTYPE}, and
6971 @item Language(s): C, C++
6973 @item Purpose: The unqualified @code{%code} or @code{%code requires}
6974 should usually be more appropriate than @code{%code top}. However,
6975 occasionally it is necessary to insert code much nearer the top of the
6976 parser implementation file. For example:
6985 @item Location(s): Near the top of the parser implementation file.
6989 @findex %code imports
6992 @item Language(s): Java
6994 @item Purpose: This is the best place to write Java import directives.
6996 @item Location(s): The parser Java file after any Java package directive and
6997 before any class definitions.
7001 Though we say the insertion locations are language-dependent, they are
7002 technically skeleton-dependent. Writers of non-standard skeletons
7003 however should choose their locations consistently with the behavior
7004 of the standard Bison skeletons.
7007 @node Multiple Parsers
7008 @section Multiple Parsers in the Same Program
7010 Most programs that use Bison parse only one language and therefore contain
7011 only one Bison parser. But what if you want to parse more than one language
7012 with the same program? Then you need to avoid name conflicts between
7013 different definitions of functions and variables such as @code{yyparse},
7014 @code{yylval}. To use different parsers from the same compilation unit, you
7015 also need to avoid conflicts on types and macros (e.g., @code{YYSTYPE})
7016 exported in the generated header.
7018 The easy way to do this is to define the @code{%define} variable
7019 @code{api.prefix}. With different @code{api.prefix}s it is guaranteed that
7020 headers do not conflict when included together, and that compiled objects
7021 can be linked together too. Specifying @samp{%define api.prefix
7022 @{@var{prefix}@}} (or passing the option @option{-Dapi.prefix=@{@var{prefix}@}}, see
7023 @ref{Invocation}) renames the interface functions and
7024 variables of the Bison parser to start with @var{prefix} instead of
7025 @samp{yy}, and all the macros to start by @var{PREFIX} (i.e., @var{prefix}
7026 upper-cased) instead of @samp{YY}.
7028 The renamed symbols include @code{yyparse}, @code{yylex}, @code{yyerror},
7029 @code{yynerrs}, @code{yylval}, @code{yylloc}, @code{yychar} and
7030 @code{yydebug}. If you use a push parser, @code{yypush_parse},
7031 @code{yypull_parse}, @code{yypstate}, @code{yypstate_new} and
7032 @code{yypstate_delete} will also be renamed. The renamed macros include
7033 @code{YYSTYPE}, @code{YYLTYPE}, and @code{YYDEBUG}, which is treated
7034 specifically --- more about this below.
7036 For example, if you use @samp{%define api.prefix @{c@}}, the names become
7037 @code{cparse}, @code{clex}, @dots{}, @code{CSTYPE}, @code{CLTYPE}, and so
7040 Users of Flex must update the signature of the generated @code{yylex}
7041 function. Since the Flex scanner usually includes the generated header of
7042 the parser (to get the definitions of the tokens, etc.), the most convenient
7043 way is to insert the declaration of @code{yylex} in the @code{provides}
7047 %define api.prefix @{c@}
7048 // Emitted in the header file, after the definition of YYSTYPE.
7051 // Tell Flex the expected prototype of yylex.
7053 int clex (CSTYPE *yylval, CLTYPE *yylloc)
7055 // Declare the scanner.
7062 The @code{%define} variable @code{api.prefix} works in two different ways.
7063 In the implementation file, it works by adding macro definitions to the
7064 beginning of the parser implementation file, defining @code{yyparse} as
7065 @code{@var{prefix}parse}, and so on:
7068 #define YYSTYPE CTYPE
7069 #define yyparse cparse
7070 #define yylval clval
7076 This effectively substitutes one name for the other in the entire parser
7077 implementation file, thus the ``original'' names (@code{yylex},
7078 @code{YYSTYPE}, @dots{}) are also usable in the parser implementation file.
7080 However, in the parser header file, the symbols are defined renamed, for
7084 extern CSTYPE clval;
7088 The macro @code{YYDEBUG} is commonly used to enable the tracing support in
7089 parsers. To comply with this tradition, when @code{api.prefix} is used,
7090 @code{YYDEBUG} (not renamed) is used as a default value:
7095 # if defined YYDEBUG
7112 Prior to Bison 2.6, a feature similar to @code{api.prefix} was provided by
7113 the obsolete directive @code{%name-prefix} (@pxref{Table of Symbols}) and
7114 the option @option{--name-prefix} (@pxref{Output Files}).
7117 @chapter Parser C-Language Interface
7118 @cindex C-language interface
7121 The Bison parser is actually a C function named @code{yyparse}. Here we
7122 describe the interface conventions of @code{yyparse} and the other
7123 functions that it needs to use.
7125 Keep in mind that the parser uses many C identifiers starting with
7126 @samp{yy} and @samp{YY} for internal purposes. If you use such an
7127 identifier (aside from those in this manual) in an action or in epilogue
7128 in the grammar file, you are likely to run into trouble.
7131 * Parser Function:: How to call @code{yyparse} and what it returns.
7132 * Push Parser Interface:: How to create, use, and destroy push parsers.
7133 * Lexical:: You must supply a function @code{yylex}
7135 * Error Reporting:: Passing error messages to the user.
7136 * Action Features:: Special features for use in actions.
7137 * Internationalization:: How to let the parser speak in the user's
7141 @node Parser Function
7142 @section The Parser Function @code{yyparse}
7145 You call the function @code{yyparse} to cause parsing to occur. This
7146 function reads tokens, executes actions, and ultimately returns when it
7147 encounters end-of-input or an unrecoverable syntax error. You can also
7148 write an action which directs @code{yyparse} to return immediately
7149 without reading further.
7152 @deftypefun int yyparse (@code{void})
7153 The value returned by @code{yyparse} is 0 if parsing was successful (return
7154 is due to end-of-input).
7156 The value is 1 if parsing failed because of invalid input, i.e., input
7157 that contains a syntax error or that causes @code{YYABORT} to be
7160 The value is 2 if parsing failed due to memory exhaustion.
7163 In an action, you can cause immediate return from @code{yyparse} by using
7168 Return immediately with value 0 (to report success).
7173 Return immediately with value 1 (to report failure).
7176 If you use a reentrant parser, you can optionally pass additional
7177 parameter information to it in a reentrant way. To do so, use the
7178 declaration @code{%parse-param}:
7180 @deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
7181 @findex %parse-param
7182 Declare that one or more
7183 @var{argument-declaration} are additional @code{yyparse} arguments.
7184 The @var{argument-declaration} is used when declaring
7185 functions or prototypes. The last identifier in
7186 @var{argument-declaration} must be the argument name.
7189 Here's an example. Write this in the parser:
7192 %parse-param @{int *nastiness@} @{int *randomness@}
7196 Then call the parser like this:
7200 int nastiness, randomness;
7201 @dots{} /* @r{Store proper data in @code{nastiness} and @code{randomness}.} */
7202 value = yyparse (&nastiness, &randomness);
7208 In the grammar actions, use expressions like this to refer to the data:
7211 exp: @dots{} @{ @dots{}; *randomness += 1; @dots{} @}
7215 Using the following:
7217 %parse-param @{int *randomness@}
7220 Results in these signatures:
7222 void yyerror (int *randomness, const char *msg);
7223 int yyparse (int *randomness);
7227 Or, if both @code{%define api.pure full} (or just @code{%define api.pure})
7228 and @code{%locations} are used:
7231 void yyerror (YYLTYPE *llocp, int *randomness, const char *msg);
7232 int yyparse (int *randomness);
7235 @node Push Parser Interface
7236 @section Push Parser Interface
7238 @findex yypstate_new
7239 You call the function @code{yypstate_new} to create a new parser instance.
7240 This function is available if either the @samp{%define api.push-pull push}
7241 or @samp{%define api.push-pull both} declaration is used. @xref{Push Decl}.
7243 @deftypefun {yypstate*} yypstate_new (@code{void})
7244 @anchor{yypstate_new}
7245 Return a valid parser instance if there is memory available, 0 otherwise.
7246 In impure mode, it will also return 0 if a parser instance is currently
7250 @findex yypstate_delete
7251 You call the function @code{yypstate_delete} to delete a parser instance.
7252 function is available if either the @samp{%define api.push-pull push} or
7253 @samp{%define api.push-pull both} declaration is used.
7256 @deftypefun void yypstate_delete (@code{yypstate *}@var{yyps})
7257 @anchor{yypstate_delete}
7258 Reclaim the memory associated with a parser instance. After this call, you
7259 should no longer attempt to use the parser instance.
7262 @findex yypush_parse
7263 You call the function @code{yypush_parse} to parse a single token. This
7264 function is available if either the @samp{%define api.push-pull push} or
7265 @samp{%define api.push-pull both} declaration is used. @xref{Push Decl}.
7267 @deftypefun int yypush_parse (@code{yypstate *}@var{yyps})
7268 @anchor{yypush_parse}
7269 The value returned by @code{yypush_parse} is the same as for @code{yyparse}
7270 with the following exception: it returns @code{YYPUSH_MORE} if more input is
7271 required to finish parsing the grammar.
7273 After @code{yypush_parse} returned, the instance may be consulted. For
7274 instance check @code{yynerrs} to see whether there were (possibly recovered)
7277 After @code{yypush_parse} returns a status other than @code{YYPUSH_MORE},
7278 the parser instance @code{yyps} may be reused for a new parse.
7281 The fact that the parser state is reusable even after an error simplifies
7282 reuse. For example, a calculator application which parses each input line
7283 as an expression can just keep reusing the same @code{yyps} even if an input
7286 You call the function @code{yypull_parse} to parse the rest of the input
7287 stream. This function is available if the @samp{%define api.push-pull both}
7288 declaration is used. @xref{Push Decl}.
7290 @deftypefun int yypull_parse (@code{yypstate *}@var{yyps})
7291 @anchor{yypull_parse}
7292 The value returned by @code{yypull_parse} is the same as for @code{yyparse}.
7294 The parser instance @code{yyps} may be reused for new parses.
7297 @deftypefun int yypstate_expected_tokens (@code{const yypstate *}yyps, @code{yysymbol_kind_t} @var{argv}@code{[]}, @code{int} @var{argc})
7298 Fill @var{argv} with the expected tokens, which never includes
7299 @code{YYSYMBOL_YYEMPTY}, @code{YYSYMBOL_YYerror}, or
7300 @code{YYSYMBOL_YYUNDEF}.
7302 Never put more than @var{argc} elements into @var{argv}, and on success
7303 return the number of tokens stored in @var{argv}. If there are more
7304 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
7305 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
7306 to @code{YYSYMBOL_YYEMPTY}.
7308 When LAC is enabled, may return a negative number on errors,
7309 such as @code{YYENOMEM} on memory exhaustion.
7311 If @var{argv} is null, return the size needed to store all the possible
7312 values, which is always less than @code{YYNTOKENS}.
7317 @section The Lexical Analyzer Function @code{yylex}
7319 @cindex lexical analyzer
7321 The @dfn{lexical analyzer} function, @code{yylex}, recognizes tokens from
7322 the input stream and returns them to the parser. Bison does not create
7323 this function automatically; you must write it so that @code{yyparse} can
7324 call it. The function is sometimes referred to as a lexical scanner.
7326 In simple programs, @code{yylex} is often defined at the end of the Bison
7327 grammar file. If @code{yylex} is defined in a separate source file, you
7328 need to arrange for the token-kind definitions to be available there. To do
7329 this, use the @option{-d} option when you run Bison, so that it will write
7330 these definitions into the separate parser header file,
7331 @file{@var{name}.tab.h}, which you can include in the other source files
7332 that need it. @xref{Invocation}.
7335 * Calling Convention:: How @code{yyparse} calls @code{yylex}.
7336 * Special Tokens:: Signaling end-of-file and errors to the parser.
7337 * Tokens from Literals:: Finding token kinds from string aliases.
7338 * Token Values:: How @code{yylex} must return the semantic value
7339 of the token it has read.
7340 * Token Locations:: How @code{yylex} must return the text location
7341 (line number, etc.) of the token, if the
7343 * Pure Calling:: How the calling convention differs in a pure parser
7344 (@pxref{Pure Decl}).
7347 @node Calling Convention
7348 @subsection Calling Convention for @code{yylex}
7350 The value that @code{yylex} returns must be the positive numeric code for
7351 the kind of token it has just found; a zero or negative value signifies
7354 When a token kind is referred to in the grammar rules by a name, that name
7355 in the parser implementation file becomes an enumerator of the enum
7356 @code{yytoken_kind_t} whose definition is the proper numeric code for that
7357 token kind. So @code{yylex} should use the name to indicate that type.
7360 When a token is referred to in the grammar rules by a character literal, the
7361 numeric code for that character is also the code for the token kind. So
7362 @code{yylex} can simply return that character code, possibly converted to
7363 @code{unsigned char} to avoid sign-extension. The null character must not
7364 be used this way, because its code is zero and that signifies end-of-input.
7366 Here is an example showing these things:
7373 if (c == EOF) /* Detect end-of-input. */
7376 else if (c == '+' || c == '-')
7377 return c; /* Assume token kind for '+' is '+'. */
7380 return INT; /* Return the kind of the token. */
7386 This interface has been designed so that the output from the @code{lex}
7387 utility can be used without change as the definition of @code{yylex}.
7390 @node Special Tokens
7391 @subsection Special Tokens
7393 In addition to the user defined tokens, Bison generates a few special tokens
7394 that @code{yylex} may return.
7396 The @code{YYEOF} token denotes the end of file, and signals to the parser
7397 that there is nothing left afterwards. @xref{Calling Convention}, for an
7400 Returning @code{YYUNDEF} tells the parser that some lexical error was found.
7401 It will emit an error message about an ``invalid token'', and enter
7402 error-recovery (@pxref{Error Recovery}). Returning an unknown token kind
7403 results in the exact same behavior.
7405 Returning @code{YYerror} requires the parser to enter error-recovery
7406 @emph{without} emitting an error message. This way the lexical analyzer can
7407 produce an accurate error messages about the invalid input (something the
7408 parser cannot do), and yet benefit from the error-recovery features of the
7419 case '0': case '1': case '2': case '3': case '4':
7420 case '5': case '6': case '7': case '8': case '9':
7427 yyerror ("syntax error: invalid character: %c", c);
7433 @node Tokens from Literals
7434 @subsection Finding Tokens by String Literals
7436 If the grammar uses literal string tokens, there are two ways that
7437 @code{yylex} can determine the token kind codes for them:
7441 If the grammar defines symbolic token names as aliases for the literal
7442 string tokens, @code{yylex} can use these symbolic names like all others.
7443 In this case, the use of the literal string tokens in the grammar file has
7444 no effect on @code{yylex}.
7446 This is the preferred approach.
7449 @code{yylex} can search for the multicharacter token in the @code{yytname}
7450 table. This method is discouraged: the primary purpose of string aliases is
7451 forging good error messages, not describing the spelling of keywords. In
7452 addition, looking for the token kind at runtime incurs a (small but
7455 The @code{yytname} table is generated only if you use the
7456 @code{%token-table} declaration. @xref{Decl Summary}.
7461 @subsection Semantic Values of Tokens
7464 In an ordinary (nonreentrant) parser, the semantic value of the token must
7465 be stored into the global variable @code{yylval}. When you are using just
7466 one data type for semantic values, @code{yylval} has that type. Thus, if
7467 the type is @code{int} (the default), you might write this in @code{yylex}:
7472 yylval = value; /* Put value onto Bison stack. */
7473 return INT; /* Return the kind of the token. */
7478 When you are using multiple data types, @code{yylval}'s type is a union made
7479 from the @code{%union} declaration (@pxref{Union Decl}). So when you store
7480 a token's value, you must use the proper member of the union. If the
7481 @code{%union} declaration looks like this:
7494 then the code in @code{yylex} might look like this:
7499 yylval.intval = value; /* Put value onto Bison stack. */
7500 return INT; /* Return the kind of the token. */
7505 @node Token Locations
7506 @subsection Textual Locations of Tokens
7509 If you are using the @samp{@@@var{n}}-feature (@pxref{Tracking Locations})
7510 in actions to keep track of the textual locations of tokens and groupings,
7511 then you must provide this information in @code{yylex}. The function
7512 @code{yyparse} expects to find the textual location of a token just parsed
7513 in the global variable @code{yylloc}. So @code{yylex} must store the proper
7514 data in that variable.
7516 By default, the value of @code{yylloc} is a structure and you need only
7517 initialize the members that are going to be used by the actions. The
7518 four members are called @code{first_line}, @code{first_column},
7519 @code{last_line} and @code{last_column}. Note that the use of this
7520 feature makes the parser noticeably slower.
7523 The data type of @code{yylloc} has the name @code{YYLTYPE}.
7526 @subsection Calling Conventions for Pure Parsers
7528 When you use the Bison declaration @code{%define api.pure full} to request a
7529 pure, reentrant parser, the global communication variables @code{yylval} and
7530 @code{yylloc} cannot be used. (@xref{Pure Decl}.) In such parsers the two
7531 global variables are replaced by pointers passed as arguments to
7532 @code{yylex}. You must declare them as shown here, and pass the information
7533 back by storing it through those pointers.
7537 yylex (YYSTYPE *lvalp, YYLTYPE *llocp)
7540 *lvalp = value; /* Put value onto Bison stack. */
7541 return INT; /* Return the kind of the token. */
7546 If the grammar file does not use the @samp{@@} constructs to refer to
7547 textual locations, then the type @code{YYLTYPE} will not be defined. In
7548 this case, omit the second argument; @code{yylex} will be called with
7551 If you wish to pass additional arguments to @code{yylex}, use
7552 @code{%lex-param} just like @code{%parse-param} (@pxref{Parser
7553 Function}). To pass additional arguments to both @code{yylex} and
7554 @code{yyparse}, use @code{%param}.
7556 @deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
7558 Specify that @var{argument-declaration} are additional @code{yylex} argument
7559 declarations. You may pass one or more such declarations, which is
7560 equivalent to repeating @code{%lex-param}.
7563 @deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
7565 Specify that @var{argument-declaration} are additional
7566 @code{yylex}/@code{yyparse} argument declaration. This is equivalent to
7567 @samp{%lex-param @{@var{argument-declaration}@} @dots{} %parse-param
7568 @{@var{argument-declaration}@} @dots{}}. You may pass one or more
7569 declarations, which is equivalent to repeating @code{%param}.
7576 %lex-param @{scanner_mode *mode@}
7577 %parse-param @{parser_mode *mode@}
7578 %param @{environment_type *env@}
7582 results in the following signatures:
7585 int yylex (scanner_mode *mode, environment_type *env);
7586 int yyparse (parser_mode *mode, environment_type *env);
7589 If @samp{%define api.pure full} is added:
7592 int yylex (YYSTYPE *lvalp, scanner_mode *mode, environment_type *env);
7593 int yyparse (parser_mode *mode, environment_type *env);
7597 and finally, if both @samp{%define api.pure full} and @code{%locations} are
7601 int yylex (YYSTYPE *lvalp, YYLTYPE *llocp,
7602 scanner_mode *mode, environment_type *env);
7603 int yyparse (parser_mode *mode, environment_type *env);
7607 @node Error Reporting
7608 @section Error Reporting
7610 During its execution the parser may have error messages to pass to the user,
7611 such as syntax error, or memory exhaustion. How this message is delivered
7612 to the user must be specified by the developer.
7615 * Error Reporting Function:: You must supply a @code{yyerror} function.
7616 * Syntax Error Reporting Function:: You can supply a @code{yyreport_syntax_error} function.
7619 @node Error Reporting Function
7620 @subsection The Error Reporting Function @code{yyerror}
7621 @cindex error reporting function
7624 @cindex syntax error
7626 The Bison parser detects a @dfn{syntax error} (or @dfn{parse error})
7627 whenever it reads a token which cannot satisfy any syntax rule. An
7628 action in the grammar can also explicitly proclaim an error, using the
7629 macro @code{YYERROR} (@pxref{Action Features}).
7631 The Bison parser expects to report the error by calling an error
7632 reporting function named @code{yyerror}, which you must supply. It is
7633 called by @code{yyparse} whenever a syntax error is found, and it
7634 receives one argument. For a syntax error, the string is normally
7635 @w{@code{"syntax error"}}.
7637 @findex %define parse.error detailed
7638 @findex %define parse.error verbose
7639 If you invoke @samp{%define parse.error detailed} (or @samp{custom}) in the
7640 Bison declarations section (@pxref{Bison Declarations}), then Bison provides
7641 a more verbose and specific error message string instead of just plain
7642 @w{@code{"syntax error"}}. However, that message sometimes contains
7643 incorrect information if LAC is not enabled (@pxref{LAC}).
7645 The parser can detect one other kind of error: memory exhaustion. This
7646 can happen when the input contains constructions that are very deeply
7647 nested. It isn't likely you will encounter this, since the Bison
7648 parser normally extends its stack automatically up to a very large limit. But
7649 if memory is exhausted, @code{yyparse} calls @code{yyerror} in the usual
7650 fashion, except that the argument string is @w{@code{"memory exhausted"}}.
7652 In some cases diagnostics like @w{@code{"syntax error"}} are
7653 translated automatically from English to some other language before
7654 they are passed to @code{yyerror}. @xref{Internationalization}.
7656 The following definition suffices in simple programs:
7661 yyerror (char const *s)
7665 fprintf (stderr, "%s\n", s);
7670 After @code{yyerror} returns to @code{yyparse}, the latter will attempt
7671 error recovery if you have written suitable error recovery grammar rules
7672 (@pxref{Error Recovery}). If recovery is impossible, @code{yyparse} will
7673 immediately return 1.
7675 Obviously, in location tracking pure parsers, @code{yyerror} should have
7676 an access to the current location. With @code{%define api.pure}, this is
7677 indeed the case for the GLR parsers, but not for the Yacc parser, for
7678 historical reasons, and this is the why @code{%define api.pure full} should be
7679 preferred over @code{%define api.pure}.
7681 When @code{%locations %define api.pure full} is used, @code{yyerror} has the
7682 following signature:
7685 void yyerror (YYLTYPE *locp, char const *msg);
7689 The prototypes are only indications of how the code produced by Bison
7690 uses @code{yyerror}. Bison-generated code always ignores the returned
7691 value, so @code{yyerror} can return any type, including @code{void}.
7692 Also, @code{yyerror} can be a variadic function; that is why the
7693 message is always passed last.
7695 Traditionally @code{yyerror} returns an @code{int} that is always
7696 ignored, but this is purely for historical reasons, and @code{void} is
7697 preferable since it more accurately describes the return type for
7701 The variable @code{yynerrs} contains the number of syntax errors
7702 reported so far. Normally this variable is global; but if you
7703 request a pure parser (@pxref{Pure Decl})
7704 then it is a local variable which only the actions can access.
7707 @node Syntax Error Reporting Function
7708 @subsection The Syntax Error Reporting Function @code{yyreport_syntax_error}
7710 @findex %define parse.error custom
7711 If you invoke @samp{%define parse.error custom} (@pxref{Bison
7712 Declarations}), then the parser no longer passes syntax error messages to
7713 @code{yyerror}, rather it delegates that task to the user by calling the
7714 @code{yyreport_syntax_error} function.
7716 The following functions and types are ``@code{static}'': they are defined in
7717 the implementation file (@file{*.c}) and available only from there. They
7718 are meant to be used from the grammar's epilogue.
7720 @deftypefun {static int} yyreport_syntax_error (@code{const yypcontext_t *}@var{ctx})
7721 Report a syntax error to the user. Return 0 on success, @code{YYENOMEM} on
7722 memory exhaustion. Whether it uses @code{yyerror} is up to the user.
7725 Use the following types and functions to build the error message.
7727 @deffn {Type} yypcontext_t
7728 An opaque type that captures the circumstances of the syntax error.
7731 @deffn {Type} yysymbol_kind_t
7732 An enum of all the grammar symbols, tokens and nonterminals. Its
7733 enumerators are forged from the symbol names:
7736 enum yysymbol_kind_t
7738 YYSYMBOL_YYEMPTY = -2, /* No symbol. */
7739 YYSYMBOL_YYEOF = 0, /* "end of file" */
7740 YYSYMBOL_YYerror = 1, /* error */
7741 YYSYMBOL_YYUNDEF = 2, /* "invalid token" */
7742 YYSYMBOL_PLUS = 3, /* "+" */
7743 YYSYMBOL_MINUS = 4, /* "-" */
7745 YYSYMBOL_VAR = 14, /* "variable" */
7746 YYSYMBOL_NEG = 15, /* NEG */
7747 YYSYMBOL_YYACCEPT = 16, /* $accept */
7748 YYSYMBOL_exp = 17, /* exp */
7749 YYSYMBOL_input = 18 /* input */
7751 typedef enum yysymbol_kind_t yysymbol_kind_t;
7755 @deftypefun {static yysymbol_kind_t} yypcontext_token (@code{const yypcontext_t *}@var{ctx})
7756 The ``unexpected'' token: the symbol kind of the lookahead token that caused
7757 the syntax error. Returns @code{YYSYMBOL_YYEMPTY} if there is no lookahead.
7760 @deftypefun {static YYLTYPE *} yypcontext_location (@code{const yypcontext_t *}@var{ctx})
7761 The location of the syntax error (that of the unexpected token).
7764 @deftypefun {static int} yypcontext_expected_tokens (@code{const yypcontext_t *}ctx, @code{yysymbol_kind_t} @var{argv}@code{[]}, @code{int} @var{argc})
7765 Fill @var{argv} with the expected tokens, which never includes
7766 @code{YYSYMBOL_YYEMPTY}, @code{YYSYMBOL_YYerror}, or
7767 @code{YYSYMBOL_YYUNDEF}.
7769 Never put more than @var{argc} elements into @var{argv}, and on success
7770 return the number of tokens stored in @var{argv}. If there are more
7771 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
7772 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
7773 to @code{YYSYMBOL_YYEMPTY}.
7775 When LAC is enabled, may return a negative number on errors,
7776 such as @code{YYENOMEM} on memory exhaustion.
7778 If @var{argv} is null, return the size needed to store all the possible
7779 values, which is always less than @code{YYNTOKENS}.
7782 @deftypefun {static const char *} yysymbol_name (@code{symbol_kind_t} @var{symbol})
7783 The name of the symbol whose kind is @var{symbol}, possibly translated.
7786 A custom syntax error function looks as follows. This implementation is
7787 inappropriate for internationalization, see the @file{c/bistromathic}
7788 example for a better alternative.
7792 yyreport_syntax_error (const yypcontext_t *ctx)
7795 YY_LOCATION_PRINT (stderr, *yypcontext_location (ctx));
7796 fprintf (stderr, ": syntax error");
7797 // Report the tokens expected at this point.
7799 enum @{ TOKENMAX = 5 @};
7800 yysymbol_kind_t expected[TOKENMAX];
7801 int n = yypcontext_expected_tokens (ctx, expected, TOKENMAX);
7803 // Forward errors to yyparse.
7806 for (int i = 0; i < n; ++i)
7807 fprintf (stderr, "%s %s",
7808 i == 0 ? ": expected" : " or", yysymbol_name (expected[i]));
7810 // Report the unexpected token.
7812 yysymbol_kind_t lookahead = yypcontext_token (ctx);
7813 if (lookahead != YYSYMBOL_YYEMPTY)
7814 fprintf (stderr, " before %s", yysymbol_name (lookahead));
7816 fprintf (stderr, "\n");
7821 You still must provide a @code{yyerror} function, used for instance to
7822 report memory exhaustion.
7824 @node Action Features
7825 @section Special Features for Use in Actions
7826 @cindex summary, action features
7827 @cindex action features summary
7829 Here is a table of Bison constructs, variables and macros that are useful in
7832 @deffn {Variable} $$
7833 Acts like a variable that contains the semantic value for the
7834 grouping made by the current rule. @xref{Actions}.
7837 @deffn {Variable} $@var{n}
7838 Acts like a variable that contains the semantic value for the
7839 @var{n}th component of the current rule. @xref{Actions}.
7842 @deffn {Variable} $<@var{typealt}>$
7843 Like @code{$$} but specifies alternative @var{typealt} in the union
7844 specified by the @code{%union} declaration. @xref{Action Types}.
7847 @deffn {Variable} $<@var{typealt}>@var{n}
7848 Like @code{$@var{n}} but specifies alternative @var{typealt} in the
7849 union specified by the @code{%union} declaration.
7850 @xref{Action Types}.
7853 @deffn {Macro} YYABORT @code{;}
7854 Return immediately from @code{yyparse}, indicating failure.
7855 @xref{Parser Function}.
7858 @deffn {Macro} YYACCEPT @code{;}
7859 Return immediately from @code{yyparse}, indicating success.
7860 @xref{Parser Function}.
7863 @deffn {Macro} YYBACKUP (@var{token}, @var{value})@code{;}
7865 Unshift a token. This macro is allowed only for rules that reduce
7866 a single value, and only when there is no lookahead token.
7867 It is also disallowed in GLR parsers.
7868 It installs a lookahead token with token kind @var{token} and
7869 semantic value @var{value}; then it discards the value that was
7870 going to be reduced by this rule.
7872 If the macro is used when it is not valid, such as when there is
7873 a lookahead token already, then it reports a syntax error with
7874 a message @samp{cannot back up} and performs ordinary error
7877 In either case, the rest of the action is not executed.
7880 @deffn {Value} YYEMPTY
7881 Value stored in @code{yychar} when there is no lookahead token.
7884 @deffn {Value} YYEOF
7885 Value stored in @code{yychar} when the lookahead is the end of the input
7889 @deffn {Macro} YYERROR @code{;}
7890 Cause an immediate syntax error. This statement initiates error
7891 recovery just as if the parser itself had detected an error; however, it
7892 does not call @code{yyerror}, and does not print any message. If you
7893 want to print an error message, call @code{yyerror} explicitly before
7894 the @samp{YYERROR;} statement. @xref{Error Recovery}.
7897 @deffn {Macro} YYRECOVERING
7898 @findex YYRECOVERING
7899 The expression @code{YYRECOVERING ()} yields 1 when the parser
7900 is recovering from a syntax error, and 0 otherwise.
7901 @xref{Error Recovery}.
7904 @deffn {Variable} yychar
7905 Variable containing either the lookahead token, or @code{YYEOF} when the
7906 lookahead is the end of the input stream, or @code{YYEMPTY} when no lookahead
7907 has been performed so the next token is not yet known.
7908 Do not modify @code{yychar} in a deferred semantic action (@pxref{GLR Semantic
7913 @deffn {Macro} yyclearin @code{;}
7914 Discard the current lookahead token. This is useful primarily in
7916 Do not invoke @code{yyclearin} in a deferred semantic action (@pxref{GLR
7918 @xref{Error Recovery}.
7921 @deffn {Macro} yyerrok @code{;}
7922 Resume generating error messages immediately for subsequent syntax
7923 errors. This is useful primarily in error rules.
7924 @xref{Error Recovery}.
7927 @deffn {Variable} yylloc
7928 Variable containing the lookahead token location when @code{yychar} is not set
7929 to @code{YYEMPTY} or @code{YYEOF}.
7930 Do not modify @code{yylloc} in a deferred semantic action (@pxref{GLR Semantic
7932 @xref{Actions and Locations}.
7935 @deffn {Variable} yylval
7936 Variable containing the lookahead token semantic value when @code{yychar} is
7937 not set to @code{YYEMPTY} or @code{YYEOF}.
7938 Do not modify @code{yylval} in a deferred semantic action (@pxref{GLR Semantic
7944 Acts like a structure variable containing information on the textual
7945 location of the grouping made by the current rule. @xref{Tracking
7948 @c Check if those paragraphs are still useful or not.
7952 @c int first_line, last_line;
7953 @c int first_column, last_column;
7957 @c Thus, to get the starting line number of the third component, you would
7958 @c use @samp{@@3.first_line}.
7960 @c In order for the members of this structure to contain valid information,
7961 @c you must make @code{yylex} supply this information about each token.
7962 @c If you need only certain members, then @code{yylex} need only fill in
7965 @c The use of this feature makes the parser noticeably slower.
7968 @deffn {Value} @@@var{n}
7970 Acts like a structure variable containing information on the textual
7971 location of the @var{n}th component of the current rule. @xref{Tracking
7975 @node Internationalization
7976 @section Parser Internationalization
7977 @cindex internationalization
7983 A Bison-generated parser can print diagnostics, including error and
7984 tracing messages. By default, they appear in English. However, Bison
7985 also supports outputting diagnostics in the user's native language. To
7986 make this work, the user should set the usual environment variables.
7987 @xref{Users, , The User's View, gettext, GNU @code{gettext} utilities}.
7988 For example, the shell command @samp{export LC_ALL=fr_CA.UTF-8} might
7989 set the user's locale to French Canadian using the UTF-8
7990 encoding. The exact set of available locales depends on the user's
7994 * Enabling I18n:: Preparing your project to support internationalization.
7995 * Token I18n:: Preparing tokens for internationalization in error messages.
7999 @subsection Enabling Internationalization
8001 The maintainer of a package that uses a Bison-generated parser enables
8002 the internationalization of the parser's output through the following
8003 steps. Here we assume a package that uses GNU Autoconf and
8008 @cindex bison-i18n.m4
8009 Into the directory containing the GNU Autoconf macros used
8010 by the package ---often called @file{m4}--- copy the
8011 @file{bison-i18n.m4} file installed by Bison under
8012 @samp{share/aclocal/bison-i18n.m4} in Bison's installation directory.
8016 cp /usr/local/share/aclocal/bison-i18n.m4 m4/bison-i18n.m4
8021 @vindex BISON_LOCALEDIR
8022 @vindex YYENABLE_NLS
8023 In the top-level @file{configure.ac}, after the @code{AM_GNU_GETTEXT}
8024 invocation, add an invocation of @code{BISON_I18N}. This macro is
8025 defined in the file @file{bison-i18n.m4} that you copied earlier. It
8026 causes @code{configure} to find the value of the
8027 @code{BISON_LOCALEDIR} variable, and it defines the source-language
8028 symbol @code{YYENABLE_NLS} to enable translations in the
8029 Bison-generated parser.
8032 In the @code{main} function of your program, designate the directory
8033 containing Bison's runtime message catalog, through a call to
8034 @samp{bindtextdomain} with domain name @samp{bison-runtime}.
8038 bindtextdomain ("bison-runtime", BISON_LOCALEDIR);
8041 Typically this appears after any other call @code{bindtextdomain
8042 (PACKAGE, LOCALEDIR)} that your package already has. Here we rely on
8043 @samp{BISON_LOCALEDIR} to be defined as a string through the
8047 In the @file{Makefile.am} that controls the compilation of the @code{main}
8048 function, make @samp{BISON_LOCALEDIR} available as a C preprocessor macro,
8049 either in @samp{DEFS} or in @samp{AM_CPPFLAGS}. For example:
8052 DEFS = @@DEFS@@ -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
8058 AM_CPPFLAGS = -DBISON_LOCALEDIR='"$(BISON_LOCALEDIR)"'
8062 Finally, invoke the command @command{autoreconf} to generate the build
8067 @subsection Token Internationalization
8069 When the @code{%define} variable @code{parse.error} is set to @code{custom}
8070 or @code{detailed}, token aliases can be internationalized:
8074 '\n' _("end of line")
8082 The remainder of the grammar may freely use either the token symbol
8083 (@code{FUN}) or its alias (@code{"function"}), but not with the
8084 internationalization marker (@code{_("function")}).
8086 If at least one token alias is internationalized, then the generated parser
8087 will use both @code{N_} and @code{_}, that must be defined
8088 (@pxref{Programmers, , The Programmer’s View, gettext, GNU @code{gettext}
8089 utilities}). They are used only on string aliases marked for translation.
8090 In other words, even if your catalog features a translation for
8091 ``function'', then with
8101 ``function'' will appear untranslated in debug traces and error messages.
8103 Unless defined by the user, the end-of-file token, @code{YYEOF}, is provided
8104 ``end of file'' as an alias. It is also internationalized if the user
8105 internationalized tokens. To map it to another string, use:
8108 %token END 0 _("end of input")
8113 @chapter The Bison Parser Algorithm
8114 @cindex Bison parser algorithm
8115 @cindex algorithm of parser
8118 @cindex parser stack
8119 @cindex stack, parser
8121 As Bison reads tokens, it pushes them onto a stack along with their
8122 semantic values. The stack is called the @dfn{parser stack}. Pushing a
8123 token is traditionally called @dfn{shifting}.
8125 For example, suppose the infix calculator has read @samp{1 + 5 *}, with a
8126 @samp{3} to come. The stack will have four elements, one for each token
8129 But the stack does not always have an element for each token read. When
8130 the last @var{n} tokens and groupings shifted match the components of a
8131 grammar rule, they can be combined according to that rule. This is called
8132 @dfn{reduction}. Those tokens and groupings are replaced on the stack by a
8133 single grouping whose symbol is the result (left hand side) of that rule.
8134 Running the rule's action is part of the process of reduction, because this
8135 is what computes the semantic value of the resulting grouping.
8137 For example, if the infix calculator's parser stack contains this:
8144 and the next input token is a newline character, then the last three
8145 elements can be reduced to 15 via the rule:
8148 expr: expr '*' expr;
8152 Then the stack contains just these three elements:
8159 At this point, another reduction can be made, resulting in the single value
8160 16. Then the newline token can be shifted.
8162 The parser tries, by shifts and reductions, to reduce the entire input down
8163 to a single grouping whose symbol is the grammar's start-symbol
8164 (@pxref{Language and Grammar}).
8166 This kind of parser is known in the literature as a bottom-up parser.
8169 * Lookahead:: Parser looks one token ahead when deciding what to do.
8170 * Shift/Reduce:: Conflicts: when either shifting or reduction is valid.
8171 * Precedence:: Operator precedence works by resolving conflicts.
8172 * Contextual Precedence:: When an operator's precedence depends on context.
8173 * Parser States:: The parser is a finite-state-machine with stack.
8174 * Reduce/Reduce:: When two rules are applicable in the same situation.
8175 * Mysterious Conflicts:: Conflicts that look unjustified.
8176 * Tuning LR:: How to tune fundamental aspects of LR-based parsing.
8177 * Generalized LR Parsing:: Parsing arbitrary context-free grammars.
8178 * Memory Management:: What happens when memory is exhausted. How to avoid it.
8182 @section Lookahead Tokens
8183 @cindex lookahead token
8185 The Bison parser does @emph{not} always reduce immediately as soon as the
8186 last @var{n} tokens and groupings match a rule. This is because such a
8187 simple strategy is inadequate to handle most languages. Instead, when a
8188 reduction is possible, the parser sometimes ``looks ahead'' at the next
8189 token in order to decide what to do.
8191 When a token is read, it is not immediately shifted; first it becomes the
8192 @dfn{lookahead token}, which is not on the stack. Now the parser can
8193 perform one or more reductions of tokens and groupings on the stack, while
8194 the lookahead token remains off to the side. When no more reductions
8195 should take place, the lookahead token is shifted onto the stack. This
8196 does not mean that all possible reductions have been done; depending on the
8197 token kind of the lookahead token, some rules may choose to delay their
8200 Here is a simple case where lookahead is needed. These three rules define
8201 expressions which contain binary addition operators and postfix unary
8202 factorial operators (@samp{!}), and allow parentheses for grouping.
8221 Suppose that the tokens @w{@samp{1 + 2}} have been read and shifted; what
8222 should be done? If the following token is @samp{)}, then the first three
8223 tokens must be reduced to form an @code{expr}. This is the only valid
8224 course, because shifting the @samp{)} would produce a sequence of symbols
8225 @w{@code{term ')'}}, and no rule allows this.
8227 If the following token is @samp{!}, then it must be shifted immediately so
8228 that @w{@samp{2 !}} can be reduced to make a @code{term}. If instead the
8229 parser were to reduce before shifting, @w{@samp{1 + 2}} would become an
8230 @code{expr}. It would then be impossible to shift the @samp{!} because
8231 doing so would produce on the stack the sequence of symbols @code{expr
8232 '!'}. No rule allows that sequence.
8237 The lookahead token is stored in the variable @code{yychar}. Its semantic
8238 value and location, if any, are stored in the variables @code{yylval} and
8239 @code{yylloc}. @xref{Action Features}.
8242 @section Shift/Reduce Conflicts
8244 @cindex shift/reduce conflicts
8245 @cindex dangling @code{else}
8246 @cindex @code{else}, dangling
8248 Suppose we are parsing a language which has if-then and if-then-else
8249 statements, with a pair of rules like this:
8254 "if" expr "then" stmt
8255 | "if" expr "then" stmt "else" stmt
8261 Here @code{"if"}, @code{"then"} and @code{"else"} are terminal symbols for
8262 specific keyword tokens.
8264 When the @code{"else"} token is read and becomes the lookahead token, the
8265 contents of the stack (assuming the input is valid) are just right for
8266 reduction by the first rule. But it is also legitimate to shift the
8267 @code{"else"}, because that would lead to eventual reduction by the second
8270 This situation, where either a shift or a reduction would be valid, is
8271 called a @dfn{shift/reduce conflict}. Bison is designed to resolve
8272 these conflicts by choosing to shift, unless otherwise directed by
8273 operator precedence declarations. To see the reason for this, let's
8274 contrast it with the other alternative.
8276 Since the parser prefers to shift the @code{"else"}, the result is to attach
8277 the else-clause to the innermost if-statement, making these two inputs
8281 if x then if y then win; else lose;
8283 if x then do; if y then win; else lose; end;
8286 But if the parser chose to reduce when possible rather than shift, the
8287 result would be to attach the else-clause to the outermost if-statement,
8288 making these two inputs equivalent:
8291 if x then if y then win; else lose;
8293 if x then do; if y then win; end; else lose;
8296 The conflict exists because the grammar as written is ambiguous: either
8297 parsing of the simple nested if-statement is legitimate. The established
8298 convention is that these ambiguities are resolved by attaching the
8299 else-clause to the innermost if-statement; this is what Bison accomplishes
8300 by choosing to shift rather than reduce. (It would ideally be cleaner to
8301 write an unambiguous grammar, but that is very hard to do in this case.)
8302 This particular ambiguity was first encountered in the specifications of
8303 Algol 60 and is called the ``dangling @code{else}'' ambiguity.
8305 To avoid warnings from Bison about predictable, legitimate shift/reduce
8306 conflicts, you can use the @code{%expect @var{n}} declaration.
8307 There will be no warning as long as the number of shift/reduce conflicts
8308 is exactly @var{n}, and Bison will report an error if there is a
8310 @xref{Expect Decl}. However, we don't
8311 recommend the use of @code{%expect} (except @samp{%expect 0}!), as an equal
8312 number of conflicts does not mean that they are the @emph{same}. When
8313 possible, you should rather use precedence directives to @emph{fix} the
8314 conflicts explicitly (@pxref{Non Operators}).
8316 The definition of @code{if_stmt} above is solely to blame for the
8317 conflict, but the conflict does not actually appear without additional
8318 rules. Here is a complete Bison grammar file that actually manifests
8332 "if" expr "then" stmt
8333 | "if" expr "then" stmt "else" stmt
8343 @section Operator Precedence
8344 @cindex operator precedence
8345 @cindex precedence of operators
8347 Another situation where shift/reduce conflicts appear is in arithmetic
8348 expressions. Here shifting is not always the preferred resolution; the
8349 Bison declarations for operator precedence allow you to specify when to
8350 shift and when to reduce.
8353 * Why Precedence:: An example showing why precedence is needed.
8354 * Using Precedence:: How to specify precedence and associativity.
8355 * Precedence Only:: How to specify precedence only.
8356 * Precedence Examples:: How these features are used in the previous example.
8357 * How Precedence:: How they work.
8358 * Non Operators:: Using precedence for general conflicts.
8361 @node Why Precedence
8362 @subsection When Precedence is Needed
8364 Consider the following ambiguous grammar fragment (ambiguous because the
8365 input @w{@samp{1 - 2 * 3}} can be parsed in two different ways):
8380 Suppose the parser has seen the tokens @samp{1}, @samp{-} and @samp{2};
8381 should it reduce them via the rule for the subtraction operator? It
8382 depends on the next token. Of course, if the next token is @samp{)}, we
8383 must reduce; shifting is invalid because no single rule can reduce the
8384 token sequence @w{@samp{- 2 )}} or anything starting with that. But if
8385 the next token is @samp{*} or @samp{<}, we have a choice: either
8386 shifting or reduction would allow the parse to complete, but with
8389 To decide which one Bison should do, we must consider the results. If
8390 the next operator token @var{op} is shifted, then it must be reduced
8391 first in order to permit another opportunity to reduce the difference.
8392 The result is (in effect) @w{@samp{1 - (2 @var{op} 3)}}. On the other
8393 hand, if the subtraction is reduced before shifting @var{op}, the result
8394 is @w{@samp{(1 - 2) @var{op} 3}}. Clearly, then, the choice of shift or
8395 reduce should depend on the relative precedence of the operators
8396 @samp{-} and @var{op}: @samp{*} should be shifted first, but not
8399 @cindex associativity
8400 What about input such as @w{@samp{1 - 2 - 5}}; should this be
8401 @w{@samp{(1 - 2) - 5}} or should it be @w{@samp{1 - (2 - 5)}}? For most
8402 operators we prefer the former, which is called @dfn{left association}.
8403 The latter alternative, @dfn{right association}, is desirable for
8404 assignment operators. The choice of left or right association is a
8405 matter of whether the parser chooses to shift or reduce when the stack
8406 contains @w{@samp{1 - 2}} and the lookahead token is @samp{-}: shifting
8407 makes right-associativity.
8409 @node Using Precedence
8410 @subsection Specifying Operator Precedence
8416 Bison allows you to specify these choices with the operator precedence
8417 declarations @code{%left} and @code{%right}. Each such declaration
8418 contains a list of tokens, which are operators whose precedence and
8419 associativity is being declared. The @code{%left} declaration makes all
8420 those operators left-associative and the @code{%right} declaration makes
8421 them right-associative. A third alternative is @code{%nonassoc}, which
8422 declares that it is a syntax error to find the same operator twice ``in a
8424 The last alternative, @code{%precedence}, allows to define only
8425 precedence and no associativity at all. As a result, any
8426 associativity-related conflict that remains will be reported as an
8427 compile-time error. The directive @code{%nonassoc} creates run-time
8428 error: using the operator in a associative way is a syntax error. The
8429 directive @code{%precedence} creates compile-time errors: an operator
8430 @emph{can} be involved in an associativity-related conflict, contrary to
8431 what expected the grammar author.
8433 The relative precedence of different operators is controlled by the
8434 order in which they are declared. The first precedence/associativity
8435 declaration in the file declares the operators whose
8436 precedence is lowest, the next such declaration declares the operators
8437 whose precedence is a little higher, and so on.
8439 @node Precedence Only
8440 @subsection Specifying Precedence Only
8443 Since POSIX Yacc defines only @code{%left}, @code{%right}, and
8444 @code{%nonassoc}, which all defines precedence and associativity, little
8445 attention is paid to the fact that precedence cannot be defined without
8446 defining associativity. Yet, sometimes, when trying to solve a
8447 conflict, precedence suffices. In such a case, using @code{%left},
8448 @code{%right}, or @code{%nonassoc} might hide future (associativity
8449 related) conflicts that would remain hidden.
8451 The dangling @code{else} ambiguity (@pxref{Shift/Reduce}) can be solved
8452 explicitly. This shift/reduce conflicts occurs in the following situation,
8453 where the period denotes the current parsing state:
8456 if @var{e1} then if @var{e2} then @var{s1} • else @var{s2}
8459 The conflict involves the reduction of the rule @samp{IF expr THEN
8460 stmt}, which precedence is by default that of its last token
8461 (@code{THEN}), and the shifting of the token @code{ELSE}. The usual
8462 disambiguation (attach the @code{else} to the closest @code{if}),
8463 shifting must be preferred, i.e., the precedence of @code{ELSE} must be
8464 higher than that of @code{THEN}. But neither is expected to be involved
8465 in an associativity related conflict, which can be specified as follows.
8472 The unary-minus is another typical example where associativity is usually
8473 over-specified, see @ref{Infix Calc}. The @code{%left} directive is
8474 traditionally used to declare the precedence of @code{NEG}, which is more
8475 than needed since it also defines its associativity. While this is harmless
8476 in the traditional example, who knows how @code{NEG} might be used in future
8477 evolutions of the grammar@dots{}
8479 @node Precedence Examples
8480 @subsection Precedence Examples
8482 In our example, we would want the following declarations:
8490 In a more complete example, which supports other operators as well, we
8491 would declare them in groups of equal precedence. For example, @code{'+'} is
8492 declared with @code{'-'}:
8495 %left '<' '>' '=' "!=" "<=" ">="
8500 @node How Precedence
8501 @subsection How Precedence Works
8503 The first effect of the precedence declarations is to assign precedence
8504 levels to the terminal symbols declared. The second effect is to assign
8505 precedence levels to certain rules: each rule gets its precedence from
8506 the last terminal symbol mentioned in the components. (You can also
8507 specify explicitly the precedence of a rule. @xref{Contextual
8510 Finally, the resolution of conflicts works by comparing the precedence
8511 of the rule being considered with that of the lookahead token. If the
8512 token's precedence is higher, the choice is to shift. If the rule's
8513 precedence is higher, the choice is to reduce. If they have equal
8514 precedence, the choice is made based on the associativity of that
8515 precedence level. The verbose output file made by @option{-v}
8516 (@pxref{Invocation}) says how each conflict was
8519 Not all rules and not all tokens have precedence. If either the rule or
8520 the lookahead token has no precedence, then the default is to shift.
8523 @subsection Using Precedence For Non Operators
8525 Using properly precedence and associativity directives can help fixing
8526 shift/reduce conflicts that do not involve arithmetic-like operators. For
8527 instance, the ``dangling @code{else}'' problem (@pxref{Shift/Reduce}) can be
8528 solved elegantly in two different ways.
8530 In the present case, the conflict is between the token @code{"else"} willing
8531 to be shifted, and the rule @samp{if_stmt: "if" expr "then" stmt}, asking
8532 for reduction. By default, the precedence of a rule is that of its last
8533 token, here @code{"then"}, so the conflict will be solved appropriately
8534 by giving @code{"else"} a precedence higher than that of @code{"then"}, for
8535 instance as follows:
8544 Alternatively, you may give both tokens the same precedence, in which case
8545 associativity is used to solve the conflict. To preserve the shift action,
8546 use right associativity:
8549 %right "then" "else"
8552 Neither solution is perfect however. Since Bison does not provide, so far,
8553 ``scoped'' precedence, both force you to declare the precedence
8554 of these keywords with respect to the other operators your grammar.
8555 Therefore, instead of being warned about new conflicts you would be unaware
8556 of (e.g., a shift/reduce conflict due to @samp{if test then 1 else 2 + 3}
8557 being ambiguous: @samp{if test then 1 else (2 + 3)} or @samp{(if test then 1
8558 else 2) + 3}?), the conflict will be already ``fixed''.
8560 @node Contextual Precedence
8561 @section Context-Dependent Precedence
8562 @cindex context-dependent precedence
8563 @cindex unary operator precedence
8564 @cindex precedence, context-dependent
8565 @cindex precedence, unary operator
8568 Often the precedence of an operator depends on the context. This sounds
8569 outlandish at first, but it is really very common. For example, a minus
8570 sign typically has a very high precedence as a unary operator, and a
8571 somewhat lower precedence (lower than multiplication) as a binary operator.
8573 The Bison precedence declarations
8574 can only be used once for a given token; so a token has
8575 only one precedence declared in this way. For context-dependent
8576 precedence, you need to use an additional mechanism: the @code{%prec}
8579 The @code{%prec} modifier declares the precedence of a particular rule by
8580 specifying a terminal symbol whose precedence should be used for that rule.
8581 It's not necessary for that symbol to appear otherwise in the rule. The
8582 modifier's syntax is:
8585 %prec @var{terminal-symbol}
8589 and it is written after the components of the rule. Its effect is to
8590 assign the rule the precedence of @var{terminal-symbol}, overriding
8591 the precedence that would be deduced for it in the ordinary way. The
8592 altered rule precedence then affects how conflicts involving that rule
8593 are resolved (@pxref{Precedence}).
8595 Here is how @code{%prec} solves the problem of unary minus. First, declare
8596 a precedence for a fictitious terminal symbol named @code{UMINUS}. There
8597 are no tokens of this type, but the symbol serves to stand for its
8607 Now the precedence of @code{UMINUS} can be used in specific rules:
8615 | '-' exp %prec UMINUS
8620 If you forget to append @code{%prec UMINUS} to the rule for unary
8621 minus, Bison silently assumes that minus has its usual precedence.
8622 This kind of problem can be tricky to debug, since one typically
8623 discovers the mistake only by testing the code.
8625 The @code{%no-default-prec;} declaration makes it easier to discover
8626 this kind of problem systematically. It causes rules that lack a
8627 @code{%prec} modifier to have no precedence, even if the last terminal
8628 symbol mentioned in their components has a declared precedence.
8630 If @code{%no-default-prec;} is in effect, you must specify @code{%prec}
8631 for all rules that participate in precedence conflict resolution.
8632 Then you will see any shift/reduce conflict until you tell Bison how
8633 to resolve it, either by changing your grammar or by adding an
8634 explicit precedence. This will probably add declarations to the
8635 grammar, but it helps to protect against incorrect rule precedences.
8637 The effect of @code{%no-default-prec;} can be reversed by giving
8638 @code{%default-prec;}, which is the default.
8642 @section Parser States
8643 @cindex finite-state machine
8644 @cindex parser state
8645 @cindex state (of parser)
8647 The function @code{yyparse} is implemented using a finite-state machine.
8648 The values pushed on the parser stack are not simply token kind codes; they
8649 represent the entire sequence of terminal and nonterminal symbols at or
8650 near the top of the stack. The current state collects all the information
8651 about previous input which is relevant to deciding what to do next.
8653 Each time a lookahead token is read, the current parser state together with
8654 the kind of lookahead token are looked up in a table. This table entry can
8655 say, ``Shift the lookahead token.'' In this case, it also specifies the new
8656 parser state, which is pushed onto the top of the parser stack. Or it can
8657 say, ``Reduce using rule number @var{n}.'' This means that a certain number
8658 of tokens or groupings are taken off the top of the stack, and replaced by
8659 one grouping. In other words, that number of states are popped from the
8660 stack, and one new state is pushed.
8662 There is one other alternative: the table can say that the lookahead token
8663 is erroneous in the current state. This causes error processing to begin
8664 (@pxref{Error Recovery}).
8667 @section Reduce/Reduce Conflicts
8668 @cindex reduce/reduce conflict
8669 @cindex conflicts, reduce/reduce
8671 A reduce/reduce conflict occurs if there are two or more rules that apply
8672 to the same sequence of input. This usually indicates a serious error
8675 For example, here is an erroneous attempt to define a sequence
8676 of zero or more @code{word} groupings.
8681 %empty @{ printf ("empty sequence\n"); @}
8683 | sequence word @{ printf ("added word %s\n", $2); @}
8689 %empty @{ printf ("empty maybeword\n"); @}
8690 | word @{ printf ("single word %s\n", $1); @}
8696 The error is an ambiguity: there is more than one way to parse a single
8697 @code{word} into a @code{sequence}. It could be reduced to a
8698 @code{maybeword} and then into a @code{sequence} via the second rule.
8699 Alternatively, nothing-at-all could be reduced into a @code{sequence}
8700 via the first rule, and this could be combined with the @code{word}
8701 using the third rule for @code{sequence}.
8703 There is also more than one way to reduce nothing-at-all into a
8704 @code{sequence}. This can be done directly via the first rule,
8705 or indirectly via @code{maybeword} and then the second rule.
8707 You might think that this is a distinction without a difference, because it
8708 does not change whether any particular input is valid or not. But it does
8709 affect which actions are run. One parsing order runs the second rule's
8710 action; the other runs the first rule's action and the third rule's action.
8711 In this example, the output of the program changes.
8713 Bison resolves a reduce/reduce conflict by choosing to use the rule that
8714 appears first in the grammar, but it is very risky to rely on this. Every
8715 reduce/reduce conflict must be studied and usually eliminated. Here is the
8716 proper way to define @code{sequence}:
8721 %empty @{ printf ("empty sequence\n"); @}
8722 | sequence word @{ printf ("added word %s\n", $2); @}
8727 Here is another common error that yields a reduce/reduce conflict:
8734 | sequence redirects
8748 | redirects redirect
8754 The intention here is to define a sequence which can contain either
8755 @code{word} or @code{redirect} groupings. The individual definitions of
8756 @code{sequence}, @code{words} and @code{redirects} are error-free, but the
8757 three together make a subtle ambiguity: even an empty input can be parsed
8758 in infinitely many ways!
8760 Consider: nothing-at-all could be a @code{words}. Or it could be two
8761 @code{words} in a row, or three, or any number. It could equally well be a
8762 @code{redirects}, or two, or any number. Or it could be a @code{words}
8763 followed by three @code{redirects} and another @code{words}. And so on.
8765 Here are two ways to correct these rules. First, to make it a single level
8776 Second, to prevent either a @code{words} or a @code{redirects}
8784 | sequence redirects
8798 | redirects redirect
8803 Yet this proposal introduces another kind of ambiguity! The input
8804 @samp{word word} can be parsed as a single @code{words} composed of two
8805 @samp{word}s, or as two one-@code{word} @code{words} (and likewise for
8806 @code{redirect}/@code{redirects}). However this ambiguity is now a
8807 shift/reduce conflict, and therefore it can now be addressed with precedence
8810 To simplify the matter, we will proceed with @code{word} and @code{redirect}
8811 being tokens: @code{"word"} and @code{"redirect"}.
8813 To prefer the longest @code{words}, the conflict between the token
8814 @code{"word"} and the rule @samp{sequence: sequence words} must be resolved
8815 as a shift. To this end, we use the same techniques as exposed above, see
8816 @ref{Non Operators}. One solution
8817 relies on precedences: use @code{%prec} to give a lower precedence to the
8822 %precedence "sequence"
8827 | sequence word %prec "sequence"
8828 | sequence redirect %prec "sequence"
8840 Another solution relies on associativity: provide both the token and the
8841 rule with the same precedence, but make them right-associative:
8844 %right "word" "redirect"
8849 | sequence word %prec "word"
8850 | sequence redirect %prec "redirect"
8855 @node Mysterious Conflicts
8856 @section Mysterious Conflicts
8857 @cindex Mysterious Conflicts
8859 Sometimes reduce/reduce conflicts can occur that don't look warranted.
8865 def: param_spec return_spec ',';
8868 | name_list ':' type
8885 | name ',' name_list
8890 It would seem that this grammar can be parsed with only a single token of
8891 lookahead: when a @code{param_spec} is being read, an @code{"id"} is a
8892 @code{name} if a comma or colon follows, or a @code{type} if another
8893 @code{"id"} follows. In other words, this grammar is LR(1).
8897 However, for historical reasons, Bison cannot by default handle all
8899 In this grammar, two contexts, that after an @code{"id"} at the beginning
8900 of a @code{param_spec} and likewise at the beginning of a
8901 @code{return_spec}, are similar enough that Bison assumes they are the
8903 They appear similar because the same set of rules would be
8904 active---the rule for reducing to a @code{name} and that for reducing to
8905 a @code{type}. Bison is unable to determine at that stage of processing
8906 that the rules would require different lookahead tokens in the two
8907 contexts, so it makes a single parser state for them both. Combining
8908 the two contexts causes a conflict later. In parser terminology, this
8909 occurrence means that the grammar is not LALR(1).
8912 @cindex canonical LR
8913 For many practical grammars (specifically those that fall into the non-LR(1)
8914 class), the limitations of LALR(1) result in difficulties beyond just
8915 mysterious reduce/reduce conflicts. The best way to fix all these problems
8916 is to select a different parser table construction algorithm. Either
8917 IELR(1) or canonical LR(1) would suffice, but the former is more efficient
8918 and easier to debug during development. @xref{LR Table Construction}, for
8921 If you instead wish to work around LALR(1)'s limitations, you
8922 can often fix a mysterious conflict by identifying the two parser states
8923 that are being confused, and adding something to make them look
8924 distinct. In the above example, adding one rule to
8925 @code{return_spec} as follows makes the problem go away:
8933 | "id" "bogus" /* This rule is never used. */
8938 This corrects the problem because it introduces the possibility of an
8939 additional active rule in the context after the @code{"id"} at the beginning of
8940 @code{return_spec}. This rule is not active in the corresponding context
8941 in a @code{param_spec}, so the two contexts receive distinct parser states.
8942 As long as the token @code{"bogus"} is never generated by @code{yylex},
8943 the added rule cannot alter the way actual input is parsed.
8945 In this particular example, there is another way to solve the problem:
8946 rewrite the rule for @code{return_spec} to use @code{"id"} directly
8947 instead of via @code{name}. This also causes the two confusing
8948 contexts to have different sets of active rules, because the one for
8949 @code{return_spec} activates the altered rule for @code{return_spec}
8950 rather than the one for @code{name}.
8956 | name_list ':' type
8968 For a more detailed exposition of LALR(1) parsers and parser generators, see
8969 @tcite{DeRemer 1982}.
8974 The default behavior of Bison's LR-based parsers is chosen mostly for
8975 historical reasons, but that behavior is often not robust. For example, in
8976 the previous section, we discussed the mysterious conflicts that can be
8977 produced by LALR(1), Bison's default parser table construction algorithm.
8978 Another example is Bison's @code{%define parse.error verbose} directive,
8979 which instructs the generated parser to produce verbose syntax error
8980 messages, which can sometimes contain incorrect information.
8982 In this section, we explore several modern features of Bison that allow you
8983 to tune fundamental aspects of the generated LR-based parsers. Some of
8984 these features easily eliminate shortcomings like those mentioned above.
8985 Others can be helpful purely for understanding your parser.
8988 * LR Table Construction:: Choose a different construction algorithm.
8989 * Default Reductions:: Disable default reductions.
8990 * LAC:: Correct lookahead sets in the parser states.
8991 * Unreachable States:: Keep unreachable parser states for debugging.
8994 @node LR Table Construction
8995 @subsection LR Table Construction
8996 @cindex Mysterious Conflict
8999 @cindex canonical LR
9000 @findex %define lr.type
9002 For historical reasons, Bison constructs LALR(1) parser tables by default.
9003 However, LALR does not possess the full language-recognition power of LR.
9004 As a result, the behavior of parsers employing LALR parser tables is often
9005 mysterious. We presented a simple example of this effect in @ref{Mysterious
9008 As we also demonstrated in that example, the traditional approach to
9009 eliminating such mysterious behavior is to restructure the grammar.
9010 Unfortunately, doing so correctly is often difficult. Moreover, merely
9011 discovering that LALR causes mysterious behavior in your parser can be
9014 Fortunately, Bison provides an easy way to eliminate the possibility of such
9015 mysterious behavior altogether. You simply need to activate a more powerful
9016 parser table construction algorithm by using the @code{%define lr.type}
9019 @deffn {Directive} {%define lr.type} @var{type}
9020 Specify the type of parser tables within the LR(1) family. The accepted
9021 values for @var{type} are:
9024 @item @code{lalr} (default)
9026 @item @code{canonical-lr}
9030 For example, to activate IELR, you might add the following directive to you
9034 %define lr.type ielr
9037 @noindent For the example in @ref{Mysterious Conflicts}, the mysterious
9038 conflict is then eliminated, so there is no need to invest time in
9039 comprehending the conflict or restructuring the grammar to fix it. If,
9040 during future development, the grammar evolves such that all mysterious
9041 behavior would have disappeared using just LALR, you need not fear that
9042 continuing to use IELR will result in unnecessarily large parser tables.
9043 That is, IELR generates LALR tables when LALR (using a deterministic parsing
9044 algorithm) is sufficient to support the full language-recognition power of
9045 LR. Thus, by enabling IELR at the start of grammar development, you can
9046 safely and completely eliminate the need to consider LALR's shortcomings.
9048 While IELR is almost always preferable, there are circumstances where LALR
9049 or the canonical LR parser tables described by Knuth @pcite{Knuth 1965} can
9050 be useful. Here we summarize the relative advantages of each parser table
9051 construction algorithm within Bison:
9056 There are at least two scenarios where LALR can be worthwhile:
9059 @item GLR without static conflict resolution.
9061 @cindex GLR with LALR
9062 When employing GLR parsers (@pxref{GLR Parsers}), if you do not resolve any
9063 conflicts statically (for example, with @code{%left} or @code{%precedence}),
9065 the parser explores all potential parses of any given input. In this case,
9066 the choice of parser table construction algorithm is guaranteed not to alter
9067 the language accepted by the parser. LALR parser tables are the smallest
9068 parser tables Bison can currently construct, so they may then be preferable.
9069 Nevertheless, once you begin to resolve conflicts statically, GLR behaves
9070 more like a deterministic parser in the syntactic contexts where those
9071 conflicts appear, and so either IELR or canonical LR can then be helpful to
9072 avoid LALR's mysterious behavior.
9074 @item Malformed grammars.
9076 Occasionally during development, an especially malformed grammar with a
9077 major recurring flaw may severely impede the IELR or canonical LR parser
9078 table construction algorithm. LALR can be a quick way to construct parser
9079 tables in order to investigate such problems while ignoring the more subtle
9080 differences from IELR and canonical LR.
9085 IELR (Inadequacy Elimination LR) is a minimal LR algorithm. That is, given
9086 any grammar (LR or non-LR), parsers using IELR or canonical LR parser tables
9087 always accept exactly the same set of sentences. However, like LALR, IELR
9088 merges parser states during parser table construction so that the number of
9089 parser states is often an order of magnitude less than for canonical LR.
9090 More importantly, because canonical LR's extra parser states may contain
9091 duplicate conflicts in the case of non-LR grammars, the number of conflicts
9092 for IELR is often an order of magnitude less as well. This effect can
9093 significantly reduce the complexity of developing a grammar.
9097 @cindex delayed syntax error detection
9100 While inefficient, canonical LR parser tables can be an interesting means to
9101 explore a grammar because they possess a property that IELR and LALR tables
9102 do not. That is, if @code{%nonassoc} is not used and default reductions are
9103 left disabled (@pxref{Default Reductions}), then, for every left context of
9104 every canonical LR state, the set of tokens accepted by that state is
9105 guaranteed to be the exact set of tokens that is syntactically acceptable in
9106 that left context. It might then seem that an advantage of canonical LR
9107 parsers in production is that, under the above constraints, they are
9108 guaranteed to detect a syntax error as soon as possible without performing
9109 any unnecessary reductions. However, IELR parsers that use LAC are also
9110 able to achieve this behavior without sacrificing @code{%nonassoc} or
9111 default reductions. For details and a few caveats of LAC, @pxref{LAC}.
9114 For a more detailed exposition of the mysterious behavior in LALR parsers
9115 and the benefits of IELR, see @tcite{Denny 2008}, and @tcite{Denny 2010
9118 @node Default Reductions
9119 @subsection Default Reductions
9120 @cindex default reductions
9121 @findex %define lr.default-reduction
9124 After parser table construction, Bison identifies the reduction with the
9125 largest lookahead set in each parser state. To reduce the size of the
9126 parser state, traditional Bison behavior is to remove that lookahead set and
9127 to assign that reduction to be the default parser action. Such a reduction
9128 is known as a @dfn{default reduction}.
9130 Default reductions affect more than the size of the parser tables. They
9131 also affect the behavior of the parser:
9134 @item Delayed @code{yylex} invocations.
9136 @cindex delayed yylex invocations
9137 @cindex consistent states
9138 @cindex defaulted states
9139 A @dfn{consistent state} is a state that has only one possible parser
9140 action. If that action is a reduction and is encoded as a default
9141 reduction, then that consistent state is called a @dfn{defaulted state}.
9142 Upon reaching a defaulted state, a Bison-generated parser does not bother to
9143 invoke @code{yylex} to fetch the next token before performing the reduction.
9144 In other words, whether default reductions are enabled in consistent states
9145 determines how soon a Bison-generated parser invokes @code{yylex} for a
9146 token: immediately when it @emph{reaches} that token in the input or when it
9147 eventually @emph{needs} that token as a lookahead to determine the next
9148 parser action. Traditionally, default reductions are enabled, and so the
9149 parser exhibits the latter behavior.
9151 The presence of defaulted states is an important consideration when
9152 designing @code{yylex} and the grammar file. That is, if the behavior of
9153 @code{yylex} can influence or be influenced by the semantic actions
9154 associated with the reductions in defaulted states, then the delay of the
9155 next @code{yylex} invocation until after those reductions is significant.
9156 For example, the semantic actions might pop a scope stack that @code{yylex}
9157 uses to determine what token to return. Thus, the delay might be necessary
9158 to ensure that @code{yylex} does not look up the next token in a scope that
9159 should already be considered closed.
9161 @item Delayed syntax error detection.
9163 @cindex delayed syntax error detection
9164 When the parser fetches a new token by invoking @code{yylex}, it checks
9165 whether there is an action for that token in the current parser state. The
9166 parser detects a syntax error if and only if either (1) there is no action
9167 for that token or (2) the action for that token is the error action (due to
9168 the use of @code{%nonassoc}). However, if there is a default reduction in
9169 that state (which might or might not be a defaulted state), then it is
9170 impossible for condition 1 to exist. That is, all tokens have an action.
9171 Thus, the parser sometimes fails to detect the syntax error until it reaches
9175 @c If there's an infinite loop, default reductions can prevent an incorrect
9176 @c sentence from being rejected.
9177 While default reductions never cause the parser to accept syntactically
9178 incorrect sentences, the delay of syntax error detection can have unexpected
9179 effects on the behavior of the parser. However, the delay can be caused
9180 anyway by parser state merging and the use of @code{%nonassoc}, and it can
9181 be fixed by another Bison feature, LAC. We discuss the effects of delayed
9182 syntax error detection and LAC more in the next section (@pxref{LAC}).
9185 For canonical LR, the only default reduction that Bison enables by default
9186 is the accept action, which appears only in the accepting state, which has
9187 no other action and is thus a defaulted state. However, the default accept
9188 action does not delay any @code{yylex} invocation or syntax error detection
9189 because the accept action ends the parse.
9191 For LALR and IELR, Bison enables default reductions in nearly all states by
9192 default. There are only two exceptions. First, states that have a shift
9193 action on the @code{error} token do not have default reductions because
9194 delayed syntax error detection could then prevent the @code{error} token
9195 from ever being shifted in that state. However, parser state merging can
9196 cause the same effect anyway, and LAC fixes it in both cases, so future
9197 versions of Bison might drop this exception when LAC is activated. Second,
9198 GLR parsers do not record the default reduction as the action on a lookahead
9199 token for which there is a conflict. The correct action in this case is to
9200 split the parse instead.
9202 To adjust which states have default reductions enabled, use the
9203 @code{%define lr.default-reduction} directive.
9205 @deffn {Directive} {%define lr.default-reduction} @var{where}
9206 Specify the kind of states that are permitted to contain default reductions.
9207 The accepted values of @var{where} are:
9209 @item @code{most} (default for LALR and IELR)
9210 @item @code{consistent}
9211 @item @code{accepting} (default for canonical LR)
9217 @findex %define parse.lac
9219 @cindex lookahead correction
9221 Canonical LR, IELR, and LALR can suffer from a couple of problems upon
9222 encountering a syntax error. First, the parser might perform additional
9223 parser stack reductions before discovering the syntax error. Such
9224 reductions can perform user semantic actions that are unexpected because
9225 they are based on an invalid token, and they cause error recovery to begin
9226 in a different syntactic context than the one in which the invalid token was
9227 encountered. Second, when verbose error messages are enabled (@pxref{Error
9228 Reporting}), the expected token list in the syntax error message can both
9229 contain invalid tokens and omit valid tokens.
9231 The culprits for the above problems are @code{%nonassoc}, default reductions
9232 in inconsistent states (@pxref{Default Reductions}), and parser state
9233 merging. Because IELR and LALR merge parser states, they suffer the most.
9234 Canonical LR can suffer only if @code{%nonassoc} is used or if default
9235 reductions are enabled for inconsistent states.
9237 LAC (Lookahead Correction) is a new mechanism within the parsing algorithm
9238 that solves these problems for canonical LR, IELR, and LALR without
9239 sacrificing @code{%nonassoc}, default reductions, or state merging. You can
9240 enable LAC with the @code{%define parse.lac} directive.
9242 @deffn {Directive} {%define parse.lac} @var{value}
9243 Enable LAC to improve syntax error handling.
9245 @item @code{none} (default)
9248 This feature is currently only available for deterministic parsers in C and C++.
9251 Conceptually, the LAC mechanism is straight-forward. Whenever the parser
9252 fetches a new token from the scanner so that it can determine the next
9253 parser action, it immediately suspends normal parsing and performs an
9254 exploratory parse using a temporary copy of the normal parser state stack.
9255 During this exploratory parse, the parser does not perform user semantic
9256 actions. If the exploratory parse reaches a shift action, normal parsing
9257 then resumes on the normal parser stacks. If the exploratory parse reaches
9258 an error instead, the parser reports a syntax error. If verbose syntax
9259 error messages are enabled, the parser must then discover the list of
9260 expected tokens, so it performs a separate exploratory parse for each token
9263 There is one subtlety about the use of LAC. That is, when in a consistent
9264 parser state with a default reduction, the parser will not attempt to fetch
9265 a token from the scanner because no lookahead is needed to determine the
9266 next parser action. Thus, whether default reductions are enabled in
9267 consistent states (@pxref{Default Reductions}) affects how soon the parser
9268 detects a syntax error: immediately when it @emph{reaches} an erroneous
9269 token or when it eventually @emph{needs} that token as a lookahead to
9270 determine the next parser action. The latter behavior is probably more
9271 intuitive, so Bison currently provides no way to achieve the former behavior
9272 while default reductions are enabled in consistent states.
9274 Thus, when LAC is in use, for some fixed decision of whether to enable
9275 default reductions in consistent states, canonical LR and IELR behave almost
9276 exactly the same for both syntactically acceptable and syntactically
9277 unacceptable input. While LALR still does not support the full
9278 language-recognition power of canonical LR and IELR, LAC at least enables
9279 LALR's syntax error handling to correctly reflect LALR's
9280 language-recognition power.
9282 There are a few caveats to consider when using LAC:
9285 @item Infinite parsing loops.
9287 IELR plus LAC does have one shortcoming relative to canonical LR. Some
9288 parsers generated by Bison can loop infinitely. LAC does not fix infinite
9289 parsing loops that occur between encountering a syntax error and detecting
9290 it, but enabling canonical LR or disabling default reductions sometimes
9293 @item Verbose error message limitations.
9295 Because of internationalization considerations, Bison-generated parsers
9296 limit the size of the expected token list they are willing to report in a
9297 verbose syntax error message. If the number of expected tokens exceeds that
9298 limit, the list is simply dropped from the message. Enabling LAC can
9299 increase the size of the list and thus cause the parser to drop it. Of
9300 course, dropping the list is better than reporting an incorrect list.
9304 Because LAC requires many parse actions to be performed twice, it can have a
9305 performance penalty. However, not all parse actions must be performed
9306 twice. Specifically, during a series of default reductions in consistent
9307 states and shift actions, the parser never has to initiate an exploratory
9308 parse. Moreover, the most time-consuming tasks in a parse are often the
9309 file I/O, the lexical analysis performed by the scanner, and the user's
9310 semantic actions, but none of these are performed during the exploratory
9311 parse. Finally, the base of the temporary stack used during an exploratory
9312 parse is a pointer into the normal parser state stack so that the stack is
9313 never physically copied. In our experience, the performance penalty of LAC
9314 has proved insignificant for practical grammars.
9317 While the LAC algorithm shares techniques that have been recognized in the
9318 parser community for years, for the publication that introduces LAC, see
9319 @tcite{Denny 2010 May}.
9321 @node Unreachable States
9322 @subsection Unreachable States
9323 @findex %define lr.keep-unreachable-state
9324 @cindex unreachable states
9326 If there exists no sequence of transitions from the parser's start state to
9327 some state @var{s}, then Bison considers @var{s} to be an @dfn{unreachable
9328 state}. A state can become unreachable during conflict resolution if Bison
9329 disables a shift action leading to it from a predecessor state.
9331 By default, Bison removes unreachable states from the parser after conflict
9332 resolution because they are useless in the generated parser. However,
9333 keeping unreachable states is sometimes useful when trying to understand the
9334 relationship between the parser and the grammar.
9336 @deffn {Directive} {%define lr.keep-unreachable-state} @var{value}
9337 Request that Bison allow unreachable states to remain in the parser tables.
9338 @var{value} must be a Boolean. The default is @code{false}.
9341 There are a few caveats to consider:
9344 @item Missing or extraneous warnings.
9346 Unreachable states may contain conflicts and may use rules not used in any
9347 other state. Thus, keeping unreachable states may induce warnings that are
9348 irrelevant to your parser's behavior, and it may eliminate warnings that are
9349 relevant. Of course, the change in warnings may actually be relevant to a
9350 parser table analysis that wants to keep unreachable states, so this
9351 behavior will likely remain in future Bison releases.
9353 @item Other useless states.
9355 While Bison is able to remove unreachable states, it is not guaranteed to
9356 remove other kinds of useless states. Specifically, when Bison disables
9357 reduce actions during conflict resolution, some goto actions may become
9358 useless, and thus some additional states may become useless. If Bison were
9359 to compute which goto actions were useless and then disable those actions,
9360 it could identify such states as unreachable and then remove those states.
9361 However, Bison does not compute which goto actions are useless.
9364 @node Generalized LR Parsing
9365 @section Generalized LR (GLR) Parsing
9367 @cindex generalized LR (GLR) parsing
9368 @cindex ambiguous grammars
9369 @cindex nondeterministic parsing
9371 Bison produces @emph{deterministic} parsers that choose uniquely
9372 when to reduce and which reduction to apply
9373 based on a summary of the preceding input and on one extra token of lookahead.
9374 As a result, normal Bison handles a proper subset of the family of
9375 context-free languages.
9376 Ambiguous grammars, since they have strings with more than one possible
9377 sequence of reductions cannot have deterministic parsers in this sense.
9378 The same is true of languages that require more than one symbol of
9379 lookahead, since the parser lacks the information necessary to make a
9380 decision at the point it must be made in a shift/reduce parser.
9381 Finally, as previously mentioned (@pxref{Mysterious Conflicts}),
9382 there are languages where Bison's default choice of how to
9383 summarize the input seen so far loses necessary information.
9385 When you use the @samp{%glr-parser} declaration in your grammar file,
9386 Bison generates a parser that uses a different algorithm, called
9387 Generalized LR (or GLR). A Bison GLR
9388 parser uses the same basic
9389 algorithm for parsing as an ordinary Bison parser, but behaves
9390 differently in cases where there is a shift/reduce conflict that has not
9391 been resolved by precedence rules (@pxref{Precedence}) or a
9392 reduce/reduce conflict. When a GLR parser encounters such a
9394 effectively @emph{splits} into a several parsers, one for each possible
9395 shift or reduction. These parsers then proceed as usual, consuming
9396 tokens in lock-step. Some of the stacks may encounter other conflicts
9397 and split further, with the result that instead of a sequence of states,
9398 a Bison GLR parsing stack is what is in effect a tree of states.
9400 In effect, each stack represents a guess as to what the proper parse
9401 is. Additional input may indicate that a guess was wrong, in which case
9402 the appropriate stack silently disappears. Otherwise, the semantics
9403 actions generated in each stack are saved, rather than being executed
9404 immediately. When a stack disappears, its saved semantic actions never
9405 get executed. When a reduction causes two stacks to become equivalent,
9406 their sets of semantic actions are both saved with the state that
9407 results from the reduction. We say that two stacks are equivalent
9408 when they both represent the same sequence of states,
9409 and each pair of corresponding states represents a
9410 grammar symbol that produces the same segment of the input token
9413 Whenever the parser makes a transition from having multiple
9414 states to having one, it reverts to the normal deterministic parsing
9415 algorithm, after resolving and executing the saved-up actions.
9416 At this transition, some of the states on the stack will have semantic
9417 values that are sets (actually multisets) of possible actions. The
9418 parser tries to pick one of the actions by first finding one whose rule
9419 has the highest dynamic precedence, as set by the @samp{%dprec}
9420 declaration. Otherwise, if the alternative actions are not ordered by
9421 precedence, but there the same merging function is declared for both
9422 rules by the @samp{%merge} declaration,
9423 Bison resolves and evaluates both and then calls the merge function on
9424 the result. Otherwise, it reports an ambiguity.
9426 It is possible to use a data structure for the GLR parsing tree that
9427 permits the processing of any LR(1) grammar in linear time (in the
9428 size of the input), any unambiguous (not necessarily
9430 quadratic worst-case time, and any general (possibly ambiguous)
9431 context-free grammar in cubic worst-case time. However, Bison currently
9432 uses a simpler data structure that requires time proportional to the
9433 length of the input times the maximum number of stacks required for any
9434 prefix of the input. Thus, really ambiguous or nondeterministic
9435 grammars can require exponential time and space to process. Such badly
9436 behaving examples, however, are not generally of practical interest.
9437 Usually, nondeterminism in a grammar is local---the parser is ``in
9438 doubt'' only for a few tokens at a time. Therefore, the current data
9439 structure should generally be adequate. On LR(1) portions of a
9440 grammar, in particular, it is only slightly slower than with the
9441 deterministic LR(1) Bison parser.
9443 For a more detailed exposition of GLR parsers, see @tcite{Scott 2000}.
9445 @node Memory Management
9446 @section Memory Management, and How to Avoid Memory Exhaustion
9447 @cindex memory exhaustion
9448 @cindex memory management
9449 @cindex stack overflow
9450 @cindex parser stack overflow
9451 @cindex overflow of parser stack
9453 The Bison parser stack can run out of memory if too many tokens are shifted and
9454 not reduced. When this happens, the parser function @code{yyparse}
9455 calls @code{yyerror} and then returns 2.
9457 Because Bison parsers have growing stacks, hitting the upper limit
9458 usually results from using a right recursion instead of a left
9459 recursion, see @ref{Recursion}.
9462 By defining the macro @code{YYMAXDEPTH}, you can control how deep the
9463 parser stack can become before memory is exhausted. Define the
9464 macro with a value that is an integer. This value is the maximum number
9465 of tokens that can be shifted (and not reduced) before overflow.
9467 The stack space allowed is not necessarily allocated. If you specify a
9468 large value for @code{YYMAXDEPTH}, the parser normally allocates a small
9469 stack at first, and then makes it bigger by stages as needed. This
9470 increasing allocation happens automatically and silently. Therefore,
9471 you do not need to make @code{YYMAXDEPTH} painfully small merely to save
9472 space for ordinary inputs that do not need much stack.
9474 However, do not allow @code{YYMAXDEPTH} to be a value so large that
9475 arithmetic overflow could occur when calculating the size of the stack
9476 space. Also, do not allow @code{YYMAXDEPTH} to be less than
9479 @cindex default stack limit
9480 The default value of @code{YYMAXDEPTH}, if you do not define it, is
9484 You can control how much stack is allocated initially by defining the
9485 macro @code{YYINITDEPTH} to a positive integer. For the deterministic
9486 parser in C, this value must be a compile-time constant
9487 unless you are assuming C99 or some other target language or compiler
9488 that allows variable-length arrays. The default is 200.
9490 Do not allow @code{YYINITDEPTH} to be greater than @code{YYMAXDEPTH}.
9492 You can generate a deterministic parser containing C++ user code from the
9493 default (C) skeleton, as well as from the C++ skeleton (@pxref{C++
9494 Parsers}). However, if you do use the default skeleton and want to allow
9495 the parsing stack to grow, be careful not to use semantic types or location
9496 types that require non-trivial copy constructors. The C skeleton bypasses
9497 these constructors when copying data to new, larger stacks.
9499 @node Error Recovery
9500 @chapter Error Recovery
9501 @cindex error recovery
9502 @cindex recovery from errors
9504 It is not usually acceptable to have a program terminate on a syntax
9505 error. For example, a compiler should recover sufficiently to parse the
9506 rest of the input file and check it for errors; a calculator should accept
9509 In a simple interactive command parser where each input is one line, it may
9510 be sufficient to allow @code{yyparse} to return 1 on error and have the
9511 caller ignore the rest of the input line when that happens (and then call
9512 @code{yyparse} again). But this is inadequate for a compiler, because it
9513 forgets all the syntactic context leading up to the error. A syntax error
9514 deep within a function in the compiler input should not cause the compiler
9515 to treat the following line like the beginning of a source file.
9518 You can define how to recover from a syntax error by writing rules to
9519 recognize the special token @code{error}. This is a terminal symbol that
9520 is always defined (you need not declare it) and reserved for error
9521 handling. The Bison parser generates an @code{error} token whenever a
9522 syntax error happens; if you have provided a rule to recognize this token
9523 in the current context, the parse can continue.
9535 The fourth rule in this example says that an error followed by a newline
9536 makes a valid addition to any @code{stmts}.
9538 What happens if a syntax error occurs in the middle of an @code{exp}? The
9539 error recovery rule, interpreted strictly, applies to the precise sequence
9540 of a @code{stmts}, an @code{error} and a newline. If an error occurs in
9541 the middle of an @code{exp}, there will probably be some additional tokens
9542 and subexpressions on the stack after the last @code{stmts}, and there
9543 will be tokens to read before the next newline. So the rule is not
9544 applicable in the ordinary way.
9546 But Bison can force the situation to fit the rule, by discarding part of the
9547 semantic context and part of the input. First it discards states and
9548 objects from the stack until it gets back to a state in which the
9549 @code{error} token is acceptable. (This means that the subexpressions
9550 already parsed are discarded, back to the last complete @code{stmts}.) At
9551 this point the @code{error} token can be shifted. Then, if the old
9552 lookahead token is not acceptable to be shifted next, the parser reads
9553 tokens and discards them until it finds a token which is acceptable. In
9554 this example, Bison reads and discards input until the next newline so that
9555 the fourth rule can apply. Note that discarded symbols are possible sources
9556 of memory leaks, see @ref{Destructor Decl}, for a means to reclaim this
9559 The choice of error rules in the grammar is a choice of strategies for
9560 error recovery. A simple and useful strategy is simply to skip the rest of
9561 the current input line or current statement if an error is detected:
9564 stmt: error ';' /* On error, skip until ';' is read. */
9567 It is also useful to recover to the matching close-delimiter of an
9568 opening-delimiter that has already been parsed. Otherwise the
9569 close-delimiter will probably appear to be unmatched, and generate another,
9570 spurious error message:
9580 Error recovery strategies are necessarily guesses. When they guess wrong,
9581 one syntax error often leads to another. In the above example, the error
9582 recovery rule guesses that an error is due to bad input within one
9583 @code{stmt}. Suppose that instead a spurious semicolon is inserted in the
9584 middle of a valid @code{stmt}. After the error recovery rule recovers from
9585 the first error, another syntax error will be found straight away, since the
9586 text following the spurious semicolon is also an invalid @code{stmt}.
9588 To prevent an outpouring of error messages, the parser will output no error
9589 message for another syntax error that happens shortly after the first; only
9590 after three consecutive input tokens have been successfully shifted will
9591 error messages resume.
9593 Note that rules which accept the @code{error} token may have actions, just
9594 as any other rules can.
9597 You can make error messages resume immediately by using the macro
9598 @code{yyerrok} in an action. If you do this in the error rule's action, no
9599 error messages will be suppressed. This macro requires no arguments;
9600 @samp{yyerrok;} is a valid C statement.
9603 The previous lookahead token is reanalyzed immediately after an error. If
9604 this is unacceptable, then the macro @code{yyclearin} may be used to clear
9605 this token. Write the statement @samp{yyclearin;} in the error rule's
9607 @xref{Action Features}.
9609 For example, suppose that on a syntax error, an error handling routine is
9610 called that advances the input stream to some point where parsing should
9611 once again commence. The next symbol returned by the lexical scanner is
9612 probably correct. The previous lookahead token ought to be discarded
9613 with @samp{yyclearin;}.
9615 @vindex YYRECOVERING
9616 The expression @code{YYRECOVERING ()} yields 1 when the parser
9617 is recovering from a syntax error, and 0 otherwise.
9618 Syntax error diagnostics are suppressed while recovering from a syntax
9621 @node Context Dependency
9622 @chapter Handling Context Dependencies
9624 The Bison paradigm is to parse tokens first, then group them into larger
9625 syntactic units. In many languages, the meaning of a token is affected by
9626 its context. Although this violates the Bison paradigm, certain techniques
9627 (known as @dfn{kludges}) may enable you to write Bison parsers for such
9631 * Semantic Tokens:: Token parsing can depend on the semantic context.
9632 * Lexical Tie-ins:: Token parsing can depend on the syntactic context.
9633 * Tie-in Recovery:: Lexical tie-ins have implications for how
9634 error recovery rules must be written.
9637 (Actually, ``kludge'' means any technique that gets its job done but is
9638 neither clean nor robust.)
9640 @node Semantic Tokens
9641 @section Semantic Info in Token Kinds
9643 The C language has a context dependency: the way an identifier is used
9644 depends on what its current meaning is. For example, consider this:
9650 This looks like a function call statement, but if @code{foo} is a typedef
9651 name, then this is actually a declaration of @code{x}. How can a Bison
9652 parser for C decide how to parse this input?
9654 The method used in GNU C is to have two different token kinds,
9655 @code{IDENTIFIER} and @code{TYPENAME}. When @code{yylex} finds an
9656 identifier, it looks up the current declaration of the identifier in order
9657 to decide which token kind to return: @code{TYPENAME} if the identifier is
9658 declared as a typedef, @code{IDENTIFIER} otherwise.
9660 The grammar rules can then express the context dependency by the choice of
9661 token kind to recognize. @code{IDENTIFIER} is accepted as an expression,
9662 but @code{TYPENAME} is not. @code{TYPENAME} can start a declaration, but
9663 @code{IDENTIFIER} cannot. In contexts where the meaning of the identifier
9664 is @emph{not} significant, such as in declarations that can shadow a
9665 typedef name, either @code{TYPENAME} or @code{IDENTIFIER} is
9666 accepted---there is one rule for each of the two token kinds.
9668 This technique is simple to use if the decision of which kinds of
9669 identifiers to allow is made at a place close to where the identifier is
9670 parsed. But in C this is not always so: C allows a declaration to
9671 redeclare a typedef name provided an explicit type has been specified
9675 typedef int foo, bar;
9679 static bar (bar); /* @r{redeclare @code{bar} as static variable} */
9680 extern foo foo (foo); /* @r{redeclare @code{foo} as function} */
9686 Unfortunately, the name being declared is separated from the declaration
9687 construct itself by a complicated syntactic structure---the ``declarator''.
9689 As a result, part of the Bison parser for C needs to be duplicated, with
9690 all the nonterminal names changed: once for parsing a declaration in
9691 which a typedef name can be redefined, and once for parsing a
9692 declaration in which that can't be done. Here is a part of the
9693 duplication, with actions omitted for brevity:
9698 declarator maybeasm '=' init
9699 | declarator maybeasm
9705 notype_declarator maybeasm '=' init
9706 | notype_declarator maybeasm
9712 Here @code{initdcl} can redeclare a typedef name, but @code{notype_initdcl}
9713 cannot. The distinction between @code{declarator} and
9714 @code{notype_declarator} is the same sort of thing.
9716 There is some similarity between this technique and a lexical tie-in
9717 (described next), in that information which alters the lexical analysis is
9718 changed during parsing by other parts of the program. The difference is
9719 here the information is global, and is used for other purposes in the
9720 program. A true lexical tie-in has a special-purpose flag controlled by
9721 the syntactic context.
9723 @node Lexical Tie-ins
9724 @section Lexical Tie-ins
9725 @cindex lexical tie-in
9727 One way to handle context-dependency is the @dfn{lexical tie-in}: a flag
9728 which is set by Bison actions, whose purpose is to alter the way tokens are
9731 For example, suppose we have a language vaguely like C, but with a special
9732 construct @samp{hex (@var{hex-expr})}. After the keyword @code{hex} comes
9733 an expression in parentheses in which all integers are hexadecimal. In
9734 particular, the token @samp{a1b} must be treated as an integer rather than
9735 as an identifier if it appears in that context. Here is how you can do it:
9742 void yyerror (char const *);
9751 | HEX '(' @{ hexflag = 1; @}
9752 expr ')' @{ hexflag = 0; $$ = $4; @}
9753 | expr '+' expr @{ $$ = make_sum ($1, $3); @}
9767 Here we assume that @code{yylex} looks at the value of @code{hexflag}; when
9768 it is nonzero, all integers are parsed in hexadecimal, and tokens starting
9769 with letters are parsed as integers if possible.
9771 The declaration of @code{hexflag} shown in the prologue of the grammar file
9772 is needed to make it accessible to the actions (@pxref{Prologue}). You must
9773 also write the code in @code{yylex} to obey the flag.
9775 @node Tie-in Recovery
9776 @section Lexical Tie-ins and Error Recovery
9778 Lexical tie-ins make strict demands on any error recovery rules you have.
9779 @xref{Error Recovery}.
9781 The reason for this is that the purpose of an error recovery rule is to
9782 abort the parsing of one construct and resume in some larger construct.
9783 For example, in C-like languages, a typical error recovery rule is to skip
9784 tokens until the next semicolon, and then start a new statement, like this:
9789 | IF '(' expr ')' stmt @{ @dots{} @}
9791 | error ';' @{ hexflag = 0; @}
9795 If there is a syntax error in the middle of a @samp{hex (@var{expr})}
9796 construct, this error rule will apply, and then the action for the
9797 completed @samp{hex (@var{expr})} will never run. So @code{hexflag} would
9798 remain set for the entire rest of the input, or until the next @code{hex}
9799 keyword, causing identifiers to be misinterpreted as integers.
9801 To avoid this problem the error recovery rule itself clears @code{hexflag}.
9803 There may also be an error recovery rule that works within expressions.
9804 For example, there could be a rule which applies within parentheses
9805 and skips to the close-parenthesis:
9811 | '(' expr ')' @{ $$ = $2; @}
9817 If this rule acts within the @code{hex} construct, it is not going to abort
9818 that construct (since it applies to an inner level of parentheses within
9819 the construct). Therefore, it should not clear the flag: the rest of
9820 the @code{hex} construct should be parsed with the flag still in effect.
9822 What if there is an error recovery rule which might abort out of the
9823 @code{hex} construct or might not, depending on circumstances? There is no
9824 way you can write the action to determine whether a @code{hex} construct is
9825 being aborted or not. So if you are using a lexical tie-in, you had better
9826 make sure your error recovery rules are not of this kind. Each rule must
9827 be such that you can be sure that it always will, or always won't, have to
9830 @c ================================================== Debugging Your Parser
9833 @chapter Debugging Your Parser
9835 Developing a parser can be a challenge, especially if you don't understand
9836 the algorithm (@pxref{Algorithm}). This chapter explains how to understand
9839 The most frequent issue users face is solving their conflicts. To fix them,
9840 the first step is understanding how they arise in a given grammar. This is
9841 made much easier by automated generation of counterexamples, cover in the
9842 first section (@pxref{Counterexamples}).
9844 In most cases though, looking at the structure of the automaton is still
9845 needed. The following sections explain how to generate and read the
9846 detailed structural description of the automaton. There are several formats
9850 as text, see @ref{Understanding};
9853 as a graph, see @ref{Graphviz};
9856 or as a markup report that can be turned, for instance, into HTML, see
9860 The last section focuses on the dynamic part of the parser: how to enable
9861 and understand the parser run-time traces (@pxref{Tracing}).
9864 * Counterexamples:: Understanding conflicts.
9865 * Understanding:: Understanding the structure of your parser.
9866 * Graphviz:: Getting a visual representation of the parser.
9867 * Xml:: Getting a markup representation of the parser.
9868 * Tracing:: Tracing the execution of your parser.
9871 @node Counterexamples
9872 @section Generation of Counterexamples
9874 Solving conflicts is probably the most delicate part of the design of an LR
9875 parser, as demonstrated by the number of sections devoted to them in this
9876 very documentation. To solve a conflict, one must understand it: when does
9877 it occur? Is it because of a flaw in the grammar? Is it rather because
9878 LR(1) cannot cope with this grammar?
9880 On difficulty is that conflicts occur in the @emph{automaton}, and it can be
9881 tricky to related them to issues in the @emph{grammar} itself. With
9882 experience and patience, analysis the detailed description of the automaton
9883 (@pxref{Understanding}) allows to find example strings that reach these conflicts.
9885 That task is made much easier thanks to the generation of counterexamples,
9886 initially developed by Chinawat Isradisaikul and Andrew Myers
9887 @pcite{Isradisaikul 2015}.
9889 As a first example, see the example grammar of @ref{Shift/Reduce}, which
9890 features on shift/reduce conflict:
9892 @c see doc/if-then-else.y
9894 $ @kbd{bison if-then-else.y}
9895 if-then-else.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
9896 if-then-else.y: @dnotice{note}: rerun with option '-Wcounterexamples' to generate conflict counterexamples
9900 Let's rerun @command{bison} with the option
9901 @option{-Wcex}/@option{-Wcounterexamples}@inlinefmt{info, (the following
9902 output is actually in color)}:
9906 if-then-else.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
9907 if-then-else.y: @dwarning{warning}: shift/reduce conflict on token "else" [@dwarning{-Wcounterexamples}]
9909 Example: @yellow{"if" expr "then"} "if" expr "then" stmt • "else" stmt
9912 @yellow{↳ "if" expr "then"} @green{stmt}
9913 @green{↳} @blue{if_stmt}
9914 @blue{↳ "if" expr "then" stmt} @red{•} @blue{"else" stmt}
9915 Example: @yellow{"if" expr "then"} @blue{"if" expr "then" stmt} @red{•} @yellow{"else" stmt}
9918 @yellow{↳ "if" expr "then"} @green{stmt} @yellow{"else" stmt}
9919 @green{↳} @blue{if_stmt}
9920 @blue{↳ "if" expr "then" stmt} @red{•}
9926 if-then-else.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
9927 if-then-else.y: @dwarning{warning}: shift/reduce conflict on token "else" [@dwarning{-Wcounterexamples}]
9929 Example: @yellow{"if" expr "then"} "if" expr "then" stmt • "else" stmt
9932 @yellow{@arrow{} "if" expr "then"} @green{stmt}
9933 @green{@arrow{}} @blue{if_stmt}
9934 @blue{@arrow{} "if" expr "then" stmt} @red{•} @blue{"else" stmt}
9935 Example: @yellow{"if" expr "then"} @blue{"if" expr "then" stmt} @red{•} @yellow{"else" stmt}
9938 @yellow{@arrow{} "if" expr "then"} @green{stmt} @yellow{"else" stmt}
9939 @green{@arrow{}} @blue{if_stmt}
9940 @blue{@arrow{} "if" expr "then" stmt} @red{•}
9945 This shows two different derivations for one single expression. That
9946 demonstrates that the grammar is ambiguous.
9950 As a more delicate example, consider the example grammar of
9951 @ref{Reduce/Reduce}, which features a reduce/reduce conflict:
9967 Bison generates the following counterexamples:
9972 $ @kbd{bison -Wcex sequence.y}
9973 sequence.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
9974 sequence.y: @dwarning{warning}: 2 reduce/reduce conflicts [@dwarning{-Wconflicts-rr}]
9977 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
9978 Example: @red{•} @green{"word"}
9981 @yellow{↳} @green{maybeword}
9982 @green{↳} @red{•} @green{"word"}
9983 Example: @red{•} @yellow{"word"}
9986 @yellow{↳} @green{sequence} @yellow{"word"}
9987 @green{↳} @red{•}
9990 sequence.y: @dwarning{warning}: reduce/reduce conflict on tokens $end, "word" [@dwarning{-Wcounterexamples}]
9992 First reduce derivation
9994 @yellow{↳} @red{•}
9996 Second reduce derivation
9998 @yellow{↳} @green{maybeword}
9999 @green{↳} @red{•}
10002 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10003 Example: @red{•} @green{"word"}
10006 @yellow{↳} @green{maybeword}
10007 @green{↳} @red{•} @green{"word"}
10008 Example: @red{•} @yellow{"word"}
10011 @yellow{↳} @green{sequence} @yellow{"word"}
10012 @green{↳} @blue{maybeword}
10013 @blue{↳} @red{•}
10016 sequence.y:8.3-45: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10017 8 | @dwarning{%empty @{ printf ("empty maybeword\n"); @}}
10018 | @dwarning{^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
10025 $ @kbd{bison -Wcex sequence.y}
10026 sequence.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10027 sequence.y: @dwarning{warning}: 2 reduce/reduce conflicts [@dwarning{-Wconflicts-rr}]
10030 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10031 Example: @red{•} @green{"word"}
10034 @yellow{@arrow{}} @green{maybeword}
10035 @green{@arrow{}} @red{•} @green{"word"}
10036 Example: @red{•} @yellow{"word"}
10039 @yellow{@arrow{}} @green{sequence} @yellow{"word"}
10040 @green{@arrow{}} @red{•}
10043 sequence.y: @dwarning{warning}: reduce/reduce conflict on tokens $end, "word" [@dwarning{-Wcounterexamples}]
10045 First reduce derivation
10047 @yellow{@arrow{}} @red{•}
10049 Second reduce derivation
10051 @yellow{@arrow{}} @green{maybeword}
10052 @green{@arrow{}} @red{•}
10055 sequence.y: @dwarning{warning}: shift/reduce conflict on token "word" [@dwarning{-Wcounterexamples}]
10056 Example: @red{•} @green{"word"}
10059 @yellow{@arrow{}} @green{maybeword}
10060 @green{@arrow{}} @red{•} @green{"word"}
10061 Example: @red{•} @yellow{"word"}
10064 @yellow{@arrow{}} @green{sequence} @yellow{"word"}
10065 @green{@arrow{}} @blue{maybeword}
10066 @blue{@arrow{}} @red{•}
10069 sequence.y:8.3-45: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10070 8 | @dwarning{%empty @{ printf ("empty maybeword\n"); @}}
10071 | @dwarning{^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
10076 Each of these three conflicts, again, prove that the grammar is ambiguous.
10077 For instance, the second conflict (the reduce/reduce one) shows that the
10078 grammar accept the empty input in two different ways.
10082 Sometimes, the search will not find an example that can be derived in two
10083 ways. In these cases, counterexample generation will provide two examples
10084 that are the same up until the dot. Most notably, this will happen when
10085 your grammar requires a stronger parser (more lookahead, LR instead of
10086 LALR). The following example isn't LR(1):
10094 expr: %empty | expr ID ','
10097 @command{bison} reports:
10101 ids.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10102 ids.y: @dwarning{warning}: shift/reduce conflict on token ID [@dwarning{-Wcounterexamples}]
10104 First example: @purple{expr} @red{•} @purple{ID ','} @green{ID} @yellow{$end}
10107 @yellow{↳} @green{s} @yellow{$end}
10108 @green{↳} @blue{a} @green{ID}
10109 @blue{↳} @purple{expr}
10110 @purple{↳ expr} @red{•} @purple{ID ','}
10111 Second example: @blue{expr} @red{•} @green{ID} @yellow{$end}
10114 @yellow{↳} @green{s} @yellow{$end}
10115 @green{↳} @blue{a} @green{ID}
10116 @blue{↳ expr} @red{•}
10119 ids.y:4.4-7: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10127 ids.y: @dwarning{warning}: 1 shift/reduce conflict [@dwarning{-Wconflicts-sr}]
10128 ids.y: @dwarning{warning}: shift/reduce conflict on token ID [@dwarning{-Wcounterexamples}]
10130 First example: @purple{expr} @red{•} @purple{ID ','} @green{ID} @yellow{$end}
10133 @yellow{@arrow{}} @green{s} @yellow{$end}
10134 @green{@arrow{}} @blue{a} @green{ID}
10135 @blue{@arrow{}} @purple{expr}
10136 @purple{@arrow{} expr} @red{•} @purple{ID ','}
10137 Second example: @blue{expr} @red{•} @green{ID} @yellow{$end}
10140 @yellow{@arrow{}} @green{s} @yellow{$end}
10141 @green{@arrow{}} @blue{a} @green{ID}
10142 @blue{@arrow{} expr} @red{•}
10145 ids.y:4.4-7: @dwarning{warning}: rule useless in parser due to conflicts [@dwarning{-Wother}]
10152 This conflict is caused by the parser not having enough information to know
10153 the difference between these two examples. The parser would need an
10154 additional lookahead token to know whether or not a comma follows the
10155 @code{ID} after @code{expr}. These types of conflicts tend to be more
10156 difficult to fix, and usually need a rework of the grammar. In this case,
10157 it can be fixed by changing around the recursion: @code{expr: ID | ',' expr
10160 Alternatively, you might also want to consider using a GLR parser
10161 (@pxref{GLR Parsers}).
10165 On occasions, it is useful to look at counterexamples @emph{in situ}: with
10166 the automaton report (@xref{Understanding}, in particular @ref{state-8,,
10169 @node Understanding
10170 @section Understanding Your Parser
10172 Bison parsers are @dfn{shift/reduce automata} (@pxref{Algorithm}). In some
10173 cases (much more frequent than one would hope), looking at this automaton is
10174 required to tune or simply fix a parser.
10176 The textual file is generated when the options @option{--report} or
10177 @option{--verbose} are specified, see @ref{Invocation}. Its name is made by
10178 removing @samp{.tab.c} or @samp{.c} from the parser implementation file
10179 name, and adding @samp{.output} instead. Therefore, if the grammar file is
10180 @file{foo.y}, then the parser implementation file is called @file{foo.tab.c}
10181 by default. As a consequence, the verbose output file is called
10184 The following grammar file, @file{calc.y}, will be used in the sequel:
10201 %nterm <sval> useless
10221 @command{bison} reports:
10224 calc.y: @dwarning{warning}: 1 nonterminal useless in grammar [@dwarning{-Wother}]
10225 calc.y: @dwarning{warning}: 1 rule useless in grammar [@dwarning{-Wother}]
10226 calc.y:19.1-7: @dwarning{warning}: nonterminal useless in grammar: useless [@dwarning{-Wother}]
10227 19 | @dwarning{useless: STR;}
10228 | @dwarning{^~~~~~~}
10229 calc.y: @dwarning{warning}: 7 shift/reduce conflicts [@dwarning{-Wconflicts-sr}]
10230 calc.y: @dnotice{note}: rerun with option '-Wcounterexamples' to generate conflict counterexamples
10233 Going back to the calc example, when given @option{--report=state},
10234 in addition to @file{calc.tab.c}, it creates a file @file{calc.output}
10235 with contents detailed below. The order of the output and the exact
10236 presentation might vary, but the interpretation is the same.
10239 @cindex token, useless
10240 @cindex useless token
10241 @cindex nonterminal, useless
10242 @cindex useless nonterminal
10243 @cindex rule, useless
10244 @cindex useless rule
10245 The first section reports useless tokens, nonterminals and rules. Useless
10246 nonterminals and rules are removed in order to produce a smaller parser, but
10247 useless tokens are preserved, since they might be used by the scanner (note
10248 the difference between ``useless'' and ``unused'' below):
10251 Nonterminals useless in grammar
10254 Terminals unused in grammar
10257 Rules useless in grammar
10262 The next section lists states that still have conflicts.
10265 State 8 conflicts: 1 shift/reduce
10266 State 9 conflicts: 1 shift/reduce
10267 State 10 conflicts: 1 shift/reduce
10268 State 11 conflicts: 4 shift/reduce
10272 Then Bison reproduces the exact grammar it used:
10277 0 $accept: exp $end
10287 and reports the uses of the symbols:
10291 Terminals, with rules where they appear
10304 Nonterminals, with rules where they appear
10310 on right: 0 1 2 3 4
10316 @cindex dotted rule
10317 @cindex rule, dotted
10318 Bison then proceeds onto the automaton itself, describing each state with
10319 its set of @dfn{items}, also known as @dfn{dotted rules}. Each item is a
10320 production rule together with a point (@samp{.}) marking the location of the
10326 0 $accept: • exp $end
10328 NUM shift, and go to state 1
10333 This reads as follows: ``state 0 corresponds to being at the very
10334 beginning of the parsing, in the initial rule, right before the start
10335 symbol (here, @code{exp}). When the parser returns to this state right
10336 after having reduced a rule that produced an @code{exp}, the control
10337 flow jumps to state 2. If there is no such transition on a nonterminal
10338 symbol, and the lookahead is a @code{NUM}, then this token is shifted onto
10339 the parse stack, and the control flow jumps to state 1. Any other
10340 lookahead triggers a syntax error.''
10342 @cindex core, item set
10343 @cindex item set core
10344 @cindex kernel, item set
10345 @cindex item set core
10346 Even though the only active rule in state 0 seems to be rule 0, the
10347 report lists @code{NUM} as a lookahead token because @code{NUM} can be
10348 at the beginning of any rule deriving an @code{exp}. By default Bison
10349 reports the so-called @dfn{core} or @dfn{kernel} of the item set, but if
10350 you want to see more detail you can invoke @command{bison} with
10351 @option{--report=itemset} to list the derived items as well:
10356 0 $accept: • exp $end
10357 1 exp: • exp '+' exp
10358 2 | • exp '-' exp
10359 3 | • exp '*' exp
10360 4 | • exp '/' exp
10363 NUM shift, and go to state 1
10369 In the state 1@dots{}
10376 $default reduce using rule 5 (exp)
10380 the rule 5, @samp{exp: NUM;}, is completed. Whatever the lookahead token
10381 (@samp{$default}), the parser will reduce it. If it was coming from State
10382 0, then, after this reduction it will return to state 0, and will jump to
10383 state 2 (@samp{exp: go to state 2}).
10388 0 $accept: exp • $end
10389 1 exp: exp • '+' exp
10390 2 | exp • '-' exp
10391 3 | exp • '*' exp
10392 4 | exp • '/' exp
10394 $end shift, and go to state 3
10395 '+' shift, and go to state 4
10396 '-' shift, and go to state 5
10397 '*' shift, and go to state 6
10398 '/' shift, and go to state 7
10402 In state 2, the automaton can only shift a symbol. For instance, because of
10403 the item @samp{exp: exp • '+' exp}, if the lookahead is @samp{+} it is
10404 shifted onto the parse stack, and the automaton jumps to state 4,
10405 corresponding to the item @samp{exp: exp '+' • exp}. Since there is no
10406 default action, any lookahead not listed triggers a syntax error.
10408 @cindex accepting state
10409 The state 3 is named the @dfn{final state}, or the @dfn{accepting
10415 0 $accept: exp $end •
10421 the initial rule is completed (the start symbol and the end-of-input were
10422 read), the parsing exits successfully.
10424 The interpretation of states 4 to 7 is straightforward, and is left to
10430 1 exp: exp '+' • exp
10432 NUM shift, and go to state 1
10439 2 exp: exp '-' • exp
10441 NUM shift, and go to state 1
10448 3 exp: exp '*' • exp
10450 NUM shift, and go to state 1
10457 4 exp: exp '/' • exp
10459 NUM shift, and go to state 1
10465 As was announced in beginning of the report, @samp{State 8 conflicts:
10471 1 exp: exp • '+' exp
10472 1 | exp '+' exp •
10473 2 | exp • '-' exp
10474 3 | exp • '*' exp
10475 4 | exp • '/' exp
10477 '*' shift, and go to state 6
10478 '/' shift, and go to state 7
10480 '/' [reduce using rule 1 (exp)]
10481 $default reduce using rule 1 (exp)
10484 Indeed, there are two actions associated to the lookahead @samp{/}:
10485 either shifting (and going to state 7), or reducing rule 1. The
10486 conflict means that either the grammar is ambiguous, or the parser lacks
10487 information to make the right decision. Indeed the grammar is
10488 ambiguous, as, since we did not specify the precedence of @samp{/}, the
10489 sentence @samp{NUM + NUM / NUM} can be parsed as @samp{NUM + (NUM /
10490 NUM)}, which corresponds to shifting @samp{/}, or as @samp{(NUM + NUM) /
10491 NUM}, which corresponds to reducing rule 1.
10493 Because in deterministic parsing a single decision can be made, Bison
10494 arbitrarily chose to disable the reduction, see @ref{Shift/Reduce}.
10495 Discarded actions are reported between square brackets.
10497 Note that all the previous states had a single possible action: either
10498 shifting the next token and going to the corresponding state, or
10499 reducing a single rule. In the other cases, i.e., when shifting
10500 @emph{and} reducing is possible or when @emph{several} reductions are
10501 possible, the lookahead is required to select the action. State 8 is
10502 one such state: if the lookahead is @samp{*} or @samp{/} then the action
10503 is shifting, otherwise the action is reducing rule 1. In other words,
10504 the first two items, corresponding to rule 1, are not eligible when the
10505 lookahead token is @samp{*}, since we specified that @samp{*} has higher
10506 precedence than @samp{+}. More generally, some items are eligible only
10507 with some set of possible lookahead tokens. When run with
10508 @option{--report=lookahead}, Bison specifies these lookahead tokens:
10513 1 exp: exp • '+' exp
10514 1 | exp '+' exp • [$end, '+', '-', '/']
10515 2 | exp • '-' exp
10516 3 | exp • '*' exp
10517 4 | exp • '/' exp
10519 '*' shift, and go to state 6
10520 '/' shift, and go to state 7
10522 '/' [reduce using rule 1 (exp)]
10523 $default reduce using rule 1 (exp)
10526 Note however that while @samp{NUM + NUM / NUM} is ambiguous (which results in
10527 the conflicts on @samp{/}), @samp{NUM + NUM * NUM} is not: the conflict was
10528 solved thanks to associativity and precedence directives. If invoked with
10529 @option{--report=solved}, Bison includes information about the solved
10530 conflicts in the report:
10533 Conflict between rule 1 and token '+' resolved as reduce (%left '+').
10534 Conflict between rule 1 and token '-' resolved as reduce (%left '-').
10535 Conflict between rule 1 and token '*' resolved as shift ('+' < '*').
10538 When given @option{--report=counterexamples}, @command{bison} will generate
10539 counterexamples within the report, augmented with the corresponding items
10540 (@pxref{Counterexamples}).
10544 shift/reduce conflict on token '/':
10545 1 exp: exp '+' exp •
10546 4 exp: exp • '/' exp
10548 Example: exp '+' exp • '/' exp
10552 ↳ exp • '/' exp
10553 Example: exp '+' exp • '/' exp
10557 ↳ exp '+' exp •
10563 shift/reduce conflict on token '/':
10564 1 exp: exp '+' exp •
10565 4 exp: exp • '/' exp
10567 Example: exp '+' exp • '/' exp
10570 @arrow{} exp '+' exp
10571 @arrow{} exp • '/' exp
10572 Example: exp '+' exp • '/' exp
10575 @arrow{} exp '/' exp
10576 @arrow{} exp '+' exp •
10581 This shows two separate derivations in the grammar for the same @code{exp}:
10582 @samp{e1 + e2 / e3}. The derivations show how your rules would parse the
10583 given example. Here, the first derivation completes a reduction when seeing
10584 @samp{/}, causing @samp{e1 + e2} to be grouped as an @code{exp}. The second
10585 derivation shifts on @samp{/}, resulting in @samp{e2 / e3} being grouped as
10586 an @code{exp}. Therefore, it is easy to see that adding @code{%precedence}
10587 directives would fix this conflict.
10589 The remaining states are similar:
10595 1 exp: exp • '+' exp
10596 2 | exp • '-' exp
10597 2 | exp '-' exp •
10598 3 | exp • '*' exp
10599 4 | exp • '/' exp
10601 '*' shift, and go to state 6
10602 '/' shift, and go to state 7
10604 '/' [reduce using rule 2 (exp)]
10605 $default reduce using rule 2 (exp)
10611 1 exp: exp • '+' exp
10612 2 | exp • '-' exp
10613 3 | exp • '*' exp
10614 3 | exp '*' exp •
10615 4 | exp • '/' exp
10617 '/' shift, and go to state 7
10619 '/' [reduce using rule 3 (exp)]
10620 $default reduce using rule 3 (exp)
10626 1 exp: exp • '+' exp
10627 2 | exp • '-' exp
10628 3 | exp • '*' exp
10629 4 | exp • '/' exp
10630 4 | exp '/' exp •
10632 '+' shift, and go to state 4
10633 '-' shift, and go to state 5
10634 '*' shift, and go to state 6
10635 '/' shift, and go to state 7
10637 '+' [reduce using rule 4 (exp)]
10638 '-' [reduce using rule 4 (exp)]
10639 '*' [reduce using rule 4 (exp)]
10640 '/' [reduce using rule 4 (exp)]
10641 $default reduce using rule 4 (exp)
10646 Observe that state 11 contains conflicts not only due to the lack of
10647 precedence of @samp{/} with respect to @samp{+}, @samp{-}, and @samp{*}, but
10648 also because the associativity of @samp{/} is not specified.
10650 Bison may also produce an HTML version of this output, via an XML file and
10651 XSLT processing (@pxref{Xml}).
10653 @c ================================================= Graphical Representation
10656 @section Visualizing Your Parser
10659 As another means to gain better understanding of the shift/reduce
10660 automaton corresponding to the Bison parser, a DOT file can be generated. Note
10661 that debugging a real grammar with this is tedious at best, and impractical
10662 most of the times, because the generated files are huge (the generation of
10663 a PDF or PNG file from it will take very long, and more often than not it will
10664 fail due to memory exhaustion). This option was rather designed for beginners,
10665 to help them understand LR parsers.
10667 This file is generated when the @option{--graph} option is specified
10668 (@pxref{Invocation}). Its name is made by removing
10669 @samp{.tab.c} or @samp{.c} from the parser implementation file name, and
10670 adding @samp{.gv} instead. If the grammar file is @file{foo.y}, the
10671 Graphviz output file is called @file{foo.gv}. A DOT file may also be
10672 produced via an XML file and XSLT processing (@pxref{Xml}).
10675 The following grammar file, @file{rr.y}, will be used in the sequel:
10680 exp: a ";" | b ".";
10686 The graphical output
10688 (see @ref{fig:graph})
10690 is very similar to the textual one, and as such it is easier understood by
10691 making direct comparisons between them. @xref{Debugging}, for a detailed
10692 analysis of the textual report.
10695 @float Figure,fig:graph
10696 @center @image{figs/example, 430pt,,,.svg}
10697 @caption{A graphical rendering of the parser.}
10701 @subheading Graphical Representation of States
10703 The items (dotted rules) for each state are grouped together in graph nodes.
10704 Their numbering is the same as in the verbose file. See the following
10705 points, about transitions, for examples
10707 When invoked with @option{--report=lookaheads}, the lookahead tokens, when
10708 needed, are shown next to the relevant rule between square brackets as a
10709 comma separated list. This is the case in the figure for the representation of
10714 The transitions are represented as directed edges between the current and
10717 @subheading Graphical Representation of Shifts
10719 Shifts are shown as solid arrows, labeled with the lookahead token for that
10720 shift. The following describes a reduction in the @file{rr.output} file:
10728 ";" shift, and go to state 6
10732 A Graphviz rendering of this portion of the graph could be:
10734 @center @image{figs/example-shift, 100pt,,,.svg}
10736 @subheading Graphical Representation of Reductions
10738 Reductions are shown as solid arrows, leading to a diamond-shaped node
10739 bearing the number of the reduction rule. The arrow is labeled with the
10740 appropriate comma separated lookahead tokens. If the reduction is the default
10741 action for the given state, there is no such label.
10743 This is how reductions are represented in the verbose file @file{rr.output}:
10750 "." reduce using rule 4 (b)
10751 $default reduce using rule 3 (a)
10754 A Graphviz rendering of this portion of the graph could be:
10756 @center @image{figs/example-reduce, 120pt,,,.svg}
10758 When unresolved conflicts are present, because in deterministic parsing
10759 a single decision can be made, Bison can arbitrarily choose to disable a
10760 reduction, see @ref{Shift/Reduce}. Discarded actions
10761 are distinguished by a red filling color on these nodes, just like how they are
10762 reported between square brackets in the verbose file.
10764 The reduction corresponding to the rule number 0 is the acceptation
10765 state. It is shown as a blue diamond, labeled ``Acc''.
10767 @subheading Graphical Representation of Gotos
10769 The @samp{go to} jump transitions are represented as dotted lines bearing
10770 the name of the rule being jumped to.
10772 @c ================================================= XML
10775 @section Visualizing your parser in multiple formats
10778 Bison supports two major report formats: textual output
10779 (@pxref{Understanding}) when invoked
10780 with option @option{--verbose}, and DOT
10781 (@pxref{Graphviz}) when invoked with
10782 option @option{--graph}. However,
10783 another alternative is to output an XML file that may then be, with
10784 @command{xsltproc}, rendered as either a raw text format equivalent to the
10785 verbose file, or as an HTML version of the same file, with clickable
10786 transitions, or even as a DOT. The @file{.output} and DOT files obtained via
10787 XSLT have no difference whatsoever with those obtained by invoking
10788 @command{bison} with options @option{--verbose} or @option{--graph}.
10790 The XML file is generated when the options @option{-x} or
10791 @option{--xml[=FILE]} are specified, see @ref{Invocation}.
10792 If not specified, its name is made by removing @samp{.tab.c} or @samp{.c}
10793 from the parser implementation file name, and adding @samp{.xml} instead.
10794 For instance, if the grammar file is @file{foo.y}, the default XML output
10795 file is @file{foo.xml}.
10797 Bison ships with a @file{data/xslt} directory, containing XSL Transformation
10798 files to apply to the XML file. Their names are non-ambiguous:
10802 Used to output a copy of the DOT visualization of the automaton.
10804 Used to output a copy of the @samp{.output} file.
10805 @item xml2xhtml.xsl
10806 Used to output an xhtml enhancement of the @samp{.output} file.
10809 Sample usage (requires @command{xsltproc}):
10811 $ @kbd{bison -x gr.y}
10813 $ @kbd{bison --print-datadir}
10814 /usr/local/share/bison
10816 $ @kbd{xsltproc /usr/local/share/bison/xslt/xml2xhtml.xsl gr.xml >gr.html}
10819 @c ================================================= Tracing
10822 @section Tracing Your Parser
10825 @cindex tracing the parser
10827 When a Bison grammar compiles properly but parses ``incorrectly'', the
10828 @code{yydebug} parser-trace feature helps figuring out why.
10831 * Enabling Traces:: Activating run-time trace support
10832 * Mfcalc Traces:: Extending @code{mfcalc} to support traces
10833 * The YYPRINT Macro:: Obsolete interface for semantic value reports
10836 @node Enabling Traces
10837 @subsection Enabling Traces
10838 There are several means to enable compilation of trace facilities, in
10839 decreasing order of preference:
10842 @item the variable @samp{parse.trace}
10843 @findex %define parse.trace
10844 Add the @samp{%define parse.trace} directive (@pxref{%define
10845 Summary}), or pass the @option{-Dparse.trace} option
10846 (@pxref{Tuning the Parser}). This is a Bison extension. Unless POSIX and
10847 Yacc portability matter to you, this is the preferred solution.
10849 @item the option @option{-t} (POSIX Yacc compliant)
10850 @itemx the option @option{--debug} (Bison extension)
10851 Use the @option{-t} option when you run Bison (@pxref{Invocation}). With
10852 @samp{%define api.prefix @{c@}}, it defines @code{CDEBUG} to 1, otherwise it
10853 defines @code{YYDEBUG} to 1.
10855 @item the directive @samp{%debug} (deprecated)
10857 Add the @code{%debug} directive (@pxref{Decl Summary}). This Bison
10858 extension is maintained for backward compatibility with previous versions of
10859 Bison; use @code{%define parse.trace} instead.
10861 @item the macro @code{YYDEBUG} (C/C++ only)
10863 Define the macro @code{YYDEBUG} to a nonzero value when you compile the
10864 parser. This is compliant with POSIX Yacc. You could use
10865 @option{-DYYDEBUG=1} as a compiler option or you could put @samp{#define
10866 YYDEBUG 1} in the prologue of the grammar file (@pxref{Prologue}).
10868 If the @code{%define} variable @code{api.prefix} is used (@pxref{Multiple
10869 Parsers}), for instance @samp{%define
10870 api.prefix @{c@}}, then if @code{CDEBUG} is defined, its value controls the
10871 tracing feature (enabled if and only if nonzero); otherwise tracing is
10872 enabled if and only if @code{YYDEBUG} is nonzero.
10875 We suggest that you always enable the trace option so that debugging is
10879 In C the trace facility outputs messages with macro calls of the form
10880 @code{YYFPRINTF (stderr, @var{format}, @var{args})} where @var{format} and
10881 @var{args} are the usual @code{printf} format and variadic arguments. If
10882 you define @code{YYDEBUG} to a nonzero value but do not define
10883 @code{YYFPRINTF}, @code{<stdio.h>} is automatically included and
10884 @code{YYFPRINTF} is defined to @code{fprintf}.
10886 Once you have compiled the program with trace facilities, the way to request
10887 a trace is to store a nonzero value in the variable @code{yydebug}. You can
10888 do this by making the C code do it (in @code{main}, perhaps), or you can
10889 alter the value with a C debugger.
10891 Each step taken by the parser when @code{yydebug} is nonzero produces a line
10892 or two of trace information, written on @code{stderr}. The trace messages
10893 tell you these things:
10897 Each time the parser calls @code{yylex}, what kind of token was read.
10900 Each time a token is shifted, the depth and complete contents of the state
10901 stack (@pxref{Parser States}).
10904 Each time a rule is reduced, which rule it is, and the complete contents of
10905 the state stack afterward.
10908 To make sense of this information, it helps to refer to the automaton
10909 description file (@pxref{Understanding}). This
10910 file shows the meaning of each state in terms of positions in various rules,
10911 and also what each state will do with each possible input token. As you
10912 read the successive trace messages, you can see that the parser is
10913 functioning according to its specification in the listing file. Eventually
10914 you will arrive at the place where something undesirable happens, and you
10915 will see which parts of the grammar are to blame.
10917 The parser implementation file is a C/C++/Java program and you can use
10918 debuggers on it, but it's not easy to interpret what it is doing. The
10919 parser function is a finite-state machine interpreter, and aside from the
10920 actions it executes the same code over and over. Only the values of
10921 variables show where in the grammar it is working.
10923 @node Mfcalc Traces
10924 @subsection Enabling Debug Traces for @code{mfcalc}
10926 The debugging information normally gives the token kind of each token read,
10927 but not its semantic value. The @code{%printer} directive allows specify
10928 how semantic values are reported, see @ref{Printer Decl}.
10930 As a demonstration of @code{%printer}, consider the multi-function
10931 calculator, @code{mfcalc} (@pxref{Multi-function Calc}). To enable run-time
10932 traces, and semantic value reports, insert the following directives in its
10935 @comment file: mfcalc.y: 2
10937 /* Generate the parser description file. */
10939 /* Enable run-time traces (yydebug). */
10940 %define parse.trace
10942 /* Formatting semantic values. */
10943 %printer @{ fprintf (yyo, "%s", $$->name); @} VAR;
10944 %printer @{ fprintf (yyo, "%s()", $$->name); @} FUN;
10945 %printer @{ fprintf (yyo, "%g", $$); @} <double>;
10948 The @code{%define} directive instructs Bison to generate run-time trace
10949 support. Then, activation of these traces is controlled at run-time by the
10950 @code{yydebug} variable, which is disabled by default. Because these traces
10951 will refer to the ``states'' of the parser, it is helpful to ask for the
10952 creation of a description of that parser; this is the purpose of (admittedly
10953 ill-named) @code{%verbose} directive.
10955 The set of @code{%printer} directives demonstrates how to format the
10956 semantic value in the traces. Note that the specification can be done
10957 either on the symbol type (e.g., @code{VAR} or @code{FUN}), or on the type
10958 tag: since @code{<double>} is the type for both @code{NUM} and @code{exp},
10959 this printer will be used for them.
10961 Here is a sample of the information provided by run-time traces. The traces
10962 are sent onto standard error.
10965 $ @kbd{echo 'sin(1-1)' | ./mfcalc -p}
10968 Reducing stack by rule 1 (line 34):
10969 -> $$ = nterm input ()
10975 This first batch shows a specific feature of this grammar: the first rule
10976 (which is in line 34 of @file{mfcalc.y} can be reduced without even having
10977 to look for the first token. The resulting left-hand symbol (@code{$$}) is
10978 a valueless (@samp{()}) @code{input} nonterminal (@code{nterm}).
10980 Then the parser calls the scanner.
10983 Next token is token FUN (sin())
10984 Shifting token FUN (sin())
10989 That token (@code{token}) is a function (@code{FUN}) whose value is
10990 @samp{sin} as formatted per our @code{%printer} specification: @samp{sin()}.
10991 The parser stores (@code{Shifting}) that token, and others, until it can do
10992 something about it.
10996 Next token is token '(' ()
10997 Shifting token '(' ()
11000 Next token is token NUM (1.000000)
11001 Shifting token NUM (1.000000)
11003 Reducing stack by rule 6 (line 44):
11004 $1 = token NUM (1.000000)
11005 -> $$ = nterm exp (1.000000)
11011 The previous reduction demonstrates the @code{%printer} directive for
11012 @code{<double>}: both the token @code{NUM} and the resulting nonterminal
11013 @code{exp} have @samp{1} as value.
11017 Next token is token '-' ()
11018 Shifting token '-' ()
11021 Next token is token NUM (1.000000)
11022 Shifting token NUM (1.000000)
11024 Reducing stack by rule 6 (line 44):
11025 $1 = token NUM (1.000000)
11026 -> $$ = nterm exp (1.000000)
11027 Stack now 0 1 6 14 24 17
11030 Next token is token ')' ()
11031 Reducing stack by rule 11 (line 49):
11032 $1 = nterm exp (1.000000)
11034 $3 = nterm exp (1.000000)
11035 -> $$ = nterm exp (0.000000)
11041 The rule for the subtraction was just reduced. The parser is about to
11042 discover the end of the call to @code{sin}.
11045 Next token is token ')' ()
11046 Shifting token ')' ()
11048 Reducing stack by rule 9 (line 47):
11049 $1 = token FUN (sin())
11051 $3 = nterm exp (0.000000)
11053 -> $$ = nterm exp (0.000000)
11059 Finally, the end-of-line allow the parser to complete the computation, and
11060 display its result.
11064 Next token is token '\n' ()
11065 Shifting token '\n' ()
11067 Reducing stack by rule 4 (line 40):
11068 $1 = nterm exp (0.000000)
11071 -> $$ = nterm line ()
11074 Reducing stack by rule 2 (line 35):
11075 $1 = nterm input ()
11077 -> $$ = nterm input ()
11082 The parser has returned into state 1, in which it is waiting for the next
11083 expression to evaluate, or for the end-of-file token, which causes the
11084 completion of the parsing.
11088 Now at end of input.
11089 Shifting token $end ()
11092 Cleanup: popping token $end ()
11093 Cleanup: popping nterm input ()
11097 @node The YYPRINT Macro
11098 @subsection The @code{YYPRINT} Macro
11101 The @code{%printer} directive was introduced in Bison 1.50 (November 2002).
11102 Before then, @code{YYPRINT} provided a similar feature, but only for
11103 terminal symbols and only with the @file{yacc.c} skeleton.
11105 @deffn {Macro} YYPRINT (@var{stream}, @var{token}, @var{value});
11107 Deprecated, will be removed eventually.
11109 If you define @code{YYPRINT}, it should take three arguments. The parser
11110 will pass a standard I/O stream, the numeric code for the token kind, and
11111 the token value (from @code{yylval}).
11113 For @file{yacc.c} only. Obsoleted by @code{%printer}.
11116 Here is an example of @code{YYPRINT} suitable for the multi-function
11117 calculator (@pxref{Mfcalc Declarations}):
11121 static void print_token_value (FILE *file, int type, YYSTYPE value);
11122 #define YYPRINT(File, Type, Value) \
11123 print_token_value (File, Type, Value)
11126 @dots{} %% @dots{} %% @dots{}
11129 print_token_value (FILE *file, yytoken_kind_t kind, YYSTYPE value)
11132 fprintf (file, "%s", value.tptr->name);
11133 else if (kind == NUM)
11134 fprintf (file, "%d", value.val);
11138 @xref{Mfcalc Traces}, for the proper use of @code{%printer}.
11140 @c ================================================= Invoking Bison
11143 @chapter Invoking Bison
11144 @cindex invoking Bison
11145 @cindex Bison invocation
11146 @cindex options for invoking Bison
11148 The usual way to invoke Bison is as follows:
11151 $ @kbd{bison @var{file}}
11154 Here @var{file} is the grammar file name, which usually ends in @samp{.y}.
11155 The parser implementation file's name is made by replacing the @samp{.y}
11156 with @samp{.tab.c} and removing any leading directory. Thus, the
11157 @samp{bison foo.y} file name yields @file{foo.tab.c}, and the @samp{bison
11158 hack/foo.y} file name yields @file{foo.tab.c}. It's also possible, in case
11159 you are writing C++ code instead of C in your grammar file, to name it
11160 @file{foo.ypp} or @file{foo.y++}. Then, the output files will take an
11161 extension like the given one as input (respectively @file{foo.tab.cpp} and
11162 @file{foo.tab.c++}). This feature takes effect with all options that
11163 manipulate file names like @option{-o} or @option{-d}.
11168 $ @kbd{bison -d @var{file.yxx}}
11171 will produce @file{file.tab.cxx} and @file{file.tab.hxx}, and
11174 $ @kbd{bison -d -o @var{output.c++} @var{file.y}}
11177 will produce @file{output.c++} and @file{output.h++}.
11179 For compatibility with POSIX, the standard Bison distribution also contains
11180 a shell script called @command{yacc} that invokes Bison with the @option{-y}
11185 The exit status of @command{bison} is:
11188 when there were no errors. Warnings, which are diagnostics about dubious
11189 constructs, do not change the exit status, unless they are turned into
11190 errors (@pxref{Werror,,@option{-Werror}}).
11193 when there were errors. No file was generated (except the reports generated
11194 by @option{--verbose}, etc.). In particular, the output files that possibly
11195 existed were not changed.
11197 @item 63 (mismatch)
11198 when @command{bison} does not meet the version requirements of the grammar
11199 file. @xref{Require Decl}. No file was generated or changed.
11204 * Bison Options:: All the options described in detail,
11205 in alphabetical order by short options.
11206 * Option Cross Key:: Alphabetical list of long options.
11207 * Yacc Library:: Yacc-compatible @code{yylex} and @code{main}.
11210 @node Bison Options
11211 @section Bison Options
11213 Bison supports both traditional single-letter options and mnemonic long
11214 option names. Long option names are indicated with @option{--} instead of
11215 @option{-}. Abbreviations for option names are allowed as long as they
11216 are unique. When a long option takes an argument, like
11217 @option{--file-prefix}, connect the option name and the argument with
11220 Here is a list of options that can be used with Bison. It is followed by a
11221 cross key alphabetized by long option.
11224 * Operation Modes:: Options controlling the global behavior of @command{bison}
11225 * Diagnostics:: Options controlling the diagnostics
11226 * Tuning the Parser:: Options changing the generated parsers
11227 * Output Files:: Options controlling the output
11230 @node Operation Modes
11231 @subsection Operation Modes
11233 Options controlling the global behavior of @command{bison}.
11235 @c Please, keep this ordered as in 'bison --help'.
11239 Print a summary of the command-line options to Bison and exit.
11243 Print the version number of Bison and exit.
11245 @item --print-localedir
11246 Print the name of the directory containing locale-dependent data.
11248 @item --print-datadir
11249 Print the name of the directory containing skeletons, CSS and XSLT.
11253 Update the grammar file (remove duplicates, update deprecated directives,
11254 etc.) and exit (i.e., do not generate any of the output files). Leaves a
11255 backup of the original file with a @code{~} appended. For instance:
11261 %define parse.error verbose
11266 $ @kbd{bison -u foo.y}
11267 foo.y:1.1-14: @dwarning{warning}: deprecated directive, use '%define parse.error verbose' [@dwarning{-Wdeprecated}]
11268 1 | @dwarning{%error-verbose}
11269 | @dwarning{^~~~~~~~~~~~~~}
11270 foo.y:2.1-27: @dwarning{warning}: %define variable 'parse.error' redefined [@dwarning{-Wother}]
11271 2 | @dwarning{%define parse.error verbose}
11272 | @dwarning{^~~~~~~~~~~~~~~~~~~~~~~~~~~}
11273 foo.y:1.1-14: previous definition
11274 1 | @dnotice{%error-verbose}
11275 | @dnotice{^~~~~~~~~~~~~~}
11276 bison: file 'foo.y' was updated (backup: 'foo.y~')
11280 %define parse.error verbose
11286 See the documentation of @option{--feature=fixit} below for more details.
11288 @item -f [@var{feature}]
11289 @itemx --feature[=@var{feature}]
11290 Activate miscellaneous @var{feature}s. @var{Feature} can be one of:
11293 @itemx diagnostics-show-caret
11294 Show caret errors, in a manner similar to GCC's
11295 @option{-fdiagnostics-show-caret}, or Clang's
11296 @option{-fcaret-diagnostics}. The location provided with the message is used
11297 to quote the corresponding line of the source file, underlining the
11298 important part of it with carets (@samp{^}). Here is an example, using the
11299 following file @file{in.y}:
11304 exp: exp '+' exp @{ $exp = $1 + $2; @};
11307 When invoked with @option{-fcaret} (or nothing), Bison will report:
11311 in.y:3.20-23: @derror{error}: ambiguous reference: '$exp'
11312 3 | exp: exp '+' exp @{ @derror{$exp} = $1 + $2; @};
11316 in.y:3.1-3: refers to: $exp at $$
11317 3 | @dnotice{exp}: exp '+' exp @{ $exp = $1 + $2; @};
11321 in.y:3.6-8: refers to: $exp at $1
11322 3 | exp: @dnotice{exp} '+' exp @{ $exp = $1 + $2; @};
11326 in.y:3.14-16: refers to: $exp at $3
11327 3 | exp: exp '+' @dnotice{exp} @{ $exp = $1 + $2; @};
11331 in.y:3.32-33: @derror{error}: $2 of 'exp' has no declared type
11332 3 | exp: exp '+' exp @{ $exp = $1 + @derror{$2}; @};
11337 Whereas, when invoked with @option{-fno-caret}, Bison will only report:
11341 in.y:3.20-23: @derror{error}: ambiguous reference: '$exp'
11342 in.y:3.1-3: refers to: $exp at $$
11343 in.y:3.6-8: refers to: $exp at $1
11344 in.y:3.14-16: refers to: $exp at $3
11345 in.y:3.32-33: @derror{error}: $2 of 'exp' has no declared type
11349 This option is activated by default.
11352 @itemx diagnostics-parseable-fixits
11353 Show machine-readable fixes, in a manner similar to GCC's and Clang's
11354 @option{-fdiagnostics-parseable-fixits}.
11356 Fix-its are generated for duplicate directives:
11361 %define api.prefix @{foo@}
11362 %define api.prefix @{bar@}
11368 $ @kbd{bison -ffixit foo.y}
11369 foo.y:2.1-24: @derror{error}: %define variable 'api.prefix' redefined
11370 2 | @derror{%define api.prefix @{bar@}}
11371 | @derror{^~~~~~~~~~~~~~~~~~~~~~~~}
11372 foo.y:1.1-24: previous definition
11373 1 | @dnotice{%define api.prefix @{foo@}}
11374 | @dnotice{^~~~~~~~~~~~~~~~~~~~~~~~}
11375 fix-it:"foo.y":@{2:1-2:25@}:""
11376 foo.y: @dwarning{warning}: fix-its can be applied. Rerun with option '--update'. [@dwarning{-Wother}]
11380 They are also generated to update deprecated directives, unless
11381 @option{-Wno-deprecated} was given:
11385 $ @kbd{cat /tmp/foo.yy}
11392 $ @kbd{bison foo.y}
11393 foo.y:1.1-14: @dwarning{warning}: deprecated directive, use '%define parse.error verbose' [@dwarning{-Wdeprecated}]
11394 1 | @dwarning{%error-verbose}
11395 | @dwarning{^~~~~~~~~~~~~~}
11396 foo.y:2.1-18: @dwarning{warning}: deprecated directive, use '%define api.prefix @{foo@}' [@dwarning{-Wdeprecated}]
11397 2 | @dwarning{%name-prefix "foo"}
11398 | @dwarning{^~~~~~~~~~~~~~~~~~}
11399 foo.y: @dwarning{warning}: fix-its can be applied. Rerun with option '--update'. [@dwarning{-Wother}]
11403 The fix-its are applied by @command{bison} itself when given the option
11404 @option{-u}/@option{--update}. See its documentation above.
11407 Do not generate the output files. The name of this feature is somewhat
11408 misleading as more than just checking the syntax is done: every stage is run
11409 (including checking for conflicts for instance), except the generation of
11416 @subsection Diagnostics
11418 Options controlling the diagnostics.
11420 @c Please, keep this ordered as in 'bison --help'.
11422 @item -W [@var{category}]
11423 @itemx --warnings[=@var{category}]
11424 Output warnings falling in @var{category}. @var{category} can be one
11427 @item @anchor{Wconflicts-sr}conflicts-sr
11428 @itemx @anchor{Wconflicts-rr}conflicts-rr
11429 S/R and R/R conflicts. These warnings are enabled by default. However, if
11430 the @code{%expect} or @code{%expect-rr} directive is specified, an
11431 unexpected number of conflicts is an error, and an expected number of
11432 conflicts is not reported, so @option{-W} and @option{--warning} then have
11433 no effect on the conflict report.
11435 @item @anchor{Wcounterexamples}counterexamples
11437 Provide counterexamples for conflicts. @xref{Counterexamples}.
11438 Counterexamples take time to compute. The option @option{-Wcex} should be
11439 used by the developer when working on the grammar; it hardly makes sense to
11442 @item @anchor{Wdangling-alias}dangling-alias
11443 Report string literals that are not bound to a token symbol.
11445 String literals, which allow for better error messages, are (too) liberally
11446 accepted by Bison, which might result in silent errors. For instance
11449 %type <exVal> cond "condition"
11453 does not define ``condition'' as a string alias to @code{cond}---nonterminal
11454 symbols do not have string aliases. It is rather equivalent to
11457 %nterm <exVal> cond
11458 %token <exVal> "condition"
11462 i.e., it gives the @samp{"condition"} token the type @code{exVal}.
11464 Also, because string aliases do not need to be defined, typos such as
11465 @samp{"baz"} instead of @samp{"bar"} will be not reported.
11467 The option @option{-Wdangling-alias} catches these situations. On
11471 %type <ival> foo "foo"
11477 @command{bison -Wdangling-alias} reports
11480 @dwarning{warning}: string literal not attached to a symbol
11481 | %type <ival> foo @dwarning{"foo"}
11483 @dwarning{warning}: string literal not attached to a symbol
11484 | foo: @dwarning{"baz"} @{@}
11488 @item @anchor{Wdeprecated}deprecated
11489 Deprecated constructs whose support will be removed in future versions of
11492 @item @anchor{Wempty-rule}empty-rule
11493 Empty rules without @code{%empty}. @xref{Empty Rules}. Disabled by
11494 default, but enabled by uses of @code{%empty}, unless
11495 @option{-Wno-empty-rule} was specified.
11497 @item @anchor{Wmidrule-values}midrule-values
11498 Warn about midrule values that are set but not used within any of the actions
11499 of the parent rule.
11500 For example, warn about unused @code{$2} in:
11503 exp: '1' @{ $$ = 1; @} '+' exp @{ $$ = $1 + $4; @};
11506 Also warn about midrule values that are used but not set.
11507 For example, warn about unset @code{$$} in the midrule action in:
11510 exp: '1' @{ $1 = 1; @} '+' exp @{ $$ = $2 + $4; @};
11513 These warnings are not enabled by default since they sometimes prove to
11514 be false alarms in existing grammars employing the Yacc constructs
11515 @code{$0} or @code{$-@var{n}} (where @var{n} is some positive integer).
11517 @item @anchor{Wprecedence}precedence
11518 Useless precedence and associativity directives. Disabled by default.
11520 Consider for instance the following grammar:
11549 @c cannot leave the location and the [-Wprecedence] for lack of
11553 @dwarning{warning}: useless precedence and associativity for "="
11554 | %nonassoc @dwarning{"="}
11558 @dwarning{warning}: useless associativity for "*", use %precedence
11559 | %left @dwarning{"*"}
11563 @dwarning{warning}: useless precedence for "("
11564 | %precedence @dwarning{"("}
11569 One would get the exact same parser with the following directives instead:
11578 @item @anchor{Wyacc}yacc
11579 Incompatibilities with POSIX Yacc.
11581 @item @anchor{Wother}other
11582 All warnings not categorized above. These warnings are enabled by default.
11584 This category is provided merely for the sake of completeness. Future
11585 releases of Bison may move warnings from this category to new, more specific
11588 @item @anchor{Wall}all
11589 All the warnings except @code{dangling-alias} and @code{yacc}.
11591 @item @anchor{Wnone}none
11592 Turn off all the warnings.
11595 See @option{-Werror}, below.
11598 A category can be turned off by prefixing its name with @samp{no-}. For
11599 instance, @option{-Wno-yacc} will hide the warnings about
11600 POSIX Yacc incompatibilities.
11602 @item @anchor{Werror}-Werror
11603 Turn enabled warnings for every @var{category} into errors, unless they are
11604 explicitly disabled by @option{-Wno-error=@var{category}}.
11606 @item -Werror=@var{category}
11607 Enable warnings falling in @var{category}, and treat them as errors.
11609 @var{category} is the same as for @option{--warnings}, with the exception that
11610 it may not be prefixed with @samp{no-} (see above).
11612 Note that the precedence of the @samp{=} and @samp{,} operators is such that
11613 the following commands are @emph{not} equivalent, as the first will not treat
11614 S/R conflicts as errors.
11617 $ @kbd{bison -Werror=yacc,conflicts-sr input.y}
11618 $ @kbd{bison -Werror=yacc,error=conflicts-sr input.y}
11622 Do not turn enabled warnings for every @var{category} into errors, unless
11623 they are explicitly enabled by @option{-Werror=@var{category}}.
11625 @item -Wno-error=@var{category}
11626 Deactivate the error treatment for this @var{category}. However, the warning
11627 itself won't be disabled, or enabled, by this option.
11630 Equivalent to @option{--color=always}.
11632 @item --color=@var{when}
11633 Control whether diagnostics are colorized, depending on @var{when}:
11637 Enable colorized diagnostics.
11641 Disable colorized diagnostics.
11643 @item auto @r{(default)}
11645 Diagnostics will be colorized if the output device is a tty, i.e. when the
11646 output goes directly to a text screen or terminal emulator window.
11649 @item --style=@var{file}
11650 Specifies the CSS style @var{file} to use when colorizing. It has an effect
11651 only when the @option{--color} option is effective. The
11652 @file{bison-default.css} file provide a good example from which to define
11653 your own style file. See the documentation of libtextstyle for more
11657 @node Tuning the Parser
11658 @subsection Tuning the Parser
11660 Options changing the generated parsers.
11662 @c Please, keep this ordered as in 'bison --help'.
11666 In the parser implementation file, define the macro @code{YYDEBUG} to 1 if
11667 it is not already defined, so that the debugging facilities are compiled.
11670 @item -D @var{name}[=@var{value}]
11671 @itemx --define=@var{name}[=@var{value}]
11672 @itemx -F @var{name}[=@var{value}]
11673 @itemx --force-define=@var{name}[=@var{value}]
11674 Each of these is equivalent to @samp{%define @var{name} @var{value}}
11675 (@pxref{%define Summary}). Note that the delimiters are part of
11676 @var{value}: @option{-Dapi.value.type=union},
11677 @option{-Dapi.value.type=@{union@}} and @option{-Dapi.value.type="union"}
11678 correspond to @samp{%define api.value.type union}, @samp{%define
11679 api.value.type @{union@}} and @samp{%define api.value.type "union"}.
11681 Bison processes multiple definitions for the same @var{name} as follows:
11685 Bison quietly ignores all command-line definitions for @var{name} except
11688 If that command-line definition is specified by a @option{-D} or
11689 @option{--define}, Bison reports an error for any @code{%define} definition
11692 If that command-line definition is specified by a @option{-F} or
11693 @option{--force-define} instead, Bison quietly ignores all @code{%define}
11694 definitions for @var{name}.
11696 Otherwise, Bison reports an error if there are multiple @code{%define}
11697 definitions for @var{name}.
11700 You should avoid using @option{-F} and @option{--force-define} in your
11701 make files unless you are confident that it is safe to quietly ignore
11702 any conflicting @code{%define} that may be added to the grammar file.
11704 @item -L @var{language}
11705 @itemx --language=@var{language}
11706 Specify the programming language for the generated parser, as if
11707 @code{%language} was specified (@pxref{Decl Summary}). Currently supported
11708 languages include C, C++, and Java. @var{language} is case-insensitive.
11711 Pretend that @code{%locations} was specified. @xref{Decl Summary}.
11713 @item -p @var{prefix}
11714 @itemx --name-prefix=@var{prefix}
11715 Pretend that @code{%name-prefix "@var{prefix}"} was specified (@pxref{Decl
11716 Summary}). Obsoleted by @option{-Dapi.prefix=@var{prefix}}. @xref{Multiple
11721 Don't put any @code{#line} preprocessor commands in the parser
11722 implementation file. Ordinarily Bison puts them in the parser
11723 implementation file so that the C compiler and debuggers will
11724 associate errors with your source file, the grammar file. This option
11725 causes them to associate errors with the parser implementation file,
11726 treating it as an independent source file in its own right.
11728 @item -S @var{file}
11729 @itemx --skeleton=@var{file}
11730 Specify the skeleton to use, similar to @code{%skeleton}
11731 (@pxref{Decl Summary}).
11733 @c You probably don't need this option unless you are developing Bison.
11734 @c You should use @option{--language} if you want to specify the skeleton for a
11735 @c different language, because it is clearer and because it will always
11736 @c choose the correct skeleton for non-deterministic or push parsers.
11738 If @var{file} does not contain a @code{/}, @var{file} is the name of a skeleton
11739 file in the Bison installation directory.
11740 If it does, @var{file} is an absolute file name or a file name relative to the
11741 current working directory.
11742 This is similar to how most shells resolve commands.
11745 @itemx --token-table
11746 Pretend that @code{%token-table} was specified. @xref{Decl Summary}.
11750 Act more like the traditional @command{yacc} command. This can cause
11751 different diagnostics to be generated (it implies @option{-Wyacc}), and may
11752 change behavior in other minor ways. Most importantly, imitate Yacc's
11753 output file name conventions, so that the parser implementation file is
11754 called @file{y.tab.c}, and the other outputs are called @file{y.output} and
11755 @file{y.tab.h}. Also, generate @code{#define} statements in addition to an
11756 @code{enum} to associate token codes with token kind names. Thus, the
11757 following shell script can substitute for Yacc, and the Bison distribution
11758 contains such a script for compatibility with POSIX:
11765 The @option{-y}/@option{--yacc} option is intended for use with traditional
11766 Yacc grammars. This option only makes sense for the default C skeleton,
11767 @file{yacc.c}. If your grammar uses Bison extensions Bison cannot be
11768 Yacc-compatible, even if this option is specified.
11772 @subsection Output Files
11774 Options controlling the output.
11776 @c Please, keep this ordered as in 'bison --help'.
11778 @item --defines[=@var{file}]
11779 Pretend that @code{%defines} was specified, i.e., write an extra output file
11780 containing definitions for the token kind names defined in the grammar, as
11781 well as a few other declarations. @xref{Decl Summary}.
11784 This is the same as @option{--defines} except @option{-d} does not accept a
11785 @var{file} argument since POSIX Yacc requires that @option{-d} can be bundled
11786 with other short options.
11788 @item -b @var{file-prefix}
11789 @itemx --file-prefix=@var{prefix}
11790 Pretend that @code{%file-prefix} was specified, i.e., specify prefix to use
11791 for all Bison output file names. @xref{Decl Summary}.
11793 @item -r @var{things}
11794 @itemx --report=@var{things}
11795 Write an extra output file containing verbose description of the comma
11796 separated list of @var{things} among:
11800 Description of the grammar, conflicts (resolved and unresolved), and
11801 parser's automaton.
11804 Implies @code{state} and augments the description of the automaton with
11805 the full set of items for each state, instead of its core only.
11808 Implies @code{state} and augments the description of the automaton with
11809 each rule's lookahead set.
11812 Implies @code{state}. Explain how conflicts were solved thanks to
11813 precedence and associativity directives.
11815 @item counterexamples
11817 Look for counterexamples for the conflicts. @xref{Counterexamples}.
11818 Counterexamples take time to compute. The option @option{-rcex} should be
11819 used by the developer when working on the grammar; it hardly makes sense to
11823 Enable all the items.
11826 Do not generate the report.
11829 @item --report-file=@var{file}
11830 Specify the @var{file} for the verbose description.
11834 Pretend that @code{%verbose} was specified, i.e., write an extra output
11835 file containing verbose descriptions of the grammar and
11836 parser. @xref{Decl Summary}.
11838 @item -o @var{file}
11839 @itemx --output=@var{file}
11840 Specify the @var{file} for the parser implementation file.
11842 The names of the other output files are constructed from @var{file} as
11843 described under the @option{-v} and @option{-d} options.
11845 @item -g [@var{file}]
11846 @itemx --graph[=@var{file}]
11847 Output a graphical representation of the parser's automaton computed by
11848 Bison, in @uref{http://www.graphviz.org/, Graphviz}
11849 @uref{http://www.graphviz.org/doc/info/lang.html, DOT} format.
11850 @code{@var{file}} is optional. If omitted and the grammar file is
11851 @file{foo.y}, the output file will be @file{foo.gv} if the @code{%required}
11852 version is 3.4 or better, @file{foo.dot} otherwise.
11854 @item -x [@var{file}]
11855 @itemx --xml[=@var{file}]
11856 Output an XML report of the parser's automaton computed by Bison.
11857 @code{@var{file}} is optional.
11858 If omitted and the grammar file is @file{foo.y}, the output file will be
11861 @item -M @var{old}=@var{new}
11862 @itemx --file-prefix-map=@var{old}=@var{new}
11863 Replace prefix @var{old} with @var{new} when writing file paths in output files
11866 @node Option Cross Key
11867 @section Option Cross Key
11869 Here is a list of options, alphabetized by long option, to help you find
11870 the corresponding short option and directive.
11872 @multitable {@option{--force-define=@var{name}[=@var{value}]}} {@option{-F @var{name}[=@var{value}]}} {@code{%nondeterministic-parser}}
11873 @headitem Long Option @tab Short Option @tab Bison Directive
11874 @include cross-options.texi
11878 @section Yacc Library
11880 The Yacc library contains default implementations of the @code{yyerror} and
11881 @code{main} functions. These default implementations are normally not
11882 useful, but POSIX requires them. To use the Yacc library, link your program
11883 with the @option{-ly} option. Note that Bison's implementation of the Yacc
11884 library is distributed under the terms of the GNU General Public License
11887 If you use the Yacc library's @code{yyerror} function, you should declare
11888 @code{yyerror} as follows:
11891 int yyerror (char const *);
11895 The @code{int} value returned by this @code{yyerror} is ignored.
11897 The implementation of Yacc library's @code{main} function is:
11902 setlocale (LC_ALL, "");
11908 so if you use it, the internationalization support is enabled (e.g., error
11909 messages are translated), and your @code{yyparse} function should have the
11910 following type signature:
11913 int yyparse (void);
11916 @c ================================================= C++ Bison
11918 @node Other Languages
11919 @chapter Parsers Written In Other Languages
11921 In addition to C, Bison can generate parsers in C++ and Java. This chapter
11922 is devoted to these languages. The reader is expected to understand how
11923 Bison works; read the introductory chapters first if you don't.
11926 * C++ Parsers:: The interface to generate C++ parser classes
11927 * Java Parsers:: The interface to generate Java parser classes
11931 @section C++ Parsers
11933 The Bison parser in C++ is an object, an instance of the class
11937 * A Simple C++ Example:: A short introduction to C++ parsers
11938 * C++ Bison Interface:: Asking for C++ parser generation
11939 * C++ Parser Interface:: Instantiating and running the parser
11940 * C++ Semantic Values:: %union vs. C++
11941 * C++ Location Values:: The position and location classes
11942 * C++ Parser Context:: You can supply a @code{report_syntax_error} function.
11943 * C++ Scanner Interface:: Exchanges between yylex and parse
11944 * A Complete C++ Example:: Demonstrating their use
11947 @node A Simple C++ Example
11948 @subsection A Simple C++ Example
11950 This tutorial about C++ parsers is based on a simple, self contained
11951 example. The following sections are the reference manual for Bison with
11952 C++, the last one showing a fully blown example (@pxref{A Complete C++
11955 To look nicer, our example will be in C++14. It is not required: Bison
11956 supports the original C++98 standard.
11958 A Bison file has three parts. In the first part, the prologue, we start by
11959 making sure we run a version of Bison which is recent enough, and that we
11963 @comment file: c++/simple.yy: 1
11965 /* Simple variant-based parser. -*- C++ -*-
11967 Copyright (C) 2018-2020 Free Software Foundation, Inc.
11969 This file is part of Bison, the GNU Compiler Compiler.
11971 This program is free software: you can redistribute it and/or modify
11972 it under the terms of the GNU General Public License as published by
11973 the Free Software Foundation, either version 3 of the License, or
11974 (at your option) any later version.
11976 This program is distributed in the hope that it will be useful,
11977 but WITHOUT ANY WARRANTY; without even the implied warranty of
11978 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
11979 GNU General Public License for more details.
11981 You should have received a copy of the GNU General Public License
11982 along with this program. If not, see <http://www.gnu.org/licenses/>. */
11986 @comment file: c++/simple.yy: 1
11992 Let's dive directly into the middle part: the grammar. Our input is a
11993 simple list of strings, that we display once the parsing is done.
11995 @comment file: c++/simple.yy: 2
12000 list @{ std::cout << $1 << '\n'; @}
12004 %nterm <std::vector<std::string>> list;
12007 %empty @{ /* Generates an empty string list */ @}
12008 | list item @{ $$ = $1; $$.push_back ($2); @}
12013 We used a vector of strings as a semantic value! To use genuine C++ objects
12014 as semantic values---not just PODs---we cannot rely on the union that Bison
12015 uses by default to store them, we need @emph{variants} (@pxref{C++
12018 @comment file: c++/simple.yy: 1
12020 %define api.value.type variant
12023 Obviously, the rule for @code{result} needs to print a vector of strings.
12024 In the prologue, we add:
12026 @comment file: c++/simple.yy: 1
12030 // Print a list of strings.
12032 operator<< (std::ostream& o, const std::vector<std::string>& ss)
12036 const char *sep = "";
12038 for (const auto& s: ss)
12050 You may want to move it into the @code{yy} namespace to avoid leaking it in
12051 your default namespace. We recommend that you keep the actions simple, and
12052 move details into auxiliary functions, as we did with @code{operator<<}.
12054 Our list of strings will be built from two types of items: numbers and
12057 @comment file: c++/simple.yy: 2
12059 %nterm <std::string> item;
12060 %token <std::string> TEXT;
12061 %token <int> NUMBER;
12065 | NUMBER @{ $$ = std::to_string ($1); @}
12070 In the case of @code{TEXT}, the implicit default action applies: @w{@code{$$
12075 Our scanner deserves some attention. The traditional interface of
12076 @code{yylex} is not type safe: since the token kind and the token value are
12077 not correlated, you may return a @code{NUMBER} with a string as semantic
12078 value. To avoid this, we use @emph{token constructors} (@pxref{Complete
12079 Symbols}). This directive:
12081 @comment file: c++/simple.yy: 1
12083 %define api.token.constructor
12087 requests that Bison generates the functions @code{make_TEXT} and
12088 @code{make_NUMBER}, but also @code{make_YYEOF}, for the end of input.
12090 Everything is in place for our scanner:
12092 @comment file: c++/simple.yy: 1
12098 // Return the next token.
12099 auto yylex () -> parser::symbol_type
12101 static int count = 0;
12102 switch (int stage = count++)
12106 return parser::make_TEXT ("I have three numbers for you.");
12109 case 1: case 2: case 3:
12110 return parser::make_NUMBER (stage);
12114 return parser::make_TEXT ("And that's all!");
12118 return parser::make_YYEOF ();
12126 In the epilogue, the third part of a Bison grammar file, we leave simple
12127 details: the error reporting function, and the main function.
12129 @comment file: c++/simple.yy: 3
12134 // Report an error to the user.
12135 auto parser::error (const std::string& msg) -> void
12137 std::cerr << msg << '\n';
12151 $ @kbd{bison simple.yy -o simple.cc}
12152 $ @kbd{g++ -std=c++14 simple.cc -o simple}
12155 @{I have three numbers for you., 1, 2, 3, And that's all!@}
12159 @node C++ Bison Interface
12160 @subsection C++ Bison Interface
12161 @c - %skeleton "lalr1.cc"
12163 @c - initial action
12165 The C++ deterministic parser is selected using the skeleton directive,
12166 @samp{%skeleton "lalr1.cc"}. @xref{Decl Summary}.
12168 When run, @command{bison} will create several entities in the @samp{yy}
12170 @findex %define api.namespace
12171 Use the @samp{%define api.namespace} directive to change the namespace name,
12172 see @ref{%define Summary}. The various classes are generated
12173 in the following files:
12176 @item @var{file}.hh
12177 (Assuming the extension of the grammar file was @samp{.yy}.) The
12178 declaration of the C++ parser class and auxiliary types. By default, this
12179 file is not generated (@pxref{Decl Summary}).
12181 @item @var{file}.cc
12182 The implementation of the C++ parser class. The basename and extension of
12183 these two files (@file{@var{file}.hh} and @file{@var{file}.cc}) follow the
12184 same rules as with regular C parsers (@pxref{Invocation}).
12187 Generated when both @code{%defines} and @code{%locations} are enabled, this
12188 file contains the definition of the classes @code{position} and
12189 @code{location}, used for location tracking. It is not generated if
12190 @samp{%define api.location.file none} is specified, or if user defined
12191 locations are used. @xref{C++ Location Values}.
12195 Useless legacy files. To get rid of then, use @samp{%require "3.2"} or
12199 All these files are documented using Doxygen; run @command{doxygen} for a
12200 complete and accurate documentation.
12202 @node C++ Parser Interface
12203 @subsection C++ Parser Interface
12205 The output files @file{@var{file}.hh} and @file{@var{file}.cc} declare and
12206 define the parser class in the namespace @code{yy}. The class name defaults
12207 to @code{parser}, but may be changed using @samp{%define api.parser.class
12208 @{@var{name}@}}. The interface of this class is detailed below. It can be
12209 extended using the @code{%parse-param} feature: its semantics is slightly
12210 changed since it describes an additional member of the parser class, and an
12211 additional argument for its constructor.
12214 @defcv {Type} {parser} {token}
12215 A structure that contains (only) the @code{token_kind_type} enumeration,
12216 which defines the tokens. To refer to the token @code{FOO}, use
12217 @code{yy::parser::token::FOO}. The scanner can use @samp{typedef
12218 yy::parser::token token;} to ``import'' the token enumeration (@pxref{Calc++
12222 @defcv {Type} {parser} {token_kind_type}
12223 An enumeration of the token kinds. Its enumerators are forged from the
12224 token names, with a possible token prefix
12225 (@pxref{api-token-prefix,,@code{api.token.prefix}}):
12231 enum token_kind_type
12233 YYEMPTY = -2, // No token.
12234 YYEOF = 0, // "end of file"
12235 YYerror = 256, // error
12236 YYUNDEF = 257, // "invalid token"
12238 MINUS = 259, // "-"
12240 VAR = 271, // "variable"
12245 /// Token kind, as returned by yylex.
12246 typedef token::token_kind_type token_kind_type;
12250 @defcv {Type} {parser} {semantic_type}
12251 The types for semantic values. @xref{C++ Semantic Values}.
12254 @defcv {Type} {parser} {location_type}
12255 The type of locations, if location tracking is enabled. @xref{C++ Location
12259 @defcv {Type} {parser} {syntax_error}
12260 This class derives from @code{std::runtime_error}. Throw instances of it
12261 from the scanner or from the actions to raise parse errors. This is
12262 equivalent with first invoking @code{error} to report the location and
12263 message of the syntax error, and then to invoke @code{YYERROR} to enter the
12264 error-recovery mode. But contrary to @code{YYERROR} which can only be
12265 invoked from user actions (i.e., written in the action itself), the
12266 exception can be thrown from functions invoked from the user action.
12269 @deftypeop {Constructor} {parser} {} parser ()
12270 @deftypeopx {Constructor} {parser} {} parser (@var{type1} @var{arg1}, ...)
12271 Build a new parser object. There are no arguments, unless
12272 @samp{%parse-param @{@var{type1} @var{arg1}@}} was used.
12275 @deftypeop {Constructor} {syntax_error} {} syntax_error (@code{const location_type&} @var{l}, @code{const std::string&} @var{m})
12276 @deftypeopx {Constructor} {syntax_error} {} syntax_error (@code{const std::string&} @var{m})
12277 Instantiate a syntax-error exception.
12280 @deftypemethod {parser} {int} operator() ()
12281 @deftypemethodx {parser} {int} parse ()
12282 Run the syntactic analysis, and return 0 on success, 1 otherwise. Both
12283 routines are equivalent, @code{operator()} being more C++ish.
12286 The whole function is wrapped in a @code{try}/@code{catch} block, so that
12287 when an exception is thrown, the @code{%destructor}s are called to release
12288 the lookahead symbol, and the symbols pushed on the stack.
12290 Exception related code in the generated parser is protected by CPP guards
12291 (@code{#if}) and disabled when exceptions are not supported (i.e., passing
12292 @option{-fno-exceptions} to the C++ compiler).
12295 @deftypemethod {parser} {std::ostream&} debug_stream ()
12296 @deftypemethodx {parser} {void} set_debug_stream (@code{std::ostream&} @var{o})
12297 Get or set the stream used for tracing the parsing. It defaults to
12301 @deftypemethod {parser} {debug_level_type} debug_level ()
12302 @deftypemethodx {parser} {void} set_debug_level (debug_level_type @var{l})
12303 Get or set the tracing level (an integral). Currently its value is either
12304 0, no trace, or nonzero, full tracing.
12307 @deftypemethod {parser} {void} error (@code{const location_type&} @var{l}, @code{const std::string&} @var{m})
12308 @deftypemethodx {parser} {void} error (@code{const std::string&} @var{m})
12309 The definition for this member function must be supplied by the user: the
12310 parser uses it to report a parser error occurring at @var{l}, described by
12311 @var{m}. If location tracking is not enabled, the second signature is used.
12315 @node C++ Semantic Values
12316 @subsection C++ Semantic Values
12317 @c - No objects in unions
12319 @c - Printer and destructor
12321 Bison supports two different means to handle semantic values in C++. One is
12322 alike the C interface, and relies on unions. As C++ practitioners know,
12323 unions are inconvenient in C++, therefore another approach is provided,
12327 * C++ Unions:: Semantic values cannot be objects
12328 * C++ Variants:: Using objects as semantic values
12332 @subsubsection C++ Unions
12334 The @code{%union} directive works as for C, see @ref{Union Decl}. In
12335 particular it produces a genuine @code{union}, which have a few specific
12339 The type @code{YYSTYPE} is defined but its use is discouraged: rather you
12340 should refer to the parser's encapsulated type
12341 @code{yy::parser::semantic_type}.
12343 Non POD (Plain Old Data) types cannot be used. C++98 forbids any instance
12344 of classes with constructors in unions: only @emph{pointers} to such objects
12345 are allowed. C++11 relaxed this constraints, but at the cost of safety.
12348 Because objects have to be stored via pointers, memory is not
12349 reclaimed automatically: using the @code{%destructor} directive is the
12350 only means to avoid leaks. @xref{Destructor Decl}.
12353 @subsubsection C++ Variants
12355 Bison provides a @emph{variant} based implementation of semantic values for
12356 C++. This alleviates all the limitations reported in the previous section,
12357 and in particular, object types can be used without pointers.
12359 To enable variant-based semantic values, set the @code{%define} variable
12360 @code{api.value.type} to @code{variant} (@pxref{%define Summary}). Then
12361 @code{%union} is ignored; instead of using the name of the fields of the
12362 @code{%union} to ``type'' the symbols, use genuine types.
12364 For instance, instead of:
12372 %token <ival> NUMBER;
12373 %token <sval> STRING;
12380 %token <int> NUMBER;
12381 %token <std::string> STRING;
12384 @code{STRING} is no longer a pointer, which should fairly simplify the user
12385 actions in the grammar and in the scanner (in particular the memory
12388 Since C++ features destructors, and since it is customary to specialize
12389 @code{operator<<} to support uniform printing of values, variants also
12390 typically simplify Bison printers and destructors.
12392 Variants are stricter than unions. When based on unions, you may play any
12393 dirty game with @code{yylval}, say storing an @code{int}, reading a
12394 @code{char*}, and then storing a @code{double} in it. This is no longer
12395 possible with variants: they must be initialized, then assigned to, and
12396 eventually, destroyed. As a matter of fact, Bison variants forbid the use
12397 of alternative types such as @samp{$<int>2} or @samp{$<std::string>$}, even
12398 in midrule actions. It is mandatory to use typed midrule actions
12399 (@pxref{Typed Midrule Actions}).
12401 @deftypemethod {semantic_type} {T&} {emplace<T>} ()
12402 @deftypemethodx {semantic_type} {T&} {emplace<T>} (@code{const T&} @var{t})
12403 Available in C++98/C++03 only. Default construct/copy-construct from
12404 @var{t}. Return a reference to where the actual value may be stored.
12405 Requires that the variant was not initialized yet.
12408 @deftypemethod {semantic_type} {T&} {emplace<T, U>} (@code{U&&...} @var{u})
12409 Available in C++11 and later only. Build a variant of type @code{T} from
12410 the variadic forwarding references @var{u...}.
12413 @strong{Warning}: We do not use Boost.Variant, for two reasons. First, it
12414 appeared unacceptable to require Boost on the user's machine (i.e., the
12415 machine on which the generated parser will be compiled, not the machine on
12416 which @command{bison} was run). Second, for each possible semantic value,
12417 Boost.Variant not only stores the value, but also a tag specifying its
12418 type. But the parser already ``knows'' the type of the semantic value, so
12419 that would be duplicating the information.
12421 We do not use C++17's @code{std::variant} either: we want to support all the
12422 C++ standards, and of course @code{std::variant} also stores a tag to record
12425 Therefore we developed light-weight variants whose type tag is external (so
12426 they are really like @code{unions} for C++ actually). There is a number of
12427 limitations in (the current implementation of) variants:
12430 Alignment must be enforced: values should be aligned in memory according to
12431 the most demanding type. Computing the smallest alignment possible requires
12432 meta-programming techniques that are not currently implemented in Bison, and
12433 therefore, since, as far as we know, @code{double} is the most demanding
12434 type on all platforms, alignments are enforced for @code{double} whatever
12435 types are actually used. This may waste space in some cases.
12438 There might be portability issues we are not aware of.
12441 As far as we know, these limitations @emph{can} be alleviated. All it takes
12442 is some time and/or some talented C++ hacker willing to contribute to Bison.
12444 @node C++ Location Values
12445 @subsection C++ Location Values
12447 When the directive @code{%locations} is used, the C++ parser supports
12448 location tracking, see @ref{Tracking Locations}.
12450 By default, two auxiliary classes define a @code{position}, a single point
12451 in a file, and a @code{location}, a range composed of a pair of
12452 @code{position}s (possibly spanning several files). If the @code{%define}
12453 variable @code{api.location.type} is defined, then these classes will not be
12454 generated, and the user defined type will be used.
12457 * C++ position:: One point in the source file
12458 * C++ location:: Two points in the source file
12459 * Exposing the Location Classes:: Using the Bison location class in your
12461 * User Defined Location Type:: Required interface for locations
12465 @subsubsection C++ @code{position}
12467 @defcv {Type} {position} {filename_type}
12468 The base type for file names. Defaults to @code{const std::string}.
12469 @xref{api-filename-type,,@code{api.filename.type}}, to change its definition.
12472 @defcv {Type} {position} {counter_type}
12473 The type used to store line and column numbers. Defined as @code{int}.
12476 @deftypeop {Constructor} {position} {} position (@code{filename_type*} @var{file} = nullptr, @code{counter_type} @var{line} = 1, @code{counter_type} @var{col} = 1)
12477 Create a @code{position} denoting a given point. Note that @code{file} is
12478 not reclaimed when the @code{position} is destroyed: memory managed must be
12482 @deftypemethod {position} {void} initialize (@code{filename_type*} @var{file} = nullptr, @code{counter_type} @var{line} = 1, @code{counter_type} @var{col} = 1)
12483 Reset the position to the given values.
12486 @deftypeivar {position} {filename_type*} file
12487 The name of the file. It will always be handled as a pointer, the parser
12488 will never duplicate nor deallocate it.
12491 @deftypeivar {position} {counter_type} line
12492 The line, starting at 1.
12495 @deftypemethod {position} {void} lines (@code{counter_type} @var{height} = 1)
12496 If @var{height} is not null, advance by @var{height} lines, resetting the
12497 column number. The resulting line number cannot be less than 1.
12500 @deftypeivar {position} {counter_type} column
12501 The column, starting at 1.
12504 @deftypemethod {position} {void} columns (@code{counter_type} @var{width} = 1)
12505 Advance by @var{width} columns, without changing the line number. The
12506 resulting column number cannot be less than 1.
12509 @deftypemethod {position} {position&} operator+= (@code{counter_type} @var{width})
12510 @deftypemethodx {position} {position} operator+ (@code{counter_type} @var{width})
12511 @deftypemethodx {position} {position&} operator-= (@code{counter_type} @var{width})
12512 @deftypemethodx {position} {position} operator- (@code{counter_type} @var{width})
12513 Various forms of syntactic sugar for @code{columns}.
12516 @deftypemethod {position} {bool} operator== (@code{const position&} @var{that})
12517 @deftypemethodx {position} {bool} operator!= (@code{const position&} @var{that})
12518 Whether @code{*this} and @code{that} denote equal/different positions.
12521 @deftypefun {std::ostream&} operator<< (@code{std::ostream&} @var{o}, @code{const position&} @var{p})
12522 Report @var{p} on @var{o} like this:
12523 @samp{@var{file}:@var{line}.@var{column}}, or
12524 @samp{@var{line}.@var{column}} if @var{file} is null.
12528 @subsubsection C++ @code{location}
12530 @deftypeop {Constructor} {location} {} location (@code{const position&} @var{begin}, @code{const position&} @var{end})
12531 Create a @code{Location} from the endpoints of the range.
12534 @deftypeop {Constructor} {location} {} location (@code{const position&} @var{pos} = position())
12535 @deftypeopx {Constructor} {location} {} location (@code{filename_type*} @var{file}, @code{counter_type} @var{line}, @code{counter_type} @var{col})
12536 Create a @code{Location} denoting an empty range located at a given point.
12539 @deftypemethod {location} {void} initialize (@code{filename_type*} @var{file} = nullptr, @code{counter_type} @var{line} = 1, @code{counter_type} @var{col} = 1)
12540 Reset the location to an empty range at the given values.
12543 @deftypeivar {location} {position} begin
12544 @deftypeivarx {location} {position} end
12545 The first, inclusive, position of the range, and the first beyond.
12548 @deftypemethod {location} {void} columns (@code{counter_type} @var{width} = 1)
12549 @deftypemethodx {location} {void} lines (@code{counter_type} @var{height} = 1)
12550 Forwarded to the @code{end} position.
12553 @deftypemethod {location} {location} operator+ (@code{counter_type} @var{width})
12554 @deftypemethodx {location} {location} operator+= (@code{counter_type} @var{width})
12555 @deftypemethodx {location} {location} operator- (@code{counter_type} @var{width})
12556 @deftypemethodx {location} {location} operator-= (@code{counter_type} @var{width})
12557 Various forms of syntactic sugar for @code{columns}.
12560 @deftypemethod {location} {location} operator+ (@code{const location&} @var{end})
12561 @deftypemethodx {location} {location} operator+= (@code{const location&} @var{end})
12562 Join two locations: starts at the position of the first one, and ends at the
12563 position of the second.
12566 @deftypemethod {location} {void} step ()
12567 Move @code{begin} onto @code{end}.
12570 @deftypemethod {location} {bool} operator== (@code{const location&} @var{that})
12571 @deftypemethodx {location} {bool} operator!= (@code{const location&} @var{that})
12572 Whether @code{*this} and @code{that} denote equal/different ranges of
12576 @deftypefun {std::ostream&} operator<< (@code{std::ostream&} @var{o}, @code{const location&} @var{p})
12577 Report @var{p} on @var{o}, taking care of special cases such as: no
12578 @code{filename} defined, or equal filename/line or column.
12581 @node Exposing the Location Classes
12582 @subsubsection Exposing the Location Classes
12584 When both @code{%defines} and @code{%locations} are enabled, Bison generates
12585 an additional file: @file{location.hh}. If you don't use locations outside
12586 of the parser, you may avoid its creation with @samp{%define
12587 api.location.file none}.
12589 However this file is useful if, for instance, your parser builds an abstract
12590 syntax tree decorated with locations: you may use Bison's @code{location}
12591 type independently of Bison's parser. You may name the file differently,
12592 e.g., @samp{%define api.location.file "include/ast/location.hh"}: this name
12593 can have directory components, or even be absolute. The way the location
12594 file is included is controlled by @code{api.location.include}.
12596 This way it is possible to have several parsers share the same location
12599 For instance, in @file{src/foo/parser.yy}, generate the
12600 @file{include/ast/loc.hh} file:
12603 // src/foo/parser.yy
12605 %define api.namespace @{foo@}
12606 %define api.location.file "include/ast/loc.hh"
12607 %define api.location.include @{<ast/loc.hh>@}
12611 and use it in @file{src/bar/parser.yy}:
12614 // src/bar/parser.yy
12616 %define api.namespace @{bar@}
12617 %code requires @{#include <ast/loc.hh>@}
12618 %define api.location.type @{bar::location@}
12621 Absolute file names are supported; it is safe in your @file{Makefile} to
12623 @option{-Dapi.location.file='"$(top_srcdir)/include/ast/loc.hh"'} to
12624 @command{bison} for @file{src/foo/parser.yy}. The generated file will not
12625 have references to this absolute path, thanks to @samp{%define
12626 api.location.include @{<ast/loc.hh>@}}. Adding @samp{-I
12627 $(top_srcdir)/include} to your @code{CPPFLAGS} will suffice for the compiler
12628 to find @file{ast/loc.hh}.
12630 @node User Defined Location Type
12631 @subsubsection User Defined Location Type
12632 @findex %define api.location.type
12634 Instead of using the built-in types you may use the @code{%define} variable
12635 @code{api.location.type} to specify your own type:
12638 %define api.location.type @{@var{LocationType}@}
12641 The requirements over your @var{LocationType} are:
12644 it must be copyable;
12647 in order to compute the (default) value of @code{@@$} in a reduction, the
12648 parser basically runs
12650 @@$.begin = @@1.begin;
12651 @@$.end = @@@var{N}.end; // The location of last right-hand side symbol.
12654 so there must be copyable @code{begin} and @code{end} members;
12657 alternatively you may redefine the computation of the default location, in
12658 which case these members are not required (@pxref{Location Default Action});
12661 if traces are enabled, then there must exist an @samp{std::ostream&
12662 operator<< (std::ostream& o, const @var{LocationType}& s)} function.
12667 In programs with several C++ parsers, you may also use the @code{%define}
12668 variable @code{api.location.type} to share a common set of built-in
12669 definitions for @code{position} and @code{location}. For instance, one
12670 parser @file{master/parser.yy} might use:
12675 %define api.namespace @{master::@}
12679 to generate the @file{master/position.hh} and @file{master/location.hh}
12680 files, reused by other parsers as follows:
12683 %define api.location.type @{master::location@}
12684 %code requires @{ #include <master/location.hh> @}
12688 @node C++ Parser Context
12689 @subsection C++ Parser Context
12691 When @samp{%define parse.error custom} is used (@pxref{Syntax Error
12692 Reporting Function}), the user must define the following function.
12694 @deftypemethod {parser} {void} report_syntax_error (@code{const context_type&}@var{ctx}) @code{const}
12695 Report a syntax error to the user. Whether it uses @code{yyerror} is up to
12699 Use the following types and functions to build the error message.
12701 @defcv {Type} {parser} {context}
12702 A type that captures the circumstances of the syntax error.
12705 @defcv {Type} {parser} {symbol_kind_type}
12706 An enum of all the grammar symbols, tokens and nonterminals. Its
12707 enumerators are forged from the symbol names:
12712 enum symbol_kind_type
12714 S_YYEMPTY = -2, // No symbol.
12715 S_YYEOF = 0, // "end of file"
12716 S_YYERROR = 1, // error
12717 S_YYUNDEF = 2, // "invalid token"
12719 S_MINUS = 4, // "-"
12721 S_VAR = 14, // "variable"
12723 S_YYACCEPT = 16, // $accept
12725 S_input = 18 // input
12728 typedef symbol_kind::symbol_kind_t symbol_kind_type;
12732 @deftypemethod {context} {const symbol_type&} lookahead () @code{const}
12733 The ``unexpected'' token: the lookahead that caused the syntax error.
12736 @deftypemethod {context} {symbol_kind_type} token () @code{const}
12737 The symbol kind of the lookahead token that caused the syntax error. Returns
12738 @code{symbol_kind::S_YYEMPTY} if there is no lookahead.
12741 @deftypemethod {context} {const location&} location () @code{const}
12742 The location of the syntax error (that of the lookahead).
12745 @deftypemethod {context} int expected_tokens (@code{symbol_kind_type} @var{argv}@code{[]}, @code{int} @var{argc}) @code{const}
12746 Fill @var{argv} with the expected tokens, which never includes
12747 @code{symbol_kind::S_YYEMPTY}, @code{symbol_kind::S_YYERROR}, or
12748 @code{symbol_kind::S_YYUNDEF}.
12750 Never put more than @var{argc} elements into @var{argv}, and on success
12751 return the number of tokens stored in @var{argv}. If there are more
12752 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
12753 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
12754 to @code{symbol_kind::S_YYEMPTY}.
12756 If @var{argv} is null, return the size needed to store all the possible
12757 values, which is always less than @code{YYNTOKENS}.
12760 @deftypemethod {parser} {const char *} symbol_name (@code{symbol_kind_t} @var{symbol}) @code{const}
12761 The name of the symbol whose kind is @var{symbol}, possibly translated.
12763 Returns a @code{std::string} when @code{parse.error} is @code{verbose}.
12766 A custom syntax error function looks as follows. This implementation is
12767 inappropriate for internationalization, see the @file{c/bistromathic}
12768 example for a better alternative.
12772 yy::parser::report_syntax_error (const context& ctx)
12775 std::cerr << ctx.location () << ": syntax error";
12776 // Report the tokens expected at this point.
12778 enum @{ TOKENMAX = 5 @};
12779 symbol_kind_type expected[TOKENMAX];
12780 int n = ctx.expected_tokens (ctx, expected, TOKENMAX);
12781 for (int i = 0; i < n; ++i)
12782 std::cerr << i == 0 ? ": expected " : " or "
12783 << symbol_name (expected[i]);
12785 // Report the unexpected token.
12787 symbol_kind_type lookahead = ctx.token ();
12788 if (lookahead != symbol_kind::S_YYEMPTY)
12789 std::cerr << " before " << symbol_name (lookahead));
12795 You still must provide a @code{yyerror} function, used for instance to
12796 report memory exhaustion.
12799 @node C++ Scanner Interface
12800 @subsection C++ Scanner Interface
12801 @c - prefix for yylex.
12802 @c - Pure interface to yylex
12805 The parser invokes the scanner by calling @code{yylex}. Contrary to C
12806 parsers, C++ parsers are always pure: there is no point in using the
12807 @samp{%define api.pure} directive. The actual interface with @code{yylex}
12808 depends whether you use unions, or variants.
12811 * Split Symbols:: Passing symbols as two/three components
12812 * Complete Symbols:: Making symbols a whole
12815 @node Split Symbols
12816 @subsubsection Split Symbols
12818 The generated parser expects @code{yylex} to have the following prototype.
12820 @deftypefun {int} yylex (@code{semantic_type*} @var{yylval}, @code{location_type*} @var{yylloc}, @var{type1} @var{arg1}, @dots{})
12821 @deftypefunx {int} yylex (@code{semantic_type*} @var{yylval}, @var{type1} @var{arg1}, @dots{})
12822 Return the next token. Its kind is the return value, its semantic value and
12823 location (if enabled) being @var{yylval} and @var{yylloc}. Invocations of
12824 @samp{%lex-param @{@var{type1} @var{arg1}@}} yield additional arguments.
12827 Note that when using variants, the interface for @code{yylex} is the same,
12828 but @code{yylval} is handled differently.
12830 Regular union-based code in Lex scanner typically looks like:
12834 yylval->ival = text_to_int (yytext);
12835 return yy::parser::token::INTEGER;
12838 yylval->sval = new std::string (yytext);
12839 return yy::parser::token::IDENTIFIER;
12843 Using variants, @code{yylval} is already constructed, but it is not
12844 initialized. So the code would look like:
12848 yylval->emplace<int> () = text_to_int (yytext);
12849 return yy::parser::token::INTEGER;
12852 yylval->emplace<std::string> () = yytext;
12853 return yy::parser::token::IDENTIFIER;
12862 yylval->emplace (text_to_int (yytext));
12863 return yy::parser::token::INTEGER;
12866 yylval->emplace (yytext);
12867 return yy::parser::token::IDENTIFIER;
12872 @node Complete Symbols
12873 @subsubsection Complete Symbols
12875 With both @code{%define api.value.type variant} and @code{%define
12876 api.token.constructor}, the parser defines the type @code{symbol_type}, and
12877 expects @code{yylex} to have the following prototype.
12879 @deftypefun {parser::symbol_type} yylex ()
12880 @deftypefunx {parser::symbol_type} yylex (@var{type1} @var{arg1}, @dots{})
12881 Return a @emph{complete} symbol, aggregating its type (i.e., the traditional
12882 value returned by @code{yylex}), its semantic value, and possibly its
12883 location. Invocations of @samp{%lex-param @{@var{type1} @var{arg1}@}} yield
12884 additional arguments.
12887 @defcv {Type} {parser} {symbol_type}
12888 A ``complete symbol'', that binds together its kind, value and (when
12889 applicable) location.
12892 @deftypemethod {symbol_type} {symbol_kind_type} kind () @code{const}
12893 The kind of this symbol.
12896 @deftypemethod {symbol_type} {const char *} name () @code{const}
12897 The name of the kind of this symbol.
12899 Returns a @code{std::string} when @code{parse.error} is @code{verbose}.
12904 For each token kind, Bison generates named constructors as follows.
12906 @deftypeop {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token}, @code{const @var{value_type}&} @var{value}, @code{const location_type&} @var{location})
12907 @deftypeopx {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token}, @code{const location_type&} @var{location})
12908 @deftypeopx {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token}, @code{const @var{value_type}&} @var{value})
12909 @deftypeopx {Constructor} {parser::symbol_type} {} {symbol_type} (@code{int} @var{token})
12910 Build a complete terminal symbol for the token kind @var{token} (including
12911 the @code{api.token.prefix}), whose semantic value, if it has one, is
12912 @var{value} of adequate @var{value_type}. Pass the @var{location} iff
12913 location tracking is enabled.
12915 Consistency between @var{token} and @var{value_type} is checked via an
12919 For instance, given the following declarations:
12922 %define api.token.prefix @{TOK_@}
12923 %token <std::string> IDENTIFIER;
12924 %token <int> INTEGER;
12929 you may use these constructors:
12932 symbol_type (int token, const std::string&, const location_type&);
12933 symbol_type (int token, const int&, const location_type&);
12934 symbol_type (int token, const location_type&);
12937 Correct matching between token kinds and value types is checked via
12938 @code{assert}; for instance, @samp{symbol_type (ID, 42)} would abort. Named
12939 constructors are preferable (see below), as they offer better type safety
12940 (for instance @samp{make_ID (42)} would not even compile), but symbol_type
12941 constructors may help when token kinds are discovered at run-time, e.g.,
12946 if (auto i = lookup_keyword (yytext))
12947 return yy::parser::symbol_type (i, loc);
12949 return yy::parser::make_ID (yytext, loc);
12956 Note that it is possible to generate and compile type incorrect code
12957 (e.g. @samp{symbol_type (':', yytext, loc)}). It will fail at run time,
12958 provided the assertions are enabled (i.e., @option{-DNDEBUG} was not passed
12959 to the compiler). Bison supports an alternative that guarantees that type
12960 incorrect code will not even compile. Indeed, it generates @emph{named
12961 constructors} as follows.
12963 @deftypemethod {parser} {symbol_type} {make_@var{token}} (@code{const @var{value_type}&} @var{value}, @code{const location_type&} @var{location})
12964 @deftypemethodx {parser} {symbol_type} {make_@var{token}} (@code{const location_type&} @var{location})
12965 @deftypemethodx {parser} {symbol_type} {make_@var{token}} (@code{const @var{value_type}&} @var{value})
12966 @deftypemethodx {parser} {symbol_type} {make_@var{token}} ()
12967 Build a complete terminal symbol for the token kind @var{token} (not
12968 including the @code{api.token.prefix}), whose semantic value, if it has one,
12969 is @var{value} of adequate @var{value_type}. Pass the @var{location} iff
12970 location tracking is enabled.
12973 For instance, given the following declarations:
12976 %define api.token.prefix @{TOK_@}
12977 %token <std::string> IDENTIFIER;
12978 %token <int> INTEGER;
12987 symbol_type make_IDENTIFIER (const std::string&, const location_type&);
12988 symbol_type make_INTEGER (const int&, const location_type&);
12989 symbol_type make_COLON (const location_type&);
12990 symbol_type make_EOF (const location_type&);
12994 which should be used in a scanner as follows.
12997 [a-z]+ return yy::parser::make_IDENTIFIER (yytext, loc);
12998 [0-9]+ return yy::parser::make_INTEGER (text_to_int (yytext), loc);
12999 ":" return yy::parser::make_COLON (loc);
13000 <<EOF>> return yy::parser::make_EOF (loc);
13003 Tokens that do not have an identifier are not accessible: you cannot simply
13004 use characters such as @code{':'}, they must be declared with @code{%token},
13005 including the end-of-file token.
13008 @node A Complete C++ Example
13009 @subsection A Complete C++ Example
13011 This section demonstrates the use of a C++ parser with a simple but complete
13012 example. This example should be available on your system, ready to compile,
13013 in the directory @dfn{.../share/doc/bison/examples/calc++}. It focuses on
13014 the use of Bison, therefore the design of the various C++ classes is very
13015 naive: no accessors, no encapsulation of members etc. We will use a Lex
13016 scanner, and more precisely, a Flex scanner, to demonstrate the various
13017 interactions. A hand-written scanner is actually easier to interface with.
13020 * Calc++ --- C++ Calculator:: The specifications
13021 * Calc++ Parsing Driver:: An active parsing context
13022 * Calc++ Parser:: A parser class
13023 * Calc++ Scanner:: A pure C++ Flex scanner
13024 * Calc++ Top Level:: Conducting the band
13027 @node Calc++ --- C++ Calculator
13028 @subsubsection Calc++ --- C++ Calculator
13030 Of course the grammar is dedicated to arithmetic, a single expression,
13031 possibly preceded by variable assignments. An environment containing
13032 possibly predefined variables such as @code{one} and @code{two}, is
13033 exchanged with the parser. An example of valid input follows.
13037 seven := one + two * three
13041 @node Calc++ Parsing Driver
13042 @subsubsection Calc++ Parsing Driver
13044 @c - A place to store error messages
13045 @c - A place for the result
13047 To support a pure interface with the parser (and the scanner) the technique
13048 of the ``parsing context'' is convenient: a structure containing all the
13049 data to exchange. Since, in addition to simply launch the parsing, there
13050 are several auxiliary tasks to execute (open the file for scanning,
13051 instantiate the parser etc.), we recommend transforming the simple parsing
13052 context structure into a fully blown @dfn{parsing driver} class.
13054 The declaration of this driver class, in @file{driver.hh}, is as follows.
13055 The first part includes the CPP guard and imports the required standard
13056 library components, and the declaration of the parser class.
13059 @comment file: calc++/driver.hh
13061 /* Driver for calc++. -*- C++ -*-
13063 Copyright (C) 2005--2015, 2018--2020 Free Software Foundation, Inc.
13065 This file is part of Bison, the GNU Compiler Compiler.
13067 This program is free software: you can redistribute it and/or modify
13068 it under the terms of the GNU General Public License as published by
13069 the Free Software Foundation, either version 3 of the License, or
13070 (at your option) any later version.
13072 This program is distributed in the hope that it will be useful,
13073 but WITHOUT ANY WARRANTY; without even the implied warranty of
13074 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13075 GNU General Public License for more details.
13077 You should have received a copy of the GNU General Public License
13078 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13082 @comment file: calc++/driver.hh
13088 # include "parser.hh"
13093 Then comes the declaration of the scanning function. Flex expects the
13094 signature of @code{yylex} to be defined in the macro @code{YY_DECL}, and the
13095 C++ parser expects it to be declared. We can factor both as follows.
13097 @comment file: calc++/driver.hh
13099 // Give Flex the prototype of yylex we want ...
13101 yy::parser::symbol_type yylex (driver& drv)
13102 // ... and declare it for the parser's sake.
13107 The @code{driver} class is then declared with its most obvious members.
13109 @comment file: calc++/driver.hh
13111 // Conducting the whole scanning and parsing of Calc++.
13117 std::map<std::string, int> variables;
13123 The main routine is of course calling the parser.
13125 @comment file: calc++/driver.hh
13127 // Run the parser on file F. Return 0 on success.
13128 int parse (const std::string& f);
13129 // The name of the file being parsed.
13131 // Whether to generate parser debug traces.
13132 bool trace_parsing;
13136 To encapsulate the coordination with the Flex scanner, it is useful to have
13137 member functions to open and close the scanning phase.
13139 @comment file: calc++/driver.hh
13141 // Handling the scanner.
13142 void scan_begin ();
13144 // Whether to generate scanner debug traces.
13145 bool trace_scanning;
13146 // The token's location used by the scanner.
13147 yy::location location;
13149 #endif // ! DRIVER_HH
13152 The implementation of the driver (@file{driver.cc}) is straightforward.
13155 @comment file: calc++/driver.cc
13157 /* Driver for calc++. -*- C++ -*-
13159 Copyright (C) 2005--2015, 2018--2020 Free Software Foundation, Inc.
13161 This file is part of Bison, the GNU Compiler Compiler.
13163 This program is free software: you can redistribute it and/or modify
13164 it under the terms of the GNU General Public License as published by
13165 the Free Software Foundation, either version 3 of the License, or
13166 (at your option) any later version.
13168 This program is distributed in the hope that it will be useful,
13169 but WITHOUT ANY WARRANTY; without even the implied warranty of
13170 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13171 GNU General Public License for more details.
13173 You should have received a copy of the GNU General Public License
13174 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13178 @comment file: calc++/driver.cc
13180 #include "driver.hh"
13181 #include "parser.hh"
13185 : trace_parsing (false), trace_scanning (false)
13187 variables["one"] = 1;
13188 variables["two"] = 2;
13193 The @code{parse} member function deserves some attention.
13195 @comment file: calc++/driver.cc
13199 driver::parse (const std::string &f)
13202 location.initialize (&file);
13204 yy::parser parse (*this);
13205 parse.set_debug_level (trace_parsing);
13206 int res = parse ();
13213 @node Calc++ Parser
13214 @subsubsection Calc++ Parser
13216 The grammar file @file{parser.yy} starts by asking for the C++ deterministic
13217 parser skeleton, the creation of the parser header file. Because the C++
13218 skeleton changed several times, it is safer to require the version you
13219 designed the grammar for.
13222 @comment file: calc++/parser.yy
13224 /* Parser for calc++. -*- C++ -*-
13226 Copyright (C) 2005--2015, 2018--2020 Free Software Foundation, Inc.
13228 This file is part of Bison, the GNU Compiler Compiler.
13230 This program is free software: you can redistribute it and/or modify
13231 it under the terms of the GNU General Public License as published by
13232 the Free Software Foundation, either version 3 of the License, or
13233 (at your option) any later version.
13235 This program is distributed in the hope that it will be useful,
13236 but WITHOUT ANY WARRANTY; without even the implied warranty of
13237 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13238 GNU General Public License for more details.
13240 You should have received a copy of the GNU General Public License
13241 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13245 @comment file: calc++/parser.yy
13247 %skeleton "lalr1.cc" // -*- C++ -*-
13248 %require "@value{VERSION}"
13253 @findex %define api.token.raw
13254 Because our scanner returns only genuine tokens and never simple characters
13255 (i.e., it returns @samp{PLUS}, not @samp{'+'}), we can avoid conversions.
13257 @comment file: calc++/parser.yy
13259 %define api.token.raw
13263 @findex %define api.token.constructor
13264 @findex %define api.value.type variant
13265 This example uses genuine C++ objects as semantic values, therefore, we
13266 require the variant-based storage of semantic values. To make sure we
13267 properly use it, we enable assertions. To fully benefit from type-safety
13268 and more natural definition of ``symbol'', we enable
13269 @code{api.token.constructor}.
13271 @comment file: calc++/parser.yy
13273 %define api.token.constructor
13274 %define api.value.type variant
13275 %define parse.assert
13279 @findex %code requires
13280 Then come the declarations/inclusions needed by the semantic values.
13281 Because the parser uses the parsing driver and reciprocally, both would like
13282 to include the header of the other, which is, of course, insane. This
13283 mutual dependency will be broken using forward declarations. Because the
13284 driver's header needs detailed knowledge about the parser class (in
13285 particular its inner types), it is the parser's header which will use a
13286 forward declaration of the driver. @xref{%code Summary}.
13288 @comment file: calc++/parser.yy
13299 The driver is passed by reference to the parser and to the scanner.
13300 This provides a simple but effective pure interface, not relying on
13303 @comment file: calc++/parser.yy
13305 // The parsing context.
13306 %param @{ driver& drv @}
13310 Then we request location tracking.
13312 @comment file: calc++/parser.yy
13318 Use the following two directives to enable parser tracing and detailed error
13319 messages. However, detailed error messages can contain incorrect
13320 information if lookahead correction is not enabled (@pxref{LAC}).
13322 @comment file: calc++/parser.yy
13324 %define parse.trace
13325 %define parse.error detailed
13326 %define parse.lac full
13331 The code between @samp{%code @{} and @samp{@}} is output in the @file{*.cc}
13332 file; it needs detailed knowledge about the driver.
13334 @comment file: calc++/parser.yy
13338 # include "driver.hh"
13345 User friendly names are provided for each symbol. To avoid name clashes in
13346 the generated files (@pxref{Calc++ Scanner}), prefix tokens with @code{TOK_}
13347 (@pxref{%define Summary}).
13349 @comment file: calc++/parser.yy
13351 %define api.token.prefix @{TOK_@}
13364 Since we use variant-based semantic values, @code{%union} is not used, and
13365 @code{%token}, @code{%nterm} and @code{%type} expect genuine types, not type
13368 @comment file: calc++/parser.yy
13370 %token <std::string> IDENTIFIER "identifier"
13371 %token <int> NUMBER "number"
13376 No @code{%destructor} is needed to enable memory deallocation during error
13377 recovery; the memory, for strings for instance, will be reclaimed by the
13378 regular destructors. All the values are printed using their
13379 @code{operator<<} (@pxref{Printer Decl}).
13381 @comment file: calc++/parser.yy
13383 %printer @{ yyo << $$; @} <*>;
13387 The grammar itself is straightforward (@pxref{Location Tracking Calc}).
13389 @comment file: calc++/parser.yy
13393 unit: assignments exp @{ drv.result = $2; @};
13397 | assignments assignment @{@};
13400 "identifier" ":=" exp @{ drv.variables[$1] = $3; @};
13406 | "identifier" @{ $$ = drv.variables[$1]; @}
13407 | exp "+" exp @{ $$ = $1 + $3; @}
13408 | exp "-" exp @{ $$ = $1 - $3; @}
13409 | exp "*" exp @{ $$ = $1 * $3; @}
13410 | exp "/" exp @{ $$ = $1 / $3; @}
13411 | "(" exp ")" @{ $$ = $2; @}
13416 Finally the @code{error} member function reports the errors.
13418 @comment file: calc++/parser.yy
13421 yy::parser::error (const location_type& l, const std::string& m)
13423 std::cerr << l << ": " << m << '\n';
13427 @node Calc++ Scanner
13428 @subsubsection Calc++ Scanner
13430 In addition to standard headers, the Flex scanner includes the driver's,
13431 then the parser's to get the set of defined tokens.
13434 @comment file: calc++/scanner.ll
13436 /* Scanner for calc++. -*- C++ -*-
13438 Copyright (C) 2005--2015, 2018--2020 Free Software Foundation, Inc.
13440 This file is part of Bison, the GNU Compiler Compiler.
13442 This program is free software: you can redistribute it and/or modify
13443 it under the terms of the GNU General Public License as published by
13444 the Free Software Foundation, either version 3 of the License, or
13445 (at your option) any later version.
13447 This program is distributed in the hope that it will be useful,
13448 but WITHOUT ANY WARRANTY; without even the implied warranty of
13449 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13450 GNU General Public License for more details.
13452 You should have received a copy of the GNU General Public License
13453 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13457 @comment file: calc++/scanner.ll
13459 %@{ /* -*- C++ -*- */
13461 # include <climits>
13462 # include <cstdlib>
13463 # include <cstring> // strerror
13465 # include "driver.hh"
13466 # include "parser.hh"
13471 @comment file: calc++/scanner.ll
13474 #if defined __clang__
13475 # define CLANG_VERSION (__clang_major__ * 100 + __clang_minor__)
13478 // Clang and ICC like to pretend they are GCC.
13479 #if defined __GNUC__ && !defined __clang__ && !defined __ICC
13480 # define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
13483 // Pacify warnings in yy_init_buffer (observed with Flex 2.6.4)
13484 // and GCC 6.4.0, 7.3.0 with -O3.
13485 #if defined GCC_VERSION && 600 <= GCC_VERSION
13486 # pragma GCC diagnostic ignored "-Wnull-dereference"
13489 // This example uses Flex's C back end, yet compiles it as C++.
13490 // So expect warnings about C style casts and NULL.
13491 #if defined CLANG_VERSION && 500 <= CLANG_VERSION
13492 # pragma clang diagnostic ignored "-Wold-style-cast"
13493 # pragma clang diagnostic ignored "-Wzero-as-null-pointer-constant"
13494 #elif defined GCC_VERSION && 407 <= GCC_VERSION
13495 # pragma GCC diagnostic ignored "-Wold-style-cast"
13496 # pragma GCC diagnostic ignored "-Wzero-as-null-pointer-constant"
13499 #define FLEX_VERSION (YY_FLEX_MAJOR_VERSION * 100 + YY_FLEX_MINOR_VERSION)
13501 // Old versions of Flex (2.5.35) generate an incomplete documentation comment.
13503 // In file included from src/scan-code-c.c:3:
13504 // src/scan-code.c:2198:21: error: empty paragraph passed to '@param' command
13505 // [-Werror,-Wdocumentation]
13506 // * @param line_number
13507 // ~~~~~~~~~~~~~~~~~^
13508 // 1 error generated.
13509 #if FLEX_VERSION < 206 && defined CLANG_VERSION
13510 # pragma clang diagnostic ignored "-Wdocumentation"
13513 // Old versions of Flex (2.5.35) use 'register'. Warnings introduced in
13514 // GCC 7 and Clang 6.
13515 #if FLEX_VERSION < 206
13516 # if defined CLANG_VERSION && 600 <= CLANG_VERSION
13517 # pragma clang diagnostic ignored "-Wdeprecated-register"
13518 # elif defined GCC_VERSION && 700 <= GCC_VERSION
13519 # pragma GCC diagnostic ignored "-Wregister"
13523 #if FLEX_VERSION < 206
13524 # if defined CLANG_VERSION
13525 # pragma clang diagnostic ignored "-Wconversion"
13526 # pragma clang diagnostic ignored "-Wdocumentation"
13527 # pragma clang diagnostic ignored "-Wshorten-64-to-32"
13528 # pragma clang diagnostic ignored "-Wsign-conversion"
13529 # elif defined GCC_VERSION
13530 # pragma GCC diagnostic ignored "-Wconversion"
13531 # pragma GCC diagnostic ignored "-Wsign-conversion"
13539 Since our calculator has no @code{#include}-like feature, we don't need
13540 @code{yywrap}. We don't need the @code{unput} and @code{input} functions
13541 either, and we parse an actual file, this is not an interactive session with
13542 the user. Finally, we enable scanner tracing.
13544 @comment file: calc++/scanner.ll
13546 %option noyywrap nounput noinput batch debug
13550 The following function will be handy to convert a string denoting a number
13551 into a @code{NUMBER} token.
13553 @comment file: calc++/scanner.ll
13556 // A number symbol corresponding to the value in S.
13557 yy::parser::symbol_type
13558 make_NUMBER (const std::string &s, const yy::parser::location_type& loc);
13563 Abbreviations allow for more readable rules.
13565 @comment file: calc++/scanner.ll
13567 id [a-zA-Z][a-zA-Z_0-9]*
13573 The following paragraph suffices to track locations accurately. Each time
13574 @code{yylex} is invoked, the begin position is moved onto the end position.
13575 Then when a pattern is matched, its width is added to the end column. When
13576 matching ends of lines, the end cursor is adjusted, and each time blanks are
13577 matched, the begin cursor is moved onto the end cursor to effectively ignore
13578 the blanks preceding tokens. Comments would be treated equally.
13580 @comment file: calc++/scanner.ll
13584 // Code run each time a pattern is matched.
13585 # define YY_USER_ACTION loc.columns (yyleng);
13591 // A handy shortcut to the location held by the driver.
13592 yy::location& loc = drv.location;
13593 // Code run each time yylex is called.
13597 @{blank@}+ loc.step ();
13598 \n+ loc.lines (yyleng); loc.step ();
13602 The rules are simple. The driver is used to report errors.
13604 @comment file: calc++/scanner.ll
13606 "-" return yy::parser::make_MINUS (loc);
13607 "+" return yy::parser::make_PLUS (loc);
13608 "*" return yy::parser::make_STAR (loc);
13609 "/" return yy::parser::make_SLASH (loc);
13610 "(" return yy::parser::make_LPAREN (loc);
13611 ")" return yy::parser::make_RPAREN (loc);
13612 ":=" return yy::parser::make_ASSIGN (loc);
13614 @{int@} return make_NUMBER (yytext, loc);
13615 @{id@} return yy::parser::make_IDENTIFIER (yytext, loc);
13618 throw yy::parser::syntax_error
13619 (loc, "invalid character: " + std::string(yytext));
13622 <<EOF>> return yy::parser::make_YYEOF (loc);
13627 You should keep your rules simple, both in the parser and in the scanner.
13628 Throwing from the auxiliary functions is then very handy to report errors.
13630 @comment file: calc++/scanner.ll
13633 yy::parser::symbol_type
13634 make_NUMBER (const std::string &s, const yy::parser::location_type& loc)
13637 long n = strtol (s.c_str(), NULL, 10);
13638 if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE))
13639 throw yy::parser::syntax_error (loc, "integer is out of range: " + s);
13640 return yy::parser::make_NUMBER ((int) n, loc);
13646 Finally, because the scanner-related driver's member-functions depend
13647 on the scanner's data, it is simpler to implement them in this file.
13649 @comment file: calc++/scanner.ll
13653 driver::scan_begin ()
13655 yy_flex_debug = trace_scanning;
13656 if (file.empty () || file == "-")
13658 else if (!(yyin = fopen (file.c_str (), "r")))
13660 std::cerr << "cannot open " << file << ": " << strerror (errno) << '\n';
13661 exit (EXIT_FAILURE);
13668 driver::scan_end ()
13675 @node Calc++ Top Level
13676 @subsubsection Calc++ Top Level
13678 The top level file, @file{calc++.cc}, poses no problem.
13681 @comment file: calc++/calc++.cc
13683 /* Main for calc++. -*- C++ -*-
13685 Copyright (C) 2005--2015, 2018--2020 Free Software Foundation, Inc.
13687 This file is part of Bison, the GNU Compiler Compiler.
13689 This program is free software: you can redistribute it and/or modify
13690 it under the terms of the GNU General Public License as published by
13691 the Free Software Foundation, either version 3 of the License, or
13692 (at your option) any later version.
13694 This program is distributed in the hope that it will be useful,
13695 but WITHOUT ANY WARRANTY; without even the implied warranty of
13696 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13697 GNU General Public License for more details.
13699 You should have received a copy of the GNU General Public License
13700 along with this program. If not, see <http://www.gnu.org/licenses/>. */
13704 @comment file: calc++/calc++.cc
13706 #include <iostream>
13707 #include "driver.hh"
13711 main (int argc, char *argv[])
13715 for (int i = 1; i < argc; ++i)
13716 if (argv[i] == std::string ("-p"))
13717 drv.trace_parsing = true;
13718 else if (argv[i] == std::string ("-s"))
13719 drv.trace_scanning = true;
13720 else if (!drv.parse (argv[i]))
13721 std::cout << drv.result << '\n';
13730 @section Java Parsers
13733 * Java Bison Interface:: Asking for Java parser generation
13734 * Java Semantic Values:: %token and %nterm vs. Java
13735 * Java Location Values:: The position and location classes
13736 * Java Parser Interface:: Instantiating and running the parser
13737 * Java Parser Context Interface:: Circumstances of a syntax error
13738 * Java Scanner Interface:: Specifying the scanner for the parser
13739 * Java Action Features:: Special features for use in actions
13740 * Java Push Parser Interface:: Instantiating and running the a push parser
13741 * Java Differences:: Differences between C/C++ and Java Grammars
13742 * Java Declarations Summary:: List of Bison declarations used with Java
13745 @node Java Bison Interface
13746 @subsection Java Bison Interface
13747 @c - %language "Java"
13749 The Java parser skeletons are selected using the @code{%language "Java"}
13750 directive or the @option{-L java}/@option{--language=java} option.
13752 @c FIXME: Documented bug.
13753 When generating a Java parser, @samp{bison @var{basename}.y} will create a
13754 single Java source file named @file{@var{basename}.java} containing the
13755 parser implementation. Using a grammar file without a @file{.y} suffix is
13756 currently broken. The basename of the parser implementation file can be
13757 changed by the @code{%file-prefix} directive or the
13758 @option{-b}/@option{--file-prefix} option. The entire parser implementation
13759 file name can be changed by the @code{%output} directive or the
13760 @option{-o}/@option{--output} option. The parser implementation file
13761 contains a single class for the parser.
13763 You can create documentation for generated parsers using Javadoc.
13765 Contrary to C parsers, Java parsers do not use global variables; the state
13766 of the parser is always local to an instance of the parser class.
13767 Therefore, all Java parsers are ``pure'', and the @code{%define api.pure}
13768 directive does nothing when used in Java.
13770 GLR parsers are currently unsupported in Java. Do not use the
13771 @code{glr-parser} directive.
13773 No header file can be generated for Java parsers. Do not use the
13774 @code{%defines} directive or the @option{-d}/@option{--defines} options.
13776 @c FIXME: Possible code change.
13777 Currently, support for tracing is always compiled in. Thus the
13778 @samp{%define parse.trace} and @samp{%token-table} directives and the
13779 @option{-t}/@option{--debug} and @option{-k}/@option{--token-table} options
13780 have no effect. This may change in the future to eliminate unused code in
13781 the generated parser, so use @samp{%define parse.trace} explicitly if
13782 needed. Also, in the future the @code{%token-table} directive might enable
13783 a public interface to access the token names and codes.
13785 Getting a ``code too large'' error from the Java compiler means the code hit
13786 the 64KB bytecode per method limitation of the Java class file. Try
13787 reducing the amount of code in actions and static initializers; otherwise,
13788 report a bug so that the parser skeleton will be improved.
13791 @node Java Semantic Values
13792 @subsection Java Semantic Values
13793 @c - No %union, specify type in %nterm/%token.
13795 @c - Printer and destructor
13797 There is no @code{%union} directive in Java parsers. Instead, the semantic
13798 values' types (class names) should be specified in the @code{%nterm} or
13799 @code{%token} directive:
13802 %nterm <Expression> expr assignment_expr term factor
13803 %nterm <Integer> number
13806 By default, the semantic stack is declared to have @code{Object} members,
13807 which means that the class types you specify can be of any class.
13808 To improve the type safety of the parser, you can declare the common
13809 superclass of all the semantic values using the @samp{%define api.value.type}
13810 directive. For example, after the following declaration:
13813 %define api.value.type @{ASTNode@}
13817 any @code{%token}, @code{%nterm} or @code{%type} specifying a semantic type
13818 which is not a subclass of @code{ASTNode}, will cause a compile-time error.
13820 @c FIXME: Documented bug.
13821 Types used in the directives may be qualified with a package name.
13822 Primitive data types are accepted for Java version 1.5 or later. Note
13823 that in this case the autoboxing feature of Java 1.5 will be used.
13824 Generic types may not be used; this is due to a limitation in the
13825 implementation of Bison, and may change in future releases.
13827 Java parsers do not support @code{%destructor}, since the language
13828 adopts garbage collection. The parser will try to hold references
13829 to semantic values for as little time as needed.
13831 Java parsers do not support @code{%printer}, as @code{toString()}
13832 can be used to print the semantic values. This however may change
13833 (in a backwards-compatible way) in future versions of Bison.
13836 @node Java Location Values
13837 @subsection Java Location Values
13839 @c - class Position
13840 @c - class Location
13842 When the directive @code{%locations} is used, the Java parser supports
13843 location tracking, see @ref{Tracking Locations}. An auxiliary user-defined
13844 class defines a @dfn{position}, a single point in a file; Bison itself
13845 defines a class representing a @dfn{location}, a range composed of a pair of
13846 positions (possibly spanning several files). The location class is an inner
13847 class of the parser; the name is @code{Location} by default, and may also be
13848 renamed using @code{%define api.location.type @{@var{class-name}@}}.
13850 The location class treats the position as a completely opaque value.
13851 By default, the class name is @code{Position}, but this can be changed
13852 with @code{%define api.position.type @{@var{class-name}@}}. This class must
13853 be supplied by the user.
13856 @deftypeivar {Location} {Position} begin
13857 @deftypeivarx {Location} {Position} end
13858 The first, inclusive, position of the range, and the first beyond.
13861 @deftypeop {Constructor} {Location} {} Location (@code{Position} @var{loc})
13862 Create a @code{Location} denoting an empty range located at a given point.
13865 @deftypeop {Constructor} {Location} {} Location (@code{Position} @var{begin}, @code{Position} @var{end})
13866 Create a @code{Location} from the endpoints of the range.
13869 @deftypemethod {Location} {String} toString ()
13870 Prints the range represented by the location. For this to work
13871 properly, the position class should override the @code{equals} and
13872 @code{toString} methods appropriately.
13876 @node Java Parser Interface
13877 @subsection Java Parser Interface
13879 The name of the generated parser class defaults to @code{YYParser}. The
13880 @code{YY} prefix may be changed using the @samp{%define api.prefix}.
13881 Alternatively, use @samp{%define api.parser.class @{@var{name}@}} to give a
13882 custom name to the class. The interface of this class is detailed below.
13884 By default, the parser class has package visibility. A declaration
13885 @samp{%define api.parser.public} will change to public visibility. Remember
13886 that, according to the Java language specification, the name of the
13887 @file{.java} file should match the name of the class in this case.
13888 Similarly, you can use @code{api.parser.abstract}, @code{api.parser.final}
13889 and @code{api.parser.strictfp} with the @code{%define} declaration to add
13890 other modifiers to the parser class. A single @samp{%define
13891 api.parser.annotations @{@var{annotations}@}} directive can be used to add
13892 any number of annotations to the parser class.
13894 The Java package name of the parser class can be specified using the
13895 @samp{%define package} directive. The superclass and the implemented
13896 interfaces of the parser class can be specified with the @code{%define
13897 api.parser.extends} and @samp{%define api.parser.implements} directives.
13899 The parser class defines an inner class, @code{Location}, that is used
13900 for location tracking (see @ref{Java Location Values}), and a inner
13901 interface, @code{Lexer} (see @ref{Java Scanner Interface}). Other than
13902 these inner class/interface, and the members described in the interface
13903 below, all the other members and fields are preceded with a @code{yy} or
13904 @code{YY} prefix to avoid clashes with user code.
13906 The parser class can be extended using the @code{%parse-param}
13907 directive. Each occurrence of the directive will add a @code{protected
13908 final} field to the parser class, and an argument to its constructor,
13909 which initialize them automatically.
13911 @deftypeop {Constructor} {YYParser} {} YYParser (@var{lex_param}, @dots{}, @var{parse_param}, @dots{})
13912 Build a new parser object with embedded @code{%code lexer}. There are
13913 no parameters, unless @code{%param}s and/or @code{%parse-param}s and/or
13914 @code{%lex-param}s are used.
13916 Use @code{%code init} for code added to the start of the constructor
13917 body. This is especially useful to initialize superclasses. Use
13918 @samp{%define init_throws} to specify any uncaught exceptions.
13921 @deftypeop {Constructor} {YYParser} {} YYParser (@code{Lexer} @var{lexer}, @var{parse_param}, @dots{})
13922 Build a new parser object using the specified scanner. There are no
13923 additional parameters unless @code{%param}s and/or @code{%parse-param}s are
13926 If the scanner is defined by @code{%code lexer}, this constructor is
13927 declared @code{protected} and is called automatically with a scanner
13928 created with the correct @code{%param}s and/or @code{%lex-param}s.
13930 Use @code{%code init} for code added to the start of the constructor
13931 body. This is especially useful to initialize superclasses. Use
13932 @samp{%define init_throws} to specify any uncaught exceptions.
13935 @deftypemethod {YYParser} {boolean} parse ()
13936 Run the syntactic analysis, and return @code{true} on success,
13937 @code{false} otherwise.
13940 @deftypemethod {YYParser} {boolean} getErrorVerbose ()
13941 @deftypemethodx {YYParser} {void} setErrorVerbose (boolean @var{verbose})
13942 Get or set the option to produce verbose error messages. These are only
13943 available with @samp{%define parse.error detailed} (or @samp{verbose}),
13944 which also turns on verbose error messages.
13947 @deftypemethod {YYParser} {void} yyerror (@code{String} @var{msg})
13948 @deftypemethodx {YYParser} {void} yyerror (@code{Position} @var{pos}, @code{String} @var{msg})
13949 @deftypemethodx {YYParser} {void} yyerror (@code{Location} @var{loc}, @code{String} @var{msg})
13950 Print an error message using the @code{yyerror} method of the scanner
13951 instance in use. The @code{Location} and @code{Position} parameters are
13952 available only if location tracking is active.
13955 @deftypemethod {YYParser} {boolean} recovering ()
13956 During the syntactic analysis, return @code{true} if recovering
13957 from a syntax error.
13958 @xref{Error Recovery}.
13961 @deftypemethod {YYParser} {java.io.PrintStream} getDebugStream ()
13962 @deftypemethodx {YYParser} {void} setDebugStream (@code{java.io.printStream} @var{o})
13963 Get or set the stream used for tracing the parsing. It defaults to
13967 @deftypemethod {YYParser} {int} getDebugLevel ()
13968 @deftypemethodx {YYParser} {void} setDebugLevel (@code{int} @var{l})
13969 Get or set the tracing level. Currently its value is either 0, no trace,
13970 or nonzero, full tracing.
13973 @deftypecv {Constant} {YYParser} {String} {bisonVersion}
13974 @deftypecvx {Constant} {YYParser} {String} {bisonSkeleton}
13975 Identify the Bison version and skeleton used to generate this parser.
13978 If you enabled token internationalization (@pxref{Token I18n}), you must
13979 provide the parser with the following function:
13981 @deftypecv {Static Method} {YYParser} {String} {i18n} (@code{string} @var{s})
13982 Return the translation of @var{s} in the user's language. As an example:
13986 static ResourceBundle myResources
13987 = ResourceBundle.getBundle("domain-name");
13988 static final String i18n(String s) @{
13989 return myResources.getString(s);
13995 @node Java Parser Context Interface
13996 @subsection Java Parser Context Interface
13998 The parser context provides information to build error reports when you
13999 invoke @samp{%define parse.error custom}.
14001 @defcv {Type} {YYParser} {SymbolKind}
14002 An enum of all the grammar symbols, tokens and nonterminals. Its
14003 enumerators are forged from the symbol names:
14006 public enum SymbolKind
14008 S_YYEOF(0), /* "end of file" */
14009 S_YYERROR(1), /* error */
14010 S_YYUNDEF(2), /* "invalid token" */
14011 S_BANG(3), /* "!" */
14012 S_PLUS(4), /* "+" */
14013 S_MINUS(5), /* "-" */
14015 S_NUM(13), /* "number" */
14016 S_NEG(14), /* NEG */
14017 S_YYACCEPT(15), /* $accept */
14018 S_input(16), /* input */
14019 S_line(17); /* line */
14024 @deftypemethod {YYParser.SymbolKind} {String} getName ()
14025 The name of this symbol, possibly translated.
14028 @deftypemethod {YYParser.Context} {YYParser.SymbolKind} getToken ()
14029 The kind of the lookahead. Return @code{null} iff there is no lookahead.
14032 @deftypemethod {YYParser.Context} {YYParser.Location} getLocation ()
14033 The location of the lookahead.
14036 @deftypemethod {YYParser.Context} {int} getExpectedTokens (@code{YYParser.SymbolKind[]} @var{argv}, @code{int} @var{argc})
14037 Fill @var{argv} with the expected tokens, which never includes
14038 @code{SymbolKind.S_YYERROR}, or @code{SymbolKind.S_YYUNDEF}.
14040 Never put more than @var{argc} elements into @var{argv}, and on success
14041 return the number of tokens stored in @var{argv}. If there are more
14042 expected tokens than @var{argc}, fill @var{argv} up to @var{argc} and return
14043 0. If there are no expected tokens, also return 0, but set @code{argv[0]}
14046 If @var{argv} is null, return the size needed to store all the possible
14047 values, which is always less than @code{YYNTOKENS}.
14051 @node Java Scanner Interface
14052 @subsection Java Scanner Interface
14055 @c - Lexer interface
14057 There are two possible ways to interface a Bison-generated Java parser
14058 with a scanner: the scanner may be defined by @code{%code lexer}, or
14059 defined elsewhere. In either case, the scanner has to implement the
14060 @code{Lexer} inner interface of the parser class. This interface also
14061 contain constants for all user-defined token names and the predefined
14062 @code{YYEOF} token.
14064 In the first case, the body of the scanner class is placed in
14065 @code{%code lexer} blocks. If you want to pass parameters from the
14066 parser constructor to the scanner constructor, specify them with
14067 @code{%lex-param}; they are passed before @code{%parse-param}s to the
14070 In the second case, the scanner has to implement the @code{Lexer} interface,
14071 which is defined within the parser class (e.g., @code{YYParser.Lexer}).
14072 The constructor of the parser object will then accept an object
14073 implementing the interface; @code{%lex-param} is not used in this
14076 In both cases, the scanner has to implement the following methods.
14078 @deftypemethod {Lexer} {void} yyerror (@code{Location} @var{loc}, @code{String} @var{msg})
14079 This method is defined by the user to emit an error message. The first
14080 parameter is omitted if location tracking is not active. Its type can be
14081 changed using @code{%define api.location.type @{@var{class-name}@}}.
14084 @deftypemethod {Lexer} {int} yylex ()
14085 Return the next token. Its type is the return value, its semantic value and
14086 location are saved and returned by the their methods in the interface. Not
14087 needed for push-only parsers.
14089 Use @samp{%define lex_throws} to specify any uncaught exceptions.
14090 Default is @code{java.io.IOException}.
14093 @deftypemethod {Lexer} {Position} getStartPos ()
14094 @deftypemethodx {Lexer} {Position} getEndPos ()
14095 Return respectively the first position of the last token that @code{yylex}
14096 returned, and the first position beyond it. These methods are not needed
14097 unless location tracking and pull parsing are active.
14099 They should return new objects for each call, to avoid that all the symbol
14100 share the same Position boundaries.
14102 The return type can be changed using @code{%define api.position.type
14103 @{@var{class-name}@}}.
14106 @deftypemethod {Lexer} {Object} getLVal ()
14107 Return the semantic value of the last token that yylex returned. Not needed
14108 for push-only parsers.
14110 The return type can be changed using @samp{%define api.value.type
14111 @{@var{class-name}@}}.
14114 @deftypemethod {Lexer} {void} reportSyntaxError (@code{YYParser.Context} @var{ctx})
14115 If you invoke @samp{%define parse.error custom} (@pxref{Bison
14116 Declarations}), then the parser no longer passes syntax error messages to
14117 @code{yyerror}, rather it delegates that task to the user by calling the
14118 @code{reportSyntaxError} function.
14120 Whether it uses @code{yyerror} is up to the user.
14122 Here is an example of a reporting function (@pxref{Java Parser Context
14126 public void reportSyntaxError(YYParser.Context ctx) @{
14127 System.err.print(ctx.getLocation() + ": syntax error");
14128 // Report the expected tokens.
14130 final int TOKENMAX = 5;
14131 YYParser.SymbolKind[] arg = new YYParser.SymbolKind[TOKENMAX];
14132 int n = ctx.getExpectedTokens(arg, TOKENMAX);
14133 for (int i = 0; i < n; ++i)
14134 System.err.print((i == 0 ? ": expected " : " or ")
14135 + arg[i].getName());
14137 // Report the unexpected token which triggered the error.
14139 YYParser.SymbolKind lookahead = ctx.getToken();
14140 if (lookahead != null)
14141 System.err.print(" before " + lookahead.getName());
14143 System.err.println("");
14148 This implementation is inappropriate for internationalization, see the
14149 @file{c/bistromathic} example for a better alternative.
14152 @node Java Action Features
14153 @subsection Special Features for Use in Java Actions
14155 The following special constructs can be uses in Java actions.
14156 Other analogous C action features are currently unavailable for Java.
14158 Use @samp{%define throws} to specify any uncaught exceptions from parser
14159 actions, and initial actions specified by @code{%initial-action}.
14162 The semantic value for the @var{n}th component of the current rule.
14163 This may not be assigned to.
14164 @xref{Java Semantic Values}.
14167 @defvar $<@var{typealt}>@var{n}
14168 Like @code{$@var{n}} but specifies a alternative type @var{typealt}.
14169 @xref{Java Semantic Values}.
14173 The semantic value for the grouping made by the current rule. As a
14174 value, this is in the base type (@code{Object} or as specified by
14175 @samp{%define api.value.type}) as in not cast to the declared subtype because
14176 casts are not allowed on the left-hand side of Java assignments.
14177 Use an explicit Java cast if the correct subtype is needed.
14178 @xref{Java Semantic Values}.
14181 @defvar $<@var{typealt}>$
14182 Same as @code{$$} since Java always allow assigning to the base type.
14183 Perhaps we should use this and @code{$<>$} for the value and @code{$$}
14184 for setting the value but there is currently no easy way to distinguish
14186 @xref{Java Semantic Values}.
14190 The location information of the @var{n}th component of the current rule.
14191 This may not be assigned to.
14192 @xref{Java Location Values}.
14196 The location information of the grouping made by the current rule.
14197 @xref{Java Location Values}.
14200 @deftypefn {Statement} return YYABORT @code{;}
14201 Return immediately from the parser, indicating failure.
14202 @xref{Java Parser Interface}.
14205 @deftypefn {Statement} return YYACCEPT @code{;}
14206 Return immediately from the parser, indicating success.
14207 @xref{Java Parser Interface}.
14210 @deftypefn {Statement} {return} YYERROR @code{;}
14211 Start error recovery (without printing an error message).
14212 @xref{Error Recovery}.
14215 @deftypefn {Function} {boolean} recovering ()
14216 Return whether error recovery is being done. In this state, the parser
14217 reads token until it reaches a known state, and then restarts normal
14219 @xref{Error Recovery}.
14222 @deftypefn {Function} {void} yyerror (@code{String} @var{msg})
14223 @deftypefnx {Function} {void} yyerror (@code{Position} @var{loc}, @code{String} @var{msg})
14224 @deftypefnx {Function} {void} yyerror (@code{Location} @var{loc}, @code{String} @var{msg})
14225 Print an error message using the @code{yyerror} method of the scanner
14226 instance in use. The @code{Location} and @code{Position} parameters are
14227 available only if location tracking is active.
14230 @node Java Push Parser Interface
14231 @subsection Java Push Parser Interface
14232 @c - define push_parse
14233 @findex %define api.push-pull
14235 Normally, Bison generates a pull parser for Java.
14236 The following Bison declaration says that you want the parser to be a push
14237 parser (@pxref{%define Summary}):
14240 %define api.push-pull push
14243 Most of the discussion about the Java pull Parser Interface, (@pxref{Java
14244 Parser Interface}) applies to the push parser interface as well.
14246 When generating a push parser, the method @code{push_parse} is created with
14247 the following signature (depending on if locations are enabled).
14249 @deftypemethod {YYParser} {void} push_parse (@code{int} @var{token}, @code{Object} @var{yylval})
14250 @deftypemethodx {YYParser} {void} push_parse (@code{int} @var{token}, @code{Object} @var{yylval}, @code{Location} @var{yyloc})
14251 @deftypemethodx {YYParser} {void} push_parse (@code{int} @var{token}, @code{Object} @var{yylval}, @code{Position} @var{yypos})
14254 The primary difference with respect to a pull parser is that the parser
14255 method @code{push_parse} is invoked repeatedly to parse each token. This
14256 function is available if either the "%define api.push-pull push" or "%define
14257 api.push-pull both" declaration is used (@pxref{%define
14258 Summary}). The @code{Location} and @code{Position}
14259 parameters are available only if location tracking is active.
14261 The value returned by the @code{push_parse} method is one of the following
14262 four constants: @code{YYABORT}, @code{YYACCEPT}, @code{YYERROR}, or
14263 @code{YYPUSH_MORE}. This new value, @code{YYPUSH_MORE}, may be returned if
14264 more input is required to finish parsing the grammar.
14266 If api.push-pull is declared as @code{both}, then the generated parser class
14267 will also implement the @code{parse} method. This method's body is a loop
14268 that repeatedly invokes the scanner and then passes the values obtained from
14269 the scanner to the @code{push_parse} method.
14271 There is one additional complication. Technically, the push parser does not
14272 need to know about the scanner (i.e. an object implementing the
14273 @code{YYParser.Lexer} interface), but it does need access to the
14274 @code{yyerror} method. Currently, the @code{yyerror} method is defined in
14275 the @code{YYParser.Lexer} interface. Hence, an implementation of that
14276 interface is still required in order to provide an implementation of
14277 @code{yyerror}. The current approach (and subject to change) is to require
14278 the @code{YYParser} constructor to be given an object implementing the
14279 @code{YYParser.Lexer} interface. This object need only implement the
14280 @code{yyerror} method; the other methods can be stubbed since they will
14281 never be invoked. The simplest way to do this is to add a trivial scanner
14282 implementation to your grammar file using whatever implementation of
14283 @code{yyerror} is desired. The following code sample shows a simple way to
14289 public Object getLVal () @{return null;@}
14290 public int yylex () @{return 0;@}
14291 public void yyerror (String s) @{System.err.println(s);@}
14295 @node Java Differences
14296 @subsection Differences between C/C++ and Java Grammars
14298 The different structure of the Java language forces several differences
14299 between C/C++ grammars, and grammars designed for Java parsers. This
14300 section summarizes these differences.
14304 Java lacks a preprocessor, so the @code{YYERROR}, @code{YYACCEPT},
14305 @code{YYABORT} symbols (@pxref{Table of Symbols}) cannot obviously be
14306 macros. Instead, they should be preceded by @code{return} when they
14307 appear in an action. The actual definition of these symbols is
14308 opaque to the Bison grammar, and it might change in the future. The
14309 only meaningful operation that you can do, is to return them.
14310 @xref{Java Action Features}.
14312 Note that of these three symbols, only @code{YYACCEPT} and
14313 @code{YYABORT} will cause a return from the @code{yyparse}
14314 method@footnote{Java parsers include the actions in a separate
14315 method than @code{yyparse} in order to have an intuitive syntax that
14316 corresponds to these C macros.}.
14319 Java lacks unions, so @code{%union} has no effect. Instead, semantic
14320 values have a common base type: @code{Object} or as specified by
14321 @samp{%define api.value.type}. Angle brackets on @code{%token}, @code{type},
14322 @code{$@var{n}} and @code{$$} specify subtypes rather than fields of
14323 an union. The type of @code{$$}, even with angle brackets, is the base
14324 type since Java casts are not allow on the left-hand side of assignments.
14325 Also, @code{$@var{n}} and @code{@@@var{n}} are not allowed on the
14326 left-hand side of assignments. @xref{Java Semantic Values}, and
14327 @ref{Java Action Features}.
14330 The prologue declarations have a different meaning than in C/C++ code.
14332 @item @code{%code imports}
14333 blocks are placed at the beginning of the Java source code. They may
14334 include copyright notices. For a @code{package} declarations, use
14335 @samp{%define api.package} instead.
14337 @item unqualified @code{%code}
14338 blocks are placed inside the parser class.
14340 @item @code{%code lexer}
14341 blocks, if specified, should include the implementation of the
14342 scanner. If there is no such block, the scanner can be any class
14343 that implements the appropriate interface (@pxref{Java Scanner
14347 Other @code{%code} blocks are not supported in Java parsers.
14348 In particular, @code{%@{ @dots{} %@}} blocks should not be used
14349 and may give an error in future versions of Bison.
14351 The epilogue has the same meaning as in C/C++ code and it can
14352 be used to define other classes used by the parser @emph{outside}
14357 @node Java Declarations Summary
14358 @subsection Java Declarations Summary
14360 This summary only include declarations specific to Java or have special
14361 meaning when used in a Java parser.
14363 @deffn {Directive} {%language "Java"}
14364 Generate a Java class for the parser.
14367 @deffn {Directive} %lex-param @{@var{type} @var{name}@}
14368 A parameter for the lexer class defined by @code{%code lexer}
14369 @emph{only}, added as parameters to the lexer constructor and the parser
14370 constructor that @emph{creates} a lexer. Default is none.
14371 @xref{Java Scanner Interface}.
14374 @deffn {Directive} %parse-param @{@var{type} @var{name}@}
14375 A parameter for the parser class added as parameters to constructor(s)
14376 and as fields initialized by the constructor(s). Default is none.
14377 @xref{Java Parser Interface}.
14380 @deffn {Directive} %token <@var{type}> @var{token} @dots{}
14381 Declare tokens. Note that the angle brackets enclose a Java @emph{type}.
14382 @xref{Java Semantic Values}.
14385 @deffn {Directive} %nterm <@var{type}> @var{nonterminal} @dots{}
14386 Declare the type of nonterminals. Note that the angle brackets enclose
14387 a Java @emph{type}.
14388 @xref{Java Semantic Values}.
14391 @deffn {Directive} %code @{ @var{code} @dots{} @}
14392 Code appended to the inside of the parser class.
14393 @xref{Java Differences}.
14396 @deffn {Directive} {%code imports} @{ @var{code} @dots{} @}
14397 Code inserted just after the @code{package} declaration.
14398 @xref{Java Differences}.
14401 @deffn {Directive} {%code init} @{ @var{code} @dots{} @}
14402 Code inserted at the beginning of the parser constructor body.
14403 @xref{Java Parser Interface}.
14406 @deffn {Directive} {%code lexer} @{ @var{code} @dots{} @}
14407 Code added to the body of a inner lexer class within the parser class.
14408 @xref{Java Scanner Interface}.
14411 @deffn {Directive} %% @var{code} @dots{}
14412 Code (after the second @code{%%}) appended to the end of the file,
14413 @emph{outside} the parser class.
14414 @xref{Java Differences}.
14417 @deffn {Directive} %@{ @var{code} @dots{} %@}
14418 Not supported. Use @code{%code imports} instead.
14419 @xref{Java Differences}.
14422 @deffn {Directive} {%define api.prefix} @{@var{prefix}@}
14423 The prefix of the parser class name @code{@var{prefix}Parser} if
14424 @samp{%define api.parser.class} is not used. Default is @code{YY}.
14425 @xref{Java Bison Interface}.
14428 @deffn {Directive} {%define api.parser.abstract}
14429 Whether the parser class is declared @code{abstract}. Default is false.
14430 @xref{Java Bison Interface}.
14433 @deffn {Directive} {%define api.parser.annotations} @{@var{annotations}@}
14434 The Java annotations for the parser class. Default is none.
14435 @xref{Java Bison Interface}.
14438 @deffn {Directive} {%define api.parser.class} @{@var{name}@}
14439 The name of the parser class. Default is @code{YYParser} or
14440 @code{@var{api.prefix}Parser}. @xref{Java Bison Interface}.
14443 @deffn {Directive} {%define api.parser.extends} @{@var{superclass}@}
14444 The superclass of the parser class. Default is none.
14445 @xref{Java Bison Interface}.
14448 @deffn {Directive} {%define api.parser.final}
14449 Whether the parser class is declared @code{final}. Default is false.
14450 @xref{Java Bison Interface}.
14453 @deffn {Directive} {%define api.parser.implements} @{@var{interfaces}@}
14454 The implemented interfaces of the parser class, a comma-separated list.
14456 @xref{Java Bison Interface}.
14459 @deffn {Directive} {%define api.parser.public}
14460 Whether the parser class is declared @code{public}. Default is false.
14461 @xref{Java Bison Interface}.
14464 @deffn {Directive} {%define api.parser.strictfp}
14465 Whether the parser class is declared @code{strictfp}. Default is false.
14466 @xref{Java Bison Interface}.
14469 @deffn {Directive} {%define init_throws} @{@var{exceptions}@}
14470 The exceptions thrown by @code{%code init} from the parser class
14471 constructor. Default is none.
14472 @xref{Java Parser Interface}.
14475 @deffn {Directive} {%define lex_throws} @{@var{exceptions}@}
14476 The exceptions thrown by the @code{yylex} method of the lexer, a
14477 comma-separated list. Default is @code{java.io.IOException}.
14478 @xref{Java Scanner Interface}.
14481 @deffn {Directive} {%define api.location.type} @{@var{class}@}
14482 The name of the class used for locations (a range between two
14483 positions). This class is generated as an inner class of the parser
14484 class by @command{bison}. Default is @code{Location}.
14485 Formerly named @code{location_type}.
14486 @xref{Java Location Values}.
14489 @deffn {Directive} {%define api.package} @{@var{package}@}
14490 The package to put the parser class in. Default is none.
14491 @xref{Java Bison Interface}.
14492 Renamed from @code{package} in Bison 3.7.
14495 @deffn {Directive} {%define api.position.type} @{@var{class}@}
14496 The name of the class used for positions. This class must be supplied by
14497 the user. Default is @code{Position}.
14498 Formerly named @code{position_type}.
14499 @xref{Java Location Values}.
14502 @deffn {Directive} {%define api.value.type} @{@var{class}@}
14503 The base type of semantic values. Default is @code{Object}.
14504 @xref{Java Semantic Values}.
14507 @deffn {Directive} {%define throws} @{@var{exceptions}@}
14508 The exceptions thrown by user-supplied parser actions and
14509 @code{%initial-action}, a comma-separated list. Default is none.
14510 @xref{Java Parser Interface}.
14513 @c ================================================= History
14516 @chapter A Brief History of the Greater Ungulates
14521 * Yacc:: The original Yacc
14522 * yacchack:: An obscure early implementation of reentrancy
14523 * Byacc:: Berkeley Yacc
14524 * Bison:: This program
14525 * Other Ungulates:: Similar programs
14529 @section The ancestral Yacc
14531 Bison originated as a workalike of a program called Yacc --- Yet Another
14532 Compiler Compiler.@footnote{Because of the acronym, the name is sometimes
14533 given as ``YACC'', but Johnson used ``Yacc'' in the descriptive paper
14535 @url{https://s3.amazonaws.com/plan9-bell-labs/7thEdMan/v7vol2b.pdf, Version
14536 7 Unix Manual}.} Yacc was written at Bell Labs as part of the very early
14537 development of Unix; one of its first uses was to develop the original
14538 Portable C Compiler, pcc. The same person, Steven C. Johnson, wrote Yacc and
14541 According to the author
14542 @footnote{@url{https://lists.gnu.org/archive/html/bison-patches/2019-02/msg00061.html}},
14543 Yacc was first invented in 1971 and reached a form recognizably similar to
14544 the C version in 1973. Johnson published @cite{A Portable Compiler: Theory
14545 and Practice} @pcite{Johnson 1978}.
14547 Yacc was not itself originally written in C but in its predecessor language,
14548 B. This goes far to explain its odd interface, which exposes a large number
14549 of global variables rather than bundling them into a C struct. All other
14550 Yacc-like programs are descended from the C port of Yacc.
14552 Yacc, through both its deployment in pcc and as a standalone tool for
14553 generating other parsers, helped drive the early spread of Unix. Yacc
14554 itself, however, passed out of use after around 1990 when workalikes
14555 with less restrictive licenses and more features became available.
14557 Original Yacc became generally available when Caldera released the sources
14558 of old versions of Unix up to V7 and 32V in 2002. By that time it had been
14559 long superseded in practical use by Bison even on Yacc's native Unix
14566 One of the deficiencies of original Yacc was its inability to produce
14567 reentrant parsers. This was first remedied by a set of drop-in
14568 modifications called ``yacchack'', published by Eric S. Raymond on USENET
14569 around 1983. This code was quickly forgotten when zoo and Berkeley Yacc
14570 became available a few years later.
14573 @section Berkeley Yacc
14576 Berkeley Yacc was originated in 1985 by Robert Corbett @pcite{Corbett
14577 1984}. It was originally named ``zoo'', but by October 1989 it became
14578 known as Berkeley Yacc or byacc.
14580 Berkeley Yacc had three advantages over the ancestral Yacc: it generated
14581 faster parsers, it could generate reentrant parsers, and the source code was
14582 released to the public domain rather than being under an AT&T proprietary
14583 license. The better performance came from implementing techniques from
14584 DeRemer and Penello's seminal paper on LALR parsing @pcite{DeRemer 1982}.
14586 Use of byacc spread rapidly due to its public domain license. However, once
14587 Bison became available, byacc itself passed out of general use.
14593 Robert Corbett actually wrote two (closely related) LALR parsers in 1985,
14594 both using the DeRemer/Penello techniques. One was ``zoo'', the other was
14595 ``Byson''. In 1987 Richard Stallman began working on Byson; the name changed
14596 to Bison and the interface became Yacc-compatible.
14598 The main visible difference between Yacc and Byson/Bison at the time of
14599 Byson's first release is that Byson supported the @code{@@@var{n}} construct
14600 (giving access to the starting and ending line number and character number
14601 associated with any of the symbols in the current rule).
14603 There was also the command @samp{%expect @var{n}} which said not to mention the
14604 conflicts if there are @var{n} shift/reduce conflicts and no reduce/reduce
14605 conflicts. In more recent versions of Bison, @code{%expect} and its
14606 @code{%expect-rr} variant for reduce/reduce conflicts can be applied to
14609 Later versions of Bison added many more new features.
14611 Bison error reporting has been improved in various ways. Notably. ancestral
14612 Yacc and Byson did not have carets in error messages.
14614 Compared to Yacc Bison uses a faster but less space-efficient encoding for
14615 the parse tables @pcite{Corbett 1984}, and more modern techniques for
14616 generating the lookahead sets @pcite{DeRemer 1982}. This approach is the
14617 standard one since then.
14619 (It has also been plausibly alleged the differences in the algorithms stem
14620 mainly from the horrible kludges that Johnson had to perpetrate to make
14621 the original Yacc fit in a PDP-11.)
14623 Named references, semantic predicates, @code{%locations},
14624 @code{%glr-parser}, @code{%printer}, %destructor, dumps to DOT,
14625 @code{%parse-param}, @code{%lex-param}, and dumps to XSLT, LAC, and IELR(1)
14626 generation are new in Bison.
14628 Bison also has many features to support C++ that were not present in the
14629 ancestral Yacc or Byson.
14631 Bison obsolesced all previous Yacc variants and workalikes generating C by
14634 @node Other Ungulates
14635 @section Other Ungulates
14637 The Yacc concept has frequently been ported to other languages. Some of the
14638 early ports are extinct along with the languages that hosted them; others
14639 have been superseded by parser skeletons shipped with Bison.
14641 However, independent implementations persist. One of the best-known
14642 still in use is David Beazley's ``PLY'' (Python Lex-Yacc) for
14643 Python. Another is goyacc, supporting the Go language. An ``ocamlyacc''
14644 is shipped as part of the Ocaml compiler suite.
14646 @c ================================================= FAQ
14649 @chapter Frequently Asked Questions
14650 @cindex frequently asked questions
14653 Several questions about Bison come up occasionally. Here some of them
14657 * Memory Exhausted:: Breaking the Stack Limits
14658 * How Can I Reset the Parser:: @code{yyparse} Keeps some State
14659 * Strings are Destroyed:: @code{yylval} Loses Track of Strings
14660 * Implementing Gotos/Loops:: Control Flow in the Calculator
14661 * Multiple start-symbols:: Factoring closely related grammars
14662 * Secure? Conform?:: Is Bison POSIX safe?
14663 * Enabling Relocatability:: Moving Bison/using it through network shares
14664 * I can't build Bison:: Troubleshooting
14665 * Where can I find help?:: Troubleshouting
14666 * Bug Reports:: Troublereporting
14667 * More Languages:: Parsers in C++, Java, and so on
14668 * Beta Testing:: Experimenting development versions
14669 * Mailing Lists:: Meeting other Bison users
14672 @node Memory Exhausted
14673 @section Memory Exhausted
14676 My parser returns with error with a @samp{memory exhausted}
14677 message. What can I do?
14680 This question is already addressed elsewhere, see @ref{Recursion}.
14682 @node How Can I Reset the Parser
14683 @section How Can I Reset the Parser
14685 The following phenomenon has several symptoms, resulting in the
14686 following typical questions:
14689 I invoke @code{yyparse} several times, and on correct input it works
14690 properly; but when a parse error is found, all the other calls fail
14691 too. How can I reset the error flag of @code{yyparse}?
14698 My parser includes support for an @samp{#include}-like feature, in which
14699 case I run @code{yyparse} from @code{yyparse}. This fails although I did
14700 specify @samp{%define api.pure full}.
14703 These problems typically come not from Bison itself, but from
14704 Lex-generated scanners. Because these scanners use large buffers for
14705 speed, they might not notice a change of input file. As a
14706 demonstration, consider the following source file,
14707 @file{first-line.l}:
14713 #include <stdlib.h>
14717 .*\n ECHO; return 1;
14721 yyparse (char const *file)
14723 yyin = fopen (file, "r");
14727 exit (EXIT_FAILURE);
14731 /* One token only. */
14733 if (fclose (yyin) != 0)
14736 exit (EXIT_FAILURE);
14754 If the file @file{input} contains
14762 then instead of getting the first line twice, you get:
14765 $ @kbd{flex -ofirst-line.c first-line.l}
14766 $ @kbd{gcc -ofirst-line first-line.c -ll}
14767 $ @kbd{./first-line}
14772 Therefore, whenever you change @code{yyin}, you must tell the
14773 Lex-generated scanner to discard its current buffer and switch to the
14774 new one. This depends upon your implementation of Lex; see its
14775 documentation for more. For Flex, it suffices to call
14776 @samp{YY_FLUSH_BUFFER} after each change to @code{yyin}. If your
14777 Flex-generated scanner needs to read from several input streams to
14778 handle features like include files, you might consider using Flex
14779 functions like @samp{yy_switch_to_buffer} that manipulate multiple
14782 If your Flex-generated scanner uses start conditions (@pxref{Start
14783 conditions, , Start conditions, flex, The Flex Manual}), you might
14784 also want to reset the scanner's state, i.e., go back to the initial
14785 start condition, through a call to @samp{BEGIN (0)}.
14787 @node Strings are Destroyed
14788 @section Strings are Destroyed
14791 My parser seems to destroy old strings, or maybe it loses track of
14792 them. Instead of reporting @samp{"foo", "bar"}, it reports
14793 @samp{"bar", "bar"}, or even @samp{"foo\nbar", "bar"}.
14796 This error is probably the single most frequent ``bug report'' sent to
14797 Bison lists, but is only concerned with a misunderstanding of the role
14798 of the scanner. Consider the following Lex code:
14804 char *yylval = NULL;
14809 .* yylval = yytext; return 1;
14817 /* Similar to using $1, $2 in a Bison action. */
14818 char *fst = (yylex (), yylval);
14819 char *snd = (yylex (), yylval);
14820 printf ("\"%s\", \"%s\"\n", fst, snd);
14826 If you compile and run this code, you get:
14829 $ @kbd{flex -osplit-lines.c split-lines.l}
14830 $ @kbd{gcc -osplit-lines split-lines.c -ll}
14831 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
14837 this is because @code{yytext} is a buffer provided for @emph{reading}
14838 in the action, but if you want to keep it, you have to duplicate it
14839 (e.g., using @code{strdup}). Note that the output may depend on how
14840 your implementation of Lex handles @code{yytext}. For instance, when
14841 given the Lex compatibility option @option{-l} (which triggers the
14842 option @samp{%array}) Flex generates a different behavior:
14845 $ @kbd{flex -l -osplit-lines.c split-lines.l}
14846 $ @kbd{gcc -osplit-lines split-lines.c -ll}
14847 $ @kbd{printf 'one\ntwo\n' | ./split-lines}
14852 @node Implementing Gotos/Loops
14853 @section Implementing Gotos/Loops
14856 My simple calculator supports variables, assignments, and functions,
14857 but how can I implement gotos, or loops?
14860 Although very pedagogical, the examples included in the document blur
14861 the distinction to make between the parser---whose job is to recover
14862 the structure of a text and to transmit it to subsequent modules of
14863 the program---and the processing (such as the execution) of this
14864 structure. This works well with so called straight line programs,
14865 i.e., precisely those that have a straightforward execution model:
14866 execute simple instructions one after the others.
14868 @cindex abstract syntax tree
14870 If you want a richer model, you will probably need to use the parser
14871 to construct a tree that does represent the structure it has
14872 recovered; this tree is usually called the @dfn{abstract syntax tree},
14873 or @dfn{AST} for short. Then, walking through this tree,
14874 traversing it in various ways, will enable treatments such as its
14875 execution or its translation, which will result in an interpreter or a
14878 This topic is way beyond the scope of this manual, and the reader is
14879 invited to consult the dedicated literature.
14882 @node Multiple start-symbols
14883 @section Multiple start-symbols
14886 I have several closely related grammars, and I would like to share their
14887 implementations. In fact, I could use a single grammar but with
14888 multiple entry points.
14891 Bison does not support multiple start-symbols, but there is a very
14892 simple means to simulate them. If @code{foo} and @code{bar} are the two
14893 pseudo start-symbols, then introduce two new tokens, say
14894 @code{START_FOO} and @code{START_BAR}, and use them as switches from the
14898 %token START_FOO START_BAR;
14905 These tokens prevents the introduction of new conflicts. As far as the
14906 parser goes, that is all that is needed.
14908 Now the difficult part is ensuring that the scanner will send these tokens
14909 first. If your scanner is hand-written, that should be straightforward. If
14910 your scanner is generated by Lex, them there is simple means to do it:
14911 recall that anything between @samp{%@{ ... %@}} after the first @code{%%} is
14912 copied verbatim in the top of the generated @code{yylex} function. Make
14913 sure a variable @code{start_token} is available in the scanner (e.g., a
14914 global variable or using @code{%lex-param} etc.), and use the following:
14917 /* @r{Prologue.} */
14922 int t = start_token;
14927 /* @r{The rules.} */
14931 @node Secure? Conform?
14932 @section Secure? Conform?
14935 Is Bison secure? Does it conform to POSIX?
14938 If you're looking for a guarantee or certification, we don't provide it.
14939 However, Bison is intended to be a reliable program that conforms to the
14940 POSIX specification for Yacc. If you run into problems, please send us a
14943 @include relocatable.texi
14945 @node I can't build Bison
14946 @section I can't build Bison
14949 I can't build Bison because @command{make} complains that
14950 @code{msgfmt} is not found.
14954 Like most GNU packages with internationalization support, that feature
14955 is turned on by default. If you have problems building in the @file{po}
14956 subdirectory, it indicates that your system's internationalization
14957 support is lacking. You can re-configure Bison with
14958 @option{--disable-nls} to turn off this support, or you can install GNU
14959 gettext from @url{https://ftp.gnu.org/gnu/gettext/} and re-configure
14960 Bison. See the file @file{ABOUT-NLS} for more information.
14963 I can't build Bison because my C compiler is too old.
14966 Except for GLR parsers (which require C99), the C code that Bison generates
14967 requires only C89 or later. However, Bison itself requires common C99
14968 features such as declarations after statements. Bison's @code{configure}
14969 script attempts to enable C99 (or later) support on compilers that default
14970 to pre-C99. If your compiler lacks these C99 features entirely, GCC may
14971 well be a better choice; or you can try upgrading to your compiler's latest
14974 @node Where can I find help?
14975 @section Where can I find help?
14978 I'm having trouble using Bison. Where can I find help?
14981 First, read this fine manual. Beyond that, you can send mail to
14982 @email{help-bison@@gnu.org}. This mailing list is intended to be
14983 populated with people who are willing to answer questions about using
14984 and installing Bison. Please keep in mind that (most of) the people on
14985 the list have aspects of their lives which are not related to Bison (!),
14986 so you may not receive an answer to your question right away. This can
14987 be frustrating, but please try not to honk them off; remember that any
14988 help they provide is purely voluntary and out of the kindness of their
14992 @section Bug Reports
14995 I found a bug. What should I include in the bug report?
14998 Before sending a bug report, make sure you are using the latest
14999 version. Check @url{https://ftp.gnu.org/pub/gnu/bison/} or one of its
15000 mirrors. Be sure to include the version number in your bug report. If
15001 the bug is present in the latest version but not in a previous version,
15002 try to determine the most recent version which did not contain the bug.
15004 If the bug is parser-related, you should include the smallest grammar
15005 you can which demonstrates the bug. The grammar file should also be
15006 complete (i.e., I should be able to run it through Bison without having
15007 to edit or add anything). The smaller and simpler the grammar, the
15008 easier it will be to fix the bug.
15010 Include information about your compilation environment, including your
15011 operating system's name and version and your compiler's name and
15012 version. If you have trouble compiling, you should also include a
15013 transcript of the build session, starting with the invocation of
15014 @code{configure}. Depending on the nature of the bug, you may be asked to
15015 send additional files as well (such as @file{config.h} or @file{config.cache}).
15017 Patches are most welcome, but not required. That is, do not hesitate to
15018 send a bug report just because you cannot provide a fix.
15020 Send bug reports to @email{bug-bison@@gnu.org}.
15022 @node More Languages
15023 @section More Languages
15026 Will Bison ever have C++ and Java support? How about @var{insert your
15027 favorite language here}?
15030 C++ and Java support is there now, and is documented. We'd love to add other
15031 languages; contributions are welcome.
15034 @section Beta Testing
15037 What is involved in being a beta tester?
15040 It's not terribly involved. Basically, you would download a test
15041 release, compile it, and use it to build and run a parser or two. After
15042 that, you would submit either a bug report or a message saying that
15043 everything is okay. It is important to report successes as well as
15044 failures because test releases eventually become mainstream releases,
15045 but only if they are adequately tested. If no one tests, development is
15046 essentially halted.
15048 Beta testers are particularly needed for operating systems to which the
15049 developers do not have easy access. They currently have easy access to
15050 recent GNU/Linux and Solaris versions. Reports about other operating
15051 systems are especially welcome.
15053 @node Mailing Lists
15054 @section Mailing Lists
15057 How do I join the help-bison and bug-bison mailing lists?
15060 See @url{http://lists.gnu.org/}.
15062 @c ================================================= Table of Symbols
15064 @node Table of Symbols
15065 @appendix Bison Symbols
15066 @cindex Bison symbols, table of
15067 @cindex symbols in Bison, table of
15069 @deffn {Variable} @@$
15070 In an action, the location of the left-hand side of the rule.
15071 @xref{Tracking Locations}.
15074 @deffn {Variable} @@@var{n}
15075 @deffnx {Symbol} @@@var{n}
15076 In an action, the location of the @var{n}-th symbol of the right-hand side
15077 of the rule. @xref{Tracking Locations}.
15079 In a grammar, the Bison-generated nonterminal symbol for a midrule action
15080 with a semantic value. @xref{Midrule Action Translation}.
15083 @deffn {Variable} @@@var{name}
15084 @deffnx {Variable} @@[@var{name}]
15085 In an action, the location of a symbol addressed by @var{name}.
15086 @xref{Tracking Locations}.
15089 @deffn {Symbol} $@@@var{n}
15090 In a grammar, the Bison-generated nonterminal symbol for a midrule action
15091 with no semantics value. @xref{Midrule Action Translation}.
15094 @deffn {Variable} $$
15095 In an action, the semantic value of the left-hand side of the rule.
15099 @deffn {Variable} $@var{n}
15100 In an action, the semantic value of the @var{n}-th symbol of the
15101 right-hand side of the rule. @xref{Actions}.
15104 @deffn {Variable} $@var{name}
15105 @deffnx {Variable} $[@var{name}]
15106 In an action, the semantic value of a symbol addressed by @var{name}.
15110 @deffn {Delimiter} %%
15111 Delimiter used to separate the grammar rule section from the
15112 Bison declarations section or the epilogue.
15113 @xref{Grammar Layout}.
15116 @c Don't insert spaces, or check the DVI output.
15117 @deffn {Delimiter} %@{@var{code}%@}
15118 All code listed between @samp{%@{} and @samp{%@}} is copied verbatim
15119 to the parser implementation file. Such code forms the prologue of
15120 the grammar file. @xref{Grammar Outline}.
15123 @deffn {Directive} %?@{@var{expression}@}
15124 Predicate actions. This is a type of action clause that may appear in
15125 rules. The expression is evaluated, and if false, causes a syntax error. In
15126 GLR parsers during nondeterministic operation,
15127 this silently causes an alternative parse to die. During deterministic
15128 operation, it is the same as the effect of YYERROR.
15129 @xref{Semantic Predicates}.
15132 @deffn {Construct} /* @dots{} */
15133 @deffnx {Construct} // @dots{}
15134 Comments, as in C/C++.
15137 @deffn {Delimiter} :
15138 Separates a rule's result from its components. @xref{Rules}.
15141 @deffn {Delimiter} ;
15142 Terminates a rule. @xref{Rules}.
15145 @deffn {Delimiter} |
15146 Separates alternate rules for the same result nonterminal.
15150 @deffn {Directive} <*>
15151 Used to define a default tagged @code{%destructor} or default tagged
15154 @xref{Destructor Decl}.
15157 @deffn {Directive} <>
15158 Used to define a default tagless @code{%destructor} or default tagless
15161 @xref{Destructor Decl}.
15164 @deffn {Symbol} $accept
15165 The predefined nonterminal whose only rule is @samp{$accept: @var{start}
15166 $end}, where @var{start} is the start symbol. @xref{Start Decl}. It cannot
15167 be used in the grammar.
15170 @deffn {Directive} %code @{@var{code}@}
15171 @deffnx {Directive} %code @var{qualifier} @{@var{code}@}
15172 Insert @var{code} verbatim into the output parser source at the
15173 default location or at the location specified by @var{qualifier}.
15174 @xref{%code Summary}.
15177 @deffn {Directive} %debug
15178 Equip the parser for debugging. @xref{Decl Summary}.
15182 @deffn {Directive} %default-prec
15183 Assign a precedence to rules that lack an explicit @samp{%prec}
15184 modifier. @xref{Contextual Precedence}.
15188 @deffn {Directive} %define @var{variable}
15189 @deffnx {Directive} %define @var{variable} @var{value}
15190 @deffnx {Directive} %define @var{variable} @{@var{value}@}
15191 @deffnx {Directive} %define @var{variable} "@var{value}"
15192 Define a variable to adjust Bison's behavior. @xref{%define Summary}.
15195 @deffn {Directive} %defines
15196 Bison declaration to create a parser header file, which is usually
15197 meant for the scanner. @xref{Decl Summary}.
15200 @deffn {Directive} %defines @var{defines-file}
15201 Same as above, but save in the file @var{defines-file}.
15202 @xref{Decl Summary}.
15205 @deffn {Directive} %destructor
15206 Specify how the parser should reclaim the memory associated to
15207 discarded symbols. @xref{Destructor Decl}.
15210 @deffn {Directive} %dprec
15211 Bison declaration to assign a precedence to a rule that is used at parse
15212 time to resolve reduce/reduce conflicts. @xref{GLR Parsers}.
15215 @deffn {Directive} %empty
15216 Bison declaration to declare make explicit that a rule has an empty
15217 right-hand side. @xref{Empty Rules}.
15220 @deffn {Symbol} $end
15221 The predefined token marking the end of the token stream. It cannot be
15222 used in the grammar.
15225 @deffn {Symbol} error
15226 A token name reserved for error recovery. This token may be used in
15227 grammar rules so as to allow the Bison parser to recognize an error in
15228 the grammar without halting the process. In effect, a sentence
15229 containing an error may be recognized as valid. On a syntax error, the
15230 token @code{error} becomes the current lookahead token. Actions
15231 corresponding to @code{error} are then executed, and the lookahead
15232 token is reset to the token that originally caused the violation.
15233 @xref{Error Recovery}.
15236 @deffn {Directive} %error-verbose
15237 An obsolete directive standing for @samp{%define parse.error verbose}.
15240 @deffn {Directive} %file-prefix "@var{prefix}"
15241 Bison declaration to set the prefix of the output files. @xref{Decl
15245 @deffn {Directive} %glr-parser
15246 Bison declaration to produce a GLR parser. @xref{GLR
15250 @deffn {Directive} %initial-action
15251 Run user code before parsing. @xref{Initial Action Decl}.
15254 @deffn {Directive} %language
15255 Specify the programming language for the generated parser.
15256 @xref{Decl Summary}.
15259 @deffn {Directive} %left
15260 Bison declaration to assign precedence and left associativity to token(s).
15261 @xref{Precedence Decl}.
15264 @deffn {Directive} %lex-param @{@var{argument-declaration}@} @dots{}
15265 Bison declaration to specifying additional arguments that
15266 @code{yylex} should accept. @xref{Pure Calling}.
15269 @deffn {Directive} %merge
15270 Bison declaration to assign a merging function to a rule. If there is a
15271 reduce/reduce conflict with a rule having the same merging function, the
15272 function is applied to the two semantic values to get a single result.
15273 @xref{GLR Parsers}.
15276 @deffn {Directive} %name-prefix "@var{prefix}"
15277 Obsoleted by the @code{%define} variable @code{api.prefix} (@pxref{Multiple
15280 Rename the external symbols (variables and functions) used in the parser so
15281 that they start with @var{prefix} instead of @samp{yy}. Contrary to
15282 @code{api.prefix}, do no rename types and macros.
15284 The precise list of symbols renamed in C parsers is @code{yyparse},
15285 @code{yylex}, @code{yyerror}, @code{yynerrs}, @code{yylval}, @code{yychar},
15286 @code{yydebug}, and (if locations are used) @code{yylloc}. If you use a
15287 push parser, @code{yypush_parse}, @code{yypull_parse}, @code{yypstate},
15288 @code{yypstate_new} and @code{yypstate_delete} will also be renamed. For
15289 example, if you use @samp{%name-prefix "c_"}, the names become
15290 @code{c_parse}, @code{c_lex}, and so on. For C++ parsers, see the
15291 @code{%define api.namespace} documentation in this section.
15296 @deffn {Directive} %no-default-prec
15297 Do not assign a precedence to rules that lack an explicit @samp{%prec}
15298 modifier. @xref{Contextual Precedence}.
15302 @deffn {Directive} %no-lines
15303 Bison declaration to avoid generating @code{#line} directives in the
15304 parser implementation file. @xref{Decl Summary}.
15307 @deffn {Directive} %nonassoc
15308 Bison declaration to assign precedence and nonassociativity to token(s).
15309 @xref{Precedence Decl}.
15312 @deffn {Directive} %nterm
15313 Bison declaration to declare nonterminals. @xref{Type Decl}.
15316 @deffn {Directive} %output "@var{file}"
15317 Bison declaration to set the name of the parser implementation file.
15318 @xref{Decl Summary}.
15321 @deffn {Directive} %param @{@var{argument-declaration}@} @dots{}
15322 Bison declaration to specify additional arguments that both
15323 @code{yylex} and @code{yyparse} should accept. @xref{Parser Function}.
15326 @deffn {Directive} %parse-param @{@var{argument-declaration}@} @dots{}
15327 Bison declaration to specify additional arguments that @code{yyparse}
15328 should accept. @xref{Parser Function}.
15331 @deffn {Directive} %prec
15332 Bison declaration to assign a precedence to a specific rule.
15333 @xref{Contextual Precedence}.
15336 @deffn {Directive} %precedence
15337 Bison declaration to assign precedence to token(s), but no associativity
15338 @xref{Precedence Decl}.
15341 @deffn {Directive} %pure-parser
15342 Deprecated version of @samp{%define api.pure} (@pxref{%define
15343 Summary}), for which Bison is more careful to warn about
15344 unreasonable usage.
15347 @deffn {Directive} %require "@var{version}"
15348 Require version @var{version} or higher of Bison. @xref{Require Decl}.
15351 @deffn {Directive} %right
15352 Bison declaration to assign precedence and right associativity to token(s).
15353 @xref{Precedence Decl}.
15356 @deffn {Directive} %skeleton
15357 Specify the skeleton to use; usually for development.
15358 @xref{Decl Summary}.
15361 @deffn {Directive} %start
15362 Bison declaration to specify the start symbol. @xref{Start Decl}.
15365 @deffn {Directive} %token
15366 Bison declaration to declare token(s) without specifying precedence.
15370 @deffn {Directive} %token-table
15371 Bison declaration to include a token name table in the parser implementation
15372 file. @xref{Decl Summary}.
15375 @deffn {Directive} %type
15376 Bison declaration to declare symbol value types. @xref{Type Decl}.
15379 @deffn {Symbol} $undefined
15380 The predefined token onto which all undefined values returned by
15381 @code{yylex} are mapped. It cannot be used in the grammar, rather, use
15385 @deffn {Directive} %union
15386 Bison declaration to specify several possible data types for semantic
15387 values. @xref{Union Decl}.
15390 @deffn {Macro} YYABORT
15391 Macro to pretend that an unrecoverable syntax error has occurred, by making
15392 @code{yyparse} return 1 immediately. The error reporting function
15393 @code{yyerror} is not called. @xref{Parser Function}.
15395 For Java parsers, this functionality is invoked using @code{return YYABORT;}
15399 @deffn {Macro} YYACCEPT
15400 Macro to pretend that a complete utterance of the language has been
15401 read, by making @code{yyparse} return 0 immediately.
15402 @xref{Parser Function}.
15404 For Java parsers, this functionality is invoked using @code{return YYACCEPT;}
15408 @deffn {Macro} YYBACKUP
15409 Macro to discard a value from the parser stack and fake a lookahead
15410 token. @xref{Action Features}.
15413 @deffn {Variable} yychar
15414 External integer variable that contains the integer value of the
15415 lookahead token. (In a pure parser, it is a local variable within
15416 @code{yyparse}.) Error-recovery rule actions may examine this variable.
15417 @xref{Action Features}.
15420 @deffn {Variable} yyclearin
15421 Macro used in error-recovery rule actions. It clears the previous
15422 lookahead token. @xref{Error Recovery}.
15425 @deffn {Macro} YYDEBUG
15426 Macro to define to equip the parser with tracing code. @xref{Tracing}.
15429 @deffn {Variable} yydebug
15430 External integer variable set to zero by default. If @code{yydebug}
15431 is given a nonzero value, the parser will output information on input
15432 symbols and parser action. @xref{Tracing}.
15435 @deffn {Value} YYEMPTY
15436 The pseudo token kind when there is no lookahead token.
15439 @deffn {Value} YYEOF
15440 The token kind denoting is the end of the input stream.
15443 @deffn {Macro} yyerrok
15444 Macro to cause parser to recover immediately to its normal mode
15445 after a syntax error. @xref{Error Recovery}.
15448 @deffn {Macro} YYERROR
15449 Cause an immediate syntax error. This statement initiates error
15450 recovery just as if the parser itself had detected an error; however, it
15451 does not call @code{yyerror}, and does not print any message. If you
15452 want to print an error message, call @code{yyerror} explicitly before
15453 the @samp{YYERROR;} statement. @xref{Error Recovery}.
15455 For Java parsers, this functionality is invoked using @code{return YYERROR;}
15459 @deffn {Function} yyerror
15460 User-supplied function to be called by @code{yyparse} on error.
15461 @xref{Error Reporting Function}.
15464 @deffn {Macro} YYFPRINTF
15465 Macro used to output run-time traces in C.
15466 @xref{Enabling Traces}.
15469 @deffn {Macro} YYINITDEPTH
15470 Macro for specifying the initial size of the parser stack.
15471 @xref{Memory Management}.
15474 @deffn {Function} yylex
15475 User-supplied lexical analyzer function, called with no arguments to get
15476 the next token. @xref{Lexical}.
15479 @deffn {Variable} yylloc
15480 External variable in which @code{yylex} should place the line and column
15481 numbers associated with a token. (In a pure parser, it is a local
15482 variable within @code{yyparse}, and its address is passed to
15484 You can ignore this variable if you don't use the @samp{@@} feature in the
15486 @xref{Token Locations}.
15487 In semantic actions, it stores the location of the lookahead token.
15488 @xref{Actions and Locations}.
15491 @deffn {Type} YYLTYPE
15492 Data type of @code{yylloc}. By default in C, a structure with four members
15493 (start/end line/column). @xref{Location Type}.
15496 @deffn {Variable} yylval
15497 External variable in which @code{yylex} should place the semantic
15498 value associated with a token. (In a pure parser, it is a local
15499 variable within @code{yyparse}, and its address is passed to
15501 @xref{Token Values}.
15502 In semantic actions, it stores the semantic value of the lookahead token.
15506 @deffn {Macro} YYMAXDEPTH
15507 Macro for specifying the maximum size of the parser stack. @xref{Memory
15511 @deffn {Variable} yynerrs
15512 Global variable which Bison increments each time it reports a syntax error.
15513 (In a pure parser, it is a local variable within @code{yyparse}. In a
15514 pure push parser, it is a member of @code{yypstate}.)
15515 @xref{Error Reporting Function}.
15518 @deffn {Function} yyparse
15519 The parser function produced by Bison; call this function to start
15520 parsing. @xref{Parser Function}.
15523 @deffn {Macro} YYPRINT
15524 Macro used to output token semantic values. For @file{yacc.c} only.
15525 Deprecated, use @code{%printer} instead (@pxref{Printer Decl}).
15526 @xref{The YYPRINT Macro}.
15529 @deffn {Function} yypstate_delete
15530 The function to delete a parser instance, produced by Bison in push mode;
15531 call this function to delete the memory associated with a parser.
15532 @xref{yypstate_delete,,@code{yypstate_delete}}. Does nothing when called
15533 with a null pointer.
15536 @deffn {Function} yypstate_new
15537 The function to create a parser instance, produced by Bison in push mode;
15538 call this function to create a new parser.
15539 @xref{yypstate_new,,@code{yypstate_new}}.
15542 @deffn {Function} yypull_parse
15543 The parser function produced by Bison in push mode; call this function to
15544 parse the rest of the input stream.
15545 @xref{yypull_parse,,@code{yypull_parse}}.
15548 @deffn {Function} yypush_parse
15549 The parser function produced by Bison in push mode; call this function to
15550 parse a single token.
15551 @xref{yypush_parse,,@code{yypush_parse}}.
15554 @deffn {Macro} YYRECOVERING
15555 The expression @code{YYRECOVERING ()} yields 1 when the parser
15556 is recovering from a syntax error, and 0 otherwise.
15557 @xref{Action Features}.
15560 @deffn {Macro} YYSTACK_USE_ALLOCA
15561 Macro used to control the use of @code{alloca} when the
15562 deterministic parser in C needs to extend its stacks. If defined to 0,
15563 the parser will use @code{malloc} to extend its stacks and memory exhaustion
15564 occurs if @code{malloc} fails (@pxref{Memory Management}). If defined to
15565 1, the parser will use @code{alloca}. Values other than 0 and 1 are
15566 reserved for future Bison extensions. If not defined,
15567 @code{YYSTACK_USE_ALLOCA} defaults to 0.
15569 In the all-too-common case where your code may run on a host with a
15570 limited stack and with unreliable stack-overflow checking, you should
15571 set @code{YYMAXDEPTH} to a value that cannot possibly result in
15572 unchecked stack overflow on any of your target hosts when
15573 @code{alloca} is called. You can inspect the code that Bison
15574 generates in order to determine the proper numeric values. This will
15575 require some expertise in low-level implementation details.
15578 @deffn {Type} YYSTYPE
15579 Deprecated in favor of the @code{%define} variable @code{api.value.type}.
15580 Data type of semantic values; @code{int} by default.
15584 @deffn {Type} yysymbol_kind_t
15585 An enum of all the symbols, tokens and nonterminals, of the grammar.
15586 @xref{Syntax Error Reporting Function}. The symbol kinds are used
15587 internally by the parser, and should not be confused with the token kinds:
15588 the symbol kind of a terminal symbol is not equal to its token kind! (Unless
15589 @samp{%define api.token.raw} was used.)
15592 @deffn {Type} yytoken_kind_t
15593 An enum of all the @dfn{token kinds} declared with @code{%token}
15594 (@pxref{Token Decl}). These are the return values for @code{yylex}. They
15595 should not be confused with the @emph{symbol kinds}, used internally by the
15599 @deffn {Value} YYUNDEF
15600 The token kind denoting an unknown token.
15609 @item Accepting state
15610 A state whose only action is the accept action.
15611 The accepting state is thus a consistent state.
15612 @xref{Understanding}.
15614 @item Backus-Naur Form (BNF; also called ``Backus Normal Form'')
15615 Formal method of specifying context-free grammars originally proposed
15616 by John Backus, and slightly improved by Peter Naur in his 1960-01-02
15617 committee document contributing to what became the Algol 60 report.
15618 @xref{Language and Grammar}.
15620 @item Consistent state
15621 A state containing only one possible action. @xref{Default Reductions}.
15623 @item Context-free grammars
15624 Grammars specified as rules that can be applied regardless of context.
15625 Thus, if there is a rule which says that an integer can be used as an
15626 expression, integers are allowed @emph{anywhere} an expression is
15627 permitted. @xref{Language and Grammar}.
15629 @item Counterexample
15630 A sequence of tokens and/or nonterminals, with one dot, that demonstrates a
15631 conflict. The dot marks the place where the conflict occurs.
15633 A @emph{unifying} counterexample is a single string that has two different
15634 parses; its existence proves that the grammar is ambiguous. When a unifying
15635 counterexample cannot be found in reasonable time, a @emph{nonunifying}
15636 counterexample is built: @emph{two} different string sharing the prefix up
15639 @xref{Counterexamples}
15641 @item Default reduction
15642 The reduction that a parser should perform if the current parser state
15643 contains no other action for the lookahead token. In permitted parser
15644 states, Bison declares the reduction with the largest lookahead set to be
15645 the default reduction and removes that lookahead set. @xref{Default
15648 @item Defaulted state
15649 A consistent state with a default reduction. @xref{Default Reductions}.
15651 @item Dynamic allocation
15652 Allocation of memory that occurs during execution, rather than at
15653 compile time or on entry to a function.
15656 Analogous to the empty set in set theory, the empty string is a
15657 character string of length zero.
15659 @item Finite-state stack machine
15660 A ``machine'' that has discrete states in which it is said to exist at
15661 each instant in time. As input to the machine is processed, the
15662 machine moves from state to state as specified by the logic of the
15663 machine. In the case of the parser, the input is the language being
15664 parsed, and the states correspond to various stages in the grammar
15665 rules. @xref{Algorithm}.
15667 @item Generalized LR (GLR)
15668 A parsing algorithm that can handle all context-free grammars, including those
15669 that are not LR(1). It resolves situations that Bison's
15670 deterministic parsing
15671 algorithm cannot by effectively splitting off multiple parsers, trying all
15672 possible parsers, and discarding those that fail in the light of additional
15673 right context. @xref{Generalized LR Parsing}.
15676 A language construct that is (in general) grammatically divisible;
15677 for example, `expression' or `declaration' in C@.
15678 @xref{Language and Grammar}.
15680 @item IELR(1) (Inadequacy Elimination LR(1))
15681 A minimal LR(1) parser table construction algorithm. That is, given any
15682 context-free grammar, IELR(1) generates parser tables with the full
15683 language-recognition power of canonical LR(1) but with nearly the same
15684 number of parser states as LALR(1). This reduction in parser states is
15685 often an order of magnitude. More importantly, because canonical LR(1)'s
15686 extra parser states may contain duplicate conflicts in the case of non-LR(1)
15687 grammars, the number of conflicts for IELR(1) is often an order of magnitude
15688 less as well. This can significantly reduce the complexity of developing a
15689 grammar. @xref{LR Table Construction}.
15691 @item Infix operator
15692 An arithmetic operator that is placed between the operands on which it
15693 performs some operation.
15696 A continuous flow of data between devices or programs.
15699 ``Token'' and ``symbol'' are each overloaded to mean either a grammar symbol
15700 (kind) or all parse info (kind, value, location) associated with occurrences
15701 of that grammar symbol from the input. To disambiguate,
15705 we use ``token kind'' and ``symbol kind'' to mean both grammar symbols and
15706 the values that represent them in a base programming language (C, C++,
15707 etc.). The names of the types of these values are typically
15708 @code{token_kind_t}, or @code{token_kind_type}, or @code{TokenKind},
15709 depending on the programming language.
15712 we use ``token'' and ``symbol'' without the word ``kind'' to mean parsed
15713 occurrences, and we append the word ``type'' to refer to the types that
15714 represent them in a base programming language.
15717 In summary: When you see ``kind'', interpret ``symbol'' or ``token'' to mean
15718 a @emph{grammar symbol}. When you don't see ``kind'' (including when you
15719 see ``type''), interpret ``symbol'' or ``token'' to mean a @emph{parsed
15722 @item LAC (Lookahead Correction)
15723 A parsing mechanism that fixes the problem of delayed syntax error
15724 detection, which is caused by LR state merging, default reductions, and the
15725 use of @code{%nonassoc}. Delayed syntax error detection results in
15726 unexpected semantic actions, initiation of error recovery in the wrong
15727 syntactic context, and an incorrect list of expected tokens in a verbose
15728 syntax error message. @xref{LAC}.
15730 @item Language construct
15731 One of the typical usage schemas of the language. For example, one of
15732 the constructs of the C language is the @code{if} statement.
15733 @xref{Language and Grammar}.
15735 @item Left associativity
15736 Operators having left associativity are analyzed from left to right:
15737 @samp{a+b+c} first computes @samp{a+b} and then combines with
15738 @samp{c}. @xref{Precedence}.
15740 @item Left recursion
15741 A rule whose result symbol is also its first component symbol; for
15742 example, @samp{expseq1 : expseq1 ',' exp;}. @xref{Recursion}.
15744 @item Left-to-right parsing
15745 Parsing a sentence of a language by analyzing it token by token from
15746 left to right. @xref{Algorithm}.
15748 @item Lexical analyzer (scanner)
15749 A function that reads an input stream and returns tokens one by one.
15752 @item Lexical tie-in
15753 A flag, set by actions in the grammar rules, which alters the way
15754 tokens are parsed. @xref{Lexical Tie-ins}.
15756 @item Literal string token
15757 A token which consists of two or more fixed characters. @xref{Symbols}.
15759 @item Lookahead token
15760 A token already read but not yet shifted. @xref{Lookahead}.
15763 The class of context-free grammars that Bison (like most other parser
15764 generators) can handle by default; a subset of LR(1).
15765 @xref{Mysterious Conflicts}.
15768 The class of context-free grammars in which at most one token of
15769 lookahead is needed to disambiguate the parsing of any piece of input.
15771 @item Nonterminal symbol
15772 A grammar symbol standing for a grammatical construct that can
15773 be expressed through rules in terms of smaller constructs; in other
15774 words, a construct that is not a token. @xref{Symbols}.
15777 A function that recognizes valid sentences of a language by analyzing
15778 the syntax structure of a set of tokens passed to it from a lexical
15781 @item Postfix operator
15782 An arithmetic operator that is placed after the operands upon which it
15783 performs some operation.
15786 Replacing a string of nonterminals and/or terminals with a single
15787 nonterminal, according to a grammar rule. @xref{Algorithm}.
15790 A reentrant subprogram is a subprogram which can be in invoked any
15791 number of times in parallel, without interference between the various
15792 invocations. @xref{Pure Decl}.
15794 @item Reverse Polish Notation
15795 A language in which all operators are postfix operators.
15797 @item Right recursion
15798 A rule whose result symbol is also its last component symbol; for
15799 example, @samp{expseq1: exp ',' expseq1;}. @xref{Recursion}.
15802 In computer languages, the semantics are specified by the actions
15803 taken for each instance of the language, i.e., the meaning of
15804 each statement. @xref{Semantics}.
15807 A parser is said to shift when it makes the choice of analyzing
15808 further input from the stream rather than reducing immediately some
15809 already-recognized rule. @xref{Algorithm}.
15811 @item Single-character literal
15812 A single character that is recognized and interpreted as is.
15813 @xref{Grammar in Bison}.
15816 The nonterminal symbol that stands for a complete valid utterance in
15817 the language being parsed. The start symbol is usually listed as the
15818 first nonterminal symbol in a language specification.
15822 A (finite) enumeration of the grammar symbols, as processed by the parser.
15826 A data structure where symbol names and associated data are stored during
15827 parsing to allow for recognition and use of existing information in repeated
15828 uses of a symbol. @xref{Multi-function Calc}.
15831 An error encountered during parsing of an input stream due to invalid
15832 syntax. @xref{Error Recovery}.
15834 @item Terminal symbol
15835 A grammar symbol that has no rules in the grammar and therefore is
15836 grammatically indivisible. The piece of text it represents is a token.
15837 @xref{Language and Grammar}.
15840 A basic, grammatically indivisible unit of a language. The symbol that
15841 describes a token in the grammar is a terminal symbol. The input of the
15842 Bison parser is a stream of tokens which comes from the lexical analyzer.
15846 A (finite) enumeration of the grammar terminals, as discriminated by the
15847 scanner. @xref{Symbols}.
15849 @item Unreachable state
15850 A parser state to which there does not exist a sequence of transitions from
15851 the parser's start state. A state can become unreachable during conflict
15852 resolution. @xref{Unreachable States}.
15855 @node GNU Free Documentation License
15856 @appendix GNU Free Documentation License
15861 @unnumbered Bibliography
15863 @c Please follow the following canvas to add more references.
15864 @c And keep sorted alphabetically.
15867 @anchor{Corbett 1984}
15868 @item [Corbett 1984]
15870 Robert Paul Corbett,
15872 Static Semantics in Compiler Error Recovery
15874 Ph.D. Dissertation, Report No. UCB/CSD 85/251,
15876 Department of Electrical Engineering and Computer Science, Compute Science
15877 Division, University of California, Berkeley, California
15881 @uref{http://xtf.lib.berkeley.edu/reports/TRWebData/accessPages/CSD-85-251.html}
15883 @anchor{Denny 2008}
15885 Joel E. Denny and Brian A. Malloy, IELR(1): Practical LR(1) Parser Tables
15886 for Non-LR(1) Grammars with Conflict Resolution, in @cite{Proceedings of the
15887 2008 ACM Symposium on Applied Computing} (SAC'08), ACM, New York, NY, USA,
15888 pp.@: 240--245. @uref{http://dx.doi.org/10.1145/1363686.1363747}
15890 @anchor{Denny 2010 May}
15891 @item [Denny 2010 May]
15892 Joel E. Denny, PSLR(1): Pseudo-Scannerless Minimal LR(1) for the
15893 Deterministic Parsing of Composite Languages, Ph.D. Dissertation, Clemson
15894 University, Clemson, SC, USA (May 2010).
15895 @uref{http://proquest.umi.com/pqdlink?did=2041473591&Fmt=7&clientId=79356&RQT=309&VName=PQD}
15897 @anchor{Denny 2010 November}
15898 @item [Denny 2010 November]
15899 Joel E. Denny and Brian A. Malloy, The IELR(1) Algorithm for Generating
15900 Minimal LR(1) Parser Tables for Non-LR(1) Grammars with Conflict Resolution,
15901 in @cite{Science of Computer Programming}, Vol.@: 75, Issue 11 (November
15902 2010), pp.@: 943--979. @uref{http://dx.doi.org/10.1016/j.scico.2009.08.001}
15904 @anchor{DeRemer 1982}
15905 @item [DeRemer 1982]
15906 Frank DeRemer and Thomas Pennello, Efficient Computation of LALR(1)
15907 Look-Ahead Sets, in @cite{ACM Transactions on Programming Languages and
15908 Systems}, Vol.@: 4, No.@: 4 (October 1982), pp.@:
15909 615--649. @uref{http://dx.doi.org/10.1145/69622.357187}
15911 @anchor{Isradisaikul 2015}
15912 @item [Isradisaikul 2015]
15913 Chinawat Isradisaikul, Andrew Myers,
15914 Finding Counterexamples from Parsing Conflicts,
15915 in @cite{Proceedings of the 36th ACM SIGPLAN Conference on
15916 Programming Language Design and Implementation} (PLDI '15),
15917 ACM, pp.@: 555--564.
15918 @uref{https://www.cs.cornell.edu/andru/papers/cupex/cupex.pdf}
15920 @anchor{Johnson 1978}
15921 @item [Johnson 1978]
15923 A portable compiler: theory and practice,
15924 in @cite{Proceedings of the 5th ACM SIGACT-SIGPLAN symposium on
15925 Principles of programming languages} (POPL '78),
15927 @uref{https://dx.doi.org/10.1145/512760.512771}.
15929 @anchor{Knuth 1965}
15931 Donald E. Knuth, On the Translation of Languages from Left to Right, in
15932 @cite{Information and Control}, Vol.@: 8, Issue 6 (December 1965), pp.@:
15933 607--639. @uref{http://dx.doi.org/10.1016/S0019-9958(65)90426-2}
15935 @anchor{Scott 2000}
15937 Elizabeth Scott, Adrian Johnstone, and Shamsa Sadaf Hussain,
15938 @cite{Tomita-Style Generalised LR Parsers}, Royal Holloway, University of
15939 London, Department of Computer Science, TR-00-12 (December 2000).
15940 @uref{http://www.cs.rhul.ac.uk/research/languages/publications/tomita_style_1.ps}
15943 @node Index of Terms
15944 @unnumbered Index of Terms
15950 @c LocalWords: texinfo setfilename settitle setchapternewpage finalout texi FSF
15951 @c LocalWords: ifinfo smallbook shorttitlepage titlepage GPL FIXME iftex FSF's
15952 @c LocalWords: akim fn cp syncodeindex vr tp synindex dircategory direntry Naur
15953 @c LocalWords: ifset vskip pt filll insertcopying sp ISBN Etienne Suvasa Multi
15954 @c LocalWords: ifnottex yyparse detailmenu GLR RPN Calc var Decls Rpcalc multi
15955 @c LocalWords: rpcalc Lexer Expr ltcalc mfcalc yylex defaultprec Donnelly Gotos
15956 @c LocalWords: yyerror pxref LR yylval cindex dfn LALR samp gpl BNF xref yypush
15957 @c LocalWords: const int paren ifnotinfo AC noindent emph expr stmt findex lr
15958 @c LocalWords: glr YYSTYPE TYPENAME prog dprec printf decl init stmtMerge POSIX
15959 @c LocalWords: pre STDC GNUC endif yy YY alloca lf stddef stdlib YYDEBUG yypull
15960 @c LocalWords: NUM exp subsubsection kbd Ctrl ctype EOF getchar isdigit nonfree
15961 @c LocalWords: ungetc stdin scanf sc calc ulator ls lm cc NEG prec yyerrok rr
15962 @c LocalWords: longjmp fprintf stderr yylloc YYLTYPE cos ln Stallman Destructor
15963 @c LocalWords: symrec val tptr FUN func struct sym enum IEC syntaxes Byacc
15964 @c LocalWords: fun putsym getsym arith funs atan ptr malloc sizeof Lex pcc
15965 @c LocalWords: strlen strcpy fctn strcmp isalpha symbuf realloc isalnum DOTDOT
15966 @c LocalWords: ptypes itype YYPRINT trigraphs yytname expseq vindex dtype Unary
15967 @c LocalWords: Rhs YYRHSLOC LE nonassoc op deffn typeless yynerrs nonterminal
15968 @c LocalWords: yychar yydebug msg YYNTOKENS YYNNTS YYNRULES YYNSTATES reentrant
15969 @c LocalWords: cparse clex deftypefun NE defmac YYACCEPT YYABORT param yypstate
15970 @c LocalWords: strncmp intval tindex lvalp locp llocp typealt YYBACKUP subrange
15971 @c LocalWords: YYEMPTY YYEOF YYRECOVERING yyclearin GE def UMINUS maybeword loc
15972 @c LocalWords: Johnstone Shamsa Sadaf Hussain Tomita TR uref YYMAXDEPTH inline
15973 @c LocalWords: YYINITDEPTH stmts ref initdcl maybeasm notype Lookahead ctx
15974 @c LocalWords: hexflag STR exdent itemset asis DYYDEBUG YYFPRINTF args Autoconf
15975 @c LocalWords: ypp yxx itemx tex leaderfill Troubleshouting sqrt Graphviz
15976 @c LocalWords: hbox hss hfill tt ly yyin fopen fclose ofirst gcc ll lookahead
15977 @c LocalWords: nbar yytext fst snd osplit ntwo strdup AST Troublereporting th
15978 @c LocalWords: YYSTACK DVI fdl printindex IELR nondeterministic nonterminals ps
15979 @c LocalWords: subexpressions declarator nondeferred config libintl postfix LAC
15980 @c LocalWords: preprocessor nonpositive unary nonnumeric typedef extern rhs sr
15981 @c LocalWords: yytokentype destructor multicharacter nonnull EBCDIC nterm LR's
15982 @c LocalWords: lvalue nonnegative XNUM CHR chr TAGLESS tagless stdout api TOK
15983 @c LocalWords: destructors Reentrancy nonreentrant subgrammar nonassociative Ph
15984 @c LocalWords: deffnx namespace xml goto lalr ielr runtime lex yacc yyps env
15985 @c LocalWords: yystate variadic Unshift NLS gettext po UTF Automake LOCALEDIR
15986 @c LocalWords: YYENABLE bindtextdomain Makefile DEFS CPPFLAGS DBISON DeRemer
15987 @c LocalWords: autoreconf Pennello multisets nondeterminism Generalised baz ACM
15988 @c LocalWords: redeclare automata Dparse localedir datadir XSLT midrule Wno
15989 @c LocalWords: multitable headitem hh basename Doxygen fno filename gdef de
15990 @c LocalWords: doxygen ival sval deftypemethod deallocate pos deftypemethodx
15991 @c LocalWords: Ctor defcv defcvx arg accessors CPP ifndef CALCXX YYerror
15992 @c LocalWords: lexer's calcxx bool LPAREN RPAREN deallocation cerrno climits
15993 @c LocalWords: cstdlib Debian undef yywrap unput noyywrap nounput zA yyleng
15994 @c LocalWords: errno strtol ERANGE str strerror iostream argc argv Javadoc PSLR
15995 @c LocalWords: bytecode initializers superclass stype ASTNode autoboxing nls
15996 @c LocalWords: toString deftypeivar deftypeivarx deftypeop YYParser strictfp
15997 @c LocalWords: superclasses boolean getErrorVerbose setErrorVerbose deftypecv
15998 @c LocalWords: getDebugStream setDebugStream getDebugLevel setDebugLevel url
15999 @c LocalWords: bisonVersion deftypecvx bisonSkeleton getStartPos getEndPos
16000 @c LocalWords: getLVal defvar deftypefn deftypefnx gotos msgfmt Corbett LALR's
16001 @c LocalWords: subdirectory Solaris nonassociativity perror schemas Malloy ints
16002 @c LocalWords: Scannerless ispell american ChangeLog smallexample CSTYPE CLTYPE
16003 @c LocalWords: clval CDEBUG cdebug deftypeopx yyterminate LocationType yyo
16004 @c LocalWords: parsers parser's documentencoding documentlanguage Wempty ss
16005 @c LocalWords: associativity subclasses precedences unresolvable runnable
16006 @c LocalWords: allocators subunit initializations unreferenced untyped dir
16007 @c LocalWords: errorVerbose subtype subtypes Wmidrule midrule's src rvalues
16008 @c LocalWords: automove evolutions Wother Wconflicts PNG lookaheads Acc sep
16009 @c LocalWords: xsltproc XSL xsl xhtml html num Wprecedence Werror fcaret gv
16010 @c LocalWords: fdiagnostics setlocale nullptr ast srcdir iff drv rgbWarning
16011 @c LocalWords: deftypefunx pragma Wnull dereference Wdocumentation elif ish
16012 @c LocalWords: Wdeprecated Wregister noinput yyloc yypos PODs sstream Wsign
16013 @c LocalWords: typename emplace Wconversion Wshorten yacchack reentrancy ou
16014 @c LocalWords: Relocatability exprs fixit Wyacc parseable fixits ffixit svg
16015 @c LocalWords: DNDEBUG cstring Wzero workalike POPL workalikes byacc UCB
16016 @c LocalWords: Penello's Penello Byson Byson's Corbett's CSD TOPLAS PDP cex
16017 @c LocalWords: Beazley's goyacc ocamlyacc SIGACT SIGPLAN colorWarning exVal
16018 @c LocalWords: setcolor rgbError colorError rgbNotice colorNotice derror
16019 @c LocalWords: colorOff maincolor inlineraw darkviolet darkcyan dwarning
16020 @c LocalWords: dnotice copyable stdint ptrdiff bufsize yyreport invariants
16021 @c LocalWords: xrefautomaticsectiontitle yysyntax yysymbol ARGMAX cond RTTI
16022 @c LocalWords: Wdangling yytoken erreur syntaxe inattendu attendait nombre
16023 @c LocalWords: YYUNDEF SymbolKind yypcontext YYENOMEM TOKENMAX getBundle
16024 @c LocalWords: ResourceBundle myResources getString getName getToken ylwrap
16025 @c LocalWords: getLocation getExpectedTokens reportSyntaxError bistromathic
16026 @c LocalWords: TokenKind Automake's rtti Wcounterexamples Chinawat PLDI
16027 @c LocalWords: Isradisaikul tcite pcite rgbGreen colorGreen rgbYellow Wcex
16028 @c LocalWords: colorYellow rgbRed colorRed rgbBlue colorBlue rgbPurple
16029 @c LocalWords: colorPurple ifhtml ifnothtml situ rcex MERCHANTABILITY Wnone
16031 @c Local Variables:
16032 @c ispell-dictionary: "american"