3 commit c22902e360e0fbbe9fd5657dcf107e03166da309
4 Author: Akim Demaille <akim.demaille@gmail.com>
5 Date: Sat Jan 23 18:40:15 2021 +0100
7 tables: fix handling for useless tokens
9 See https://github.com/akimd/bison/issues/72#issuecomment-766153154
11 commit 2c294c132528ede23d8ae4959783a67e9ff05ac5
12 Author: Vincent Imbimbo <vmi6@cornell.edu>
13 Date: Sat Jan 23 13:25:18 2021 -0500
15 cex: fix state-item pruning
17 See https://lists.gnu.org/r/bug-bison/2021-01/msg00002.html
20 The current approach is correct, but with poor performances. Bitsets need
21 to support 'assign' and 'shift'. And instead of extending POS_SET just for
22 the out-of-range new values, we need something like doubling the size.
25 This is an internal detail. But it's very handy, we should make it public.
26 It already has leaked in the documentation by accident.
29 There is no test with "Parse on stack %ld rejected by rule %d" in it.
32 Tests with typed %merge: 716 717 718 740 741 742 746 747 748
34 716: Duplicate representation of merged trees: glr.c FAILED (glr-regression.at:517)
35 740: Leaked semantic values if user action cuts parse: glr.c FAILED (glr-regression.at:1230)
36 746: Incorrect lookahead during nondeterministic GLR: glr.c FAILED (glr-regression.at:1610)
38 Document typed merges.
41 Clarify that rule numbers in the skeletons are 1-based.
44 There are many macros that should obey api.prefix: YY_CPLUSPLUS, YY_MOVE,
47 ** YYDEBUG etc. in C++
48 Discourage the use of YYDEBUG in C++ (see thread with Jot). Stop supporting
49 #define YYSTYPE by the user.
51 Add value_type as a synonym for semantic_type.
54 And add tests in calc.at, to prepare work for D.
56 ** YYERROR and yynerrs
57 We are missing some cases. Write a test case, and check all the skeletons.
61 Don't do this (counterexample.c):
63 // This is the fastest way to get the tail node from the gl_list API.
65 list_get_end (gl_list_t list)
67 gl_list_node_t sentinel = gl_list_add_last (list, NULL);
68 gl_list_node_t res = gl_list_previous_node (list, sentinel);
69 gl_list_remove_node (list, sentinel);
73 *** Ambiguous rewriting
74 If the user is stupid enough to have equal rules, then the derivations are
77 Reduce/reduce conflict on tokens $end, "+", "⊕":
81 First derivation exp ::=[ exp "+" exp • ]
83 Second derivation exp ::=[ exp "+" exp • ]
85 Do we care about this? In color, we use twice the same color here, but we
86 could try to use the same color for the same rule.
89 Show the counterexamples. This is going to be really hard and/or painful.
90 Unless we play it dumb (little structure).
93 - How about not evaluating incomplete lines when the text is not finished
98 - Should i18n be part of the Lexer? Currently it's a static method of
101 - is there a migration path that would allow to use TokenKinds in
104 - define the tokens as an enum too.
106 - promote YYEOF rather than EOF.
109 https://git.savannah.gnu.org/gitweb/?p=gettext.git;a=blob;f=gettext-runtime/intl/plural.y;h=a712255af4f2f739c93336d4ff6556d932a426a5;hb=HEAD
111 should be updated to not use YYERRCODE. Returning an undef token is good
116 Stop hard-coding "Calc". Adjust local.at (look for FIXME).
118 ** A dev warning for b4_
119 Maybe we should check for m4_ and b4_ leaking out of the m4 processing, as
120 Autoconf does. It would have caught over-quotation issues.
123 I feel it's ugly to use the GNU style to declare functions in the doc. It
124 generates tons of white space in the page, and may contribute to bad page
131 The YYUNDEFTOK could be assigned a semantic value so that yyerror could be
132 used to report invalid lexemes.
135 Consider deprecating impure push parsers. They add a lot of complexity, for
136 a bad feature. On the other hand, that would make it much harder to sit
137 push parsers on top of pull parser. Which is currently not relevant, since
138 push parsers are measurably slower.
140 ** %define parse.error formatted
141 How about pushing Bistromathic's yyreport_syntax_error as another standard
142 way to generate the error message, and leave to the user the task of
143 providing the message formats? Currently in bistro, it reads:
146 error_format_string (int argc)
150 default: /* Avoid compiler warnings. */
151 case 0: return _("%@: syntax error");
152 case 1: return _("%@: syntax error: unexpected %u");
153 // TRANSLATORS: '%@' is a location in a file, '%u' is an
154 // "unexpected token", and '%0e', '%1e'... are expected tokens
157 // For instance on the expression "1 + * 2", you'd get
159 // 1.5: syntax error: expected - or ( or number or function or variable before *
160 case 2: return _("%@: syntax error: expected %0e before %u");
161 case 3: return _("%@: syntax error: expected %0e or %1e before %u");
162 case 4: return _("%@: syntax error: expected %0e or %1e or %2e before %u");
163 case 5: return _("%@: syntax error: expected %0e or %1e or %2e or %3e before %u");
164 case 6: return _("%@: syntax error: expected %0e or %1e or %2e or %3e or %4e before %u");
165 case 7: return _("%@: syntax error: expected %0e or %1e or %2e or %3e or %4e or %5e before %u");
166 case 8: return _("%@: syntax error: expected %0e or %1e or %2e or %3e or %4e or %5e or %6e before %u");
170 The message would have to be generated in a string, and pushed to yyerror.
171 Which will be a pain in the neck in yacc.c.
173 If we want to do that, we should think very carefully about the syntax of
176 ** yyclearin does not invoke the lookahead token's %destructor
177 https://lists.gnu.org/r/bug-bison/2018-02/msg00000.html
180 > Modifying yyclearin so that it calls yydestruct seems like the simplest
181 > solution to this issue, but it is conceivable that such a change would
182 > break programs which already perform some kind of workaround in order to
183 > destruct the lookahead symbol. So it might be necessary to use some kind of
184 > compatibility %define, or to create a new replacement macro with a
185 > different name such as yydiscardin.
187 > At a minimum, the fact that yyclearin does not invoke the %destructor
188 > should be highlighted in the documentation, since it is not at all obvious.
192 Les catégories d'avertissements incluent :
193 conflicts-sr conflits S/R (activé par défaut)
194 conflicts-rr conflits R/R (activé par défaut)
195 dangling-alias l'alias chaîne n'est pas attaché à un symbole
196 deprecated construction obsolète
197 empty-rule règle vide sans %empty
198 midrule-values valeurs de règle intermédiaire non définies ou inutilisées
199 precedence priorité et associativité inutiles
200 yacc incompatibilités avec POSIX Yacc
201 other tous les autres avertissements (activé par défaut)
202 all tous les avertissements sauf « dangling-alias » et « yacc »
203 no-CATEGORY désactiver les avertissements dans CATEGORIE
204 none désactiver tous les avertissements
205 error[=CATEGORY] traiter les avertissements comme des erreurs
207 Line -1 and -3 should mention CATEGORIE, not CATEGORY.
210 ** Rewrite glr.cc (currently glr2.cc)
212 We can probably replace setjmp/longjmp with exceptions. That would help
213 tremendously other languages such as D and Java that probably have no
214 similar feature. If we remove jumps, we probably no longer need _Noreturn,
215 so simplify `b4_attribute_define([noreturn])` into `b4_attribute_define`.
217 After discussing with Valentin, it was decided that it's better to stay with
218 jumps, since in some places exceptions are ruled out from C++.
221 Move to our coding conventions. In particular names such as yy_glr_stack,
225 It should be a member of the parser object, see lalr1.cc. Let the parser
226 object decide what the debug stream is, rather than open coding std::cerr.
229 There are many places where pointers should be replaced with references.
230 Some occurrences were fixed, but now some have improper names:
232 -yygetToken (int *yycharp, ]b4_namespace_ref[::]b4_parser_class[& yyparser][]b4_pure_if([, glr_stack* yystackp])[]b4_user_formals[)
233 +yygetToken (int& yycharp, ]b4_namespace_ref[::]b4_parser_class[& yyparser][]b4_pure_if([, glr_stack* yystackp])[]b4_user_formals[)
235 yycharp is no longer a Pointer. And yystackp should probably also be a reference.
238 Currently all the assertions are enabled. Once we are confident in glr2.cc,
239 let parse.assert use the same approach as in lalr1.cc.
242 Stop using std::cerr everywhere.
245 When glr2.cc fully replaces glr.cc, get rid of the glr.cc scaffolding in
249 ** Unit rules / Injection rules (Akim Demaille)
250 Maybe we could expand unit rules (or "injections", see
251 https://homepages.cwi.nl/~daybuild/daily-books/syntax/2-sdf/sdf.html), i.e.,
260 exp: exp '+' exp | exp '&' exp;
262 when there are no actions. This can significantly speed up some grammars.
263 I can't find the papers. In particular the book 'LR parsing: Theory and
264 Practice' is impossible to find, but according to 'Parsing Techniques: a
265 Practical Guide', it includes information about this issue. Does anybody
268 ** clean up (Akim Demaille)
269 Do not work on these items now, as I (Akim) have branches with a lot of
270 changes in this area (hitting several files), and no desire to have to fix
271 conflicts. Addressing these items will happen after my branches have been
275 Introduce a goto struct, and use it in place of from_state/to_state.
276 Rename states1 as path, length as pathlen.
277 Introduce inline functions for things such as nullable[*rp - ntokens]
278 where we need to map from symbol number to nterm number.
280 There are probably a significant part of the relations management that
281 should be migrated on top of a bitsetv.
284 It should probably take a "state*" instead of two arguments.
287 The "automaton" and "set" categories are not so useful. We should probably
288 introduce lr(0) and lalr, just the way we have ielr categories. The
289 "closure" function is too verbose, it should probably have its own category.
291 "set" can still be used for summarizing the important sets. That would make
292 tests easy to maintain.
295 Rename these guys as "diagnostics.*" (or "diagnose.*"), since that's the
296 name they have in GCC, clang, etc. Likewise for the complain_* series of
300 states/nstates, rules/nrules, ..., ritem/nritems
303 *** m4: slot, type, type_tag
304 The meaning of type_tag varies depending on api.value.type. We should avoid
305 that and using clear definitions with stable semantics.
307 * D programming language
308 There's a number of features that are missing, here sorted in _suggested_
309 order of implementation.
311 When copying code from other skeletons, keep the comments exactly as they
312 are. Keep the same variable names. If you change the wording in one place,
313 do it in the others too. In other words: make sure to keep the
314 maintenance *simple* by avoiding any gratuitous difference.
317 Check when gdc and ldc.
319 ** Token Constructors
320 It is possible to mix incorrectly kinds and values, and for instance:
322 return parser.Symbol (TokenKind.NUM, "Hello, World!\n");
324 attaches a string value to NUM kind (wrong, of course). When
325 api.token.constructor is set, in C++, Bison generated "token constructors":
326 parser.make_NUM. parser.make_PLUS, parser.make_STRING, etc. The previous
329 return parser.make_NUM ("Hello, World!\n");
331 which would easily be caught by the type checker.
334 Add support for push parser. Do not start a nice skeleton, just enhance the
335 current one to support push parsers. This is going to be a tougher nut to
338 First, you need to understand well how the push parser is expected to work.
341 - look at examples/c/pushcalc
342 - create an example of a Java push parser.
343 - have a look at the generated parser in Java, which has the advantage of
344 being already based on a parser object, instead of just a function.
346 The C case is harder to read, but it may help too. Keep in mind that
347 because there's no object to maintain state, the C push parser uses some
348 struct (yypstate) to preserve this state. We don't need this in D, the
349 parser object will suffice.
351 I think working directly on the skeleton to add push-parser support is not
352 the simplest path. I suggest that you (1) transform a generated parser into
353 a push parser by hand, and then (2) transform lalr1.d to generate such a
356 Use `git commit` frequently to make sure you keep track of your progress.
358 *** (1.a) Prepare pull parser by hand
359 Copy again one of the D examples into say examples/d/pushcalc. Also
360 check-in the generated parser to facilitate experimentation.
362 - find local variables of yyparse should become members of the parser object
363 (so that we preserve state from one call to the next).
365 - do it in your generated D parser. We don't need an equivalent for
366 yypstate, because we already have it: that the parser object itself.
368 - have your *pull*-parser (i.e., the good old yy::parser::parse()) work
369 properly this way. Write and run tests. That's one of the reasons I
370 suggest using examples/d/calc as a starting point: it already has tests,
371 you can/should add more.
373 At this point you have a pull-parser which you prepared to turn into a
376 *** (1.b) Turn pull parser into push parser by hand
378 - look again at how push parsers are implemented in Java/C to see what needs
379 to change in yyparse so that the control is inverted: parse() will
380 be *given* the tokens, instead of having to call yylex itself. When I say
381 "look at C", I think your best option are (i) yacc.c (look for b4_push_if)
382 and (ii) examples/c/pushcalc.
384 - rename parse() as push_parse(Symbol yyla) (or push_parse(TokenKind, Value,
385 Location)) that takes the symbol as argument. That's the push parser we
388 - define a new parse() function which has the same signature as the usual
389 pull-parser, that repeatedly calls the push_parse function. Something
396 status = this->push_parse (yylex());
397 } while (status == YYPUSH_MORE);
401 - show me that parser, so that we can validate the approach.
403 *** (2) Port that into the skeleton
404 - once we agree on the API of the push parser, implement it into lalr1.d.
405 You will probaby need help on this regard, but imitation, again, should
408 - have example/d/pushcalc work properly and pass tests
410 - add tests in the "real" test suite. Do that in tests/calc.at. I can
416 This is very ambitious. That's the final boss. There are currently no
417 "clean" implementation to get inspiration from.
419 glr.c is very clean but:
421 - is a different skeleton from yacc.c
423 glr.cc is (currently) an ugly hack: a C++ shell around glr.c. Valentin
424 Tolmer is currently rewriting glr.cc to be clean C++, but he is not
425 finished. There will be a lot a common code between lalr1.cc and glr.cc, so
426 eventually I would like them to be fused into a single skeleton, supporting
427 both deterministic and generalized parsing.
429 It would be great for D to also support this.
431 The basic ideas of GLR are explained here:
433 https://www.codeproject.com/Articles/5259825/GLR-Parsing-in-Csharp-How-to-Use-The-Most-Powerful
435 * Better error messages
436 The users are not provided with enough tools to forge their error messages.
437 See for instance "Is there an option to change the message produced by
438 YYERROR_VERBOSE?" by Simon Sobisch, on bison-help.
441 https://www.cs.tufts.edu/~nr/cs257/archive/clinton-jefferey/lr-error-messages.pdf
442 https://research.swtch.com/yyerror
443 http://gallium.inria.fr/~fpottier/publis/fpottier-reachability-cc2016.pdf
446 Fix data/skeletons/yacc.c so that it defines YYPTRDIFF_T properly for modern
447 and older C++ compilers. Currently the code defaults to defining it to
448 'long' for non-GCC compilers, but it should use the proper C++ magic to
449 define it to the same type as the C ptrdiff_t type.
452 Several features are not available in all the back-ends.
454 - push parsers: glr.c, glr.cc, lalr1.cc (not very difficult)
455 - token constructors: Java, C, D (a bit difficult)
456 - glr: D, Java (super difficult)
459 ** Autotest has quotation issues
460 tests/input.at:1730:AT_SETUP([%define errors])
464 $ ./tests/testsuite -l | grep errors | sed q
465 38: input.at:1730 errors
468 ** Get rid of YYPRINT and b4_toknum
469 Besides yytoknum is wrong when api.token.raw is defined.
471 ** Better design for diagnostics
472 The current implementation of diagnostics is ad hoc, it grew organically.
473 It works as a series of calls to several functions, with dependency of the
474 latter calls on the former. For instance:
476 complain (&sym->location,
477 sym->content->status == needed ? complaint : Wother,
478 _("symbol %s is used, but is not defined as a token"
479 " and has no rules; did you mean %s?"),
480 quote_n (0, sym->tag),
481 quote_n (1, best->tag));
482 if (feature_flag & feature_caret)
483 location_caret_suggestion (sym->location, best->tag, stderr);
485 We should rewrite this in a more FP way:
487 1. build a rich structure that denotes the (complete) diagnostic.
488 "Complete" in the sense that it also contains the suggestions, the list
489 of possible matches, etc.
491 2. send this to the pretty-printing routine. The diagnostic structure
492 should be sufficient so that we can generate all the 'format' of
493 diagnostics, including the fixits.
495 If properly done, this diagnostic module can be detached from Bison and be
496 put in gnulib. It could be used, for instance, for errors caught by
499 There's certainly already something alike in GCC. At least that's the
500 impression I get from reading the "-fdiagnostics-format=FORMAT" part of this
503 https://gcc.gnu.org/onlinedocs/gcc/Diagnostic-Message-Formatting-Options.html
505 ** Graphviz display code thoughts
506 The code for the --graph option is over two files: print_graph, and
507 graphviz. This is because Bison used to also produce VCG graphs, but since
508 this is no longer true, maybe we could consider these files for fusion.
510 An other consideration worth noting is that print_graph.c (correct me if I
511 am wrong) should contain generic functions, whereas graphviz.c and other
512 potential files should contain just the specific code for that output
513 format. It will probably prove difficult to tell if the implementation is
514 actually generic whilst only having support for a single format, but it
515 would be nice to keep stuff a bit tidier: right now, the construction of the
516 bitset used to show reductions is in the graphviz-specific code, and on the
517 opposite side we have some use of \l, which is graphviz-specific, in what
518 should be generic code.
520 Little effort seems to have been given to factoring these files and their
521 print{,-xml} counterpart. We would very much like to re-use the pretty format
522 of states from .output for the graphs, etc.
524 Since graphviz dies on medium-to-big grammars, maybe consider an other tool?
527 Check it too when checking the different kinds of parsers. And be
528 sure to check that the initial-action is performed once per parsing.
531 b4_shared_declarations is no longer what it is. Make it
532 b4_parser_declaration for instance.
534 ** yychar in lalr1.cc
535 There is a large difference bw maint and master on the handling of
536 yychar (which was removed in lalr1.cc). See what needs to be
540 /* User semantic actions sometimes alter yychar, and that requires
541 that yytoken be updated with the new translation. We take the
542 approach of translating immediately before every use of yytoken.
543 One alternative is translating here after every semantic action,
544 but that translation would be missed if the semantic action
545 invokes YYABORT, YYACCEPT, or YYERROR immediately after altering
546 yychar. In the case of YYABORT or YYACCEPT, an incorrect
547 destructor might then be invoked immediately. In the case of
548 YYERROR, subsequent parser actions might lead to an incorrect
549 destructor call or verbose syntax error message before the
550 lookahead is translated. */
552 /* Make sure we have latest lookahead translation. See comments at
553 user semantic actions for why this is necessary. */
554 yytoken = yytranslate_ (yychar);
557 ** Get rid of fake #lines [Bison: ...]
558 Possibly as simple as checking whether the column number is nonnegative.
560 I have seen messages like the following from GCC.
562 <built-in>:0: fatal error: opening dependency file .deps/libltdl/argz.Tpo: No such file or directory
565 ** Discuss about %printer/%destroy in the case of C++.
566 It would be very nice to provide the symbol classes with an operator<<
567 and a destructor. Unfortunately the syntax we have chosen for
568 %destroy and %printer make them hard to reuse. For instance, the user
569 is invited to write something like
571 %printer { debug_stream() << $$; } <my_type>;
573 which is hard to reuse elsewhere since it wants to use
574 "debug_stream()" to find the stream to use. The same applies to
575 %destroy: we told the user she could use the members of the Parser
576 class in the printers/destructors, which is not good for an operator<<
577 since it is no longer bound to a particular parser, it's just a
581 ** Rewrite glr.cc in C++ (Valentin Tolmer)
582 As a matter of fact, it would be very interesting to see how much we can
583 share between lalr1.cc and glr.cc. Most of the skeletons should be common.
584 It would be a very nice source of inspiration for the other languages.
586 Valentin Tolmer is working on this.
588 * From lalr1.cc to yacc.c
590 Merging the three stacks in lalr1.cc simplified the code, prompted for
591 other improvements and also made it faster (probably because memory
592 management is performed once instead of three times). I suggest that
593 we do the same in yacc.c.
595 (Some time later): it's also very nice to have three stacks: it's more dense
596 as we don't lose bits to padding. For instance the typical stack for states
597 will use 8 bits, while it is likely to consume 32 bits in a struct.
599 We need trustworthy benchmarks for Bison, for all our backends. Akim has a
600 few things scattered around; we need to put them in the repo, and make them
606 Some statistics about the grammar and the parser would be useful,
607 especially when asking the user to send some information about the
608 grammars she is working on. We should probably also include some
609 information about the variables (I'm not sure for instance we even
610 specify what LR variant was used).
613 How would Paul like to display the conflicted actions? In particular,
614 what when two reductions are possible on a given lookahead token, but one is
615 part of $default. Should we make the two reductions explicit, or just
616 keep $default? See the following point.
618 ** Disabled Reductions
619 See 'tests/conflicts.at (Defaulted Conflicted Reduction)', and decide
623 Extend with error productions. The hard part will probably be finding
624 the right rule so that a single state does not exhibit too many yet
625 undocumented ''features''. Maybe an empty action ought to be
626 presented too. Shall we try to make a single grammar with all these
627 features, or should we have several very small grammars?
631 Well, only if there is really some demand for it.
634 https://github.com/scfc/bison-php/blob/master/data/lalr1.php
637 https://lists.gnu.org/r/bison-patches/2013-09/msg00000.html and following
639 ** Multiple start symbols
640 Would be very useful when parsing closely related languages. The idea is to
641 declare several start symbols, for instance
648 and to generate parse(), parse_stmt() and parse_expr(). Technically, the
649 above grammar would be transformed into
652 %token YY_START_STMT YY_START_EXPR
654 yy_start: YY_START_STMT stmt | YY_START_EXPR expr
656 so that there are no new conflicts in the grammar (as would undoubtedly
657 happen with yy_start: stmt | expr). Then adjust the skeletons so that this
658 initial token (YY_START_STMT, YY_START_EXPR) be shifted first in the
659 corresponding parse function.
661 *** Number of useless symbols
665 [[input.y: warning: 2 nonterminals useless in grammar [-Wother]
666 input.y: warning: 2 rules useless in grammar [-Wother]
667 input.y:2.8-10: error: start symbol exp does not derive any sentence]])
669 We should say "1 nonterminal": the other one is $accept, which should not
670 participate in the count.
673 Do we want to disallow terminal start symbols? The limitation is not
674 technical. Can it be useful to someone to "parse" a token?
677 This is a popular demand. We already made many changes in the parser that
678 should make this reasonably easy to implement.
680 Bruce Mardle <marblypup@yahoo.co.uk>
681 https://lists.gnu.org/r/bison-patches/2015-09/msg00000.html
683 However, there are many other things to do before having such a feature,
684 because I don't want a % equivalent to #include (which we all learned to
685 hate). I want something that builds "modules" of grammars, and assembles
686 them together, paying attention to keep separate bits separated, in pseudo
690 There is demand for push parsers in C++.
692 ** Generate code instead of tables
693 This is certainly quite a lot of work. See
694 https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.50.4539.
697 We should find a means to provide an access to values deep in the
698 stack. For instance, instead of
700 baz: qux { $$ = $<foo>-1 + $<bar>0 + $1; }
702 we should be able to have:
704 foo($foo) bar($bar) baz($bar): qux($qux) { $baz = $foo + $bar + $qux; }
706 Or something like this.
709 It should be possible to have %if/%else/%endif. The implementation is
710 not clear: should it be lexical or syntactic. Vadim Maslow thinks it
711 must be in the scanner: we must not parse what is in a switched off
712 part of %if. Akim Demaille thinks it should be in the parser, so as
713 to avoid falling into another CPP mistake.
715 (Later): I'm sure there's actually good case for this. People who need that
716 feature can use m4/cpp on top of Bison. I don't think it is worth the
717 trouble in Bison itself.
720 There are couple of available extensions of Bison targeting some XML
721 output. Some day we should consider including them. One issue is
722 that they seem to be quite orthogonal to the parsing technique, and
723 seem to depend mostly on the possibility to have some code triggered
724 for each reduction. As a matter of fact, such hooks could also be
725 used to generate the yydebug traces. Some generic scheme probably
728 XML output for GNU Bison and gcc
729 http://www.cs.may.ie/~jpower/Research/bisonXML/
731 XML output for GNU Bison
732 http://yaxx.sourceforge.net/
734 * Coding system independence
737 Currently Bison assumes 8-bit bytes (i.e. that UCHAR_MAX is
738 255). It also assumes that the 8-bit character encoding is
739 the same for the invocation of 'bison' as it is for the
740 invocation of 'cc', but this is not necessarily true when
741 people run bison on an ASCII host and then use cc on an EBCDIC
742 host. I don't think these topics are worth our time
743 addressing (unless we find a gung-ho volunteer for EBCDIC or
744 PDP-10 ports :-) but they should probably be documented
747 More importantly, Bison does not currently allow NUL bytes in
748 tokens, either via escapes (e.g., "x\0y") or via a NUL byte in
749 the source code. This should get fixed.
754 Must we keep %token-table?
759 It is unfortunate that there is a total order for precedence. It
760 makes it impossible to have modular precedence information. We should
761 move to partial orders (sounds like series/parallel orders to me).
763 This is a prerequisite for modules.
765 * Pre and post actions.
766 From: Florian Krohm <florian@edamail.fishkill.ibm.com>
767 Subject: YYACT_EPILOGUE
768 To: bug-bison@gnu.org
769 X-Sent: 1 week, 4 days, 14 hours, 38 minutes, 11 seconds ago
771 The other day I had the need for explicitly building the parse tree. I
772 used %locations for that and defined YYLLOC_DEFAULT to call a function
773 that returns the tree node for the production. Easy. But I also needed
774 to assign the S-attribute to the tree node. That cannot be done in
775 YYLLOC_DEFAULT, because it is invoked before the action is executed.
776 The way I solved this was to define a macro YYACT_EPILOGUE that would
777 be invoked after the action. For reasons of symmetry I also added
778 YYACT_PROLOGUE. Although I had no use for that I can envision how it
779 might come in handy for debugging purposes.
780 All is needed is to add
783 YYACT_EPILOGUE (yyval, (yyvsp - yylen), yylen, yyloc, (yylsp - yylen));
785 YYACT_EPILOGUE (yyval, (yyvsp - yylen), yylen);
788 at the proper place to bison.simple. Ditto for YYACT_PROLOGUE.
790 I was wondering what you think about adding YYACT_PROLOGUE/EPILOGUE
791 to bison. If you're interested, I'll work on a patch.
794 Equip the parser with a means to create the (visual) parse tree.
799 # LocalWords: Cex gnulib gl Bistromathic TokenKinds yylex enum YYEOF EOF
800 # LocalWords: YYerror gettext af hb YYERRCODE undef calc FIXME dev yyerror
801 # LocalWords: Autoconf YYUNDEFTOK lexemes parsers Bistromathic's yyreport
802 # LocalWords: const argc yacc yyclearin lookahead destructor Rici incluent
803 # LocalWords: yydestruct yydiscardin catégories d'avertissements sr activé
804 # LocalWords: conflits défaut rr l'alias chaîne n'est attaché un symbole
805 # LocalWords: obsolète règle vide midrule valeurs de intermédiaire ou avec
806 # LocalWords: définies inutilisées priorité associativité inutiles POSIX
807 # LocalWords: incompatibilités tous les autres avertissements sauf dans rp
808 # LocalWords: désactiver CATEGORIE traiter comme des erreurs glr Akim bool
809 # LocalWords: Demaille arith lalr goto struct pathlen nullable ntokens lr
810 # LocalWords: nterm bitsetv ielr ritem nstates nrules nritems yysymbol EQ
811 # LocalWords: SymbolKind YYEMPTY YYUNDEF YYTNAME NUM yyntokens yytname sed
812 # LocalWords: nonterminals yykind yycode YYNAMES yynames init getName conv
813 # LocalWords: TokenKind ival yychar yylval yylexer Tolmer hoc
814 # LocalWords: Sobisch YYPTRDIFF ptrdiff Autotest YYPRINT toknum yytoknum
815 # LocalWords: sym Wother stderr FP fixits xgettext fdiagnostics Graphviz
816 # LocalWords: graphviz VCG bitset xml bw maint yytoken YYABORT deps
817 # LocalWords: YYACCEPT yytranslate nonnegative destructors yyerrlab repo
818 # LocalWords: backends stmt expr yy Mardle baz qux Vadim Maslow CPP cpp
819 # LocalWords: yydebug gcc UCHAR EBCDIC gung PDP NUL Pre Florian Krohm utf
820 # LocalWords: YYACT YYLLOC YYLSP yyval yyvsp yylen yyloc yylsp endif
821 # LocalWords: ispell american
827 ispell-dictionary: "american"
830 Copyright (C) 2001-2004, 2006, 2008-2015, 2018-2021 Free Software
833 This file is part of Bison, the GNU Compiler Compiler.
835 Permission is granted to copy, distribute and/or modify this document
836 under the terms of the GNU Free Documentation License, Version 1.3 or
837 any later version published by the Free Software Foundation; with no
838 Invariant Sections, with no Front-Cover Texts, and with no Back-Cover
839 Texts. A copy of the license is included in the "GNU Free
840 Documentation License" file as part of this distribution.