3 <style|<tuple|book|fangle|header-book|tmdoc-keyboard>>
6 <hide-preamble|<assign|LyX|<macro|L<space|-0.1667em><move|Y|0fn|-0.25em><space|-0.125em>X>><assign|par-first|0fn><assign|par-par-sep|0.5fn>>
8 <doc-data|<doc-title|fangle>|<doc-author-data|<author-name|Sam
9 Liddicott>|<\author-address>
11 </author-address>>|<doc-date|August 2009>>
13 <section*|Introduction>
15 <name|Fangle> is a tool for fangled literate programming. Newfangled is
16 defined as <em|New and often needlessly novel> by
17 <name|TheFreeDictionary.com>.
19 In this case, fangled means yet another not-so-new<footnote|but improved.>
20 method for literate programming.
22 <name|Literate Programming> has a long history starting with the great
23 <name|Donald Knuth> himself, whose literate programming tools seem to make
24 use of as many escape sequences for semantic markup as <TeX> (also by
27 <name|Norman Ramsey> wrote the <name|Noweb> set of tools
28 (<verbatim|notangle>, <verbatim|noweave> and <verbatim|noroots>) and
29 helpfully reduced the amount of magic character sequences to pretty much
30 just <verbatim|\<less\>\<less\>>, <verbatim|\<gtr\>\<gtr\>> and
31 <verbatim|@>, and in doing so brought the wonders of literate programming
34 While using the <LyX> editor for <LaTeX> editing I had various troubles
35 with the noweb tools, some of which were my fault, some of which were
36 noweb's fault and some of which were <LyX>'s fault.
38 <name|Noweb> generally brought literate programming to the masses through
39 removing some of the complexity of the original literate programming, but
40 this would be of no advantage to me if the <LyX> / <LaTeX> combination
41 brought more complications in their place.
43 <name|Fangle> was thus born (originally called <name|Newfangle>) as an awk
44 replacement for notangle, adding some important features, like better
45 integration with <LyX> and <LaTeX> (and later <TeXmacs>), multiple output
46 format conversions, and fixing notangle bugs like indentation when using -L
49 Significantly, fangle is just one program which replaces various programs
50 in <name|Noweb>. Noweave is done away with and implemented directly as
51 <LaTeX> macros, and noroots is implemented as a function of the untangler
54 Fangle is written in awk for portability reasons, awk being available for
55 most platforms. A Python version<\footnote>
56 hasn't anyone implemented awk in python yet?
57 </footnote> was considered for the benefit of <LyX> but a scheme version
58 for <TeXmacs> will probably materialise first; as <TeXmacs> macro
59 capabilities help make edit-time and format-time rendering of fangle chunks
60 simple enough for my weak brain.
62 As an extension to many literate-programming styles, Fangle permits code
63 chunks to take parameters and thus operate somewhat like C pre-processor
64 macros, or like C++ templates. Name parameters (or even local
65 <em|variables> in the callers scope) are anticipated, as parameterized
66 chunks <emdash> useful though they are <emdash> are hard to comprehend in
67 the literate document.
69 <section*|License><new-page*><label|License>
71 Fangle is licensed under the GPL 3 (or later).
73 This doesn't mean that sources generated by fangle must be licensed under
76 This doesn't mean that you can't use or distribute fangle with sources of
77 an incompatible license, but it means you must make the source of fangle
80 As fangle is currently written in awk, an interpreted language, this should
83 <\nf-chunk|gpl3-copyright>
84 <item>fangle - fully featured notangle replacement in awk
88 <item>Copyright (C) 2009-2010 Sam Liddicott
89 \<less\>sam@liddicott.com\<gtr\>
93 <item>This program is free software: you can redistribute it and/or
96 <item>it under the terms of the GNU General Public License as published
99 <item>the Free Software Foundation, either version 3 of the License, or
101 <item>(at your option) any later version.
105 <item>This program is distributed in the hope that it will be useful,
107 <item>but WITHOUT ANY WARRANTY; without even the implied warranty of
109 <item>MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. \ See the
111 <item>GNU General Public License for more details.
115 <item>You should have received a copy of the GNU General Public License
117 <item>along with this program. \ If not, see
118 \<less\>http://www.gnu.org/licenses/\<gtr\>.
121 <\table-of-contents|toc>
126 <chapter|Introduction to Literate Programming>
128 Todo: Should really follow on from a part-0 explanation of what literate
131 <chapter|Running Fangle>
133 Fangle is a replacement for <name|noweb>, which consists of
134 <verbatim|notangle>, <verbatim|noroots> and <verbatim|noweave>.
136 Like <verbatim|notangle> and <verbatim|noroots>, <verbatim|fangle> can read
137 multiple named files, or from stdin.
139 <section|Listing roots>
141 The -r option causes fangle to behave like noroots.
143 <code*|fangle -r filename.tex>
145 will print out the fangle roots of a tex file.\
147 Unlike the <verbatim|noroots> command, the printed roots are not enclosed
148 in angle brackets e.g. <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>>,
149 unless at least one of the roots is defined using the <verbatim|notangle>
150 notation <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>=>.
152 Also, unlike noroots, it prints out all roots --- not just those that are
153 not used elsewhere. I find that a root not being used doesn't make it
154 particularly top level <emdash> and so-called top level roots could also be
155 included in another root as well.\
157 My convention is that top level roots to be extracted begin with
158 <verbatim|./> and have the form of a filename.
160 Makefile.inc, discussed in <reference|makefile.inc>, can automatically
161 extract all such sources prefixed with <verbatim|./>
163 <section|Extracting roots>
165 notangle's <verbatim|-R> and <verbatim|-L> options are supported.
167 If you are using <LyX> or <LaTeX>, the standard way to extract a file would
170 <verbatim|fangle -R./Makefile.inc fangle.tex \<gtr\> ./Makefile.inc>
172 If you are using <TeXmacs>, the standard way to extract a file would
175 <verbatim|fangle -R./Makefile.inc fangle.txt \<gtr\> ./Makefile.inc>
177 <TeXmacs> users would obtain the text file with a <em|verbatim> export from
178 <TeXmacs> which can be done on the command line with <verbatim|texmacs -s
179 -c fangle.tm fangle.txt -q>
181 Unlike the <verbatim|noroots> command, the <verbatim|<verbatim|-L>> option
182 to generate C pre-preocessor <verbatim|#file> style line-number
183 directives,does not break indenting of the generated file..
185 Also, thanks to mode tracking (described in <reference|modes>) the
186 <verbatim|-L> option does not interrupt (and break) multi-line C macros
189 This does mean that sometimes the compiler might calculate the source line
190 wrongly when generating error messages in such cases, but there isn't any
191 other way around if multi-line macros include other chunks.
193 Future releases will include a mapping file so that line/character
194 references from the C compiler can be converted to the correct part of the
197 <section|Formatting the document>
199 The noweave replacement built into the editing and formatting environment
200 for <TeXmacs>, <LyX> (which uses <LaTeX>), and even for raw <LaTeX>.
202 Use of fangle with <TeXmacs>, <LyX> and <LaTeX> are explained the the next
205 <chapter|Using Fangle with <LaTeX>>
207 Because the noweave replacement is impemented in <LaTeX>, there is no
208 processing stage required before running the <LaTeX> command. Of course,
209 <LaTeX> may need running two or more times, so that the code chunk
210 references can be fully calculated.
212 The formatting is managed by a set of macros shown in
213 <reference|latex-source>, and can be included with:
215 <verbatim|\\usepackage{fangle.sty}>
217 Norman Ramsay's origial <filename|noweb.sty> package is currently required
218 as it is used for formatting the code chunk captions.
220 The <filename|listings.sty> package is required, and is used for formatting
221 the code chunks and syntax highlighting.
223 The <filename|xargs.sty> package is also required, and makes writing
224 <LaTeX> macro so much more pleasant.
226 <todo|Add examples of use of Macros>
228 <chapter|Using Fangle with <LyX>>
230 <LyX> uses the same <LaTeX> macros shown in <reference|latex-source> as
231 part of a <LyX> module file <filename|fangle.module>, which automatically
232 includes the macros in the document pre-amble provided that the fangle
233 <LyX> module is used in the document.
235 <section|Installing the <LyX> module>
237 Copy <filename|fangle.module> to your <LyX> layouts directory, which for
238 unix users will be <filename|~/.lyx/layouts>
240 In order to make the new literate styles availalble, you will need to
241 reconfigure <LyX> by clicking Tools-\<gtr\>Reconfigure, and then re-start
244 <section|Obtaining a decent mono font>
246 The syntax high-lighting features of <name|lstlistings> makes use of bold;
247 however a mono-space tt font is used to typeset the listings. Obtaining a
248 <with|font-family|tt|<strong|bold> tt font> can be impossibly difficult and
249 amazingly easy. I spent many hours at it, following complicated
250 instructions from those who had spend many hours over it, and was finally
251 delivered the simple solution on the lyx mailing list.
255 The simple way was to add this to my preamble:
258 \\usepackage{txfonts}
260 \\renewcommand{\\ttdefault}{txtt}
267 The next simplest way was to use ams poor-mans-bold, by adding this to the
273 %\\renewcommand{\\ttdefault}{txtt}
275 %somehow make \\pmb be the command for bold, forgot how, sorry, above
279 It works, but looks wretched on the dvi viewer.
281 <subsection|Luximono>
283 The lstlistings documention suggests using Luximono.
285 Luximono was installed according to the instructions in Ubuntu Forums
286 thread 1159181<\footnote>
287 http://ubuntuforums.org/showthread.php?t=1159181
288 </footnote> with tips from miknight<\footnote>
289 http://miknight.blogspot.com/2005/11/how-to-install-luxi-mono-font-in.html
290 </footnote> stating that <verbatim|sudo updmap --enable MixedMap ul9.map>
291 is required. It looks fine in PDF and PS view but still looks rotten in dvi
294 <section|Formatting your Lyx document>
296 It is not necessary to base your literate document on any of the original
297 <LyX> literate classes; so select a regular class for your document type.
299 Add the new module <em|Fangle Literate Listings> and also <em|Logical
300 Markup> which is very useful.
302 In the drop-down style listbox you should notice a new style defined,
305 When you wish to insert a literate chunk, you enter it's plain name in the
306 Chunk style, instead of the old <name|noweb> method that uses
307 <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>=> type tags. In the line (or
308 paragraph) following the chunk name, you insert a listing with:
309 Insert-\<gtr\>Program Listing.
311 Inside the white listing box you can type (or paste using
312 <kbd|shift+ctrl+V>) your listing. There is no need to use <kbd|ctrl+enter>
313 at the end of lines as with some older <LyX> literate techniques --- just
314 press enter as normal.
316 <subsection|Customising the listing appearance>
318 The code is formatted using the <name|lstlistings> package. The chunk style
319 doesn't just define the chunk name, but can also define any other chunk
320 options supported by the lstlistings package <verbatim|\\lstset> command.
321 In fact, what you type in the chunk style is raw latex. If you want to set
322 the chunk language without having to right-click the listing, just add
323 <verbatim|,lanuage=C> after the chunk name. (Currently the language will
324 affect all subsequent listings, so you may need to specify
325 <verbatim|,language=> quite a lot).
327 <todo|so fix the bug>
329 Of course you can do this by editing the listings box advanced properties
330 by right-clicking on the listings box, but that takes longer, and you can't
331 see at-a-glance what the advanced settings are while editing the document;
332 also advanced settings apply only to that box --- the chunk settings apply
333 through the rest of the document<\footnote>
334 It ought to apply only to subsequent chunks of the same name. I'll fix
338 <todo|So make sure they only apply to chunks of that name>
340 <subsection|Global customisations>
342 As lstlistings is used to set the code chunks, it's <verbatim|\\lstset>
343 command can be used in the pre-amble to set some document wide settings.
345 If your source has many words with long sequences of capital letters, then
346 <verbatim|columns=fullflexible> may be a good idea, or the capital letters
347 will get crowded. (I think lstlistings ought to use a slightly smaller font
348 for captial letters so that they still fit).
350 The font family <verbatim|\\ttfamily> looks more normal for code, but has
351 no bold (an alternate typewriter font is used).\
353 With <verbatim|\\ttfamily>, I must also specify
354 <verbatim|columns=fullflexible> or the wrong letter spacing is used.
356 In my <LaTeX> pre-amble I usually specialise my code format with:
358 <\nf-chunk|document-preamble>
361 <item>numbers=left, stepnumber=1, numbersep=5pt,
363 <item>breaklines=false,
365 <item>basicstyle=\\footnotesize\\ttfamily,
367 <item>numberstyle=\\tiny,
371 <item>columns=fullflexible,
373 <item>numberfirstline=true
380 <section|Configuring the build script>
382 You can invoke code extraction and building from the <LyX> menu option
383 Document-\<gtr\>Build Program.
385 First, make sure you don't have a conversion defined for Lyx-\<gtr\>Program
387 From the menu Tools-\<gtr\>Preferences, add a conversion from
388 Latex(Plain)-\<gtr\>Program as:
391 set -x ; fangle -Rlyx-build $$i \|\
393 \ \ env LYX_b=$$b LYX_i=$$i LYX_o=$$o LYX_p=$$p LYX_r=$$r bash
396 (But don't cut-n-paste it from this document or you may be be pasting a
397 multi-line string which will break your lyx preferences file).\
399 I hope that one day, <LyX> will set these into the environment when calling
402 You may also want to consider adding options to this conversion...
404 <verbatim|parselog=/usr/share/lyx/scripts/listerrors>
406 ...but if you do you will lose your stderr<\footnote>
407 There is some bash plumbing to get a copy of stderr but this footnote is
411 Now, a shell script chunk called <filename|lyx-build> will be extracted and
412 run whenever you choose the Document-\<gtr\>Build Program menu item.
414 This document was originally managed using <LyX> and lyx-build script for
415 this document is shown here for historical reference.\
418 lyx -e latex fangle.lyx && \\
420 \ \ fangle fangle.lyx \<gtr\> ./autoboot
423 This looks simple enough, but as mentioned, fangle has to be had from
424 somewhere before it can be extracted.
428 When the lyx-build chunk is executed, the current directory will be a
429 temporary directory, and <verbatim|LYX_SOURCE> will refer to the tex file
430 in this temporary directory. This is unfortunate as our makefile wants to
431 run from the project directory where the Lyx file is kept.
433 We can extract the project directory from <verbatim|$$r>, and derive the
434 probable Lyx filename from the noweb file that Lyx generated.
436 <\nf-chunk|lyx-build-helper>
437 <item>PROJECT_DIR="$LYX_r"
439 <item>LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
441 <item>TEX_DIR="$LYX_p"
443 <item>TEX_SRC="$TEX_DIR/$LYX_i"
446 And then we can define a lyx-build fragment similar to the autoboot
449 <\nf-chunk|lyx-build>
452 <item><nf-ref|lyx-build-helper|>
454 <item>cd $PROJECT_DIR \|\| exit 1
458 <item>#/usr/bin/fangle -filter ./notanglefix-filter \\
460 <item># \ -R./Makefile.inc "../../noweb-lyx/noweb-lyx3.lyx" \\
462 <item># \ \| sed '/NOWEB_SOURCE=/s/=.*/=samba4-dfs.lyx/' \\
464 <item># \ \<gtr\> ./Makefile.inc
468 <item>#make -f ./Makefile.inc fangle_sources
473 <chapter|Using Fangle with <TeXmacs>>
475 <todo|Write this chapter>
477 <chapter|Fangle with Makefiles><label|makefile.inc>
479 Here we describe a <filename|Makefile.inc> that you can include in your own
480 Makefiles, or glue as a recursive make to other projects.
482 <filename|Makefile.inc> will cope with extracting all the other source
483 files from this or any specified literate document and keeping them up to
486 It may also be included by a <verbatim|Makefile> or <verbatim|Makefile.am>
487 defined in a literate document to automatically deal with the extraction of
488 source files and documents during normal builds.
490 Thus, if <verbatim|Makefile.inc> is included into a main project makefile
491 it add rules for the source files, capable of extracting the source files
492 from the literate document.
494 <section|A word about makefiles formats>
496 Whitespace formatting is very important in a Makefile. The first character
497 of each action line must be a TAB.\
500 target: pre-requisite
507 This requires that the literate programming environment have the ability to
508 represent a TAB character in a way that fangle will generate an actual TAB
511 We also adopt a convention that code chunks whose names beginning with
512 <verbatim|./> should always be automatically extracted from the document.
513 Code chunks whose names do not begin with <verbatim|./> are for internal
514 reference. Such chunks may be extracted directly, but will not be
515 automatically extracted by this Makefile.
517 <section|Extracting Sources>
519 Our makefile has two parts; variables must be defined before the targets
522 As we progress through this chapter, explaining concepts, we will be adding
523 lines to <nf-ref|Makefile.inc-vars|> and <nf-ref|Makefile.inc-targets|>
524 which are included in <nf-ref|./Makefile.inc|> below.
526 <\nf-chunk|./Makefile.inc>
527 <item><nf-ref|Makefile.inc-vars|>
529 <item><nf-ref|Makefile.inc-default-targets|>
531 <item><nf-ref|Makefile.inc-targets|>
534 We first define a placeholder for the tool <verbatim|fangle> in case it
535 cannot be found in the path.
537 <\nf-chunk|Makefile.inc-vars>
541 We also define a placeholder for <verbatim|LITERATE_SOURCE> to hold the
542 name of this document. This will normally be passed on the command line.
544 <\nf-chunk|Makefile.inc-vars>
545 <item>LITERATE_SOURCE=
548 Fangle cannot process <LyX> or <TeXmacs> documents directly, so the first
549 stage is to convert these to more suitable text based formats<\footnote>
550 <LyX> and <TeXmacs> formats are text-based, but not suitable for fangle
553 <subsection|Converting from <LyX> to <LaTeX>><label|Converting-from-Lyx>
555 The first stage will always be to convert the <LyX> file to a <LaTeX> file.
556 Fangle must run on a <TeX> file because the <LyX> command
557 <verbatim|server-goto-file-line><\footnote>
558 The Lyx command <verbatim|server-goto-file-line> is used to position the
559 Lyx cursor at the compiler errors.
560 </footnote> requries that the line number provided be a line of the <TeX>
561 file and always maps this the line in the <LyX> docment. We use
562 <verbatim|server-goto-file-line> when moving the cursor to error lines
563 during compile failures.
565 The command <verbatim|lyx -e literate fangle.lyx> will produce
566 <verbatim|fangle.tex>, a <TeX> file; so we define a make target to be the
567 same as the <LyX> file but with the <verbatim|.tex> extension.
569 The <verbatim|EXTRA_DIST> is for automake support so that the <TeX> files
570 will automaticaly be distributed with the source, to help those who don't
571 have <LyX> installed.
573 <\nf-chunk|Makefile.inc-vars>
574 <item>LYX_SOURCE=$(LITERATE_SOURCE) # but only the .lyx files
576 <item>TEX_SOURCE=$(LYX_SOURCE:.lyx=.tex)
578 <item>EXTRA_DIST+=$(TEX_SOURCE)
581 We then specify that the <TeX> source is to be generated from the <LyX>
584 <\nf-chunk|Makefile.inc-targets>
585 <item>.SUFFIXES: .tex .lyx
589 <item><nf-tab>lyx -e latex $\<less\>
593 <item><nf-tab>rm -f -- $(TEX_SOURCE)
595 <item>clean: clean_tex
598 <subsection|Converting from <TeXmacs>><label|Converting-from-Lyx>
600 Fangle cannot process <TeXmacs> files directly<\footnote>
601 but this is planned when <TeXmacs> uses xml as it's native format
602 </footnote>, but must first convert them to text files.
604 The command <verbatim|texmacs -c fangle.tm fangle.txt -q> will produce
605 <verbatim|fangle.txt>, a text file; so we define a make target to be the
606 same as the <TeXmacs> file but with the <verbatim|.txt> extension.
608 The <verbatim|EXTRA_DIST> is for automake support so that the <TeX> files
609 will automaticaly be distributed with the source, to help those who don't
610 have <LyX> installed.
612 <\nf-chunk|Makefile.inc-vars>
613 <item>TEXMACS_SOURCE=$(LITERATE_SOURCE) # but only the .tm files
615 <item>TXT_SOURCE=$(LITERATE_SOURCE:.tm=.txt)
617 <item>EXTRA_DIST+=$(TXT_SOURCE)
620 <todo|Add loop around each $\<less\> so multiple targets can be specified>
622 <\nf-chunk|Makefile.inc-targets>
623 <item>.SUFFIXES: .txt .tm
627 <item><nf-tab>texmacs -s -c $\<less\> $@ -q
629 <item>.PHONEY: clean_txt
633 <item><nf-tab>rm -f -- $(TXT_SOURCE)
635 <item>clean: clean_txt
638 <section|Extracting Program Source>
640 The program source is extracted using fangle, which is designed to operate
641 on text or a <LaTeX> documents<\footnote>
642 <LaTeX> documents are just slightly special text documents
645 <\nf-chunk|Makefile.inc-vars>
646 <item>FANGLE_SOURCE=$(TXT_SOURCE)
649 The literate document can result in any number of source files, but not all
650 of these will be changed each time the document is updated. We certainly
651 don't want to update the timestamps of these files and cause the whole
652 source tree to be recompiled just because the literate explanation was
653 revised. We use <verbatim|CPIF> from the <em|Noweb> tools to avoid updating
654 the file if the content has not changed, but should probably write our own.
656 However, if a source file is not updated, then the fangle file will always
657 have a newer time-stamp and the makefile would always re-attempt to extact
658 a newer source file which would be a waste of time.
660 Because of this, we use a stamp file which is always updated each time the
661 sources are fully extracted from the <LaTeX> document. If the stamp file is
662 newer than the document, then we can avoid an attempt to re-extract any of
663 the sources. Because this stamp file is only updated when extraction is
664 complete, it is safe for the user to interrupt the build-process
667 We use <verbatim|echo> rather than <verbatim|touch> to update the stamp
668 file beause the <verbatim|touch> command does not work very well over an
669 <verbatim|sshfs> mount \ that I was using.
671 <\nf-chunk|Makefile.inc-vars>
672 <item>FANGLE_SOURCE_STAMP=$(FANGLE_SOURCE).stamp
675 <\nf-chunk|Makefile.inc-targets>
676 <item>$(FANGLE_SOURCE_STAMP): $(FANGLE_SOURCE) \\
678 <item><nf-tab> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ $(FANGLE_SOURCES) ; \\
680 <item><nf-tab>echo -n \<gtr\> $(FANGLE_SOURCE_STAMP)
684 <item><nf-tab>rm -f $(FANGLE_SOURCE_STAMP)
686 <item>clean: clean_stamp
689 <section|Extracting Source Files>
691 We compute <verbatim|FANGLE_SOURCES> to hold the names of all the source
692 files defined in the document. We compute this only once, by means of
693 <verbatim|:=> in assignent. The sed deletes the any
694 <verbatim|\<less\>\<less\>> and <verbatim|\<gtr\>\<gtr\>> which may
695 surround the roots names (for compatibility with Noweb's noroots command).
697 As we use chunk names beginning with <filename|./> to denote top level
698 fragments that should be extracted, we filter out all fragments that do not
699 begin with <filename|./>
702 <verbatim|FANGLE_PREFIX> is set to <verbatim|./> by default, but whatever
703 it may be overridden to, the prefix is replaced by a literal
704 <verbatim|./> before extraction so that files will be extracted in the
705 current directory whatever the prefix. This helps namespace or
706 sub-project prefixes like <verbatim|documents:> for chunks like
707 <verbatim|documents:docbook/intro.xml>
710 <todo|This doesn't work though, because it loses the full name and doesn't
711 know what to extact!>
713 <\nf-chunk|Makefile.inc-vars>
714 <item>FANGLE_PREFIX:=\\.\\/
716 <item>FANGLE_SOURCES:=$(shell \\
718 <item> \ $(FANGLE) -r $(FANGLE_SOURCE) \|\\
720 <item> \ sed -e 's/^[\<less\>][\<less\>]//;s/[\<gtr\>][\<gtr\>]$$//;/^$(FANGLE_PREFIX)/!d'
723 <item> \ \ \ \ \ -e 's/^$(FANGLE_PREFIX)/\\.\\//' )
726 The target below, <verbatim|echo_fangle_sources> is a helpful debugging
727 target and shows the names of the files that would be extracted.
729 <\nf-chunk|Makefile.inc-targets>
730 <item>.PHONY: echo_fangle_sources
732 <item>echo_fangle_sources: ; @echo $(FANGLE_SOURCES)
735 We define a convenient target called <verbatim|fangle_sources> so that
736 <verbatim|make -f fangle_sources> will re-extract the source if the
737 literate document has been updated.\
739 <\nf-chunk|Makefile.inc-targets>
740 <item>.PHONY: fangle_sources
742 <item>fangle_sources: $(FANGLE_SOURCE_STAMP)
745 And also a convenient target to remove extracted sources.
747 <\nf-chunk|Makefile.inc-targets>
748 <item>.PHONY: clean_fangle_sources
750 <item>clean_fangle_sources: ; \\
752 <item> \ \ \ \ \ \ \ rm -f -- $(FANGLE_SOURCE_STAMP) $(FANGLE_SOURCES)
755 We now look at the extraction of the source files.
757 This makefile macro <verbatim|if_extension> takes 4 arguments: the filename
758 <verbatim|$(1)>, some extensions to match <verbatim|$(2)> and a shell
759 command to return if the filename does match the exensions <verbatim|$(3)>,
760 and a shell command to return if it does not match the extensions
763 <\nf-chunk|Makefile.inc-vars>
764 <item>if_extension=$(if $(findstring $(suffix $(1)),$(2)),$(3),$(4))
767 For some source files like C files, we want to output the line number and
768 filename of the original <LaTeX> document from which the source
770 I plan to replace this option with a separate mapping file so as not to
771 pollute the generated source, and also to allow a code pretty-printing
772 reformatter like <verbatim|indent> be able to re-format the file and
773 adjust for changes through comparing the character streams.
776 To make this easier we define the file extensions for which we want to do
779 <\nf-chunk|Makefile.inc-vars>
780 <item>C_EXTENSIONS=.c .h
783 We can then use the <verbatim|if_extensions> macro to define a macro which
784 expands out to the <verbatim|-L> option if fangle is being invoked in a C
785 source file, so that C compile errors will refer to the line number in the
788 <\nf-chunk|Makefile.inc-vars>
791 <item>nf_line=-L -T$(TABS)
793 <item>fangle=$(FANGLE) $(call if_extension,$(2),$(C_EXTENSIONS),$(nf_line))
797 We can use a similar trick to define an indent macro which takes just the
798 filename as an argument and can return a pipeline stage calling the indent
799 command. Indent can be turned off with <verbatim|make fangle_sources
802 <\nf-chunk|Makefile.inc-vars>
803 <item>indent_options=-npro -kr -i8 -ts8 -sob -l80 -ss -ncs
805 <item>indent=$(call if_extension,$(1),$(C_EXTENSIONS), \| indent
809 We now define the pattern for extracting a file. The files are written
810 using noweb's <verbatim|cpif> so that the file timestamp will not be
811 touched if the contents haven't changed. This avoids the need to rebuild
812 the entire project because of a typographical change in the documentation,
813 or if none or a few C source files have changed.
815 <\nf-chunk|Makefile.inc-vars>
816 <item>fangle_extract=@mkdir -p $(dir $(1)) && \\
818 <item> \ $(call fangle,$(2),$(1)) \<gtr\> "$(1).tmp" && \\
820 <item> \ cat "$(1).tmp" $(indent) \| cpif "$(1)" \\
822 <item> \ && rm -f -- "$(1).tmp" \|\| \\
824 <item> \ (echo error fangling $(1) from $(2) ; exit 1)
827 We define a target which will extract or update all sources. To do this we
828 first defined a makefile template that can do this for any source file in
829 the <LaTeX> document.
831 <\nf-chunk|Makefile.inc-vars>
832 <item>define FANGLE_template
836 <item><nf-tab>$$(call fangle_extract,$(1),$(2))
838 <item> \ FANGLE_TARGETS+=$(1)
843 We then enumerate the discovered <verbatim|FANGLE_SOURCES> to generate a
844 makefile rule for each one using the makefile template we defined above.
846 <\nf-chunk|Makefile.inc-targets>
847 <item>$(foreach source,$(FANGLE_SOURCES),\\
849 <item> \ $(eval $(call FANGLE_template,$(source),$(FANGLE_SOURCE))) \\
854 These will all be built with <verbatim|FANGLE_SOURCE_STAMP>.
856 We also remove the generated sources on a make distclean.
858 <\nf-chunk|Makefile.inc-targets>
859 <item>_distclean: clean_fangle_sources
862 <section|Extracting Documentation>
864 We then identify the intermediate stages of the documentation and their
865 build and clean targets.
867 <\nf-chunk|Makefile.inc-default-targets>
868 <item>.PHONEY : clean_pdf
871 <subsection|Formatting <TeX>>
873 <subsubsection|Running pdflatex>
875 We produce a pdf file from the tex file.
877 <\nf-chunk|Makefile.inc-vars>
878 <item>FANGLE_PDF+=$(TEX_SOURCE:.tex=.pdf)
881 We run pdflatex twice to be sure that the contents and aux files are up to
882 date. We certainly are <em|required> to run pdflatex at least twice if
883 these files do not exist.
885 <\nf-chunk|Makefile.inc-targets>
886 <item>.SUFFIXES: .tex .pdf
890 <item><nf-tab>pdflatex $\<less\> && pdflatex $\<less\>
896 <item><nf-tab>rm -f -- $(FANGLE_PDF) $(TEX_SOURCE:.tex=.toc) \\
898 <item><nf-tab> \ $(TEX_SOURCE:.tex=.log) $(TEX_SOURCE:.tex=.aux)
900 <item>clean_pdf: clean_pdf_tex
903 <subsection|Formatting <TeXmacs>>
905 <TeXmacs> can produce a PDF file directly.
907 <\nf-chunk|Makefile.inc-vars>
908 <item>FANGLE_PDF+=$(LITERATE_SOURCE:.tm=.pdf)
912 Outputting the PDF may not be enough to update the links and page
915 we need to update twice, generate a pdf, update twice mode and generate a
918 Basically the PDF export of <TeXmacs> is pretty rotten and doesn't work
919 properly from the CLI
922 <\nf-chunk|Makefile.inc-targets>
923 <item>.SUFFIXES: .tm .pdf
927 <item><nf-tab>texmacs -s -c $\<less\> $@ -q
931 <item>clean_pdf_texmacs:
933 <item><nf-tab>rm -f -- $(FANGLE_PDF)
935 <item>clean_pdf: clean_pdf_texmacs
938 <subsection|Building the Documentation as a Whole>
940 Currently we only build pdf as a final format, but <verbatim|FANGLE_DOCS>
941 may later hold other output formats.
943 <\nf-chunk|Makefile.inc-vars>
944 <item>FANGLE_DOCS=$(FANGLE_PDF)
947 We also define <verbatim|fangle_docs> as a convenient phony target.
949 <\nf-chunk|Makefile.inc-targets>
950 <item>.PHONY: fangle_docs
952 <item>fangle_docs: $(FANGLE_DOCS)
954 <item>docs: fangle_docs
957 And define a convenient <verbatim|clean_fangle_docs> which we add to the
960 <\nf-chunk|Makefile.inc-targets>
961 <item>.PHONEY: clean_fangle_docs
963 <item>clean_fangle_docs: clean_tex clean_pdf
965 <item>clean: clean_fangle_docs
969 <item>distclean_fangle_docs: clean_tex clean_fangle_docs
971 <item>distclean: clean distclean_fangle_docs
974 <section|Other helpers>
976 If <filename|Makefile.inc> is included into <filename|Makefile>, then
977 extracted files can be updated with this command:
979 <verbatim|make fangle_sources>
983 <verbatim|make -f Makefile.inc fangle_sources>
985 <section|Boot-strapping the extraction>
987 As well as having the makefile extract or update the source files as part
988 of it's operation, it also seems convenient to have the makefile
989 re-extracted itself from <em|this> document.
991 It would also be convenient to have the code that extracts the makefile
992 from this document to also be part of this document, however we have to
993 start somewhere and this unfortunately requires us to type at least a few
994 words by hand to start things off.
996 Therefore we will have a minimal root fragment, which, when extracted, can
997 cope with extracting the rest of the source. This shell script fragment can
998 do that. It's name is <verbatim|*> <emdash> out of regard for <name|Noweb>,
999 but when extracted might better be called <verbatim|autoupdate>.
1008 <item>MAKE_SRC="${1:-${NW_LYX:-../../noweb-lyx/noweb-lyx3.lyx}}"
1010 <item>MAKE_SRC=`dirname "$MAKE_SRC"`/`basename "$MAKE_SRC" .lyx`
1012 <item>NOWEB_SRC="${2:-${NOWEB_SRC:-$MAKE_SRC.lyx}}"
1014 <item>lyx -e latex $MAKE_SRC
1018 <item>fangle -R./Makefile.inc ${MAKE_SRC}.tex \\
1020 <item> \ \| sed "/FANGLE_SOURCE=/s/^/#/;T;aNOWEB_SOURCE=$FANGLE_SRC" \\
1022 <item> \ \| cpif ./Makefile.inc
1026 <item>make -f ./Makefile.inc fangle_sources
1029 The general Makefile can be invoked with <filename|./autoboot> and can also
1030 be included into any automake file to automatically re-generate the source
1033 The <em|autoboot> can be extracted with this command:
1036 lyx -e latex fangle.lyx && \\
1038 \ \ fangle fangle.lyx \<gtr\> ./autoboot
1041 This looks simple enough, but as mentioned, fangle has to be had from
1042 somewhere before it can be extracted.
1044 On a unix system this will extract <filename|fangle.module> and the
1045 <filename|fangle> awk script, and run some basic tests.\
1047 <todo|cross-ref to test chapter when it is a chapter all on its own>
1049 <section|Incorporating Makefile.inc into existing projects>
1051 If you are writing a literate module of an existing non-literate program
1052 you may find it easier to use a slight recursive make instead of directly
1053 including <verbatim|Makefile.inc> in the projects makefile.\
1055 This way there is less chance of definitions in <verbatim|Makefile.inc>
1056 interfering with definitions in the main makefile, or with definitions in
1057 other <verbatim|Makefile.inc> from other literate modules of the same
1060 To do this we add some <em|glue> to the project makefile that invokes
1061 Makefile.inc in the right way. The glue works by adding a <verbatim|.PHONY>
1062 target to call the recursive make, and adding this target as an additional
1063 pre-requisite to the existing targets.
1065 <paragraph|Example>Sub-module of existing system
1067 In this example, we are building <verbatim|module.so> as a literate module
1068 of a larger project.
1070 We will show the sort glue that can be inserted into the projects Makefile
1071 <emdash> or more likely <emdash> a regular Makefile included in or invoked
1072 by the projects Makefile.
1074 <\nf-chunk|makefile-glue>
1075 <item>module_srcdir=modules/module
1077 <item>MODULE_SOURCE=module.tm
1079 <item>MODULE_STAMP=$(MODULE_SOURCE).stamp
1082 The existing build system may already have a build target for
1083 <filename|module.o>, but we just add another pre-requisite to that. In this
1084 case we use <filename|module.tm.stamp> as a pre-requisite, the stamp file's
1085 modified time indicating when all sources were extracted<\footnote>
1086 If the projects build system does not know how to build the module from
1087 the extracted sources, then just add build actions here as normal.
1090 <\nf-chunk|makefile-glue>
1091 <item>$(module_srcdir)/module.o: $(module_srcdir)/$(MODULE_STAMP)
1094 The target for this new pre-requisite will be generated by a recursive make
1095 using <filename|Makefile.inc> which will make sure that the source is up to
1096 date, before it is built by the main projects makefile.
1098 <\nf-chunk|makefile-glue>
1099 <item>$(module_srcdir)/$(MODULE_STAMP): $(module_srcdir)/$(MODULE_SOURCE)
1101 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc fangle_sources
1102 LITERATE_SOURCE=$(MODULE_SOURCE)
1105 We can do similar glue for the docs, clean and distclean targets. In this
1106 example the main prject was using a double colon for these targets, so we
1107 must use the same in our glue.
1109 <\nf-chunk|makefile-glue>
1110 <item>docs:: docs_module
1112 <item>.PHONY: docs_module
1116 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc docs
1117 LITERATE_SOURCE=$(MODULE_SOURCE)
1121 <item>clean:: clean_module
1123 <item>.PHONEY: clean_module
1127 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc clean
1128 LITERATE_SOURCE=$(MODULE_SOURCE)
1132 <item>distclean:: distclean_module
1134 <item>.PHONY: distclean_module
1136 <item>distclean_module:
1138 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc distclean
1139 LITERATE_SOURCE=$(MODULE_SOURCE)
1142 We could do similarly for install targets to install the generated docs.
1146 <chapter|Fangle awk source code>
1148 We use the copyright notice from chapter <reference|License>.
1150 <\nf-chunk|./fangle>
1151 <item>#! /usr/bin/awk -f
1153 <item># <nf-ref|gpl3-copyright|>
1156 We also use code from <person|Arnold Robbins> public domain getopt (1993
1157 revision) defined in <reference|getopt>, and naturally want to attribute
1160 <\nf-chunk|./fangle>
1161 <item># NOTE: Arnold Robbins public domain getopt for awk is also used:
1163 <item><nf-ref|getopt.awk-header|>
1165 <item><nf-ref|getopt.awk-getopt()|>
1170 And include the following chunks (which are explained further on) to make
1173 <\nf-chunk|./fangle>
1174 <item><nf-ref|helper-functions|>
1176 <item><nf-ref|mode-tracker|>
1178 <item><nf-ref|parse_chunk_args|>
1180 <item><nf-ref|chunk-storage-functions|>
1182 <item><nf-ref|output_chunk_names()|>
1184 <item><nf-ref|output_chunks()|>
1186 <item><nf-ref|write_chunk()|>
1188 <item><nf-ref|expand_chunk_args()|>
1192 <item><nf-ref|begin|>
1194 <item><nf-ref|recognize-chunk|>
1199 <section|AWK tricks>
1201 The portable way to erase an array in awk is to split the empty string, so
1202 we define a fangle macro that can split an array, like this:
1204 <\nf-chunk|awk-delete-array>
1205 <item>split("", <nf-arg|ARRAY>);
1206 </nf-chunk|awk|<tuple|ARRAY>>
1208 For debugging it is sometimes convenient to be able to dump the contents of
1209 an array to <verbatim|stderr>, and so this macro is also useful.
1211 <\nf-chunk|dump-array>
1212 <item>print "\\nDump: <nf-arg|ARRAY>\\n--------\\n" \<gtr\>
1215 <item>for (_x in <nf-arg|ARRAY>) {
1217 <item> \ print _x "=" <nf-arg|ARRAY>[_x] "\\n" \<gtr\> "/dev/stderr";
1221 <item>print "========\\n" \<gtr\> "/dev/stderr";
1222 </nf-chunk|awk|<tuple|ARRAY>>
1224 <section|Catching errors>
1226 Fatal errors are issued with the error function:
1229 <item>function error(message)
1233 <item> \ print "ERROR: " FILENAME ":" FNR " " message \<gtr\>
1241 and likewise for non-fatal warnings:
1244 <item>function warning(message)
1248 <item> \ print "WARNING: " FILENAME ":" FNR " " message \<gtr\>
1251 <item> \ warnings++;
1256 and debug output too:
1259 <item>function debug_log(message)
1263 <item> \ print "DEBUG: " FILENAME ":" FNR " " message \<gtr\>
1269 <todo|append=helper-functions>
1271 <\nf-chunk|helper-functions>
1272 <item><nf-ref|error()|>
1275 <chapter|<TeXmacs> args>
1277 <TeXmacs> functions with arguments<\footnote>
1278 or function declarations with parameters
1279 </footnote> appear like this:
1281 <math|<math-tt|blah(><wide*|<wide|<math-tt|I came, I saw, I
1282 conquered>|\<wide-overbrace\>><rsup|argument 1><wide|<math-tt|<key|^K>>,
1283 |\<wide-overbrace\>><rsup|sep.><wide|and then went home
1284 asd|\<wide-overbrace\>><rsup|argument 3><wide|<math-tt|<key|^K>><math-tt|)>|\<wide-overbrace\>><rsup|term.>|\<wide-underbrace\>><rsub|arguments>>
1286 Arguments commence after the opening parenthesis. The first argument runs
1287 up till the next <key|^K>.\
1289 If the following character is a <key|,> then another argument follows. If
1290 the next character after the <key|,> is a space character, then it is also
1291 eaten. The fangle stylesheet emits <key|^K><key|,><key|space> as
1292 separators, but the fangle untangler will forgive a missing space.
1294 If the following character is <key|)> then this is a terminator and there
1295 are no more arguments.
1297 <\nf-chunk|constants>
1298 <item>ARG_SEPARATOR=sprintf("%c", 11);
1301 To process the <verbatim|text> in this fashion, we split the string on
1306 <\nf-chunk|get_chunk_args>
1307 <item>function get_texmacs_chunk_args(text, args, \ \ a, done) {
1309 <item> \ split(text, args, ARG_SEPARATOR);
1315 <item> \ for (a=1; (a in args); a++) if (a\<gtr\>1) {
1317 <item> \ \ \ if (args[a] == "" \|\| substr(args[a], 1, 1) == ")") done=1;
1319 <item> \ \ \ if (done) {
1321 <item> \ \ \ \ \ delete args[a];
1323 <item> \ \ \ \ \ break;
1329 <item> \ \ \ if (substr(args[a], 1, 2) == ", ") args[a]=substr(args[a],
1332 <item> \ \ \ else if (substr(args[a], 1, 1) == ",")
1333 args[a]=substr(args[a], 2); \
1340 <chapter|<LaTeX> and lstlistings>
1342 <todo|Split LyX and TeXmacs parts>
1344 For <LyX> and <LaTeX>, the <verbatim|lstlistings> package is used to format
1345 the lines of code chunks. You may recal from chapter XXX that arguments to
1346 a chunk definition are pure <LaTeX> code. This means that fangle needs to
1347 be able to parse <LaTeX> a little.
1349 <LaTeX> arguments to <verbatim|lstlistings> macros are a comma seperated
1350 list of key-value pairs, and values containing commas are enclosed in
1351 <verbatim|{> braces <verbatim|}> (which is to be expected for <LaTeX>).
1353 A sample expressions is:
1355 <verbatim|name=thomas, params={a, b}, something, something-else>
1357 but we see that this is just a simpler form of this expression:
1359 <verbatim|name=freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc>
1361 We may consider that we need a function that can parse such <LaTeX>
1362 expressions and assign the values to an AWK associated array, perhaps using
1363 a recursive parser into a multi-dimensional hash<\footnote>
1364 as AWK doesn't have nested-hash support
1365 </footnote>, resulting in:
1367 <tabular|<tformat|<cwith|2|6|1|2|cell-lborder|0.5pt>|<cwith|2|6|1|2|cell-rborder|0.5pt>|<cwith|2|6|1|2|cell-bborder|0.5pt>|<cwith|2|6|1|2|cell-tborder|0.5pt>|<cwith|1|1|1|2|cell-lborder|0.5pt>|<cwith|1|1|1|2|cell-rborder|0.5pt>|<cwith|1|1|1|2|cell-bborder|0.5pt>|<cwith|1|1|1|2|cell-tborder|0.5pt>|<table|<row|<cell|key>|<cell|value>>|<row|<cell|a[name]>|<cell|freddie>>|<row|<cell|a[foo,
1368 bar]>|<cell|baz>>|<row|<cell|a[foo, quux,
1369 quirk]>|<cell|>>|<row|<cell|a[foo, quux,
1370 a]>|<cell|fleeg>>|<row|<cell|a[etc]>|<cell|>>>>>
1372 Yet, also, on reflection it seems that sometimes such nesting is not
1373 desirable, as the braces are also used to delimit values that contain
1374 commas --- we may consider that
1376 <verbatim|name={williamson, freddie}>
1378 should assign <verbatim|williamson, freddie> to <verbatim|name>.
1380 In fact we are not so interested in the detail so as to be bothered by
1381 this, which turns out to be a good thing for two reasons. Firstly <TeX> has
1382 a malleable parser with no strict syntax, and secondly whether or not
1383 <verbatim|williamson> and <verbatim|freddie> should count as two items will
1384 be context dependant anyway.
1386 We need to parse this latex for only one reason; which is that we are
1387 extending lstlistings to add some additional arguments which will be used
1388 to express chunk parameters and other chunk options.
1390 <section|Additional lstlstings parameters>
1392 Further on we define a <verbatim|\\Chunk> <LaTeX> macro whose arguments
1393 will consist of a the chunk name, optionally followed by a comma and then a
1394 comma separated list of arguments. In fact we will just need to prefix
1395 <verbatim|name=> to the arguments to in order to create valid lstlistings
1398 There will be other arguments supported too;\
1401 <item*|params>As an extension to many literate-programming styles, fangle
1402 permits code chunks to take parameters and thus operate somewhat like C
1403 pre-processor macros, or like C++ templates. Chunk parameters are
1404 declared with a chunk argument called params, which holds a semi-colon
1405 separated list of parameters, like this:
1407 <verbatim|achunk,language=C,params=name;address>
1409 <item*|addto>a named chunk that this chunk is to be included into. This
1410 saves the effort of having to declare another listing of the named chunk
1411 merely to include this one.
1414 Function get_chunk_args() will accept two paramters, text being the text to
1415 parse, and values being an array to receive the parsed values as described
1416 above. The optional parameter path is used during recursion to build up the
1417 multi-dimensional array path.
1419 <\nf-chunk|./fangle>
1420 <item><nf-ref|get_chunk_args()|>
1423 <\nf-chunk|get_chunk_args()>
1424 <item>function get_tex_chunk_args(text, values,
1426 <item> \ # optional parameters
1428 <item> \ path, # hierarchical precursors
1430 <item> \ # local vars
1435 The strategy is to parse the name, and then look for a value. If the value
1436 begins with a brace <verbatim|{>, then we recurse and consume as much of
1437 the text as necessary, returning the remaining text when we encounter a
1438 leading close-brace <verbatim|}>. This being the strategy --- and executed
1439 in a loop --- we realise that we must first look for the closing brace
1440 (perhaps preceded by white space) in order to terminate the recursion, and
1441 returning remaining text.
1443 <\nf-chunk|get_chunk_args()>
1446 <item> \ split("", values);
1448 <item> \ while(length(text)) {
1450 <item> \ \ \ if (match(text, "^ *}(.*)", a)) {
1452 <item> \ \ \ \ \ return a[1];
1456 <item> \ \ \ <nf-ref|parse-chunk-args|>
1460 <item> \ return text;
1465 We can see that the text could be inspected with this regex:
1467 <\nf-chunk|parse-chunk-args>
1468 <item>if (! match(text, " *([^,=]*[^,= ]) *(([,=]) *(([^,}]*) *,*
1471 <item> \ return text;
1476 and that <verbatim|a> will have the following values:
1478 <tabular|<tformat|<cwith|2|7|1|2|cell-lborder|0.5pt>|<cwith|2|7|1|2|cell-rborder|0.5pt>|<cwith|2|7|1|2|cell-bborder|0.5pt>|<cwith|2|7|1|2|cell-tborder|0.5pt>|<cwith|1|1|1|2|cell-lborder|0.5pt>|<cwith|1|1|1|2|cell-rborder|0.5pt>|<cwith|1|1|1|2|cell-bborder|0.5pt>|<cwith|1|1|1|2|cell-tborder|0.5pt>|<table|<row|<cell|a[n]>|<cell|assigned
1479 text>>|<row|<cell|1>|<cell|freddie>>|<row|<cell|2>|<cell|=freddie,
1480 foo={bar=baz, quux={quirk, a=fleeg}}, etc>>|<row|<cell|3>|<cell|=>>|<row|<cell|4>|<cell|freddie,
1481 foo={bar=baz, quux={quirk, a=fleeg}}, etc>>|<row|<cell|5>|<cell|freddie>>|<row|<cell|6>|<cell|,
1482 foo={bar=baz, quux={quirk, a=fleeg}}, etc>>>>>
1484 <verbatim|a[3]> will be either <verbatim|=> or <verbatim|,> and signify
1485 whether the option named in <verbatim|a[1]> has a value or not
1488 If the option does have a value, then if the expression
1489 <verbatim|substr(a[4],1,1)> returns a brace <verbatim|{> it will signify
1490 that we need to recurse:
1492 <\nf-chunk|parse-chunk-args>
1495 <item>if (a[3] == "=") {
1497 <item> \ if (substr(a[4],1,1) == "{") {
1499 <item> \ \ \ text = get_tex_chunk_args(substr(a[4],2), values, path name
1504 <item> \ \ \ values[path name]=a[5];
1506 <item> \ \ \ text = a[6];
1512 <item> \ values[path name]="";
1514 <item> \ text = a[2];
1519 We can test this function like this:
1521 <\nf-chunk|gca-test.awk>
1522 <item><nf-ref|get_chunk_args()|>
1526 <item> \ SUBSEP=".";
1530 <item> \ print get_tex_chunk_args("name=freddie, foo={bar=baz,
1531 quux={quirk, a=fleeg}}, etc", a);
1533 <item> \ for (b in a) {
1535 <item> \ \ \ print "a[" b "] =\<gtr\> " a[b];
1542 which should give this output:
1544 <\nf-chunk|gca-test.awk-results>
1545 <item>a[foo.quux.quirk] =\<gtr\>\
1547 <item>a[foo.quux.a] =\<gtr\> fleeg
1549 <item>a[foo.bar] =\<gtr\> baz
1551 <item>a[etc] =\<gtr\>\
1553 <item>a[name] =\<gtr\> freddie
1556 <section|Parsing chunk arguments><label|Chunk Arguments>
1558 Arguments to paramterized chunks are expressed in round brackets as a comma
1559 separated list of optional arguments. For example, a chunk that is defined
1562 <verbatim|\\Chunk{achunk, params=name ; address}>
1564 could be invoked as:
1566 <verbatim|\\chunkref{achunk}(John Jones, jones@example.com)>
1568 An argument list may be as simple as in <verbatim|\\chunkref{pull}(thing,
1569 otherthing)> or as complex as:
1571 <verbatim|\\chunkref{pull}(things[x, y], get_other_things(a, "(all)"))>
1573 --- which for all it's commas and quotes and parenthesis represents only
1574 two parameters: <verbatim|things[x, y]> and <verbatim|get_other_things(a,
1577 If we simply split parameter list on commas, then the comma in
1578 <verbatim|things[x,y]> would split into two seperate arguments:
1579 <verbatim|things[x> and <verbatim|y]>--- neither of which make sense on
1582 One way to prevent this would be by refusing to split text between matching
1583 delimiters, such as <verbatim|[>, <verbatim|]>, <verbatim|(>, <verbatim|)>,
1584 <verbatim|{>, <verbatim|}> and most likely also <verbatim|">, <verbatim|">
1585 and <verbatim|'>, <verbatim|'>. Of course this also makes it impossible to
1586 pass such mis-matched code fragments as parameters, but I think that it
1587 would be hard for readers to cope with authors who would pass such code
1588 unbalanced fragments as chunk parameters<\footnote>
1589 I know that I couldn't cope with users doing such things, and although
1590 the GPL3 license prevents me from actually forbidding anyone from trying,
1591 if they want it to work they'll have to write the code themselves and not
1592 expect any support from me.
1595 Unfortunately, the full set of matching delimiters may vary from language
1596 to language. In certain C++ template contexts, <verbatim|\<less\>> and
1597 <verbatim|\<gtr\>> would count as delimiters, and yet in other contexts
1600 This puts me in the unfortunate position of having to parse-somewhat all
1601 programming languages without knowing what they are!
1603 However, if this universal mode-tracking is possible, then parsing the
1604 arguments would be trivial. Such a mode tracker is described in chapter
1605 <reference|modes> and used here with simplicity.
1607 <\nf-chunk|parse_chunk_args>
1608 <item>function parse_chunk_args(language, text, values, mode,
1610 <item> \ # local vars
1612 <item> \ c, context, rest)
1616 <item> \ <nf-ref|new-mode-tracker|<tuple|context|language|mode>>
1618 <item> \ rest = mode_tracker(context, text, values);
1620 <item> \ # extract values
1622 <item> \ for(c=1; c \<less\>= context[0, "values"]; c++) {
1624 <item> \ \ \ values[c] = context[0, "values", c];
1628 <item> \ return rest;
1633 <section|Expanding parameters in the text>
1635 Within the body of the chunk, the parameters are referred to with:
1636 <verbatim|${name}> and <verbatim|${address}>. There is a strong case that a
1637 <LaTeX> style notation should be used, like <verbatim|\\param{name}> which
1638 would be expressed in the listing as <verbatim|=\<less\>\\param{name}\<gtr\>>
1639 and be rendered as <verbatim|<nf-arg|name>>. Such notation would make me go
1640 blind, but I do intend to adopt it.
1642 We therefore need a function <verbatim|expand_chunk_args> which will take a
1643 block of text, a list of permitted parameters, and the arguments which must
1644 substitute for the parameters.\
1646 Here we split the text on <verbatim|${> which means that all parts except
1647 the first will begin with a parameter name which will be terminated by
1648 <verbatim|}>. The split function will consume the literal <verbatim|${> in
1651 <\nf-chunk|expand_chunk_args()>
1652 <item>function expand_chunk_args(text, params, args, \
1654 <item> \ p, text_array, next_text, v, t, l)
1658 <item> \ if (split(text, text_array, "\\\\${")) {
1660 <item> \ \ \ <nf-ref|substitute-chunk-args|>
1666 <item> \ return text;
1671 First, we produce an associative array of substitution values indexed by
1672 parameter names. This will serve as a cache, allowing us to look up the
1673 replacement values as we extract each name.
1675 <\nf-chunk|substitute-chunk-args>
1676 <item>for(p in params) {
1678 <item> \ v[params[p]]=args[p];
1683 We accumulate substituted text in the variable text. As the first part of
1684 the split function is the part before the delimiter --- which is
1685 <verbatim|${> in our case --- this part will never contain a parameter
1686 reference, so we assign this directly to the result kept in
1689 <\nf-chunk|substitute-chunk-args>
1690 <item>text=text_array[1];
1693 We then iterate over the remaining values in the array, and substitute each
1694 reference for it's argument.
1696 <\nf-chunk|substitute-chunk-args>
1697 <item>for(t=2; t in text_array; t++) {
1699 <item> \ <nf-ref|substitute-chunk-arg|>
1704 After the split on <verbatim|${> a valid parameter reference will consist
1705 of valid parameter name terminated by a close-brace <verbatim|}>. A valid
1706 character name begins with the underscore or a letter, and may contain
1707 letters, digits or underscores.
1709 A valid looking reference that is not actually the name of a parameter will
1710 be and not substituted. This is good because there is nothing to substitute
1711 anyway, and it avoids clashes when writing code for languages where
1712 <verbatim|${...}> is a valid construct --- such constructs will not be
1713 interfered with unless the parameter name also matches.
1715 <\nf-chunk|substitute-chunk-arg>
1716 <item>if (match(text_array[t], "^([a-zA-Z_][a-zA-Z0-9_]*)}", l) &&
1718 <item> \ \ \ l[1] in v)\
1722 <item> \ text = text v[l[1]] substr(text_array[t], length(l[1])+2);
1726 <item> \ text = text "${" text_array[t];
1731 <chapter|Language Modes & Quoting><label|modes>
1733 <verbatim|lstlistings> and <verbatim|fangle> both recognize source
1734 languages, and perform some basic parsing and syntax highlighting in the
1735 rendered document<\footnote>
1736 although lstlisting supports many more languages
1737 </footnote>. <verbatim|lstlistings> can detect strings and comments within
1738 a language definition and perform suitable rendering, such as italics for
1739 comments, and visible-spaces within strings.
1741 Fangle similarly can recognize strings, and comments, etc, within a
1742 language, so that any chunks included with <verbatim|\\chunkref{a-chunk}>
1743 or <nf-ref|a-chunk|> can be suitably escape or quoted.
1745 <section|Modes to keep code together>
1747 As an example, the C language has a few parse modes, which affect the
1748 interpretation of characters.
1750 One parse mode is the string mode. The string mode is commenced by an
1751 un-escaped quotation mark <verbatim|"> and terminated by the same. Within
1752 the string mode, only one additional mode can be commenced, it is the
1753 backslash mode <verbatim|\\>, which is always terminated by the following
1756 Another mode is <verbatim|[> which is terminated by a <verbatim|]> (unless
1757 it occurs in a string).
1759 Consider this fragment of C code:
1761 <math|<math-tt|do_something><wide|<around*|(|<math-tt|things><wide|<around|[|<math-tt|x>,
1762 <math-tt|y>|]>|\<wide-overbrace\>><rsup|2. <math-tt|[> mode><math-tt|,
1763 get_other_things><wide|<around|(|<math-tt|a>,
1764 <wide*|<text|"><math-tt|<around|(|all|)>><text|">|\<wide-underbrace\>><rsub|4.
1765 <text|"> mode>|)>|\<wide-overbrace\>><rsup|3. <math-tt|(>
1766 mode>|)>|\<wide-overbrace\>><rsup|1. <math-tt|(> mode>>
1770 Mode nesting prevents the close parenthesis in the quoted string (part 4)
1771 from terminating the parenthesis mode (part 3).
1773 Each language has a set of modes, the default mode being the null mode.
1774 Each mode can lead to other modes.
1776 <section|Modes affect included chunks>
1778 For instance, consider this chunk with <verbatim|language=perl>:
1780 <\nf-chunk|test:example-perl>
1781 <item>print "hello world $0\\n";
1784 If it were included in a chunk with <verbatim|language=sh>, like this:
1786 <\nf-chunk|test:example-sh>
1787 <item>perl -e "<nf-ref|test:example-perl|>"
1790 we might want fangle would to generate output like this:
1792 <\nf-chunk|test:example-sh.result>
1793 <item>perl -e "print \\"hello world \\$0\\\\n\\";"
1796 See that the double quote <verbatim|">, back-slash <verbatim|\\> and
1797 <verbatim|$> have been quoted with a back-slash to protect them from shell
1800 If that were then included in a chunk with language=make, like this:
1802 <\nf-chunk|test:example-makefile>
1803 <item>target: pre-req
1805 <item><nf-tab><nf-ref|test:example-sh|>
1808 We would need the output to look like this --- note the <verbatim|$$> as
1809 the single <verbatim|$> has been makefile-quoted with another <verbatim|$>.
1811 <\nf-chunk|test:example-makefile.result>
1812 <item>target: pre-req
1814 <item><nf-tab>perl -e "print \\"hello world \\$$0\\\\n\\";"
1817 <section|Modes operation>
1819 In order to make this work, we must define a mode-tracker supporting each
1820 language, that can detect the various quoting modes, and provide a
1821 transformation that may be applied to any included text so that included
1822 text will be interpreted correctly after any interpolation that it may be
1823 subject to at run-time.
1825 For example, the sed transformation for text to be inserted into shell
1826 double-quoted strings would be something like:
1828 <verbatim|s/\\\\/\\\\\\\\/g;s/$/\\\\$/g;s/"/\\\\"/g;>
1830 which would protect <verbatim|\\ $ ">
1832 The mode tracker must also nested mode-changes, as in this shell example:
1834 <verbatim|echo "hello `id ...`">
1836 <phantom|<verbatim|echo "hello `id >><math|\<uparrow\>>
1838 Any shell special characters inserted at the point marked
1839 <math|\<uparrow\>> would need to be escaped if their plain-text meaning is
1840 to be preserved, including <verbatim|`> <verbatim|\|> <verbatim|*> among
1841 others. The set of characters that need escaping in the back-ticks
1842 <verbatim|`> is not the same as the set that need escaing in the
1843 double-quotes <verbatim|">. However, in shell syntax, a <verbatim|"> at the
1844 point marked <math|\<uparrow\>> does not close the leading <verbatim|"> and
1845 so would not need additional escaping because of the nesting of the two
1848 <todo|MAYBE>Escaping need not occur if the format and mode of the included
1849 chunk matches that of the including chunk.
1851 As each chunk is output a new mode tracker for that language is initialized
1852 in it's normal state. As text is output for that chunk the output mode is
1853 tracked. When a new chunk is included, a transformation appropriate to that
1854 mode is selected and pushed onto a stack of transformations. Any text to be
1855 output is passed through this stack of transformations.
1857 It remains to consider if the chunk-include function should return it's
1858 generated text so that the caller can apply any transformations (and
1859 formatting), or if it should apply the stack of transformations itself.
1861 Note that the transformed included text should have the property of not
1862 being able to change the mode in the current chunk.
1864 <todo|Note chunk parameters should probably also be transformed>
1866 <section|Quoting scenarios>
1868 <subsection|Direct quoting>
1870 He we give examples of various quoting scenarios and discuss what the
1871 expected outcome might be and how this could be obtained.
1873 <\with|par-columns|2>
1874 <\nf-chunk|test:q:1>
1875 <item>echo "$(<nf-ref|test:q:1-inc|>)"
1878 <\nf-chunk|test:q:1-inc>
1883 Should this examples produce <verbatim|echo "$(echo "hello")"> or
1884 <verbatim|echo "\\$(echo \\"hello\\")"> ?
1886 This depends on what the author intended, but we must provde a way to
1887 express that intent.
1889 We might argue that as both chunks have <verbatim|lang=sh> the intent must
1890 have been to quote the included chunk <emdash> but consider that this might
1891 be shell script that writes shell script.
1893 If <nf-ref|test:q:1-inc|> had <verbatim|lang=text> then it certainly would
1894 have been right to quote it, which leads us to ask: in what ways can we
1895 reduce quoting if lang of the included chunk is compatible with the lang of
1896 the including chunk?
1898 If we take a completely nested approach then even though <verbatim|$(> mode
1899 might do no quoting of it's own, <verbatim|"> mode will still do it's own
1900 quoting. We need a model where the nested <verbatim|$(> mode will prevent
1901 <verbatim|"> from quoting.
1903 This leads rise to the <em|tunneling> feature. In bash, the <verbatim|$(>
1904 gives rise to a new top-level parsing scenario, so we need to enter the
1905 <em|null> mode, and also ignore any quoting and then undo-this when the
1906 <verbatim|$(> mode is terminated by the corresponding close <verbatim|)>.
1908 We shall say that tunneling is when a mode in a language ignores other
1909 modes in the same language and arrives back at an earlier <em|null> mode of
1912 In example <nf-ref|test:q:1|> above, the nesting of modes is: <em|null>,
1913 <verbatim|">, <verbatim|$(>
1915 When mode <verbatim|$(> is commenced, the stack of nest modes will be
1916 traversed. If the <em|null> mode can be found in the same language, without
1917 the language varying, then a tunnel will be established so that the
1918 intervening modes, <verbatim|"> in this case, can be skipped when the modes
1919 are enumerated to quote the texted being emitted.
1921 In such a case, the correct result would be:
1923 <\nf-chunk|test:q:1.result>
1924 <item>echo "$(echo "hello")"
1927 <section|Language Mode Definitions>
1929 All modes definitions are stored in a single multi-dimensional hash. The
1930 first index is the language, and the second index is the mode-identifier.
1931 The third indexes hold properties: terminators, and optionally, submodes,
1932 delimiters, and tunnel targets.
1934 A useful set of mode definitions for a nameless general C-type language is
1935 shown here. (Don't be confused by the double backslash escaping needed in
1936 awk. One set of escaping is for the string, and the second set of escaping
1940 TODO: Add =\<less\>\\mode{}\<gtr\> command which will allow us to signify
1943 \ regex and thus fangle will quote it for us.
1946 Submodes are entered by the characters \ <verbatim|"> <verbatim|'>
1947 <verbatim|{> <verbatim|(> <verbatim|[> <verbatim|/*>
1949 <\nf-chunk|common-mode-definitions>
1950 <item>modes[<nf-arg|language>, "", \ "submodes"]="\\\\\\\\\|\\"\|'\|{\|\\\\(\|\\\\[";
1951 </nf-chunk||<tuple|language>>
1953 In the default mode, a comma surrounded by un-important white space is a
1954 delimiter of language items<\footnote>
1955 whatever a <em|language item> might be
1958 <\nf-chunk|common-mode-definitions>
1959 <item>modes[<nf-arg|language>, "", \ "delimiters"]=" *, *";
1960 </nf-chunk||language>
1962 and should pass this test:<todo|Why do the tests run in ?(? mode and not ??
1965 <\nf-chunk|test:mode-definitions>
1966 <item>parse_chunk_args("c-like", "1,2,3", a, "");
1968 <item>if (a[1] != "1") e++;
1970 <item>if (a[2] != "2") e++;
1972 <item>if (a[3] != "3") e++;
1974 <item>if (length(a) != 3) e++;
1976 <item><nf-ref|pca-test.awk:summary|>
1980 <item>parse_chunk_args("c-like", "joe, red", a, "");
1982 <item>if (a[1] != "joe") e++;
1984 <item>if (a[2] != "red") e++;
1986 <item>if (length(a) != 2) e++;
1988 <item><nf-ref|pca-test.awk:summary|>
1992 <item>parse_chunk_args("c-like", "${colour}", a, "");
1994 <item>if (a[1] != "${colour}") e++;
1996 <item>if (length(a) != 1) e++;
1998 <item><nf-ref|pca-test.awk:summary|>
2001 Nested modes are identified by a backslash, a double or single quote,
2002 various bracket styles or a <verbatim|/*> comment.
2004 For each of these sub-modes modes we must also identify at a mode
2005 terminator, and any sub-modes or delimiters that may be entered<\footnote>
2006 Because we are using the sub-mode characters as the mode identifier it
2007 means we can't currently have a mode character dependant on it's context;
2008 i.e. <verbatim|{> can't behave differently when it is inside
2012 <subsection|Backslash>
2014 The backslash mode has no submodes or delimiters, and is terminated by any
2015 character. Note that we are not so much interested in evaluating or
2016 interpolating content as we are in delineating content. It is no matter
2017 that a double backslash (<verbatim|\\\\>) may represent a single backslash
2018 while a backslash-newline may represent white space, but it does matter
2019 that the newline in a backslash newline should not be able to terminate a C
2020 pre-processor statement; and so the newline will be consumed by the
2021 backslash however it is to be interpreted.
2023 <\nf-chunk|common-mode-definitions>
2024 <item>modes[<nf-arg|language>, "\\\\", "terminators"]=".";
2027 <subsection|Strings>
2029 Common languages support two kinds of strings quoting, double quotes and
2032 In a string we have one special mode, which is the backslash. This may
2033 escape an embedded quote and prevent us thinking that it should terminate
2036 <\nf-chunk|mode:common-string>
2037 <item>modes[<nf-arg|language>, <nf-arg|quote>, "submodes"]="\\\\\\\\";
2038 </nf-chunk||<tuple|language|quote>>
2040 Otherwise, the string will be terminated by the same character that
2043 <\nf-chunk|mode:common-string>
2044 <item>modes[<nf-arg|language>, <nf-arg|quote>,
2045 "terminators"]=<nf-arg|quote>;
2046 </nf-chunk||language>
2048 In C type languages, certain escape sequences exist in strings. We need to
2049 define mechanism to enclode any chunks included in this mode using those
2050 escape sequences. These are expressed in two parts, s meaning search, and r
2053 The first substitution is to replace a backslash with a double backslash.
2054 We do this first as other substitutions may introduce a backslash which we
2055 would not then want to escape again here.
2057 Note: Backslashes need double-escaping in the search pattern but not in the
2058 replacement string, hence we are replacing a literal <verbatim|\\> with a
2059 literal <verbatim|\\\\>.
2061 <\nf-chunk|mode:common-string>
2062 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2063 ++escapes[<nf-arg|language>, <nf-arg|quote>], "s"]="\\\\\\\\";
2065 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2066 \ \ escapes[<nf-arg|language>, <nf-arg|quote>], "r"]="\\\\\\\\";
2067 </nf-chunk||language>
2069 If the quote character occurs in the text, it should be preceded by a
2070 backslash, otherwise it would terminate the string unexpectedly.
2072 <\nf-chunk|mode:common-string>
2073 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2074 ++escapes[<nf-arg|language>, <nf-arg|quote>], "s"]=<nf-arg|quote>;
2076 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2077 \ \ escapes[<nf-arg|language>, <nf-arg|quote>], "r"]="\\\\"
2079 </nf-chunk||language>
2081 Any newlines in the string, must be replaced by <verbatim|\\n>.
2083 <\nf-chunk|mode:common-string>
2084 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2085 ++escapes[<nf-arg|language>, <nf-arg|quote>], "s"]="\\n";
2087 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2088 \ \ escapes[<nf-arg|language>, <nf-arg|quote>], "r"]="\\\\n";
2089 </nf-chunk||language>
2091 For the common modes, we define this string handling for double and single
2094 <\nf-chunk|common-mode-definitions>
2095 <item><nf-ref|mode:common-string|<tuple|<nf-arg|language>|"\\"">>
2097 <item><nf-ref|mode:common-string|<tuple|<nf-arg|language>|"'">>
2100 Working strings should pass this test:
2102 <\nf-chunk|test:mode-definitions>
2103 <item>parse_chunk_args("c-like", "say \\"I said, \\\\\\"Hello, how are
2104 you\\\\\\".\\", for me", a, "");
2106 <item>if (a[1] != "say \\"I said, \\\\\\"Hello, how are you\\\\\\".\\"")
2109 <item>if (a[2] != "for me") e++;
2111 <item>if (length(a) != 2) e++;
2113 <item><nf-ref|pca-test.awk:summary|>
2116 <subsection|Parentheses, Braces and Brackets>
2118 Where quotes are closed by the same character, parentheses, brackets and
2119 braces are closed by an alternate character.
2121 <\nf-chunk|mode:common-brackets>
2122 <item>modes[<nf-arg|language>, <nf-arg|open>, \ "submodes"
2123 ]="\\\\\\\\\|\\"\|{\|\\\\(\|\\\\[\|'\|/\\\\*";
2125 <item>modes[<nf-arg|language>, <nf-arg|open>, \ "delimiters"]=" *, *";
2127 <item>modes[<nf-arg|language>, <nf-arg|open>,
2128 \ "terminators"]=<nf-arg|close>;
2129 </nf-chunk||<tuple|language|open|close>>
2131 Note that the open is NOT a regex but the close token IS. <todo|When we can
2132 quote regex we won't have to put the slashes in here>
2134 <\nf-chunk|common-mode-definitions>
2135 <item><nf-ref|mode:common-brackets|<tuple|<nf-arg|language>|"{"|"}">>
2137 <item><nf-ref|mode:common-brackets|<tuple|<nf-arg|language>|"["|"\\\\]">>
2139 <item><nf-ref|mode:common-brackets|<tuple|<nf-arg|language>|"("|"\\\\)">>
2142 <subsection|Customizing Standard Modes>
2144 <\nf-chunk|mode:add-submode>
2145 <item>modes[<nf-arg|language>, <nf-arg|mode>, "submodes"] =
2146 modes[<nf-arg|language>, <nf-arg|mode>, "submodes"] "\|"
2148 </nf-chunk||<tuple|language|mode|submode>>
2150 <\nf-chunk|mode:add-escapes>
2151 <item>escapes[<nf-arg|language>, <nf-arg|mode>,
2152 ++escapes[<nf-arg|language>, <nf-arg|mode>], "s"]=<nf-arg|search>;
2154 <item>escapes[<nf-arg|language>, <nf-arg|mode>,
2155 \ \ escapes[<nf-arg|language>, <nf-arg|mode>], "r"]=<nf-arg|replace>;
2156 </nf-chunk||<tuple|language|mode|search|replace>>
2160 <subsection|Comments>
2162 We can define <verbatim|/* comment */> style comments and
2163 <verbatim|//comment> style comments to be added to any language:
2165 <\nf-chunk|mode:multi-line-comments>
2166 <item><nf-ref|mode:add-submode|<tuple|<nf-arg|language>|""|"/\\\\*">>
2168 <item>modes[<nf-arg|language>, "/*", "terminators"]="\\\\*/";
2169 </nf-chunk||<tuple|language>>
2171 <\nf-chunk|mode:single-line-slash-comments>
2172 <item><nf-ref|mode:add-submode|<tuple|<nf-arg|language>|""|"//">>
2174 <item>modes[<nf-arg|language>, "//", "terminators"]="\\n";
2176 <item><nf-ref|mode:add-escapes|<tuple|<nf-arg|language>|"//"|"\\n"|"\\n//">>
2177 </nf-chunk||language>
2179 We can also define <verbatim|# comment> style comments (as used in awk and
2180 shell scripts) in a similar manner.
2182 <todo|I'm having to use # for hash and \textbackslash{} for \ and have
2183 hacky work-arounds in the parser for now>
2185 <\nf-chunk|mode:add-hash-comments>
2186 <item><nf-ref|mode:add-submode|<tuple|<nf-arg|language>|""|"#">>
2188 <item>modes[<nf-arg|language>, "#", "terminators"]="\\n";
2190 <item><nf-ref|mode:add-escapes|<tuple|<nf-arg|language>|"#"|"\\n"|"\\n#">>
2191 </nf-chunk||<tuple|language>>
2193 In C, the <verbatim|#> denotes pre-processor directives which can be
2196 <\nf-chunk|mode:add-hash-defines>
2197 <item><nf-ref|mode:add-submode|<tuple|<nf-arg|language>|""|"#">>
2199 <item>modes[<nf-arg|language>, "#", "submodes" ]="\\\\\\\\";
2201 <item>modes[<nf-arg|language>, "#", "terminators"]="\\n";
2203 <item><nf-ref|mode:add-escapes|<tuple|<nf-arg|language>|"#"|"\\n"|"\\\\\\\\\\n">>
2204 </nf-chunk||<tuple|language>>
2206 <\nf-chunk|mode:quote-dollar-escape>
2207 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2208 ++escapes[<nf-arg|language>, <nf-arg|quote>], "s"]="\\\\$";
2210 <item>escapes[<nf-arg|language>, <nf-arg|quote>,
2211 \ \ escapes[<nf-arg|language>, <nf-arg|quote>], "r"]="\\\\$";
2212 </nf-chunk||<tuple|language|quote>>
2214 We can add these definitions to various languages
2216 <\nf-chunk|mode-definitions>
2217 <item><nf-ref|common-mode-definitions|<tuple|"c-like">>
2221 <item><nf-ref|common-mode-definitions|<tuple|"c">>
2223 <item><nf-ref|mode:multi-line-comments|<tuple|"c">>
2225 <item><nf-ref|mode:single-line-slash-comments|<tuple|"c">>
2227 <item><nf-ref|mode:add-hash-defines|<tuple|"c">>
2231 <item><nf-ref|common-mode-definitions|<tuple|"awk">>
2233 <item><nf-ref|mode:add-hash-comments|<tuple|"awk">>
2235 <item><nf-ref|mode:add-naked-regex|<tuple|"awk">>
2238 The awk definitions should allow a comment block like this:
2240 <nf-chunk|test:comment-quote|<item># Comment:
2241 <nf-ref|test:comment-text|>|awk|>
2243 <\nf-chunk|test:comment-text>
2244 <item>Now is the time for
2246 <item>the quick brown fox to bring lemonade
2251 to come out like this:
2253 <\nf-chunk|test:comment-quote:result>
2254 <item># Comment: Now is the time for
2256 <item>#the quick brown fox to bring lemonade
2261 The C definition for such a block should have it come out like this:
2263 <\nf-chunk|test:comment-quote:C-result>
2264 <item># Comment: Now is the time for\\
2266 <item>the quick brown fox to bring lemonade\\
2273 This pattern is incomplete, but meant to detect naked regular expressions
2274 in awk and perl; e.g. <verbatim|/.*$/>, however required capabilities are
2277 Current it only detects regexes anchored with ^ as used in fangle.
2279 For full regex support, modes need to be named not after their starting
2280 character, but some other more fully qualified name.
2282 <\nf-chunk|mode:add-naked-regex>
2283 <item><nf-ref|mode:add-submode|<tuple|<nf-arg|language>|""|"/\\\\^">>
2285 <item>modes[<nf-arg|language>, "/^", "terminators"]="/";
2286 </nf-chunk||<tuple|language>>
2290 <\nf-chunk|mode-definitions>
2291 <item><nf-ref|common-mode-definitions|<tuple|"perl">>
2293 <item><nf-ref|mode:multi-line-comments|<tuple|"perl">>
2295 <item><nf-ref|mode:add-hash-comments|<tuple|"perl">>
2298 Still need to add add <verbatim|s/>, submode <verbatim|/>, terminate both
2299 with <verbatim|//>. This is likely to be impossible as perl regexes can
2304 Shell single-quote strings are different to other strings and have no
2305 escape characters. The only special character is the single quote
2306 <verbatim|'> which always closes the string. Therefore we cannot use
2307 <nf-ref|common-mode-definitions|<tuple|"sh">> but we will invoke most of
2308 it's definition apart from single-quote strings.\
2310 <\nf-chunk|mode-definitions>
2311 <item>modes["sh", "", \ "submodes"]="\\\\\\\\\|\\"\|'\|{\|\\\\(\|\\\\[\|\\\\$\\\\(";
2313 <item>modes["sh", "\\\\", "terminators"]=".";
2317 <item>modes["sh", "\\"", "submodes"]="\\\\\\\\\|\\\\$\\\\(";
2319 <item>modes["sh", "\\"", "terminators"]="\\"";
2321 <item>escapes["sh", "\\"", ++escapes["sh", "\\""], "s"]="\\\\\\\\";
2323 <item>escapes["sh", "\\"", \ \ escapes["sh", "\\""], "r"]="\\\\\\\\";
2325 <item>escapes["sh", "\\"", ++escapes["sh", "\\""], "s"]="\\"";
2327 <item>escapes["sh", "\\"", \ \ escapes["sh", "\\""], "r"]="\\\\" "\\"";
2329 <item>escapes["sh", "\\"", ++escapes["sh", "\\""], "s"]="\\n";
2331 <item>escapes["sh", "\\"", \ \ escapes["sh", "\\""], "r"]="\\\\n";
2335 <item>modes["sh", "'", "terminators"]="'";
2337 <item>escapes["sh", "'", ++escapes["sh", "'"], "s"]="'";
2339 <item>escapes["sh", "'", \ \ escapes["sh", "'"], "r"]="'\\\\'" "'";
2341 <item><nf-ref|mode:common-brackets|<tuple|"sh"|"$("|"\\\\)">>
2343 <item><nf-ref|mode:add-tunnel|<tuple|"sh"|"$("|"">>
2345 <item><nf-ref|mode:common-brackets|<tuple|"sh"|"{"|"}">>
2347 <item><nf-ref|mode:common-brackets|<tuple|"sh"|"["|"\\\\]">>
2349 <item><nf-ref|mode:common-brackets|<tuple|"sh"|"("|"\\\\)">>
2351 <item><nf-ref|mode:add-hash-comments|<tuple|"sh">>
2353 <item><nf-ref|mode:quote-dollar-escape|<tuple|"sh"|"\\"">>
2356 The definition of add-tunnel is:
2358 <\nf-chunk|mode:add-tunnel>
2359 <item>escapes[<nf-arg|language>, <nf-arg|mode>,
2360 ++escapes[<nf-arg|language>, <nf-arg|mode>], "tunnel"]=<nf-arg|tunnel>;
2361 </nf-chunk||<tuple|language|mode|tunnel>>
2365 For makefiles, we currently recognize 2 modes: the <em|null> mode and
2366 <nf-tab> mode, which is tabbed mode and contains the makefile recipie. In
2367 the <em|null> mode the only escape is <verbatim|$> which must be converted
2370 Tabbed mode is harder to manage, as the GNU Make Manual says in the section
2371 on <hlink|splitting lines|http://www.gnu.org/s/hello/manual/make/Splitting-Lines.html>.
2372 There is no way to escape a multi-line text that occurs as part of a
2375 Despite this sad fact, if the newline's in the shell script all occur at
2376 points of top-level shell syntax, then we could replace them with
2377 <verbatim| ;\\n<nf-tab>>and largely get the right effect.
2379 <\with|par-columns|2>
2380 <\nf-chunk|test:make:1>
2383 <item><nf-tab>echo making
2385 <item><nf-tab><nf-ref|test:make:1-inc|$@>
2390 <\nf-chunk|test:make:1-inc>
2391 <item>if test "<nf-arg|target>" = "all"
2393 <item>then echo yes, all
2395 <item>else echo not all
2398 </nf-chunk|sh|<tuple|target>>
2401 The two chunks about could reasonably produce this:
2403 <\nf-chunk|test:make:1.result>
2406 <item><nf-tab>echo making
2408 <item><nf-tab>if test "$@" = "all" ;\\
2410 <item><nf-tab>then echo yes, all ;\\
2412 <item><nf-tab>else echo not all ;\\
2417 But will more likely produce this:
2419 <\nf-chunk|test:make:1.result-actual>
2422 <item><nf-tab>echo making
2424 <item><nf-tab>if test "$$@" = "all" ;\\
2426 <item><nf-tab> then echo yes, all ;\\
2428 <item><nf-tab> else echo not all ;\\
2433 The chunk argument <verbatim|$@> has been quoted (which would have been
2434 fine if we were passing the name of a shell variable), and the other shell
2435 lines are (harmlessly) indented by 1 space as part of fangle
2436 indent-matching which should have taken into account the expanded tab size,
2437 and should generally take into account the expanded prefix of the line
2438 whose indent it is trying to match, but which in this case we want to have
2442 The $@ was passed from a make fragment. In what cases should it be
2445 Do we need to track the language of sources of arguments?
2448 A more ugly work-around until this problem can be solved would be to use
2451 <\nf-chunk|test:make:2>
2454 <item><nf-tab>echo making
2456 <item><nf-tab>ARG="$@"; <nf-ref|test:make:1-inc|$ARG>
2459 which produces this output which is more useful (because it works):
2461 <\nf-chunk|test:make:2.result>
2464 <item><nf-tab>echo making test
2466 <item><nf-tab>ARG="$@"; if test "$$ARG" = "all" ;\\
2468 <item><nf-tab> \ \ \ \ \ \ \ \ \ \ then echo yes, all ;\\
2470 <item><nf-tab> \ \ \ \ \ \ \ \ \ \ else echo not all ;\\
2472 <item><nf-tab> \ \ \ \ \ \ \ \ \ \ fi
2475 If, however, the shell fragment contained strings with literal newline
2476 characters then there would be no easy way to escape these and preserve the
2477 value of the string.
2479 A different style of makefile construction might be used <emdash> the
2480 recipe could be stored in a <hlink|target specific
2481 variable|http://www.gnu.org/s/hello/manual/make/Target_002dspecific.html>
2482 which contains the recipe with a more normal escape mechanism.
2486 <\nf-chunk|mode-definitions>
2487 <item>modes["make", "", \ "submodes"]="<nf-tab>";
2489 <item>modes["make", "<nf-tab>", "terminators"]="\\\\n";
2491 <item>escapes["make", "<nf-tab>", ++escapes["make", "<nf-tab>"],
2494 <item>escapes["make", "<nf-tab>", escapes["make", "<nf-tab>"], "r"]="
2497 <item>escapes["make", "<nf-tab>", ++escapes["make", "<nf-tab>"],
2500 <item>escapes["make", "<nf-tab>", escapes["make", "<nf-tab>"], "r"]="$$";
2503 Note also that the tab character is hard-wired into the pattern, and that
2504 the make variable <verbatim|.RECIPEPREFIX> might change this to something
2507 <section|Some tests>
2509 Also, the parser must return any spare text at the end that has not been
2510 processed due to a mode terminator being found.
2512 <\nf-chunk|test:mode-definitions>
2513 <item>rest = parse_chunk_args("c-like", "1, 2, 3) spare", a, "(");
2515 <item>if (a[1] != 1) e++;
2517 <item>if (a[2] != 2) e++;
2519 <item>if (a[3] != 3) e++;
2521 <item>if (length(a) != 3) e++;
2523 <item>if (rest != " spare") e++;
2525 <item><nf-ref|pca-test.awk:summary|>
2528 We must also be able to parse the example given earlier.
2530 <\nf-chunk|test:mode-definitions>
2531 <item>parse_chunk_args("c-like", "things[x, y], get_other_things(a,
2532 \\"(all)\\"), 99", a, "(");
2534 <item>if (a[1] != "things[x, y]") e++;
2536 <item>if (a[2] != "get_other_things(a, \\"(all)\\")") e++;
2538 <item>if (a[3] != "99") e++;
2540 <item>if (length(a) != 3) e++;
2542 <item><nf-ref|pca-test.awk:summary|>
2545 <section|A non-recursive mode tracker>
2547 <subsection|Constructor>
2549 The mode tracker holds its state in a stack based on a numerically indexed
2550 hash. This function, when passed an empty hash, will intialize it.
2552 <\nf-chunk|new_mode_tracker()>
2553 <item>function new_mode_tracker(context, language, mode) {
2555 <item> \ context[""] = 0;
2557 <item> \ context[0, "language"] = language;
2559 <item> \ context[0, "mode"] = mode;
2564 Because awk functions cannot return an array, we must create the array
2565 first and pass it in, so we have a fangle macro to do this:
2567 <\nf-chunk|new-mode-tracker>
2568 <item><nf-ref|awk-delete-array|<tuple|context>>
2570 <item>new_mode_tracker(<nf-arg|context>, <nf-arg|language>,
2572 </nf-chunk|awk|<tuple|context|language|mode>>
2574 <subsection|Management>
2576 And for tracking modes, we dispatch to a mode-tracker action based on the
2579 <\nf-chunk|mode_tracker>
2580 <item>function push_mode_tracker(context, language, mode,
2582 <item> \ # local vars
2588 <item> \ if (! ("" in context)) {
2590 <item> \ \ \ <nf-ref|new-mode-tracker|<tuple|context|language|mode>>
2592 <item> \ \ \ return;
2596 <item> \ \ \ top = context[""];
2598 <item># \ \ \ if (context[top, "language"] == language && mode=="") mode
2599 = context[top, "mode"];
2601 <item> \ \ \ if (context[top, "language"] == language && context[top,
2602 "mode"] == mode) return top;
2604 <item> \ \ \ old_top = top;
2608 <item> \ \ \ context[top, "language"] = language;
2610 <item> \ \ \ context[top, "mode"] = mode;
2612 <item> \ \ \ context[""] = top;
2616 <item> \ return old_top;
2621 <\nf-chunk|mode_tracker>
2622 <item>function dump_mode_tracker(context, \
2628 <item> \ for(c=0; c \<less\>= context[""]; c++) {
2630 <item> \ \ \ printf(" %2d \ \ %s:%s\\n", c, context[c, "language"],
2631 context[c, "mode"]) \<gtr\> "/dev/stderr";
2633 <item> \ \ \ for(d=1; ( (c, "values", d) in context); d++) {
2635 <item> \ \ \ \ \ printf(" \ \ %2d %s\\n", d, context[c, "values", d])
2636 \<gtr\> "/dev/stderr";
2645 <\nf-chunk|mode_tracker>
2646 <item>function pop_mode_tracker(context, context_origin)
2650 <item> \ if ( (context_origin) && ("" in context) && context[""] !=
2651 (1+context_origin) && context[""] != context_origin) return 0;
2653 <item> \ context[""] = context_origin;
2660 This implies that any chunk must be syntactically whole; for instance, this
2663 <\nf-chunk|test:whole-chunk>
2666 <item> \ <nf-ref|test:say-hello|>
2671 <\nf-chunk|test:say-hello>
2672 <item>print "hello";
2675 But this is not fine; the chunk <nf-ref|test:hidden-else|> is not properly
2678 <\nf-chunk|test:partial-chunk>
2681 <item> \ <nf-ref|test:hidden-else|>
2686 <\nf-chunk|test:hidden-else>
2687 <item> \ print "I'm fine";
2691 <item> \ print "I'm not";
2694 These tests will check for correct behaviour:
2696 <\nf-chunk|test:cromulence>
2697 <item>echo Cromulence test
2699 <item>passtest $FANGLE -Rtest:whole-chunk $TXT_SRC &\<gtr\>/dev/null \|\|
2700 ( echo "Whole chunk failed" && exit 1 )
2702 <item>failtest $FANGLE -Rtest:partial-chunk $TXT_SRC &\<gtr\>/dev/null
2703 \|\| ( echo "Partial chunk failed" && exit 1 )
2706 <subsection|Tracker>
2708 We must avoid recursion as a language construct because we intend to employ
2709 mode-tracking to track language mode of emitted code, and the code is
2710 emitted from a function which is itself recursive, so instead we implement
2711 psuedo-recursion using our own stack based on a hash.
2713 <\nf-chunk|mode_tracker()>
2714 <item>function mode_tracker(context, text, values,\
2716 <item> \ # optional parameters
2718 <item> \ # local vars
2720 <item> \ mode, submodes, language,
2722 <item> \ cindex, c, a, part, item, name, result, new_values, new_mode,\
2724 <item> \ delimiters, terminators)
2729 We could be re-commencing with a valid context, so we need to setup the
2730 state according to the last context.
2732 <\nf-chunk|mode_tracker()>
2733 <item> \ cindex = context[""] + 0;
2735 <item> \ mode = context[cindex, "mode"];
2737 <item> \ language = context[cindex, "language" ];
2740 First we construct a single large regex combining the possible sub-modes
2741 for the current mode along with the terminators for the current mode.
2743 <\nf-chunk|parse_chunk_args-reset-modes>
2744 <item> \ submodes=modes[language, mode, "submodes"];
2748 <item> \ if ((language, mode, "delimiters") in modes) {
2750 <item> \ \ \ delimiters = modes[language, mode, "delimiters"];
2752 <item> \ \ \ if (length(submodes)\<gtr\>0) submodes = submodes "\|";
2754 <item> \ \ \ submodes=submodes delimiters;
2756 <item> \ } else delimiters="";
2758 <item> \ if ((language, mode, "terminators") in modes) {
2760 <item> \ \ \ terminators = modes[language, mode, "terminators"];
2762 <item> \ \ \ if (length(submodes)\<gtr\>0) submodes = submodes "\|";
2764 <item> \ \ \ submodes=submodes terminators;
2766 <item> \ } else terminators="";
2769 If we don't find anything to match on --- probably because the language is
2770 not supported --- then we return the entire text without matching anything.
2772 <\nf-chunk|parse_chunk_args-reset-modes>
2773 <item> if (! length(submodes)) return text;
2776 <\nf-chunk|mode_tracker()>
2777 <item><nf-ref|parse_chunk_args-reset-modes|>
2780 We then iterate the text (until there is none left) looking for sub-modes
2781 or terminators in the regex.
2783 <\nf-chunk|mode_tracker()>
2784 <item> \ while((cindex \<gtr\>= 0) && length(text)) {
2786 <item> \ \ \ if (match(text, "(" submodes ")", a)) {
2789 A bug that creeps in regularly during development is bad regexes of zero
2790 length which result in an infinite loop (as no text is consumed), so I
2791 catch that right away with this test.
2793 <\nf-chunk|mode_tracker()>
2794 <item> \ \ \ \ \ if (RLENGTH\<less\>1) {
2796 <item> \ \ \ \ \ \ \ error(sprintf("Internal error, matched zero length
2797 submode, should be impossible - likely regex computation error\\n" \\
2799 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "Language=%s\\nmode=%s\\nmatch=%s\\n",
2800 language, mode, submodes));
2805 part is defined as the text up to the sub-mode or terminator, and this is
2806 appended to item --- which is the current text being gathered. If a mode
2807 has a delimiter, then item is reset each time a delimiter is found.
2809 <math|<wide|<with|mode|prog|"><wide*|hello|\<wide-underbrace\>><rsub|item>,
2810 <wide*|there|\<wide-underbrace\>><rsub|item><with|mode|prog|">|\<wide-overbrace\>><rsup|item>,
2811 \ <wide|he said.|\<wide-overbrace\>><rsup|item>>
2813 <\nf-chunk|mode_tracker()>
2814 <item> \ \ \ \ \ part = substr(text, 1, RSTART -1);
2816 <item> \ \ \ \ \ item = item part;
2819 We must now determine what was matched. If it was a terminator, then we
2820 must restore the previous mode.
2822 <\nf-chunk|mode_tracker()>
2823 <item> \ \ \ \ \ if (match(a[1], "^" terminators "$")) {
2825 <item>#printf("%2d EXIT \ MODE [%s] by [%s] [%s]\\n", cindex, mode, a[1],
2826 text) \<gtr\> "/dev/stderr"
2828 <item> \ \ \ \ \ \ \ context[cindex, "values", ++context[cindex,
2831 <item> \ \ \ \ \ \ \ delete context[cindex];
2833 <item> \ \ \ \ \ \ \ context[""] = --cindex;
2835 <item> \ \ \ \ \ \ \ if (cindex\<gtr\>=0) {
2837 <item> \ \ \ \ \ \ \ \ \ mode = context[cindex, "mode"];
2839 <item> \ \ \ \ \ \ \ \ \ language = context[cindex, "language"];
2841 <item> \ \ \ \ \ \ \ \ \ <nf-ref|parse_chunk_args-reset-modes|>
2843 <item> \ \ \ \ \ \ \ }
2845 <item> \ \ \ \ \ \ \ item = item a[1];
2847 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2853 If a delimiter was matched, then we must store the current item in the
2854 parsed values array, and reset the item.
2856 <\nf-chunk|mode_tracker()>
2857 <item> \ \ \ \ \ else if (match(a[1], "^" delimiters "$")) {
2859 <item> \ \ \ \ \ \ \ if (cindex==0) {
2861 <item> \ \ \ \ \ \ \ \ \ context[cindex, "values", ++context[cindex,
2864 <item> \ \ \ \ \ \ \ \ \ item = "";
2866 <item> \ \ \ \ \ \ \ } else {
2868 <item> \ \ \ \ \ \ \ \ \ item = item a[1];
2870 <item> \ \ \ \ \ \ \ }
2872 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2878 otherwise, if a new submode is detected (all submodes have terminators), we
2879 must create a nested parse context until we find the terminator for this
2882 <\nf-chunk|mode_tracker()>
2883 <item> else if ((language, a[1], "terminators") in modes) {
2885 <item> \ \ \ \ \ \ \ #check if new_mode is defined
2887 <item> \ \ \ \ \ \ \ item = item a[1];
2889 <item>#printf("%2d ENTER MODE [%s] in [%s]\\n", cindex, a[1], text)
2890 \<gtr\> "/dev/stderr"
2892 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2895 <item> \ \ \ \ \ \ \ context[""] = ++cindex;
2897 <item> \ \ \ \ \ \ \ context[cindex, "mode"] = a[1];
2899 <item> \ \ \ \ \ \ \ context[cindex, "language"] = language;
2901 <item> \ \ \ \ \ \ \ mode = a[1];
2903 <item> \ \ \ \ \ \ \ <nf-ref|parse_chunk_args-reset-modes|>
2905 <item> \ \ \ \ \ } else {
2907 <item> \ \ \ \ \ \ \ error(sprintf("Submode '%s' set unknown mode in
2908 text: %s\\nLanguage %s Mode %s\\n", a[1], text, language, mode));
2910 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2918 In the final case, we parsed to the end of the string. If the string was
2919 entire, then we should have no nested mode context, but if the string was
2920 just a fragment we may have a mode context which must be preserved for the
2921 next fragment. Todo: Consideration ought to be given if sub-mode strings
2922 are split over two fragments.
2924 <\nf-chunk|mode_tracker()>
2927 <item> \ \ \ \ \ context[cindex, "values", ++context[cindex, "values"]] =
2930 <item> \ \ \ \ \ text = "";
2932 <item> \ \ \ \ \ item = "";
2940 <item> \ context["item"] = item;
2944 <item> \ if (length(item)) context[cindex, "values", ++context[cindex,
2947 <item> \ return text;
2952 <subsubsection|One happy chunk>
2954 All the mode tracker chunks are referred to here:
2956 <\nf-chunk|mode-tracker>
2957 <item><nf-ref|new_mode_tracker()|>
2959 <item><nf-ref|mode_tracker()|>
2962 <subsubsection|Tests>
2964 We can test this function like this:
2966 <\nf-chunk|pca-test.awk>
2967 <item><nf-ref|error()|>
2969 <item><nf-ref|mode-tracker|>
2971 <item><nf-ref|parse_chunk_args()|>
2975 <item> \ SUBSEP=".";
2977 <item> \ <nf-ref|mode-definitions|>
2981 <item> \ <nf-ref|test:mode-definitions|>
2986 <\nf-chunk|pca-test.awk:summary>
2989 <item> \ printf "Failed " e
2991 <item> \ for (b in a) {
2993 <item> \ \ \ print "a[" b "] =\<gtr\> " a[b];
2999 <item> \ print "Passed"
3008 which should give this output:
3010 <\nf-chunk|pca-test.awk-results>
3011 <item>a[foo.quux.quirk] =\<gtr\>\
3013 <item>a[foo.quux.a] =\<gtr\> fleeg
3015 <item>a[foo.bar] =\<gtr\> baz
3017 <item>a[etc] =\<gtr\>\
3019 <item>a[name] =\<gtr\> freddie
3022 <section|Escaping and Quoting>
3024 For the time being and to get around <TeXmacs> inability to export a
3025 <kbd|TAB> character, the right arrow <with|mode|math|\<mapsto\>> whose
3026 UTF-8 sequence is ...
3030 Another special character is used, the left-arrow
3031 <with|mode|math|\<mapsfrom\>> with UTF-8 sequence 0xE2 0x86 0xA4 is used to
3032 strip any preceding white space as a way of un-tabbing and removing indent
3033 that has been applied <emdash> this is important for bash here documents,
3034 and the like. It's a filthy hack.
3036 <todo|remove the hack>
3038 <\nf-chunk|mode_tracker>
3041 <item>function untab(text) {
3043 <item> \ gsub("[[:space:]]*\\xE2\\x86\\xA4","", text);
3045 <item> \ return text;
3050 Each nested mode can optionally define a set of transforms to be applied to
3051 any text that is included from another language.
3053 This code can perform transforms from index c downwards.
3055 <\nf-chunk|mode_tracker>
3056 <item>function transform_escape(context, text, top,
3058 <item> \ c, cp, cpl, s, r)
3062 <item> \ for(c = top; c \<gtr\>= 0; c--) {
3064 <item> \ \ \ if ( (context[c, "language"], context[c, "mode"]) in
3067 <item> \ \ \ \ \ cpl = escapes[context[c, "language"], context[c,
3070 <item> \ \ \ \ \ for (cp = 1; cp \<less\>= cpl; cp ++) {
3072 <item> \ \ \ \ \ \ \ s = escapes[context[c, "language"], context[c,
3075 <item> \ \ \ \ \ \ \ r = escapes[context[c, "language"], context[c,
3078 <item> \ \ \ \ \ \ \ if (length(s)) {
3080 <item> \ \ \ \ \ \ \ \ \ gsub(s, r, text);
3082 <item> \ \ \ \ \ \ \ }
3084 <item> \ \ \ \ \ \ \ if ( (context[c, "language"], context[c, "mode"],
3085 cp, "t") in escapes ) {
3087 <item> \ \ \ \ \ \ \ \ \ quotes[src, "t"] = escapes[context[c,
3088 "language"], context[c, "mode"], cp, "t"];
3090 <item> \ \ \ \ \ \ \ }
3098 <item> \ return text;
3102 <item>function dump_escaper(quotes, r, cc) {
3104 <item> \ for(cc=1; cc\<less\>=c; cc++) {
3106 <item> \ \ \ printf("%2d s[%s] r[%s]\\n", cc, quotes[cc, "s"], quotes[cc,
3107 "r"]) \<gtr\> "/dev/stderr"
3114 <\nf-chunk|test:escapes>
3115 <item>echo escapes test
3117 <item>passtest $FANGLE -Rtest:comment-quote $TXT_SRC &\<gtr\>/dev/null
3118 \|\| ( echo "Comment-quote failed" && exit 1 )
3121 <chapter|Recognizing Chunks>
3123 Fangle recognizes noweb chunks, but as we also want better <LaTeX>
3124 integration we will recognize any of these:
3127 <item>notangle chunks matching the pattern
3128 <verbatim|^\<less\>\<less\>.*?\<gtr\>\<gtr\>=>
3130 <item>chunks beginning with <verbatim|\\begin{lstlistings}>, possibly
3131 with <verbatim|\\Chunk{...}> on the previous line
3133 <item>an older form I have used, beginning with
3134 <verbatim|\\begin{Chunk}[options]> --- also more suitable for plain
3135 <LaTeX> users<\footnote>
3136 Is there such a thing as plain <LaTeX>?
3140 <section|Chunk start>
3142 The variable chunking is used to signify that we are processing a code
3143 chunk and not document. In such a state, input lines will be assigned to
3144 the current chunk; otherwise they are ignored.
3146 <subsection|<TeXmacs>>
3148 We don't handle <TeXmacs> files natively yet, but rather instead emit
3149 unicode character sequences to mark up the text-export file which we do
3152 These hacks detect the unicode character sequences and retro-fit in the old
3155 We convert <math|\<mapsto\>> into a tab character.
3157 <\nf-chunk|recognize-chunk>
3162 <item># \ gsub("\\n*$","");
3164 <item># \ gsub("\\n", " ");
3170 <item>/\\xE2\\x86\\xA6/ {
3172 <item> \ gsub("\\\\xE2\\\\x86\\\\xA6", "\\x09");
3177 <TeXmacs> back-tick handling is obscure, and a cut-n-paste back-tick from a
3178 shell window comes out as a unicode sequence<\footnote>
3179 that won't export to html, except as a NULL character (literal 0x00)
3180 </footnote> that is fixed-up here.
3182 <\nf-chunk|recognize-chunk>
3185 <item>/\\xE2\\x80\\x98/ {
3187 <item> \ gsub("\\\\xE2\\\\x80\\\\x98", "`");
3192 In the <TeXmacs> output, the start of a chunk will appear like this:
3194 <verbatim| \ 5b\<less\>example-chunk<key|^K>[1](arg1,<key|^K>
3195 arg2<key|^K><key|^K>), lang=C\<gtr\> <math|\<equiv\>>>
3197 We detect the the start of a <TeXmacs> chunk by detecting the
3198 <math|\<equiv\>> symbol which occurs near the end of the line. We obtain
3199 the chunk name, the chunk parameters, and the chunk language.
3201 <\nf-chunk|recognize-chunk>
3204 <item>/\\xE2\\x89\\xA1/ {
3206 <item> \ if (match($0, "^ *([^[ ]* \|)\<less\>([^[
3207 ]*)\\\\[[0-9]*\\\\][(](.*)[)].*, lang=([^ ]*)\<gtr\>", line)) {
3209 <item> \ \ \ next_chunk_name=line[2];
3211 <item> \ \ \ get_texmacs_chunk_args(line[3], next_chunk_params);
3213 <item> \ \ \ gsub(ARG_SEPARATOR ",? ?", ";", line[3]);
3215 <item> \ \ \ params = "params=" line[3];
3217 <item> \ \ \ if ((line[4])) {
3219 <item> \ \ \ \ \ params = params ",language=" line[4]
3223 <item> \ \ \ get_tex_chunk_args(params, next_chunk_opts);
3225 <item> \ \ \ new_chunk(next_chunk_name, next_chunk_opts,
3228 <item> \ \ \ texmacs_chunking = 1;
3232 <item> \ \ \ # warning(sprintf("Unexpected chunk match: %s\\n", $_))
3241 <subsection|lstlistings>
3243 Our current scheme is to recognize the new lstlisting chunks, but these may
3244 be preceded by a <verbatim|\\Chunk> command which in <LyX> is a more
3245 convenient way to pass the chunk name to the
3246 <verbatim|\\begin{lstlistings}> command, and a more visible way to specify
3247 other <verbatim|lstset> settings.
3249 The arguments to the <verbatim|\\Chunk> command are a name, and then a
3250 comma-seperated list of key-value pairs after the manner of
3251 <verbatim|\\lstset>. (In fact within the <LaTeX> <verbatim|\\Chunk> macro
3252 (section <reference|sub:The-chunk-command>) the text <verbatim|name=> is
3253 prefixed to the argument which is then literally passed to
3254 <verbatim|\\lstset>).
3256 <\nf-chunk|recognize-chunk>
3257 <item>/^\\\\Chunk{/ {
3259 <item> \ if (match($0, "^\\\\\\\\Chunk{ *([^ ,}]*),?(.*)}", line)) {
3261 <item> \ \ \ next_chunk_name = line[1];
3263 <item> \ \ \ get_tex_chunk_args(line[2], next_chunk_opts);
3272 We also make a basic attempt to parse the name out of the
3273 <verbatim|\\lstlistings[name=chunk-name]> text, otherwise we fall back to
3274 the name found in the previous chunk command. This attempt is very basic
3275 and doesn't support commas or spaces or square brackets as part of the
3276 chunkname. We also recognize <verbatim|\\begin{Chunk}> which is convenient
3277 for some users<\footnote>
3278 but not yet supported in the <LaTeX> macros
3281 <\nf-chunk|recognize-chunk>
3282 <item>/^\\\\begin{lstlisting}\|^\\\\begin{Chunk}/ {
3284 <item> \ if (match($0, "}.*[[,] *name= *{? *([^], }]*)", line)) {
3286 <item> \ \ \ new_chunk(line[1]);
3290 <item> \ \ \ new_chunk(next_chunk_name, next_chunk_opts);
3294 <item> \ chunking=1;
3301 <section|Chunk Body>
3303 <subsection|<TeXmacs>>
3305 A chunk body in <TeXmacs> ends with <verbatim|\|________>... if it is the
3306 final chunklet of a chunk, or if there are further chunklets it ends with
3307 <verbatim|\|\\/\\/\\/>... which is a depiction of a jagged line of torn
3310 <\nf-chunk|recognize-chunk>
3311 <item>/^ *\\\|____________*/ && texmacs_chunking {
3313 <item> \ active_chunk="";
3315 <item> \ texmacs_chunking=0;
3317 <item> \ chunking=0;
3321 <item>/^ *\\\|\\/\\\\/ && texmacs_chunking {
3323 <item> \ texmacs_chunking=0;
3325 <item> \ chunking=0;
3327 <item> \ active_chunk="";
3332 It has been observed that not every line of output when a <TeXmacs> chunk
3333 is active is a line of chunk. This may no longer be true, but we set a
3334 variable <verbatim|texmacs_chunk> if the current line is a chunk line.
3336 Initially we set this to zero...
3338 <\nf-chunk|recognize-chunk>
3339 <item>texmacs_chunk=0;
3342 ...and then we look to see if the current line is a chunk line.
3344 <TeXmacs> lines look like this: <verbatim| \ 3 \| main() {> so we detect
3345 the lines by leading white space, digits, more whiter space and a vertical
3346 bar followed by at least once space.
3348 If we find such a line, we remove this line-header and set
3349 <verbatim|texmacs_chunk=1> as well as <verbatim|chunking=1>
3351 <\nf-chunk|recognize-chunk>
3352 <item>/^ *[1-9][0-9]* *\\\| / {
3354 <item> \ if (texmacs_chunking) {
3356 <item> \ \ \ chunking=1;
3358 <item> \ \ \ texmacs_chunk=1;
3360 <item> \ \ \ gsub("^ *[1-9][0-9]* *\\\\\| ", "")
3367 When <TeXmacs> chunking, lines that commence with <verbatim|\\/> or
3368 <verbatim|__> are not chunk content but visual framing, and are skipped.
3370 <\nf-chunk|recognize-chunk>
3371 <item>/^ *\\.\\/\\\\/ && texmacs_chunking {
3377 <item>/^ *__*$/ && texmacs_chunking {
3384 Any other line when <TeXmacs> chunking is considered to be a line-wrapped
3387 <\nf-chunk|recognize-chunk>
3388 <item>texmacs_chunking {
3390 <item> \ if (! texmacs_chunk) {
3392 <item> \ \ \ # must be a texmacs continued line
3394 <item> \ \ \ chunking=1;
3396 <item> \ \ \ texmacs_chunk=1;
3403 This final chunklet seems bogus and probably stops <LyX> working.
3405 <\nf-chunk|recognize-chunk>
3406 <item>! texmacs_chunk {
3408 <item># \ texmacs_chunking=0;
3410 <item> \ chunking=0;
3417 We recognize notangle style chunks too:
3419 <\nf-chunk|recognize-chunk>
3420 <item>/^[\<less\>]\<less\>.*[\<gtr\>]\<gtr\>=/ {
3422 <item> \ if (match($0, "^[\<less\>]\<less\>(.*)[\<gtr\>]\<gtr\>= *$",
3425 <item> \ \ \ chunking=1;
3427 <item> \ \ \ notangle_mode=1;
3429 <item> \ \ \ new_chunk(line[1]);
3440 Likewise, we need to recognize when a chunk ends.
3442 <subsection|lstlistings>
3444 The <verbatim|e> in <verbatim|[e]nd{lislisting}> is surrounded by square
3445 brackets so that when this document is processed, this chunk doesn't
3446 terminate early when the lstlistings package recognizes it's own
3447 end-string!<\footnote>
3448 This doesn't make sense as the regex is anchored with ^, which this line
3449 does not begin with!
3452 <\nf-chunk|recognize-chunk>
3453 <item>/^\\\\[e]nd{lstlisting}\|^\\\\[e]nd{Chunk}/ {
3455 <item> \ chunking=0;
3457 <item> \ active_chunk="";
3466 <\nf-chunk|recognize-chunk>
3469 <item> \ chunking=0;
3471 <item> \ active_chunk="";
3476 All other recognizers are only of effect if we are chunking; there's no
3477 point in looking at lines if they aren't part of a chunk, so we just ignore
3478 them as efficiently as we can.
3480 <\nf-chunk|recognize-chunk>
3481 <item>! chunking { next; }
3484 <section|Chunk contents>
3486 Chunk contents are any lines read while <verbatim|chunking> is true. Some
3487 chunk contents are special in that they refer to other chunks, and will be
3488 replaced by the contents of these chunks when the file is generated.
3490 <label|sub:ORS-chunk-text>We add the output record separator <verbatim|ORS>
3491 to the line now, because we will set <verbatim|ORS> to the empty string
3492 when we generate the output<\footnote>
3493 So that we can partial print lines using <verbatim|print> instead of
3494 <verbatim|printf>. <todo|This does't make sense>
3497 <\nf-chunk|recognize-chunk>
3498 <item>length(active_chunk) {
3500 <item> \ <nf-ref|process-chunk-tabs|>
3502 <item> \ <nf-ref|process-chunk|>
3507 If a chunk just consisted of plain text, we could handle the chunk like
3510 <\nf-chunk|process-chunk-simple>
3511 <item>chunk_line(active_chunk, $0 ORS);
3514 but in fact a chunk can include references to other chunks. Chunk includes
3515 are traditionally written as <verbatim|\<less\>\<less\>chunk-name\<gtr\>\<gtr\>>
3516 but we support other variations, some of which are more suitable for
3517 particular editing systems.
3519 However, we also process tabs at this point. A tab at input can be replaced
3520 by a number of spaces defined by the <verbatim|tabs> variable, set by the
3521 <verbatim|-T> option. Of course this is poor tab behaviour, we should
3522 probably have the option to use proper counted tab-stops and process this
3525 <\nf-chunk|process-chunk-tabs>
3526 <item>if (length(tabs)) {
3528 <item> \ gsub("\\t", tabs);
3533 <subsection|lstlistings><label|sub:lst-listings-includes>
3535 If <verbatim|\\lstset{escapeinside={=\<less\>}{\<gtr\>}}> is set, then we
3536 can use <verbatim|<nf-ref|chunk-name|>> in listings. The sequence
3537 <verbatim|=\<less\>> was chosen because:
3540 <item>it is a better mnemonic than <verbatim|\<less\>\<less\>chunk-name\<gtr\>\<gtr\>>
3541 in that the <verbatim|=> sign signifies equivalence or substitutability.
3543 <item>and because <verbatim|=\<less\>> is not valid in C or any language
3546 <item>and also because lstlistings doesn't like <verbatim|\<gtr\>\<gtr\>>
3547 as an end delimiter for the <em|texcl> escape, so we must make do with a
3548 single <verbatim|\<gtr\>> which is better complemented by
3549 <verbatim|=\<less\>> than by <verbatim|\<less\>\<less\>>.
3552 Unfortunately the <verbatim|=\<less\>...\<gtr\>> that we use re-enters a
3553 <LaTeX> parsing mode in which some characters are special, e.g. <verbatim|#
3554 \\> and so these cause trouble if used in arguments to
3555 <verbatim|\\chunkref>. At some point I must fix the <LaTeX> command
3556 <verbatim|\\chunkref> so that it can accept these literally, but until
3557 then, when writing chunkref argumemts that need these characters, I must
3558 use the forms <verbatim|\\textbackslash{}> and <verbatim|\\#>; so I also
3559 define a hacky chunk <verbatim|delatex> to be used further on whose purpose
3560 it is to remove these from any arguments parsed by fangle.
3565 <item>gsub("\\\\\\\\#", "#", ${text});
3567 <item>gsub("\\\\\\\\textbackslash{}", "\\\\", ${text});
3569 <item>gsub("\\\\\\\\\\\\^", "^", ${text});
3570 </nf-chunk||<tuple|text>>
3572 As each chunk line may contain more than one chunk include, we will split
3573 out chunk includes in an iterative fashion<\footnote>
3574 Contrary to our use of split when substituting parameters in chapter
3575 <reference|Here-we-split>
3578 First, as long as the chunk contains a <verbatim|\\chunkref> command we
3579 take as much as we can up to the first <verbatim|\\chunkref> command.
3581 <TeXmacs> text output uses <math|\<langle\>>...<math|\<rangle\>> which
3582 comes out as unicode sequences <verbatim|0xC2> <verbatim|0xAB> ...
3583 <verbatim|0xC2> <verbatim|0xBB>. Modern awk will interpret
3584 <verbatim|[^\\xC2\\xBB]> as a single unicode character if <verbatim|LANG>
3585 is set correctly to the sub-type <verbatim|UTF-8>, e.g.
3586 <verbatim|LANG=en_GB.UTF-8>, otherwise <verbatim|[^\\xC2\\xBB]> will be
3587 treated as a two character negated match <emdash> but this should not
3588 interfere with the function.
3590 <\nf-chunk|process-chunk>
3595 <item>while(match(chunk,"(\\xC2\\xAB)([^\\xC2\\xBB]*)
3596 [^\\xC2\\xBB]*\\xC2\\xBB", line) \|\|
3598 <item> \ \ \ \ \ match(chunk,\
3600 <item> \ \ \ \ \ \ \ \ \ \ \ "([=]\<less\>\\\\\\\\chunkref{([^}\<gtr\>]*)}(\\\\(.*\\\\)\|)\<gtr\>\|\<less\>\<less\>([a-zA-Z_][-a-zA-Z0-9_]*)\<gtr\>\<gtr\>)",\
3602 <item> \ \ \ \ \ \ \ \ \ \ \ line)\\
3606 <item> \ chunklet = substr(chunk, 1, RSTART - 1);
3609 We keep track of the indent count, by counting the number of literal
3610 characters found. We can then preserve this indent on each output line when
3611 multi-line chunks are expanded.
3613 We then process this first part literal text, and set the chunk which is
3614 still to be processed to be the text after the <verbatim|\\chunkref>
3615 command, which we will process next as we continue around the loop.
3617 <\nf-chunk|process-chunk>
3618 <item> \ indent += length(chunklet);
3620 <item> \ chunk_line(active_chunk, chunklet);
3622 <item> \ chunk = substr(chunk, RSTART + RLENGTH);
3625 We then consider the type of chunk command we have found, whether it is the
3626 fangle style command beginning with <verbatim|=\<less\>> the older notangle
3627 style beginning with <verbatim|\<less\>\<less\>>.
3629 Fangle chunks may have parameters contained within square brackets. These
3630 will be matched in <verbatim|line[3]> and are considered at this stage of
3631 processing to be part of the name of the chunk to be included.
3633 <\nf-chunk|process-chunk>
3634 <item> \ if (substr(line[1], 1, 1) == "=") {
3636 <item> \ \ \ # chunk name up to }
3638 <item> \ \ \ \ \ \ \ <nf-ref|delatex|<tuple|line[3]>>
3640 <item> \ \ \ chunk_include(active_chunk, line[2] line[3], indent);
3642 <item> \ } else if (substr(line[1], 1, 1) == "\<less\>") {
3644 <item> \ \ \ chunk_include(active_chunk, line[4], indent);
3646 <item> \ } else if (line[1] == "\\xC2\\xAB") {
3648 <item> \ \ \ chunk_include(active_chunk, line[2], indent);
3652 <item> \ \ \ error("Unknown chunk fragment: " line[1]);
3659 The loop will continue until there are no more chunkref statements in the
3660 text, at which point we process the final part of the chunk.
3662 <\nf-chunk|process-chunk>
3665 <item>chunk_line(active_chunk, chunk);
3668 <label|lone-newline>We add the newline character as a chunklet on it's own,
3669 to make it easier to detect new lines and thus manage indentation when
3670 processing the output.
3672 <\nf-chunk|process-chunk>
3673 <item>chunk_line(active_chunk, "\\n");
3678 We will also permit a chunk-part number to follow in square brackets, so
3679 that <verbatim|<nf-ref|chunk-name[1]|>> will refer to the first part only.
3680 This can make it easy to include a C function prototype in a header file,
3681 if the first part of the chunk is just the function prototype without the
3682 trailing semi-colon. The header file would include the prototype with the
3683 trailing semi-colon, like this:
3685 <verbatim|<nf-ref|chunk-name[1]|>>
3687 This is handled in section <reference|sub:Chunk-parts>
3689 We should perhaps introduce a notion of language specific chunk options; so
3690 that perhaps we could specify:
3692 <verbatim|=\<less\>\\chunkref{chunk-name[function-declaration]}>
3694 which applies a transform <verbatim|function-declaration> to the chunk ---
3695 which in this case would extract a function prototype from a function.
3698 <chapter|Processing Options>
3700 At the start, first we set the default options.
3702 <\nf-chunk|default-options>
3707 <item>notangle_mode=0;
3714 Then we use getopt the standard way, and null out ARGV afterwards in the
3717 <\nf-chunk|read-options>
3718 <item>Optind = 1 \ \ \ # skip ARGV[0]
3720 <item>while(getopt(ARGC, ARGV, "R:LdT:hr")!=-1) {
3722 <item> \ <nf-ref|handle-options|>
3726 <item>for (i=1; i\<less\>Optind; i++) { ARGV[i]=""; }
3729 This is how we handle our options:
3731 <\nf-chunk|handle-options>
3732 <item>if (Optopt == "R") root = Optarg;
3734 <item>else if (Optopt == "r") root="";
3736 <item>else if (Optopt == "L") linenos = 1;
3738 <item>else if (Optopt == "d") debug = 1;
3740 <item>else if (Optopt == "T") tabs = indent_string(Optarg+0);
3742 <item>else if (Optopt == "h") help();
3744 <item>else if (Optopt == "?") help();
3747 We do all of this at the beginning of the program
3752 <item> \ <nf-ref|constants|>
3754 <item> \ <nf-ref|mode-definitions|>
3756 <item> \ <nf-ref|default-options|>
3760 <item> \ <nf-ref|read-options|>
3765 And have a simple help function
3768 <item>function help() {
3770 <item> \ print "Usage:"
3772 <item> \ print " \ fangle [-L] -R\<less\>rootname\<gtr\> [source.tex
3775 <item> \ print " \ fangle -r [source.tex ...]"
3777 <item> \ print " \ If the filename, source.tex is not specified then
3782 <item> \ print "-L causes the C statement: #line \<less\>lineno\<gtr\>
3783 \\"filename\\"" to be issued"
3785 <item> \ print "-R causes the named root to be written to stdout"
3787 <item> \ print "-r lists all roots in the file (even those used
3795 <chapter|Generating the Output>
3797 We generate output by calling output_chunk, or listing the chunk names.
3799 <\nf-chunk|generate-output>
3800 <item>if (length(root)) output_chunk(root);
3802 <item>else output_chunk_names();
3805 We also have some other output debugging:
3807 <\nf-chunk|debug-output>
3810 <item> \ print "------ chunk names "
3812 <item> \ output_chunk_names();
3814 <item> \ print "====== chunks"
3816 <item> \ output_chunks();
3818 <item> \ print "++++++ debug"
3820 <item> \ for (a in chunks) {
3822 <item> \ \ \ print a "=" chunks[a];
3829 We do both of these at the end. We also set <verbatim|ORS=""> because each
3830 chunklet is not necessarily a complete line, and we already added
3831 <verbatim|ORS> to each input line in section
3832 <reference|sub:ORS-chunk-text>.
3837 <item> \ <nf-ref|debug-output|>
3841 <item> \ <nf-ref|generate-output|>
3846 We write chunk names like this. If we seem to be running in notangle
3847 compatibility mode, then we enclose the name like this
3848 <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>> the same way notangle does:
3850 <\nf-chunk|output_chunk_names()>
3851 <item>function output_chunk_names( \ \ c, prefix, suffix)\
3855 <item> \ if (notangle_mode) {
3857 <item> \ \ \ prefix="\<less\>\<less\>";
3859 <item> \ \ \ suffix="\<gtr\>\<gtr\>";
3863 <item> \ for (c in chunk_names) {
3865 <item> \ \ \ print prefix c suffix "\\n";
3872 This function would write out all chunks
3874 <\nf-chunk|output_chunks()>
3875 <item>function output_chunks( \ a)\
3879 <item> \ for (a in chunk_names) {
3881 <item> \ \ \ output_chunk(a);
3889 <item>function output_chunk(chunk) {
3891 <item> \ newline = 1;
3893 <item> \ lineno_needed = linenos;
3897 <item> \ write_chunk(chunk);
3904 <section|Assembling the Chunks>
3906 <verbatim|chunk_path> holds a string consisting of the names of all the
3907 chunks that resulted in this chunk being output. It should probably also
3908 contain the source line numbers at which each inclusion also occured.
3910 We first initialize the mode tracker for this chunk.
3912 <\nf-chunk|write_chunk()>
3913 <item>function write_chunk(chunk_name) {
3915 <item> \ <nf-ref|awk-delete-array|<tuple|context>>
3917 <item> \ return write_chunk_r(chunk_name, context);
3923 <item>function write_chunk_r(chunk_name, context, indent, tail,
3925 <item> \ # optional vars
3927 <item> \ <with|font-shape|italic|chunk_path>, chunk_args,\
3929 <item> \ # local vars
3931 <item> \ context_origin,
3933 <item> \ chunk_params, part, max_part, part_line, frag, max_frag, text,\
3935 <item> \ chunklet, only_part, call_chunk_args, new_context)
3939 <item> \ if (debug) debug_log("write_chunk_r(" chunk_name ")");
3942 <subsection|Chunk Parts><label|sub:Chunk-parts>
3944 As mentioned in section <reference|sub:lstlistings-includes>, a chunk name
3945 may contain a part specifier in square brackets, limiting the parts that
3948 <\nf-chunk|write_chunk()>
3949 <item> \ if (match(chunk_name, "^(.*)\\\\[([0-9]*)\\\\]$",
3950 chunk_name_parts)) {
3952 <item> \ \ \ chunk_name = chunk_name_parts[1];
3954 <item> \ \ \ only_part = chunk_name_parts[2];
3959 We then create a mode tracker
3961 <\nf-chunk|write_chunk()>
3962 <item> \ context_origin = push_mode_tracker(context, chunks[chunk_name,
3966 We extract into <verbatim|chunk_params> the names of the parameters that
3967 this chunk accepts, whose values were (optionally) passed in
3968 <verbatim|chunk_args>.
3970 <\nf-chunk|write_chunk()>
3971 <item> \ split(chunks[chunk_name, "params"], chunk_params, " *; *");
3974 To assemble a chunk, we write out each part.
3976 <\nf-chunk|write_chunk()>
3977 <item> \ if (! (chunk_name in chunk_names)) {
3979 <item> \ \ \ error(sprintf(_"The root module
3980 \<less\>\<less\>%s\<gtr\>\<gtr\> was not defined.\\nUsed by: %s",\\
3982 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ chunk_name, chunk_path));
3988 <item> \ max_part = chunks[chunk_name, "part"];
3990 <item> \ for(part = 1; part \<less\>= max_part; part++) {
3992 <item> \ \ \ if (! only_part \|\| part == only_part) {
3994 <item> \ \ \ \ \ <nf-ref|write-part|>
4000 <item> \ if (! pop_mode_tracker(context, context_origin)) {
4002 <item> \ \ \ dump_mode_tracker(context);
4004 <item> \ \ \ error(sprintf(_"Module %s did not close context
4005 properly.\\nUsed by: %s\\n", chunk_name, chunk_path));
4012 A part can either be a chunklet of lines, or an include of another chunk.
4014 Chunks may also have parameters, specified in LaTeX style with braces after
4015 the chunk name --- looking like this in the document: chunkname{param1,
4016 param2}. Arguments are passed in square brackets:
4017 <verbatim|\\chunkref{chunkname}[arg1, arg2]>.
4019 Before we process each part, we check that the source position hasn't
4020 changed unexpectedly, so that we can know if we need to output a new
4021 file-line directive.
4023 <\nf-chunk|write-part>
4024 <item><nf-ref|check-source-jump|>
4028 <item>chunklet = chunks[chunk_name, "part", part];
4030 <item>if (chunks[chunk_name, "part", part, "type"] == part_type_chunk) {
4032 <item> \ <nf-ref|write-included-chunk|>
4034 <item>} else if (chunklet SUBSEP "line" in chunks) {
4036 <item> \ <nf-ref|write-chunklets|>
4040 <item> \ # empty last chunklet
4045 To write an included chunk, we must detect any optional chunk arguments in
4046 parenthesis. Then we recurse calling <verbatim|write_chunk()>.
4048 <\nf-chunk|write-included-chunk>
4049 <item>if (match(chunklet, "^([^\\\\[\\\\(]*)\\\\((.*)\\\\)$",
4052 <item> \ chunklet = chunklet_parts[1];
4056 <item>gsub(sprintf("%c",11), "", chunklet);
4058 <item>gsub(sprintf("%c",11), "", chunklet_parts[2]);
4060 <item> \ parse_chunk_args("c-like", chunklet_parts[2], call_chunk_args,
4063 <item> \ for (c in call_chunk_args) {
4065 <item> \ \ \ call_chunk_args[c] = expand_chunk_args(call_chunk_args[c],
4066 chunk_params, chunk_args);
4072 <item> \ split("", call_chunk_args);
4078 <item>write_chunk_r(chunklet, context,
4080 <item> \ \ \ \ \ \ \ \ \ \ \ chunks[chunk_name, "part", part, "indent"]
4083 <item> \ \ \ \ \ \ \ \ \ \ \ chunks[chunk_name, "part", part, "tail"],
4085 <item> \ \ \ \ \ \ \ \ \ \ \ chunk_path "\\n \ \ \ \ \ \ \ \ "
4088 <item> \ \ \ \ \ \ \ \ \ \ \ call_chunk_args);
4091 Before we output a chunklet of lines, we first emit the file and line
4092 number if we have one, and if it is safe to do so.
4094 Chunklets are generally broken up by includes, so the start of a chunklet
4095 is a good place to do this. Then we output each line of the chunklet.
4097 When it is not safe, such as in the middle of a multi-line macro
4098 definition, <verbatim|lineno_suppressed> is set to true, and in such a case
4099 we note that we want to emit the line statement when it is next safe.
4101 <\nf-chunk|write-chunklets>
4102 <item>max_frag = chunks[chunklet, "line"];
4104 <item>for(frag = 1; frag \<less\>= max_frag; frag++) {
4106 <item> \ <nf-ref|write-file-line|>
4109 We then extract the chunklet text and expand any arguments.
4111 <\nf-chunk|write-chunklets>
4114 <item> \ text = chunks[chunklet, frag];
4118 <item> \ /* check params */
4120 <item> \ text = expand_chunk_args(text, chunk_params, chunk_args);
4123 If the text is a single newline (which we keep separate - see
4124 <reference|lone-newline>) then we increment the line number. In the case
4125 where this is the last line of a chunk and it is not a top-level chunk we
4126 replace the newline with an empty string --- because the chunk that
4127 included this chunk will have the newline at the end of the line that
4128 included this chunk.
4130 We also note by <verbatim|newline = 1> that we have started a new line, so
4131 that indentation can be managed with the following piece of text.
4133 <\nf-chunk|write-chunklets>
4136 <item> if (text == "\\n") {
4138 <item> \ \ \ lineno++;
4140 <item> \ \ \ if (part == max_part && frag == max_frag &&
4141 length(chunk_path)) {
4143 <item> \ \ \ \ \ text = "";
4145 <item> \ \ \ \ \ break;
4147 <item> \ \ \ } else {
4149 <item> \ \ \ \ \ newline = 1;
4154 If this text does not represent a newline, but we see that we are the first
4155 piece of text on a newline, then we prefix our text with the current
4159 <verbatim|newline> is a global output-state variable, but the
4160 <verbatim|indent> is not.
4163 <\nf-chunk|write-chunklets>
4164 <item> \ } else if (length(text) \|\| length(tail)) {
4166 <item> \ \ \ if (newline) text = indent text;
4168 <item> \ \ \ newline = 0;
4175 Tail will soon no longer be relevant once mode-detection is in place.
4177 <\nf-chunk|write-chunklets>
4178 <item> \ text = text tail;
4180 <item> \ mode_tracker(context, text);
4182 <item> \ print untab(transform_escape(context, text, context_origin));
4185 If a line ends in a backslash --- suggesting continuation --- then we
4186 supress outputting file-line as it would probably break the continued
4189 <\nf-chunk|write-chunklets>
4190 <item> \ if (linenos) {
4192 <item> \ \ \ lineno_suppressed = substr(lastline, length(lastline)) ==
4200 Of course there is no point in actually outputting the source filename and
4201 line number (file-line) if they don't say anything new! We only need to
4202 emit them if they aren't what is expected, or if we we not able to emit one
4203 when they had changed.
4205 <\nf-chunk|write-file-line>
4206 <item>if (newline && lineno_needed && ! lineno_suppressed) {
4208 <item> \ filename = a_filename;
4210 <item> \ lineno = a_lineno;
4212 <item> \ print "#line " lineno " \\"" filename "\\"\\n"
4214 <item> \ lineno_needed = 0;
4219 We check if a new file-line is needed by checking if the source line
4220 matches what we (or a compiler) would expect.
4222 <\nf-chunk|check-source-jump>
4223 <item>if (linenos && (chunk_name SUBSEP "part" SUBSEP part SUBSEP
4224 "FILENAME" in chunks)) {
4226 <item> \ a_filename = chunks[chunk_name, "part", part, "FILENAME"];
4228 <item> \ a_lineno = chunks[chunk_name, "part", part, "LINENO"];
4230 <item> \ if (a_filename != filename \|\| a_lineno != lineno) {
4232 <item> \ \ \ lineno_needed++;
4239 <chapter|Storing Chunks>
4241 Awk has pretty limited data structures, so we will use two main hashes.
4242 Uninterrupted sequences of a chunk will be stored in chunklets and the
4243 chunklets used in a chunk will be stored in <verbatim|chunks>.
4245 <\nf-chunk|constants>
4246 <item>part_type_chunk=1;
4251 The params mentioned are not chunk parameters for parameterized chunks, as
4252 mentioned in <reference|Chunk Arguments>, but the lstlistings style
4253 parameters used in the <verbatim|\\Chunk> command<\footnote>
4254 The <verbatim|params> parameter is used to hold the parameters for
4255 parameterized chunks
4258 <\nf-chunk|chunk-storage-functions>
4259 <item>function new_chunk(chunk_name, opts, args,
4261 <item> \ # local vars
4263 <item> \ p, append )
4267 <item> \ # HACK WHILE WE CHANGE TO ( ) for PARAM CHUNKS
4269 <item> \ gsub("\\\\(\\\\)$", "", chunk_name);
4271 <item> \ if (! (chunk_name in chunk_names)) {
4273 <item> \ \ \ if (debug) print "New chunk " chunk_name;
4275 <item> \ \ \ chunk_names[chunk_name];
4277 <item> \ \ \ for (p in opts) {
4279 <item> \ \ \ \ \ chunks[chunk_name, p] = opts[p];
4281 <item> \ \ \ \ \ if (debug) print "chunks[" chunk_name "," p "] = "
4286 <item> \ \ \ for (p in args) {
4288 <item> \ \ \ \ \ chunks[chunk_name, "params", p] = args[p];
4292 <item> \ \ \ if ("append" in opts) {
4294 <item> \ \ \ \ \ append=opts["append"];
4296 <item> \ \ \ \ \ if (! (append in chunk_names)) {
4298 <item> \ \ \ \ \ \ \ warning("Chunk " chunk_name " is appended to chunk "
4299 append " which is not defined yet");
4301 <item> \ \ \ \ \ \ \ new_chunk(append);
4305 <item> \ \ \ \ \ chunk_include(append, chunk_name);
4307 <item> \ \ \ \ \ chunk_line(append, ORS);
4313 <item> \ active_chunk = chunk_name;
4315 <item> \ prime_chunk(chunk_name);
4320 <\nf-chunk|chunk-storage-functions>
4323 <item>function prime_chunk(chunk_name)
4327 <item> \ chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] = \\
4329 <item> \ \ \ \ \ \ \ \ chunk_name SUBSEP "chunklet" SUBSEP ""
4330 ++chunks[chunk_name, "chunklet"];
4332 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"],
4333 "FILENAME"] = FILENAME;
4335 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "LINENO"]
4342 <item>function chunk_line(chunk_name, line){
4344 <item> \ chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
4346 <item> \ \ \ \ \ \ \ \ ++chunks[chunk_name, "chunklet",
4347 chunks[chunk_name, "chunklet"], "line"] \ ] = line;
4354 Chunk include represents a <em|chunkref> statement, and stores the
4355 requirement to include another chunk. The parameter indent represents the
4356 quanity of literal text characters that preceded this <em|chunkref>
4357 statement and therefore by how much additional lines of the included chunk
4360 <\nf-chunk|chunk-storage-functions>
4361 <item>function chunk_include(chunk_name, chunk_ref, indent, tail)
4365 <item> \ chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] =
4368 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "type" ]
4371 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "indent"
4372 ] = indent_string(indent);
4374 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "tail" ]
4377 <item> \ prime_chunk(chunk_name);
4384 The indent is calculated by indent_string, which may in future convert some
4385 spaces into tab characters. This function works by generating a printf
4386 padded format string, like <verbatim|%22s> for an indent of 22, and then
4387 printing an empty string using that format.
4389 <\nf-chunk|chunk-storage-functions>
4390 <item>function indent_string(indent) {
4392 <item> \ return sprintf("%" indent "s", "");
4397 <chapter|getopt><label|cha:getopt>
4399 I use Arnold Robbins public domain getopt (1993 revision). This is probably
4400 the same one that is covered in chapter 12 of “Edition 3 of GAWK:
4401 Effective AWK Programming: A User's Guide for GNU Awk” but as that is
4402 licensed under the GNU Free Documentation License, Version 1.3, which
4403 conflicts with the GPL3, I can't use it from there (or it's accompanying
4404 explanations), so I do my best to explain how it works here.
4406 The getopt.awk header is:
4408 <\nf-chunk|getopt.awk-header>
4409 <item># getopt.awk --- do C library getopt(3) function in awk
4413 <item># Arnold Robbins, arnold@skeeve.com, Public Domain
4417 <item># Initial version: March, 1991
4419 <item># Revised: May, 1993
4424 The provided explanation is:
4426 <\nf-chunk|getopt.awk-notes>
4427 <item># External variables:
4429 <item># \ \ \ Optind -- index in ARGV of first nonoption argument
4431 <item># \ \ \ Optarg -- string value of argument to current option
4433 <item># \ \ \ Opterr -- if nonzero, print our own diagnostic
4435 <item># \ \ \ Optopt -- current option letter
4441 <item># \ \ \ -1 \ \ \ \ at end of options
4443 <item># \ \ \ ? \ \ \ \ \ for unrecognized option
4445 <item># \ \ \ \<less\>c\<gtr\> \ \ \ a character representing the current
4450 <item># Private Data:
4452 <item># \ \ \ _opti \ -- index in multi-flag option, e.g., -abc
4457 The function follows. The final two parameters, <verbatim|thisopt> and
4458 <verbatim|i> are local variables and not parameters --- as indicated by the
4459 multiple spaces preceding them. Awk doesn't care, the multiple spaces are a
4460 convention to help us humans.
4462 <\nf-chunk|getopt.awk-getopt()>
4463 <item>function getopt(argc, argv, options, \ \ \ thisopt, i)
4467 <item> \ \ \ if (length(options) == 0) \ \ \ # no options given
4469 <item> \ \ \ \ \ \ \ return -1
4471 <item> \ \ \ if (argv[Optind] == "--") { \ # all done
4473 <item> \ \ \ \ \ \ \ Optind++
4475 <item> \ \ \ \ \ \ \ _opti = 0
4477 <item> \ \ \ \ \ \ \ return -1
4479 <item> \ \ \ } else if (argv[Optind] !~ /^-[^: \\t\\n\\f\\r\\v\\b]/) {
4481 <item> \ \ \ \ \ \ \ _opti = 0
4483 <item> \ \ \ \ \ \ \ return -1
4487 <item> \ \ \ if (_opti == 0)
4489 <item> \ \ \ \ \ \ \ _opti = 2
4491 <item> \ \ \ thisopt = substr(argv[Optind], _opti, 1)
4493 <item> \ \ \ Optopt = thisopt
4495 <item> \ \ \ i = index(options, thisopt)
4497 <item> \ \ \ if (i == 0) {
4499 <item> \ \ \ \ \ \ \ if (Opterr)
4501 <item> \ \ \ \ \ \ \ \ \ \ \ printf("%c -- invalid option\\n",
4503 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ thisopt)
4504 \<gtr\> "/dev/stderr"
4506 <item> \ \ \ \ \ \ \ if (_opti \<gtr\>= length(argv[Optind])) {
4508 <item> \ \ \ \ \ \ \ \ \ \ \ Optind++
4510 <item> \ \ \ \ \ \ \ \ \ \ \ _opti = 0
4512 <item> \ \ \ \ \ \ \ } else
4514 <item> \ \ \ \ \ \ \ \ \ \ \ _opti++
4516 <item> \ \ \ \ \ \ \ return "?"
4521 At this point, the option has been found and we need to know if it takes
4524 <\nf-chunk|getopt.awk-getopt()>
4525 <item> \ \ \ if (substr(options, i + 1, 1) == ":") {
4527 <item> \ \ \ \ \ \ \ # get option argument
4529 <item> \ \ \ \ \ \ \ if (length(substr(argv[Optind], _opti + 1)) \<gtr\>
4532 <item> \ \ \ \ \ \ \ \ \ \ \ Optarg = substr(argv[Optind], _opti + 1)
4534 <item> \ \ \ \ \ \ \ else
4536 <item> \ \ \ \ \ \ \ \ \ \ \ Optarg = argv[++Optind]
4538 <item> \ \ \ \ \ \ \ _opti = 0
4542 <item> \ \ \ \ \ \ \ Optarg = ""
4544 <item> \ \ \ if (_opti == 0 \|\| _opti \<gtr\>= length(argv[Optind])) {
4546 <item> \ \ \ \ \ \ \ Optind++
4548 <item> \ \ \ \ \ \ \ _opti = 0
4552 <item> \ \ \ \ \ \ \ _opti++
4554 <item> \ \ \ return thisopt
4559 A test program is built in, too
4561 <\nf-chunk|getopt.awk-begin>
4564 <item> \ \ \ Opterr = 1 \ \ \ # default is to diagnose
4566 <item> \ \ \ Optind = 1 \ \ \ # skip ARGV[0]
4568 <item> \ \ \ # test program
4570 <item> \ \ \ if (_getopt_test) {
4572 <item> \ \ \ \ \ \ \ while ((_go_c = getopt(ARGC, ARGV, "ab:cd")) != -1)
4574 <item> \ \ \ \ \ \ \ \ \ \ \ printf("c = \<less\>%c\<gtr\>, optarg =
4575 \<less\>%s\<gtr\>\\n",
4577 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ _go_c,
4580 <item> \ \ \ \ \ \ \ printf("non-option arguments:\\n")
4582 <item> \ \ \ \ \ \ \ for (; Optind \<less\> ARGC; Optind++)
4584 <item> \ \ \ \ \ \ \ \ \ \ \ printf("\\tARGV[%d] = \<less\>%s\<gtr\>\\n",
4586 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Optind,
4594 The entire getopt.awk is made out of these chunks in order
4596 <\nf-chunk|getopt.awk>
4597 <item><nf-ref|getopt.awk-header|>
4601 <item><nf-ref|getopt.awk-notes|>
4603 <item><nf-ref|getopt.awk-getopt()|>
4605 <item><nf-ref|getopt.awk-begin|>
4608 Although we only want the header and function:
4611 <item># try: locate getopt.awk for the full original file
4613 <item># as part of your standard awk installation
4615 <item><nf-ref|getopt.awk-header|>
4619 <item><nf-ref|getopt.awk-getopt()|>
4622 <chapter|Fangle LaTeX source code><label|latex-source>
4624 <section|fangle module>
4626 Here we define a <LyX> <verbatim|.module> file that makes it convenient to
4627 use <LyX> for writing such literate programs.
4629 This file <verbatim|./fangle.module> can be installed in your personal
4630 <verbatim|.lyx/layouts> folder. You will need to Tools Reconfigure so that
4631 <LyX> notices it. It adds a new format Chunk, which should precede every
4632 listing and contain the chunk name.
4634 <\nf-chunk|./fangle.module>
4635 <item>#\\DeclareLyXModule{Fangle Literate Listings}
4637 <item>#DescriptionBegin
4639 <item># \ Fangle literate listings allow one to write
4641 <item># \ \ literate programs after the fashion of noweb, but without
4644 <item># \ \ to use noweave to generate the documentation. Instead the
4647 <item># \ \ package is extended in conjunction with the noweb package to
4650 <item># \ \ to code formating directly as latex.
4652 <item># \ The fangle awk script
4654 <item>#DescriptionEnd
4658 <item><nf-ref|gpl3-copyright.hashed|>
4668 <item><nf-ref|./fangle.sty|>
4674 <item><nf-ref|chunkstyle|>
4678 <item><nf-ref|chunkref|>
4679 </nf-chunk|lyx-module|>
4681 Because <LyX> modules are not yet a language supported by fangle or
4682 lstlistings, we resort to this fake awk chunk below in order to have each
4683 line of the GPL3 license commence with a #
4685 <\nf-chunk|gpl3-copyright.hashed>
4686 <item>#<nf-ref|gpl3-copyright|>
4691 <subsection|The Chunk style>
4693 The purpose of the <name|chunk> style is to make it easier for <LyX> users
4694 to provide the name to <verbatim|lstlistings>. Normally this requires
4695 right-clicking on the listing, choosing settings, advanced, and then typing
4696 <verbatim|name=chunk-name>. This has the further disadvantage that the name
4697 (and other options) are not generally visible during document editing.
4699 The chunk style is defined as a <LaTeX> command, so that all text on the
4700 same line is passed to the <verbatim|LaTeX> command <verbatim|Chunk>. This
4701 makes it easy to parse using <verbatim|fangle>, and easy to pass these
4702 options on to the listings package. The first word in a chunk section
4703 should be the chunk name, and will have <verbatim|name=> prepended to it.
4704 Any other words are accepted arguments to <verbatim|lstset>.
4706 We set PassThru to 1 because the user is actually entering raw latex.
4708 <\nf-chunk|chunkstyle>
4711 <item> \ LatexType \ \ \ \ \ \ \ \ \ \ \ \ Command
4713 <item> \ LatexName \ \ \ \ \ \ \ \ \ \ \ \ Chunk
4715 <item> \ Margin \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ First_Dynamic
4717 <item> \ LeftMargin \ \ \ \ \ \ \ \ \ \ \ Chunk:xxx
4719 <item> \ LabelSep \ \ \ \ \ \ \ \ \ \ \ \ \ xx
4721 <item> \ LabelType \ \ \ \ \ \ \ \ \ \ \ \ Static
4723 <item> \ LabelString \ \ \ \ \ \ \ \ \ \ "Chunk:"
4725 <item> \ Align \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Left
4727 <item> \ PassThru \ \ \ \ \ \ \ \ \ \ \ \ \ 1
4732 To make the label very visible we choose a larger font coloured red.
4734 <\nf-chunk|chunkstyle>
4737 <item> \ \ \ Family \ \ \ \ \ \ \ \ \ \ \ \ \ Sans
4739 <item> \ \ \ Size \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Large
4741 <item> \ \ \ Series \ \ \ \ \ \ \ \ \ \ \ \ \ Bold
4743 <item> \ \ \ Shape \ \ \ \ \ \ \ \ \ \ \ \ \ \ Italic
4745 <item> \ \ \ Color \ \ \ \ \ \ \ \ \ \ \ \ \ \ red
4752 <subsection|The chunkref style>
4754 We also define the Chunkref style which can be used to express cross
4755 references to chunks.
4757 <\nf-chunk|chunkref>
4758 <item>InsetLayout Chunkref
4760 <item> \ LyxType \ \ \ \ \ \ \ \ \ \ \ \ \ \ charstyle
4762 <item> \ LatexType \ \ \ \ \ \ \ \ \ \ \ \ Command
4764 <item> \ LatexName \ \ \ \ \ \ \ \ \ \ \ \ chunkref
4766 <item> \ PassThru \ \ \ \ \ \ \ \ \ \ \ \ \ 1
4768 <item> \ LabelFont \ \ \ \ \ \ \ \ \ \ \ \
4770 <item> \ \ \ Shape \ \ \ \ \ \ \ \ \ \ \ \ \ \ Italic
4772 <item> \ \ \ Color \ \ \ \ \ \ \ \ \ \ \ \ \ \ red
4779 <section|Latex Macros><label|sec:Latex-Macros>
4781 We require the listings, noweb and xargs packages. As noweb defines it's
4782 own <verbatim|\\code> environment, we re-define the one that <LyX> logical
4783 markup module expects here.
4785 <\nf-chunk|./fangle.sty>
4786 <item>\\usepackage{listings}%
4788 <item>\\usepackage{noweb}%
4790 <item>\\usepackage{xargs}%
4792 <item>\\renewcommand{\\code}[1]{\\texttt{#1}}%
4795 We also define a <verbatim|CChunk> macro, for use as:
4796 <verbatim|\\begin{CChunk}> which will need renaming to
4797 <verbatim|\\begin{Chunk}> when I can do this without clashing with
4800 <\nf-chunk|./fangle.sty>
4801 <item>\\lstnewenvironment{Chunk}{\\relax}{\\relax}%
4804 We also define a suitable <verbatim|\\lstset> of parameters that suit the
4805 literate programming style after the fashion of <name|noweave>.
4807 <\nf-chunk|./fangle.sty>
4808 <item>\\lstset{numbers=left, stepnumber=5, numbersep=5pt,
4810 <item> \ \ \ \ \ \ \ breaklines=false,basicstyle=\\ttfamily,
4812 <item> \ \ \ \ \ \ \ numberstyle=\\tiny, language=C}%
4815 We also define a notangle-like mechanism for escaping to <LaTeX> from the
4816 listing, and by which we can refer to other listings. We declare the
4817 <verbatim|=\<less\>...\<gtr\>> sequence to contain <LaTeX> code, and
4818 include another like this chunk: <verbatim|<nf-ref|chunkname|>>. However,
4819 because <verbatim|=\<less\>...\<gtr\>> is already defined to contain
4820 <LaTeX> code for this document --- this is a fangle document after all ---
4821 the code fragment below effectively contains the <LaTeX> code:
4822 <verbatim|}{>. To avoid problems with document generation, I had to declare
4823 an lstlistings property: <verbatim|escapeinside={}> for this listing only;
4824 which in <LyX> was done by right-clicking the listings inset, choosing
4825 settings-\<gtr\>advanced. Therefore <verbatim|=\<less\>> isn't interpreted
4826 literally here, in a listing when the escape sequence is already defined as
4827 shown... we need to somehow escape this representation...
4829 <\nf-chunk|./fangle.sty>
4830 <item>\\lstset{escapeinside={=\<less\>}{\<gtr\>}}%
4833 Although our macros will contain the <verbatim|@> symbol, they will be
4834 included in a <verbatim|\\makeatletter> section by <LyX>; however we keep
4835 the commented out <verbatim|\\makeatletter> as a reminder. The listings
4836 package likes to centre the titles, but noweb titles are specially
4837 formatted and must be left aligned. The simplest way to do this turned out
4838 to be by removing the definition of <verbatim|\\lst@maketitle>. This may
4839 interact badly if other listings want a regular title or caption. We
4840 remember the old maketitle in case we need it.
4842 <\nf-chunk|./fangle.sty>
4843 <item>%\\makeatletter
4845 <item>%somehow re-defining maketitle gives us a left-aligned title
4847 <item>%which is extactly what our specially formatted title needs!
4849 <item>\\global\\let\\fangle@lst@maketitle\\lst@maketitle%
4851 <item>\\global\\def\\lst@maketitle{}%
4854 <subsection|The chunk command><label|sub:The-chunk-command>
4856 Our chunk command accepts one argument, and calls <verbatim|\\ltset>.
4857 Although <verbatim|\\ltset> will note the name, this is erased when the
4858 next <verbatim|\\lstlisting> starts, so we make a note of this in
4859 <verbatim|\\lst@chunkname> and restore in in lstlistings Init hook.
4861 <\nf-chunk|./fangle.sty>
4862 <item>\\def\\Chunk#1{%
4864 <item> \ \\lstset{title={\\fanglecaption},name=#1}%
4866 <item> \ \\global\\edef\\lst@chunkname{\\lst@intname}%
4870 <item>\\def\\lst@chunkname{\\empty}%
4873 <subsubsection|Chunk parameters>
4875 Fangle permits parameterized chunks, and requires the paramters to be
4876 specified as listings options. The fangle script uses this, and although we
4877 don't do anything with these in the <LaTeX> code right now, we need to stop
4878 the listings package complaining.
4880 <\nf-chunk|./fangle.sty>
4881 <item>\\lst@Key{params}\\relax{\\def\\fangle@chunk@params{#1}}%
4884 As it is common to define a chunk which then needs appending to another
4885 chunk, and annoying to have to declare a single line chunk to manage the
4886 include, we support an append= option.
4888 <\nf-chunk|./fangle.sty>
4889 <item>\\lst@Key{append}\\relax{\\def\\fangle@chunk@append{#1}}%
4892 <subsection|The noweb styled caption>
4894 We define a public macro <verbatim|\\fanglecaption> which can be set as a
4895 regular title. By means of <verbatim|\\protect>, It expands to
4896 <verbatim|\\fangle@caption> at the appopriate time when the caption is
4899 <nf-chunk|./fangle.sty|\\def\\fanglecaption{\\protect\\fangle@caption}%||>
4902 22c <math|\<langle\>>some-chunk 19b<math|\<rangle\>><math|\<equiv\>>+
4903 \ \ <math|\<vartriangleleft\>>22b 24d<math|\<vartriangleright\>>
4907 In this example, the current chunk is 22c, and therefore the third chunk
4910 It's name is some-chunk.\
4912 The first chunk with this name (19b) occurs as the second chunk on page
4915 The previous chunk (22d) with the same name is the second chunk on page
4918 The next chunk (24d) is the fourth chunk on page 24.
4919 </big-figure|Noweb Heading<label|noweb heading>>
4921 The general noweb output format compactly identifies the current chunk, and
4922 references to the first chunk, and the previous and next chunks that have
4925 This means that we need to keep a counter for each chunk-name, that we use
4926 to count chunks of the same name.
4928 <subsection|The chunk counter>
4930 It would be natural to have a counter for each chunk name, but TeX would
4931 soon run out of counters<\footnote>
4932 ...soon did run out of counters and so I had to re-write the LaTeX macros
4933 to share a counter as described here.
4934 </footnote>, so we have one counter which we save at the end of a chunk and
4935 restore at the beginning of a chunk.
4937 <\nf-chunk|./fangle.sty>
4938 <item>\\newcounter{fangle@chunkcounter}%
4941 We construct the name of this variable to store the counter to be the text
4942 <verbatim|lst-chunk-> prefixed onto the chunks own name, and store it in
4943 <verbatim|\\chunkcount>.\
4945 We save the counter like this:
4947 <nf-chunk|save-counter|\\global\\expandafter\\edef\\csname
4948 \\chunkcount\\endcsname{\\arabic{fangle@chunkcounter}}%||>
4950 and restore the counter like this:
4952 <nf-chunk|restore-counter|\\setcounter{fangle@chunkcounter}{\\csname
4953 \\chunkcount\\endcsname}%||>
4955 If there does not already exist a variable whose name is stored in
4956 <verbatim|\\chunkcount>, then we know we are the first chunk with this
4957 name, and then define a counter.\
4959 Although chunks of the same name share a common counter, they must still be
4960 distinguished. We use is the internal name of the listing, suffixed by the
4961 counter value. So the first chunk might be <verbatim|something-1> and the
4962 second chunk be <verbatim|something-2>, etc.
4964 We also calculate the name of the previous chunk if we can (before we
4965 increment the chunk counter). If this is the first chunk of that name, then
4966 <verbatim|\\prevchunkname> is set to <verbatim|\\relax> which the noweb
4967 package will interpret as not existing.
4969 <\nf-chunk|./fangle.sty>
4970 <item>\\def\\fangle@caption{%
4972 <item> \ \\edef\\chunkcount{lst-chunk-\\lst@intname}%
4974 <item> \ \\@ifundefined{\\chunkcount}{%
4976 <item> \ \ \ \\expandafter\\gdef\\csname \\chunkcount\\endcsname{0}%
4978 <item> \ \ \ \\setcounter{fangle@chunkcounter}{\\csname
4979 \\chunkcount\\endcsname}%
4981 <item> \ \ \ \\let\\prevchunkname\\relax%
4985 <item> \ \ \ \\setcounter{fangle@chunkcounter}{\\csname
4986 \\chunkcount\\endcsname}%
4988 <item> \ \ \ \\edef\\prevchunkname{\\lst@intname-\\arabic{fangle@chunkcounter}}%
4993 After incrementing the chunk counter, we then define the name of this
4994 chunk, as well as the name of the first chunk.
4996 <\nf-chunk|./fangle.sty>
4997 <item> \ \\addtocounter{fangle@chunkcounter}{1}%
4999 <item> \ \\global\\expandafter\\edef\\csname
5000 \\chunkcount\\endcsname{\\arabic{fangle@chunkcounter}}%
5002 <item> \ \\edef\\chunkname{\\lst@intname-\\arabic{fangle@chunkcounter}}%
5004 <item> \ \\edef\\firstchunkname{\\lst@intname-1}%
5007 We now need to calculate the name of the next chunk. We do this by
5008 temporarily skipping the counter on by one; however there may not actually
5009 be another chunk with this name! We detect this by also defining a label
5010 for each chunk based on the chunkname. If there is a next chunkname then it
5011 will define a label with that name. As labels are persistent, we can at
5012 least tell the second time <LaTeX> is run. If we don't find such a defined
5013 label then we define <verbatim|\\nextchunkname> to <verbatim|\\relax>.
5015 <\nf-chunk|./fangle.sty>
5016 <item> \ \\addtocounter{fangle@chunkcounter}{1}%
5018 <item> \ \\edef\\nextchunkname{\\lst@intname-\\arabic{fangle@chunkcounter}}%
5020 <item> \ \\@ifundefined{r@label-\\nextchunkname}{\\let\\nextchunkname\\relax}{}%
5023 The noweb package requires that we define a <verbatim|\\sublabel> for every
5024 chunk, with a unique name, which is then used to print out it's navigation
5027 We also define a regular label for this chunk, as was mentioned above when
5028 we calculated <verbatim|\\nextchunkname>. This requires <LaTeX> to be run
5029 at least twice after new chunk sections are added --- but noweb requried
5032 <\nf-chunk|./fangle.sty>
5033 <item> \ \\sublabel{\\chunkname}%
5035 <item>% define this label for every chunk instance, so we
5037 <item>% can tell when we are the last chunk of this name
5039 <item> \ \\label{label-\\chunkname}%
5042 We also try and add the chunk to the list of listings, but I'm afraid we
5043 don't do very well. We want each chunk name listing once, with all of it's
5046 <\nf-chunk|./fangle.sty>
5047 <item> \ \\addcontentsline{lol}{lstlisting}{\\lst@name~[\\protect\\subpageref{\\chunkname}]}%
5050 We then call the noweb output macros in the same way that noweave generates
5051 them, except that we don't need to call <verbatim|\\nwstartdeflinemarkup>
5052 or <verbatim|\\nwenddeflinemarkup> <emdash> and if we do, it messes up the
5055 <\nf-chunk|./fangle.sty>
5056 <item> \ \\nwmargintag{%
5060 <item> \ \ \ \ \ \\nwtagstyle{}%
5062 <item> \ \ \ \ \ \\subpageref{\\chunkname}%
5072 <item> \ \ \ {\\lst@name}%
5076 <item> \ \ \ \ \ \\nwtagstyle{}\\/%
5078 <item> \ \ \ \ \ \\@ifundefined{fangle@chunk@params}{}{%
5080 <item> \ \ \ \ \ \ \ (\\fangle@chunk@params)%
5084 <item> \ \ \ \ \ [\\csname \\chunkcount\\endcsname]~%
5086 <item> \ \ \ \ \ \\subpageref{\\firstchunkname}%
5090 <item> \ \ \ \\@ifundefined{fangle@chunk@append}{}{%
5092 <item> \ \ \ \\ifx{}\\fangle@chunk@append{x}\\else%
5094 <item> \ \ \ \ \ \ \ ,~add~to~\\fangle@chunk@append%
5100 <item>\\global\\def\\fangle@chunk@append{}%
5102 <item>\\lstset{append=x}%
5108 <item> \ \\ifx\\relax\\prevchunkname\\endmoddef\\else\\plusendmoddef\\fi%
5110 <item>% \ \\nwstartdeflinemarkup%
5112 <item> \ \\nwprevnextdefs{\\prevchunkname}{\\nextchunkname}%
5114 <item>% \ \\nwenddeflinemarkup%
5119 Originally this was developed as a <verbatim|listings> aspect, in the Init
5120 hook, but it was found easier to affect the title without using a hook
5121 <emdash> <verbatim|\\lst@AddToHookExe{PreSet}> is still required to set the
5122 listings name to the name passed to the <verbatim|\\Chunk> command, though.
5124 <\nf-chunk|./fangle.sty>
5125 <item>%\\lst@BeginAspect{fangle}
5127 <item>%\\lst@Key{fangle}{true}[t]{\\lstKV@SetIf{#1}{true}}
5129 <item>\\lst@AddToHookExe{PreSet}{\\global\\let\\lst@intname\\lst@chunkname}
5131 <item>\\lst@AddToHook{Init}{}%\\fangle@caption}
5133 <item>%\\lst@EndAspect
5136 <subsection|Cross references>
5138 We define the <verbatim|\\chunkref> command which makes it easy to generate
5139 visual references to different code chunks, e.g.
5141 <block|<tformat|<table|<row|<cell|Macro>|<cell|Appearance>>|<row|<cell|<verbatim|\\chunkref{preamble}>>|<cell|>>|<row|<cell|<verbatim|\\chunkref[3]{preamble}>>|<cell|>>|<row|<cell|<verbatim|\\chunkref{preamble}[arg1,
5144 Chunkref can also be used within a code chunk to include another code
5145 chunk. The third optional parameter to chunkref is a comma sepatarated list
5146 of arguments, which will replace defined parameters in the chunkref.
5149 Darn it, if I have: <verbatim|=\<less\>\\chunkref{new-mode-tracker}[{chunks[chunk_name,
5150 "language"]},{mode}]\<gtr\>> the inner braces (inside [ ]) cause _ to
5151 signify subscript even though we have <verbatim|lst@ReplaceIn>
5154 <\nf-chunk|./fangle.sty>
5155 <item>\\def\\chunkref@args#1,{%
5157 <item> \ \\def\\arg{#1}%
5159 <item> \ \\lst@ReplaceIn\\arg\\lst@filenamerpl%
5163 <item> \ \\@ifnextchar){\\relax}{, \\chunkref@args}%
5167 <item>\\newcommand\\chunkref[2][0]{%
5169 <item> \ \\@ifnextchar({\\chunkref@i{#1}{#2}}{\\chunkref@i{#1}{#2}()}%
5173 <item>\\def\\chunkref@i#1#2(#3){%
5175 <item> \ \\def\\zero{0}%
5177 <item> \ \\def\\chunk{#2}%
5179 <item> \ \\def\\chunkno{#1}%
5181 <item> \ \\def\\chunkargs{#3}%
5183 <item> \ \\ifx\\chunkno\\zero%
5185 <item> \ \ \ \\def\\chunkname{#2-1}%
5189 <item> \ \ \ \\def\\chunkname{#2-\\chunkno}%
5193 <item> \ \\let\\lst@arg\\chunk%
5195 <item> \ \\lst@ReplaceIn\\chunk\\lst@filenamerpl%
5197 <item> \ \\LA{%\\moddef{%
5199 <item> \ \ \ {\\chunk}%
5203 <item> \ \ \ \ \ \\nwtagstyle{}\\/%
5205 <item> \ \ \ \ \ \\ifx\\chunkno\\zero%
5207 <item> \ \ \ \ \ \\else%
5209 <item> \ \ \ \ \ [\\chunkno]%
5211 <item> \ \ \ \ \ \\fi%
5213 <item> \ \ \ \ \ \\ifx\\chunkargs\\empty%
5215 <item> \ \ \ \ \ \\else%
5217 <item> \ \ \ \ \ \ \ (\\chunkref@args #3,)%
5219 <item> \ \ \ \ \ \\fi%
5221 <item> \ \ \ \ \ ~\\subpageref{\\chunkname}%
5227 <item> \ \\RA%\\endmoddef%
5232 <subsection|The end>
5234 <\nf-chunk|./fangle.sty>
5237 <item>%\\makeatother
5240 <chapter|Extracting fangle>
5242 <section|Extracting from Lyx>
5244 To extract from <LyX>, you will need to configure <LyX> as explained in
5245 section <reference|Configuring-the-build>.
5247 <label|lyx-build-script>And this lyx-build scrap will extract fangle for
5250 <\nf-chunk|lyx-build>
5257 <item><nf-ref|lyx-build-helper|>
5259 <item>cd $PROJECT_DIR \|\| exit 1
5263 <item>/usr/local/bin/fangle -R./fangle $TEX_SRC \<gtr\> ./fangle
5265 <item>/usr/local/bin/fangle -R./fangle.module $TEX_SRC \<gtr\>
5270 <item>export FANGLE=./fangle
5272 <item>export TMP=${TMP:-/tmp}
5274 <item><nf-ref|test:*|>
5277 With a lyx-build-helper
5279 <\nf-chunk|lyx-build-helper>
5280 <item>PROJECT_DIR="$LYX_r"
5282 <item>LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
5284 <item>TEX_DIR="$LYX_p"
5286 <item>TEX_SRC="$TEX_DIR/$LYX_i"
5288 <item>TXT_SRC="$TEX_SRC"
5291 <section|Extracting documentation>
5293 <\nf-chunk|./gen-www>
5294 <item>#python -m elyxer --css lyx.css $LYX_SRC \| \\
5296 <item># \ iconv -c -f utf-8 -t ISO-8859-1//TRANSLIT \| \\
5298 <item># \ sed 's/UTF-8"\\(.\\)\<gtr\>/ISO-8859-1"\\1\<gtr\>/' \<gtr\>
5299 www/docs/fangle.html
5303 <item>python -m elyxer --css lyx.css --iso885915 --html --destdirectory
5304 www/docs/fangle.e \\
5306 <item> \ \ \ \ \ \ fangle.lyx \<gtr\> www/docs/fangle.e/fangle.html
5310 <item>( mkdir -p www/docs/fangle && cd www/docs/fangle && \\
5312 <item> \ lyx -e latex ../../../fangle.lyx && \\
5314 <item> \ htlatex ../../../fangle.tex "xhtml,fn-in" && \\
5316 <item> \ sed -i -e 's/\<less\>!--l\\. [0-9][0-9]* *--\<gtr\>//g'
5323 <item>( mkdir -p www/docs/literate && cd www/docs/literate && \\
5325 <item> \ lyx -e latex ../../../literate.lyx && \\
5327 <item> \ htlatex ../../../literate.tex "xhtml,fn-in" && \\
5329 <item> \ sed -i -e 's/\<less\>!--l\\. [0-9][0-9]* *--\<gtr\>$//g'
5335 <section|Extracting from the command line>
5337 First you will need the tex output, then you can extract:
5339 <\nf-chunk|lyx-build-manual>
5340 <item>lyx -e latex fangle.lyx
5342 <item>fangle -R./fangle fangle.tex \<gtr\> ./fangle
5344 <item>fangle -R./fangle.module fangle.tex \<gtr\> ./fangle.module
5358 <item>export SRC="${SRC:-./fangle.tm}"
5360 <item>export FANGLE="${FANGLE:-./fangle}"
5362 <item>export TMP="${TMP:-/tmp}"
5364 <item>export TESTDIR="$TMP/$USER/fangle.tests"
5366 <item>export TXT_SRC="${TXT_SRC:-$TESTDIR/fangle.txt}"
5370 <item>mkdir -p "$TESTDIR"
5374 <item>tm -s -c "$SRC" "$TXT_SRC" -q
5378 <item><nf-ref|test:helpers|>
5382 <item> \ <nf-ref|test:run-tests|>
5388 <item># test current fangle
5390 <item>echo Testing current fangle
5396 <item># extract new fangle
5398 <item>echo testing new fangle
5400 <item>$FANGLE -R./fangle "$TXT_SRC" \<gtr\> "$TESTDIR/fangle"
5402 <item>export FANGLE="$TESTDIR/fangle"
5408 <item># Now check that it can extract a fangle that also passes the
5411 <item>echo testing if new fangle can generate itself
5413 <item>$FANGLE -R./fangle "$TXT_SRC" \<gtr\> "$TESTDIR/fangle.new"
5415 <item>passtest diff -bwu "$FANGLE" "$TESTDIR/fangle.new"
5417 <item>export FANGLE="$TESTDIR/fangle.new"
5422 <\nf-chunk|test:run-tests>
5425 <item>$FANGLE -Rpca-test.awk $TXT_SRC \| awk -f - \|\| exit 1
5427 <item><nf-ref|test:cromulence|>
5429 <item><nf-ref|test:escapes|>
5431 <item><nf-ref|test:test-chunk|<tuple|test:example-sh>>
5433 <item><nf-ref|test:test-chunk|<tuple|test:example-makefile>>
5435 <item><nf-ref|test:test-chunk|<tuple|test:q:1>>
5437 <item><nf-ref|test:test-chunk|<tuple|test:make:1>>
5439 <item><nf-ref|test:test-chunk|<tuple|test:make:2>>
5441 <item><nf-ref|test:chunk-params|>
5444 <\nf-chunk|test:helpers>
5449 <item> \ then echo "Passed $TEST"
5451 <item> \ else echo "Failed $TEST"
5453 <item> \ \ \ \ \ \ return 1
5465 <item> \ then echo "Passed $TEST"
5467 <item> \ else echo "Failed $TEST"
5469 <item> \ \ \ \ \ \ return 1
5476 This chunk will render a named chunk and compare it to another rendered
5479 <\nf-chunk|test:test-chunk>
5480 <item><nf-ref|test:test-chunk-result|<tuple|<nf-arg|chunk>|<nf-arg|chunk>.result>>
5481 </nf-chunk|sh|<tuple|chunk>>
5483 <\nf-chunk|test:test-chunk-result>
5484 <item>TEST="<nf-arg|result>" passtest diff -u --label "<nf-arg|chunk>"
5485 \<less\>( $FANGLE -R<nf-arg|chunk> $TXT_SRC ) \\
5487 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ --label
5488 "<nf-arg|result>" \<less\>( $FANGLE -R<nf-arg|result> $TXT_SRC )
5489 </nf-chunk|sh|<tuple|chunk|result>>
5491 <chapter|Chunk Parameters>
5495 <\nf-chunk|test:lyx:chunk-params:sub>
5496 <item>I see a ${THING},
5498 <item>a ${THING} of colour ${colour},\
5500 <item>and looking closer =\<less\>\\chunkref{test:lyx:chunk-params:sub:sub}(${colour})\<gtr\>
5501 </nf-chunk||<tuple|THING|colour>>
5503 <\nf-chunk|test:lyx:chunk-params:sub:sub>
5504 <item>a funny shade of ${colour}
5505 </nf-chunk||<tuple|colour>>
5507 <\nf-chunk|test:lyx:chunk-params:text>
5508 <item>What do you see? "=\<less\>\\chunkref{test:lyx:chunk-params:sub}(joe,
5514 Should generate output:
5516 <\nf-chunk|test:lyx:chunk-params:result>
5517 <item>What do you see? "I see a joe,
5519 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ a joe of colour red,\
5521 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ and looking closer a funny shade
5527 And this chunk will perform the test:
5529 <\nf-chunk|test:chunk-params>
5530 <item><nf-ref|test:test-chunk-result|<tuple|test:lyx:chunk-params:text|test:lyx:chunk-params:result>>
5536 <\nf-chunk|test:chunk-params:sub>
5537 <item>I see a <nf-arg|THING>,
5539 <item>a <nf-arg|THING> of colour <nf-arg|colour>,\
5541 <item>and looking closer <nf-ref|test:chunk-params:sub:sub|<tuple|<nf-arg|colour>>>
5542 </nf-chunk||<tuple|THING|colour>>
5544 <\nf-chunk|test:chunk-params:sub:sub>
5545 <item>a funny shade of <nf-arg|colour>
5546 </nf-chunk||<tuple|colour>>
5548 <\nf-chunk|test:chunk-params:text>
5549 <item>What do you see? "<nf-ref|test:chunk-params:sub|<tuple|joe|red>>"
5554 Should generate output:
5556 <\nf-chunk|test:chunk-params:result>
5557 <item>What do you see? "I see a joe,
5559 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ a joe of colour red,\
5561 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ and looking closer a funny shade
5567 And this chunk will perform the test:
5569 <\nf-chunk|test:chunk-params>
5570 <item><nf-ref|test:test-chunk-result|<tuple|test:chunk-params:text|test:chunk-params:result>>
5574 <chapter|Compile-log-lyx><label|Compile-log-lyx>
5576 <\nf-chunk|Chunk:./compile-log-lyx>
5579 <item># can't use gtkdialog -i, cos it uses the "source" command which
5580 ubuntu sh doesn't have
5586 <item> \ errors="/tmp/compile.log.$$"
5588 <item># \ if grep '^[^ ]*:\\( In \\\|[0-9][0-9]*: [^ ]*:\\)' \<gtr\>
5591 <item>if grep '^[^ ]*(\\([0-9][0-9]*\\)) *: *\\(error\\\|warning\\)'
5596 <item> \ \ \ sed -i -e 's/^[^ ]*[/\\\\]\\([^/\\\\]*\\)(\\([ 0-9][
5597 0-9]*\\)) *: */\\1:\\2\|\\2\|/' $errors
5599 <item> \ \ \ COMPILE_DIALOG='
5601 <item> \<less\>vbox\<gtr\>
5603 <item> \ \<less\>text\<gtr\>
5605 <item> \ \ \ \<less\>label\<gtr\>Compiler errors:\<less\>/label\<gtr\>
5607 <item> \ \<less\>/text\<gtr\>
5609 <item> \ \<less\>tree exported_column="0"\<gtr\>
5611 <item> \ \ \ \<less\>variable\<gtr\>LINE\<less\>/variable\<gtr\>
5613 <item> \ \ \ \<less\>height\<gtr\>400\<less\>/height\<gtr\>\<less\>width\<gtr\>800\<less\>/width\<gtr\>
5615 <item> \ \ \ \<less\>label\<gtr\>File \| Line \|
5616 Message\<less\>/label\<gtr\>
5618 <item> \ \ \ \<less\>action\<gtr\>'". $SELF ; "'lyxgoto
5619 $LINE\<less\>/action\<gtr\>
5621 <item> \ \ \ \<less\>input\<gtr\>'"cat $errors"'\<less\>/input\<gtr\>
5623 <item> \ \<less\>/tree\<gtr\>
5625 <item> \ \<less\>hbox\<gtr\>
5627 <item> \ \ \<less\>button\<gtr\>\<less\>label\<gtr\>Build\<less\>/label\<gtr\>
5629 <item> \ \ \ \ \<less\>action\<gtr\>lyxclient -c "LYXCMD:build-program"
5630 &\<less\>/action\<gtr\>
5632 <item> \ \ \<less\>/button\<gtr\>
5634 <item> \ \ \<less\>button ok\<gtr\>\<less\>/button\<gtr\>
5636 <item> \ \<less\>/hbox\<gtr\>
5638 <item> \<less\>/vbox\<gtr\>
5642 <item> \ \ \ export COMPILE_DIALOG
5644 <item> \ \ \ ( gtkdialog --program=COMPILE_DIALOG ; rm $errors ) &
5648 <item> \ \ \ rm $errors
5658 <item> \ file="${LINE%:*}"
5660 <item> \ line="${LINE##*:}"
5662 <item> \ extraline=`cat $file \| head -n $line \| tac \| sed
5663 '/^\\\\\\\\begin{lstlisting}/q' \| wc -l`
5665 <item> \ extraline=`expr $extraline - 1`
5667 <item> \ lyxclient -c "LYXCMD:command-sequence server-goto-file-row $file
5668 $line ; char-forward ; repeat $extraline paragraph-down ;
5669 paragraph-up-select"
5677 <item>if test -z "$COMPILE_DIALOG"
5679 <item>then main "$@"\
5689 <associate|info-flag|short>
5690 <associate|page-medium|paper>
5691 <associate|page-screen-height|982016tmpt>
5692 <associate|page-screen-margin|false>
5693 <associate|page-screen-width|1686528tmpt>
5694 <associate|page-show-hf|true>
5695 <associate|preamble|false>
5696 <associate|sfactor|5>