18 Fangle is a tool for fangled literate programming. Newfangled is defined as New and often needlessly novel by TheFreeDictionary.com.
19 In this case, fangled means yet another not-so-new1. but improved. ^1 method for literate programming.
20 Literate Programming has a long history starting with the great Donald Knuth himself, whose literate programming tools seem to make use of as many escape sequences for semantic markup as TeX (also by Donald Knuth).
21 Norman Ramsey wrote the Noweb set of tools (notangle, noweave and noroots) and helpfully reduced the amount of magic character sequences to pretty much just <<, >> and @, and in doing so brought the wonders of literate programming within my reach.
22 While using the L Y X editor for LaTeX editing I had various troubles with the noweb tools, some of which were my fault, some of which were noweb's fault and some of which were L Y X's fault.
23 Noweb generally brought literate programming to the masses through removing some of the complexity of the original literate programming, but this would be of no advantage to me if the L Y X / LaTeX combination brought more complications in their place.
24 Fangle was thus born (originally called Newfangle) as an awk replacement for notangle, adding some important features, like better integration with L Y X and LaTeX (and later TeXmacs), multiple output format conversions, and fixing notangle bugs like indentation when using -L for line numbers.
25 Significantly, fangle is just one program which replaces various programs in Noweb. Noweave is done away with and implemented directly as LaTeX macros, and noroots is implemented as a function of the untangler fangle.
26 Fangle is written in awk for portability reasons, awk being available for most platforms. A Python version2. hasn't anyone implemented awk in python yet? ^2 was considered for the benefit of L Y X but a scheme version for TeXmacs will probably materialise first; as TeXmacs macro capabilities help make edit-time and format-time rendering of fangle chunks simple enough for my weak brain.
27 As an extension to many literate-programming styles, Fangle permits code chunks to take parameters and thus operate somewhat like C pre-processor macros, or like C++ templates. Name parameters (or even local variables in the callers scope) are anticipated, as parameterized chunks — useful though they are — are hard to comprehend in the literate document.
29 Fangle is licensed under the GPL 3 (or later).
30 This doesn't mean that sources generated by fangle must be licensed under the GPL 3.
31 This doesn't mean that you can't use or distribute fangle with sources of an incompatible license, but it means you must make the source of fangle available too.
32 As fangle is currently written in awk, an interpreted language, this should not be too hard.
34 4a <gpl3-copyright[1](
\v), lang=text> ≡
35 ________________________________________________________________________
36 1 | fangle - fully featured notangle replacement in awk
38 3 | Copyright (C) 2009-2010 Sam Liddicott <sam@liddicott.com>
40 5 | This program is free software: you can redistribute it and/or modify
41 6 | it under the terms of the GNU General Public License as published by
42 7 | the Free Software Foundation, either version 3 of the License, or
43 8 | (at your option) any later version.
45 10 | This program is distributed in the hope that it will be useful,
46 11 | but WITHOUT ANY WARRANTY; without even the implied warranty of
47 12 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
48 13 | GNU General Public License for more details.
50 15 | You should have received a copy of the GNU General Public License
51 16 | along with this program. If not, see <http://www.gnu.org/licenses/>.
52 |________________________________________________________________________
59 1 Introduction to Literate Programming 11
62 2.2 Extracting roots 13
63 2.3 Formatting the document 13
64 3 Using Fangle with L^ A T_ E X 15
65 4 Using Fangle with L Y X 17
66 4.1 Installing the L Y X module 17
67 4.2 Obtaining a decent mono font 17
71 4.3 Formatting your Lyx document 18
72 4.3.1 Customising the listing appearance 18
73 4.3.2 Global customisations 18
74 4.4 Configuring the build script 19
76 5 Using Fangle with T_ E X_( M A CS) 21
77 6 Fangle with Makefiles 23
78 6.1 A word about makefiles formats 23
79 6.2 Extracting Sources 23
80 6.2.1 Converting from L Y X to L^ A T_ E X 24
81 6.2.2 Converting from T_ E X_( M A CS) 24
82 6.3 Extracting Program Source 25
83 6.4 Extracting Source Files 25
84 6.5 Extracting Documentation 27
85 6.5.1 Formatting T_ E X 28
86 6.5.1.1 Running pdflatex 28
87 6.5.2 Formatting T_ E X_( M A CS) 28
88 6.5.3 Building the Documentation as a Whole 28
90 6.7 Boot-strapping the extraction 29
91 6.8 Incorporating Makefile.inc into existing projects 30
94 7 Fangle awk source code 35
96 7.2 Catching errors 36
97 8 T_ E X_( M A CS) args 37
98 9 L^ A T_ E X and lstlistings 39
99 9.1 Additional lstlstings parameters 39
100 9.2 Parsing chunk arguments 41
101 9.3 Expanding parameters in the text 42
102 10 Language Modes & Quoting 45
103 10.1 Modes to keep code together 45
104 10.2 Modes affect included chunks 45
105 10.3 Modes operation 46
106 10.4 Quoting scenarios 47
107 10.4.1 Direct quoting 47
108 10.5 Language Mode Definitions 47
111 10.5.3 Parentheses, Braces and Brackets 50
112 10.5.4 Customizing Standard Modes 50
119 10.7 A non-recursive mode tracker 54
120 10.7.1 Constructor 54
123 10.7.3.1 One happy chunk 59
125 10.8 Escaping and Quoting 59
126 11 Recognizing Chunks 61
128 11.1.1 T_ E X_( M A CS) 61
129 11.1.2 lstlistings 62
131 11.2.1 T_ E X_( M A CS) 63
134 11.3.1 lstlistings 64
136 11.4 Chunk contents 65
137 11.4.1 lstlistings 66
138 12 Processing Options 69
139 13 Generating the Output 71
140 13.1 Assembling the Chunks 72
141 13.1.1 Chunk Parts 72
144 16 Fangle LaTeX source code 83
145 16.1 fangle module 83
146 16.1.1 The Chunk style 83
147 16.1.2 The chunkref style 84
149 16.2.1 The chunk command 85
150 16.2.1.1 Chunk parameters 86
151 16.2.2 The noweb styled caption 86
152 16.2.3 The chunk counter 86
153 16.2.4 Cross references 89
155 17 Extracting fangle 91
156 17.1 Extracting from Lyx 91
157 17.2 Extracting documentation 91
158 17.3 Extracting from the command line 92
161 19 Chunk Parameters 97
163 19.2 T_ E X_( M A CS) 97
164 20 Compile-log-lyx 99
166 Chapter 1Introduction to Literate Programming
167 Todo: Should really follow on from a part-0 explanation of what literate programming is.
168 Chapter 2Running Fangle
169 Fangle is a replacement for noweb, which consists of notangle, noroots and noweave.
170 Like notangle and noroots, fangle can read multiple named files, or from stdin.
172 The -r option causes fangle to behave like noroots.
173 fangle -r filename.tex
174 will print out the fangle roots of a tex file.
175 Unlike the noroots command, the printed roots are not enclosed in angle brackets e.g. <<name>>, unless at least one of the roots is defined using the notangle notation <<name>>=.
176 Also, unlike noroots, it prints out all roots --- not just those that are not used elsewhere. I find that a root not being used doesn't make it particularly top level — and so-called top level roots could also be included in another root as well.
177 My convention is that top level roots to be extracted begin with ./ and have the form of a filename.
178 Makefile.inc, discussed in 6, can automatically extract all such sources prefixed with ./
180 notangle's -R and -L options are supported.
181 If you are using L Y X or LaTeX, the standard way to extract a file would be:
182 fangle -R./Makefile.inc fangle.tex > ./Makefile.inc
183 If you are using TeXmacs, the standard way to extract a file would similarly be:
184 fangle -R./Makefile.inc fangle.txt > ./Makefile.inc
185 TeXmacs users would obtain the text file with a verbatim export from TeXmacs which can be done on the command line with texmacs -s -c fangle.tm fangle.txt -q
186 Unlike the noroots command, the -L option to generate C pre-preocessor #file style line-number directives,does not break indenting of the generated file..
187 Also, thanks to mode tracking (described in 10) the -L option does not interrupt (and break) multi-line C macros either.
188 This does mean that sometimes the compiler might calculate the source line wrongly when generating error messages in such cases, but there isn't any other way around if multi-line macros include other chunks.
189 Future releases will include a mapping file so that line/character references from the C compiler can be converted to the correct part of the source document.
190 2.3 Formatting the document
191 The noweave replacement built into the editing and formatting environment for TeXmacs, L Y X (which uses LaTeX), and even for raw LaTeX.
192 Use of fangle with TeXmacs, L Y X and LaTeX are explained the the next few chapters.
193 Chapter 3Using Fangle with LaTeX
194 Because the noweave replacement is impemented in LaTeX, there is no processing stage required before running the LaTeX command. Of course, LaTeX may need running two or more times, so that the code chunk references can be fully calculated.
195 The formatting is managed by a set of macros shown in 16, and can be included with:
196 \usepackage{fangle.sty}
197 Norman Ramsay's origial noweb.sty package is currently required as it is used for formatting the code chunk captions.
198 The listings.sty package is required, and is used for formatting the code chunks and syntax highlighting.
199 The xargs.sty package is also required, and makes writing LaTeX macro so much more pleasant.
200 To do: Add examples of use of Macros
202 Chapter 4Using Fangle with L Y X
203 L Y X uses the same LaTeX macros shown in 16 as part of a L Y X module file fangle.module, which automatically includes the macros in the document pre-amble provided that the fangle L Y X module is used in the document.
204 4.1 Installing the L Y X module
205 Copy fangle.module to your L Y X layouts directory, which for unix users will be ~/.lyx/layouts
206 In order to make the new literate styles availalble, you will need to reconfigure L Y X by clicking Tools->Reconfigure, and then re-start L Y X.
207 4.2 Obtaining a decent mono font
208 The syntax high-lighting features of lstlistings makes use of bold; however a mono-space tt font is used to typeset the listings. Obtaining a bold tt font can be impossibly difficult and amazingly easy. I spent many hours at it, following complicated instructions from those who had spend many hours over it, and was finally delivered the simple solution on the lyx mailing list.
210 The simple way was to add this to my preamble:
212 \renewcommand{\ttdefault}{txtt}
215 The next simplest way was to use ams poor-mans-bold, by adding this to the pre-amble:
217 %\renewcommand{\ttdefault}{txtt}
218 %somehow make \pmb be the command for bold, forgot how, sorry, above line not work
219 It works, but looks wretched on the dvi viewer.
221 The lstlistings documention suggests using Luximono.
222 Luximono was installed according to the instructions in Ubuntu Forums thread 11591811. http://ubuntuforums.org/showthread.php?t=1159181 ^1 with tips from miknight2. http://miknight.blogspot.com/2005/11/how-to-install-luxi-mono-font-in.html ^2 stating that sudo updmap --enable MixedMap ul9.map is required. It looks fine in PDF and PS view but still looks rotten in dvi view.
223 4.3 Formatting your Lyx document
224 It is not necessary to base your literate document on any of the original L Y X literate classes; so select a regular class for your document type.
225 Add the new module Fangle Literate Listings and also Logical Markup which is very useful.
226 In the drop-down style listbox you should notice a new style defined, called Chunk.
227 When you wish to insert a literate chunk, you enter it's plain name in the Chunk style, instead of the old noweb method that uses <<name>>= type tags. In the line (or paragraph) following the chunk name, you insert a listing with: Insert->Program Listing.
228 Inside the white listing box you can type (or paste using shift+ctrl+V) your listing. There is no need to use ctrl+enter at the end of lines as with some older L Y X literate techniques --- just press enter as normal.
229 4.3.1 Customising the listing appearance
230 The code is formatted using the lstlistings package. The chunk style doesn't just define the chunk name, but can also define any other chunk options supported by the lstlistings package \lstset command. In fact, what you type in the chunk style is raw latex. If you want to set the chunk language without having to right-click the listing, just add ,lanuage=C after the chunk name. (Currently the language will affect all subsequent listings, so you may need to specify ,language= quite a lot).
231 To do: so fix the bug
233 Of course you can do this by editing the listings box advanced properties by right-clicking on the listings box, but that takes longer, and you can't see at-a-glance what the advanced settings are while editing the document; also advanced settings apply only to that box --- the chunk settings apply through the rest of the document3. It ought to apply only to subsequent chunks of the same name. I'll fix that later ^3.
234 To do: So make sure they only apply to chunks of that name
236 4.3.2 Global customisations
237 As lstlistings is used to set the code chunks, it's \lstset command can be used in the pre-amble to set some document wide settings.
238 If your source has many words with long sequences of capital letters, then columns=fullflexible may be a good idea, or the capital letters will get crowded. (I think lstlistings ought to use a slightly smaller font for captial letters so that they still fit).
239 The font family \ttfamily looks more normal for code, but has no bold (an alternate typewriter font is used).
240 With \ttfamily, I must also specify columns=fullflexible or the wrong letter spacing is used.
241 In my LaTeX pre-amble I usually specialise my code format with:
243 19a <document-preamble[1](
\v), lang=tex> ≡
244 ________________________________________________________________________
246 2 | numbers=left, stepnumber=1, numbersep=5pt,
247 3 | breaklines=false,
248 4 | basicstyle=\footnotesize\ttfamily,
249 5 | numberstyle=\tiny,
251 7 | columns=fullflexible,
252 8 | numberfirstline=true
254 |________________________________________________________________________
258 4.4 Configuring the build script
259 You can invoke code extraction and building from the L Y X menu option Document->Build Program.
260 First, make sure you don't have a conversion defined for Lyx->Program
261 From the menu Tools->Preferences, add a conversion from Latex(Plain)->Program as:
262 set -x ; fangle -Rlyx-build $$i |
263 env LYX_b=$$b LYX_i=$$i LYX_o=$$o LYX_p=$$p LYX_r=$$r bash
264 (But don't cut-n-paste it from this document or you may be be pasting a multi-line string which will break your lyx preferences file).
265 I hope that one day, L Y X will set these into the environment when calling the build script.
266 You may also want to consider adding options to this conversion...
267 parselog=/usr/share/lyx/scripts/listerrors
268 ...but if you do you will lose your stderr4. There is some bash plumbing to get a copy of stderr but this footnote is too small ^4.
269 Now, a shell script chunk called lyx-build will be extracted and run whenever you choose the Document->Build Program menu item.
270 This document was originally managed using L Y X and lyx-build script for this document is shown here for historical reference.
271 lyx -e latex fangle.lyx && \
272 fangle fangle.lyx > ./autoboot
273 This looks simple enough, but as mentioned, fangle has to be had from somewhere before it can be extracted.
275 When the lyx-build chunk is executed, the current directory will be a temporary directory, and LYX_SOURCE will refer to the tex file in this temporary directory. This is unfortunate as our makefile wants to run from the project directory where the Lyx file is kept.
276 We can extract the project directory from $$r, and derive the probable Lyx filename from the noweb file that Lyx generated.
278 19b <lyx-build-helper[1](
\v), lang=sh> ≡ 91b⊳
279 ________________________________________________________________________
280 1 | PROJECT_DIR="$LYX_r"
281 2 | LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
283 4 | TEX_SRC="$TEX_DIR/$LYX_i"
284 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
285 And then we can define a lyx-build fragment similar to the autoboot fragment
287 20a <lyx-build[1](
\v), lang=sh> ≡ 91a⊳
288 ________________________________________________________________________
290 2 | «lyx-build-helper 19b»
291 3 | cd $PROJECT_DIR || exit 1
293 5 | #/usr/bin/fangle -filter ./notanglefix-filter \
294 6 | # -R./Makefile.inc "../../noweb-lyx/noweb-lyx3.lyx" \
295 7 | # | sed '/NOWEB_SOURCE=/s/=.*/=samba4-dfs.lyx/' \
296 8 | # > ./Makefile.inc
298 10 | #make -f ./Makefile.inc fangle_sources
299 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
301 Chapter 5Using Fangle with TeXmacs
302 To do: Write this chapter
304 Chapter 6Fangle with Makefiles
305 Here we describe a Makefile.inc that you can include in your own Makefiles, or glue as a recursive make to other projects.
306 Makefile.inc will cope with extracting all the other source files from this or any specified literate document and keeping them up to date.
307 It may also be included by a Makefile or Makefile.am defined in a literate document to automatically deal with the extraction of source files and documents during normal builds.
308 Thus, if Makefile.inc is included into a main project makefile it add rules for the source files, capable of extracting the source files from the literate document.
309 6.1 A word about makefiles formats
310 Whitespace formatting is very important in a Makefile. The first character of each action line must be a TAB.
311 target: pre-requisite
314 This requires that the literate programming environment have the ability to represent a TAB character in a way that fangle will generate an actual TAB character.
315 We also adopt a convention that code chunks whose names beginning with ./ should always be automatically extracted from the document. Code chunks whose names do not begin with ./ are for internal reference. Such chunks may be extracted directly, but will not be automatically extracted by this Makefile.
316 6.2 Extracting Sources
317 Our makefile has two parts; variables must be defined before the targets that use them.
318 As we progress through this chapter, explaining concepts, we will be adding lines to <Makefile.inc-vars 23b> and <Makefile.inc-targets 24c> which are included in <./Makefile.inc 23a> below.
320 23a <./Makefile.inc[1](
\v), lang=make> ≡
321 ________________________________________________________________________
322 1 | «Makefile.inc-vars 23b»
323 2 | «Makefile.inc-default-targets 28a»
324 3 | «Makefile.inc-targets 24c»
325 |________________________________________________________________________
328 We first define a placeholder for the tool fangle in case it cannot be found in the path.
330 23b <Makefile.inc-vars[1](
\v), lang=> ≡ 24a⊳
331 ________________________________________________________________________
333 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
334 We also define a placeholder for LITERATE_SOURCE to hold the name of this document. This will normally be passed on the command line.
336 24a <Makefile.inc-vars[2](
\v) ⇑23b, lang=> +≡ ⊲23b 24b▿
337 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
339 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
340 Fangle cannot process L Y X or TeXmacs documents directly, so the first stage is to convert these to more suitable text based formats1. L Y X and TeXmacs formats are text-based, but not suitable for fangle ^1.
341 6.2.1 Converting from L Y X to LaTeX
342 The first stage will always be to convert the L Y X file to a LaTeX file. Fangle must run on a TeX file because the L Y X command server-goto-file-line2. The Lyx command server-goto-file-line is used to position the Lyx cursor at the compiler errors. ^2 requries that the line number provided be a line of the TeX file and always maps this the line in the L Y X docment. We use server-goto-file-line when moving the cursor to error lines during compile failures.
343 The command lyx -e literate fangle.lyx will produce fangle.tex, a TeX file; so we define a make target to be the same as the L Y X file but with the .tex extension.
344 The EXTRA_DIST is for automake support so that the TeX files will automaticaly be distributed with the source, to help those who don't have L Y X installed.
346 24b <Makefile.inc-vars[3](
\v) ⇑23b, lang=> +≡ ▵24a 24d▿
347 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
348 3 | LYX_SOURCE=$(LITERATE_SOURCE) # but only the .lyx files
349 4 | TEX_SOURCE=$(LYX_SOURCE:.lyx=.tex)
350 5 | EXTRA_DIST+=$(TEX_SOURCE)
351 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
352 We then specify that the TeX source is to be generated from the L Y X source.
354 24c <Makefile.inc-targets[1](
\v), lang=> ≡ 25a⊳
355 ________________________________________________________________________
356 1 | .SUFFIXES: .tex .lyx
360 5 | ↦rm -f -- $(TEX_SOURCE)
362 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
363 6.2.2 Converting from TeXmacs
364 Fangle cannot process TeXmacs files directly3. but this is planned when TeXmacs uses xml as it's native format ^3, but must first convert them to text files.
365 The command texmacs -c fangle.tm fangle.txt -q will produce fangle.txt, a text file; so we define a make target to be the same as the TeXmacs file but with the .txt extension.
366 The EXTRA_DIST is for automake support so that the TeX files will automaticaly be distributed with the source, to help those who don't have L Y X installed.
368 24d <Makefile.inc-vars[4](
\v) ⇑23b, lang=> +≡ ▵24b 25b⊳
369 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
370 6 | TEXMACS_SOURCE=$(LITERATE_SOURCE) # but only the .tm files
371 7 | TXT_SOURCE=$(LITERATE_SOURCE:.tm=.txt)
372 8 | EXTRA_DIST+=$(TXT_SOURCE)
373 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
374 To do: Add loop around each $< so multiple targets can be specified
377 25a <Makefile.inc-targets[2](
\v) ⇑24c, lang=> +≡ ⊲24c 25d▿
378 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
379 7 | .SUFFIXES: .txt .tm
381 9 | ↦texmacs -s -c $< $@ -q
382 10 | .PHONEY: clean_txt
384 12 | ↦rm -f -- $(TXT_SOURCE)
385 13 | clean: clean_txt
386 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
387 6.3 Extracting Program Source
388 The program source is extracted using fangle, which is designed to operate on text or a LaTeX documents4. LaTeX documents are just slightly special text documents ^4.
390 25b <Makefile.inc-vars[5](
\v) ⇑23b, lang=> +≡ ⊲24d 25c▿
391 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
392 9 | FANGLE_SOURCE=$(TXT_SOURCE)
393 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
394 The literate document can result in any number of source files, but not all of these will be changed each time the document is updated. We certainly don't want to update the timestamps of these files and cause the whole source tree to be recompiled just because the literate explanation was revised. We use CPIF from the Noweb tools to avoid updating the file if the content has not changed, but should probably write our own.
395 However, if a source file is not updated, then the fangle file will always have a newer time-stamp and the makefile would always re-attempt to extact a newer source file which would be a waste of time.
396 Because of this, we use a stamp file which is always updated each time the sources are fully extracted from the LaTeX document. If the stamp file is newer than the document, then we can avoid an attempt to re-extract any of the sources. Because this stamp file is only updated when extraction is complete, it is safe for the user to interrupt the build-process mid-extraction.
397 We use echo rather than touch to update the stamp file beause the touch command does not work very well over an sshfs mount that I was using.
399 25c <Makefile.inc-vars[6](
\v) ⇑23b, lang=> +≡ ▵25b 26a⊳
400 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
401 10 | FANGLE_SOURCE_STAMP=$(FANGLE_SOURCE).stamp
402 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
404 25d <Makefile.inc-targets[3](
\v) ⇑24c, lang=> +≡ ▵25a 26b⊳
405 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
406 14 | $(FANGLE_SOURCE_STAMP): $(FANGLE_SOURCE) \
407 15 | ↦ $(FANGLE_SOURCES) ; \
408 16 | ↦echo -n > $(FANGLE_SOURCE_STAMP)
410 18 | ↦rm -f $(FANGLE_SOURCE_STAMP)
411 19 | clean: clean_stamp
412 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
413 6.4 Extracting Source Files
414 We compute FANGLE_SOURCES to hold the names of all the source files defined in the document. We compute this only once, by means of := in assignent. The sed deletes the any << and >> which may surround the roots names (for compatibility with Noweb's noroots command).
415 As we use chunk names beginning with ./ to denote top level fragments that should be extracted, we filter out all fragments that do not begin with ./
416 Note 1. FANGLE_PREFIX is set to ./ by default, but whatever it may be overridden to, the prefix is replaced by a literal ./ before extraction so that files will be extracted in the current directory whatever the prefix. This helps namespace or sub-project prefixes like documents: for chunks like documents:docbook/intro.xml
417 To do: This doesn't work though, because it loses the full name and doesn't know what to extact!
420 26a <Makefile.inc-vars[7](
\v) ⇑23b, lang=> +≡ ⊲25c 26e▿
421 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
422 11 | FANGLE_PREFIX:=\.\/
423 12 | FANGLE_SOURCES:=$(shell \
424 13 | $(FANGLE) -r $(FANGLE_SOURCE) |\
425 14 | sed -e 's/^[<][<]//;s/[>][>]$$//;/^$(FANGLE_PREFIX)/!d' \
426 15 | -e 's/^$(FANGLE_PREFIX)/\.\//' )
427 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
428 The target below, echo_fangle_sources is a helpful debugging target and shows the names of the files that would be extracted.
430 26b <Makefile.inc-targets[4](
\v) ⇑24c, lang=> +≡ ⊲25d 26c▿
431 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
432 20 | .PHONY: echo_fangle_sources
433 21 | echo_fangle_sources: ; @echo $(FANGLE_SOURCES)
434 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
435 We define a convenient target called fangle_sources so that make -f fangle_sources will re-extract the source if the literate document has been updated.
437 26c <Makefile.inc-targets[5](
\v) ⇑24c, lang=> +≡ ▵26b 26d▿
438 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
439 22 | .PHONY: fangle_sources
440 23 | fangle_sources: $(FANGLE_SOURCE_STAMP)
441 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
442 And also a convenient target to remove extracted sources.
444 26d <Makefile.inc-targets[6](
\v) ⇑24c, lang=> +≡ ▵26c 27e⊳
445 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
446 24 | .PHONY: clean_fangle_sources
447 25 | clean_fangle_sources: ; \
448 26 | rm -f -- $(FANGLE_SOURCE_STAMP) $(FANGLE_SOURCES)
449 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
450 We now look at the extraction of the source files.
451 This makefile macro if_extension takes 4 arguments: the filename $(1), some extensions to match $(2) and a shell command to return if the filename does match the exensions $(3), and a shell command to return if it does not match the extensions $(4).
453 26e <Makefile.inc-vars[8](
\v) ⇑23b, lang=> +≡ ▵26a 26f▿
454 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
455 16 | if_extension=$(if $(findstring $(suffix $(1)),$(2)),$(3),$(4))
456 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
457 For some source files like C files, we want to output the line number and filename of the original LaTeX document from which the source came5. I plan to replace this option with a separate mapping file so as not to pollute the generated source, and also to allow a code pretty-printing reformatter like indent be able to re-format the file and adjust for changes through comparing the character streams. ^5.
458 To make this easier we define the file extensions for which we want to do this.
460 26f <Makefile.inc-vars[9](
\v) ⇑23b, lang=> +≡ ▵26e 27a⊳
461 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
462 17 | C_EXTENSIONS=.c .h
463 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
464 We can then use the if_extensions macro to define a macro which expands out to the -L option if fangle is being invoked in a C source file, so that C compile errors will refer to the line number in the TeX document.
466 27a <Makefile.inc-vars[10](
\v) ⇑23b, lang=> +≡ ⊲26f 27b▿
467 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
469 19 | nf_line=-L -T$(TABS)
470 20 | fangle=$(FANGLE) $(call if_extension,$(2),$(C_EXTENSIONS),$(nf_line)) -R"$(2)" $(1)
471 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
472 We can use a similar trick to define an indent macro which takes just the filename as an argument and can return a pipeline stage calling the indent command. Indent can be turned off with make fangle_sources indent=
474 27b <Makefile.inc-vars[11](
\v) ⇑23b, lang=> +≡ ▵27a 27c▿
475 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
476 21 | indent_options=-npro -kr -i8 -ts8 -sob -l80 -ss -ncs
477 22 | indent=$(call if_extension,$(1),$(C_EXTENSIONS), | indent $(indent_options))
478 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
479 We now define the pattern for extracting a file. The files are written using noweb's cpif so that the file timestamp will not be touched if the contents haven't changed. This avoids the need to rebuild the entire project because of a typographical change in the documentation, or if none or a few C source files have changed.
481 27c <Makefile.inc-vars[12](
\v) ⇑23b, lang=> +≡ ▵27b 27d▿
482 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
483 23 | fangle_extract=@mkdir -p $(dir $(1)) && \
484 24 | $(call fangle,$(2),$(1)) > "$(1).tmp" && \
485 25 | cat "$(1).tmp" $(indent) | cpif "$(1)" \
486 26 | && rm -f -- "$(1).tmp" || \
487 27 | (echo error fangling $(1) from $(2) ; exit 1)
488 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
489 We define a target which will extract or update all sources. To do this we first defined a makefile template that can do this for any source file in the LaTeX document.
491 27d <Makefile.inc-vars[13](
\v) ⇑23b, lang=> +≡ ▵27c 28b⊳
492 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
493 28 | define FANGLE_template
495 30 | ↦$$(call fangle_extract,$(1),$(2))
496 31 | FANGLE_TARGETS+=$(1)
498 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
499 We then enumerate the discovered FANGLE_SOURCES to generate a makefile rule for each one using the makefile template we defined above.
501 27e <Makefile.inc-targets[7](
\v) ⇑24c, lang=> +≡ ⊲26d 27f▿
502 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
503 27 | $(foreach source,$(FANGLE_SOURCES),\
504 28 | $(eval $(call FANGLE_template,$(source),$(FANGLE_SOURCE))) \
506 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
507 These will all be built with FANGLE_SOURCE_STAMP.
508 We also remove the generated sources on a make distclean.
510 27f <Makefile.inc-targets[8](
\v) ⇑24c, lang=> +≡ ▵27e 28c⊳
511 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
512 30 | _distclean: clean_fangle_sources
513 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
514 6.5 Extracting Documentation
515 We then identify the intermediate stages of the documentation and their build and clean targets.
517 28a <Makefile.inc-default-targets[1](
\v), lang=> ≡
518 ________________________________________________________________________
519 1 | .PHONEY : clean_pdf
520 |________________________________________________________________________
524 6.5.1.1 Running pdflatex
525 We produce a pdf file from the tex file.
527 28b <Makefile.inc-vars[14](
\v) ⇑23b, lang=> +≡ ⊲27d 28d▿
528 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
529 33 | FANGLE_PDF+=$(TEX_SOURCE:.tex=.pdf)
530 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
531 We run pdflatex twice to be sure that the contents and aux files are up to date. We certainly are required to run pdflatex at least twice if these files do not exist.
533 28c <Makefile.inc-targets[9](
\v) ⇑24c, lang=> +≡ ⊲27f 28e▿
534 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
535 31 | .SUFFIXES: .tex .pdf
537 33 | ↦pdflatex $< && pdflatex $<
540 36 | ↦rm -f -- $(FANGLE_PDF) $(TEX_SOURCE:.tex=.toc) \
541 37 | ↦ $(TEX_SOURCE:.tex=.log) $(TEX_SOURCE:.tex=.aux)
542 38 | clean_pdf: clean_pdf_tex
543 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
544 6.5.2 Formatting TeXmacs
545 TeXmacs can produce a PDF file directly.
547 28d <Makefile.inc-vars[15](
\v) ⇑23b, lang=> +≡ ▵28b 28f▿
548 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
549 34 | FANGLE_PDF+=$(LITERATE_SOURCE:.tm=.pdf)
550 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
551 To do: Outputting the PDF may not be enough to update the links and page references. I think
552 we need to update twice, generate a pdf, update twice mode and generate a new PDF.
553 Basically the PDF export of TeXmacs is pretty rotten and doesn't work properly from the CLI
556 28e <Makefile.inc-targets[10](
\v) ⇑24c, lang=> +≡ ▵28c 29a⊳
557 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
558 39 | .SUFFIXES: .tm .pdf
560 41 | ↦texmacs -s -c $< $@ -q
562 43 | clean_pdf_texmacs:
563 44 | ↦rm -f -- $(FANGLE_PDF)
564 45 | clean_pdf: clean_pdf_texmacs
565 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
566 6.5.3 Building the Documentation as a Whole
567 Currently we only build pdf as a final format, but FANGLE_DOCS may later hold other output formats.
569 28f <Makefile.inc-vars[16](
\v) ⇑23b, lang=> +≡ ▵28d
570 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
571 35 | FANGLE_DOCS=$(FANGLE_PDF)
572 |________________________________________________________________________
575 We also define fangle_docs as a convenient phony target.
577 29a <Makefile.inc-targets[11](
\v) ⇑24c, lang=> +≡ ⊲28e 29b▿
578 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
579 46 | .PHONY: fangle_docs
580 47 | fangle_docs: $(FANGLE_DOCS)
581 48 | docs: fangle_docs
582 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
583 And define a convenient clean_fangle_docs which we add to the regular clean target
585 29b <Makefile.inc-targets[12](
\v) ⇑24c, lang=> +≡ ▵29a
586 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
587 49 | .PHONEY: clean_fangle_docs
588 50 | clean_fangle_docs: clean_tex clean_pdf
589 51 | clean: clean_fangle_docs
591 53 | distclean_fangle_docs: clean_tex clean_fangle_docs
592 54 | distclean: clean distclean_fangle_docs
593 |________________________________________________________________________
597 If Makefile.inc is included into Makefile, then extracted files can be updated with this command:
600 make -f Makefile.inc fangle_sources
601 6.7 Boot-strapping the extraction
602 As well as having the makefile extract or update the source files as part of it's operation, it also seems convenient to have the makefile re-extracted itself from this document.
603 It would also be convenient to have the code that extracts the makefile from this document to also be part of this document, however we have to start somewhere and this unfortunately requires us to type at least a few words by hand to start things off.
604 Therefore we will have a minimal root fragment, which, when extracted, can cope with extracting the rest of the source. This shell script fragment can do that. It's name is * — out of regard for Noweb, but when extracted might better be called autoupdate.
608 29c <*[1](
\v), lang=sh> ≡
609 ________________________________________________________________________
612 3 | MAKE_SRC="${1:-${NW_LYX:-../../noweb-lyx/noweb-lyx3.lyx}}"
613 4 | MAKE_SRC=‘dirname "$MAKE_SRC"‘/‘basename "$MAKE_SRC" .lyx‘
614 5 | NOWEB_SRC="${2:-${NOWEB_SRC:-$MAKE_SRC.lyx}}"
615 6 | lyx -e latex $MAKE_SRC
617 8 | fangle -R./Makefile.inc ${MAKE_SRC}.tex \
618 9 | | sed "/FANGLE_SOURCE=/s/^/#/;T;aNOWEB_SOURCE=$FANGLE_SRC" \
619 10 | | cpif ./Makefile.inc
621 12 | make -f ./Makefile.inc fangle_sources
622 |________________________________________________________________________
625 The general Makefile can be invoked with ./autoboot and can also be included into any automake file to automatically re-generate the source files.
626 The autoboot can be extracted with this command:
627 lyx -e latex fangle.lyx && \
628 fangle fangle.lyx > ./autoboot
629 This looks simple enough, but as mentioned, fangle has to be had from somewhere before it can be extracted.
630 On a unix system this will extract fangle.module and the fangle awk script, and run some basic tests.
631 To do: cross-ref to test chapter when it is a chapter all on its own
633 6.8 Incorporating Makefile.inc into existing projects
634 If you are writing a literate module of an existing non-literate program you may find it easier to use a slight recursive make instead of directly including Makefile.inc in the projects makefile.
635 This way there is less chance of definitions in Makefile.inc interfering with definitions in the main makefile, or with definitions in other Makefile.inc from other literate modules of the same project.
636 To do this we add some glue to the project makefile that invokes Makefile.inc in the right way. The glue works by adding a .PHONY target to call the recursive make, and adding this target as an additional pre-requisite to the existing targets.
637 Example Sub-module of existing system
638 In this example, we are building module.so as a literate module of a larger project.
639 We will show the sort glue that can be inserted into the projects Makefile — or more likely — a regular Makefile included in or invoked by the projects Makefile.
641 30a <makefile-glue[1](
\v), lang=> ≡ 30b▿
642 ________________________________________________________________________
643 1 | module_srcdir=modules/module
644 2 | MODULE_SOURCE=module.tm
645 3 | MODULE_STAMP=$(MODULE_SOURCE).stamp
646 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
647 The existing build system may already have a build target for module.o, but we just add another pre-requisite to that. In this case we use module.tm.stamp as a pre-requisite, the stamp file's modified time indicating when all sources were extracted6. If the projects build system does not know how to build the module from the extracted sources, then just add build actions here as normal. ^6.
649 30b <makefile-glue[2](
\v) ⇑30a, lang=make> +≡ ▵30a 30c▿
650 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
651 4 | $(module_srcdir)/module.o: $(module_srcdir)/$(MODULE_STAMP)
652 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
653 The target for this new pre-requisite will be generated by a recursive make using Makefile.inc which will make sure that the source is up to date, before it is built by the main projects makefile.
655 30c <makefile-glue[3](
\v) ⇑30a, lang=> +≡ ▵30b 31a⊳
656 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
657 5 | $(module_srcdir)/$(MODULE_STAMP): $(module_srcdir)/$(MODULE_SOURCE)
658 6 | ↦$(MAKE) -C $(module_srcdir) -f Makefile.inc fangle_sources LITERATE_SOURCE=$(MODULE_SOURCE)
659 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
660 We can do similar glue for the docs, clean and distclean targets. In this example the main prject was using a double colon for these targets, so we must use the same in our glue.
662 31a <makefile-glue[4](
\v) ⇑30a, lang=> +≡ ⊲30c
663 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
664 7 | docs:: docs_module
665 8 | .PHONY: docs_module
667 10 | ↦$(MAKE) -C $(module_srcdir) -f Makefile.inc docs LITERATE_SOURCE=$(MODULE_SOURCE)
669 12 | clean:: clean_module
670 13 | .PHONEY: clean_module
672 15 | ↦$(MAKE) -C $(module_srcdir) -f Makefile.inc clean LITERATE_SOURCE=$(MODULE_SOURCE)
674 17 | distclean:: distclean_module
675 18 | .PHONY: distclean_module
676 19 | distclean_module:
677 20 | ↦$(MAKE) -C $(module_srcdir) -f Makefile.inc distclean LITERATE_SOURCE=$(MODULE_SOURCE)
678 |________________________________________________________________________
681 We could do similarly for install targets to install the generated docs.
683 Chapter 7Fangle awk source code
684 We use the copyright notice from chapter 2.
686 35a <./fangle[1](
\v), lang=awk> ≡ 35b▿
687 ________________________________________________________________________
688 1 | #! /usr/bin/awk -f
689 2 | # «gpl3-copyright 4a»
690 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
691 We also use code from Arnold Robbins public domain getopt (1993 revision) defined in 81a, and naturally want to attribute this appropriately.
693 35b <./fangle[2](
\v) ⇑35a, lang=> +≡ ▵35a 35c▿
694 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
695 3 | # NOTE: Arnold Robbins public domain getopt for awk is also used:
696 4 | «getopt.awk-header 79a»
697 5 | «getopt.awk-getopt() 79c»
699 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
700 And include the following chunks (which are explained further on) to make up the program:
702 35c <./fangle[3](
\v) ⇑35a, lang=> +≡ ▵35b 40a⊳
703 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
704 7 | «helper-functions 36d»
705 8 | «mode-tracker 59a»
706 9 | «parse_chunk_args 42a»
707 10 | «chunk-storage-functions 77b»
708 11 | «output_chunk_names() 71d»
709 12 | «output_chunks() 71e»
710 13 | «write_chunk() 72a»
711 14 | «expand_chunk_args() 42b»
714 17 | «recognize-chunk 61a»
716 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
718 The portable way to erase an array in awk is to split the empty string, so we define a fangle macro that can split an array, like this:
720 35d <awk-delete-array[1](ARRAY
\v\v), lang=awk> ≡
721 ________________________________________________________________________
722 1 | split("", ${ARRAY});
723 |________________________________________________________________________
726 For debugging it is sometimes convenient to be able to dump the contents of an array to stderr, and so this macro is also useful.
728 35e <dump-array[1](ARRAY
\v\v), lang=awk> ≡
729 ________________________________________________________________________
730 1 | print "\nDump: ${ARRAY}\n--------\n" > "/dev/stderr";
731 2 | for (_x in ${ARRAY}) {
732 3 | print _x "=" ${ARRAY}[_x] "\n" > "/dev/stderr";
734 5 | print "========\n" > "/dev/stderr";
735 |________________________________________________________________________
739 Fatal errors are issued with the error function:
741 36a <error()[1](
\v), lang=awk> ≡ 36b▿
742 ________________________________________________________________________
743 1 | function error(message)
745 3 | print "ERROR: " FILENAME ":" FNR " " message > "/dev/stderr";
748 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
749 and likewise for non-fatal warnings:
751 36b <error()[2](
\v) ⇑36a, lang=awk> +≡ ▵36a 36c▿
752 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
753 6 | function warning(message)
755 8 | print "WARNING: " FILENAME ":" FNR " " message > "/dev/stderr";
758 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
759 and debug output too:
761 36c <error()[3](
\v) ⇑36a, lang=awk> +≡ ▵36b
762 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
763 11 | function debug_log(message)
765 13 | print "DEBUG: " FILENAME ":" FNR " " message > "/dev/stderr";
767 |________________________________________________________________________
770 To do: append=helper-functions
773 36d <helper-functions[1](
\v), lang=> ≡
774 ________________________________________________________________________
776 |________________________________________________________________________
779 Chapter 8TeXmacs args
780 TeXmacs functions with arguments1. or function declarations with parameters ^1 appear like this:
781 blah((I came, I saw, I conquered)<wide-overbrace>^(argument 1)(^K, )<wide-overbrace>^(sep.)(and then went home asd)<wide-overbrace>^(argument 3)(^K))<wide-overbrace>^(term.)_arguments
782 Arguments commence after the opening parenthesis. The first argument runs up till the next ^K.
783 If the following character is a , then another argument follows. If the next character after the , is a space character, then it is also eaten. The fangle stylesheet emits ^K,space as separators, but the fangle untangler will forgive a missing space.
784 If the following character is ) then this is a terminator and there are no more arguments.
786 37a <constants[1](
\v), lang=> ≡ 77a⊳
787 ________________________________________________________________________
788 1 | ARG_SEPARATOR=sprintf("%c", 11);
789 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
790 To process the text in this fashion, we split the string on ^K
793 37b <get_chunk_args[1](
\v), lang=> ≡
794 ________________________________________________________________________
795 1 | function get_texmacs_chunk_args(text, args, a, done) {
796 2 | split(text, args, ARG_SEPARATOR);
799 5 | for (a=1; (a in args); a++) if (a>1) {
800 6 | if (args[a] == "" || substr(args[a], 1, 1) == ")") done=1;
806 12 | if (substr(args[a], 1, 2) == ", ") args[a]=substr(args[a], 3);
807 13 | else if (substr(args[a], 1, 1) == ",") args[a]=substr(args[a], 2);
810 |________________________________________________________________________
813 Chapter 9LaTeX and lstlistings
814 To do: Split LyX and TeXmacs parts
816 For L Y X and LaTeX, the lstlistings package is used to format the lines of code chunks. You may recal from chapter XXX that arguments to a chunk definition are pure LaTeX code. This means that fangle needs to be able to parse LaTeX a little.
817 LaTeX arguments to lstlistings macros are a comma seperated list of key-value pairs, and values containing commas are enclosed in { braces } (which is to be expected for LaTeX).
818 A sample expressions is:
819 name=thomas, params={a, b}, something, something-else
820 but we see that this is just a simpler form of this expression:
821 name=freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
822 We may consider that we need a function that can parse such LaTeX expressions and assign the values to an AWK associated array, perhaps using a recursive parser into a multi-dimensional hash1. as AWK doesn't have nested-hash support ^1, resulting in:
827 a[foo, quux, a] fleeg
830 Yet, also, on reflection it seems that sometimes such nesting is not desirable, as the braces are also used to delimit values that contain commas --- we may consider that
831 name={williamson, freddie}
832 should assign williamson, freddie to name.
833 In fact we are not so interested in the detail so as to be bothered by this, which turns out to be a good thing for two reasons. Firstly TeX has a malleable parser with no strict syntax, and secondly whether or not williamson and freddie should count as two items will be context dependant anyway.
834 We need to parse this latex for only one reason; which is that we are extending lstlistings to add some additional arguments which will be used to express chunk parameters and other chunk options.
835 9.1 Additional lstlstings parameters
836 Further on we define a \Chunk LaTeX macro whose arguments will consist of a the chunk name, optionally followed by a comma and then a comma separated list of arguments. In fact we will just need to prefix name= to the arguments to in order to create valid lstlistings arguments.
837 There will be other arguments supported too;
838 params.As an extension to many literate-programming styles, fangle permits code chunks to take parameters and thus operate somewhat like C pre-processor macros, or like C++ templates. Chunk parameters are declared with a chunk argument called params, which holds a semi-colon separated list of parameters, like this:
839 achunk,language=C,params=name;address
840 addto.a named chunk that this chunk is to be included into. This saves the effort of having to declare another listing of the named chunk merely to include this one.
841 Function get_chunk_args() will accept two paramters, text being the text to parse, and values being an array to receive the parsed values as described above. The optional parameter path is used during recursion to build up the multi-dimensional array path.
843 40a <./fangle[4](
\v) ⇑35a, lang=> +≡ ⊲35c
844 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
845 19 | «get_chunk_args() 40b»
846 |________________________________________________________________________
850 40b <get_chunk_args()[1](
\v), lang=> ≡ 40c▿
851 ________________________________________________________________________
852 1 | function get_tex_chunk_args(text, values,
853 2 | # optional parameters
854 3 | path, # hierarchical precursors
857 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
858 The strategy is to parse the name, and then look for a value. If the value begins with a brace {, then we recurse and consume as much of the text as necessary, returning the remaining text when we encounter a leading close-brace }. This being the strategy --- and executed in a loop --- we realise that we must first look for the closing brace (perhaps preceded by white space) in order to terminate the recursion, and returning remaining text.
860 40c <get_chunk_args()[2](
\v) ⇑40b, lang=> +≡ ▵40b
861 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
863 7 | split("", values);
864 8 | while(length(text)) {
865 9 | if (match(text, "^ *}(.*)", a)) {
868 12 | «parse-chunk-args 40d»
872 |________________________________________________________________________
875 We can see that the text could be inspected with this regex:
877 40d <parse-chunk-args[1](
\v), lang=> ≡ 41a⊳
878 ________________________________________________________________________
879 1 | if (! match(text, " *([^,=]*[^,= ]) *(([,=]) *(([^,}]*) *,* *(.*))|)$", a)) {
882 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
883 and that a will have the following values:
886 2 =freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
888 4 freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
890 6 , foo={bar=baz, quux={quirk, a=fleeg}}, etc
892 a[3] will be either = or , and signify whether the option named in a[1] has a value or not (respectively).
893 If the option does have a value, then if the expression substr(a[4],1,1) returns a brace { it will signify that we need to recurse:
895 41a <parse-chunk-args[2](
\v) ⇑40d, lang=> +≡ ⊲40d
896 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
898 5 | if (a[3] == "=") {
899 6 | if (substr(a[4],1,1) == "{") {
900 7 | text = get_tex_chunk_args(substr(a[4],2), values, path name SUBSEP);
902 9 | values[path name]=a[5];
906 13 | values[path name]="";
909 |________________________________________________________________________
912 We can test this function like this:
914 41b <gca-test.awk[1](
\v), lang=> ≡
915 ________________________________________________________________________
916 1 | «get_chunk_args() 40b»
920 5 | print get_tex_chunk_args("name=freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc", a);
922 7 | print "a[" b "] => " a[b];
925 |________________________________________________________________________
928 which should give this output:
930 41c <gca-test.awk-results[1](
\v), lang=> ≡
931 ________________________________________________________________________
932 1 | a[foo.quux.quirk] =>
933 2 | a[foo.quux.a] => fleeg
934 3 | a[foo.bar] => baz
936 5 | a[name] => freddie
937 |________________________________________________________________________
940 9.2 Parsing chunk arguments
941 Arguments to paramterized chunks are expressed in round brackets as a comma separated list of optional arguments. For example, a chunk that is defined with:
942 \Chunk{achunk, params=name ; address}
944 \chunkref{achunk}(John Jones, jones@example.com)
945 An argument list may be as simple as in \chunkref{pull}(thing, otherthing) or as complex as:
946 \chunkref{pull}(things[x, y], get_other_things(a, "(all)"))
947 --- which for all it's commas and quotes and parenthesis represents only two parameters: things[x, y] and get_other_things(a, "(all)").
948 If we simply split parameter list on commas, then the comma in things[x,y] would split into two seperate arguments: things[x and y]--- neither of which make sense on their own.
949 One way to prevent this would be by refusing to split text between matching delimiters, such as [, ], (, ), {, } and most likely also ", " and ', '. Of course this also makes it impossible to pass such mis-matched code fragments as parameters, but I think that it would be hard for readers to cope with authors who would pass such code unbalanced fragments as chunk parameters2. I know that I couldn't cope with users doing such things, and although the GPL3 license prevents me from actually forbidding anyone from trying, if they want it to work they'll have to write the code themselves and not expect any support from me. ^2.
950 Unfortunately, the full set of matching delimiters may vary from language to language. In certain C++ template contexts, < and > would count as delimiters, and yet in other contexts they would not.
951 This puts me in the unfortunate position of having to parse-somewhat all programming languages without knowing what they are!
952 However, if this universal mode-tracking is possible, then parsing the arguments would be trivial. Such a mode tracker is described in chapter 10 and used here with simplicity.
954 42a <parse_chunk_args[1](
\v), lang=> ≡
955 ________________________________________________________________________
956 1 | function parse_chunk_args(language, text, values, mode,
958 3 | c, context, rest)
960 5 | «new-mode-tracker
\v(context
\v, language
\v, mode
\v) 55a»
961 6 | rest = mode_tracker(context, text, values);
963 8 | for(c=1; c <= context[0, "values"]; c++) {
964 9 | values[c] = context[0, "values", c];
968 |________________________________________________________________________
971 9.3 Expanding parameters in the text
972 Within the body of the chunk, the parameters are referred to with: ${name} and ${address}. There is a strong case that a LaTeX style notation should be used, like \param{name} which would be expressed in the listing as =<\param{name}> and be rendered as ${name}. Such notation would make me go blind, but I do intend to adopt it.
973 We therefore need a function expand_chunk_args which will take a block of text, a list of permitted parameters, and the arguments which must substitute for the parameters.
974 Here we split the text on ${ which means that all parts except the first will begin with a parameter name which will be terminated by }. The split function will consume the literal ${ in each case.
976 42b <expand_chunk_args()[1](
\v), lang=> ≡
977 ________________________________________________________________________
978 1 | function expand_chunk_args(text, params, args,
979 2 | p, text_array, next_text, v, t, l)
981 4 | if (split(text, text_array, "\\${")) {
982 5 | «substitute-chunk-args 43a»
987 |________________________________________________________________________
990 First, we produce an associative array of substitution values indexed by parameter names. This will serve as a cache, allowing us to look up the replacement values as we extract each name.
992 43a <substitute-chunk-args[1](
\v), lang=> ≡ 43b▿
993 ________________________________________________________________________
994 1 | for(p in params) {
995 2 | v[params[p]]=args[p];
997 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
998 We accumulate substituted text in the variable text. As the first part of the split function is the part before the delimiter --- which is ${ in our case --- this part will never contain a parameter reference, so we assign this directly to the result kept in $text.
1000 43b <substitute-chunk-args[2](
\v) ⇑43a, lang=> +≡ ▵43a 43c▿
1001 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1002 4 | text=text_array[1];
1003 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1004 We then iterate over the remaining values in the array, and substitute each reference for it's argument.
1006 43c <substitute-chunk-args[3](
\v) ⇑43a, lang=> +≡ ▵43b
1007 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1008 5 | for(t=2; t in text_array; t++) {
1009 6 | «substitute-chunk-arg 43d»
1011 |________________________________________________________________________
1014 After the split on ${ a valid parameter reference will consist of valid parameter name terminated by a close-brace }. A valid character name begins with the underscore or a letter, and may contain letters, digits or underscores.
1015 A valid looking reference that is not actually the name of a parameter will be and not substituted. This is good because there is nothing to substitute anyway, and it avoids clashes when writing code for languages where ${...} is a valid construct --- such constructs will not be interfered with unless the parameter name also matches.
1017 43d <substitute-chunk-arg[1](
\v), lang=> ≡
1018 ________________________________________________________________________
1019 1 | if (match(text_array[t], "^([a-zA-Z_][a-zA-Z0-9_]*)}", l) &&
1022 4 | text = text v[l[1]] substr(text_array[t], length(l[1])+2);
1024 6 | text = text "${" text_array[t];
1026 |________________________________________________________________________
1029 Chapter 10Language Modes & Quoting
1030 lstlistings and fangle both recognize source languages, and perform some basic parsing and syntax highlighting in the rendered document1. although lstlisting supports many more languages ^1. lstlistings can detect strings and comments within a language definition and perform suitable rendering, such as italics for comments, and visible-spaces within strings.
1031 Fangle similarly can recognize strings, and comments, etc, within a language, so that any chunks included with \chunkref{a-chunk} or <a-chunk ?> can be suitably escape or quoted.
1032 10.1 Modes to keep code together
1033 As an example, the C language has a few parse modes, which affect the interpretation of characters.
1034 One parse mode is the string mode. The string mode is commenced by an un-escaped quotation mark " and terminated by the same. Within the string mode, only one additional mode can be commenced, it is the backslash mode \, which is always terminated by the following character.
1035 Another mode is [ which is terminated by a ] (unless it occurs in a string).
1036 Consider this fragment of C code:
1037 do_something((things([x, y])<wide-overbrace>^(2. [ mode), get_other_things((a, "(all)"_(4. " mode)))<wide-overbrace>^(3. ( mode)))<wide-overbrace>^(1. ( mode)
1039 Mode nesting prevents the close parenthesis in the quoted string (part 4) from terminating the parenthesis mode (part 3).
1040 Each language has a set of modes, the default mode being the null mode. Each mode can lead to other modes.
1041 10.2 Modes affect included chunks
1042 For instance, consider this chunk with language=perl:
1044 45a <test:example-perl[1](
\v), lang=perl> ≡
1045 ________________________________________________________________________
1046 1 | print "hello world $0\n";
1047 |________________________________________________________________________
1050 If it were included in a chunk with language=sh, like this:
1052 45b <test:example-sh[1](
\v), lang=sh> ≡
1053 ________________________________________________________________________
1054 1 | perl -e "«test:example-perl 45a»"
1055 |________________________________________________________________________
1058 we might want fangle would to generate output like this:
1060 46a <test:example-sh.result[1](
\v), lang=sh> ≡
1061 ________________________________________________________________________
1062 1 | perl -e "print \"hello world \$0\\n\";"
1063 |________________________________________________________________________
1066 See that the double quote ", back-slash \ and $ have been quoted with a back-slash to protect them from shell interpretation.
1067 If that were then included in a chunk with language=make, like this:
1069 46b <test:example-makefile[1](
\v), lang=make> ≡
1070 ________________________________________________________________________
1072 2 | ↦«test:example-sh 45b»
1073 |________________________________________________________________________
1076 We would need the output to look like this --- note the $$ as the single $ has been makefile-quoted with another $.
1078 46c <test:example-makefile.result[1](
\v), lang=make> ≡
1079 ________________________________________________________________________
1081 2 | ↦perl -e "print \"hello world \$$0\\n\";"
1082 |________________________________________________________________________
1085 10.3 Modes operation
1086 In order to make this work, we must define a mode-tracker supporting each language, that can detect the various quoting modes, and provide a transformation that may be applied to any included text so that included text will be interpreted correctly after any interpolation that it may be subject to at run-time.
1087 For example, the sed transformation for text to be inserted into shell double-quoted strings would be something like:
1088 s/\\/\\\\/g;s/$/\\$/g;s/"/\\"/g;
1089 which would protect \ $ "
1090 The mode tracker must also nested mode-changes, as in this shell example:
1091 echo "hello ‘id ...‘"
1093 Any shell special characters inserted at the point marked ↑ would need to be escaped if their plain-text meaning is to be preserved, including ‘ | * among others. The set of characters that need escaping in the back-ticks ‘ is not the same as the set that need escaing in the double-quotes ". However, in shell syntax, a " at the point marked ↑ does not close the leading " and so would not need additional escaping because of the nesting of the two modes.
1095 Escaping need not occur if the format and mode of the included chunk matches that of the including chunk.
1096 As each chunk is output a new mode tracker for that language is initialized in it's normal state. As text is output for that chunk the output mode is tracked. When a new chunk is included, a transformation appropriate to that mode is selected and pushed onto a stack of transformations. Any text to be output is passed through this stack of transformations.
1097 It remains to consider if the chunk-include function should return it's generated text so that the caller can apply any transformations (and formatting), or if it should apply the stack of transformations itself.
1098 Note that the transformed included text should have the property of not being able to change the mode in the current chunk.
1099 To do: Note chunk parameters should probably also be transformed
1101 10.4 Quoting scenarios
1102 10.4.1 Direct quoting
1103 He we give examples of various quoting scenarios and discuss what the expected outcome might be and how this could be obtained.
1105 47a <test:q:1[1](
\v), lang=sh> ≡
1106 ________________________________________________________________________
1107 1 | echo "$(«test:q:1-inc 47b»)"
1108 |________________________________________________________________________
1112 47b <test:q:1-inc[1](
\v), lang=sh> ≡
1113 ________________________________________________________________________
1115 |________________________________________________________________________
1118 Should this examples produce echo "$(echo "hello")" or echo "\$(echo \"hello\")" ?
1119 This depends on what the author intended, but we must provde a way to express that intent.
1120 We might argue that as both chunks have lang=sh the intent must have been to quote the included chunk — but consider that this might be shell script that writes shell script.
1121 If <test:q:1-inc 47b> had lang=text then it certainly would have been right to quote it, which leads us to ask: in what ways can we reduce quoting if lang of the included chunk is compatible with the lang of the including chunk?
1122 If we take a completely nested approach then even though $( mode might do no quoting of it's own, " mode will still do it's own quoting. We need a model where the nested $( mode will prevent " from quoting.
1123 This leads rise to the tunneling feature. In bash, the $( gives rise to a new top-level parsing scenario, so we need to enter the null mode, and also ignore any quoting and then undo-this when the $( mode is terminated by the corresponding close ).
1124 We shall say that tunneling is when a mode in a language ignores other modes in the same language and arrives back at an earlier null mode of the same language.
1125 In example <test:q:1 47a> above, the nesting of modes is: null, ", $(
1126 When mode $( is commenced, the stack of nest modes will be traversed. If the null mode can be found in the same language, without the language varying, then a tunnel will be established so that the intervening modes, " in this case, can be skipped when the modes are enumerated to quote the texted being emitted.
1127 In such a case, the correct result would be:
1129 47c <test:q:1.result[1](
\v), lang=sh> ≡
1130 ________________________________________________________________________
1131 1 | echo "$(echo "hello")"
1132 |________________________________________________________________________
1135 10.5 Language Mode Definitions
1136 All modes definitions are stored in a single multi-dimensional hash. The first index is the language, and the second index is the mode-identifier. The third indexes hold properties: terminators, and optionally, submodes, delimiters, and tunnel targets.
1137 A useful set of mode definitions for a nameless general C-type language is shown here. (Don't be confused by the double backslash escaping needed in awk. One set of escaping is for the string, and the second set of escaping is for the regex).
1138 To do: TODO: Add =<\mode{}> command which will allow us to signify that a string is
1139 regex and thus fangle will quote it for us.
1141 Submodes are entered by the characters " ' { ( [ /*
1143 48a <common-mode-definitions[1](language
\v\v), lang=> ≡ 48b▿
1144 ________________________________________________________________________
1145 1 | modes[${language}, "", "submodes"]="\\\\|\"|'|{|\\(|\\[";
1146 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1147 In the default mode, a comma surrounded by un-important white space is a delimiter of language items2. whatever a language item might be ^2.
1149 48b <common-mode-definitions[2](language
\v\v) ⇑48a, lang=> +≡ ▵48a 48d▿
1150 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1151 2 | modes[${language}, "", "delimiters"]=" *, *";
1152 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1153 and should pass this test:
1154 To do: Why do the tests run in ?(? mode and not ?? mode
1157 48c <test:mode-definitions[1](
\v), lang=> ≡ 49g⊳
1158 ________________________________________________________________________
1159 1 | parse_chunk_args("c-like", "1,2,3", a, "");
1160 2 | if (a[1] != "1") e++;
1161 3 | if (a[2] != "2") e++;
1162 4 | if (a[3] != "3") e++;
1163 5 | if (length(a) != 3) e++;
1164 6 | «pca-test.awk:summary 59c»
1166 8 | parse_chunk_args("c-like", "joe, red", a, "");
1167 9 | if (a[1] != "joe") e++;
1168 10 | if (a[2] != "red") e++;
1169 11 | if (length(a) != 2) e++;
1170 12 | «pca-test.awk:summary 59c»
1172 14 | parse_chunk_args("c-like", "${colour}", a, "");
1173 15 | if (a[1] != "${colour}") e++;
1174 16 | if (length(a) != 1) e++;
1175 17 | «pca-test.awk:summary 59c»
1176 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1177 Nested modes are identified by a backslash, a double or single quote, various bracket styles or a /* comment.
1178 For each of these sub-modes modes we must also identify at a mode terminator, and any sub-modes or delimiters that may be entered3. Because we are using the sub-mode characters as the mode identifier it means we can't currently have a mode character dependant on it's context; i.e. { can't behave differently when it is inside [. ^3.
1180 The backslash mode has no submodes or delimiters, and is terminated by any character. Note that we are not so much interested in evaluating or interpolating content as we are in delineating content. It is no matter that a double backslash (\\) may represent a single backslash while a backslash-newline may represent white space, but it does matter that the newline in a backslash newline should not be able to terminate a C pre-processor statement; and so the newline will be consumed by the backslash however it is to be interpreted.
1182 48d <common-mode-definitions[3](language
\v\v) ⇑48a, lang=> +≡ ▵48b 49f⊳
1183 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1184 3 | modes[${language}, "\\", "terminators"]=".";
1185 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1187 Common languages support two kinds of strings quoting, double quotes and single quotes.
1188 In a string we have one special mode, which is the backslash. This may escape an embedded quote and prevent us thinking that it should terminate the string.
1190 49a <mode:common-string[1](language
\v, quote
\v\v), lang=> ≡ 49b▿
1191 ________________________________________________________________________
1192 1 | modes[${language}, ${quote}, "submodes"]="\\\\";
1193 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1194 Otherwise, the string will be terminated by the same character that commenced it.
1196 49b <mode:common-string[2](language
\v, quote
\v\v) ⇑49a, lang=> +≡ ▵49a 49c▿
1197 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1198 2 | modes[${language}, ${quote}, "terminators"]=${quote};
1199 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1200 In C type languages, certain escape sequences exist in strings. We need to define mechanism to enclode any chunks included in this mode using those escape sequences. These are expressed in two parts, s meaning search, and r meaning replace.
1201 The first substitution is to replace a backslash with a double backslash. We do this first as other substitutions may introduce a backslash which we would not then want to escape again here.
1202 Note: Backslashes need double-escaping in the search pattern but not in the replacement string, hence we are replacing a literal \ with a literal \\.
1204 49c <mode:common-string[3](language
\v, quote
\v\v) ⇑49a, lang=> +≡ ▵49b 49d▿
1205 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1206 3 | escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]="\\\\";
1207 4 | escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="\\\\";
1208 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1209 If the quote character occurs in the text, it should be preceded by a backslash, otherwise it would terminate the string unexpectedly.
1211 49d <mode:common-string[4](language
\v, quote
\v\v) ⇑49a, lang=> +≡ ▵49c 49e▿
1212 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1213 5 | escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]=${quote};
1214 6 | escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="\\" ${quote};
1215 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1216 Any newlines in the string, must be replaced by \n.
1218 49e <mode:common-string[5](language
\v, quote
\v\v) ⇑49a, lang=> +≡ ▵49d
1219 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1220 7 | escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]="\n";
1221 8 | escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="\\n";
1222 |________________________________________________________________________
1225 For the common modes, we define this string handling for double and single quotes.
1227 49f <common-mode-definitions[4](language
\v\v) ⇑48a, lang=> +≡ ⊲48d 50b⊳
1228 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1229 4 | «mode:common-string
\v(${language}
\v, "\""
\v) 49a»
1230 5 | «mode:common-string
\v(${language}
\v, "'"
\v) 49a»
1231 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1232 Working strings should pass this test:
1234 49g <test:mode-definitions[2](
\v) ⇑48c, lang=> +≡ ⊲48c 54b⊳
1235 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1236 18 | parse_chunk_args("c-like", "say \"I said, \\\"Hello, how are you\\\".\", for me", a, "");
1237 19 | if (a[1] != "say \"I said, \\\"Hello, how are you\\\".\"") e++;
1238 20 | if (a[2] != "for me") e++;
1239 21 | if (length(a) != 2) e++;
1240 22 | «pca-test.awk:summary 59c»
1241 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1242 10.5.3 Parentheses, Braces and Brackets
1243 Where quotes are closed by the same character, parentheses, brackets and braces are closed by an alternate character.
1245 50a <mode:common-brackets[1](language
\v, open
\v, close
\v\v), lang=> ≡
1246 ________________________________________________________________________
1247 1 | modes[${language}, ${open}, "submodes" ]="\\\\|\"|{|\\(|\\[|'|/\\*";
1248 2 | modes[${language}, ${open}, "delimiters"]=" *, *";
1249 3 | modes[${language}, ${open}, "terminators"]=${close};
1250 |________________________________________________________________________
1253 Note that the open is NOT a regex but the close token IS.
1254 To do: When we can quote regex we won't have to put the slashes in here
1257 50b <common-mode-definitions[5](language
\v\v) ⇑48a, lang=> +≡ ⊲49f
1258 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1259 6 | «mode:common-brackets
\v(${language}
\v, "{"
\v, "}"
\v) 50a»
1260 7 | «mode:common-brackets
\v(${language}
\v, "["
\v, "\\]"
\v) 50a»
1261 8 | «mode:common-brackets
\v(${language}
\v, "("
\v, "\\)"
\v) 50a»
1262 |________________________________________________________________________
1265 10.5.4 Customizing Standard Modes
1267 50c <mode:add-submode[1](language
\v, mode
\v, submode
\v\v), lang=> ≡
1268 ________________________________________________________________________
1269 1 | modes[${language}, ${mode}, "submodes"] = modes[${language}, ${mode}, "submodes"] "|" ${submode};
1270 |________________________________________________________________________
1274 50d <mode:add-escapes[1](language
\v, mode
\v, search
\v, replace
\v\v), lang=> ≡
1275 ________________________________________________________________________
1276 1 | escapes[${language}, ${mode}, ++escapes[${language}, ${mode}], "s"]=${search};
1277 2 | escapes[${language}, ${mode}, escapes[${language}, ${mode}], "r"]=${replace};
1278 |________________________________________________________________________
1283 We can define /* comment */ style comments and //comment style comments to be added to any language:
1285 50e <mode:multi-line-comments[1](language
\v\v), lang=> ≡
1286 ________________________________________________________________________
1287 1 | «mode:add-submode
\v(${language}
\v, ""
\v, "/\\*"
\v) 50c»
1288 2 | modes[${language}, "/*", "terminators"]="\\*/";
1289 |________________________________________________________________________
1293 50f <mode:single-line-slash-comments[1](language
\v\v), lang=> ≡
1294 ________________________________________________________________________
1295 1 | «mode:add-submode
\v(${language}
\v, ""
\v, "//"
\v) 50c»
1296 2 | modes[${language}, "//", "terminators"]="\n";
1297 3 | «mode:add-escapes
\v(${language}
\v, "//"
\v, "\n"
\v, "\n//"
\v) 50d»
1298 |________________________________________________________________________
1301 We can also define # comment style comments (as used in awk and shell scripts) in a similar manner.
1302 To do: I'm having to use # for hash and ¯extbackslash{} for and have hacky work-arounds in the parser for now
1305 50g <mode:add-hash-comments[1](language
\v\v), lang=> ≡
1306 ________________________________________________________________________
1307 1 | «mode:add-submode
\v(${language}
\v, ""
\v, "#"
\v) 50c»
1308 2 | modes[${language}, "#", "terminators"]="\n";
1309 3 | «mode:add-escapes
\v(${language}
\v, "#"
\v, "\n"
\v, "\n#"
\v) 50d»
1310 |________________________________________________________________________
1313 In C, the # denotes pre-processor directives which can be multi-line
1315 51a <mode:add-hash-defines[1](language
\v\v), lang=> ≡
1316 ________________________________________________________________________
1317 1 | «mode:add-submode
\v(${language}
\v, ""
\v, "#"
\v) 50c»
1318 2 | modes[${language}, "#", "submodes" ]="\\\\";
1319 3 | modes[${language}, "#", "terminators"]="\n";
1320 4 | «mode:add-escapes
\v(${language}
\v, "#"
\v, "\n"
\v, "\\\\\n"
\v) 50d»
1321 |________________________________________________________________________
1325 51b <mode:quote-dollar-escape[1](language
\v, quote
\v\v), lang=> ≡
1326 ________________________________________________________________________
1327 1 | escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]="\\$";
1328 2 | escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="\\$";
1329 |________________________________________________________________________
1332 We can add these definitions to various languages
1334 51c <mode-definitions[1](
\v), lang=> ≡ 52b⊳
1335 ________________________________________________________________________
1336 1 | «common-mode-definitions
\v("c-like"
\v) 48a»
1338 3 | «common-mode-definitions
\v("c"
\v) 48a»
1339 4 | «mode:multi-line-comments
\v("c"
\v) 50e»
1340 5 | «mode:single-line-slash-comments
\v("c"
\v) 50f»
1341 6 | «mode:add-hash-defines
\v("c"
\v) 51a»
1343 8 | «common-mode-definitions
\v("awk"
\v) 48a»
1344 9 | «mode:add-hash-comments
\v("awk"
\v) 50g»
1345 10 | «mode:add-naked-regex
\v("awk"
\v) 52a»
1346 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1347 The awk definitions should allow a comment block like this:
1349 51d <test:comment-quote[1](
\v), lang=awk> ≡
1350 ________________________________________________________________________
1351 1 | # Comment: «test:comment-text 51e»
1352 |________________________________________________________________________
1356 51e <test:comment-text[1](
\v), lang=> ≡
1357 ________________________________________________________________________
1358 1 | Now is the time for
1359 2 | the quick brown fox to bring lemonade
1361 |________________________________________________________________________
1364 to come out like this:
1366 51f <test:comment-quote:result[1](
\v), lang=> ≡
1367 ________________________________________________________________________
1368 1 | # Comment: Now is the time for
1369 2 | #the quick brown fox to bring lemonade
1371 |________________________________________________________________________
1374 The C definition for such a block should have it come out like this:
1376 51g <test:comment-quote:C-result[1](
\v), lang=> ≡
1377 ________________________________________________________________________
1378 1 | # Comment: Now is the time for\
1379 2 | the quick brown fox to bring lemonade\
1381 |________________________________________________________________________
1385 This pattern is incomplete, but meant to detect naked regular expressions in awk and perl; e.g. /.*$/, however required capabilities are not present.
1386 Current it only detects regexes anchored with ^ as used in fangle.
1387 For full regex support, modes need to be named not after their starting character, but some other more fully qualified name.
1389 52a <mode:add-naked-regex[1](language
\v\v), lang=> ≡
1390 ________________________________________________________________________
1391 1 | «mode:add-submode
\v(${language}
\v, ""
\v, "/\\^"
\v) 50c»
1392 2 | modes[${language}, "/^", "terminators"]="/";
1393 |________________________________________________________________________
1398 52b <mode-definitions[2](
\v) ⇑51c, lang=> +≡ ⊲51c 52c▿
1399 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1400 11 | «common-mode-definitions
\v("perl"
\v) 48a»
1401 12 | «mode:multi-line-comments
\v("perl"
\v) 50e»
1402 13 | «mode:add-hash-comments
\v("perl"
\v) 50g»
1403 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1404 Still need to add add s/, submode /, terminate both with //. This is likely to be impossible as perl regexes can contain perl.
1406 Shell single-quote strings are different to other strings and have no escape characters. The only special character is the single quote ' which always closes the string. Therefore we cannot use <common-mode-definitions
\v("sh"
\v) 48a> but we will invoke most of it's definition apart from single-quote strings.
1408 52c <mode-definitions[3](
\v) ⇑51c, lang=awk> +≡ ▵52b 54a⊳
1409 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1410 14 | modes["sh", "", "submodes"]="\\\\|\"|'|{|\\(|\\[|\\$\\(";
1411 15 | modes["sh", "\\", "terminators"]=".";
1413 17 | modes["sh", "\"", "submodes"]="\\\\|\\$\\(";
1414 18 | modes["sh", "\"", "terminators"]="\"";
1415 19 | escapes["sh", "\"", ++escapes["sh", "\""], "s"]="\\\\";
1416 20 | escapes["sh", "\"", escapes["sh", "\""], "r"]="\\\\";
1417 21 | escapes["sh", "\"", ++escapes["sh", "\""], "s"]="\"";
1418 22 | escapes["sh", "\"", escapes["sh", "\""], "r"]="\\" "\"";
1419 23 | escapes["sh", "\"", ++escapes["sh", "\""], "s"]="\n";
1420 24 | escapes["sh", "\"", escapes["sh", "\""], "r"]="\\n";
1422 26 | modes["sh", "'", "terminators"]="'";
1423 27 | escapes["sh", "'", ++escapes["sh", "'"], "s"]="'";
1424 28 | escapes["sh", "'", escapes["sh", "'"], "r"]="'\\'" "'";
1425 29 | «mode:common-brackets
\v("sh"
\v, "$("
\v, "\\)"
\v) 50a»
1426 30 | «mode:add-tunnel
\v("sh"
\v, "$("
\v, ""
\v) 52d»
1427 31 | «mode:common-brackets
\v("sh"
\v, "{"
\v, "}"
\v) 50a»
1428 32 | «mode:common-brackets
\v("sh"
\v, "["
\v, "\\]"
\v) 50a»
1429 33 | «mode:common-brackets
\v("sh"
\v, "("
\v, "\\)"
\v) 50a»
1430 34 | «mode:add-hash-comments
\v("sh"
\v) 50g»
1431 35 | «mode:quote-dollar-escape
\v("sh"
\v, "\""
\v) 51b»
1432 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1433 The definition of add-tunnel is:
1435 52d <mode:add-tunnel[1](language
\v, mode
\v, tunnel
\v\v), lang=> ≡
1436 ________________________________________________________________________
1437 1 | escapes[${language}, ${mode}, ++escapes[${language}, ${mode}], "tunnel"]=${tunnel};
1438 |________________________________________________________________________
1442 For makefiles, we currently recognize 2 modes: the null mode and ↦ mode, which is tabbed mode and contains the makefile recipie. In the null mode the only escape is $ which must be converted to $$.
1443 Tabbed mode is harder to manage, as the GNU Make Manual says in the section on splitting lines4. http://www.gnu.org/s/hello/manual/make/Splitting-Lines.html ^4. There is no way to escape a multi-line text that occurs as part of a makefile recipe.
1444 Despite this sad fact, if the newline's in the shell script all occur at points of top-level shell syntax, then we could replace them with ;\n↦and largely get the right effect.
1446 53a <test:make:1[1](
\v), lang=make> ≡
1447 ________________________________________________________________________
1450 3 | ↦«test:make:1-inc
\v($@) 53b»
1452 |________________________________________________________________________
1456 53b <test:make:1-inc[1](target
\v\v), lang=sh> ≡
1457 ________________________________________________________________________
1458 1 | if test "${target}" = "all"
1459 2 | then echo yes, all
1460 3 | else echo not all
1462 |________________________________________________________________________
1465 The two chunks about could reasonably produce this:
1467 53c <test:make:1.result[1](
\v), lang=make> ≡
1468 ________________________________________________________________________
1471 3 | ↦if test "$@" = "all" ;\
1472 4 | ↦then echo yes, all ;\
1473 5 | ↦else echo not all ;\
1475 |________________________________________________________________________
1478 But will more likely produce this:
1480 53d <test:make:1.result-actual[1](
\v), lang=make> ≡
1481 ________________________________________________________________________
1484 3 | ↦if test "$$@" = "all" ;\
1485 4 | ↦ then echo yes, all ;\
1486 5 | ↦ else echo not all ;\
1488 |________________________________________________________________________
1491 The chunk argument $@ has been quoted (which would have been fine if we were passing the name of a shell variable), and the other shell lines are (harmlessly) indented by 1 space as part of fangle indent-matching which should have taken into account the expanded tab size, and should generally take into account the expanded prefix of the line whose indent it is trying to match, but which in this case we want to have no effect at all!
1492 To do: The $@ was passed from a make fragment. In what cases should it be converted to $$@?
1493 Do we need to track the language of sources of arguments?
1495 A more ugly work-around until this problem can be solved would be to use this notation:
1497 53e <test:make:2[1](
\v), lang=make> ≡
1498 ________________________________________________________________________
1501 3 | ↦ARG="$@"; «test:make:1-inc
\v($ARG) 53b»
1502 |________________________________________________________________________
1505 which produces this output which is more useful (because it works):
1507 53f <test:make:2.result[1](
\v), lang=make> ≡
1508 ________________________________________________________________________
1510 2 | ↦echo making test
1511 3 | ↦ARG="$@"; if test "$$ARG" = "all" ;\
1512 4 | ↦ then echo yes, all ;\
1513 5 | ↦ else echo not all ;\
1515 |________________________________________________________________________
1518 If, however, the shell fragment contained strings with literal newline characters then there would be no easy way to escape these and preserve the value of the string.
1519 A different style of makefile construction might be used — the recipe could be stored in a target specific variable5. http://www.gnu.org/s/hello/manual/make/Target_002dspecific.html ^5 which contains the recipe with a more normal escape mechanism.
1522 54a <mode-definitions[4](
\v) ⇑51c, lang=awk> +≡ ⊲52c
1523 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1524 36 | modes["make", "", "submodes"]="↦";
1525 37 | modes["make", "↦", "terminators"]="\\n";
1526 38 | escapes["make", "↦", ++escapes["make", "↦"], "s"]="\\n";
1527 39 | escapes["make", "↦", escapes["make", "↦"], "r"]=" ;\\\n↦";
1528 40 | escapes["make", "↦", ++escapes["make", "↦"], "s"]="\\$";
1529 41 | escapes["make", "↦", escapes["make", "↦"], "r"]="$$";
1530 |________________________________________________________________________
1533 Note also that the tab character is hard-wired into the pattern, and that the make variable .RECIPEPREFIX might change this to something else.
1535 Also, the parser must return any spare text at the end that has not been processed due to a mode terminator being found.
1537 54b <test:mode-definitions[3](
\v) ⇑48c, lang=> +≡ ⊲49g 54c▿
1538 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1539 23 | rest = parse_chunk_args("c-like", "1, 2, 3) spare", a, "(");
1540 24 | if (a[1] != 1) e++;
1541 25 | if (a[2] != 2) e++;
1542 26 | if (a[3] != 3) e++;
1543 27 | if (length(a) != 3) e++;
1544 28 | if (rest != " spare") e++;
1545 29 | «pca-test.awk:summary 59c»
1546 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1547 We must also be able to parse the example given earlier.
1549 54c <test:mode-definitions[4](
\v) ⇑48c, lang=> +≡ ▵54b
1550 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1551 30 | parse_chunk_args("c-like", "things[x, y], get_other_things(a, \"(all)\"), 99", a, "(");
1552 31 | if (a[1] != "things[x, y]") e++;
1553 32 | if (a[2] != "get_other_things(a, \"(all)\")") e++;
1554 33 | if (a[3] != "99") e++;
1555 34 | if (length(a) != 3) e++;
1556 35 | «pca-test.awk:summary 59c»
1557 |________________________________________________________________________
1560 10.7 A non-recursive mode tracker
1562 The mode tracker holds its state in a stack based on a numerically indexed hash. This function, when passed an empty hash, will intialize it.
1564 54d <new_mode_tracker()[1](
\v), lang=> ≡
1565 ________________________________________________________________________
1566 1 | function new_mode_tracker(context, language, mode) {
1567 2 | context[""] = 0;
1568 3 | context[0, "language"] = language;
1569 4 | context[0, "mode"] = mode;
1571 |________________________________________________________________________
1574 Because awk functions cannot return an array, we must create the array first and pass it in, so we have a fangle macro to do this:
1576 55a <new-mode-tracker[1](context
\v, language
\v, mode
\v\v), lang=awk> ≡
1577 ________________________________________________________________________
1578 1 | «awk-delete-array
\v(context
\v) 35d»
1579 2 | new_mode_tracker(${context}, ${language}, ${mode});
1580 |________________________________________________________________________
1584 And for tracking modes, we dispatch to a mode-tracker action based on the current language
1586 55b <mode_tracker[1](
\v), lang=awk> ≡ 55c▿
1587 ________________________________________________________________________
1588 1 | function push_mode_tracker(context, language, mode,
1592 5 | if (! ("" in context)) {
1593 6 | «new-mode-tracker
\v(context
\v, language
\v, mode
\v) 55a»
1596 9 | top = context[""];
1597 10 | # if (context[top, "language"] == language && mode=="") mode = context[top, "mode"];
1598 11 | if (context[top, "language"] == language && context[top, "mode"] == mode) return top;
1601 14 | context[top, "language"] = language;
1602 15 | context[top, "mode"] = mode;
1603 16 | context[""] = top;
1605 18 | return old_top;
1607 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1609 55c <mode_tracker[2](
\v) ⇑55b, lang=> +≡ ▵55b 55d▿
1610 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1611 20 | function dump_mode_tracker(context,
1614 23 | for(c=0; c <= context[""]; c++) {
1615 24 | printf(" %2d %s:%s\n", c, context[c, "language"], context[c, "mode"]) > "/dev/stderr";
1616 25 | for(d=1; ( (c, "values", d) in context); d++) {
1617 26 | printf(" %2d %s\n", d, context[c, "values", d]) > "/dev/stderr";
1621 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1623 55d <mode_tracker[3](
\v) ⇑55b, lang=> +≡ ▵55c 60a⊳
1624 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1625 30 | function pop_mode_tracker(context, context_origin)
1627 32 | if ( (context_origin) && ("" in context) && context[""] != (1+context_origin) && context[""] != context_origin) return 0;
1628 33 | context[""] = context_origin;
1631 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1632 This implies that any chunk must be syntactically whole; for instance, this is fine:
1634 55e <test:whole-chunk[1](
\v), lang=> ≡
1635 ________________________________________________________________________
1637 2 | «test:say-hello 56a»
1639 |________________________________________________________________________
1643 56a <test:say-hello[1](
\v), lang=> ≡
1644 ________________________________________________________________________
1646 |________________________________________________________________________
1649 But this is not fine; the chunk <test:hidden-else 56c> is not properly cromulent.
1651 56b <test:partial-chunk[1](
\v), lang=> ≡
1652 ________________________________________________________________________
1654 2 | «test:hidden-else 56c»
1656 |________________________________________________________________________
1660 56c <test:hidden-else[1](
\v), lang=> ≡
1661 ________________________________________________________________________
1662 1 | print "I'm fine";
1664 3 | print "I'm not";
1665 |________________________________________________________________________
1668 These tests will check for correct behaviour:
1670 56d <test:cromulence[1](
\v), lang=> ≡
1671 ________________________________________________________________________
1672 1 | echo Cromulence test
1673 2 | passtest $FANGLE -Rtest:whole-chunk $TXT_SRC &>/dev/null || ( echo "Whole chunk failed" && exit 1 )
1674 3 | failtest $FANGLE -Rtest:partial-chunk $TXT_SRC &>/dev/null || ( echo "Partial chunk failed" && exit 1 )
1675 |________________________________________________________________________
1679 We must avoid recursion as a language construct because we intend to employ mode-tracking to track language mode of emitted code, and the code is emitted from a function which is itself recursive, so instead we implement psuedo-recursion using our own stack based on a hash.
1681 56e <mode_tracker()[1](
\v), lang=awk> ≡ 56f▿
1682 ________________________________________________________________________
1683 1 | function mode_tracker(context, text, values,
1684 2 | # optional parameters
1686 4 | mode, submodes, language,
1687 5 | cindex, c, a, part, item, name, result, new_values, new_mode,
1688 6 | delimiters, terminators)
1690 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1691 We could be re-commencing with a valid context, so we need to setup the state according to the last context.
1693 56f <mode_tracker()[2](
\v) ⇑56e, lang=> +≡ ▵56e 57b⊳
1694 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1695 8 | cindex = context[""] + 0;
1696 9 | mode = context[cindex, "mode"];
1697 10 | language = context[cindex, "language" ];
1698 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1699 First we construct a single large regex combining the possible sub-modes for the current mode along with the terminators for the current mode.
1701 56g <parse_chunk_args-reset-modes[1](
\v), lang=> ≡ 57a⊳
1702 ________________________________________________________________________
1703 1 | submodes=modes[language, mode, "submodes"];
1705 3 | if ((language, mode, "delimiters") in modes) {
1706 4 | delimiters = modes[language, mode, "delimiters"];
1707 5 | if (length(submodes)>0) submodes = submodes "|";
1708 6 | submodes=submodes delimiters;
1709 7 | } else delimiters="";
1710 8 | if ((language, mode, "terminators") in modes) {
1711 9 | terminators = modes[language, mode, "terminators"];
1712 10 | if (length(submodes)>0) submodes = submodes "|";
1713 11 | submodes=submodes terminators;
1714 12 | } else terminators="";
1715 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1716 If we don't find anything to match on --- probably because the language is not supported --- then we return the entire text without matching anything.
1718 57a <parse_chunk_args-reset-modes[2](
\v) ⇑56g, lang=> +≡ ⊲56g
1719 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1720 13 | if (! length(submodes)) return text;
1721 |________________________________________________________________________
1725 57b <mode_tracker()[3](
\v) ⇑56e, lang=> +≡ ⊲56f 57c▿
1726 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1727 11 | «parse_chunk_args-reset-modes 56g»
1728 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1729 We then iterate the text (until there is none left) looking for sub-modes or terminators in the regex.
1731 57c <mode_tracker()[4](
\v) ⇑56e, lang=> +≡ ▵57b 57d▿
1732 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1733 12 | while((cindex >= 0) && length(text)) {
1734 13 | if (match(text, "(" submodes ")", a)) {
1735 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1736 A bug that creeps in regularly during development is bad regexes of zero length which result in an infinite loop (as no text is consumed), so I catch that right away with this test.
1738 57d <mode_tracker()[5](
\v) ⇑56e, lang=> +≡ ▵57c 57e▿
1739 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1740 14 | if (RLENGTH<1) {
1741 15 | error(sprintf("Internal error, matched zero length submode, should be impossible - likely regex computation error\n" \
1742 16 | "Language=%s\nmode=%s\nmatch=%s\n", language, mode, submodes));
1744 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1745 part is defined as the text up to the sub-mode or terminator, and this is appended to item --- which is the current text being gathered. If a mode has a delimiter, then item is reset each time a delimiter is found.
1746 ("hello_item, there_item")<wide-overbrace>^item, (he said.)<wide-overbrace>^item
1748 57e <mode_tracker()[6](
\v) ⇑56e, lang=> +≡ ▵57d 57f▿
1749 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1750 18 | part = substr(text, 1, RSTART -1);
1751 19 | item = item part;
1752 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1753 We must now determine what was matched. If it was a terminator, then we must restore the previous mode.
1755 57f <mode_tracker()[7](
\v) ⇑56e, lang=> +≡ ▵57e 58a⊳
1756 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1757 20 | if (match(a[1], "^" terminators "$")) {
1758 21 | #printf("%2d EXIT MODE [%s] by [%s] [%s]\n", cindex, mode, a[1], text) > "/dev/stderr"
1759 22 | context[cindex, "values", ++context[cindex, "values"]] = item;
1760 23 | delete context[cindex];
1761 24 | context[""] = --cindex;
1762 25 | if (cindex>=0) {
1763 26 | mode = context[cindex, "mode"];
1764 27 | language = context[cindex, "language"];
1765 28 | «parse_chunk_args-reset-modes 56g»
1767 30 | item = item a[1];
1768 31 | text = substr(text, 1 + length(part) + length(a[1]));
1770 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1771 If a delimiter was matched, then we must store the current item in the parsed values array, and reset the item.
1773 58a <mode_tracker()[8](
\v) ⇑56e, lang=> +≡ ⊲57f 58b▿
1774 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1775 33 | else if (match(a[1], "^" delimiters "$")) {
1776 34 | if (cindex==0) {
1777 35 | context[cindex, "values", ++context[cindex, "values"]] = item;
1780 38 | item = item a[1];
1782 40 | text = substr(text, 1 + length(part) + length(a[1]));
1784 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1785 otherwise, if a new submode is detected (all submodes have terminators), we must create a nested parse context until we find the terminator for this mode.
1787 58b <mode_tracker()[9](
\v) ⇑56e, lang=> +≡ ▵58a 58c▿
1788 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1789 42 | else if ((language, a[1], "terminators") in modes) {
1790 43 | #check if new_mode is defined
1791 44 | item = item a[1];
1792 45 | #printf("%2d ENTER MODE [%s] in [%s]\n", cindex, a[1], text) > "/dev/stderr"
1793 46 | text = substr(text, 1 + length(part) + length(a[1]));
1794 47 | context[""] = ++cindex;
1795 48 | context[cindex, "mode"] = a[1];
1796 49 | context[cindex, "language"] = language;
1798 51 | «parse_chunk_args-reset-modes 56g»
1800 53 | error(sprintf("Submode '%s' set unknown mode in text: %s\nLanguage %s Mode %s\n", a[1], text, language, mode));
1801 54 | text = substr(text, 1 + length(part) + length(a[1]));
1804 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1805 In the final case, we parsed to the end of the string. If the string was entire, then we should have no nested mode context, but if the string was just a fragment we may have a mode context which must be preserved for the next fragment. Todo: Consideration ought to be given if sub-mode strings are split over two fragments.
1807 58c <mode_tracker()[10](
\v) ⇑56e, lang=> +≡ ▵58b
1808 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1810 58 | context[cindex, "values", ++context[cindex, "values"]] = item text;
1816 64 | context["item"] = item;
1818 66 | if (length(item)) context[cindex, "values", ++context[cindex, "values"]] = item;
1821 |________________________________________________________________________
1824 10.7.3.1 One happy chunk
1825 All the mode tracker chunks are referred to here:
1827 59a <mode-tracker[1](
\v), lang=> ≡
1828 ________________________________________________________________________
1829 1 | «new_mode_tracker() 54d»
1830 2 | «mode_tracker() 56e»
1831 |________________________________________________________________________
1835 We can test this function like this:
1837 59b <pca-test.awk[1](
\v), lang=awk> ≡
1838 ________________________________________________________________________
1840 2 | «mode-tracker 59a»
1841 3 | «parse_chunk_args() ?»
1844 6 | «mode-definitions 51c»
1846 8 | «test:mode-definitions 48c»
1848 |________________________________________________________________________
1852 59c <pca-test.awk:summary[1](
\v), lang=awk> ≡
1853 ________________________________________________________________________
1855 2 | printf "Failed " e
1857 4 | print "a[" b "] => " a[b];
1864 |________________________________________________________________________
1867 which should give this output:
1869 59d <pca-test.awk-results[1](
\v), lang=> ≡
1870 ________________________________________________________________________
1871 1 | a[foo.quux.quirk] =>
1872 2 | a[foo.quux.a] => fleeg
1873 3 | a[foo.bar] => baz
1875 5 | a[name] => freddie
1876 |________________________________________________________________________
1879 10.8 Escaping and Quoting
1880 For the time being and to get around TeXmacs inability to export a TAB character, the right arrow ↦ whose UTF-8 sequence is ...
1883 Another special character is used, the left-arrow ↤ with UTF-8 sequence 0xE2 0x86 0xA4 is used to strip any preceding white space as a way of un-tabbing and removing indent that has been applied — this is important for bash here documents, and the like. It's a filthy hack.
1884 To do: remove the hack
1887 60a <mode_tracker[4](
\v) ⇑55b, lang=> +≡ ⊲55d 60b▿
1888 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1890 36 | function untab(text) {
1891 37 | gsub("[[:space:]]*\xE2\x86\xA4","", text);
1894 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1895 Each nested mode can optionally define a set of transforms to be applied to any text that is included from another language.
1896 This code can perform transforms from index c downwards.
1898 60b <mode_tracker[5](
\v) ⇑55b, lang=awk> +≡ ▵60a 58c⊳
1899 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1900 40 | function transform_escape(context, text, top,
1901 41 | c, cp, cpl, s, r)
1903 43 | for(c = top; c >= 0; c--) {
1904 44 | if ( (context[c, "language"], context[c, "mode"]) in escapes) {
1905 45 | cpl = escapes[context[c, "language"], context[c, "mode"]];
1906 46 | for (cp = 1; cp <= cpl; cp ++) {
1907 47 | s = escapes[context[c, "language"], context[c, "mode"], cp, "s"];
1908 48 | r = escapes[context[c, "language"], context[c, "mode"], cp, "r"];
1909 49 | if (length(s)) {
1910 50 | gsub(s, r, text);
1912 52 | if ( (context[c, "language"], context[c, "mode"], cp, "t") in escapes ) {
1913 53 | quotes[src, "t"] = escapes[context[c, "language"], context[c, "mode"], cp, "t"];
1920 60 | function dump_escaper(quotes, r, cc) {
1921 61 | for(cc=1; cc<=c; cc++) {
1922 62 | printf("%2d s[%s] r[%s]\n", cc, quotes[cc, "s"], quotes[cc, "r"]) > "/dev/stderr"
1925 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1927 60c <test:escapes[1](
\v), lang=sh> ≡
1928 ________________________________________________________________________
1929 1 | echo escapes test
1930 2 | passtest $FANGLE -Rtest:comment-quote $TXT_SRC &>/dev/null || ( echo "Comment-quote failed" && exit 1 )
1931 |________________________________________________________________________
1934 Chapter 11Recognizing Chunks
1935 Fangle recognizes noweb chunks, but as we also want better LaTeX integration we will recognize any of these:
1936 • notangle chunks matching the pattern ^<<.*?>>=
1937 • chunks beginning with \begin{lstlistings}, possibly with \Chunk{...} on the previous line
1938 • an older form I have used, beginning with \begin{Chunk}[options] --- also more suitable for plain LaTeX users1. Is there such a thing as plain LaTeX? ^1.
1940 The variable chunking is used to signify that we are processing a code chunk and not document. In such a state, input lines will be assigned to the current chunk; otherwise they are ignored.
1942 We don't handle TeXmacs files natively yet, but rather instead emit unicode character sequences to mark up the text-export file which we do process.
1943 These hacks detect the unicode character sequences and retro-fit in the old TeX parsing.
1944 We convert ↦ into a tab character.
1946 61a <recognize-chunk[1](
\v), lang=> ≡ 61b▿
1947 ________________________________________________________________________
1950 2 | # gsub("\n*$","");
1951 3 | # gsub("\n", " ");
1954 6 | /\xE2\x86\xA6/ {
1955 7 | gsub("\\xE2\\x86\\xA6", "\x09");
1957 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1958 TeXmacs back-tick handling is obscure, and a cut-n-paste back-tick from a shell window comes out as a unicode sequence2. that won't export to html, except as a NULL character (literal 0x00) ^2 that is fixed-up here.
1960 61b <recognize-chunk[2](
\v) ⇑61a, lang=> +≡ ▵61a 62a⊳
1961 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1963 10 | /\xE2\x80\x98/ {
1964 11 | gsub("\\xE2\\x80\\x98", "‘");
1966 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1967 In the TeXmacs output, the start of a chunk will appear like this:
1968 5b<example-chunk^K[1](arg1,^K arg2^K^K), lang=C> ≡
1969 We detect the the start of a TeXmacs chunk by detecting the ≡ symbol which occurs near the end of the line. We obtain the chunk name, the chunk parameters, and the chunk language.
1971 62a <recognize-chunk[3](
\v) ⇑61a, lang=> +≡ ⊲61b 62b▿
1972 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1974 14 | /\xE2\x89\xA1/ {
1975 15 | if (match($0, "^ *([^[ ]* |)<([^[ ]*)\\[[0-9]*\\][(](.*)[)].*, lang=([^ ]*)>", line)) {
1976 16 | next_chunk_name=line[2];
1977 17 | get_texmacs_chunk_args(line[3], next_chunk_params);
1978 18 | gsub(ARG_SEPARATOR ",? ?", ";", line[3]);
1979 19 | params = "params=" line[3];
1980 20 | if ((line[4])) {
1981 21 | params = params ",language=" line[4]
1983 23 | get_tex_chunk_args(params, next_chunk_opts);
1984 24 | new_chunk(next_chunk_name, next_chunk_opts, next_chunk_params);
1985 25 | texmacs_chunking = 1;
1987 27 | # warning(sprintf("Unexpected chunk match: %s\n", $_))
1991 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1993 Our current scheme is to recognize the new lstlisting chunks, but these may be preceded by a \Chunk command which in L Y X is a more convenient way to pass the chunk name to the \begin{lstlistings} command, and a more visible way to specify other lstset settings.
1994 The arguments to the \Chunk command are a name, and then a comma-seperated list of key-value pairs after the manner of \lstset. (In fact within the LaTeX \Chunk macro (section 16.2.1) the text name= is prefixed to the argument which is then literally passed to \lstset).
1996 62b <recognize-chunk[4](
\v) ⇑61a, lang=awk> +≡ ▵62a 62c▿
1997 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
1999 32 | if (match($0, "^\\\\Chunk{ *([^ ,}]*),?(.*)}", line)) {
2000 33 | next_chunk_name = line[1];
2001 34 | get_tex_chunk_args(line[2], next_chunk_opts);
2005 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2006 We also make a basic attempt to parse the name out of the \lstlistings[name=chunk-name] text, otherwise we fall back to the name found in the previous chunk command. This attempt is very basic and doesn't support commas or spaces or square brackets as part of the chunkname. We also recognize \begin{Chunk} which is convenient for some users3. but not yet supported in the LaTeX macros ^3.
2008 62c <recognize-chunk[5](
\v) ⇑61a, lang=> +≡ ▵62b 63a⊳
2009 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2010 38 | /^\\begin{lstlisting}|^\\begin{Chunk}/ {
2011 39 | if (match($0, "}.*[[,] *name= *{? *([^], }]*)", line)) {
2012 40 | new_chunk(line[1]);
2014 42 | new_chunk(next_chunk_name, next_chunk_opts);
2019 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2022 A chunk body in TeXmacs ends with |________... if it is the final chunklet of a chunk, or if there are further chunklets it ends with |\/\/\/... which is a depiction of a jagged line of torn paper.
2024 63a <recognize-chunk[6](
\v) ⇑61a, lang=> +≡ ⊲62c 63b▿
2025 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2026 47 | /^ *\|____________*/ && texmacs_chunking {
2027 48 | active_chunk="";
2028 49 | texmacs_chunking=0;
2031 52 | /^ *\|\/\\/ && texmacs_chunking {
2032 53 | texmacs_chunking=0;
2034 55 | active_chunk="";
2036 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2037 It has been observed that not every line of output when a TeXmacs chunk is active is a line of chunk. This may no longer be true, but we set a variable texmacs_chunk if the current line is a chunk line.
2038 Initially we set this to zero...
2040 63b <recognize-chunk[7](
\v) ⇑61a, lang=> +≡ ▵63a 63c▿
2041 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2042 57 | texmacs_chunk=0;
2043 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2044 ...and then we look to see if the current line is a chunk line.
2045 TeXmacs lines look like this: 3 | main() { so we detect the lines by leading white space, digits, more whiter space and a vertical bar followed by at least once space.
2046 If we find such a line, we remove this line-header and set texmacs_chunk=1 as well as chunking=1
2048 63c <recognize-chunk[8](
\v) ⇑61a, lang=> +≡ ▵63b 63d▿
2049 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2050 58 | /^ *[1-9][0-9]* *\| / {
2051 59 | if (texmacs_chunking) {
2053 61 | texmacs_chunk=1;
2054 62 | gsub("^ *[1-9][0-9]* *\\| ", "")
2057 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2058 When TeXmacs chunking, lines that commence with \/ or __ are not chunk content but visual framing, and are skipped.
2060 63d <recognize-chunk[9](
\v) ⇑61a, lang=> +≡ ▵63c 64a⊳
2061 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2062 65 | /^ *\.\/\\/ && texmacs_chunking {
2065 68 | /^ *__*$/ && texmacs_chunking {
2068 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2069 Any other line when TeXmacs chunking is considered to be a line-wrapped line.
2071 64a <recognize-chunk[10](
\v) ⇑61a, lang=> +≡ ⊲63d 64b▿
2072 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2073 71 | texmacs_chunking {
2074 72 | if (! texmacs_chunk) {
2075 73 | # must be a texmacs continued line
2077 75 | texmacs_chunk=1;
2080 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2081 This final chunklet seems bogus and probably stops L Y X working.
2083 64b <recognize-chunk[11](
\v) ⇑61a, lang=> +≡ ▵64a 64c▿
2084 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2085 78 | ! texmacs_chunk {
2086 79 | # texmacs_chunking=0;
2089 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2091 We recognize notangle style chunks too:
2093 64c <recognize-chunk[12](
\v) ⇑61a, lang=awk> +≡ ▵64b 64d▿
2094 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2095 82 | /^[<]<.*[>]>=/ {
2096 83 | if (match($0, "^[<]<(.*)[>]>= *$", line)) {
2098 85 | notangle_mode=1;
2099 86 | new_chunk(line[1]);
2103 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2105 Likewise, we need to recognize when a chunk ends.
2107 The e in [e]nd{lislisting} is surrounded by square brackets so that when this document is processed, this chunk doesn't terminate early when the lstlistings package recognizes it's own end-string!4. This doesn't make sense as the regex is anchored with ^, which this line does not begin with! ^4
2109 64d <recognize-chunk[13](
\v) ⇑61a, lang=> +≡ ▵64c 65a⊳
2110 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2111 90 | /^\\[e]nd{lstlisting}|^\\[e]nd{Chunk}/ {
2113 92 | active_chunk="";
2116 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2119 65a <recognize-chunk[14](
\v) ⇑61a, lang=> +≡ ⊲64d 65b▿
2120 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2123 97 | active_chunk="";
2125 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2126 All other recognizers are only of effect if we are chunking; there's no point in looking at lines if they aren't part of a chunk, so we just ignore them as efficiently as we can.
2128 65b <recognize-chunk[15](
\v) ⇑61a, lang=> +≡ ▵65a 65c▿
2129 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2130 99 | ! chunking { next; }
2131 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2133 Chunk contents are any lines read while chunking is true. Some chunk contents are special in that they refer to other chunks, and will be replaced by the contents of these chunks when the file is generated.
2134 We add the output record separator ORS to the line now, because we will set ORS to the empty string when we generate the output5. So that we can partial print lines using print instead of printf.
2135 To do: This does't make sense
2138 65c <recognize-chunk[16](
\v) ⇑61a, lang=> +≡ ▵65b
2139 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2140 100 | length(active_chunk) {
2141 101 | «process-chunk-tabs 65e»
2142 102 | «process-chunk 66b»
2144 |________________________________________________________________________
2147 If a chunk just consisted of plain text, we could handle the chunk like this:
2149 65d <process-chunk-simple[1](
\v), lang=> ≡
2150 ________________________________________________________________________
2151 1 | chunk_line(active_chunk, $0 ORS);
2152 |________________________________________________________________________
2155 but in fact a chunk can include references to other chunks. Chunk includes are traditionally written as <<chunk-name>> but we support other variations, some of which are more suitable for particular editing systems.
2156 However, we also process tabs at this point. A tab at input can be replaced by a number of spaces defined by the tabs variable, set by the -T option. Of course this is poor tab behaviour, we should probably have the option to use proper counted tab-stops and process this on output.
2158 65e <process-chunk-tabs[1](
\v), lang=> ≡
2159 ________________________________________________________________________
2160 1 | if (length(tabs)) {
2161 2 | gsub("\t", tabs);
2163 |________________________________________________________________________
2167 If \lstset{escapeinside={=<}{>}} is set, then we can use <chunk-name ?> in listings. The sequence =< was chosen because:
2168 1.it is a better mnemonic than <<chunk-name>> in that the = sign signifies equivalence or substitutability.
2169 2.and because =< is not valid in C or any language I can think of.
2170 3.and also because lstlistings doesn't like >> as an end delimiter for the texcl escape, so we must make do with a single > which is better complemented by =< than by <<.
2171 Unfortunately the =<...> that we use re-enters a LaTeX parsing mode in which some characters are special, e.g. # \ and so these cause trouble if used in arguments to \chunkref. At some point I must fix the LaTeX command \chunkref so that it can accept these literally, but until then, when writing chunkref argumemts that need these characters, I must use the forms \textbackslash{} and \#; so I also define a hacky chunk delatex to be used further on whose purpose it is to remove these from any arguments parsed by fangle.
2173 66a <delatex[1](text
\v\v), lang=> ≡
2174 ________________________________________________________________________
2176 2 | gsub("\\\\#", "#", ${text});
2177 3 | gsub("\\\\textbackslash{}", "\\", ${text});
2178 4 | gsub("\\\\\\^", "^", ${text});
2179 |________________________________________________________________________
2182 As each chunk line may contain more than one chunk include, we will split out chunk includes in an iterative fashion6. Contrary to our use of split when substituting parameters in chapter ? ^6.
2183 First, as long as the chunk contains a \chunkref command we take as much as we can up to the first \chunkref command.
2184 TeXmacs text output uses ⟨...⟩ which comes out as unicode sequences 0xC2 0xAB ... 0xC2 0xBB. Modern awk will interpret [^\xC2\xBB] as a single unicode character if LANG is set correctly to the sub-type UTF-8, e.g. LANG=en_GB.UTF-8, otherwise [^\xC2\xBB] will be treated as a two character negated match — but this should not interfere with the function.
2186 66b <process-chunk[1](
\v), lang=> ≡ 66c▿
2187 ________________________________________________________________________
2190 3 | while(match(chunk,"(\xC2\xAB)([^\xC2\xBB]*) [^\xC2\xBB]*\xC2\xBB", line) ||
2192 5 | "([=]<\\\\chunkref{([^}>]*)}(\\(.*\\)|)>|<<([a-zA-Z_][-a-zA-Z0-9_]*)>>)",
2195 8 | chunklet = substr(chunk, 1, RSTART - 1);
2196 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2197 We keep track of the indent count, by counting the number of literal characters found. We can then preserve this indent on each output line when multi-line chunks are expanded.
2198 We then process this first part literal text, and set the chunk which is still to be processed to be the text after the \chunkref command, which we will process next as we continue around the loop.
2200 66c <process-chunk[2](
\v) ⇑66b, lang=> +≡ ▵66b 67a⊳
2201 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2202 9 | indent += length(chunklet);
2203 10 | chunk_line(active_chunk, chunklet);
2204 11 | chunk = substr(chunk, RSTART + RLENGTH);
2205 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2206 We then consider the type of chunk command we have found, whether it is the fangle style command beginning with =< the older notangle style beginning with <<.
2207 Fangle chunks may have parameters contained within square brackets. These will be matched in line[3] and are considered at this stage of processing to be part of the name of the chunk to be included.
2209 67a <process-chunk[3](
\v) ⇑66b, lang=> +≡ ⊲66c 67b▿
2210 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2211 12 | if (substr(line[1], 1, 1) == "=") {
2212 13 | # chunk name up to }
2213 14 | «delatex
\v(line[3]
\v) 66a»
2214 15 | chunk_include(active_chunk, line[2] line[3], indent);
2215 16 | } else if (substr(line[1], 1, 1) == "<") {
2216 17 | chunk_include(active_chunk, line[4], indent);
2217 18 | } else if (line[1] == "\xC2\xAB") {
2218 19 | chunk_include(active_chunk, line[2], indent);
2220 21 | error("Unknown chunk fragment: " line[1]);
2222 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2223 The loop will continue until there are no more chunkref statements in the text, at which point we process the final part of the chunk.
2225 67b <process-chunk[4](
\v) ⇑66b, lang=> +≡ ▵67a 67c▿
2226 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2228 24 | chunk_line(active_chunk, chunk);
2229 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2230 We add the newline character as a chunklet on it's own, to make it easier to detect new lines and thus manage indentation when processing the output.
2232 67c <process-chunk[5](
\v) ⇑66b, lang=> +≡ ▵67b
2233 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2234 25 | chunk_line(active_chunk, "\n");
2235 |________________________________________________________________________
2238 We will also permit a chunk-part number to follow in square brackets, so that <chunk-name[1] ?> will refer to the first part only. This can make it easy to include a C function prototype in a header file, if the first part of the chunk is just the function prototype without the trailing semi-colon. The header file would include the prototype with the trailing semi-colon, like this:
2240 This is handled in section 13.1.1
2241 We should perhaps introduce a notion of language specific chunk options; so that perhaps we could specify:
2242 =<\chunkref{chunk-name[function-declaration]}
2243 which applies a transform function-declaration to the chunk --- which in this case would extract a function prototype from a function.
2246 Chapter 12Processing Options
2247 At the start, first we set the default options.
2249 69a <default-options[1](
\v), lang=> ≡
2250 ________________________________________________________________________
2253 3 | notangle_mode=0;
2256 |________________________________________________________________________
2259 Then we use getopt the standard way, and null out ARGV afterwards in the normal AWK fashion.
2261 69b <read-options[1](
\v), lang=> ≡
2262 ________________________________________________________________________
2263 1 | Optind = 1 # skip ARGV[0]
2264 2 | while(getopt(ARGC, ARGV, "R:LdT:hr")!=-1) {
2265 3 | «handle-options 69c»
2267 5 | for (i=1; i<Optind; i++) { ARGV[i]=""; }
2268 |________________________________________________________________________
2271 This is how we handle our options:
2273 69c <handle-options[1](
\v), lang=> ≡
2274 ________________________________________________________________________
2275 1 | if (Optopt == "R") root = Optarg;
2276 2 | else if (Optopt == "r") root="";
2277 3 | else if (Optopt == "L") linenos = 1;
2278 4 | else if (Optopt == "d") debug = 1;
2279 5 | else if (Optopt == "T") tabs = indent_string(Optarg+0);
2280 6 | else if (Optopt == "h") help();
2281 7 | else if (Optopt == "?") help();
2282 |________________________________________________________________________
2285 We do all of this at the beginning of the program
2287 69d <begin[1](
\v), lang=> ≡
2288 ________________________________________________________________________
2291 3 | «mode-definitions 51c»
2292 4 | «default-options 69a»
2294 6 | «read-options 69b»
2296 |________________________________________________________________________
2299 And have a simple help function
2301 69e <help()[1](
\v), lang=> ≡
2302 ________________________________________________________________________
2303 1 | function help() {
2305 3 | print " fangle [-L] -R<rootname> [source.tex ...]"
2306 4 | print " fangle -r [source.tex ...]"
2307 5 | print " If the filename, source.tex is not specified then stdin is used"
2309 7 | print "-L causes the C statement: #line <lineno> \"filename\"" to be issued"
2310 8 | print "-R causes the named root to be written to stdout"
2311 9 | print "-r lists all roots in the file (even those used elsewhere)"
2314 |________________________________________________________________________
2317 Chapter 13Generating the Output
2318 We generate output by calling output_chunk, or listing the chunk names.
2320 71a <generate-output[1](
\v), lang=> ≡
2321 ________________________________________________________________________
2322 1 | if (length(root)) output_chunk(root);
2323 2 | else output_chunk_names();
2324 |________________________________________________________________________
2327 We also have some other output debugging:
2329 71b <debug-output[1](
\v), lang=> ≡
2330 ________________________________________________________________________
2332 2 | print "------ chunk names "
2333 3 | output_chunk_names();
2334 4 | print "====== chunks"
2335 5 | output_chunks();
2336 6 | print "++++++ debug"
2337 7 | for (a in chunks) {
2338 8 | print a "=" chunks[a];
2341 |________________________________________________________________________
2344 We do both of these at the end. We also set ORS="" because each chunklet is not necessarily a complete line, and we already added ORS to each input line in section 11.4.
2346 71c <end[1](
\v), lang=> ≡
2347 ________________________________________________________________________
2349 2 | «debug-output 71b»
2351 4 | «generate-output 71a»
2353 |________________________________________________________________________
2356 We write chunk names like this. If we seem to be running in notangle compatibility mode, then we enclose the name like this <<name>> the same way notangle does:
2358 71d <output_chunk_names()[1](
\v), lang=> ≡
2359 ________________________________________________________________________
2360 1 | function output_chunk_names( c, prefix, suffix)
2362 3 | if (notangle_mode) {
2366 7 | for (c in chunk_names) {
2367 8 | print prefix c suffix "\n";
2370 |________________________________________________________________________
2373 This function would write out all chunks
2375 71e <output_chunks()[1](
\v), lang=> ≡
2376 ________________________________________________________________________
2377 1 | function output_chunks( a)
2379 3 | for (a in chunk_names) {
2380 4 | output_chunk(a);
2384 8 | function output_chunk(chunk) {
2386 10 | lineno_needed = linenos;
2388 12 | write_chunk(chunk);
2391 |________________________________________________________________________
2394 13.1 Assembling the Chunks
2395 chunk_path holds a string consisting of the names of all the chunks that resulted in this chunk being output. It should probably also contain the source line numbers at which each inclusion also occured.
2396 We first initialize the mode tracker for this chunk.
2398 72a <write_chunk()[1](
\v), lang=awk> ≡ 72b▿
2399 ________________________________________________________________________
2400 1 | function write_chunk(chunk_name) {
2401 2 | «awk-delete-array
\v(context
\v) 35d»
2402 3 | return write_chunk_r(chunk_name, context);
2405 6 | function write_chunk_r(chunk_name, context, indent, tail,
2407 8 | chunk_path, chunk_args,
2409 10 | context_origin,
2410 11 | chunk_params, part, max_part, part_line, frag, max_frag, text,
2411 12 | chunklet, only_part, call_chunk_args, new_context)
2413 14 | if (debug) debug_log("write_chunk_r(" chunk_name ")");
2414 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2416 As mentioned in section ?, a chunk name may contain a part specifier in square brackets, limiting the parts that should be emitted.
2418 72b <write_chunk()[2](
\v) ⇑72a, lang=> +≡ ▵72a 72c▿
2419 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2420 15 | if (match(chunk_name, "^(.*)\\[([0-9]*)\\]$", chunk_name_parts)) {
2421 16 | chunk_name = chunk_name_parts[1];
2422 17 | only_part = chunk_name_parts[2];
2424 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2425 We then create a mode tracker
2427 72c <write_chunk()[3](
\v) ⇑72a, lang=> +≡ ▵72b 73a⊳
2428 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2429 19 | context_origin = push_mode_tracker(context, chunks[chunk_name, "language"], "");
2430 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2431 We extract into chunk_params the names of the parameters that this chunk accepts, whose values were (optionally) passed in chunk_args.
2433 73a <write_chunk()[4](
\v) ⇑72a, lang=> +≡ ⊲72c 73b▿
2434 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2435 20 | split(chunks[chunk_name, "params"], chunk_params, " *; *");
2436 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2437 To assemble a chunk, we write out each part.
2439 73b <write_chunk()[5](
\v) ⇑72a, lang=> +≡ ▵73a
2440 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2441 21 | if (! (chunk_name in chunk_names)) {
2442 22 | error(sprintf(_"The root module <<%s>> was not defined.\nUsed by: %s",\
2443 23 | chunk_name, chunk_path));
2446 26 | max_part = chunks[chunk_name, "part"];
2447 27 | for(part = 1; part <= max_part; part++) {
2448 28 | if (! only_part || part == only_part) {
2449 29 | «write-part 73c»
2452 32 | if (! pop_mode_tracker(context, context_origin)) {
2453 33 | dump_mode_tracker(context);
2454 34 | error(sprintf(_"Module %s did not close context properly.\nUsed by: %s\n", chunk_name, chunk_path));
2457 |________________________________________________________________________
2460 A part can either be a chunklet of lines, or an include of another chunk.
2461 Chunks may also have parameters, specified in LaTeX style with braces after the chunk name --- looking like this in the document: chunkname{param1, param2}. Arguments are passed in square brackets: \chunkref{chunkname}[arg1, arg2].
2462 Before we process each part, we check that the source position hasn't changed unexpectedly, so that we can know if we need to output a new file-line directive.
2464 73c <write-part[1](
\v), lang=> ≡
2465 ________________________________________________________________________
2466 1 | «check-source-jump 75d»
2468 3 | chunklet = chunks[chunk_name, "part", part];
2469 4 | if (chunks[chunk_name, "part", part, "type"] == part_type_chunk) {
2470 5 | «write-included-chunk 73d»
2471 6 | } else if (chunklet SUBSEP "line" in chunks) {
2472 7 | «write-chunklets 74a»
2474 9 | # empty last chunklet
2476 |________________________________________________________________________
2479 To write an included chunk, we must detect any optional chunk arguments in parenthesis. Then we recurse calling write_chunk().
2481 73d <write-included-chunk[1](
\v), lang=> ≡
2482 ________________________________________________________________________
2483 1 | if (match(chunklet, "^([^\\[\\(]*)\\((.*)\\)$", chunklet_parts)) {
2484 2 | chunklet = chunklet_parts[1];
2486 4 | gsub(sprintf("%c",11), "", chunklet);
2487 5 | gsub(sprintf("%c",11), "", chunklet_parts[2]);
2488 6 | parse_chunk_args("c-like", chunklet_parts[2], call_chunk_args, "(");
2489 7 | for (c in call_chunk_args) {
2490 8 | call_chunk_args[c] = expand_chunk_args(call_chunk_args[c], chunk_params, chunk_args);
2493 11 | split("", call_chunk_args);
2496 14 | write_chunk_r(chunklet, context,
2497 15 | chunks[chunk_name, "part", part, "indent"] indent,
2498 16 | chunks[chunk_name, "part", part, "tail"],
2499 17 | chunk_path "\n " chunk_name,
2500 18 | call_chunk_args);
2501 |________________________________________________________________________
2504 Before we output a chunklet of lines, we first emit the file and line number if we have one, and if it is safe to do so.
2505 Chunklets are generally broken up by includes, so the start of a chunklet is a good place to do this. Then we output each line of the chunklet.
2506 When it is not safe, such as in the middle of a multi-line macro definition, lineno_suppressed is set to true, and in such a case we note that we want to emit the line statement when it is next safe.
2508 74a <write-chunklets[1](
\v), lang=> ≡ 74b▿
2509 ________________________________________________________________________
2510 1 | max_frag = chunks[chunklet, "line"];
2511 2 | for(frag = 1; frag <= max_frag; frag++) {
2512 3 | «write-file-line 75c»
2513 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2514 We then extract the chunklet text and expand any arguments.
2516 74b <write-chunklets[2](
\v) ⇑74a, lang=> +≡ ▵74a 74c▿
2517 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2519 5 | text = chunks[chunklet, frag];
2521 7 | /* check params */
2522 8 | text = expand_chunk_args(text, chunk_params, chunk_args);
2523 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2524 If the text is a single newline (which we keep separate - see 6) then we increment the line number. In the case where this is the last line of a chunk and it is not a top-level chunk we replace the newline with an empty string --- because the chunk that included this chunk will have the newline at the end of the line that included this chunk.
2525 We also note by newline = 1 that we have started a new line, so that indentation can be managed with the following piece of text.
2527 74c <write-chunklets[3](
\v) ⇑74a, lang=> +≡ ▵74b 74d▿
2528 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2530 10 | if (text == "\n") {
2532 12 | if (part == max_part && frag == max_frag && length(chunk_path)) {
2538 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2539 If this text does not represent a newline, but we see that we are the first piece of text on a newline, then we prefix our text with the current indent.
2540 Note 1. newline is a global output-state variable, but the indent is not.
2542 74d <write-chunklets[4](
\v) ⇑74a, lang=> +≡ ▵74c 75a⊳
2543 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2544 18 | } else if (length(text) || length(tail)) {
2545 19 | if (newline) text = indent text;
2549 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2550 Tail will soon no longer be relevant once mode-detection is in place.
2552 75a <write-chunklets[5](
\v) ⇑74a, lang=> +≡ ⊲74d 75b▿
2553 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2554 23 | text = text tail;
2555 24 | mode_tracker(context, text);
2556 25 | print untab(transform_escape(context, text, context_origin));
2557 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2558 If a line ends in a backslash --- suggesting continuation --- then we supress outputting file-line as it would probably break the continued lines.
2560 75b <write-chunklets[6](
\v) ⇑74a, lang=> +≡ ▵75a
2561 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2563 27 | lineno_suppressed = substr(lastline, length(lastline)) == "\\";
2566 |________________________________________________________________________
2569 Of course there is no point in actually outputting the source filename and line number (file-line) if they don't say anything new! We only need to emit them if they aren't what is expected, or if we we not able to emit one when they had changed.
2571 75c <write-file-line[1](
\v), lang=> ≡
2572 ________________________________________________________________________
2573 1 | if (newline && lineno_needed && ! lineno_suppressed) {
2574 2 | filename = a_filename;
2575 3 | lineno = a_lineno;
2576 4 | print "#line " lineno " \"" filename "\"\n"
2577 5 | lineno_needed = 0;
2579 |________________________________________________________________________
2582 We check if a new file-line is needed by checking if the source line matches what we (or a compiler) would expect.
2584 75d <check-source-jump[1](
\v), lang=> ≡
2585 ________________________________________________________________________
2586 1 | if (linenos && (chunk_name SUBSEP "part" SUBSEP part SUBSEP "FILENAME" in chunks)) {
2587 2 | a_filename = chunks[chunk_name, "part", part, "FILENAME"];
2588 3 | a_lineno = chunks[chunk_name, "part", part, "LINENO"];
2589 4 | if (a_filename != filename || a_lineno != lineno) {
2590 5 | lineno_needed++;
2593 |________________________________________________________________________
2596 Chapter 14Storing Chunks
2597 Awk has pretty limited data structures, so we will use two main hashes. Uninterrupted sequences of a chunk will be stored in chunklets and the chunklets used in a chunk will be stored in chunks.
2599 77a <constants[2](
\v) ⇑37a, lang=> +≡ ⊲37a
2600 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2601 2 | part_type_chunk=1;
2603 |________________________________________________________________________
2606 The params mentioned are not chunk parameters for parameterized chunks, as mentioned in 9.2, but the lstlistings style parameters used in the \Chunk command1. The params parameter is used to hold the parameters for parameterized chunks ^1.
2608 77b <chunk-storage-functions[1](
\v), lang=> ≡ 77c▿
2609 ________________________________________________________________________
2610 1 | function new_chunk(chunk_name, opts, args,
2614 5 | # HACK WHILE WE CHANGE TO ( ) for PARAM CHUNKS
2615 6 | gsub("\\(\\)$", "", chunk_name);
2616 7 | if (! (chunk_name in chunk_names)) {
2617 8 | if (debug) print "New chunk " chunk_name;
2618 9 | chunk_names[chunk_name];
2619 10 | for (p in opts) {
2620 11 | chunks[chunk_name, p] = opts[p];
2621 12 | if (debug) print "chunks[" chunk_name "," p "] = " opts[p];
2623 14 | for (p in args) {
2624 15 | chunks[chunk_name, "params", p] = args[p];
2626 17 | if ("append" in opts) {
2627 18 | append=opts["append"];
2628 19 | if (! (append in chunk_names)) {
2629 20 | warning("Chunk " chunk_name " is appended to chunk " append " which is not defined yet");
2630 21 | new_chunk(append);
2632 23 | chunk_include(append, chunk_name);
2633 24 | chunk_line(append, ORS);
2636 27 | active_chunk = chunk_name;
2637 28 | prime_chunk(chunk_name);
2639 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2641 77c <chunk-storage-functions[2](
\v) ⇑77b, lang=> +≡ ▵77b 78a⊳
2642 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2644 31 | function prime_chunk(chunk_name)
2646 33 | chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] = \
2647 34 | chunk_name SUBSEP "chunklet" SUBSEP "" ++chunks[chunk_name, "chunklet"];
2648 35 | chunks[chunk_name, "part", chunks[chunk_name, "part"], "FILENAME"] = FILENAME;
2649 36 | chunks[chunk_name, "part", chunks[chunk_name, "part"], "LINENO"] = FNR + 1;
2652 39 | function chunk_line(chunk_name, line){
2653 40 | chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
2654 41 | ++chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"], "line"] ] = line;
2657 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2658 Chunk include represents a chunkref statement, and stores the requirement to include another chunk. The parameter indent represents the quanity of literal text characters that preceded this chunkref statement and therefore by how much additional lines of the included chunk should be indented.
2660 78a <chunk-storage-functions[3](
\v) ⇑77b, lang=> +≡ ⊲77c 78b▿
2661 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2662 44 | function chunk_include(chunk_name, chunk_ref, indent, tail)
2664 46 | chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] = chunk_ref;
2665 47 | chunks[chunk_name, "part", chunks[chunk_name, "part"], "type" ] = part_type_chunk;
2666 48 | chunks[chunk_name, "part", chunks[chunk_name, "part"], "indent" ] = indent_string(indent);
2667 49 | chunks[chunk_name, "part", chunks[chunk_name, "part"], "tail" ] = tail;
2668 50 | prime_chunk(chunk_name);
2671 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2672 The indent is calculated by indent_string, which may in future convert some spaces into tab characters. This function works by generating a printf padded format string, like %22s for an indent of 22, and then printing an empty string using that format.
2674 78b <chunk-storage-functions[4](
\v) ⇑77b, lang=> +≡ ▵78a
2675 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2676 53 | function indent_string(indent) {
2677 54 | return sprintf("%" indent "s", "");
2679 |________________________________________________________________________
2683 I use Arnold Robbins public domain getopt (1993 revision). This is probably the same one that is covered in chapter 12 of âĂIJEdition 3 of GAWK: Effective AWK Programming: A User's Guide for GNU AwkâĂİ but as that is licensed under the GNU Free Documentation License, Version 1.3, which conflicts with the GPL3, I can't use it from there (or it's accompanying explanations), so I do my best to explain how it works here.
2684 The getopt.awk header is:
2686 79a <getopt.awk-header[1](
\v), lang=> ≡
2687 ________________________________________________________________________
2688 1 | # getopt.awk --- do C library getopt(3) function in awk
2690 3 | # Arnold Robbins, arnold@skeeve.com, Public Domain
2692 5 | # Initial version: March, 1991
2693 6 | # Revised: May, 1993
2695 |________________________________________________________________________
2698 The provided explanation is:
2700 79b <getopt.awk-notes[1](
\v), lang=> ≡
2701 ________________________________________________________________________
2702 1 | # External variables:
2703 2 | # Optind -- index in ARGV of first nonoption argument
2704 3 | # Optarg -- string value of argument to current option
2705 4 | # Opterr -- if nonzero, print our own diagnostic
2706 5 | # Optopt -- current option letter
2709 8 | # -1 at end of options
2710 9 | # ? for unrecognized option
2711 10 | # <c> a character representing the current option
2713 12 | # Private Data:
2714 13 | # _opti -- index in multi-flag option, e.g., -abc
2716 |________________________________________________________________________
2719 The function follows. The final two parameters, thisopt and i are local variables and not parameters --- as indicated by the multiple spaces preceding them. Awk doesn't care, the multiple spaces are a convention to help us humans.
2721 79c <getopt.awk-getopt()[1](
\v), lang=> ≡ 80a⊳
2722 ________________________________________________________________________
2723 1 | function getopt(argc, argv, options, thisopt, i)
2725 3 | if (length(options) == 0) # no options given
2727 5 | if (argv[Optind] == "--") { # all done
2731 9 | } else if (argv[Optind] !~ /^-[^: \t\n\f\r\v\b]/) {
2735 13 | if (_opti == 0)
2737 15 | thisopt = substr(argv[Optind], _opti, 1)
2738 16 | Optopt = thisopt
2739 17 | i = index(options, thisopt)
2742 20 | printf("%c -- invalid option\n",
2743 21 | thisopt) > "/dev/stderr"
2744 22 | if (_opti >= length(argv[Optind])) {
2751 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2752 At this point, the option has been found and we need to know if it takes any arguments.
2754 80a <getopt.awk-getopt()[2](
\v) ⇑79c, lang=> +≡ ⊲79c
2755 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2756 29 | if (substr(options, i + 1, 1) == ":") {
2757 30 | # get option argument
2758 31 | if (length(substr(argv[Optind], _opti + 1)) > 0)
2759 32 | Optarg = substr(argv[Optind], _opti + 1)
2761 34 | Optarg = argv[++Optind]
2765 38 | if (_opti == 0 || _opti >= length(argv[Optind])) {
2772 |________________________________________________________________________
2775 A test program is built in, too
2777 80b <getopt.awk-begin[1](
\v), lang=> ≡
2778 ________________________________________________________________________
2780 2 | Opterr = 1 # default is to diagnose
2781 3 | Optind = 1 # skip ARGV[0]
2783 5 | if (_getopt_test) {
2784 6 | while ((_go_c = getopt(ARGC, ARGV, "ab:cd")) != -1)
2785 7 | printf("c = <%c>, optarg = <%s>\n",
2787 9 | printf("non-option arguments:\n")
2788 10 | for (; Optind < ARGC; Optind++)
2789 11 | printf("\tARGV[%d] = <%s>\n",
2790 12 | Optind, ARGV[Optind])
2793 |________________________________________________________________________
2796 The entire getopt.awk is made out of these chunks in order
2798 80c <getopt.awk[1](
\v), lang=> ≡
2799 ________________________________________________________________________
2800 1 | «getopt.awk-header 79a»
2802 3 | «getopt.awk-notes 79b»
2803 4 | «getopt.awk-getopt() 79c»
2804 5 | «getopt.awk-begin 80b»
2805 |________________________________________________________________________
2808 Although we only want the header and function:
2810 81a <getopt[1](
\v), lang=> ≡
2811 ________________________________________________________________________
2812 1 | # try: locate getopt.awk for the full original file
2813 2 | # as part of your standard awk installation
2814 3 | «getopt.awk-header 79a»
2816 5 | «getopt.awk-getopt() 79c»
2817 |________________________________________________________________________
2820 Chapter 16Fangle LaTeX source code
2822 Here we define a L Y X .module file that makes it convenient to use L Y X for writing such literate programs.
2823 This file ./fangle.module can be installed in your personal .lyx/layouts folder. You will need to Tools Reconfigure so that L Y X notices it. It adds a new format Chunk, which should precede every listing and contain the chunk name.
2825 83a <./fangle.module[1](
\v), lang=lyx-module> ≡
2826 ________________________________________________________________________
2827 1 | #\DeclareLyXModule{Fangle Literate Listings}
2828 2 | #DescriptionBegin
2829 3 | # Fangle literate listings allow one to write
2830 4 | # literate programs after the fashion of noweb, but without having
2831 5 | # to use noweave to generate the documentation. Instead the listings
2832 6 | # package is extended in conjunction with the noweb package to implement
2833 7 | # to code formating directly as latex.
2834 8 | # The fangle awk script
2837 11 | «gpl3-copyright.hashed 83b»
2842 16 | «./fangle.sty 84d»
2845 19 | «chunkstyle 84a»
2848 |________________________________________________________________________
2851 Because L Y X modules are not yet a language supported by fangle or lstlistings, we resort to this fake awk chunk below in order to have each line of the GPL3 license commence with a #
2853 83b <gpl3-copyright.hashed[1](
\v), lang=awk> ≡
2854 ________________________________________________________________________
2855 1 | #«gpl3-copyright 4a»
2857 |________________________________________________________________________
2860 16.1.1 The Chunk style
2861 The purpose of the chunk style is to make it easier for L Y X users to provide the name to lstlistings. Normally this requires right-clicking on the listing, choosing settings, advanced, and then typing name=chunk-name. This has the further disadvantage that the name (and other options) are not generally visible during document editing.
2862 The chunk style is defined as a LaTeX command, so that all text on the same line is passed to the LaTeX command Chunk. This makes it easy to parse using fangle, and easy to pass these options on to the listings package. The first word in a chunk section should be the chunk name, and will have name= prepended to it. Any other words are accepted arguments to lstset.
2863 We set PassThru to 1 because the user is actually entering raw latex.
2865 84a <chunkstyle[1](
\v), lang=> ≡ 84b▿
2866 ________________________________________________________________________
2868 2 | LatexType Command
2870 4 | Margin First_Dynamic
2871 5 | LeftMargin Chunk:xxx
2873 7 | LabelType Static
2874 8 | LabelString "Chunk:"
2878 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2879 To make the label very visible we choose a larger font coloured red.
2881 84b <chunkstyle[2](
\v) ⇑84a, lang=> +≡ ▵84a
2882 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2891 |________________________________________________________________________
2894 16.1.2 The chunkref style
2895 We also define the Chunkref style which can be used to express cross references to chunks.
2897 84c <chunkref[1](
\v), lang=> ≡
2898 ________________________________________________________________________
2899 1 | InsetLayout Chunkref
2900 2 | LyxType charstyle
2901 3 | LatexType Command
2902 4 | LatexName chunkref
2909 |________________________________________________________________________
2913 We require the listings, noweb and xargs packages. As noweb defines it's own \code environment, we re-define the one that L Y X logical markup module expects here.
2915 84d <./fangle.sty[1](
\v), lang=tex> ≡ 85a⊳
2916 ________________________________________________________________________
2917 1 | \usepackage{listings}%
2918 2 | \usepackage{noweb}%
2919 3 | \usepackage{xargs}%
2920 4 | \renewcommand{\code}[1]{\texttt{#1}}%
2921 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2922 We also define a CChunk macro, for use as: \begin{CChunk} which will need renaming to \begin{Chunk} when I can do this without clashing with \Chunk.
2924 85a <./fangle.sty[2](
\v) ⇑84d, lang=> +≡ ⊲84d 85b▿
2925 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2926 5 | \lstnewenvironment{Chunk}{\relax}{\relax}%
2927 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2928 We also define a suitable \lstset of parameters that suit the literate programming style after the fashion of noweave.
2930 85b <./fangle.sty[3](
\v) ⇑84d, lang=> +≡ ▵85a 85c▿
2931 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2932 6 | \lstset{numbers=left, stepnumber=5, numbersep=5pt,
2933 7 | breaklines=false,basicstyle=\ttfamily,
2934 8 | numberstyle=\tiny, language=C}%
2935 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2936 We also define a notangle-like mechanism for escaping to LaTeX from the listing, and by which we can refer to other listings. We declare the =<...> sequence to contain LaTeX code, and include another like this chunk: <chunkname ?>. However, because =<...> is already defined to contain LaTeX code for this document --- this is a fangle document after all --- the code fragment below effectively contains the LaTeX code: }{. To avoid problems with document generation, I had to declare an lstlistings property: escapeinside={} for this listing only; which in L Y X was done by right-clicking the listings inset, choosing settings->advanced. Therefore =< isn't interpreted literally here, in a listing when the escape sequence is already defined as shown... we need to somehow escape this representation...
2938 85c <./fangle.sty[4](
\v) ⇑84d, lang=> +≡ ▵85b 85d▿
2939 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2940 9 | \lstset{escapeinside={=<}{>}}%
2941 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2942 Although our macros will contain the @ symbol, they will be included in a \makeatletter section by L Y X; however we keep the commented out \makeatletter as a reminder. The listings package likes to centre the titles, but noweb titles are specially formatted and must be left aligned. The simplest way to do this turned out to be by removing the definition of \lst@maketitle. This may interact badly if other listings want a regular title or caption. We remember the old maketitle in case we need it.
2944 85d <./fangle.sty[5](
\v) ⇑84d, lang=> +≡ ▵85c 85e▿
2945 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2947 11 | %somehow re-defining maketitle gives us a left-aligned title
2948 12 | %which is extactly what our specially formatted title needs!
2949 13 | \global\let\fangle@lst@maketitle\lst@maketitle%
2950 14 | \global\def\lst@maketitle{}%
2951 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2952 16.2.1 The chunk command
2953 Our chunk command accepts one argument, and calls \ltset. Although \ltset will note the name, this is erased when the next \lstlisting starts, so we make a note of this in \lst@chunkname and restore in in lstlistings Init hook.
2955 85e <./fangle.sty[6](
\v) ⇑84d, lang=> +≡ ▵85d 86a⊳
2956 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2958 16 | \lstset{title={\fanglecaption},name=#1}%
2959 17 | \global\edef\lst@chunkname{\lst@intname}%
2961 19 | \def\lst@chunkname{\empty}%
2962 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2963 16.2.1.1 Chunk parameters
2964 Fangle permits parameterized chunks, and requires the paramters to be specified as listings options. The fangle script uses this, and although we don't do anything with these in the LaTeX code right now, we need to stop the listings package complaining.
2966 86a <./fangle.sty[7](
\v) ⇑84d, lang=> +≡ ⊲85e 86b▿
2967 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2968 20 | \lst@Key{params}\relax{\def\fangle@chunk@params{#1}}%
2969 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2970 As it is common to define a chunk which then needs appending to another chunk, and annoying to have to declare a single line chunk to manage the include, we support an append= option.
2972 86b <./fangle.sty[8](
\v) ⇑84d, lang=> +≡ ▵86a 86c▿
2973 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2974 21 | \lst@Key{append}\relax{\def\fangle@chunk@append{#1}}%
2975 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2976 16.2.2 The noweb styled caption
2977 We define a public macro \fanglecaption which can be set as a regular title. By means of \protect, It expands to \fangle@caption at the appopriate time when the caption is emitted.
2979 86c <./fangle.sty[9](
\v) ⇑84d, lang=> +≡ ▵86b 86d▿
2980 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2981 \def\fanglecaption{\protect\fangle@caption}%
2982 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
2983 22c ⟨some-chunk 19b⟩≡+ ⊲22b 24d⊳
2985 In this example, the current chunk is 22c, and therefore the third chunk on page 22.
2986 It's name is some-chunk.
2987 The first chunk with this name (19b) occurs as the second chunk on page 19.
2988 The previous chunk (22d) with the same name is the second chunk on page 22.
2989 The next chunk (24d) is the fourth chunk on page 24.
2991 Figure 1. Noweb Heading
2993 The general noweb output format compactly identifies the current chunk, and references to the first chunk, and the previous and next chunks that have the same name.
2994 This means that we need to keep a counter for each chunk-name, that we use to count chunks of the same name.
2995 16.2.3 The chunk counter
2996 It would be natural to have a counter for each chunk name, but TeX would soon run out of counters1. ...soon did run out of counters and so I had to re-write the LaTeX macros to share a counter as described here. ^1, so we have one counter which we save at the end of a chunk and restore at the beginning of a chunk.
2998 86d <./fangle.sty[10](
\v) ⇑84d, lang=> +≡ ▵86c 87c⊳
2999 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3000 22 | \newcounter{fangle@chunkcounter}%
3001 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3002 We construct the name of this variable to store the counter to be the text lst-chunk- prefixed onto the chunks own name, and store it in \chunkcount.
3003 We save the counter like this:
3005 87a <save-counter[1](
\v), lang=> ≡
3006 ________________________________________________________________________
3007 \global\expandafter\edef\csname \chunkcount\endcsname{\arabic{fangle@chunkcounter}}%
3008 |________________________________________________________________________
3011 and restore the counter like this:
3013 87b <restore-counter[1](
\v), lang=> ≡
3014 ________________________________________________________________________
3015 \setcounter{fangle@chunkcounter}{\csname \chunkcount\endcsname}%
3016 |________________________________________________________________________
3019 If there does not already exist a variable whose name is stored in \chunkcount, then we know we are the first chunk with this name, and then define a counter.
3020 Although chunks of the same name share a common counter, they must still be distinguished. We use is the internal name of the listing, suffixed by the counter value. So the first chunk might be something-1 and the second chunk be something-2, etc.
3021 We also calculate the name of the previous chunk if we can (before we increment the chunk counter). If this is the first chunk of that name, then \prevchunkname is set to \relax which the noweb package will interpret as not existing.
3023 87c <./fangle.sty[11](
\v) ⇑84d, lang=> +≡ ⊲86d 87d▿
3024 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3025 23 | \def\fangle@caption{%
3026 24 | \edef\chunkcount{lst-chunk-\lst@intname}%
3027 25 | \@ifundefined{\chunkcount}{%
3028 26 | \expandafter\gdef\csname \chunkcount\endcsname{0}%
3029 27 | \setcounter{fangle@chunkcounter}{\csname \chunkcount\endcsname}%
3030 28 | \let\prevchunkname\relax%
3032 30 | \setcounter{fangle@chunkcounter}{\csname \chunkcount\endcsname}%
3033 31 | \edef\prevchunkname{\lst@intname-\arabic{fangle@chunkcounter}}%
3035 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3036 After incrementing the chunk counter, we then define the name of this chunk, as well as the name of the first chunk.
3038 87d <./fangle.sty[12](
\v) ⇑84d, lang=> +≡ ▵87c 87e▿
3039 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3040 33 | \addtocounter{fangle@chunkcounter}{1}%
3041 34 | \global\expandafter\edef\csname \chunkcount\endcsname{\arabic{fangle@chunkcounter}}%
3042 35 | \edef\chunkname{\lst@intname-\arabic{fangle@chunkcounter}}%
3043 36 | \edef\firstchunkname{\lst@intname-1}%
3044 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3045 We now need to calculate the name of the next chunk. We do this by temporarily skipping the counter on by one; however there may not actually be another chunk with this name! We detect this by also defining a label for each chunk based on the chunkname. If there is a next chunkname then it will define a label with that name. As labels are persistent, we can at least tell the second time LaTeX is run. If we don't find such a defined label then we define \nextchunkname to \relax.
3047 87e <./fangle.sty[13](
\v) ⇑84d, lang=> +≡ ▵87d 88a⊳
3048 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3049 37 | \addtocounter{fangle@chunkcounter}{1}%
3050 38 | \edef\nextchunkname{\lst@intname-\arabic{fangle@chunkcounter}}%
3051 39 | \@ifundefined{r@label-\nextchunkname}{\let\nextchunkname\relax}{}%
3052 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3053 The noweb package requires that we define a \sublabel for every chunk, with a unique name, which is then used to print out it's navigation hints.
3054 We also define a regular label for this chunk, as was mentioned above when we calculated \nextchunkname. This requires LaTeX to be run at least twice after new chunk sections are added --- but noweb requried that anyway.
3056 88a <./fangle.sty[14](
\v) ⇑84d, lang=> +≡ ⊲87e 88b▿
3057 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3058 40 | \sublabel{\chunkname}%
3059 41 | % define this label for every chunk instance, so we
3060 42 | % can tell when we are the last chunk of this name
3061 43 | \label{label-\chunkname}%
3062 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3063 We also try and add the chunk to the list of listings, but I'm afraid we don't do very well. We want each chunk name listing once, with all of it's references.
3065 88b <./fangle.sty[15](
\v) ⇑84d, lang=> +≡ ▵88a 88c▿
3066 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3067 44 | \addcontentsline{lol}{lstlisting}{\lst@name~[\protect\subpageref{\chunkname}]}%
3068 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3069 We then call the noweb output macros in the same way that noweave generates them, except that we don't need to call \nwstartdeflinemarkup or \nwenddeflinemarkup — and if we do, it messes up the output somewhat.
3071 88c <./fangle.sty[16](
\v) ⇑84d, lang=> +≡ ▵88b 88d▿
3072 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3076 48 | \subpageref{\chunkname}%
3083 55 | \nwtagstyle{}\/%
3084 56 | \@ifundefined{fangle@chunk@params}{}{%
3085 57 | (\fangle@chunk@params)%
3087 59 | [\csname \chunkcount\endcsname]~%
3088 60 | \subpageref{\firstchunkname}%
3090 62 | \@ifundefined{fangle@chunk@append}{}{%
3091 63 | \ifx{}\fangle@chunk@append{x}\else%
3092 64 | ,~add~to~\fangle@chunk@append%
3095 67 | \global\def\fangle@chunk@append{}%
3096 68 | \lstset{append=x}%
3099 71 | \ifx\relax\prevchunkname\endmoddef\else\plusendmoddef\fi%
3100 72 | % \nwstartdeflinemarkup%
3101 73 | \nwprevnextdefs{\prevchunkname}{\nextchunkname}%
3102 74 | % \nwenddeflinemarkup%
3104 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3105 Originally this was developed as a listings aspect, in the Init hook, but it was found easier to affect the title without using a hook — \lst@AddToHookExe{PreSet} is still required to set the listings name to the name passed to the \Chunk command, though.
3107 88d <./fangle.sty[17](
\v) ⇑84d, lang=> +≡ ▵88c 89a⊳
3108 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3109 76 | %\lst@BeginAspect{fangle}
3110 77 | %\lst@Key{fangle}{true}[t]{\lstKV@SetIf{#1}{true}}
3111 78 | \lst@AddToHookExe{PreSet}{\global\let\lst@intname\lst@chunkname}
3112 79 | \lst@AddToHook{Init}{}%\fangle@caption}
3113 80 | %\lst@EndAspect
3114 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3115 16.2.4 Cross references
3116 We define the \chunkref command which makes it easy to generate visual references to different code chunks, e.g.
3119 \chunkref[3]{preamble}
3120 \chunkref{preamble}[arg1, arg2]
3122 Chunkref can also be used within a code chunk to include another code chunk. The third optional parameter to chunkref is a comma sepatarated list of arguments, which will replace defined parameters in the chunkref.
3123 Note 1. Darn it, if I have: =<\chunkref{new-mode-tracker}[{chunks[chunk_name, "language"]},{mode}]> the inner braces (inside [ ]) cause _ to signify subscript even though we have lst@ReplaceIn
3125 89a <./fangle.sty[18](
\v) ⇑84d, lang=> +≡ ⊲88d 90a⊳
3126 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3127 81 | \def\chunkref@args#1,{%
3129 83 | \lst@ReplaceIn\arg\lst@filenamerpl%
3131 85 | \@ifnextchar){\relax}{, \chunkref@args}%
3133 87 | \newcommand\chunkref[2][0]{%
3134 88 | \@ifnextchar({\chunkref@i{#1}{#2}}{\chunkref@i{#1}{#2}()}%
3136 90 | \def\chunkref@i#1#2(#3){%
3138 92 | \def\chunk{#2}%
3139 93 | \def\chunkno{#1}%
3140 94 | \def\chunkargs{#3}%
3141 95 | \ifx\chunkno\zero%
3142 96 | \def\chunkname{#2-1}%
3144 98 | \def\chunkname{#2-\chunkno}%
3146 100 | \let\lst@arg\chunk%
3147 101 | \lst@ReplaceIn\chunk\lst@filenamerpl%
3148 102 | \LA{%\moddef{%
3151 105 | \nwtagstyle{}\/%
3152 106 | \ifx\chunkno\zero%
3156 110 | \ifx\chunkargs\empty%
3158 112 | (\chunkref@args #3,)%
3160 114 | ~\subpageref{\chunkname}%
3163 117 | \RA%\endmoddef%
3165 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3168 90a <./fangle.sty[19](
\v) ⇑84d, lang=> +≡ ⊲89a
3169 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3172 |________________________________________________________________________
3175 Chapter 17Extracting fangle
3176 17.1 Extracting from Lyx
3177 To extract from L Y X, you will need to configure L Y X as explained in section ?.
3178 And this lyx-build scrap will extract fangle for me.
3180 91a <lyx-build[2](
\v) ⇑20a, lang=sh> +≡ ⊲20a
3181 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3185 14 | «lyx-build-helper 19b»
3186 15 | cd $PROJECT_DIR || exit 1
3188 17 | /usr/local/bin/fangle -R./fangle $TEX_SRC > ./fangle
3189 18 | /usr/local/bin/fangle -R./fangle.module $TEX_SRC > ./fangle.module
3191 20 | export FANGLE=./fangle
3192 21 | export TMP=${TMP:-/tmp}
3194 |________________________________________________________________________
3197 With a lyx-build-helper
3199 91b <lyx-build-helper[2](
\v) ⇑19b, lang=sh> +≡ ⊲19b
3200 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3201 5 | PROJECT_DIR="$LYX_r"
3202 6 | LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
3203 7 | TEX_DIR="$LYX_p"
3204 8 | TEX_SRC="$TEX_DIR/$LYX_i"
3205 9 | TXT_SRC="$TEX_SRC"
3206 |________________________________________________________________________
3209 17.2 Extracting documentation
3211 91c <./gen-www[1](
\v), lang=> ≡
3212 ________________________________________________________________________
3213 1 | #python -m elyxer --css lyx.css $LYX_SRC | \
3214 2 | # iconv -c -f utf-8 -t ISO-8859-1//TRANSLIT | \
3215 3 | # sed 's/UTF-8"\(.\)>/ISO-8859-1"\1>/' > www/docs/fangle.html
3217 5 | python -m elyxer --css lyx.css --iso885915 --html --destdirectory www/docs/fangle.e \
3218 6 | fangle.lyx > www/docs/fangle.e/fangle.html
3220 8 | ( mkdir -p www/docs/fangle && cd www/docs/fangle && \
3221 9 | lyx -e latex ../../../fangle.lyx && \
3222 10 | htlatex ../../../fangle.tex "xhtml,fn-in" && \
3223 11 | sed -i -e 's/<!--l\. [0-9][0-9]* *-->//g' fangle.html
3226 14 | ( mkdir -p www/docs/literate && cd www/docs/literate && \
3227 15 | lyx -e latex ../../../literate.lyx && \
3228 16 | htlatex ../../../literate.tex "xhtml,fn-in" && \
3229 17 | sed -i -e 's/<!--l\. [0-9][0-9]* *-->$//g' literate.html
3231 |________________________________________________________________________
3234 17.3 Extracting from the command line
3235 First you will need the tex output, then you can extract:
3237 92a <lyx-build-manual[1](
\v), lang=sh> ≡
3238 ________________________________________________________________________
3239 1 | lyx -e latex fangle.lyx
3240 2 | fangle -R./fangle fangle.tex > ./fangle
3241 3 | fangle -R./fangle.module fangle.tex > ./fangle.module
3242 |________________________________________________________________________
3249 95a <test:*[1](
\v), lang=> ≡
3250 ________________________________________________________________________
3253 3 | export SRC="${SRC:-./fangle.tm}"
3254 4 | export FANGLE="${FANGLE:-./fangle}"
3255 5 | export TMP="${TMP:-/tmp}"
3256 6 | export TESTDIR="$TMP/$USER/fangle.tests"
3257 7 | export TXT_SRC="${TXT_SRC:-$TESTDIR/fangle.txt}"
3259 9 | mkdir -p "$TESTDIR"
3261 11 | tm -s -c "$SRC" "$TXT_SRC" -q
3263 13 | «test:helpers 95c»
3265 15 | «test:run-tests 95b»
3268 18 | # test current fangle
3269 19 | echo Testing current fangle
3272 22 | # extract new fangle
3273 23 | echo testing new fangle
3274 24 | $FANGLE -R./fangle "$TXT_SRC" > "$TESTDIR/fangle"
3275 25 | export FANGLE="$TESTDIR/fangle"
3278 28 | # Now check that it can extract a fangle that also passes the tests!
3279 29 | echo testing if new fangle can generate itself
3280 30 | $FANGLE -R./fangle "$TXT_SRC" > "$TESTDIR/fangle.new"
3281 31 | passtest diff -bwu "$FANGLE" "$TESTDIR/fangle.new"
3282 32 | export FANGLE="$TESTDIR/fangle.new"
3284 |________________________________________________________________________
3288 95b <test:run-tests[1](
\v), lang=sh> ≡
3289 ________________________________________________________________________
3291 2 | $FANGLE -Rpca-test.awk $TXT_SRC | awk -f - || exit 1
3292 3 | «test:cromulence 56d»
3293 4 | «test:escapes 60c»
3294 5 | «test:test-chunk
\v(test:example-sh
\v) 96a»
3295 6 | «test:test-chunk
\v(test:example-makefile
\v) 96a»
3296 7 | «test:test-chunk
\v(test:q:1
\v) 96a»
3297 8 | «test:test-chunk
\v(test:make:1
\v) 96a»
3298 9 | «test:test-chunk
\v(test:make:2
\v) 96a»
3299 10 | «test:chunk-params 97e»
3300 |________________________________________________________________________
3304 95c <test:helpers[1](
\v), lang=> ≡
3305 ________________________________________________________________________
3308 3 | then echo "Passed $TEST"
3309 4 | else echo "Failed $TEST"
3316 11 | then echo "Passed $TEST"
3317 12 | else echo "Failed $TEST"
3321 |________________________________________________________________________
3324 This chunk will render a named chunk and compare it to another rendered nameed chunk
3326 96a <test:test-chunk[1](chunk
\v\v), lang=sh> ≡
3327 ________________________________________________________________________
3328 1 | «test:test-chunk-result
\v(${chunk}
\v, ${chunk}.result
\v) 96b»
3329 |________________________________________________________________________
3333 96b <test:test-chunk-result[1](chunk
\v, result
\v\v), lang=sh> ≡
3334 ________________________________________________________________________
3335 1 | TEST="${result}" passtest diff -u --label "${chunk}" <( $FANGLE -R${chunk} $TXT_SRC ) \
3336 2 | --label "${result}" <( $FANGLE -R${result} $TXT_SRC )
3337 |________________________________________________________________________
3340 Chapter 19Chunk Parameters
3343 97a <test:lyx:chunk-params:sub[1](THING
\v, colour
\v\v), lang=> ≡
3344 ________________________________________________________________________
3345 1 | I see a ${THING},
3346 2 | a ${THING} of colour ${colour},
3347 3 | and looking closer =<\chunkref{test:lyx:chunk-params:sub:sub}(${colour})>
3348 |________________________________________________________________________
3352 97b <test:lyx:chunk-params:sub:sub[1](colour
\v\v), lang=> ≡
3353 ________________________________________________________________________
3354 1 | a funny shade of ${colour}
3355 |________________________________________________________________________
3359 97c <test:lyx:chunk-params:text[1](
\v), lang=> ≡
3360 ________________________________________________________________________
3361 1 | What do you see? "=<\chunkref{test:lyx:chunk-params:sub}(joe, red)>"
3363 |________________________________________________________________________
3366 Should generate output:
3368 97d <test:lyx:chunk-params:result[1](
\v), lang=> ≡
3369 ________________________________________________________________________
3370 1 | What do you see? "I see a joe,
3371 2 | a joe of colour red,
3372 3 | and looking closer a funny shade of red"
3374 |________________________________________________________________________
3377 And this chunk will perform the test:
3379 97e <test:chunk-params[1](
\v), lang=> ≡ 98b⊳
3380 ________________________________________________________________________
3381 1 | «test:test-chunk-result
\v(test:lyx:chunk-params:text
\v, test:lyx:chunk-params:result
\v) 96b» || exit 1
3382 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3385 97f <test:chunk-params:sub[1](THING
\v, colour
\v\v), lang=> ≡
3386 ________________________________________________________________________
3387 1 | I see a ${THING},
3388 2 | a ${THING} of colour ${colour},
3389 3 | and looking closer «test:chunk-params:sub:sub
\v(${colour}
\v) 97g»
3390 |________________________________________________________________________
3394 97g <test:chunk-params:sub:sub[1](colour
\v\v), lang=> ≡
3395 ________________________________________________________________________
3396 1 | a funny shade of ${colour}
3397 |________________________________________________________________________
3401 97h <test:chunk-params:text[1](
\v), lang=> ≡ 96a⊳
3402 ________________________________________________________________________
3403 1 | What do you see? "«test:chunk-params:sub
\v(joe
\v, red
\v) 97f»"
3405 |/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3406 Should generate output:
3408 98a <test:chunk-params:result[1](
\v), lang=> ≡
3409 ________________________________________________________________________
3410 1 | What do you see? "I see a joe,
3411 2 | a joe of colour red,
3412 3 | and looking closer a funny shade of red"
3414 |________________________________________________________________________
3417 And this chunk will perform the test:
3419 98b <test:chunk-params[2](
\v) ⇑97e, lang=> +≡ ⊲97e
3420 ./\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
3421 2 | «test:test-chunk-result
\v(test:chunk-params:text
\v, test:chunk-params:result
\v) 96b» || exit 1
3422 |________________________________________________________________________
3425 Chapter 20Compile-log-lyx
3427 99a <Chunk:./compile-log-lyx[1](
\v), lang=sh> ≡
3428 ________________________________________________________________________
3430 2 | # can't use gtkdialog -i, cos it uses the "source" command which ubuntu sh doesn't have
3433 5 | errors="/tmp/compile.log.$$"
3434 6 | # if grep '^[^ ]*:\( In \|[0-9][0-9]*: [^ ]*:\)' > $errors
3435 7 | if grep '^[^ ]*(\([0-9][0-9]*\)) *: *\(error\|warning\)' > $errors
3437 9 | sed -i -e 's/^[^ ]*[/\\]\([^/\\]*\)(\([ 0-9][ 0-9]*\)) *: */\1:\2|\2|/' $errors
3438 10 | COMPILE_DIALOG='
3441 13 | <label>Compiler errors:</label>
3443 15 | <tree exported_column="0">
3444 16 | <variable>LINE</variable>
3445 17 | <height>400</height><width>800</width>
3446 18 | <label>File | Line | Message</label>
3447 19 | <action>'". $SELF ; "'lyxgoto $LINE</action>
3448 20 | <input>'"cat $errors"'</input>
3451 23 | <button><label>Build</label>
3452 24 | <action>lyxclient -c "LYXCMD:build-program" &</action>
3454 26 | <button ok></button>
3458 30 | export COMPILE_DIALOG
3459 31 | ( gtkdialog --program=COMPILE_DIALOG ; rm $errors ) &
3466 38 | file="${LINE%:*}"
3467 39 | line="${LINE##*:}"
3468 40 | extraline=‘cat $file | head -n $line | tac | sed '/^\\\\begin{lstlisting}/q' | wc -l‘
3469 41 | extraline=‘expr $extraline - 1‘
3470 42 | lyxclient -c "LYXCMD:command-sequence server-goto-file-row $file $line ; char-forward ; repeat $extraline paragraph-down ; paragraph-up-select"
3474 46 | if test -z "$COMPILE_DIALOG"
3477 |________________________________________________________________________