3 <style|<tuple|book|fangle|header-book>>
6 <hide-preamble|<assign|LyX|<macro|L<space|-0.1667em><move|Y|0fn|-0.25em><space|-0.125em>X>><assign|par-first|0fn><assign|par-par-sep|0.5fn>>
8 <doc-data|<doc-title|fangle>|<doc-author-data|<author-name|Sam
9 Liddicott>|<\author-address>
11 </author-address>>|<doc-date|August 2009>>
13 <section*|Introduction>
15 <name|Fangle> is a tool for fangled literate programming. Newfangled is
16 defined as <em|New and often needlessly novel> by
17 <name|TheFreeDictionary.com>.
19 In this case, fangled means yet another not-so-new<footnote|but improved.>
20 method for literate programming.
22 <name|Literate Programming> has a long history starting with the great
23 <name|Donald Knuth> himself, whose literate programming tools seem to make
24 use of as many escape sequences for semantic markup as <TeX> (also by
27 <name|Norman Ramsey> wrote the <name|Noweb> set of tools
28 (<verbatim|notangle>, <verbatim|noweave> and <verbatim|noroots>) and
29 helpfully reduced the amount of magic character sequences to pretty much
30 just <verbatim|\<less\>\<less\>>, <verbatim|\<gtr\>\<gtr\>> and
31 <verbatim|@>, and in doing so brought the wonders of literate programming
34 While using the <LyX> editor for <LaTeX> editing I had various troubles
35 with the noweb tools, some of which were my fault, some of which were
36 noweb's fault and some of which were <LyX>'s fault.
38 <name|Noweb> generally brought literate programming to the masses through
39 removing some of the complexity of the original literate programming, but
40 this would be of no advantage to me if the <LyX> / <LaTeX> combination
41 brought more complications in their place.
43 <name|Fangle> was thus born (originally called <name|Newfangle>) as an awk
44 replacement for notangle, adding some important features, like better
45 integration with <LyX> and <LaTeX> (and later <TeXmacs>), multiple output
46 format conversions, and fixing notangle bugs like indentation when using -L
49 Significantly, fangle is just one program which replaces various programs
50 in <name|Noweb>. Noweave is done away with and implemented directly as
51 <LaTeX> macros, and noroots is implemented as a function of the untangler
54 Fangle is written in awk for portability reasons, awk being available for
55 most platforms. A Python version<\footnote>
56 hasn't anyone implemented awk in python yet?
57 </footnote> was considered for the benefit of <LyX> but a scheme version
58 for <TeXmacs> will probably materialise first; as <TeXmacs> macro
59 capabilities help make edit-time and format-time rendering of fangle chunks
60 simple enough for my weak brain.
62 As an extension to many literate-programming styles, Fangle permits code
63 chunks to take parameters and thus operate somewhat like C pre-processor
64 macros, or like C++ templates. Name parameters (or even local
65 <em|variables> in the callers scope) are anticipated, as parameterized
66 chunks <emdash> useful though they are <emdash> are hard to comprehend in
67 the literate document.
69 <section*|License><new-page*><label|License>
71 Fangle is licensed under the GPL 3 (or later).
73 This doesn't mean that sources generated by fangle must be licensed under
76 This doesn't mean that you can't use or distribute fangle with sources of
77 an incompatible license, but it means you must make the source of fangle
80 As fangle is currently written in awk, an interpreted language, this should
83 <\nf-chunk|gpl3-copyright>
84 <item>fangle - fully featured notangle replacement in awk
88 <item>Copyright (C) 2009-2010 Sam Liddicott
89 \<less\>sam@liddicott.com\<gtr\>
93 <item>This program is free software: you can redistribute it and/or
96 <item>it under the terms of the GNU General Public License as published
99 <item>the Free Software Foundation, either version 3 of the License, or
101 <item>(at your option) any later version.
105 <item>This program is distributed in the hope that it will be useful,
107 <item>but WITHOUT ANY WARRANTY; without even the implied warranty of
109 <item>MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. \ See the
111 <item>GNU General Public License for more details.
115 <item>You should have received a copy of the GNU General Public License
117 <item>along with this program. \ If not, see
118 \<less\>http://www.gnu.org/licenses/\<gtr\>.
121 <\table-of-contents|toc>
122 Introduction <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
123 <no-break><pageref|auto-1>
125 License <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
126 <no-break><pageref|auto-2>
128 <vspace*|2fn><with|font-series|bold|math-font-series|bold|font-size|1.19|I<space|2spc>Using
129 Fangle> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
130 <no-break><pageref|auto-3><vspace|1fn>
132 <vspace*|1fn><with|font-series|bold|math-font-series|bold|1<space|2spc>Introduction
133 to Literate Programming> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
134 <no-break><pageref|auto-4><vspace|0.5fn>
136 <vspace*|1fn><with|font-series|bold|math-font-series|bold|2<space|2spc>Running
137 Fangle> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
138 <no-break><pageref|auto-5><vspace|0.5fn>
140 2.1<space|2spc>Listing roots <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
141 <no-break><pageref|auto-6>
143 2.2<space|2spc>Extracting roots <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
144 <no-break><pageref|auto-7>
146 2.3<space|2spc>Formatting the document
147 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
148 <no-break><pageref|auto-8>
150 <vspace*|1fn><with|font-series|bold|math-font-series|bold|3<space|2spc>Using
151 Fangle with L<rsup|<space|-0.8spc><move|A|0fn|-0.1fn>><space|-0.2spc>T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X>
152 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
153 <no-break><pageref|auto-9><vspace|0.5fn>
155 <vspace*|1fn><with|font-series|bold|math-font-series|bold|4<space|2spc>Using
156 Fangle with L<space|-0.1667em><move|Y|0fn|-0.25em><space|-0.125em>X>
157 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
158 <no-break><pageref|auto-10><vspace|0.5fn>
160 4.1<space|2spc>Installing the L<space|-0.1667em><move|Y|0fn|-0.25em><space|-0.125em>X
161 module <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
162 <no-break><pageref|auto-11>
164 4.2<space|2spc>Obtaining a decent mono font
165 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
166 <no-break><pageref|auto-12>
168 <with|par-left|1.5fn|4.2.1<space|2spc>txfonts
169 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
170 <no-break><pageref|auto-13>>
172 <with|par-left|1.5fn|4.2.2<space|2spc>ams pmb
173 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
174 <no-break><pageref|auto-14>>
176 <with|par-left|1.5fn|4.2.3<space|2spc>Luximono
177 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
178 <no-break><pageref|auto-15>>
180 4.3<space|2spc>Formatting your Lyx document
181 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
182 <no-break><pageref|auto-16>
184 <with|par-left|1.5fn|4.3.1<space|2spc>Customising the listing appearance
185 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
186 <no-break><pageref|auto-17>>
188 <with|par-left|1.5fn|4.3.2<space|2spc>Global customisations
189 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
190 <no-break><pageref|auto-18>>
192 4.4<space|2spc>Configuring the build script
193 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
194 <no-break><pageref|auto-19>
196 <with|par-left|1.5fn|4.4.1<space|2spc>...
197 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
198 <no-break><pageref|auto-20>>
200 <vspace*|1fn><with|font-series|bold|math-font-series|bold|5<space|2spc>Using
201 Fangle with T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X<rsub|<space|-0.4spc><move|<resize|M<space|-0.2spc>A<space|-0.4spc>CS||||0.5fn>|0fn|-0.1fn>>>
202 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
203 <no-break><pageref|auto-21><vspace|0.5fn>
205 <vspace*|1fn><with|font-series|bold|math-font-series|bold|6<space|2spc>Fangle
206 with Makefiles> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
207 <no-break><pageref|auto-22><vspace|0.5fn>
209 6.1<space|2spc>A word about makefiles formats
210 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
211 <no-break><pageref|auto-23>
213 6.2<space|2spc>Extracting Sources <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
214 <no-break><pageref|auto-24>
216 <with|par-left|1.5fn|6.2.1<space|2spc>Converting from
217 L<space|-0.1667em><move|Y|0fn|-0.25em><space|-0.125em>X to
218 L<rsup|<space|-0.8spc><move|A|0fn|-0.1fn>><space|-0.2spc>T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X
219 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
220 <no-break><pageref|auto-25>>
222 <with|par-left|1.5fn|6.2.2<space|2spc>Converting from
223 T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X<rsub|<space|-0.4spc><move|<resize|M<space|-0.2spc>A<space|-0.4spc>CS||||0.5fn>|0fn|-0.1fn>>
224 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
225 <no-break><pageref|auto-26>>
227 6.3<space|2spc>Extracting Program Source
228 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
229 <no-break><pageref|auto-27>
231 6.4<space|2spc>Extracting Source Files
232 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
233 <no-break><pageref|auto-28>
235 6.5<space|2spc>Extracting Documentation
236 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
237 <no-break><pageref|auto-29>
239 <with|par-left|1.5fn|6.5.1<space|2spc>Formatting
240 T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X
241 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
242 <no-break><pageref|auto-30>>
244 <with|par-left|3fn|6.5.1.1<space|2spc>Running pdflatex
245 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
246 <no-break><pageref|auto-31>>
248 <with|par-left|1.5fn|6.5.2<space|2spc>Formatting
249 T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X<rsub|<space|-0.4spc><move|<resize|M<space|-0.2spc>A<space|-0.4spc>CS||||0.5fn>|0fn|-0.1fn>>
250 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
251 <no-break><pageref|auto-32>>
253 <with|par-left|1.5fn|6.5.3<space|2spc>Building the Documentation as a
254 Whole <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
255 <no-break><pageref|auto-33>>
257 6.6<space|2spc>Other helpers <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
258 <no-break><pageref|auto-34>
260 6.7<space|2spc>Boot-strapping the extraction
261 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
262 <no-break><pageref|auto-35>
264 6.8<space|2spc>Incorporating Makefile.inc into existing projects
265 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
266 <no-break><pageref|auto-36>
268 <with|par-left|6fn|Example <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
269 <no-break><pageref|auto-37><vspace|0.15fn>>
271 <vspace*|2fn><with|font-series|bold|math-font-series|bold|font-size|1.19|II<space|2spc>Source
272 Code> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
273 <no-break><pageref|auto-38><vspace|1fn>
275 <vspace*|1fn><with|font-series|bold|math-font-series|bold|7<space|2spc>Fangle
276 awk source code> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
277 <no-break><pageref|auto-39><vspace|0.5fn>
279 7.1<space|2spc>AWK tricks <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
280 <no-break><pageref|auto-40>
282 7.2<space|2spc>Catching errors <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
283 <no-break><pageref|auto-41>
285 <vspace*|1fn><with|font-series|bold|math-font-series|bold|8<space|2spc>L<rsup|<space|-0.8spc><move|A|0fn|-0.1fn>><space|-0.2spc>T<rsub|<space|-0.4spc><move|<resize|<with|math-level|0|E>||||0.5fn>|0fn|-0.1fn>><space|-0.4spc>X
286 and lstlistings> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
287 <no-break><pageref|auto-42><vspace|0.5fn>
289 8.1<space|2spc>Additional lstlstings parameters
290 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
291 <no-break><pageref|auto-43>
293 8.2<space|2spc>Parsing chunk arguments
294 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
295 <no-break><pageref|auto-44>
297 8.3<space|2spc>Expanding parameters in the text
298 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
299 <no-break><pageref|auto-45>
301 <vspace*|1fn><with|font-series|bold|math-font-series|bold|9<space|2spc>Language
302 Modes & Quoting> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
303 <no-break><pageref|auto-46><vspace|0.5fn>
305 9.1<space|2spc>Modes <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
306 <no-break><pageref|auto-47>
308 <with|par-left|1.5fn|9.1.1<space|2spc>Modes to keep code together
309 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
310 <no-break><pageref|auto-48>>
312 <with|par-left|1.5fn|9.1.2<space|2spc>Modes affect included chunks
313 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
314 <no-break><pageref|auto-49>>
316 9.2<space|2spc>Language Mode Definitions
317 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
318 <no-break><pageref|auto-50>
320 <with|par-left|1.5fn|9.2.1<space|2spc>Backslash
321 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
322 <no-break><pageref|auto-51>>
324 <with|par-left|1.5fn|9.2.2<space|2spc>Strings
325 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
326 <no-break><pageref|auto-52>>
328 <with|par-left|1.5fn|9.2.3<space|2spc>Parentheses, Braces and Brackets
329 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
330 <no-break><pageref|auto-53>>
332 <with|par-left|1.5fn|9.2.4<space|2spc>Customizing Standard Modes
333 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
334 <no-break><pageref|auto-54>>
336 <with|par-left|1.5fn|9.2.5<space|2spc>Comments
337 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
338 <no-break><pageref|auto-55>>
340 <with|par-left|1.5fn|9.2.6<space|2spc>Regex
341 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
342 <no-break><pageref|auto-56>>
344 <with|par-left|1.5fn|9.2.7<space|2spc>Perl
345 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
346 <no-break><pageref|auto-57>>
348 <with|par-left|1.5fn|9.2.8<space|2spc>sh
349 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
350 <no-break><pageref|auto-58>>
352 9.3<space|2spc>Some tests <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
353 <no-break><pageref|auto-59>
355 9.4<space|2spc>A non-recursive mode tracker
356 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
357 <no-break><pageref|auto-60>
359 <with|par-left|1.5fn|9.4.1<space|2spc>Constructor
360 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
361 <no-break><pageref|auto-61>>
363 <with|par-left|1.5fn|9.4.2<space|2spc>Management
364 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
365 <no-break><pageref|auto-62>>
367 <with|par-left|1.5fn|9.4.3<space|2spc>Tracker
368 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
369 <no-break><pageref|auto-63>>
371 <with|par-left|3fn|9.4.3.1<space|2spc>One happy chunk
372 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
373 <no-break><pageref|auto-64>>
375 <with|par-left|3fn|9.4.3.2<space|2spc>Tests
376 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
377 <no-break><pageref|auto-65>>
379 9.5<space|2spc>Escaping and Quoting <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
380 <no-break><pageref|auto-66>
382 <vspace*|1fn><with|font-series|bold|math-font-series|bold|10<space|2spc>Recognizing
383 Chunks> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
384 <no-break><pageref|auto-67><vspace|0.5fn>
386 10.1<space|2spc>Chunk start <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
387 <no-break><pageref|auto-68>
389 <with|par-left|1.5fn|10.1.1<space|2spc>lstlistings
390 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
391 <no-break><pageref|auto-69>>
393 <with|par-left|1.5fn|10.1.2<space|2spc>Noweb
394 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
395 <no-break><pageref|auto-70>>
397 10.2<space|2spc>Chunk end <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
398 <no-break><pageref|auto-71>
400 <with|par-left|1.5fn|10.2.1<space|2spc>lstlistings
401 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
402 <no-break><pageref|auto-72>>
404 <with|par-left|1.5fn|10.2.2<space|2spc>noweb
405 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
406 <no-break><pageref|auto-73>>
408 10.3<space|2spc>Chunk contents <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
409 <no-break><pageref|auto-74>
411 <with|par-left|1.5fn|10.3.1<space|2spc>lstlistings
412 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
413 <no-break><pageref|auto-75>>
415 <vspace*|1fn><with|font-series|bold|math-font-series|bold|11<space|2spc>Processing
416 Options> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
417 <no-break><pageref|auto-76><vspace|0.5fn>
419 <vspace*|1fn><with|font-series|bold|math-font-series|bold|12<space|2spc>Generating
420 the Output> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
421 <no-break><pageref|auto-77><vspace|0.5fn>
423 12.1<space|2spc>Assembling the Chunks
424 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
425 <no-break><pageref|auto-78>
427 <with|par-left|1.5fn|12.1.1<space|2spc>Chunk Parts
428 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
429 <no-break><pageref|auto-79>>
431 <vspace*|1fn><with|font-series|bold|math-font-series|bold|13<space|2spc>Storing
432 Chunks> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
433 <no-break><pageref|auto-80><vspace|0.5fn>
435 <vspace*|1fn><with|font-series|bold|math-font-series|bold|14<space|2spc>getopt>
436 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
437 <no-break><pageref|auto-81><vspace|0.5fn>
439 <vspace*|1fn><with|font-series|bold|math-font-series|bold|15<space|2spc>Fangle
440 LaTeX source code> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
441 <no-break><pageref|auto-82><vspace|0.5fn>
443 15.1<space|2spc>fangle module <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
444 <no-break><pageref|auto-83>
446 <with|par-left|1.5fn|15.1.1<space|2spc>The Chunk style
447 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
448 <no-break><pageref|auto-84>>
450 <with|par-left|1.5fn|15.1.2<space|2spc>The chunkref style
451 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
452 <no-break><pageref|auto-85>>
454 15.2<space|2spc>Latex Macros <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
455 <no-break><pageref|auto-86>
457 <with|par-left|1.5fn|15.2.1<space|2spc>The chunk command
458 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
459 <no-break><pageref|auto-87>>
461 <with|par-left|3fn|15.2.1.1<space|2spc>Chunk parameters
462 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
463 <no-break><pageref|auto-88>>
465 <with|par-left|1.5fn|15.2.2<space|2spc>The noweb styled caption
466 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
467 <no-break><pageref|auto-89>>
469 <with|par-left|1.5fn|15.2.3<space|2spc>The chunk counter
470 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
471 <no-break><pageref|auto-91>>
473 <with|par-left|1.5fn|15.2.4<space|2spc>Cross references
474 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
475 <no-break><pageref|auto-92>>
477 <with|par-left|1.5fn|15.2.5<space|2spc>The end
478 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
479 <no-break><pageref|auto-93>>
481 <vspace*|1fn><with|font-series|bold|math-font-series|bold|16<space|2spc>Extracting
482 fangle> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
483 <no-break><pageref|auto-94><vspace|0.5fn>
485 16.1<space|2spc>Extracting from Lyx <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
486 <no-break><pageref|auto-95>
488 16.2<space|2spc>Extracting documentation
489 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
490 <no-break><pageref|auto-96>
492 16.3<space|2spc>Extracting from the command line
493 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
494 <no-break><pageref|auto-97>
496 16.4<space|2spc>Testing <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
497 <no-break><pageref|auto-98>
499 <vspace*|2fn><with|font-series|bold|math-font-series|bold|font-size|1.19|III<space|2spc>Tests>
500 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
501 <no-break><pageref|auto-99><vspace|1fn>
503 <vspace*|1fn><with|font-series|bold|math-font-series|bold|17<space|2spc>Chunk
504 Parameters> <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
505 <no-break><pageref|auto-100><vspace|0.5fn>
507 <vspace*|1fn><with|font-series|bold|math-font-series|bold|18<space|2spc>Compile-log-lyx>
508 <datoms|<macro|x|<repeat|<arg|x>|<with|font-series|medium|<with|font-size|1|<space|0.2fn>.<space|0.2fn>>>>>|<htab|5mm>>
509 <no-break><pageref|auto-101><vspace|0.5fn>
514 <chapter|Introduction to Literate Programming>
516 Todo: Should really follow on from a part-0 explanation of what literate
519 <chapter|Running Fangle>
521 Fangle is a replacement for <name|noweb>, which consists of
522 <verbatim|notangle>, <verbatim|noroots> and <verbatim|noweave>.
524 Like <verbatim|notangle> and <verbatim|noroots>, <verbatim|fangle> can read
525 multiple named files, or from stdin.
527 <section|Listing roots>
529 The -r option causes fangle to behave like noroots.
531 <code*|fangle -r filename.tex>
533 will print out the fangle roots of a tex file.\
535 Unlike the <verbatim|noroots> command, the printed roots are not enclosed
536 in angle brackets e.g. <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>>,
537 unless at least one of the roots is defined using the <verbatim|notangle>
538 notation <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>=>.
540 Also, unlike noroots, it prints out all roots --- not just those that are
541 not used elsewhere. I find that a root not being used doesn't make it
542 particularly top level <emdash> and so-called top level roots could also be
543 included in another root as well.\
545 My convention is that top level roots to be extracted begin with
546 <verbatim|./> and have the form of a filename.
548 Makefile.inc, discussed in <reference|makefile.inc>, can automatically
549 extract all such sources prefixed with <verbatim|./>
551 <section|Extracting roots>
553 notangle's <verbatim|-R> and <verbatim|-L> options are supported.
555 If you are using <LyX> or <LaTeX>, the standard way to extract a file would
558 <verbatim|fangle -R./Makefile.inc fangle.tex \<gtr\> ./Makefile.inc>
560 If you are using <TeXmacs>, the standard way to extract a file would
563 <verbatim|fangle -R./Makefile.inc fangle.txt \<gtr\> ./Makefile.inc>
565 <TeXmacs> users would obtain the text file with a <em|verbatim> export from
566 <TeXmacs> which can be done on the command line with <verbatim|texmacs -s
567 -c fangle.tm fangle.txt -q>
569 Unlike the <verbatim|noroots> command, the <verbatim|<verbatim|-L>> option
570 to generate C pre-preocessor <verbatim|#file> style line-number
571 directives,does not break indenting of the generated file..
573 Also, thanks to mode tracking (described in <reference|modes>) the
574 <verbatim|-L> option does not interrupt (and break) multi-line C macros
577 This does mean that sometimes the compiler might calculate the source line
578 wrongly when generating error messages in such cases, but there isn't any
579 other way around if multi-line macros include other chunks.
581 Future releases will include a mapping file so that line/character
582 references from the C compiler can be converted to the correct part of the
585 <section|Formatting the document>
587 The noweave replacement built into the editing and formatting environment
588 for <TeXmacs>, <LyX> (which uses <LaTeX>), and even for raw <LaTeX>.
590 Use of fangle with <TeXmacs>, <LyX> and <LaTeX> are explained the the next
593 <chapter|Using Fangle with <LaTeX>>
595 Because the noweave replacement is impemented in <LaTeX>, there is no
596 processing stage required before running the <LaTeX> command. Of course,
597 <LaTeX> may need running two or more times, so that the code chunk
598 references can be fully calculated.
600 The formatting is managed by a set of macros shown in
601 <reference|latex-source>, and can be included with:
603 <verbatim|\\usepackage{fangle.sty}>
605 Norman Ramsay's origial <filename|noweb.sty> package is currently required
606 as it is used for formatting the code chunk captions.
608 The <filename|listings.sty> package is required, and is used for formatting
609 the code chunks and syntax highlighting.
611 The <filename|xargs.sty> package is also required, and makes writing
612 <LaTeX> macro so much more pleasant.
614 <todo|Add examples of use of Macros>
616 <chapter|Using Fangle with <LyX>>
618 <LyX> uses the same <LaTeX> macros shown in <reference|latex-source> as
619 part of a <LyX> module file <filename|fangle.module>, which automatically
620 includes the macros in the document pre-amble provided that the fangle
621 <LyX> module is used in the document.
623 <section|Installing the <LyX> module>
625 Copy <filename|fangle.module> to your <LyX> layouts directory, which for
626 unix users will be <filename|~/.lyx/layouts>
628 In order to make the new literate styles availalble, you will need to
629 reconfigure <LyX> by clicking Tools-\<gtr\>Reconfigure, and then re-start
632 <section|Obtaining a decent mono font>
634 The syntax high-lighting features of <name|lstlistings> makes use of bold;
635 however a mono-space tt font is used to typeset the listings. Obtaining a
636 <with|font-family|tt|<strong|bold> tt font> can be impossibly difficult and
637 amazingly easy. I spent many hours at it, following complicated
638 instructions from those who had spend many hours over it, and was finally
639 delivered the simple solution on the lyx mailing list.
643 The simple way was to add this to my preamble:
646 \\usepackage{txfonts}
648 \\renewcommand{\\ttdefault}{txtt}
655 The next simplest way was to use ams poor-mans-bold, by adding this to the
661 %\\renewcommand{\\ttdefault}{txtt}
663 %somehow make \\pmb be the command for bold, forgot how, sorry, above
667 It works, but looks wretched on the dvi viewer.
669 <subsection|Luximono>
671 The lstlistings documention suggests using Luximono.
673 Luximono was installed according to the instructions in Ubuntu Forums
674 thread 1159181<\footnote>
675 http://ubuntuforums.org/showthread.php?t=1159181
676 </footnote> with tips from miknight<\footnote>
677 http://miknight.blogspot.com/2005/11/how-to-install-luxi-mono-font-in.html
678 </footnote> stating that <verbatim|sudo updmap --enable MixedMap ul9.map>
679 is required. It looks fine in PDF and PS view but still looks rotten in dvi
682 <section|Formatting your Lyx document>
684 It is not necessary to base your literate document on any of the original
685 <LyX> literate classes; so select a regular class for your document type.
687 Add the new module <em|Fangle Literate Listings> and also <em|Logical
688 Markup> which is very useful.
690 In the drop-down style listbox you should notice a new style defined,
693 When you wish to insert a literate chunk, you enter it's plain name in the
694 Chunk style, instead of the old <name|noweb> method that uses
695 <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>=> type tags. In the line (or
696 paragraph) following the chunk name, you insert a listing with:
697 Insert-\<gtr\>Program Listing.
699 Inside the white listing box you can type (or paste using
700 <kbd|shift+ctrl+V>) your listing. There is no need to use <kbd|ctrl+enter>
701 at the end of lines as with some older <LyX> literate techniques --- just
702 press enter as normal.
704 <subsection|Customising the listing appearance>
706 The code is formatted using the <name|lstlistings> package. The chunk style
707 doesn't just define the chunk name, but can also define any other chunk
708 options supported by the lstlistings package <verbatim|\\lstset> command.
709 In fact, what you type in the chunk style is raw latex. If you want to set
710 the chunk language without having to right-click the listing, just add
711 <verbatim|,lanuage=C> after the chunk name. (Currently the language will
712 affect all subsequent listings, so you may need to specify
713 <verbatim|,language=> quite a lot).
715 <todo|so fix the bug>
717 Of course you can do this by editing the listings box advanced properties
718 by right-clicking on the listings box, but that takes longer, and you can't
719 see at-a-glance what the advanced settings are while editing the document;
720 also advanced settings apply only to that box --- the chunk settings apply
721 through the rest of the document<\footnote>
722 It ought to apply only to subsequent chunks of the same name. I'll fix
726 <todo|So make sure they only apply to chunks of that name>
728 <subsection|Global customisations>
730 As lstlistings is used to set the code chunks, it's <verbatim|\\lstset>
731 command can be used in the pre-amble to set some document wide settings.
733 If your source has many words with long sequences of capital letters, then
734 <verbatim|columns=fullflexible> may be a good idea, or the capital letters
735 will get crowded. (I think lstlistings ought to use a slightly smaller font
736 for captial letters so that they still fit).
738 The font family <verbatim|\\ttfamily> looks more normal for code, but has
739 no bold (an alternate typewriter font is used).\
741 With <verbatim|\\ttfamily>, I must also specify
742 <verbatim|columns=fullflexible> or the wrong letter spacing is used.
744 In my <LaTeX> pre-amble I usually specialise my code format with:
746 <\nf-chunk|document-preamble>
749 <item>numbers=left, stepnumber=1, numbersep=5pt,
751 <item>breaklines=false,
753 <item>basicstyle=\\footnotesize\\ttfamily,
755 <item>numberstyle=\\tiny,
759 <item>columns=fullflexible,
761 <item>numberfirstline=true
768 <section|Configuring the build script>
770 You can invoke code extraction and building from the <LyX> menu option
771 Document-\<gtr\>Build Program.
773 First, make sure you don't have a conversion defined for Lyx-\<gtr\>Program
775 From the menu Tools-\<gtr\>Preferences, add a conversion from
776 Latex(Plain)-\<gtr\>Program as:
779 set -x ; fangle -Rlyx-build $$i \|\
781 \ \ env LYX_b=$$b LYX_i=$$i LYX_o=$$o LYX_p=$$p LYX_r=$$r bash
784 (But don't cut-n-paste it from this document or you may be be pasting a
785 multi-line string which will break your lyx preferences file).\
787 I hope that one day, <LyX> will set these into the environment when calling
790 You may also want to consider adding options to this conversion...
792 <verbatim|parselog=/usr/share/lyx/scripts/listerrors>
794 ...but if you do you will lose your stderr<\footnote>
795 There is some bash plumbing to get a copy of stderr but this footnote is
799 Now, a shell script chunk called <filename|lyx-build> will be extracted and
800 run whenever you choose the Document-\<gtr\>Build Program menu item.
802 This document was originally managed using <LyX> and lyx-build script for
803 this document is shown here for historical reference.\
806 lyx -e latex fangle.lyx && \\
808 \ \ fangle fangle.lyx \<gtr\> ./autoboot
811 This looks simple enough, but as mentioned, fangle has to be had from
812 somewhere before it can be extracted.
816 When the lyx-build chunk is executed, the current directory will be a
817 temporary directory, and <verbatim|LYX_SOURCE> will refer to the tex file
818 in this temporary directory. This is unfortunate as our makefile wants to
819 run from the project directory where the Lyx file is kept.
821 We can extract the project directory from <verbatim|$$r>, and derive the
822 probable Lyx filename from the noweb file that Lyx generated.
824 <\nf-chunk|lyx-build-helper>
825 <item>PROJECT_DIR="$LYX_r"
827 <item>LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
829 <item>TEX_DIR="$LYX_p"
831 <item>TEX_SRC="$TEX_DIR/$LYX_i"
834 And then we can define a lyx-build fragment similar to the autoboot
837 <\nf-chunk|lyx-build>
840 <item>=\<less\>\\chunkref{lyx-build-helper}\<gtr\>
842 <item>cd $PROJECT_DIR \|\| exit 1
846 <item>#/usr/bin/fangle -filter ./notanglefix-filter \\
848 <item># \ -R./Makefile.inc "../../noweb-lyx/noweb-lyx3.lyx" \\
850 <item># \ \| sed '/NOWEB_SOURCE=/s/=.*/=samba4-dfs.lyx/' \\
852 <item># \ \<gtr\> ./Makefile.inc
856 <item>#make -f ./Makefile.inc fangle_sources
861 <chapter|Using Fangle with <TeXmacs>>
863 <todo|Write this chapter>
865 <chapter|Fangle with Makefiles><label|makefile.inc>
867 Here we describe a <filename|Makefile.inc> that you can include in your own
868 Makefiles, or glue as a recursive make to other projects.
870 <filename|Makefile.inc> will cope with extracting all the other source
871 files from this or any specified literate document and keeping them up to
874 It may also be included by a <verbatim|Makefile> or <verbatim|Makefile.am>
875 defined in a literate document to automatically deal with the extraction of
876 source files and documents during normal builds.
878 Thus, if <verbatim|Makefile.inc> is included into a main project makefile
879 it add rules for the source files, capable of extracting the source files
880 from the literate document.
882 <section|A word about makefiles formats>
884 Whitespace formatting is very important in a Makefile. The first character
885 of each action line must be a TAB.\
888 target: pre-requisite
895 This requires that the literate programming environment have the ability to
896 represent a TAB character in a way that fangle will generate an actual TAB
899 We also adopt a convention that code chunks whose names beginning with
900 <verbatim|./> should always be automatically extracted from the document.
901 Code chunks whose names do not begin with <verbatim|./> are for internal
902 reference. Such chunks may be extracted directly, but will not be
903 automatically extracted by this Makefile.
905 <section|Extracting Sources>
907 Our makefile has two parts; variables must be defined before the targets
910 As we progress through this chapter, explaining concepts, we will be adding
911 lines to <nf-ref|Makefile.inc-vars|> and <nf-ref|Makefile.inc-targets|>
912 which are included in <nf-ref|./Makefile.inc|> below.
914 <\nf-chunk|./Makefile.inc>
915 <item><nf-ref|Makefile.inc-vars|>
917 <item><nf-ref|Makefile.inc-targets|>
920 We first define a placeholder for <verbatim|LITERATE_SOURCE> to hold the
921 name of this document. This will normally be passed on the command line.
923 <\nf-chunk|Makefile.inc-vars>
924 <item>LITERATE_SOURCE=
927 Fangle cannot process <LyX> or <TeXmacs> documents directly, so the first
928 stage is to convert these to more suitable text based formats<\footnote>
929 <LyX> and <TeXmacs> formats are text-based, but not suitable for fangle
932 <subsection|Converting from <LyX> to <LaTeX>><label|Converting-from-Lyx>
934 The first stage will always be to convert the <LyX> file to a <LaTeX> file.
935 Fangle must run on a <TeX> file because the <LyX> command
936 <verbatim|server-goto-file-line><\footnote>
937 The Lyx command <verbatim|server-goto-file-line> is used to position the
938 Lyx cursor at the compiler errors.
939 </footnote> requries that the line number provided be a line of the <TeX>
940 file and always maps this the line in the <LyX> docment. We use
941 <verbatim|server-goto-file-line> when moving the cursor to error lines
942 during compile failures.
944 The command <verbatim|lyx -e literate fangle.lyx> will produce
945 <verbatim|fangle.tex>, a <TeX> file; so we define a make target to be the
946 same as the <LyX> file but with the <verbatim|.tex> extension.
948 The <verbatim|EXTRA_DIST> is for automake support so that the <TeX> files
949 will automaticaly be distributed with the source, to help those who don't
950 have <LyX> installed.
952 <\nf-chunk|Makefile.inc-vars>
953 <item>TEX_SOURCE=$(LYX_SOURCE:.lyx=.tex)
955 <item>EXTRA_DIST+=$(TEX_SOURCE)
958 We then specify that the <TeX> source is to be generated from the <LyX>
961 <\nf-chunk|Makefile.inc-targets>
962 <item>$(TEX_SOURCE): $(LYX_SOURCE)
964 <item><nf-tab>lyx -e latex $\<less\>
968 <item><nf-tab>rm -f -- $(TEX_SOURCE)
970 <item>clean: clean_tex
973 <subsection|Converting from <TeXmacs>><label|Converting-from-Lyx>
975 Fangle cannot process <TeXmacs> files directly<\footnote>
976 but this is planned when <TeXmacs> uses xml as it's native format
977 </footnote>, but must first convert them to text files.
979 The command <verbatim|texmacs -c fangle.tm fangle.txt -q> will produce
980 <verbatim|fangle.txt>, a text file; so we define a make target to be the
981 same as the <TeXmacs> file but with the <verbatim|.txt> extension.
983 The <verbatim|EXTRA_DIST> is for automake support so that the <TeX> files
984 will automaticaly be distributed with the source, to help those who don't
985 have <LyX> installed.
987 <\nf-chunk|Makefile.inc-vars>
988 <item>TXT_SOURCE=$(LITERATE_SOURCE:.tm=.txt)
990 <item>EXTRA_DIST+=$(TXT_SOURCE)
993 <todo|Add loop around each $\<less\> so multiple targets can be specified>
995 <\nf-chunk|Makefile.inc-targets>
996 <item>$(TXT_SOURCE): $(LITERATE_SOURCE)
998 <item><nf-tab>texmacs -c $\<less\> $(TXT_SOURCE) -q
1002 <item><nf-tab>rm -f -- $(TXT_SOURCE)
1004 <item>clean: clean_txt
1007 <section|Extracting Program Source>
1009 The program source is extracted using fangle, which is designed to operate
1010 on text or a <LaTeX> documents<\footnote>
1011 <LaTeX> documents are just slightly special text documents
1014 <\nf-chunk|Makefile.inc-vars>
1015 <item>FANGLE_SOURCE=$(TEX_SOURCE) $(TXT_SOURCE)
1018 The literate document can result in any number of source files, but not all
1019 of these will be changed each time the document is updated. We certainly
1020 don't want to update the timestamps of these files and cause the whole
1021 source tree to be recompiled just because the literate explanation was
1022 revised. We use <verbatim|CPIF> from the <em|Noweb> tools to avoid updating
1023 the file if the content has not changed, but should probably write our own.
1025 However, if a source file is not updated, then the fangle file will always
1026 have a newer time-stamp and the makefile would always re-attempt to extact
1027 a newer source file which would be a waste of time.
1029 Because of this, we use a stamp file which is always updated each time the
1030 sources are fully extracted from the <LaTeX> document. If the stamp file is
1031 newer than the document, then we can avoid an attempt to re-extract any of
1032 the sources. Because this stamp file is only updated when extraction is
1033 complete, it is safe for the user to interrupt the build-process
1036 We use <verbatim|echo> rather than <verbatim|touch> to update the stamp
1037 file beause the <verbatim|touch> command does not work very well over an
1038 <verbatim|sshfs>mount \ that I was using.
1040 <\nf-chunk|Makefile.inc-vars>
1041 <item>FANGLE_SOURCE_STAMP=$(FANGLE_SOURCE).stamp
1044 <\nf-chunk|Makefile.inc-targets>
1045 <item>$(FANGLE_SOURCE_STAMP): $(FANGLE_SOURCE) \\
1047 <item><nf-tab> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ $(FANGLE_SOURCES) ; \\
1049 <item><nf-tab>echo -n \<gtr\> $(FANGLE_SOURCE_STAMP)
1053 <item><nf-tab>rm -f $(FANGLE_SOURCE_STAMP)
1055 <item>clean: clean_stamp
1058 <section|Extracting Source Files>
1060 We compute <verbatim|FANGLE_SOURCES> to hold the names of all the source
1061 files defined in the document. We compute this only once, by means of
1062 <verbatim|:=> in assignent. The sed deletes the any
1063 <verbatim|\<less\>\<less\>> and <verbatim|\<gtr\>\<gtr\>> which may
1064 surround the roots names (for compatibility with Noweb's noroots command).
1066 As we use chunk names beginning with <filename|./> to denote top level
1067 fragments that should be extracted, we filter out all fragments that do not
1068 begin with <filename|./>
1071 <verbatim|FANGLE_PREFIX> is set to <verbatim|./> by default, but whatever
1072 it may be overridden to, the prefix is replaced by a literal
1073 <verbatim|./> before extraction so that files will be extracted in the
1074 current directory whatever the prefix. This helps namespace or
1075 sub-project prefixes like <verbatim|documents:> for chunks like
1076 <verbatim|documents:docbook/intro.xml>
1079 <todo|This doesn't work though, because it loses the full name and doesn't
1080 know what to extact!>
1082 <\nf-chunk|Makefile.inc-vars>
1083 <item>FANGLE_PREFIX:=\\.\\/
1085 <item>FANGLE_SOURCES:=$(shell \\
1087 <item> \ fangle -r $(FANGLE_SOURCE) \|\\
1089 <item> \ sed -e 's/^[\<less\>][\<less\>]//;s/[\<gtr\>][\<gtr\>]$$//;/^$(FANGLE_PREFIX)/!d'
1092 <item> \ \ \ \ \ -e 's/^$(FANGLE_PREFIX)/\\.\\//' )
1095 The target below, <verbatim|echo_fangle_sources> is a helpful debugging
1096 target and shows the names of the files that would be extracted.
1098 <\nf-chunk|Makefile.inc-targets>
1099 <item>.PHONY: echo_fangle_sources
1101 <item>echo_fangle_sources: ; @echo $(FANGLE_SOURCES)
1104 We define a convenient target called <verbatim|fangle_sources> so that
1105 <verbatim|make -f fangle_sources> will re-extract the source if the
1106 literate document has been updated.\
1108 <\nf-chunk|Makefile.inc-targets>
1109 <item>.PHONY: fangle_sources
1111 <item>fangle_sources: $(FANGLE_SOURCE_STAMP)
1114 And also a convenient target to remove extracted sources.
1116 <\nf-chunk|Makefile.inc-targets>
1117 <item>.PHONY: clean_fangle_sources
1119 <item>clean_fangle_sources: ; \\
1121 <item> \ \ \ \ \ \ \ rm -f -- $(FANGLE_SOURCE_STAMP) $(FANGLE_SOURCES)
1124 We now look at the extraction of the source files.
1126 This makefile macro <verbatim|if_extension> takes 4 arguments: the filename
1127 <verbatim|$(1)>, some extensions to match <verbatim|$(2)> and a shell
1128 command to return if the filename does match the exensions <verbatim|$(3)>,
1129 and a shell command to return if it does not match the extensions
1132 <\nf-chunk|Makefile.inc-vars>
1133 <item>if_extension=$(if $(findstring $(suffix $(1)),$(2)),$(3),$(4))
1136 For some source files like C files, we want to output the line number and
1137 filename of the original <LaTeX> document from which the source
1139 I plan to replace this option with a separate mapping file so as not to
1140 pollute the generated source, and also to allow a code pretty-printing
1141 reformatter like <verbatim|indent> be able to re-format the file and
1142 adjust for changes through comparing the character streams.
1145 To make this easier we define the file extensions for which we want to do
1148 <\nf-chunk|Makefile.inc-vars>
1149 <item>C_EXTENSIONS=.c .h
1152 We can then use the <verbatim|if_extensions> macro to define a macro which
1153 expands out to the <verbatim|-L> option if fangle is being invoked in a C
1154 source file, so that C compile errors will refer to the line number in the
1157 <\nf-chunk|Makefile.inc-vars>
1160 <item>nf_line=-L -T$(TABS)
1162 <item>fangle=fangle $(call if_extension,$(2),$(C_EXTENSIONS),$(nf_line))
1166 We can use a similar trick to define an indent macro which takes just the
1167 filename as an argument and can return a pipeline stage calling the indent
1168 command. Indent can be turned off with <verbatim|make fangle_sources
1171 <\nf-chunk|Makefile.inc-vars>
1172 <item>indent_options=-npro -kr -i8 -ts8 -sob -l80 -ss -ncs
1174 <item>indent=$(call if_extension,$(1),$(C_EXTENSIONS), \| indent
1178 We now define the pattern for extracting a file. The files are written
1179 using noweb's <verbatim|cpif> so that the file timestamp will not be
1180 touched if the contents haven't changed. This avoids the need to rebuild
1181 the entire project because of a typographical change in the documentation,
1182 or if none or a few C source files have changed.
1184 <\nf-chunk|Makefile.inc-vars>
1185 <item>fangle_extract=@mkdir -p $(dir $(1)) && \\
1187 <item> \ $(call fangle,$(2),$(1)) \<gtr\> "$(1).tmp" && \\
1189 <item> \ cat "$(1).tmp" $(indent) \| cpif "$(1)" \\
1191 <item> \ && rm -- "$(1).tmp" \|\| \\
1193 <item> \ (echo error newfangling $(1) from $(2) ; exit 1)
1196 We define a target which will extract or update all sources. To do this we
1197 first defined a makefile template that can do this for any source file in
1198 the <LaTeX> document.
1200 <\nf-chunk|Makefile.inc-vars>
1201 <item>define FANGLE_template
1205 <item><nf-tab>$$(call fangle_extract,$(1),$(2))
1207 <item> \ FANGLE_TARGETS+=$(1)
1212 We then enumerate the discovered <verbatim|FANGLE_SOURCES> to generate a
1213 makefile rule for each one using the makefile template we defined above.
1215 <\nf-chunk|Makefile.inc-targets>
1216 <item>$(foreach source,$(FANGLE_SOURCES),\\
1218 <item> \ $(eval $(call FANGLE_template,$(source),$(FANGLE_SOURCE))) \\
1223 These will all be built with <verbatim|FANGLE_SOURCE_STAMP>.
1225 We also remove the generated sources on a make distclean.
1227 <\nf-chunk|Makefile.inc-targets>
1228 <item>_distclean: clean_fangle_sources
1231 <section|Extracting Documentation>
1233 We then identify the intermediate stages of the documentation and their
1234 build and clean targets.
1236 <subsection|Formatting <TeX>>
1238 <subsubsection|Running pdflatex>
1240 We produce a pdf file from the tex file.
1242 <\nf-chunk|Makefile.inc-vars>
1243 <item>FANGLE_PDF=$(TEX_SOURCE:.tex=.pdf)
1246 We run pdflatex twice to be sure that the contents and aux files are up to
1247 date. We certainly are <em|required> to run pdflatex at least twice if
1248 these files do not exist.
1250 <\nf-chunk|Makefile.inc-targets>
1251 <item>$(FANGLE_PDF): $(TEX_SOURCE)
1253 <item><nf-tab>pdflatex $\<less\> && pdflatex $\<less\>
1259 <item><nf-tab>rm -f -- $(FANGLE_PDF) $(TEX_SOURCE:.tex=.toc) \\
1261 <item><nf-tab> \ $(TEX_SOURCE:.tex=.log) $(TEX_SOURCE:.tex=.aux)
1264 <subsection|Formatting <TeXmacs>>
1266 <TeXmacs> can produce a PDF file directly.
1268 <\nf-chunk|Makefile.inc-vars>
1269 <item>FANGLE_PDF=$(TEX_SOURCE:.tm=.pdf)
1273 Outputting the PDF may not be enough to update the links and page
1276 we need to update twice, generate a pdf, update twice mode and generate a
1279 Basically the PDF export of <TeXmacs> is pretty rotten and doesn't work
1280 properly from the CLI
1283 <\nf-chunk|Makefile.inc-targets>
1284 <item>$(FANGLE_PDF): $(TEXMACS_SOURCE)
1286 <item><nf-tab>texmacs -c $(TEXMACS_SOURCE) $\<less\> -q
1292 <item><nf-tab>rm -f -- $(FANGLE_PDF)
1295 <subsection|Building the Documentation as a Whole>
1297 Currently we only build pdf as a final format, but <verbatim|FANGLE_DOCS>
1298 may later hold other output formats.
1300 <\nf-chunk|Makefile.inc-vars>
1301 <item>FANGLE_DOCS=$(FANGLE_PDF)
1304 We also define <verbatim|fangle_docs> as a convenient phony target.
1306 <\nf-chunk|Makefile.inc-targets>
1307 <item>.PHONY: fangle_docs
1309 <item>fangle_docs: $(FANGLE_DOCS)
1311 <item>docs: fangle_docs
1314 And define a convenient <verbatim|clean_fangle_docs> which we add to the
1315 regular clean target
1317 <\nf-chunk|Makefile.inc-targets>
1318 <item>.PHONEY: clean_fangle_docs
1320 <item>clean_fangle_docs: clean_tex clean_pdf
1322 <item>clean: clean_fangle_docs
1326 <item>distclean_fangle_docs: clean_tex clean_fangle_docs
1328 <item>distclean: clean distclean_fangle_docs
1331 <section|Other helpers>
1333 If <filename|Makefile.inc> is included into <filename|Makefile>, then
1334 extracted files can be updated with this command:
1336 <verbatim|make fangle_sources>
1340 <verbatim|make -f Makefile.inc fangle_sources>
1342 <section|Boot-strapping the extraction>
1344 As well as having the makefile extract or update the source files as part
1345 of it's operation, it also seems convenient to have the makefile
1346 re-extracted itself from <em|this> document.
1348 It would also be convenient to have the code that extracts the makefile
1349 from this document to also be part of this document, however we have to
1350 start somewhere and this unfortunately requires us to type at least a few
1351 words by hand to start things off.
1353 Therefore we will have a minimal root fragment, which, when extracted, can
1354 cope with extracting the rest of the source. This shell script fragment can
1355 do that. It's name is <verbatim|*> <emdash> out of regard for <name|Noweb>,
1356 but when extracted might better be called <verbatim|autoupdate>.
1365 <item>MAKE_SRC="${1:-${NW_LYX:-../../noweb-lyx/noweb-lyx3.lyx}}"
1367 <item>MAKE_SRC=\0dirname "$MAKE_SRC"\0/\0basename "$MAKE_SRC" .lyx\0
1369 <item>NOWEB_SRC="${2:-${NOWEB_SRC:-$MAKE_SRC.lyx}}"
1371 <item>lyx -e latex $MAKE_SRC
1375 <item>fangle -R./Makefile.inc ${MAKE_SRC}.tex \\
1377 <item> \ \| sed "/FANGLE_SOURCE=/s/^/#/;T;aNOWEB_SOURCE=$FANGLE_SRC" \\
1379 <item> \ \| cpif ./Makefile.inc
1383 <item>make -f ./Makefile.inc fangle_sources
1386 The general Makefile can be invoked with <filename|./autoboot> and can also
1387 be included into any automake file to automatically re-generate the source
1390 The <em|autoboot> can be extracted with this command:
1393 lyx -e latex fangle.lyx && \\
1395 \ \ fangle fangle.lyx \<gtr\> ./autoboot
1398 This looks simple enough, but as mentioned, fangle has to be had from
1399 somewhere before it can be extracted.
1401 On a unix system this will extract <filename|fangle.module> and the
1402 <filename|fangle> awk script, and run some basic tests.\
1404 <todo|cross-ref to test chapter when it is a chapter all on its own>
1406 <section|Incorporating Makefile.inc into existing projects>
1408 If you are writing a literate module of an existing non-literate program
1409 you may find it easier to use a slight recursive make instead of directly
1410 including <verbatim|Makefile.inc> in the projects makefile.\
1412 This way there is less chance of definitions in <verbatim|Makefile.inc>
1413 interfering with definitions in the main makefile, or with definitions in
1414 other <verbatim|Makefile.inc> from other literate modules of the same
1417 To do this we add some <em|glue> to the project makefile that invokes
1418 Makefile.inc in the right way. The glue works by adding a <verbatim|.PHONY>
1419 target to call the recursive make, and adding this target as an additional
1420 pre-requisite to the existing targets.
1422 <paragraph|Example>Sub-module of existing system
1424 In this example, we are building <verbatim|module.so> as a literate module
1425 of a larger project.
1427 We will show the sort glue that can be inserted into the projects Makefile
1428 <emdash> or more likely <emdash> a regular Makefile included in or invoked
1429 by the projects Makefile.
1431 <\nf-chunk|makefile-glue>
1432 <item>module_srcdir=modules/module
1434 <item>MODULE_SOURCE=module.tm
1436 <item>MODULE_STAMP=$(MODULE_SOURCE).stamp
1439 The existing build system may already have a build target for
1440 <filename|module.o>, but we just add another pre-requisite to that. In this
1441 case we use <filename|module.tm.stamp> as a pre-requisite, the stamp file's
1442 modified time indicating when all sources were extracted<\footnote>
1443 If the projects build system does not know how to build the module from
1444 the extracted sources, then just add build actions here as normal.
1447 <\nf-chunk|makefile-glue>
1448 <item>$(module_srcdir)/module.o: $(module_srcdir)/$(MODULE_STAMP)
1451 The target for this new pre-requisite will be generated by a recursive make
1452 using <filename|Makefile.inc> which will make sure that the source is up to
1453 date, before it is built by the main projects makefile.
1455 <\nf-chunk|makefile-glue>
1456 <item>$(module_srcdir)/$(MODULE_STAMP): $(module_srcdir)/$(MODULE_SOURCE)
1458 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc fangle_sources
1459 LITERATE_SOURCE=$(MODULE_SOURCE)
1462 We can do similar glue for the docs, clean and distclean targets. In this
1463 example the main prject was using a double colon for these targets, so we
1464 must use the same in our glue.
1466 <\nf-chunk|makefile-glue>
1467 <item>docs:: docs_module
1469 <item>.PHONY: docs_module
1473 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc docs
1474 LITERATE_SOURCE=$(MODULE_SOURCE)
1478 <item>clean:: clean_module
1480 <item>.PHONEY: clean_module
1484 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc clean
1485 LITERATE_SOURCE=$(MODULE_SOURCE)
1489 <item>distclean:: distclean_module
1491 <item>.PHONY: distclean_module
1493 <item>distclean_module:
1495 <item><nf-tab>$(MAKE) -C $(module_srcdir) -f Makefile.inc distclean
1496 LITERATE_SOURCE=$(MODULE_SOURCE)
1499 We could do similarly for install targets to install the generated docs.
1503 <chapter|Fangle awk source code>
1505 We use the copyright notice from chapter <reference|License>.
1507 <\nf-chunk|./fangle>
1508 <item>#! /usr/bin/awk -f
1510 <item># <nf-ref|gpl3-copyright|>
1513 We also use code from <person|Arnold Robbins> public domain getopt (1993
1514 revision) defined in <reference|getopt>, and naturally want to attribute
1517 <\nf-chunk|./fangle>
1520 <item># NOTE: Arnold Robbins public domain getopt for awk is also used:
1522 <item><nf-ref|getopt.awk-header|>
1526 <item><nf-ref|getopt.awk-getopt()|>
1531 And include the following chunks (which are explained further on) to make
1534 <\nf-chunk|./fangle>
1535 <item><nf-ref|helper-functions|>
1537 <item><nf-ref|mode-tracker|>
1539 <item><nf-ref|parse_chunk_args|>
1541 <item><nf-ref|chunk-storage-functions|>
1543 <item><nf-ref|output_chunk_names()|>
1545 <item><nf-ref|output_chunks()|>
1547 <item><nf-ref|write_chunk()|>
1549 <item><nf-ref|expand_chunk_args()|>
1553 <item><nf-ref|begin|>
1555 <item><nf-ref|recognize-chunk|>
1560 <section|AWK tricks>
1562 The portable way to erase an array in awk is to split the empty string, so
1563 we define a fangle macro that can split an array, like this:
1565 <nf-chunk|awk-delete-array|split("", <nf-arg|ARRAY>);|awk|<tuple|ARRAY>>
1567 For debugging it is sometimes convenient to be able to dump the contents of
1568 an array to <verbatim|stderr>, and so this macro is also useful.
1570 <\nf-chunk|dump-array>
1571 <item>print "\\nDump: <nf-arg|ARRAY>\\n--------\\n" \<gtr\>
1574 <item>for (_x in <nf-arg|ARRAY>) {
1576 <item> \ print _x "=" <nf-arg|ARRAY>[_x] "\\n" \<gtr\> "/dev/stderr";
1580 <item>print "========\\n" \<gtr\> "/dev/stderr";
1581 </nf-chunk|awk|<tuple|ARRAY>>
1583 <section|Catching errors>
1585 Fatal errors are issued with the error function:
1588 <item>function error(message)
1592 <item> \ print "ERROR: " FILENAME ":" FNR " " message \<gtr\>
1600 and likewise for non-fatal warnings:
1603 <item>function warning(message)
1607 <item> \ print "WARNING: " FILENAME ":" FNR " " message \<gtr\>
1610 <item> \ warnings++;
1615 <todo|append=helper-functions>
1617 <chapter|<LaTeX> and lstlistings>
1619 <todo|Split LyX and TeXmacs parts>
1621 For <LyX> and <LaTeX>, the <verbatim|lstlistings> package is used to format
1622 the lines of code chunks. You may recal from chapter XXX that arguments to
1623 a chunk definition are pure <LaTeX> code. This means that fangle needs to
1624 be able to parse <LaTeX> a little.
1626 <LaTeX> arguments to <verbatim|lstlistings> macros are a comma seperated
1627 list of key-value pairs, and values containing commas are enclosed in
1628 <verbatim|{> braces <verbatim|}> (which is to be expected for <LaTeX>).
1630 A sample expressions is:
1632 <verbatim|name=thomas, params={a, b}, something, something-else>
1634 but we see that this is just a simpler form of this expression:
1636 <verbatim|name=freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc>
1638 We may consider that we need a function that can parse such <LaTeX>
1639 expressions and assign the values to an AWK associated array, perhaps using
1640 a recursive parser into a multi-dimensional hash<\footnote>
1641 as AWK doesn't have nested-hash support
1642 </footnote>, resulting in:
1644 <tabular|<tformat|<cwith|2|6|1|2|cell-lborder|0.5pt>|<cwith|2|6|1|2|cell-rborder|0.5pt>|<cwith|2|6|1|2|cell-bborder|0.5pt>|<cwith|2|6|1|2|cell-tborder|0.5pt>|<cwith|1|1|1|2|cell-lborder|0.5pt>|<cwith|1|1|1|2|cell-rborder|0.5pt>|<cwith|1|1|1|2|cell-bborder|0.5pt>|<cwith|1|1|1|2|cell-tborder|0.5pt>|<table|<row|<cell|key>|<cell|value>>|<row|<cell|a[name]>|<cell|freddie>>|<row|<cell|a[foo,
1645 bar]>|<cell|baz>>|<row|<cell|a[foo, quux,
1646 quirk]>|<cell|>>|<row|<cell|a[foo, quux,
1647 a]>|<cell|fleeg>>|<row|<cell|a[etc]>|<cell|>>>>>
1649 Yet, also, on reflection it seems that sometimes such nesting is not
1650 desirable, as the braces are also used to delimit values that contain
1651 commas --- we may consider that
1653 <verbatim|name={williamson, freddie}>
1655 should assign <verbatim|williamson, freddie> to <verbatim|name>.
1657 In fact we are not so interested in the detail so as to be bothered by
1658 this, which turns out to be a good thing for two reasons. Firstly <TeX> has
1659 a malleable parser with no strict syntax, and secondly whether or not
1660 <verbatim|williamson> and <verbatim|freddie> should count as two items will
1661 be context dependant anyway.
1663 We need to parse this latex for only one reason; which is that we are
1664 extending lstlistings to add some additional arguments which will be used
1665 to express chunk parameters and other chunk options.
1667 <section|Additional lstlstings parameters>
1669 Further on we define a <verbatim|\\Chunk> <LaTeX> macro whose arguments
1670 will consist of a the chunk name, optionally followed by a comma and then a
1671 comma separated list of arguments. In fact we will just need to prefix
1672 <verbatim|name=> to the arguments to in order to create valid lstlistings
1675 There will be other arguments supported too;\
1678 <item*|params>As an extension to many literate-programming styles, fangle
1679 permits code chunks to take parameters and thus operate somewhat like C
1680 pre-processor macros, or like C++ templates. Chunk parameters are
1681 declared with a chunk argument called params, which holds a semi-colon
1682 separated list of parameters, like this:
1684 <verbatim|achunk,language=C,params=name;address>
1686 <item*|addto>a named chunk that this chunk is to be included into. This
1687 saves the effort of having to declare another listing of the named chunk
1688 merely to include this one.
1691 Function get_chunk_args() will accept two paramters, text being the text to
1692 parse, and values being an array to receive the parsed values as described
1693 above. The optional parameter path is used during recursion to build up the
1694 multi-dimensional array path.
1696 <\nf-chunk|./fangle>
1697 <item>=\<less\>\\chunkref{get_chunk_args()}\<gtr\>
1700 <\nf-chunk|get_chunk_args()>
1701 <item>function get_chunk_args(text, values,
1703 <item> \ # optional parameters
1705 <item> \ path, # hierarchical precursors
1707 <item> \ # local vars
1712 The strategy is to parse the name, and then look for a value. If the value
1713 begins with a brace <verbatim|{>, then we recurse and consume as much of
1714 the text as necessary, returning the remaining text when we encounter a
1715 leading close-brace <verbatim|}>. This being the strategy --- and executed
1716 in a loop --- we realise that we must first look for the closing brace
1717 (perhaps preceded by white space) in order to terminate the recursion, and
1718 returning remaining text.
1720 <\nf-chunk|get_chunk_args()>
1723 <item> \ split("", next_chunk_args);
1725 <item> \ while(length(text)) {
1727 <item> \ \ \ if (match(text, "^ *}(.*)", a)) {
1729 <item> \ \ \ \ \ return a[1];
1733 <item> \ \ \ =\<less\>\\chunkref{parse-chunk-args}\<gtr\>
1737 <item> \ return text;
1742 We can see that the text could be inspected with this regex:
1744 <\nf-chunk|parse-chunk-args>
1745 <item>if (! match(text, " *([^,=]*[^,= ]) *(([,=]) *(([^,}]*) *,*
1748 <item> \ return text;
1753 and that <verbatim|a> will have the following values:
1755 <tabular|<tformat|<cwith|2|7|1|2|cell-lborder|0.5pt>|<cwith|2|7|1|2|cell-rborder|0.5pt>|<cwith|2|7|1|2|cell-bborder|0.5pt>|<cwith|2|7|1|2|cell-tborder|0.5pt>|<cwith|1|1|1|2|cell-lborder|0.5pt>|<cwith|1|1|1|2|cell-rborder|0.5pt>|<cwith|1|1|1|2|cell-bborder|0.5pt>|<cwith|1|1|1|2|cell-tborder|0.5pt>|<table|<row|<cell|a[n]>|<cell|assigned
1756 text>>|<row|<cell|1>|<cell|freddie>>|<row|<cell|2>|<cell|=freddie,
1757 foo={bar=baz, quux={quirk, a=fleeg}}, etc>>|<row|<cell|3>|<cell|=>>|<row|<cell|4>|<cell|freddie,
1758 foo={bar=baz, quux={quirk, a=fleeg}}, etc>>|<row|<cell|5>|<cell|freddie>>|<row|<cell|6>|<cell|,
1759 foo={bar=baz, quux={quirk, a=fleeg}}, etc>>>>>
1761 <verbatim|a[3]> will be either <verbatim|=> or <verbatim|,> and signify
1762 whether the option named in <verbatim|a[1]> has a value or not
1765 If the option does have a value, then if the expression
1766 <verbatim|substr(a[4],1,1)> returns a brace <verbatim|{> it will signify
1767 that we need to recurse:
1769 <\nf-chunk|parse-chunk-args>
1772 <item>if (a[3] == "=") {
1774 <item> \ if (substr(a[4],1,1) == "{") {
1776 <item> \ \ \ text = get_chunk_args(substr(a[4],2), values, path name
1781 <item> \ \ \ values[path name]=a[5];
1783 <item> \ \ \ text = a[6];
1789 <item> \ values[path name]="";
1791 <item> \ text = a[2];
1796 We can test this function like this:
1798 <\nf-chunk|gca-test.awk>
1799 <item>=\<less\>\\chunkref{get_chunk_args()}\<gtr\>
1803 <item> \ SUBSEP=".";
1807 <item> \ print get_chunk_args("name=freddie, foo={bar=baz, quux={quirk,
1808 a=fleeg}}, etc", a);
1810 <item> \ for (b in a) {
1812 <item> \ \ \ print "a[" b "] =\<gtr\> " a[b];
1819 which should give this output:
1821 <\nf-chunk|gca-test.awk-results>
1822 <item>a[foo.quux.quirk] =\<gtr\>\
1824 <item>a[foo.quux.a] =\<gtr\> fleeg
1826 <item>a[foo.bar] =\<gtr\> baz
1828 <item>a[etc] =\<gtr\>\
1830 <item>a[name] =\<gtr\> freddie
1833 <section|Parsing chunk arguments><label|Chunk Arguments>
1835 Arguments to paramterized chunks are expressed in round brackets as a comma
1836 separated list of optional arguments. For example, a chunk that is defined
1839 <verbatim|\\Chunk{achunk, params=name ; address}>
1841 could be invoked as:
1843 <verbatim|\\chunkref{achunk}(John Jones, jones@example.com)>
1845 An argument list may be as simple as in <verbatim|\\chunkref{pull}(thing,
1846 otherthing)> or as complex as:
1848 <verbatim|\\chunkref{pull}(things[x, y], get_other_things(a, "(all)"))>
1850 --- which for all it's commas and quotes and parenthesis represents only
1851 two parameters: <verbatim|things[x, y]> and <verbatim|get_other_things(a,
1854 If we simply split parameter list on commas, then the comma in
1855 <verbatim|things[x,y]> would split into two seperate arguments:
1856 <verbatim|things[x> and <verbatim|y]>--- neither of which make sense on
1859 One way to prevent this would be by refusing to split text between matching
1860 delimiters, such as <verbatim|[>, <verbatim|]>, <verbatim|(>, <verbatim|)>,
1861 <verbatim|{>, <verbatim|}> and most likely also <verbatim|">, <verbatim|">
1862 and <verbatim|'>, <verbatim|'>. Of course this also makes it impossible to
1863 pass such mis-matched code fragments as parameters, but I think that it
1864 would be hard for readers to cope with authors who would pass such code
1865 unbalanced fragments as chunk parameters<\footnote>
1866 I know that I couldn't cope with users doing such things, and although
1867 the GPL3 license prevents me from actually forbidding anyone from trying,
1868 if they want it to work they'll have to write the code themselves and not
1869 expect any support from me.
1872 Unfortunately, the full set of matching delimiters may vary from language
1873 to language. In certain C++ template contexts, <verbatim|\<less\>> and
1874 <verbatim|\<gtr\>> would count as delimiters, and yet in other contexts
1877 This puts me in the unfortunate position of having to parse-somewhat all
1878 programming languages without knowing what they are!
1880 However, if this universal mode-tracking is possible, then parsing the
1881 arguments would be trivial. Such a mode tracker is described in chapter
1882 <reference|modes> and used here with simplicity.
1884 <\nf-chunk|parse_chunk_args>
1885 <item>function parse_chunk_args(language, text, values, mode,
1887 <item> \ # local vars
1889 <item> \ c, context, rest)
1893 <item> \ =\<less\>\\chunkref{new-mode-tracker}(context, language,
1896 <item> \ rest = mode_tracker(context, text, values);
1898 <item> \ # extract values
1900 <item> \ for(c=1; c \<less\>= context[0, "values"]; c++) {
1902 <item> \ \ \ values[c] = context[0, "values", c];
1906 <item> \ return rest;
1911 <section|Expanding parameters in the text>
1913 Within the body of the chunk, the parameters are referred to with:
1914 <verbatim|${name}> and <verbatim|${address}>. There is a strong case that a
1915 <LaTeX> style notation should be used, like <verbatim|\\param{name}> which
1916 would be expressed in the listing as <verbatim|=\<less\>\\param{name}\<gtr\>>
1917 and be rendered as <verbatim|<nf-arg|name>>. Such notation would make me go
1918 blind, but I do intend to adopt it.
1920 We therefore need a function <verbatim|expand_chunk_args> which will take a
1921 block of text, a list of permitted parameters, and the arguments which must
1922 substitute for the parameters.\
1924 Here we split the text on <verbatim|${> which means that all parts except
1925 the first will begin with a parameter name which will be terminated by
1926 <verbatim|}>. The split function will consume the literal <verbatim|${> in
1929 <\nf-chunk|expand_chunk_args()>
1930 <item>function expand_chunk_args(text, params, args, \
1932 <item> \ p, text_array, next_text, v, t, l)
1936 <item> \ if (split(text, text_array, "\\\\${")) {
1938 <item> \ \ \ =\<less\>\\chunkref{substitute-chunk-args}\<gtr\>
1944 <item> \ return text;
1949 First, we produce an associative array of substitution values indexed by
1950 parameter names. This will serve as a cache, allowing us to look up the
1951 replacement values as we extract each name.
1953 <\nf-chunk|substitute-chunk-args>
1954 <item>for(p in params) {
1956 <item> \ v[params[p]]=args[p];
1961 We accumulate substituted text in the variable text. As the first part of
1962 the split function is the part before the delimiter --- which is
1963 <verbatim|${> in our case --- this part will never contain a parameter
1964 reference, so we assign this directly to the result kept in
1968 <item>text=text_array[1];
1971 We then iterate over the remaining values in the array<\footnote>
1972 I don't know why I think that it will enumerate the array in order, but
1974 </footnote><todo|fix or prove it>, and substitute each reference for it's
1977 <\nf-chunk|substitute-chunk-args>
1978 <item>for(t=2; t in text_array; t++) {
1980 <item> \ =\<less\>\\chunkref{substitute-chunk-arg}\<gtr\>
1985 After the split on <verbatim|${> a valid parameter reference will consist
1986 of valid parameter name terminated by a close-brace <verbatim|}>. A valid
1987 character name begins with the underscore or a letter, and may contain
1988 letters, digits or underscores.
1990 A valid looking reference that is not actually the name of a parameter will
1991 be and not substituted. This is good because there is nothing to substitute
1992 anyway, and it avoids clashes when writing code for languages where
1993 <verbatim|${...}> is a valid construct --- such constructs will not be
1994 interfered with unless the parameter name also matches.
1996 <\nf-chunk|substitute-chunk-arg>
1997 <item>if (match(text_array[t], "^([a-zA-Z_][a-zA-Z0-9_]*)}", l) &&
1999 <item> \ \ \ l[1] in v)\
2003 <item> \ text = text v[l[1]] substr(text_array[t], length(l[1])+2);
2007 <item> \ text = text "${" text_array[t];
2012 <chapter|Language Modes & Quoting><label|modes>
2016 <verbatim|lstlistings> and <verbatim|fangle> both recognize source
2017 languages, and perform some basic parsing. <verbatim|lstlistings> can
2018 detect strings and comments within a language definition and perform
2019 suitable rendering, such as italics for comments, and visible-spaces within
2022 Fangle similarly can recognize strings, and comments, etc, within a
2023 language, so that any chunks included with <verbatim|\\chunkref> can be
2024 suitably escape or quoted.
2026 <subsection|Modes to keep code together>
2028 As an example, in the C language there are a few parse modes, affecting the
2029 interpretation of characters.
2031 One parse mode is the strings mode. The string mode is commenced by an
2032 un-escaped quotation mark <verbatim|"> and terminated by the same. Within
2033 the string mode, only one additional mode can be commenced, it is the
2034 backslash mode <verbatim|\\>, which is always terminated after the folloing
2037 Another mode is <verbatim|[> which is terminated by a <verbatim|]> (unless
2038 it occurs in a string).
2040 Consider this fragment of C code:
2044 <math|things<wide|<around|[|x, y|]>|\<wide-overbrace\>><rsup|1. [ mode>,
2045 get_other_things<wide|<around|(|a, <wide*|<text|"><around|(|all|)><text|">|\<wide-underbrace\>><rsub|3.
2046 " mode>|)>|\<wide-overbrace\>><rsup|2. ( mode>>
2050 Mode nesting prevents the close parenthesis in quote mode (part 3) from
2051 terminating the parenthesis mode (part 2).
2053 Each language has a set of modes, the default mode being the null mode.
2054 Each mode can lead to other modes.
2056 <subsection|Modes affect included chunks>
2058 For instance, consider this chunk with language=perl:
2060 <nf-chunk|example-perl|print "hello world $0\\n";|perl|>
2062 If it were included in a chunk with <verbatim|language=sh>, like this:
2064 <nf-chunk|example-sh|perl -e "=\<less\>\\chunkref{example-perl}\<gtr\>"|sh|>
2066 fangle would need to generate output like this:
2068 <verbatim|perl -e "print \\"hello world \\$0\\\\n\\";" >
2070 See that the double quote <verbatim|">, back-slash <verbatim|\\> and
2071 <verbatim|$> have been quoted with a back-slash to protect them from shell
2074 If that were then included in a chunk with language=make, like this:
2076 <\nf-chunk|example-makefile>
2077 <item>target: pre-req
2079 <item><htab|5mm>=\<less\>\\chunkref{example-sh}\<gtr\>
2082 We would need the output to look like this --- note the <verbatim|$$>:
2087 \ \ \ \ \ \ \ \ perl -e "print \\"hello world \\$$0\\\\n\\";"
2090 In order to make this work, we need to define a mode-tracker supporting
2091 each language, that can detect the various quoting modes, and provide a
2092 transformation that must be applied to any included text so that included
2093 text will be interpreted correctly after any interpolation that it may be
2094 subject to at run-time.
2096 For example, the sed transformation for text to be inserted into shell
2097 double-quoted strings would be something like:
2099 <verbatim|s/\\\\/\\\\\\\\/g;s/$/\\\\$/g;s/"/\\\\"/g;>
2101 which protects <verbatim|\\ $ ">.
2103 <todo|I don't think this example is true>The mode tracker must also track
2104 nested mode-changes, as in this sh example.
2106 <verbatim|echo "hello \0id ...\0">
2108 <phantom|<verbatim|echo "hello \0id >><math|\<uparrow\>>
2110 Any characters inserted at the point marked <math|\<uparrow\>> would need
2111 to be escaped, including <verbatim|\0> <verbatim|\|> <verbatim|*> among
2112 others. First it would need escaping for the back-ticks <verbatim|\0>, and
2113 then for the double-quotes <verbatim|">.
2115 <todo|MAYBE>Escaping need not occur if the format and mode of the included
2116 chunk matches that of the including chunk.
2118 As each chunk is output a new mode tracker for that language is initialized
2119 in it's normal state. As text is output for that chunk the output mode is
2120 tracked. When a new chunk is included, a transformation appropriate to that
2121 mode is selected and pushed onto a stack of transformations. Any text to be
2122 output is first passed through this stack of transformations.
2124 It remains to consider if the chunk-include function should return it's
2125 generated text so that the caller can apply any transformations (and
2126 formatting), or if it should apply the stack of transformations itself.
2128 Note that the transformed text should have the property of not being able
2129 to change the mode in the current chunk.
2131 <todo|Note chunk parameters should probably also be transformed>
2133 <section|Language Mode Definitions>
2135 All modes are stored in a single multi-dimensional hash. The first index is
2136 the language, and the second index is the mode-identifier. The third
2137 indexes are terminators, and optionally, submodes, and delimiters.
2139 A useful set of mode definitions for a nameless general C-type language is
2140 shown here. (Don't be confused by the double backslash escaping needed in
2141 awk. One set of escaping is for the string, and the second set of escaping
2145 TODO: Add =\<less\>\\mode{}\<gtr\> command which will allow us to signify
2148 \ regex and thus fangle will quote it for us.
2151 Submodes are entered by the characters \ <verbatim|"> <verbatim|'>
2152 <verbatim|{> <verbatim|(> <verbatim|[> <verbatim|/*>
2154 <\nf-chunk|common-mode-definitions>
2155 <item>modes[${language}, "", \ "submodes"
2156 ]="\\\\\\\\\|\\"\|'\|{\|\\\\(\|\\\\[";
2157 </nf-chunk||language>
2159 In the default mode, a comma surrounded by un-important white space is a
2160 delimiter of language items.
2162 <\nf-chunk|common-mode-definitions>
2163 <item>modes[${language}, "", \ "delimiters"]=" *, *";
2164 </nf-chunk||language>
2166 and should pass this test:<todo|Why do the tests run in ?(? mode and not ??
2169 <\nf-chunk|test:mode-definitions>
2170 <item>parse_chunk_args("c-like", "1,2,3", a, "");
2172 <item>if (a[1] != "1") e++;
2174 <item>if (a[2] != "2") e++;
2176 <item>if (a[3] != "3") e++;
2178 <item>if (length(a) != 3) e++;
2180 <item>=\<less\>\\chunkref{pca-test.awk:summary}\<gtr\>
2184 <item>parse_chunk_args("c-like", "joe, red", a, "");
2186 <item>if (a[1] != "joe") e++;
2188 <item>if (a[2] != "red") e++;
2190 <item>if (length(a) != 2) e++;
2192 <item>=\<less\>\\chunkref{pca-test.awk:summary}\<gtr\>
2196 <item>parse_chunk_args("c-like", "${colour}", a, "");
2198 <item>if (a[1] != "${colour}") e++;
2200 <item>if (length(a) != 1) e++;
2202 <item>=\<less\>\\chunkref{pca-test.awk:summary}\<gtr\>
2205 Nested modes are identified by a backslash, a double or single quote,
2206 various bracket styles or a <verbatim|/*> comment.
2208 For each of these sub-modes modes we must also identify at a mode
2209 terminator, and any sub-modes or delimiters that may be entered<\footnote>
2210 Because we are using the sub-mode characters as the mode identifier it
2211 means we can't currently have a mode character dependant on it's context;
2212 i.e. <verbatim|{> can't behave differently when it is inside
2216 <subsection|Backslash>
2218 The backslash mode has no submodes or delimiters, and is terminated by any
2219 character. Note that we are not so much interested in evaluating or
2220 interpolating content as we are in delineating content. It is no matter
2221 that a double backslash (<verbatim|\\\\>) may represent a single backslash
2222 while a backslash-newline may represent white space, but it does matter
2223 that the newline in a backslash newline should not be able to terminate a C
2224 pre-processor statement; and so the newline will be consumed by the
2225 backslash however it is to be interpreted.
2227 <nf-chunk|common-mode-definitions|modes[${language}, "\\\\",
2228 "terminators"]=".";||>
2230 <subsection|Strings>
2232 Common languages support two kinds of strings quoting, double quotes and
2235 In a string we have one special mode, which is the backslash. This may
2236 escape an embedded quote and prevent us thinking that it should terminate
2239 <\nf-chunk|mode:common-string>
2240 <item>modes[${language}, "\\\\", "terminators"]=".";
2241 </nf-chunk||language>
2243 Otherwise, the string will be terminated by the same character that
2246 <\nf-chunk|mode:common-string>
2247 <item>modes[${language}, ${quote}, "terminators"]=${quote};
2248 </nf-chunk||language>
2250 In C type languages, certain escape sequences exist in strings. We need to
2251 define mechanism to enclode any chunks included in this mode using those
2252 escape sequences. These are expressed in two parts, s meaning search, and r
2255 The first substitution is to replace a backslash with a double backslash.
2256 We do this first as other substitutions may introduce a backslash which we
2257 would not then want to escape again here.
2259 Note: Backslashes need double-escaping in the search pattern but not in the
2260 replacement string, hence we are replacing a literal <verbatim|\\> with a
2261 literal <verbatim|\\\\>.
2264 <item>escapes[${language}, ${quote}, ++escapes[${language}, ${quote}],
2267 <item>escapes[${language}, ${quote}, \ \ escapes[${language}, ${quote}],
2271 If the quote character occurs in the text, it should be preceded by a
2272 backslash, otherwise it would terminate the string unexpectedly.
2275 <item>escapes[${language}, ${quote}, ++escapes[${language}, ${quote}],
2278 <item>escapes[${language}, ${quote}, \ \ escapes[${language}, ${quote}],
2279 "r"]="\\\\" ${quote};
2282 Any newlines in the string, must be replaced by <verbatim|\\n>.
2285 <item>escapes[${language}, ${quote}, ++escapes[${language}, ${quote}],
2288 <item>escapes[${language}, ${quote}, \ \ escapes[${language}, ${quote}],
2292 For the common modes, we define this string handling for double and single
2295 <\nf-chunk|common-mode-definitions>
2296 <item>=\<less\>\\chunkref{mode:common-string}(${language},
2297 "\\textbackslash{}"")\<gtr\>
2299 <item>=\<less\>\\chunkref{mode:common-string}(${language}, "'")\<gtr\>
2302 Working strings should pass this test:
2304 <\nf-chunk|test:mode-definitions>
2305 <item>parse_chunk_args("c-like", "say \\"I said, \\\\\\"Hello, how are
2306 you\\\\\\".\\", for me", a, "");
2308 <item>if (a[1] != "say \\"I said, \\\\\\"Hello, how are you\\\\\\".\\"")
2311 <item>if (a[2] != "for me") e++;
2313 <item>if (length(a) != 2) e++;
2315 <item>=\<less\>\\chunkref{pca-test.awk:summary}\<gtr\>
2318 <subsection|Parentheses, Braces and Brackets>
2320 Where quotes are closed by the same character, parentheses, brackets and
2321 braces are closed by an alternate character.
2323 <\nf-chunk|mode:common-brackets>
2324 <item>modes[<nf-arg|language>, <nf-arg|open>, \ "submodes"
2325 ]="\\\\\\\\\|\\"\|{\|\\\\(\|\\\\[\|'\|/\\\\*";
2327 <item>modes[<nf-arg|language>, <nf-arg|open>, \ "delimiters"]=" *, *";
2329 <item>modes[<nf-arg|language>, <nf-arg|open>,
2330 \ "terminators"]=<nf-arg|close>;
2331 </nf-chunk||<tuple|language|open|close>>
2333 Note that the open is NOT a regex but the close token IS. <todo|When we can
2334 quote regex we won't have to put the slashes in here>
2336 <\nf-chunk|common-mode-definitions>
2337 <item>=\<less\>\\chunkref{mode:common-brackets}(${language}, "{",
2340 <item>=\<less\>\\chunkref{mode:common-brackets}(${language}, "[",
2341 "\\textbackslash{}\\textbackslash{}]")\<gtr\>
2343 <item>=\<less\>\\chunkref{mode:common-brackets}(${language}, "(",
2344 "\\textbackslash{}\\textbackslash{})")\<gtr\>
2347 <subsection|Customizing Standard Modes>
2349 <\nf-chunk|mode:add-submode>
2350 <item>modes[${language}, ${mode}, "submodes"] = modes[${language},
2351 ${mode}, "submodes"] "\|" ${submode};
2352 </nf-chunk||<tuple|language|mode|submode>>
2354 <\nf-chunk|mode:add-escapes>
2355 <item>escapes[${language}, ${mode}, ++escapes[${language}, ${mode}],
2358 <item>escapes[${language}, ${mode}, \ \ escapes[${language}, ${mode}],
2360 </nf-chunk||<tuple|language|mode|search|replace>>
2364 <subsection|Comments>
2366 We can define <verbatim|/* comment */> style comments and
2367 <verbatim|//comment> style comments to be added to any language:
2369 <\nf-chunk|mode:multi-line-comments>
2370 <item>=\<less\>\\chunkref{mode:add-submode}(${language}, "",
2371 "/\\textbackslash{}\\textbackslash{}*")\<gtr\>
2373 <item>modes[${language}, "/*", "terminators"]="\\\\*/";
2374 </nf-chunk||<tuple|language>>
2376 <\nf-chunk|mode:single-line-slash-comments>
2377 <item>=\<less\>\\chunkref{mode:add-submode}(${language}, "", "//")\<gtr\>
2379 <item>modes[${language}, "//", "terminators"]="\\n";
2381 <item>=\<less\>\\chunkref{mode:add-escapes}(${language}, "//",
2382 "\\textbackslash{}n", "\\textbackslash{}n//")\<gtr\>
2383 </nf-chunk||language>
2385 We can also define <verbatim|# comment> style comments (as used in awk and
2386 shell scripts) in a similar manner.
2388 <todo|I'm having to use # for hash and \textbackslash{} for \ and have
2389 hacky work-arounds in the parser for now>
2391 <\nf-chunk|mode:add-hash-comments>
2392 <item>=\<less\>\\chunkref{mode:add-submode}(${language}, "",
2395 <item>modes[${language}, "#", "terminators"]="\\n";
2397 <item>=\<less\>\\chunkref{mode:add-escapes}(${language}, "\\#",
2398 "\\textbackslash{}n", "\\textbackslash{}n\\#")\<gtr\>
2399 </nf-chunk||<tuple|language>>
2401 In C, the <verbatim|#> denotes pre-processor directives which can be
2404 <\nf-chunk|mode:add-hash-defines>
2405 <item>=\<less\>\\chunkref{mode:add-submode}(${language}, "",
2408 <item>modes[${language}, "#", "submodes" ]="\\\\\\\\";
2410 <item>modes[${language}, "#", "terminators"]="\\n";
2412 <item>=\<less\>\\chunkref{mode:add-escapes}(${language}, "\\#",
2413 "\\textbackslash{}n", "\\textbackslash{}\\textbackslash{}\\textbackslash{}\\textbackslash{}\\textbackslash{}n")\<gtr\>
2414 </nf-chunk||<tuple|language>>
2416 <\nf-chunk|mode:quote-dollar-escape>
2417 <item>escapes[${language}, ${quote}, ++escapes[${language}, ${quote}],
2420 <item>escapes[${language}, ${quote}, \ \ escapes[${language}, ${quote}],
2422 </nf-chunk||<tuple|language|quote>>
2424 We can add these definitions to various languages
2426 <\nf-chunk|mode-definitions>
2427 <item><nf-ref|common-mode-definitions|<tuple|"c-like">>
2431 <item><nf-ref|common-mode-definitions|<tuple|"c">>
2433 <item>=\<less\>\\chunkref{mode:multi-line-comments}("c")\<gtr\>
2435 <item>=\<less\>\\chunkref{mode:single-line-slash-comments}("c")\<gtr\>
2437 <item>=\<less\>\\chunkref{mode:add-hash-defines}("c")\<gtr\>
2441 <item>=\<less\>\\chunkref{common-mode-definitions}("awk")\<gtr\>
2443 <item>=\<less\>\\chunkref{mode:add-hash-comments}("awk")\<gtr\>
2445 <item>=\<less\>\\chunkref{mode:add-naked-regex}("awk")\<gtr\>
2448 The awk definitions should allow a comment block like this:
2450 <nf-chunk|test:comment-quote|# Comment:
2451 =\<less\>\\chunkref{test:comment-text}\<gtr\>|awk|>
2453 <\nf-chunk|test:comment-text>
2454 <item>Now is the time for
2456 <item>the quick brown fox to bring lemonade
2461 to come out like this:
2463 <\nf-chunk|test:comment-quote:result>
2464 <item># Comment: Now is the time for
2466 <item>#the quick brown fox to bring lemonade
2471 The C definition for such a block should have it come out like this:
2473 <\nf-chunk|test:comment-quote:C-result>
2474 <item># Comment: Now is the time for\\
2476 <item>the quick brown fox to bring lemonade\\
2483 This pattern is incomplete, but meant to detect naked regular expressions
2484 in awk and perl; e.g. <verbatim|/.*$/>, however required capabilities are
2487 Current it only detects regexes anchored with ^ as used in fangle.
2489 For full regex support, modes need to be named not after their starting
2490 character, but some other more fully qualified name.
2492 <\nf-chunk|mode:add-naked-regex>
2493 <item>=\<less\>\\chunkref{mode:add-submode}(${language}, "",
2494 "/\\textbackslash{}\\textbackslash{}\\^")\<gtr\>
2496 <item>modes[${language}, "/^", "terminators"]="/";
2497 </nf-chunk||<tuple|language>>
2501 <\nf-chunk|mode-definitions>
2502 <item>=\<less\>\\chunkref{common-mode-definitions}("perl")\<gtr\>
2504 <item>=\<less\>\\chunkref{mode:multi-line-comments}("perl")\<gtr\>
2506 <item>=\<less\>\\chunkref{mode:add-hash-comments}("perl")\<gtr\>
2509 Still need to add add <verbatim|s/>, submode <verbatim|/>, terminate both
2510 with <verbatim|//>. This is likely to be impossible as perl regexes can
2515 <\nf-chunk|mode-definitions>
2516 <item>=\<less\>\\chunkref{common-mode-definitions}("sh")\<gtr\>
2518 <item>#\<less\>\\chunkref{mode:common-string}("sh",
2519 "\\textbackslash{}"")\<gtr\>
2521 <item>#\<less\>\\chunkref{mode:common-string}("sh", "'")\<gtr\>
2523 <item>=\<less\>\\chunkref{mode:add-hash-comments}("sh")\<gtr\>
2525 <item>=\<less\>\\chunkref{mode:quote-dollar-escape}("sh", "\\"")\<gtr\>
2528 <section|Some tests>
2530 Also, the parser must return any spare text at the end that has not been
2531 processed due to a mode terminator being found.
2533 <\nf-chunk|test:mode-definitions>
2534 <item>rest = parse_chunk_args("c-like", "1, 2, 3) spare", a, "(");
2536 <item>if (a[1] != 1) e++;
2538 <item>if (a[2] != 2) e++;
2540 <item>if (a[3] != 3) e++;
2542 <item>if (length(a) != 3) e++;
2544 <item>if (rest != " spare") e++;
2546 <item>=\<less\>\\chunkref{pca-test.awk:summary}\<gtr\>
2549 We must also be able to parse the example given earlier.
2551 <\nf-chunk|test:mode-definitions>
2552 <item>parse_chunk_args("c-like", "things[x, y], get_other_things(a,
2553 \\"(all)\\"), 99", a, "(");
2555 <item>if (a[1] != "things[x, y]") e++;
2557 <item>if (a[2] != "get_other_things(a, \\"(all)\\")") e++;
2559 <item>if (a[3] != "99") e++;
2561 <item>if (length(a) != 3) e++;
2563 <item>=\<less\>\\chunkref{pca-test.awk:summary}\<gtr\>
2566 <section|A non-recursive mode tracker>
2568 <subsection|Constructor>
2570 The mode tracker holds its state in a stack based on a hash. This function,
2571 when passed an empty hash will intialize it.
2573 <\nf-chunk|new-mode-tracker()>
2574 <item>function new_mode_tracker(context, language, mode) {
2576 <item> \ context[""] = 0;
2578 <item> \ context[0, "language"] = language;
2580 <item> \ context[0, "mode"] = mode;
2585 Because awk functions cannot return an array, we must create the array
2586 first and pass it in, so we have a fangle macro to do this:
2588 <\nf-chunk|new-mode-tracker>
2589 <item>=\<less\>\\chunkref{awk-delete-array}(${context})\<gtr\>
2591 <item>new_mode_tracker(${context}, ${language}, ${mode});
2592 </nf-chunk|awk|<tuple|context|language|mode>>
2594 <subsection|Management>
2596 And for tracking modes, we dispatch to a mode-tracker action based on the
2599 <\nf-chunk|mode_tracker>
2600 <item>function push_mode_tracker(context, language, mode,
2602 <item> \ # local vars
2608 <item> \ if (! ("" in context)) {
2610 <item> \ \ \ =\<less\>\\chunkref{new-mode-tracker}(context, language,
2615 <item> \ \ \ top = context[""];
2617 <item> \ \ \ if (context[top, "language"] == language && mode=="") mode =
2618 context[top, "mode"];
2622 <item> \ \ \ context[top, "language"] = language;
2624 <item> \ \ \ context[top, "mode"] = mode;
2626 <item> \ \ \ context[""] = top;
2633 <\nf-chunk|mode_tracker>
2634 <item>function dump_mode_tracker(context, \
2640 <item> \ for(c=0; c \<less\>= context[""]; c++) {
2642 <item> \ \ \ printf(" %2d \ \ %s:%s\\n", c, context[c, "language"],
2643 context[c, "mode"]) \<gtr\> "/dev/stderr";
2645 <item> \ \ \ for(d=1; ( (c, "values", d) in context); d++) {
2647 <item> \ \ \ \ \ printf(" \ \ %2d %s\\n", d, context[c, "values", d])
2648 \<gtr\> "/dev/stderr";
2657 <\nf-chunk|mode_tracker>
2658 <item>function finalize_mode_tracker(context)
2662 <item> \ if ( ("" in context) && context[""] != 0) return 0;
2669 This implies that any chunk must be syntactically whole; for instance, this
2672 <\nf-chunk|test:whole-chunk>
2675 <item> \ =\<less\>\\chunkref{test:say-hello}\<gtr\>
2680 <\nf-chunk|test:say-hello>
2681 <item>print "hello";
2684 But this is not fine; the chunk <nf-ref|test:hidden-else|> is not properly
2687 <\nf-chunk|test:partial-chunk>
2690 <item> \ =\<less\>\\chunkref{test:hidden-else}\<gtr\>
2695 <\nf-chunk|test:hidden-else>
2696 <item> \ print "I'm fine";
2700 <item> \ print "I'm not";
2703 These tests will check for correct behaviour:
2705 <\nf-chunk|test:cromulence>
2706 <item>echo Cromulence test
2708 <item>passtest $FANGLE -Rtest:whole-chunk $TEX_SRC &\<gtr\>/dev/null \|\|
2709 ( echo "Whole chunk failed" && exit 1 )
2711 <item>failtest $FANGLE -Rtest:partial-chunk $TEX_SRC &\<gtr\>/dev/null
2712 \|\| ( echo "Partial chunk failed" && exit 1 )
2715 <subsection|Tracker>
2717 We must avoid recursion as a language construct because we intend to employ
2718 mode-tracking to track language mode of emitted code, and the code is
2719 emitted from a function which is itself recursive, so instead we implement
2720 psuedo-recursion using our own stack based on a hash.
2722 <\nf-chunk|mode_tracker()>
2723 <item>function mode_tracker(context, text, values,\
2725 <item> \ # optional parameters
2727 <item> \ # local vars
2729 <item> \ mode, submodes, language,
2731 <item> \ cindex, c, a, part, item, name, result, new_values, new_mode,\
2733 <item> \ delimiters, terminators)
2738 We could be re-commencing with a valid context, so we need to setup the
2739 state according to the last context.
2741 <\nf-chunk|mode_tracker()>
2742 <item> \ cindex = context[""] + 0;
2744 <item> \ mode = context[cindex, "mode"];
2746 <item> \ language = context[cindex, "language" ];
2749 First we construct a single large regex combining the possible sub-modes
2750 for the current mode along with the terminators for the current mode.
2752 <\nf-chunk|parse_chunk_args-reset-modes>
2753 <item> \ submodes=modes[language, mode, "submodes"];
2757 <item> \ if ((language, mode, "delimiters") in modes) {
2759 <item> \ \ \ delimiters = modes[language, mode, "delimiters"];
2761 <item> \ \ \ if (length(submodes)\<gtr\>0) submodes = submodes "\|";
2763 <item> \ \ \ submodes=submodes delimiters;
2765 <item> \ } else delimiters="";
2767 <item> \ if ((language, mode, "terminators") in modes) {
2769 <item> \ \ \ terminators = modes[language, mode, "terminators"];
2771 <item> \ \ \ if (length(submodes)\<gtr\>0) submodes = submodes "\|";
2773 <item> \ \ \ submodes=submodes terminators;
2775 <item> \ } else terminators="";
2778 If we don't find anything to match on --- probably because the language is
2779 not supported --- then we return the entire text without matching anything.
2781 <\nf-chunk|parse_chunk_args-reset-modes>
2782 <item> if (! length(submodes)) return text;
2785 <\nf-chunk|mode_tracker()>
2786 <item>=\<less\>\\chunkref{parse_chunk_args-reset-modes}\<gtr\>
2789 We then iterate the text (until there is none left) looking for sub-modes
2790 or terminators in the regex.
2792 <\nf-chunk|mode_tracker()>
2793 <item> \ while((cindex \<gtr\>= 0) && length(text)) {
2795 <item> \ \ \ if (match(text, "(" submodes ")", a)) {
2798 A bug that creeps in regularly during development is bad regexes of zero
2799 length which result in an infinite loop (as no text is consumed), so I
2800 catch that right away with this test.
2802 <\nf-chunk|mode_tracker()>
2803 <item> \ \ \ \ \ if (RLENGTH\<less\>1) {
2805 <item> \ \ \ \ \ \ \ error(sprintf("Internal error, matched zero length
2806 submode, should be impossible - likely regex computation error\\n" \\
2808 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ "Language=%s\\nmode=%s\\nmatch=%s\\n",
2809 language, mode, submodes));
2814 part is defined as the text up to the sub-mode or terminator, and this is
2815 appended to item --- which is the current text being gathered. If a mode
2816 has a delimiter, then item is reset each time a delimiter is found.
2818 <math|<wide|<with|mode|prog|"><wide*|hello|\<wide-underbrace\>><rsub|item>,
2819 <wide*|there|\<wide-underbrace\>><rsub|item><with|mode|prog|">|\<wide-overbrace\>><rsup|item>,
2820 \ <wide|he said.|\<wide-overbrace\>><rsup|item>>
2822 <\nf-chunk|mode_tracker()>
2823 <item> \ \ \ \ \ part = substr(text, 1, RSTART -1);
2825 <item> \ \ \ \ \ item = item part;
2828 We must now determine what was matched. If it was a terminator, then we
2829 must restore the previous mode.
2831 <\nf-chunk|mode_tracker()>
2832 <item> \ \ \ \ \ if (match(a[1], "^" terminators "$")) {
2834 <item>#printf("%2d EXIT \ MODE [%s] by [%s] [%s]\\n", cindex, mode, a[1],
2835 text) \<gtr\> "/dev/stderr"
2837 <item> \ \ \ \ \ \ \ context[cindex, "values", ++context[cindex,
2840 <item> \ \ \ \ \ \ \ delete context[cindex];
2842 <item> \ \ \ \ \ \ \ context[""] = --cindex;
2844 <item> \ \ \ \ \ \ \ if (cindex\<gtr\>=0) {
2846 <item> \ \ \ \ \ \ \ \ \ mode = context[cindex, "mode"];
2848 <item> \ \ \ \ \ \ \ \ \ language = context[cindex, "language"];
2850 <item> \ \ \ \ \ \ \ \ \ =\<less\>\\chunkref{parse_chunk_args-reset-modes}\<gtr\>
2852 <item> \ \ \ \ \ \ \ }
2854 <item> \ \ \ \ \ \ \ item = item a[1];
2856 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2862 If a delimiter was matched, then we must store the current item in the
2863 parsed values array, and reset the item.
2865 <\nf-chunk|mode_tracker()>
2866 <item> \ \ \ \ \ else if (match(a[1], "^" delimiters "$")) {
2868 <item> \ \ \ \ \ \ \ if (cindex==0) {
2870 <item> \ \ \ \ \ \ \ \ \ context[cindex, "values", ++context[cindex,
2873 <item> \ \ \ \ \ \ \ \ \ item = "";
2875 <item> \ \ \ \ \ \ \ } else {
2877 <item> \ \ \ \ \ \ \ \ \ item = item a[1];
2879 <item> \ \ \ \ \ \ \ }
2881 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2887 otherwise, if a new submode is detected (all submodes have terminators), we
2888 must create a nested parse context until we find the terminator for this
2891 <\nf-chunk|mode_tracker()>
2892 <item> else if ((language, a[1], "terminators") in modes) {
2894 <item> \ \ \ \ \ \ \ #check if new_mode is defined
2896 <item> \ \ \ \ \ \ \ item = item a[1];
2898 <item>#printf("%2d ENTER MODE [%s] in [%s]\\n", cindex, a[1], text)
2899 \<gtr\> "/dev/stderr"
2901 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2904 <item> \ \ \ \ \ \ \ context[""] = ++cindex;
2906 <item> \ \ \ \ \ \ \ context[cindex, "mode"] = a[1];
2908 <item> \ \ \ \ \ \ \ context[cindex, "language"] = language;
2910 <item> \ \ \ \ \ \ \ mode = a[1];
2912 <item> \ \ \ \ \ \ \ =\<less\>\\chunkref{parse_chunk_args-reset-modes}\<gtr\>
2914 <item> \ \ \ \ \ } else {
2916 <item> \ \ \ \ \ \ \ error(sprintf("Submode '%s' set unknown mode in
2917 text: %s\\nLanguage %s Mode %s\\n", a[1], text, language, mode));
2919 <item> \ \ \ \ \ \ \ text = substr(text, 1 + length(part) +
2927 In the final case, we parsed to the end of the string. If the string was
2928 entire, then we should have no nested mode context, but if the string was
2929 just a fragment we may have a mode context which must be preserved for the
2930 next fragment. Todo: Consideration ought to be given if sub-mode strings
2931 are split over two fragments.
2933 <\nf-chunk|mode_tracker()>
2936 <item> \ \ \ \ \ context[cindex, "values", ++context[cindex, "values"]] =
2939 <item> \ \ \ \ \ text = "";
2941 <item> \ \ \ \ \ item = "";
2949 <item> \ context["item"] = item;
2953 <item> \ if (length(item)) context[cindex, "values", ++context[cindex,
2956 <item> \ return text;
2961 <subsubsection|One happy chunk>
2963 All the mode tracker chunks are referred to here:
2965 <\nf-chunk|mode-tracker>
2966 <item>=\<less\>\\chunkref{new_mode_tracker()}\<gtr\>
2968 <item>=\<less\>\\chunkref{mode_tracker()}\<gtr\>
2971 <subsubsection|Tests>
2973 We can test this function like this:
2975 <\nf-chunk|pca-test.awk>
2976 <item>=\<less\>\\chunkref{error()}\<gtr\>
2978 <item>=\<less\>\\chunkref{mode-tracker}\<gtr\>
2980 <item>=\<less\>\\chunkref{parse_chunk_args()}\<gtr\>
2984 <item> \ SUBSEP=".";
2986 <item> \ =\<less\>\\chunkref{mode-definitions}\<gtr\>
2990 <item> \ =\<less\>\\chunkref{test:mode-definitions}\<gtr\>
2995 <\nf-chunk|pca-test.awk:summary>
2998 <item> \ printf "Failed " e
3000 <item> \ for (b in a) {
3002 <item> \ \ \ print "a[" b "] =\<gtr\> " a[b];
3008 <item> \ print "Passed"
3017 which should give this output:
3019 <\nf-chunk|pca-test.awk-results>
3020 <item>a[foo.quux.quirk] =\<gtr\>\
3022 <item>a[foo.quux.a] =\<gtr\> fleeg
3024 <item>a[foo.bar] =\<gtr\> baz
3026 <item>a[etc] =\<gtr\>\
3028 <item>a[name] =\<gtr\> freddie
3031 <section|Escaping and Quoting>
3033 Each nested mode can optionally define a set of transforms to be applied to
3034 any text that is included from another language.
3036 This code can perform transforms
3038 <\nf-chunk|mode_tracker>
3039 <item>function transform_escape(s, r, text,
3041 <item> \ \ \ # optional
3045 <item> \ \ \ \ \ \ \ # local vars
3047 <item> \ \ \ \ \ \ \ c)
3051 <item> \ for(c=1; c \<less\>= max && (c in s); c++) {
3053 <item> \ \ \ gsub(s[c], r[c], text);
3057 <item> \ return text;
3062 This function must append from index c onwards, and escape transforms from
3063 the supplied context, and return c + number of new transforms.
3065 <\nf-chunk|mode_tracker>
3066 <item>function mode_escaper(context, s, r, src,
3068 <item> \ c, cp, cpl)
3072 <item> \ \ \ \ \ \ \ for(c = context[""]; c \<gtr\>= 0; c--) {
3074 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ if ( (context[c, "language"],
3075 context[c, "mode"]) in escapes) {
3077 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ cpl =
3078 escapes[context[c, "language"], context[c, "mode"]];
3080 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ for (cp = 1; cp
3081 \<less\>= cpl; cp ++) {
3083 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ ++src;
3085 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ s[src]
3086 = escapes[context[c, "language"], context[c, "mode"], cp, "s"];
3088 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ r[src]
3089 = escapes[context[c, "language"], context[c, "mode"], cp, "r"];
3091 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ }
3093 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ }
3095 <item> \ \ \ \ \ \ \ }
3097 <item> \ \ \ \ \ \ \ return src;
3101 <item>function dump_escaper(c, s, r, cc) {
3103 <item> \ for(cc=1; cc\<less\>=c; cc++) {
3105 <item> \ \ \ printf("%2d s[%s] r[%s]\\n", cc, s[cc], r[cc]) \<gtr\>
3113 <\nf-chunk|test:escapes>
3114 <item>echo escapes test
3116 <item>passtest $FANGLE -Rtest:comment-quote $TEX_SRC &\<gtr\>/dev/null
3117 \|\| ( echo "Comment-quote failed" && exit 1 )
3120 <chapter|Recognizing Chunks>
3122 Fangle recognizes noweb chunks, but as we also want better <LaTeX>
3123 integration we will recognize any of these:
3126 <item>notangle chunks matching the pattern
3127 <verbatim|^\<less\>\<less\>.*?\<gtr\>\<gtr\>=>
3129 <item>chunks beginning with <verbatim|\\begin{lstlistings}>, possibly
3130 with <verbatim|\\Chunk{...}> on the previous line
3132 <item>an older form I have used, beginning with
3133 <verbatim|\\begin{Chunk}[options]> --- also more suitable for plain
3134 <LaTeX> users<\footnote>
3135 Is there such a thing as plain <LaTeX>?
3139 <section|Chunk start>
3141 The variable chunking is used to signify that we are processing a code
3142 chunk and not document. In such a state, input lines will be assigned to
3143 the current chunk; otherwise they are ignored.
3145 <subsection|lstlistings>
3147 Our current scheme is to recognize the new lstlisting chunks, but these may
3148 be preceded by a <verbatim|\\Chunk> command which in <LyX> is a more
3149 convenient way to pass the chunk name to the
3150 <verbatim|\\begin{lstlistings}> command, and a more visible way to specify
3151 other <verbatim|lstset> settings.
3153 The arguments to the <verbatim|\\Chunk> command are a name, and then a
3154 comma-seperated list of key-value pairs after the manner of
3155 <verbatim|\\lstset>. (In fact within the <LaTeX> <verbatim|\\Chunk> macro
3156 (section <reference|sub:The-chunk-command>) the text <verbatim|name=> is
3157 prefixed to the argument which is then literally passed to
3158 <verbatim|\\lstset>).
3160 <\nf-chunk|recognize-chunk>
3161 <item>/^\\\\Chunk{/ {
3163 <item> \ if (match($0, "^\\\\\\\\Chunk{ *([^ ,}]*),?(.*)}", line)) {
3165 <item> \ \ \ next_chunk_name = line[1];
3167 <item> \ \ \ get_chunk_args(line[2], next_chunk_args);
3176 We also make a basic attempt to parse the name out of the
3177 <verbatim|\\lstlistings[name=chunk-name]> text, otherwise we fall back to
3178 the name found in the previous chunk command. This attempt is very basic
3179 and doesn't support commas or spaces or square brackets as part of the
3180 chunkname. We also recognize <verbatim|\\begin{Chunk}> which is convenient
3181 for some users<\footnote>
3182 but not yet supported in the <LaTeX> macros
3185 <\nf-chunk|recognize-chunk>
3186 <item>/^\\\\begin{lstlisting}\|^\\\\begin{Chunk}/ {
3188 <item> \ if (match($0, "}.*[[,] *name= *{? *([^], }]*)", line)) {
3190 <item> \ \ \ new_chunk(line[1]);
3194 <item> \ \ \ new_chunk(next_chunk_name, next_chunk_args);
3198 <item> \ chunking=1;
3207 We recognize notangle style chunks too:
3209 <\nf-chunk|recognize-chunk>
3210 <item>/^[\<less\>]\<less\>.*[\<gtr\>]\<gtr\>=/ {
3212 <item> \ if (match($0, "^[\<less\>]\<less\>(.*)[\<gtr\>]\<gtr\>= *$",
3215 <item> \ \ \ chunking=1;
3217 <item> \ \ \ notangle_mode=1;
3219 <item> \ \ \ new_chunk(line[1]);
3230 Likewise, we need to recognize when a chunk ends.
3232 <subsection|lstlistings>
3234 The <verbatim|e> in <verbatim|[e]nd{lislisting}> is surrounded by square
3235 brackets so that when this document is processed, this chunk doesn't
3236 terminate early when the lstlistings package recognizes it's own
3237 end-string!<\footnote>
3238 This doesn't make sense as the regex is anchored with ^, which this line
3239 does not begin with!
3242 <\nf-chunk|recognize-chunk>
3243 <item>/^\\\\[e]nd{lstlisting}\|^\\\\[e]nd{Chunk}/ {
3245 <item> \ chunking=0;
3247 <item> \ active_chunk="";
3256 <\nf-chunk|recognize-chunk>
3259 <item> \ chunking=0;
3261 <item> \ active_chunk="";
3266 All other recognizers are only of effect if we are chunking; there's no
3267 point in looking at lines if they aren't part of a chunk, so we just ignore
3268 them as efficiently as we can.
3270 <\nf-chunk|recognize-chunk>
3271 <item>! chunking { next; }
3274 <section|Chunk contents>
3276 Chunk contents are any lines read while <verbatim|chunking> is true. Some
3277 chunk contents are special in that they refer to other chunks, and will be
3278 replaced by the contents of these chunks when the file is generated.
3280 <label|sub:ORS-chunk-text>We add the output record separator <verbatim|ORS>
3281 to the line now, because we will set <verbatim|ORS> to the empty string
3282 when we generate the output<\footnote>
3283 So that we can partial print lines using <verbatim|print> instead of
3284 <verbatim|printf>. <todo|This does't make sense>
3287 <\nf-chunk|recognize-chunk>
3288 <item>length(active_chunk) {
3290 <item> \ =\<less\>\\chunkref{process-chunk-tabs}\<gtr\>
3292 <item> \ =\<less\>\\chunkref{process-chunk}\<gtr\>
3297 If a chunk just consisted of plain text, we could handle the chunk like
3300 <\nf-chunk|process-chunk-simple>
3301 <item>chunk_line(active_chunk, $0 ORS);
3304 but in fact a chunk can include references to other chunks. Chunk includes
3305 are traditionally written as <verbatim|\<less\>\<less\>chunk-name\<gtr\>\<gtr\>>
3306 but we support other variations, some of which are more suitable for
3307 particular editing systems.
3309 However, we also process tabs at this point, a tab at input can be replaced
3310 by a number of spaces defined by the <verbatim|tabs> variable, set by the
3311 <verbatim|-T> option. Of course this is poor tab behaviour, we should
3312 probably have the option to use proper counted tab-stops and process this
3315 <\nf-chunk|process-chunk-tabs>
3316 <item>if (length(tabs)) {
3318 <item> \ gsub("\\t", tabs);
3323 <subsection|lstlistings><label|sub:lst-listings-includes>
3325 If <verbatim|\\lstset{escapeinside={=\<less\>}{\<gtr\>}}> is set, then we
3326 can use <verbatim|=\<less\>\\chunkref{chunk-name}\<gtr\>> in listings. The
3327 sequence <verbatim|=\<less\>> was chosen because:
3330 <item>it is a better mnemonic than <verbatim|\<less\>\<less\>chunk-name\<gtr\>\<gtr\>>
3331 in that the <verbatim|=> sign signifies equivalence or substitutability.
3333 <item>and because <verbatim|=\<less\>> is not valid in C or any language
3336 <item>and also because lstlistings doesn't like <verbatim|\<gtr\>\<gtr\>>
3337 as an end delimiter for the <em|texcl> escape, so we must make do with a
3338 single <verbatim|\<gtr\>> which is better complemented by
3339 <verbatim|=\<less\>> than by <verbatim|\<less\>\<less\>>.
3342 Unfortunately the <verbatim|=\<less\>...\<gtr\>> that we use re-enters a
3343 <LaTeX> parsing mode in which some characters are special, e.g. <verbatim|#
3344 \\> and so these cause trouble if used in arguments to
3345 <verbatim|\\chunkref>. At some point I must fix the <LaTeX> command
3346 <verbatim|\\chunkref> so that it can accept these literally, but until
3347 then, when writing chunkref argumemts that need these characters, I must
3348 use the forms <verbatim|\\textbackslash{}> and <verbatim|\\#>; so I also
3349 define a hacky chunk <verbatim|delatex> to be used further on whose purpose
3350 it is to remove these from any arguments parsed by fangle.
3355 <item>gsub("\\\\\\\\#", "#", ${text});
3357 <item>gsub("\\\\\\\\textbackslash{}", "\\\\", ${text});
3359 <item>gsub("\\\\\\\\\\\\^", "^", ${text});
3360 </nf-chunk||<tuple|text>>
3362 As each chunk line may contain more than one chunk include, we will split
3363 out chunk includes in an iterative fashion<\footnote>
3364 Contrary to our use of split when substituting parameters in chapter
3365 <reference|Here-we-split>
3368 First, as long as the chunk contains a <verbatim|\\chunkref> command we
3369 take as much as we can up to the first <verbatim|\\chunkref> command.
3371 <\nf-chunk|process-chunk>
3376 <item>while(match(chunk,\
3378 <item> \ \ \ \ \ \ \ \ \ \ \ "([=]\<less\>\\\\\\\\chunkref{([^}\<gtr\>]*)}(\\\\(.*\\\\)\|)\<gtr\>\|\<less\>\<less\>([a-zA-Z_][-a-zA-Z0-9_]*)\<gtr\>\<gtr\>)",\
3380 <item> \ \ \ \ \ \ \ \ \ \ \ line)\\
3384 <item> \ chunklet = substr(chunk, 1, RSTART - 1);
3387 We keep track of the indent count, by counting the number of literal
3388 characters found. We can then preserve this indent on each output line when
3389 multi-line chunks are expanded.
3391 We then process this first part literal text, and set the chunk which is
3392 still to be processed to be the text after the <verbatim|\\chunkref>
3393 command, which we will process next as we continue around the loop.
3395 <\nf-chunk|process-chunk>
3396 <item> \ indent += length(chunklet);
3398 <item> \ chunk_line(active_chunk, chunklet);
3400 <item> \ chunk = substr(chunk, RSTART + RLENGTH);
3403 We then consider the type of chunk command we have found, whether it is the
3404 fangle style command beginning with <verbatim|=\<less\>> the older notangle
3405 style beginning with <verbatim|\<less\>\<less\>>.
3407 Fangle chunks may have parameters contained within square brackets. These
3408 will be matched in <verbatim|line[3]> and are considered at this stage of
3409 processing to be part of the name of the chunk to be included.
3411 <\nf-chunk|process-chunk>
3412 <item> \ if (substr(line[1], 1, 1) == "=") {
3414 <item> \ \ \ # chunk name up to }
3416 <item> \ \ \ \ \ \ \ =\<less\>\\chunkref{delatex}(line[3])\<gtr\>
3418 <item> \ \ \ chunk_include(active_chunk, line[2] line[3], indent);
3420 <item> \ } else if (substr(line[1], 1, 1) == "\<less\>") {
3422 <item> \ \ \ chunk_include(active_chunk, line[4], indent);
3426 <item> \ \ \ error("Unknown chunk fragment: " line[1]);
3435 The loop will continue until there are no more chunkref statements in the
3436 text, at which point we process the final part of the chunk.
3438 <\nf-chunk|process-chunk>
3441 <item>chunk_line(active_chunk, chunk);
3444 <label|lone-newline>We add the newline character as a chunklet on it's own,
3445 to make it easier to detect new lines and thus manage indentation when
3446 processing the output.
3448 <\nf-chunk|process-chunk>
3449 <item>chunk_line(active_chunk, "\n");
3454 We will also permit a chunk-part number to follow in square brackets, so
3455 that <verbatim|=\<less\>\\chunkref{chunk-name[1]}\<gtr\>> will refer to the
3456 first part only. This can make it easy to include a C function prototype in
3457 a header file, if the first part of the chunk is just the function
3458 prototype without the trailing semi-colon. The header file would include
3459 the prototype with the trailing semi-colon, like this:
3461 <verbatim|=\<less\>\\chunkref{chunk-name[1]}\<gtr\>>
3463 This is handled in section <reference|sub:Chunk-parts>
3465 We should perhaps introduce a notion of language specific chunk options; so
3466 that perhaps we could specify:
3468 <verbatim|=\<less\>\\chunkref{chunk-name[function-declaration]}>
3470 which applies a transform <verbatim|function-declaration> to the chunk ---
3471 which in this case would extract a function prototype from a function.
3474 <chapter|Processing Options>
3476 At the start, first we set the default options.
3478 <\nf-chunk|default-options>
3483 <item>notangle_mode=0;
3490 Then we use getopt the standard way, and null out ARGV afterwards in the
3493 <\nf-chunk|read-options>
3494 <item>Optind = 1 \ \ \ # skip ARGV[0]
3496 <item>while(getopt(ARGC, ARGV, "R:LdT:hr")!=-1) {
3498 <item> \ =\<less\>\\chunkref{handle-options}\<gtr\>
3502 <item>for (i=1; i\<less\>Optind; i++) { ARGV[i]=""; }
3505 This is how we handle our options:
3507 <\nf-chunk|handle-options>
3508 <item>if (Optopt == "R") root = Optarg;
3510 <item>else if (Optopt == "r") root="";
3512 <item>else if (Optopt == "L") linenos = 1;
3514 <item>else if (Optopt == "d") debug = 1;
3516 <item>else if (Optopt == "T") tabs = indent_string(Optarg+0);
3518 <item>else if (Optopt == "h") help();
3520 <item>else if (Optopt == "?") help();
3523 We do all of this at the beginning of the program
3528 <item> \ =\<less\>\\chunkref{constants}\<gtr\>
3530 <item> \ =\<less\>\\chunkref{mode-definitions}\<gtr\>
3532 <item> \ =\<less\>\\chunkref{default-options}\<gtr\>
3536 <item> \ =\<less\>\\chunkref{read-options}\<gtr\>
3541 And have a simple help function
3544 <item>function help() {
3546 <item> \ print "Usage:"
3548 <item> \ print " \ fangle [-L] -R\<less\>rootname\<gtr\> [source.tex
3551 <item> \ print " \ fangle -r [source.tex ...]"
3553 <item> \ print " \ If the filename, source.tex is not specified then
3558 <item> \ print "-L causes the C statement: #line \<less\>lineno\<gtr\>
3559 \\"filename\\"" to be issued"
3561 <item> \ print "-R causes the named root to be written to stdout"
3563 <item> \ print "-r lists all roots in the file (even those used
3571 <chapter|Generating the Output>
3573 We generate output by calling output_chunk, or listing the chunk names.
3575 <\nf-chunk|generate-output>
3576 <item>if (length(root)) output_chunk(root);
3578 <item>else output_chunk_names();
3581 We also have some other output debugging:
3583 <\nf-chunk|debug-output>
3586 <item> \ print "------ chunk names "
3588 <item> \ output_chunk_names();
3590 <item> \ print "====== chunks"
3592 <item> \ output_chunks();
3594 <item> \ print "++++++ debug"
3596 <item> \ for (a in chunks) {
3598 <item> \ \ \ print a "=" chunks[a];
3605 We do both of these at the end. We also set <verbatim|ORS=""> because each
3606 chunklet is not necessarily a complete line, and we already added
3607 <verbatim|ORS> to each input line in section
3608 <reference|sub:ORS-chunk-text>.
3613 <item> \ =\<less\>\\chunkref{debug-output}\<gtr\>
3617 <item> \ =\<less\>\\chunkref{generate-output}\<gtr\>
3622 We write chunk names like this. If we seem to be running in notangle
3623 compatibility mode, then we enclose the name like this
3624 <verbatim|\<less\>\<less\>name\<gtr\>\<gtr\>> the same way notangle does:
3626 <\nf-chunk|output_chunk_names()>
3627 <item>function output_chunk_names( \ \ c, prefix, suffix)\
3631 <item> \ if (notangle_mode) {
3633 <item> \ \ \ prefix="\<less\>\<less\>";
3635 <item> \ \ \ suffix="\<gtr\>\<gtr\>";
3639 <item> \ for (c in chunk_names) {
3641 <item> \ \ \ print prefix c suffix "\\n";
3648 This function would write out all chunks
3650 <\nf-chunk|output_chunks()>
3651 <item>function output_chunks( \ a)\
3655 <item> \ for (a in chunk_names) {
3657 <item> \ \ \ output_chunk(chunk_names[a]);
3665 <item>function output_chunk(chunk) {
3667 <item> \ newline = 1;
3669 <item> \ lineno_needed = linenos;
3673 <item> \ write_chunk(chunk);
3680 <section|Assembling the Chunks>
3682 <verbatim|chunk_path> holds a string consisting of the names of all the
3683 chunks that resulted in this chunk being output. It should probably also
3684 contain the source line numbers at which each inclusion also occured.
3686 We first initialize the mode tracker for this chunk.
3688 <\nf-chunk|write_chunk()>
3689 <item>function write_chunk(chunk_name) {
3691 <item> \ =\<less\>\\chunkref{awk-delete-array}(context)\<gtr\>
3693 <item> \ return write_chunk_r(chunk_name, context);
3699 <item>function write_chunk_r(chunk_name, context, indent, tail,
3701 <item> \ # optional vars
3703 <item> \ <with|font-shape|italic|chunk-path>, chunk_args,\
3705 <item> \ s, r, src, new_src,\
3707 <item> \ # local vars
3709 <item> \ chunk_params, part, max_part, part_line, frag, max_frag, text,\
3711 <item> \ chunklet, only_part, call_chunk_args, new_context)
3716 <subsection|Chunk Parts><label|sub:Chunk-parts>
3718 As mentioned in section <reference|sub:lstlistings-includes>, a chunk name
3719 may contain a part specifier in square brackets, limiting the parts that
3722 <\nf-chunk|write_chunk()>
3723 <item> \ if (match(chunk_name, "^(.*)\\\\[([0-9]*)\\\\]$",
3724 chunk_name_parts)) {
3726 <item> \ \ \ chunk_name = chunk_name_parts[1];
3728 <item> \ \ \ only_part = chunk_name_parts[2];
3733 We then create a mode tracker
3735 <\nf-chunk|write_chunk()>
3736 <item> =\<less\>\\chunkref{new-mode-tracker}(context, chunks[chunk_name,
3737 "language"], "")\<gtr\>
3740 We extract into <verbatim|chunk_params> the names of the parameters that
3741 this chunk accepts, whose values were (optionally) passed in
3742 <verbatim|chunk_args>.
3744 <\nf-chunk|write_chunk()>
3745 <item> split(chunks[chunk_name, "params"], chunk_params, " *; *");
3748 To assemble a chunk, we write out each part.
3750 <\nf-chunk|write_chunk()>
3751 <item> \ if (! (chunk_name in chunk_names)) {
3753 <item> \ \ \ error(sprintf(_"The root module
3754 \<less\>\<less\>%s\<gtr\>\<gtr\> was not defined.\\nUsed by: %s",\\
3756 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ chunk_name, chunk_path));
3762 <item> \ max_part = chunks[chunk_name, "part"];
3764 <item> \ for(part = 1; part \<less\>= max_part; part++) {
3766 <item> \ \ \ if (! only_part \|\| part == only_part) {
3768 <item> \ \ \ \ \ =\<less\>\\chunkref{write-part}\<gtr\>
3774 <item> \ if (! finalize_mode_tracker(context)) {
3776 <item> \ \ \ dump_mode_tracker(context);
3778 <item> \ \ \ error(sprintf(_"Module %s did not close context
3779 properly.\\nUsed by: %s\\n", chunk_name, chunk_path));
3786 A part can either be a chunklet of lines, or an include of another chunk.
3788 Chunks may also have parameters, specified in LaTeX style with braces after
3789 the chunk name --- looking like this in the document: chunkname{param1,
3790 param2}. Arguments are passed in square brackets:
3791 <verbatim|\\chunkref{chunkname}[arg1, arg2]>.
3793 Before we process each part, we check that the source position hasn't
3794 changed unexpectedly, so that we can know if we need to output a new
3795 file-line directive.
3797 <\nf-chunk|write-part>
3798 <item>=\<less\>\\chunkref{check-source-jump}\<gtr\>
3802 <item>chunklet = chunks[chunk_name, "part", part];
3804 <item>if (chunks[chunk_name, "part", part, "type"] == part_type_chunk) {
3806 <item> \ =\<less\>\\chunkref{write-included-chunk}\<gtr\>
3808 <item>} else if (chunklet SUBSEP "line" in chunks) {
3810 <item> \ =\<less\>\\chunkref{write-chunklets}\<gtr\>
3814 <item> \ # empty last chunklet
3819 To write an included chunk, we must detect any optional chunk arguments in
3820 parenthesis. Then we recurse calling <verbatim|write_chunk()>.
3822 <\nf-chunk|write-included-chunk>
3823 <item>if (match(chunklet, "^([^\\\\[\\\\(]*)\\\\((.*)\\\\)$",
3826 <item> \ chunklet = chunklet_parts[1];
3828 <item> \ parse_chunk_args("c-like", chunklet_parts[2], call_chunk_args,
3831 <item> \ for (c in call_chunk_args) {
3833 <item> \ \ \ call_chunk_args[c] = expand_chunk_args(call_chunk_args[c],
3834 chunk_params, chunk_args);
3840 <item> \ split("", call_chunk_args);
3844 <item># update the transforms arrays
3846 <item>new_src = mode_escaper(context, s, r, src);
3848 <item>=\<less\>\\chunkref{awk-delete-array}(new_context)\<gtr\>
3850 <item>write_chunk_r(chunklet, new_context,
3852 <item> \ \ \ \ \ \ \ \ \ \ \ chunks[chunk_name, "part", part, "indent"]
3855 <item> \ \ \ \ \ \ \ \ \ \ \ chunks[chunk_name, "part", part, "tail"],
3857 <item> \ \ \ \ \ \ \ \ \ \ \ chunk_path "\\n \ \ \ \ \ \ \ \ "
3860 <item> \ \ \ \ \ \ \ \ \ \ \ call_chunk_args,
3862 <item> \ \ \ \ \ \ \ \ \ \ \ s, r, new_src);
3865 Before we output a chunklet of lines, we first emit the file and line
3866 number if we have one, and if it is safe to do so.
3868 Chunklets are generally broken up by includes, so the start of a chunklet
3869 is a good place to do this. Then we output each line of the chunklet.
3871 When it is not safe, such as in the middle of a multi-line macro
3872 definition, <verbatim|lineno_suppressed> is set to true, and in such a case
3873 we note that we want to emit the line statement when it is next safe.
3875 <\nf-chunk|write-chunklets>
3876 <item>max_frag = chunks[chunklet, "line"];
3878 <item>for(frag = 1; frag \<less\>= max_frag; frag++) {
3880 <item> \ =\<less\>\\chunkref{write-file-line}\<gtr\>
3883 We then extract the chunklet text and expand any arguments.
3885 <\nf-chunk|write-chunklets>
3888 <item> \ text = chunks[chunklet, frag];
3892 <item> \ /* check params */
3894 <item> \ text = expand_chunk_args(text, chunk_params, chunk_args);
3897 If the text is a single newline (which we keep separate - see
3898 <reference|lone-newline>) then we increment the line number. In the case
3899 where this is the last line of a chunk and it is not a top-level chunk we
3900 replace the newline with an empty string --- because the chunk that
3901 included this chunk will have the newline at the end of the line that
3902 included this chunk.
3904 We also note by <verbatim|newline = 1> that we have started a new line, so
3905 that indentation can be managed with the following piece of text.
3907 <\nf-chunk|write-chunklets>
3910 <item> if (text == "\\n") {
3912 <item> \ \ \ lineno++;
3914 <item> \ \ \ if (part == max_part && frag == max_frag &&
3915 length(chunk_path)) {
3917 <item> \ \ \ \ \ text = "";
3919 <item> \ \ \ \ \ break;
3921 <item> \ \ \ } else {
3923 <item> \ \ \ \ \ newline = 1;
3928 If this text does not represent a newline, but we see that we are the first
3929 piece of text on a newline, then we prefix our text with the current
3933 <verbatim|newline> is a global output-state variable, but the
3934 <verbatim|indent> is not.
3937 <\nf-chunk|write-chunklets>
3938 <item> \ } else if (length(text) \|\| length(tail)) {
3940 <item> \ \ \ if (newline) text = indent text;
3942 <item> \ \ \ newline = 0;
3949 Tail will soon no longer be relevant once mode-detection is in place.
3951 <\nf-chunk|write-chunklets>
3952 <item> \ text = text tail;
3954 <item> \ mode_tracker(context, text);
3956 <item> \ print transform_escape(s, r, text, src);
3959 If a line ends in a backslash --- suggesting continuation --- then we
3960 supress outputting file-line as it would probably break the continued
3963 <\nf-chunk|write-chunklets>
3964 <item> \ if (linenos) {
3966 <item> \ \ \ lineno_suppressed = substr(lastline, length(lastline)) ==
3974 Of course there is no point in actually outputting the source filename and
3975 line number (file-line) if they don't say anything new! We only need to
3976 emit them if they aren't what is expected, or if we we not able to emit one
3977 when they had changed.
3979 <\nf-chunk|write-file-line>
3980 <item>if (newline && lineno_needed && ! lineno_suppressed) {
3982 <item> \ filename = a_filename;
3984 <item> \ lineno = a_lineno;
3986 <item> \ print "#line " lineno " \\"" filename "\\"\\n"
3988 <item> \ lineno_needed = 0;
3993 We check if a new file-line is needed by checking if the source line
3994 matches what we (or a compiler) would expect.
3996 <\nf-chunk|check-source-jump>
3997 <item>if (linenos && (chunk_name SUBSEP "part" SUBSEP part SUBSEP
3998 "FILENAME" in chunks)) {
4000 <item> \ a_filename = chunks[chunk_name, "part", part, "FILENAME"];
4002 <item> \ a_lineno = chunks[chunk_name, "part", part, "LINENO"];
4004 <item> \ if (a_filename != filename \|\| a_lineno != lineno) {
4006 <item> \ \ \ lineno_needed++;
4013 <chapter|Storing Chunks>
4015 Awk has pretty limited data structures, so we will use two main hashes.
4016 Uninterrupted sequences of a chunk will be stored in chunklets and the
4017 chunklets used in a chunk will be stored in <verbatim|chunks>.
4019 <\nf-chunk|constants>
4020 <item>part_type_chunk=1;
4025 The params mentioned are not chunk parameters for parameterized chunks, as
4026 mentioned in <reference|Chunk Arguments>, but the lstlistings style
4027 parameters used in the <verbatim|\\Chunk> command<\footnote>
4028 The <verbatim|params> parameter is used to hold the parameters for
4029 parameterized chunks
4032 <\nf-chunk|chunk-storage-functions>
4033 <item>function new_chunk(chunk_name, params,
4035 <item> \ # local vars
4037 <item> \ p, append )
4041 <item> \ # HACK WHILE WE CHANGE TO ( ) for PARAM CHUNKS
4043 <item> \ gsub("\\\\(\\\\)$", "", chunk_name);
4045 <item> \ if (! (chunk_name in chunk_names)) {
4047 <item> \ \ \ if (debug) print "New chunk " chunk_name;
4049 <item> \ \ \ chunk_names[chunk_name];
4051 <item> \ \ \ for (p in params) {
4053 <item> \ \ \ \ \ chunks[chunk_name, p] = params[p];
4057 <item> \ \ \ if ("append" in params) {
4059 <item> \ \ \ \ \ append=params["append"];
4061 <item> \ \ \ \ \ if (! (append in chunk_names)) {
4063 <item> \ \ \ \ \ \ \ warning("Chunk " chunk_name " is appended to chunk "
4064 append " which is not defined yet");
4066 <item> \ \ \ \ \ \ \ new_chunk(append);
4070 <item> \ \ \ \ \ chunk_include(append, chunk_name);
4072 <item> \ \ \ \ \ chunk_line(append, ORS);
4078 <item> \ active_chunk = chunk_name;
4080 <item> \ prime_chunk(chunk_name);
4085 <\nf-chunk|chunk-storage-functions>
4088 <item>function prime_chunk(chunk_name)
4092 <item> \ chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] = \\
4094 <item> \ \ \ \ \ \ \ \ chunk_name SUBSEP "chunklet" SUBSEP ""
4095 ++chunks[chunk_name, "chunklet"];
4097 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"],
4098 "FILENAME"] = FILENAME;
4100 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "LINENO"]
4107 <item>function chunk_line(chunk_name, line){
4109 <item> \ chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
4111 <item> \ \ \ \ \ \ \ \ ++chunks[chunk_name, "chunklet",
4112 chunks[chunk_name, "chunklet"], "line"] \ ] = line;
4119 Chunk include represents a <em|chunkref> statement, and stores the
4120 requirement to include another chunk. The parameter indent represents the
4121 quanity of literal text characters that preceded this <em|chunkref>
4122 statement and therefore by how much additional lines of the included chunk
4125 <\nf-chunk|chunk-storage-functions>
4126 <item>function chunk_include(chunk_name, chunk_ref, indent, tail)
4130 <item> \ chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] =
4133 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "type" ]
4136 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "indent"
4137 ] = indent_string(indent);
4139 <item> \ chunks[chunk_name, "part", chunks[chunk_name, "part"], "tail" ]
4142 <item> \ prime_chunk(chunk_name);
4149 The indent is calculated by indent_string, which may in future convert some
4150 spaces into tab characters. This function works by generating a printf
4151 padded format string, like <verbatim|%22s> for an indent of 22, and then
4152 printing an empty string using that format.
4154 <\nf-chunk|chunk-storage-functions>
4155 <item>function indent_string(indent) {
4157 <item> \ return sprintf("%" indent "s", "");
4162 <chapter|getopt><label|cha:getopt>
4164 I use Arnold Robbins public domain getopt (1993 revision). This is probably
4165 the same one that is covered in chapter 12 of “Edition 3 of GAWK:
4166 Effective AWK Programming: A User's Guide for GNU Awk” but as that is
4167 licensed under the GNU Free Documentation License, Version 1.3, which
4168 conflicts with the GPL3, I can't use it from there (or it's accompanying
4169 explanations), so I do my best to explain how it works here.
4171 The getopt.awk header is:
4173 <\nf-chunk|getopt.awk-header>
4174 <item># getopt.awk --- do C library getopt(3) function in awk
4178 <item># Arnold Robbins, arnold@skeeve.com, Public Domain
4182 <item># Initial version: March, 1991
4184 <item># Revised: May, 1993
4189 The provided explanation is:
4191 <\nf-chunk|getopt.awk-notes>
4192 <item># External variables:
4194 <item># \ \ \ Optind -- index in ARGV of first nonoption argument
4196 <item># \ \ \ Optarg -- string value of argument to current option
4198 <item># \ \ \ Opterr -- if nonzero, print our own diagnostic
4200 <item># \ \ \ Optopt -- current option letter
4206 <item># \ \ \ -1 \ \ \ \ at end of options
4208 <item># \ \ \ ? \ \ \ \ \ for unrecognized option
4210 <item># \ \ \ \<less\>c\<gtr\> \ \ \ a character representing the current
4215 <item># Private Data:
4217 <item># \ \ \ _opti \ -- index in multi-flag option, e.g., -abc
4222 The function follows. The final two parameters, <verbatim|thisopt> and
4223 <verbatim|i> are local variables and not parameters --- as indicated by the
4224 multiple spaces preceding them. Awk doesn't care, the multiple spaces are a
4225 convention to help us humans.
4227 <\nf-chunk|getopt.awk-getopt()>
4228 <item>function getopt(argc, argv, options, \ \ \ thisopt, i)
4232 <item> \ \ \ if (length(options) == 0) \ \ \ # no options given
4234 <item> \ \ \ \ \ \ \ return -1
4236 <item> \ \ \ if (argv[Optind] == "--") { \ # all done
4238 <item> \ \ \ \ \ \ \ Optind++
4240 <item> \ \ \ \ \ \ \ _opti = 0
4242 <item> \ \ \ \ \ \ \ return -1
4244 <item> \ \ \ } else if (argv[Optind] !~ /^-[^: \\t\\n\\f\\r\\v\\b]/) {
4246 <item> \ \ \ \ \ \ \ _opti = 0
4248 <item> \ \ \ \ \ \ \ return -1
4252 <item> \ \ \ if (_opti == 0)
4254 <item> \ \ \ \ \ \ \ _opti = 2
4256 <item> \ \ \ thisopt = substr(argv[Optind], _opti, 1)
4258 <item> \ \ \ Optopt = thisopt
4260 <item> \ \ \ i = index(options, thisopt)
4262 <item> \ \ \ if (i == 0) {
4264 <item> \ \ \ \ \ \ \ if (Opterr)
4266 <item> \ \ \ \ \ \ \ \ \ \ \ printf("%c -- invalid option\\n",
4268 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ thisopt)
4269 \<gtr\> "/dev/stderr"
4271 <item> \ \ \ \ \ \ \ if (_opti \<gtr\>= length(argv[Optind])) {
4273 <item> \ \ \ \ \ \ \ \ \ \ \ Optind++
4275 <item> \ \ \ \ \ \ \ \ \ \ \ _opti = 0
4277 <item> \ \ \ \ \ \ \ } else
4279 <item> \ \ \ \ \ \ \ \ \ \ \ _opti++
4281 <item> \ \ \ \ \ \ \ return "?"
4286 At this point, the option has been found and we need to know if it takes
4289 <\nf-chunk|getopt.awk-getopt()>
4290 <item> \ \ \ if (substr(options, i + 1, 1) == ":") {
4292 <item> \ \ \ \ \ \ \ # get option argument
4294 <item> \ \ \ \ \ \ \ if (length(substr(argv[Optind], _opti + 1)) \<gtr\>
4297 <item> \ \ \ \ \ \ \ \ \ \ \ Optarg = substr(argv[Optind], _opti + 1)
4299 <item> \ \ \ \ \ \ \ else
4301 <item> \ \ \ \ \ \ \ \ \ \ \ Optarg = argv[++Optind]
4303 <item> \ \ \ \ \ \ \ _opti = 0
4307 <item> \ \ \ \ \ \ \ Optarg = ""
4309 <item> \ \ \ if (_opti == 0 \|\| _opti \<gtr\>= length(argv[Optind])) {
4311 <item> \ \ \ \ \ \ \ Optind++
4313 <item> \ \ \ \ \ \ \ _opti = 0
4317 <item> \ \ \ \ \ \ \ _opti++
4319 <item> \ \ \ return thisopt
4324 A test program is built in, too
4326 <\nf-chunk|getopt.awk-begin>
4329 <item> \ \ \ Opterr = 1 \ \ \ # default is to diagnose
4331 <item> \ \ \ Optind = 1 \ \ \ # skip ARGV[0]
4333 <item> \ \ \ # test program
4335 <item> \ \ \ if (_getopt_test) {
4337 <item> \ \ \ \ \ \ \ while ((_go_c = getopt(ARGC, ARGV, "ab:cd")) != -1)
4339 <item> \ \ \ \ \ \ \ \ \ \ \ printf("c = \<less\>%c\<gtr\>, optarg =
4340 \<less\>%s\<gtr\>\\n",
4342 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ _go_c,
4345 <item> \ \ \ \ \ \ \ printf("non-option arguments:\\n")
4347 <item> \ \ \ \ \ \ \ for (; Optind \<less\> ARGC; Optind++)
4349 <item> \ \ \ \ \ \ \ \ \ \ \ printf("\\tARGV[%d] = \<less\>%s\<gtr\>\\n",
4351 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Optind,
4359 The entire getopt.awk is made out of these chunks in order
4361 <\nf-chunk|getopt.awk>
4362 <item>=\<less\>\\chunkref{getopt.awk-header}\<gtr\>
4366 <item>=\<less\>\\chunkref{getopt.awk-notes}\<gtr\>
4368 <item>=\<less\>\\chunkref{getopt.awk-getopt()}\<gtr\>
4370 <item>=\<less\>\\chunkref{getopt.awk-begin}\<gtr\>
4373 Although we only want the header and function:
4376 <item># try: locate getopt.awk for the full original file
4378 <item># as part of your standard awk installation
4380 <item>=\<less\>\\chunkref{getopt.awk-header}\<gtr\>
4384 <item>=\<less\>\\chunkref{getopt.awk-getopt()}\<gtr\>
4387 <chapter|Fangle LaTeX source code><label|latex-source>
4389 <section|fangle module>
4391 Here we define a <LyX> <verbatim|.module> file that makes it convenient to
4392 use <LyX> for writing such literate programs.
4394 This file <verbatim|./fangle.module> can be installed in your personal
4395 <verbatim|.lyx/layouts> folder. You will need to Tools Reconfigure so that
4396 <LyX> notices it. It adds a new format Chunk, which should precede every
4397 listing and contain the chunk name.
4399 <\nf-chunk|./fangle.module>
4400 <item>#\\DeclareLyXModule{Fangle Literate Listings}
4402 <item>#DescriptionBegin
4404 <item># \ Fangle literate listings allow one to write
4406 <item># \ \ literate programs after the fashion of noweb, but without
4409 <item># \ \ to use noweave to generate the documentation. Instead the
4412 <item># \ \ package is extended in conjunction with the noweb package to
4415 <item># \ \ to code formating directly as latex.
4417 <item># \ The fangle awk script
4419 <item>#DescriptionEnd
4423 <item>=\<less\>\\chunkref{gpl3-copyright.hashed}\<gtr\>
4433 <item>=\<less\>\\chunkref{./fangle.sty}\<gtr\>
4439 <item>=\<less\>\\chunkref{chunkstyle}\<gtr\>
4443 <item>=\<less\>\\chunkref{chunkref}\<gtr\>
4444 </nf-chunk|lyx-module|>
4446 Because <LyX> modules are not yet a language supported by fangle or
4447 lstlistings, we resort to this fake awk chunk below in order to have each
4448 line of the GPL3 license commence with a #
4450 <\nf-chunk|gpl3-copyright.hashed>
4451 <item>#=\<less\>\\chunkref{gpl3-copyright}\<gtr\>
4456 <subsection|The Chunk style>
4458 The purpose of the <name|chunk> style is to make it easier for <LyX> users
4459 to provide the name to <verbatim|lstlistings>. Normally this requires
4460 right-clicking on the listing, choosing settings, advanced, and then typing
4461 <verbatim|name=chunk-name>. This has the further disadvantage that the name
4462 (and other options) are not generally visible during document editing.
4464 The chunk style is defined as a <LaTeX> command, so that all text on the
4465 same line is passed to the <verbatim|LaTeX> command <verbatim|Chunk>. This
4466 makes it easy to parse using <verbatim|fangle>, and easy to pass these
4467 options on to the listings package. The first word in a chunk section
4468 should be the chunk name, and will have <verbatim|name=> prepended to it.
4469 Any other words are accepted arguments to <verbatim|lstset>.
4471 We set PassThru to 1 because the user is actually entering raw latex.
4473 <\nf-chunk|chunkstyle>
4476 <item> \ LatexType \ \ \ \ \ \ \ \ \ \ \ \ Command
4478 <item> \ LatexName \ \ \ \ \ \ \ \ \ \ \ \ Chunk
4480 <item> \ Margin \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ First_Dynamic
4482 <item> \ LeftMargin \ \ \ \ \ \ \ \ \ \ \ Chunk:xxx
4484 <item> \ LabelSep \ \ \ \ \ \ \ \ \ \ \ \ \ xx
4486 <item> \ LabelType \ \ \ \ \ \ \ \ \ \ \ \ Static
4488 <item> \ LabelString \ \ \ \ \ \ \ \ \ \ "Chunk:"
4490 <item> \ Align \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Left
4492 <item> \ PassThru \ \ \ \ \ \ \ \ \ \ \ \ \ 1
4497 To make the label very visible we choose a larger font coloured red.
4499 <\nf-chunk|chunkstyle>
4502 <item> \ \ \ Family \ \ \ \ \ \ \ \ \ \ \ \ \ Sans
4504 <item> \ \ \ Size \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ Large
4506 <item> \ \ \ Series \ \ \ \ \ \ \ \ \ \ \ \ \ Bold
4508 <item> \ \ \ Shape \ \ \ \ \ \ \ \ \ \ \ \ \ \ Italic
4510 <item> \ \ \ Color \ \ \ \ \ \ \ \ \ \ \ \ \ \ red
4517 <subsection|The chunkref style>
4519 We also define the Chunkref style which can be used to express cross
4520 references to chunks.
4522 <\nf-chunk|chunkref>
4523 <item>InsetLayout Chunkref
4525 <item> \ LyxType \ \ \ \ \ \ \ \ \ \ \ \ \ \ charstyle
4527 <item> \ LatexType \ \ \ \ \ \ \ \ \ \ \ \ Command
4529 <item> \ LatexName \ \ \ \ \ \ \ \ \ \ \ \ chunkref
4531 <item> \ PassThru \ \ \ \ \ \ \ \ \ \ \ \ \ 1
4533 <item> \ LabelFont \ \ \ \ \ \ \ \ \ \ \ \
4535 <item> \ \ \ Shape \ \ \ \ \ \ \ \ \ \ \ \ \ \ Italic
4537 <item> \ \ \ Color \ \ \ \ \ \ \ \ \ \ \ \ \ \ red
4544 <section|Latex Macros><label|sec:Latex-Macros>
4546 We require the listings, noweb and xargs packages. As noweb defines it's
4547 own <verbatim|\\code> environment, we re-define the one that <LyX> logical
4548 markup module expects here.
4550 <\nf-chunk|./fangle.sty>
4551 <item>\\usepackage{listings}%
4553 <item>\\usepackage{noweb}%
4555 <item>\\usepackage{xargs}%
4557 <item>\\renewcommand{\\code}[1]{\\texttt{#1}}%
4560 We also define a <verbatim|CChunk> macro, for use as:
4561 <verbatim|\\begin{CChunk}> which will need renaming to
4562 <verbatim|\\begin{Chunk}> when I can do this without clashing with
4565 <\nf-chunk|./fangle.sty>
4566 <item>\\lstnewenvironment{Chunk}{\\relax}{\\relax}%
4569 We also define a suitable <verbatim|\\lstset> of parameters that suit the
4570 literate programming style after the fashion of <name|noweave>.
4572 <\nf-chunk|./fangle.sty>
4573 <item>\\lstset{numbers=left, stepnumber=5, numbersep=5pt,
4575 <item> \ \ \ \ \ \ \ breaklines=false,basicstyle=\\ttfamily,
4577 <item> \ \ \ \ \ \ \ numberstyle=\\tiny, language=C}%
4580 We also define a notangle-like mechanism for escaping to <LaTeX> from the
4581 listing, and by which we can refer to other listings. We declare the
4582 <verbatim|=\<less\>...\<gtr\>> sequence to contain <LaTeX> code, and
4583 include another like this chunk: <verbatim|=\<less\>\\chunkref{chunkname}\<gtr\>>.
4584 However, because <verbatim|=\<less\>...\<gtr\>> is already defined to
4585 contain <LaTeX> code for this document --- this is a fangle document after
4586 all --- the code fragment below effectively contains the <LaTeX> code:
4587 <verbatim|}{>. To avoid problems with document generation, I had to declare
4588 an lstlistings property: <verbatim|escapeinside={}> for this listing only;
4589 which in <LyX> was done by right-clicking the listings inset, choosing
4590 settings-\<gtr\>advanced. Therefore <verbatim|=\<less\>> isn't interpreted
4591 literally here, in a listing when the escape sequence is already defined as
4592 shown... we need to somehow escape this representation...
4594 <\nf-chunk|./fangle.sty>
4595 <item>\\lstset{escapeinside={=\<less\>}{\<gtr\>}}%
4598 Although our macros will contain the <verbatim|@> symbol, they will be
4599 included in a <verbatim|\\makeatletter> section by <LyX>; however we keep
4600 the commented out <verbatim|\\makeatletter> as a reminder. The listings
4601 package likes to centre the titles, but noweb titles are specially
4602 formatted and must be left aligned. The simplest way to do this turned out
4603 to be by removing the definition of <verbatim|\\lst@maketitle>. This may
4604 interact badly if other listings want a regular title or caption. We
4605 remember the old maketitle in case we need it.
4607 <\nf-chunk|./fangle.sty>
4608 <item>%\\makeatletter
4610 <item>%somehow re-defining maketitle gives us a left-aligned title
4612 <item>%which is extactly what our specially formatted title needs!
4614 <item>\\global\\let\\fangle@lst@maketitle\\lst@maketitle%
4616 <item>\\global\\def\\lst@maketitle{}%
4619 <subsection|The chunk command><label|sub:The-chunk-command>
4621 Our chunk command accepts one argument, and calls <verbatim|\\ltset>.
4622 Although <verbatim|\\ltset> will note the name, this is erased when the
4623 next <verbatim|\\lstlisting> starts, so we make a note of this in
4624 <verbatim|\\lst@chunkname> and restore in in lstlistings Init hook.
4626 <\nf-chunk|./fangle.sty>
4627 <item>\\def\\Chunk#1{%
4629 <item> \ \\lstset{title={\\fanglecaption},name=#1}%
4631 <item> \ \\global\\edef\\lst@chunkname{\\lst@intname}%
4635 <item>\\def\\lst@chunkname{\\empty}%
4638 <subsubsection|Chunk parameters>
4640 Fangle permits parameterized chunks, and requires the paramters to be
4641 specified as listings options. The fangle script uses this, and although we
4642 don't do anything with these in the <LaTeX> code right now, we need to stop
4643 the listings package complaining.
4645 <\nf-chunk|./fangle.sty>
4646 <item>\\lst@Key{params}\\relax{\\def\\fangle@chunk@params{#1}}%
4649 As it is common to define a chunk which then needs appending to another
4650 chunk, and annoying to have to declare a single line chunk to manage the
4651 include, we support an append= option.
4653 <\nf-chunk|./fangle.sty>
4654 <item>\\lst@Key{append}\\relax{\\def\\fangle@chunk@append{#1}}%
4657 <subsection|The noweb styled caption>
4659 We define a public macro <verbatim|\\fanglecaption> which can be set as a
4660 regular title. By means of <verbatim|\\protect>, It expands to
4661 <verbatim|\\fangle@caption> at the appopriate time when the caption is
4664 <nf-chunk|./fangle.sty|\\def\\fanglecaption{\\protect\\fangle@caption}%||>
4667 22c <math|\<langle\>>some-chunk 19b<math|\<rangle\>><math|\<equiv\>>+
4668 \ \ <math|\<vartriangleleft\>>22b 24d<math|\<vartriangleright\>>
4672 In this example, the current chunk is 22c, and therefore the third chunk
4675 It's name is some-chunk.\
4677 The first chunk with this name (19b) occurs as the second chunk on page
4680 The previous chunk (22d) with the same name is the second chunk on page
4683 The next chunk (24d) is the fourth chunk on page 24.
4684 </big-figure|Noweb Heading<label|noweb heading>>
4686 The general noweb output format compactly identifies the current chunk, and
4687 references to the first chunk, and the previous and next chunks that have
4690 This means that we need to keep a counter for each chunk-name, that we use
4691 to count chunks of the same name.
4693 <subsection|The chunk counter>
4695 It would be natural to have a counter for each chunk name, but TeX would
4696 soon run out of counters<\footnote>
4697 ...soon did run out of counters and so I had to re-write the LaTeX macros
4698 to share a counter as described here.
4699 </footnote>, so we have one counter which we save at the end of a chunk and
4700 restore at the beginning of a chunk.
4702 <\nf-chunk|./fangle.sty>
4703 <item>\\newcounter{fangle@chunkcounter}%
4706 We construct the name of this variable to store the counter to be the text
4707 <verbatim|lst-chunk-> prefixed onto the chunks own name, and store it in
4708 <verbatim|\\chunkcount>.\
4710 We save the counter like this:
4712 <nf-chunk|save-counter|\\global\\expandafter\\edef\\csname
4713 \\chunkcount\\endcsname{\\arabic{fangle@chunkcounter}}%||>
4715 and restore the counter like this:
4717 <nf-chunk|restore-counter|\\setcounter{fangle@chunkcounter}{\\csname
4718 \\chunkcount\\endcsname}%||>
4720 If there does not already exist a variable whose name is stored in
4721 <verbatim|\\chunkcount>, then we know we are the first chunk with this
4722 name, and then define a counter.\
4724 Although chunks of the same name share a common counter, they must still be
4725 distinguished. We use is the internal name of the listing, suffixed by the
4726 counter value. So the first chunk might be <verbatim|something-1> and the
4727 second chunk be <verbatim|something-2>, etc.
4729 We also calculate the name of the previous chunk if we can (before we
4730 increment the chunk counter). If this is the first chunk of that name, then
4731 <verbatim|\\prevchunkname> is set to <verbatim|\\relax> which the noweb
4732 package will interpret as not existing.
4734 <\nf-chunk|./fangle.sty>
4735 <item>\\def\\fangle@caption{%
4737 <item> \ \\edef\\chunkcount{lst-chunk-\\lst@intname}%
4739 <item> \ \\@ifundefined{\\chunkcount}{%
4741 <item> \ \ \ \\expandafter\\gdef\\csname \\chunkcount\\endcsname{0}%
4743 <item> \ \ \ \\setcounter{fangle@chunkcounter}{\\csname
4744 \\chunkcount\\endcsname}%
4746 <item> \ \ \ \\let\\prevchunkname\\relax%
4750 <item> \ \ \ \\setcounter{fangle@chunkcounter}{\\csname
4751 \\chunkcount\\endcsname}%
4753 <item> \ \ \ \\edef\\prevchunkname{\\lst@intname-\\arabic{fangle@chunkcounter}}%
4758 After incrementing the chunk counter, we then define the name of this
4759 chunk, as well as the name of the first chunk.
4761 <\nf-chunk|./fangle.sty>
4762 <item> \ \\addtocounter{fangle@chunkcounter}{1}%
4764 <item> \ \\global\\expandafter\\edef\\csname
4765 \\chunkcount\\endcsname{\\arabic{fangle@chunkcounter}}%
4767 <item> \ \\edef\\chunkname{\\lst@intname-\\arabic{fangle@chunkcounter}}%
4769 <item> \ \\edef\\firstchunkname{\\lst@intname-1}%
4772 We now need to calculate the name of the next chunk. We do this by
4773 temporarily skipping the counter on by one; however there may not actually
4774 be another chunk with this name! We detect this by also defining a label
4775 for each chunk based on the chunkname. If there is a next chunkname then it
4776 will define a label with that name. As labels are persistent, we can at
4777 least tell the second time <LaTeX> is run. If we don't find such a defined
4778 label then we define <verbatim|\\nextchunkname> to <verbatim|\\relax>.
4780 <\nf-chunk|./fangle.sty>
4781 <item> \ \\addtocounter{fangle@chunkcounter}{1}%
4783 <item> \ \\edef\\nextchunkname{\\lst@intname-\\arabic{fangle@chunkcounter}}%
4785 <item> \ \\@ifundefined{r@label-\\nextchunkname}{\\let\\nextchunkname\\relax}{}%
4788 The noweb package requires that we define a <verbatim|\\sublabel> for every
4789 chunk, with a unique name, which is then used to print out it's navigation
4792 We also define a regular label for this chunk, as was mentioned above when
4793 we calculated <verbatim|\\nextchunkname>. This requires <LaTeX> to be run
4794 at least twice after new chunk sections are added --- but noweb requried
4797 <\nf-chunk|./fangle.sty>
4798 <item> \ \\sublabel{\\chunkname}%
4800 <item>% define this label for every chunk instance, so we
4802 <item>% can tell when we are the last chunk of this name
4804 <item> \ \\label{label-\\chunkname}%
4807 We also try and add the chunk to the list of listings, but I'm afraid we
4808 don't do very well. We want each chunk name listing once, with all of it's
4811 <\nf-chunk|./fangle.sty>
4812 <item> \ \\addcontentsline{lol}{lstlisting}{\\lst@name~[\\protect\\subpageref{\\chunkname}]}%
4815 We then call the noweb output macros in the same way that noweave generates
4816 them, except that we don't need to call <verbatim|\\nwstartdeflinemarkup>
4817 or <verbatim|\\nwenddeflinemarkup> <emdash> and if we do, it messes up the
4820 <\nf-chunk|./fangle.sty>
4821 <item> \ \\nwmargintag{%
4825 <item> \ \ \ \ \ \\nwtagstyle{}%
4827 <item> \ \ \ \ \ \\subpageref{\\chunkname}%
4837 <item> \ \ \ {\\lst@name}%
4841 <item> \ \ \ \ \ \\nwtagstyle{}\\/%
4843 <item> \ \ \ \ \ \\@ifundefined{fangle@chunk@params}{}{%
4845 <item> \ \ \ \ \ \ \ (\\fangle@chunk@params)%
4849 <item> \ \ \ \ \ [\\csname \\chunkcount\\endcsname]~%
4851 <item> \ \ \ \ \ \\subpageref{\\firstchunkname}%
4855 <item> \ \ \ \\@ifundefined{fangle@chunk@append}{}{%
4857 <item> \ \ \ \\ifx{}\\fangle@chunk@append{x}\\else%
4859 <item> \ \ \ \ \ \ \ ,~add~to~\\fangle@chunk@append%
4865 <item>\\global\\def\\fangle@chunk@append{}%
4867 <item>\\lstset{append=x}%
4873 <item> \ \\ifx\\relax\\prevchunkname\\endmoddef\\else\\plusendmoddef\\fi%
4875 <item>% \ \\nwstartdeflinemarkup%
4877 <item> \ \\nwprevnextdefs{\\prevchunkname}{\\nextchunkname}%
4879 <item>% \ \\nwenddeflinemarkup%
4884 Originally this was developed as a <verbatim|listings> aspect, in the Init
4885 hook, but it was found easier to affect the title without using a hook
4886 <emdash> <verbatim|\\lst@AddToHookExe{PreSet}> is still required to set the
4887 listings name to the name passed to the <verbatim|\\Chunk> command, though.
4889 <\nf-chunk|./fangle.sty>
4890 <item>%\\lst@BeginAspect{fangle}
4892 <item>%\\lst@Key{fangle}{true}[t]{\\lstKV@SetIf{#1}{true}}
4894 <item>\\lst@AddToHookExe{PreSet}{\\global\\let\\lst@intname\\lst@chunkname}
4896 <item>\\lst@AddToHook{Init}{}%\\fangle@caption}
4898 <item>%\\lst@EndAspect
4901 <subsection|Cross references>
4903 We define the <verbatim|\\chunkref> command which makes it easy to generate
4904 visual references to different code chunks, e.g.
4906 <block|<tformat|<table|<row|<cell|Macro>|<cell|Appearance>>|<row|<cell|<verbatim|\\chunkref{preamble}>>|<cell|>>|<row|<cell|<verbatim|\\chunkref[3]{preamble}>>|<cell|>>|<row|<cell|<verbatim|\\chunkref{preamble}[arg1,
4909 Chunkref can also be used within a code chunk to include another code
4910 chunk. The third optional parameter to chunkref is a comma sepatarated list
4911 of arguments, which will replace defined parameters in the chunkref.
4914 Darn it, if I have: <verbatim|=\<less\>\\chunkref{new-mode-tracker}[{chunks[chunk_name,
4915 "language"]},{mode}]\<gtr\>> the inner braces (inside [ ]) cause _ to
4916 signify subscript even though we have <verbatim|lst@ReplaceIn>
4919 <\nf-chunk|./fangle.sty>
4920 <item>\\def\\chunkref@args#1,{%
4922 <item> \ \\def\\arg{#1}%
4924 <item> \ \\lst@ReplaceIn\\arg\\lst@filenamerpl%
4928 <item> \ \\@ifnextchar){\\relax}{, \\chunkref@args}%
4932 <item>\\newcommand\\chunkref[2][0]{%
4934 <item> \ \\@ifnextchar({\\chunkref@i{#1}{#2}}{\\chunkref@i{#1}{#2}()}%
4938 <item>\\def\\chunkref@i#1#2(#3){%
4940 <item> \ \\def\\zero{0}%
4942 <item> \ \\def\\chunk{#2}%
4944 <item> \ \\def\\chunkno{#1}%
4946 <item> \ \\def\\chunkargs{#3}%
4948 <item> \ \\ifx\\chunkno\\zero%
4950 <item> \ \ \ \\def\\chunkname{#2-1}%
4954 <item> \ \ \ \\def\\chunkname{#2-\\chunkno}%
4958 <item> \ \\let\\lst@arg\\chunk%
4960 <item> \ \\lst@ReplaceIn\\chunk\\lst@filenamerpl%
4962 <item> \ \\LA{%\\moddef{%
4964 <item> \ \ \ {\\chunk}%
4968 <item> \ \ \ \ \ \\nwtagstyle{}\\/%
4970 <item> \ \ \ \ \ \\ifx\\chunkno\\zero%
4972 <item> \ \ \ \ \ \\else%
4974 <item> \ \ \ \ \ [\\chunkno]%
4976 <item> \ \ \ \ \ \\fi%
4978 <item> \ \ \ \ \ \\ifx\\chunkargs\\empty%
4980 <item> \ \ \ \ \ \\else%
4982 <item> \ \ \ \ \ \ \ (\\chunkref@args #3,)%
4984 <item> \ \ \ \ \ \\fi%
4986 <item> \ \ \ \ \ ~\\subpageref{\\chunkname}%
4992 <item> \ \\RA%\\endmoddef%
4997 <subsection|The end>
4999 <\nf-chunk|./fangle.sty>
5002 <item>%\\makeatother
5005 <chapter|Extracting fangle>
5007 <section|Extracting from Lyx>
5009 To extract from <LyX>, you will need to configure <LyX> as explained in
5010 section <reference|Configuring-the-build>.
5012 <label|lyx-build-script>And this lyx-build scrap will extract fangle for
5015 <\nf-chunk|lyx-build>
5022 <item>=\<less\>\\chunkref{lyx-build-helper}\<gtr\>
5024 <item>cd $PROJECT_DIR \|\| exit 1
5028 <item>/usr/local/bin/fangle -R./fangle $TEX_SRC \<gtr\> ./fangle
5030 <item>/usr/local/bin/fangle -R./fangle.module $TEX_SRC \<gtr\>
5035 <item>=\<less\>\\chunkref{test:helpers}\<gtr\>
5037 <item>export FANGLE=./fangle
5039 <item>export TMP=${TMP:-/tmp}
5041 <item>=\<less\>\\chunkref{test:run-tests}\<gtr\>
5043 <item># Now check that we can extract a fangle that also passes the
5046 <item>$FANGLE -R./fangle $TEX_SRC \<gtr\> ./new-fangle
5048 <item>export FANGLE=./new-fangle
5050 <item>=\<less\>\\chunkref{test:run-tests}\<gtr\>
5053 <\nf-chunk|test:run-tests>
5056 <item>$FANGLE -Rpca-test.awk $TEX_SRC \| awk -f - \|\| exit 1
5058 <item>=\<less\>\\chunkref{test:cromulence}\<gtr\>
5060 <item>=\<less\>\\chunkref{test:escapes}\<gtr\>
5062 <item>=\<less\>\\chunkref{test:chunk-params}\<gtr\>
5065 With a lyx-build-helper
5067 <\nf-chunk|lyx-build-helper>
5068 <item>PROJECT_DIR="$LYX_r"
5070 <item>LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
5072 <item>TEX_DIR="$LYX_p"
5074 <item>TEX_SRC="$TEX_DIR/$LYX_i"
5077 <section|Extracting documentation>
5079 <\nf-chunk|./gen-www>
5080 <item>#python -m elyxer --css lyx.css $LYX_SRC \| \\
5082 <item># \ iconv -c -f utf-8 -t ISO-8859-1//TRANSLIT \| \\
5084 <item># \ sed 's/UTF-8"\\(.\\)\<gtr\>/ISO-8859-1"\\1\<gtr\>/' \<gtr\>
5085 www/docs/fangle.html
5089 <item>python -m elyxer --css lyx.css --iso885915 --html --destdirectory
5090 www/docs/fangle.e \\
5092 <item> \ \ \ \ \ \ fangle.lyx \<gtr\> www/docs/fangle.e/fangle.html
5096 <item>( mkdir -p www/docs/fangle && cd www/docs/fangle && \\
5098 <item> \ lyx -e latex ../../../fangle.lyx && \\
5100 <item> \ htlatex ../../../fangle.tex "xhtml,fn-in" && \\
5102 <item> \ sed -i -e 's/\<less\>!--l\\. [0-9][0-9]* *--\<gtr\>//g'
5109 <item>( mkdir -p www/docs/literate && cd www/docs/literate && \\
5111 <item> \ lyx -e latex ../../../literate.lyx && \\
5113 <item> \ htlatex ../../../literate.tex "xhtml,fn-in" && \\
5115 <item> \ sed -i -e 's/\<less\>!--l\\. [0-9][0-9]* *--\<gtr\>$//g'
5121 <section|Extracting from the command line>
5123 First you will need the tex output, then you can extract:
5125 <\nf-chunk|lyx-build-manual>
5126 <item>lyx -e latex fangle.lyx
5128 <item>fangle -R./fangle fangle.tex \<gtr\> ./fangle
5130 <item>fangle -R./fangle.module fangle.tex \<gtr\> ./fangle.module
5135 <\nf-chunk|test:helpers>
5140 <item> \ then echo "Passed"
5142 <item> \ else echo "Failed"
5144 <item> \ \ \ \ \ \ return 1
5156 <item> \ then echo "Passed"
5158 <item> \ else echo "Failed"
5160 <item> \ \ \ \ \ \ return 1
5169 <chapter|Chunk Parameters>
5171 <\nf-chunk|test:chunk-params:sub>
5172 <item>I see a ${THING},
5174 <item>a ${THING} of colour ${colour},\
5176 <item>and looking closer =\<less\>\\chunkref{test:chunk-params:sub:sub}(${colour})\<gtr\>
5177 </nf-chunk||<tuple|THING|colour>>
5179 <\nf-chunk|test:chunk-params:sub:sub>
5180 <item>a funny shade of ${colour}
5181 </nf-chunk||<tuple|colour>>
5183 <\nf-chunk|test:chunk-params:text>
5184 <item>What do you see? "=\<less\>\\chunkref{test:chunk-params:sub}(joe,
5190 Should generate output:
5192 <\nf-chunk|test:chunk-params:result>
5193 <item>What do you see? "I see a joe,
5195 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ a joe of colour red,\
5197 <item> \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ and looking closer a funny shade
5203 And this chunk will perform the test:
5205 <\nf-chunk|test:chunk-params>
5206 <item>$FANGLE -Rtest:chunk-params:result $TEX_SRC \<gtr\> $TMP/answer
5209 <item>$FANGLE -Rtest:chunk-params:text $TEX_SRC \<gtr\> $TMP/result \|\|
5212 <item>passtest diff $TMP/answer $TMP/result \|\| (echo
5213 test:chunk-params:text failed ; exit 1)
5216 <chapter|Compile-log-lyx><label|Compile-log-lyx>
5218 <\nf-chunk|Chunk: ./compile-log-lyx>
5221 <item># can't use gtkdialog -i, cos it uses the "source" command which
5222 ubuntu sh doesn't have
5228 <item> \ errors="/tmp/compile.log.$$"
5230 <item># \ if grep '^[^ ]*:\\( In \\\|[0-9][0-9]*: [^ ]*:\\)' \<gtr\>
5233 <item>if grep '^[^ ]*(\\([0-9][0-9]*\\)) *: *\\(error\\\|warning\\)'
5238 <item> \ \ \ sed -i -e 's/^[^ ]*[/\\\\]\\([^/\\\\]*\\)(\\([ 0-9][
5239 0-9]*\\)) *: */\\1:\\2\|\\2\|/' $errors
5241 <item> \ \ \ COMPILE_DIALOG='
5243 <item> \<less\>vbox\<gtr\>
5245 <item> \ \<less\>text\<gtr\>
5247 <item> \ \ \ \<less\>label\<gtr\>Compiler errors:\<less\>/label\<gtr\>
5249 <item> \ \<less\>/text\<gtr\>
5251 <item> \ \<less\>tree exported_column="0"\<gtr\>
5253 <item> \ \ \ \<less\>variable\<gtr\>LINE\<less\>/variable\<gtr\>
5255 <item> \ \ \ \<less\>height\<gtr\>400\<less\>/height\<gtr\>\<less\>width\<gtr\>800\<less\>/width\<gtr\>
5257 <item> \ \ \ \<less\>label\<gtr\>File \| Line \|
5258 Message\<less\>/label\<gtr\>
5260 <item> \ \ \ \<less\>action\<gtr\>'". $SELF ; "'lyxgoto
5261 $LINE\<less\>/action\<gtr\>
5263 <item> \ \ \ \<less\>input\<gtr\>'"cat $errors"'\<less\>/input\<gtr\>
5265 <item> \ \<less\>/tree\<gtr\>
5267 <item> \ \<less\>hbox\<gtr\>
5269 <item> \ \ \<less\>button\<gtr\>\<less\>label\<gtr\>Build\<less\>/label\<gtr\>
5271 <item> \ \ \ \ \<less\>action\<gtr\>lyxclient -c "LYXCMD:build-program"
5272 &\<less\>/action\<gtr\>
5274 <item> \ \ \<less\>/button\<gtr\>
5276 <item> \ \ \<less\>button ok\<gtr\>\<less\>/button\<gtr\>
5278 <item> \ \<less\>/hbox\<gtr\>
5280 <item> \<less\>/vbox\<gtr\>
5284 <item> \ \ \ export COMPILE_DIALOG
5286 <item> \ \ \ ( gtkdialog --program=COMPILE_DIALOG ; rm $errors ) &
5290 <item> \ \ \ rm $errors
5300 <item> \ file="${LINE%:*}"
5302 <item> \ line="${LINE##*:}"
5304 <item> \ extraline=\0cat $file \| head -n $line \| tac \| sed
5305 '/^\\\\\\\\begin{lstlisting}/q' \| wc -l\0
5307 <item> \ extraline=\0expr $extraline - 1\0
5309 <item> \ lyxclient -c "LYXCMD:command-sequence server-goto-file-row $file
5310 $line ; char-forward ; repeat $extraline paragraph-down ;
5311 paragraph-up-select"
5319 <item>if test -z "$COMPILE_DIALOG"
5321 <item>then main "$@"\
5331 <associate|info-flag|short>
5332 <associate|page-medium|paper>
5333 <associate|page-screen-height|982016tmpt>
5334 <associate|page-screen-margin|false>
5335 <associate|page-screen-width|1686528tmpt>
5336 <associate|preamble|false>
5337 <associate|sfactor|5>