1 #LyX 1.6.4 created this file. For more info see http://www.lyx.org/
8 %\definecolor{darkgreen}{rgb}{0,0.5,0}
9 \lstset{numbers=left, stepnumber=1, numbersep=5pt, breaklines=false,
10 basicstyle=\footnotesize\ttfamily,
11 %keywordstyle=\color{darkgreen},
12 numberstyle=\tiny,language=C,columns=fullflexible,
15 \use_default_options true
24 \font_typewriter default
25 \font_default_family default
32 \paperfontsize default
35 \pdf_title "Newfangle"
36 \pdf_author "Sam Liddicott"
37 \pdf_subject "Literate Programing"
38 \pdf_keywords "notangle noweb noweave literate programming cweb"
40 \pdf_bookmarksnumbered false
41 \pdf_bookmarksopen false
42 \pdf_bookmarksopenlevel 1
54 \paperorientation portrait
57 \paragraph_separation skip
59 \quotes_language english
62 \paperpagestyle default
63 \tracking_changes false
83 \begin_layout Chapter*
87 \begin_layout Standard
92 is a tool for newfangled literate programming.
93 Newfangled is defined as
95 New and often needlessly novel
104 \begin_layout Standard
105 In this case, newfangled means yet another new and improved method for literate
109 \begin_layout Standard
114 has a long history starting with the great
118 himself, whose literate programming tools seem to make use of as many escaped
119 abbreviations for semantic markup as TeX itself.
122 \begin_layout Standard
131 set of tools (notangle, noweave and noroots) and helpfully reduced the
132 amount of magic character sequences to just
133 \begin_inset Flex CharStyle:Code
136 \begin_layout Plain Layout
143 \begin_inset Flex CharStyle:Code
146 \begin_layout Plain Layout
152 , and in doing so brought the wonders of literate programming within my
156 \begin_layout Standard
157 While using the LyX editor for LaTeX editing I had various troubles with
158 the noweb tools, some of which were my fault, some of which were noweb's
159 fault and some of which were LyX's fault.
162 \begin_layout Standard
167 generally brought literate programming to the masses through removing some
168 of the complexity of the original literate programming, but this would
169 be of no advantage to me if the LyX / LaTeX combination brought more complicati
173 \begin_layout Standard
178 was thus born --- as an awk replacement for notangle, adding some important
179 features, like better integration with LyX and LaTeX, multiple output format
180 conversions, and fixing notangle bugs like indentation when using -L for
184 \begin_layout Standard
185 Significantly, newfangle is just one program which replaces various programs
187 Specifically noweave is done away with and implemented directly as LaTeX
188 macros, and noroots is implemented as a function of the untangler
195 \begin_layout Standard
196 Newfangle is written in awk for portability reasons, awk being available
198 A python conversion will probably be attempted for the benefit of LyX.
199 (Hasn't anyone implemented awk in python yet?)
202 \begin_layout Standard
203 As an extension to many literate-programming styles, Newfangle permits code
204 chunks to take parameters and thus operate somewhat like C pre-processor
205 macros, or like C++ templates.
208 \begin_layout Section*
212 \begin_layout Enumerate
213 ^^ is always going to be a problem, see texbytopic 1.2.2 (Work out what I
217 \begin_layout Enumerate
218 copy over up to date Makefile guide from noweb-lyx document
221 \begin_layout Enumerate
222 Make chunk-name settings only apply to chunks with that name
225 \begin_layout Enumerate
226 indent of multi-line chunks may be mode dependant (i.e.
227 not in string literals)
230 \begin_layout Enumerate
231 support chunk-param usage =<
236 \begin_layout Enumerate
237 trim spaces from param
240 \begin_layout Enumerate
241 add support for other commands in =<...>, starting with
243 label which takes the line-number within the chunk, and maybe should also
244 take the chunk name/page
247 \begin_layout Enumerate
248 cant have listing inside a ruled box
251 \begin_layout Enumerate
252 when a parameterized chunk is included as well as the #line emission, say
253 what the paremeters were for that invocation.
256 \begin_layout Enumerate
257 with 2 macro expansions on one line ${} ${} the first is too greedy and
261 \begin_layout Enumerate
264 chunkref[3]{preamble} to include a certain chunk needs to work in newfangle.awk
265 instead of failing to be recognized at all
268 \begin_layout Enumerate
269 make in-listins labels track the chunk ref too, and make
271 chunref{[2],thing}> resolve to 41c (or d, or whatever chunk the 2nd chunk
275 \begin_layout Enumerate
278 chunkref in text needs a trailing space maybe, it keeps butting up to the
282 \begin_layout Enumerate
283 because the white-space indent is output by the parent chunk, the #line
284 is that of the parent chunk.
285 White space indents must be passed to the child chunk
288 \begin_layout Chapter*
292 \begin_layout Standard
293 \begin_inset CommandInset label
299 Newfangle is licensed under the GPL 3
300 \begin_inset CommandInset citation
307 This doesn't mean that you can't use or distribute newfangle with sources
308 of an incompatible license, but it means you must make the source of newfangle
313 gpl3-copyright,language=
316 \begin_layout Standard
317 \begin_inset listings
321 \begin_layout Plain Layout
323 newfangle - fully featured notangle replacement in awk
326 \begin_layout Plain Layout
330 \begin_layout Plain Layout
332 Copyright (C) Sam Liddicott 2009
335 \begin_layout Plain Layout
339 \begin_layout Plain Layout
341 This program is free software: you can redistribute it and/or modify
344 \begin_layout Plain Layout
346 it under the terms of the GNU General Public License as published by
349 \begin_layout Plain Layout
351 the Free Software Foundation, either version 3 of the License, or
354 \begin_layout Plain Layout
356 (at your option) any later version.
359 \begin_layout Plain Layout
363 \begin_layout Plain Layout
365 This program is distributed in the hope that it will be useful,
368 \begin_layout Plain Layout
370 but WITHOUT ANY WARRANTY; without even the implied warranty of
373 \begin_layout Plain Layout
375 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
379 \begin_layout Plain Layout
381 GNU General Public License for more details.
384 \begin_layout Plain Layout
388 \begin_layout Plain Layout
390 You should have received a copy of the GNU General Public License
393 \begin_layout Plain Layout
395 along with this program.
396 If not, see <http://www.gnu.org/licenses/>.
399 \begin_layout Plain Layout
408 \begin_layout Standard
409 \begin_inset CommandInset toc
410 LatexCommand tableofcontents
421 \begin_layout Chapter
425 \begin_layout Standard
426 Newfangle is a replacement for noweb, which consists of
427 \begin_inset Flex CharStyle:Code
430 \begin_layout Plain Layout
437 \begin_inset Flex CharStyle:Code
440 \begin_layout Plain Layout
447 \begin_inset Flex CharStyle:Code
450 \begin_layout Plain Layout
459 \begin_layout Standard
461 \begin_inset Flex CharStyle:Code
464 \begin_layout Plain Layout
471 \begin_inset Flex CharStyle:Code
474 \begin_layout Plain Layout
480 it can read multiple named files, or from stdin.
483 \begin_layout Section
487 \begin_layout Standard
488 The -r option causes newfangle to behave like noroots.
491 \begin_layout LyX-Code
492 newfangle -r filename.tex
495 \begin_layout Standard
496 will print out the newfangle roots of a tex file.
500 \begin_layout Standard
502 \begin_inset Flex CharStyle:Code
505 \begin_layout Plain Layout
511 command, the roots are not enclosed in
512 \begin_inset Flex CharStyle:Code
515 \begin_layout Plain Layout
521 , unless at least one of the roots is defined using the
522 \begin_inset Flex CharStyle:Code
525 \begin_layout Plain Layout
532 \begin_inset Flex CharStyle:Code
535 \begin_layout Plain Layout
544 \begin_layout Standard
545 Also, unlike noroots, it prints out all roots --- not just those that are
547 I find that a root not being used, doesn't make it particularly top level.
548 My convention is that top level roots to be extracted begin with
549 \begin_inset Flex CharStyle:Code
552 \begin_layout Plain Layout
558 and have the form of a filename.
561 \begin_layout Section
565 \begin_layout Standard
567 \begin_inset Flex CharStyle:Code
570 \begin_layout Plain Layout
577 \begin_inset Flex CharStyle:Code
580 \begin_layout Plain Layout
586 options are supported.
589 \begin_layout Standard
590 The standard way to extract a file would be:
593 \begin_layout LyX-Code
594 newfangle -R./Makefile.inc newfangle.tex > ./Makefile.inc
597 \begin_layout Standard
599 \begin_inset Flex CharStyle:Code
602 \begin_layout Plain Layout
609 \begin_inset Flex CharStyle:Code
612 \begin_layout Plain Layout
618 option does not break indenting; also the
619 \begin_inset Flex CharStyle:Code
622 \begin_layout Plain Layout
628 option does not interrupt (and break) multi-line C macros --- or indeed
629 any line ending with a backslash.
630 This does mean that sometimes the compiler might calculate the source line
631 wrongly when generating error messages in such cases, but there isn't any
632 other way around if multi-line macros include other chunks.
635 \begin_layout Section
636 Formatting source in LaTeX
639 \begin_layout Standard
640 The noweave replacement is a set of LaTeX macros dependant upon
644 , and which can be included with:
647 \begin_layout LyX-Code
650 usepackage{newfangle.sty}
653 \begin_layout Standard
654 The LaTeX macros are shown in section
655 \begin_inset CommandInset ref
657 reference "sec:Latex-Macros"
661 , and are part of a LyX module file
662 \begin_inset Flex CharStyle:Code
665 \begin_layout Plain Layout
671 , which automatically includes the macros in the document pre-amble when
672 the newfangle LyX module is used.
675 \begin_layout Standard
676 Because the noweave replacement is impemented in LaTeX, there is no processing
677 stage required before running the
678 \begin_inset Flex CharStyle:Code
681 \begin_layout Plain Layout
688 LaTeX may need running two or more times, so that the code chunk references
689 can be fully calculated.
692 \begin_layout Standard
694 \begin_inset Flex CharStyle:Code
697 \begin_layout Plain Layout
703 package is required as it is used for formatting the code chunk captions
706 \begin_layout Standard
708 \begin_inset Flex CharStyle:Code
711 \begin_layout Plain Layout
717 package is also required, as it is used for formatting the code chunks
721 \begin_layout Standard
723 \begin_inset Flex CharStyle:Code
726 \begin_layout Plain Layout
732 package is also required.
735 \begin_layout Chapter
736 Literate Programming with Newfangle
739 \begin_layout Standard
741 Should really follow on from a part-0 explanation of what literate programming
745 \begin_layout Chapter
746 Using Newfangle with LyX
749 \begin_layout Section
753 \begin_layout Subsection
754 Installing the LyX module
757 \begin_layout Standard
759 \begin_inset Flex CharStyle:Code
762 \begin_layout Plain Layout
768 to your LyX layouts directory, which for unix users will be
769 \begin_inset Flex CharStyle:Code
772 \begin_layout Plain Layout
781 \begin_layout Standard
782 You will need to reconfigure LyX by clicking Tools\SpecialChar \menuseparator
783 Reconfigure, and then
787 \begin_layout Subsection
788 \begin_inset CommandInset label
790 name "sub:Configuring-the-build"
794 Configuring the build script
797 \begin_layout Standard
798 Make sure you don't have a conversion defined for Lyx → Program
801 \begin_layout Standard
802 From the menu Tools\SpecialChar \menuseparator
803 Preferences, add a conversion from Latex(Plain) → Program
807 \begin_layout LyX-Code
808 set -x ; newfangle -Rlyx-build $$i |
811 \begin_layout LyX-Code
812 env LYX_b=$$b LYX_i=$$i LYX_o=$$o LYX_p=$$p LYX_r=$$r bash
815 \begin_layout Standard
816 (But don't cut-n-paste it from this document or you'll be pasting a multi-line
817 string which will break your lyx preferences file).
821 \begin_layout Standard
822 I hope that one day, LyX will set these into the environment when calling
826 \begin_layout Standard
827 You may also want to consider adding options to this conversion\SpecialChar \ldots{}
831 \begin_layout LyX-Code
832 parselog=/usr/share/lyx/scripts/listerrors
835 \begin_layout Standard
836 \SpecialChar \ldots{}
837 but if you do you will lose your stderr
841 \begin_layout Plain Layout
842 There is some bash plumbing to get a copy of stderr but this footnote is
851 \begin_layout Standard
852 Now, a shell script chunk called
853 \begin_inset Flex CharStyle:Code
856 \begin_layout Plain Layout
862 will be extracted and run whenever you choose the Document\SpecialChar \menuseparator
867 \begin_layout Standard
868 The lyx-build script for this document is in section
869 \begin_inset CommandInset ref
871 reference "lyx-build-script"
875 and on a unix system will extract
876 \begin_inset Flex CharStyle:Code
879 \begin_layout Plain Layout
886 \begin_inset Flex CharStyle:Code
889 \begin_layout Plain Layout
898 \begin_layout Subsection
899 Preparing your Lyx document
902 \begin_layout Standard
903 It is not necessary to base your literate document on any of the original
904 LyX literate classes; so select a regular class for your document type.
907 \begin_layout Standard
923 \begin_layout Standard
924 In the drop-down style listbox you should notice a new style defined, called
932 \begin_layout Standard
933 When you wish to insert a literate chunk, you enter it's plain name in the
934 Chunk style, instead of the older method that used
935 \begin_inset Flex CharStyle:Code
938 \begin_layout Plain Layout
945 Following the chunk name, you insert a listing with: Insert\SpecialChar \menuseparator
949 \begin_layout Standard
950 Inside the white listing box you can type (or paste using shift+ctrl+V)
952 There is not need to use ctrl+enter at the end of lines as with some older
953 LyX literate techniques --- just press enter as normal.
956 \begin_layout Subsubsection
957 Customising the listing appearance
960 \begin_layout Standard
961 In the final document, the code is formatted using the
966 The chunk style doesn't just define the chunk name, but can also define
967 any other chunk options supported by the lstlistings package
968 \begin_inset Flex CharStyle:Code
971 \begin_layout Plain Layout
980 In fact, what you type in the chunk style is raw latex.
981 If you want to set the chunk language without having to right-click the
983 \begin_inset Flex CharStyle:Code
986 \begin_layout Plain Layout
992 after the chunk name.
995 \begin_layout Standard
996 Of course you can do this by editing the listings box advanced properties
997 by right-clicking on the listings box, but that takes longer, and you can't
998 see at-a-glance what the advanced settings are while editing the document;
999 also advanced settings apply only to that box --- the chunk settings apply
1000 through the rest of the document
1004 \begin_layout Plain Layout
1005 It ought to apply only to subsequent chunks of the same name.
1012 \begin_inset Note Note
1015 \begin_layout Plain Layout
1016 So make sure they only apply to chunks of that name
1024 \begin_layout Subsubsection
1025 Global customisations
1028 \begin_layout Standard
1033 is used to set the code chunks, it's
1034 \begin_inset Flex CharStyle:Code
1037 \begin_layout Plain Layout
1045 command can be used in the pre-amble to set some document wide settings.
1048 \begin_layout Standard
1049 If your source has many words with long sequences of capital letters, then
1051 \begin_inset Flex CharStyle:Code
1054 \begin_layout Plain Layout
1055 columns=fullflexible
1060 may be a good idea, or the capital letters will get crowded.
1061 (I think lstlistings ought to use a slightly smaller font for captial letters
1062 so that they still fit).
1065 \begin_layout Standard
1067 \begin_inset Flex CharStyle:Code
1070 \begin_layout Plain Layout
1078 looks more normal for code, but has no bold (unless luximono is used, but
1079 it doesn't work for me); so I use
1080 \begin_inset Flex CharStyle:Code
1083 \begin_layout Plain Layout
1093 \begin_inset Flex CharStyle:Code
1096 \begin_layout Plain Layout
1105 \begin_inset Flex CharStyle:Code
1108 \begin_layout Plain Layout
1109 columns=fullflexible
1114 is used or the wrong letter spacing is used.
1117 \begin_layout Standard
1118 In my LeTeX pre-amble I usually specialise my code format with:
1122 document-preamble,language=tex
1125 \begin_layout Standard
1126 \begin_inset listings
1130 \begin_layout Plain Layout
1137 \begin_layout Plain Layout
1141 definecolor{darkgreen}{rgb}{0,0.5,0}
1144 \begin_layout Plain Layout
1148 lstset{numbers=left, stepnumber=5, numbersep=5pt, breaklines=false,
1151 \begin_layout Plain Layout
1160 \begin_layout Plain Layout
1167 \begin_layout Plain Layout
1171 tiny,language=C,columns=fullflexible,
1174 \begin_layout Plain Layout
1176 numberfirstline=true
1179 \begin_layout Plain Layout
1189 \begin_layout Chapter
1190 Newfangle with Makefiles
1193 \begin_layout Standard
1194 \begin_inset Note Note
1197 \begin_layout Plain Layout
1198 This chapter needs revising
1204 \begin_inset Note Greyedout
1207 \begin_layout Plain Layout
1208 This chapter needs revising
1213 Here we describe a Makefile.inc that you can include in your own Makefiles,
1214 or glue as a recursive make to other projects.
1217 \begin_layout Standard
1218 The Makefile.inc described here was put together for a Samba4 vfs module,
1219 but can be used in any Make project, including automake projects.
1222 \begin_layout Section
1223 A word about makefiles formats
1226 \begin_layout Standard
1227 Whitespace formatting is very important in a Makefile.
1228 The first character of each command line must be a TAB.
1231 \begin_layout LyX-Code
1232 target: pre-requisite
1233 \begin_inset Newline newline
1237 \begin_inset Newline newline
1243 \begin_layout Standard
1244 But a TAB is pretty hard to enter into most of the Lyx formats and insets
1246 An alternative is to use a semi-colon after the pre-requisite, and a backslash
1247 at the end of each line (except the last).
1248 Then any whitespace (or none) can prefix each action.
1251 \begin_layout LyX-Code
1252 target: pre-requisite ;
1255 \begin_inset Newline newline
1261 \begin_inset Newline newline
1267 \begin_layout Standard
1268 This is the style that we use and it works pretty well for GNU make at least.
1271 \begin_layout Standard
1272 We also adopt a convention that code chunks whose names beginning with ./
1273 should always be automatically extracted from the document.
1274 Code chunks whose names do not begin with ./ are for internal reference.
1275 (This doesn't prevent such chunks from being extracted directly).
1278 \begin_layout Section
1279 Boot-strapping the extraction
1282 \begin_layout Subsection
1286 \begin_layout Standard
1287 \begin_inset CommandInset label
1289 name "sub:Bootstrap-Using-a-Makefile"
1293 It seems convenient to have the makefile extract or update the C source
1294 files as part of it's operation.
1295 It also seems convenient to have the makefile itself extracted from this
1299 \begin_layout Standard
1300 It would also be convenient to have the code to extract the makefile from
1301 this document to also be part of this document, however we have to start
1302 somewhere and this unfortunately requires us to type at least a few words
1303 by hand to start things off.
1306 \begin_layout Standard
1307 Therefore we will have a minimal root fragment, which, when extracted, can
1308 cope with extracting the rest of the source.
1309 perhaps with this shell script, which could be called
1314 \begin_inset Note Note
1317 \begin_layout Plain Layout
1318 FIX THIS CHUNK AND TEST IT
1330 \begin_layout Standard
1331 \begin_inset listings
1335 \begin_layout Plain Layout
1340 \begin_layout Plain Layout
1344 \begin_layout Plain Layout
1346 MAKE_SRC="${1:-${NW_LYX:-../../noweb-lyx/noweb-lyx3.lyx}}"
1349 \begin_layout Plain Layout
1351 MAKE_SRC=`dirname "$MAKE_SRC"`/`basename "$MAKE_SRC" .lyx`
1354 \begin_layout Plain Layout
1356 NOWEB_SRC="${2:-${NOWEB_SRC:-$MAKE_SRC.lyx}}"
1359 \begin_layout Plain Layout
1361 lyx -e latex $MAKE_SRC
1364 \begin_layout Plain Layout
1368 \begin_layout Plain Layout
1370 newfangle -R./Makefile.inc ${MAKE_SRC}.tex
1375 \begin_layout Plain Layout
1377 | sed "/NEWFANGLE_SOURCE=/s/^/#/;T;aNOWEB_SOURCE=$NEWFANGLE_SRC"
1382 \begin_layout Plain Layout
1384 | cpif ./Makefile.inc
1387 \begin_layout Plain Layout
1391 \begin_layout Plain Layout
1393 make -f ./Makefile.inc newfangle_sources
1401 \begin_layout Standard
1402 The general Makefile can be invoked with
1406 and can also be included into any automake file to automatically re-generate
1410 \begin_layout Standard
1415 can be extracted with this command:
1418 \begin_layout LyX-Code
1419 lyx -e latex newfangle.lyx &&
1424 \begin_layout LyX-Code
1425 newfangle newfangle.lyx > ./autoboot
1428 \begin_layout Standard
1429 This looks simple enough, but as mentioned, newfangle has to be had from
1430 somewhere before it can be extracted.
1433 \begin_layout Subsection
1434 \begin_inset Note Note
1437 \begin_layout Plain Layout
1438 MERGE THIS WITH THE SECTIONS OF THIS DOCUMENT
1443 \SpecialChar \ldots{}
1447 \begin_layout Standard
1448 When the lyx-build chunk is executed, the current directory will be a temporary
1450 \begin_inset Flex CharStyle:Code
1453 \begin_layout Plain Layout
1459 will refer to the tex file in this temporary directory.
1460 This is unfortunate as our makefile wants to run from the project directory
1461 where the Lyx file is kept.
1464 \begin_layout Standard
1465 We can extract the project directory from $$r, and derive the probable Lyx
1466 filename from the noweb file that Lyx generated.
1473 \begin_layout Standard
1474 \begin_inset listings
1478 \begin_layout Plain Layout
1480 PROJECT_DIR="$LYX_r"
1483 \begin_layout Plain Layout
1485 LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
1488 \begin_layout Plain Layout
1493 \begin_layout Plain Layout
1495 TEX_SRC="$TEX_DIR/$LYX_i"
1503 \begin_layout Standard
1504 And then we can define a lyx-build fragment similar to the autoboot fragment
1511 \begin_layout Standard
1512 \begin_inset listings
1516 \begin_layout Plain Layout
1521 \begin_layout Plain Layout
1525 chunkref{lyx-build-helper}>
1528 \begin_layout Plain Layout
1530 cd $PROJECT_DIR || exit 1
1533 \begin_layout Plain Layout
1537 \begin_layout Plain Layout
1539 #/usr/bin/newfangle -filter ./notanglefix-filter
1544 \begin_layout Plain Layout
1546 # -R./Makefile.inc "../../noweb-lyx/noweb-lyx3.lyx"
1551 \begin_layout Plain Layout
1553 # | sed '/NOWEB_SOURCE=/s/=.*/=samba4-dfs.lyx/'
1558 \begin_layout Plain Layout
1563 \begin_layout Plain Layout
1568 \begin_layout Plain Layout
1570 #make -f ./Makefile.inc newfangle_sources
1578 \begin_layout Section
1582 \begin_layout Subsection
1583 Including Makefile.inc
1586 \begin_layout Standard
1587 \begin_inset CommandInset label
1589 name "sub:Keeping-extracted-files"
1593 Makefile.inc will cope with extracting all the other source files from this
1594 document and keeping them up to date.
1598 \begin_layout Standard
1599 It may also be included by a Makefile or Makefile.am defined in a Lyx document
1600 to automatically deal with the extraction of source files and documents.
1603 \begin_layout Standard
1604 A makefile has two parts; variables must be defined before the targets that
1612 \begin_layout Standard
1613 \begin_inset listings
1617 \begin_layout Plain Layout
1621 chunkref{Makefile.inc-vars}>
1624 \begin_layout Plain Layout
1628 chunkref{Makefile.inc-targets}>
1636 \begin_layout Standard
1638 \begin_inset Flex CharStyle:Code
1641 \begin_layout Plain Layout
1647 to hold the name of this Lyx file.
1654 \begin_layout Standard
1655 \begin_inset listings
1659 \begin_layout Plain Layout
1664 \begin_layout Plain Layout
1666 LITERATE_SOURCE=$(LYX_SOURCE)
1674 \begin_layout Subsection
1675 Recursive use of Makefile.inc
1678 \begin_layout Standard
1679 The makefile glue described here is used when building Samba4 vfs modules.
1682 \begin_layout Standard
1683 If you are defining a module of an existing program you may find it easier
1684 to use a slight recursive make instead of including the makefile directly.
1685 This way there is less chance of definitions in Makefile.inc interfering
1686 with definitions in the main makefile, or with definitions in other Makefile.inc
1687 from other noweb modules.
1690 \begin_layout Standard
1691 The glue works by adding a .PHONY target to call the recursive make, and
1692 adding this target as an additional pre-requisite to the existing targets.
1695 \begin_layout Standard
1696 In this example, the existing build system already has a build target for
1698 \begin_inset Flex CharStyle:Code
1701 \begin_layout Plain Layout
1707 , so we just add another pre-requisite to that.
1709 \begin_inset Flex CharStyle:Code
1712 \begin_layout Plain Layout
1718 as a pre-requisite, the stamp file's modified time indicating when all
1719 sources were extracted.
1726 \begin_layout Standard
1727 \begin_inset listings
1731 \begin_layout Plain Layout
1733 $(example_srcdir)/example.o: $(example_srcdir)/example.tex.stamp
1741 \begin_layout Standard
1742 The target for this new pre-requisite is generated by a recursive make using
1743 Makefile.inc which will make sure that the source is up to date, before
1744 it is built by the main projects makefile.
1751 \begin_layout Standard
1752 \begin_inset listings
1756 \begin_layout Plain Layout
1758 $(example_srcdir)/example.tex.stamp: $(example_srcdir)/example.tex ;
1763 \begin_layout Plain Layout
1765 cd $(example_srcdir) &&
1770 \begin_layout Plain Layout
1772 $(MAKE) -f Makefile.inc newfangle_sources
1780 \begin_layout Standard
1781 We can do similar glue for the docs, clean and distclean targets.
1782 In this example our build system is using a double colon for these targets,
1783 so we use the same in our glue.
1790 \begin_layout Standard
1791 \begin_inset listings
1795 \begin_layout Plain Layout
1800 \begin_layout Plain Layout
1802 .PHONY: docs_example
1805 \begin_layout Plain Layout
1807 docs_example:: ; cd $(example_srcdir) &&
1812 \begin_layout Plain Layout
1814 $(MAKE) -f Makefile.inc docs
1817 \begin_layout Plain Layout
1821 \begin_layout Plain Layout
1823 clean:: clean_example
1826 \begin_layout Plain Layout
1828 .PHONEY: clean_example
1831 \begin_layout Plain Layout
1833 clean_example: ; cd $(example_srcdir) &&
1838 \begin_layout Plain Layout
1840 $(MAKE) -f Makefile.inc clean
1843 \begin_layout Plain Layout
1847 \begin_layout Plain Layout
1849 distclean:: distclean_example
1852 \begin_layout Plain Layout
1854 .PHONY: distclean_example
1857 \begin_layout Plain Layout
1859 distclean_example: ; cd $(example_srcdir) &&
1864 \begin_layout Plain Layout
1866 $(MAKE) -f Makefile.inc distclean
1874 \begin_layout Standard
1875 We could do similarly for install targets to install the generated docs.
1878 \begin_layout Subsection
1879 \begin_inset CommandInset label
1881 name "sub:Converting-from-Lyx"
1885 Converting from Lyx to LaTeX
1888 \begin_layout Standard
1889 The first stage will always be to convert the Lyx file to a LaTeX file;
1890 this must be so not only because newfangle needs to to run on a TeX file,
1891 but also because the Lyx command
1893 server-goto-file-line
1897 \begin_layout Plain Layout
1900 server-goto-file-line
1902 is used to position the Lyx cursor at the compiler errors.
1909 insists that the line number provided is a line in the TeX file, and always
1910 reverse maps this to derive the line in the Lyx docment.
1911 \begin_inset Note Note
1914 \begin_layout Plain Layout
1915 The tex file should probably be an automake extra dist sources or something,
1916 so that it gets produced and packaged by make dist
1924 \begin_layout Standard
1925 The command [[lyx -e literate noweb-lyx.lyx]] will produce [[noweb-lyx.nw]]
1926 a tex file, so we define the noweb target to be the same as the Lyx file
1927 but with the .nw extension.
1934 \begin_layout Standard
1935 \begin_inset listings
1939 \begin_layout Plain Layout
1941 TEX_SOURCE=$(LYX_SOURCE:.lyx=.tex)
1950 Makefile.inc-targets
1953 \begin_layout Standard
1954 \begin_inset listings
1958 \begin_layout Plain Layout
1960 $(TEX_SOURCE): $(LYX_SOURCE) ;
1965 \begin_layout Plain Layout
1970 \begin_layout Plain Layout
1972 clean_tex: ; rm -f -- $(TEX_SOURCE)
1980 \begin_layout Subsection
1981 Extracting Program Source
1984 \begin_layout Standard
1985 The program source is extracted using newfangle, which is designed to operate
1986 on a LaTeX document.
1994 \begin_layout Standard
1995 \begin_inset listings
1999 \begin_layout Plain Layout
2001 NEWFANGLE_SOURCE=$(TEX_SOURCE)
2009 \begin_layout Standard
2010 The Lyx document can result in any number of source documents, but not all
2011 of these will be changed each time the Lyx document is updated.
2012 We certainly don't want to update the timestamps of these files and cause
2013 the whole source tree to be recompiled just because the Lyx document was
2018 \begin_layout Standard
2019 To solve this problem we use a stamp file which is always updated each time
2020 the sources are extracted from the LaTeX document.
2021 If the stamp file is older than the LaTeX document, then we can make an
2022 attempt to re-extract the sources.
2029 \begin_layout Standard
2030 \begin_inset listings
2034 \begin_layout Plain Layout
2036 NEWFANGLE_SOURCE_STAMP=$(NEWFANGLE_SOURCE).stamp
2045 Makefile.inc-targets
2048 \begin_layout Standard
2049 \begin_inset listings
2053 \begin_layout Plain Layout
2055 $(NEWFANGLE_SOURCE_STAMP): $(NEWFANGLE_SOURCE)
2060 \begin_layout Plain Layout
2062 $(NEWFANGLE_SOURCES) ;
2067 \begin_layout Plain Layout
2069 echo > $(NEWFANGLE_SOURCE_STAMP)
2072 \begin_layout Plain Layout
2074 clean_stamp: ; rm -f $(NEWFANGLE_SOURCE_STAMP)
2077 \begin_layout Plain Layout
2087 \begin_layout Subsection
2088 Extracting C sources
2091 \begin_layout Standard
2093 \begin_inset Flex CharStyle:Code
2096 \begin_layout Plain Layout
2102 to hold the names of all the C source files defined in this document.
2103 We compute this only once, by means of := in assignent.
2104 The sed deletes the any <
2105 \begin_inset space \hspace*{}
2110 \begin_inset space \hspace*{}
2114 > which may surround the roots names (for noroots compatibility).
2118 \begin_layout Standard
2119 As we use chunk names beginning with ./ to denote top level fragments that
2120 should be extracted, we filter out all fragments that do not begin with
2128 \begin_layout Standard
2129 \begin_inset listings
2133 \begin_layout Plain Layout
2142 \begin_layout Plain Layout
2144 NEWFANGLE_SOURCES:=$(shell
2149 \begin_layout Plain Layout
2151 newfangle -r $(NEWFANGLE_SOURCE) |
2156 \begin_layout Plain Layout
2158 sed -e 's/^[<][<]//;s/[>][>]$$//;/^$(NEWFANGLE_PREFIX)/!d'
2163 \begin_layout Plain Layout
2165 -e 's/^$(NEWFANGLE_PREFIX)/
2172 \begin_layout Plain Layout
2183 Makefile.inc-targets
2186 \begin_layout Standard
2187 \begin_inset listings
2191 \begin_layout Plain Layout
2193 .PHONY: echo_newfangle_sources
2196 \begin_layout Plain Layout
2198 echo_newfangle_sources: ; @echo $(NEWFANGLE_SOURCES)
2206 \begin_layout Standard
2207 We define a convenient target called
2208 \begin_inset Flex CharStyle:Code
2211 \begin_layout Plain Layout
2217 to re-extract the source if the LaTeX file has been updated.
2221 Makefile.inc-targets
2224 \begin_layout Standard
2225 \begin_inset listings
2229 \begin_layout Plain Layout
2231 .PHONY: newfangle_sources
2234 \begin_layout Plain Layout
2236 newfangle_sources: $(NEWFANGLE_SOURCE_STAMP)
2244 \begin_layout Standard
2245 And also a convenient target to remove extracted sources.
2249 Makefile.inc-targets
2252 \begin_layout Standard
2253 \begin_inset listings
2257 \begin_layout Plain Layout
2259 .PHONY: clean_newfangle_sources
2262 \begin_layout Plain Layout
2264 clean_newfangle_sources: ;
2269 \begin_layout Plain Layout
2271 rm -f -- $(NEWFANGLE_SOURCE_STAMP) $(NEWFANGLE_SOURCES)
2279 \begin_layout Standard
2281 \begin_inset Flex CharStyle:Code
2284 \begin_layout Plain Layout
2290 macro takes 4 arguments: the filename (1), some extensions to match (2)
2291 and a some shell command to return if the filename matches the exentions
2299 \begin_layout Standard
2300 \begin_inset listings
2304 \begin_layout Plain Layout
2306 if_extension=$(if $(findstring $(suffix $(1)),$(2)),$(3),$(4))
2314 \begin_layout Standard
2315 For some source files like C files, we want to output the line number and
2316 filename of the original LaTeX document from which the source came.
2319 \begin_layout Standard
2320 To make this easier we define the file extensions for which we want to do
2328 \begin_layout Standard
2329 \begin_inset listings
2333 \begin_layout Plain Layout
2343 \begin_layout Standard
2344 We can then use the if_extensions macro to define a macro which expands
2346 \begin_inset Flex CharStyle:Code
2349 \begin_layout Plain Layout
2355 option if newfangle is being invoked in a C source file, so that C compile
2356 errors will refer to the line number in the Lyx document.
2364 \begin_layout Standard
2365 \begin_inset listings
2369 \begin_layout Plain Layout
2374 \begin_layout Plain Layout
2376 nf_line=-L -T$(TABS)
2379 \begin_layout Plain Layout
2386 \begin_layout Plain Layout
2388 $(call if_extension,$(2),$(C_EXTENSIONS),$(nf_line))
2393 \begin_layout Plain Layout
2403 \begin_layout Standard
2404 We can use a similar trick to define an
2408 macro which takes just the filename as an argument and can return a pipeline
2409 stage calling the indent command.
2410 Indent can be turned off with
2411 \begin_inset Flex CharStyle:Code
2414 \begin_layout Plain Layout
2415 make newfangle_sources indent=
2427 \begin_layout Standard
2428 \begin_inset listings
2432 \begin_layout Plain Layout
2434 indent_options=-npro -kr -i8 -ts8 -sob -l80 -ss -ncs
2437 \begin_layout Plain Layout
2439 indent=$(call if_extension,$(1),$(C_EXTENSIONS),
2444 \begin_layout Plain Layout
2446 | indent $(indent_options))
2454 \begin_layout Standard
2455 We now define the pattern for extracting a file.
2456 The files are written using noweb's
2462 \begin_layout Plain Layout
2465 So you still need noweb installed in order to use cpif
2471 \begin_inset Note Note
2474 \begin_layout Plain Layout
2477 Write an awk version
2484 so that the file timestamp will not be touched if the contents haven't
2486 This avoids the need to rebuild the entire project because of a typographical
2487 change in the documentation, or if only a few C source files have changed.
2494 \begin_layout Standard
2495 \begin_inset listings
2499 \begin_layout Plain Layout
2501 newfangle_extract=@mkdir -p $(dir $(1)) &&
2506 \begin_layout Plain Layout
2508 $(call newfangle,$(2),$(1)) > "$(1).tmp" &&
2513 \begin_layout Plain Layout
2515 cat "$(1).tmp" $(indent) | cpif "$(1)"
2520 \begin_layout Plain Layout
2522 && rm -- "$(1).tmp" ||
2527 \begin_layout Plain Layout
2529 (echo error newfangling $(1) from $(2) ; exit 1)
2537 \begin_layout Standard
2538 We define a target which will extract or update all sources.
2539 To do this we first defined a makefile template that can do this for any
2540 source file in the LaTeX document.
2547 \begin_layout Standard
2548 \begin_inset listings
2552 \begin_layout Plain Layout
2554 define NEWFANGLE_template
2557 \begin_layout Plain Layout
2564 \begin_layout Plain Layout
2566 $$(call newfangle_extract,$(1),$(2))
2569 \begin_layout Plain Layout
2571 NEWFANGLE_TARGETS+=$(1)
2574 \begin_layout Plain Layout
2584 \begin_layout Standard
2585 We then enumerate the discovered
2586 \begin_inset Flex CharStyle:Code
2589 \begin_layout Plain Layout
2595 to generate a makefile rule for each one using the makefile template we
2600 Makefile.inc-targets
2603 \begin_layout Standard
2604 \begin_inset listings
2608 \begin_layout Plain Layout
2610 $(foreach source,$(NEWFANGLE_SOURCES),
2615 \begin_layout Plain Layout
2617 $(eval $(call NEWFANGLE_template,$(source),$(NEWFANGLE_SOURCE)))
2622 \begin_layout Plain Layout
2632 \begin_layout Standard
2633 These will all be built with NEWFANGLE_SOURCE_STAMP.
2636 \begin_layout Standard
2637 We also remove the generated sources on a
2645 Makefile.inc-targets
2648 \begin_layout Standard
2649 \begin_inset listings
2653 \begin_layout Plain Layout
2655 _distclean: clean_newfangle_sources
2663 \begin_layout Subsection
2664 Extracting Documentation
2667 \begin_layout Standard
2668 We then identify the intermediate stages of the documentation and their
2669 build and clean targets.
2672 \begin_layout Subsubsection
2676 \begin_layout Standard
2677 We produce a pdf file from the tex file.
2684 \begin_layout Standard
2685 \begin_inset listings
2689 \begin_layout Plain Layout
2691 NEWFANGLE_PDF=$(TEX_SOURCE:.tex=.pdf)
2699 \begin_layout Standard
2700 We run pdflatex twice to be sure that the contents and aux files are up
2702 We certainly are required to run pdflatex twice if these files do not exist!
2706 Makefile.inc-targets
2709 \begin_layout Standard
2710 \begin_inset listings
2714 \begin_layout Plain Layout
2716 $(NEWFANGLE_PDF): $(TEX_SOURCE); pdflatex $< && pdflatex $<
2719 \begin_layout Plain Layout
2721 clean_pdf: ; rm -f -- $(NEWFANGLE_PDF)
2726 \begin_layout Plain Layout
2728 $(TEX_SOURCE:.tex=.toc)
2733 \begin_layout Plain Layout
2735 $(TEX_SOURCE:.tex=.log)
2740 \begin_layout Plain Layout
2742 $(TEX_SOURCE:.tex=.aux)
2750 \begin_layout Subsubsection
2754 \begin_layout Standard
2755 Currently we only build pdf as a final format, but NEWFANGLE_DOCS may later
2756 hold other output formats.
2763 \begin_layout Standard
2764 \begin_inset listings
2768 \begin_layout Plain Layout
2770 NEWFANGLE_DOCS=$(NEWFANGLE_PDF)
2778 \begin_layout Standard
2779 We also define newfangle_docs as a convenient phony target<
2783 Makefile.inc-targets
2786 \begin_layout Standard
2787 \begin_inset listings
2791 \begin_layout Plain Layout
2793 .PHONY: newfangle_docs
2796 \begin_layout Plain Layout
2798 newfangle_docs: $(NEWFANGLE_DOCS)
2801 \begin_layout Plain Layout
2803 docs: newfangle_docs
2811 \begin_layout Standard
2812 And define a convenient clean_noweb_docs which we add to the regular clean
2817 Makefile.inc-targets
2820 \begin_layout Standard
2821 \begin_inset listings
2825 \begin_layout Plain Layout
2827 .PHONEY: clean_newfangle_docs
2830 \begin_layout Plain Layout
2832 clean_newfangle_docs: clean_tex clean_pdf
2835 \begin_layout Plain Layout
2837 clean: clean_newfangle_docs
2840 \begin_layout Plain Layout
2844 \begin_layout Plain Layout
2846 distclean_newfangle_docs: clean_tex clean_newfangle_docs
2849 \begin_layout Plain Layout
2851 distclean: clean distclean_newfangle_docs
2859 \begin_layout Subsection
2863 \begin_layout Standard
2864 If Makefile.inc is included into Makefile, then extracted files can be updated
2868 \begin_layout LyX-Code
2869 make newfangle_sources
2872 \begin_layout Standard
2876 \begin_layout LyX-Code
2877 make -f Makefile.inc newfangle_sources
2884 \begin_layout Chapter
2885 Newfangle awk source code
2888 \begin_layout Standard
2889 We use the copyright notice from chapter
2890 \begin_inset CommandInset ref
2892 reference "cha:License"
2900 ./newfangle,language=awk,morestring=[b]{/},morekeywords=else
2903 \begin_layout Standard
2904 \begin_inset listings
2908 \begin_layout Plain Layout
2913 \begin_layout Plain Layout
2917 chunkref{gpl3-copyright}>
2920 \begin_layout Plain Layout
2929 \begin_layout Standard
2930 We also use code from Arnold Robbins public domain getopt (1993 revision)
2932 \begin_inset CommandInset ref
2934 reference "cha:getopt"
2938 , and naturally want to attribute this appropriately.
2941 \begin_layout Standard
2942 \begin_inset listings
2946 \begin_layout Plain Layout
2950 \begin_layout Plain Layout
2952 # NOTE: Arnold Robbins public domain getopt for awk is also used:
2955 \begin_layout Plain Layout
2959 chunkref{getopt.awk-header}>
2962 \begin_layout Plain Layout
2966 \begin_layout Plain Layout
2970 chunkref{getopt.awk-getopt()}>
2973 \begin_layout Plain Layout
2982 \begin_layout Standard
2983 And include the following chunks
2990 \begin_layout Standard
2991 \begin_inset listings
2995 \begin_layout Plain Layout
2999 chunkref{helper-functions}>
3002 \begin_layout Plain Layout
3006 chunkref{mode-tracker}>
3009 \begin_layout Plain Layout
3013 chunkref{parse_chunk_args}>
3016 \begin_layout Plain Layout
3020 chunkref{chunk-storage-functions}>
3023 \begin_layout Plain Layout
3027 chunkref{output_chunk_names()}>
3030 \begin_layout Plain Layout
3034 chunkref{output_chunks()}>
3037 \begin_layout Plain Layout
3041 chunkref{write_chunk()}>
3044 \begin_layout Plain Layout
3048 chunkref{expand_chunk_args()}>
3051 \begin_layout Plain Layout
3058 \begin_layout Plain Layout
3062 chunkref{recognize-chunk}>
3065 \begin_layout Plain Layout
3077 \begin_layout Section
3081 \begin_layout Standard
3082 The portable way to erase an array in awk is to split the empty string,
3087 awk-delete-array,params=ARRAY
3090 \begin_layout Standard
3091 \begin_inset listings
3095 \begin_layout Plain Layout
3097 split("", ${ARRAY});
3106 dump-array,params=ARRAY
3109 \begin_layout Standard
3110 \begin_inset listings
3114 \begin_layout Plain Layout
3125 \begin_layout Plain Layout
3127 for (_x in ${ARRAY}) {
3130 \begin_layout Plain Layout
3132 print _x "=" ${ARRAY}[_x] "
3137 \begin_layout Plain Layout
3142 \begin_layout Plain Layout
3158 \begin_layout Section
3162 \begin_layout Standard
3163 Fatal errors are issued with the error function:
3167 error(),append=helper-functions
3170 \begin_layout Standard
3171 \begin_inset listings
3175 \begin_layout Plain Layout
3177 function error(message)
3180 \begin_layout Plain Layout
3185 \begin_layout Plain Layout
3187 print "ERROR: " FILENAME ":" FNR " " message > "/dev/stderr";
3190 \begin_layout Plain Layout
3195 \begin_layout Plain Layout
3205 \begin_layout Standard
3206 \begin_inset listings
3210 \begin_layout Plain Layout
3212 function warning(message)
3215 \begin_layout Plain Layout
3220 \begin_layout Plain Layout
3222 print "WARNING: " FILENAME ":" FNR " " message > "/dev/stderr";
3225 \begin_layout Plain Layout
3230 \begin_layout Plain Layout
3240 \begin_layout Chapter
3244 \begin_layout Standard
3245 LaTeX arguments to lstlistings macros are a comma seperated list of key-value
3247 Values containing commas are enclosed in
3248 \begin_inset Flex CharStyle:Code
3251 \begin_layout Plain Layout
3258 \begin_inset Flex CharStyle:Code
3261 \begin_layout Plain Layout
3267 , which is to be expected for LaTeX.
3270 \begin_layout Standard
3271 A sample expressions is:
3274 \begin_layout LyX-Code
3275 name=thomas, params={a, b}, something, something-else
3278 \begin_layout Standard
3279 but we see that this is just a simpler form of this expression:
3282 \begin_layout LyX-Code
3283 name=freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
3286 \begin_layout Standard
3287 We may consider that we need a function that can parse such LaTeX expressions
3288 and assign the values to an
3292 associated array, perhaps using a recursive parser into a multi-dimensional
3297 \begin_layout Plain Layout
3298 as AWK doesn't have nested-hash support
3306 \begin_layout Standard
3307 \begin_inset Tabular
3308 <lyxtabular version="3" rows="6" columns="2">
3310 <column alignment="left" valignment="top" width="0">
3311 <column alignment="left" valignment="top" width="0">
3313 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3316 \begin_layout Plain Layout
3322 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3325 \begin_layout Plain Layout
3333 <cell alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
3336 \begin_layout Plain Layout
3342 <cell alignment="left" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3345 \begin_layout Plain Layout
3353 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3356 \begin_layout Plain Layout
3362 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3365 \begin_layout Plain Layout
3373 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3376 \begin_layout Plain Layout
3382 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3385 \begin_layout Plain Layout
3393 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3396 \begin_layout Plain Layout
3402 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3405 \begin_layout Plain Layout
3413 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3416 \begin_layout Plain Layout
3422 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3425 \begin_layout Plain Layout
3439 \begin_layout Standard
3440 Oon reflection it seems that sometimes such nesting is not desirable, as
3441 the braces are also used to delimit values that contain commas --- we may
3445 \begin_layout LyX-Code
3446 name={williamson, freddie}
3449 \begin_layout Standard
3451 \begin_inset Flex CharStyle:Code
3454 \begin_layout Plain Layout
3461 \begin_inset Flex CharStyle:Code
3464 \begin_layout Plain Layout
3473 \begin_layout Standard
3474 In fact we are not so interested in the detail so as to be bothered by this,
3475 which turns out to be a good thing for two reasons.
3476 Firstly LaTeX has a malleable parser with no strict syntax, and secondly
3478 \begin_inset Flex CharStyle:Code
3481 \begin_layout Plain Layout
3488 \begin_inset Flex CharStyle:Code
3491 \begin_layout Plain Layout
3497 should count as two items will be context dependant anyway.
3500 \begin_layout Standard
3501 We need to parse this latex for only one reason; which is that we are extending
3502 lstlistings to add some additional arguments which will be used to express
3503 chunk parameters and other chunk options.
3506 \begin_layout Section
3507 Additional lstlstings parameters
3510 \begin_layout Standard
3511 Further on we define a
3512 \begin_inset Flex CharStyle:Code
3515 \begin_layout Plain Layout
3523 LaTeX macro whose arguments will consist of a the chunk name, optionally
3524 followed by a comma and then a comma separated list of arguments.
3525 In fact we will just need to prefix
3526 \begin_inset Flex CharStyle:Code
3529 \begin_layout Plain Layout
3535 to the arguments to in order to create valid lstlistings arguments.
3539 \begin_layout Standard
3540 There will be other arguments supported too;
3543 \begin_layout Description
3544 params As an extension to many literate-programming styles, newfangle permits
3545 code chunks to take parameters and thus operate somewhat like C pre-processor
3546 macros, or like C++ templates.
3547 Chunk parameters are declared with a chunk argument called
3548 \begin_inset Flex CharStyle:Code
3551 \begin_layout Plain Layout
3557 , which holds a semi-colon separated list of parameters, like this:
3560 \begin_layout LyX-Code
3561 achunk,language=C,params=name;address
3564 \begin_layout Description
3565 addto a named chunk that this chunk is to be included into.
3566 This saves the effort of having to declare another listing of the named
3567 chunk merely to include this one.
3570 \begin_layout Standard
3572 \begin_inset Flex Chunkref
3575 \begin_layout Plain Layout
3581 will accept two paramters,
3582 \begin_inset Flex CharStyle:Code
3585 \begin_layout Plain Layout
3591 being the text to parse, and
3592 \begin_inset Flex CharStyle:Code
3595 \begin_layout Plain Layout
3601 being an array to receive the parsed values as described above.
3602 The optional parameter
3603 \begin_inset Flex CharStyle:Code
3606 \begin_layout Plain Layout
3612 is used during recursion to build up the multi-dimensional array path.
3619 \begin_layout Standard
3620 \begin_inset listings
3624 \begin_layout Plain Layout
3628 chunkref{get_chunk_args()}>
3640 \begin_layout Standard
3641 \begin_inset listings
3645 \begin_layout Plain Layout
3647 function get_chunk_args(text, values,
3650 \begin_layout Plain Layout
3652 # optional parameters
3655 \begin_layout Plain Layout
3657 path, # hierarchical precursors
3660 \begin_layout Plain Layout
3665 \begin_layout Plain Layout
3675 \begin_layout Standard
3676 The strategy is to parse the name, and then look for a value.
3677 If the value begins with a brace
3678 \begin_inset Flex CharStyle:Code
3681 \begin_layout Plain Layout
3687 , then we recurse and consume as much of the text as necessary, returning
3688 the remaining text when we encounter a leading close-brace
3689 \begin_inset Flex CharStyle:Code
3692 \begin_layout Plain Layout
3699 This being the strategy --- and executed in a loop --- we realise that
3700 we must first look for the closing brace (perhaps preceded by white space)
3701 in order to terminate the recursion, and returning remaining text.
3704 \begin_layout Standard
3705 \begin_inset listings
3709 \begin_layout Plain Layout
3714 \begin_layout Plain Layout
3716 split("", next_chunk_args);
3719 \begin_layout Plain Layout
3721 while(length(text)) {
3724 \begin_layout Plain Layout
3726 if (match(text, "^ *}(.*)", a)) {
3729 \begin_layout Plain Layout
3734 \begin_layout Plain Layout
3739 \begin_layout Plain Layout
3743 chunkref{parse-chunk-args}>
3746 \begin_layout Plain Layout
3751 \begin_layout Plain Layout
3756 \begin_layout Plain Layout
3766 \begin_layout Standard
3767 \begin_inset Note Note
3770 \begin_layout Plain Layout
3771 Use BNF package here
3776 We can see that the text could be inspected with this regex:
3783 \begin_layout Standard
3784 \begin_inset listings
3788 \begin_layout Plain Layout
3790 if (! match(text, " *([^,=]*[^,= ]) *(([,=]) *(([^,}]*) *,* *(.*))|)$", a))
3794 \begin_layout Plain Layout
3799 \begin_layout Plain Layout
3809 \begin_layout Standard
3811 \begin_inset Flex CharStyle:Code
3814 \begin_layout Plain Layout
3820 will have the following values:
3823 \begin_layout Standard
3824 \begin_inset Tabular
3825 <lyxtabular version="3" rows="7" columns="2">
3827 <column alignment="center" valignment="top" width="0">
3828 <column alignment="left" valignment="top" width="0">
3830 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3833 \begin_layout Plain Layout
3839 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3842 \begin_layout Plain Layout
3850 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3853 \begin_layout Plain Layout
3859 <cell alignment="left" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3862 \begin_layout Plain Layout
3870 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3873 \begin_layout Plain Layout
3879 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3882 \begin_layout Plain Layout
3883 =freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
3890 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3893 \begin_layout Plain Layout
3899 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3902 \begin_layout Plain Layout
3910 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3913 \begin_layout Plain Layout
3919 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3922 \begin_layout Plain Layout
3923 freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
3930 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3933 \begin_layout Plain Layout
3939 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3942 \begin_layout Plain Layout
3950 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3953 \begin_layout Plain Layout
3959 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3962 \begin_layout Plain Layout
3963 , foo={bar=baz, quux={quirk, a=fleeg}}, etc
3976 \begin_layout Standard
3978 \begin_inset Flex CharStyle:Code
3981 \begin_layout Plain Layout
3988 \begin_inset Flex CharStyle:Code
3991 \begin_layout Plain Layout
3997 and signify whether the option named in
3998 \begin_inset Flex CharStyle:Code
4001 \begin_layout Plain Layout
4007 has a value or not (respectively).
4010 \begin_layout Standard
4011 If the option does have a value, then if the expression
4012 \begin_inset Flex CharStyle:Code
4015 \begin_layout Plain Layout
4022 \begin_inset Flex CharStyle:Code
4025 \begin_layout Plain Layout
4031 it will signify that we need to recurse:
4034 \begin_layout Standard
4035 \begin_inset listings
4039 \begin_layout Plain Layout
4044 \begin_layout Plain Layout
4049 \begin_layout Plain Layout
4051 if (substr(a[4],1,1) == "{") {
4054 \begin_layout Plain Layout
4056 text = get_chunk_args(substr(a[4],2), values, path name SUBSEP);
4059 \begin_layout Plain Layout
4064 \begin_layout Plain Layout
4066 values[path name]=a[5];
4069 \begin_layout Plain Layout
4074 \begin_layout Plain Layout
4079 \begin_layout Plain Layout
4084 \begin_layout Plain Layout
4086 values[path name]="";
4089 \begin_layout Plain Layout
4094 \begin_layout Plain Layout
4104 \begin_layout Standard
4105 We can test this function like this:
4112 \begin_layout Standard
4113 \begin_inset listings
4117 \begin_layout Plain Layout
4121 chunkref{get_chunk_args()}>
4124 \begin_layout Plain Layout
4129 \begin_layout Plain Layout
4134 \begin_layout Plain Layout
4138 \begin_layout Plain Layout
4140 print get_chunk_args("name=freddie, foo={bar=baz, quux={quirk, a=fleeg}},
4144 \begin_layout Plain Layout
4149 \begin_layout Plain Layout
4151 print "a[" b "] => " a[b];
4154 \begin_layout Plain Layout
4159 \begin_layout Plain Layout
4169 \begin_layout Standard
4170 which should give this output:
4174 gca-test.awk-results
4177 \begin_layout Standard
4178 \begin_inset listings
4182 \begin_layout Plain Layout
4184 a[foo.quux.quirk] =>
4187 \begin_layout Plain Layout
4189 a[foo.quux.a] => fleeg
4192 \begin_layout Plain Layout
4197 \begin_layout Plain Layout
4202 \begin_layout Plain Layout
4212 \begin_layout Section
4213 Parsing chunk arguments
4216 \begin_layout Standard
4217 \begin_inset CommandInset label
4219 name "cha:Chunk Arguments"
4226 \begin_layout Standard
4227 Arguments to paramterized chunks are expressed in round brackets as a comma
4228 separated list of optional arguments.
4229 For example, a chunk that is defined with:
4232 \begin_layout LyX-Code
4235 Chunk{achunk, params=name ; address}
4238 \begin_layout Standard
4239 could be invoked as:
4242 \begin_layout LyX-Code
4245 chunkref{achunk}(John Jones, jones@example.com)
4248 \begin_layout Standard
4249 An argument list may be as simple as in
4250 \begin_inset Flex CharStyle:Code
4253 \begin_layout Plain Layout
4256 chunkref{pull}(thing, otherthing)
4264 \begin_layout LyX-Code
4267 chunkref{pull}(things[x, y], get_other_things(a, "(all)"))
4270 \begin_layout Standard
4271 --- which for all it's commas and quotes and parenthesis represents only
4273 \begin_inset Flex CharStyle:Code
4276 \begin_layout Plain Layout
4283 \begin_inset Flex CharStyle:Code
4286 \begin_layout Plain Layout
4287 get_other_things(a, "(all)")
4295 \begin_layout Standard
4296 If we simply split parameter list on commas, then the comma in
4297 \begin_inset Flex CharStyle:Code
4300 \begin_layout Plain Layout
4306 would split into two seperate arguments:
4307 \begin_inset Flex CharStyle:Code
4310 \begin_layout Plain Layout
4317 \begin_inset Flex CharStyle:Code
4320 \begin_layout Plain Layout
4326 --- neither of which make sense on their own.
4329 \begin_layout Standard
4330 One way to prevent this would be by refusing to split text between matching
4332 \begin_inset Flex CharStyle:Code
4335 \begin_layout Plain Layout
4342 \begin_inset Flex CharStyle:Code
4345 \begin_layout Plain Layout
4352 \begin_inset Flex CharStyle:Code
4355 \begin_layout Plain Layout
4362 \begin_inset Flex CharStyle:Code
4365 \begin_layout Plain Layout
4372 \begin_inset Flex CharStyle:Code
4375 \begin_layout Plain Layout
4382 \begin_inset Flex CharStyle:Code
4385 \begin_layout Plain Layout
4391 and most likely also
4392 \begin_inset Flex CharStyle:Code
4395 \begin_layout Plain Layout
4402 \begin_inset Flex CharStyle:Code
4405 \begin_layout Plain Layout
4412 \begin_inset Flex CharStyle:Code
4415 \begin_layout Plain Layout
4422 \begin_inset Flex CharStyle:Code
4425 \begin_layout Plain Layout
4432 Of course this also makes it impossible to pass such mis-matched code fragments
4433 as parameters, but I think that it would be hard for readers to cope with
4434 authors who would pass such code unbalanced fragments as chunk parameters
4438 \begin_layout Plain Layout
4439 I know that I couldn't cope with users doing such things, and although the
4440 GPL3 license prevents me from actually forbidding anyone from trying, if
4441 they want it to work they'll have to write the code themselves and not
4442 expect any support from me.
4450 \begin_layout Standard
4451 Unfortunately, the full set of matching delimiters may vary from language
4453 In certain C++ template contexts,
4454 \begin_inset Flex CharStyle:Code
4457 \begin_layout Plain Layout
4464 \begin_inset Flex CharStyle:Code
4467 \begin_layout Plain Layout
4473 would count as delimiters, and yet in other contexts they would not.
4476 \begin_layout Standard
4477 This puts me in the unfortunate position of having to parse-somewhat all
4478 programming languages without knowing what they are!
4481 \begin_layout Standard
4482 However, if this universal mode-tracking is possible, then parsing the arguments
4484 Such a mode tracker is described in chapter
4485 \begin_inset CommandInset ref
4487 reference "cha:modes"
4491 and used here with simplicity.
4498 \begin_layout Standard
4499 \begin_inset listings
4503 \begin_layout Plain Layout
4505 function parse_chunk_args(language, text, values, mode,
4508 \begin_layout Plain Layout
4513 \begin_layout Plain Layout
4518 \begin_layout Plain Layout
4523 \begin_layout Plain Layout
4527 chunkref{new-mode-tracker}(context, language, mode)>
4530 \begin_layout Plain Layout
4532 rest = mode_tracker(context, text, values);
4535 \begin_layout Plain Layout
4540 \begin_layout Plain Layout
4542 for(c=1; c <= context[0, "values"]; c++) {
4545 \begin_layout Plain Layout
4547 values[c] = context[0, "values", c];
4550 \begin_layout Plain Layout
4555 \begin_layout Plain Layout
4560 \begin_layout Plain Layout
4570 \begin_layout Section
4571 Expanding parameters in the text
4574 \begin_layout Standard
4575 Within the body of the chunk, the parameters are referred to with:
4576 \begin_inset Flex CharStyle:Code
4579 \begin_layout Plain Layout
4586 \begin_inset Flex CharStyle:Code
4589 \begin_layout Plain Layout
4596 There is a strong case that a LaTeX style notation should be used, like
4599 param{name} which would be expressed in the listing as =<
4601 param{name}> and be rendered as
4602 \begin_inset listings
4606 \begin_layout Plain Layout
4616 Such notation would make me go blind, but I do intend to adopt it
4620 \begin_layout Plain Layout
4621 Probably when LyX's listings inset can recognize and render it, so that
4622 I no longer have to type =<\SpecialChar \ldots{}
4631 \begin_layout Standard
4632 We therefore need a function
4633 \begin_inset Flex CharStyle:Code
4636 \begin_layout Plain Layout
4642 which will take a block of text, a list of permitted parameters, and the
4643 arguments which must substitute for the parameters.
4647 \begin_layout Standard
4648 \begin_inset CommandInset label
4650 name "Here-we-split"
4654 Here we split the text on
4655 \begin_inset Flex CharStyle:Code
4658 \begin_layout Plain Layout
4664 which means that all parts except the first will begin with a parameter
4665 name which will be terminated by
4666 \begin_inset Flex CharStyle:Code
4669 \begin_layout Plain Layout
4676 The split function will consume the literal
4677 \begin_inset Flex CharStyle:Code
4680 \begin_layout Plain Layout
4693 \begin_layout Standard
4694 \begin_inset listings
4698 \begin_layout Plain Layout
4700 function expand_chunk_args(text, params, args,
4703 \begin_layout Plain Layout
4705 p, text_array, next_text, v, t, l)
4708 \begin_layout Plain Layout
4713 \begin_layout Plain Layout
4715 if (split(text, text_array, "
4722 \begin_layout Plain Layout
4726 chunkref{substitute-chunk-args}>
4729 \begin_layout Plain Layout
4734 \begin_layout Plain Layout
4738 \begin_layout Plain Layout
4743 \begin_layout Plain Layout
4753 \begin_layout Standard
4754 First, we produce an associative array of substitution values indexed by
4756 This will serve as a cache, allowing us to look up the replacement values
4757 as we extract each name.
4761 substitute-chunk-args
4764 \begin_layout Standard
4765 \begin_inset listings
4769 \begin_layout Plain Layout
4774 \begin_layout Plain Layout
4776 v[params[p]]=args[p];
4779 \begin_layout Plain Layout
4789 \begin_layout Standard
4790 We accumulate substituted text in the variable
4791 \begin_inset Flex CharStyle:Code
4794 \begin_layout Plain Layout
4801 As the first part of the split function is the part before the delimiter
4803 \begin_inset Flex CharStyle:Code
4806 \begin_layout Plain Layout
4812 in our case --- this part will never contain a parameter reference, so
4813 we assign this directly to the result kept in
4814 \begin_inset Flex CharStyle:Code
4817 \begin_layout Plain Layout
4824 \begin_inset listings
4828 \begin_layout Plain Layout
4838 \begin_layout Standard
4839 We then iterate over the remaining values in the array
4843 \begin_layout Plain Layout
4844 I don't know why I think that it will enumerate the array in order, but
4851 \begin_inset Note Note
4854 \begin_layout Plain Layout
4855 So fix it or porve it
4860 , and substitute each reference for it's argument.
4863 \begin_layout Standard
4864 \begin_inset listings
4868 \begin_layout Plain Layout
4870 for(t=2; t in text_array; t++) {
4873 \begin_layout Plain Layout
4877 chunkref{substitute-chunk-arg}>
4880 \begin_layout Plain Layout
4890 \begin_layout Standard
4892 \begin_inset Flex CharStyle:Code
4895 \begin_layout Plain Layout
4901 a valid parameter reference will consist of valid parameter name terminated
4903 \begin_inset Flex CharStyle:Code
4906 \begin_layout Plain Layout
4913 A valid character name begins with the underscore or a letter, and may
4914 contain letters, digits or underscores.
4917 \begin_layout Standard
4918 A valid looking reference that is not actually the name of a parameter will
4919 be and not substituted.
4920 This is good because there is nothing to substitute anyway, and it avoids
4921 clashes when writing code for languages where ${\SpecialChar \ldots{}
4922 } is a valid construct
4923 --- such constructs will not be interfered with unless the parameter name
4928 substitute-chunk-arg
4931 \begin_layout Standard
4932 \begin_inset listings
4936 \begin_layout Plain Layout
4938 if (match(text_array[t], "^([a-zA-Z_][a-zA-Z0-9_]*)}", l) &&
4941 \begin_layout Plain Layout
4946 \begin_layout Plain Layout
4951 \begin_layout Plain Layout
4953 text = text v[l[1]] substr(text_array[t], length(l[1])+2);
4956 \begin_layout Plain Layout
4961 \begin_layout Plain Layout
4963 text = text "${" text_array[t];
4966 \begin_layout Plain Layout
4976 \begin_layout Chapter
4980 \begin_layout Standard
4981 \begin_inset CommandInset label
4990 \begin_layout Section
4994 \begin_layout Standard
4995 lstlistings and newfangle both recognize source languages, and perform some
4997 lstlistings can detect strings and comments within a language definition
4998 and perform suitable rendering, such as italics for comments, and visible-space
5002 \begin_layout Standard
5003 Newfangle similarly can recognize strings, and comments, etc, within a language,
5004 so that any chunks included with
5005 \begin_inset Flex CharStyle:Code
5008 \begin_layout Plain Layout
5016 can be suitably escape or quoted.
5019 \begin_layout Subsection
5020 Modes to keep code together
5023 \begin_layout Standard
5024 As an example, in the C language there are a few parse modes, affecting
5025 the interpretation of characters.
5028 \begin_layout Standard
5029 One parse mode is the strings mode.
5030 The string mode is commenced by an un-escaped quotation mark
5031 \begin_inset Flex CharStyle:Code
5034 \begin_layout Plain Layout
5040 and terminated by the same.
5041 Within the string mode, only one additional mode can be commenced, it is
5043 \begin_inset Flex CharStyle:Code
5046 \begin_layout Plain Layout
5054 , which is always terminated by the folloing character.
5057 \begin_layout Standard
5059 \begin_inset Flex CharStyle:Code
5062 \begin_layout Plain Layout
5068 which is terminated by a
5069 \begin_inset Flex CharStyle:Code
5072 \begin_layout Plain Layout
5078 (unless it occurs in a string).
5081 \begin_layout Standard
5082 Consider this line of C code:
5085 \begin_layout Standard
5086 \begin_inset Formula $\mathtt{things\underbrace{[\mathtt{x,\ y}]}_{1.\ [\ mode},\ get\_other\_things\underbrace{(\mathtt{a,\overbrace{"\mathtt{(all)}"}})}_{2.\ (\ mode}^{3.\ "\ mode}}$
5092 \begin_layout Standard
5093 Mode nesting prevents the close parenthesis in quote mode (part 3) from
5094 terminating the parenthesis mode (part 2).
5097 \begin_layout Standard
5098 Each language has a set of modes, the default mode being the null mode.
5099 Each mode can lead to other modes.
5102 \begin_layout Subsection
5103 Modes to included chunks
5106 \begin_layout Standard
5107 For instance, consider this chunk with
5108 \begin_inset Flex CharStyle:Code
5111 \begin_layout Plain Layout
5121 example-perl,language=perl
5124 \begin_layout Standard
5125 \begin_inset listings
5129 \begin_layout Plain Layout
5139 \begin_layout Standard
5140 If it were included in a chunk with
5141 \begin_inset Flex CharStyle:Code
5144 \begin_layout Plain Layout
5154 example-sh,language=sh
5157 \begin_layout Standard
5158 \begin_inset listings
5162 \begin_layout Plain Layout
5166 chunkref{example-perl}>"
5174 \begin_layout Standard
5175 newfangle would need to generate output like this if it were to work:
5178 \begin_layout LyX-Code
5186 \begin_layout Standard
5187 See that the double quote
5188 \begin_inset Flex CharStyle:Code
5191 \begin_layout Plain Layout
5198 \begin_inset Flex CharStyle:Code
5201 \begin_layout Plain Layout
5207 in the regex have been quoted with a back-slash to protect them from shell
5211 \begin_layout Standard
5212 If that were then included in a chunk with
5213 \begin_inset Flex CharStyle:Code
5216 \begin_layout Plain Layout
5226 example-makefile,language=make
5229 \begin_layout Standard
5230 \begin_inset listings
5234 \begin_layout Plain Layout
5239 \begin_layout Plain Layout
5243 chunkref{example-sh}>
5251 \begin_layout Standard
5252 We would need the output to look like this --- note the $$:
5255 \begin_layout LyX-Code
5259 \begin_layout LyX-Code
5267 \begin_layout Standard
5268 In order to make this work, we need to define a mode-tracker supporting
5269 each language, that can detect the various quoting modes, and provide a
5270 transformation that must be applied to any included text so that included
5271 text will be interpreted correctly after any interpolation that it may
5272 be subject to at run-time.
5275 \begin_layout Standard
5276 For example, the sed transformation for text to be inserted into sh double-quote
5277 d strings would be something like:
5280 \begin_layout LyX-Code
5304 \begin_layout Standard
5306 \begin_inset Flex CharStyle:Code
5309 \begin_layout Plain Layout
5320 \begin_layout Standard
5321 \begin_inset Note Note
5324 \begin_layout Plain Layout
5325 I don't think this example is true
5330 The mode tracker must also track nested mode-changes, as in this
5331 \begin_inset Flex CharStyle:Code
5334 \begin_layout Plain Layout
5343 \begin_layout LyX-Code
5344 echo "hello `id \SpecialChar \ldots{}
5348 \begin_layout LyX-Code
5352 \begin_layout Plain Layout
5358 hphantom{echo "hello `id}
5366 \begin_layout Standard
5367 Any characters inserted at the point marked ↑ would need to be escaped,
5369 \begin_inset Flex CharStyle:Code
5372 \begin_layout Plain Layout
5379 First it would need escaping for the back-ticks `, and then for the double-quot
5383 \begin_layout Standard
5384 Escaping need not occur if the format and mode of the included chunk matches
5385 that of the including chunk.
5388 \begin_layout Standard
5389 As each chunk is output a new mode tracker for that language is initialized
5390 in it's normal state.
5391 As text is output for that chunk the output mode is tracked.
5392 When a new chunk is included, a transformation appropriate to that mode
5393 is selected and pushed onto a stack of transformations.
5394 Any text to be output is first passed through this stack of transformations.
5397 \begin_layout Standard
5398 It remains to consider if the chunk-include function should return it's
5399 generated text so that the caller can apply any transformations (and formatting
5400 ), or if it should apply the stack of transformations itself.
5403 \begin_layout Standard
5404 Note that the transformed text should have the property of not being able
5405 to change the mode in the current chunk.
5408 \begin_layout Standard
5409 \begin_inset Note Note
5412 \begin_layout Plain Layout
5413 Note chunk parameters should probably also be transformed
5421 \begin_layout Section
5422 Language Mode Definitions
5425 \begin_layout Standard
5426 All modes are stored in a single multi-dimensional hash.
5427 The first index is the language, and the second index is the mode-identifier.
5428 The third indexes are terminators, and optionally, submodes, and delimiters.
5431 \begin_layout Standard
5432 A useful set of mode definitions for a nameless general C-type language
5434 (Don't be confused by the double backslash escaping needed in awk.
5435 One set of escaping is for the string, and the second set of escaping is
5438 \begin_inset Note Note
5441 \begin_layout Plain Layout
5444 mode{}> command which will allow us to signify that a string is regex and
5445 thus newfangle will quote it for us.
5453 \begin_layout Standard
5454 Submodes are entered by the characters
5455 \begin_inset Flex CharStyle:Code
5458 \begin_layout Plain Layout
5467 \begin_inset Flex CharStyle:Code
5470 \begin_layout Plain Layout
5477 \begin_inset Flex CharStyle:Code
5480 \begin_layout Plain Layout
5487 \begin_inset Flex CharStyle:Code
5490 \begin_layout Plain Layout
5497 \begin_inset Flex CharStyle:Code
5500 \begin_layout Plain Layout
5507 \begin_inset Flex CharStyle:Code
5510 \begin_layout Plain Layout
5517 \begin_inset Flex CharStyle:Code
5520 \begin_layout Plain Layout
5530 common-mode-definitions,params=language
5533 \begin_layout Standard
5534 \begin_inset listings
5538 \begin_layout Plain Layout
5540 modes[${language}, "", "submodes" ]="
5566 \begin_layout Standard
5567 In the default mode, a comma surrounded by un-important white space is a
5568 delimiter of language items.
5571 \begin_layout Standard
5572 \begin_inset listings
5576 \begin_layout Plain Layout
5578 modes[${language}, "", "delimiters"]=" *, *";
5586 \begin_layout Standard
5587 and should pass this test:
5590 \begin_layout Standard
5591 \begin_inset Note Note
5594 \begin_layout Plain Layout
5595 Why do the tests run in
5596 \begin_inset Quotes eld
5600 \begin_inset Quotes erd
5604 \begin_inset Quotes eld
5608 \begin_inset Quotes erd
5620 test:mode-definitions
5623 \begin_layout Standard
5624 \begin_inset listings
5628 \begin_layout Plain Layout
5630 parse_chunk_args("", "1,2,3", a, "");
5633 \begin_layout Plain Layout
5635 if (a[1] != "1") e++;
5638 \begin_layout Plain Layout
5640 if (a[2] != "2") e++;
5643 \begin_layout Plain Layout
5645 if (a[3] != "3") e++;
5648 \begin_layout Plain Layout
5650 if (length(a) != 3) e++;
5653 \begin_layout Plain Layout
5657 chunkref{pca-test.awk:summary}>
5660 \begin_layout Plain Layout
5664 \begin_layout Plain Layout
5666 parse_chunk_args("", "joe, red", a, "");
5669 \begin_layout Plain Layout
5671 if (a[1] != "joe") e++;
5674 \begin_layout Plain Layout
5676 if (a[2] != "red") e++;
5679 \begin_layout Plain Layout
5681 if (length(a) != 2) e++;
5684 \begin_layout Plain Layout
5688 chunkref{pca-test.awk:summary}>
5691 \begin_layout Plain Layout
5695 \begin_layout Plain Layout
5697 parse_chunk_args("", "${colour}", a, "");
5700 \begin_layout Plain Layout
5702 if (a[1] != "${colour}") e++;
5705 \begin_layout Plain Layout
5707 if (length(a) != 1) e++;
5710 \begin_layout Plain Layout
5714 chunkref{pca-test.awk:summary}>
5722 \begin_layout Standard
5723 Nested modes are identified by a backslash, a double or single quote, various
5724 bracket styles or a /* comment.
5727 \begin_layout Standard
5728 For each of these sub-modes modes we must also identify at a mode terminator,
5729 and any sub-modes or delimiters that may be entered
5733 \begin_layout Plain Layout
5734 Because we are using the sub-mode characters as the mode identifier it means
5735 we can't currently have a mode character dependant on it's context; i.e.
5737 \begin_inset Flex CharStyle:Code
5740 \begin_layout Plain Layout
5746 can't behave differently when it is inside
5747 \begin_inset Flex CharStyle:Code
5750 \begin_layout Plain Layout
5764 \begin_layout Subsection
5768 \begin_layout Standard
5769 The backslash mode has no submodes or delimiters, and is terminated by any
5771 Note that we are not so much interested in evaluating or interpolating
5772 content as we are in delineating content.
5773 It is no matter that a double backslash (
5774 \begin_inset Flex CharStyle:Code
5777 \begin_layout Plain Layout
5787 ) may represent a single backslash while a backslash-newline may represent
5788 white space, but it does matter that the newline in a backslash newline
5789 should not be able to terminate a C pre-processor statement; and so the
5790 newline will be consumed by the backslash however it is to be interpreted.
5794 common-mode-definitions
5797 \begin_layout Standard
5798 \begin_inset listings
5802 \begin_layout Plain Layout
5804 modes[${language}, "
5808 ", "terminators"]=".";
5816 \begin_layout Subsection
5820 \begin_layout Standard
5821 Common languages support two kinds of strings quoting, double quotes and
5826 mode:common-string,params=language;quote
5829 \begin_layout Standard
5830 In a string we have one special mode, which is the backslash.
5831 This may escape an embedded quote and prevent us thinking that it should
5832 terminate the string.
5836 \begin_layout Standard
5837 \begin_inset listings
5841 \begin_layout Plain Layout
5843 modes[${language}, ${quote}, "submodes" ]="
5859 \begin_layout Standard
5860 Otherwise, the string will be terminated by the same character that commenced
5864 \begin_layout Standard
5865 \begin_inset listings
5869 \begin_layout Plain Layout
5871 modes[${language}, ${quote}, "terminators"]=${quote};
5879 \begin_layout Standard
5880 In C type languages, certain escape sequences exist in strings.
5881 We need to define mechanism to enclode any chunks included in this mode
5882 using those escape sequences.
5883 These are expressed in two parts, s meaning search, and r meaning replace.
5886 \begin_layout Standard
5887 The first substitution is to replace a backslash with a double backslash.
5888 We do this first as other substitutions may introduce a backslash which
5889 we would not then want to escape again here.
5892 \begin_layout Standard
5893 \begin_inset listings
5897 \begin_layout Plain Layout
5899 escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]="
5906 \begin_layout Plain Layout
5908 escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="
5924 \begin_layout Standard
5925 If the quote character occurs in the text, it should be preceded by a backslash,
5926 otherwise it would terminate the string unexpectedly.
5929 \begin_layout Standard
5930 \begin_inset listings
5934 \begin_layout Plain Layout
5936 escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]=${quote};
5939 \begin_layout Plain Layout
5941 escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="
5953 \begin_layout Standard
5954 Any newlines in the string, must be replaced by
5955 \begin_inset Flex CharStyle:Code
5958 \begin_layout Plain Layout
5969 \begin_layout Standard
5970 \begin_inset listings
5974 \begin_layout Plain Layout
5976 escapes[${language}, ${quote}, ++escapes[${language}, ${quote}], "s"]="
5981 \begin_layout Plain Layout
5983 escapes[${language}, ${quote}, escapes[${language}, ${quote}], "r"]="
5995 \begin_layout Standard
5996 For the common modes, we define this string handling for double and single
6001 common-mode-definitions,params=language
6004 \begin_layout Standard
6005 \begin_inset listings
6009 \begin_layout Plain Layout
6013 chunkref{mode:common-string}(${language}, "
6018 \begin_layout Plain Layout
6022 chunkref{mode:common-string}(${language}, "'")>
6030 \begin_layout Standard
6031 Working strings should pass this test:
6035 test:mode-definitions
6038 \begin_layout Standard
6039 \begin_inset listings
6043 \begin_layout Plain Layout
6045 parse_chunk_args("", "say
6064 \begin_layout Plain Layout
6085 \begin_layout Plain Layout
6087 if (a[2] != "for me") e++;
6090 \begin_layout Plain Layout
6092 if (length(a) != 2) e++;
6095 \begin_layout Plain Layout
6099 chunkref{pca-test.awk:summary}>
6107 \begin_layout Subsection
6108 Parentheses, Braces and Brackets
6111 \begin_layout Standard
6112 Where quotes are closed by the same character, parentheses, brackets and
6113 braces are closed by an alternate character.
6117 mode:common-brackets,params=language;open;close
6120 \begin_layout Standard
6121 \begin_inset listings
6125 \begin_layout Plain Layout
6127 modes[${language}, ${open}, "submodes" ]="
6152 \begin_layout Plain Layout
6154 modes[${language}, ${open}, "delimiters"]=" *, *";
6157 \begin_layout Plain Layout
6159 modes[${language}, ${open}, "terminators"]=${close};
6167 \begin_layout Standard
6168 Note that the open is NOT a regex but the close token IS.
6170 \begin_inset Note Note
6173 \begin_layout Plain Layout
6174 WHen we can quote regex we won't have to put the slashes in here
6183 common-mode-definitions,params=language
6186 \begin_layout Standard
6187 \begin_inset listings
6191 \begin_layout Plain Layout
6195 chunkref{mode:common-brackets}(${language}, "{", "}")>
6198 \begin_layout Plain Layout
6202 chunkref{mode:common-brackets}(${language}, "[", "
6209 \begin_layout Plain Layout
6213 chunkref{mode:common-brackets}(${language}, "(", "
6225 \begin_layout Subsection
6226 Customizing Standard Modes
6230 mode:add-submode,params=language;mode;submode
6233 \begin_layout Standard
6234 \begin_inset listings
6238 \begin_layout Plain Layout
6240 modes[${language}, ${mode}, "submodes"] = modes[${language}, ${mode}, "submodes"
6250 mode:add-escapes,params=language;mode;search;replace
6253 \begin_layout Standard
6254 \begin_inset listings
6258 \begin_layout Plain Layout
6260 escapes[${language}, ${mode}, ++escapes[${language}, ${mode}], "s"]=${search};
6263 \begin_layout Plain Layout
6265 escapes[${language}, ${mode}, escapes[${language}, ${mode}], "r"]=${replace};
6273 \begin_layout Subsection
6277 \begin_layout Standard
6279 \begin_inset Flex CharStyle:Code
6282 \begin_layout Plain Layout
6289 \begin_inset Flex CharStyle:Code
6292 \begin_layout Plain Layout
6298 style comments to be added to any language:
6302 mode:multi-line-comments,params=language
6305 \begin_layout Standard
6306 \begin_inset listings
6310 \begin_layout Plain Layout
6314 chunkref{mode:add-submode}(${language}, "", "/
6321 \begin_layout Plain Layout
6323 modes[${language}, "/*", "terminators"]="
6336 mode:single-line-slash-comments,params=language
6339 \begin_layout Standard
6340 \begin_inset listings
6344 \begin_layout Plain Layout
6348 chunkref{mode:add-submode}(${language}, "", "//")>
6351 \begin_layout Plain Layout
6353 modes[${language}, "//", "terminators"]="
6358 \begin_layout Plain Layout
6362 chunkref{mode:add-escapes}(${language}, "//", "
6364 textbackslash{}n", "
6366 textbackslash{}n//")>
6374 \begin_layout Standard
6376 \begin_inset Flex CharStyle:Code
6379 \begin_layout Plain Layout
6385 style comments (as used in awk and shell scripts) in a similar manner.
6389 mode:add-hash-comments,params=language
6390 \begin_inset Note Note
6393 \begin_layout Plain Layout
6400 and have hacky work-arounds in the parser for now
6408 \begin_layout Standard
6409 \begin_inset listings
6413 \begin_layout Plain Layout
6417 chunkref{mode:add-submode}(${language}, "", "
6422 \begin_layout Plain Layout
6424 modes[${language}, "#", "terminators"]="
6429 \begin_layout Plain Layout
6433 chunkref{mode:add-escapes}(${language}, "
6437 textbackslash{}n", "
6449 \begin_layout Standard
6451 \begin_inset Flex CharStyle:Code
6454 \begin_layout Plain Layout
6460 denotes pre-processor directives which can be multi-line
6464 mode:add-hash-defines,params=language
6467 \begin_layout Standard
6468 \begin_inset listings
6472 \begin_layout Plain Layout
6476 chunkref{mode:add-submode}(${language}, "", "
6481 \begin_layout Plain Layout
6483 modes[${language}, "#", "submodes" ]="
6494 \begin_layout Plain Layout
6496 modes[${language}, "#", "terminators"]="
6501 \begin_layout Plain Layout
6505 chunkref{mode:add-escapes}(${language}, "
6509 textbackslash{}n", "
6527 \begin_layout Standard
6528 We can add these definitions to various languages
6535 \begin_layout Standard
6536 \begin_inset listings
6540 \begin_layout Plain Layout
6544 chunkref{common-mode-definitions}("")>
6547 \begin_layout Plain Layout
6551 \begin_layout Plain Layout
6555 chunkref{common-mode-definitions}("c")>
6558 \begin_layout Plain Layout
6562 chunkref{mode:multi-line-comments}("c")>
6565 \begin_layout Plain Layout
6569 chunkref{mode:single-line-slash-comments}("c")>
6572 \begin_layout Plain Layout
6576 chunkref{mode:add-hash-defines}("c")>
6579 \begin_layout Plain Layout
6583 \begin_layout Plain Layout
6587 chunkref{common-mode-definitions}("awk")>
6590 \begin_layout Plain Layout
6594 chunkref{mode:add-hash-comments}("awk")>
6602 \begin_layout Standard
6603 The awk definitions should allow a comment block like this:
6607 test:comment-quote,language=awk
6610 \begin_layout Standard
6611 \begin_inset listings
6615 \begin_layout Plain Layout
6619 chunkref{test:comment-text}>
6628 test:comment-text,language=
6631 \begin_layout Standard
6632 \begin_inset listings
6636 \begin_layout Plain Layout
6641 \begin_layout Plain Layout
6643 the quick brown fox to bring lemonade
6646 \begin_layout Plain Layout
6656 \begin_layout Standard
6657 to come out like this:
6661 test:comment-quote:result
6664 \begin_layout Standard
6665 \begin_inset listings
6669 \begin_layout Plain Layout
6671 # Comment: Now is the time for
6674 \begin_layout Plain Layout
6676 #the quick brown fox to bring lemonade
6679 \begin_layout Plain Layout
6689 \begin_layout Standard
6690 The C definition for such a block should have it come out like this:
6694 test:comment-quote:C-result
6697 \begin_layout Standard
6698 \begin_inset listings
6702 \begin_layout Plain Layout
6704 # Comment: Now is the time for
6709 \begin_layout Plain Layout
6711 the quick brown fox to bring lemonade
6716 \begin_layout Plain Layout
6726 \begin_layout Section
6730 \begin_layout Standard
6731 Also, the parser must return any spare text at the end that has not been
6732 processed due to a mode terminator being found.
6736 test:mode-definitions
6739 \begin_layout Standard
6740 \begin_inset listings
6744 \begin_layout Plain Layout
6746 rest = parse_chunk_args("", "1, 2, 3) spare", a, "(");
6749 \begin_layout Plain Layout
6754 \begin_layout Plain Layout
6759 \begin_layout Plain Layout
6764 \begin_layout Plain Layout
6766 if (length(a) != 3) e++;
6769 \begin_layout Plain Layout
6771 if (rest != " spare") e++;
6774 \begin_layout Plain Layout
6778 chunkref{pca-test.awk:summary}>
6786 \begin_layout Standard
6787 We must also be able to parse the example given earlier.
6791 test:mode-definitions
6794 \begin_layout Standard
6795 \begin_inset listings
6799 \begin_layout Plain Layout
6801 parse_chunk_args("", "things[x, y], get_other_things(a,
6808 \begin_layout Plain Layout
6810 if (a[1] != "things[x, y]") e++;
6813 \begin_layout Plain Layout
6815 if (a[2] != "get_other_things(a,
6822 \begin_layout Plain Layout
6824 if (a[3] != "99") e++;
6827 \begin_layout Plain Layout
6829 if (length(a) != 3) e++;
6832 \begin_layout Plain Layout
6836 chunkref{pca-test.awk:summary}>
6844 \begin_layout Section
6845 A non-recursive mode tracker
6848 \begin_layout Subsection
6852 \begin_layout Standard
6853 The mode tracker holds its state in a stack based on a hash.
6854 This function, when passed an empty hash will intialize it.
6861 \begin_layout Standard
6862 \begin_inset listings
6866 \begin_layout Plain Layout
6868 function new_mode_tracker(context, language, mode) {
6871 \begin_layout Plain Layout
6876 \begin_layout Plain Layout
6878 context[0, "language"] = language;
6881 \begin_layout Plain Layout
6883 context[0, "mode"] = mode;
6886 \begin_layout Plain Layout
6896 \begin_layout Standard
6897 Because awk functions cannot return an array, we must create the array first
6898 and pass it in, so we have a newfangle macro to do this:
6902 new-mode-tracker,params=context;language;mode
6905 \begin_layout Standard
6906 \begin_inset listings
6910 \begin_layout Plain Layout
6914 chunkref{awk-delete-array}(${context})>
6917 \begin_layout Plain Layout
6919 new_mode_tracker(${context}, ${language}, ${mode});
6927 \begin_layout Subsection
6931 \begin_layout Standard
6932 And for tracking modes, we dispatch to a mode-tracker action based on the
6940 \begin_layout Standard
6941 \begin_inset listings
6945 \begin_layout Plain Layout
6947 function push_mode_tracker(context, language, mode,
6950 \begin_layout Plain Layout
6955 \begin_layout Plain Layout
6960 \begin_layout Plain Layout
6965 \begin_layout Plain Layout
6967 if (! ("" in context)) {
6970 \begin_layout Plain Layout
6974 chunkref{new-mode-tracker}(context, language, mode)>
6977 \begin_layout Plain Layout
6982 \begin_layout Plain Layout
6987 \begin_layout Plain Layout
6989 if (context[top, "language"] == language && mode=="") mode = context[top,
6993 \begin_layout Plain Layout
6998 \begin_layout Plain Layout
7000 context[top, "language"] = language;
7003 \begin_layout Plain Layout
7005 context[top, "mode"] = mode;
7008 \begin_layout Plain Layout
7013 \begin_layout Plain Layout
7018 \begin_layout Plain Layout
7028 \begin_layout Standard
7029 \begin_inset listings
7033 \begin_layout Plain Layout
7035 function finalize_mode_tracker(context,
7038 \begin_layout Plain Layout
7043 \begin_layout Plain Layout
7048 \begin_layout Plain Layout
7053 \begin_layout Plain Layout
7055 if ( ("" in context) && context[""] != 0) return 0;
7058 \begin_layout Plain Layout
7063 \begin_layout Plain Layout
7073 \begin_layout Standard
7074 This implies that any chunk must be syntactically whole; for instance, this
7082 \begin_layout Standard
7083 \begin_inset listings
7087 \begin_layout Plain Layout
7092 \begin_layout Plain Layout
7096 chunkref{test:say-hello}>
7099 \begin_layout Plain Layout
7113 \begin_layout Standard
7114 \begin_inset listings
7118 \begin_layout Plain Layout
7128 \begin_layout Standard
7129 But this is not fine; the chunk
7130 \begin_inset Flex Chunkref
7133 \begin_layout Plain Layout
7139 is not properly cromulent.
7146 \begin_layout Standard
7147 \begin_inset listings
7151 \begin_layout Plain Layout
7156 \begin_layout Plain Layout
7160 chunkref{test:hidden-else}>
7163 \begin_layout Plain Layout
7177 \begin_layout Standard
7178 \begin_inset listings
7182 \begin_layout Plain Layout
7187 \begin_layout Plain Layout
7192 \begin_layout Plain Layout
7202 \begin_layout Standard
7203 These tests will check for correct behaviour:
7210 \begin_layout Standard
7211 \begin_inset listings
7215 \begin_layout Plain Layout
7217 echo Cromulence test
7220 \begin_layout Plain Layout
7222 passtest ./newfangle -Rtest:whole-chunk $TEX_SRC &>/dev/null || ( echo "Whole
7223 chunk failed" && exit 1 )
7226 \begin_layout Plain Layout
7228 failtest ./newfangle -Rtest:partial-chunk $TEX_SRC &>/dev/null || ( echo
7229 "Partial chunk failed" && exit 1 )
7237 \begin_layout Subsection
7241 \begin_layout Standard
7242 We must avoid recursion as a language construct because we intend to employ
7243 mode-tracking to track language mode of emitted code, and the code is emitted
7244 from a function which is itself recursive, so instead we implement psuedo-recur
7245 sion using our own stack based on a hash.
7252 \begin_layout Standard
7253 \begin_inset listings
7257 \begin_layout Plain Layout
7259 function mode_tracker(context, text, values,
7262 \begin_layout Plain Layout
7264 # optional parameters
7267 \begin_layout Plain Layout
7272 \begin_layout Plain Layout
7274 mode, submodes, language,
7277 \begin_layout Plain Layout
7279 cindex, c, a, part, item, name, result, new_values, new_mode,
7282 \begin_layout Plain Layout
7284 delimiters, terminators)
7287 \begin_layout Plain Layout
7297 \begin_layout Standard
7298 We could be re-commencing with a valid context, so we need to setup the
7299 state according to the last context.
7302 \begin_layout Standard
7303 \begin_inset listings
7307 \begin_layout Plain Layout
7309 cindex = context[""] + 0;
7312 \begin_layout Plain Layout
7314 mode = context[cindex, "mode"];
7317 \begin_layout Plain Layout
7319 language = context[cindex, "language" ];
7327 \begin_layout Standard
7328 First we construct a single large regex combining the possible sub-modes
7329 for the current mode along with the terminators for the current mode.
7333 parse_chunk_args-reset-modes
7336 \begin_layout Standard
7337 \begin_inset listings
7341 \begin_layout Plain Layout
7343 submodes=modes[language, mode, "submodes"];
7346 \begin_layout Plain Layout
7350 \begin_layout Plain Layout
7352 if ((language, mode, "delimiters") in modes) {
7355 \begin_layout Plain Layout
7357 delimiters = modes[language, mode, "delimiters"];
7360 \begin_layout Plain Layout
7362 if (length(submodes)>0) submodes = submodes "|";
7365 \begin_layout Plain Layout
7367 submodes=submodes delimiters;
7370 \begin_layout Plain Layout
7372 } else delimiters="";
7375 \begin_layout Plain Layout
7377 if ((language, mode, "terminators") in modes) {
7380 \begin_layout Plain Layout
7382 terminators = modes[language, mode, "terminators"];
7385 \begin_layout Plain Layout
7387 if (length(submodes)>0) submodes = submodes "|";
7390 \begin_layout Plain Layout
7392 submodes=submodes terminators;
7395 \begin_layout Plain Layout
7397 } else terminators="";
7405 \begin_layout Standard
7406 If we don't find anything to match on --- probably because the language
7407 is not supported --- then we return the entire text without matching anything.
7410 \begin_layout Standard
7411 \begin_inset listings
7415 \begin_layout Plain Layout
7417 if (! length(submodes)) return text;
7429 \begin_layout Standard
7430 \begin_inset listings
7434 \begin_layout Plain Layout
7438 chunkref{parse_chunk_args-reset-modes}>
7446 \begin_layout Standard
7447 We then iterate the text (until there is none left) looking for sub-modes
7448 or terminators in the regex.
7451 \begin_layout Standard
7452 \begin_inset listings
7456 \begin_layout Plain Layout
7458 while((cindex >= 0) && length(text)) {
7461 \begin_layout Plain Layout
7463 if (match(text, "(" submodes ")", a)) {
7471 \begin_layout Standard
7472 A bug that creeps in regularly during development is bad regexes of zero
7473 length which result in an infinite loop (as no text is consumed), so I
7474 catch that right away with this test.
7477 \begin_layout Standard
7478 \begin_inset listings
7482 \begin_layout Plain Layout
7487 \begin_layout Plain Layout
7489 error(sprintf("Internal error, matched zero length submode, should
7490 be impossible - likely regex computation error
7497 \begin_layout Plain Layout
7505 n", language, mode, submodes));
7508 \begin_layout Plain Layout
7518 \begin_layout Standard
7519 \begin_inset Flex CharStyle:Code
7522 \begin_layout Plain Layout
7528 is defined as the text up to the sub-mode or terminator, and this is appended
7530 \begin_inset Flex CharStyle:Code
7533 \begin_layout Plain Layout
7539 --- which is the current text being gathered.
7540 If a mode has a delimiter, then item is reset each time a delimiter is
7544 \begin_layout Standard
7545 \begin_inset Formula $\mathtt{\overbrace{"\overbrace{hello}^{item},\ \overbrace{there}^{item}"}^{item},\ \overbrace{he\ said.}^{item}}$
7551 \begin_layout Standard
7552 \begin_inset listings
7556 \begin_layout Plain Layout
7558 part = substr(text, 1, RSTART -1);
7561 \begin_layout Plain Layout
7571 \begin_layout Standard
7572 We must now determine what was matched.
7573 If it was a terminator, then we must restore the previous mode.
7576 \begin_layout Standard
7577 \begin_inset listings
7581 \begin_layout Plain Layout
7583 if (match(a[1], "^" terminators "$")) {
7586 \begin_layout Plain Layout
7588 #printf("%2d EXIT MODE [%s] by [%s] [%s]
7590 n", cindex, mode, a[1], text) > "/dev/stderr"
7593 \begin_layout Plain Layout
7595 context[cindex, "values", ++context[cindex, "values"]] = item;
7598 \begin_layout Plain Layout
7600 delete context[cindex];
7603 \begin_layout Plain Layout
7605 context[""] = --cindex;
7608 \begin_layout Plain Layout
7613 \begin_layout Plain Layout
7615 mode = context[cindex, "mode"];
7618 \begin_layout Plain Layout
7620 language = context[cindex, "language"];
7623 \begin_layout Plain Layout
7627 chunkref{parse_chunk_args-reset-modes}>
7630 \begin_layout Plain Layout
7635 \begin_layout Plain Layout
7640 \begin_layout Plain Layout
7642 text = substr(text, 1 + length(part) + length(a[1]));
7645 \begin_layout Plain Layout
7655 \begin_layout Standard
7656 If a delimiter was matched, then we must store the current item in the parsed
7657 values array, and reset the item.
7660 \begin_layout Standard
7661 \begin_inset listings
7665 \begin_layout Plain Layout
7667 else if (match(a[1], "^" delimiters "$")) {
7670 \begin_layout Plain Layout
7675 \begin_layout Plain Layout
7677 context[cindex, "values", ++context[cindex, "values"]] = item;
7680 \begin_layout Plain Layout
7685 \begin_layout Plain Layout
7690 \begin_layout Plain Layout
7695 \begin_layout Plain Layout
7700 \begin_layout Plain Layout
7702 text = substr(text, 1 + length(part) + length(a[1]));
7705 \begin_layout Plain Layout
7715 \begin_layout Standard
7716 otherwise, if a new submode is detected (all submodes have terminators),
7717 we must create a nested parse context until we find the terminator for
7721 \begin_layout Standard
7722 \begin_inset listings
7726 \begin_layout Plain Layout
7728 else if ((language, a[1], "terminators") in modes) {
7731 \begin_layout Plain Layout
7733 #check if new_mode is defined
7736 \begin_layout Plain Layout
7741 \begin_layout Plain Layout
7743 #printf("%2d ENTER MODE [%s] in [%s]
7745 n", cindex, a[1], text) > "/dev/stderr"
7748 \begin_layout Plain Layout
7750 text = substr(text, 1 + length(part) + length(a[1]));
7753 \begin_layout Plain Layout
7755 context[""] = ++cindex;
7758 \begin_layout Plain Layout
7760 context[cindex, "mode"] = a[1];
7763 \begin_layout Plain Layout
7765 context[cindex, "language"] = language;
7768 \begin_layout Plain Layout
7773 \begin_layout Plain Layout
7777 chunkref{parse_chunk_args-reset-modes}>
7780 \begin_layout Plain Layout
7785 \begin_layout Plain Layout
7787 error(sprintf("Submode '%s' set unknown mode in text: %s
7789 nLanguage %s Mode %s
7791 n", a[1], text, language, mode));
7794 \begin_layout Plain Layout
7796 text = substr(text, 1 + length(part) + length(a[1]));
7799 \begin_layout Plain Layout
7804 \begin_layout Plain Layout
7814 \begin_layout Standard
7815 In the final case, we parsed to the end of the string.
7816 If the string was entire, then we should have no nested mode context, but
7817 if the string was just a fragment we may have a mode context which must
7818 be preserved for the next fragment.
7819 Todo: Consideration ought to be given if sub-mode strings are split over
7821 \begin_inset Note Note
7824 \begin_layout Plain Layout
7825 Consideration ought to be given if sub-mode strings are split over two fragments.
7833 \begin_layout Standard
7834 \begin_inset listings
7838 \begin_layout Plain Layout
7843 \begin_layout Plain Layout
7845 context[cindex, "values", ++context[cindex, "values"]] = item text;
7848 \begin_layout Plain Layout
7853 \begin_layout Plain Layout
7858 \begin_layout Plain Layout
7863 \begin_layout Plain Layout
7868 \begin_layout Plain Layout
7872 \begin_layout Plain Layout
7874 context["item"] = item;
7877 \begin_layout Plain Layout
7881 \begin_layout Plain Layout
7883 if (length(item)) context[cindex, "values", ++context[cindex, "values"]]
7887 \begin_layout Plain Layout
7892 \begin_layout Plain Layout
7902 \begin_layout Subsubsection
7906 \begin_layout Standard
7907 All the mode tracker chunks are referred to here:
7914 \begin_layout Standard
7915 \begin_inset listings
7919 \begin_layout Plain Layout
7923 chunkref{new_mode_tracker()}>
7926 \begin_layout Plain Layout
7930 chunkref{mode_tracker()}>
7938 \begin_layout Subsubsection
7942 \begin_layout Standard
7943 We can test this function like this:
7950 \begin_layout Standard
7951 \begin_inset listings
7955 \begin_layout Plain Layout
7962 \begin_layout Plain Layout
7966 chunkref{mode-tracker}>
7969 \begin_layout Plain Layout
7973 chunkref{parse_chunk_args()}>
7976 \begin_layout Plain Layout
7981 \begin_layout Plain Layout
7986 \begin_layout Plain Layout
7990 chunkref{mode-definitions}>
7993 \begin_layout Plain Layout
7997 \begin_layout Plain Layout
8001 chunkref{test:mode-definitions}>
8004 \begin_layout Plain Layout
8015 pca-test.awk:summary
8018 \begin_layout Standard
8019 \begin_inset listings
8023 \begin_layout Plain Layout
8028 \begin_layout Plain Layout
8033 \begin_layout Plain Layout
8038 \begin_layout Plain Layout
8040 print "a[" b "] => " a[b];
8043 \begin_layout Plain Layout
8048 \begin_layout Plain Layout
8053 \begin_layout Plain Layout
8058 \begin_layout Plain Layout
8063 \begin_layout Plain Layout
8068 \begin_layout Plain Layout
8078 \begin_layout Standard
8079 which should give this output:
8083 pca-test.awk-results
8086 \begin_layout Standard
8087 \begin_inset listings
8091 \begin_layout Plain Layout
8093 a[foo.quux.quirk] =>
8096 \begin_layout Plain Layout
8098 a[foo.quux.a] => fleeg
8101 \begin_layout Plain Layout
8106 \begin_layout Plain Layout
8111 \begin_layout Plain Layout
8121 \begin_layout Section
8122 Escaping and Quoting
8125 \begin_layout Standard
8126 Each nested mode can optionally define a set of transforms to be applied
8127 to any text that is included from another language.
8130 \begin_layout Standard
8131 This code can perform transforms
8138 \begin_layout Standard
8139 \begin_inset listings
8143 \begin_layout Plain Layout
8145 function transform_escape(s, r, text,
8148 \begin_layout Plain Layout
8153 \begin_layout Plain Layout
8158 \begin_layout Plain Layout
8163 \begin_layout Plain Layout
8168 \begin_layout Plain Layout
8173 \begin_layout Plain Layout
8175 for(c=1; c <= max && (c in s); c++) {
8178 \begin_layout Plain Layout
8180 gsub(s[c], r[c], text);
8183 \begin_layout Plain Layout
8188 \begin_layout Plain Layout
8193 \begin_layout Plain Layout
8203 \begin_layout Standard
8204 This function must append from index
8205 \begin_inset Flex CharStyle:Code
8208 \begin_layout Plain Layout
8214 onwards, and escape transforms from the supplied context, and return c
8215 + number of new transforms.
8218 \begin_layout Standard
8219 \begin_inset listings
8223 \begin_layout Plain Layout
8225 function mode_escaper(context, s, r, src,
8228 \begin_layout Plain Layout
8233 \begin_layout Plain Layout
8238 \begin_layout Plain Layout
8240 for(c = context[""]; c >= 0; c--) {
8243 \begin_layout Plain Layout
8245 if ( (context[c, "language"], context[c, "mode"]) in escapes) {
8248 \begin_layout Plain Layout
8250 cpl = escapes[context[c, "language"], context[c, "mode"]];
8253 \begin_layout Plain Layout
8255 for (cp = 1; cp <= cpl; cp ++) {
8258 \begin_layout Plain Layout
8263 \begin_layout Plain Layout
8265 s[src] = escapes[context[c, "language"], context[c, "mode"], c, "s"];
8268 \begin_layout Plain Layout
8270 r[src] = escapes[context[c, "language"], context[c, "mode"], c, "r"];
8273 \begin_layout Plain Layout
8278 \begin_layout Plain Layout
8283 \begin_layout Plain Layout
8288 \begin_layout Plain Layout
8293 \begin_layout Plain Layout
8307 \begin_layout Standard
8308 \begin_inset listings
8312 \begin_layout Plain Layout
8317 \begin_layout Plain Layout
8319 passtest ./newfangle -Rtest:comment-quote $TEX_SRC &>/dev/null || ( echo
8320 "Comment-quote failed" && exit 1 )
8328 \begin_layout Chapter
8332 \begin_layout Standard
8333 Newfangle recognizes noweb chunks, but as we also want better LaTeX integration
8334 we will recognize any of these:
8337 \begin_layout Itemize
8338 notangle chunks matching the pattern
8339 \begin_inset Flex CharStyle:Code
8342 \begin_layout Plain Layout
8344 \begin_inset space \hspace*{}
8349 \begin_inset space \hspace*{}
8361 \begin_layout Itemize
8362 a chunks beginning with
8363 \begin_inset Flex CharStyle:Code
8366 \begin_layout Plain Layout
8376 Chunk{\SpecialChar \ldots{}
8377 } on the previous line
8380 \begin_layout Itemize
8381 an older form I have used, beginning with
8382 \begin_inset Flex CharStyle:Code
8385 \begin_layout Plain Layout
8388 begin{Chunk}[options]
8393 --- also more suitable for plain LaTeX users
8397 \begin_layout Plain Layout
8398 Is there such a thing as plain LaTeX?
8406 \begin_layout Section
8410 \begin_layout Standard
8412 \begin_inset Flex CharStyle:Code
8415 \begin_layout Plain Layout
8421 is used to signify that we are processing a code chunk and not document.
8422 In such a state, input lines will be assigned to the current chunk; otherwise
8426 \begin_layout Subsection
8430 \begin_layout Standard
8431 Our current scheme is to recognize the new lstlisting chunks, but these
8432 may be preceded by a
8433 \begin_inset Flex CharStyle:Code
8436 \begin_layout Plain Layout
8444 command which in LyX is a more convenient way to pass the chunk name to
8446 \begin_inset Flex CharStyle:Code
8449 \begin_layout Plain Layout
8457 command, and a more visible way to specify other
8458 \begin_inset Flex CharStyle:Code
8461 \begin_layout Plain Layout
8470 \begin_layout Standard
8471 The arguments to the
8472 \begin_inset Flex CharStyle:Code
8475 \begin_layout Plain Layout
8483 command are a name, and then a comma-seperated list of key-value pairs
8485 \begin_inset Flex CharStyle:Code
8488 \begin_layout Plain Layout
8497 (In fact within the LaTeX
8498 \begin_inset Flex CharStyle:Code
8501 \begin_layout Plain Layout
8510 \begin_inset CommandInset ref
8512 reference "sub:The-chunk-command"
8517 \begin_inset Flex CharStyle:Code
8520 \begin_layout Plain Layout
8526 is prefixed to the argument which is then literally passed to
8527 \begin_inset Flex CharStyle:Code
8530 \begin_layout Plain Layout
8545 \begin_layout Standard
8546 \begin_inset listings
8550 \begin_layout Plain Layout
8559 \begin_layout Plain Layout
8569 Chunk{ *([^ ,}]*),?(.*)}", line)) {
8572 \begin_layout Plain Layout
8574 next_chunk_name = line[1];
8577 \begin_layout Plain Layout
8579 get_chunk_args(line[2], next_chunk_args);
8582 \begin_layout Plain Layout
8587 \begin_layout Plain Layout
8592 \begin_layout Plain Layout
8602 \begin_layout Standard
8603 We also make a basic attempt to parse the name out of the
8604 \begin_inset Flex CharStyle:Code
8607 \begin_layout Plain Layout
8611 \begin_inset space \hspace{}
8620 text, otherwise we fall back to the name found in the previous chunk command.
8621 This attempt is very basic and doesn't support commas or spaces or square
8622 brackets as part of the chunkname.
8624 \begin_inset Flex CharStyle:Code
8627 \begin_layout Plain Layout
8635 which is convenient for some users
8639 \begin_layout Plain Layout
8640 but not yet supported in the LaTeX macros
8646 \begin_inset Note Note
8649 \begin_layout Plain Layout
8658 \begin_layout Standard
8659 \begin_inset listings
8663 \begin_layout Plain Layout
8676 \begin_layout Plain Layout
8678 if (match($0, "}.*[[,] *name= *{? *([^], }]*)", line)) {
8681 \begin_layout Plain Layout
8686 \begin_layout Plain Layout
8691 \begin_layout Plain Layout
8693 new_chunk(next_chunk_name, next_chunk_args);
8696 \begin_layout Plain Layout
8701 \begin_layout Plain Layout
8706 \begin_layout Plain Layout
8711 \begin_layout Plain Layout
8721 \begin_layout Subsection
8725 \begin_layout Standard
8726 We recognize notangle style chunks too:
8733 \begin_layout Standard
8734 \begin_inset listings
8738 \begin_layout Plain Layout
8743 \begin_layout Plain Layout
8745 if (match($0, "^[<]<(.*)[>]>= *$", line)) {
8748 \begin_layout Plain Layout
8753 \begin_layout Plain Layout
8758 \begin_layout Plain Layout
8763 \begin_layout Plain Layout
8768 \begin_layout Plain Layout
8773 \begin_layout Plain Layout
8783 \begin_layout Section
8787 \begin_layout Standard
8788 Likewise, we need to recognize when a chunk ends.
8791 \begin_layout Subsection
8795 \begin_layout Standard
8797 \begin_inset Flex CharStyle:Code
8800 \begin_layout Plain Layout
8807 \begin_inset Flex CharStyle:Code
8810 \begin_layout Plain Layout
8816 is surrounded by square brackets so that when this document is processed,
8817 this chunk doesn't terminate early when the lstlistings package recognizes
8818 it's own end-string!
8819 \begin_inset Note Greyedout
8822 \begin_layout Plain Layout
8823 This doesn't make sense as the regex is anchored with ^, which this line
8824 does not begin with!
8830 \begin_inset Note Note
8833 \begin_layout Plain Layout
8846 \begin_layout Standard
8847 \begin_inset listings
8851 \begin_layout Plain Layout
8864 \begin_layout Plain Layout
8869 \begin_layout Plain Layout
8874 \begin_layout Plain Layout
8879 \begin_layout Plain Layout
8889 \begin_layout Subsection
8897 \begin_layout Standard
8898 \begin_inset listings
8902 \begin_layout Plain Layout
8907 \begin_layout Plain Layout
8912 \begin_layout Plain Layout
8917 \begin_layout Plain Layout
8927 \begin_layout Standard
8928 All other recognizers are only of effect if we are chunking; there's no
8929 point in looking at lines if they aren't part of a chunk, so we just ignore
8930 them as efficiently as we can.
8937 \begin_layout Standard
8938 \begin_inset listings
8942 \begin_layout Plain Layout
8944 ! chunking { next; }
8952 \begin_layout Section
8956 \begin_layout Standard
8957 Chunk contents are any lines read while
8958 \begin_inset Flex CharStyle:Code
8961 \begin_layout Plain Layout
8968 Some chunk contents are special in that they refer to other chunks, and
8969 will be replaced by the contents of these chunks when the file is generated.
8972 \begin_layout Standard
8973 \begin_inset CommandInset label
8975 name "sub:ORS-chunk-text"
8979 We add the output record separator
8980 \begin_inset Flex CharStyle:Code
8983 \begin_layout Plain Layout
8989 to the line now, because we will set
8990 \begin_inset Flex CharStyle:Code
8993 \begin_layout Plain Layout
8999 to the empty string when we generate the output
9003 \begin_layout Plain Layout
9004 So that we can print partial lines using
9005 \begin_inset Flex CharStyle:Code
9008 \begin_layout Plain Layout
9015 \begin_inset Flex CharStyle:Code
9018 \begin_layout Plain Layout
9036 \begin_layout Standard
9037 \begin_inset listings
9041 \begin_layout Plain Layout
9043 length(active_chunk) {
9046 \begin_layout Plain Layout
9050 chunkref{process-chunk-tabs}>
9053 \begin_layout Plain Layout
9057 chunkref{process-chunk}>
9060 \begin_layout Plain Layout
9070 \begin_layout Standard
9071 If a chunk just consisted of plain text, we could handle the chunk like
9076 process-chunk-simple
9079 \begin_layout Standard
9080 \begin_inset listings
9084 \begin_layout Plain Layout
9086 chunk_line(active_chunk, $0 ORS);
9094 \begin_layout Standard
9095 but in fact a chunk can include references to other chunks.
9096 Chunk includes are traditionally written as
9097 \begin_inset Flex CharStyle:Code
9100 \begin_layout Plain Layout
9106 , but we support other variations.
9109 \begin_layout Standard
9110 However, we also process tabs at this point, a tab at input can be replaced
9111 by a number of spaces defined by the
9112 \begin_inset Flex CharStyle:Code
9115 \begin_layout Plain Layout
9121 variable, set by the
9122 \begin_inset Flex CharStyle:Code
9125 \begin_layout Plain Layout
9132 Of course this is poor tab behaviour, we should probably have the option
9133 to use proper counted tab-stops and process this on output.
9140 \begin_layout Standard
9141 \begin_inset listings
9145 \begin_layout Plain Layout
9150 \begin_layout Plain Layout
9157 \begin_layout Plain Layout
9167 \begin_layout Subsection
9168 \begin_inset CommandInset label
9170 name "sub:lstlistings-includes"
9177 \begin_layout Standard
9179 \begin_inset Flex CharStyle:Code
9182 \begin_layout Plain Layout
9185 lstset{escapeinside={=<}{>}}
9190 is set, then we can use
9191 \begin_inset Flex CharStyle:Code
9194 \begin_layout Plain Layout
9198 \begin_inset space \hspace{}
9209 \begin_inset Flex CharStyle:Code
9212 \begin_layout Plain Layout
9221 \begin_layout Enumerate
9222 it is a better mnemonic than
9223 \begin_inset Flex CharStyle:Code
9226 \begin_layout Plain Layout
9232 in that the = sign signifies equivalent or substitutability,
9235 \begin_layout Enumerate
9236 and because =< is not valid in C or in any language I can think of
9239 \begin_layout Enumerate
9240 and also because lstlistings doesn't like
9241 \begin_inset Flex CharStyle:Code
9244 \begin_layout Plain Layout
9250 as an end delimiter for the
9254 escape, so we must make do with a single
9255 \begin_inset Flex CharStyle:Code
9258 \begin_layout Plain Layout
9264 , which is better matched by
9265 \begin_inset Flex CharStyle:Code
9268 \begin_layout Plain Layout
9275 \begin_inset Flex CharStyle:Code
9278 \begin_layout Plain Layout
9287 \begin_layout Standard
9289 \begin_inset Note Note
9292 \begin_layout Plain Layout
9299 \begin_inset Flex CharStyle:Code
9302 \begin_layout Plain Layout
9303 =<\SpecialChar \ldots{}
9309 that we use re-enters a LaTeX parsing mode in which some characters are
9312 \begin_inset Flex CharStyle:Code
9315 \begin_layout Plain Layout
9323 , and so these cause trouble if used in arguments to
9324 \begin_inset Flex CharStyle:Code
9327 \begin_layout Plain Layout
9336 At some point I must fix the LaTeX command
9337 \begin_inset Flex CharStyle:Code
9340 \begin_layout Plain Layout
9348 so that it can accept these literally, but until then, when writing chunkref
9349 argumemts that need these characters, I must use the forms
9350 \begin_inset Flex CharStyle:Code
9353 \begin_layout Plain Layout
9362 \begin_inset Flex CharStyle:Code
9365 \begin_layout Plain Layout
9373 ; so I also define a hacky chunk
9374 \begin_inset Flex CharStyle:Code
9377 \begin_layout Plain Layout
9383 whose purpose it is to remove these from any arguments parsed by newfangle,
9384 and used further on.
9391 \begin_layout Standard
9392 \begin_inset listings
9396 \begin_layout Plain Layout
9401 \begin_layout Plain Layout
9414 \begin_layout Plain Layout
9436 \begin_layout Standard
9437 As each chunk line may contain more than one chunk include, we will split
9438 out chunk includes in an iterative fashion
9442 \begin_layout Plain Layout
9443 Contrary to our use of
9444 \begin_inset Flex CharStyle:Code
9447 \begin_layout Plain Layout
9453 when substituting parameters in chapter
9454 \begin_inset CommandInset ref
9456 reference "Here-we-split"
9468 \begin_layout Standard
9469 First, as long as the chunk contains a
9470 \begin_inset Flex CharStyle:Code
9473 \begin_layout Plain Layout
9481 command we take as much as we can up to the first
9482 \begin_inset Flex CharStyle:Code
9485 \begin_layout Plain Layout
9500 \begin_layout Standard
9501 \begin_inset listings
9505 \begin_layout Plain Layout
9510 \begin_layout Plain Layout
9515 \begin_layout Plain Layout
9520 \begin_layout Plain Layout
9538 )|)>|<<([a-zA-Z_][-a-zA-Z0-9_]*)>>)",
9541 \begin_layout Plain Layout
9548 \begin_layout Plain Layout
9553 \begin_layout Plain Layout
9555 chunklet = substr(chunk, 1, RSTART - 1);
9563 \begin_layout Standard
9564 We keep track of the indent count, by counting the number of literal characters
9566 We can then preserve this indent on each output line when multi-line chunks
9570 \begin_layout Standard
9571 We then process this first part literal text, and set the chunk which is
9572 still to be processed to be the text after the
9573 \begin_inset Flex CharStyle:Code
9576 \begin_layout Plain Layout
9584 command, which we will process next as we continue around the loop.
9587 \begin_layout Standard
9588 \begin_inset listings
9592 \begin_layout Plain Layout
9594 indent += length(chunklet);
9597 \begin_layout Plain Layout
9599 chunk_line(active_chunk, chunklet);
9602 \begin_layout Plain Layout
9604 chunk = substr(chunk, RSTART + RLENGTH);
9612 \begin_layout Standard
9613 We then consider the type of chunk command we have found, whether it is
9614 the newfangle style command beginning with
9615 \begin_inset Flex CharStyle:Code
9618 \begin_layout Plain Layout
9624 or the older notangle style beginning with
9625 \begin_inset Flex CharStyle:Code
9628 \begin_layout Plain Layout
9638 \begin_layout Standard
9639 Newfangle chunks may have parameters contained within square brackets.
9640 These will be matched in
9641 \begin_inset Flex CharStyle:Code
9644 \begin_layout Plain Layout
9650 and are considered at this stage of processing to be part of the name of
9651 the chunk to be included.
9654 \begin_layout Standard
9655 \begin_inset listings
9659 \begin_layout Plain Layout
9661 if (substr(line[1], 1, 1) == "=") {
9664 \begin_layout Plain Layout
9666 # chunk name up to }
9669 \begin_layout Plain Layout
9673 chunkref{delatex}(line[3])>
9676 \begin_layout Plain Layout
9678 chunk_include(active_chunk, line[2] line[3], indent);
9681 \begin_layout Plain Layout
9683 } else if (substr(line[1], 1, 1) == "<") {
9686 \begin_layout Plain Layout
9688 chunk_include(active_chunk, line[4], indent);
9691 \begin_layout Plain Layout
9696 \begin_layout Plain Layout
9698 error("Unknown chunk fragment: " line[1]);
9701 \begin_layout Plain Layout
9711 \begin_layout Standard
9712 The loop will continue until there are no more chunkref statements in the
9713 text, at which point we process the final part of the chunk.
9716 \begin_layout Standard
9717 \begin_inset listings
9721 \begin_layout Plain Layout
9726 \begin_layout Plain Layout
9728 chunk_line(active_chunk, chunk);
9736 \begin_layout Standard
9737 \begin_inset CommandInset label
9743 We add the newline character as a chunklet on it's own, to make it easier
9744 to detect new lines and thus manage indentation when processing the output.
9747 \begin_layout Standard
9748 \begin_inset listings
9752 \begin_layout Plain Layout
9754 chunk_line(active_chunk, "
9764 \begin_layout Standard
9765 We will also permit a chunk-part number to follow in square brackets, so
9767 \begin_inset Flex CharStyle:Code
9770 \begin_layout Plain Layout
9773 chunkref{chunk-name[1]}>
9778 will refer to the first part only.
9779 This can make it easy to include a C function prototype in a header file,
9780 if the first part of the chunk is just the function prototype without the
9781 trailing semi-colon.
9782 The header file would include the prototype with the trailing semi-colon,
9786 \begin_layout LyX-Code
9789 chunkref{chunk-name[1]}>;
9792 \begin_layout Standard
9793 This is handled in section
9794 \begin_inset CommandInset ref
9796 reference "sub:Chunk-parts"
9803 \begin_layout Standard
9804 We should perhaps introduce a notion of language specific chunk options;
9805 so that perhaps we could specify:
9808 \begin_layout LyX-Code
9811 chunkref{chunk-name[function-declaration]}>;
9814 \begin_layout Standard
9815 which applies a transform
9816 \begin_inset Flex CharStyle:Code
9819 \begin_layout Plain Layout
9820 function-declaration
9825 to the chunk --- which in this case would extract a function prototype
9827 \begin_inset Note Note
9830 \begin_layout Plain Layout
9839 \begin_layout Chapter
9843 \begin_layout Standard
9844 At the start, first we set the default options.
9851 \begin_layout Standard
9852 \begin_inset listings
9856 \begin_layout Plain Layout
9861 \begin_layout Plain Layout
9866 \begin_layout Plain Layout
9871 \begin_layout Plain Layout
9876 \begin_layout Plain Layout
9886 \begin_layout Standard
9887 Then we use getopt the standard way, and null out ARGV afterwards in the
9895 \begin_layout Standard
9896 \begin_inset listings
9900 \begin_layout Plain Layout
9902 Optind = 1 # skip ARGV[0]
9905 \begin_layout Plain Layout
9907 while(getopt(ARGC, ARGV, "R:LdT:hr")!=-1) {
9910 \begin_layout Plain Layout
9914 chunkref{handle-options}>
9917 \begin_layout Plain Layout
9922 \begin_layout Plain Layout
9924 for (i=1; i<Optind; i++) { ARGV[i]=""; }
9932 \begin_layout Standard
9933 This is how we handle our options:
9940 \begin_layout Standard
9941 \begin_inset listings
9945 \begin_layout Plain Layout
9947 if (Optopt == "R") root = Optarg;
9950 \begin_layout Plain Layout
9952 else if (Optopt == "r") root="";
9955 \begin_layout Plain Layout
9957 else if (Optopt == "L") linenos = 1;
9960 \begin_layout Plain Layout
9962 else if (Optopt == "d") debug = 1;
9965 \begin_layout Plain Layout
9967 else if (Optopt == "T") tabs = indent_string(Optarg+0);
9970 \begin_layout Plain Layout
9972 else if (Optopt == "h") help();
9975 \begin_layout Plain Layout
9977 else if (Optopt == "?") help();
9985 \begin_layout Standard
9986 We do all of this at the beginning of the program
9993 \begin_layout Standard
9994 \begin_inset listings
9998 \begin_layout Plain Layout
10003 \begin_layout Plain Layout
10007 chunkref{constants}>
10010 \begin_layout Plain Layout
10014 chunkref{mode-definitions}>
10017 \begin_layout Plain Layout
10021 chunkref{default-options}>
10024 \begin_layout Plain Layout
10028 \begin_layout Plain Layout
10032 chunkref{read-options}>
10035 \begin_layout Plain Layout
10045 \begin_layout Standard
10046 And have a simple help function
10049 \begin_layout Chunk
10053 \begin_layout Standard
10054 \begin_inset listings
10058 \begin_layout Plain Layout
10063 \begin_layout Plain Layout
10068 \begin_layout Plain Layout
10070 print " newfangle [-L] -R<rootname> [source.tex ...]"
10073 \begin_layout Plain Layout
10075 print " newfangle -r [source.tex ...]"
10078 \begin_layout Plain Layout
10080 print " If the filename, source.tex is not specified then stdin is used"
10083 \begin_layout Plain Layout
10088 \begin_layout Plain Layout
10090 print "-L causes the C statement: #line <lineno>
10097 \begin_layout Plain Layout
10099 print "-R causes the named root to be written to stdout"
10102 \begin_layout Plain Layout
10104 print "-r lists all roots in the file (even those used elsewhere)"
10107 \begin_layout Plain Layout
10112 \begin_layout Plain Layout
10122 \begin_layout Chapter
10123 Generating the output
10126 \begin_layout Standard
10127 We generate output by calling output_chunk, or listing the chunk names.
10130 \begin_layout Chunk
10134 \begin_layout Standard
10135 \begin_inset listings
10139 \begin_layout Plain Layout
10141 if (length(root)) output_chunk(root);
10144 \begin_layout Plain Layout
10146 else output_chunk_names();
10154 \begin_layout Standard
10155 We also have some other output debugging:
10158 \begin_layout Chunk
10162 \begin_layout Standard
10163 \begin_inset listings
10167 \begin_layout Plain Layout
10172 \begin_layout Plain Layout
10174 print "------ chunk names "
10177 \begin_layout Plain Layout
10179 output_chunk_names();
10182 \begin_layout Plain Layout
10184 print "====== chunks"
10187 \begin_layout Plain Layout
10192 \begin_layout Plain Layout
10194 print "++++++ debug"
10197 \begin_layout Plain Layout
10199 for (a in chunks) {
10202 \begin_layout Plain Layout
10204 print a "=" chunks[a];
10207 \begin_layout Plain Layout
10212 \begin_layout Plain Layout
10222 \begin_layout Standard
10223 We do both of these at the end.
10225 \begin_inset Flex CharStyle:Code
10228 \begin_layout Plain Layout
10234 because each chunklet is not necessarily a complete line, and we already
10236 \begin_inset Flex CharStyle:Code
10239 \begin_layout Plain Layout
10245 to each input line in section
10246 \begin_inset CommandInset ref
10248 reference "sub:ORS-chunk-text"
10255 \begin_layout Chunk
10259 \begin_layout Standard
10260 \begin_inset listings
10264 \begin_layout Plain Layout
10269 \begin_layout Plain Layout
10273 chunkref{debug-output}>
10276 \begin_layout Plain Layout
10281 \begin_layout Plain Layout
10285 chunkref{generate-output}>
10288 \begin_layout Plain Layout
10298 \begin_layout Standard
10299 We write chunk names like this.
10300 If we seem to be running in notangle compatibility mode, then we enclose
10302 \begin_inset Flex CharStyle:Code
10305 \begin_layout Plain Layout
10311 the same way notangle does:
10314 \begin_layout Chunk
10315 output_chunk_names()
10318 \begin_layout Standard
10319 \begin_inset listings
10323 \begin_layout Plain Layout
10325 function output_chunk_names( c, prefix, suffix)
10328 \begin_layout Plain Layout
10333 \begin_layout Plain Layout
10335 if (notangle_mode) {
10338 \begin_layout Plain Layout
10343 \begin_layout Plain Layout
10348 \begin_layout Plain Layout
10353 \begin_layout Plain Layout
10355 for (c in chunk_names) {
10358 \begin_layout Plain Layout
10360 print prefix c suffix "
10365 \begin_layout Plain Layout
10370 \begin_layout Plain Layout
10380 \begin_layout Standard
10381 This function would write out all chunks
10384 \begin_layout Chunk
10388 \begin_layout Standard
10389 \begin_inset listings
10393 \begin_layout Plain Layout
10395 function output_chunks( a)
10398 \begin_layout Plain Layout
10403 \begin_layout Plain Layout
10405 for (a in chunk_names) {
10408 \begin_layout Plain Layout
10410 output_chunk(chunk_names[a]);
10413 \begin_layout Plain Layout
10418 \begin_layout Plain Layout
10423 \begin_layout Plain Layout
10427 \begin_layout Plain Layout
10429 function output_chunk(chunk) {
10432 \begin_layout Plain Layout
10437 \begin_layout Plain Layout
10439 lineno_needed = linenos;
10442 \begin_layout Plain Layout
10446 \begin_layout Plain Layout
10448 write_chunk(chunk);
10451 \begin_layout Plain Layout
10456 \begin_layout Plain Layout
10465 \begin_layout Section
10466 Assembling the chunks
10469 \begin_layout Standard
10470 \begin_inset Flex CharStyle:Code
10473 \begin_layout Plain Layout
10479 holds a string consisting of the names of all the chunks that resulted
10480 in this chunk being output.
10482 \begin_inset Note Note
10485 \begin_layout Plain Layout
10486 Make sure it includes the line numbers too...
10492 It should probably also contain the source line numbers at which each inclusion
10496 \begin_layout Chunk
10500 \begin_layout Standard
10501 We first initialize the mode tracker for this chunk.
10504 \begin_layout Standard
10505 \begin_inset listings
10509 \begin_layout Plain Layout
10511 function write_chunk(chunk_name) {
10514 \begin_layout Plain Layout
10518 chunkref{awk-delete-array}(context)>
10521 \begin_layout Plain Layout
10523 return write_chunk_r(chunk_name, context);
10526 \begin_layout Plain Layout
10536 \begin_layout Chunk
10537 write_chunk(),emph={chunk_path}
10540 \begin_layout Standard
10541 \begin_inset listings
10545 \begin_layout Plain Layout
10547 function write_chunk_r(chunk_name, context, indent, tail,
10550 \begin_layout Plain Layout
10555 \begin_layout Plain Layout
10557 chunk_path, chunk_args,
10560 \begin_layout Plain Layout
10562 s, r, src, new_src,
10565 \begin_layout Plain Layout
10570 \begin_layout Plain Layout
10572 chunk_params, part, max_part, part_line, frag, max_frag, text,
10575 \begin_layout Plain Layout
10577 chunklet, only_part, call_chunk_args)
10580 \begin_layout Plain Layout
10590 \begin_layout Subsection
10591 \begin_inset CommandInset label
10593 name "sub:Chunk-parts"
10600 \begin_layout Standard
10601 As mentioned in section
10602 \begin_inset CommandInset ref
10604 reference "sub:lstlistings-includes"
10608 , a chunk name may contain a part specifier in square brackets, limiting
10609 the parts that should be emitted.
10612 \begin_layout Standard
10613 \begin_inset listings
10617 \begin_layout Plain Layout
10619 if (match(chunk_name, "^(.*)
10627 ]$", chunk_name_parts)) {
10630 \begin_layout Plain Layout
10632 chunk_name = chunk_name_parts[1];
10635 \begin_layout Plain Layout
10637 only_part = chunk_name_parts[2];
10640 \begin_layout Plain Layout
10650 \begin_layout Standard
10651 We then create a mode tracker
10654 \begin_layout Standard
10655 \begin_inset listings
10659 \begin_layout Plain Layout
10663 chunkref{new-mode-tracker}(context, chunks[chunk_name, "language"], "")>
10671 \begin_layout Standard
10673 \begin_inset Flex CharStyle:Code
10676 \begin_layout Plain Layout
10682 the names of the parameters that this chunk accepts, whose values were
10683 (optionally) passed in
10684 \begin_inset Flex CharStyle:Code
10687 \begin_layout Plain Layout
10696 \begin_layout Standard
10697 \begin_inset listings
10701 \begin_layout Plain Layout
10703 split(chunks[chunk_name, "params"], chunk_params, " *; *");
10711 \begin_layout Standard
10712 To assemble a chunk, we write out each part.
10715 \begin_layout Chunk
10719 \begin_layout Standard
10720 \begin_inset listings
10724 \begin_layout Plain Layout
10726 if (! (chunk_name in chunk_names)) {
10729 \begin_layout Plain Layout
10731 error(sprintf(_"The root module <<%s>> was not defined.
10738 \begin_layout Plain Layout
10740 chunk_name, chunk_path));
10743 \begin_layout Plain Layout
10748 \begin_layout Plain Layout
10752 \begin_layout Plain Layout
10754 max_part = chunks[chunk_name, "part"];
10757 \begin_layout Plain Layout
10759 for(part = 1; part <= max_part; part++) {
10762 \begin_layout Plain Layout
10764 if (! only_part || part == only_part) {
10767 \begin_layout Plain Layout
10771 chunkref{write-part}>
10774 \begin_layout Plain Layout
10779 \begin_layout Plain Layout
10784 \begin_layout Plain Layout
10786 if (! finalize_mode_tracker(context)) {
10789 \begin_layout Plain Layout
10791 error(sprintf(_"Module %s did not close context properly.
10795 n", chunk_name, chunk_path));
10798 \begin_layout Plain Layout
10803 \begin_layout Plain Layout
10813 \begin_layout Standard
10814 A part can either be a chunklet of lines, or an include of another chunk.
10817 \begin_layout Standard
10818 Chunks may also have parameters, specified in LaTeX style with braces after
10819 the chunk name --- looking like this in the document:
10820 \begin_inset Flex CharStyle:Code
10823 \begin_layout Plain Layout
10824 chunkname{param1, param2}
10830 Arguments are passed in square brackets:
10831 \begin_inset Flex CharStyle:Code
10834 \begin_layout Plain Layout
10837 chunkref{chunkname}[arg1, arg2]
10845 \begin_layout Standard
10846 Before we process each part, we check that the source position hasn't changed
10847 unexpectedly, so that we can know if we need to output a new file-line
10851 \begin_layout Chunk
10855 \begin_layout Standard
10856 \begin_inset listings
10860 \begin_layout Plain Layout
10864 chunkref{check-source-jump}>
10867 \begin_layout Plain Layout
10871 \begin_layout Plain Layout
10873 chunklet = chunks[chunk_name, "part", part];
10876 \begin_layout Plain Layout
10878 if (chunks[chunk_name, "part", part, "type"] == part_type_chunk) {
10881 \begin_layout Plain Layout
10885 chunkref{write-included-chunk}>
10888 \begin_layout Plain Layout
10890 } else if (chunklet SUBSEP "line" in chunks) {
10893 \begin_layout Plain Layout
10897 chunkref{write-chunklets}>
10900 \begin_layout Plain Layout
10905 \begin_layout Plain Layout
10907 # empty last chunklet
10910 \begin_layout Plain Layout
10920 \begin_layout Standard
10921 To write an included chunk, we must detect any optional chunk arguments
10923 Then we recurse calling
10924 \begin_inset Flex Chunkref
10927 \begin_layout Plain Layout
10936 \begin_layout Chunk
10937 write-included-chunk
10940 \begin_layout Standard
10941 \begin_inset listings
10945 \begin_layout Plain Layout
10947 if (match(chunklet, "^([^
10963 )$", chunklet_parts)) {
10966 \begin_layout Plain Layout
10968 chunklet = chunklet_parts[1];
10971 \begin_layout Plain Layout
10973 parse_chunk_args("", chunklet_parts[2], call_chunk_args, "(");
10976 \begin_layout Plain Layout
10978 for (c in call_chunk_args) {
10981 \begin_layout Plain Layout
10983 call_chunk_args[c] = expand_chunk_args(call_chunk_args[c], chunk_params,
10987 \begin_layout Plain Layout
10992 \begin_layout Plain Layout
10997 \begin_layout Plain Layout
10999 split("", call_chunk_args);
11002 \begin_layout Plain Layout
11007 \begin_layout Plain Layout
11009 # update the transforms arrays
11012 \begin_layout Plain Layout
11014 new_src = mode_escaper(context, s, r, src);
11017 \begin_layout Plain Layout
11019 write_chunk_r(chunklet, context,
11022 \begin_layout Plain Layout
11024 chunks[chunk_name, "part", part, "indent"] indent,
11027 \begin_layout Plain Layout
11029 chunks[chunk_name, "part", part, "tail"],
11032 \begin_layout Plain Layout
11039 \begin_layout Plain Layout
11044 \begin_layout Plain Layout
11054 \begin_layout Standard
11055 Before we output a chunklet of lines, we first emit the file and line number
11056 if we have one, and if it is safe to do so.
11060 \begin_layout Standard
11061 Chunklets are generally broken up by includes, so the start of a chunklet
11062 is a good place to do this.
11063 Then we output each line of the chunklet.
11066 \begin_layout Standard
11067 When it is not safe, such as in the middle of a multi-line macro definition,
11069 \begin_inset Flex CharStyle:Code
11072 \begin_layout Plain Layout
11078 is set to true, and in such a case we note that we want to emit the line
11079 statement when it is next safe.
11082 \begin_layout Chunk
11086 \begin_layout Standard
11087 \begin_inset listings
11091 \begin_layout Plain Layout
11093 max_frag = chunks[chunklet, "line"];
11096 \begin_layout Plain Layout
11098 for(frag = 1; frag <= max_frag; frag++) {
11101 \begin_layout Plain Layout
11105 chunkref{write-file-line}>
11113 \begin_layout Standard
11114 We then extract the chunklet text and expand any arguments.
11117 \begin_layout Standard
11118 \begin_inset listings
11122 \begin_layout Plain Layout
11126 \begin_layout Plain Layout
11128 text = chunks[chunklet, frag];
11131 \begin_layout Plain Layout
11136 \begin_layout Plain Layout
11141 \begin_layout Plain Layout
11143 text = expand_chunk_args(text, chunk_params, chunk_args);
11151 \begin_layout Standard
11152 If the text is a single newline (which we keep separate - see
11153 \begin_inset CommandInset ref
11155 reference "lone-newline"
11159 ) then we increment the line number.
11160 In the case where this is the last line of a chunk and it is not a top-level
11161 chunk we replace the newline with an empty string --- because the chunk
11162 that included this chunk will have the newline at the end of the line that
11163 included this chunk.
11166 \begin_layout Standard
11168 \begin_inset Flex CharStyle:Code
11171 \begin_layout Plain Layout
11177 that we have started a new line, so that indentation can be managed with
11178 the following piece of text.
11181 \begin_layout Standard
11182 \begin_inset listings
11186 \begin_layout Plain Layout
11190 \begin_layout Plain Layout
11197 \begin_layout Plain Layout
11202 \begin_layout Plain Layout
11204 if (part == max_part && frag == max_frag && length(chunk_path)) {
11207 \begin_layout Plain Layout
11212 \begin_layout Plain Layout
11217 \begin_layout Plain Layout
11222 \begin_layout Plain Layout
11227 \begin_layout Plain Layout
11237 \begin_layout Standard
11238 If this text does not represent a newline, but we see that we are the first
11239 piece of text on a newline, then we prefix our text with the current indent.
11241 \begin_inset Flex CharStyle:Code
11244 \begin_layout Plain Layout
11250 is a global output-state variable, but the
11251 \begin_inset Flex CharStyle:Code
11254 \begin_layout Plain Layout
11264 \begin_layout Standard
11265 \begin_inset listings
11269 \begin_layout Plain Layout
11271 } else if (length(text) || length(tail)) {
11274 \begin_layout Plain Layout
11276 if (newline) text = indent text;
11279 \begin_layout Plain Layout
11284 \begin_layout Plain Layout
11289 \begin_layout Plain Layout
11298 \begin_layout Standard
11299 Tail will soon no longer be relevant once mode-detection is in place.
11302 \begin_layout Standard
11303 \begin_inset listings
11307 \begin_layout Plain Layout
11312 \begin_layout Plain Layout
11314 mode_tracker(context, text);
11317 \begin_layout Plain Layout
11319 print transform_escape(s, r, text, src);
11327 \begin_layout Standard
11328 If a line ends in a backslash --- suggesting continuation --- then we supress
11329 outputting file-line as it would probably break the continued lines.
11333 \begin_layout Standard
11334 \begin_inset listings
11338 \begin_layout Plain Layout
11343 \begin_layout Plain Layout
11345 lineno_suppressed = substr(lastline, length(lastline)) == "
11352 \begin_layout Plain Layout
11357 \begin_layout Plain Layout
11367 \begin_layout Standard
11368 Of course there is no point in actually outputting the source filename and
11369 line number (file-line) if they don't say anything new! We only need to
11370 emit them if they aren't what is expected, or if we we not able to emit
11371 one when they had changed.
11374 \begin_layout Chunk
11378 \begin_layout Standard
11379 \begin_inset listings
11383 \begin_layout Plain Layout
11385 if (newline && lineno_needed && ! lineno_suppressed) {
11388 \begin_layout Plain Layout
11390 filename = a_filename;
11393 \begin_layout Plain Layout
11398 \begin_layout Plain Layout
11400 print "#line " lineno "
11409 \begin_layout Plain Layout
11414 \begin_layout Plain Layout
11424 \begin_layout Standard
11425 We check if a new file-line is needed by checking if the source line matches
11426 what we (or a compiler) would expect.
11430 \begin_layout Chunk
11434 \begin_layout Standard
11435 \begin_inset listings
11439 \begin_layout Plain Layout
11441 if (linenos && (chunk_name SUBSEP "part" SUBSEP part SUBSEP "FILENAME" in
11445 \begin_layout Plain Layout
11447 a_filename = chunks[chunk_name, "part", part, "FILENAME"];
11450 \begin_layout Plain Layout
11452 a_lineno = chunks[chunk_name, "part", part, "LINENO"];
11455 \begin_layout Plain Layout
11457 if (a_filename != filename || a_lineno != lineno) {
11460 \begin_layout Plain Layout
11465 \begin_layout Plain Layout
11470 \begin_layout Plain Layout
11480 \begin_layout Chapter
11484 \begin_layout Standard
11485 Awk has pretty limited data structures, so we will use two main hashes.
11486 Uninterrupted sequences of a chunk will be stored in
11487 \begin_inset Flex CharStyle:Code
11490 \begin_layout Plain Layout
11496 and the chunklets used in a chunk will be stored in
11497 \begin_inset Flex CharStyle:Code
11500 \begin_layout Plain Layout
11509 \begin_layout Chunk
11513 \begin_layout Standard
11514 \begin_inset listings
11518 \begin_layout Plain Layout
11523 \begin_layout Plain Layout
11533 \begin_layout Standard
11535 \begin_inset Flex CharStyle:Code
11538 \begin_layout Plain Layout
11544 mentioned are not chunk parameters for parameterized chunks, as mentioned
11546 \begin_inset CommandInset ref
11548 reference "cha:Chunk Arguments"
11552 , but the lstlistings style parameters used in the
11553 \begin_inset Flex CharStyle:Code
11556 \begin_layout Plain Layout
11568 \begin_layout Plain Layout
11570 \begin_inset Flex CharStyle:Code
11573 \begin_layout Plain Layout
11579 parameter is used to hold the parameters for parameterized chunks
11587 \begin_layout Chunk
11588 chunk-storage-functions
11591 \begin_layout Standard
11592 \begin_inset listings
11596 \begin_layout Plain Layout
11598 function new_chunk(chunk_name, params,
11601 \begin_layout Plain Layout
11606 \begin_layout Plain Layout
11611 \begin_layout Plain Layout
11616 \begin_layout Plain Layout
11618 # HACK WHILE WE CHANGE TO ( ) for PARAM CHUNKS
11621 \begin_layout Plain Layout
11631 )$", "", chunk_name);
11634 \begin_layout Plain Layout
11636 if (! (chunk_name in chunk_names)) {
11639 \begin_layout Plain Layout
11641 if (debug) print "New chunk " chunk_name;
11644 \begin_layout Plain Layout
11646 chunk_names[chunk_name];
11649 \begin_layout Plain Layout
11651 for (p in params) {
11654 \begin_layout Plain Layout
11656 chunks[chunk_name, p] = params[p];
11659 \begin_layout Plain Layout
11664 \begin_layout Plain Layout
11666 if ("append" in params) {
11669 \begin_layout Plain Layout
11671 append=params["append"];
11674 \begin_layout Plain Layout
11676 if (! (append in chunk_names)) {
11679 \begin_layout Plain Layout
11681 warning("Chunk " chunk_name " is appended to chunk " append " which
11682 is not defined yet");
11685 \begin_layout Plain Layout
11690 \begin_layout Plain Layout
11695 \begin_layout Plain Layout
11697 chunk_include(append, chunk_name);
11700 \begin_layout Plain Layout
11702 chunk_line(append, ORS);
11705 \begin_layout Plain Layout
11710 \begin_layout Plain Layout
11715 \begin_layout Plain Layout
11717 active_chunk = chunk_name;
11720 \begin_layout Plain Layout
11722 prime_chunk(chunk_name);
11725 \begin_layout Plain Layout
11735 \begin_layout Standard
11736 \begin_inset listings
11740 \begin_layout Plain Layout
11744 \begin_layout Plain Layout
11746 function prime_chunk(chunk_name)
11749 \begin_layout Plain Layout
11754 \begin_layout Plain Layout
11756 chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] =
11761 \begin_layout Plain Layout
11763 chunk_name SUBSEP "chunklet" SUBSEP "" ++chunks[chunk_name, "chunklet"]
11767 \begin_layout Plain Layout
11769 chunks[chunk_name, "part", chunks[chunk_name, "part"], "FILENAME"] = FILENAME;
11772 \begin_layout Plain Layout
11774 chunks[chunk_name, "part", chunks[chunk_name, "part"], "LINENO"] = FNR
11778 \begin_layout Plain Layout
11783 \begin_layout Plain Layout
11787 \begin_layout Plain Layout
11789 function chunk_line(chunk_name, line){
11792 \begin_layout Plain Layout
11794 chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
11797 \begin_layout Plain Layout
11799 ++chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
11803 \begin_layout Plain Layout
11808 \begin_layout Plain Layout
11817 \begin_layout Standard
11818 Chunk include represents a
11822 statement, and stores the requirement to include another chunk.
11823 The parameter indent represents the quanity of literal text characters
11828 statement and therefore by how much additional lines of the included chunk
11829 should be indented.
11832 \begin_layout Standard
11833 \begin_inset listings
11837 \begin_layout Plain Layout
11839 function chunk_include(chunk_name, chunk_ref, indent, tail)
11842 \begin_layout Plain Layout
11847 \begin_layout Plain Layout
11849 chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] = chunk_ref;
11852 \begin_layout Plain Layout
11854 chunks[chunk_name, "part", chunks[chunk_name, "part"], "type" ] = part_type_ch
11858 \begin_layout Plain Layout
11860 chunks[chunk_name, "part", chunks[chunk_name, "part"], "indent" ] = indent_str
11864 \begin_layout Plain Layout
11866 chunks[chunk_name, "part", chunks[chunk_name, "part"], "tail" ] = tail;
11869 \begin_layout Plain Layout
11871 prime_chunk(chunk_name);
11874 \begin_layout Plain Layout
11879 \begin_layout Plain Layout
11888 \begin_layout Standard
11889 The indent is calculated by indent_string, which may in future convert some
11890 spaces into tab characters.
11891 This function works by generating a printf padded format string, like
11892 \begin_inset Flex CharStyle:Code
11895 \begin_layout Plain Layout
11901 for an indent of 22, and then printing an empty string using that format.
11904 \begin_layout Standard
11905 \begin_inset listings
11909 \begin_layout Plain Layout
11911 function indent_string(indent) {
11914 \begin_layout Plain Layout
11916 return sprintf("%" indent "s", "");
11919 \begin_layout Plain Layout
11929 \begin_layout Chapter
11930 \begin_inset CommandInset label
11939 \begin_layout Standard
11940 I use Arnold Robbins public domain getopt (1993 revision).
11941 This is probably the same one that is covered in chapter 12 of
11942 \begin_inset Quotes eld
11945 Edition 3 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk
11946 \begin_inset Quotes erd
11949 but as that is licensed under the GNU Free Documentation License, Version
11950 1.3, which conflicts with the GPL3, I can't use it from there (or it's accompany
11951 ing explanations), so I do my best to explain how it works here.
11954 \begin_layout Standard
11955 The getopt.awk header is:
11958 \begin_layout Chunk
11959 getopt.awk-header,language=awk,morestring=[b]{/},morekeywords=else
11962 \begin_layout Standard
11963 \begin_inset listings
11967 \begin_layout Plain Layout
11969 # getopt.awk --- do C library getopt(3) function in awk
11972 \begin_layout Plain Layout
11977 \begin_layout Plain Layout
11979 # Arnold Robbins, arnold@skeeve.com, Public Domain
11982 \begin_layout Plain Layout
11987 \begin_layout Plain Layout
11989 # Initial version: March, 1991
11992 \begin_layout Plain Layout
11994 # Revised: May, 1993
11997 \begin_layout Plain Layout
12006 \begin_layout Standard
12007 The provided explanation is:
12010 \begin_layout Chunk
12014 \begin_layout Standard
12015 \begin_inset listings
12019 \begin_layout Plain Layout
12021 # External variables:
12024 \begin_layout Plain Layout
12026 # Optind -- index in ARGV of first nonoption argument
12029 \begin_layout Plain Layout
12031 # Optarg -- string value of argument to current option
12034 \begin_layout Plain Layout
12036 # Opterr -- if nonzero, print our own diagnostic
12039 \begin_layout Plain Layout
12041 # Optopt -- current option letter
12044 \begin_layout Plain Layout
12048 \begin_layout Plain Layout
12053 \begin_layout Plain Layout
12055 # -1 at end of options
12058 \begin_layout Plain Layout
12060 # ? for unrecognized option
12063 \begin_layout Plain Layout
12065 # <c> a character representing the current option
12068 \begin_layout Plain Layout
12072 \begin_layout Plain Layout
12077 \begin_layout Plain Layout
12079 # _opti -- index in multi-flag option, e.g., -abc
12082 \begin_layout Plain Layout
12091 \begin_layout Standard
12092 The function follows.
12093 The final two parameters,
12094 \begin_inset Flex CharStyle:Code
12097 \begin_layout Plain Layout
12104 \begin_inset Flex CharStyle:Code
12107 \begin_layout Plain Layout
12113 are local variables and not parameters --- as indicated by the multiple
12114 spaces preceding them.
12115 Awk doesn't care, the multiple spaces are a convention to help us humans.
12118 \begin_layout Chunk
12119 getopt.awk-getopt()
12122 \begin_layout Standard
12123 \begin_inset listings
12127 \begin_layout Plain Layout
12129 function getopt(argc, argv, options, thisopt, i)
12132 \begin_layout Plain Layout
12137 \begin_layout Plain Layout
12139 if (length(options) == 0) # no options given
12142 \begin_layout Plain Layout
12147 \begin_layout Plain Layout
12149 if (argv[Optind] == "--") { # all done
12152 \begin_layout Plain Layout
12157 \begin_layout Plain Layout
12162 \begin_layout Plain Layout
12167 \begin_layout Plain Layout
12169 } else if (argv[Optind] !~ /^-[^:
12184 \begin_layout Plain Layout
12189 \begin_layout Plain Layout
12194 \begin_layout Plain Layout
12199 \begin_layout Plain Layout
12204 \begin_layout Plain Layout
12209 \begin_layout Plain Layout
12211 thisopt = substr(argv[Optind], _opti, 1)
12214 \begin_layout Plain Layout
12219 \begin_layout Plain Layout
12221 i = index(options, thisopt)
12224 \begin_layout Plain Layout
12229 \begin_layout Plain Layout
12234 \begin_layout Plain Layout
12236 printf("%c -- invalid option
12241 \begin_layout Plain Layout
12243 thisopt) > "/dev/stderr"
12246 \begin_layout Plain Layout
12248 if (_opti >= length(argv[Optind])) {
12251 \begin_layout Plain Layout
12256 \begin_layout Plain Layout
12261 \begin_layout Plain Layout
12266 \begin_layout Plain Layout
12271 \begin_layout Plain Layout
12276 \begin_layout Plain Layout
12286 \begin_layout Standard
12287 At this point, the option has been found and we need to know if it takes
12291 \begin_layout Standard
12292 \begin_inset listings
12296 \begin_layout Plain Layout
12298 if (substr(options, i + 1, 1) == ":") {
12301 \begin_layout Plain Layout
12303 # get option argument
12306 \begin_layout Plain Layout
12308 if (length(substr(argv[Optind], _opti + 1)) > 0)
12311 \begin_layout Plain Layout
12313 Optarg = substr(argv[Optind], _opti + 1)
12316 \begin_layout Plain Layout
12321 \begin_layout Plain Layout
12323 Optarg = argv[++Optind]
12326 \begin_layout Plain Layout
12331 \begin_layout Plain Layout
12336 \begin_layout Plain Layout
12341 \begin_layout Plain Layout
12343 if (_opti == 0 || _opti >= length(argv[Optind])) {
12346 \begin_layout Plain Layout
12351 \begin_layout Plain Layout
12356 \begin_layout Plain Layout
12361 \begin_layout Plain Layout
12366 \begin_layout Plain Layout
12371 \begin_layout Plain Layout
12378 A test program is built in, too
12381 \begin_layout Chunk
12385 \begin_layout Standard
12386 \begin_inset listings
12390 \begin_layout Plain Layout
12395 \begin_layout Plain Layout
12397 Opterr = 1 # default is to diagnose
12400 \begin_layout Plain Layout
12402 Optind = 1 # skip ARGV[0]
12405 \begin_layout Plain Layout
12410 \begin_layout Plain Layout
12412 if (_getopt_test) {
12415 \begin_layout Plain Layout
12417 while ((_go_c = getopt(ARGC, ARGV, "ab:cd")) != -1)
12420 \begin_layout Plain Layout
12422 printf("c = <%c>, optarg = <%s>
12427 \begin_layout Plain Layout
12432 \begin_layout Plain Layout
12434 printf("non-option arguments:
12439 \begin_layout Plain Layout
12441 for (; Optind < ARGC; Optind++)
12444 \begin_layout Plain Layout
12453 \begin_layout Plain Layout
12455 Optind, ARGV[Optind])
12458 \begin_layout Plain Layout
12463 \begin_layout Plain Layout
12473 \begin_layout Standard
12474 The entire getopt.awk is made out of these chunks in order
12477 \begin_layout Chunk
12481 \begin_layout Standard
12482 \begin_inset listings
12486 \begin_layout Plain Layout
12490 chunkref{getopt.awk-header}>
12493 \begin_layout Plain Layout
12497 \begin_layout Plain Layout
12501 chunkref{getopt.awk-notes}>
12504 \begin_layout Plain Layout
12508 chunkref{getopt.awk-getopt()}>
12511 \begin_layout Plain Layout
12515 chunkref{getopt.awk-begin}>
12523 \begin_layout Standard
12524 Although we only want the header and function:
12527 \begin_layout Chunk
12531 \begin_layout Standard
12532 \begin_inset listings
12536 \begin_layout Plain Layout
12538 # try: locate getopt.awk for the full original file
12541 \begin_layout Plain Layout
12543 # as part of your standard awk installation
12546 \begin_layout Plain Layout
12550 chunkref{getopt.awk-header}>
12553 \begin_layout Plain Layout
12557 \begin_layout Plain Layout
12561 chunkref{getopt.awk-getopt()}>
12569 \begin_layout Chapter
12570 Newfangle LaTeX source code
12573 \begin_layout Section
12577 \begin_layout Standard
12578 Here we define a Lyx .module file that makes it convenient to use LyX for
12579 writing such literate programs.
12582 \begin_layout Standard
12584 \begin_inset Flex CharStyle:Code
12587 \begin_layout Plain Layout
12593 can be installed in your personal
12594 \begin_inset Flex CharStyle:Code
12597 \begin_layout Plain Layout
12598 .lyx/layouts folder
12604 You will need to Tools Reconfigure so that LyX notices it.
12605 It adds a new format Chunk, which should precede every listing and contain
12610 \begin_layout Chunk
12611 ./newfangle.module,language=
12614 \begin_layout Standard
12615 \begin_inset listings
12619 \begin_layout Plain Layout
12623 DeclareLyXModule{Newfangle Literate Listings}
12626 \begin_layout Plain Layout
12631 \begin_layout Plain Layout
12633 # Newfangle literate listings allow one to write
12636 \begin_layout Plain Layout
12638 # literate programs after the fashion of noweb, but without having
12641 \begin_layout Plain Layout
12643 # to use noweave to generate the documentation.
12644 Instead the listings
12647 \begin_layout Plain Layout
12649 # package is extended in conjunction with the noweb package to implement
12652 \begin_layout Plain Layout
12654 # to code formating directly as latex.
12657 \begin_layout Plain Layout
12659 # The newfangle awk script
12662 \begin_layout Plain Layout
12667 \begin_layout Plain Layout
12671 \begin_layout Plain Layout
12676 \begin_layout Plain Layout
12680 \begin_layout Plain Layout
12685 \begin_layout Plain Layout
12689 chunkref{./newfangle.sty}>
12692 \begin_layout Plain Layout
12697 \begin_layout Plain Layout
12701 \begin_layout Plain Layout
12705 chunkref{chunkstyle}>
12708 \begin_layout Plain Layout
12712 \begin_layout Plain Layout
12716 chunkref{chunkref}>
12724 \begin_layout Subsection
12728 \begin_layout Standard
12733 style is to make it easier for LyX users to provide the name to
12734 \begin_inset Flex CharStyle:Code
12737 \begin_layout Plain Layout
12746 Normally this requires right-clicking on the listing, choosing settings,
12747 advanced, and then typing
12748 \begin_inset Flex CharStyle:Code
12751 \begin_layout Plain Layout
12758 This has the further disadvantage that the name (and other options) are
12759 not generally visible during document editing.
12762 \begin_layout Standard
12763 The chunk style is defined as a LaTeX command, so that all text on the same
12764 line is passed to the LaTeX command
12765 \begin_inset Flex CharStyle:Code
12768 \begin_layout Plain Layout
12775 This makes it easy to parse using
12776 \begin_inset Flex CharStyle:Code
12779 \begin_layout Plain Layout
12785 , and easy to pass these options on to the listings package.
12786 The first word in a chunk section should be the chunk name, and will have
12788 \begin_inset Flex CharStyle:Code
12791 \begin_layout Plain Layout
12798 Any other words are accepted arguments to
12799 \begin_inset Flex CharStyle:Code
12802 \begin_layout Plain Layout
12813 \begin_layout Standard
12814 We set PassThru to 1 because the user is actually entering raw latex.
12817 \begin_layout Chunk
12821 \begin_layout Standard
12822 \begin_inset listings
12826 \begin_layout Plain Layout
12831 \begin_layout Plain Layout
12836 \begin_layout Plain Layout
12841 \begin_layout Plain Layout
12843 Margin First_Dynamic
12846 \begin_layout Plain Layout
12848 LeftMargin Chunk:xxx
12851 \begin_layout Plain Layout
12856 \begin_layout Plain Layout
12861 \begin_layout Plain Layout
12863 LabelString "Chunk:"
12866 \begin_layout Plain Layout
12871 \begin_layout Plain Layout
12876 \begin_layout Plain Layout
12885 \begin_layout Standard
12886 To make the label very visible we choose a larger font coloured red.
12889 \begin_layout Standard
12890 \begin_inset listings
12894 \begin_layout Plain Layout
12899 \begin_layout Plain Layout
12904 \begin_layout Plain Layout
12909 \begin_layout Plain Layout
12914 \begin_layout Plain Layout
12919 \begin_layout Plain Layout
12924 \begin_layout Plain Layout
12929 \begin_layout Plain Layout
12939 \begin_layout Subsection
12943 \begin_layout Standard
12944 We also define the Chunkref style which can be used to express cross references
12948 \begin_layout Chunk
12952 \begin_layout Standard
12953 \begin_inset listings
12957 \begin_layout Plain Layout
12959 InsetLayout Chunkref
12962 \begin_layout Plain Layout
12967 \begin_layout Plain Layout
12972 \begin_layout Plain Layout
12977 \begin_layout Plain Layout
12982 \begin_layout Plain Layout
12987 \begin_layout Plain Layout
12992 \begin_layout Plain Layout
12997 \begin_layout Plain Layout
13002 \begin_layout Plain Layout
13012 \begin_layout Section
13013 \begin_inset CommandInset label
13015 name "sec:Latex-Macros"
13022 \begin_layout Standard
13036 As noweb defines it's own
13037 \begin_inset Flex CharStyle:Code
13040 \begin_layout Plain Layout
13048 environment, we re-define the one that LyX logical markup module expects
13052 \begin_layout Chunk
13053 ./newfangle.sty,language=tex,basicstyle=
13058 \begin_layout Standard
13059 \begin_inset listings
13063 \begin_layout Plain Layout
13067 usepackage{listings}%
13070 \begin_layout Plain Layout
13077 \begin_layout Plain Layout
13084 \begin_layout Plain Layout
13100 \begin_layout Standard
13102 \begin_inset Flex CharStyle:Code
13105 \begin_layout Plain Layout
13112 \begin_inset Flex CharStyle:Code
13115 \begin_layout Plain Layout
13123 which will need renaming to
13124 \begin_inset Flex CharStyle:Code
13127 \begin_layout Plain Layout
13135 when I can do this without clashing with
13136 \begin_inset Flex CharStyle:Code
13139 \begin_layout Plain Layout
13150 \begin_layout Standard
13151 \begin_inset listings
13155 \begin_layout Plain Layout
13159 lstnewenvironment{Chunk}{
13171 \begin_layout Standard
13172 We also define a suitable
13173 \begin_inset Flex CharStyle:Code
13176 \begin_layout Plain Layout
13184 of parameters that suit the literate programming style after the fashion
13192 \begin_layout Standard
13193 \begin_inset listings
13197 \begin_layout Plain Layout
13201 lstset{numbers=left, stepnumber=5, numbersep=5pt,
13204 \begin_layout Plain Layout
13206 breaklines=false,basicstyle=
13211 \begin_layout Plain Layout
13223 \begin_layout Standard
13224 We also define a notangle-like mechanism for
13228 to LaTeX from the listing, and by which we can refer to other listings.
13230 \begin_inset Flex CharStyle:Code
13233 \begin_layout Plain Layout
13234 =<\SpecialChar \ldots{}
13240 sequence to contain LaTeX code, and include another like this chunk:
13241 \begin_inset Flex CharStyle:Code
13244 \begin_layout Plain Layout
13247 chunkref{chunkname}>
13254 \begin_inset Flex CharStyle:Code
13257 \begin_layout Plain Layout
13258 =<\SpecialChar \ldots{}
13264 is already defined to contain LaTeX code for this document --- this is
13269 document after all --- the code fragment below effectively contains the
13271 \begin_inset Flex CharStyle:Code
13274 \begin_layout Plain Layout
13281 To avoid problems with document generation, I had to declare an lstlistings
13283 \begin_inset Flex CharStyle:Code
13286 \begin_layout Plain Layout
13292 for this listing only; which in LyX was done by right-clicking the listings
13294 \begin_inset Flex CharStyle:Code
13297 \begin_layout Plain Layout
13303 \SpecialChar \menuseparator
13305 \begin_inset Flex CharStyle:Code
13308 \begin_layout Plain Layout
13317 \begin_layout Standard
13318 \begin_inset Note Note
13321 \begin_layout Plain Layout
13322 =< isn't enjoyed literally here, in a listing when the escape sequence is
13323 already defined as shown...
13324 we need to somehow escape this representation...
13332 \begin_layout Standard
13333 \begin_inset listings
13334 lstparams "escapeinside={}"
13338 \begin_layout Plain Layout
13342 lstset{escapeinside={=<}{>}}%
13350 \begin_layout Standard
13351 Although our macros will contain the @ symbol, they will be included in
13353 \begin_inset Flex CharStyle:Code
13356 \begin_layout Plain Layout
13364 section by LyX; however we keep the commented out
13365 \begin_inset Flex CharStyle:Code
13368 \begin_layout Plain Layout
13377 The listings package likes to centre the titles, but noweb titles are specially
13378 formatted and must be left aligned.
13379 The simplest way to do this turned out to be by removing the definition
13381 \begin_inset Flex CharStyle:Code
13384 \begin_layout Plain Layout
13393 This may interact badly if other listings want a regular title or caption.
13394 We remember the old maketitle in case we need it.
13397 \begin_layout Standard
13398 \begin_inset listings
13402 \begin_layout Plain Layout
13409 \begin_layout Plain Layout
13411 %somehow re-defining maketitle gives us a left-aligned title
13414 \begin_layout Plain Layout
13416 %which is extactly what our specially formatted title needs!
13419 \begin_layout Plain Layout
13427 newfangle@lst@maketitle
13432 \begin_layout Plain Layout
13448 \begin_layout Subsection
13449 \begin_inset CommandInset label
13451 name "sub:The-chunk-command"
13458 \begin_layout Standard
13459 Our chunk command accepts one argument, and calls
13460 \begin_inset Flex CharStyle:Code
13463 \begin_layout Plain Layout
13473 \begin_inset Flex CharStyle:Code
13476 \begin_layout Plain Layout
13484 will note the name, this is erased when the next
13485 \begin_inset Flex CharStyle:Code
13488 \begin_layout Plain Layout
13496 starts, so we make a note of this in
13497 \begin_inset Flex CharStyle:Code
13500 \begin_layout Plain Layout
13508 and restore in in lstlistings Init hook.
13511 \begin_layout Standard
13512 \begin_inset listings
13516 \begin_layout Plain Layout
13525 \begin_layout Plain Layout
13531 newfanglecaption},name=#1}%
13534 \begin_layout Plain Layout
13547 \begin_layout Plain Layout
13552 \begin_layout Plain Layout
13568 \begin_layout Subsubsection
13572 \begin_layout Standard
13573 Newfangle permits parameterized chunks, and requires the paramters to be
13574 specified as listings options.
13575 The newfangle script uses this, and although we don't do anything with
13576 these in the LaTeX code right now, we need to stop the listings package
13580 \begin_layout Standard
13581 \begin_inset listings
13585 \begin_layout Plain Layout
13595 newfangle@chunk@params{#1}}%
13603 \begin_layout Standard
13604 As it is common to define a chunk which then needs appending to another
13605 chunk, and annoying to have to declare a single line chunk to manage the
13606 include, we support an
13607 \begin_inset Flex CharStyle:Code
13610 \begin_layout Plain Layout
13620 \begin_layout Standard
13621 \begin_inset listings
13625 \begin_layout Plain Layout
13635 newfangle@chunk@append{#1}}%
13643 \begin_layout Subsection
13644 The noweb styled caption
13647 \begin_layout Standard
13648 We define a public macro
13649 \begin_inset Flex CharStyle:Code
13652 \begin_layout Plain Layout
13660 which can be set as a regular title.
13662 \begin_inset Flex CharStyle:Code
13665 \begin_layout Plain Layout
13674 \begin_inset Flex CharStyle:Code
13677 \begin_layout Plain Layout
13685 at the appriate time when the caption is emitted.
13688 \begin_layout Standard
13689 \begin_inset listings
13693 \begin_layout Plain Layout
13703 newfangle@caption}%
13711 \begin_layout Standard
13712 \begin_inset Float figure
13718 \begin_layout Plain Layout
13719 \begin_inset Box Boxed
13728 height_special "totalheight"
13731 \begin_layout Plain Layout
13733 \begin_inset space \qquad{}
13741 \begin_inset Formula $\equiv+$
13745 \begin_inset space \qquad{}
13749 \begin_inset space \qquad{}
13753 \begin_inset space \qquad{}
13757 \begin_inset Formula $\triangleleft$
13761 \begin_inset space \quad{}
13765 \begin_inset Formula $\triangleright$
13771 \begin_layout Plain Layout
13774 In this example, the current chunk is 22c, and therefore the third chunk
13778 \begin_layout Plain Layout
13789 \begin_layout Plain Layout
13792 The first chunk with this name (19b) occurs as the second chunk on page
13796 \begin_layout Plain Layout
13799 The previous chunk (22d) with the same name is the second chunk on page
13803 \begin_layout Plain Layout
13806 The next chunk (24d) is the fourth chunk on page 24.
13809 \begin_layout Plain Layout
13810 \begin_inset Caption
13812 \begin_layout Plain Layout
13828 The general noweb output format compactly identifies the current chunk,
13829 and references to the first chunk, and the previous and next chunks that
13830 have the same name.
13834 \begin_layout Standard
13835 This means that we need to keep a counter for each chunk-name, that we use
13836 to count chunks of the same name.
13840 \begin_layout Subsection
13844 \begin_layout Standard
13845 It would be natural to have a counter for each chunk name, but TeX would
13846 soon run out of counters
13850 \begin_layout Plain Layout
13851 \SpecialChar \ldots{}
13856 run out of counters and so I had to re-write the LaTeX macros to share
13857 a counter as described here
13862 , so we have one counter which we save at the end of a chunk and restore
13863 at the beginning of a chunk.
13866 \begin_layout Standard
13867 \begin_inset listings
13871 \begin_layout Plain Layout
13875 newcounter{newfangle@chunkcounter}%
13883 \begin_layout Standard
13884 We construct the name of this variable to store the counter to be the text
13886 \begin_inset Flex CharStyle:Code
13889 \begin_layout Plain Layout
13895 prefixed onto the chunks own name, and store it in
13896 \begin_inset Flex CharStyle:Code
13899 \begin_layout Plain Layout
13911 \begin_layout Standard
13912 We save the counter like this:
13915 \begin_layout Chunk
13919 \begin_layout Standard
13920 \begin_inset listings
13924 \begin_layout Plain Layout
13940 arabic{newfangle@chunkcounter}}%
13948 \begin_layout Standard
13949 and restore the counter like this:
13952 \begin_layout Chunk
13956 \begin_layout Standard
13957 \begin_inset listings
13961 \begin_layout Plain Layout
13965 setcounter{newfangle@chunkcounter}{
13979 \begin_layout Chunk
13983 \begin_layout Standard
13984 If there does not already exist a variable whose name is stored in
13985 \begin_inset Flex CharStyle:Code
13988 \begin_layout Plain Layout
13996 , then we know we are the first chunk with this name, and then define a
14001 \begin_layout Standard
14002 Although chunks of the same name share a common counter, they must still
14004 We use is the internal name of the listing, suffixed by the counter value.
14005 So the first chunk might be
14006 \begin_inset Flex CharStyle:Code
14009 \begin_layout Plain Layout
14015 and the second chunk be
14016 \begin_inset Flex CharStyle:Code
14019 \begin_layout Plain Layout
14028 \begin_layout Standard
14029 We also calculate the name of the previous chunk if we can (before we increment
14030 the chunk counter).
14031 If this is the first chunk of that name, then
14032 \begin_inset Flex CharStyle:Code
14035 \begin_layout Plain Layout
14044 \begin_inset Flex CharStyle:Code
14047 \begin_layout Plain Layout
14055 which the noweb package will interpret as not existing.
14058 \begin_layout Standard
14059 \begin_inset listings
14063 \begin_layout Plain Layout
14069 newfangle@caption{%
14072 \begin_layout Plain Layout
14078 chunkcount{lst-chunk-
14083 \begin_layout Plain Layout
14092 \begin_layout Plain Layout
14107 \begin_layout Plain Layout
14111 setcounter{newfangle@chunkcounter}{
14120 \begin_layout Plain Layout
14131 \begin_layout Plain Layout
14136 \begin_layout Plain Layout
14140 setcounter{newfangle@chunkcounter}{
14149 \begin_layout Plain Layout
14159 arabic{newfangle@chunkcounter}}%
14162 \begin_layout Plain Layout
14172 \begin_layout Standard
14173 After incrementing the chunk counter, we then define the name of this chunk,
14174 as well as the name of the first chunk.
14177 \begin_layout Standard
14178 \begin_inset listings
14182 \begin_layout Plain Layout
14186 addtocounter{newfangle@chunkcounter}{1}%
14189 \begin_layout Plain Layout
14205 arabic{newfangle@chunkcounter}}%
14208 \begin_layout Plain Layout
14218 arabic{newfangle@chunkcounter}}%
14221 \begin_layout Plain Layout
14237 \begin_layout Standard
14238 We now need to calculate the name of the next chunk.
14239 We do this by temporarily skipping the counter on by one; however there
14240 may not actually be another chunk with this name! We detect this by also
14241 defining a label for each chunk based on the chunkname.
14242 If there is a next chunkname then it will define a label with that name.
14243 As labels are persistent, we can at least tell the second time LaTeX is
14245 If we don't find such a defined label then we define
14246 \begin_inset Flex CharStyle:Code
14249 \begin_layout Plain Layout
14258 \begin_inset Flex CharStyle:Code
14261 \begin_layout Plain Layout
14272 \begin_layout Standard
14273 \begin_inset listings
14277 \begin_layout Plain Layout
14281 addtocounter{newfangle@chunkcounter}{1}%
14284 \begin_layout Plain Layout
14294 arabic{newfangle@chunkcounter}}%
14297 \begin_layout Plain Layout
14301 @ifundefined{r@label-
14317 \begin_layout Standard
14318 The noweb package requires that we define a
14319 \begin_inset Flex CharStyle:Code
14322 \begin_layout Plain Layout
14330 for every chunk, with a unique name, which is then used to print out it's
14334 \begin_layout Standard
14335 We also define a regular label for this chunk, as was mentioned above when
14337 \begin_inset Flex CharStyle:Code
14340 \begin_layout Plain Layout
14349 This requires LaTeX to be run at least twice after new chunk sections are
14350 added --- but noweb requried that anyway.
14353 \begin_layout Standard
14354 \begin_inset listings
14358 \begin_layout Plain Layout
14367 \begin_layout Plain Layout
14369 % define this label for every chunk instance, so we
14372 \begin_layout Plain Layout
14374 % can tell when we are the last chunk of this name
14377 \begin_layout Plain Layout
14391 \begin_layout Standard
14392 We also try and add the chunk to the list of listings, but I'm afraid we
14393 don't do very well.
14394 We want each chunk name listing once, with all of it's references.
14397 \begin_layout Standard
14398 \begin_inset listings
14402 \begin_layout Plain Layout
14406 addcontentsline{lol}{lstlisting}{
14422 \begin_layout Standard
14423 We then call the noweb output macros in the same way that noweave generates
14424 them, except that we don't need to call
14425 \begin_inset Flex CharStyle:Code
14428 \begin_layout Plain Layout
14431 nwstartdeflinemarkup
14437 \begin_inset Flex CharStyle:Code
14440 \begin_layout Plain Layout
14448 -- and if we do it messes up the output somewhat.
14451 \begin_layout Standard
14452 \begin_inset listings
14456 \begin_layout Plain Layout
14463 \begin_layout Plain Layout
14468 \begin_layout Plain Layout
14475 \begin_layout Plain Layout
14484 \begin_layout Plain Layout
14489 \begin_layout Plain Layout
14494 \begin_layout Plain Layout
14499 \begin_layout Plain Layout
14506 \begin_layout Plain Layout
14513 \begin_layout Plain Layout
14518 \begin_layout Plain Layout
14527 \begin_layout Plain Layout
14531 @ifundefined{newfangle@chunk@params}{}{%
14534 \begin_layout Plain Layout
14538 newfangle@chunk@params)%
14541 \begin_layout Plain Layout
14546 \begin_layout Plain Layout
14557 \begin_layout Plain Layout
14566 \begin_layout Plain Layout
14571 \begin_layout Plain Layout
14575 @ifundefined{newfangle@chunk@append}{}{%
14578 \begin_layout Plain Layout
14584 newfangle@chunk@append{x}
14589 \begin_layout Plain Layout
14593 newfangle@chunk@append%
14596 \begin_layout Plain Layout
14603 \begin_layout Plain Layout
14608 \begin_layout Plain Layout
14616 newfangle@chunk@append{}%
14619 \begin_layout Plain Layout
14626 \begin_layout Plain Layout
14631 \begin_layout Plain Layout
14636 \begin_layout Plain Layout
14655 \begin_layout Plain Layout
14659 nwstartdeflinemarkup%
14662 \begin_layout Plain Layout
14673 \begin_layout Plain Layout
14677 nwenddeflinemarkup%
14680 \begin_layout Plain Layout
14690 \begin_layout Standard
14691 Originally this was developed as a
14692 \begin_inset Flex CharStyle:Code
14695 \begin_layout Plain Layout
14701 aspect, in the Init hook, but it was found easier to affect the title without
14703 \begin_inset Flex CharStyle:Code
14706 \begin_layout Plain Layout
14709 lst@AddToHookExe{PreSet}
14714 is still required to set the listings name to the name passed to the
14715 \begin_inset Flex CharStyle:Code
14718 \begin_layout Plain Layout
14729 \begin_layout Standard
14730 \begin_inset listings
14734 \begin_layout Plain Layout
14738 lst@BeginAspect{newfangle}
14741 \begin_layout Plain Layout
14745 lst@Key{newfangle}{true}[t]{
14747 lstKV@SetIf{#1}{true}}
14750 \begin_layout Plain Layout
14754 lst@AddToHookExe{PreSet}{
14765 \begin_layout Plain Layout
14769 lst@AddToHook{Init}{}%
14774 \begin_layout Plain Layout
14786 \begin_layout Subsection
14790 \begin_layout Standard
14793 chunkref command which makes it easy to generate visual references to different
14797 \begin_layout Standard
14798 \begin_inset Tabular
14799 <lyxtabular version="3" rows="4" columns="2">
14801 <column alignment="center" valignment="top" width="0">
14802 <column alignment="center" valignment="top" width="0">
14804 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
14807 \begin_layout Plain Layout
14813 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
14816 \begin_layout Plain Layout
14824 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
14827 \begin_layout Plain Layout
14835 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
14838 \begin_layout Plain Layout
14842 \begin_layout Plain Layout
14858 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
14861 \begin_layout Plain Layout
14864 chunkref[3]{preamble}
14869 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
14872 \begin_layout Plain Layout
14876 \begin_layout Plain Layout
14880 chunkref[3]{preamble}
14892 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
14895 \begin_layout Plain Layout
14898 chunkref{preamble}[arg1, arg2]
14903 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
14906 \begin_layout Plain Layout
14910 \begin_layout Plain Layout
14914 chunkref{preamble}[arg1, arg2]
14932 \begin_layout Standard
14933 Chunkref can also be used within a code chunk to include another code chunk.
14934 The third optional parameter to chunkref is a comma sepatarated list of
14935 arguments, which will replace defined parameters in the chunkref.
14936 \begin_inset Note Note
14939 \begin_layout Plain Layout
14940 Darn it, if I have: =<
14942 chunkref{new-mode-tracker}[{chunks[chunk_name, "language"]},{mode}]> the
14943 inner braces (inside [ ]) cause _ to signify subscript even though we have
14952 \begin_layout Standard
14953 \begin_inset listings
14957 \begin_layout Plain Layout
14966 \begin_layout Plain Layout
14975 \begin_layout Plain Layout
14986 \begin_layout Plain Layout
14993 \begin_layout Plain Layout
15004 \begin_layout Plain Layout
15009 \begin_layout Plain Layout
15018 \begin_layout Plain Layout
15024 chunkref@i{#1}{#2}}{
15026 chunkref@i{#1}{#2}()}%
15029 \begin_layout Plain Layout
15034 \begin_layout Plain Layout
15040 chunkref@i#1#2(#3){%
15043 \begin_layout Plain Layout
15052 \begin_layout Plain Layout
15061 \begin_layout Plain Layout
15070 \begin_layout Plain Layout
15079 \begin_layout Plain Layout
15090 \begin_layout Plain Layout
15099 \begin_layout Plain Layout
15106 \begin_layout Plain Layout
15117 \begin_layout Plain Layout
15124 \begin_layout Plain Layout
15135 \begin_layout Plain Layout
15146 \begin_layout Plain Layout
15155 \begin_layout Plain Layout
15162 \begin_layout Plain Layout
15167 \begin_layout Plain Layout
15176 \begin_layout Plain Layout
15187 \begin_layout Plain Layout
15194 \begin_layout Plain Layout
15201 \begin_layout Plain Layout
15208 \begin_layout Plain Layout
15219 \begin_layout Plain Layout
15226 \begin_layout Plain Layout
15230 chunkref@args #3,)%
15233 \begin_layout Plain Layout
15240 \begin_layout Plain Layout
15249 \begin_layout Plain Layout
15254 \begin_layout Plain Layout
15259 \begin_layout Plain Layout
15268 \begin_layout Plain Layout
15278 \begin_layout Subsection
15282 \begin_layout Standard
15283 \begin_inset listings
15287 \begin_layout Plain Layout
15292 \begin_layout Plain Layout
15304 \begin_layout Chapter
15305 Extracting newfangle
15308 \begin_layout Section
15309 Extracting from Lyx
15312 \begin_layout Standard
15313 To extract from LyX, you will need to configure LyX as explained in section
15315 \begin_inset CommandInset ref
15317 reference "sub:Configuring-the-build"
15324 \begin_layout Standard
15325 \begin_inset CommandInset label
15327 name "lyx-build-script"
15331 And this lyx-build scrap will extract newfangle for me.
15334 \begin_layout Chunk
15335 lyx-build,language=sh
15338 \begin_layout Standard
15339 \begin_inset listings
15343 \begin_layout Plain Layout
15348 \begin_layout Plain Layout
15353 \begin_layout Plain Layout
15357 \begin_layout Plain Layout
15361 chunkref{lyx-build-helper}>
15364 \begin_layout Plain Layout
15366 cd $PROJECT_DIR || exit 1
15369 \begin_layout Plain Layout
15373 \begin_layout Plain Layout
15375 /usr/local/bin/newfangle -R./newfangle $TEX_SRC > ./newfangle
15378 \begin_layout Plain Layout
15380 /usr/local/bin/newfangle -R./newfangle.module $TEX_SRC > ./newfangle.module
15383 \begin_layout Plain Layout
15387 \begin_layout Plain Layout
15391 chunkref{test:helpers}>
15394 \begin_layout Plain Layout
15399 \begin_layout Plain Layout
15401 ./newfangle -Rpca-test.awk $TEX_SRC | awk -f - || exit 1
15404 \begin_layout Plain Layout
15408 chunkref{test:cromulence}>
15411 \begin_layout Plain Layout
15415 chunkref{test:escapes}>
15423 \begin_layout Standard
15424 With a lyx-build-helper
15427 \begin_layout Chunk
15428 lyx-build-helper,language=sh
15431 \begin_layout Standard
15432 \begin_inset listings
15436 \begin_layout Plain Layout
15438 PROJECT_DIR="$LYX_r"
15441 \begin_layout Plain Layout
15443 LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
15446 \begin_layout Plain Layout
15451 \begin_layout Plain Layout
15453 TEX_SRC="$TEX_DIR/$LYX_i"
15461 \begin_layout Section
15462 Extracting from the command line
15465 \begin_layout Standard
15466 First you will need the tex output, then you can extract:
15469 \begin_layout Chunk
15470 lyx-build-manual,language=sh
15473 \begin_layout Standard
15474 \begin_inset listings
15478 \begin_layout Plain Layout
15480 lyx -e latex newfangle.lyx
15483 \begin_layout Plain Layout
15485 newfangle -R./newfangle newfangle.tex > ./newfangle
15488 \begin_layout Plain Layout
15490 newfangle -R./newfangle.module newfangle.tex > ./newfangle.module
15498 \begin_layout Section
15502 \begin_layout Chunk
15506 \begin_layout Standard
15507 \begin_inset listings
15511 \begin_layout Plain Layout
15516 \begin_layout Plain Layout
15521 \begin_layout Plain Layout
15526 \begin_layout Plain Layout
15531 \begin_layout Plain Layout
15536 \begin_layout Plain Layout
15541 \begin_layout Plain Layout
15546 \begin_layout Plain Layout
15550 \begin_layout Plain Layout
15555 \begin_layout Plain Layout
15560 \begin_layout Plain Layout
15565 \begin_layout Plain Layout
15570 \begin_layout Plain Layout
15575 \begin_layout Plain Layout
15580 \begin_layout Plain Layout
15594 \begin_layout Chapter
15598 \begin_layout Chunk
15599 tests-sub,language=,params=THING;colour
15602 \begin_layout Standard
15603 \begin_inset listings
15607 \begin_layout Plain Layout
15609 I see a ${THING} of
15612 \begin_layout Plain Layout
15617 \begin_layout Plain Layout
15621 chunkref{tests-sub-sub}(${colour})>
15629 \begin_layout Chunk
15630 tests-sub-sub,params=colour
15633 \begin_layout Standard
15634 \begin_inset listings
15638 \begin_layout Plain Layout
15640 a funny shade of ${colour}
15648 \begin_layout Chunk
15652 \begin_layout Standard
15653 \begin_inset listings
15657 \begin_layout Plain Layout
15659 What do you see? "=<
15661 chunkref{tests-sub}(joe, red)>"
15664 \begin_layout Plain Layout
15674 \begin_layout Standard
15675 Should generate output:
15678 \begin_layout Chunk
15682 \begin_layout Standard
15683 \begin_inset listings
15687 \begin_layout Plain Layout
15689 What do you see? "I see a joe of
15692 \begin_layout Plain Layout
15697 \begin_layout Plain Layout
15699 looking closer a funny shade of red"
15702 \begin_layout Plain Layout
15707 \begin_layout Plain Layout