1 #LyX 1.6.4 created this file. For more info see http://www.lyx.org/
8 %\definecolor{darkgreen}{rgb}{0,0.5,0}
9 \lstset{numbers=left, stepnumber=1, numbersep=5pt, breaklines=false,
10 basicstyle=\footnotesize\ttfamily,
11 %keywordstyle=\color{darkgreen},
12 numberstyle=\tiny,language=C,columns=fullflexible,
15 \use_default_options true
24 \font_typewriter default
25 \font_default_family default
32 \paperfontsize default
35 \pdf_title "Newfangle"
36 \pdf_author "Sam Liddicott"
37 \pdf_subject "Literate Programing"
38 \pdf_keywords "notangle noweb noweave literate programming cweb"
40 \pdf_bookmarksnumbered false
41 \pdf_bookmarksopen false
42 \pdf_bookmarksopenlevel 1
54 \paperorientation portrait
57 \paragraph_separation skip
59 \quotes_language english
62 \paperpagestyle default
63 \tracking_changes false
83 \begin_layout Chapter*
87 \begin_layout Standard
92 is a tool for newfangled literate programming.
93 Newfangled is defined as
95 New and often needlessly novel
104 \begin_layout Standard
105 In this case, newfangled means yet another new and improved method for literate
109 \begin_layout Standard
114 has a long history starting with the great
118 whose literate programming tools seem to make use of as many escaped abbreviati
119 ons for semantic markup as TeX itself.
122 \begin_layout Standard
131 set of tools (notangle, noweave and noroots) and helpfully reduced the
132 amount of magic character sequences to just
133 \begin_inset Flex CharStyle:Code
136 \begin_layout Plain Layout
143 \begin_inset Flex CharStyle:Code
146 \begin_layout Plain Layout
152 , and in doing so brought the wonders of literate programming within my
156 \begin_layout Standard
157 Using LyX for LaTeX editing, I had various troubles with the noweb tools,
158 some of which were my fault, some of which were noweb's fault and some
159 of which were LyX's fault.
162 \begin_layout Standard
167 generally brought literate programming to the masses through removing some
168 of the complexity of the original literate programming, but this would
169 be of no advantage to me if the LyX / LaTeX combination brought more complicati
173 \begin_layout Standard
178 was thus born --- as an awk replacement for notangle, adding some important
179 features, like better integration with LyX and LaTeX, multiple output format
180 conversions, and fixing notangle bugs like indentation when using -L for
184 \begin_layout Standard
185 Significantly, newfangle is just one program which replaces various programs
187 Specifically noweave is done away with and implemented directly as LaTeX
188 macros, and noroots is implemented as a function of the untangler
195 \begin_layout Standard
196 Newfangle is written in awk for portability reasons, awk being available
198 A python conversion will probably be attempted for the benefit of LyX.
199 (Hasn't anyone implemented awk in python yet?)
202 \begin_layout Section*
206 \begin_layout Enumerate
207 ^^ is always going to be a problem, see texbytopic 1.2.2 (Work out what I
211 \begin_layout Enumerate
212 copy over up to date Makefile guide from noweb-lyx document
215 \begin_layout Enumerate
216 Make chunk-name settings only apply to chunks with that name
219 \begin_layout Enumerate
220 indent of multi-line chunks may be mode dependant (i.e.
221 not in string literals)
224 \begin_layout Enumerate
225 support chunk-param usage =<
230 \begin_layout Enumerate
231 trim spaces from param
234 \begin_layout Enumerate
235 add support for other commands in =<...>, starting with
237 label which takes the line-number within the chunk, and maybe should also
238 take the chunk name/page
241 \begin_layout Enumerate
242 cant have listing inside a ruled box
245 \begin_layout Enumerate
246 when a parameterized chunk is included as well as the #line emission, say
247 what the paremeters were for that invocation.
250 \begin_layout Enumerate
251 with 2 macro expansions on one line ${} ${} the first is too greedy and
255 \begin_layout Enumerate
258 chunkref[3]{preamble} to include a certain chunk needs to work in newfangle.awk
259 instead of failing to be recognized at all
262 \begin_layout Enumerate
263 make in-listins labels track the chunk ref too, and make
265 chunref{[2],thing}> resolve to 41c (or d, or whatever chunk the 2nd chunk
269 \begin_layout Enumerate
272 chunkref in text needs a trailing space maybe, it keeps butting up to the
276 \begin_layout Enumerate
277 because the white-space indent is output by the parent chunk, the #line
278 is that of the parent chunk.
279 White space indents must be passed to the child chunk
282 \begin_layout Chapter*
286 \begin_layout Standard
287 \begin_inset CommandInset label
293 Newfangle is licensed under the GPL 3
294 \begin_inset CommandInset citation
301 This doesn't mean that you can't use or distribute newfangle with sources
302 of an incompatible license, but it means you must make the source of newfangle
307 gpl3-copyright,language=
310 \begin_layout Standard
311 \begin_inset listings
315 \begin_layout Plain Layout
317 #newfangle - fully featured notangle replacement in awk
320 \begin_layout Plain Layout
325 \begin_layout Plain Layout
327 #Copyright (C) Sam Liddicott 2009
330 \begin_layout Plain Layout
335 \begin_layout Plain Layout
337 #This program is free software: you can redistribute it and/or modify
340 \begin_layout Plain Layout
342 #it under the terms of the GNU General Public License as published by
345 \begin_layout Plain Layout
347 #the Free Software Foundation, either version 3 of the License, or
350 \begin_layout Plain Layout
352 #(at your option) any later version.
355 \begin_layout Plain Layout
360 \begin_layout Plain Layout
362 #This program is distributed in the hope that it will be useful,
365 \begin_layout Plain Layout
367 #but WITHOUT ANY WARRANTY; without even the implied warranty of
370 \begin_layout Plain Layout
372 #MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
376 \begin_layout Plain Layout
378 #GNU General Public License for more details.
381 \begin_layout Plain Layout
386 \begin_layout Plain Layout
388 #You should have received a copy of the GNU General Public License
391 \begin_layout Plain Layout
393 #along with this program.
394 If not, see <http://www.gnu.org/licenses/>.
402 \begin_layout Standard
403 \begin_inset CommandInset toc
404 LatexCommand tableofcontents
415 \begin_layout Chapter
419 \begin_layout Standard
420 Newfangle is a replacement for noweb, which consists of
421 \begin_inset Flex CharStyle:Code
424 \begin_layout Plain Layout
431 \begin_inset Flex CharStyle:Code
434 \begin_layout Plain Layout
441 \begin_inset Flex CharStyle:Code
444 \begin_layout Plain Layout
453 \begin_layout Standard
455 \begin_inset Flex CharStyle:Code
458 \begin_layout Plain Layout
465 \begin_inset Flex CharStyle:Code
468 \begin_layout Plain Layout
474 it can read multiple named files, or from stdin.
477 \begin_layout Section
481 \begin_layout Standard
482 The -r option causes newfangle to behave like noroots.
485 \begin_layout LyX-Code
486 newfangle -r filename.tex
489 \begin_layout Standard
490 will print out the newfangle roots of a tex file.
494 \begin_layout Standard
496 \begin_inset Flex CharStyle:Code
499 \begin_layout Plain Layout
505 command, the roots are not enclosed in
506 \begin_inset Flex CharStyle:Code
509 \begin_layout Plain Layout
515 , unless at least one of the roots is defined using the
516 \begin_inset Flex CharStyle:Code
519 \begin_layout Plain Layout
526 \begin_inset Flex CharStyle:Code
529 \begin_layout Plain Layout
538 \begin_layout Standard
539 Also, unlike noroots, it prints out all roots --- not just those that are
541 I find that a root not being used, doesn't make it particularly top level.
542 My convention is that top level roots to be extracted begin with
543 \begin_inset Flex CharStyle:Code
546 \begin_layout Plain Layout
552 and have the form of a filename.
555 \begin_layout Section
559 \begin_layout Standard
561 \begin_inset Flex CharStyle:Code
564 \begin_layout Plain Layout
571 \begin_inset Flex CharStyle:Code
574 \begin_layout Plain Layout
580 options are supported.
583 \begin_layout Standard
584 The standard way to extract a file would be:
587 \begin_layout LyX-Code
588 newfangle -R./Makefile.inc newfangle.tex > ./Makefile.inc
591 \begin_layout Standard
593 \begin_inset Flex CharStyle:Code
596 \begin_layout Plain Layout
603 \begin_inset Flex CharStyle:Code
606 \begin_layout Plain Layout
612 option does not break indenting; also the
613 \begin_inset Flex CharStyle:Code
616 \begin_layout Plain Layout
622 option does not interrupt (and break) multi-line C macros --- or indeed
623 any line ending with a backslash.
624 This does mean that sometimes the compiler might calculate the source line
625 wrongly when generating error messages in such cases, but there isn't any
626 other way around if multi-line macros include other chunks.
629 \begin_layout Section
630 Formatting source in LaTeX
633 \begin_layout Standard
634 The noweave replacement is a set of LaTeX macros dependant upon
638 , and which can be included with:
641 \begin_layout LyX-Code
644 usepackage{newfangle.sty}
647 \begin_layout Standard
648 The LaTeX macros are shown in section
649 \begin_inset CommandInset ref
651 reference "sec:Latex-Macros"
655 , and are part of a LyX module file
656 \begin_inset Flex CharStyle:Code
659 \begin_layout Plain Layout
665 , which automatically includes the macros in the document pre-amble when
666 the newfangle LyX module is used.
669 \begin_layout Standard
670 Because the noweave replacement is impemented in LaTeX, there is no processing
671 stage required before running the
672 \begin_inset Flex CharStyle:Code
675 \begin_layout Plain Layout
682 LaTeX may need running two or more times, so that the code chunk references
683 can be fully calculated.
686 \begin_layout Standard
688 \begin_inset Flex CharStyle:Code
691 \begin_layout Plain Layout
697 package is required as it is used for formatting the code chunk captions
700 \begin_layout Standard
702 \begin_inset Flex CharStyle:Code
705 \begin_layout Plain Layout
711 package is also required, as it is used for formatting the code chunks
715 \begin_layout Standard
717 \begin_inset Flex CharStyle:Code
720 \begin_layout Plain Layout
726 package is also required.
729 \begin_layout Chapter
730 Literate Programming with Newfangle
733 \begin_layout Standard
735 Should really follow on from a part-0 explanation of what literate programming
739 \begin_layout Chapter
740 Using Newfangle with LyX
743 \begin_layout Section
747 \begin_layout Subsection
748 Installing the LyX module
751 \begin_layout Standard
753 \begin_inset Flex CharStyle:Code
756 \begin_layout Plain Layout
762 to your LyX layouts directory, which for unix users will be
763 \begin_inset Flex CharStyle:Code
766 \begin_layout Plain Layout
775 \begin_layout Standard
776 You will need to reconfigure LyX by clicking Tools\SpecialChar \menuseparator
777 Reconfigure, and then
781 \begin_layout Subsection
782 \begin_inset CommandInset label
784 name "sub:Configuring-the-build"
788 Configuring the build script
791 \begin_layout Standard
792 Make sure you don't have a conversion defined for Lyx → Program
795 \begin_layout Standard
796 From the menu Tools\SpecialChar \menuseparator
797 Preferences, add a conversion from Latex(Plain) → Program
801 \begin_layout LyX-Code
802 set -x ; newfangle -Rlyx-build $$i |
805 \begin_layout LyX-Code
806 env LYX_b=$$b LYX_i=$$i LYX_o=$$o LYX_p=$$p LYX_r=$$r bash
809 \begin_layout Standard
810 (But don't cut-n-paste it from this document or you'll be pasting a multi-line
811 string which will break your lyx preferences file).
815 \begin_layout Standard
816 I hope that one day, LyX will set these into the environment when calling
820 \begin_layout Standard
821 You may also want to consider adding options to this conversion\SpecialChar \ldots{}
825 \begin_layout LyX-Code
826 parselog=/usr/share/lyx/scripts/listerrors
829 \begin_layout Standard
830 \SpecialChar \ldots{}
831 but if you do you will lose your stderr
835 \begin_layout Plain Layout
836 There is some bash plumbing to get a copy of stderr but this footnote is
845 \begin_layout Standard
846 Now, a shell script chunk called
847 \begin_inset Flex CharStyle:Code
850 \begin_layout Plain Layout
856 will be extracted and run whenever you choose the Document\SpecialChar \menuseparator
861 \begin_layout Standard
862 The lyx-build script for this document is in section
863 \begin_inset CommandInset ref
865 reference "lyx-build-script"
869 and on a unix system will extract
870 \begin_inset Flex CharStyle:Code
873 \begin_layout Plain Layout
880 \begin_inset Flex CharStyle:Code
883 \begin_layout Plain Layout
892 \begin_layout Subsection
893 Preparing your Lyx document
896 \begin_layout Standard
897 It is not necessary to base your literate document on any of the original
898 LyX literate classes; so select a regular class for your document type.
901 \begin_layout Standard
917 \begin_layout Standard
918 In the drop-down style listbox you should notice a new style defined, called
926 \begin_layout Standard
927 When you wish to insert a literate chunk, you enter it's plain name in the
928 Chunk style, instead of the older method that used
929 \begin_inset Flex CharStyle:Code
932 \begin_layout Plain Layout
939 Following the chunk name, you insert a listing with: Insert\SpecialChar \menuseparator
943 \begin_layout Standard
944 Inside the white listing box you can type (or paste using shift+ctrl+V)
946 There is not need to use ctrl+enter at the end of lines as with some older
947 LyX literate techniques --- just press enter as normal.
950 \begin_layout Subsubsection
951 Customising the listing appearance
954 \begin_layout Standard
955 In the final document, the code is formatted using the
960 The chunk style doesn't just define the chunk name, but can also define
961 any other chunk options supported by the lstlistings package
962 \begin_inset Flex CharStyle:Code
965 \begin_layout Plain Layout
974 In fact, what you type in the chunk style is raw latex.
975 If you want to set the chunk language without having to right-click the
977 \begin_inset Flex CharStyle:Code
980 \begin_layout Plain Layout
986 after the chunk name.
989 \begin_layout Standard
990 Of course you can do this by editing the listings box advanced properties
991 by right-clicking on the listings box, but that takes longer, and you can't
992 see at-a-glance what the advanced settings are while editing the document;
993 also advanced settings apply only to that box --- the chunk settings apply
994 through the rest of the document
998 \begin_layout Plain Layout
999 It ought to apply only to subsequent chunks of the same name.
1006 \begin_inset Note Note
1009 \begin_layout Plain Layout
1010 So make sure they only apply to chunks of that name
1018 \begin_layout Subsubsection
1019 Global customisations
1022 \begin_layout Standard
1027 is used to set the code chunks, it's
1028 \begin_inset Flex CharStyle:Code
1031 \begin_layout Plain Layout
1039 command can be used in the pre-amble to set some document wide settings.
1042 \begin_layout Standard
1043 If your source has many words with long sequences of capital letters, then
1045 \begin_inset Flex CharStyle:Code
1048 \begin_layout Plain Layout
1049 columns=fullflexible
1054 may be a good idea, or the capital letters will get crowded.
1055 (I think lstlistings ought to use a slightly smaller font for captial letters
1056 so that they still fit).
1059 \begin_layout Standard
1061 \begin_inset Flex CharStyle:Code
1064 \begin_layout Plain Layout
1072 looks more normal for code, but has no bold (unless luximono is used, but
1073 it doesn't work for me); so I use
1074 \begin_inset Flex CharStyle:Code
1077 \begin_layout Plain Layout
1087 \begin_inset Flex CharStyle:Code
1090 \begin_layout Plain Layout
1099 \begin_inset Flex CharStyle:Code
1102 \begin_layout Plain Layout
1103 columns=fullflexible
1108 is used or the wrong letter spacing is used.
1111 \begin_layout Standard
1112 In my LeTeX pre-amble I usually specialise my code format with:
1116 document-preamble,language=tex
1119 \begin_layout Standard
1120 \begin_inset listings
1124 \begin_layout Plain Layout
1131 \begin_layout Plain Layout
1135 definecolor{darkgreen}{rgb}{0,0.5,0}
1138 \begin_layout Plain Layout
1142 lstset{numbers=left, stepnumber=5, numbersep=5pt, breaklines=false,
1145 \begin_layout Plain Layout
1154 \begin_layout Plain Layout
1161 \begin_layout Plain Layout
1165 tiny,language=C,columns=fullflexible,
1168 \begin_layout Plain Layout
1170 numberfirstline=true
1173 \begin_layout Plain Layout
1183 \begin_layout Chapter
1184 Newfangle with Makefiles
1187 \begin_layout Standard
1188 \begin_inset Note Note
1191 \begin_layout Plain Layout
1192 This chapter needs revising
1198 \begin_inset Note Greyedout
1201 \begin_layout Plain Layout
1202 This chapter needs revising
1207 Here we describe a Makefile.inc that you can include in your own Makefiles,
1208 or glue as a recursive make to other projects.
1211 \begin_layout Standard
1212 The Makefile.inc described here was put together for a Samba4 vfs module,
1213 but can be used in any Make project, including automake projects.
1216 \begin_layout Section
1217 A word about makefiles formats
1220 \begin_layout Standard
1221 Whitespace formatting is very important in a Makefile.
1222 The first character of each command line must be a TAB.
1225 \begin_layout LyX-Code
1226 target: pre-requisite
1227 \begin_inset Newline newline
1231 \begin_inset Newline newline
1237 \begin_layout Standard
1238 But a TAB is pretty hard to enter into most of the Lyx formats and insets
1240 An alternative is to use a semi-colon after the pre-requisite, and a backslash
1241 at the end of each line (except the last).
1242 Then any whitespace (or none) can prefix each action.
1245 \begin_layout LyX-Code
1246 target: pre-requisite ;
1249 \begin_inset Newline newline
1255 \begin_inset Newline newline
1261 \begin_layout Standard
1262 This is the style that we use and it works pretty well for GNU make at least.
1265 \begin_layout Standard
1266 We also adopt a convention that code chunks whose names beginning with ./
1267 should always be automatically extracted from the document.
1268 Code chunks whose names do not begin with ./ are for internal reference.
1269 (This doesn't prevent such chunks from being extracted directly).
1272 \begin_layout Section
1273 Boot-strapping the extraction
1276 \begin_layout Subsection
1280 \begin_layout Standard
1281 \begin_inset CommandInset label
1283 name "sub:Bootstrap-Using-a-Makefile"
1287 It seems convenient to have the makefile extract or update the C source
1288 files as part of it's operation.
1289 It also seems convenient to have the makefile itself extracted from this
1293 \begin_layout Standard
1294 It would also be convenient to have the code to extract the makefile from
1295 this document to also be part of this document, however we have to start
1296 somewhere and this unfortunately requires us to type at least a few words
1297 by hand to start things off.
1300 \begin_layout Standard
1301 Therefore we will have a minimal root fragment, which, when extracted, can
1302 cope with extracting the rest of the source.
1303 perhaps with this shell script, which could be called
1308 \begin_inset Note Note
1311 \begin_layout Plain Layout
1312 FIX THIS CHUNK AND TEST IT
1324 \begin_layout Standard
1325 \begin_inset listings
1329 \begin_layout Plain Layout
1334 \begin_layout Plain Layout
1338 \begin_layout Plain Layout
1340 MAKE_SRC="${1:-${NW_LYX:-../../noweb-lyx/noweb-lyx3.lyx}}"
1343 \begin_layout Plain Layout
1345 MAKE_SRC=`dirname "$MAKE_SRC"`/`basename "$MAKE_SRC" .lyx`
1348 \begin_layout Plain Layout
1350 NOWEB_SRC="${2:-${NOWEB_SRC:-$MAKE_SRC.lyx}}"
1353 \begin_layout Plain Layout
1355 lyx -e latex $MAKE_SRC
1358 \begin_layout Plain Layout
1362 \begin_layout Plain Layout
1364 newfangle -R./Makefile.inc ${MAKE_SRC}.tex
1369 \begin_layout Plain Layout
1371 | sed "/NEWFANGLE_SOURCE=/s/^/#/;T;aNOWEB_SOURCE=$NEWFANGLE_SRC"
1376 \begin_layout Plain Layout
1378 | cpif ./Makefile.inc
1381 \begin_layout Plain Layout
1385 \begin_layout Plain Layout
1387 make -f ./Makefile.inc newfangle_sources
1395 \begin_layout Standard
1396 The general Makefile can be invoked with
1400 and can also be included into any automake file to automatically re-generate
1404 \begin_layout Standard
1409 can be extracted with this command:
1412 \begin_layout LyX-Code
1413 lyx -e latex newfangle.lyx &&
1418 \begin_layout LyX-Code
1419 newfangle newfangle.lyx > ./autoboot
1422 \begin_layout Standard
1423 This looks simple enough, but as mentioned, newfangle has to be had from
1424 somewhere before it can be extracted.
1427 \begin_layout Subsection
1428 \begin_inset Note Note
1431 \begin_layout Plain Layout
1432 MERGE THIS WITH THE SECTIONS OF THIS DOCUMENT
1437 \SpecialChar \ldots{}
1441 \begin_layout Standard
1442 When the lyx-build chunk is executed, the current directory will be a temporary
1444 \begin_inset Flex CharStyle:Code
1447 \begin_layout Plain Layout
1453 will refer to the tex file in this temporary directory.
1454 This is unfortunate as our makefile wants to run from the project directory
1455 where the Lyx file is kept.
1458 \begin_layout Standard
1459 We can extract the project directory from $$r, and derive the probable Lyx
1460 filename from the noweb file that Lyx generated.
1467 \begin_layout Standard
1468 \begin_inset listings
1472 \begin_layout Plain Layout
1474 PROJECT_DIR="$LYX_r"
1477 \begin_layout Plain Layout
1479 LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
1482 \begin_layout Plain Layout
1487 \begin_layout Plain Layout
1489 TEX_SRC="$TEX_DIR/$LYX_i"
1497 \begin_layout Standard
1498 And then we can define a lyx-build fragment similar to the autoboot fragment
1505 \begin_layout Standard
1506 \begin_inset listings
1510 \begin_layout Plain Layout
1515 \begin_layout Plain Layout
1519 chunkref{lyx-build-helper}>
1522 \begin_layout Plain Layout
1524 cd $PROJECT_DIR || exit 1
1527 \begin_layout Plain Layout
1531 \begin_layout Plain Layout
1533 #/usr/bin/newfangle -filter ./notanglefix-filter
1538 \begin_layout Plain Layout
1540 # -R./Makefile.inc "../../noweb-lyx/noweb-lyx3.lyx"
1545 \begin_layout Plain Layout
1547 # | sed '/NOWEB_SOURCE=/s/=.*/=samba4-dfs.lyx/'
1552 \begin_layout Plain Layout
1557 \begin_layout Plain Layout
1562 \begin_layout Plain Layout
1564 #make -f ./Makefile.inc newfangle_sources
1572 \begin_layout Section
1576 \begin_layout Subsection
1577 Including Makefile.inc
1580 \begin_layout Standard
1581 \begin_inset CommandInset label
1583 name "sub:Keeping-extracted-files"
1587 Makefile.inc will cope with extracting all the other source files from this
1588 document and keeping them up to date.
1592 \begin_layout Standard
1593 It may also be included by a Makefile or Makefile.am defined in a Lyx document
1594 to automatically deal with the extraction of source files and documents.
1597 \begin_layout Standard
1598 A makefile has two parts; variables must be defined before the targets that
1606 \begin_layout Standard
1607 \begin_inset listings
1611 \begin_layout Plain Layout
1615 chunkref{Makefile.inc-vars}>
1618 \begin_layout Plain Layout
1622 chunkref{Makefile.inc-targets}>
1630 \begin_layout Standard
1632 \begin_inset Flex CharStyle:Code
1635 \begin_layout Plain Layout
1641 to hold the name of this Lyx file.
1648 \begin_layout Standard
1649 \begin_inset listings
1653 \begin_layout Plain Layout
1658 \begin_layout Plain Layout
1660 LITERATE_SOURCE=$(LYX_SOURCE)
1668 \begin_layout Subsection
1669 Recursive use of Makefile.inc
1672 \begin_layout Standard
1673 The makefile glue described here is used when building Samba4 vfs modules.
1676 \begin_layout Standard
1677 If you are defining a module of an existing program you may find it easier
1678 to use a slight recursive make instead of including the makefile directly.
1679 This way there is less chance of definitions in Makefile.inc interfering
1680 with definitions in the main makefile, or with definitions in other Makefile.inc
1681 from other noweb modules.
1684 \begin_layout Standard
1685 The glue works by adding a .PHONY target to call the recursive make, and
1686 adding this target as an additional pre-requisite to the existing targets.
1689 \begin_layout Standard
1690 In this example, the existing build system already has a build target for
1692 \begin_inset Flex CharStyle:Code
1695 \begin_layout Plain Layout
1701 , so we just add another pre-requisite to that.
1703 \begin_inset Flex CharStyle:Code
1706 \begin_layout Plain Layout
1712 as a pre-requisite, the stamp file's modified time indicating when all
1713 sources were extracted.
1720 \begin_layout Standard
1721 \begin_inset listings
1725 \begin_layout Plain Layout
1727 $(example_srcdir)/example.o: $(example_srcdir)/example.tex.stamp
1735 \begin_layout Standard
1736 The target for this new pre-requisite is generated by a recursive make using
1737 Makefile.inc which will make sure that the source is up to date, before
1738 it is built by the main projects makefile.
1745 \begin_layout Standard
1746 \begin_inset listings
1750 \begin_layout Plain Layout
1752 $(example_srcdir)/example.tex.stamp: $(example_srcdir)/example.tex ;
1757 \begin_layout Plain Layout
1759 cd $(example_srcdir) &&
1764 \begin_layout Plain Layout
1766 $(MAKE) -f Makefile.inc newfangle_sources
1774 \begin_layout Standard
1775 We can do similar glue for the docs, clean and distclean targets.
1776 In this example our build system is using a double colon for these targets,
1777 so we use the same in our glue.
1784 \begin_layout Standard
1785 \begin_inset listings
1789 \begin_layout Plain Layout
1794 \begin_layout Plain Layout
1796 .PHONY: docs_example
1799 \begin_layout Plain Layout
1801 docs_example:: ; cd $(example_srcdir) &&
1806 \begin_layout Plain Layout
1808 $(MAKE) -f Makefile.inc docs
1811 \begin_layout Plain Layout
1815 \begin_layout Plain Layout
1817 clean:: clean_example
1820 \begin_layout Plain Layout
1822 .PHONEY: clean_example
1825 \begin_layout Plain Layout
1827 clean_example: ; cd $(example_srcdir) &&
1832 \begin_layout Plain Layout
1834 $(MAKE) -f Makefile.inc clean
1837 \begin_layout Plain Layout
1841 \begin_layout Plain Layout
1843 distclean:: distclean_example
1846 \begin_layout Plain Layout
1848 .PHONY: distclean_example
1851 \begin_layout Plain Layout
1853 distclean_example: ; cd $(example_srcdir) &&
1858 \begin_layout Plain Layout
1860 $(MAKE) -f Makefile.inc distclean
1868 \begin_layout Standard
1869 We could do similarly for install targets to install the generated docs.
1872 \begin_layout Subsection
1873 \begin_inset CommandInset label
1875 name "sub:Converting-from-Lyx"
1879 Converting from Lyx to LaTeX
1882 \begin_layout Standard
1883 The first stage will always be to convert the Lyx file to a LaTeX file;
1884 this must be so not only because newfangle needs to to run on a TeX file,
1885 but also because the Lyx command
1887 server-goto-file-line
1891 \begin_layout Plain Layout
1894 server-goto-file-line
1896 is used to position the Lyx cursor at the compiler errors.
1903 insists that the line number provided is a line in the TeX file, and always
1904 reverse maps this to derive the line in the Lyx docment.
1905 \begin_inset Note Note
1908 \begin_layout Plain Layout
1909 The tex file should probably be an automake extra dist sources or something,
1910 so that it gets produced and packaged by make dist
1918 \begin_layout Standard
1919 The command [[lyx -e literate noweb-lyx.lyx]] will produce [[noweb-lyx.nw]]
1920 a tex file, so we define the noweb target to be the same as the Lyx file
1921 but with the .nw extension.
1928 \begin_layout Standard
1929 \begin_inset listings
1933 \begin_layout Plain Layout
1935 TEX_SOURCE=$(LYX_SOURCE:.lyx=.tex)
1944 Makefile.inc-targets
1947 \begin_layout Standard
1948 \begin_inset listings
1952 \begin_layout Plain Layout
1954 $(TEX_SOURCE): $(LYX_SOURCE) ;
1959 \begin_layout Plain Layout
1964 \begin_layout Plain Layout
1966 clean_tex: ; rm -f -- $(TEX_SOURCE)
1974 \begin_layout Subsection
1975 Extracting Program Source
1978 \begin_layout Standard
1979 The program source is extracted using newfangle, which is designed to operate
1980 on a LaTeX document.
1988 \begin_layout Standard
1989 \begin_inset listings
1993 \begin_layout Plain Layout
1995 NEWFANGLE_SOURCE=$(TEX_SOURCE)
2003 \begin_layout Standard
2004 The Lyx document can result in any number of source documents, but not all
2005 of these will be changed each time the Lyx document is updated.
2006 We certainly don't want to update the timestamps of these files and cause
2007 the whole source tree to be recompiled just because the Lyx document was
2012 \begin_layout Standard
2013 To solve this problem we use a stamp file which is always updated each time
2014 the sources are extracted from the LaTeX document.
2015 If the stamp file is older than the LaTeX document, then we can make an
2016 attempt to re-extract the sources.
2023 \begin_layout Standard
2024 \begin_inset listings
2028 \begin_layout Plain Layout
2030 NEWFANGLE_SOURCE_STAMP=$(NEWFANGLE_SOURCE).stamp
2039 Makefile.inc-targets
2042 \begin_layout Standard
2043 \begin_inset listings
2047 \begin_layout Plain Layout
2049 $(NEWFANGLE_SOURCE_STAMP): $(NEWFANGLE_SOURCE)
2054 \begin_layout Plain Layout
2056 $(NEWFANGLE_SOURCES) ;
2061 \begin_layout Plain Layout
2063 echo > $(NEWFANGLE_SOURCE_STAMP)
2066 \begin_layout Plain Layout
2068 clean_stamp: ; rm -f $(NEWFANGLE_SOURCE_STAMP)
2071 \begin_layout Plain Layout
2081 \begin_layout Subsection
2082 Extracting C sources
2085 \begin_layout Standard
2087 \begin_inset Flex CharStyle:Code
2090 \begin_layout Plain Layout
2096 to hold the names of all the C source files defined in this document.
2097 We compute this only once, by means of := in assignent.
2098 The sed deletes the any <
2099 \begin_inset space \hspace*{}
2104 \begin_inset space \hspace*{}
2108 > which may surround the roots names (for noroots compatibility).
2112 \begin_layout Standard
2113 As we use chunk names beginning with ./ to denote top level fragments that
2114 should be extracted, we filter out all fragments that do not begin with
2122 \begin_layout Standard
2123 \begin_inset listings
2127 \begin_layout Plain Layout
2136 \begin_layout Plain Layout
2138 NEWFANGLE_SOURCES:=$(shell
2143 \begin_layout Plain Layout
2145 newfangle -r $(NEWFANGLE_SOURCE) |
2150 \begin_layout Plain Layout
2152 sed -e 's/^[<][<]//;s/[>][>]$$//;/^$(NEWFANGLE_PREFIX)/!d'
2157 \begin_layout Plain Layout
2159 -e 's/^$(NEWFANGLE_PREFIX)/
2166 \begin_layout Plain Layout
2177 Makefile.inc-targets
2180 \begin_layout Standard
2181 \begin_inset listings
2185 \begin_layout Plain Layout
2187 .PHONY: echo_newfangle_sources
2190 \begin_layout Plain Layout
2192 echo_newfangle_sources: ; @echo $(NEWFANGLE_SOURCES)
2200 \begin_layout Standard
2201 We define a convenient target called
2202 \begin_inset Flex CharStyle:Code
2205 \begin_layout Plain Layout
2211 to re-extract the source if the LaTeX file has been updated.
2215 Makefile.inc-targets
2218 \begin_layout Standard
2219 \begin_inset listings
2223 \begin_layout Plain Layout
2225 .PHONY: newfangle_sources
2228 \begin_layout Plain Layout
2230 newfangle_sources: $(NEWFANGLE_SOURCE_STAMP)
2238 \begin_layout Standard
2239 And also a convenient target to remove extracted sources.
2243 Makefile.inc-targets
2246 \begin_layout Standard
2247 \begin_inset listings
2251 \begin_layout Plain Layout
2253 .PHONY: clean_newfangle_sources
2256 \begin_layout Plain Layout
2258 clean_newfangle_sources: ;
2263 \begin_layout Plain Layout
2265 rm -f -- $(NEWFANGLE_SOURCE_STAMP) $(NEWFANGLE_SOURCES)
2273 \begin_layout Standard
2275 \begin_inset Flex CharStyle:Code
2278 \begin_layout Plain Layout
2284 macro takes 4 arguments: the filename (1), some extensions to match (2)
2285 and a some shell command to return if the filename matches the exentions
2293 \begin_layout Standard
2294 \begin_inset listings
2298 \begin_layout Plain Layout
2300 if_extension=$(if $(findstring $(suffix $(1)),$(2)),$(3),$(4))
2308 \begin_layout Standard
2309 For some source files like C files, we want to output the line number and
2310 filename of the original LaTeX document from which the source came.
2313 \begin_layout Standard
2314 To make this easier we define the file extensions for which we want to do
2322 \begin_layout Standard
2323 \begin_inset listings
2327 \begin_layout Plain Layout
2337 \begin_layout Standard
2338 We can then use the if_extensions macro to define a macro which expands
2340 \begin_inset Flex CharStyle:Code
2343 \begin_layout Plain Layout
2349 option if newfangle is being invoked in a C source file, so that C compile
2350 errors will refer to the line number in the Lyx document.
2358 \begin_layout Standard
2359 \begin_inset listings
2363 \begin_layout Plain Layout
2368 \begin_layout Plain Layout
2370 nf_line=-L -T$(TABS)
2373 \begin_layout Plain Layout
2380 \begin_layout Plain Layout
2382 $(call if_extension,$(2),$(C_EXTENSIONS),$(nf_line))
2387 \begin_layout Plain Layout
2397 \begin_layout Standard
2398 We can use a similar trick to define an
2402 macro which takes just the filename as an argument and can return a pipeline
2403 stage calling the indent command.
2404 Indent can be turned off with
2405 \begin_inset Flex CharStyle:Code
2408 \begin_layout Plain Layout
2409 make newfangle_sources indent=
2421 \begin_layout Standard
2422 \begin_inset listings
2426 \begin_layout Plain Layout
2428 indent_options=-npro -kr -i8 -ts8 -sob -l80 -ss -ncs
2431 \begin_layout Plain Layout
2433 indent=$(call if_extension,$(1),$(C_EXTENSIONS),
2438 \begin_layout Plain Layout
2440 | indent $(indent_options))
2448 \begin_layout Standard
2449 We now define the pattern for extracting a file.
2450 The files are written using noweb's
2456 \begin_layout Plain Layout
2459 So you still need noweb installed in order to use cpif
2465 \begin_inset Note Note
2468 \begin_layout Plain Layout
2471 Write an awk version
2478 so that the file timestamp will not be touched if the contents haven't
2480 This avoids the need to rebuild the entire project because of a typographical
2481 change in the documentation, or if only a few C source files have changed.
2488 \begin_layout Standard
2489 \begin_inset listings
2493 \begin_layout Plain Layout
2495 newfangle_extract=@mkdir -p $(dir $(1)) &&
2500 \begin_layout Plain Layout
2502 $(call newfangle,$(2),$(1)) > "$(1).tmp" &&
2507 \begin_layout Plain Layout
2509 cat "$(1).tmp" $(indent) | cpif "$(1)"
2514 \begin_layout Plain Layout
2516 && rm -- "$(1).tmp" ||
2521 \begin_layout Plain Layout
2523 (echo error newfangling $(1) from $(2) ; exit 1)
2531 \begin_layout Standard
2532 We define a target which will extract or update all sources.
2533 To do this we first defined a makefile template that can do this for any
2534 source file in the LaTeX document.
2541 \begin_layout Standard
2542 \begin_inset listings
2546 \begin_layout Plain Layout
2548 define NEWFANGLE_template
2551 \begin_layout Plain Layout
2558 \begin_layout Plain Layout
2560 $$(call newfangle_extract,$(1),$(2))
2563 \begin_layout Plain Layout
2565 NEWFANGLE_TARGETS+=$(1)
2568 \begin_layout Plain Layout
2578 \begin_layout Standard
2579 We then enumerate the discovered
2580 \begin_inset Flex CharStyle:Code
2583 \begin_layout Plain Layout
2589 to generate a makefile rule for each one using the makefile template we
2594 Makefile.inc-targets
2597 \begin_layout Standard
2598 \begin_inset listings
2602 \begin_layout Plain Layout
2604 $(foreach source,$(NEWFANGLE_SOURCES),
2609 \begin_layout Plain Layout
2611 $(eval $(call NEWFANGLE_template,$(source),$(NEWFANGLE_SOURCE)))
2616 \begin_layout Plain Layout
2626 \begin_layout Standard
2627 These will all be built with NEWFANGLE_SOURCE_STAMP.
2630 \begin_layout Standard
2631 We also remove the generated sources on a
2639 Makefile.inc-targets
2642 \begin_layout Standard
2643 \begin_inset listings
2647 \begin_layout Plain Layout
2649 _distclean: clean_newfangle_sources
2657 \begin_layout Subsection
2658 Extracting Documentation
2661 \begin_layout Standard
2662 We then identify the intermediate stages of the documentation and their
2663 build and clean targets.
2666 \begin_layout Subsubsection
2670 \begin_layout Standard
2671 We produce a pdf file from the tex file.
2678 \begin_layout Standard
2679 \begin_inset listings
2683 \begin_layout Plain Layout
2685 NEWFANGLE_PDF=$(TEX_SOURCE:.tex=.pdf)
2693 \begin_layout Standard
2694 We run pdflatex twice to be sure that the contents and aux files are up
2696 We certainly are required to run pdflatex twice if these files do not exist!
2700 Makefile.inc-targets
2703 \begin_layout Standard
2704 \begin_inset listings
2708 \begin_layout Plain Layout
2710 $(NEWFANGLE_PDF): $(TEX_SOURCE); pdflatex $< && pdflatex $<
2713 \begin_layout Plain Layout
2715 clean_pdf: ; rm -f -- $(NEWFANGLE_PDF)
2720 \begin_layout Plain Layout
2722 $(TEX_SOURCE:.tex=.toc)
2727 \begin_layout Plain Layout
2729 $(TEX_SOURCE:.tex=.log)
2734 \begin_layout Plain Layout
2736 $(TEX_SOURCE:.tex=.aux)
2744 \begin_layout Subsubsection
2748 \begin_layout Standard
2749 Currently we only build pdf as a final format, but NEWFANGLE_DOCS may later
2750 hold other output formats.
2757 \begin_layout Standard
2758 \begin_inset listings
2762 \begin_layout Plain Layout
2764 NEWFANGLE_DOCS=$(NEWFANGLE_PDF)
2772 \begin_layout Standard
2773 We also define newfangle_docs as a convenient phony target<
2777 Makefile.inc-targets
2780 \begin_layout Standard
2781 \begin_inset listings
2785 \begin_layout Plain Layout
2787 .PHONY: newfangle_docs
2790 \begin_layout Plain Layout
2792 newfangle_docs: $(NEWFANGLE_DOCS)
2795 \begin_layout Plain Layout
2797 docs: newfangle_docs
2805 \begin_layout Standard
2806 And define a convenient clean_noweb_docs which we add to the regular clean
2811 Makefile.inc-targets
2814 \begin_layout Standard
2815 \begin_inset listings
2819 \begin_layout Plain Layout
2821 .PHONEY: clean_newfangle_docs
2824 \begin_layout Plain Layout
2826 clean_newfangle_docs: clean_tex clean_pdf
2829 \begin_layout Plain Layout
2831 clean: clean_newfangle_docs
2834 \begin_layout Plain Layout
2838 \begin_layout Plain Layout
2840 distclean_newfangle_docs: clean_tex clean_newfangle_docs
2843 \begin_layout Plain Layout
2845 distclean: clean distclean_newfangle_docs
2853 \begin_layout Subsection
2857 \begin_layout Standard
2858 If Makefile.inc is included into Makefile, then extracted files can be updated
2862 \begin_layout LyX-Code
2863 make newfangle_sources
2866 \begin_layout Standard
2870 \begin_layout LyX-Code
2871 make -f Makefile.inc newfangle_sources
2878 \begin_layout Chapter
2879 Newfangle awk source code
2882 \begin_layout Standard
2883 We use the copyright notice from chapter
2884 \begin_inset CommandInset ref
2886 reference "cha:License"
2894 ./newfangle,language=awk,morestring=[b]{/},morekeywords=else
2897 \begin_layout Standard
2898 \begin_inset listings
2902 \begin_layout Plain Layout
2907 \begin_layout Plain Layout
2911 chunkref{gpl3-copyright}>
2919 \begin_layout Standard
2920 We also use code from Arnold Robbins public domain getopt (1993 revision)
2922 \begin_inset CommandInset ref
2924 reference "cha:getopt"
2928 , and naturally want to attribute this appropriately.
2931 \begin_layout Standard
2932 \begin_inset listings
2936 \begin_layout Plain Layout
2940 \begin_layout Plain Layout
2942 # NOTE: Arnold Robbins public domain getopt for awk is also used:
2945 \begin_layout Plain Layout
2949 chunkref{getopt.awk-header}>
2952 \begin_layout Plain Layout
2956 \begin_layout Plain Layout
2960 chunkref{getopt.awk-getopt()}>
2963 \begin_layout Plain Layout
2972 \begin_layout Standard
2973 And include the following chunks
2980 \begin_layout Standard
2981 \begin_inset listings
2985 \begin_layout Plain Layout
2989 chunkref{helper-functions}>
2992 \begin_layout Plain Layout
2996 chunkref{mode-tracker}>
2999 \begin_layout Plain Layout
3003 chunkref{chunk-storage-functions}>
3006 \begin_layout Plain Layout
3010 chunkref{output_chunk_names()}>
3013 \begin_layout Plain Layout
3017 chunkref{output_chunks()}>
3020 \begin_layout Plain Layout
3024 chunkref{write_chunk()}>
3027 \begin_layout Plain Layout
3031 chunkref{expand_chunk_args()}>
3034 \begin_layout Plain Layout
3041 \begin_layout Plain Layout
3045 chunkref{recognize-chunk}>
3048 \begin_layout Plain Layout
3060 \begin_layout Section
3064 \begin_layout Standard
3065 The portable way to erase an array in awk is to split the empty string,
3070 awk-delete-array,params=ARRAY
3073 \begin_layout Standard
3074 \begin_inset listings
3078 \begin_layout Plain Layout
3080 split("", ${ARRAY});
3089 dump-array,params=ARRAY
3092 \begin_layout Standard
3093 \begin_inset listings
3097 \begin_layout Plain Layout
3108 \begin_layout Plain Layout
3110 for (_x in ${ARRAY}) {
3113 \begin_layout Plain Layout
3115 print _x "=" ${ARRAY}[_x] "
3120 \begin_layout Plain Layout
3125 \begin_layout Plain Layout
3141 \begin_layout Section
3145 \begin_layout Standard
3146 Fatal errors are issued with the error function:
3150 error(),append=helper-functions
3153 \begin_layout Standard
3154 \begin_inset listings
3158 \begin_layout Plain Layout
3160 function error(message)
3163 \begin_layout Plain Layout
3168 \begin_layout Plain Layout
3170 print "ERROR: " FILENAME ":" FNR " " message > "/dev/stderr";
3173 \begin_layout Plain Layout
3178 \begin_layout Plain Layout
3188 \begin_layout Standard
3189 \begin_inset listings
3193 \begin_layout Plain Layout
3195 function warning(message)
3198 \begin_layout Plain Layout
3203 \begin_layout Plain Layout
3205 print "WARNING: " FILENAME ":" FNR " " message > "/dev/stderr";
3208 \begin_layout Plain Layout
3213 \begin_layout Plain Layout
3223 \begin_layout Chapter
3227 \begin_layout Standard
3228 LaTeX arguments to lstlistings macros are a comma seperated list of key-value
3230 Values containing commas are enclosed in { braces }.
3233 \begin_layout Standard
3234 We need a function that can parse such an expression and assign the values
3242 \begin_layout Standard
3243 A sample expressions is:
3246 \begin_layout LyX-Code
3247 name=thomas, params={a, b}, something, something-else
3250 \begin_layout Standard
3251 but we see that this is just a simpler form of this expression:
3254 \begin_layout LyX-Code
3255 name=freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
3258 \begin_layout Standard
3259 And that it would be a good idea to use a recursive parser into a multi-dimensio
3264 \begin_layout Plain Layout
3265 as AWK doesn't have nested-hash support
3273 \begin_layout Standard
3274 \begin_inset Tabular
3275 <lyxtabular version="3" rows="6" columns="2">
3277 <column alignment="left" valignment="top" width="0">
3278 <column alignment="left" valignment="top" width="0">
3280 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3283 \begin_layout Plain Layout
3289 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3292 \begin_layout Plain Layout
3300 <cell alignment="left" valignment="top" topline="true" leftline="true" usebox="none">
3303 \begin_layout Plain Layout
3309 <cell alignment="left" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3312 \begin_layout Plain Layout
3320 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3323 \begin_layout Plain Layout
3329 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3332 \begin_layout Plain Layout
3340 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3343 \begin_layout Plain Layout
3349 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3352 \begin_layout Plain Layout
3360 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3363 \begin_layout Plain Layout
3369 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3372 \begin_layout Plain Layout
3380 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3383 \begin_layout Plain Layout
3389 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3392 \begin_layout Plain Layout
3406 \begin_layout Standard
3407 On reflection it seems that sometimes such nesting is not desirable, as
3408 the braces are also used to delimit values that contain commas --- we may
3412 \begin_layout LyX-Code
3413 name={williamson, freddie}
3416 \begin_layout Standard
3418 \begin_inset Flex CharStyle:Code
3421 \begin_layout Plain Layout
3428 \begin_inset Flex CharStyle:Code
3431 \begin_layout Plain Layout
3437 --- so I may change this behaviour.
3438 \begin_inset Note Note
3441 \begin_layout Plain Layout
3450 \begin_layout Standard
3452 \begin_inset Flex Chunkref
3455 \begin_layout Plain Layout
3461 will accept two paramters,
3462 \begin_inset Flex CharStyle:Code
3465 \begin_layout Plain Layout
3471 being the text to parse, and
3472 \begin_inset Flex CharStyle:Code
3475 \begin_layout Plain Layout
3481 being an array to receive the parsed values as described above.
3482 The optional parameter
3483 \begin_inset Flex CharStyle:Code
3486 \begin_layout Plain Layout
3492 is used during recursion to build up the multi-dimensional array path.
3499 \begin_layout Standard
3500 \begin_inset listings
3504 \begin_layout Plain Layout
3508 chunkref{get_chunk_args()}>
3520 \begin_layout Standard
3521 \begin_inset listings
3525 \begin_layout Plain Layout
3527 function get_chunk_args(text, values,
3530 \begin_layout Plain Layout
3532 # optional parameters
3535 \begin_layout Plain Layout
3537 path, # hierarchical precursors
3540 \begin_layout Plain Layout
3545 \begin_layout Plain Layout
3555 \begin_layout Standard
3556 The strategy is to parse the name, and then look for a value.
3557 If the value begins with a brace
3558 \begin_inset Flex CharStyle:Code
3561 \begin_layout Plain Layout
3567 , then we recurse and consume as much of the text as necessary, returning
3568 the remaining text when we encounter a leading close-brace
3569 \begin_inset Flex CharStyle:Code
3572 \begin_layout Plain Layout
3579 This being the strategy --- and executed in a loop --- we realise that
3580 we must first look for the closing brace (perhaps preceded by white space)
3581 in order to terminate the recursion, and returning remaining text.
3584 \begin_layout Standard
3585 \begin_inset listings
3589 \begin_layout Plain Layout
3594 \begin_layout Plain Layout
3596 split("", next_chunk_args);
3599 \begin_layout Plain Layout
3601 while(length(text)) {
3604 \begin_layout Plain Layout
3606 if (match(text, "^ *}(.*)", a)) {
3609 \begin_layout Plain Layout
3614 \begin_layout Plain Layout
3619 \begin_layout Plain Layout
3623 chunkref{parse-chunk-args}>
3626 \begin_layout Plain Layout
3631 \begin_layout Plain Layout
3636 \begin_layout Plain Layout
3646 \begin_layout Standard
3647 \begin_inset Note Note
3650 \begin_layout Plain Layout
3651 Use BNF package here
3656 We can see that the text could be inspected with this regex:
3663 \begin_layout Standard
3664 \begin_inset listings
3668 \begin_layout Plain Layout
3670 if (! match(text, " *([^,=]*[^,= ]) *(([,=]) *(([^,}]*) *,* *(.*))|)$", a))
3674 \begin_layout Plain Layout
3679 \begin_layout Plain Layout
3689 \begin_layout Standard
3691 \begin_inset Flex CharStyle:Code
3694 \begin_layout Plain Layout
3700 will have the following values:
3703 \begin_layout Standard
3704 \begin_inset Tabular
3705 <lyxtabular version="3" rows="7" columns="2">
3707 <column alignment="center" valignment="top" width="0">
3708 <column alignment="left" valignment="top" width="0">
3710 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3713 \begin_layout Plain Layout
3719 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3722 \begin_layout Plain Layout
3730 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3733 \begin_layout Plain Layout
3739 <cell alignment="left" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3742 \begin_layout Plain Layout
3750 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3753 \begin_layout Plain Layout
3759 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3762 \begin_layout Plain Layout
3763 =freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
3770 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3773 \begin_layout Plain Layout
3779 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3782 \begin_layout Plain Layout
3790 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3793 \begin_layout Plain Layout
3799 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3802 \begin_layout Plain Layout
3803 freddie, foo={bar=baz, quux={quirk, a=fleeg}}, etc
3810 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
3813 \begin_layout Plain Layout
3819 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
3822 \begin_layout Plain Layout
3830 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
3833 \begin_layout Plain Layout
3839 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
3842 \begin_layout Plain Layout
3843 , foo={bar=baz, quux={quirk, a=fleeg}}, etc
3856 \begin_layout Standard
3858 \begin_inset Flex CharStyle:Code
3861 \begin_layout Plain Layout
3868 \begin_inset Flex CharStyle:Code
3871 \begin_layout Plain Layout
3877 and signify whether the option named in
3878 \begin_inset Flex CharStyle:Code
3881 \begin_layout Plain Layout
3887 has a value or not (respectively).
3890 \begin_layout Standard
3891 If the option does have a value, then if the expression
3892 \begin_inset Flex CharStyle:Code
3895 \begin_layout Plain Layout
3902 \begin_inset Flex CharStyle:Code
3905 \begin_layout Plain Layout
3911 it will signify that we need to recurse:
3914 \begin_layout Standard
3915 \begin_inset listings
3919 \begin_layout Plain Layout
3924 \begin_layout Plain Layout
3929 \begin_layout Plain Layout
3931 if (substr(a[4],1,1) == "{") {
3934 \begin_layout Plain Layout
3936 text = get_chunk_args(substr(a[4],2), values, path name SUBSEP);
3939 \begin_layout Plain Layout
3944 \begin_layout Plain Layout
3946 values[path name]=a[5];
3949 \begin_layout Plain Layout
3954 \begin_layout Plain Layout
3959 \begin_layout Plain Layout
3964 \begin_layout Plain Layout
3966 values[path name]="";
3969 \begin_layout Plain Layout
3974 \begin_layout Plain Layout
3984 \begin_layout Standard
3985 We can test this function like this:
3992 \begin_layout Standard
3993 \begin_inset listings
3997 \begin_layout Plain Layout
4001 chunkref{get_chunk_args()}>
4004 \begin_layout Plain Layout
4009 \begin_layout Plain Layout
4014 \begin_layout Plain Layout
4018 \begin_layout Plain Layout
4020 print get_chunk_args("name=freddie, foo={bar=baz, quux={quirk, a=fleeg}},
4024 \begin_layout Plain Layout
4029 \begin_layout Plain Layout
4031 print "a[" b "] => " a[b];
4034 \begin_layout Plain Layout
4039 \begin_layout Plain Layout
4049 \begin_layout Standard
4050 which should give this output:
4054 gca-test.awk-results
4057 \begin_layout Standard
4058 \begin_inset listings
4062 \begin_layout Plain Layout
4064 a[foo.quux.quirk] =>
4067 \begin_layout Plain Layout
4069 a[foo.quux.a] => fleeg
4072 \begin_layout Plain Layout
4077 \begin_layout Plain Layout
4082 \begin_layout Plain Layout
4092 \begin_layout Chapter
4093 Expanding chunk arguments
4096 \begin_layout Standard
4097 \begin_inset CommandInset label
4099 name "cha:Chunk Arguments"
4104 \begin_inset Note Note
4107 \begin_layout Plain Layout
4108 Explain this in the documentation section too
4113 As an extension to many literate-programming styles, newfangle permits code
4114 chunks to take parameters and thus operate somewhat like C pre-processor
4115 macros, or like C++ templates.
4118 \begin_layout Standard
4119 Chunk parameters are declared with a chunk argument called
4120 \begin_inset Flex CharStyle:Code
4123 \begin_layout Plain Layout
4129 , which holds a semi-colon separated list of parameters, like this:
4132 \begin_layout LyX-Code
4133 achunk,language=C,params=name;address
4136 \begin_layout Standard
4137 When such a chunk is included, the arguments are expressed in round brackets
4138 as a comma separated list of optional arguments:
4139 \begin_inset Note Note
4142 \begin_layout Plain Layout
4143 We ought to support qouting in {} like ({Jones, John}, Jones@example.com)
4151 \begin_layout LyX-Code
4154 chunkref{achunk}(John Jones, jones@example.com)
4157 \begin_layout Standard
4158 Within the body of a chunk, the parameters are referred to with:
4159 \begin_inset Flex CharStyle:Code
4162 \begin_layout Plain Layout
4169 \begin_inset Flex CharStyle:Code
4172 \begin_layout Plain Layout
4179 There is a strong case that a LaTeX style notation should be used, like
4182 param{name} which would be expressed in the listing as =<
4184 param{name}> and be rendered as
4185 \begin_inset listings
4189 \begin_layout Plain Layout
4199 Such notation would make me go blind, but I do intend to adopt it.
4202 \begin_layout Standard
4203 We therefore need a function
4204 \begin_inset Flex CharStyle:Code
4207 \begin_layout Plain Layout
4213 which will take a block of text, a list of permitted parameters and the
4214 arguments which must substitute for the parameters.
4218 \begin_layout Standard
4219 We also need to be able to parse a parameter list into an array of parameters.
4222 \begin_layout Section
4223 Parsing argument lists
4226 \begin_layout Standard
4227 An argument list may be as simple as in
4228 \begin_inset Flex CharStyle:Code
4231 \begin_layout Plain Layout
4234 chunkref{pull}(thing, otherthing)
4242 \begin_layout LyX-Code
4245 chunkref{pull}(things[x, y], get_other_things(a, "(all)"))
4248 \begin_layout Standard
4249 --- which for all it's commas and quotes and parenthesis represents only
4251 \begin_inset Flex CharStyle:Code
4254 \begin_layout Plain Layout
4261 \begin_inset Flex CharStyle:Code
4264 \begin_layout Plain Layout
4265 get_other_things(a, "(all)")
4273 \begin_layout Standard
4274 If we simply split parameter list on commas, then the comma in
4275 \begin_inset Flex CharStyle:Code
4278 \begin_layout Plain Layout
4284 would split into two seperate arguments:
4285 \begin_inset Flex CharStyle:Code
4288 \begin_layout Plain Layout
4295 \begin_inset Flex CharStyle:Code
4298 \begin_layout Plain Layout
4304 --- neither of which make sense on their own.
4307 \begin_layout Standard
4308 One way to prevent this would be by refusing to split text between maching
4310 \begin_inset Flex CharStyle:Code
4313 \begin_layout Plain Layout
4320 \begin_inset Flex CharStyle:Code
4323 \begin_layout Plain Layout
4330 \begin_inset Flex CharStyle:Code
4333 \begin_layout Plain Layout
4340 \begin_inset Flex CharStyle:Code
4343 \begin_layout Plain Layout
4350 \begin_inset Flex CharStyle:Code
4353 \begin_layout Plain Layout
4360 \begin_inset Flex CharStyle:Code
4363 \begin_layout Plain Layout
4369 and most likely also
4370 \begin_inset Flex CharStyle:Code
4373 \begin_layout Plain Layout
4380 \begin_inset Flex CharStyle:Code
4383 \begin_layout Plain Layout
4390 \begin_inset Flex CharStyle:Code
4393 \begin_layout Plain Layout
4400 \begin_inset Flex CharStyle:Code
4403 \begin_layout Plain Layout
4410 Of course this also makes it impossible to pass such mis-matched code fragments
4411 as parameters, but I think that it would be hard for readers to cope with
4412 authors who would pass such code unbalanced fragments as chunk parameters
4416 \begin_layout Plain Layout
4417 I know that I couldn't cope with users doing such things, and although the
4418 GPL3 license prevents me from actually forbidding anyone from trying, if
4419 they want it to work they'll have to write the code themselves and not
4420 expect any support from me.
4428 \begin_layout Standard
4429 Unfortunately, the full set of matching delimiters may vary from language
4431 In certain C++ template contexts,
4432 \begin_inset Flex CharStyle:Code
4435 \begin_layout Plain Layout
4442 \begin_inset Flex CharStyle:Code
4445 \begin_layout Plain Layout
4451 would count as delimiters, and yet in other contexts they would not.
4454 \begin_layout Standard
4455 This puts me in the unfortunate position of having to parse-somewhat all
4456 programming languages without knowing what they are!
4459 \begin_layout Standard
4460 This is the basis of mode-tracking, by tracking parse-modes for different
4464 \begin_layout Subsection
4468 \begin_layout Standard
4469 In the C language there are a few parse modes, affecting the interpretation
4473 \begin_layout Standard
4474 One parse mode is the strings mode.
4475 The string mode is commenced by an un-escaped quotation mark
4476 \begin_inset Flex CharStyle:Code
4479 \begin_layout Plain Layout
4485 and terminated by the same.
4486 Within the string mode, only one additional mode can be commenced, it is
4488 \begin_inset Flex CharStyle:Code
4491 \begin_layout Plain Layout
4499 , which is always terminated by the folloing character.
4502 \begin_layout Standard
4504 \begin_inset Flex CharStyle:Code
4507 \begin_layout Plain Layout
4513 which is terminated by a
4514 \begin_inset Flex CharStyle:Code
4517 \begin_layout Plain Layout
4523 (unless it occurs in a string).
4526 \begin_layout Standard
4527 Consider this line of C code:
4530 \begin_layout Standard
4531 \begin_inset Formula $\mathtt{things\underbrace{[\mathtt{x,\ y}]}_{1.\ [\ mode},\ get\_other\_things\underbrace{(\mathtt{a,\overbrace{"\mathtt{(all)}"}})}_{2.\ (\ mode}^{3.\ "\ mode}}$
4537 \begin_layout Standard
4538 Mode nesting prevents the close parenthesis in quote mode (part 3) from
4539 terminating the parenthesis mode (part 2).
4542 \begin_layout Standard
4543 Each language has a set of modes, the default mode being the null mode.
4544 Each mode can lead to other modes.
4547 \begin_layout Standard
4548 Modes are stored in a multi-dimensional hash.
4549 The first index is the language, and the second is the mode-identifier.
4550 The third indexes are terminators, and optionally, submodes, and delimiters.
4553 \begin_layout Standard
4554 A useful set of mode definitions for a nameless general C-type language
4556 (Don't be confused by the double backslash escaping needed in awk.
4557 One set of escaping is for the string, and the second set of escaping is
4565 \begin_layout Standard
4566 \begin_inset listings
4570 \begin_layout Plain Layout
4572 modes["", "", "delimeters"]=" *, *";
4575 \begin_layout Plain Layout
4577 modes["", "", "submodes" ]="
4607 \begin_layout Standard
4608 In the default mode, a comma surrounded by un-important white space is a
4609 delimiter of language items; and should pass this test:
4613 test:mode-definitions
4616 \begin_layout Standard
4617 \begin_inset listings
4621 \begin_layout Plain Layout
4623 parse_chunk_args("", "1,2,3)", a, "(");
4626 \begin_layout Plain Layout
4628 if (a[1] != "1") e++;
4631 \begin_layout Plain Layout
4633 if (a[2] != "2") e++;
4636 \begin_layout Plain Layout
4638 if (a[3] != "3") e++;
4641 \begin_layout Plain Layout
4643 if (length(a) != 3) e++;
4646 \begin_layout Plain Layout
4650 chunkref{pca-test.awk:summary}>
4653 \begin_layout Plain Layout
4657 \begin_layout Plain Layout
4659 parse_chunk_args("", "joe, red", a, "(");
4662 \begin_layout Plain Layout
4664 if (a[1] != "joe") e++;
4667 \begin_layout Plain Layout
4669 if (a[2] != "red") e++;
4672 \begin_layout Plain Layout
4674 if (length(a) != 2) e++;
4677 \begin_layout Plain Layout
4681 chunkref{pca-test.awk:summary}>
4684 \begin_layout Plain Layout
4688 \begin_layout Plain Layout
4690 parse_chunk_args("", "${colour}", a, "(");
4693 \begin_layout Plain Layout
4695 if (a[1] != "${colour}") e++;
4698 \begin_layout Plain Layout
4700 if (length(a) != 1) e++;
4703 \begin_layout Plain Layout
4707 chunkref{pca-test.awk:summary}>
4715 \begin_layout Standard
4716 Nested modes are identified by a backslash, a double or single quote, various
4717 bracket styles or a /* comment.
4720 \begin_layout Standard
4721 For each of these sub-modes modes we must also identify at a mode terminator,
4722 and any sub-modes or delimiters that may be entered
4726 \begin_layout Plain Layout
4727 Because we are using the sub-mode characters as the mode identifier it means
4728 we can't currently have a mode character dependant on it's context; i.e.
4730 \begin_inset Flex CharStyle:Code
4733 \begin_layout Plain Layout
4739 can't behave differently when it is inside
4740 \begin_inset Flex CharStyle:Code
4743 \begin_layout Plain Layout
4757 \begin_layout Subsubsection
4761 \begin_layout Standard
4762 The backslash mode has no submodes or delimiters, and is terminated by any
4764 Note that we are not so much interested in evaluating or interpolating
4765 content as we are in delineating content.
4766 It is no matter that a double backslash (
4767 \begin_inset Flex CharStyle:Code
4770 \begin_layout Plain Layout
4780 ) may represent a single backslash while a backslash-newline may represent
4781 white space, but it does matter that the newline in a backslash newline
4782 should not be able to terminate a C pre-processor statement; and so the
4783 newline will be consumed by the backslash however it is to be interpreted.
4790 \begin_layout Standard
4791 \begin_inset listings
4795 \begin_layout Plain Layout
4801 ", "terminators"]=".";
4809 \begin_layout Subsubsection
4813 \begin_layout Standard
4814 In a string we have one special mode, which is the backslash.
4815 This may escape an embedded quote and prevent us thinking that it should
4816 terminate the string.
4817 Otherwise, the string will be terminated by a double-quote.
4820 \begin_layout Standard
4821 \begin_inset listings
4825 \begin_layout Plain Layout
4840 \begin_layout Plain Layout
4844 "", "terminators"]="
4854 \begin_layout Standard
4855 Working strings should pass this test:
4859 test:mode-definitions
4862 \begin_layout Standard
4863 \begin_inset listings
4867 \begin_layout Plain Layout
4869 parse_chunk_args("", "say
4885 ", for me", a, "(");
4888 \begin_layout Plain Layout
4909 \begin_layout Plain Layout
4911 if (a[2] != "for me") e++;
4914 \begin_layout Plain Layout
4916 if (length(a) != 2) e++;
4919 \begin_layout Plain Layout
4923 chunkref{pca-test.awk:summary}>
4935 \begin_layout Standard
4936 \begin_inset listings
4940 \begin_layout Plain Layout
4942 modes["", "{", "submodes" ]="
4967 \begin_layout Plain Layout
4969 modes["", "{", "delimeters"]=" *, *";
4972 \begin_layout Plain Layout
4974 modes["", "{", "terminators"]="}";
4977 \begin_layout Plain Layout
4979 modes["", "[", "submodes" ]="
5004 \begin_layout Plain Layout
5006 modes["", "[", "delimiters"]=" *, *";
5009 \begin_layout Plain Layout
5011 modes["", "[", "terminators"]="
5018 \begin_layout Plain Layout
5020 modes["", "(", "submodes" ]="
5045 \begin_layout Plain Layout
5047 modes["", "(", "delimiters"]=" *, *";
5050 \begin_layout Plain Layout
5052 modes["", "(", "terminators"]="
5059 \begin_layout Plain Layout
5061 modes["", "'", "submodes" ]="
5072 \begin_layout Plain Layout
5074 modes["", "'", "terminators"]="'";
5077 \begin_layout Plain Layout
5079 modes["", "/*", "submodes"]="
5086 \begin_layout Plain Layout
5088 modes["", "/*", "terminators"]="*/";
5091 \begin_layout Plain Layout
5093 modes["", "//", "submodes"]="
5098 \begin_layout Plain Layout
5100 modes["", "//", "terminators"]="
5105 \begin_layout Plain Layout
5107 modes["", "", "submodes" ]="
5137 \begin_layout Standard
5138 This test should also pass.
5142 test:mode-definitions
5145 \begin_layout Standard
5146 \begin_inset listings
5150 \begin_layout Plain Layout
5152 parse_chunk_args("", "things[x, y], get_other_things(a,
5159 \begin_layout Plain Layout
5161 if (a[1] != "things[x, y]") e++;
5164 \begin_layout Plain Layout
5166 if (a[2] != "get_other_things(a,
5173 \begin_layout Plain Layout
5175 if (a[3] != "99") e++;
5178 \begin_layout Plain Layout
5180 if (length(a) != 3) e++;
5183 \begin_layout Plain Layout
5187 chunkref{pca-test.awk:summary}>
5199 \begin_layout Standard
5200 \begin_inset listings
5204 \begin_layout Plain Layout
5206 function parse_chunk_args(language, text, values,
5209 \begin_layout Plain Layout
5211 # optional parameters
5214 \begin_layout Plain Layout
5219 \begin_layout Plain Layout
5221 path, # hierarchical precursors
5224 \begin_layout Plain Layout
5226 stack, # delimiters to be matched
5229 \begin_layout Plain Layout
5234 \begin_layout Plain Layout
5239 \begin_layout Plain Layout
5241 c, a, part, item, name, result, new_values, new_mode, delimiters)
5244 \begin_layout Plain Layout
5249 \begin_layout Plain Layout
5251 # split(chunklet_parts[2], call_chunk_args, " *, *");
5259 \begin_layout Standard
5260 The strategy is to parse the name, and then look for a value.
5261 If the value begins with a brace
5262 \begin_inset Flex CharStyle:Code
5265 \begin_layout Plain Layout
5271 , then we recurse and consume as much of the text as necessary, returning
5272 the remaining text when we encounter a leading close-brace
5273 \begin_inset Flex CharStyle:Code
5276 \begin_layout Plain Layout
5283 This being the strategy --- and executed in a loop --- we realise that
5284 we must first look for the closing brace (perhaps preceded by white space)
5285 in order to terminate the recursion, and returning remaining text.
5288 \begin_layout Standard
5289 \begin_inset listings
5293 \begin_layout Plain Layout
5295 submodes=modes[language, mode, "submodes"];
5298 \begin_layout Plain Layout
5300 if ((language, mode, "delimiters") in modes) {
5303 \begin_layout Plain Layout
5305 delimiters = modes[language, mode, "delimiters"];
5308 \begin_layout Plain Layout
5310 submodes=submodes "|" delimiters;
5313 \begin_layout Plain Layout
5318 \begin_layout Plain Layout
5320 if ((language, mode, "terminators") in modes) {
5323 \begin_layout Plain Layout
5325 submodes=submodes "|" modes[language, mode, "terminators"];
5328 \begin_layout Plain Layout
5333 \begin_layout Plain Layout
5335 while(length(text)) {
5338 \begin_layout Plain Layout
5340 if (match(text, "(" submodes ")", a)) {
5343 \begin_layout Plain Layout
5348 \begin_layout Plain Layout
5350 error("Internal error, matched zero length submode, should be impossible
5351 - likely regex computation error");
5354 \begin_layout Plain Layout
5359 \begin_layout Plain Layout
5361 part = substr(text, 1, RSTART -1);
5364 \begin_layout Plain Layout
5369 \begin_layout Plain Layout
5371 if (match(a[1], "^" modes[language, mode, "terminators"] "$")) {
5374 \begin_layout Plain Layout
5379 \begin_layout Plain Layout
5381 return result item a[1];
5384 \begin_layout Plain Layout
5389 \begin_layout Plain Layout
5391 if (match(a[1], "^" delimiters "$")) {
5394 \begin_layout Plain Layout
5399 \begin_layout Plain Layout
5401 text = substr(text, 1 + length(part) + length(a[1]));
5404 \begin_layout Plain Layout
5406 result = result item a[1];
5409 \begin_layout Plain Layout
5414 \begin_layout Plain Layout
5416 } else if ((language, a[1], "terminators") in modes) {
5419 \begin_layout Plain Layout
5424 \begin_layout Plain Layout
5426 #check if new_mode is defined
5429 \begin_layout Plain Layout
5431 text = substr(text, 1 + length(part) + length(a[1]));
5434 \begin_layout Plain Layout
5436 s = parse_chunk_args(language, text,new_values,new_mode);
5439 \begin_layout Plain Layout
5441 text = substr(text, length(s) +1);
5444 \begin_layout Plain Layout
5449 \begin_layout Plain Layout
5454 \begin_layout Plain Layout
5456 error(sprintf("Submode '%s' set unknown mode in text: %s", new_mode,
5460 \begin_layout Plain Layout
5462 text = substr(text, 1 + length(part) + length(a[1]));
5465 \begin_layout Plain Layout
5470 \begin_layout Plain Layout
5475 \begin_layout Plain Layout
5477 result = result item text;
5480 \begin_layout Plain Layout
5485 \begin_layout Plain Layout
5490 \begin_layout Plain Layout
5495 \begin_layout Plain Layout
5500 \begin_layout Plain Layout
5505 \begin_layout Plain Layout
5507 if (length(item)) values[++c] = item;
5510 \begin_layout Plain Layout
5515 \begin_layout Plain Layout
5525 \begin_layout Standard
5526 We can test this function like this:
5533 \begin_layout Standard
5534 \begin_inset listings
5538 \begin_layout Plain Layout
5545 \begin_layout Plain Layout
5549 chunkref{parse_chunk_args()}>
5552 \begin_layout Plain Layout
5557 \begin_layout Plain Layout
5562 \begin_layout Plain Layout
5566 chunkref{mode-definitions}>
5569 \begin_layout Plain Layout
5573 \begin_layout Plain Layout
5577 chunkref{test:mode-definitions}>
5580 \begin_layout Plain Layout
5591 pca-test.awk:summary
5594 \begin_layout Standard
5595 \begin_inset listings
5599 \begin_layout Plain Layout
5604 \begin_layout Plain Layout
5609 \begin_layout Plain Layout
5614 \begin_layout Plain Layout
5616 print "a[" b "] => " a[b];
5619 \begin_layout Plain Layout
5624 \begin_layout Plain Layout
5629 \begin_layout Plain Layout
5634 \begin_layout Plain Layout
5639 \begin_layout Plain Layout
5644 \begin_layout Plain Layout
5654 \begin_layout Standard
5655 which should give this output:
5659 pca-test.awk-results
5662 \begin_layout Standard
5663 \begin_inset listings
5667 \begin_layout Plain Layout
5669 a[foo.quux.quirk] =>
5672 \begin_layout Plain Layout
5674 a[foo.quux.a] => fleeg
5677 \begin_layout Plain Layout
5682 \begin_layout Plain Layout
5687 \begin_layout Plain Layout
5697 \begin_layout Section
5698 Expanding parameters
5701 \begin_layout Standard
5702 \begin_inset CommandInset label
5704 name "Here-we-split"
5708 Here we split the text on
5709 \begin_inset Flex CharStyle:Code
5712 \begin_layout Plain Layout
5718 which means that all parts except the first will begin with a parameter
5720 The split function will consume the literal
5721 \begin_inset Flex CharStyle:Code
5724 \begin_layout Plain Layout
5737 \begin_layout Standard
5738 \begin_inset listings
5742 \begin_layout Plain Layout
5744 function expand_chunk_args(text, params, args,
5747 \begin_layout Plain Layout
5749 p, text_array, next_text, v, t, l)
5752 \begin_layout Plain Layout
5757 \begin_layout Plain Layout
5759 if (split(text, text_array, "
5766 \begin_layout Plain Layout
5770 chunkref{substitute-chunk-args}>
5773 \begin_layout Plain Layout
5778 \begin_layout Plain Layout
5783 \begin_layout Plain Layout
5793 \begin_layout Standard
5794 First, we produce an associative array of substitution values indexed by
5799 substitute-chunk-args
5802 \begin_layout Standard
5803 \begin_inset listings
5807 \begin_layout Plain Layout
5812 \begin_layout Plain Layout
5814 v[params[p]]=args[p];
5817 \begin_layout Plain Layout
5827 \begin_layout Standard
5828 We accumulate substituted text in the variable
5829 \begin_inset Flex CharStyle:Code
5832 \begin_layout Plain Layout
5839 As the first part of the split function is the part before the delimiter
5841 \begin_inset Flex CharStyle:Code
5844 \begin_layout Plain Layout
5850 in our case --- this part will never contain a parameter reference, so
5851 we assign this directly to the result kept in
5852 \begin_inset Flex CharStyle:Code
5855 \begin_layout Plain Layout
5862 \begin_inset listings
5866 \begin_layout Plain Layout
5876 \begin_layout Standard
5877 We then iterate over the remaining values in the array
5881 \begin_layout Plain Layout
5882 I don't know why I think that it will enumerate the array in order, but
5889 \begin_inset Note Note
5892 \begin_layout Plain Layout
5893 So fix it or porve it
5898 , and substitute each reference for it's argument.
5901 \begin_layout Standard
5902 \begin_inset listings
5906 \begin_layout Plain Layout
5908 for(t in text_array) if (t>1) {
5911 \begin_layout Plain Layout
5915 chunkref{substitute-chunk-arg}>
5918 \begin_layout Plain Layout
5928 \begin_layout Standard
5930 \begin_inset Flex CharStyle:Code
5933 \begin_layout Plain Layout
5939 a valid parameter reference will consist of valid parameter name terminated
5941 \begin_inset Flex CharStyle:Code
5944 \begin_layout Plain Layout
5951 A valid character name begins with the underscore or a letter, and may
5952 contain letters, digits or underscores.
5955 \begin_layout Standard
5956 A valid looking reference that is not actually the name of a parameter will
5957 be and not substituted.
5958 This is good because there is nothing to substitute anyway, and it avoids
5959 clashes when writing code for languages where ${\SpecialChar \ldots{}
5960 } is a valid construct
5961 --- such constructs will not be interfered with unless the parameter name
5966 substitute-chunk-arg
5969 \begin_layout Standard
5970 \begin_inset listings
5974 \begin_layout Plain Layout
5976 if (match(text_array[t], "^([a-zA-Z_][a-zA-Z0-9_]*)}", l) &&
5979 \begin_layout Plain Layout
5984 \begin_layout Plain Layout
5989 \begin_layout Plain Layout
5991 text = text v[l[1]] substr(text_array[t], length(l[1])+2);
5994 \begin_layout Plain Layout
5999 \begin_layout Plain Layout
6001 text = text "${" text_array[t];
6004 \begin_layout Plain Layout
6014 \begin_layout Chapter
6018 \begin_layout Standard
6019 Newfangle recognizes noweb chunks, but as we also want better LaTeX integration
6020 we will recognize any of these:
6023 \begin_layout Itemize
6024 notangle chunks matching the pattern
6025 \begin_inset Flex CharStyle:Code
6028 \begin_layout Plain Layout
6030 \begin_inset space \hspace*{}
6035 \begin_inset space \hspace*{}
6047 \begin_layout Itemize
6048 a chunks beginning with
6049 \begin_inset Flex CharStyle:Code
6052 \begin_layout Plain Layout
6062 Chunk{\SpecialChar \ldots{}
6063 } on the previous line
6066 \begin_layout Itemize
6067 an older form I have used, beginning with
6068 \begin_inset Flex CharStyle:Code
6071 \begin_layout Plain Layout
6074 begin{Chunk}[options]
6079 --- also more suitable for plain LaTeX users
6083 \begin_layout Plain Layout
6084 Is there such a thing as plain LaTeX?
6092 \begin_layout Section
6096 \begin_layout Standard
6098 \begin_inset Flex CharStyle:Code
6101 \begin_layout Plain Layout
6107 is used to signify that we are processing a code chunk and not document.
6108 In such a state, input lines will be assigned to the current chunk; otherwise
6112 \begin_layout Subsection
6116 \begin_layout Standard
6117 Our current scheme is to recognize the new lstlisting chunks, but these
6118 may be preceded by a
6119 \begin_inset Flex CharStyle:Code
6122 \begin_layout Plain Layout
6130 command which in LyX is a more convenient way to pass the chunk name to
6132 \begin_inset Flex CharStyle:Code
6135 \begin_layout Plain Layout
6143 command, and a more visible way to specify other
6144 \begin_inset Flex CharStyle:Code
6147 \begin_layout Plain Layout
6156 \begin_layout Standard
6157 The arguments to the
6158 \begin_inset Flex CharStyle:Code
6161 \begin_layout Plain Layout
6169 command are a name, and then a comma-seperated list of key-value pairs
6171 \begin_inset Flex CharStyle:Code
6174 \begin_layout Plain Layout
6183 (In fact within the LaTeX
6184 \begin_inset Flex CharStyle:Code
6187 \begin_layout Plain Layout
6196 \begin_inset CommandInset ref
6198 reference "sub:The-chunk-command"
6203 \begin_inset Flex CharStyle:Code
6206 \begin_layout Plain Layout
6212 is prefixed to the argument which is then literally passed to
6213 \begin_inset Flex CharStyle:Code
6216 \begin_layout Plain Layout
6231 \begin_layout Standard
6232 \begin_inset listings
6236 \begin_layout Plain Layout
6245 \begin_layout Plain Layout
6255 Chunk{ *([^ ,}]*),?(.*)}", line)) {
6258 \begin_layout Plain Layout
6260 next_chunk_name = line[1];
6263 \begin_layout Plain Layout
6265 get_chunk_args(line[2], next_chunk_args);
6268 \begin_layout Plain Layout
6273 \begin_layout Plain Layout
6278 \begin_layout Plain Layout
6288 \begin_layout Standard
6289 We also make a basic attempt to parse the name out of the
6290 \begin_inset Flex CharStyle:Code
6293 \begin_layout Plain Layout
6297 \begin_inset space \hspace{}
6306 text, otherwise we fall back to the name found in the previous chunk command.
6307 This attempt is very basic and doesn't support commas or spaces or square
6308 brackets as part of the chunkname.
6310 \begin_inset Flex CharStyle:Code
6313 \begin_layout Plain Layout
6321 which is convenient for some users
6325 \begin_layout Plain Layout
6326 but not yet supported in the LaTeX macros
6332 \begin_inset Note Note
6335 \begin_layout Plain Layout
6344 \begin_layout Standard
6345 \begin_inset listings
6349 \begin_layout Plain Layout
6362 \begin_layout Plain Layout
6364 if (match($0, "}.*[[,] *name= *{? *([^], }]*)", line)) {
6367 \begin_layout Plain Layout
6372 \begin_layout Plain Layout
6377 \begin_layout Plain Layout
6379 new_chunk(next_chunk_name, next_chunk_args);
6382 \begin_layout Plain Layout
6387 \begin_layout Plain Layout
6392 \begin_layout Plain Layout
6397 \begin_layout Plain Layout
6407 \begin_layout Subsection
6411 \begin_layout Standard
6412 We recognize notangle style chunks too:
6419 \begin_layout Standard
6420 \begin_inset listings
6424 \begin_layout Plain Layout
6429 \begin_layout Plain Layout
6431 if (match($0, "^[<]<(.*)[>]>= *$", line)) {
6434 \begin_layout Plain Layout
6439 \begin_layout Plain Layout
6444 \begin_layout Plain Layout
6449 \begin_layout Plain Layout
6454 \begin_layout Plain Layout
6459 \begin_layout Plain Layout
6469 \begin_layout Section
6473 \begin_layout Standard
6474 Likewise, we need to recognize when a chunk ends.
6477 \begin_layout Subsection
6481 \begin_layout Standard
6483 \begin_inset Flex CharStyle:Code
6486 \begin_layout Plain Layout
6493 \begin_inset Flex CharStyle:Code
6496 \begin_layout Plain Layout
6502 is surrounded by square brackets so that when this document is processed,
6503 this chunk doesn't terminate early when the lstlistings package recognizes
6504 it's own end-string!
6505 \begin_inset Note Greyedout
6508 \begin_layout Plain Layout
6509 This doesn't make sense as the regex is anchored with ^, which this line
6510 does not begin with!
6516 \begin_inset Note Note
6519 \begin_layout Plain Layout
6532 \begin_layout Standard
6533 \begin_inset listings
6537 \begin_layout Plain Layout
6550 \begin_layout Plain Layout
6555 \begin_layout Plain Layout
6560 \begin_layout Plain Layout
6565 \begin_layout Plain Layout
6575 \begin_layout Subsection
6583 \begin_layout Standard
6584 \begin_inset listings
6588 \begin_layout Plain Layout
6593 \begin_layout Plain Layout
6598 \begin_layout Plain Layout
6603 \begin_layout Plain Layout
6613 \begin_layout Standard
6614 All other recognizers are only of effect if we are chunking; there's no
6615 point in looking at lines if they aren't part of a chunk, so we just ignore
6616 them as efficiently as we can.
6623 \begin_layout Standard
6624 \begin_inset listings
6628 \begin_layout Plain Layout
6630 ! chunking { next; }
6638 \begin_layout Section
6642 \begin_layout Standard
6643 Chunk contents are any lines read while
6644 \begin_inset Flex CharStyle:Code
6647 \begin_layout Plain Layout
6654 Some chunk contents are special in that they refer to other chunks, and
6655 will be replaced by the contents of these chunks when the file is generated.
6658 \begin_layout Standard
6659 \begin_inset CommandInset label
6661 name "sub:ORS-chunk-text"
6665 We add the output record separator
6666 \begin_inset Flex CharStyle:Code
6669 \begin_layout Plain Layout
6675 to the line now, because we will set
6676 \begin_inset Flex CharStyle:Code
6679 \begin_layout Plain Layout
6685 to the empty string when we generate the output
6689 \begin_layout Plain Layout
6690 So that we can print partial lines using
6691 \begin_inset Flex CharStyle:Code
6694 \begin_layout Plain Layout
6701 \begin_inset Flex CharStyle:Code
6704 \begin_layout Plain Layout
6722 \begin_layout Standard
6723 \begin_inset listings
6727 \begin_layout Plain Layout
6729 length(active_chunk) {
6732 \begin_layout Plain Layout
6736 chunkref{process-chunk-tabs}>
6739 \begin_layout Plain Layout
6743 chunkref{process-chunk}>
6746 \begin_layout Plain Layout
6756 \begin_layout Standard
6757 If a chunk just consisted of plain text, we could handle the chunk like
6762 process-chunk-simple
6765 \begin_layout Standard
6766 \begin_inset listings
6770 \begin_layout Plain Layout
6772 chunk_line(active_chunk, $0 ORS);
6780 \begin_layout Standard
6781 but in fact a chunk can include references to other chunks.
6782 Chunk includes are traditionally written as
6783 \begin_inset Flex CharStyle:Code
6786 \begin_layout Plain Layout
6792 , but we support other variations.
6795 \begin_layout Standard
6796 However, we also process tabs at this point, a tab at input can be replaced
6797 by a number of spaces defined by the
6798 \begin_inset Flex CharStyle:Code
6801 \begin_layout Plain Layout
6807 variable, set by the
6808 \begin_inset Flex CharStyle:Code
6811 \begin_layout Plain Layout
6818 Of course this is poor tab behaviour, we should probably have the option
6819 to use proper counted tab-stops and process this on output.
6826 \begin_layout Standard
6827 \begin_inset listings
6831 \begin_layout Plain Layout
6836 \begin_layout Plain Layout
6843 \begin_layout Plain Layout
6853 \begin_layout Subsection
6854 \begin_inset CommandInset label
6856 name "sub:lstlistings-includes"
6863 \begin_layout Standard
6865 \begin_inset Flex CharStyle:Code
6868 \begin_layout Plain Layout
6871 lstset{escapeinside={=<}{>}}
6876 is set, then we can use
6877 \begin_inset Flex CharStyle:Code
6880 \begin_layout Plain Layout
6884 \begin_inset space \hspace{}
6895 \begin_inset Flex CharStyle:Code
6898 \begin_layout Plain Layout
6907 \begin_layout Enumerate
6908 it is a better mnemonic than
6909 \begin_inset Flex CharStyle:Code
6912 \begin_layout Plain Layout
6918 in that the = sign signifies equivalent or substitutability,
6921 \begin_layout Enumerate
6922 and because =< is not valid in C or in any language I can think of
6925 \begin_layout Enumerate
6926 and also because lstlistings doesn't like
6927 \begin_inset Flex CharStyle:Code
6930 \begin_layout Plain Layout
6936 as an end delimiter for the
6940 escape, so we must make do with a single
6941 \begin_inset Flex CharStyle:Code
6944 \begin_layout Plain Layout
6950 , which is better matched by
6951 \begin_inset Flex CharStyle:Code
6954 \begin_layout Plain Layout
6961 \begin_inset Flex CharStyle:Code
6964 \begin_layout Plain Layout
6973 \begin_layout Standard
6974 As each chunk line may contain more than one chunk include, we will split
6975 out chunk includes in an iterative fashion
6979 \begin_layout Plain Layout
6980 Contrary to our use of
6981 \begin_inset Flex CharStyle:Code
6984 \begin_layout Plain Layout
6990 when substituting parameters in chapter
6991 \begin_inset CommandInset ref
6993 reference "Here-we-split"
7005 \begin_layout Standard
7006 First, as long as the chunk contains a
7007 \begin_inset Flex CharStyle:Code
7010 \begin_layout Plain Layout
7018 command we take as much as we can up to the first
7019 \begin_inset Flex CharStyle:Code
7022 \begin_layout Plain Layout
7037 \begin_layout Standard
7038 \begin_inset listings
7042 \begin_layout Plain Layout
7047 \begin_layout Plain Layout
7052 \begin_layout Plain Layout
7057 \begin_layout Plain Layout
7075 )|)>|<<([a-zA-Z_][-a-zA-Z0-9_]*)>>)",
7078 \begin_layout Plain Layout
7085 \begin_layout Plain Layout
7090 \begin_layout Plain Layout
7092 chunklet = substr(chunk, 1, RSTART - 1);
7100 \begin_layout Standard
7101 We keep track of the indent count, by counting the number of literal characters
7103 We can then preserve this indent on each output line when multi-line chunks
7107 \begin_layout Standard
7108 We then process this first part literal text, and set the chunk which is
7109 still to be processed to be the text after the
7110 \begin_inset Flex CharStyle:Code
7113 \begin_layout Plain Layout
7121 command, which we will process next as we continue around the loop.
7124 \begin_layout Standard
7125 \begin_inset listings
7129 \begin_layout Plain Layout
7131 indent += length(chunklet);
7134 \begin_layout Plain Layout
7136 chunk_line(active_chunk, chunklet);
7139 \begin_layout Plain Layout
7141 chunk = substr(chunk, RSTART + RLENGTH);
7149 \begin_layout Standard
7150 We then consider the type of chunk command we have found, whether it is
7151 the newfangle style command beginning with
7152 \begin_inset Flex CharStyle:Code
7155 \begin_layout Plain Layout
7161 or the older notangle style beginning with
7162 \begin_inset Flex CharStyle:Code
7165 \begin_layout Plain Layout
7175 \begin_layout Standard
7176 Newfangle chunks may have parameters contained within square brackets.
7177 These will be matched in
7178 \begin_inset Flex CharStyle:Code
7181 \begin_layout Plain Layout
7187 and are considered at this stage of processing to be part of the name of
7188 the chunk to be included.
7191 \begin_layout Standard
7192 \begin_inset listings
7196 \begin_layout Plain Layout
7198 if (substr(line[1], 1, 1) == "=") {
7201 \begin_layout Plain Layout
7203 # chunk name up to }
7206 \begin_layout Plain Layout
7208 chunk_include(active_chunk, line[2] line[3], indent);
7211 \begin_layout Plain Layout
7213 } else if (substr(line[1], 1, 1) == "<") {
7216 \begin_layout Plain Layout
7218 chunk_include(active_chunk, line[4], indent);
7221 \begin_layout Plain Layout
7226 \begin_layout Plain Layout
7228 error("Unknown chunk fragment: " line[1]);
7231 \begin_layout Plain Layout
7241 \begin_layout Standard
7242 The loop will continue until there are no more chunkref statements in the
7243 text, at which point we process the final part of the chunk.
7246 \begin_layout Standard
7247 \begin_inset listings
7251 \begin_layout Plain Layout
7256 \begin_layout Plain Layout
7258 chunk_line(active_chunk, chunk);
7266 \begin_layout Standard
7267 \begin_inset CommandInset label
7273 We add the newline character as a chunklet on it's own, to make it easier
7274 to detect new lines and thus manage indentation when processing the output.
7277 \begin_layout Standard
7278 \begin_inset listings
7282 \begin_layout Plain Layout
7284 chunk_line(active_chunk, "
7294 \begin_layout Standard
7295 We will also permit a chunk-part number to follow in square brackets, so
7297 \begin_inset Flex CharStyle:Code
7300 \begin_layout Plain Layout
7303 chunkref{chunk-name[1]}>
7308 will refer to the first part only.
7309 This can make it easy to include a C function prototype in a header file,
7310 if the first part of the chunk is just the function prototype without the
7311 trailing semi-colon.
7312 The header file would include the prototype with the trailing semi-colon,
7316 \begin_layout LyX-Code
7319 chunkref{chunk-name[1]}>;
7322 \begin_layout Standard
7323 This is handled in section
7324 \begin_inset CommandInset ref
7326 reference "sub:Chunk-parts"
7333 \begin_layout Standard
7334 We should perhaps introduce a notion of language specific chunk options;
7335 so that perhaps we could specify:
7338 \begin_layout LyX-Code
7341 chunkref{chunk-name[function-declaration]}>;
7344 \begin_layout Standard
7345 which applies a transform
7346 \begin_inset Flex CharStyle:Code
7349 \begin_layout Plain Layout
7350 function-declaration
7355 to the chunk --- which in this case would extract a function prototype
7357 \begin_inset Note Note
7360 \begin_layout Plain Layout
7369 \begin_layout Chapter
7373 \begin_layout Standard
7374 At the start, first we set the default options.
7381 \begin_layout Standard
7382 \begin_inset listings
7386 \begin_layout Plain Layout
7391 \begin_layout Plain Layout
7396 \begin_layout Plain Layout
7401 \begin_layout Plain Layout
7406 \begin_layout Plain Layout
7416 \begin_layout Standard
7417 Then we use getopt the standard way, and null out ARGV afterwards in the
7425 \begin_layout Standard
7426 \begin_inset listings
7430 \begin_layout Plain Layout
7432 Optind = 1 # skip ARGV[0]
7435 \begin_layout Plain Layout
7437 while(getopt(ARGC, ARGV, "R:LdT:hr")!=-1) {
7440 \begin_layout Plain Layout
7444 chunkref{handle-options}>
7447 \begin_layout Plain Layout
7452 \begin_layout Plain Layout
7454 for (i=1; i<Optind; i++) { ARGV[i]=""; }
7462 \begin_layout Standard
7463 This is how we handle our options:
7470 \begin_layout Standard
7471 \begin_inset listings
7475 \begin_layout Plain Layout
7477 if (Optopt == "R") root = Optarg;
7480 \begin_layout Plain Layout
7482 else if (Optopt == "r") root="";
7485 \begin_layout Plain Layout
7487 else if (Optopt == "L") linenos = 1;
7490 \begin_layout Plain Layout
7492 else if (Optopt == "d") debug = 1;
7495 \begin_layout Plain Layout
7497 else if (Optopt == "T") tabs = indent_string(Optarg+0);
7500 \begin_layout Plain Layout
7502 else if (Optopt == "h") help();
7505 \begin_layout Plain Layout
7507 else if (Optopt == "?") help();
7515 \begin_layout Standard
7516 We do all of this at the beginning of the program
7523 \begin_layout Standard
7524 \begin_inset listings
7528 \begin_layout Plain Layout
7533 \begin_layout Plain Layout
7537 chunkref{constants}>
7540 \begin_layout Plain Layout
7544 chunkref{mode-definitions}>
7547 \begin_layout Plain Layout
7551 chunkref{default-options}>
7554 \begin_layout Plain Layout
7558 \begin_layout Plain Layout
7562 chunkref{read-options}>
7565 \begin_layout Plain Layout
7575 \begin_layout Standard
7576 And have a simple help function
7583 \begin_layout Standard
7584 \begin_inset listings
7588 \begin_layout Plain Layout
7593 \begin_layout Plain Layout
7598 \begin_layout Plain Layout
7600 print " newfangle [-L] -R<rootname> [source.tex ...]"
7603 \begin_layout Plain Layout
7605 print " newfangle -r [source.tex ...]"
7608 \begin_layout Plain Layout
7610 print " If the filename, source.tex is not specified then stdin is used"
7613 \begin_layout Plain Layout
7618 \begin_layout Plain Layout
7620 print "-L causes the C statement: #line <lineno>
7627 \begin_layout Plain Layout
7629 print "-R causes the named root to be written to stdout"
7632 \begin_layout Plain Layout
7634 print "-r lists all roots in the file (even those used elsewhere)"
7637 \begin_layout Plain Layout
7642 \begin_layout Plain Layout
7652 \begin_layout Chapter
7653 Chunk Language Modes
7656 \begin_layout Standard
7657 \begin_inset CommandInset label
7664 \begin_inset Note Greyedout
7667 \begin_layout Plain Layout
7668 This feature is in-development and does not work yet
7674 \begin_inset Note Note
7677 \begin_layout Plain Layout
7686 \begin_layout Standard
7687 lstlistings and newfangle both recognize source languages, and perform some
7689 lstlistings can detect strings and comments within a language definition
7690 and perform suitable rendering, such as italics for comments, and visible-space
7694 \begin_layout Standard
7695 Newfangle similarly can recognize strings, and comments, etc, within a language,
7696 so that any chunks included with
7697 \begin_inset Flex CharStyle:Code
7700 \begin_layout Plain Layout
7708 can be suitably quoted.
7711 \begin_layout Standard
7712 For instance, consider this chunk with
7713 \begin_inset Flex CharStyle:Code
7716 \begin_layout Plain Layout
7726 example-perl,language=perl
7729 \begin_layout Standard
7730 \begin_inset listings
7734 \begin_layout Plain Layout
7744 \begin_layout Standard
7745 If it were included in a chunk with
7746 \begin_inset Flex CharStyle:Code
7749 \begin_layout Plain Layout
7759 example-sh,language=sh
7762 \begin_layout Standard
7763 \begin_inset listings
7767 \begin_layout Plain Layout
7771 chunkref{example-perl}>"
7779 \begin_layout Standard
7780 would need to generate output like this if it were to work:
7783 \begin_layout LyX-Code
7791 \begin_layout Standard
7792 See that the double quote " as part of the regex has been quoted with a
7793 slash to protect it from shell interpretation.
7796 \begin_layout Standard
7797 If that were then included in a chunk with
7798 \begin_inset Flex CharStyle:Code
7801 \begin_layout Plain Layout
7811 example-makefile,language=make
7814 \begin_layout Standard
7815 \begin_inset listings
7819 \begin_layout Plain Layout
7824 \begin_layout Plain Layout
7828 chunkref{example-sh}>
7836 \begin_layout Standard
7837 We would need the output to look like this --- note the $$:
7840 \begin_layout LyX-Code
7844 \begin_layout LyX-Code
7852 \begin_layout Standard
7853 In order to make this work, we need to define a mode-tracker for each supported
7854 language, that can detect the various quoting modes, and provide a transformati
7855 on that must be applied to any included text so that included text will
7856 be interpreted correctly after the additional interpolation that it will
7857 be subject to at run-time.
7860 \begin_layout Standard
7861 For example, the transformation for text to be inserted into sh double-quoted
7862 strings would be something like:
7865 \begin_layout LyX-Code
7889 \begin_layout Standard
7891 \begin_inset Flex CharStyle:Code
7894 \begin_layout Plain Layout
7905 \begin_layout Standard
7906 \begin_inset Note Note
7909 \begin_layout Plain Layout
7910 I don't think this example is true
7915 The mode tracker must also track nested mode-changes, as in this
7916 \begin_inset Flex CharStyle:Code
7919 \begin_layout Plain Layout
7928 \begin_layout LyX-Code
7929 echo "hello `id **`"
7932 \begin_layout Standard
7933 Any literal text inserted at the point marked ** would need to be escaped
7934 in all kinds of ways, including
7935 \begin_inset Flex CharStyle:Code
7938 \begin_layout Plain Layout
7945 First it would need escaping for the back-ticks `, and then for the double-quot
7949 \begin_layout Standard
7950 Escaping need not occur if the format and mode of the included chunk matches
7951 that of the including chunk, which would suggest that any back-ticks might
7952 need to be part of the included chunk and not including chunk
7953 \begin_inset Note Note
7956 \begin_layout Plain Layout
7957 or is it the other way around?
7965 \begin_layout Standard
7966 As each chunk is output a new mode tracker for that language is initialized
7967 in it's normal state.
7968 As text is output for that chunk the output mode is tracked.
7969 When a new chunk is included, a transformation appropriate to that mode
7970 is selected and pushed onto a stack of transformations.
7971 Any text to be output is first passed through this stack of transformations.
7974 \begin_layout Standard
7975 It remains to consider if the chunk-include function should return it's
7976 generated text so that the caller can apply any transformations (and formatting
7977 ), or if it should apply the stack of transformations itself.
7980 \begin_layout Standard
7981 Note that the transformed text should have the property of not being able
7982 to change the mode in the current chunk.
7985 \begin_layout Standard
7986 \begin_inset Note Note
7989 \begin_layout Plain Layout
7990 Note chunk parameters should probably also be transformed
8002 \begin_layout Standard
8003 \begin_inset listings
8007 \begin_layout Plain Layout
8009 function new_mode(language, mode) {
8012 \begin_layout Plain Layout
8014 mode["language"] = language;
8017 \begin_layout Plain Layout
8022 \begin_layout Plain Layout
8032 \begin_layout Standard
8033 Because awk functions cannot return an array, we must create the array first
8038 new-mode,params=language;mode
8041 \begin_layout Standard
8042 \begin_inset listings
8046 \begin_layout Plain Layout
8050 chunkref{awk-delete-array}(${mode})>
8053 \begin_layout Plain Layout
8055 new_mode(${language}, ${mode});
8063 \begin_layout Standard
8064 And for tracking modes, we dispatch to a mode-tracker action based on the
8072 \begin_layout Standard
8073 \begin_inset listings
8077 \begin_layout Plain Layout
8079 function track_mode(mode, text) {
8082 \begin_layout Plain Layout
8084 if (mode["language"] == "C") {
8087 \begin_layout Plain Layout
8091 chunkref{track-mode-C}>
8094 \begin_layout Plain Layout
8099 \begin_layout Plain Layout
8104 \begin_layout Plain Layout
8114 \begin_layout Standard
8115 For each mode, we look for a character that has the power to change the
8119 \begin_layout Standard
8124 \labelwidthstring 00.00.0000
8125 \begin_inset Quotes eld
8128 enters double-quote mode
8132 \labelwidthstring 00.00.0000
8133 ' enters single-quote mode
8137 \labelwidthstring 00.00.0000
8140 enters multi-line mode
8144 \labelwidthstring 00.00.0000
8145 # enters #define sub-mode, needs
8147 escaping end of line too
8151 \labelwidthstring 00.00.0000
8152 /* enters comment mode
8155 \begin_layout Standard
8156 In double-quote mode, escape
8159 \begin_inset Quotes eld
8165 \begin_layout Standard
8169 \begin_inset Quotes eld
8175 \begin_layout Standard
8176 \begin_inset Quotes eld
8182 \begin_layout Standard
8183 newline needs to close and re-open string or something
8186 \begin_layout Standard
8187 in single-quote mode escape
8190 \begin_inset Quotes eld
8200 \begin_layout Standard
8201 \begin_inset listings
8205 \begin_layout Plain Layout
8218 \begin_layout Standard
8219 \begin_inset listings
8223 \begin_layout Plain Layout
8227 chunkref{new_mode()}>
8230 \begin_layout Plain Layout
8234 chunkref{parse_chunk_args}>
8242 \begin_layout Chapter
8243 Generating the output
8246 \begin_layout Standard
8247 We generate output by calling output_chunk, or listing the chunk names.
8254 \begin_layout Standard
8255 \begin_inset listings
8259 \begin_layout Plain Layout
8261 if (length(root)) output_chunk(root);
8264 \begin_layout Plain Layout
8266 else output_chunk_names();
8274 \begin_layout Standard
8275 We also have some other output debugging:
8282 \begin_layout Standard
8283 \begin_inset listings
8287 \begin_layout Plain Layout
8292 \begin_layout Plain Layout
8294 print "------ chunk names "
8297 \begin_layout Plain Layout
8299 output_chunk_names();
8302 \begin_layout Plain Layout
8304 print "====== chunks"
8307 \begin_layout Plain Layout
8312 \begin_layout Plain Layout
8314 print "++++++ debug"
8317 \begin_layout Plain Layout
8322 \begin_layout Plain Layout
8324 print a "=" chunks[a];
8327 \begin_layout Plain Layout
8332 \begin_layout Plain Layout
8342 \begin_layout Standard
8343 We do both of these at the end.
8345 \begin_inset Flex CharStyle:Code
8348 \begin_layout Plain Layout
8354 because each chunklet is not necessarily a complete line, and we already
8356 \begin_inset Flex CharStyle:Code
8359 \begin_layout Plain Layout
8365 to each input line in section
8366 \begin_inset CommandInset ref
8368 reference "sub:ORS-chunk-text"
8379 \begin_layout Standard
8380 \begin_inset listings
8384 \begin_layout Plain Layout
8389 \begin_layout Plain Layout
8393 chunkref{debug-output}>
8396 \begin_layout Plain Layout
8401 \begin_layout Plain Layout
8405 chunkref{generate-output}>
8408 \begin_layout Plain Layout
8418 \begin_layout Standard
8419 We write chunk names like this.
8420 If we seem to be running in notangle compatibility mode, then we enclose
8422 \begin_inset Flex CharStyle:Code
8425 \begin_layout Plain Layout
8431 the same way notangle does:
8435 output_chunk_names()
8438 \begin_layout Standard
8439 \begin_inset listings
8443 \begin_layout Plain Layout
8445 function output_chunk_names( c, prefix, suffix)
8448 \begin_layout Plain Layout
8453 \begin_layout Plain Layout
8455 if (notangle_mode) {
8458 \begin_layout Plain Layout
8463 \begin_layout Plain Layout
8468 \begin_layout Plain Layout
8473 \begin_layout Plain Layout
8475 for (c in chunk_names) {
8478 \begin_layout Plain Layout
8480 print prefix c suffix "
8485 \begin_layout Plain Layout
8490 \begin_layout Plain Layout
8500 \begin_layout Standard
8501 This function would write out all chunks
8508 \begin_layout Standard
8509 \begin_inset listings
8513 \begin_layout Plain Layout
8515 function output_chunks( a)
8518 \begin_layout Plain Layout
8523 \begin_layout Plain Layout
8525 for (a in chunk_names) {
8528 \begin_layout Plain Layout
8530 output_chunk(chunk_names[a]);
8533 \begin_layout Plain Layout
8538 \begin_layout Plain Layout
8543 \begin_layout Plain Layout
8547 \begin_layout Plain Layout
8549 function output_chunk(chunk) {
8552 \begin_layout Plain Layout
8557 \begin_layout Plain Layout
8559 lineno_needed = linenos;
8562 \begin_layout Plain Layout
8566 \begin_layout Plain Layout
8571 \begin_layout Plain Layout
8576 \begin_layout Plain Layout
8585 \begin_layout Section
8586 Assembling the chunks
8589 \begin_layout Standard
8590 \begin_inset Flex CharStyle:Code
8593 \begin_layout Plain Layout
8599 holds a string consisting of the names of all the chunks that resulted
8600 in this chunk being output.
8602 \begin_inset Note Note
8605 \begin_layout Plain Layout
8606 Make sure it includes the line numbers too...
8612 It should probably also contain the source line numbers at which each inclusion
8617 write_chunk(),emph={chunk_path}
8620 \begin_layout Standard
8621 \begin_inset listings
8625 \begin_layout Plain Layout
8627 function write_chunk(chunk_name, indent, tail,
8630 \begin_layout Plain Layout
8635 \begin_layout Plain Layout
8637 chunk_path, chunk_args,
8640 \begin_layout Plain Layout
8645 \begin_layout Plain Layout
8647 part, max_part, part_line, frag, max_frag, text,
8650 \begin_layout Plain Layout
8652 chunklet, only_part, call_chunk_args, mode)
8655 \begin_layout Plain Layout
8665 \begin_layout Subsection
8666 \begin_inset CommandInset label
8668 name "sub:Chunk-parts"
8675 \begin_layout Standard
8676 As mentioned in section
8677 \begin_inset CommandInset ref
8679 reference "sub:lstlistings-includes"
8683 , a chunk name may contain a part specifier in square brackets, limiting
8684 the parts that should be emitted.
8687 \begin_layout Standard
8688 \begin_inset listings
8692 \begin_layout Plain Layout
8694 if (match(chunk_name, "^(.*)
8702 ]$", chunk_name_parts)) {
8705 \begin_layout Plain Layout
8707 chunk_name = chunk_name_parts[1];
8710 \begin_layout Plain Layout
8712 only_part = chunk_name_parts[2];
8715 \begin_layout Plain Layout
8725 \begin_layout Standard
8726 We first create the mode tracker for this chunk.
8729 \begin_layout Standard
8732 chunkref{awk-delete-array}(mode)>
8735 \begin_layout Standard
8736 \begin_inset listings
8740 \begin_layout Plain Layout
8745 \begin_layout Plain Layout
8747 new_mode(chunks[chunk_name, "language"], mode);
8755 \begin_layout Standard
8758 chunkref{new-mode}(chunks[chunk_name, "language"], mode)>
8761 \begin_layout Standard
8763 \begin_inset Flex CharStyle:Code
8766 \begin_layout Plain Layout
8772 the names of the parameters that this chunk accepts, whose values were
8773 (optionally) passed in
8774 \begin_inset Flex CharStyle:Code
8777 \begin_layout Plain Layout
8786 \begin_layout Standard
8787 \begin_inset listings
8791 \begin_layout Plain Layout
8793 split(chunks[chunk_name, "params"], chunk_params, " *; *");
8801 \begin_layout Standard
8802 To assemble a chunk, we write out each part.
8809 \begin_layout Standard
8810 \begin_inset listings
8814 \begin_layout Plain Layout
8816 if (! (chunk_name in chunk_names)) {
8819 \begin_layout Plain Layout
8821 error(sprintf(_"The root module <<%s>> was not defined.
8828 \begin_layout Plain Layout
8830 chunk_name, chunk_path));
8833 \begin_layout Plain Layout
8838 \begin_layout Plain Layout
8842 \begin_layout Plain Layout
8844 max_part = chunks[chunk_name, "part"];
8847 \begin_layout Plain Layout
8849 for(part = 1; part <= max_part; part++) {
8852 \begin_layout Plain Layout
8854 if (! only_part || part == only_part) {
8857 \begin_layout Plain Layout
8861 chunkref{write-part}>
8864 \begin_layout Plain Layout
8869 \begin_layout Plain Layout
8874 \begin_layout Plain Layout
8884 \begin_layout Standard
8885 A part can either be a chunklet of lines, or an include of another chunk.
8888 \begin_layout Standard
8889 Chunks may also have parameters, specified in LaTeX style with braces after
8890 the chunk name --- looking like this in the document:
8891 \begin_inset Flex CharStyle:Code
8894 \begin_layout Plain Layout
8895 chunkname{param1, param2}
8901 Arguments are passed in square brackets:
8902 \begin_inset Flex CharStyle:Code
8905 \begin_layout Plain Layout
8908 chunkref{chunkname}[arg1, arg2]
8916 \begin_layout Standard
8917 Before we process each part, we check that the source position hasn't changed
8918 unexpectedly, so that we can know if we need to output a new file-line
8926 \begin_layout Standard
8927 \begin_inset listings
8931 \begin_layout Plain Layout
8935 chunkref{check-source-jump}>
8938 \begin_layout Plain Layout
8942 \begin_layout Plain Layout
8944 chunklet = chunks[chunk_name, "part", part];
8947 \begin_layout Plain Layout
8949 if (chunks[chunk_name, "part", part, "type"] == part_type_chunk) {
8952 \begin_layout Plain Layout
8956 chunkref{write-included-chunk}>
8959 \begin_layout Plain Layout
8961 } else if (chunklet SUBSEP "line" in chunks) {
8964 \begin_layout Plain Layout
8968 chunkref{write-chunklets}>
8971 \begin_layout Plain Layout
8976 \begin_layout Plain Layout
8978 # empty last chunklet
8981 \begin_layout Plain Layout
8991 \begin_layout Standard
8992 To write an included chunk, we must detect any optional chunk arguments
8994 Then we recurse calling
8995 \begin_inset Flex Chunkref
8998 \begin_layout Plain Layout
9008 write-included-chunk
9011 \begin_layout Standard
9012 \begin_inset listings
9016 \begin_layout Plain Layout
9018 if (match(chunklet, "^([^
9030 )$", chunklet_parts)) {
9033 \begin_layout Plain Layout
9035 chunklet = chunklet_parts[1];
9038 \begin_layout Plain Layout
9040 parse_chunk_args("", chunklet_parts[2], call_chunk_args, "(");
9043 \begin_layout Plain Layout
9045 for (c in call_chunk_args) {
9048 \begin_layout Plain Layout
9050 call_chunk_args[c] = expand_chunk_args(call_chunk_args[c], chunk_params,
9054 \begin_layout Plain Layout
9059 \begin_layout Plain Layout
9064 \begin_layout Plain Layout
9066 split("", call_chunk_args);
9069 \begin_layout Plain Layout
9074 \begin_layout Plain Layout
9076 write_chunk(chunklet,
9079 \begin_layout Plain Layout
9081 chunks[chunk_name, "part", part, "indent"] indent,
9084 \begin_layout Plain Layout
9086 chunks[chunk_name, "part", part, "tail"],
9089 \begin_layout Plain Layout
9096 \begin_layout Plain Layout
9106 \begin_layout Standard
9107 Before we output a chunklet of lines, we first emit the file and line number
9108 if we have one, and if it is safe to do so.
9112 \begin_layout Standard
9113 Chunklets are generally broken up by includes, so the start of a chunklet
9114 is a good place to do this.
9115 Then we output each line of the chunklet.
9118 \begin_layout Standard
9119 When it is not safe, such as in the middle of a multi-line macro definition,
9121 \begin_inset Flex CharStyle:Code
9124 \begin_layout Plain Layout
9130 is set to true, and in such a case we note that we want to emit the line
9131 statement when it is next safe.
9138 \begin_layout Standard
9139 \begin_inset listings
9143 \begin_layout Plain Layout
9145 max_frag = chunks[chunklet, "line"];
9148 \begin_layout Plain Layout
9150 for(frag = 1; frag <= max_frag; frag++) {
9153 \begin_layout Plain Layout
9157 chunkref{write-file-line}>
9165 \begin_layout Standard
9166 We then extract the chunklet text and expand any arguments.
9169 \begin_layout Standard
9170 \begin_inset listings
9174 \begin_layout Plain Layout
9178 \begin_layout Plain Layout
9180 text = chunks[chunklet, frag];
9183 \begin_layout Plain Layout
9188 \begin_layout Plain Layout
9193 \begin_layout Plain Layout
9195 text = expand_chunk_args(text, chunk_params, chunk_args);
9203 \begin_layout Standard
9204 If the text is a single newline (which we keep separate - see
9205 \begin_inset CommandInset ref
9207 reference "lone-newline"
9211 ) then we increment the line number.
9212 In the case where this is the last line of a chunk and it is not a top-level
9213 chunk we replace the newline with an empty string --- because the chunk
9214 that included this chunk will have the newline at the end of the line that
9215 included this chunk.
9218 \begin_layout Standard
9220 \begin_inset Flex CharStyle:Code
9223 \begin_layout Plain Layout
9229 that we have started a new line, so that indentation can be managed with
9230 the following piece of text.
9233 \begin_layout Standard
9234 \begin_inset listings
9238 \begin_layout Plain Layout
9242 \begin_layout Plain Layout
9249 \begin_layout Plain Layout
9254 \begin_layout Plain Layout
9256 if (part == max_part && frag == max_frag && length(chunk_path)) {
9259 \begin_layout Plain Layout
9264 \begin_layout Plain Layout
9269 \begin_layout Plain Layout
9274 \begin_layout Plain Layout
9279 \begin_layout Plain Layout
9289 \begin_layout Standard
9290 If this text does not represent a newline, but we see that we are the first
9291 piece of text on a newline, then we prefix our text with the current indent.
9293 \begin_inset Flex CharStyle:Code
9296 \begin_layout Plain Layout
9302 is a global output-state variable, but the
9303 \begin_inset Flex CharStyle:Code
9306 \begin_layout Plain Layout
9316 \begin_layout Standard
9317 \begin_inset listings
9321 \begin_layout Plain Layout
9323 } else if (length(text) || length(tail)) {
9326 \begin_layout Plain Layout
9328 if (newline) text = indent text;
9331 \begin_layout Plain Layout
9336 \begin_layout Plain Layout
9341 \begin_layout Plain Layout
9350 \begin_layout Standard
9351 Tail will soon no longer be relevant once mode-detection is in place.
9354 \begin_layout Standard
9355 \begin_inset listings
9359 \begin_layout Plain Layout
9364 \begin_layout Plain Layout
9366 # track_mode(mode, text);
9369 \begin_layout Plain Layout
9379 \begin_layout Standard
9380 If a line ends in a backslash --- suggesting continuation --- then we supress
9381 outputting file-line as it would probably break the continued lines.
9385 \begin_layout Standard
9386 \begin_inset listings
9390 \begin_layout Plain Layout
9395 \begin_layout Plain Layout
9397 lineno_suppressed = substr(lastline, length(lastline)) == "
9404 \begin_layout Plain Layout
9409 \begin_layout Plain Layout
9419 \begin_layout Standard
9420 Of course there is no point in actually outputting the source filename and
9421 line number (file-line) if they don't say anything new! We only need to
9422 emit them if they aren't what is expected, or if we we not able to emit
9423 one when they had changed.
9430 \begin_layout Standard
9431 \begin_inset listings
9435 \begin_layout Plain Layout
9437 if (newline && lineno_needed && ! lineno_suppressed) {
9440 \begin_layout Plain Layout
9442 filename = a_filename;
9445 \begin_layout Plain Layout
9450 \begin_layout Plain Layout
9452 print "#line " lineno "
9461 \begin_layout Plain Layout
9466 \begin_layout Plain Layout
9476 \begin_layout Standard
9477 We check if a new file-line is needed by checking if the source line matches
9478 what we (or a compiler) would expect.
9486 \begin_layout Standard
9487 \begin_inset listings
9491 \begin_layout Plain Layout
9493 if (linenos && (chunk_name SUBSEP "part" SUBSEP part SUBSEP "FILENAME" in
9497 \begin_layout Plain Layout
9499 a_filename = chunks[chunk_name, "part", part, "FILENAME"];
9502 \begin_layout Plain Layout
9504 a_lineno = chunks[chunk_name, "part", part, "LINENO"];
9507 \begin_layout Plain Layout
9509 if (a_filename != filename || a_lineno != lineno) {
9512 \begin_layout Plain Layout
9517 \begin_layout Plain Layout
9522 \begin_layout Plain Layout
9532 \begin_layout Chapter
9536 \begin_layout Standard
9537 Awk has pretty limited data structures, so we will use two main hashes.
9538 Uninterrupted sequences of a chunk will be stored in
9539 \begin_inset Flex CharStyle:Code
9542 \begin_layout Plain Layout
9548 and the chunklets used in a chunk will be stored in
9549 \begin_inset Flex CharStyle:Code
9552 \begin_layout Plain Layout
9565 \begin_layout Standard
9566 \begin_inset listings
9570 \begin_layout Plain Layout
9575 \begin_layout Plain Layout
9585 \begin_layout Standard
9587 \begin_inset Flex CharStyle:Code
9590 \begin_layout Plain Layout
9596 mentioned are not chunk parameters for parameterized chunks, as mentioned
9598 \begin_inset CommandInset ref
9600 reference "cha:Chunk Arguments"
9604 , but the lstlistings style parameters used in the
9605 \begin_inset Flex CharStyle:Code
9608 \begin_layout Plain Layout
9620 \begin_layout Plain Layout
9622 \begin_inset Flex CharStyle:Code
9625 \begin_layout Plain Layout
9631 parameter is used to hold the parameters for parameterized chunks
9640 chunk-storage-functions
9643 \begin_layout Standard
9644 \begin_inset listings
9648 \begin_layout Plain Layout
9650 function new_chunk(chunk_name, params,
9653 \begin_layout Plain Layout
9658 \begin_layout Plain Layout
9663 \begin_layout Plain Layout
9668 \begin_layout Plain Layout
9670 # HACK WHILE WE CHANGE TO ( ) for PARAM CHUNKS
9673 \begin_layout Plain Layout
9683 )$", "", chunk_name);
9686 \begin_layout Plain Layout
9688 if (! (chunk_name in chunk_names)) {
9691 \begin_layout Plain Layout
9693 if (debug) print "New chunk " chunk_name;
9696 \begin_layout Plain Layout
9698 chunk_names[chunk_name];
9701 \begin_layout Plain Layout
9706 \begin_layout Plain Layout
9708 chunks[chunk_name, p] = params[p];
9711 \begin_layout Plain Layout
9716 \begin_layout Plain Layout
9718 if ("append" in params) {
9721 \begin_layout Plain Layout
9723 append=params["append"];
9726 \begin_layout Plain Layout
9728 if (! (append in chunk_names)) {
9731 \begin_layout Plain Layout
9733 warning("Chunk " chunk_name " is appended to chunk " append " which
9734 is not defined yet");
9737 \begin_layout Plain Layout
9742 \begin_layout Plain Layout
9747 \begin_layout Plain Layout
9749 chunk_include(append, chunk_name);
9752 \begin_layout Plain Layout
9754 chunk_line(append, ORS);
9757 \begin_layout Plain Layout
9762 \begin_layout Plain Layout
9767 \begin_layout Plain Layout
9769 active_chunk = chunk_name;
9772 \begin_layout Plain Layout
9774 prime_chunk(chunk_name);
9777 \begin_layout Plain Layout
9787 \begin_layout Standard
9788 \begin_inset listings
9792 \begin_layout Plain Layout
9796 \begin_layout Plain Layout
9798 function prime_chunk(chunk_name)
9801 \begin_layout Plain Layout
9806 \begin_layout Plain Layout
9808 chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] =
9813 \begin_layout Plain Layout
9815 chunk_name SUBSEP "chunklet" SUBSEP "" ++chunks[chunk_name, "chunklet"]
9819 \begin_layout Plain Layout
9821 chunks[chunk_name, "part", chunks[chunk_name, "part"], "FILENAME"] = FILENAME;
9824 \begin_layout Plain Layout
9826 chunks[chunk_name, "part", chunks[chunk_name, "part"], "LINENO"] = FNR
9830 \begin_layout Plain Layout
9835 \begin_layout Plain Layout
9839 \begin_layout Plain Layout
9841 function chunk_line(chunk_name, line){
9844 \begin_layout Plain Layout
9846 chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
9849 \begin_layout Plain Layout
9851 ++chunks[chunk_name, "chunklet", chunks[chunk_name, "chunklet"],
9855 \begin_layout Plain Layout
9860 \begin_layout Plain Layout
9869 \begin_layout Standard
9870 Chunk include represents a
9874 statement, and stores the requirement to include another chunk.
9875 The parameter indent represents the quanity of literal text characters
9880 statement and therefore by how much additional lines of the included chunk
9884 \begin_layout Standard
9885 \begin_inset listings
9889 \begin_layout Plain Layout
9891 function chunk_include(chunk_name, chunk_ref, indent, tail)
9894 \begin_layout Plain Layout
9899 \begin_layout Plain Layout
9901 chunks[chunk_name, "part", ++chunks[chunk_name, "part"] ] = chunk_ref;
9904 \begin_layout Plain Layout
9906 chunks[chunk_name, "part", chunks[chunk_name, "part"], "type" ] = part_type_ch
9910 \begin_layout Plain Layout
9912 chunks[chunk_name, "part", chunks[chunk_name, "part"], "indent" ] = indent_str
9916 \begin_layout Plain Layout
9918 chunks[chunk_name, "part", chunks[chunk_name, "part"], "tail" ] = tail;
9921 \begin_layout Plain Layout
9923 prime_chunk(chunk_name);
9926 \begin_layout Plain Layout
9931 \begin_layout Plain Layout
9940 \begin_layout Standard
9941 The indent is calculated by indent_string, which may in future convert some
9942 spaces into tab characters.
9943 This function works by generating a printf padded format string, like
9944 \begin_inset Flex CharStyle:Code
9947 \begin_layout Plain Layout
9953 for an indent of 22, and then printing an empty string using that format.
9956 \begin_layout Standard
9957 \begin_inset listings
9961 \begin_layout Plain Layout
9963 function indent_string(indent) {
9966 \begin_layout Plain Layout
9968 return sprintf("%" indent "s", "");
9971 \begin_layout Plain Layout
9981 \begin_layout Chapter
9982 \begin_inset CommandInset label
9991 \begin_layout Standard
9992 I use Arnold Robbins public domain getopt (1993 revision).
9993 This is probably the same one that is covered in chapter 12 of
9994 \begin_inset Quotes eld
9997 Edition 3 of GAWK: Effective AWK Programming: A User's Guide for GNU Awk
9998 \begin_inset Quotes erd
10001 but as that is licensed under the GNU Free Documentation License, Version
10002 1.3, which conflicts with the GPL3, I can't use it from there (or it's accompany
10003 ing explanations), so I do my best to explain how it works here.
10006 \begin_layout Standard
10007 The getopt.awk header is:
10010 \begin_layout Chunk
10011 getopt.awk-header,language=awk,morestring=[b]{/},morekeywords=else
10014 \begin_layout Standard
10015 \begin_inset listings
10019 \begin_layout Plain Layout
10021 # getopt.awk --- do C library getopt(3) function in awk
10024 \begin_layout Plain Layout
10029 \begin_layout Plain Layout
10031 # Arnold Robbins, arnold@skeeve.com, Public Domain
10034 \begin_layout Plain Layout
10039 \begin_layout Plain Layout
10041 # Initial version: March, 1991
10044 \begin_layout Plain Layout
10046 # Revised: May, 1993
10054 \begin_layout Standard
10055 The provided explanation is:
10058 \begin_layout Chunk
10062 \begin_layout Standard
10063 \begin_inset listings
10067 \begin_layout Plain Layout
10069 # External variables:
10072 \begin_layout Plain Layout
10074 # Optind -- index in ARGV of first nonoption argument
10077 \begin_layout Plain Layout
10079 # Optarg -- string value of argument to current option
10082 \begin_layout Plain Layout
10084 # Opterr -- if nonzero, print our own diagnostic
10087 \begin_layout Plain Layout
10089 # Optopt -- current option letter
10092 \begin_layout Plain Layout
10096 \begin_layout Plain Layout
10101 \begin_layout Plain Layout
10103 # -1 at end of options
10106 \begin_layout Plain Layout
10108 # ? for unrecognized option
10111 \begin_layout Plain Layout
10113 # <c> a character representing the current option
10116 \begin_layout Plain Layout
10120 \begin_layout Plain Layout
10125 \begin_layout Plain Layout
10127 # _opti -- index in multi-flag option, e.g., -abc
10135 \begin_layout Standard
10136 The function follows.
10137 The final two parameters,
10138 \begin_inset Flex CharStyle:Code
10141 \begin_layout Plain Layout
10148 \begin_inset Flex CharStyle:Code
10151 \begin_layout Plain Layout
10157 are local variables and not parameters --- as indicated by the multiple
10158 spaces preceding them.
10159 Awk doesn't care, the multiple spaces are a convention to help us humans.
10162 \begin_layout Chunk
10163 getopt.awk-getopt()
10166 \begin_layout Standard
10167 \begin_inset listings
10171 \begin_layout Plain Layout
10173 function getopt(argc, argv, options, thisopt, i)
10176 \begin_layout Plain Layout
10181 \begin_layout Plain Layout
10183 if (length(options) == 0) # no options given
10186 \begin_layout Plain Layout
10191 \begin_layout Plain Layout
10193 if (argv[Optind] == "--") { # all done
10196 \begin_layout Plain Layout
10201 \begin_layout Plain Layout
10206 \begin_layout Plain Layout
10211 \begin_layout Plain Layout
10213 } else if (argv[Optind] !~ /^-[^:
10228 \begin_layout Plain Layout
10233 \begin_layout Plain Layout
10238 \begin_layout Plain Layout
10243 \begin_layout Plain Layout
10248 \begin_layout Plain Layout
10253 \begin_layout Plain Layout
10255 thisopt = substr(argv[Optind], _opti, 1)
10258 \begin_layout Plain Layout
10263 \begin_layout Plain Layout
10265 i = index(options, thisopt)
10268 \begin_layout Plain Layout
10273 \begin_layout Plain Layout
10278 \begin_layout Plain Layout
10280 printf("%c -- invalid option
10285 \begin_layout Plain Layout
10287 thisopt) > "/dev/stderr"
10290 \begin_layout Plain Layout
10292 if (_opti >= length(argv[Optind])) {
10295 \begin_layout Plain Layout
10300 \begin_layout Plain Layout
10305 \begin_layout Plain Layout
10310 \begin_layout Plain Layout
10315 \begin_layout Plain Layout
10320 \begin_layout Plain Layout
10330 \begin_layout Standard
10331 At this point, the option has been found and we need to know if it takes
10335 \begin_layout Standard
10336 \begin_inset listings
10340 \begin_layout Plain Layout
10342 if (substr(options, i + 1, 1) == ":") {
10345 \begin_layout Plain Layout
10347 # get option argument
10350 \begin_layout Plain Layout
10352 if (length(substr(argv[Optind], _opti + 1)) > 0)
10355 \begin_layout Plain Layout
10357 Optarg = substr(argv[Optind], _opti + 1)
10360 \begin_layout Plain Layout
10365 \begin_layout Plain Layout
10367 Optarg = argv[++Optind]
10370 \begin_layout Plain Layout
10375 \begin_layout Plain Layout
10380 \begin_layout Plain Layout
10385 \begin_layout Plain Layout
10387 if (_opti == 0 || _opti >= length(argv[Optind])) {
10390 \begin_layout Plain Layout
10395 \begin_layout Plain Layout
10400 \begin_layout Plain Layout
10405 \begin_layout Plain Layout
10410 \begin_layout Plain Layout
10415 \begin_layout Plain Layout
10422 A test program is built in, too
10425 \begin_layout Chunk
10429 \begin_layout Standard
10430 \begin_inset listings
10434 \begin_layout Plain Layout
10439 \begin_layout Plain Layout
10441 Opterr = 1 # default is to diagnose
10444 \begin_layout Plain Layout
10446 Optind = 1 # skip ARGV[0]
10449 \begin_layout Plain Layout
10454 \begin_layout Plain Layout
10456 if (_getopt_test) {
10459 \begin_layout Plain Layout
10461 while ((_go_c = getopt(ARGC, ARGV, "ab:cd")) != -1)
10464 \begin_layout Plain Layout
10466 printf("c = <%c>, optarg = <%s>
10471 \begin_layout Plain Layout
10476 \begin_layout Plain Layout
10478 printf("non-option arguments:
10483 \begin_layout Plain Layout
10485 for (; Optind < ARGC; Optind++)
10488 \begin_layout Plain Layout
10497 \begin_layout Plain Layout
10499 Optind, ARGV[Optind])
10502 \begin_layout Plain Layout
10507 \begin_layout Plain Layout
10517 \begin_layout Standard
10518 The entire getopt.awk is made out of these chunks in order
10521 \begin_layout Chunk
10525 \begin_layout Standard
10526 \begin_inset listings
10530 \begin_layout Plain Layout
10534 chunkref{getopt.awk-header}>
10537 \begin_layout Plain Layout
10541 \begin_layout Plain Layout
10545 chunkref{getopt.awk-notes}>
10548 \begin_layout Plain Layout
10552 chunkref{getopt.awk-getopt()}>
10555 \begin_layout Plain Layout
10559 chunkref{getopt.awk-begin}>
10567 \begin_layout Standard
10568 Although we only want the header and function:
10571 \begin_layout Chunk
10575 \begin_layout Standard
10576 \begin_inset listings
10580 \begin_layout Plain Layout
10582 # try: locate getopt.awk for the full original file
10585 \begin_layout Plain Layout
10587 # as part of your standard awk installation
10590 \begin_layout Plain Layout
10594 chunkref{getopt.awk-header}>
10597 \begin_layout Plain Layout
10601 \begin_layout Plain Layout
10605 chunkref{getopt.awk-getopt()}>
10613 \begin_layout Chapter
10614 Newfangle LaTeX source code
10617 \begin_layout Section
10621 \begin_layout Standard
10622 Here we define a Lyx .module file that makes it convenient to use LyX for
10623 writing such literate programs.
10626 \begin_layout Standard
10628 \begin_inset Flex CharStyle:Code
10631 \begin_layout Plain Layout
10637 can be installed in your personal
10638 \begin_inset Flex CharStyle:Code
10641 \begin_layout Plain Layout
10642 .lyx/layouts folder
10648 You will need to Tools Reconfigure so that LyX notices it.
10649 It adds a new format Chunk, which should precede every listing and contain
10654 \begin_layout Chunk
10655 ./newfangle.module,language=
10658 \begin_layout Standard
10659 \begin_inset listings
10663 \begin_layout Plain Layout
10667 DeclareLyXModule{Newfangle Literate Listings}
10670 \begin_layout Plain Layout
10675 \begin_layout Plain Layout
10677 # Newfangle literate listings allow one to write
10680 \begin_layout Plain Layout
10682 # literate programs after the fashion of noweb, but without having
10685 \begin_layout Plain Layout
10687 # to use noweave to generate the documentation.
10688 Instead the listings
10691 \begin_layout Plain Layout
10693 # package is extended in conjunction with the noweb package to implement
10696 \begin_layout Plain Layout
10698 # to code formating directly as latex.
10701 \begin_layout Plain Layout
10703 # The newfangle awk script
10706 \begin_layout Plain Layout
10711 \begin_layout Plain Layout
10715 \begin_layout Plain Layout
10720 \begin_layout Plain Layout
10724 \begin_layout Plain Layout
10729 \begin_layout Plain Layout
10733 chunkref{./newfangle.sty}>
10736 \begin_layout Plain Layout
10741 \begin_layout Plain Layout
10745 \begin_layout Plain Layout
10749 chunkref{chunkstyle}>
10752 \begin_layout Plain Layout
10756 \begin_layout Plain Layout
10760 chunkref{chunkref}>
10768 \begin_layout Subsection
10772 \begin_layout Standard
10777 style is to make it easier for LyX users to provide the name to
10778 \begin_inset Flex CharStyle:Code
10781 \begin_layout Plain Layout
10790 Normally this requires right-clicking on the listing, choosing settings,
10791 advanced, and then typing
10792 \begin_inset Flex CharStyle:Code
10795 \begin_layout Plain Layout
10802 This has the further disadvantage that the name (and other options) are
10803 not generally visible during document editing.
10806 \begin_layout Standard
10807 The chunk style is defined as a LaTeX command, so that all text on the same
10808 line is passed to the LaTeX command
10809 \begin_inset Flex CharStyle:Code
10812 \begin_layout Plain Layout
10819 This makes it easy to parse using
10820 \begin_inset Flex CharStyle:Code
10823 \begin_layout Plain Layout
10829 , and easy to pass these options on to the listings package.
10830 The first word in a chunk section should be the chunk name, and will have
10832 \begin_inset Flex CharStyle:Code
10835 \begin_layout Plain Layout
10842 Any other words are accepted arguments to
10843 \begin_inset Flex CharStyle:Code
10846 \begin_layout Plain Layout
10857 \begin_layout Standard
10858 We set PassThru to 1 because the user is actually entering raw latex.
10861 \begin_layout Chunk
10865 \begin_layout Standard
10866 \begin_inset listings
10870 \begin_layout Plain Layout
10875 \begin_layout Plain Layout
10880 \begin_layout Plain Layout
10885 \begin_layout Plain Layout
10887 Margin First_Dynamic
10890 \begin_layout Plain Layout
10892 LeftMargin Chunk:xxx
10895 \begin_layout Plain Layout
10900 \begin_layout Plain Layout
10905 \begin_layout Plain Layout
10907 LabelString "Chunk:"
10910 \begin_layout Plain Layout
10915 \begin_layout Plain Layout
10920 \begin_layout Plain Layout
10929 \begin_layout Standard
10930 To make the label very visible we choose a larger font coloured red.
10933 \begin_layout Standard
10934 \begin_inset listings
10938 \begin_layout Plain Layout
10943 \begin_layout Plain Layout
10948 \begin_layout Plain Layout
10953 \begin_layout Plain Layout
10958 \begin_layout Plain Layout
10963 \begin_layout Plain Layout
10968 \begin_layout Plain Layout
10973 \begin_layout Plain Layout
10983 \begin_layout Subsection
10987 \begin_layout Standard
10988 We also define the Chunkref style which can be used to express cross references
10992 \begin_layout Chunk
10996 \begin_layout Standard
10997 \begin_inset listings
11001 \begin_layout Plain Layout
11003 InsetLayout Chunkref
11006 \begin_layout Plain Layout
11011 \begin_layout Plain Layout
11016 \begin_layout Plain Layout
11021 \begin_layout Plain Layout
11026 \begin_layout Plain Layout
11031 \begin_layout Plain Layout
11036 \begin_layout Plain Layout
11041 \begin_layout Plain Layout
11046 \begin_layout Plain Layout
11056 \begin_layout Section
11057 \begin_inset CommandInset label
11059 name "sec:Latex-Macros"
11066 \begin_layout Standard
11080 As noweb defines it's own
11081 \begin_inset Flex CharStyle:Code
11084 \begin_layout Plain Layout
11092 environment, we re-define the one that LyX logical markup module expects
11096 \begin_layout Chunk
11097 ./newfangle.sty,language=tex,basicstyle=
11102 \begin_layout Standard
11103 \begin_inset listings
11107 \begin_layout Plain Layout
11111 usepackage{listings}%
11114 \begin_layout Plain Layout
11121 \begin_layout Plain Layout
11128 \begin_layout Plain Layout
11144 \begin_layout Standard
11146 \begin_inset Flex CharStyle:Code
11149 \begin_layout Plain Layout
11156 \begin_inset Flex CharStyle:Code
11159 \begin_layout Plain Layout
11167 which will need renaming to
11168 \begin_inset Flex CharStyle:Code
11171 \begin_layout Plain Layout
11179 when I can do this without clashing with
11180 \begin_inset Flex CharStyle:Code
11183 \begin_layout Plain Layout
11194 \begin_layout Standard
11195 \begin_inset listings
11199 \begin_layout Plain Layout
11203 lstnewenvironment{Chunk}{
11215 \begin_layout Standard
11216 We also define a suitable
11217 \begin_inset Flex CharStyle:Code
11220 \begin_layout Plain Layout
11228 of parameters that suit the literate programming style after the fashion
11236 \begin_layout Standard
11237 \begin_inset listings
11241 \begin_layout Plain Layout
11245 lstset{numbers=left, stepnumber=5, numbersep=5pt,
11248 \begin_layout Plain Layout
11250 breaklines=false,basicstyle=
11255 \begin_layout Plain Layout
11267 \begin_layout Standard
11268 We also define a notangle-like mechanism for
11272 to LaTeX from the listing, and by which we can refer to other listings.
11274 \begin_inset Flex CharStyle:Code
11277 \begin_layout Plain Layout
11278 =<\SpecialChar \ldots{}
11284 sequence to contain LaTeX code, and include another like this chunk:
11285 \begin_inset Flex CharStyle:Code
11288 \begin_layout Plain Layout
11291 chunkref{chunkname}>
11298 \begin_inset Flex CharStyle:Code
11301 \begin_layout Plain Layout
11302 =<\SpecialChar \ldots{}
11308 is already defined to contain LaTeX code for this document --- this is
11313 document after all --- the code fragment below effectively contains the
11315 \begin_inset Flex CharStyle:Code
11318 \begin_layout Plain Layout
11325 To avoid problems with document generation, I had to declare an lstlistings
11327 \begin_inset Flex CharStyle:Code
11330 \begin_layout Plain Layout
11336 for this listing only; which in LyX was done by right-clicking the listings
11338 \begin_inset Flex CharStyle:Code
11341 \begin_layout Plain Layout
11347 \SpecialChar \menuseparator
11349 \begin_inset Flex CharStyle:Code
11352 \begin_layout Plain Layout
11361 \begin_layout Standard
11362 \begin_inset Note Note
11365 \begin_layout Plain Layout
11366 =< isn't enjoyed literally here, in a listing when the escape sequence is
11367 already defined as shown...
11368 we need to somehow escape this representation...
11376 \begin_layout Standard
11377 \begin_inset listings
11378 lstparams "escapeinside={}"
11382 \begin_layout Plain Layout
11386 lstset{escapeinside={=<}{>}}%
11394 \begin_layout Standard
11395 Although our macros will contain the @ symbol, they will be included in
11397 \begin_inset Flex CharStyle:Code
11400 \begin_layout Plain Layout
11408 section by LyX; however we keep the commented out
11409 \begin_inset Flex CharStyle:Code
11412 \begin_layout Plain Layout
11421 The listings package likes to centre the titles, but noweb titles are specially
11422 formatted and must be left aligned.
11423 The simplest way to do this turned out to be by removing the definition
11425 \begin_inset Flex CharStyle:Code
11428 \begin_layout Plain Layout
11437 This may interact badly if other listings want a regular title or caption.
11438 We remember the old maketitle in case we need it.
11441 \begin_layout Standard
11442 \begin_inset listings
11446 \begin_layout Plain Layout
11453 \begin_layout Plain Layout
11455 %somehow re-defining maketitle gives us a left-aligned title
11458 \begin_layout Plain Layout
11460 %which is extactly what our specially formatted title needs!
11463 \begin_layout Plain Layout
11471 newfangle@lst@maketitle
11476 \begin_layout Plain Layout
11492 \begin_layout Subsection
11493 \begin_inset CommandInset label
11495 name "sub:The-chunk-command"
11502 \begin_layout Standard
11503 Our chunk command accepts one argument, and calls
11504 \begin_inset Flex CharStyle:Code
11507 \begin_layout Plain Layout
11517 \begin_inset Flex CharStyle:Code
11520 \begin_layout Plain Layout
11528 will note the name, this is erased when the next
11529 \begin_inset Flex CharStyle:Code
11532 \begin_layout Plain Layout
11540 starts, so we make a note of this in
11541 \begin_inset Flex CharStyle:Code
11544 \begin_layout Plain Layout
11552 and restore in in lstlistings Init hook.
11555 \begin_layout Standard
11556 \begin_inset listings
11560 \begin_layout Plain Layout
11569 \begin_layout Plain Layout
11575 newfanglecaption},name=#1}%
11578 \begin_layout Plain Layout
11591 \begin_layout Plain Layout
11596 \begin_layout Plain Layout
11612 \begin_layout Subsubsection
11616 \begin_layout Standard
11617 Newfangle permits parameterized chunks, and requires the paramters to be
11618 specified as listings options.
11619 The newfangle script uses this, and although we don't do anything with
11620 these in the LaTeX code right now, we need to stop the listings package
11624 \begin_layout Standard
11625 \begin_inset listings
11629 \begin_layout Plain Layout
11639 newfangle@chunk@params{#1}}%
11647 \begin_layout Standard
11648 As it is common to define a chunk which then needs appending to another
11649 chunk, and annoying to have to declare a single line chunk to manage the
11650 include, we support an
11651 \begin_inset Flex CharStyle:Code
11654 \begin_layout Plain Layout
11664 \begin_layout Standard
11665 \begin_inset listings
11669 \begin_layout Plain Layout
11679 newfangle@chunk@append{#1}}%
11687 \begin_layout Subsection
11688 The noweb styled caption
11691 \begin_layout Standard
11692 We define a public macro
11693 \begin_inset Flex CharStyle:Code
11696 \begin_layout Plain Layout
11704 which can be set as a regular title.
11706 \begin_inset Flex CharStyle:Code
11709 \begin_layout Plain Layout
11718 \begin_inset Flex CharStyle:Code
11721 \begin_layout Plain Layout
11729 at the appriate time when the caption is emitted.
11732 \begin_layout Standard
11733 \begin_inset listings
11737 \begin_layout Plain Layout
11747 newfangle@caption}%
11755 \begin_layout Standard
11756 \begin_inset Float figure
11762 \begin_layout Plain Layout
11763 \begin_inset Box Boxed
11772 height_special "totalheight"
11775 \begin_layout Plain Layout
11777 \begin_inset space \qquad{}
11785 \begin_inset Formula $\equiv+$
11789 \begin_inset space \qquad{}
11793 \begin_inset space \qquad{}
11797 \begin_inset space \qquad{}
11801 \begin_inset Formula $\triangleleft$
11805 \begin_inset space \quad{}
11809 \begin_inset Formula $\triangleright$
11815 \begin_layout Plain Layout
11818 In this example, the current chunk is 22c, and therefore the third chunk
11822 \begin_layout Plain Layout
11833 \begin_layout Plain Layout
11836 The first chunk with this name (19b) occurs as the second chunk on page
11840 \begin_layout Plain Layout
11843 The previous chunk (22d) with the same name is the second chunk on page
11847 \begin_layout Plain Layout
11850 The next chunk (24d) is the fourth chunk on page 24.
11853 \begin_layout Plain Layout
11854 \begin_inset Caption
11856 \begin_layout Plain Layout
11872 The general noweb output format compactly identifies the current chunk,
11873 and references to the first chunk, and the previous and next chunks that
11874 have the same name.
11878 \begin_layout Standard
11879 This means that we need to keep a counter for each chunk-name, that we use
11880 to count chunks of the same name.
11884 \begin_layout Subsection
11888 \begin_layout Standard
11889 It would be natural to have a counter for each chunk name, but TeX would
11890 soon run out of counters
11894 \begin_layout Plain Layout
11895 \SpecialChar \ldots{}
11900 run out of counters and so I had to re-write the LaTeX macros to share
11901 a counter as described here
11906 , so we have one counter which we save at the end of a chunk and restore
11907 at the beginning of a chunk.
11910 \begin_layout Standard
11911 \begin_inset listings
11915 \begin_layout Plain Layout
11919 newcounter{newfangle@chunkcounter}%
11927 \begin_layout Standard
11928 We construct the name of this variable to store the counter to be the text
11930 \begin_inset Flex CharStyle:Code
11933 \begin_layout Plain Layout
11939 prefixed onto the chunks own name, and store it in
11940 \begin_inset Flex CharStyle:Code
11943 \begin_layout Plain Layout
11955 \begin_layout Standard
11956 We save the counter like this:
11959 \begin_layout Chunk
11963 \begin_layout Standard
11964 \begin_inset listings
11968 \begin_layout Plain Layout
11984 arabic{newfangle@chunkcounter}}%
11992 \begin_layout Standard
11993 and restore the counter like this:
11996 \begin_layout Chunk
12000 \begin_layout Standard
12001 \begin_inset listings
12005 \begin_layout Plain Layout
12009 setcounter{newfangle@chunkcounter}{
12023 \begin_layout Chunk
12027 \begin_layout Standard
12028 If there does not already exist a variable whose name is stored in
12029 \begin_inset Flex CharStyle:Code
12032 \begin_layout Plain Layout
12040 , then we know we are the first chunk with this name, and then define a
12045 \begin_layout Standard
12046 Although chunks of the same name share a common counter, they must still
12048 We use is the internal name of the listing, suffixed by the counter value.
12049 So the first chunk might be
12050 \begin_inset Flex CharStyle:Code
12053 \begin_layout Plain Layout
12059 and the second chunk be
12060 \begin_inset Flex CharStyle:Code
12063 \begin_layout Plain Layout
12072 \begin_layout Standard
12073 We also calculate the name of the previous chunk if we can (before we increment
12074 the chunk counter).
12075 If this is the first chunk of that name, then
12076 \begin_inset Flex CharStyle:Code
12079 \begin_layout Plain Layout
12088 \begin_inset Flex CharStyle:Code
12091 \begin_layout Plain Layout
12099 which the noweb package will interpret as not existing.
12102 \begin_layout Standard
12103 \begin_inset listings
12107 \begin_layout Plain Layout
12113 newfangle@caption{%
12116 \begin_layout Plain Layout
12122 chunkcount{lst-chunk-
12127 \begin_layout Plain Layout
12136 \begin_layout Plain Layout
12151 \begin_layout Plain Layout
12155 setcounter{newfangle@chunkcounter}{
12164 \begin_layout Plain Layout
12175 \begin_layout Plain Layout
12180 \begin_layout Plain Layout
12184 setcounter{newfangle@chunkcounter}{
12193 \begin_layout Plain Layout
12203 arabic{newfangle@chunkcounter}}%
12206 \begin_layout Plain Layout
12216 \begin_layout Standard
12217 After incrementing the chunk counter, we then define the name of this chunk,
12218 as well as the name of the first chunk.
12221 \begin_layout Standard
12222 \begin_inset listings
12226 \begin_layout Plain Layout
12230 addtocounter{newfangle@chunkcounter}{1}%
12233 \begin_layout Plain Layout
12249 arabic{newfangle@chunkcounter}}%
12252 \begin_layout Plain Layout
12262 arabic{newfangle@chunkcounter}}%
12265 \begin_layout Plain Layout
12281 \begin_layout Standard
12282 We now need to calculate the name of the next chunk.
12283 We do this by temporarily skipping the counter on by one; however there
12284 may not actually be another chunk with this name! We detect this by also
12285 defining a label for each chunk based on the chunkname.
12286 If there is a next chunkname then it will define a label with that name.
12287 As labels are persistent, we can at least tell the second time LaTeX is
12289 If we don't find such a defined label then we define
12290 \begin_inset Flex CharStyle:Code
12293 \begin_layout Plain Layout
12302 \begin_inset Flex CharStyle:Code
12305 \begin_layout Plain Layout
12316 \begin_layout Standard
12317 \begin_inset listings
12321 \begin_layout Plain Layout
12325 addtocounter{newfangle@chunkcounter}{1}%
12328 \begin_layout Plain Layout
12338 arabic{newfangle@chunkcounter}}%
12341 \begin_layout Plain Layout
12345 @ifundefined{r@label-
12361 \begin_layout Standard
12362 The noweb package requires that we define a
12363 \begin_inset Flex CharStyle:Code
12366 \begin_layout Plain Layout
12374 for every chunk, with a unique name, which is then used to print out it's
12378 \begin_layout Standard
12379 We also define a regular label for this chunk, as was mentioned above when
12381 \begin_inset Flex CharStyle:Code
12384 \begin_layout Plain Layout
12393 This requires LaTeX to be run at least twice after new chunk sections are
12394 added --- but noweb requried that anyway.
12397 \begin_layout Standard
12398 \begin_inset listings
12402 \begin_layout Plain Layout
12411 \begin_layout Plain Layout
12413 % define this label for every chunk instance, so we
12416 \begin_layout Plain Layout
12418 % can tell when we are the last chunk of this name
12421 \begin_layout Plain Layout
12435 \begin_layout Standard
12436 We also try and add the chunk to the list of listings, but I'm afraid we
12437 don't do very well.
12438 We want each chunk name listing once, with all of it's references.
12441 \begin_layout Standard
12442 \begin_inset listings
12446 \begin_layout Plain Layout
12450 addcontentsline{lol}{lstlisting}{
12466 \begin_layout Standard
12467 We then call the noweb output macros in the same way that noweave generates
12468 them, except that we don't need to call
12469 \begin_inset Flex CharStyle:Code
12472 \begin_layout Plain Layout
12475 nwstartdeflinemarkup
12481 \begin_inset Flex CharStyle:Code
12484 \begin_layout Plain Layout
12492 -- and if we do it messes up the output somewhat.
12495 \begin_layout Standard
12496 \begin_inset listings
12500 \begin_layout Plain Layout
12507 \begin_layout Plain Layout
12512 \begin_layout Plain Layout
12519 \begin_layout Plain Layout
12528 \begin_layout Plain Layout
12533 \begin_layout Plain Layout
12538 \begin_layout Plain Layout
12543 \begin_layout Plain Layout
12550 \begin_layout Plain Layout
12557 \begin_layout Plain Layout
12562 \begin_layout Plain Layout
12571 \begin_layout Plain Layout
12575 @ifundefined{newfangle@chunk@params}{}{%
12578 \begin_layout Plain Layout
12582 newfangle@chunk@params)%
12585 \begin_layout Plain Layout
12590 \begin_layout Plain Layout
12601 \begin_layout Plain Layout
12610 \begin_layout Plain Layout
12615 \begin_layout Plain Layout
12619 @ifundefined{newfangle@chunk@append}{}{%
12622 \begin_layout Plain Layout
12628 newfangle@chunk@append{x}
12633 \begin_layout Plain Layout
12637 newfangle@chunk@append%
12640 \begin_layout Plain Layout
12647 \begin_layout Plain Layout
12652 \begin_layout Plain Layout
12660 newfangle@chunk@append{}%
12663 \begin_layout Plain Layout
12670 \begin_layout Plain Layout
12675 \begin_layout Plain Layout
12680 \begin_layout Plain Layout
12699 \begin_layout Plain Layout
12703 nwstartdeflinemarkup%
12706 \begin_layout Plain Layout
12717 \begin_layout Plain Layout
12721 nwenddeflinemarkup%
12724 \begin_layout Plain Layout
12734 \begin_layout Standard
12735 Originally this was developed as a
12736 \begin_inset Flex CharStyle:Code
12739 \begin_layout Plain Layout
12745 aspect, in the Init hook, but it was found easier to affect the title without
12747 \begin_inset Flex CharStyle:Code
12750 \begin_layout Plain Layout
12753 lst@AddToHookExe{PreSet}
12758 is still required to set the listings name to the name passed to the
12759 \begin_inset Flex CharStyle:Code
12762 \begin_layout Plain Layout
12773 \begin_layout Standard
12774 \begin_inset listings
12778 \begin_layout Plain Layout
12782 lst@BeginAspect{newfangle}
12785 \begin_layout Plain Layout
12789 lst@Key{newfangle}{true}[t]{
12791 lstKV@SetIf{#1}{true}}
12794 \begin_layout Plain Layout
12798 lst@AddToHookExe{PreSet}{
12809 \begin_layout Plain Layout
12813 lst@AddToHook{Init}{}%
12818 \begin_layout Plain Layout
12830 \begin_layout Subsection
12834 \begin_layout Standard
12837 chunkref command which makes it easy to generate visual references to different
12841 \begin_layout Standard
12842 \begin_inset Tabular
12843 <lyxtabular version="3" rows="4" columns="2">
12845 <column alignment="center" valignment="top" width="0">
12846 <column alignment="center" valignment="top" width="0">
12848 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
12851 \begin_layout Plain Layout
12857 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
12860 \begin_layout Plain Layout
12868 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
12871 \begin_layout Plain Layout
12879 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
12882 \begin_layout Plain Layout
12886 \begin_layout Plain Layout
12902 <cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
12905 \begin_layout Plain Layout
12908 chunkref[3]{preamble}
12913 <cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
12916 \begin_layout Plain Layout
12920 \begin_layout Plain Layout
12924 chunkref[3]{preamble}
12936 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" usebox="none">
12939 \begin_layout Plain Layout
12942 chunkref{preamble}[arg1, arg2]
12947 <cell alignment="center" valignment="top" topline="true" bottomline="true" leftline="true" rightline="true" usebox="none">
12950 \begin_layout Plain Layout
12954 \begin_layout Plain Layout
12958 chunkref{preamble}[arg1, arg2]
12976 \begin_layout Standard
12977 Chunkref can also be used within a code chunk to include another code chunk.
12978 The third optional parameter to chunkref is a comma sepatarated list of
12979 arguments, which will replace defined parameters in the chunkref.
12980 \begin_inset Note Note
12983 \begin_layout Plain Layout
12984 Darn it, if I have: =<
12986 chunkref{new-mode}[{chunks[chunk_name, "language"]},{mode}]> the inner braces
12987 (inside [ ]) cause _ to signify subscript even though we have lst@ReplaceIn
12995 \begin_layout Standard
12996 \begin_inset listings
13000 \begin_layout Plain Layout
13009 \begin_layout Plain Layout
13018 \begin_layout Plain Layout
13029 \begin_layout Plain Layout
13036 \begin_layout Plain Layout
13047 \begin_layout Plain Layout
13052 \begin_layout Plain Layout
13061 \begin_layout Plain Layout
13067 chunkref@i{#1}{#2}}{
13069 chunkref@i{#1}{#2}()}%
13072 \begin_layout Plain Layout
13077 \begin_layout Plain Layout
13083 chunkref@i#1#2(#3){%
13086 \begin_layout Plain Layout
13095 \begin_layout Plain Layout
13104 \begin_layout Plain Layout
13113 \begin_layout Plain Layout
13122 \begin_layout Plain Layout
13133 \begin_layout Plain Layout
13142 \begin_layout Plain Layout
13149 \begin_layout Plain Layout
13160 \begin_layout Plain Layout
13167 \begin_layout Plain Layout
13178 \begin_layout Plain Layout
13189 \begin_layout Plain Layout
13198 \begin_layout Plain Layout
13205 \begin_layout Plain Layout
13210 \begin_layout Plain Layout
13219 \begin_layout Plain Layout
13230 \begin_layout Plain Layout
13237 \begin_layout Plain Layout
13244 \begin_layout Plain Layout
13251 \begin_layout Plain Layout
13262 \begin_layout Plain Layout
13269 \begin_layout Plain Layout
13273 chunkref@args #3,)%
13276 \begin_layout Plain Layout
13283 \begin_layout Plain Layout
13292 \begin_layout Plain Layout
13297 \begin_layout Plain Layout
13302 \begin_layout Plain Layout
13311 \begin_layout Plain Layout
13321 \begin_layout Subsection
13325 \begin_layout Standard
13326 \begin_inset listings
13330 \begin_layout Plain Layout
13335 \begin_layout Plain Layout
13347 \begin_layout Chapter
13348 Extracting newfangle
13351 \begin_layout Section
13352 Extracting from Lyx
13355 \begin_layout Standard
13356 To extract from LyX, you will need to configure LyX as explained in section
13358 \begin_inset CommandInset ref
13360 reference "sub:Configuring-the-build"
13367 \begin_layout Standard
13368 \begin_inset CommandInset label
13370 name "lyx-build-script"
13374 And this lyx-build scrap will extract newfangle for me.
13377 \begin_layout Chunk
13378 lyx-build,language=sh
13381 \begin_layout Standard
13382 \begin_inset listings
13386 \begin_layout Plain Layout
13391 \begin_layout Plain Layout
13396 \begin_layout Plain Layout
13400 \begin_layout Plain Layout
13404 chunkref{lyx-build-helper}>
13407 \begin_layout Plain Layout
13409 cd $PROJECT_DIR || exit 1
13412 \begin_layout Plain Layout
13416 \begin_layout Plain Layout
13418 /usr/local/bin/newfangle -R./newfangle $TEX_SRC > ./newfangle
13421 \begin_layout Plain Layout
13423 /usr/local/bin/newfangle -R./newfangle.module $TEX_SRC > ./newfangle.module
13426 \begin_layout Plain Layout
13430 \begin_layout Plain Layout
13435 \begin_layout Plain Layout
13437 ./newfangle -Rpca-test.awk $TEX_SRC | awk -f -
13445 \begin_layout Standard
13446 With a lyx-build-helper
13449 \begin_layout Chunk
13450 lyx-build-helper,language=sh
13453 \begin_layout Standard
13454 \begin_inset listings
13458 \begin_layout Plain Layout
13460 PROJECT_DIR="$LYX_r"
13463 \begin_layout Plain Layout
13465 LYX_SRC="$PROJECT_DIR/${LYX_i%.tex}.lyx"
13468 \begin_layout Plain Layout
13473 \begin_layout Plain Layout
13475 TEX_SRC="$TEX_DIR/$LYX_i"
13483 \begin_layout Section
13484 Extracting from the command line
13487 \begin_layout Standard
13488 First you will need the tex output, then you can extract:
13491 \begin_layout Chunk
13492 lyx-build-manual,language=sh
13495 \begin_layout Standard
13496 \begin_inset listings
13500 \begin_layout Plain Layout
13502 lyx -e latex newfangle.lyx
13505 \begin_layout Plain Layout
13507 newfangle -R./newfangle newfangle.tex > ./newfangle
13510 \begin_layout Plain Layout
13512 newfangle -R./newfangle.module newfangle.tex > ./newfangle.module
13524 \begin_layout Chapter
13528 \begin_layout Chunk
13529 tests-sub,params=THING;colour
13532 \begin_layout Standard
13533 \begin_inset listings
13537 \begin_layout Plain Layout
13539 I see a ${THING} of
13542 \begin_layout Plain Layout
13547 \begin_layout Plain Layout
13551 chunkref{tests-sub-sub}(${colour})>
13559 \begin_layout Chunk
13560 tests-sub-sub,params=colour
13563 \begin_layout Standard
13564 \begin_inset listings
13568 \begin_layout Plain Layout
13570 a funny shade of ${colour}
13578 \begin_layout Chunk
13582 \begin_layout Standard
13583 \begin_inset listings
13587 \begin_layout Plain Layout
13589 What do you see? "=<
13591 chunkref{tests-sub}(joe, red)>"
13594 \begin_layout Plain Layout
13604 \begin_layout Standard
13605 Should generate output:
13608 \begin_layout Chunk
13612 \begin_layout Standard
13613 \begin_inset listings
13617 \begin_layout Plain Layout
13619 What do you see? "I see a joe of
13622 \begin_layout Plain Layout
13627 \begin_layout Plain Layout
13629 looking closer a funny shade of red"
13632 \begin_layout Plain Layout
13637 \begin_layout Plain Layout