3 :Copyright: 2006 Guenter Milde.
4 Released under the terms of the GNU General Public License
12 The Philosophy of Literate Programming
13 ++++++++++++++++++++++++++++++++++++++
17 The `Literate Programming FAQ`_ gives the following definition:
19 *Literate programming* is the combination of documentation and source
20 together in a fashion suited for reading by human beings.
22 and the `Wikipedia`_ says:
24 *Literate programming* is a philosophy of computer programming based on
25 the premise that a computer program should be written with human
26 readability as a primary goal, similarly to a work of literature.
28 According to this philosophy, programmers should aim for a "literate"
29 style in their programming just as writers aim for an intelligible and
30 articulate style in their writing. This philosophy contrasts with the
31 mainstream view that the programmer's primary or sole objective is to
32 create source code and that documentation should only be a secondary
35 The philosophy of *literate programming* and the term itself were the
36 creation of Donald Knuth, the author of `The Art of Computer Programming`,
37 the TeX typesetting system, and other works of the programming art.
39 I believe that the time is ripe for significantly better documentation of
40 programs, and that we can best achieve this by considering programs to be
41 works of literature. Hence, my title: "Literate Programming."
43 Let us change our traditional attitude to the construction of programs:
44 Instead of imagining that our main task is to instruct a computer what to
45 do, let us concentrate rather on explaining to humans what we want the
48 -- Donald E. Knuth, `Literate Programming`_, 1984
50 John Max Skaller put the philosophy of literate programming in one sentence
52 The idea is that you do not document programs (after the fact), but
53 write documents that *contain* the programs.
55 -- John Max Skaller in a `Charming Python interview`_.
58 .. _`Literate Programming FAQ`: http://www.literateprogramming.com/lpfaq.pdf
59 .. _Literate Programming: WEB_
60 .. _Charming Python interview:
61 http://www-128.ibm.com/developerworks/library/l-pyth7.html
64 Literate Programming Technique and Literate Programming Systems
65 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
67 Donald Knuth not only conceived the *philosophy* of literate programming,
68 he also wrote the first literate programming *system* `WEB`_ that
69 implements the *technique* of literate programming:
71 .. _`technique of literate programming`:
73 WEB itself is chiefly a combination of two other languages:
75 (1) a document formatting language and
76 (2) a programming language.
78 My prototype WEB system uses TEX as the document formatting language
79 and PASCAL as the programming language, but the same principles would
80 apply equally well if other languages were substituted. [...]. The main
81 point is that WEB is inherently bilingual, and that such a combination
82 of languages proves to be much more powerful than either single
83 language by itself. WEB does not make the other languages obsolete; on
84 the contrary, it enhances them.
86 -- Donald E. Knuth, `Literate Programming`_, 1984
88 .. _`literate programming system`:
90 There exists a famous *statement of requirements* for a literate programming
93 .. _statement of requirements:
95 *Literate programming systems* have the following properties:
97 1. Code and extended, detailed comments are intermingled.
99 2. The code sections can be written in whatever order is best for people
100 to understand, and are re-ordered automatically when the computer needs
103 3. The program and its documentation can be handsomely typeset into a
104 single article that explains the program and how it works. Indices and
105 cross-references are generated automatically.
107 POD only does task 1, but the other tasks are much more important.
109 -- Mark-Jason Dominus `POD is not Literate Programming`_
113 Literate Programming with reStructured Text
114 +++++++++++++++++++++++++++++++++++++++++++
116 PyLit fails the `statement of requirements`_ for a literate programming
117 system, as it does not provide for code re-ordering, automatic indices and
120 According to the Wikipedia, PyLit qualifies as *semi-literate* (while POD is
121 classified as `embedded documentation`_):
123 It is the fact that documentation can be written freely whereas code must
124 be marked in a special way that makes the difference between semi-literate
125 programming and excessive documenting, where the documentation is embedded
126 into the code as comments.
128 -- `Wikipedia entry on literate programming`_
130 However, it is argued that PyLit is an *educated choice* as tool for the
131 `technique of literate programming`_:
133 * PyLit is inherently bilingual. The document containing the program is
136 (1) a document formatting language (`reStructured Text`_) and
137 (2) a programming language.
139 * Program and documentation can be typeset in a single article as well as
140 converted to a hyperlinked HTML document representing the program's
141 "web-like" structure. For indexing and cross-references, PyLit relies
142 on the rich features of the `reStructured Text`_ document formatting
145 Dispensing with code re-ordering enables the `dual-source`_ concept. Code
146 re-ordering is considered less important when literate programming is not
147 seen as an *alternative* to structured programming but a *variant* used
148 in combination with structured, object oriented or functional programming
151 The computer science and programming pioneers, like Wirth and Knuth,
152 used another programming style than we recommend today. One notable
153 difference is the use of procedure abstractions. When Wirth and Knuth
154 wrote their influential books and programming, procedure calls were used
155 with care, partly because they were relatively expensive. Consequently,
156 they needed an extra level of structuring within a program or procedure.
157 'Programming by stepwise refinement' as well as 'literate programming'
158 can be seen as techniques and representations to remedy the problem. The
159 use of many small (procedure) abstractions goes in another direction. If
160 you have attempted to use 'literate programming' on a program (maybe an
161 object-oriented program) with lots of small abstractions, you will
162 probably have realized that there is a misfit between, on the one hand,
163 being forced to name literal program fragments (scraps) and on the
164 other, named program abstractions. We do not want to leave the
165 impression that this is a major conceptual problem, but it is the
166 experience of the author that we need a variation of literate
167 programming, which is well-suited to programs with many, small (named)
170 -- Kurt Nørmark: `Literate Programming - Issues and Problems`_
172 Ordering code for optimal presentation to humans is still a problem in
173 many "traditional" programming languages. Pythons dynamic name resolution
174 allows a great deal of flexibility in the order of code.
176 .. _POD is not Literate Programming:
177 http://www.perl.com/pub/a/tchrist/litprog.html
178 .. _Wikipedia entry on literate programming:
179 .. _Wikipedia: http://en.wikipedia.org/wiki/Literate_programming
180 .. _embedded documentation:
181 http://en.wikipedia.org/wiki/Literate_programming#Embedded_documentation
182 .. _`Literate Programming - Issues and Problems`:
183 http://www.cs.auc.dk/~normark/litpro/issues-and-problems.html
184 .. _dual-source: /features/index.html#dual-source
191 There are many literate programming systems available, both sophisticated
192 and lightweight. This non-comprehensive list compares them to PyLit.
198 The first published literate programming system was WEB_ by Donald Knuth. It
199 uses Pascal as programming language and TeX as document formatting language.
201 CWEB_ is a newer version of WEB for the C programming language. Other
202 implementations of the concept are the "multi-lingual" noweb_ and
203 FunnelWeb_. All of them use TeX as document formatting language.
205 Some of the problems with WEB-like systems are:
207 * The steep learning curve for mastering TeX, special WEB commands and
208 the actual programming language in parallel discourages potential literate
211 * While a printout has the professional look of a TeX-typeset essay, the
212 programmer will spend most time viewing and editing the source. A WEB
213 source in a text editor is not "suited for reading by human beings":
215 In Knuth's work, beautiful literate programs were woven from ugly mixes of
216 markups in combined program and documentation files. The literate program
217 was directly targeted towards paper and books. Of obvious reasons,
218 programmers do not very often print their programs on paper. (And as
219 argued above, few of us intend to publish our programs in a book). Within
220 hours, such a program print out is no longer up-to-date. And furthermore,
221 being based on a paper representation, we cannot use the computers dynamic
222 potentials, such as outlining and searching.
224 -- Kurt Nørmark: `Literate Programming - Issues and Problems`_
227 .. _`Literate Programming - Issues and Problems`:
228 http://www.cs.auc.dk/~normark/litpro/issues-and-problems.html
230 How does PyLit differ from the web-like literate programming frameworks
231 (For a full discussion of PyLit's concepts see the Features_ page.):
233 * PyLit is not a complete `literate programming system`_ but a simple tool.
234 It does not support re-arranging of named code chunks.
236 * PyLit uses `reStructured Text`_ as document formatting language.
237 The literate program source is legible in any text editor.
239 Even the code source is "in a fashion suited for reading by human beings",
240 in opposition to source code produced by a full grown web system:
242 Just consider the output of ``tangle`` to be object code: You can run
243 it, but you don't want to look at it.
245 -- http://www.perl.com/pub/a/tchrist/litprog.html
247 * PyLit is a *bidirectional* text <-> code converter.
249 + You can start a literate program from a "normal" code source by
250 converting to a text source and adding the documentation parts.
251 + You can edit code in its "native" mode in any decent text editor or IDE.
252 + You can debug and fix code with the standard tools using the code source
253 Line numbers are the same in text and code format. Once ready, forward
254 your changes to the text version with a code->text conversion.
256 - PyLit cannot support export to several code files from one text source.
259 .. _docutils: http://docutils.sourceforge.net/
260 .. _Features: /features/index.html
262 .. _WEB: http://www.literateprogramming.com/knuthweb.pdf
263 .. _CWEB: http://www-cs-faculty.stanford.edu/~knuth/cweb.html
264 .. _noweb: http://www.eecs.harvard.edu/~nr/noweb/
265 .. _FunnelWeb: http://www.ross.net/funnelweb/
272 SourceBrowser_ is a complex tool generating a hyper-linked representation of
273 complex software projects.
275 SourceBrowser is a documentation meta-programming tool that generates a
276 wiki representation of a source tree.
278 It provides a *Source Annotation Language* for writing a documentation
279 meta-program and a tool for generating a static wiki for the host program.
281 -- SourceBrowser_ home page
283 .. _SourceBrowser: http://web.media.mit.edu/~vyzo/srcb/doc/index.html
289 Interscript_ is a complex framework that uses Python both for its
290 implementation and to supply scripting services to client sourceware.
292 Interscript is different, because the fundamental paradigm is extended so
293 that there are three kinds of LP sections in file:
296 2. Documentation sections
297 3. Scripting sections
299 where the scripting sections [...] control and possibly generate both
300 documentation and code.
302 -- Interscript_ homepage
304 .. _Interscript: http://interscript.sourceforge.net/
309 Ly_, the "lyterate programming thingy." is an engine for Literate Programming.
310 The design considerations look very much like the ones for PyLit:
312 So why would anybody use Ly instead of the established Literate
313 Programming tools like WEB or noweb?
315 * Elegance. I do not consider the source code of WEB or noweb to be
316 particularly nice. I especially wanted a syntax that allows me to easily
317 write a short paragraph, then two or three lines of code, then another
318 short paragraph. Ly accomplishes this by distinguishing code from text
320 * HTML. WEB and CWEB are targeted at outputting TeX. This is a valid
321 choice, but not mine, as I want to create documents with WWW links, and
322 as I want to re-create documents each time I change something-- both of
323 these features don't go well together with a format suited to printout,
324 like TeX is. (Of course you can create HTML from LaTeX, but if my output
325 format is HTML, I want to be able to write HTML directly.)
326 * Python. I originally created Ly for a Python-based project, and
327 therefore I wanted it to run everywhere where Python runs. WEB and noweb
328 are mostly targeted at UNIX-like platforms.
332 However, Ly introduces its own language with
334 * a syntax for code chunks,
335 * some special ways for entering HTML tags ("currently undocumented").
337 whereas PyLit relies on the established `reStructured Text`_ for
338 documentation markup and linking.
340 .. _Ly: http://lyterate.sourceforge.net/intro.html
341 .. _reStructured Text: http://docutils.sourceforge.net/rst.html
347 XMLTangle_ is an generic tangle program written in Python. It differs from
348 PyLit mainly by its choice of XML as documenting language (making the source
349 document harder to read for a human being).
351 .. _XMLTangle: http://literatexml.sourceforge.net/xmltangle/
354 Lightweight literate programming
355 --------------------------------
357 `Lightweight literate programming`_ is one more example of a Python based
358 literate programming framework. One XML source is processed to either text or
359 code. The XML source itself is rather hard to read for human beings.
361 .. _`Lightweight literate programming`:
362 http://infohost.nmt.edu/~shipman/soft/litprog/
368 Pyreport_ is quite similar to PyLit in its focus on Python and the use of
369 reStructuredText. It is a combination of `documentation generator`__
370 (processing embedded documentation) and report tool (processing Python
373 __ http://en.wikipedia.org/wiki/Documentation_generator
375 pyreport is a program that runs a python script and captures its output,
376 compiling it to a pretty report in a PDF or an html file. It can display
377 the output embedded in the code that produced it and can process special
378 comments (literate comments) according to markup languages (rst or LaTeX)
379 to compile a very readable document.
381 This allows for extensive literate programming in python, for generating
382 reports out of calculations written in python, and for making nice
385 However, it is not a tool to "write documents that contain programs".
388 .. _pyreport: http://gael-varoquaux.info/computers/pyreport/