1 #+OPTIONS: H:3 num:nil toc:2 \n:nil @:t ::t |:t ^:t -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
2 #+STARTUP: align fold nodlcheck hidestars oddeven lognotestate hideblocks
3 #+SEQ_TODO: TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
4 #+TAGS: Write(w) Update(u) Fix(f) Check(c)
6 #+AUTHOR: Dan Davison, Eric Schulte
7 #+EMAIL: davison at stats dot ox dot ac dot uk
13 <p>executable source code blocks in org-mode</p>
17 <img src="images/tower-of-babel.png" alt="images/tower-of-babel.png"
18 title="And the Lord said, Behold, the people is one, and they have all one language; and this they begin to do; and now nothing will be restrained from them, which they have imagined to do. Genesis 11:1-9"/>
21 <a href="http://www.flickr.com/photos/23379658@N05/" title=""><b>Martijn Streefkerk</b></a>
29 :CUSTOM_ID: introduction
31 Org-babel provides the following modifications to [[http://orgmode.org/manual/Literal-examples.html][the existing
32 support]] for blocks of source code examples in the org-mode core.
33 1. source code execution
34 2. arguments to source code blocks
35 3. exportation of source code blocks to files (literate programming)
39 :CUSTOM_ID: getting-started
42 1) Grab the latest code from the git repo at [[http://github.com/eschulte/org-babel/tree/master][github/org-babel]]
44 git clone git://github.com/eschulte/org-babel.git
47 2) Add the following lines to your .emacs, replacing the path as
48 appropriate. A good place to check that things are up and running
49 would the examples in [[* Basic org-babel functionality][Basic org-babel functionality]].
50 #+begin_src emacs-lisp
51 (add-to-list 'load-path "/path/to/org-babel/lisp")
52 (require 'org-babel-init)
55 3) Then activate the subset of supported Org-babel languages which
56 you will want to be able to execute on your system. The
57 following can be added to your .emacs and used to activate
58 languages. It includes a brief list of the requirements for each
60 #+begin_src emacs-lisp
61 ;; Uncomment each of the following require lines if you want org-babel
62 ;; to support that language. Each language has a comment explaining
63 ;; it's dependencies. See the related files in lisp/langs for more
64 ;; detailed explanations of requirements.
65 ;; (require 'org-babel-R) ;; R and ess-mode
66 ;; (require 'org-babel-asymptote) ;; asymptote
67 ;; (require 'org-babel-css) ;; none
68 ;; (require 'org-babel-ditaa) ;; ditaa
69 ;; (require 'org-babel-dot) ;; dot
70 ;; (require 'org-babel-gnuplot) ;; gnuplot, and gnuplot-mode
71 ;; (require 'org-babel-ocaml) ;; ocaml, and tuareg-mode
72 ;; (require 'org-babel-python) ;; python, and python-mode
73 ;; (require 'org-babel-ruby) ;; ruby, irb, ruby-mode, and inf-ruby mode
74 ;; (require 'org-babel-sass) ;; sass, sass-mode
75 ;; (require 'org-babel-sql) ;; none
77 ;; Once you've activated languages, load the library of babel for
78 ;; pre-built helpers in the languages you will be using.
79 (org-babel-load-library-of-babel)
82 * Basic org-babel functionality
84 :CUSTOM_ID: basic-functionality
86 *** Source code execution
88 :CUSTOM_ID: source-code-execution
90 For interpreted languages such as shell, python, R, etc, org-babel
91 allows source blocks to be executed: the code is passed to the
92 interpreter and you have control over what is done with the
93 results of excecution. E.g. place point anywhere in the following
94 block and use C-c C-c to run the code:
96 [[http://www.ruby-lang.org/][Ruby]] source code
98 "This file was last evaluated on #{Date.today}"
101 Results of Ruby evaluation
103 : This file was last evaluated on 2009-08-09
105 [[http://www.r-project.org/][R]] source code
106 #+begin_src R :results value
112 Results of R evaluation
117 *** What happens to the results?
121 Org-babel provides two fundamentally different modes for capturing
122 the results of code evaluation, specified by the :results header
125 This means that the 'result' of code evaluation is defined to be
126 the *value* of the last statement in the block. Thus with this
127 setting, one can view the code block as a function with a return
128 value. And not only can one view it that way, but you can
129 actually use the return value of one source block as input for
130 another (see later). This setting is the default.
132 With this setting, org-babel captures all the text output of the
133 code block and places it in the org buffer. One can think of this
134 as a 'scripting' mode: the code block contains a series of
135 commands, and you get the output of all the commands. Unlike in
136 the 'functional' mode specified by =:results value=, the code
137 block has no return value. (This mode will be familiar to Sweave
139 **** Additional :results settings
141 *** Arguments to source code blocks
143 :CUSTOM_ID: arguments-to-source-code-blocks
145 In addition to evaluation of code blocks, org-babel allows them to
146 be parameterised (i.e. have arguments). Thus source code blocks
147 now have the status of *functions*.
149 Inputs for fibonacci-seq
151 #+tblname: fibonacci-inputs
152 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
153 | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
155 in the Org-mode buffer this looks like
156 : #+tblname: fibonacci-inputs
157 : | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
158 : | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
160 [[http://www.gnu.org/software/emacs/manual/elisp.html][Emacs Lisp]] source code
161 #+srcname: fibonacci-seq
162 #+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
164 (if (or (= n 0) (= n 1))
166 (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
168 (mapcar (lambda (row)
169 (mapcar #'fibonacci row)) fib-inputs)
172 in the Org-mode buffer this looks like
173 : #+srcname: fibonacci-seq
174 : #+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
175 : (defun fibonacci (n)
176 : (if (or (= n 0) (= n 1))
178 : (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
180 : (mapcar (lambda (row)
181 : (mapcar #'fibonacci row)) fib-inputs)
184 Results of Emacs Lisp code evaluation
186 | 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
187 | 1 | 3 | 8 | 21 | 55 | 144 | 377 | 987 | 2584 | 6765 |
189 * A meta-programming language for org-mode
191 :CUSTOM_ID: meta-programming-language
194 Since information can pass freely between source-code blocks and
195 org-mode tables you can mix and match languages using each language
196 for those tasks to which it is suited. This makes Org-mode files with
197 Org-babel into a kind of meta-functional programming language in which
198 functions from many languages can work together.
200 As an example, lets take some system diagnostics in the shell, and
201 then graph them with R.
204 #+srcname: directories
205 #+begin_src bash :results replace
206 cd ~ && du -sc * |grep -v total
208 2. Results of the shell source code (on my system, grab this org-mode
209 files and try running it on your own)
210 #+resname: directories
212 | 12156104 | "Documents" |
213 | 3482440 | "Downloads" |
214 | 2901720 | "Library" |
216 | 16548024 | "Music" |
218 | 7649472 | "Pictures" |
226 3. R source code (which calls the previous shell source code)
227 #+srcname: directory-pie
228 #+begin_src R :var dirs = directories :session R-pie-example
229 pie(dirs[,1], labels = dirs[,2])
231 4. Results of R code [[file:images/dirs.png]]
233 * Spreadsheet plugins for org-mode in any language
235 :CUSTOM_ID: spreadsheet
238 *NOTE*: Maybe in-addition-to/in-stead-of this example we should do a
239 more traditional "spreadsheet" example with R [Eric]
241 Not only can Org-babel pass entire tables of data to source code
242 blocks (see [[arguments-to-source-code-blocks]]), Org-babel can also be
243 used to call source code blocks from *within* tables using the
244 Org-mode's [[http://orgmode.org/manual/The-spreadsheet.html#The-spreadsheet][existing spreadsheet functionality]].
246 In fact the functional test suite for Org-babel is implemented as a
247 large Org-mode table. To run the entire test suite you simple
248 evaluate the table =C-u C-c C-c=, and all of the tests are run
249 updating the table with pass/fail statistics.
251 Here's a sample of our test suite.
253 #+TBLNAME: org-babel-tests
254 | functionality | block | arg | expected | results | pass |
255 |------------------+--------------+-----+-------------+-------------+------|
256 | basic evaluation | | | | | pass |
257 |------------------+--------------+-----+-------------+-------------+------|
258 | emacs lisp | basic-elisp | 2 | 4 | 4 | pass |
259 | shell | basic-shell | | 6 | 6 | pass |
260 | ruby | basic-ruby | | org-babel | org-babel | pass |
261 | python | basic-python | | hello world | hello world | pass |
262 | R | basic-R | | 13 | 13 | pass |
263 #+TBLFM: $5='(if (= (length $3) 1) (progn (message (format "running %S" '(sbe $2 (n $3)))) (sbe $2 (n $3))) (sbe $2))::$6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))
264 #+TBLFM: $5=""::$6=""
266 *** code blocks for tests
268 #+srcname: basic-elisp
269 #+begin_src emacs-lisp :var n=7
273 #+srcname: basic-shell
274 #+begin_src sh :results silent
278 #+srcname: date-simple
279 #+begin_src sh :results silent
283 #+srcname: basic-ruby
284 #+begin_src ruby :results silent
288 #+srcname: basic-python
289 #+begin_src python :results silent
294 #+begin_src R :results silent
301 :CUSTOM_ID: library-of-babel
303 What about those source code blocks which are so useful you want to
304 have them available in every org-mode buffer?
306 The [[file:library-of-babel.org][Library of Babel]] is an extensible collection of ready-made and
307 easily-shortcut-callable source-code blocks for handling common
308 tasks. Org-babel comes pre-populated with the source-code blocks
309 located in the [[file:library-of-babel.org][library-of-babel.org]] file. It is possible to add
310 source-code blocks from any org-mode file to the library by calling
312 #+srcname: add-file-to-lob
313 #+begin_src emacs-lisp
314 (org-babel-lob-ingest "path/to/file.org")
317 * Reproducible Research
319 :CUSTOM_ID: reproducable-research
322 An article about computational science in a scientific publication is
323 not the scholarship itself, it is merely advertising of the
324 scholarship. The actual scholarship is the complete software
325 development environment and the complete set of instructions which
326 generated the figures.
331 [[http://reproducibleresearch.net/index.php/Main_Page][Reproducible Research]] (RR) is the practice of distributing along with
332 an article of research all data, code, and tools required to reproduce
333 the results discussed in the paper. As such the paper becomes not
334 only a document describing the research but a complete laboratory
335 reproducing the research.
337 Org-mode already has exceptional support for [[http://orgmode.org/manual/Exporting.html#Exporting][exporting to html and
338 LaTeX]]. Org-babel makes Org-mode a tool for RR by *activating* the
339 data and source code embedded into Org-mode documents making the
340 entire document executable. This makes it not only possible, but
341 natural to distribute research in a format that encourages readers to
342 recreate your results, and perform their own analysis.
344 Existing RR tools like [[http://en.wikipedia.org/wiki/Sweave][Sweave]] provide for the embedding of [[http://www.r-project.org/][R]] code into
345 LaTeX documents. While this is very useful, such documents often
346 still require a large degree of "glue code" in the form of external
347 shell scripts, python scripts, and Makefiles. To my knowledge
348 Org-babl is the only RR tool which allows multiple languages and data
349 to coexist and cooperate inside of a single document.
351 * Literate programming
353 :CUSTOM_ID: literate-programming
357 Let us change our traditional attitude to the construction of
358 programs: Instead of imagining that our main task is to instruct a
359 /computer/ what to do, let us concentrate rather on explaining to
360 /human beings/ what we want a computer to do.
362 The practitioner of literate programming can be regarded as an
363 essayist, whose main concern is with exposition and excellence of
364 style. Such an author, with thesaurus in hand, chooses the names of
365 variables carefully and explains what each variable means. He or she
366 strives for a program that is comprehensible because its concepts have
367 been introduced in an order that is best for human understanding,
368 using a mixture of formal and informal methods that reinforce each
374 Org-babel supports [[http://en.wikipedia.org/wiki/Literate_programming][Literate Programming]] (LP) by allowing the act of
375 programming to take place inside of Org-mode documents. The Org-mode
376 file can then be exported (*woven* in LP speak) to html or LaTeX for
377 consumption by a human, and the embedded source code can be extracted
378 (*tangled* in LP speak) into structured source code files for
379 consumption by a computer.
381 To support these operations Org-babel relies on Org-mode's [[http://orgmode.org/manual/Exporting.html#Exporting][existing
382 exporting functionality]] for *weaving* of documentation, and on the
383 =org-babel-tangle= function which makes use of [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] [[reference-expansion][reference syntax]]
384 for *tangling* of code files.
386 The [[literate-programming-example][following example]] demonstrates the process of *tangling* in
389 *** Literate Programming Example
391 :CUSTOM_ID: literate-programming-example
394 Tangling functionality is controlled by the =tangle= family of
395 [[header-arguments]]. These arguments can be used to turn tangling on or
396 off (the default) on the source code block, or the outline heading
399 The following demonstrates the combination of three source code blocks
400 into a single source code file using =org-babel-tangle=.
402 The following two blocks will not be tangled by default since they
403 have no =tangle= header arguments.
405 #+srcname: hello-world-prefix
406 #+begin_src sh :exports none
407 echo "/-----------------------------------------------------------\\"
410 : #+srcname: hello-world-prefix
411 : #+begin_src sh :exports none
412 : echo "/-----------------------------------------------------------\\"
415 #+srcname: hello-world-postfix
416 #+begin_src sh :exports none
417 echo "\-----------------------------------------------------------/"
420 : #+srcname: hello-world-postfix
421 : #+begin_src sh :exports none
422 : echo "\-----------------------------------------------------------/"
426 The third block does have a =tangle= header argument indicating the
427 name of the file to which it should be written. It also has [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]]
428 style references to the two previous source code blocks which will be
429 expanded during tangling to include them in the output file as well.
431 #+srcname: hello-world
432 #+begin_src sh :tangle hello :exports none
433 # <<hello-world-prefix>>
434 echo "| hello world |"
435 # <<hello-world-postfix>>
438 : #+srcname: hello-world
439 : #+begin_src sh :tangle hello :exports none
440 : # <<hello-world-prefix>>
441 : echo "| hello world |"
442 : # <<hello-world-postfix>>
445 Calling =org-babel-tangle= will result in the following being written
446 to the =hello.sh= file.
448 #+srcname: hello-world-output
451 # generated by org-babel-tangle
453 # [[file:~/src/org-babel/org-babel-worg.org::#literate-programming-example][block-16]]
454 # <<hello-world-prefix>>
455 echo "/-----------------------------------------------------------\\"
457 echo "| hello world |"
458 # <<hello-world-postfix>>
459 echo "\-----------------------------------------------------------/"
463 * Reference / Documentation
465 :CUSTOM_ID: reference-and-documentation
468 *** Source Code block syntax
470 The basic syntax of source-code blocks is as follows:
473 : #+begin_src language header-arguments
477 - name :: This name is associated with the source-code block. This is
478 similar to the =#+TBLNAME= lines which can be used to name tables
479 in org-mode files. By referencing the srcname of a source-code
480 block it is possible to evaluate the block for other places,
481 files, or from inside tables.
482 - language :: The language of the code in the source-code block, valid
483 values must be members of `org-babel-interpreters'.
484 - header-arguments :: Header arguments control many facets of the
485 input to, evaluation of, and output of source-code blocks. See
486 the [[* Header Arguments][Header Arguments]] section for a complete review of available
488 - body :: The actual source code which will be evaluated. This can be
489 edited with `org-edit-special'.
493 :CUSTOM_ID: header-arguments
496 - results :: results arguments specify what should be done with the
497 output of source-code blocks
498 - The following options are mutually exclusive, and specify how the
499 results should be collected from the source-code block
502 - The following options are mutually exclusive and specify what type
503 of results the code block will return
504 - vector :: specifies that the results should be interpreted as a
505 multidimensional vector (even if the vector is
506 trivial), and will be inserted into the org-mode file
508 - scalar :: specifies that the results should be interpreted as a
509 scalar value, and will be inserted into the org-mode
511 - file :: specifies that the results should be interpreted as the
512 path to a file, and will be inserted into the org-mode
514 - The following options specify how the results should be inserted
515 into the org-mode file
516 - replace :: the current results replace any previously inserted
517 results from the code block
518 - silent :: rather than being inserted into the org-mode file the
519 results are echoed into the message bar
520 - exports :: exports arguments specify what should be included in html
521 or latex exports of the org-mode file
522 - code :: the body of code is included into the exported file
523 - results :: the results of evaluating the code is included in the
525 - both :: both the code and results are included in the exported
527 - none :: nothing is included in the exported file
528 - tangle :: tangle arguments specify whether or not the source-code
529 block should be included in tangled extraction of
531 - yes :: the source-code block is exported to a source-code file
532 named after the basename (name w/o extension) of the
534 - no :: (default) the source-code block is not exported to a
536 - other :: any other string passed to the =tangle= header argument
537 is interpreted as a file basename to which the block will
540 *** Noweb reference syntax
541 The [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]] Literate Programming system allows named blocks of code to
542 be referenced by using a =<<code-block-name>>= syntax. When a
543 document is tangled these references are replaced with the named code.
544 An example is provided in the [[literate-programming-example]] in this