1 #+OPTIONS: H:3 num:nil toc:2 \n:nil @:t ::t |:t ^:t -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
2 #+STARTUP: align fold nodlcheck hidestars oddeven lognotestate
3 #+SEQ_TODO: TODO(t) INPROGRESS(i) WAITING(w@) | DONE(d) CANCELED(c@)
4 #+TAGS: Write(w) Update(u) Fix(f) Check(c)
6 #+AUTHOR: Dan Davison, Eric Schulte
7 #+EMAIL: davison at stats dot ox dot ac dot uk
13 <p>executable source code blocks in org-mode</p>
17 <img src="images/tower-of-babel.png" alt="images/tower-of-babel.png"
18 title="And the Lord said, Behold, the people is one, and they have all one language; and this they begin to do; and now nothing will be restrained from them, which they have imagined to do. Genesis 11:1-9"/>
21 <a href="http://www.flickr.com/photos/23379658@N05/" title=""><b>Martijn Streefkerk</b></a>
29 :CUSTOM_ID: introduction
31 Org-babel provides the following modifications to [[http://orgmode.org/manual/Literal-examples.html][the existing
32 support]] for blocks of source code examples in the org-mode core.
33 1. source code execution
34 2. arguments to source code blocks
35 3. exportation of source code blocks to files (literate programming)
39 :CUSTOM_ID: getting-started
41 Grab the latest code from the git repo at [[http://github.com/eschulte/org-babel/tree/master][github/org-babel]]
43 git clone git://github.com/eschulte/org-babel.git
46 And add the following lines to your .emacs, replacing the path as
47 appropriate. A good place to check that things are up and running
48 would the examples in [[* Basic org-babel functionality][Basic org-babel functionality]].
49 #+begin_src emacs-lisp
50 (add-to-list 'load-path "/path/to/org-babel/lisp")
51 (require 'org-babel-init)
54 * Basic org-babel functionality
56 :CUSTOM_ID: basic-functionality
58 *** Source code execution
60 :CUSTOM_ID: source-code-execution
62 For interpreted languages such as shell, python, R, etc, org-babel
63 allows source blocks to be executed: the code is passed to the
64 interpreter and you have control over what is done with the
65 results of excecution. E.g. place point anywhere in the following
66 block and use C-c C-c to run the code:
68 [[http://www.ruby-lang.org/][Ruby]] source code
70 "This file was last evaluated on #{Date.today}"
73 Results of Ruby evaluation
75 : This file was last evaluated on 2009-08-09
77 [[http://www.r-project.org/][R]] source code
78 #+begin_src R :results value
84 Results of R evaluation
89 *** What happens to the results?
93 Org-babel provides two fundamentally different modes for capturing
94 the results of code evaluation, specified by the :results header
97 This means that the 'result' of code evaluation is defined to be
98 the *value* of the last statement in the block. Thus with this
99 setting, one can view the code block as a function with a return
100 value. And not only can one view it that way, but you can
101 actually use the return value of one source block as input for
102 another (see later). This setting is the default.
104 With this setting, org-babel captures all the text output of the
105 code block and places it in the org buffer. One can think of this
106 as a 'scripting' mode: the code block contains a series of
107 commands, and you get the output of all the commands. Unlike in
108 the 'functional' mode specified by =:results value=, the code
109 block has no return value. (This mode will be familiar to Sweave
111 **** Additional :results settings
113 *** Arguments to source code blocks
115 :CUSTOM_ID: arguments-to-source-code-blocks
117 In addition to evaluation of code blocks, org-babel allows them to
118 be parameterised (i.e. have arguments). Thus source code blocks
119 now have the status of *functions*.
121 Inputs for fibonacci-seq
123 #+tblname: fibonacci-inputs
124 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
125 | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
127 in the Org-mode buffer this looks like
128 : #+tblname: fibonacci-inputs
129 : | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
130 : | 2 | 4 | 6 | 8 | 10 | 12 | 14 | 16 | 18 | 20 |
132 [[http://www.gnu.org/software/emacs/manual/elisp.html][Emacs Lisp]] source code
133 #+srcname: fibonacci-seq
134 #+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
136 (if (or (= n 0) (= n 1))
138 (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
140 (mapcar (lambda (row)
141 (mapcar #'fibonacci row)) fib-inputs)
144 in the Org-mode buffer this looks like
145 : #+srcname: fibonacci-seq
146 : #+begin_src emacs-lisp :var fib-inputs=fibonacci-inputs
147 : (defun fibonacci (n)
148 : (if (or (= n 0) (= n 1))
150 : (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))
152 : (mapcar (lambda (row)
153 : (mapcar #'fibonacci row)) fib-inputs)
156 Results of Emacs Lisp code evaluation
158 | 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
159 | 1 | 3 | 8 | 21 | 55 | 144 | 377 | 987 | 2584 | 6765 |
161 * A meta-programming language for org-mode
163 :CUSTOM_ID: meta-programming-language
166 Since information can pass freely between source-code blocks and
167 org-mode tables you can mix and match languages using each language
168 for those tasks to which it is suited. This makes Org-mode files with
169 Org-babel into a kind of meta-functional programming language in which
170 functions from many languages can work together.
172 As an example, lets take some system diagnostics in the shell, and
173 then graph them with R.
176 #+srcname: directories
177 #+begin_src bash :results replace
178 cd ~ && du -sc * |grep -v total
180 2. Results of the shell source code (on my system, grab this org-mode
181 files and try running it on your own)
182 #+resname: directories
184 | 12156104 | "Documents" |
185 | 3482440 | "Downloads" |
186 | 2901720 | "Library" |
188 | 16548024 | "Music" |
190 | 7649472 | "Pictures" |
198 3. R source code (which calls the previous shell source code)
199 #+srcname: directory-pie
200 #+begin_src R :var dirs = directories :session R-pie-example
201 pie(dirs[,1], labels = dirs[,2])
203 4. Results of R code [[file:images/dirs.png]]
205 * Spreadsheet plugins for org-mode in any language
207 :CUSTOM_ID: spreadsheet
210 *NOTE*: Maybe in-addition-to/in-stead-of this example we should do a
211 more traditional "spreadsheet" example with R [Eric]
213 Not only can Org-babel pass entire tables of data to source code
214 blocks (see [[arguments-to-source-code-blocks]]), Org-babel can also be
215 used to call source code blocks from *within* tables using the
216 Org-mode's [[http://orgmode.org/manual/The-spreadsheet.html#The-spreadsheet][existing spreadsheet functionality]].
218 In fact the functional test suite for Org-babel is implemented as a
219 large Org-mode table. To run the entire test suite you simple
220 evaluate the table =C-u C-c C-c=, and all of the tests are run
221 updating the table with pass/fail statistics.
223 Here's a sample of our test suite.
225 #+TBLNAME: org-babel-tests
226 | functionality | block | arg | expected | results | pass |
227 |------------------+--------------+-----+-------------+-------------+------|
228 | basic evaluation | | | | | pass |
229 |------------------+--------------+-----+-------------+-------------+------|
230 | emacs lisp | basic-elisp | 2 | 4 | 4 | pass |
231 | shell | basic-shell | | 6 | 6 | pass |
232 | ruby | basic-ruby | | org-babel | org-babel | pass |
233 | python | basic-python | | hello world | hello world | pass |
234 | R | basic-R | | 13 | 13 | pass |
235 #+TBLFM: $5='(if (= (length $3) 1) (progn (message (format "running %S" '(sbe $2 (n $3)))) (sbe $2 (n $3))) (sbe $2))::$6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))
236 #+TBLFM: $5=""::$6=""
238 *** code blocks for tests
240 #+srcname: basic-elisp
241 #+begin_src emacs-lisp :var n=7
245 #+srcname: basic-shell
246 #+begin_src sh :results silent
250 #+srcname: date-simple
251 #+begin_src sh :results silent
255 #+srcname: basic-ruby
256 #+begin_src ruby :results silent
260 #+srcname: basic-python
261 #+begin_src python :results silent
266 #+begin_src R :results silent
273 :CUSTOM_ID: library-of-babel
275 What about those source code blocks which are so useful you want to
276 have them available in every org-mode buffer?
278 The [[file:library-of-babel.org][Library of Babel]] is an extensible collection of ready-made and
279 easily-shortcut-callable source-code blocks for handling common
280 tasks. Org-babel comes pre-populated with the source-code blocks
281 located in the [[file:library-of-babel.org][library-of-babel.org]] file. It is possible to add
282 source-code blocks from any org-mode file to the library by calling
284 #+srcname: add-file-to-lob
285 #+begin_src emacs-lisp
286 (org-babel-lob-ingest "path/to/file.org")
289 * Reproducible Research
291 :CUSTOM_ID: reproducable-research
294 An article about computational science in a scientific publication is
295 not the scholarship itself, it is merely advertising of the
296 scholarship. The actual scholarship is the complete software
297 development environment and the complete set of instructions which
298 generated the figures. -- D. Donoho
301 [[http://reproducibleresearch.net/index.php/Main_Page][Reproducible Research]] (RR) is the practice of distributing along with
302 an article of research all data, code, and tools required to reproduce
303 the results discussed in the paper. As such the paper becomes not
304 only a document describing the research but a complete laboratory
305 reproducing the research.
307 Org-mode already has exceptional support for [[http://orgmode.org/manual/Exporting.html#Exporting][exporting to html and
308 LaTeX]]. Org-babel makes Org-mode a tool for RR by *activating* the
309 data and source code embedded into Org-mode documents making the
310 entire document executable. This makes it not only possible, but
311 natural to distribute research in a format that encourages readers to
312 recreate your results, and perform their own analysis.
314 Existing RR tools like [[http://en.wikipedia.org/wiki/Sweave][Sweave]] provide for the embedding of [[http://www.r-project.org/][R]] code into
315 LaTeX documents. While this is very useful, such documents often
316 still require a large degree of "glue code" in the form of external
317 shell scripts, python scripts, and Makefiles. To our knowledge
318 Org-babel is the only RR tool which allows multiple languages and data
319 to coexist and cooperate inside of a single document.
321 * Literate programming
323 :CUSTOM_ID: literate-programming
326 - org-babel-load-file
327 * Reference / Documentation
329 :CUSTOM_ID: reference-and-documentation
332 *** Source Code block syntax
334 The basic syntax of source-code blocks is as follows:
337 : #+begin_src language header-arguments
341 - name :: This name is associated with the source-code block. This is
342 similar to the =#+TBLNAME= lines which can be used to name tables
343 in org-mode files. By referencing the srcname of a source-code
344 block it is possible to evaluate the block for other places,
345 files, or from inside tables.
346 - language :: The language of the code in the source-code block, valid
347 values must be members of `org-babel-interpreters'.
348 - header-arguments :: Header arguments control many facets of the
349 input to, evaluation of, and output of source-code blocks. See
350 the [[* Header Arguments][Header Arguments]] section for a complete review of available
352 - body :: The actual source code which will be evaluated. This can be
353 edited with `org-edit-special'.
355 **** Header Arguments
357 - results :: results arguments specify what should be done with the
358 output of source-code blocks
359 - The following options are mutually exclusive, and specify how the
360 results should be collected from the source-code block
363 - The following options are mutually exclusive and specify what type
364 of results the code block will return
365 - vector :: specifies that the results should be interpreted as a
366 multidimensional vector (even if the vector is
367 trivial), and will be inserted into the org-mode file
369 - scalar :: specifies that the results should be interpreted as a
370 scalar value, and will be inserted into the org-mode
372 - file :: specifies that the results should be interpreted as the
373 path to a file, and will be inserted into the org-mode
375 - The following options specify how the results should be inserted
376 into the org-mode file
377 - replace :: the current results replace any previously inserted
378 results from the code block
379 - silent :: rather than being inserted into the org-mode file the
380 results are echoed into the message bar
381 - exports :: exports arguments specify what should be included in html
382 or latex exports of the org-mode file
383 - code :: the body of code is included into the exported file
384 - results :: the results of evaluating the code is included in the
386 - both :: both the code and results are included in the exported
388 - none :: nothing is included in the exported file
389 - tangle :: tangle arguments specify whether or not the source-code
390 block should be included in tangled extraction of
392 - yes :: the source-code block is exported to a source-code file
393 named after the basename (name w/o extension) of the
395 - no :: (default) the source-code block is not exported to a
397 - other :: any other string passed to the =tangle= header argument
398 is interpreted as a file basename to which the block will