1 #+TITLE: org-babel --- facilitating communication between programming languages and people
2 #+SEQ_TODO: PROPOSED TODO STARTED | DONE DEFERRED REJECTED
3 #+OPTIONS: H:3 num:nil toc:1 \n:nil @:t ::t |:t ^:t -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
4 #+STARTUP: oddeven hideblocks
7 ** PROPOSED further work on dependencies of header args?
8 For example, pp and code should probably imply value. It would be
9 possible in principle to have a general mechanism for specifying
10 and resolving dependencies, which would be used by
12 ** TODO =:hide= header argument for automatically folding source blocks
15 My suggestion is that if a source block has the :hide header argument
16 it should be closed by default as if the user had pressed tab. The
17 user could then press tab at the "#BEGIN_SRC ..." line to show the
18 content of the block and, maybe, the block could be closed again if
19 the cursor leaves the block. This can be useful for other blocks as
23 ** PROPOSED allow hiding of code blocks with <tab> on srcname line?
24 ** PROPOSED Allow hiding of results blocks?
26 ** PROPOSED Support passing of data to source block on stdin?
27 ** PROPOSED add data serialization language result types (XML, YAML, JSON, etc...)
28 these could be cached in source-code blocks of the appropriate
29 serialization language, and could be very useful, especially for
30 languages (like ruby) which support dumping object to/from these
31 serialization languages.
33 ** STARTED Export issues
34 *** STARTED reference source blocks that are themselves excluded from export
35 *** STARTED restrictions on locations of org-exp-blocks interblocks
36 *** STARTED Suitable export of #+srcname and #+resname lines
37 *** STARTED inline source code blocks [5/8]
38 Like the =\R{ code }= blocks
40 not sure what the format should be, maybe just something simple
41 like =src_lang[]{}= where lang is the name of the source code
42 language to be evaluated, =[]= is optional and contains any header
43 arguments and ={}= contains the code.
45 **** DONE evaluation with \C-c\C-c
46 Putting aside the header argument issue for now we can just run these
47 with the following default header arguments
48 - =:results= :: silent
49 - =:exports= :: results
51 **** DONE inline exportation
52 Need to add an interblock hook (or some such) through org-exp-blocks
53 **** DONE header arguments
54 We should make it possible to use header arguments.
55 **** DONE Bring export of inline code back to life
56 **** DONE Uses session even when not requested
57 **** TODO fontification
58 we should color these blocks differently
60 **** TODO refine html exportation
61 should use a span class, and should show original source in tool-tip
62 *** STARTED Allow export of results of #+lob lines
63 *** STARTED Inline fragment not evaluated unless another code block in buffer
65 ** PROPOSED allow references to bound emacs lisp variables?
66 I don't *think* we can do this currently. Something like this?
67 #+begin_src emacs-lisp
68 (defun org-babel-ref-literal (ref)
69 "Determine if the right side of a header argument variable
70 assignment is a literal value or is a reference to some external
71 resource. If REF is literal then return it's value, otherwise
73 (if (boundp (intern ref)) (eval (intern ref))
74 (let ((out (org-babel-read ref)))
76 (if (string-match "^\"\\(.+\\)\"$" ref)
81 Need to be careful that an attempt is *not* made to interpret quoted
82 strings as elisp variables. It would allows stuff like this
84 #+begin_src emacs-lisp :results silent
86 evecfile (concat dir "/" "evecs")
90 #+srcname: pcaplot(evecf=evecfile, numpcs=numpcs)
91 #+begin_src R :file pca.png
92 x <- matrix(scan(evecf), ncol=numpcs)
93 plot(x[,1:2], pch="+")
96 ** PROPOSED jumping between results and source blocks (evaluation from results)
97 see discussion on the Org-mode list
98 http://thread.gmane.org/gmane.emacs.orgmode/18407/focus=18419
99 ** TODO add a =:requires= header argument
100 this header argument would specify a named source-code block which
101 should be appended to the body of the current source-code block before
109 #+begin_src python :requires setup
113 The initial implementation of this should be fairly easy and
114 straightforward. It may become more complicated when it comes to
115 - ensuring that blocks aren't repeated on tangling
116 - ensuring that blocks aren't repeated during session based evaluation
118 *** additional comment (Juan Reyero)
119 Sounds like a good solution. Another possibility would be to add an
120 option that makes chunks dependent on other chunks that appear earlier
121 in the buffer. It is less general, but possibly simpler to implement
122 (you don't have to worry about circular dependencies) and less
123 verbose. If you could assume a functional style without side effects
124 you could even track which chunks are up-to-date, and only re-compute
125 from the first one not up-to-date in the buffer onwards to the chunk
126 you are being asked to process. This could be yet another option.
127 ** PROPOSED :results org should be org block
132 ** TODO Work on tangling
133 *** TODO allow tangle to be called on a single source block
134 this should have a reasonable binding
135 *** PROPOSED make tangled files read-only?
136 With a file-local variable setting, yea that makes sense. Maybe
137 the header should reference the related org-mode file.
138 *** PROPOSED make tangled files executable?
139 At least if using shebang line
140 *** PROPOSED optionally do not output comment and links
141 ** STARTED sha1 hash based caching
143 :CUSTOM_ID: sha1-caching
146 I've put the beginnings of a simple cache of evaluation results in
149 - We'd want some way to re-evaluate without using the cached result
151 - I was considering using org-attach to store the cached data in the
152 attach dir for the tree. But how do we deal with unwanted accumulation
153 of files? In linux at least it's nice that /tmp gets cleared when I
154 reboot (org-babel should do more cleaning up of temp files)
156 - We could byte-compile the cached file (good for large results?)
161 I wonder if we should consider some cashing of images, also for
162 export. I think we could have an alist with sha1 hashes as keys and
163 image files as values. The sha1 hash could be made from the entire
164 code and the command that is used to create the image..
171 (sha1 stuff) seems to work.
173 org-feed.el has a (require 'sha1) and org-publish.el uses it too.
178 ** TODO support for working with =*Org Edit Src Example*= buffers [5/7]
179 *** DONE Patch against org source.
180 I've worked on several related changes to source code edit buffer
181 behaviour in the org core. My current patch (below) does the
182 following. Detailed explanation / working notes are below.
183 - C-x s offers to save edit buffers
184 - C-x C-c offers to save edit buffers
185 - C-x k warns that you're killing an edit buffer
186 - If you do kill an edit buffer, the overlay in the parent buffer is removed
187 - Edit buffers are named *Org Src <orgbuf>[<lang>]*, where
188 <orgbuf> is the name of the org-mode buffer containing this
189 source code block, and lang is the language major mode. The
190 latter might be unnecessary?
192 These changes were added to the main org repository in commit
193 4b6988bf36cb458c9d113ee4332e016990c1eb04
195 **** Detailed working notes to go with that patch
196 ***** Recap of current org-src-mode
198 If you use C-c ' to work on code in a begin_source block, the code
199 buffer is put in minor mode org-src-mode, which features the
200 following two useful key-bindings:
202 | C-x s | org-edit-src-save | save the code in the source code block in the parent org file |
203 | C-c ' | org-edit-src-exit | return to the parent org file with new code |
205 Furthermore, while the edit buffer is alive, the originating code
206 block is subject to a special overlay which links to the edit
207 buffer when you click on it.
209 This is all excellent, and I use it daily, but I think there's
210 still a couple of improvements that we should make.
213 C-x k kills the buffer without questions; the overlay remains, but
214 now links to a deleted buffer.
215 ***** Proposed bug II
216 C-x C-c kills a modified edit buffer silently, without offering to
217 save your work. I have lost work like that a number of times
219 ***** Proposed bug III
220 C-x s does not offer to save a modified edit buffer
221 ***** Notes on solution
222 ****** write-contents-functions
223 A good start seems to be to use org-src-mode-hook to add
224 org-edit-src-save to the write-contents-functions list. This
225 means that when it comes to saving, org-edit-src-save will be
226 called and no subsequent attempt will be made to save the buffer
227 in the normal way. (This should obviate the remapping of C-x C-s
228 to org-edit-src-save in org-src.el)
229 ****** buffer-offer-save
230 We also want to set this to t.
232 ****** Where does this get us?
234 - C-x s still does *not* offer to save the edit buffer. That's
235 because buffer-file-name is nil.
237 - C-x C-c does ask us whether we want to save the
238 edit buffer. However, since buffer-file-name is nil it asks us
239 for a file name. The check in org-edit-src-exit throws an error
240 unless the buffer is named '* Org Edit '...
242 - C-x k kills the buffer silently, leaving a broken overlay
243 link. If buffer-file-name were set, it would have warned that
244 the buffer was modified.
246 ****** buffer-file-name
247 So, that all suggests that we need to set buffer-file-name, even
248 though we don't really want to associate this buffer with a file
249 in the normal way. As for the file name, my current suggestion
250 is parent-org-filename[edit-buffer-name].
252 [I had to move the (org-src-mode) call to the end of
253 org-edit-src-code to make sure that the required variables were
254 defined when the hook was called.]
256 ****** And so where are we now?
257 - C-x s *does* offer to save the edit buffer, but in saving
258 produces a warning that the edit buffer is modified.
259 - C-x k now gives a warning that the edit buffer is modified
261 - C-x C-c is working as desired, except that again we get
262 warnings that the edit buffer is modified, once when we save,
263 and again just before exiting emacs.
264 - And C-c ' now issues a warning that the edit buffer is
265 modified when we leave it, which we don't want.
266 ****** So, we need to get rid of the buffer modification warnings.
267 I've made buffer-file-name nil inside the let binding in
270 - C-x s behaves as desired, except that as was already the case,
271 the edit buffer is always considered modified, and so repeated
272 invocations keep saving it.
273 - As was already the case, C-x k always gives a warning that the
274 edit buffer has been modified.
275 - C-x C-c is as desired (offers to save the edit buffer) except
276 that it warns of the modified buffer just before exiting.
277 - C-c ' is as it should be (silent)
279 We've got the desired behaviour, at the cost of being forced to
280 assign a buffer-file-name to the edit buffer. The consequence is
281 that the edit buffer is considered to always be modified, since
282 a file of that name is never actually written to (doesn't even
283 exist). I couldn't see a way to trick emacs into believing that
284 the buffer was unmodified since last save. But in any case, I
285 think there's an argument that these modifications warnings are
286 a good thing, because one should not leave active edit buffers
287 around: you should always have exited with C-c ' first.
289 *** TODO Doesn't currently work with ess-load-file
290 ess-load-file contains these two lines
291 #+begin_src emacs-lisp
292 (let ((source-buffer (get-file-buffer filename)))
293 (if (ess-check-source filename)
294 (error "Buffer %s has not been saved" (buffer-name source-buffer)))
297 which have the effect of, in the course of saving, deleting the buffer
298 `source-buffer', and then attempting to use it subsequently. The only
299 solution I have thought of so far is submitting a patch to ess which
300 would, e.g. reverse the order of those two lines (perform the error
301 check outside the let binding).
303 In fact, even after doing that there are further problems generated by
304 the fact that the edit buffer has an associated filename for which the
305 file doesn't exist. I think this worked OK in the past when the edit
306 buffer had no associated filename. So this is a problem which needs
307 addressing. Maybe defadvice could be used on ess functions where
308 necessary to make org/org-babel play nicely with ess?
310 **** DONE C-x s steals focus
311 With two modified edit buffers open, make one of them the current
312 buffer and issue C-x s. It will offer to save both of them, but
313 the second one to be saved will become the current buffer at the
315 *** DONE name edit buffer according to #+srcname (and language?)
316 See above patch agains org.
317 *** DONE optionally evaluate header references when we switch to =*Org Edit Src*= buffer
318 That seems to imply that the header references need to be evaluated
319 and transformed into the target language object when we hit C-c ' to
320 enter the *Org Edit Src* buffer [DED]
322 Good point, I heartily agree that this should be supported [Eric]
324 (or at least before the first time we attempt to evaluate code in that
325 buffer -- I suppose there might be an argument for lazy evaluation, in
326 case someone hits C-c ' but is "just looking" and not actually
327 evaluating anything.) Of course if evaluating the reference is
328 computationally intensive then the user might have to wait before they
329 get the *Org Edit Src* buffer. [DED]
331 I fear that it may be hard to anticipate when the references will be
332 needed, some major-modes do on-the-fly evaluation while the buffer is
333 being edited. I think that we should either do this before the buffer
334 is opened or not at all, specifically I think we should resolve
335 references if the user calls C-c ' with a prefix argument. Does that
336 sound reasonable? [Eric]
340 [Dan] So now that we have org-src-mode and org-src-mode-hook, I guess
341 org-babel should do this by using the hook to make sure that, when C-c
342 C-' is issued on a source block, any references are resolved and
343 assignments are made in the appropriate session.
345 #+tblname: my-little-table
349 #+srcname: resolve-vars-on-edit
350 #+begin_src ruby :var table=my-little-table :results silent :session test
351 table.size.times.do |n|
356 *** TODO set buffer-local-process variables appropriately [DED]
357 I think something like this would be great. You've probably
358 already thought of this, but just to note it down: it would be really
359 nice if org-babel's notion of a buffer's 'session/process' played
360 nicely with ESS's notion of the buffer's session/process. ESS keeps
361 the current process name for a buffer in a buffer-local variable
362 ess-local-process-name. So one thing we will probably want to do is
363 make sure that the *Org Edit Src Example* buffer sets that variable
366 I had not thought of that, but I agree whole heartedly. [Eric]
368 Once this is done every variable should be able to dump regions into
369 their inferior-process buffer using major-mode functions.
370 *** REJECTED send code to inferior process
371 Another thought on this topic: I think we will want users to send
372 chunks of code to the interpreter from within the *Org Edit Src*
373 buffer, and I think that's what you have in mind already. In ESS that
374 is done using the ess-eval-* functions. [DED]
376 I think we can leave this up to the major-mode in the source code
377 buffer, as almost every source-code major mode will have functions for
378 doing things like sending regions to the inferior process. If
379 anything we might need to set the value of the buffer local inferior
380 process variable. [Eric]
382 *** DONE some possible requests/proposed changes for Carsten [4/4]
383 While I remember, some possible requests/proposed changes for Carsten
384 come to mind in that regard:
386 **** DONE Remap C-x C-s to save the source to the org buffer?
387 I've done this personally and I find it essential. I'm using
388 #+begin_src emacs-lisp
389 (defun org-edit-src-save ()
390 "Update the parent org buffer with the edited source code, save
391 the parent org-buffer, and return to the source code edit
400 (define-key org-exit-edit-mode-map "\C-x\C-s" 'org-edit-src-save)
404 I think this is great, but I think it should be implemented in the
407 **** DONE Rename buffer and minor mode?
408 Something shorter than *Org Edit Src Example* for the buffer
409 name. org-babel is bringing org's source code interaction to a
410 level of maturity where the 'example' is no longer
411 appropriate. And if further keybindings are going to be added to
412 the minor mode then maybe org-edit-src-mode is a better name than
415 Maybe we should name the buffer with a combination of the source
416 code and the session. I think that makes sense.
418 [ES] Are you also suggesting a new org-edit-src minor mode?
419 [DED] org-exit-edit-mode is a minor mode that already exists:
421 Minor mode installing a single key binding, "C-c '" to exit special edit.
423 org-edit-src-save now has a binding in that mode, so I guess all
424 I'm saying at this stage is that it's a bit of a misnomer. But
425 perhaps we will also have more functionality to add to that minor
426 mode, making it even more of a misnomer. Perhaps something like
427 org-src-mode would be better.
428 **** DONE Changed minor mode name and added hooks
430 **** DONE a hook called when the src edit buffer is created
431 This should be implemented in the org-mode core
432 ** TODO resolve references to other org buffers/files
433 This would allow source blocks to call upon tables, source-blocks,
434 and results in other org buffers/files.
437 - [[file:lisp/org-babel-ref.el::TODO%20allow%20searching%20for%20names%20in%20other%20buffers][org-babel-ref.el:searching-in-other-buffers]]
438 - [[file:lisp/org-babel.el::defun%20org-babel%20find%20named%20result%20name][org-babel.el#org-babel-find-named-result]]
439 ** TODO resolve references to other non-org files
440 - tabular data in .csv, .tsv etc format
441 - files of interpreted code: anything stopping us giving such files
442 similar status to a source code block?
443 - Would be nice to allow org and non-org files to be remote
444 ** TODO command line execution
445 Allow source code blocks to be called form the command line. This
446 will be easy using the =sbe= function in [[file:lisp/org-babel-table.el][org-babel-table.el]].
448 This will rely upon [[* resolve references to other buffers][resolve references to other buffers]].
449 ** TODO LoB: re-implement plotting and analysis functions from org-R
450 I'll do this soon, now that we things are a bit more settled and we
451 have column names in R.
452 ** TODO Improved error checking
453 E.g. when trying to execute sass block, I did not have sass
454 installed, and so shell-command returned code 127, but org-babel
455 did not warn me that anything had gone wrong.
456 *** Error checking in R
457 A simple thing to do is to wrap the R code in try(...), as in the
458 patch below. That results in some improved behaviour:
459 - You get the error message from R
460 - Execution halts at first error
462 #+begin_src R :results output :session *R*
470 #+begin_src R :results output :session *R*
476 : Error in log("a") : Non-numeric argument to mathematical function
479 diff --git a/contrib/babel/lisp/langs/org-babel-R.el b/contrib/babel/lisp/langs/org-babel-R.el
480 index 1ef21db..45f8409 100644
481 --- a/contrib/babel/lisp/langs/org-babel-R.el
482 +++ b/contrib/babel/lisp/langs/org-babel-R.el
483 @@ -103,8 +103,8 @@ last statement in BODY, as elisp."
484 (out-tmp-file (make-temp-file "R-out-functional-results")))
487 - (with-temp-file in-tmp-file (insert body))
488 - (shell-command-to-string (format "R --slave --no-save < '%s' > '%s'"
489 + (with-temp-file in-tmp-file (insert (concat "try({" body "})")))
490 + (shell-command-to-string (format "R --slave --no-save < '%s' > '%s' 2>&1"
491 in-tmp-file out-tmp-file))
492 (with-temp-buffer (insert-file-contents out-tmp-file) (buffer-string)))
494 @@ -124,7 +124,7 @@ last statement in BODY, as elisp."
495 (format "write.table(.Last.value, file=\"%s\", sep=\"\\t\", na=\"nil\",row.names=FALSE, col.names=%s, quote=FALSE)" tmp-file (if column-names-p "TRUE" "FALSE"))
496 org-babel-R-eoe-indicator) "\n"))
498 - (mapconcat #'org-babel-chomp (list body org-babel-R-eoe-indicator) "\n"))))
499 + (mapconcat #'org-babel-chomp (list (concat "try({" body "})") org-babel-R-eoe-indicator) "\n"))))
500 (raw (org-babel-comint-with-output buffer org-babel-R-eoe-output nil
501 (insert full-body) (inferior-ess-send-input)))
503 diff --git a/contrib/babel/lisp/org-babel-ref.el b/contrib/babel/lisp/org-babel-ref.el
504 index 0e8695f..060f880 100644
505 --- a/contrib/babel/lisp/org-babel-ref.el
506 +++ b/contrib/babel/lisp/org-babel-ref.el
507 @@ -139,7 +139,7 @@ return nil."
508 ('results-line (org-babel-read-result))
509 ('table (org-babel-read-table))
511 - (setq result (org-babel-execute-src-block t nil args))
512 + (setq result (org-babel-execute-src-block t (org-babel-get-src-block-info) args))
513 (if (symbolp result) (format "%S" result) result))
514 ('lob (setq result (org-babel-execute-src-block t lob-info args)))))))
516 *** DEFERRED figure out how to handle errors during evaluation
517 I expect it will be hard to do this properly, but ultimately it
518 would be nice to be able to specify somewhere to receive STDERR,
519 and to be warned if it is non-empty.
521 Probably simpler in non-session evaluation than session? At least
522 the mechanism will be different I guess.
524 R has a try function, with error handling, along the lines of
525 python. I bet ruby does too. Maybe more of an issue for functional
526 style; in my proposed scripting style the error just gets dumped to
527 the org buffer and the user is thus alerted.
529 For now I think the current behavior of returning any error
530 messages generated by the source language is sufficient.
531 ** TODO Finalise argument-passing syntax
533 #+srcname: unnamedargs(x=7)
538 #+lob: unnamedargs(5)
540 #+resname: unnamedargs(5)
543 In general we need to have a full set of rules for how a string
544 of supplied arguments (some possibly named) interact with the
545 arguments in the definition (some possibly with defaults) to give
546 values to the variables in the function body.
547 ** STARTED share org-babel [3/7]
548 how should we share org-babel?
549 *** DONE post to org-mode
550 *** TODO post to ess mailing list
551 I'd like to not rush in to this, get some feedback from the org
552 list first and let my R usage of org-babel settle down. [DD]
553 *** DONE create a org-babel page on worg
554 **** DONE Getting hold of it instructions
555 - What about non-git users?
556 - Are we moving/copying to contrib/?
557 **** DEFERRED Fixed width HTML output created by =...= is ugly!
558 *** TODO create a short screencast demonstrating org-babel in action
559 *** PROPOSED a peer-reviewed publication?
561 The following notes are biased towards statistics-oriented
562 journals because ESS and Sweave are written by people associated
563 with / in statistics departments. But I am sure there are suitable
564 journals out there for an article on using org mode for
565 reproducible research (and literate programming etc).
567 Clearly, we would invite Carsten to be involved with this.
569 ESS is described in a peer-reviewed journal article:
570 Emacs Speaks Statistics: A Multiplatform, Multipackage Development Environment for Statistical Analysis [Abstract]
571 Journal of Computational & Graphical Statistics 13(1), 247-261
572 Rossini, A.J, Heiberger, R.M., Sparapani, R.A., Maechler, M., Hornik, K. (2004)
573 [[http://www.amstat.org/publications/jcgs.cfm][Journal of Computational and Graphical Statistics]]
575 Also [[http://www.amstat.org/publications/jss.cfm][Journal of Statistical Software]] Established in 1996, the
576 Journal of Statistical Software publishes articles, book reviews,
577 code snippets, and software reviews. The contents are freely
578 available online. For both articles and code snippets, the source
579 code is published along with the paper.
583 Friedrich Leisch and Anthony J. Rossini. Reproducible statistical
584 research. Chance, 16(2):46-50, 2003. [ bib ]
588 Friedrich Leisch. Sweave: Dynamic generation of statistical reports
589 using literate data analysis. In Wolfgang Härdle and Bernd Rönz,
590 editors, Compstat 2002 - Proceedings in Computational Statistics,
591 pages 575-580. Physica Verlag, Heidelberg, 2002. ISBN 3-7908-1517-9.
595 We could also look at the Journals publishing these [[http://www.reproducibleresearch.net/index.php/RR_links#Articles_about_RR_.28chronologically.29][Reproducible
598 *** PROPOSED an article in [[http://journal.r-project.org/][The R Journal]]
599 This looks good. It seems that their main topic to software tools for
600 use by R programmers, and Org-babel is certainly that.
602 *** existing similar tools
603 try to collect pointers to similar tools
605 Reproducible Research
606 - [[http://en.wikipedia.org/wiki/Sweave][Sweave]]
609 - [[http://www.cs.tufts.edu/~nr/noweb/][Noweb]]
610 - [[http://www-cs-faculty.stanford.edu/~knuth/cweb.html][Cweb]]
611 - [[http://www.lri.fr/~filliatr/ocamlweb/][OCamlWeb]]
613 Meta Functional Programming
616 Programmable Spreadsheet
620 we need to think up some good examples
622 **** interactive tutorials
623 This could be a place to use [[* org-babel assertions][org-babel assertions]].
625 for example the first step of a tutorial could assert that the version
626 of the software-package (or whatever) is equal to some value, then
627 source-code blocks could be used with confidence (and executed
628 directly from) the rest of the tutorial.
630 **** answering a text-book question w/code example
631 org-babel is an ideal environment enabling both the development and
632 demonstrationg of the code snippets required as answers to many
635 **** something using tables
636 maybe something along the lines of calculations from collected grades
639 Maybe something like the following which outputs sizes of directories
640 under the home directory, and then instead of the trivial =emacs-lisp=
641 block we could use an R block to create a nice pie chart of the
645 #+begin_src bash :results replace
649 #+begin_src emacs-lisp :var sizes=sizes :results replace
652 *** DONE Answer to question on list
653 From: Hector Villafuerte <hectorvd@gmail.com>
654 Subject: [Orgmode] Merge tables
655 Date: Wed, 19 Aug 2009 10:08:40 -0600
656 To: emacs-orgmode@gnu.org
659 I've just discovered Org and are truly impressed with it; using it for
662 Here's what I want to do: I have 2 tables with the same number of rows
663 (one row per subject). I would like to make just one big table by
664 copying the second table to the right of the first one. This is a
665 no-brainer in a spreadsheet but my attempts in Org have failed. Any
668 By the way, thanks for this great piece of software!
672 **** Suppose the tables are as follows
685 **** Here is an answer using R in org-babel
687 #+srcname: column-bind(a=tab1, b=tab2)
688 #+begin_src R :colnames t
692 #+resname: column-bind
693 | "a" | "b" | "c" | "d" | "e" | "f" |
694 |-----+-----+-----+-----+-----+-----|
695 | 1 | 2 | 3 | 4 | 5 | 6 |
696 | 7 | 8 | 9 | 10 | 11 | 12 |
700 Use org-table-export, do it in external spreadsheet software,
701 then org-table-import
702 ** PROPOSED allow for stripping of header rows from table data
703 maybe controlled by an argument
704 ** PROPOSED Control precision of numerical output
705 Does org have an option controlling precision of numbers in tables?
706 ** PROPOSED allow `anonymous' function block with function call args?
707 My question here is simply whether we're going to allow
708 #+begin_src python(arg=ref)
712 but with preference given to
713 #+srcname blockname(arg=ref)
714 ** PROPOSED allow :result as synonym for :results?
715 ** PROPOSED Creating presentations
716 The [[mairix:t:@@9854.1246500519@gamaville.dokosmarshall.org][recent thread]] containing posts by Nick Dokos and Sebastian
717 Vaubán on exporting to beamer looked very interesting, but I
718 haven't had time to try it out yet. I would really like it if,
719 eventually, we can generate a presentation (with graphics generated
720 by code blocks) from the same org file that contains all the notes
721 and code etc. I just wanted that to be on record in this document;
722 I don't have anything more profound to say about it at the moment,
723 and I'm not sure to what extent it is an org-babel issue.
724 ** PROPOSED conversion between org-babel and noweb (e.g. .Rnw) format
725 I haven't thought about this properly. Just noting it down. What
726 Sweave uses is called "R noweb" (.Rnw).
728 I found a good description of noweb in the following article (see
729 the [[http://www.cs.tufts.edu/~nr/pubs/lpsimp.pdf][pdf]]).
731 I think there are two parts to noweb, the construction of
732 documentation and the extraction of source-code (with notangle).
734 *documentation*: org-mode handles all of our documentation needs in
735 a manner that I believe is superior to noweb.
737 *source extraction* At this point I don't see anyone writing large
738 applications with 100% of the source code contained in org-babel
739 files, rather I see org-babel files containing things like
740 - notes with active code chunks
741 - interactive tutorials
742 - requirements documents with code running test suites
743 - and of course experimental reports with the code to run the
744 experiment, and perform analysis
746 Basically I think the scope of the programs written in org-babel
747 (at least initially) will be small enough that it wont require the
748 addition of a tangle type program to extract all of the source code
749 into a running application.
751 On the other hand, since we already have named blocks of source
752 code which reference other blocks on which they rely, this
753 shouldn't be too hard to implement either on our own, or possibly
754 relying on something like noweb/notangle.
755 ** DONE allow 'output mode to return stdout as value?
756 We do allow this now. It turned out to be necessary for lob calls using output mode.
758 Maybe we should allow this. In fact, if block x is called
759 with :results output, and it references blocks y and z, then
760 shouldn't the output of x contain a concatenation of the outputs of
761 y and z, together with x's own output? That would raise the
762 question of what happens if y is defined with :results output and z
763 with :results value. I guess z's (possibly vector/tabular) output
764 would be inside a literal example block containing the whole lot.
765 ** DONE Reworking output: verbatim, pretty-print, vector/scalar etc
766 See emails on subject between ES, BA, DED.
767 *** Draft of conclusions of email discussion
768 - We add two new :results options (which only take effect
769 with :results value):
770 - [X] parseable (code) output
771 - [X] pretty-printed output
772 - We get rid of :results scalar: [EMS] I don't see why results
773 scalar is on the chopping block? Did we want another name for
774 this value, or is it overcome by the verbatim/code options? For
775 now I propose we leave the scalar option unchanged.
776 - [X] We rename :results vector -> table (even though this will be
777 the default it is nice to have a name for it, and we need a
778 symbolic name for implementing -- and allowing users to
779 change -- the default behavior)
780 - [X] We force :results output to be empty for elisp
782 - output as code block
783 - possible :results option names
787 *** Pretty-printed output
788 - output as literal text block (not as code block)
789 - possible :results option names
794 *** Language-specific implementation
796 | language | parseable | pretty | Notes |
797 |------------+---------------+-----------------------------+---------------------------------|
798 | emacs-lisp | pp | pp | |
799 | ruby | ?pp | pp | is pp output parseable? |
800 | python | pprint.pprint | pprint.pprint | |
802 | R | dput | default interpreter output? | |
803 | shell | NA | NA | no such thing as :results value |
806 *** DONE pretty print source results
810 add a result type for the display of source-code objects. results of
811 this type should be wrapped in source-code blocks.
813 inspired by Benny's pp patch for emacs-lisp (below)
816 diff --git a/contrib/babel/lisp/langs/org-babel-emacs-lisp.el b/contrib/babel/lisp/langs/org-babel-emacs-lisp.el
817 index 39f5cc7..60671ac 100644
818 --- a/contrib/babel/lisp/langs/org-babel-emacs-lisp.el
819 +++ b/contrib/babel/lisp/langs/org-babel-emacs-lisp.el
821 "Execute a block of emacs-lisp code with org-babel. This
822 function is called by `org-babel-execute-src-block' via multiple-value-bind."
823 (message "executing emacs-lisp code block...")
824 - (save-window-excursion
825 - (let ((print-level nil) (print-length nil))
826 - (eval `(let ,(mapcar (lambda (var) `(,(car var) ',(cdr var))) vars)
827 - ,(read (concat "(progn " body ")")))))))
828 + (let ((results (cdr (assoc :results params))))
829 + (save-window-excursion
830 + (let ((print-level nil) (print-length nil))
831 + (eval `(let ,(mapcar (lambda (var) `(,(car var) ',(cdr var))) vars)
832 + ,(read (concat "(progn " (if (string-match "pp$" results)
833 + (concat "(pp " body ")")
837 (provide 'org-babel-emacs-lisp)
838 ;;; org-babel-emacs-lisp.el ends here
841 something similar for ruby exists here
842 http://www.ruby-doc.org/stdlib/libdoc/pp/rdoc/index.html
843 *** DONE Finalise behaviour regarding vector/scalar output
845 *** DONE Stop spaces causing vector output
846 This simple example of multilingual chaining produces vector output if
847 there are spaces in the message and scalar otherwise.
851 #+srcname: msg-from-R(msg=msg-from-python)
853 paste(msg, "und R", sep=" ")
857 : org-babel speaks elisp y python und R
859 #+srcname: msg-from-python(msg=msg-from-elisp)
864 #+srcname: msg-from-elisp(msg="org-babel speaks")
865 #+begin_src emacs-lisp
866 (concat msg " elisp")
869 ** DONE support for passing paths to files between source blocks
870 Maybe this should be it's own result type (in addition to scalars and
871 vectors). The reason being that some source-code blocks (for example
872 ditaa or anything that results in the creation of a file) may want to
873 pass a file path back to org-mode which could then be inserted into
874 the org-mode buffer as a link to the file...
876 This would allow for display of images upon export providing
877 functionality similar to =org-exp-blocks= only in a more general
879 ** DEFERRED optional timestamp for output
880 *DEFERRED*: I'm deferring this in deference to the better caching
881 system proposed by Carsten. (see [[sha1-caching]])
883 Add option to place an (inactive) timestamp at the #+resname, to
884 record when that output was generated.
886 *** source code block timestamps (optional addition)
887 [Eric] If we did this would we then want to place a timestamp on the
888 source-code block, so that we would know if the results are
889 current or out of date? This would have the effect of caching the
890 results of calculations and then only re-running if the
891 source-code has changed. For the caching to work we would need to
892 check not only the timestamp on a source-code block, but also the
893 timestamps of any tables or source-code blocks referenced by the
894 original source-code block.
896 [Dan] I do remember getting frustrated by Sweave always having to
897 re-do everything, so this could be desirable, as long as it's easy
898 to over-ride of course. I'm not sure it should be the default
899 behaviour unless we are very confident that it works well.
901 **** maintaining source-code block timestamps
902 It may make sense to add a hook to `org-edit-special' which could
903 update the source-code blocks timestamp. If the user edits the
904 contents of a source-code block directly I can think of no
905 efficient way of maintaining the timestamp.
906 ** DEFERRED source-name visible in LaTeX and html exports
907 Maybe this should be done in backend specific manners.
909 The listings package may provide for naming a source-code block...
911 Actually there is no obvious simple and attractive way to implement
912 this. Closing this issue for now.
913 ** DEFERRED Support rownames and other org babel table features?
915 The full org table features are detailed in the manual [[http://orgmode.org/manual/Advanced-features.html#Advanced-features][here]].
918 Perhaps add a :rownames header arg. This would be an integer
919 (usually 1) which would have the effect of post-processing all the
920 variables created in the R session in the following way: if the
921 integer is j, set the row names to the contents of column j and
922 delete column j. Perhaps it is artificial to allow this integer to
923 take any value other than 1. The default would be nil which would
924 mean no such behaviour.
926 Actually I don't know about that. If multiple variables are passed
927 in, it's not appropriate to alter them all in the same way. The
928 rownames specification would normally refer to just one of the
929 variables. For now maybe just say this has to be done in R. E.g.
931 #+TBLNAME: sample-sizes
932 | collection | size | exclude | include | exclude2 | include2 |
933 |-----------------+------+---------+---------+----------+----------|
934 | 58C | 2936 | 8 | 2928 | 256 | 2680 |
935 | MS | 5852 | 771 | 5081 | 771 | 5081 |
936 | NBS | 2929 | 64 | 2865 | 402 | 2527 |
937 | POBI | 2717 | 1 | 2716 | 1 | 2716 |
938 | 58C+MS+NBS+POBI | | | 13590 | | 13004 |
939 #+TBLFM: @2$4=@2$2 - @2$3::@2$6=@2$2 - @2$5::@3$4=@3$2-@3$3::@3$6=@3$2 - @3$5::@4$4=@4$2 - @4$3::@4$6=@4$2 - @4$5::@5$4=@5$2-@5$3::@5$6=@5$2 - @5$5::@6$4=vsum(@2$4..@5$4)::@6$6=vsum(@2$6..@5$6)
941 #+srcname: make-size-table(size=sample-sizes)
943 rownames(size) <- size[,1]
949 [I don't think it's as problematic as this makes out]
950 This is non-trivial, but may be worth doing, in particular to
951 develop a nice framework for sending data to/from R.
953 In R, indexing vector elements, and rows and columns, using
954 strings rather than integers is an important part of the
956 - elements of a vector may have names
957 - matrices and data.frames may have "column names" and "row names"
958 which can be used for indexing
959 - In a data frame, row names *must* be unique
967 > mat <- matrix(1:4, nrow=2, ncol=2, dimnames=list(c("r1","r2"), c("c1","c2")))
972 > # The names are separate from the data: they do not interfere with operations on the data
979 > df <- data.frame(var1=1:26, var2=26:1, row.names=letters)
981 [1] 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1
987 So it's tempting to try to provide support for this in org-babel. For example
988 - allow R to refer to columns of a :var reference by their names
989 - When appropriate, results from R appear in the org buffer with "named
992 However none (?) of the other languages we are currently supporting
993 really have a native matrix type, let alone "column names" or "row
994 names". Names are used in e.g. python and perl to refer to entries
997 It currently seems to me that support for this in org-babel would
998 require setting rules about when org tables are considered to have
999 named columns/fields, and ensuring that (a) languages with a notion
1000 of named columns/fields use them appropriately and (b) languages
1001 with no such notion do not treat then as data.
1003 - Org allows something that *looks* like column names to be separated
1005 - Org also allows a row to *function* as column names when special
1006 markers are placed in the first column. An hline is unnecessary
1007 (indeed hlines are purely cosmetic in org [correct?]
1008 - Org does not have a notion of "row names" [correct?]
1010 The full org table functionality exeplified [[http://orgmode.org/manual/Advanced-features.html#Advanced-features][here]] has features that
1011 we would not support in e.g. R (like names for the row below).
1013 **** Initial statement: allow tables with hline to be passed as args into R
1014 This doesn't seem to work at the moment (example below). It would
1015 also be nice to have a natural way for the column names of the org
1016 table to become the column names of the R data frame, and to have
1017 the option to specify that the first column is to be used as row
1018 names in R (these must be unique). But this might require a bit of
1023 | col1 | col2 | col3 |
1024 |------+---------+------|
1032 #+begin_src R :var tabel=egtable :colnames t
1037 | "col1" | "col2" | "col3" |
1038 |--------+-----------+--------|
1040 | 4 | "schulte" | 6 |
1043 Another example is in the [[*operations%20in%20on%20tables][grades example]].
1044 ** DEFERRED use textConnection to pass tsv to R?
1045 When passing args from the org buffer to R, the following route is
1046 used: arg in buffer -> elisp -> tsv on file -> data frame in R. I
1047 think it would be possible to avoid having to write to file by
1048 constructing an R expression in org-babel-R-assign-elisp, something
1051 #+begin_src emacs-lisp
1052 (org-babel-R-input-command
1053 (format "%s <- read.table(textConnection(\"%s\"), sep=\"\\t\", as.is=TRUE)"
1054 name (orgtbl-to-tsv value '(:sep "\t" :fmt org-babel-R-quote-tsv-field))))
1057 I haven't tried to implement this yet as it's basically just
1058 fiddling with something that works. The only reason for it I can
1059 think of would be efficiency and I haven't tested that.
1061 This Didn't work after an initial test. I still think this is a
1062 good idea (I also think we should try to do something similar when
1063 writing out results frmo R to elisp) however as it wouldn't result
1064 in any functional changes I'm bumping it down to deferred for
1069 #+tblname: quick-test
1072 #+srcname: quick-test-src-blk
1073 #+begin_src R :var vec=quick-test
1083 ** DEFERRED Rework Interaction with Running Processes [2/5]
1084 *** DONE robust to errors interrupting execution
1086 #+srcname: long-runner-ruby
1087 #+begin_src ruby :results silent
1089 :patton_is_an_grumpy
1092 *** DEFERRED use =C-g= keyboard-quit to push processing into the background
1093 This may be possible using the `run-with-timer' command.
1095 I have no idea how this could work...
1097 #+srcname: long-runner-ruby
1098 #+begin_src ruby :results silent
1100 :patton_is_an_grumpy
1103 *** TODO ability to select which of multiple sessions is being used
1104 Increasingly it is looking like we're going to want to run all
1105 source code blocks in comint buffer (sessions). Which will have
1107 1) allowing background execution
1108 2) maintaining state between source-blocks
1109 - allowing inline blocks w/o header arguments
1112 (like ess-switch-process in .R buffers)
1114 Maybe this could be packaged into a header argument, something
1115 like =:R_session= which could accept either the name of the
1116 session to use, or the string =prompt=, in which case we could use
1117 the =ess-switch-process= command to select a new process.
1119 *** TODO evaluation of shell code as background process?
1120 After C-c C-c on an R code block, the process may appear to
1121 block, but C-g can be used to reclaim control of the .org buffer,
1122 without interrupting the R evalution. However I believe this is not
1123 true of bash/sh evaluation. [Haven't tried other languages] Perhaps
1124 a solution is just to background the individual shell commands.
1126 The other languages (aside from emacs lisp) are run through the
1127 shell, so if we find a shell solution it should work for them as
1130 Adding an ampersand seems to be a supported way to run commands in
1131 the background (see [[http://www.emacswiki.org/emacs/ExecuteExternalCommand#toc4][external-commands]]). Although a more extensible
1132 solution may involve the use of the [[elisp:(progn (describe-function 'call-process-region) nil)][call-process-region]] function.
1134 Going to try this out in a new file [[file:lisp/org-babel-proc.el][org-babel-proc.el]]. This should
1135 contain functions for asynchronously running generic shell commands
1136 in the background, and then returning their input.
1138 **** partial update of org-mode buffer
1139 The sleekest solution to this may be using a comint buffer, and
1140 then defining a filter function which would incrementally interpret
1141 the results as they are returned, including insertion into the
1142 org-mode buffer. This may actually cause more problems than it is
1143 worth, what with the complexities of identifying the types of
1144 incrementally returned results, and the need for maintenance of a
1145 process marker in the org buffer.
1147 **** 'working' spinner
1148 It may be nice and not too difficult to place a spinner on/near the
1149 evaluating source code block
1151 *** TODO conversion of output from interactive shell, R (and python) sessions to org-babel buffers
1152 [DED] This would be a nice feature I think. Although an org-babel
1153 purist would say that it's working the wrong way round... After
1154 some interactive work in a *R* buffer, you save the buffer, maybe
1155 edit out some lines, and then convert it to org-babel format for
1156 posterity. Same for a shell session either in a *shell* buffer, or
1157 pasted from another terminal emulator. And python of course.
1158 ** DEFERRED improve the source-block snippet
1159 any real improvement seems somewhat beyond the ability of yasnippet
1162 [[file:~/src/emacs-starter-kit/src/snippets/text-mode/rst-mode/chap::name%20Chapter%20title][file:~/src/emacs-starter-kit/src/snippets/text-mode/rst-mode/chap::name Chapter title]]
1164 ,#name : Chapter title
1167 ${1:$(make-string (string-width text) ?\=)}
1172 [[file:snippets/org-mode/sb][sb -- snippet]]
1174 waiting for guidance from those more familiar with yasnippets
1176 ** REJECTED re-implement R evaluation using ess-command or ess-execute
1177 I don't have any complaints with the current R evaluation code or
1178 behaviour, but I think it would be good to use the ESS functions
1179 from a political point of view. Plus of course it has the normal
1180 benefits of an API (insulates us from any underlying changes etc). [DED]
1182 I'll look into this. I believe that I looked at and rejected these
1183 functions initially but now I can't remember why. I agree with
1184 your overall point about using API's where available. I will take
1185 a look back at these and either switch to using the ess commands,
1186 or at least articulate under this TODO the reasons for using our
1187 custom R-interaction commands. [Eric]
1191 Lets just replace =org-babel-R-input-command= with =ess-execute=.
1193 I tried this, and although it works in some situations, I find that
1194 =ess-command= will often just hang indefinitely without returning
1195 results. Also =ess-execute= will occasionally hang, and pops up
1196 the buffer containing the results of the command's execution, which
1197 is undesirable. For now these functions can not be used. Maybe
1198 someone more familiar with the ESS code can recommend proper usage
1199 of =ess-command= or some other lower-level function which could be
1200 used in place of [[file:lisp/org-babel-R.el::defun%20org-babel%20R%20input%20command%20command][org-babel-R-input-command]].
1204 #+begin_quote ess-command
1205 (ess-command COM &optional BUF SLEEP NO-PROMPT-CHECK)
1207 Send the ESS process command COM and delete the output
1208 from the ESS process buffer. If an optional second argument BUF exists
1209 save the output in that buffer. BUF is erased before use.
1210 COM should have a terminating newline.
1211 Guarantees that the value of .Last.value will be preserved.
1212 When optional third arg SLEEP is non-nil, `(sleep-for (* a SLEEP))'
1213 will be used in a few places where `a' is proportional to `ess-cmd-delay'.
1216 #+begin_quote ess-execute
1217 (ess-execute COMMAND &optional INVERT BUFF MESSAGE)
1219 Send a command to the ESS process.
1220 A newline is automatically added to COMMAND. Prefix arg (or second arg
1221 INVERT) means invert the meaning of
1222 `ess-execute-in-process-buffer'. If INVERT is 'buffer, output is
1223 forced to go to the process buffer. If the output is going to a
1224 buffer, name it *BUFF*. This buffer is erased before use. Optional
1225 fourth arg MESSAGE is text to print at the top of the buffer (defaults
1226 to the command if BUFF is not given.)
1229 *** out current setup
1231 1) The body of the R source code block is wrapped in a function
1232 2) The function is called inside of a =write.table= function call
1233 writing the results to a table
1234 3) The table is read using =org-table-import=
1235 ** DONE Incorporate [[http://github.com/bandresen/org-babel-screen][Benny Andresen's GNU screen interpreter]]
1236 ** DONE Ensure that #+lob calls honour header args
1237 Currently, the header args defined in a source block are not
1238 honoured when that source block is referenced by a #+lob call.
1240 Maybe we will have to either stop using an emacs-lisp block in the
1241 lob implementation, or provide some mechanism for the emacs-lisp
1242 blocks to pass unused header arguments through to their :var
1243 blocks. At first glance the former seems much cleaner. [Eric]
1244 **** Allow header args on lob line
1245 commit da1f07620862a2f8701597fbd6d8ceca93183840
1247 Here's a source block that defines the action we want to do.
1249 #+begin_src ditaa :file blue.png :cmdline -r :exports none
1259 And here's some more text.
1261 **** Another heading
1264 Here is where we actually want the image to appear
1266 #+lob: myplot() :file blue.png
1270 ** DONE dynamic clock tables (as input to source blocks)
1273 : #+BEGIN: clocktable :maxlevel 2 :block today :scope tree1 :link t :name todays-clock
1276 These actually work given the current setup, the tables simply need to
1277 be named using a =#+TBLNAME:= line.
1279 : #+TBLNAME: todays-times
1280 : #+BEGIN: clocktable :maxlevel 2 :block today :scope tree1 :link t :name todays-clock
1282 ** DONE figure out how to handle graphic output
1284 This is listed under [[* graphical output][graphical output]] in out objectives.
1286 This should take advantage of the =:results file= option, and
1287 languages which almost always produce graphical output should set
1288 =:results file= to true by default (this is currently done for the
1289 gnuplot and ditaa languages). That would handle placing these results
1290 in the buffer. Then if there is a combination of =silent= and =file=
1291 =:results= headers we could drop the results to a temp buffer and pop
1294 Display of file results is addressed in the [[* =\C-c \C-o= to open results of source block][open-results-task]].
1296 I think this is done for now. With the ability of the file option it
1297 is now possible to save images directly to a file. Then calling
1298 =\C-c\C-o= with point on the source block will open the related
1301 *** R graphics to screen means session evaluation
1302 If R graphical output is going to screen then evaluation must be
1303 in a session, otherwise the graphics will disappear as soon as the
1306 *** Adding to a discussion started in email
1307 I'm not deeply wedded to these ideas, just noting them down. I'm
1308 probably just thinking of R and haven't really thought about how
1309 this fits with the other graphics-generating languages.
1311 > I used the approach below to get graphical file output
1312 > today, which is one idea at least. Maybe it could be linked up with
1313 > your :results file variable. (Or do we need a :results image for R?)
1316 I don't think we need a special image results variable, but I may be
1317 missing what the code below accomplishes. Would the task I added about
1318 adding org-open-at-point functionality to source code blocks take care
1321 Dan: I'm not sure. I think the ability for a script to generate both
1322 text and graphical output might be a natural expectation, at least for
1328 > #+srcname: cohort-scatter-plots-2d(org_babel_graphical_output_file="cohort-scatter-plots-2d.png")
1330 > if(exists("org_babel_output_file"))
1331 > png(filename=org_babel_graphical_output_file, width=1000, height=1000)
1332 > ## plotting code in here
1333 > if(exists("org_babel_graphical_output_file")) dev.off()
1336 Dan: Yes, the results :file option is nice for dealing with graphical
1337 output, and that could well be enough. Something based on the scheme
1338 above would have a couple of points in its favour:
1339 1. It's easy to switch between output going to on-screen graphics and
1340 output going to file: Output will go to screen unless a string variable
1341 with a standard name (e.g. ""org_babel_graphical_output_file"")
1342 exists in which case it will go to the file indicated by the value
1344 2. The block can return a result / script output, as well as produce
1347 In interactive use we might want to allow the user to choose between
1348 screen and file output. In non-interactive use such as export, it
1349 would be file output (subject to the :exports directives).
1350 ** DONE new results types (org, html, latex)
1351 Thanks to Tom Short for this recommendation.
1353 - raw or org :: in which case the results are implemented raw, unquoted
1354 into the org-mode file. This would also handle links as
1355 source block output.
1356 - html :: the results are inserted inside of a #+BEGIN_HTML block
1357 - latex :: the results are inserted inside of a #+BEGIN_LATEX block
1360 : #+begin_src R :session *R* :results org
1361 : cat("***** This is a table\n")
1362 : cat("| 1 | 2 | 3 |\n")
1363 : cat("[[http://google.com][Google it here]]\n"
1367 : ***** This is a table
1369 [[http://google.com][: Google it here]]
1371 We actually might want to remove the =#+resname= line if the results
1372 type is org-mode, not sure... Either way I don't think there is a
1373 good way to capture/remove org type results.
1376 #+srcname: latex-results
1377 #+begin_src emacs-lisp :results latex
1378 "this should be inside of a LaTeX block"
1383 this should be inside of a LaTeX block
1387 #+srcname: html-results
1388 #+begin_src emacs-lisp :results html
1389 "this should be inside of a HTML block
1402 this should be inside of a HTML block
1415 Added a =raw= results header argument, which will insert the results
1416 of a source-code block into an org buffer un-escaped. Also, if the
1417 results look like a table, then the table will be aligned.
1419 #+srcname: raw-table-demonstration
1420 #+begin_src ruby :results output raw
1421 puts "| root | square |"
1424 puts "| #{n} | #{n*n} |"
1442 Not sure how/if this would work, but it may be desirable.
1443 ** DONE org-bable-tangle: no default extension if one already exists
1444 ** DONE take default values for header args from properties
1445 Use file-wide and subtree wide properties to set default values for
1448 [DED] One thing I'm finding when working with R is that an org file
1449 may contain many source blocks, but that I just want to evaluate a
1450 subset of them. Typically this is in order to take up where I left
1451 off: I need to recreate a bunch of variables in the session
1452 environment. I'm thinking maybe we want to use a tag-based
1453 mechanism similar to :export: and :noexport: to control evaluation
1454 on a per-subtree basis.
1456 *** test-header with properties
1462 Ahh... as is so often the case, just had to wrap
1463 `org-babel-params-from-properties' in a `save-match-data' form.
1465 #+tblname: why-def-props-cause-probs
1468 #+srcname: default-props-implementation
1469 #+begin_src emacs-lisp :tangle no :var my-lis=why-def-props-cause-probs :results silent
1470 (+ (length my-lis) def)
1473 ** DONE new reference syntax *inside* source code blocks
1474 This is from an email discussion on the org-mode mailing list with
1475 Sébastien. The goal here is to mimic the source-block reference style
1476 of Noweb. Upon export and/or tangle these references could be
1477 replaced with the actual body of the referenced source-code block.
1479 See the following for an example.
1481 #+srcname: ems-ruby-print-header
1483 puts "---------------------------header---------------------------"
1486 #+srcname: emacs-ruby-print-footer
1488 puts "---------------------------footer---------------------------"
1491 #+srcname: ems-ruby-print-message
1492 #+begin_src ruby :file ruby-noweb.rb
1493 # <<ems-ruby-print-header>>
1495 # <<ems-ruby-print-footer>>
1498 Upon export the previous source-code block would result in a file
1499 being generated at =ruby-noweb.rb= with the following contents
1501 : puts "---------------------------header---------------------------"
1503 : puts "---------------------------footer---------------------------"
1505 the body of a source-code block with all =<<src-name>>= references
1506 expanded can now be returned by `org-babel-expand-noweb-references'.
1507 This function is now called by default on all source-code blocks on
1510 ** DONE re-work tangling system
1511 Sometimes when tangling a file (e.g. when extracting elisp from a
1512 org-mode file) we want to get nearly every source-code block.
1514 Sometimes we want to only extract those source-code blocks which
1515 reference a indicate that they should be extracted (e.g. traditional
1516 literate programming along the Noweb model)
1518 I'm not sure how we can devise a single simple tangling system that
1519 naturally fits both of these use cases.
1522 the =tangle= header argument will default to =no= meaning source-code
1523 blocks will *not* be exported by default. In order for a source-code
1524 block to be tangled it needs to have an output file specified. This
1525 can happen in two ways...
1527 1) a file-wide default output file can be passed to `org-babel-tangle'
1528 which will then be used for all blocks
1529 2) if the value of the =tangle= header argument is anything other than
1530 =no= or =yes= then it is used as the file name
1532 #+srcname: test-new-tangling
1533 #+begin_src emacs-lisp
1534 (org-babel-load-file "test-tangle.org")
1535 (if (string= test-tangle-advert "use org-babel-tangle for all your emacs initialization files!!")
1543 ** DONE =\C-c \C-o= to open results of source block
1544 by adding a =defadvice= to =org-open-at-point= we can use the common
1545 =\C-c \C-o= keybinding to open the results of a source-code block.
1546 This would be especially useful for source-code blocks which generate
1547 graphical results and insert a file link as the results in the
1548 org-mode buffer. (see [[* figure out how to handle graphic output][TODO figure out how to handle graphic output]]).
1549 This could also act reasonably with other results types...
1551 - file :: use org-open-at-point to open the file
1552 - scalar :: open results unquoted in a new buffer
1553 - tabular :: export the table to a new buffer and open that buffer
1555 when called with a prefix argument the block is re-run
1557 #+srcname: task-opening-results-of-blocks
1558 #+begin_src ditaa :results replace :file blue.png :cmdline -r
1569 [[file:blue.png][blue.png]]
1571 #+srcname: task-open-vector
1572 #+begin_src emacs-lisp
1580 #+srcname: task-open-scalar
1581 #+begin_src ruby :results output
1597 ** DONE Stop spaces causing vector output
1598 This simple example of multilingual chaining produces vector output if
1599 there are spaces in the message and scalar otherwise.
1603 #+srcname: msg-from-R(msg=msg-from-python)
1605 paste(msg, "und R", sep=" ")
1609 : org-babel speaks elisp y python und R
1611 #+srcname: msg-from-python(msg=msg-from-elisp)
1616 #+srcname: msg-from-elisp(msg="org-babel speaks")
1617 #+begin_src emacs-lisp
1618 (concat msg " elisp")
1621 ** DONE add =:tangle= family of header arguments
1623 - no :: don't include source-code block when tangling
1624 - yes :: do include source-code block when tangling
1626 this is tested in [[file:test-tangle.org::*Emacs%20Lisp%20initialization%20stuff][test-tangle.org]]
1628 ** DONE extensible library of callable source blocks
1630 This is covered by the [[file:library-of-babel.org][Library of Babel]], which will contain
1631 ready-made source blocks designed to carry out useful common tasks.
1632 *** Initial statement [Eric]
1633 Much of the power of org-R seems to be in it's helper functions for
1634 the quick graphing of tables. Should we try to re-implement these
1635 functions on top of org-babel?
1637 I'm thinking this may be useful both to add features to org-babel-R and
1638 also to potentially suggest extensions of the framework. For example
1639 one that comes to mind is the ability to treat a source-code block
1640 like a function which accepts arguments and returns results. Actually
1641 this can be it's own TODO (see [[* source blocks as functions][source blocks as functions]]).
1642 *** Objectives [Dan]
1643 - We want to provide convenient off-the-shelf actions
1644 (e.g. plotting data) that make use of our new code evaluation
1645 environment but do not require any actual coding.
1646 *** Initial Design proposal [Dan]
1647 - *Input data* will be specified using the same mechanism as :var
1648 references, thus the input data may come from a table, or
1649 another source block, and it is initially available as an elisp
1651 - We introduce a new #+ line, e.g. #+BABELDO. C-c C-c on that
1652 line will apply an *action* to the referenced data.
1653 - *Actions correspond to source blocks*: our library of available
1654 actions will be a library of org-babel source blocks. Thus the
1655 code for executing an action, and the code for dealing with the
1656 output of the action will be the same code as for executing
1657 source blocks in general
1658 - Optionally, the user can have the relevant source block inserted
1659 into the org buffer after the (say) #+BABELDO line. This will
1660 allow the user to fine tune the action by modifying the code
1661 (especially useful for plots).
1662 - So maybe a #+BABELDO line will have header args
1663 - :data (a reference to a table or source code block)
1664 - :action (or should that be :srcname?) which will be something
1665 like :action pie-chart, referring to a source block which will
1666 be executed with the :data referent passed in using a :var arg.
1667 - :showcode or something controlling whether to show the code
1669 *** Modification to design
1670 I'm implementing this, at least initially, as a new interpreter
1671 named 'babel', which has an empty body. 'babel' blocks take
1672 a :srcname header arg, and look for the source-code block with
1673 that name. They then execute the referenced block, after first
1674 appending their own header args on to the target block's header
1677 If the target block is in the library of babel (a.o.t. e.g. the
1678 current buffer), then the code in the block will refer to the
1679 input data with a name dictated by convention (e.g. __data__
1680 (something which is syntactically legal in all languages...). Thus
1681 the babel block will use a :var __data__ = whatever header arg to
1682 reference the data to be plotted.
1684 ** DONE Column names in R input/output
1685 This has been implemented: Automatic on input to R; optional in
1686 output. Note that this equates column names with the header row in
1687 an org table; whereas org actually has a mechanism whereby a row
1688 with a '!' in the first field defines column names. I have not
1689 attempted to support these org table mechanisms yet. See [[*Support%20rownames%20and%20other%20org%20babel%20table%20features][this
1690 DEFERRED todo item]].
1691 ** DONE use example block for large amounts of stdout output?
1692 We're currently `examplizing' with : at the beginning of the line,
1693 but should larger amounts of output be in a
1694 \#+begin_example...\#+end_example block? What's the cutoff? > 1
1695 line? This would be nice as it would allow folding of lengthy
1696 output. Sometimes one will want to see stdout just to check
1697 everything looks OK, and then fold it away.
1699 I'm addressing this in branch 'examplizing-output'.
1700 Yea, that makes sense. (either that or allow folding of large
1701 blocks escaped with =:=).
1703 Proposed cutoff of 10 lines, we can save this value in a user
1704 customizable variable.
1705 *** DONE add ability to remove such results
1706 ** DONE exclusive =exports= params
1708 #+srcname: implement-export-exclusivity
1715 ** DONE LoB: allow output in buffer
1716 ** DONE allow default header arguments by language
1717 org-babel-default-header-args:lang-name
1719 An example of when this is useful is for languages which always return
1720 files as their results (e.g. [[*** ditaa][ditaa]], and [[*** gnuplot][gnuplot]]).
1721 ** DONE singe-function tangling and loading elisp from literate org-mode file [3/3]
1723 This function should tangle the org-mode file for elisp, and then call
1724 `load-file' on the resulting tangled file.
1726 #+srcname: test-loading-embedded-emacs-lisp
1727 #+begin_src emacs-lisp :results replace
1728 (setq test-tangle-advert nil)
1729 (setq test-tangle-loading nil)
1730 (setq results (list :before test-tangle-loading test-tangle-advert))
1731 (org-babel-load-file "test-tangle.org")
1732 (setq results (list (list :after test-tangle-loading test-tangle-advert) results))
1733 (delete-file "test-tangle.el")
1737 #+resname: test-loading-embedded-emacs-lisp
1738 | :before | nil | nil |
1739 | :after | "org-babel tangles" | "use org-babel-tangle for all your emacs initialization files!!" |
1741 *** DONE add optional language limiter to org-babel-tangle
1742 This should check to see if there is any need to re-export
1744 *** DONE ensure that org-babel-tangle returns the path to the tangled file(s)
1746 #+srcname: test-return-value-of-org-babel-tangle
1747 #+begin_src emacs-lisp :results replace
1748 (mapcar #'file-name-nondirectory (org-babel-tangle-file "test-tangle.org" "emacs-lisp"))
1752 | "test-tangle.el" |
1754 *** DONE only tangle the file if it's actually necessary
1755 ** DONE add a function to jump to a source-block by name
1756 I've had an initial stab at that in org-babel-find-named-block
1757 (library-of-babel branch).
1759 At the same time I introduced org-babel-named-src-block-regexp, to
1760 match src-blocks with srcname.
1762 This is now working with the command
1763 `org-babel-goto-named-source-block', all we need is a good key
1766 ** DONE add =:none= session argument (for purely functional execution) [4/4]
1767 This would allow source blocks to be run in their own new process
1769 - These blocks could then also be run in the background (since we can
1770 detach and just wait for the process to signal that it has terminated)
1771 - We wouldn't be drowning in session buffers after running the tests
1772 - we can re-use much of the session code to run in a more /functional/
1775 While session provide a lot of cool features, like persistent
1776 environments, [[* DONE function to bring up inferior-process buffer][pop-to-session]], and hints at exportation for
1777 org-babel-tangle, they also have some down sides and I'm thinking that
1778 session-based execution maybe shouldn't be the default behavior.
1780 Down-sides to sessions
1781 - *much* more complicated than functional evaluation
1782 - maintaining the state of the session has weird issues
1783 - waiting for evaluation to finish
1784 - prompt issues like [[* TODO weird escaped characters in shell prompt break shell evaluation][shell-prompt-escapes-bug]]
1785 - can't run in background
1786 - litter emacs with session buffers
1790 #+srcname: ruby-task-no-session
1791 #+begin_src ruby :results replace output
1797 #+resname: ruby-task-no-session
1802 #+srcname: task-python-none-session
1803 #+begin_src python :session none :results replace value
1809 #+resname: task-python-none-session
1814 #+srcname: task-session-none-sh
1815 #+begin_src sh :results replace
1820 #+resname: task-session-none-sh
1826 #+srcname: task-no-session-R
1827 #+begin_src R :results replace output
1834 #+resname: task-no-session-R
1838 ** DONE fully purge org-babel-R of direct comint interaction
1839 try to remove all code under the [[file:lisp/org-babel-R.el::functions%20for%20evaluation%20of%20R%20code][;; functions for evaluation of R code]] line
1841 ** DONE Create objects in top level (global) environment [5/5]
1844 *** initial requirement statement [DED]
1845 At the moment, objects created by computations performed in the
1846 code block are evaluated in the scope of the
1847 code-block-function-body and therefore disappear when the code
1848 block is evaluated {unless you employ some extra trickery like
1849 assign('name', object, env=globalenv()) }. I think it will be
1850 desirable to also allow for a style wherein objects that are
1851 created in one code block persist in the R global environment and
1852 can be re-used in a separate block.
1854 This is what Sweave does, and while I'm not saying we have to be
1855 the same as Sweave, it wouldn't be hard for us to provide the same
1856 behaviour in this case; if we don't, we risk undeservedly being
1857 written off as an oddity by some.
1859 IOW one aspect of org-babel is that of a sort of functional
1860 meta-programming language. This is crazy, in a very good
1861 way. Nevertheless, wrt R I think there's going to be a lot of value
1862 in providing for a working style in which the objects are stored in
1863 the R session, rather than elisp/org buffer. This will be a very
1864 familiar working style to lots of people.
1866 There are no doubt a number of different ways of accomplishing
1867 this, the simplest being a hack like adding
1870 for(objname in ls())
1871 assign(objname, get(objname), envir=globalenv())
1874 to the source code block function body. (Maybe wrap it in an on.exit() call).
1876 However this may deserve to be thought about more carefully, perhaps
1877 with a view to having a uniform approach across languages. E.g. shell
1878 code blocks have the same semantics at the moment (no persistence of
1879 variables across code blocks), because the body is evaluated in a new
1880 bash shell process rather than a running shell. And I guess the same
1881 is true for python. However, in both these cases, you could imagine
1882 implementing the alternative in which the body is evaluated in a
1883 persistent interactive session. It's just that it's particularly
1884 natural for R, seeing as both ESS and org-babel evaluate commands in a
1885 single persistent R session.
1889 Thanks for bringing this up. I think you are absolutely correct that we
1890 should provide support for a persistent environment (maybe called a
1891 *session*) in which to evaluate code blocks. I think the current setup
1892 demonstrates my personal bias for a functional style of programming
1893 which is certainly not ideal in all contexts.
1895 While the R function you mention does look like an elegant solution, I
1896 think we should choose an implementation that would be the same across
1897 all source code types. Specifically I think we should allow the user to
1898 specify an optional *session* as a header variable (when not present we
1899 assume a default session for each language). The session name could be
1900 used to name a comint buffer (like the *R* buffer) in which all
1901 evaluation would take place (within which variables would retain their
1902 values --at least once I remove some of the functional method wrappings
1903 currently in place-- ).
1905 This would allow multiple environments to be used in the same buffer,
1906 and once this setup was implemented we should be able to fairly easily
1907 implement commands for jumping between source code blocks and the
1908 related session buffers, as well as for dumping the last N commands from
1909 a session into a new or existing source code block.
1911 Please let me know if you foresee any problems with this proposed setup,
1912 or if you think any parts might be confusing for people coming from
1913 Sweave. I'll hopefully find some time to work on this later in the
1916 *** can functional and interpreted/interactive models coexist?
1918 Even though both of these use the same =*R*= buffer the value of =a=
1919 is not preserved because it is assigned inside of a functional
1922 #+srcname: task-R-sessions
1929 #+srcname: task-R-same-session
1934 This functional wrapper was implemented in order to efficiently return
1935 the results of the execution of the entire source code block. However
1936 it inhibits the evaluation of source code blocks in the top level,
1937 which would allow for persistence of variable assignment across
1938 evaluations. How can we allow *both* evaluation in the top level, and
1939 efficient capture of the return value of an entire source code block
1940 in a language independent manner?
1942 Possible solutions...
1943 1) we can't so we will have to implement two types of evaluation
1944 depending on which is appropriate (functional or imperative)
1945 2) we remove the functional wrapper and parse the source code block
1946 into it's top level statements (most often but not always on line
1947 breaks) so that we can isolate the final segment which is our
1949 3) we add some sort of "#+return" line to the code block
1950 4) we take advantage of each languages support for meta-programming
1951 through =eval= type functions, and use said to evaluate the entire
1952 blocks in such a way that their environment can be combined with the
1953 global environment, and their results are still captured.
1954 5) I believe that most modern languages which support interactive
1955 sessions have support for a =last_result= type function, which
1956 returns the result of the last input without re-calculation. If
1957 widely enough present this would be the ideal solution to a
1958 combination of functional and imperative styles.
1960 None of these solutions seem very desirable, but for now I don't see
1961 what else would be possible.
1963 Of these options I was leaning towards (1) and (4) but now believe
1964 that if it is possible option (5) will be ideal.
1966 **** (1) both functional and imperative evaluation
1968 - can take advantage of built in functions for sending regions to the
1970 - retains the proven tested and working functional wrappers
1973 - introduces the complication of keeping track of which type of
1974 evaluation is best suited to a particular context
1975 - the current functional wrappers may require some changes in order to
1976 include the existing global context
1978 **** (4) exploit language meta-programming constructs to explicitly evaluate code
1980 - only one type of evaluation
1983 - some languages may not have sufficient meta-programming constructs
1985 **** (5) exploit some =last_value= functionality if present
1987 Need to ensure that most languages have such a function, those without
1988 will simply have to implement their own similar solution...
1990 | language | =last_value= function |
1991 |------------+-----------------------------|
1995 | shell | see [[* last command for shells][last command for shells]] |
1996 | emacs-lisp | see [[* emacs-lisp will be a special case][special-case]] |
1998 #+srcname: task-last-value
2003 ***** last command for shells
2004 Do this using the =tee= shell command, and continually pipe the output
2007 Got this idea from the following [[http://linux.derkeiler.com/Mailing-Lists/Fedora/2004-01/0898.html][email-thread]].
2009 suggested from mailing list
2011 #+srcname: bash-save-last-output-to-file
2015 bash -c "$line" | tee /tmp/last.out1
2016 mv /tmp/last.out1 /tmp/last.out
2020 another proposed solution from the above thread
2022 #+srcname: bash-save-in-variable
2025 # so - Save Output. Saves output of command in OUT shell variable.
2035 "^M": " | tee /tmp/h_lastcmd.out ^[k"
2038 export __=/tmp/h_lastcmd.out
2040 If you try it, Alt-k will stand for the old Enter; use "command $__" to
2041 access the last output.
2047 Herculano de Lima Einloft Neto
2050 ***** emacs-lisp will be a special case
2051 While it is possible for emacs-lisp to be run in a console type
2052 environment (see the =elim= function) it is *not* possible to run
2053 emacs-lisp in a different *session*. Meaning any variable set top
2054 level of the console environment will be set *everywhere* inside
2055 emacs. For this reason I think that it doesn't make any sense to
2056 worry about session support for emacs-lisp.
2058 *** Further thoughts on 'scripting' vs. functional approaches
2060 These are just thoughts, I don't know how sure I am about this.
2061 And again, perhaps I'm not saying anything very radical, just that
2062 it would be nice to have some options supporting things like
2063 receiving text output in the org buffer.
2065 I can see that you've already gone some way down the road towards
2066 the 'last value' approach, so sorry if my comments come rather
2067 late. I am concerned that we are not giving sufficient attention
2068 to stdout / the text that is returned by the interpreters. In
2069 contrast, many of our potential users will be accustomed to a
2070 'scripting' approach, where they are outputting text at various
2071 points in the code block, not just at the end. I am leaning
2072 towards thinking that we should have 2 modes of evaluation:
2073 'script' mode, and 'functional' mode.
2075 In script mode, evaluation of a code block would result in *all*
2076 text output from that code block appearing as output in the org
2077 buffer, presumably as an #+begin_example...#+end_example. There
2078 could be an :echo option controlling whether the input commands
2079 also appear in the output. [This is like Sweave].
2081 In functional mode, the *result* of the code block is available as
2082 an elisp object, and may appear in the org buffer as an org
2083 table/string, via the mechanisms you have developed already.
2085 One thing I'm wondering about is whether, in script mode, there
2086 simply should not be a return value. Perhaps this is not so
2087 different from what exists: script mode would be new, and what
2088 exists currently would be functional mode.
2090 I think it's likely that, while code evaluation will be exciting
2091 to people, a large majority of our users in a large majority of
2092 their usage will not attempt to actually use the return value from
2093 a source code block in any meaningful way. In that case, it seems
2094 rather restrictive to only allow them to see output from the end
2097 Instead I think the most accessible way to introduce org-babel to
2098 people, at least while they are learning it, is as an immensely
2099 powerful environment in which to embed their 'scripts', which now
2100 also allows them to 'run' their 'scripts'. Especially as such
2101 people are likely to be the least capable of the user-base, a
2102 possible design-rule would be to make the scripting style of usage
2103 easy (default?), perhaps requiring a special option to enable a
2104 functional style. Those who will use the functional style won't
2105 have a problem understanding what's going on, whereas the 'skript
2106 kiddies' might not even know the syntax for defining a function in
2107 their language of choice. And of course we can allow the user to
2108 set a variable in their .emacs controlling the preference, so that
2109 functional users are not inconveniennced by having to provide
2110 header args the whole time.
2112 Please don't get the impression that I am down-valuing the
2113 functional style of org-babel. I am constantly horrified at the
2114 messy 'scripts' that my colleagues produce in perl or R or
2115 whatever! Nevertheless that seems to be how a lot of people work.
2117 I think you were leaning towards the last-value approach because
2118 it offered the possibility of unified code supporting both the
2119 single evaluation environment and the functional style. If you
2120 agree with any of the above then perhaps it will impact upon this
2121 and mean that the code in the two branches has to differ a bit. In
2122 that case, functional mode could perhaps after all evaluate each
2123 code block in its own environment, thus (re)approaching 'true'
2124 functional programming (side-effects are hard to achieve).
2128 echo "There are `wc -l files` files in this directory"
2132 *** even more thoughts on evaluation, results, models and options
2134 Thanks Dan, These comments are invaluable.
2136 What do you think about this as a new list of priorities/requirements
2137 for the execution of source-code blocks.
2140 1) we want the evaluation of the source code block to take place in a
2141 session which can persist state (variables, current directory,
2143 2) source code blocks can specify their session with a header argument
2144 3) each session should correspond to an Emacs comint buffer so that the
2145 user can drop into the session and experiment with live code
2148 1) each source-code block generates some form of results which (as
2149 we have already implemented) is transfered into emacs-lisp
2150 after which it can be inserted into the org-mode buffer, or
2151 used by other source-code blocks
2152 2) when the results are translated into emacs-lisp, forced to be
2153 interpreted as a scalar (dumping their raw values into the
2154 org-mode buffer), as a vector (which is often desirable with R
2155 code blocks), or interpreted on the fly (the default option).
2156 Note that this is very nearly currently implemented through the
2157 [[* DONE results-type header (vector/file)][results-type-header]].
2158 3) there should be *two* means of collecting results from the
2159 execution of a source code block. *Either* the value of the
2160 last statement of the source code block, or the collection of
2161 all that has been passed to STDOUT during the evaluation.
2163 **** header argument or return line (*header argument*)
2165 Rather than using a header argument to specify how the return value
2166 should be passed back, I'm leaning towards the use of a =#+RETURN=
2167 line inside the block. If such a line *is not present* then we
2168 default to using STDOUT to collect results, but if such a line *is
2169 present* then we use it's value as the results of the block. I
2170 think this will allow for the most elegant specification between
2171 functional and script execution. This also cleans up some issues
2172 of implementation and finding which statement is the last
2175 Having given this more thought, I think a header argument is
2176 preferable. The =#+return:= line adds new complicating syntax for
2177 something that does little more than we would accomplish through
2178 the addition of a header argument. The only benefit being that we
2179 know where the final statement starts, which is not an issue in
2180 those languages which contain 'last value' operators.
2182 new header =:results= arguments
2183 - script :: explicitly states that we want to use STDOUT to
2184 initialize our results
2185 - return_last :: stdout is ignored instead the *value* of the final
2186 statement in the block is returned
2187 - echo :: means echo the contents of the source-code block along
2188 with the results (this implies the *script* =:results=
2191 *** DONE rework evaluation lang-by-lang [4/4]
2193 This should include...
2194 - functional results working with the comint buffer
2196 - script :: return the output of STDOUT
2197 - write a macro which runs the first redirection, executes the
2198 body, then runs the second redirection
2199 - last :: return the value of the last statement
2202 - sessions in comint buffers
2204 **** DONE Ruby [4/4]
2205 - [X] functional results working with comint
2206 - [X] script results
2207 - [X] ensure scalar/vector results args are taken into consideration
2208 - [X] ensure callable by other source block
2210 #+srcname: ruby-use-last-output
2211 #+begin_src ruby :results replace
2218 #+resname: ruby-use-last-output
2221 #+srcname: task-call-use-last-output
2222 #+begin_src ruby :var last=ruby-use-last-output :results replace
2223 last.flatten.size + 1
2226 #+resname: task-call-use-last-output
2231 #+srcname: first-ruby-session-task
2232 #+begin_src ruby :session schulte :results silent
2236 #+srcname: second-ruby-session-task
2237 #+begin_src ruby :session schulte :results silent
2241 #+srcname: without-the-right-session
2242 #+begin_src ruby :results silent
2248 - [X] functional results working with comint
2249 - [X] script results
2250 - [X] ensure scalar/vector results args are taken into consideration
2251 - [X] ensure callable by other source block
2253 To redirect output to a file, you can use the =sink()= command.
2256 #+begin_src R :results value vector silent
2263 #+srcname: task-R-use-other-output
2264 #+begin_src R :var twoentyseven=task_R_B() :results replace value
2269 #+resname: task-R-use-other-output
2272 **** DONE Python [4/4]
2273 - [X] functional results working with comint
2274 - [X] script results
2275 - [X] ensure scalar/vector results args are taken into consideration
2276 - [X] ensure callable by other source block
2278 #+srcname: task-new-eval-for-python
2279 #+begin_src python :results silent output scalar
2285 #+srcname: task-use-new-eval
2286 #+begin_src python :var tasking=task-new-eval-for-python() :results replace
2290 #+resname: task-use-new-eval
2293 **** DONE Shells [4/4]
2294 - [X] functional results working with comint
2295 - [X] script results
2296 - [X] ensure scalar/vector results args are taken into consideration
2297 - [X] ensure callable by other source block
2299 #+srcname: task-shell-new-evaluation
2300 #+begin_src sh :results silent value scalar
2305 #+srcname: task-call-other-shell
2306 #+begin_src sh :var other=task-shell-new-evaluation() :results replace scalar
2307 echo $other ' is the old date'
2310 #+resname: task-call-other-shell
2311 : $ Fri Jun 12 13:08:37 PDT 2009 is the old date
2313 *** DONE implement a *session* header argument [4/4]
2314 =:session= header argument to override the default *session* buffer
2318 #+srcname: task-ruby-named-session
2319 #+begin_src ruby :session schulte :results replace
2320 schulte = :in_schulte
2323 #+resname: task-ruby-named-session
2326 #+srcname: another-in-schulte
2327 #+begin_src ruby :session schulte
2331 #+resname: another-in-schulte
2338 #+srcname: python-session-task
2339 #+begin_src python :session what :results silent
2343 #+srcname: python-get-from-session
2344 #+begin_src python :session what :results replace
2348 #+resname: python-get-from-session
2353 #+srcname: task-shell-sessions
2354 #+begin_src sh :session what
2358 #+srcname: task-shell-sessions-what
2359 #+begin_src sh :session what :results replace
2363 #+resname: task-shell-sessions-what
2368 #+srcname: task-R-session
2369 #+begin_src R :session what :results replace
2375 #+resname: task-R-session
2378 #+srcname: another-task-R-session
2379 #+begin_src R :session what :results replace
2383 *** DONE function to bring up inferior-process buffer [4/4]
2385 This should be callable from inside of a source-code block in an
2386 org-mode buffer. It should evaluate the header arguments, then bring
2387 up the inf-proc buffer using =pop-to-buffer=.
2389 For lack of a better place, lets add this to the `org-metadown-hook'
2392 To give this a try, place the cursor on a source block with variables,
2393 (optionally git a prefix argument) then hold meta and press down.
2397 #+srcname: task-ruby-pop-to-session
2398 #+begin_src ruby :var num=9 :var another="something else"
2399 num.times{|n| puts another}
2404 #+srcname: task-python-pop-to-session
2405 #+begin_src python :var num=9 :var another="something else"
2410 #+srcname: task-R-pop-to-session
2411 #+begin_src R :var a=9 :var b=8
2417 #+srcname: task-shell-pop-sessions
2418 #+begin_src sh :var NAME="eric"
2422 *** DEFERRED function to dump last N lines from inf-proc buffer into the current source block
2424 Callable with a prefix argument to specify how many lines should be
2425 dumped into the source-code buffer.
2427 *** REJECTED comint notes
2429 Implementing comint integration in [[file:lisp/org-babel-comint.el][org-babel-comint.el]].
2432 - handling of outputs
2433 - split raw output from process by prompts
2434 - a ring of the outputs, buffer-local, `org-babel-comint-output-ring'
2435 - a switch for dumping all outputs to a buffer
2436 - inputting commands
2438 Lets drop all this language specific stuff, and just use
2439 org-babel-comint to split up our outputs, and return either the last
2440 value of an execution or the combination of values from the
2443 **** comint filter functions
2444 : ;; comint-input-filter-functions hook process-in-a-buffer
2445 : ;; comint-output-filter-functions hook function modes.
2446 : ;; comint-preoutput-filter-functions hook
2447 : ;; comint-input-filter function ...
2449 #+srcname: obc-filter-ruby
2450 #+begin_src ruby :results last
2458 ** DONE Remove protective commas from # comments before evaluating
2459 org inserts protective commas in front of ## comments in language
2460 modes that use them. We need to remove them prior to sending code
2463 #+srcname: testing-removal-of-protective-comas
2465 ,# this one might break it??
2469 ** DONE pass multiple reference arguments into R
2470 Can we do this? I wasn't sure how to supply multiple 'var' header
2471 args. Just delete this if I'm being dense.
2473 This should be working, see the following example...
2475 #+srcname: two-arg-example
2476 #+begin_src R :var n=2 :var m=8
2480 #+resname: two-arg-example
2483 ** DONE ensure that table ranges work
2484 when a table range is passed to org-babel as an argument, it should be
2485 interpreted as a vector.
2488 | 2 | 3 | Fixnum:1 |
2489 | 3 | 4 | Array:123456 |
2493 #+TBLFM: @1$3='(sbe simple-sbe-example (n 4))::@2$3='(sbe task-table-range (n @1$1..@6$1))::@3$3='(sbe task-table-range (n (@1$1..@6$1)))
2495 #+srcname: simple-sbe-example
2496 #+begin_src emacs-lisp
2500 #+srcname: task-table-range
2501 #+begin_src ruby :var n=simple-sbe-example
2505 #+srcname: simple-results
2506 #+begin_src emacs-lisp :var n=task-table-range(n=(1 2 3))
2510 #+resname: simple-results
2513 #+srcname: task-arr-referent
2514 #+begin_src ruby :var ar=(1 2 3)
2518 #+resname: task-arr-referent
2521 ** DONE global variable indicating default to vector output
2522 how about an alist... =org-babel-default-header-args= this may already
2523 exist... just execute the following and all source blocks will default
2526 #+begin_src emacs-lisp
2527 (setq org-babel-default-header-args '((:results . "vector")))
2530 ** DONE name named results if source block is named
2531 currently this isn't happening although it should be
2533 #+srcname: test-naming-named-source-blocks
2534 #+begin_src emacs-lisp
2538 #+resname: test-naming-named-source-blocks
2540 ** DONE (simple caching) check for named results before source blocks
2541 see the TODO comment in [[file:lisp/org-babel-ref.el::TODO%20This%20should%20explicitly%20look%20for%20resname%20lines%20before][org-babel-ref.el#org-babel-ref-resolve-reference]]
2542 ** DONE set =:results silent= when eval with prefix argument
2544 #+begin_src emacs-lisp
2547 ** DONE results-type header (vector/file) [3/3]
2548 In response to a point in Dan's email. We should allow the user to
2549 force scalar or vector results. This could be done with a header
2550 argument, and the default behavior could be controlled through a
2551 configuration variable.
2553 #+srcname: task-trivial-vector
2554 #+begin_src ruby :results replace vector
2561 since it doesn't make sense to turn a vector into a scalar, lets
2562 just add a two values...
2564 - vector :: forces the results to be a vector (potentially 1 dimensional)
2565 - file :: this throws an error if the result isn't a string, and
2566 tries to treat it as a path to a file.
2568 I'm just going to cram all of these into the =:results= header
2569 argument. Then if we allow multiple header arguments it should
2570 work out, for example one possible header argument string could be
2571 =:results replace vector file=, which would *replace* any existing
2572 results forcing the results into an org-mode table, and
2573 interpreting any strings as file paths.
2575 *** DONE multiple =:results= headers
2577 #+srcname: multiple-result-headers
2578 #+begin_src ruby :results replace silent
2584 *** DONE file result types
2585 When inserting into an org-mode buffer create a link with the path
2586 being the value, and optionally the display being the
2587 =file-name-nondirectory= if it exists.
2589 #+srcname: task-file-result
2590 #+begin_src python :results replace file
2595 [[something][something]]
2598 This will be useful because blocks like =ditaa= and =dot= can return
2599 the string path of their files, and can add =file= to their results
2602 *** DONE vector result types
2604 #+srcname: task-force-results
2605 #+begin_src emacs-lisp :results vector
2612 ** DONE results name
2613 In order to do this we will need to start naming our results.
2614 Since the source blocks are named with =#+srcname:= lines we can
2615 name results with =#+resname:= lines (if the source block has no
2616 name then no name is given to the =#+resname:= line on creation,
2617 otherwise the name of the source block is used).
2619 This will have the additional benefit of allowing results and
2620 source blocks to be located in different places in a buffer (and
2621 eventually in different buffers entirely).
2623 #+srcname: developing-resnames
2624 #+begin_src emacs-lisp :results silent
2628 Once source blocks are able to find their own =#+resname:= lines
2631 #+srcname: sbe-w-new-results
2632 #+begin_src emacs-lisp :results replace
2633 (sbe "developing-resnames")
2639 *** TODO change the results insertion functions to use these lines
2641 *** TODO teach references to resolve =#+resname= lines.
2643 ** DONE org-babel tests org-babel [1/1]
2644 since we are accumulating this nice collection of source-code blocks
2645 in the sandbox section we should make use of them as unit tests.
2646 What's more, we should be able to actually use org-babel to run these
2649 We would just need to cycle over every source code block under the
2650 sandbox, run it, and assert that the return value is equal to what we
2653 I have the feeling that this should be possible using only org-babel
2654 functions with minimal or no additional elisp. It would be very cool
2655 for org-babel to be able to test itself.
2657 This is now done, see [[* Tests]].
2659 *** DEFERRED org-babel assertions (may not be necessary)
2660 These could be used to make assertions about the results of a
2661 source-code block. If the assertion fails then the point could be
2662 moved to the block, and error messages and highlighting etc... could
2665 ** DONE make C-c C-c work anywhere within source code block?
2666 This seems like it would be nice to me, but perhaps it would be
2667 inefficient or ugly in implementation? I suppose you could search
2668 forward, and if you find #+end_src before you find #+begin_src,
2669 then you're inside one. [DED]
2671 Agreed, I think inside of the =#+srcname: line= would be useful as
2674 #+srcname: testing-out-cc
2675 #+begin_src emacs-lisp
2679 ** DONE integration with org tables
2680 We should make it easy to call org-babel source blocks from org-mode
2681 table formulas. This is practical now that it is possible to pass
2682 arguments to org-babel source blocks.
2684 See the related [[* (sandbox) integration w/org tables][sandbox]] header for tests/examples.
2686 *** digging in org-table.el
2687 In the past [[file:~/src/org/lisp/org-table.el::org%20table%20el%20The%20table%20editor%20for%20Org%20mode][org-table.el]] has proven difficult to work with.
2689 Should be a hook in [[file:~/src/org/lisp/org-table.el::defun%20org%20table%20eval%20formula%20optional%20arg%20equation][org-table-eval-formula]].
2691 Looks like I need to change this [[file:~/src/org/lisp/org-table.el::if%20lispp][if statement]] (line 2239) into a cond
2694 ** DONE source blocks as functions
2696 Allow source code blocks to be called like functions, with arguments
2697 specified. We are already able to call a source-code block and assign
2698 it's return result to a variable. This would just add the ability to
2699 specify the values of the arguments to the source code block assuming
2700 any exist. For an example see
2702 When a variable appears in a header argument, how do we differentiate
2703 between it's value being a reference or a literal value? I guess this
2704 could work just like a programming language. If it's escaped or in
2705 quotes, then we count it as a literal, otherwise we try to look it up
2708 ** DONE folding of code blocks? [2/2]
2709 [DED] In similar way to using outline-minor-mode for folding function
2710 bodies, can we fold code blocks? #+begin whatever statements are
2711 pretty ugly, and in any case when you're thinking about the overall
2712 game plan you don't necessarily want to see the code for each Step.
2714 *** DONE folding of source code block
2715 Sounds good, and wasn't too hard to implement. Code blocks should
2716 now be fold-able in the same manner as headlines (by pressing TAB
2719 *** REJECTED folding of results
2720 So, lets do a three-stage tab cycle... First fold the src block,
2721 then fold the results, then unfold.
2723 There's no way to tell if the results are a table or not w/o
2724 actually executing the block which would be too expensive of an
2727 ** DONE selective export of text, code, figures
2728 [DED] The org-babel buffer contains everything (code, headings and
2729 notes/prose describing what you're up to, textual/numeric/graphical
2730 code output, etc). However on export to html / LaTeX one might want
2731 to include only a subset of that content. For example you might
2732 want to create a presentation of what you've done which omits the
2735 [EMS] So I think this should be implemented as a property which can
2736 be set globally or on the outline header level (I need to review
2737 the mechanics of org-mode properties). And then as a source block
2738 header argument which will apply only to a specific source code
2739 block. A header argument of =:export= with values of
2741 - =code= :: just show the code in the source code block
2742 - =none= :: don't show the code or the results of the evaluation
2743 - =results= :: just show the results of the code evaluation (don't
2744 show the actual code)
2745 - =both= :: show both the source code, and the results
2747 this will be done in [[* (sandbox) selective export][(sandbox) selective export]].
2749 ** DONE a header argument specifying silent evaluation (no output)
2750 This would be useful across all types of source block. Currently
2751 there is a =:replace t= option to control output, this could be
2752 generalized to an =:output= option which could take the following
2753 options (maybe more)
2755 - =t= :: this would be the default, and would simply insert the
2756 results after the source block
2757 - =replace= :: to replace any results which may already be there
2758 - =silent= :: this would inhibit any insertion of the results
2760 This is now implemented see the example in the [[* silent evaluation][sandbox]]
2762 ** DONE assign variables from tables in R
2763 This is now working (see [[* (sandbox table) R][(sandbox-table)-R]]). Although it's not that
2764 impressive until we are able to print table results from R.
2766 ** DONE insert 2-D R results as tables
2767 everything is working but R and shell
2773 This has already been tackled by Dan in [[file:existing_tools/org-R.el::defconst%20org%20R%20write%20org%20table%20def][org-R:check-dimensions]]. The
2774 functions there should be useful in combination with [[http://cran.r-project.org/doc/manuals/R-data.html#Export-to-text-files][R-export-to-csv]]
2775 as a means of converting multidimensional R objects to emacs lisp.
2777 It may be as simple as first checking if the data is multidimensional,
2778 and then, if so using =write= to write the data out to a temporary
2779 file from which emacs can read the data in using =org-table-import=.
2781 Looking into this further, is seems that there is no such thing as a
2782 scalar in R [[http://tolstoy.newcastle.edu.au/R/help/03a/3733.html][R-scalar-vs-vector]] In that light I am not sure how to
2783 deal with trivial vectors (scalars) in R. I'm tempted to just treat
2784 them as vectors, but then that would lead to a proliferation of
2785 trivial 1-cell tables...
2787 ** DONE allow variable initialization from source blocks
2788 Currently it is possible to initialize a variable from an org-mode
2789 table with a block argument like =table=sandbox= (note that the
2790 variable doesn't have to named =table=) as in the following example
2796 #+begin_src emacs-lisp :var table=sandbox :results replace
2797 (message (format "table = %S" table))
2800 : "table = ((1 2 3) (4 \"schulte\" 6))"
2802 It would be good to allow initialization of variables from the results
2803 of other source blocks in the same manner. This would probably
2804 require the addition of =#+SRCNAME: example= lines for the naming of
2805 source blocks, also the =table=sandbox= syntax may have to be expanded
2806 to specify whether the target is a source code block or a table
2807 (alternately we could just match the first one with the given name
2808 whether it's a table or a source code block).
2810 At least initially I'll try to implement this so that there is no need
2811 to specify whether the reference is to a table or a source-code block.
2812 That seems to be simpler both in terms of use and implementation.
2814 This is now working for emacs-lisp, ruby and python (and mixtures of
2815 the three) source blocks. See the examples in the [[* (sandbox) referencing other source blocks][sandbox]].
2817 This is currently working only with emacs lisp as in the following
2818 example in the [[* emacs lisp source reference][emacs lisp source reference]].
2821 ** TODO Add languages [11/16]
2822 I'm sure there are many more that aren't listed here. Please add
2823 them, and bubble any that you particularly care about up to the top.
2825 Any new language should be implemented in a org-babel-lang.el file.
2826 Follow the pattern set by [[file:lisp/org-babel-script.el][org-babel-script.el]], [[file:lisp/org-babel-shell.el][org-babel-shell.el]] and
2827 [[file:lisp/org-babel-R.el][org-babel-R.el]].
2838 let fac n = if n == 0 then 1 else n * fac (n - 1)
2850 | 1 | 2 | 3 | 4 | 5 |
2852 **** allow non-interactive evaluation
2854 *** STARTED ocaml [2/3]
2856 - [X] Working for the simple case (no arguments, simple output)
2857 - [X] correct handling of vector/list output
2858 - [ ] ability to import arguments
2865 | n -> fib(n - 1) + fib(n - 2) in
2887 [|"ocaml"; "array"|]
2891 | "ocaml" | "array" |
2894 This could probably be added to [[file:lisp/org-babel-script.el][org-babel-script.el]]
2898 - support for sessions
2899 - add more useful header arguments (user, passwd, database, etc...)
2900 - support for more engines (currently only supports mysql)
2901 - what's a reasonable way to drop table data into SQL?
2903 #+srcname: sql-example
2904 #+begin_src sql :engine mysql
2910 | "information_schema" |
2914 Sass is a very nice extension of CSS, which is much nicer to read and
2915 write (see [[http://sass-lang.com/][sass-lang]]).
2917 #+srcname: sass-example
2918 #+begin_src sass :file stylesheet.css :results file
2928 [[file:stylesheet.css][stylesheet.css]]
2931 trivial [[file:lisp/langs/org-babel-css.el][org-babel-css.el]]
2934 (see [[* file result types][file result types]])
2936 #+srcname: implementing-ditaa
2937 #+begin_src ditaa :results replace :file blue.png :cmdline -r
2947 #+resname: implementing-ditaa
2948 [[file:blue.png][blue.png]]
2950 *** DONE gnuplot [7/7]
2951 (see [[* file result types][file result types]])
2953 #+PLOT: title:"Citas" ind:1 deps:(3) type:2d with:histograms set:"yrange [0:]"
2954 #+TBLNAME: gnuplot-data
2955 | independent var | first dependent var | second dependent var |
2956 |-----------------+---------------------+----------------------|
2957 | 0.1 | 0.425 | 0.375 |
2958 | 0.2 | 0.3125 | 0.3375 |
2959 | 0.3 | 0.24999993 | 0.28333338 |
2960 | 0.4 | 0.275 | 0.28125 |
2961 | 0.5 | 0.26 | 0.27 |
2962 | 0.6 | 0.25833338 | 0.24999993 |
2963 | 0.7 | 0.24642845 | 0.23928553 |
2964 | 0.8 | 0.23125 | 0.2375 |
2965 | 0.9 | 0.23333323 | 0.2333332 |
2966 | 1 | 0.2225 | 0.22 |
2967 | 1.1 | 0.20909075 | 0.22272708 |
2968 | 1.2 | 0.19999998 | 0.21458333 |
2969 | 1.3 | 0.19615368 | 0.21730748 |
2971 #+srcname: implementing-gnuplot
2972 #+begin_src gnuplot :var data=gnuplot-data :results silent
2973 set title "Implementing Gnuplot"
2974 plot data using 1:2 with lines
2977 **** DONE add variables
2978 gnuplot 4.2 and up support user defined variables. This is how
2979 we will handle variables with org-babel (meaning we will need to
2980 require gnuplot 4.2 and up for variable support, which can be
2981 install using [[http://www.macports.org/install.php][macports]] on Mac OSX).
2983 - scalar variables should be replaced in the body of the gnuplot code
2984 - vector variables should be exported to tab-separated files, and
2985 the variable names should be replaced with the path to the files
2987 **** DONE direct plotting w/o session
2988 **** DEFERRED gnuplot support for column/row names
2989 This should be implemented along the lines of the [[* STARTED Column (and row) names of tables in R input/output][R-colname-support]].
2991 We can do something similar to the :labels param in org-plot, we just
2992 have to be careful to ensure that each label is aligned with the
2995 This may be walking too close to an entirely prebuilt plotting tool
2996 rather than straight gnuplot code evaluation. For now I think this
2999 **** DONE a =file= header argument
3000 to specify a file holding the results
3002 #+srcname: gnuplot-to-file-implementation
3003 #+begin_src gnuplot :file plot.png :var data=gnuplot-data
3004 plot data using 1:2, data using 1:3 with lines
3008 [[file:plot.png][plot.png]]
3010 **** DONE helpers from org-plot.el
3011 There are a variety of helpers in org-plot which can be fit nicely
3012 into custom gnuplot header arguments.
3014 These should all be in place by now.
3016 **** DEFERRED header argument specifying 3D data
3019 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
3020 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
3021 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
3022 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 1 | 1 | 0 | 0 |
3023 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
3024 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
3025 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 | 0 |
3026 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
3027 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
3028 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
3029 | 0 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 0 | 0 | 1 | 0 |
3030 | 0 | 0 | 1 | 1 | 0 | 0 | 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 1 | 0 | 0 |
3031 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
3032 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
3034 #+srcname: implementing-gnuplot-grid-plots
3035 #+begin_src gnuplot :vars data=org-grid
3039 **** DONE gnuplot sessions
3040 Working on this, we won't support multiple sessions as `gnuplot-mode'
3041 isn't setup for such things.
3043 Also we can't display results with the default :none session, so for
3044 gnuplot we really want the default behavior to be :default, and to
3045 only run a :none session when explicitly specified.
3047 #+srcname: implementing-gnuplot-sessions
3048 #+begin_src gnuplot :var data=gnuplot-data :session none :file session.png
3049 set title "Implementing Gnuplot Sessions"
3050 plot data using 1:2 with lines
3054 [[file:session.png][session.png]]
3057 (see [[* file result types][file result types]])
3059 #+srcname: implementing-dot-support
3060 #+begin_src dot :file test-dot.png :cmdline -Tpng
3061 digraph data_relationships {
3062 "data_requirement" [shape=Mrecord, label="{DataRequirement|description\lformat\l}"]
3063 "data_product" [shape=Mrecord, label="{DataProduct|name\lversion\lpoc\lformat\l}"]
3064 "data_requirement" -> "data_product"
3069 [[file:test-dot.png][test-dot.png]]
3072 (see [[* file result types][file result types]])
3074 for information on asymptote see http://asymptote.sourceforge.net
3076 #+begin_src asymptote :file asymptote-test.png
3081 real f(real t) {return 1+cos(t);}
3083 path g=polargraph(f,0,2pi,operator ..)--cycle;
3086 xaxis("$x$",above=true);
3087 yaxis("$y$",above=true);
3089 dot("$(a,0)$",(1,0),N);
3090 dot("$(2a,0)$",(2,0),N+E);
3094 [[file:asymptote-test.png][asymptote-test.png]]
3104 ** TODO tangle needs to check that buffer has file name
3105 ** TODO :results value fails with result ending in newline
3107 #+begin_src python :results value
3114 ** PROPOSED (org-babel-merge-params nil) --> ((:tangle . "") (:exports . "") (:results . ""))
3116 ** TODO ensure file output goes to correct destination with relative file paths
3117 E.g. if I use :file pca.png, but the working directory of the R
3118 session is "/tmp", then we need to ensure that the org file link points
3119 to the location of the file created by R.
3120 ** TODO (R) session evaluation fails if no new line terminating output
3122 #+srcname: nonewline
3123 #+begin_src R :session
3124 cat("There's no newline ending this")
3127 ** PROPOSED Do we deal with indented code blocks correctly?
3128 I.e. if there is whitespace before #+begin and #+end.
3129 ** TODO Juan Reyero bug report
3131 > ... However I can't replicate this
3132 > behaviour under linux. I get
3137 > for all three examples.
3139 > I'm using org-version 6.31trans in emacs-version 23.0.91.1 under ubuntu
3140 > jaunty with python 2.6.2. Is this definitely replicable under OSX?
3142 Yes, definitely. I am using emacs version 22.3.1, and python 2.6.1.
3143 I have stripped bare my .emacs, and still:
3145 #+begin_src python :session :results value
3154 ** PROPOSED should we allow :results file without explicitly giving path?
3155 I.e. should we create a file in /tmp or in the current directory?
3156 ** TODO should delete temp files after evaluation
3157 ** TODO tangle is outputting multiple shebang lines with python code
3158 ** TODO o-b-execute-subtree places output outside of folded subtree
3160 Try M-x org-babel-execute-subtree with the subtree folded and
3161 point at the beginning of the heading line. The 7 output doesn't
3162 appear and the 6 output ends up outside the folded heading.
3174 ** TODO point should go to end of comint buffer after execution
3175 ** DEFERRED external shell execution can't isolate return values
3176 I have no idea how to do this as of yet. The result is that when
3177 shell functions are run w/o a session there is no difference between
3178 the =output= and =value= result arguments.
3180 Yea, I don't know how to do this either. I searched extensively on
3181 how to isolate the *last* output of a series of shell commands (see
3182 [[* last command for
3183 shells][last command for shells]]). The results of the search were basically
3184 that it was not possible (or at least not accomplish-able with a
3185 reasonable amount of effort).
3187 That fact combined with the tendancy to always use standard out in
3188 shell scripts led me to treat these two options (=output= and =value=)
3189 as identical in shell evaluation. Not ideal but maybe good enough for
3192 In the `results' branch I've changed this so that they're not quite
3193 identical: output results in raw stdout contents, whereas value
3194 converts it to elisp, perhaps to a table if it looks tabular. This is
3195 the same for the other languages. [Dan]
3196 ** DONE Maintain correct indentation in tangled (e.g.) python code
3197 This is solved by setting new variable org-src-preserve-indentation
3198 to t, either globally or buffer locally.
3211 ***** Do stuff method
3214 print('Doing stuff')
3218 This has invalid indentation
3219 #!/usr/bin/env python
3220 # generated by org-babel-tangle
3222 # [[file:/tmp/z.org::*Class][block-1]]
3225 #!/usr/bin/env python
3226 # generated by org-babel-tangle
3228 # [[file:/tmp/z.org::*Init%20method][block-2]]
3232 #!/usr/bin/env python
3233 # generated by org-babel-tangle
3235 # [[file:/tmp/z.org::*Do%20stuff%20method][block-3]]
3237 print('Doing stuff')
3239 ** DONE prompt characters appearing in output with R
3240 See commit 6947d0ecd2fbcc9e7d1330143ae0a51f2fe347bb which makes a
3241 start on solving this.
3243 org-babel: prevent comint prompt characters from appearing in R output.
3245 This is a hack, but is does fix the problem (on my system at least),
3246 and it does demonstrate where the problem lies. It seems that the
3247 'raw' output often contains lines consisting of multiple concatenated
3248 elements, each element being of the form 'x ' where x is either '>' or
3249 '+'. '+' is inferior-ess-secondary-prompt, and '>' is (related to)
3250 inferior-ess-primary-prompt. The problem can be triggered by leaving
3251 blank lines in the middle of R code, and using :results output.
3254 #+begin_src R :session *R* :results output
3272 ** DONE changed srcname doesn't change resname
3284 **** another sub-header
3286 and finally the results
3290 ** DONE blank srcname takes srcname from next line
3291 Commit 1c7009e28e5c16992b2c219808acff35b8d38c2b
3293 #+begin_src R :session *R*
3299 ** DONE stripping indentation from source-code blocks
3300 This is a problem in [[file:lisp/org-babel-exp.el][org-babel-exp.el]].
3302 ** DONE failing to write srcname to resname when evaluating a named block
3304 #+srcname: please-name-my-result
3305 #+begin_src emacs-lisp
3309 #+resname: please-name-my-result
3312 ** DONE Python session evaluation bug
3313 The following block evaluates correctly with :session none
3314 (set :results to output), but fails with session-based evaluation
3315 (with :results value, as below, you see the error message)
3317 I edebug'ed it and it seems fine until [[file:lisp/langs/org-babel-python.el::comint%20session%20evaluation%20org%20babel%20comint%20in%20buffer%20buffer%20let%20full%20body][we go to comint]].
3319 #+begin_src python :session pysession :results value
3321 from subprocess import *
3326 cc = [('58C','NBS'),
3330 for outdir in ['none', 'noscots', 'lax', 'strict']:
3331 outdir = os.path.join('exclusion-study', outdir)
3332 for case, control in cc:
3333 outfile = os.path.join(outdir, '%s-vs-%s-direct' % (case, control))
3334 cmd = 'snptest %s -frequentist 1 -hwe ' % ('-gen_gz' if format == '.gen.gz' else '')
3335 cmd += '-cases %s %s ' % (case + format, case + '.sample')
3337 cmd += '-controls %s %s ' % (control + format, control + '.sample')
3338 cmd += '-exclude_samples %s ' % os.path.join(outdir, 'exclusions')
3339 cmd += '-o %s ' % outfile
3340 cmd += '-chunk %d ' % chunk
3341 cmd += '> %s' % outfile + '.log'
3348 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls NBS.gen.gz NBS.sample -exclude_samples exclusion-study/none/exclusions -o exclusion-study/none/58C-vs-NBS-direct -chunk 10000 > exclusion-study/none/58C-vs-NBS-direct.log
3349 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/none/exclusions -o exclusion-study/none/58C-vs-POBI-direct -chunk 10000 > exclusion-study/none/58C-vs-POBI-direct.log
3350 snptest -gen_gz -frequentist 1 -hwe -cases NBS.gen.gz NBS.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/none/exclusions -o exclusion-study/none/NBS-vs-POBI-direct -chunk 10000 > exclusion-study/none/NBS-vs-POBI-direct.log
3351 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls NBS.gen.gz NBS.sample -exclude_samples exclusion-study/noscots/exclusions -o exclusion-study/noscots/58C-vs-NBS-direct -chunk 10000 > exclusion-study/noscots/58C-vs-NBS-direct.log
3352 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/noscots/exclusions -o exclusion-study/noscots/58C-vs-POBI-direct -chunk 10000 > exclusion-study/noscots/58C-vs-POBI-direct.log
3353 snptest -gen_gz -frequentist 1 -hwe -cases NBS.gen.gz NBS.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/noscots/exclusions -o exclusion-study/noscots/NBS-vs-POBI-direct -chunk 10000 > exclusion-study/noscots/NBS-vs-POBI-direct.log
3354 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls NBS.gen.gz NBS.sample -exclude_samples exclusion-study/lax/exclusions -o exclusion-study/lax/58C-vs-NBS-direct -chunk 10000 > exclusion-study/lax/58C-vs-NBS-direct.log
3355 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/lax/exclusions -o exclusion-study/lax/58C-vs-POBI-direct -chunk 10000 > exclusion-study/lax/58C-vs-POBI-direct.log
3356 snptest -gen_gz -frequentist 1 -hwe -cases NBS.gen.gz NBS.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/lax/exclusions -o exclusion-study/lax/NBS-vs-POBI-direct -chunk 10000 > exclusion-study/lax/NBS-vs-POBI-direct.log
3357 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls NBS.gen.gz NBS.sample -exclude_samples exclusion-study/strict/exclusions -o exclusion-study/strict/58C-vs-NBS-direct -chunk 10000 > exclusion-study/strict/58C-vs-NBS-direct.log
3358 snptest -gen_gz -frequentist 1 -hwe -cases 58C.gen.gz 58C.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/strict/exclusions -o exclusion-study/strict/58C-vs-POBI-direct -chunk 10000 > exclusion-study/strict/58C-vs-POBI-direct.log
3359 snptest -gen_gz -frequentist 1 -hwe -cases NBS.gen.gz NBS.sample -controls POBI.gen.gz POBI.sample -exclude_samples exclusion-study/strict/exclusions -o exclusion-study/strict/NBS-vs-POBI-direct -chunk 10000 > exclusion-study/strict/NBS-vs-POBI-direct.log
3362 ** DONE require users to explicitly turn on each language
3363 As we continue to add more languages to org-babel, many of which will
3364 require new major-modes we need to re-think how languages are added to
3367 Currently we are requiring all available languages in the
3368 [[file:lisp/org-babel-init.el][org-babel-init.el]] file. I think we need to change this to a user
3369 setting so that only the language which have their requirements met
3370 (in terms of system executables and emacs major modes) are loaded. It
3371 is one more step for install, but it seems to me to be the only
3376 we add something like the following to the instillation instructions
3378 #+begin_src emacs-lisp
3379 ;; Uncomment each of the following require lines if you want org-babel
3380 ;; to support that language. Each language has a comment explaining
3381 ;; it's dependencies. See the related files in lisp/langs for more
3382 ;; detailed explanations of requirements.
3384 ;; (require 'org-babel-R) ;; ess-mode
3385 ;; (require 'org-babel-asymptote) ;; asymptote be installed on your system
3386 ;; (require 'org-babel-css) ;; none
3387 ;; (require 'org-babel-ditaa) ;; ditaa be installed on your system
3388 ;; (require 'org-babel-dot) ;; dot be installed on your system
3389 ;; (require 'org-babel-gnuplot) ;; gnuplot-mode
3390 ;; (require 'org-babel-python) ;; python-mode
3391 ;; (require 'org-babel-ruby) ;; inf-ruby mode, ruby and irb must be installed on your system
3392 ;; (require 'org-babel-sql) ;; none
3395 note that =org-babel-sh=, =org-babel-emacs-lisp= are not included in
3396 the list as they can safely be assumed to work on any system.
3399 we should come up with a way to gracefully degrade when support for a
3400 specific language is missing
3402 > To demonstrate creation of documents, open the "test-export.org" file in
3403 > the base of the org-babel directory, and export it as you would any
3404 > other org-mode file. The "exports" header argument controls how
3405 > source-code blocks are exported, with the following options
3407 > - none :: no part of the source-code block is exported in the document
3408 > - results :: only the output of the evaluated block is exported
3409 > - code :: the code itself is exported
3410 > - both :: both the code and results are exported
3412 I have this error showing up:
3414 executing Ruby source code block
3415 apply: Searching for program: no such file or directory, irb
3416 ** DONE problem with newlines in output when :results value
3418 #+begin_src python :results value
3419 '\n'.join(map(str, range(4)))
3428 Whereas I was hoping for
3435 *Note*: to generate the above you can try using the new =raw= results
3438 #+begin_src python :results value raw
3439 '|'+'|\n|'.join(map(str, range(4)))+'|'
3448 This is now working, it doesn't return as a table because the value
3449 returned is technically a string. To return the table mentioned above
3450 try something like the following.
3453 [[0], [1], [2], [3]]
3462 This is some sort of non-printing char / quoting issue I think. Note
3465 #+begin_src python :results value
3466 '\\n'.join(map(str, range(4)))
3473 #+begin_src python :results output
3474 print('\n'.join(map(str, range(4))))
3483 *** collapsing consecutive newlines in string output
3485 This is an example of the same bug
3487 #+srcname: multi-line-string-output
3488 #+begin_src ruby :results output
3489 "the first line ends here
3492 and this is the second one
3497 This doesn't produce anything at all now. I believe that's because
3498 I've changed things so that :results output really does *not* get the
3499 value of the block, only the STDOUT. So if we add a print statement
3502 #+srcname: multi-line-string-output
3503 #+begin_src ruby :results output
3504 print "the first line ends here
3507 and this is the second one
3513 : the first line ends here
3516 : and this is the second one
3520 However, the behaviour with :results value is wrong
3522 #+srcname: multi-line-string-value
3524 "the first line ends here
3527 and this is the second one
3534 ** DONE o-b-execute-subtree overwrites heading when subtree is folded
3535 I'm not seeing this any more, so am closing this bug.
3537 Try M-x org-babel-execute-subtree with the subtree folded and
3538 point at the beginning of the heading line.
3550 ** DONE non-orgtbl formatted lists
3553 #+srcname: this-doesn't-match-orgtbl
3554 #+begin_src emacs-lisp :results replace
3555 '((:results . "replace"))
3559 | (:results . "replace") |
3561 #+srcname: this-probably-also-wont-work
3562 #+begin_src emacs-lisp :results replace
3569 ** DONE adding blank line when source-block produces no output
3571 #+srcname: show-org-babel-trim
3573 find . \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o -path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path \*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) -prune -o -type f \( -iname \*.el \) -exec grep -i -nH -e org-babel-trim {} \;
3576 ** DONE Allow source blocks to be recognised when #+ are not first characters on the line
3577 I think Carsten has recently altered the core so that #+ can have
3578 preceding whitespace, at least for literal/code examples. org-babel
3579 should support this.
3581 #+srcname: testing-indentation
3582 #+begin_src emacs-lisp :results silent
3583 (message "i'm indented")
3586 #+srcname: testing-non-indentation
3587 #+begin_src emacs-lisp :results silent
3588 (message "I'm not indented")
3591 #+srcname: i-resolve-references-to-the-indented
3592 #+begin_src emacs-lisp :var speech=testing-indentation :results silent
3593 (message "I said %s" speech)
3596 ** DONE are the org-babel-trim s necessary?
3597 at the end of e.g. org-babel-R-evaluate, org-babel-python-evaluate, but
3598 not org-babel-ruby-evaluate
3600 I think it depends on the language, if we find that extra blank
3601 lines are being inserted in a particular language that is a good
3602 indication that the trim or chomp functions may be appropriate.
3604 org-babel-trim and the related org-babel-chomp are use throughout
3607 #+srcname: show-org-babel-trim-usage
3608 #+begin_src sh :results output
3609 find lisp/ \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o -path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path \*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) -prune -o -type f \( -iname \*.el \) -exec grep -i -nH org-babel-trim {} \;
3614 lisp//langs/org-babel-python.el:49: vars "\n") "\n" (org-babel-trim body) "\n")) ;; then the source block body
3615 lisp//langs/org-babel-python.el:143: (org-remove-indentation (org-babel-trim body)) "[\r\n]")))
3616 lisp//langs/org-babel-python.el:158: #'org-babel-trim
3617 lisp//langs/org-babel-python.el:166: (reverse (mapcar #'org-babel-trim raw)))))))
3618 lisp//langs/org-babel-python.el:169: (output (org-babel-trim (mapconcat #'identity (reverse (cdr results)) "\n")))
3619 lisp//langs/org-babel-python.el:170: (value (org-babel-python-table-or-string (org-babel-trim (car results)))))))))
3620 lisp//langs/org-babel-ruby.el:149: (mapcar #'org-babel-trim raw)))))))
3621 lisp//langs/org-babel-sh.el:148: (mapcar #'org-babel-trim raw)))))))
3622 lisp//langs/org-babel-sh.el:151: (output (org-babel-trim (mapconcat #'org-babel-trim (reverse results) "\n")))
3623 lisp//org-babel-ref.el:161: (mapcar #'org-babel-trim (reverse (cons buffer return)))))
3624 lisp//org-babel.el:198: (with-temp-buffer (insert (org-babel-trim body)) (copy-region-as-kill (point-min) (point-max)))
3625 lisp//org-babel.el:465: (org-babel-trim
3626 lisp//org-babel.el:706:(defun org-babel-trim (string &optional regexp)
3629 #+srcname: show-org-babel-chomp-usage
3630 #+begin_src sh :results output
3631 find lisp/ \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o -path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path \*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) -prune -o -type f \( -iname \*.el \) -exec grep -i -nH org-babel-chomp {} \;
3636 lisp//langs/org-babel-R.el:122: (full-body (mapconcat #'org-babel-chomp
3637 lisp//langs/org-babel-R.el:143: (delete nil (mapcar #'extractor (mapcar #'org-babel-chomp raw))) "\n"))))))))
3638 lisp//langs/org-babel-ruby.el:143: #'org-babel-chomp
3639 lisp//langs/org-babel-sh.el:142: (full-body (mapconcat #'org-babel-chomp
3640 lisp//org-babel-tangle.el:163: (insert (format "\n%s\n" (org-babel-chomp body)))
3641 lisp//org-babel.el:362: (org-babel-chomp (match-string 2 arg)))
3642 lisp//org-babel.el:698:(defun org-babel-chomp (string &optional regexp)
3643 lisp//org-babel.el:707: "Like `org-babel-chomp' only it runs on both the front and back of the string"
3644 lisp//org-babel.el:708: (org-babel-chomp (org-babel-reverse-string
3645 lisp//org-babel.el:709: (org-babel-chomp (org-babel-reverse-string string) regexp)) regexp))
3648 ** DONE LoB is not populated on startup
3649 org-babel-library-of-babel is nil for me on startup. I have to
3650 evaluate the [[file:lisp/org-babel-lob.el::][org-babel-lob-ingest]] line manually.
3652 #+tblname: R-plot-example-data
3659 #+lob: R-plot(data=R-plot-example-data)
3663 I've added a section to [[file:lisp/org-babel-init.el][org-babel-init.el]] which will load the
3664 library of babel on startup.
3666 Note that this needs to be done in [[file:lisp/org-babel-init.el][org-babel-init.el]] rather than in
3667 [[file:lisp/org-babel-lob.el][org-babel-lob.el]], not entirely sure why, something about it being
3670 Also, I'm now having the file closed if it wasn't being visited by
3671 a buffer before being loaded.
3673 ** DONE use new merge function [[file:lisp/org-babel-ref.el::t%20nil%20org%20combine%20plists%20args%20nil][here]]?
3674 And at other occurrences of org-combine-plists?
3675 ** DONE creeping blank lines
3676 There's still inappropriate addition of blank lines in some circumstances.
3678 Hmm, it's a bit confusing. It's to do with o-b-remove-result. LoB
3679 removes the entire (#+resname and result) and starts from scratch,
3680 whereas #+begin_src only removes the result. I haven't worked out
3681 what the correct fix is yet. Maybe the right thing to do is to make
3682 sure that those functions (o-b-remove-result et al.) are neutral
3683 with respect to newlines. Sounds easy, but...
3693 Compare the results of
3694 #+lob: adder(a=5, b=17)
3696 #+resname: python-add(a=5, b=17)
3698 --------------------------------
3706 ---------------------
3707 ** DONE #+srcname arg parsing bug
3708 #+srcname: test-zz(arg=adder(a=1, b=1))
3717 #+srcname: test-zz-nasty(arg=adder(a=adder(a=19,b=adder(a=5,b=2)),b=adder(a=adder(a=1,b=9),b=adder(a=1,b=3))))
3722 #+resname: test-zz-nasty
3727 #+srcname: test-zz-hdr-arg
3728 #+begin_src python :var arg=adder(a=adder(a=19,b=adder(a=5,b=2)),b=adder(a=adder(a=1,b=9),b=adder(a=1,b=3)))
3735 ** DONE Fix nested evaluation and default args
3736 The current parser / evaluator fails with greater levels of nested
3737 function block calls (example below).
3739 *** Initial statement [ded]
3740 If we want to overcome this I think we'd have to redesign some of
3741 the evaluation mechanism. Seeing as we are also facing issues like
3742 dealing with default argument values, and seeing as we now know
3743 how we want the library of babel to behave in addition to the
3744 source blocks, now might be a good time to think about this. It
3745 would be nice to do the full thing at some point, but otoh we may
3746 not consider it a massive priority.
3748 AIui, there are two stages: (i) construct a parse tree, and (ii)
3749 evaluate it and return the value at the root. In the parse tree
3750 each node represents an unevaluated value (either a literal value
3751 or a reference). Node v may have descendent nodes, which represent
3752 values upon which node v's evaluation depends. Once that tree is
3753 constructed, then we evaluate the nodes from the tips towards the
3754 root (a post-order traversal).
3756 [This would also provide a solution for concatenating the STDOUTs
3757 of called blocks, which is a [[*allow%20output%20mode%20to%20return%20stdout%20as%20value][task below]]; we concatenate them in
3758 whatever order the traversal is done in.]
3760 In addition to the variable references (i.e. daughter nodes), each
3761 node would contain the information needed to evaluate that node
3762 (e.g. lang body). Then we would pass a function postorder over the
3763 tree which would call o-b-execute-src-block at each node, finally
3764 returning the value at the root.
3766 Fwiw I made a very tentative small start at stubbing this out in
3767 org-babel-call.el in the 'evaluation' branch. And I've made a start
3768 at sketching a parsing algorithm below.
3769 **** Parse tree algorithm
3770 Seeing as we're just trying to parse a string like
3771 f(a=1,b=g(c=2,d=3)) it shouldn't be too hard. But of course there
3772 are 'proper' parsers written in elisp out there,
3773 e.g. [[http://cedet.sourceforge.net/semantic.shtml][Semantic]]. Perhaps we can find what we need -- our syntax is
3774 pretty much the same as python and R isn't it?
3776 Or, a complete hack, but maybe it would be we easy to transform it
3777 to XML and then parse that with some existing tool?
3779 But if we're doing it ourselves, something very vaguely like this?
3780 (I'm sure there're lots of problems with this)
3782 #+srcname: org-babel-call-parse(call)
3784 ## we are currently reading a reference name: the name of the root function
3785 whereami = "refname"
3786 node = root = Node()
3787 for c in call_string:
3790 whereami = "varname" # now we're reading a variable name
3793 node.daughters = [node.daughters, new]
3796 whereami = "refname"
3798 whereami = "varname"
3805 if whereami = "varname":
3806 node.varnames[varnum] += c
3807 elif whereami = "refname":
3811 *** discussion / investigation
3812 I believe that this issue should be addressed as a bug rather than as
3813 a point for new development. The code in [[file:lisp/org-babel-ref.el][org-babel-ref.el]] already
3814 resolves variable references in a recursive manner which *should* work
3815 in the same manner regardless of the depth of the number of nested
3816 function calls. This recursive evaluation has the effect of
3817 implicitly constructing the parse tree that your are thinking of
3818 constructing explicitly.
3820 Through using some of the commented out debugging statements in
3821 [[file:lisp/org-babel-ref.el][org-babel-ref.el]] I have looked at what may be going wrong in the
3822 current evaluation setup, and it seems that nested variables are being
3823 set using the =:var= header argument, and these variables are being
3824 overridden by the *default* variables which are being entered through
3825 the new functional syntax (see the demonstration header below).
3827 I believe that once this bug is fixed we should be back to fully
3828 resolution of nested arguments. We should capture this functionality
3829 in a test to ensure that we continue to test it as we move forward. I
3830 can take a look at implementing this once I get a chance.
3832 Looks like the problem may be in [[file:lisp/org-babel.el::defun%20org%20babel%20merge%20params%20rest%20plists][org-babel-merge-params]], which seems
3833 to be trampling the provided :vars values.
3835 Nope, now it seems that we are actually looking up the results line,
3836 rather than the actual source-code block, which would make sense given
3837 that the results-line will return the same value regardless of the
3838 arguments supplied. See the output of this [[file:lisp/org-babel-ref.el::message%20type%20S%20type%20debugging][debug-statement]].
3840 We need to be sure that we don't read from a =#+resname:= line when we
3841 have a non-nil set of arguments.
3844 After uncommenting the debugging statements located [[file:lisp/org-babel-ref.el::message%20format%20first%20second%20S%20S%20new%20refere%20new%20referent%20debugging][here]] and more
3845 importantly [[file:lisp/org-babel-ref.el::message%20nested%20args%20S%20args%20debugging][here]], we can see that the current reference code does
3846 evaluate the references correctly, and it uses the =:var= header
3847 argument to set =a=8=, however the default variables specified using
3848 the functional syntax in =adder(a=3, b=2)= is overriding this
3851 ***** doesn't work with functional syntax
3853 #+srcname: adder-func(a=3, b=2)
3858 #+resname: adder-func
3861 #+srcname: after-adder-func(arg=adder-func(a=8))
3866 #+resname: after-adder-func
3869 ***** still does work with =:var= syntax
3871 so it looks like regardless of the syntax used we're not overriding
3872 the default argument values.
3874 #+srcname: adder-header
3875 #+begin_src python :var a=3 :var b=2
3879 #+resname: adder-header
3882 #+srcname: after-adder-header
3883 #+begin_src python :var arg=adder-header(a=8, b=0)
3887 #+resname: after-adder-header
3890 *** Set of test cases
3891 **** Both defaults provided in definition
3892 #+srcname: adder1(a=10,b=20)
3899 ****** DONE Rely on defaults
3907 ******* DONE empty parens () not recognised as lob call
3908 E.g. remove spaces between parens above
3910 updated [[file:lisp/org-babel-lob.el::defvar%20org%20babel%20lob%20one%20liner%20regexp%20lob%20t%20n%20n%20t%20n][org-babel-lob-one-liner-regexp]]
3912 ****** DONE One supplied, one default
3915 #+resname: adder1(a=0)
3922 #+resname: adder1(b=0)
3926 ****** DONE Both supplied
3927 #+lob: adder1(a=1,b=2)
3929 #+resname: adder1(a=1,b=2)
3932 **** One arg lacks default in definition
3933 #+srcname: adder2(a=10,b)
3937 ****** DEFERRED Rely on defaults (one of which is missing)
3942 ## should be error: b has no default
3944 Maybe we should let the programming language handle this case. For
3945 example python spits out an error in the =#+lob= line above. Maybe
3946 rather than catching these errors our-selves we should raise an error
3947 when the source-block returns an error. I'll propose a [[* PROPOSED raise elisp error when source-blocks return errors][task]] for this
3948 idea, I'm not sure how/if it would work...
3950 ****** DEFERRED Default over-ridden
3953 See the above [[* DEFERRED Rely on defaults (one of which is missing)][deferred]] and the new proposed [[* PROPOSED raise elisp error when source-blocks return errors][task]], I think it may be
3954 more flexible to allow the source block language to handle the error.
3957 ## should be error: b has no default
3959 ****** DONE Missing default supplied
3962 #+resname: adder2(b=1)
3969 ****** DONE One over-ridden, one supplied
3970 #+lob: adder2(a=1,b=2)
3972 #+resname: adder2(a=1,b=2)
3977 *** Example that fails
3979 #+srcname: adder(a=0, b=99)
3992 #+srcname: level-one-nesting()
3993 #+begin_src python :var arg=adder(a=one(),b=one())
3997 #+resname: level-one-nesting
4001 #+srcname: level-one-nesting()
4002 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()),b=adder(a=one(),b=one()))
4009 *** DONE deeply nested arguments still fails
4011 #+srcname: deeply-nested-args-bug
4012 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()),b=adder(a=one(),b=one()))
4019 **** Used to result in this error
4020 : supplied params=nil
4021 : new-refere="adder", new-referent="a=adder(a=one(),b=one()),b=adder(a=one(),b=one())"
4022 : args=((:var . "a=adder(a=one()") (:var . "b=one())") (:var . "b=adder(a=one()") (:var . "b=one())"))
4024 : supplied params=((:var . "a=adder(a=one()") (:var . "b=one())") (:var . "b=adder(a=one()") (:var . "b=one())"))
4025 : new-refere="adder", new-referent="a=one("
4026 : args=((:var . "a=one("))
4028 : supplied params=((:var . "a=one("))
4029 : reference 'one(' not found in this buffer
4031 Need to change the regexp in [[file:lisp/org-babel-ref.el::assign%20any%20arguments%20to%20pass%20to%20source%20block][org-babel-ref-resolve-reference]] so that
4032 it only matches when the parenthesis are balanced. Maybe look at
4033 [[http://www.gnu.org/software/emacs/elisp/html_node/List-Motion.html][this]].
4035 *** DONE Still some problems with deeply nested arguments and defaults
4037 **** DONE Parsing / defaults bug
4038 Try inserting a space between 'a=0,' and 'b=0' and comparing results
4039 #+srcname: parsing-defaults-bug()
4040 #+begin_src python :var arg=adder(a=adder(a=0,b=0))
4044 #+resname: parsing-defaults-bug
4048 #+srcname: deeply-nested-args-bug-orig()
4049 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()),b=adder(a=adder(a=3, b=4),b=one()))
4053 #+resname: deeply-nested-args-bug-orig
4058 **** DONE Nesting problem II
4059 This generates parsing errors
4061 Fixed: c2bef96b7f644c05be5a38cad6ad1d28723533aa
4063 #+srcname: deeply-nested-args-bug-II-1()
4064 #+begin_src python :var arg=adder(a=adder(a=one(),b=adder(a=2,b=4)))
4068 #+resname: deeply-nested-args-bug-II-1
4072 #+srcname: deeply-nested-args-bug-II-original()
4073 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()),b=adder(a=one(),b=adder(a=1,b=4)))
4077 #+resname: deeply-nested-args-bug-II-original
4082 **** DONE Why does this give 8?
4083 It was picking up the wrong definition of adder
4084 #+srcname: deeply-nested-args-bug-2()
4085 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()))
4089 #+resname: deeply-nested-args-bug-2
4093 **** DONE Problem with empty argument list
4094 This gives empty list with () and 'no output' with ( )
4096 I think this is OK now.
4099 #+begin_src python :var arg=adder( )
4106 #+srcname: deeply-nested-args-bug-orig()
4107 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()),b=adder(a=adder(a=3, b=4),b=one()))
4111 #+resname: deeply-nested-args-bug-orig
4116 **** DONE Nesting problem II
4117 This generates parsing errors
4119 Fixed: c2bef96b7f644c05be5a38cad6ad1d28723533aa
4121 #+srcname: deeply-nested-args-bug-II-1()
4122 #+begin_src python :var arg=adder(a=adder(a=one(),b=adder(a=2,b=4)))
4126 #+resname: deeply-nested-args-bug-II-1
4130 #+srcname: deeply-nested-args-bug-II-original()
4131 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()),b=adder(a=one(),b=adder(a=1,b=4)))
4135 #+resname: deeply-nested-args-bug-II-original
4140 **** DONE Why does this give 8?
4141 It was picking up the wrong definition of adder
4142 #+srcname: deeply-nested-args-bug-2()
4143 #+begin_src python :var arg=adder(a=adder(a=one(),b=one()))
4147 #+resname: deeply-nested-args-bug-2
4151 **** DONE Problem with empty argument list
4152 This gives empty list with () and 'no output' with ( )
4154 I think this is OK now.
4157 #+begin_src python :var arg=adder( )
4190 #+srcname: test-zz(arg=adder(a=adder(a=19,b=adder(a=5,b=2)),b=adder(a=adder(a=1,b=9),b=adder(a=1,b=3))))
4196 *** DONE Arg lacking default
4197 This would be good thing to address soon. I'm imagining that
4198 e.g. here, the 'caller' block would return the answer 30. I believe
4199 there's a few issues here: i.e. the naked 'a' without a reference
4200 is not understood; the default arg b=6 is not understood.
4202 #+srcname: adder-with-arg-lacking-default(a, b=6)
4209 #+srcname: caller(var=adder-with-arg-lacking-default(a=24))
4210 #+begin_src python :results silent
4214 ** DONE allow srcname to omit function call parentheses
4215 Someone needs to revisit those regexps. Is there an argument for
4216 moving some of the regexps used to match function calls into
4217 defvars? (i.e. in o-b.el and o-b-ref.el)
4219 This seems to work now. It still might be a good idea to separate
4220 out some of the key regexps into defvars at some point.
4222 #+srcname: omit-parens-test
4223 #+begin_src ruby :results output
4232 ** DONE avoid stripping whitespace from output when :results output
4233 This may be partly solved by using o-b-chomp rather than o-b-trim
4234 in the o-b-LANG-evaluate functions.
4235 ** DONE function calls in #+srcname: refs
4237 My srcname references don't seem to be working for function
4238 calls. This needs fixing.
4245 srcname function call doesn't work for calling a source block
4246 #+srcname: caller(var1=called())
4259 They do work for a simple reference
4260 #+srcname: caller2(var1=56)
4269 and they do work for :var header arg
4271 #+begin_src python :var var1=called()
4277 ** DONE LoB: with output to buffer, not working in buffers other than library-of-babel.org
4279 I haven't fixed this yet. org-babel-ref-resolve-reference moves
4280 point around, inside a save-excursion. Somehow when it comes to
4281 inserting the results (after possible further recursive calls to
4282 org-babel-ref-resolve-reference), point hasn't gone back to the
4285 #+tblname: test-data
4290 #+lob: R-plot(data=test-data)
4292 #+lob: python-add(a=2, b=9)
4294 #+resname: python-add(a=2, b=9)
4298 I think this got fixed in the bugfixes before merging results into master.
4299 ** DONE cursor movement when evaluating source blocks
4300 E.g. the pie chart example. Despite the save-window-excursion in
4301 org-babel-execute:R. (I never learned how to do this properly: org-R
4302 jumps all over the place...)
4304 I don't see this now [ded]
4306 ** DONE LoB: calls fail if reference has single character name
4307 commit 21d058869df1ff23f4f8cc26f63045ac9c0190e2
4308 **** This doesn't work
4309 #+lob: R-plot(data=X)
4328 #+lob: R-plot(data=XX)
4329 ** DONE make :results replace the default?
4330 I'm tending to think that appending results to pre-existing results
4331 creates mess, and that the cleaner `replace' option should be the
4332 default. E.g. when a source block creates an image, we would want
4333 that to be updated, rather than have a new one be added.
4337 ** DONE ruby evaluation not working under ubuntu emacs 23
4338 With emacs 23.0.91.1 on ubuntu, for C-h f run-ruby I have the
4339 following, which seems to conflict with [[file:lisp/langs/org-babel-ruby.el::let%20session%20buffer%20save%20window%20excursion%20run%20ruby%20nil%20session%20current%20buffer][this line]] in org-babel-ruby.el.
4342 run-ruby is an interactive compiled Lisp function.
4346 Run an inferior Ruby process, input and output via buffer *ruby*.
4347 If there is a process already running in `*ruby*', switch to that buffer.
4348 With argument, allows you to edit the command line (default is value
4349 of `ruby-program-name'). Runs the hooks `inferior-ruby-mode-hook'
4350 (after the `comint-mode-hook' is run).
4351 (Type C-h m in the process buffer for a list of commands.)
4354 So, I may have a non-standard inf-ruby.el. Here's my version of
4358 run-ruby is an interactive Lisp function in `inf-ruby.el'.
4360 (run-ruby &optional COMMAND NAME)
4362 Run an inferior Ruby process, input and output via buffer *ruby*.
4363 If there is a process already running in `*ruby*', switch to that buffer.
4364 With argument, allows you to edit the command line (default is value
4365 of `ruby-program-name'). Runs the hooks `inferior-ruby-mode-hook'
4366 (after the `comint-mode-hook' is run).
4367 (Type C-h m in the process buffer for a list of commands.)
4370 It seems we could either bundle my version of inf-ruby.el (as it's
4371 the newest). Or we could change the use of `run-ruby' so that it
4372 is robust across multiple distributions. I think I'd prefer the
4373 former, unless the older version of inf-ruby is actually bundled
4374 with emacs, in which case maybe we should go out of our way to
4375 support it. Thoughts?
4377 I think for now I'll just include the latest [[file:util/inf-ruby.el][inf-ruby.el]] in the
4378 newly created utility directory. I doubt anyone would have a
4379 problem using the latest version of this file.
4380 ** DONE test failing forcing vector results with =test-forced-vector-results= ruby code block
4381 Note that this only seems to happen the *second* time the test table
4384 #+srcname: bug-trivial-vector
4385 #+begin_src emacs-lisp :results vector silent
4389 #+srcname: bug-forced-vector-results
4390 #+begin_src ruby :var triv=test-trivial-vector :results silent
4394 mysteriously this seems to be fixed...
4395 ** DONE defunct R sessions
4396 Sometimes an old R session will turn defunct, and newly inserted code
4397 will not be evaluated (leading to a hang).
4399 This seems to be fixed by using `inferior-ess-send-input' rather than `comint-send-input'.
4400 ** DONE ruby fails on first call to non-default session
4402 #+srcname: bug-new-session
4403 #+begin_src ruby :session is-new
4407 ** DONE when reading results from =#+resname= line
4409 Errors when trying to read from resname lines.
4411 #+resname: bug-in-resname
4414 #+srcname: bug-in-resname-reader
4415 #+begin_src emacs-lisp :var buggy=bug-in-resname() :results silent
4419 ** DONE R-code broke on "org-babel" rename
4421 #+srcname: bug-R-babels
4426 ** DONE error on trivial R results
4428 So I know it's generally not a good idea to squash error without
4429 handling them, but in this case the error almost always means that
4430 there was no file contents to be read by =org-table-import=, so I
4433 #+srcname: bug-trivial-r1
4434 #+begin_src R :results replace
4435 pie(c(1, 2, 3), labels = c(1, 2, 3))
4438 #+srcname: bug-trivial-r2
4439 #+begin_src R :results replace
4443 #+resname: bug-trivial-r2
4446 #+srcname: bug-trivial-r3
4447 #+begin_src R :results replace
4451 #+resname: bug-trivial-r3
4456 ** DONE ruby new variable creation (multi-line ruby blocks)
4457 Actually it looks like we were dropping all but the last line.
4459 #+srcname: multi-line-ruby-test
4460 #+begin_src ruby :var table=bug-numerical-table :results replace
4462 table.each{|n| total += n}
4469 ** DONE R code execution seems to choke on certain inputs
4470 Currently the R code seems to work on vertical (but not landscape)
4473 #+srcname: little-fake
4474 #+begin_src emacs-lisp
4479 #+begin_src R :var num=little-fake
4486 #+srcname: set-debug-on-error
4487 #+begin_src emacs-lisp :results silent
4488 (setq debug-on-error t)
4491 #+srcname: bug-numerical-table
4492 #+begin_src emacs-lisp :results silent
4499 #+srcname: bug-R-number-evaluation
4500 #+begin_src R :var table=bug-numerical-table
4509 #+tblname: bug-vert-table
4514 #+srcname: bug-R-vertical-table
4515 #+begin_src R :var table=bug-vert-table :results silent
4519 ** DONE org bug/request: prevent certain org behaviour within code blocks
4520 E.g. [[]] gets recognised as a link (when there's text inside the
4521 brackets). This is bad for R code at least, and more generally
4522 could be argued to be inappropriate. Is it difficult to get org to
4523 ignore text in code blocks? [DED]
4525 I believe Carsten addressed this recently on the mailing list with
4526 the comment that it was indeed a difficult issue. I believe this
4527 may be one area where we could wait for an upstream (org-mode) fix.
4529 [Dan] Carsten has fixed this now in the core.
4531 ** DONE with :results replace, non-table output doesn't replace table output
4532 And vice versa. E.g. Try this first with table and then with len(table) [DED]
4533 #+begin_src python :var table=sandbox :results replace
4538 | 4 | "schulte" | 6 |
4541 Yes, this is certainly a problem. I fear that if we begin replacing
4542 anything immediately following a source block (regardless of whether
4543 it matches the type of our current results) we may accidentally delete
4544 hand written portions of the user's org-mode buffer.
4546 I think that the best solution here would be to actually start
4547 labeling results with a line that looks something like...
4551 This would have a couple of benefits...
4552 1) we wouldn't have to worry about possibly deleting non-results
4553 (which is currently an issue)
4554 2) we could reliably replace results even if there are different types
4555 3) we could reference the results of a source-code block in variable
4556 definitions, which would be useful if for example we don't wish to
4557 re-run a source-block every time because it is long-running.
4559 Thoughts? If no-one objects, I believe I will implement the labeling
4562 ** DONE extra quotes for nested string
4563 Well R appears to be reading the tables without issue...
4565 these *should* be quoted
4567 #+begin_src sh :results replace
4572 | "README.markdown" |
4575 | "existing_tools" |
4579 | "test-export.html" |
4580 | "test-export.org" |
4582 #+srcname: test-quotes
4583 #+begin_src ruby :var tab=ls
4589 #+srcname: test-quotes
4590 #+begin_src R :var tab=ls
4596 ** DONE simple ruby arrays not working
4598 As an example eval the following. Adding a line to test
4600 #+tblname: simple-ruby-array
4603 #+srcname: ruby-array-test
4604 #+begin_src ruby :var ar = simple-ruby-array :results silent
4608 ** DONE space trailing language name
4609 fix regexp so it works when there's a space trailing the language name
4611 #+srcname: test-trailing-space
4616 ** DONE Args out of range error
4618 The following block resulted in the error below [DED]. It ran without
4619 error directly in the shell.
4622 for platf in ill aff ; do
4623 for pop in CEU YRI ASI ; do
4624 rm -f $platf/hapmap-genos-$pop-all $platf/hapmap-rs-all
4625 cat $platf/hapmap-genos-$pop-* > $platf/hapmap-genos-$pop-all
4626 cat $platf/hapmap-rs-* > $platf/hapmap-rs-all
4631 executing source block with sh...
4632 finished executing source block
4633 string-equal: Args out of range: "", -1, 0
4635 the error =string-equal: Args out of range: "", -1, 0= looks like what
4636 used to be output when the block returned an empty results string.
4637 This should be fixed in the current version, you should now see the
4638 following message =no result returned by source block=.
4640 ** DONE ruby arrays not recognized as such
4642 Something is wrong in [[file:lisp/org-babel-script.el]] related to the
4643 recognition of ruby arrays as such.
4645 #+begin_src ruby :results replace
4651 #+begin_src python :results replace
4656 ** DEFERRED weird escaped characters in shell prompt break shell evaluation
4657 E.g. this doesn't work. Should the shell sessions set a sane prompt
4658 when they start up? Or is it a question of altering
4659 comint-prompt-regexp? Or altering org-babel regexps?
4662 black=30 ; red=31 ; green=32 ; yellow=33 ; blue=34 ; magenta=35 ; cyan=36 ; white=37
4665 export PS1="\[\033[${prompt_col}m\]\w${prompt_char} \[\033[0m\]"
4668 I just pushed a good amount of changes, could you see if your shell
4669 problems still exist?
4671 The problem's still there. Specifically, aIui, at [[file:lisp/langs/org-babel-sh.el::raw%20org%20babel%20comint%20with%20output%20buffer%20org%20babel%20sh%20eoe%20output%20nil%20insert%20full%20body%20comint%20send%20input%20nil%20t][this line]] of
4672 org-babel-sh.el, raw gets the value
4674 ("" "
\e[0m Sun Jun 14 19:26:24 EDT 2009\n" "
\e[0m org_babel_sh_eoe\n" "
\e[0m ")
4676 and therefore (member org-babel-sh-eoe-output ...) fails
4678 I think that `comint-prompt-regexp' needs to be altered to match
4679 the shell prompt. This shouldn't be too difficult to do by hand,
4680 using the `regexp-builder' command and should probably be part of
4681 the user's regular emacs init. I can't think of a way for us to
4682 set this automatically, and we are SOL without a regexp to match
4684 ** REJECTED elisp reference fails for literal number
4685 That's a bug in Dan's elisp, not in org-babel.
4686 #+srcname: elisp-test(a=4)
4687 #+begin_src emacs-lisp
4695 Evaluate all the cells in this table for a comprehensive test of the
4696 org-babel functionality.
4698 *Note*: if you have customized =org-babel-default-header-args= then some
4699 of these tests may fail.
4701 #+TBLNAME: org-babel-tests
4702 | functionality | block | arg | expected | results | pass |
4703 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4704 | basic evaluation | | | | | pass |
4705 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4706 | emacs lisp | basic-elisp | | 5 | 5 | pass |
4707 | shell | basic-shell | | 6 | 6 | pass |
4708 | ruby | basic-ruby | | org-babel | org-babel | pass |
4709 | python | basic-python | | hello world | hello world | pass |
4710 | R | basic-R | | 13 | 13 | pass |
4711 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4712 | tables | | | | | pass |
4713 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4714 | emacs lisp | table-elisp | | 3 | 3 | pass |
4715 | ruby | table-ruby | | 1-2-3 | 1-2-3 | pass |
4716 | python | table-python | | 5 | 5 | pass |
4717 | R | table-R | | 3.5 | 3.5 | pass |
4718 | R: col names in R | table-R-colnames | | -3 | -3 | pass |
4719 | R: col names in org | table-R-colnames-org | | 169 | 169 | pass |
4720 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4721 | source block references | | | | | pass |
4722 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4723 | all languages | chained-ref-last | | Array | Array | pass |
4724 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4725 | source block functions | | | | | pass |
4726 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4727 | emacs lisp | defun-fibb | | fibbd | fibbd | pass |
4728 | run over | Fibonacci | 0 | 1 | 1 | pass |
4729 | a | Fibonacci | 1 | 1 | 1 | pass |
4730 | variety | Fibonacci | 2 | 2 | 2 | pass |
4731 | of | Fibonacci | 3 | 3 | 3 | pass |
4732 | different | Fibonacci | 4 | 5 | 5 | pass |
4733 | arguments | Fibonacci | 5 | 8 | 8 | pass |
4734 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4735 | bugs and tasks | | | | | pass |
4736 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4737 | simple ruby arrays | ruby-array-test | | 3 | 3 | pass |
4738 | R number evaluation | bug-R-number-evaluation | | 2 | 2 | pass |
4739 | multi-line ruby blocks | multi-line-ruby-test | | 2 | 2 | pass |
4740 | forcing vector results | test-forced-vector-results | | Array | Array | pass |
4741 | deeply nested arguments | deeply-nested-args-bug | | 4 | 4 | pass |
4742 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4743 | sessions | | | | | pass |
4744 |-------------------------+----------------------------+-----+-------------+-------------+----------------------------------|
4745 | set ruby session | set-ruby-session-var | | :set | #ERROR | expected ":set" but was "#ERROR" |
4746 | get from ruby session | get-ruby-session-var | | 3 | #ERROR | expected "3" but was "#ERROR" |
4747 | set python session | set-python-session-var | | set | set | pass |
4748 | get from python session | get-python-session-var | | 4 | 4 | pass |
4749 | set R session | set-R-session-var | | set | set | pass |
4750 | get from R session | get-R-session-var | | 5 | 5 | pass |
4751 #+TBLFM: $5='(if (= (length $3) 1) (progn (message (format "running %S" '(sbe $2 (n $3)))) (sbe $2 (n $3))) (sbe $2))::$6='(if (string= $4 $5) "pass" (format "expected %S but was %S" $4 $5))
4752 #+TBLFM: $5=""::$6=""
4754 The second TBLFM line (followed by replacing '[]' with '') can be used
4755 to blank out the table results, in the absence of a better method.
4759 #+srcname: basic-elisp
4760 #+begin_src emacs-lisp :results silent
4765 #+srcname: basic-shell
4766 #+begin_src sh :results silent
4771 #+srcname: date-simple
4772 #+begin_src sh :results silent
4776 #+srcname: basic-ruby
4777 #+begin_src ruby :results silent
4782 #+srcname: basic-python
4783 #+begin_src python :results silent
4789 #+begin_src R :results silent
4797 #+tblname: test-table
4801 #+tblname: test-table-colnames
4802 | var1 | var2 | var3 |
4803 |------+------+------|
4807 #+srcname: table-elisp
4808 #+begin_src emacs-lisp :results silent :var table=test-table
4809 (length (car table))
4813 #+srcname: table-ruby
4814 #+begin_src ruby :results silent :var table=test-table
4815 table.first.join("-")
4819 #+srcname: table-python
4820 #+begin_src python :var table=test-table
4824 #+srcname: table-R(table=test-table)
4829 #+srcname: table-R-colnames(table=test-table-colnames)
4830 #+begin_src R :results silent
4831 sum(table$var2 - table$var3)
4834 #+srcname: R-square(x=default-name-doesnt-exist)
4835 #+begin_src R :colnames t
4839 This should return 169. The fact that R is able to use the column name
4840 to index the data frame (x$var3) proves that a table with column names
4841 (a header row) has been recognised as input for the R-square function
4842 block, and that the R-square block has output an elisp table with
4843 column names, and that the colnames have again been recognised when
4844 creating the R variables in this block.
4845 #+srcname: table-R-colnames-org(x = R-square(x=test-table-colnames))
4855 Lets pass a references through all of our languages...
4857 Lets start by reversing the table from the previous examples
4859 #+srcname: chained-ref-first
4860 #+begin_src python :var table = test-table
4865 #+resname: chained-ref-first
4869 Take the first part of the list
4871 #+srcname: chained-ref-second
4872 #+begin_src R :var table = chained-ref-first
4876 #+resname: chained-ref-second
4880 Turn the numbers into string
4882 #+srcname: chained-ref-third
4883 #+begin_src emacs-lisp :var table = chained-ref-second
4884 (mapcar (lambda (el) (format "%S" el)) table)
4887 #+resname: chained-ref-third
4890 and Check that it is still a list
4892 #+srcname: chained-ref-last
4893 #+begin_src ruby :var table=chained-ref-third
4898 ** source blocks as functions
4900 #+srcname: defun-fibb
4901 #+begin_src emacs-lisp :results silent
4902 (defun fibbd (n) (if (< n 2) 1 (+ (fibbd (- n 1)) (fibbd (- n 2)))))
4906 #+srcname: fibonacci
4907 #+begin_src emacs-lisp :results silent :var n=7
4917 ** sbe tests (these don't seem to be working...)
4918 Testing the insertion of results into org-mode tables.
4920 #+srcname: multi-line-output
4921 #+begin_src ruby :results replace
4922 "the first line ends here
4925 and this is the second one
4931 : the first line ends here\n\n\n and this is the second one\n\neven a third
4933 #+srcname: multi-line-error
4934 #+begin_src ruby :results replace
4935 raise "oh nooooooooooo"
4941 | the first line ends here... | -:5: warning: parenthesize argument(s) for future version... |
4942 #+TBLFM: $1='(sbe "multi-line-output")::$2='(sbe "multi-line-error")
4944 ** forcing results types tests
4946 #+srcname: test-trivial-vector
4947 #+begin_src emacs-lisp :results vector silent
4951 #+srcname: test-forced-vector-results
4952 #+begin_src ruby :var triv=test-trivial-vector :results silent
4958 #+srcname: set-ruby-session-var
4959 #+begin_src ruby :session :results silent
4964 #+srcname: get-ruby-session-var
4965 #+begin_src ruby :session :results silent
4969 #+srcname: set-python-session-var
4970 #+begin_src python :session
4975 #+srcname: get-python-session-var
4976 #+begin_src python :session
4980 #+srcname: set-R-session-var
4981 #+begin_src R :session
4986 #+srcname: get-R-session-var
4987 #+begin_src R :session