2 #+TITLE: Org-mode Testing
3 #+PROPERTY: results silent
7 The only dependency is [[http://www.emacswiki.org/emacs/ErtTestLibrary][ERT]] the Emacs testing library which ships with
8 Emacs24. If you are running an older version of Emacs and don't
9 already have ERT installed it can be installed from its old [[https://github.com/ohler/ert][git
12 * Non-interactive batch testing from the command line
14 The simplest way to run the Org-mode test suite is from the command
15 line with the following invocation. Note that the paths below are
16 relative to the base of the Org-mode directory.
18 #+BEGIN_SRC sh :dir (expand-file-name "..")
19 # For Emacs earlier than 24, add -L /path/to/ert
21 -L lisp/ -L testing/ -L testing/lisp -l lisp/org.el \
22 -l lisp/org-id.el -l testing/org-test.el \
23 --eval "(progn (org-reload) (setq org-confirm-babel-evaluate nil))" \
24 -f org-test-run-batch-tests
27 The options in the above command are explained below.
29 | -Q | ignores any personal configuration ensuring a vanilla Emacs instance is used |
30 | --batch | runs Emacs in "batch" mode with no gui and termination after execution |
31 | -l | loads Org-mode and the org mode test suite defined in testing/org-test.el |
32 | --eval | reloads Org-mode and allows evaluation of code blocks by the tests |
33 | -f | actually runs the tests using the `org-test-run-batch-tests' function |
35 * Interactive testing from within Emacs
37 To run the Org-mode test suite from a current Emacs instance simply
38 load and run the test suite with the following commands.
40 1) First load the test suite.
41 #+BEGIN_SRC emacs-lisp :var here=(buffer-file-name)
42 (add-to-list 'load-path (file-name-directory here))
46 2) Then run the test suite,
47 #+BEGIN_SRC emacs-lisp
48 (org-test-run-all-tests)
51 or when a test fails run it interactively and investigate the
52 problem in the ERT results buffer.
55 Use this as a demo example of a failing test
56 #+BEGIN_SRC emacs-lisp
57 (ert-deftest test-org/org-link-escape-ascii-character-demo-of-fail ()
58 (should (string= "%5B" ;; expected is right
59 (org-link-escape "[")))
60 (should (string= "%5C" ;; expected is wrong, "%5D" would be right
61 (org-link-escape "]"))))
63 or evaluate the ert-deftest form of the test you want to run. Then
64 "M-x ert RET test-org/org-link-escape-ascii-character-demo-of-fail RET"
65 When not visible yet switch to the ERT results buffer named
66 "\*ert\*". When a test failed the ERT results buffer shows the
67 details of the first "should" that failed. See
68 (info "(ert)Running Tests Interactively") on how to re-run, start
71 How to run all tests of a single test file:
72 "M-x ert-delete-all-tests RET", confirm. Open the file
73 ./lisp/test-*.el, "M-x eval-buffer RET", "M-x ert RET t RET"
75 Consider to set pp-escape-newlines nil before running the test when
76 looking at "should" in the ERT results buffer. Especially when
77 using "l" to look at passed test results and possibly missing an
78 appropriate setting of pp-escape-newlines made only temporarily for
79 the running time of the test as e. g. tests using
80 org-test-table-target-expect-tblfm do.
84 - If the value of the =org-babel-no-eval-on-ctrl-c-ctrl-c= is non-nil
85 then it will result in some test failure, as there are tests which
86 rely on this behavior.