1 \input texinfo @c -*-texinfo-*-
3 @setfilename etest.info
4 @settitle Emacs Testing Framework
8 * ETest: (etest). The Emacs Testing Framework
12 Copyright @copyright{} 2008 Philip Jackson.
14 Permission is granted to copy, distribute and/or modify this document
15 under the terms of the GNU Free Documentation License, Version 1.1 or
16 any later version published by the Free Software Foundation; with no
17 Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A
18 copy of the license is included in the section entitled "GNU Free
19 Documentation License".
24 @title ETest - The Emacs Testing Framework
27 @vskip 0pt plus 1filll
36 @node Top, Introduction, (dir), (dir)
40 * Introduction:: Introduction to ETest.
41 * Fetching ETest:: How to download ETest.
42 * Installation:: How to get emacs to wear ETest.
43 * Usage:: How to use ETest.
44 * The results buffer:: Display of test results.
45 * The Tests:: Functions available for testing.
49 @node Introduction, Fetching ETest, Top, Top
52 ETest (or etest, if you like) is the Emacs Testing Framework. It is a
53 small modular system for writing unit tests in Emacs Lisp.
55 At time of writing ETest consists of two files. @file{etest.el}
56 provides the core test functionality. It defines the function
57 (actually, macro) @code{etest} which is usually one's entrance into a
58 test run. The file @file{etest-result-mode.el} adds functions that
59 allow the visualisation of a run with syntax highlighting, folding and
62 @node Fetching ETest, Installation, Introduction, Top
63 @chapter Fetching ETest
65 At the time of writing you must download ETest using git:
67 @code{git clone http://repo.or.cz/r/ETest.git etest}
69 Though it will always be bleeding edge @code{master} should always be
70 in a @emph{working} state.
72 @node Installation, Usage, Fetching ETest, Top
75 To install ETest use one of the following methods:
77 @section Compiling ETest
79 Once you've fetched ETest change into the new directory and run
80 @code{make} if all goes well run @code{make install} (you may need
81 escalated privileges for this step).
83 @section Manual Installation
85 If you don't want to byte-compile for some reason then you can just
86 copy the @code{.el} files into a directory in your @var{load-path}.
88 If you would rather add the unpacked library @emph{to} your load path
89 then the following will work assuming the etest directory is in your
93 (add-to-list 'load-path "~/etest")
96 If you want to run tests from a lisp buffer add the following to your
100 ;; The only keybinding we make... hopefully
101 (add-hook 'emacs-lisp-mode-hook
103 (local-set-key (kbd "C-c t") 'etest-execute)))
106 And to have @code{.etest} files load @code{emacs-lisp-mode} put the
107 following into your initialistation file:
110 ;; Load lisp mode when editing an etest file
111 (add-to-list 'auto-mode-alist '("\\.etest$" . emacs-lisp-mode))
114 @node Usage, The results buffer, Installation, Top
117 First you must evaluate @code{(require 'etest)} then you can start
118 using the @code{etest} macro.
120 Using ETest is very simple. Once you have installed the modules then
121 you can simply run your first test like this:
128 This should pop up a results buffer showing you the outcome of the
129 run. In this case all should be ok because, well, 1 is a pass
130 according to the @code{ok} test.
133 * Example working practice:: How one might use etest.
136 @node Example working practice, , Usage, Usage
137 @section Example working practice
139 Generally (at least the way I work) is to have a @code{.etest} file
140 which matches all but the extension of the filename of the file I am
141 to test. So if I were testing a file called @code{find-cmd.el} I would
142 write my tests in a file called @code{find-cmd.etest} (in the same
143 directory as @code{find-cmd.el}) and then when I hit @kbd{C-c t} (in
144 @code{emacs-lisp-mode}) the run would happen and the results pop up in
145 their normal fashion.
147 Thanks to @file{etest-execute.el} there are several options for the
148 location of an etest file (one at a time). In order of execution they
151 1. The buffer local variable @code{etest-file} is set to the filename
154 2. A matching etest file is in the directories listed in
155 @code{etest-load-path}.
157 3. A matching etest file is in the current working directory
158 (@code{default-directory}).
160 4. A matching etest file is in one of the directories mentioned in
163 At the moment it's impossible to have two etest structures defined in
164 one file @emph{and} see the results in one results buffer (at least in
165 the result mode that is bundled with ETest) as an @code{erase-buffer}
166 is executed (effectively) for each invocation of the @code{etest}
169 @node The results buffer, The Tests, Usage, Top
170 @chapter The results buffer
172 The results buffer is where you can see (and manipulate) the results
173 of a test run in a human friendly format. It will always popup when
174 etest is run and let you know how things went.
177 * Example output:: What the output looks like.
178 * Bindings:: What commands you have at your disposal.
181 @node Example output, Bindings, The results buffer, The results buffer
182 @section Example output
184 Given the hypothetical tests:
188 ("Numeric comparisons"
190 (ok (> 1 0) "one is more than 0")
191 (ok (< 1 0) "one is less than 0"))
194 (ok (< 1 -5.2) "one is less than 5.2"))))
198 Once run give us the following in the results buffer:
201 * Numeric comparisons
203 ok ................ one is more than 0
205 not ok ............ one is less than 0
208 ok ................ (ok (> 10 9.99))
210 not ok ............ one is less than 5.2
215 All headings are foldable as are comments.
217 @node Bindings, , Example output, The results buffer
223 Bury this results buffer.
225 @findex etest-rm-etest-rm-cycle-comments
226 Shift the values in @code{etest-rm-comment-visibility-types} and use
227 the @code{car} of that list to determine the visibility of comments.
229 @findex etest-rm-toggle-headline
230 Toggle the visibility of a heading or test comment.
233 @node The Tests, , The results buffer, Top
236 Tests are always run within the @code{etest} form and usually always
237 evaluate their arguments. Tests will always have a defined number of
238 required arguments (for example @code{ok} requires one argument. Each
239 test also allows for one optional argument which is a custom
240 documentation string. If this argument is omitted then ETest will
241 generate one in its place. So, for example, if you used
242 @code{(etest (ok 1))} the doc string would be @code{"(ok 1)"} if you used
243 @code{(etest (ok 1 "Foo"))} the doc string would be @code{"Foo"}.
246 * Test Structure:: Basic structure of tests.
247 * Builtin Simple Tests:: Boolean checks.
248 * Builtin Equality Tests:: Are two things similar?
249 * Builtin Error Tests:: Test exception handling.
250 * Builtin String Tests:: Check a string matches an RE.
251 * Defining your own tests:: Extend ETest.
252 * Leaving tests until later:: Using todo and skip.
255 @node Test Structure, Builtin Simple Tests, The Tests, The Tests
256 @section Test Structure
258 Tests can be grouped within headings by simply using a string as the
259 first element of a form like this:
268 You can nest headings to your hearts content.
270 This will produce a set of results that follow the same hierarchical
271 pattern as the tests themselves. For example the above produces the
276 (:result t :comments "got: '1'" :doc "(ok 1)")))
280 The output in the results buffer is:
284 ok ................. (ok 1)
288 @node Builtin Simple Tests, Builtin Equality Tests, Test Structure, The Tests
289 @section Builtin Simple Tests
291 These basic tests allow you, basically, to check if a value is either
296 @code{ok} will only pass if its argument produces a non-nil result.
305 @code{null} will only pass if its argument produces a nil result.
312 @node Builtin Equality Tests, Builtin Error Tests, Builtin Simple Tests, The Tests
313 @section Builtin Equality Tests
315 The following functions map to their lisp counterparts and so don't
316 really require much explanation. Evaluate each of the examples to
319 Each take two forms which, post evaluation, are the objects to
332 (etest (eql 1.1 1.1))
339 (etest (equal '(1 2) '(1 2)))
343 @node Builtin Error Tests, Builtin String Tests, Builtin Equality Tests, The Tests
344 @section Builtin Error Tests
346 These two functions each take one form.
350 This test will pass if an exception is raised. For example, cause a
351 divide by zero error (@code{(arith-error)}):
354 (etest (error (/ 1 0)))
360 This test will pass if no exception is raised. For example, a valid
361 division will not raise an error:
364 (etest (noerror (/ 0 1)))
368 @node Builtin String Tests, Defining your own tests, Builtin Error Tests, The Tests
369 @section Builtin String Tests
373 @code{like} takes two arguments a string and a regexp to test it against:
376 (etest (like "Hello" "^H\\(e\\)"))
380 Produces this in the results buffer:
383 ok .................. (like "Hello" "^H\\(e\\)")
389 The grouping within the regular expression only affects the comments.
391 @node Defining your own tests, Leaving tests until later, Builtin String Tests, The Tests
392 @section Defining your own tests
394 Defining your own tests is fairly trivial and where ETest becomes
397 Each test must return a plist that has @code{:result} and, optionally,
398 @code{:comments} in it.
400 @code{:result} represents whether the test passed or failed non-nil
401 for a pass and nil for a fail.
403 The optional @code{:comments} are newline separated strings that might
404 help the user in their diagnosis of a problem. Comments should follow
405 the conventions set by the 'builtin' tests using keywords such as
408 To have ETest recognise the test as valid the @code{deftest} function
409 should be used. For example if I wanted to create a function that
410 took two arguments and tested the first was numerically greater than
411 the other I might do this:
414 (defun etest-greater-than (one two)
415 (let* ((res (> one two)))
417 :comments (unless res
418 (format "one: '%d'\ntwo: '%d'" one two)))))
422 We let emacs take care of type errors (and any other type of error)
423 which is what all of the builtins do. Also it's worth noting that if
424 you want your arguments evaling you have to do it yourself.
426 Now we let ETest know this new function exists:
429 (deftest '(> 2) 'etest-greater-than)
433 So, the new function will be called @code{>}, it will take two arguments
434 and it calls maps to @code{etest-greater-than}.
437 (deftest '(> 2) 'etest-greater-than)
441 Now you can mix @code{>} with other tests:
446 (> 1 2 "one is more than two"))
450 Which in the results buffer produces:
453 ok .................. (ok "something")
455 not ok .............. one is more than two
461 @node Leaving tests until later, , Defining your own tests, The Tests
462 @section Leaving tests until later
464 @subsection Todo tests
466 If you wrap a test in a todo keyword:
472 Then the test will appear to pass but with an extra keyword in the
473 result (@code{:todo}) with the value @code{t}.
475 The result mode takes account of this by highlighting todoed tests and
476 showing you what happened when ETest actually ran the test (catching