4 The point of the testing library (available by sourcing the
5 test-lib.sh file) is to assist with rapidly writing robust
6 tests that produce TAP-compliant output. (For a quick primer
7 on TAP see the README file in the section "TAP - A Quick Overview".)
10 ----------------------------
11 Why Use the Testing Library?
12 ----------------------------
16 * hopefully helps to allow tests to be written more quickly
17 * catches common failures (signal exits, not found etc.)
18 * avoids reproducing commonly needed code in every test
19 * prevents unwanted "other" lines from ending up in the TAP output
20 * allows test results to be aggregated without using "prove"
21 * supports easier test debugging
28 Check out the `README` file for a quick overview of everything in the test
29 directory and a brief overview of TAP.
31 This file, `README-TESTLIB`, serves as a reference of available variables,
32 options and functions in the testing library that can be used by test writers
33 to create tests. This file does not, however, provide any "how-to" instruction
34 as that's reserved for the `README-WRITING-TESTS` file.
36 The `README-WRITING-TESTS` file describes how to quickly get started writing
37 tests using the testing library and should be perused by anyone who finds the
38 simplistic example in the `README` file to be insufficient. It does not,
39 however, serve as a comprehensive reference for the testing library.
46 Note there is a difference between the way test scripts passed to the various
47 test_expect/test_tolerate functions behave compared to Git's testing library.
48 In this testing library the scripts passed to those functions are evaluated
49 inside a subshell (test_eval_inner_ uses (...) instead of {...}). This is
50 done intentionally so that any errors in the test itself do not abort the
51 testing process (and yes, that can happen). This means that variables set in
52 such test scripts DO NOT PERSIST after running the test scripts. But see the
53 TESTLIB_TEST_NO_SUBSHELL variable.
62 This directory contains the testing library source code scripts (i.e.
63 `test-lib.sh`, `test-lib-main.sh` and `test-lib-functions.sh`). This
64 variable is exported. Should be constant for any given test suite run.
65 If not set, defaults to the real directory of
66 "${TEST_DIRECTORY:-.}/test-lib.sh" (if test-lib.sh is a symblic link
67 somewhere else then that somewhere else directory will be the actual
68 default for TESTLIB_DIRECTORY, NOT the result of dirname).
72 This directory will be empty (i.e. no files in it) _AND_ will *NOT* have
73 any write permissions enabled. Usually this is "$TESTLIB_DIRECTORY/empty".
77 This directory contains the test scripts to be run (e.g. `t1234-foo.sh`)
78 and will often be the same as TESTLIB_DIRECTORY. It is also exported.
79 Should be constant for any given test suite run. This directory MUST
80 contain a `test-lib.sh` file even if it's only a symbolic link to the
81 real `test-lib.sh` file (n.b. it _MUST NOT_ be a hard link).
83 - TEST_OUTPUT_DIRECTORY
85 This is the directory containing output from the tests. It defaults to
86 TEST_DIRECTORY. Another exported variable. Should be constant for any
91 This directory is named after the specific test being run, for example,
92 `trash directory.t0000-foo.sh`. By default it's located in the
93 TEST_OUTPUT_DIRECTORY but the `--root` option to a test script (options
94 are parsed by the "`. ./test-lib.sh`" line) can change that. Using the
95 `--root` option to specify a directory on a RAM disk can accelerate
96 testing. This variable changes for every test and the directory it
97 specifies is automatically created and destroyed (`test-lib.sh` makes
102 This variable can be used to skip whole tests or parts of tests. It should
103 contain a whitespace-separated list of shell patterns. If any of the
104 pattern(s) match the test or subtest is skipped. The patterns are matched
107 1. the basename of the test with the first `'-'` and everything
108 following stripped off
110 2. value from (1) suffixed with `'.'` and the number of the subtest
111 where the first subtest is number 1
113 If the test file is `t/t1234-hello-there.sh` then the value used for (1) is
114 "t1234". For (2) and the second subtest the value is "t1234.2".
116 So setting TESTLIB_SKIP_TESTS="t1234 t1235.[235]" will skip test t1234
117 entirely and subtests 2, 3 and 5 of t1235.
119 - TESTLIB_NO_TOLERATE
121 If this is set to any non-empty value then all `test_tolerate_failure`
122 calls are turned into `test_expect_success` calls.
124 - TESTLIB_TEST_CHAIN_LINT
126 This variable defaults to true (any non-empty value). When true it results
127 in a fatal "Bail out!" error if any test script code (the last argument to
128 the `test_expect_success` function for example) does not have all of its
129 statements connected by `&&`. This variable may be set to the empty string
130 _BEFORE_ sourcing `test-lib.sh` to disable this. Alternatively, the
131 checker can be "fooled" by enclosing code in `{ ` ... `;}` or putting it in
132 a separate function that's called.
134 - TESTLIB_TEST_NO_SUBSHELL
136 Normally the various test_expect_.../test_tolerate_... functions eval the
137 test scripts they've been passed inside a subshell so that unexpected
138 failures do not abort the test process. This precludes, however, making
139 any lasting variable changes in the passed in script. This is a good
140 thing. However, if `TESTLIB_TEST_NO_SUBSHELL` is set to a non-empty string
141 then the scripts will NOT be eval'd in a subshell. Try to avoid using
142 this. Note that it may be toggled in between
143 test_expect.../test_tolerate_... calls to only enable it for one test for
146 - TEST_NO_CREATE_REPO
148 Since the focus of the testing library is on Git or Git-related software
149 all "trash directory...."s are initialized as a non-bare empty Git
150 repository by default. Setting `TEST_NO_CREATE_REPO` _BEFORE_ sourcing
151 the `test-lib.sh` testing library file will instead result in the "trash"
152 directory for the test just being an empty directory instead. The
153 function `test_create_repo` can always be used to easily create a
154 repository (or two or more) if desired.
158 If set to a non-empty value before calling the `test_done` function and
159 zero tests have been run, then the value will be appended to the output
160 `1..0 # SKIP ` plan line. If the test matches `$TESTLIB_SKIP_TESTS` when
161 `./test-lib.sh` is sourced, `skip_all` will be set to a suitable value and
162 `test_done` called immediately.
166 This variable _must_ be set to a non-empty value _before_ sourcing the
167 `test-lib.sh` script. Other than checking for non-empty, there are no
168 requirements for the value. It is only used by the `-h`|`--help` options
169 (which are parsed when `test-lib.sh` is sourced). The only formatting
170 that occurse before it's output is that leading and trailing blank lines
171 are trimmed off. Recommended format is like a Git check in comment with
172 one summary line and then any optional body separated from the summary
173 line by one blank line if present. Ideally it should explain enough about
174 the test so that if the test is failing someone other than the test author
175 can debug the problem.
177 - test_external_has_tap
179 The default setting for this variable is `0` and should normally be left
180 set to `0`. Setting it to any other value causes the output of test plan
181 and test result TAP lines to be suppressed. It can be useful when one
182 testing library test is using the testing library to run a set of tests
183 thereby preventing the recursive testing library invocation from upsetting
184 the TAP output of the parent.
191 When `test-lib.sh` is sourced, it parses the options passed to the test. When
192 running tests via the Makefile, setting "TESTLIB_TEST_OPTS" will pass those
193 options to _every_ test run by the Makefile.
195 * --chain-lint | --no-chain-lint
197 Set TESTLIB_TEST_CHAIN_LINT to 1 (`--chain-lint`) or 0 (`--no-chain-lint`).
198 See the description of TESTLIB_TEST_CHAIN_LINT above for details.
200 * --color | --no-color
202 Override automatic color detection (color output is normally enabled when
203 output is to a terminal that supports color sequences) and activate
204 (--color) or deactivate (--no-color) color output. Note that when the
205 environment variable `HARNESS_ACTIVE` is non-empty, color is ALWAYS
206 SUPPRESSED when TAP lines are output and the `#` first character of
207 comment lines also has its color suppressed in that case.
211 Normally any command preceded by `test_debug` becomes a nop. (The testing
212 library itself does not use `test_debug` anywhere.) With `--debug` these
213 lines are activated and actually run instead of being nops.
215 Using this option also prevents the temporary trash directory from being
216 removed even for a successful test.
220 Show the `$test_description` to standard output and `exit 0`. NO TAP lines
221 of any kind are output when using this option (nor are the
222 $test_description lines prefixed with the `#` comment character).
226 When `--immediate` is used, if a test_expect_success... etc. call fails
227 then `exit 1` immediately after showing the `not ok` and quoted (with
228 leading `#` characters) test script.
230 * -l | --long | --long-tests | --expensive
232 Set and export `TESTLIB_TEST_LONG=t`. The `EXPENSIVE` prereq checks for
233 this and only succeeds when `TESTLIB_TEST_LONG` is non-empty.
237 Suppresses some output lines such as "ok" (only when NOT running under a
238 harness) and various output from test_external... functions. Mainly
239 useful to produce more compact output when NOT running under a TAP harness.
243 Causes the "trash directories" to be created as subdirectories of `<path>`
244 instead of as subdirectories of `$TEST_OUTPUT_DIRECTORY`. Specifying a
245 RAM disk location as `<path>` can significantly speed up testing.
247 * --run=<run-list> | -r <run-list>
249 The <run-list> must contain comma or whitespace separated "selectors". But
250 note that when using "TESTLIB_TEST_OPTS" only comma separated will work.
252 A selector must match either !?\d+ or !?\d*-\d* which is to say it's either
253 <num> or <num>- or -<num> or <num>-<num> optionally prefixed with `!`. The
254 "<num>-" form is a shortcut for <num>-infinity and the "-<num>" form is a
255 shortcut for "1-<num>".
257 If the first (or only) selector is prefixed with `!` then it's an exclusion
258 list otherwise it's an inclusion list.
260 The "selectors" are processed in left to right order and if the number of
261 the subtest matches the range it will be run if the selector was NOT
262 prefixed with `!` otherwise it will be excluded.
264 An "inclusion" list includes and runs ONLY those subtests that are
265 specifically matched by a selector NOT prefixed with `!`. (Note that
266 a subsequent selector can again exclude a previously included match and
269 An "exclusion" list runs everything except tests that are specifically
270 excluded by a match. In other words an "exclusion" list is just a shortcut
271 to avoid adding a "1-" selector to the front of <run-list>.
273 For example, "1-4,!3" will run only subtests 1, 2 and 4 but so will
274 "1,2,4". On the other hand, "!3,1-4" will run ALL subtests except 3
275 (because it's equivalent to "1-,!3,1-4") as will just "!3".
279 All standard output and standard error from running the test is copied to
280 $TEST_OUTPUT_DIRECTORY/test-results/$(basename $0 .sh).out and as a side
281 effect standard error is redirected to standard out. The final exit code
282 is written to $TEST_OUTPUT_DIRECTORY/test-results/$(basename $0 .sh).exit
285 This is accomplished by running another copy of the test script and
286 redirecting both its standard output and standard error through the `tee`
287 utility and when that copy exits, so too will the "tee" process and it will
288 then use the same exit code as its exit code (unless something went wrong
289 and no exit code was recorded in which case it exits with a result of 1).
291 The environment variable TESTLIB_TEST_TEE_OUTPUT_FILE is set (and exported)
292 to the name of the file the output is being "tee"'d into.
296 When `--verbose` is used the `test_pause` function will actually pause
297 a test by entering a subshell and `test_external_without_stderr` will show
298 both stdout and stderr output on failure. Test scripts passed to the
299 test_expect_success... functions are run with standard output and standard
300 error left unchanged (normally it's sent to /dev/null).
302 Automatically enabled by `-x`.
304 Note that `--verbose` is not allowed when $HARNESS_ACTIVE.
308 Enables the `--tee` option and arranges for the standard output and
309 standard error that are enabled by `--verbose` to end up in the same log
310 file that `--tee` creates while not disrupting the output going to the
313 This setting supersedes `--verbose` in that if both are given the output
314 ends up in the log but not on normal standard output or standard error.
316 Also if `--verbose-only=...` is used together with this option then only
317 the subtests selected by the `--verbose-only` option value will have their
318 output captured into the log -- the other subtests will have it sent to
321 Note that `--verbose-log` *IS* compatible with $HARNESS_ACTIVE.
323 * --verbose-only=<pattern-list>
325 The current subtest number (sub tests are numbered starting at 1) is
326 matched against each pattern in the the whitespace-separated
327 filename-wildcard-pattern list <pattern-list> and if there's a match then
328 `--verbose` is active for that subtest and if there is *NOT* a match then
329 `--verbose` is NOT active for that subtest.
331 For example, `--verbose-only="[1-3] 5"` will enable verbose for
332 subtests 1, 2, 3 and 5 and will DISABLE verbose mode for subtests
333 4,6,7,8,9,10,11,12,13,14,15 etc. The fancy <run-list> processing of the
334 `--run` option's argument is *NOT* available here.
336 Using this option with a non-empty <pattern-list> value completely
337 supersedes any explicit `--verbose` option (before or after).
338 (Using it with an empty value just silently has no effect.)
340 Note that `--verbose-only=...` is not allowed when $HARNESS_ACTIVE unless
341 `--verbose-log` is also used.
345 Set the `--verbose` option and temporarily enable the shell's `set -x`
346 option while evaluating the scripts passed to test_expect_success....
348 Note, however, that tracing will be suppressed unless verbose is also
349 enabled. Normally that will always be the case since `-x` also enables
350 `--verbose`. However, if `--verbose-only` is used then verbose will be
351 toggled on a per-subtest basis and as a result the `-x` option will only
352 be in effect for subtests selected by the `--verbose-only=...` option.
354 Note that since this option sets `--verbose` it's not normally allowed
355 when $HARNESS_ACTIVE but setting `--verbose-log` will allow it and send the
356 output to the log file.
363 These are functions that are intended to ultimately emit either a TAP test plan
364 line or a TAP test (aka "subtest") result line.
366 These functions are intended to be called from the main body of a test script
367 (e.g. t-????-*.sh) after having sourced the `./test-lib.sh` script.
373 The purpose of these functions is to output the single, required, TAP test
374 plan line. Additionally the `test_done` function performs some extra work.
378 Outputs a TAP test plan line.
380 MUST be called AFTER any test result functions. (TAP supports outputting
381 a test plan line either _before all_ test result lines or _after all_
384 It may be tempting to output the plan line without using this function.
385 Don't do that. Failure to use this function will result in test cleanup
386 functions failing to run and the test results aggregation provided by the
387 testing library breaking.
389 If `$test_external_has_tap` is set to something other than `0` (the
390 default) then no test plan line will be output but normal test cleanup will
393 If `$skip_all` is set to a non-empty value AND 0 tests were run then the
394 `$skip_all` message will be appended to a `1..0 # SKIP ' line (which is,
395 again, suppressed if `$test_external_has_tap` is not `0`).
397 The `test_done` function then calls either `exit 0` if there were no
398 unexpected failures or `exit 1` if there were. Just before calling
399 `exit 0` (but _not_ `exit 1`) the `test_at_end_hook_` function is called
400 which, by default, always succeeds and does nothing but can be re-defined
401 by a test script if such a hook is needed.
403 Note that if the test name appears in `$TESTLIB_SKIP_TESTS` (see the
404 description above) then `$skip_all` is automatically set to a suitable
405 value and `test_done` is called immediately when `./test-lib.sh` is
410 This function allows a test plan line to be output first. If `$skip_all`
411 is set it just passes through to `test_done`. Otherwise it outputs the
412 test plan line using the first argument as the total test count and then
413 sets `test_wrote_plan_count=$1` so that the required subsequent call to
414 `test_done` does not write a second test plan line. (But if the value of
415 `$test_external_has_tap`is not 0 it will not output the test plan line.)
417 If the argument is 0, then `skip_all` will be set to the second argument
418 (or a suitable default if there is no second argument) and `test_done` will
419 be called immediately.
421 Using `test_plan` is inconvenient in that the number of subtests in the
422 file must be known in advance. Sometimes that's not possible if the number
423 of tests to be run varies depending on test conditions. Other times it's
424 just damn inconvenient to have to count all those subtests. ;)
426 However, outputting the test plan line in advance makes those fancy test
427 harnesses (like `prove`) show a "1/n" progress rather than "1/?" which is
428 nice to have and that is the reason the `test_plan` function is provided.
430 Also if a different number of tests are run than are planned the overall
431 test will fail with an error.
433 The `test_done` function MUST STILL BE CALLED at the end of the test script
434 even when `test_plan` is used at the beginning otherwise test aggregation
435 and test cleanup will fail to take place!
437 Note that if it's not already clear from the description, use of the
438 `test_plan` function is completely optional, but use of `test_done` is not.
441 Test Result Functions
442 ~~~~~~~~~~~~~~~~~~~~~
444 The purpose of these functions is to output one `ok` or `not ok` TAP test
445 result line each time they're called.
447 All of these functions take an optional first argument which is a whitespace
448 and/or comma separated list of prerequisites that must be present or the
449 test will be skipped. This is shown as `[<prereqs>]`. Each prereq in the
450 list may be optionally prefixed with a single `!` to require the item *not*
453 All of these functions expect the next argument to be a brief test description
454 that will be used in the `ok` or `not ok` output line. This is shown as
457 The "external" functions expect two additional arguments which are the
458 "external" command to be run and its single argument (use `eval` and a suitable
459 string to work around this requirement for exactly two arguments). These are
460 shown as `<cmd>` `<arg>`.
462 The non-"external" functions expect one additional argument which is the
463 test script to be "eval"'d. However, if the test script argument is `-` then
464 the test script will be read from standard input (allowing use of a HERE doc to
465 simplify quoting issues), but this is slightly inefficient so avoid using it
466 unless it's really needed. This is shown as `<script>`. Note that scripts are
467 normally "eval"'d inside a subshell to avoid side-effects and uncaught
468 failures, but this can be altered with the `TESTLIB_TEST_NO_SUBSHELL` variable
469 (see description above).
471 Test scripts are "eval"'d as the last line of a function so they may exit early
472 by using a `return` statement or, if `$TESTLIB_TEST_NO_SUBSHELL` is _NOT_ set
473 (which is the default), by using an `exit` statement.
475 - test_expect_success [<prereqs>] <desc> <script>
477 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
478 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
479 it succeeds (exit code is 0) then this subtest succeeds and outputs an "ok"
480 line otherwise it outputs a "not ok" line.
482 - test_expect_failure [<prereqs>] <desc> <script>
484 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
485 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
486 fails (exit code is _not_ 0) then this subtest outputs a "not ok # todo"
487 line. If the script succeeds a "ok # todo" line is output which will
488 ultimately produce some warnings and/or complaints about a "known breakage"
489 vanishing or "todo passing". See the `test_tolerate_failure` function to
492 - test_tolerate_failure [<prereqs>] <desc> <script>
494 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
495 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
496 fails (exit code is _not_ 0) then this subtest outputs a "not ok # todo"
497 line. If the script succeeds a plain "ok" line is output (which avoids
498 any complaints about "vanishing" breakages). However, if the variable
499 `TESTLIB_NO_TOLERATE` is non-empty all `test_tolerate_failure` calls are
500 turned into `test_expect_success` calls instead (meaning a failure causes
501 a regular `not ok` to be output instead of a `not ok # todo`).
503 This function is useful for testing things that probably don't work the
504 way they should but are reluctantly tolerated and if they do start working
505 better no fuss should be made about the test suddendly passing.
507 It's a nice way to document those "yes, we still have this ugly thing here
508 that we wish we didn't" behaviors. It can also be used to distinguish
509 "tier 1" supported platforms (those which pass all tests even when
510 `TESTLIB_NO_TOLERATE=1` is set) and "tier 2" supported platforms (those
511 that pass all tests as long as `TESTLIB_NO_TOLERATE` is _not_ set). (Tier
512 3 would be those platforms that always fail at least one test.)
514 - test_external [<prereqs>] <desc> <cmd> <arg>
516 Behaves just like `test_expect_success` with a script of "<cmd> <arg>"
517 (assuming <cmd> and <arg> are first suitably quoted) except that stdout is
518 always left on stdout even without `--verbose`.
520 Use "eval" for <cmd> and pass a script as <arg> to use as a substitute
521 `test_expect_success` that does not redirect stdout or to escape the
522 two argument restriction.
524 - test_external_without_stderr [<prereqs>] <desc> <cmd> <arg>
526 *IMPORTANT*: This function outputs _two_ test result lines!
528 If using the `test_plan` function then each `test_external_without_stderr`
529 function call counts as *TWO* planned tests!
531 It works just like `test_external` (in fact, it calls test_external first)
532 and then it performs a second subtest to see if _anything_ was output to
533 stderr by "<cmd> <arg>" in which case the second subtest fails otherwise
534 it succeeds. If the `test_external` first subtest ends up being skipped
535 then nothing will be output to stderr and the second subtest will always
536 succeed (it will not be skipped). In other words, the second subtest
537 result line is always a plain "ok" or "not ok" and never has any "# skip"
538 or "# todo" added to it even when the first subtest result line does.
540 Note that the special prerequiste `LASTOK` behaves as expected here --
541 that is if the first `test_external` call is skipped then its state
542 _does NOT change_ regardless of the outcome of the second "stderr"
543 subtest check. In addition, if the the `test_external` test is _not_
544 skipped then the `LASTOK` prerequisite is guarantted to only be true
545 provided _both_ the `test_external` call suceeds _and_ no "stderr" output
546 is generated. In other words, it just does the right thing here so don't
547 worry about it and go ahead and use it.
550 Standard Test Prerequisites
551 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
553 The test result functions all take an optional first argument which is a comma
554 and/or whitespace separated list of positive or negative (prefixed with `!`)
555 prerequisites for the test to be run. *All* off the listed prerequisites for
556 each test _must_ be satisfied or it will be skipped.
558 Additional prerequisites may be set using the `test_set_prereq` function but
559 there are a number of pre-defined prerequisites that may be tested:
563 The `git var GIT_AUTHOR_IDENT` command succeeds when both `GIT_AUTHOR_NAME`
564 and `GIT_AUTHOR_EMAIL` are unset.
568 The backslash character (`\`) is allowed in path/file names. Normally
569 defined _except_ for `MINGW` and `CYGWIN`.
571 - `CASE_INSENSITIVE_FS`
573 File names are case-insensitive (i.e. "foo" and "Foo" refer to the same
578 Whether or not the `run_with_limited_cmdline` function appears to work
579 (it does `ulimit -s 128` in a subshell and runs the command assuming that
580 will limit the size of the command line argument list).
584 Set when the shell's `$COLUMNS` variable can be successfully set to 1.
588 The CygWin environment is active.
592 The process ID before and immediately after an `exec` call stays the
593 same. Normally defined for all but `MINGW`.
597 The `TESTLIB_TEST_LONG` variable is non-empty (see the `--long-tests` aka
598 `--expensive` test option).
602 True if Git thinks the executable bit is supported properly (i.e.
603 Git's `core.filemode` is set to true after `git init`).
607 The grep command strips off CR. Normally only defined for `MINGW` and
612 This is a special prerequisite that always succeeds for the first subtest
613 in a test script and for every subtest thereafter for which the most
614 immediately preceeding non-skipped test was "ok" (regardless of the
615 presence any "# TODO").
617 In other words, this can be used to skip a subtest if the preceeding
618 subtest did not succeed. Since "skipped" tests do not change the state
619 of this prerequisite a single failure can cause a long chain of following
620 subtests that use this prerequisite to be skipped.
624 The MinGW environment is active.
628 The native environment uses CR+LF line endings (normally only defined
633 The `uid -u` command succeeded and returned a non-0 value.
637 The file system automatically converts NFD unicode to NFC unicode.
641 The `mkfifo` command is supported and works.
645 Sane, POSIX-style permissions. *Not* normally defined for `MINGW`.
649 File permission bits are honored (i.e. after doing `chmod a-r file` then
650 a subsequent `cat file` will fail). When running as root these "sanity"
655 The sed command strips off CR. Normally only defined for `MINGW` and
660 Symbolic links created using `ln -s` work.
664 The result of `test -x /usr/bin/time` is true.
671 These are convenience functions and while most of them are provided to be
672 called from the body of a script passed to one of the test_expect_...
673 functions, several of them can also be used from the main body of a test
674 script (`say`, `say_color`, `test_debug` and `test_pause` being prime examples
677 These functions are intended to be called only _after_ the `./test-lib.sh`
678 script has been sourced.
682 Can be used to exit a test script on an unexpected error, should be used
683 after a `||` to preserve the exit code as in:
685 perform_some_function perhaps with arguments || die
687 - error <msg> <goes> <here>
689 Cause an instant test `Bail out!` error (including <msg> <goes> <here>).
691 The original Git version of this function just did an exit after showing
692 the error without using `Bail out!`, but these errors are terminal and the
693 entire test suite should be stopped if one occurs and that is what will now
694 happen when this function is called.
696 - list_contains <commalist> <pattern>
698 Returns true if comma separated list <commalist> contains any item that
699 matches the filename matching wildcard <pattern>.
701 A shortcut for those uncomfortable with the `case` statement.
703 - sane_unset <varname>...
705 An alias for `unset <varname>... || :` to workaround non-conforming shells
706 that return a non-zero status if any of the variables were not already set.
708 - say <msg> <goes> <here>
710 A convenient shortcut for `say_color info <msg> <goes> <here>`
712 - say_tap <msg> <goes> <here>
714 A convenient shortcut for `say_color_tap info <msg> <goes> <here>`
716 - say_color <color> <msg> <goes> <here>
718 Possible <color> values are:
720 * "" (the empty string)
721 plain text with no color, suppressed when `--quiet` in effect
723 used for testing failures, typically shown in red
725 used for test skip messages, typically shown in a light color
727 warnings are shown in this, typically yellow (or brown)
729 used when a complete test script passes, usually green
731 info messages and non-harness TAP messages, usually a light color
733 turns off color, used internally, should not be used by callers
735 Using anything else for <color> ends up outputting plain text followed by
736 the color reset code.
738 If `--quiet` is in effect, anything output using `""` (the empty string)
739 as the color name will be supressed entirely.
741 If `--no-color` is in effect (perhaps implicitly) the color code sequences
742 are omitted from the output.
744 If `$HARNESS_ACTIVE` and the first character of <msg> is `#` then that
745 single character is always output _without_ any preceding color codes (the
746 first character to be in color will be the following one unless, of course,
747 the color name is `""` or `--no-color` is in effect).
749 - say_color_tap <color> <msg> <goes> <here>
751 When `$HARNESS_ACTIVE` this always outputs "<msg> <goes> <here>" _without_
752 any color sequences regardless of the presence of the `--quiet` option.
754 If there is no `$HARNESS_ACTIVE` then this call is identical to `say_color`
755 with the exact same arguments.
759 This function is called by `test_done` if there are no unexpected failures.
760 By default it's defined to be just `:` but should be re-defined be each
761 test script file as needed. Note that it _will_ be called when the entire
762 test script is skipped with a `1..0` plan line.
768 $TESTLIB_TEST_CMP <arg>...
770 Useful because `diff -u` will be used where supported instead of `cmp`.
772 - test_cmp_bin <arg>...
778 There's no real compelling reason to use this other than to look nice
781 - test_cmp_rev <expect-rev> <actual-rev>
783 Save output of `git rev-parse --verify` on each argument to a file and
784 then run `test_cmp expect.rev actual.rev` on the resulting two files.
786 - test_commit [--notick] [--signoff] <message> [<file> [<contents> [<tag>]]]
788 Create a new Git commit with "<message>" as the commit message optionally
789 having a signed-off-by line (`--signoff`) using the next tick time by
790 calling `test_tick` (unless `--notick` is used).
792 <file> defaults to "<message>.t", <contents> and <tag> default to
795 The "<contents>" will be written (using `echo`) to <file> which is then
796 added and committed using <message> and then tagged using <message>.
798 - test_copy_bytes <count>
800 This is a shortcut for a call to the `dd` utility (unlike the Git version
801 which uses perl) to transfer <count> bytes (or up to EOF) from stdin to
804 - test_create_repo <path>
808 (mkdir -p "<path>" && cd "<path>" && git init)
810 Except that an empty template directory is used and if that doesn't succeed
811 in suppressing creation of a hooks directory the hooks directory will be
812 renamed to `hooks-disabled`.
814 Note that since Git will happily reinitialize an existing Git repository
815 this command can succeed on a pre-existing repository without altering its
818 - test_debug [<cmd> [<arg>...]]
820 This function does nothing (it's a fancy comment) _unless_ `--debug` is
821 used in which case <cmd> [<arg>...] is executed with standard output and
822 standard error redirected to the original standard error of the test
823 script (in other words, you don't need to add `--verbose` to see the
824 output when `--debug` is active).
826 Note that this is different than the original Git version that only takes
827 a single argument and "eval"s it (while not guaranteeing anything about
828 where the output goes). To get the Git "eval" equivalent, use `eval` as
829 the first <arg> and the string to be "eval"'d as the second.
831 - test_dir_is_empty <dirpath>
833 Fail verbosely (with directory contents) unless <dirpath> is an existing
834 and empty (only `.` and `..` entries) directory.
836 - test_env [VAR=VAL]... <cmd> <arg>...
838 In a subshell set and export each `VAR=VAL` and then run `<cmd> <arg>...`.
840 Obviously, since it's a subshell no variable changes will persist, but this
841 makes it easy to temporarily change a variable while running some shell
844 - test_expect_code <status> <cmd> [<arg>...]
846 Runs `<cmd> <arg>...` and fails verbosely unless the resulting status code
849 - test_have_prereq <PREREQ>...
851 Test a list of zero or more whitespace and/or comma separated prereqs
852 (the same as can be given to any of the test result functions). As with
853 the test result functions, prefixing a prerequisite with a single `!`
854 requires that it _not_ be present. Result code is 0 if the requirements
855 are satisfied, non-zero if not.
857 - test_line_count <op> <val> <file>
859 Verbosely fails if `test $(( $(wc -l < "<file>") )) "<op>" "<val>"` fails.
861 - test_match_signal <signum> <exitcode>
863 Returns true if <exitcode> represents and exit from signal <signum> (which
864 must be numeric). This function only exists to accomodate `ksh` which
865 uses 256+<signum> instead of the POSIX 128+<signum> for signal exit codes.
867 - test_merge <message> <commit>
871 test_tick && git merge -m "<message>" <commit> && git tag "<message>"
873 - test_might_fail <cmd> [<arg>...]
875 Just a shortcut for `test_must_fail ok=success <cmd> <arg>...`.
877 This is useful when it's okay for <cmd> to succeed or for it to fail
878 provided the failure is a "normal" (i.e. non-signal) failure.
880 - test_must_be_empty <file>
882 Verbosely complain (and fail) showing <file>'s contents if it's not empty
883 (i.e. -s <file> succeeds).
885 - test_must_fail [ok=<commalist>] <cmd> [<arg>...]
887 Runs `<cmd> <arg>...` and verbosely converts all result codes into either
888 0 or 1 where "normal" failures become 0 and others become 1.
890 This function should be used when a failure is required but only if the
891 failure is *not* the result of a signal, command not found or command not
894 If <commalist> contains "success" then a result code of 0 is _not_
895 converted to 1 (it stays 0). If <commalist> contains "sigpipe" then an
896 exit due to `SIGPIPE` is converted to status 0 (otherwise it becomes 1).
898 - test_path_is_dir <path> [<msg>]
900 Fail verbosely (including <msg> if given) if `test -d <path>` fails.
902 - test_path_is_file <path> [<msg>]
904 Fail verbosely (including <msg> if given) if `test -f <path>` fails.
906 - test_path_is_missing <path> [<msg>...]
908 Fail verbosely (including <msg>... if given) if `test -e <path>` _succeeds_.
912 Spawn an interactive `$SHELL` to pause testing until it exits -- causes an
913 error if used without `--verbose`. Should be used for debugging only.
915 - test_seq [<start>] <end>
917 <start> defaults to 1.
919 Output each of the numbers from <start> to <end> inclusive (incrementing by
920 1) each followed by a newline to stdout. If <end> is less than <start>
923 - test_set_editor <path-to-editor>
925 Set and export the `EDITOR` environment variable but using tricks so that
926 <path-to-editor> is allowed to contain any value (including spaces, quotes,
927 etc.) without needing any special quoting.
929 - test_set_prereq <PREREQ>
931 Indicate that the test prerequisite <PREREQ> is available (should be a
932 single word in ALL CAPS). It may then be used as a prereq for any of the
933 test result functions or be explicitly tested with `test_have_prereq`.
935 - test_skip_or_die <mode> <msg>
937 Don't use this function, it's confusing, use either `test_plan 0` or
940 If <mode> is `auto` it's the same as `test_plan 0 <msg>`.
942 If <mode> is `true` it's the same as `error <msg>`.
944 If <mode> is anything else it's a more verbose call to `error`.
948 Set and export `GIT_COMMITTER_DATE` and `GIT_AUTHOR_DATE` to the same
949 value (initially 1112911993 -- the time of the first Git commit) increasing
950 it by exactly 60 seconds each time `test_tick` is called.
952 - test_tristate <varname>
954 Examines contents of `$<varname>` and sets `$<varname>` to one of `false`
955 `true` or `auto`. An unset variable becomes `auto` and an empty string
956 variable becomes `false` otherwise there should be no surprises except
957 that anything that is not a Git boolean or `auto` is treated as `true`.
959 The original version from the Git test library uses Git itself to do this
960 conversion, but this implementation is strictly shell built-ins.
962 - test_when_finished <arg>...
964 This function only works inside a test script!
966 Causes "<arg>..." to be "eval"'d when the current test script (the
967 <script> arg to a test result function) exits. May be used more than
968 once to schedule multiple items.
970 Currently only works when `TESTLIB_TEST_NO_SUBSHELL` is set.
972 - test_write_lines <arg>...
976 printf '%s\n' <arg>...
978 Although since it's more characters to type perhaps it's not so lazy. ;)
980 - verbose <cmd> [<arg>...]
982 Run `<cmd> <arg>...` and verbosely complain (showing command and args) if
983 it fails, but the exit status on failure will be converted to 1.
985 - write_script <file> [<shell-path>] < <script-text>
987 Write a shebang line using <shell-path> (default is `$SHELL_PATH`) to
988 <file> and then append <script-text> to it and then finally add execute
989 permission to the resulting <file>.
993 If <expletive> is omitted, `y` is used.
995 Outputs "<expletive>" (and a trailing newline) 99 times.
997 This is basically the same as the POSIX `yes` utility except that output
998 is limited to 99 lines and it's implemented entirely as shell code.