4 The point of the testing library (available by sourcing the
5 test-lib.sh file) is to assist with rapidly writing robust
6 tests that produce TAP-compliant output. (For a quick primer
7 on TAP see the README file in the section "TAP - A Quick Overview".)
10 ----------------------------
11 Why Use the Testing Library?
12 ----------------------------
16 * hopefully helps to allow tests to be written more quickly
17 * catches common failures (signal exits, not found etc.)
18 * avoids reproducing commonly needed code in every test
19 * prevents unwanted "other" lines from ending up in the TAP output
20 * allows test results to be aggregated without using "prove"
21 * supports easier test debugging
28 Check out the `README` file for a quick overview of everything in the test
29 directory and a brief overview of TAP.
31 This file, `README-TESTLIB`, serves as a reference of available variables,
32 options and functions in the testing library that can be used by test writers
33 to create tests. This file does not, however, provide any "how-to" instruction
34 as that's reserved for the `README-WRITING-TESTS` file.
36 The `README-WRITING-TESTS` file describes how to quickly get started writing
37 tests using the testing library and should be perused by anyone who finds the
38 simplistic example in the `README` file to be insufficient. It does not,
39 however, serve as a comprehensive reference for the testing library.
41 To quickly find the testing library function and/or variable needed refer to
42 the [Quick Reference] section below.
49 Note there is a difference between the way test scripts passed to the various
50 test_expect/test_tolerate functions behave compared to Git's testing library.
51 In this testing library the scripts passed to those functions are evaluated
52 inside a subshell (test_eval_inner_ uses (...) instead of {...}). This is
53 done intentionally so that any errors in the test itself do not abort the
54 testing process (and yes, that can happen). It also allows `set -e` to be
55 activated by default in the subshell (which it is). As a side effect, this
56 means that variables set in such test scripts DO NOT PERSIST after running the
57 test scripts. The `test_when_finished` function can be used to raise up some
58 information out of the (possibly nested) subshell as it runs after the
59 subtests's subshell has exited -- this is an enhancement compared to Git's
60 version of `test_when_finished` which doesn't support any subshells at all.
62 See the TESTLIB_TEST_NO_SUBSHELL variable to disable (NOT recommended) the use
70 Variable | Description
71 --------------------------------|----------------------------------------------
72 TESTLIB_DIRECTORY | directory where testing library code lives
73 EMPTY_DIRECTORY | empty directory with no write permission
74 TEST_DIRECTORY | directory with tNNNN-*.sh scripts to run
75 TEST_HELPER_DIRECTORY | directory with "helpers" (auto-added to PATH)
76 TEST_OUTPUT_DIRECTORY | test output ends up in here
77 TRASH_DIRECTORY | test specific temporary "trash" directory
78 TRASHTMP_DIRECTORY | test specific temporary `test_get_temp` dir
79 TESTLIB_SKIP_TESTS | test suite wide test/subtest skipping pats
80 TESTLIB_NO_TOLERATE | `test_tolerate_failure` means expect success
81 TESTLIB_TEST_CHAIN_LINT | can be used to disable `&&` chaining test
82 TESTLIB_TEST_NO_SUBSHELL | prevent subshell for subtests (do not use)
83 TESTLIB_TEST_TAP_ONLY | limit output to (almost) only TAP lines
84 TEST_NO_CREATE_REPO | suppress initial `test_create_repo .`
85 color | either "" (--no-color) or "--color" (--color)
86 skip_all | `test_done` skip value for `1..0` plan
87 test_auh | `--allow-unrelated-histories` if Git >= 2.9
88 test_description | must be non-empty before `. ./test-lib.sh`
89 test_external_has_tap | avoid changing -- will suppress TAP lines
90 test_hash_algo | either `sha1` or `sha256` (only if supported)
91 test_tick | last commit timestamp used see same function
92 this_test | basename $0 | sed 's/-*//' (usually tNNNN)
94 TAP function | Description
95 --------------------------------|----------------------------------------------
96 test_done | uncounted plan line and counts (call last)
97 test_expect_success | expect success from a script (no #TODO)
98 test_expect_failure | expect failure from a script (with #TODO)
99 test_external | test external command
100 test_external_without_stderr | test external command & require empty stderr
101 test_plan | output TAP plan line (call first)
102 test_tolerate_failure | expect success or failure from a script
104 Utility function | Description
105 --------------------------------|----------------------------------------------
106 die | fatally exit the test (or subshell) right now
107 error | Bail out! now and kill all further testing
108 git_init | git -c init.defaultBranch=master init "$@"
109 list_contains | check comma-separated list against wildcards
110 sane_unset | safe unset for broken sh implementations
111 say | say_color info ... shortcut
112 say_color | output in the named (special names) color
113 say_color_tap | if $HARNESS_ACTIVE always output w/o color
114 say_tap | say_color_tap info ... shortcut
115 test_at_end_hook_ | called by `test_done` default is `:`
116 test_clear_when_finished | make code run just after current subtest exit
117 test_cmp | fail if two text files do not match
118 test_cmp_bin | fail if two binary files do not match
119 test_cmp_rev | fail if two Git revspecs resolve differently
120 test_commit | make a new commit optionally tagging it
121 test_config | make test_when_finished reverted config mod
122 test_config_global | like `test_config` but `--global` config
123 test_copy_bytes | transfer specified # of bytes from in to out
124 test_create_repo | create dir (if not exists) and `git_init` it
125 test_debug | with `--debug` run args otherwise nop
126 test_diff | fancy (same semantics) `test_cmp` using git
127 test_dir_is_empty | verbose fail if directory not empty
128 test_env | like /usr/bin/env but works for functions too
129 test_expect_code | fail unless command has specified exit status
130 test_get_temp | create a new, unique empty temp file (or dir)
131 test_have_prereq | check for a prereq
132 test_line_count | overblown `test` plus `wc` utility
133 test_match_signal | bad bad bad ksh
134 test_merge | `test_tick` & `git merge` & `git tag` shortcut
135 test_might_fail | succeeds for all but abnormal exits
136 test_must_be_empty | verbose fail if `test -s ...` is true
137 test_must_fail | succeed for all non-0 and non-abnormal exits
138 test_path_is_dir | verbose failure if `test -d ...` fails
139 test_path_is_file | verbose failure if `test -f ...` fails
140 test_path_is_missing | verbose failure if `test -e ...` fails
141 test_pause | interactive $SHELL if `--verbose` else error
142 test_seq | shell-only version of 1 and 2 arg seq
143 test_set_editor | properly quote value and set+export EDITOR
144 test_set_prereq | set specified prereq to true
145 test_skip_or_die | prefer `test_plan 0 <msg>` or `error <msg>`
146 test_tick | set or increment(+60) the `test_tick` var
147 test_tristate | reset variable to `true`, `false` or `auto`
148 test_unconfig | safe `git config --unset-all` when unset
149 test_when_finished | run code right after current subtest exits
150 test_write_lines | overblown `printf '%s\n'` shortcut
151 verbose | verbosely show a command's failures
152 wc | whitespace safe version of wc
153 write_script | make executable sh script from stdin
154 yes | 99 times max shell-only version of yes
163 This directory contains the testing library source code scripts (i.e.
164 `test-lib.sh`, `test-lib-main.sh` and `test-lib-functions.sh`). This
165 variable is exported. Should be constant for any given test suite run.
166 If not set, defaults to the real directory of
167 "${TEST_DIRECTORY:-.}/test-lib.sh" (if test-lib.sh is a symblic link
168 somewhere else then that somewhere else directory will be the actual
169 default for TESTLIB_DIRECTORY, NOT the result of dirname).
173 This directory will be empty (i.e. no files in it) _AND_ will *NOT* have
174 any write permissions enabled. Usually this is "$TESTLIB_DIRECTORY/empty".
178 This directory contains the test scripts to be run (e.g. `t1234-foo.sh`)
179 and will often be the same as TESTLIB_DIRECTORY. It is also exported.
180 Should be constant for any given test suite run. This directory MUST
181 contain a `test-lib.sh` file even if it's only a symbolic link to the
182 real `test-lib.sh` file (n.b. it _MUST NOT_ be a hard link).
184 - TEST_HELPER_DIRECTORY
186 The test helpers are located in here. It defaults to
187 "$TEST_DIRECTORY/helper" if not set and that directory exists. It is
188 automatically added to the `PATH`. Helper scripts should use this to
189 locate any auxilliary files they might need to use that are checked into
190 version control alongside them. This variable is also exported.
192 - TEST_OUTPUT_DIRECTORY
194 This is the directory containing output from the tests. It defaults to
195 TEST_DIRECTORY. Another exported variable. Should be constant for any
196 given test suite run.
200 This directory is named after the specific test being run, for example,
201 `trash directory.t0000-foo.sh`. By default it's located in the
202 TEST_OUTPUT_DIRECTORY but the `--root` option to a test script (options
203 are parsed by the "`. ./test-lib.sh`" line) can change that. Using the
204 `--root` option to specify a directory on a RAM disk can accelerate
205 testing. This variable changes for every test and the directory it
206 specifies is automatically created and destroyed (`test-lib.sh` makes
211 This directory is named after the specific test being run, for example,
212 `trash tmp directory.t0000-foo.sh`. By default it's located in the
213 TEST_OUTPUT_DIRECTORY but the `--root` option to a test script (options
214 are parsed by the "`. ./test-lib.sh`" line) can change that. Using the
215 `--root` option to specify a directory on a RAM disk can accelerate
216 testing. This variable changes for every test and the directory it
217 specifies is automatically created and destroyed (`test-lib.sh` makes
220 However, unlike TRASH_DIRECTORY, this directory is created on-demand the
221 first time the `test_get_temp` function is called (or a lazy prereq is
222 evaluated) and so should not be used directly. Rather it exists primarily
223 to support creation of temporary test files that live completely outside
224 of TRASH_DIRECTORY and so can be created and used without concern about
225 any possible unintended consequences of temporary files lying around in
226 the TRASH_DIRECTORY (or a subdirectory thereof) causing unexpected test
231 This variable can be used to skip whole tests or parts of tests. It should
232 contain a whitespace-separated list of shell patterns. If any of the
233 pattern(s) match the test or subtest is skipped. The patterns are matched
236 1. the basename of the test with the first `'-'` and everything
237 following stripped off
239 2. value from (1) suffixed with `'.'` and the number of the subtest
240 where the first subtest is number 1
242 If the test file is `t/t1234-hello-there.sh` then the value used for (1) is
243 "t1234". For (2) and the second subtest the value is "t1234.2".
245 So setting TESTLIB_SKIP_TESTS="t1234 t1235.[235]" will skip test t1234
246 entirely and subtests 2, 3 and 5 of t1235.
248 - TESTLIB_NO_TOLERATE
250 If this is set to any non-empty value then all `test_tolerate_failure`
251 calls are turned into `test_expect_success` calls.
253 - TESTLIB_TEST_CHAIN_LINT
255 This variable defaults to true (any non-empty value). When true it results
256 in a fatal "Bail out!" error if any test script code (the last argument to
257 the `test_expect_success` function for example) does not have all of its
258 statements connected by `&&`. This variable may be set to the string "0"
259 _BEFORE_ sourcing `test-lib.sh` to disable this. Alternatively, the
260 checker can be "fooled" by enclosing code in `{ ` ... `;}` or putting it in
261 a separate function that's called.
263 - TESTLIB_TEST_NO_SUBSHELL
265 Normally the various test_expect_.../test_tolerate_... functions eval the
266 test scripts they've been passed inside a subshell so that unexpected
267 failures do not abort the test process. This precludes, however, making
268 any lasting variable changes in the passed in script. This is a good
269 thing. However, if `TESTLIB_TEST_NO_SUBSHELL` is set to a non-empty string
270 then the scripts will NOT be eval'd in a subshell. Try to avoid using
271 this. Note that it may be toggled in between
272 test_expect.../test_tolerate_... calls to only enable it for one test for
275 Use of this variable to disable subtest subshells is deprecated and may go
276 away at some point! Avoid this if at all possible! The
277 `test_when_finished` function (possibly combined with `test_get_temp`) can
278 be used to raise information out of a subtest subshell when required.
280 - TESTLIB_TEST_TAP_ONLY
282 Setting this to a value other than empty or `0` suppresses extraneous
283 failure output (such as the script). Incompatible with verbose mode (but
284 allowed with verbose log mode). If set to the special value -1 it will
285 auto-disable when verbose mode is used.
287 Note that when TESTLIB_TEST_TAP_ONLY is enabled one additional line will
288 be output for each failure showing the file and line number, but _only_ if
289 the line number is available. This extra line can be suppressed by using
290 `--quiet` when TESTLIB_TEST_TAP_ONLY is enabled.
292 Set `TESTLIB_TEST_TAP_ONLY = -1` in your `config.mak` and get happy.
294 - TESTLIB_GIT_DEFAULT_HASH
296 Select the default hash algorithm for Git to use when creating new
297 repositories. If this is not set, it defaults to `sha1`.
299 Essentially TESTLIB_GIT_DEFAULT_HASH provides a default `--hash=<hash>`
300 option for the individual tests if no explicit `--hash=<hash>` option has
303 If TESTLIB_GIT_DEFAULT_HASH is not the empty string ("") and is neither
304 `sha1` nor `sha256` a fatal error will occur.
306 If TESTLIB_GIT_DEFAULT_HASH is set to `sha256` and the version of Git is
307 not at least version 2.29.0, a fatal error will occur.
309 - TEST_NO_CREATE_REPO
311 Since the focus of the testing library is on Git or Git-related software
312 all "trash directory...."s are initialized as a non-bare empty Git
313 repository by default. Setting `TEST_NO_CREATE_REPO` _BEFORE_ sourcing
314 the `test-lib.sh` testing library file will instead result in the "trash"
315 directory for the test just being an empty directory instead. The
316 function `test_create_repo` can always be used to easily create a
317 repository (or two or more) if desired.
321 Set automatically or explicity using the `--color` or `--no-color` options.
322 It will be the empty string if "no color" is in effect or it will be the
323 string "--color" if "color" is in effect. It can therefore be directly
324 used as an argument to git commands when producing debugging output.
328 If set to a non-empty value before calling the `test_done` function and
329 zero tests have been run, then the value will be appended to the output
330 `1..0 # SKIP ` plan line. If the test matches `$TESTLIB_SKIP_TESTS` when
331 `./test-lib.sh` is sourced, `skip_all` will be set to a suitable value and
332 `test_done` called immediately.
336 Will automatically be set to empty unless the detected Git version is at
337 least version 2.9 in which case it will be set to the value
338 `--allow-unrelated-histories`. Inteded for use with `git merge`. The
339 `test_merge` function automatically supplies this.
343 This variable _must_ be set to a non-empty value _before_ sourcing the
344 `test-lib.sh` script. Other than checking for non-empty, there are no
345 requirements for the value. It is only used by the `-h`|`--help` options
346 (which are parsed when `test-lib.sh` is sourced). The only formatting
347 that occurs before it is output is that leading and trailing blank lines
348 are trimmed off. Recommended format is like a Git check in comment with
349 one summary line and then any optional body separated from the summary
350 line by one blank line if present. Ideally it should explain enough about
351 the test so that if the test is failing someone other than the test author
352 can debug the problem.
354 - test_external_has_tap
356 The default setting for this variable is `0` and should normally be left
357 set to `0`. Setting it to any other value causes the output of test plan
358 and test result TAP lines to be suppressed. It can be useful when one
359 testing library test is using the testing library to run a set of tests
360 thereby preventing the recursive testing library invocation from upsetting
361 the TAP output of the parent.
365 This will be set to either `sha1` or `sha256`. It indicates which hash
366 algorithm Git will use for `git_init` and `test_asv`. The initial value
367 will be taken from `TESTLIB_GIT_DEFAULT_HASH` with `sha1` being used if
368 that is not set. The `--hash=<hash>` option can be used to run a test with
369 a specific hash algorithm (overriding any `TESTLIB_GIT_DEFAULT_HASH`
370 value). However, an error will be generated if an attempt is made to use
371 the `sha256` algorithm with a version of Git prior to the 2.29.0 release.
375 Initially unset. Records the last unix epoch seconds value that was used
376 by the `test_tick` function to set both the `GIT_COMMITTER_DATE` and
377 `GIT_AUTHOR_DATE` variables. Note that the `test_tick` function always
378 appends " -0700" to the value of "$test_tick" when constructing the value
379 for `GIT_COMMITTER_DATE` and `GIT_AUTHOR_DATE`.
381 If `test_tick` is unset or empty when the `test_tick` function is called
382 then it will be set to 1112911993 (2005-04-07 15:13:13 -0700) and that
383 value used. Otherwise it will be advanced by 60 and the resulting value
386 See also the description of the `test_tick` utility function below.
390 This variable is set automatically by the ". ./test-lib.sh" line.
392 The value of "$this_test" is the basename of the current test file upto
393 but excluding the first "-" in the name. For example "t/t1000-it-works.sh"
394 has a "$this_test" value of "t1000". Since tests are expected to be
395 located in the "$TEST_DIRECTORY" directory, any test that needs to make use
396 of auxilliary files should locate them in a "$TEST_DIRECTORY/$this_test"
397 directory that is checked in alongside the test so in the case of the
398 aforementioned "t/t1000-it-works.sh" this might be the directory structure
399 if two auxilliary files "aux1" and "aux2" were present:
405 And all three of them would be checked into version control.
412 When `test-lib.sh` is sourced, it parses the options passed to the test. When
413 running tests via the Makefile, setting "TESTLIB_TEST_OPTS" will pass those
414 options to _every_ test run by the Makefile.
416 * --chain-lint | --no-chain-lint
418 Set TESTLIB_TEST_CHAIN_LINT to 1 (`--chain-lint`) or 0 (`--no-chain-lint`).
419 See the description of TESTLIB_TEST_CHAIN_LINT above for details.
421 * --color | --no-color
423 Override automatic color detection (color output is normally enabled when
424 output is to a terminal that supports color sequences) and activate
425 (--color) or deactivate (--no-color) color output. Note that when the
426 environment variable `HARNESS_ACTIVE` is non-empty, color is ALWAYS
427 SUPPRESSED when TAP lines are output and the `#` first character of
428 comment lines also has its color suppressed in that case.
432 Normally any command preceded by `test_debug` becomes a nop. (The testing
433 library itself does not use `test_debug` anywhere.) With `--debug` these
434 lines are activated and actually run instead of being nops.
436 Using this option also prevents the temporary trash directory from being
437 removed even for a successful test.
441 The value of <hash> must be either `sha1` or `sha256`. If this option is
442 not used, the `TESTLIB_GIT_DEFAULT_HASH` environment variable will select
443 the hash to be used and if that's not set, `sha1` will be used.
445 If this option is used, then the `TESTLIB_GIT_DEFAULT_HASH` environment
446 variable will be set to <hash> and exported to fully effect an override.
448 Attempting to use `sha256` with a version of Git prior to version 2.29.0
449 will cause a fatal error.
453 Show the `$test_description` to standard output and `exit 0`. NO TAP lines
454 of any kind are output when using this option (nor are the
455 $test_description lines prefixed with the `#` comment character).
459 When `--immediate` is used, if a test_expect_success... etc. call fails
460 then `exit 1` immediately after showing the `not ok` and quoted (with
461 leading `#` characters) test script.
463 * -l | --long | --long-tests | --expensive
465 Set and export `TESTLIB_TEST_LONG=t`. The `EXPENSIVE` prereq checks for
466 this and only succeeds when `TESTLIB_TEST_LONG` is non-empty.
468 * -q | --quiet | --no-quiet
470 Suppresses some output lines such as "ok" (only when NOT running under a
471 harness) and various output from test_external... functions. Mainly
472 useful to produce more compact output when NOT running under a TAP harness.
474 Ignored if `--verbose` is also active.
476 When combined with `--run` the skipping test message(s) that would
477 otherwise be output when tests are skipped by a `--run` option are
478 suppressed unless $HARNESS_ACTIVE.
480 Use of `--run` will implicitly partially activate the `--quiet` option
481 unless the `--no-quiet` option is also given. The "partially" part means
482 that it will only suppress the skipping test messages but not any other
483 messages that would ordinarily be suppressed with an explicit `--quiet`.
485 Use an explicit `--quiet` or `--no-quiet` to leave "partially quiet" mode
486 when the `--run` option is in use.
490 Causes the "trash directories" to be created as subdirectories of `<path>`
491 instead of as subdirectories of `$TEST_OUTPUT_DIRECTORY`. Specifying a
492 RAM disk location as `<path>` can significantly speed up testing.
494 * --run=<run-list> | -r <run-list>
496 The <run-list> must contain comma or whitespace separated "selectors". But
497 note that when using "TESTLIB_TEST_OPTS" only comma separated will work.
499 A selector must match either !?\d+ or !?\d*-\d* which is to say it's either
500 <num> or <num>- or -<num> or <num>-<num> optionally prefixed with `!`. The
501 "<num>-" form is a shortcut for <num>-infinity and the "-<num>" form is a
502 shortcut for "1-<num>".
504 If the first (or only) selector is prefixed with `!` then it's an exclusion
505 list otherwise it's an inclusion list.
507 The "selectors" are processed in left to right order and if the number of
508 the subtest matches the range it will be run if the selector was NOT
509 prefixed with `!` otherwise it will be excluded.
511 An "inclusion" list includes and runs ONLY those subtests that are
512 specifically matched by a selector NOT prefixed with `!`. (Note that
513 a subsequent selector can again exclude a previously included match and
516 An "exclusion" list runs everything except tests that are specifically
517 excluded by a match. In other words an "exclusion" list is just a shortcut
518 to avoid adding a "1-" selector to the front of <run-list>.
520 For example, "1-4,!3" will run only subtests 1, 2 and 4 but so will
521 "1,2,4". On the other hand, "!3,1-4" will run ALL subtests except 3
522 (because it's equivalent to "1-,!3,1-4") as will just "!3".
524 Using the `--run` option (with a non-empty list) implicitly partially
525 activates the `--quiet` option (see the desciption of the `--quiet`
526 option) and moves the test plan line to the end (if there was a `test_plan`
527 call at the beginning), but adding an explicit `--no-quiet` (or `--quiet`)
528 will override all that.
530 * --tap-only | --no-tap-only
532 Set `TESTLIB_TEST_TAP_ONLY` to `1` or `0` respectively. See the
533 description for `TESTLIB_TEST_TAP_ONLY`. Use of ``--tap-only`` is
534 incompatible with ``--verbose`` unless ``--verbose-log`` is also used.
535 Does _not_ cause `TESTLIB_TEST_TAP_ONLY` to be exported (unless it already
540 All standard output and standard error from running the test is copied to
541 $TEST_OUTPUT_DIRECTORY/test-results/$(basename $0 .sh).out and as a side
542 effect standard error is redirected to standard out. The final exit code
543 is written to $TEST_OUTPUT_DIRECTORY/test-results/$(basename $0 .sh).exit
546 This is accomplished by running another copy of the test script and
547 redirecting both its standard output and standard error through the `tee`
548 utility and when that copy exits, so too will the "tee" process and it will
549 then use the same exit code as its exit code (unless something went wrong
550 and no exit code was recorded in which case it exits with a result of 1).
552 The environment variable TESTLIB_TEST_TEE_OUTPUT_FILE is set (and exported)
553 to the name of the file the output is being "tee"'d into.
557 When `--verbose` is used the `test_pause` function will actually pause
558 a test by entering a subshell and `test_external_without_stderr` will show
559 both stdout and stderr output on failure. Test scripts passed to the
560 test_expect_success... functions are run with standard output and standard
561 error left unchanged (normally it's sent to /dev/null).
563 Automatically enabled by `-x`.
565 Note that `--verbose` is not allowed when $HARNESS_ACTIVE.
569 Enables the `--tee` option and arranges for the standard output and
570 standard error that are enabled by `--verbose` to end up in the same log
571 file that `--tee` creates while not disrupting the output going to the
574 This setting supersedes `--verbose` in that if both are given the output
575 ends up in the log but not on normal standard output or standard error.
577 Also if `--verbose-only=...` is used together with this option then only
578 the subtests selected by the `--verbose-only` option value will have their
579 output captured into the log -- the other subtests will have it sent to
582 Note that `--verbose-log` *IS* compatible with $HARNESS_ACTIVE.
584 * --verbose-only=<run-list>
586 The current subtest number (sub tests are numbered starting at 1) is
587 matched against <run-list> (see description for `--run` option) and if
588 there's a match then `--verbose` is active for that subtest and if there is
589 *NOT* a match then `--verbose` is NOT active for that subtest.
591 For example, `--verbose-only="1-3,5"` will enable verbose for
592 subtests 1, 2, 3 and 5 and will DISABLE verbose mode for subtests
593 4,6,7,8,9,10,11,12,13,14,15 etc.
595 Using this option with a non-empty <run-list> value completely supersedes
596 any explicit `--verbose` option (before or after). (Using it with an empty
597 value just silently has no effect.)
599 Note that `--verbose-only=...` is not allowed when $HARNESS_ACTIVE unless
600 `--verbose-log` is also used.
604 Set the `--verbose` option and temporarily enable the shell's `set -x`
605 option while evaluating the scripts passed to test_expect_success....
607 Note, however, that tracing will be suppressed unless verbose is also
608 enabled. Normally that will always be the case since `-x` also enables
609 `--verbose`. However, if `--verbose-only` is used then verbose will be
610 toggled on a per-subtest basis and as a result the `-x` option will only
611 be in effect for subtests selected by the `--verbose-only=...` option.
613 Note that since this option sets `--verbose` it's not normally allowed
614 when $HARNESS_ACTIVE but setting `--verbose-log` will allow it and send the
615 output to the log file.
622 These are functions that are intended to ultimately emit either a TAP test plan
623 line or a TAP test (aka "subtest") result line.
625 These functions are intended to be called from the main body of a test script
626 (e.g. t-????-*.sh) after having sourced the `./test-lib.sh` script.
632 The purpose of these functions is to output the single, required, TAP test
633 plan line. Additionally the `test_done` function performs some extra work.
637 Outputs a TAP test plan line.
639 MUST be called AFTER any test result functions. (TAP supports outputting
640 a test plan line either _before all_ test result lines or _after all_
643 It may be tempting to output the plan line without using this function.
644 Don't do that. Failure to use this function will result in test cleanup
645 functions failing to run and the test results aggregation provided by the
646 testing library breaking.
648 If `$test_external_has_tap` is set to something other than `0` (the
649 default) then no test plan line will be output but normal test cleanup will
652 If `$skip_all` is set to a non-empty value AND 0 tests were run then the
653 `$skip_all` message will be appended to a `1..0 # SKIP ' line (which is,
654 again, suppressed if `$test_external_has_tap` is not `0`).
656 The `test_done` function then calls either `exit 0` if there were no
657 unexpected failures or `exit 1` if there were. Just before calling
658 `exit 0` (but _not_ `exit 1`) the `test_at_end_hook_` function is called
659 which, by default, always succeeds and does nothing but can be re-defined
660 by a test script if such a hook is needed.
662 Note that if the test name appears in `$TESTLIB_SKIP_TESTS` (see the
663 description above) then `$skip_all` is automatically set to a suitable
664 value and `test_done` is called immediately when `./test-lib.sh` is
667 The `test_done` function DOES NOT RETURN! It _always_ ends up calling
668 exit with an exit status and therefore _never_ returns to the caller!
670 - test_plan <count> <count==0> <optional> <alternate> <skip> <msg> <here>
672 This function allows a test plan line to be output first. If `$skip_all`
673 is set it just passes through to `test_done`. Otherwise it outputs the
674 test plan line using the first argument as the total test count and then
675 sets `test_wrote_plan_count=$1` so that the required subsequent call to
676 `test_done` does not write a second test plan line. (But if the value of
677 `$test_external_has_tap` is not 0 it will not output the test plan line.)
679 If the argument is 0, then `skip_all` will be set to the following
680 arguments (or a suitable default if there are no following arguments) and
681 `test_done` will be called immediately (which will cause an immediate
682 exit of the test script at that point).
684 Using `test_plan` is inconvenient in that the number of subtests in the
685 file must be known in advance. Sometimes that's not possible if the number
686 of tests to be run varies depending on test conditions. Other times it's
687 just damn inconvenient to have to count all those subtests. ;)
689 To document that a `test_plan` number has been deliberately omitted, a
690 single argument of `?` can be used (e.g. `test_plan '?'` -- the "?" needs
691 to be quoted in some fashion to avoid wildcard expansion). Using the
692 value "`?`" has the same effect as not having any `test_plan` line at all
693 but documents the intent not to have one.
695 If `test_plan` is never called (and the entire test was not skipped as a
696 result of a `TESTLIB_SKIP_TESTS` setting) a message urging addition of a
697 `test_plan` call will be output. With support for a count of `?`, it's
698 always possible to add a `test_plan` call to the script.
700 Outputting the test plan line in advance makes those fancy test harnesses
701 (like `prove`) show a "1/n" progress rather than "1/?" which is nice to
702 have and that is the reason the `test_plan` function is provided (and the
703 prodding message output if it's not been called at `test_done` time).
705 Also if a different number of tests are run than are planned the overall
706 test will fail with an error which provides yet another test of the test
707 script itself which is not available without a call to `test_plan`.
709 The `test_done` function MUST STILL BE CALLED at the end of the test script
710 even when `test_plan` is used at the beginning otherwise test aggregation
711 and test cleanup will fail to take place! The only exception is when a
712 `test_plan 0` (possibly with optional arguments) call is _always_ used as
713 it will implicitly call `test_done` before returning (and therefore never
714 actually return at all).
716 Note that if it's not already clear from the description, use of the
717 `test_plan` function is completely optional, but use of `test_done` is not.
720 Test Result Functions
721 ~~~~~~~~~~~~~~~~~~~~~
723 The purpose of these functions is to output one `ok` or `not ok` TAP test
724 result line each time they're called.
726 All of these functions take an optional first argument which is a whitespace
727 and/or comma separated list of prerequisites that must be present or the
728 test will be skipped. This is shown as `[<prereqs>]`. Each prereq in the
729 list may be optionally prefixed with a single `!` to require the item *not*
732 All of these functions expect the next argument to be a brief test description
733 that will be used in the `ok` or `not ok` output line. This is shown as
736 The "external" functions expect two additional arguments which are the
737 "external" command to be run and its single argument (use `eval` and a suitable
738 string to work around this requirement for exactly two arguments). These are
739 shown as `<cmd>` `<arg>`.
741 The non-"external" functions expect one additional argument which is the
742 test script to be "eval"'d. However, if the test script argument is `-` then
743 the test script will be read from standard input (allowing use of a HERE doc to
744 simplify quoting issues), but this is slightly inefficient so avoid using it
745 unless it's really needed. This is shown as `<script>`. Note that scripts are
746 normally "eval"'d inside a subshell to avoid side-effects and uncaught
747 failures, but this can be altered with the `TESTLIB_TEST_NO_SUBSHELL` variable
748 (see description above).
750 Test scripts are "eval"'d as the last line of a function so they may exit early
751 by using a `return` statement or, if `$TESTLIB_TEST_NO_SUBSHELL` is _NOT_ set
752 (which is the default), by using an `exit` statement.
754 - test_expect_success [<prereqs>] <desc> <script>
756 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
757 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
758 it succeeds (exit code is 0) then this subtest succeeds and outputs an "ok"
759 line otherwise it outputs a "not ok" line.
761 - test_expect_failure [<prereqs>] <desc> <script>
763 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
764 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
765 it fails (exit code is _not_ 0) then this subtest outputs a "not ok # todo"
766 line. If the script succeeds a "ok # todo" line is output which will
767 ultimately produce some warnings and/or complaints about a "known breakage"
768 vanishing or "todo passing". See the `test_tolerate_failure` function to
771 - test_tolerate_failure [<prereqs>] <desc> <script>
773 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
774 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
775 it fails (exit code is _not_ 0) then this subtest outputs a "not ok # todo"
776 line. If the script succeeds a plain "ok" line is output (which avoids
777 any complaints about "vanishing" breakages). However, if the variable
778 `TESTLIB_NO_TOLERATE` is non-empty all `test_tolerate_failure` calls are
779 turned into `test_expect_success` calls instead (meaning a failure causes
780 a regular `not ok` to be output instead of a `not ok # todo`).
782 This function is useful for testing things that probably don't work the
783 way they should but are reluctantly tolerated and if they do start working
784 better no fuss should be made about the test suddenly passing.
786 It's a nice way to document those "yes, we still have this ugly thing here
787 that we wish we didn't" behaviors. It can also be used to distinguish
788 "tier 1" supported platforms (those which pass all tests even when
789 `TESTLIB_NO_TOLERATE=1` is set) and "tier 2" supported platforms (those
790 that pass all tests as long as `TESTLIB_NO_TOLERATE` is _not_ set). (Tier
791 3 would be those platforms that always fail at least one test.)
793 - test_external [<prereqs>] <desc> <cmd> <arg>
795 Behaves just like `test_expect_success` with a script of "<cmd> <arg>"
796 (assuming <cmd> and <arg> are first suitably quoted) except that stdout is
797 always left on stdout even without `--verbose`.
799 Use "eval" for <cmd> and pass a script as <arg> to use as a substitute
800 `test_expect_success` that does not redirect stdout or to escape the
801 two argument restriction.
803 - test_external_without_stderr [<prereqs>] <desc> <cmd> <arg>
805 *IMPORTANT*: This function outputs _two_ test result lines!
807 If using the `test_plan` function then each `test_external_without_stderr`
808 function call counts as *TWO* planned tests!
810 It works just like `test_external` (in fact, it calls test_external first)
811 and then it performs a second subtest to see if _anything_ was output to
812 stderr by "<cmd> <arg>" in which case the second subtest fails otherwise
813 it succeeds. If the `test_external` first subtest ends up being skipped
814 then nothing will be output to stderr and the second subtest will always
815 succeed (it will not be skipped). In other words, the second subtest
816 result line is always a plain "ok" or "not ok" and never has any "# skip"
817 or "# todo" added to it even when the first subtest result line does.
819 Note that the special prerequiste `LASTOK` behaves as expected here --
820 that is if the first `test_external` call is skipped then its state
821 _does NOT change_ regardless of the outcome of the second "stderr"
822 subtest check. In addition, if the `test_external` test is _not_
823 skipped then the `LASTOK` prerequisite is guaranteed to only be true
824 provided _both_ the `test_external` call suceeds _and_ no "stderr" output
825 is generated. In other words, it just does the right thing here so don't
826 worry about it and go ahead and use it.
829 Standard Test Prerequisites
830 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
832 The test result functions all take an optional first argument which is a comma
833 and/or whitespace separated list of positive or negative (prefixed with `!`)
834 prerequisites for the test to be run. *All* of the listed prerequisites for
835 each test _must_ be satisfied or it will be skipped.
837 Additional prerequisites may be set using the `test_set_prereq` function but
838 there are a number of pre-defined prerequisites that may be tested:
842 The `git var GIT_AUTHOR_IDENT` command succeeds when both `GIT_AUTHOR_NAME`
843 and `GIT_AUTHOR_EMAIL` are unset.
847 The backslash character (`\`) is allowed in path/file names. Normally
848 defined _except_ for `MINGW` and `CYGWIN`.
850 - `CASE_INSENSITIVE_FS`
852 File names are case-insensitive (i.e. "foo" and "Foo" refer to the same
857 Whether or not the `run_with_limited_cmdline` function appears to work
858 (it does `ulimit -s 128` in a subshell and runs the command assuming that
859 will limit the size of the command line argument list).
863 Set when the shell's `$COLUMNS` variable can be successfully set to 1.
867 The CygWin environment is active.
871 The process ID before and immediately after an `exec` call stays the
872 same. Normally defined for all but `MINGW`.
876 The `TESTLIB_TEST_LONG` variable is non-empty (see the `--long-tests` aka
877 `--expensive` test option).
881 True if Git thinks the executable bit is supported properly (i.e.
882 Git's `core.filemode` is set to true after `git_init`).
886 This prerequisite is never cached. It evaluates to true if the current
887 value of the `test_hash_algo` variable is `sha1`.
891 True if the Git executable being used to run the tests is at least version
892 2.29.0. If true it means `git init --object-format=sha256` can be used.
893 (As well as `git init --object-format=sha1` if necessary.)
897 The grep command strips off CR. Normally only defined for `MINGW` and
902 This is a special prerequisite that always succeeds for the first subtest
903 in a test script and for every subtest thereafter for which the most
904 immediately preceeding non-skipped test was "ok" (regardless of the
905 presence any "# TODO").
907 In other words, this can be used to skip a subtest if the preceeding
908 subtest did not succeed. Since "skipped" tests do not change the state
909 of this prerequisite a single failure can cause a long chain of following
910 subtests that use this prerequisite to be skipped.
914 The MinGW environment is active.
918 The native environment uses CR+LF line endings (normally only defined
923 The `id -u` command succeeded and returned a non-0 value.
927 The file system automatically converts NFD unicode to NFC unicode.
931 The `mkfifo` command is supported and works.
935 Sane, POSIX-style permissions. *Not* normally defined for `MINGW`.
939 File permission bits are honored (i.e. after doing `chmod a-r file` then
940 a subsequent `cat file` will fail). When running as root these "sanity"
945 The sed command strips off CR. Normally only defined for `MINGW` and
950 Symbolic links created using `ln -s` work.
954 The result of `test -x /usr/bin/time` is true.
961 These are convenience functions and while most of them are provided to be
962 called from the body of a script passed to one of the test_expect_...
963 functions, several of them can also be used from the main body of a test
964 script (`say`, `say_color`, `test_debug` and `test_pause` being prime examples
967 These functions are intended to be called only _after_ the `./test-lib.sh`
968 script has been sourced. See the `README-TESTLIB-TG` file for additional
969 available utility functions that are TopGit-specific.
971 - die <optional> <message> <here>
973 Can be used to exit a test script on an unexpected error, should be used
974 after a `||` to preserve the exit code as in:
976 perform_some_function perhaps with arguments || die
978 If the <optional> <message> <here> is present, it will be included in the
979 fatal error message (along with the exit code).
981 - error <msg> <goes> <here>
983 Cause an instant test `Bail out!` error (including <msg> <goes> <here>).
985 The original Git version of this function just did an exit after showing
986 the error without using `Bail out!`, but these errors are terminal and the
987 entire test suite should be stopped if one occurs and that is what will now
988 happen when this function is called.
990 - git_init [<arg>...]
992 Runs `git -c init.defaultBranch=master init "$@"`. This is a convenience
993 function that provides reproducibility in the face of changing Git defaults
994 in the future. Replace any "git init" calls with "git_init" in tests that
995 depend on the name of the initial branch being "master" to avoid future
998 When running Git version 2.29.0 or later, the
999 `--object-format=$test_hash_algo` option will also be passed to `git init`
1000 immediately preceding the `"$@"` arguments. Note that to override the
1001 automatically passed `--object-format=$test_hash_algo` option, simply
1002 provide another `--object-format=<hash>` option. As far as `git init` is
1003 concerned, the last `--object-format=<hash>` option wins.
1005 - list_contains <commalist> <pattern>
1007 Returns true if comma separated list <commalist> contains any item that
1008 matches the filename matching wildcard <pattern>.
1010 A shortcut for those uncomfortable with the `case` statement.
1012 - sane_unset <varname>...
1014 An alias for `unset <varname>... || :` to workaround non-conforming shells
1015 that return a non-zero status if any of the variables were not already set.
1017 - say <msg> <goes> <here>
1019 A convenient shortcut for `say_color info <msg> <goes> <here>`
1021 - say_tap <msg> <goes> <here>
1023 A convenient shortcut for `say_color_tap info <msg> <goes> <here>`
1025 - say_color <color> <msg> <goes> <here>
1027 Possible <color> values are:
1029 * "" (the empty string)
1030 plain text with no color, suppressed when `--quiet` in effect
1032 used for testing failures, typically shown in red
1034 used for test skip messages, typically shown in a light color
1036 warnings are shown in this, typically yellow (or brown)
1038 used when a complete test script passes, usually green
1040 info messages and non-harness TAP messages, usually a light color
1042 turns off color, used internally, should not be used by callers
1044 Using anything else for <color> ends up outputting plain text followed by
1045 the color reset code.
1047 If `--quiet` is in effect, anything output using `""` (the empty string)
1048 as the color name will be supressed entirely.
1050 If `--no-color` is in effect (perhaps implicitly) the color code sequences
1051 are omitted from the output.
1053 If `$HARNESS_ACTIVE` and the first character of <msg> is `#` then that
1054 single character is always output _without_ any preceding color codes (the
1055 first character to be in color will be the following one unless, of course,
1056 the color name is `""` or `--no-color` is in effect).
1058 - say_color_tap <color> <msg> <goes> <here>
1060 When `$HARNESS_ACTIVE` this always outputs "<msg> <goes> <here>" _without_
1061 any color sequences regardless of the presence of the `--quiet` option.
1063 If there is no `$HARNESS_ACTIVE` then this call is identical to `say_color`
1064 with the exact same arguments.
1068 Output the value of the hash algorithm specific value cached with the
1069 id <id> to standard output. This will typically be used as
1070 `"$(test_asv <id>)"`. The test_v_asv function does not cause a subshell to
1071 be created and is therefore recommended since it's more efficient.
1073 A fatal error will occur if no value has been set for <id> using the
1074 `test_asv_cache` function.
1076 See further details in the description for `test_asv_cache`.
1078 - test_asv_cache <database>
1080 Read each line of <database> using the shell's `read` function into three
1081 variables. The first is the <id> to be used with `test_asv` and
1082 `test_v_asv`, the second is either `sha1` or `sha256` and the rest of the
1083 line is the algorithm specific value to associate with that database entry.
1085 Blank lines and lines with the first non-space character a `#` are ignored.
1087 If the <database> is given as a single hyphen (-) then the database lines
1088 will be read from standard input.
1094 01 sha1 value for sha1
1098 # this is a comment too
1100 v33 sha1 hash is sha1
1101 v33 sha256 hash is sha256
1104 The value of <id> must be a valid shell variable name after having a letter
1105 prefix added. In other words, if <id> is "5" then prefixing a letter such
1106 as "z" produces "z5" which is a valid shell variable name. In general,
1107 the <id> must consist of only alphanumeric characters plus underscore (`_`).
1109 In the case of duplicate <id> plus hash keys, the last one wins.
1111 Note that since the shell's read function is used with the standard setting
1112 for `$IFS` extra whitespace may be used around the <id> and hash name and
1113 the value will have whitespace trimmed from its beginning and end. In
1114 addition, since the shell's read function supports line continuation with
1115 a backslash (`\`) as the last character on a line, the value for each entry
1116 may be split across lines using that feature if desired.
1118 The `test_asv_cache` function may be invoked more than once. Subsequent
1119 invocations add to the cache (replacing any items with the same <id> plus
1120 hash key). Since each individual test runs in its own subshell (unless
1121 `TESTLIB_TEST_NO_SUBSHELL` has been used), the effects of calls to
1122 `test_asv_cache` from inside an individual test will not persist after that
1123 test has completed (successfully or not).
1125 Providing a hash name of other than `sha1` or `sha256` will produce a
1126 fatal error. Note that it is *not* an error to provide a database with
1127 `sha256` entries even when running a version of Git prior to 2.29.0 -- they
1128 will just never be produced by `test_asv` or `test_v_asv` in that case.
1130 After executing the above example, the following code can be used:
1132 test_asv 01 # outputs "value for sha1" or "value for sha256"
1133 test_asv v33 # outputs "hash is sha1" or "hash is sha256"
1134 test_v_asv ex1 01 # sets variable `ex1` to "value for sha1"
1135 # or "value for sha256"
1136 test_v_asv ex2 v33 # sets variable `ex2` to "hash is sha1"
1137 # or "hash is sha256"
1139 Which value is chosen by `test_asv` or `test_v_asv` depends solely on the
1140 current value of `test_hash_algo` at the time `test_asv` or `test_v_asv`
1145 This function is called by `test_done` if there are no unexpected failures.
1146 By default it's defined to be just `:` but should be re-defined by each
1147 test script file as needed. Note that it _will_ be called when the entire
1148 test script is skipped with a `1..0` plan line.
1150 - test_clear_when_finished
1152 Any `test_when_finished` items already scheduled in the subtest script
1153 so far are cleared (i.e. removed). Calling `test_clear_when_finished`
1154 before `test_when_finished` has first been called silently does nothing
1161 $TESTLIB_TEST_CMP <arg>...
1163 Useful because `diff -u` will be used where supported instead of `cmp`.
1164 See also `test_diff` for an alternative.
1166 - test_cmp_bin <arg>...
1172 There's no real compelling reason to use this other than to look nice
1175 - test_cmp_rev <expect-rev> <actual-rev>
1177 Save output of `git rev-parse --verify` on each argument to a file and
1178 then run `test_cmp expect.rev actual.rev` on the resulting two files.
1180 The temporary files are actually placed in TRASHTMP_DIRECTORY so there's
1181 no need to be concerned about contaminating a working tree that's being
1182 used for testing or needing to do a `cd` to avoid that.
1184 - test_commit [--notick] [--signoff] <message> [<file> [<contents> [<tag>]]]
1186 Create a new Git commit with "<message>" as the commit message optionally
1187 having a signed-off-by line (`--signoff`) using the next tick time by
1188 calling `test_tick` (unless `--notick` is used).
1190 <file> defaults to "<message>.t", <contents> and <tag> default to
1193 The "<contents>" will be written (using `printf '%s\n'`) to <file> which is
1194 then added and committed using <message> and then tagged using <message>.
1196 It will not be tagged if <tag> is the empty string or contains any
1197 whitespace or invalid ref name characters. (Including at least one '^' or
1198 '~' is a simple way to keep a single "word" that's a valid filename while
1199 at the same time suppressing creation of a tag while not needing to
1200 supply anything other than the <message> argument.)
1202 - test_config [-C <dir>] <name> <value> [...]
1204 Run `git [-C <dir>] config <name> <value> [...]` now and schedule a
1205 `git [-C <dir>] config --unset-all <name>` (via `test_unconfig`) using
1206 `test_when_finished`.
1208 This function allows for easy temporary configuration changes that are
1209 automatically undone when the current test finishes.
1211 - test_config_global <name> <value> [...]
1213 Run `git config --global <name> <value> [...]` now and schedule a
1214 `git [-C <dir>] config --unset-all --global <name>` (via `test_unconfig`)
1215 using `test_when_finished`.
1217 This function allows for easy temporary global configuration changes that
1218 are automatically undone when the current test finishes.
1220 - test_copy_bytes <count>
1222 This is a shortcut for a call to the `dd` utility (unlike the Git version
1223 which uses perl) to transfer <count> bytes (or up to EOF) from stdin to
1226 - test_create_repo <path>
1230 (mkdir -p "<path>" && cd "<path>" && git_init)
1232 Except that an empty template directory is used and if that doesn't succeed
1233 in suppressing creation of a hooks directory the hooks directory will be
1234 renamed to `hooks-disabled`.
1236 Note that since Git will happily reinitialize an existing Git repository
1237 this command can succeed on a pre-existing repository without altering its
1240 - test_debug [<cmd> [<arg>...]]
1242 This function does nothing (it's a fancy comment) _unless_ `--debug` is
1243 used in which case <cmd> [<arg>...] is executed with standard output and
1244 standard error redirected to the original standard error of the test
1245 script (in other words, you don't need to add `--verbose` to see the
1246 output when `--debug` is active).
1248 Note that this is different than the original Git version that only takes
1249 a single argument and "eval"s it (while not guaranteeing anything about
1250 where the output goes). To get the Git "eval" equivalent, use `eval` as
1251 the first <arg> and the string to be "eval"'d as the second.
1253 - test_diff <arg>...
1257 git --no-pager -c core.abbrev=16 diff --no-color --exit-code \
1258 --no-prefix --no-index <arg>...
1260 Always gives nice diffs and they're always `-u` diffs.
1261 This should be usable as a drop-in replacement for `test_cmp` everywhere
1262 and its output should be deterministic as no dates or times are included.
1264 - test_dir_is_empty <dirpath>
1266 Fail verbosely (with directory contents) unless <dirpath> is an existing
1267 and empty (only `.` and `..` entries) directory.
1269 - test_env [VAR=VAL]... <cmd> <arg>...
1271 In a subshell set and export each `VAR=VAL` and then run `<cmd> <arg>...`.
1273 Obviously, since it's a subshell no variable changes will persist, but this
1274 makes it easy to temporarily change a variable while running some shell
1277 - test_expect_code <status> <cmd> [<arg>...]
1279 Runs `<cmd> <arg>...` and fails verbosely unless the resulting status code
1282 - test_get_temp [-d] [<name>]
1284 Create a new temporary file (or directory with -d) in the temporary
1285 directory $TRASHTMP_DIRECTORY with optional pattern prefix "<name>." and
1286 output the resulting unique name of the newly created empty file (or
1289 The $TRASHTMP_DIRECTORY is created on-demand the first time this function
1290 is called and is guaranteed to be outside of $TRASH_DIRECTORY but still
1291 unique to the test script currently being run.
1293 - test_have_prereq <PREREQ>...
1295 Test a list of zero or more whitespace and/or comma separated prereqs
1296 (the same as can be given to any of the test result functions). As with
1297 the test result functions, prefixing a prerequisite with a single `!`
1298 requires that it _not_ be present. Result code is 0 if the requirements
1299 are satisfied, non-zero if not.
1301 - test_line_count <op> <val> <file>
1303 Verbosely fails if `test $(( $(wc -l < "<file>") )) "<op>" "<val>"` fails.
1305 - test_match_signal <signum> <exitcode>
1307 Returns true if <exitcode> represents an exit from signal <signum> (which
1308 must be numeric). This function only exists to accomodate `ksh` which
1309 uses 256+<signum> instead of the POSIX 128+<signum> for signal exit codes.
1311 - test_merge <message> [<opt>...] <commit>
1315 test_tick && git merge -m "$@" && git tag "<message>"
1317 Except that `--allow-unrelated-histories` is automatically supplied for
1318 Git 2.9 or later and the tag is skipped if "<message>" is not tagish.
1320 - test_might_fail <cmd> [<arg>...]
1322 Just a shortcut for `test_must_fail ok=success <cmd> <arg>...`.
1324 This is useful when it's okay for <cmd> to succeed or for it to fail
1325 provided the failure is a "normal" (i.e. non-signal) failure.
1327 - test_must_be_empty <file>
1329 Verbosely complain (and fail) showing <file>'s contents if it's not empty
1330 (i.e. -s <file> succeeds).
1332 - test_must_fail [ok=<commalist>] <cmd> [<arg>...]
1334 Runs `<cmd> <arg>...` and verbosely converts all result codes into either
1335 0 or 1 where "normal" failures become 0 and others become 1.
1337 This function should be used when a failure is required but only if the
1338 failure is *not* the result of a signal, command not found or command not
1341 If <commalist> contains "success" then a result code of 0 is _not_
1342 converted to 1 (it stays 0). If <commalist> contains "sigpipe" then an
1343 exit due to `SIGPIPE` is converted to status 0 (otherwise it becomes 1).
1345 - test_path_is_dir <path> [<msg>]
1347 Fail verbosely (including <msg> if given) if `test -d <path>` fails.
1349 - test_path_is_file <path> [<msg>]
1351 Fail verbosely (including <msg> if given) if `test -f <path>` fails.
1353 - test_path_is_missing <path> [<msg>...]
1355 Fail verbosely (including <msg>... if given) if `test -e <path>` _succeeds_.
1359 Spawn an interactive `$SHELL` to pause testing until it exits -- causes an
1360 error if used without `--verbose`. Should be used for debugging only.
1362 Something like this can be used to leave it in a script (although this is
1365 test_debug eval "want_no_verbose || test_pause"
1367 which will cause the test in question to spawn a subshell whenever it's run
1368 with both the --verbose and --debug options.
1370 - test_seq [<start>] <end>
1372 <start> defaults to 1.
1374 Output each of the numbers from <start> to <end> inclusive (incrementing by
1375 1) each followed by a newline to stdout. If <end> is less than <start>
1378 - test_set_editor <path-to-editor>
1380 Set and export the `EDITOR` environment variable but using tricks so that
1381 <path-to-editor> is allowed to contain any value (including spaces, quotes,
1382 etc.) without needing any special quoting.
1384 - test_set_hash_algo <algorithm>
1386 The value of <algorithm> must be either `sha1` or `sha256`. Attempting to
1387 set a value of `sha256` with a version of Git prior to version 2.29.0 will
1388 cause a fatal error. The value of both the `GIT_DEFAULT_HASH` environment
1389 variable and the `test_hash_algo` variable are set by this function. The
1390 `TESTLIB_GIT_DEFAULT_HASH` environment variable is, however, left untouched.
1392 An implicit `test_set_hash_algo ${TESTLIB_GIT_DEFAULT_HASH:-sha1}` happens
1393 early during test setup.
1395 - test_set_prereq <PREREQ>
1397 Indicate that the test prerequisite <PREREQ> is available (should be a
1398 single word in ALL CAPS). It may then be used as a prereq for any of the
1399 test result functions or be explicitly tested with `test_have_prereq`.
1401 Note that for this to work inside a subtest and have the result be
1402 available to subsequent subtests, it needs to be preceded by
1403 `test_when_finished` (and also be run without that if `test_have_prereq` is
1404 intended to work subsequently *inside* the same subtest's subshell).
1406 - test_skip_or_die <mode> <msg>
1408 Don't use this function, it's confusing, use either `test_plan 0` or
1411 If <mode> is `auto` it's the same as `test_plan 0 <msg>`.
1413 If <mode> is `true` it's the same as `error <msg>`.
1415 If <mode> is anything else it's a more verbose call to `error`.
1419 Set and export `GIT_COMMITTER_DATE` and `GIT_AUTHOR_DATE` to the same
1420 value (initially 1112911993 -- the time of the first Git commit) increasing
1421 it by exactly 60 seconds each time `test_tick` is called.
1423 Note that when a test script starts running neither `GIT_COMMITTER_DATE`
1424 nor `GIT_AUTHOR_DATE` are initially set. It's not until after the first
1425 time `test_tick` is called (it's called implicitly by `test_commit` and
1426 `tg_test_create_branch` unless they're given the `--notick` option) that
1427 both of them are set (unless the test script otherwise sets them itself).
1429 See also the description of the `test_tick` variable above.
1431 - test_tristate <varname>
1433 Examines contents of `$<varname>` and sets `$<varname>` to one of `false`
1434 `true` or `auto`. An unset variable becomes `auto` and an empty string
1435 variable becomes `false` otherwise there should be no surprises except
1436 that anything that is not a Git boolean or `auto` is treated as `true`.
1438 The original version from the Git test library uses Git itself to do this
1439 conversion, but this implementation is strictly shell built-ins.
1441 - test_unconfig [-C <dir>] <arg>...
1443 Runs `git [-C <dir>] config --unset-all <arg>...` immediately but ignores
1444 any error resulting from the value(s) to be unset not actually being
1445 currently set at all (status 5).
1447 Can be used for global values simply by making the first <arg> be `--global`
1448 and the rest the global arg(s) to unset.
1450 - test_v_asv <varname> <id>
1452 Set <varname> to the value of the hash algorithm specific value cached with
1455 A fatal error will occur if no value has been set for <id> using the
1456 `test_asv_cache` function.
1458 See further details in the description for `test_asv_cache`.
1460 - test_v_git_mt <varname> <object-type> [<hash>]
1462 With an <object-type> of `blob`, `tree` or 'null`, set <varname> to the
1463 correct $test_hash_algo hash algorithm value for an empty Git
1464 <object-type> object. (A `null` <object-type> returns a string of 0's of
1465 the proper length for $test_hash_algo.)
1467 If <hash> is provided, it must be `sha1` or `sha256` or a fatal error will
1468 occur. If <hash> is non-empty it will be used as the hash algorithm
1469 instead of $test_hash_algo.
1471 An unknown <object-type> will produce a fatal error.
1473 - test_when_finished <cmd> [<arg>...]
1475 This function only works inside a test script!
1477 Causes <cmd> [<arg>...] to be run when the current test script (the
1478 <script> arg to a test result function) exits. May be used more than
1479 once to schedule multiple items.
1481 This function differs from Git's version in that <cmd> [<arg>...]
1482 are automagically quoted and stored in a "test_when_finished_<n>.sh"
1483 script that then gets sourced outside of any subshell that may have
1484 been created by the test script at test exit time. (The "<n>" value
1485 is the number of the subtest within the test script and the script
1486 itself is stored in the TRASHTMP_DIRECTORY.)
1488 This means that `TESTLIB_TEST_NO_SUBSHELL` has no effect on this
1489 functionality nor do subshells created implicitly (or explicitly) by
1490 the test script itself.
1492 Git's version uses eval. To get the same effect make the <cmd> be
1493 "eval". However, that's not recommended as the eval will take place
1494 outside of any test script subshells which means none of the variables
1495 created inside the test script's subshells will be accessible at that time.
1496 Instead use the normal form (it's much simpler to use anyway) which causes
1497 any variables to be evaluated immediately when `test_when_finished` is
1498 called to schedule the "when finished" operation.
1500 This approach, however, precludes using `test_when_finished` to perform
1501 some action using a variable that has not yet been set. In that case the
1502 "eval" form will be needed and probably `TESTLIB_TEST_NO_SUBSHELL` will
1503 also have to be set to true to make it work.
1505 See also `test_clear_when_finished`.
1507 - test_write_lines <arg>...
1509 A lazy shortcut for:
1511 printf '%s\n' <arg>...
1513 Although since it's more characters to type perhaps it's not so lazy. ;)
1515 - verbose <cmd> [<arg>...]
1517 Run `<cmd> <arg>...` and verbosely complain (showing command and args) if
1518 it fails, but the exit status on failure will be converted to 1.
1520 - wc [-c|-m] [-lw] [<file>...]
1522 This is a wrapper around the wc utility. Some systems produce formatted
1523 output with extra spaces in it that this wrapper conveniently removes.
1525 BEWARE! The output from the system's native wc will have all leading and
1526 trailing spaces removed and all sequences of internal whitespace converted
1529 Although it's not been disabled, avoid using wc with any <file> arguments!
1531 Parsing out the filenames from the result is error prone (and they will
1532 all end up on one line if there's more than one and any internal runs of
1533 whitespace in the name(s) will each be converted to a single space courtesy
1536 Instead redirect standard input to the file or files to be counted and
1537 use `cat` if there's more than one file, something like this:
1539 lines_1=$(wc -l < file1)
1540 lines_2_plus_3=$(cat file2 file3 | wc -l)
1542 To test line counts in a test script, use something like this:
1544 printf "%s\n" one two three four >testfile &&
1545 lines=$(wc -l <testfile) &&
1548 While the output of a command can be piped directly to wc that means
1549 the command's result status will be discarded (in favor of wc's if it's
1550 the last command in the pipeline) so that should generally be avoided
1551 (although the `cat` example above is probably okay to use).
1553 If you're using wc to count something on its standard input then you can
1554 safely use it just like the POSIX version without needing to be concerned
1555 about bogus whitespace showing up in the result which is the whole point
1556 of this wrapper in the first place.
1558 - write_script <file> [<shell-path>] < <script-text>
1560 Write a shebang line using <shell-path> (default is `$SHELL_PATH`) to
1561 <file> and then append <script-text> to it and then finally add execute
1562 permission to the resulting <file>.
1566 If <expletive> is omitted, `y` is used.
1568 Outputs "<expletive>" (and a trailing newline) 99 times.
1570 This is basically the same as the POSIX `yes` utility except that output
1571 is limited to 99 lines and it's implemented entirely as shell code.