| blob | 5d9f702e8089b160d426688bec43ec42f595e0bc |
1 Testing Library
2 ===============
4 The point of the testing library (available by sourcing the
5 test-lib.sh file) is to assist with rapidly writing robust
6 tests that produce TAP-compliant output. (For a quick primer
7 on TAP see the README file in the section "TAP - A Quick Overview".)
10 ----------------------------
11 Why Use the Testing Library?
12 ----------------------------
14 Because it:
16 * hopefully helps to allow tests to be written more quickly
17 * catches common failures (signal exits, not found etc.)
18 * avoids reproducing commonly needed code in every test
19 * prevents unwanted "other" lines from ending up in the TAP output
20 * allows test results to be aggregated without using "prove"
21 * supports easier test debugging
24 ---------------
25 Getting Started
26 ---------------
28 Check out the `README` file for a quick overview of everything in the test
29 directory and a brief overview of TAP.
31 This file, `README-TESTLIB`, serves as a reference of available variables,
32 options and functions in the testing library that can be used by test writers
33 to create tests. This file does not, however, provide any "how-to" instruction
34 as that's reserved for the `README-WRITING-TESTS` file.
36 The `README-WRITING-TESTS` file describes how to quickly get started writing
37 tests using the testing library and should be perused by anyone who finds the
38 simplistic example in the `README` file to be insufficient. It does not,
39 however, serve as a comprehensive reference for the testing library.
41 To quickly find the testing library function and/or variable needed refer to
42 the [Quick Reference] section below.
45 -------------
46 Running Tests
47 -------------
49 Note there is a difference between the way test scripts passed to the various
50 test_expect/test_tolerate functions behave compared to Git's testing library.
51 In this testing library the scripts passed to those functions are evaluated
52 inside a subshell (test_eval_inner_ uses (...) instead of {...}). This is
53 done intentionally so that any errors in the test itself do not abort the
54 testing process (and yes, that can happen). It also allows `set -e` to be
55 activated by default in the subshell (which it is). As a side effect, this
56 means that variables set in such test scripts DO NOT PERSIST after running the
57 test scripts. The `test_when_finished` function can be used to raise up some
58 information out of the (possibly nested) subshell as it runs after the
59 subtests's subshell has exited -- this is an enhancement compared to Git's
60 version of `test_when_finished` which doesn't support any subshells at all.
62 See the TESTLIB_TEST_NO_SUBSHELL variable to disable (NOT recommended) the use
63 of a subshell.
66 ---------------
67 Quick Reference
68 ---------------
70 Variable | Description
71 --------------------------------|----------------------------------------------
72 TESTLIB_DIRECTORY | directory where testing library code lives
73 EMPTY_DIRECTORY | empty directory with no write permission
74 TEST_DIRECTORY | directory with tNNNN-*.sh scripts to run
75 TEST_HELPER_DIRECTORY | directory with "helpers" (auto-added to PATH)
76 TEST_OUTPUT_DIRECTORY | test output ends up in here
77 TRASH_DIRECTORY | test specific temporary "trash" directory
78 TRASHTMP_DIRECTORY | test specific temporary `test_get_temp` dir
79 TESTLIB_SKIP_TESTS | test suite wide test/subtest skipping pats
80 TESTLIB_NO_TOLERATE | `test_tolerate_failure` means expect success
81 TESTLIB_TEST_CHAIN_LINT | can be used to disable `&&` chaining test
82 TESTLIB_TEST_NO_SUBSHELL | prevent subshell for subtests (do not use)
83 TESTLIB_TEST_TAP_ONLY | limit output to (almost) only TAP lines
84 TEST_NO_CREATE_REPO | suppress initial `test_create_repo .`
85 color | either "" (--no-color) or "--color" (--color)
86 skip_all | `test_done` skip value for `1..0` plan
87 test_auh | `--allow-unrelated-histories` if Git >= 2.9
88 test_description | must be non-empty before `. ./test-lib.sh`
89 test_external_has_tap | avoid changing -- will suppress TAP lines
90 test_tick | last commit timestamp used see same function
91 this_test | basename $0 | sed 's/-*//' (usually tNNNN)
93 TAP function | Description
94 --------------------------------|----------------------------------------------
95 test_done | uncounted plan line and counts (call last)
96 test_expect_success | expect success from a script (no #TODO)
97 test_expect_failure | expect failure from a script (with #TODO)
98 test_external | test external command
99 test_external_without_stderr | test external command & require empty stderr
100 test_plan | output TAP plan line (call first)
101 test_tolerate_failure | expect success or failure from a script
103 Utility function | Description
104 --------------------------------|----------------------------------------------
105 die | fatally exit the test (or subshell) right now
106 error | Bail out! now and kill all further testing
107 list_contains | check comma-separated list against wildcards
108 sane_unset | safe unset for broken sh implementations
109 say | say_color info ... shortcut
110 say_color | output in the named (special names) color
111 say_color_tap | if $HARNESS_ACTIVE always output w/o color
112 say_tap | say_color_tap info ... shortcut
113 test_at_end_hook_ | called by `test_done` default is `:`
114 test_clear_when_finished | make code run just after current subtest exit
115 test_cmp | fail if two text files do not match
116 test_cmp_bin | fail if two binary files do not match
117 test_cmp_rev | fail if two Git revspecs resolve differently
118 test_commit | make a new commit optionally tagging it
119 test_config | make test_when_finished reverted config mod
120 test_config_global | like `test_config` but `--global` config
121 test_copy_bytes | transfer specified # of bytes from in to out
122 test_create_repo | create dif (if not exists) and `git init` it
123 test_debug | with `--debug` run args otherwise nop
124 test_diff | fancy (same semantics) `test_cmp` using git
125 test_dir_is_empty | verbose fail if directory not empty
126 test_env | like /usr/bin/env but works for functions too
127 test_expect_code | fail unless command has specified exit status
128 test_get_temp | create a new, unique empty temp file (or dir)
129 test_have_prereq | check for a prereq
130 test_line_count | overblown `test` plus `wc` utility
131 test_match_signal | bad bad bad ksh
132 test_merge | `test_tick` & `git merge` & `git tag` shortcut
133 test_might_fail | succeeds for all but abnormal exits
134 test_must_be_empty | verbose fail if `test -s ...` is true
135 test_must_fail | succeed for all non-0 and non-abnormal exits
136 test_path_is_dir | verbose failure if `test -d ...` fails
137 test_path_is_file | verbose failure if `test -f ...` fails
138 test_path_is_missing | verbose failure if `test -e ...` fails
139 test_pause | interactive $SHELL if `--verbose` else error
140 test_seq | shell-only version of 1 and 2 arg seq
141 test_set_editor | properly quote value and set+export EDITOR
142 test_set_prereq | set specified prereq to true
143 test_skip_or_die | prefer `test_plan 0 <msg>` or `error <msg>`
144 test_tick | set or increment(+60) the `test_tick` var
145 test_tristate | reset variable to `true`, `false` or `auto`
146 test_unconfig | safe `git config --unset-all` when unset
147 test_when_finished | run code right after current subtest exits
148 test_write_lines | overblown `printf '%s\n'` shortcut
149 verbose | verbosely show a command's failures
150 wc | whitespace safe version of wc
151 write_script | make executable sh script from stdin
152 yes | 99 times max shell-only version of yes
155 ---------
156 Variables
157 ---------
159 - TESTLIB_DIRECTORY
161 This directory contains the testing library source code scripts (i.e.
162 `test-lib.sh`, `test-lib-main.sh` and `test-lib-functions.sh`). This
163 variable is exported. Should be constant for any given test suite run.
164 If not set, defaults to the real directory of
165 "${TEST_DIRECTORY:-.}/test-lib.sh" (if test-lib.sh is a symblic link
166 somewhere else then that somewhere else directory will be the actual
167 default for TESTLIB_DIRECTORY, NOT the result of dirname).
169 - EMPTY_DIRECTORY
171 This directory will be empty (i.e. no files in it) _AND_ will *NOT* have
172 any write permissions enabled. Usually this is "$TESTLIB_DIRECTORY/empty".
174 - TEST_DIRECTORY
176 This directory contains the test scripts to be run (e.g. `t1234-foo.sh`)
177 and will often be the same as TESTLIB_DIRECTORY. It is also exported.
178 Should be constant for any given test suite run. This directory MUST
179 contain a `test-lib.sh` file even if it's only a symbolic link to the
180 real `test-lib.sh` file (n.b. it _MUST NOT_ be a hard link).
182 - TEST_HELPER_DIRECTORY
184 The test helpers are located in here. It defaults to
185 "$TEST_DIRECTORY/helper" if not set and that directory exists. It is
186 automatically added to the `PATH`. Helper scripts should use this to
187 locate any auxilliary files they might need to use that are checked into
188 version control alongside them. This variable is also exported.
190 - TEST_OUTPUT_DIRECTORY
192 This is the directory containing output from the tests. It defaults to
193 TEST_DIRECTORY. Another exported variable. Should be constant for any
194 given test suite run.
196 - TRASH_DIRECTORY
198 This directory is named after the specific test being run, for example,
199 `trash directory.t0000-foo.sh`. By default it's located in the
200 TEST_OUTPUT_DIRECTORY but the `--root` option to a test script (options
201 are parsed by the "`. ./test-lib.sh`" line) can change that. Using the
202 `--root` option to specify a directory on a RAM disk can accelerate
203 testing. This variable changes for every test and the directory it
204 specifies is automatically created and destroyed (`test-lib.sh` makes
205 that happen).
207 - TRASHTMP_DIRECTORY
209 This directory is named after the specific test being run, for example,
210 `trash tmp directory.t0000-foo.sh`. By default it's located in the
211 TEST_OUTPUT_DIRECTORY but the `--root` option to a test script (options
212 are parsed by the "`. ./test-lib.sh`" line) can change that. Using the
213 `--root` option to specify a directory on a RAM disk can accelerate
214 testing. This variable changes for every test and the directory it
215 specifies is automatically created and destroyed (`test-lib.sh` makes
216 that happen).
218 However, unlike TRASH_DIRECTORY, this directory is created on-demand the
219 first time the `test_get_temp` function is called (or a lazy prereq is
220 evaluated) and so should not be used directly. Rather it exists primarily
221 to support creation of temporary test files that live completely outside
222 of TRASH_DIRECTORY and so can be created and used without concern about
223 any possible unintended consequences of temporary files lying around in
224 the TRASH_DIRECTORY (or a subdirectory thereof) causing unexpected test
225 results.
227 - TESTLIB_SKIP_TESTS
229 This variable can be used to skip whole tests or parts of tests. It should
230 contain a whitespace-separated list of shell patterns. If any of the
231 pattern(s) match the test or subtest is skipped. The patterns are matched
232 against:
234 1. the basename of the test with the first `'-'` and everything
235 following stripped off
237 2. value from (1) suffixed with `'.'` and the number of the subtest
238 where the first subtest is number 1
240 If the test file is `t/t1234-hello-there.sh` then the value used for (1) is
241 "t1234". For (2) and the second subtest the value is "t1234.2".
243 So setting TESTLIB_SKIP_TESTS="t1234 t1235.[235]" will skip test t1234
244 entirely and subtests 2, 3 and 5 of t1235.
246 - TESTLIB_NO_TOLERATE
248 If this is set to any non-empty value then all `test_tolerate_failure`
249 calls are turned into `test_expect_success` calls.
251 - TESTLIB_TEST_CHAIN_LINT
253 This variable defaults to true (any non-empty value). When true it results
254 in a fatal "Bail out!" error if any test script code (the last argument to
255 the `test_expect_success` function for example) does not have all of its
256 statements connected by `&&`. This variable may be set to the string "0"
257 _BEFORE_ sourcing `test-lib.sh` to disable this. Alternatively, the
258 checker can be "fooled" by enclosing code in `{ ` ... `;}` or putting it in
259 a separate function that's called.
261 - TESTLIB_TEST_NO_SUBSHELL
263 Normally the various test_expect_.../test_tolerate_... functions eval the
264 test scripts they've been passed inside a subshell so that unexpected
265 failures do not abort the test process. This precludes, however, making
266 any lasting variable changes in the passed in script. This is a good
267 thing. However, if `TESTLIB_TEST_NO_SUBSHELL` is set to a non-empty string
268 then the scripts will NOT be eval'd in a subshell. Try to avoid using
269 this. Note that it may be toggled in between
270 test_expect.../test_tolerate_... calls to only enable it for one test for
271 example.
273 Use of this variable to disable subtest subshells is deprecated and may go
274 away at some point! Avoid this if at all possible! The
275 `test_when_finished` function (possibly combined with `test_get_temp`) can
276 be used to raise information out of a subtest subshell when required.
278 - TESTLIB_TEST_TAP_ONLY
280 Setting this to a value other than empty or `0` suppresses extraneous
281 failure output (such as the script). Incompatible with verbose mode (but
282 allowed with verbose log mode). If set to the special value -1 it will
283 auto-disable when verbose mode is used.
285 Note that when TESTLIB_TEST_TAP_ONLY is enabled one additional line will
286 be output for each failure showing the file and line number, but _only_ if
287 the line number is available. This extra line can be suppressed by using
288 `--quiet` when TESTLIB_TEST_TAP_ONLY is enabled.
290 Set `TESTLIB_TEST_TAP_ONLY = -1` in your `config.mak` and get happy.
292 - TEST_NO_CREATE_REPO
294 Since the focus of the testing library is on Git or Git-related software
295 all "trash directory...."s are initialized as a non-bare empty Git
296 repository by default. Setting `TEST_NO_CREATE_REPO` _BEFORE_ sourcing
297 the `test-lib.sh` testing library file will instead result in the "trash"
298 directory for the test just being an empty directory instead. The
299 function `test_create_repo` can always be used to easily create a
300 repository (or two or more) if desired.
302 - color
304 Set automatically or explicity using the `--color` or `--no-color` options.
305 It will be the empty string if "no color" is in effect or it will be the
306 string "--color" if "color" is in effect. It can therefore be directly
307 used as an argument to git commands when producing debugging output.
309 - skip_all
311 If set to a non-empty value before calling the `test_done` function and
312 zero tests have been run, then the value will be appended to the output
313 `1..0 # SKIP ` plan line. If the test matches `$TESTLIB_SKIP_TESTS` when
314 `./test-lib.sh` is sourced, `skip_all` will be set to a suitable value and
315 `test_done` called immediately.
317 - test_auh
319 Will automatically be set to empty unless the detected Git version is at
320 least version 2.9 in which case it will be set to the value
321 `--allow-unrelated-histories`. Inteded for use with `git merge`. The
322 `test_merge` function automatically supplies this.
324 - test_description
326 This variable _must_ be set to a non-empty value _before_ sourcing the
327 `test-lib.sh` script. Other than checking for non-empty, there are no
328 requirements for the value. It is only used by the `-h`|`--help` options
329 (which are parsed when `test-lib.sh` is sourced). The only formatting
330 that occurs before it is output is that leading and trailing blank lines
331 are trimmed off. Recommended format is like a Git check in comment with
332 one summary line and then any optional body separated from the summary
333 line by one blank line if present. Ideally it should explain enough about
334 the test so that if the test is failing someone other than the test author
335 can debug the problem.
337 - test_external_has_tap
339 The default setting for this variable is `0` and should normally be left
340 set to `0`. Setting it to any other value causes the output of test plan
341 and test result TAP lines to be suppressed. It can be useful when one
342 testing library test is using the testing library to run a set of tests
343 thereby preventing the recursive testing library invocation from upsetting
344 the TAP output of the parent.
346 - test_tick
348 Initially unset. Records the last unix epoch seconds value that was used
349 by the `test_tick` function to set both the `GIT_COMMITTER_DATE` and
350 `GIT_AUTHOR_DATE` variables. Note that the `test_tick` function always
351 appends " -0700" to the value of "$test_tick" when constructing the value
352 for `GIT_COMMITTER_DATE` and `GIT_AUTHOR_DATE`.
354 If `test_tick` is unset or empty when the `test_tick` function is called
355 then it will be set to 1112911993 (2005-04-07 15:13:13 -0700) and that
356 value used. Otherwise it will be advanced by 60 and the resulting value
357 used.
359 See also the description of the `test_tick` utility function below.
361 - this_test
363 This variable is set automatically by the ". ./test-lib.sh" line.
365 The value of "$this_test" is the basename of the current test file upto
366 but excluding the first "-" in the name. For example "t/t1000-it-works.sh"
367 has a "$this_test" value of "t1000". Since tests are expected to be
368 located in the "$TEST_DIRECTORY" directory, any test that needs to make use
369 of auxilliary files should locate them in a "$TEST_DIRECTORY/$this_test"
370 directory that is checked in alongside the test so in the case of the
371 aforementioned "t/t1000-it-works.sh" this might be the directory structure
372 if two auxilliary files "aux1" and "aux2" were present:
374 t/t1000-it-works.sh
375 t/t1000/aux1
376 t/t1000/aux2
378 And all three of them would be checked into version control.
381 ------------
382 Test Options
383 ------------
385 When `test-lib.sh` is sourced, it parses the options passed to the test. When
386 running tests via the Makefile, setting "TESTLIB_TEST_OPTS" will pass those
387 options to _every_ test run by the Makefile.
389 * --chain-lint | --no-chain-lint
391 Set TESTLIB_TEST_CHAIN_LINT to 1 (`--chain-lint`) or 0 (`--no-chain-lint`).
392 See the description of TESTLIB_TEST_CHAIN_LINT above for details.
394 * --color | --no-color
396 Override automatic color detection (color output is normally enabled when
397 output is to a terminal that supports color sequences) and activate
398 (--color) or deactivate (--no-color) color output. Note that when the
399 environment variable `HARNESS_ACTIVE` is non-empty, color is ALWAYS
400 SUPPRESSED when TAP lines are output and the `#` first character of
401 comment lines also has its color suppressed in that case.
403 * -d | --debug
405 Normally any command preceded by `test_debug` becomes a nop. (The testing
406 library itself does not use `test_debug` anywhere.) With `--debug` these
407 lines are activated and actually run instead of being nops.
409 Using this option also prevents the temporary trash directory from being
410 removed even for a successful test.
412 * -h | --help
414 Show the `$test_description` to standard output and `exit 0`. NO TAP lines
415 of any kind are output when using this option (nor are the
416 $test_description lines prefixed with the `#` comment character).
418 * -i | --immediate
420 When `--immediate` is used, if a test_expect_success... etc. call fails
421 then `exit 1` immediately after showing the `not ok` and quoted (with
422 leading `#` characters) test script.
424 * -l | --long | --long-tests | --expensive
426 Set and export `TESTLIB_TEST_LONG=t`. The `EXPENSIVE` prereq checks for
427 this and only succeeds when `TESTLIB_TEST_LONG` is non-empty.
429 * -q | --quiet | --no-quiet
431 Suppresses some output lines such as "ok" (only when NOT running under a
432 harness) and various output from test_external... functions. Mainly
433 useful to produce more compact output when NOT running under a TAP harness.
435 Ignored if `--verbose` is also active.
437 When combined with `--run` the skipping test message(s) that would
438 otherwise be output when tests are skipped by a `--run` option are
439 suppressed unless $HARNESS_ACTIVE.
441 Use of `--run` will implicitly partially activate the `--quiet` option
442 unless the `--no-quiet` option is also given. The "partially" part means
443 that it will only suppress the skipping test messages but not any other
444 messages that would ordinarily be suppressed with an explicit `--quiet`.
446 Use an explicit `--quiet` or `--no-quiet` to leave "partially quiet" mode
447 when the `--run` option is in use.
449 * --root=<path>
451 Causes the "trash directories" to be created as subdirectories of `<path>`
452 instead of as subdirectories of `$TEST_OUTPUT_DIRECTORY`. Specifying a
453 RAM disk location as `<path>` can significantly speed up testing.
455 * --run=<run-list> | -r <run-list>
457 The <run-list> must contain comma or whitespace separated "selectors". But
458 note that when using "TESTLIB_TEST_OPTS" only comma separated will work.
460 A selector must match either !?\d+ or !?\d*-\d* which is to say it's either
461 <num> or <num>- or -<num> or <num>-<num> optionally prefixed with `!`. The
462 "<num>-" form is a shortcut for <num>-infinity and the "-<num>" form is a
463 shortcut for "1-<num>".
465 If the first (or only) selector is prefixed with `!` then it's an exclusion
466 list otherwise it's an inclusion list.
468 The "selectors" are processed in left to right order and if the number of
469 the subtest matches the range it will be run if the selector was NOT
470 prefixed with `!` otherwise it will be excluded.
472 An "inclusion" list includes and runs ONLY those subtests that are
473 specifically matched by a selector NOT prefixed with `!`. (Note that
474 a subsequent selector can again exclude a previously included match and
475 vice versa).
477 An "exclusion" list runs everything except tests that are specifically
478 excluded by a match. In other words an "exclusion" list is just a shortcut
479 to avoid adding a "1-" selector to the front of <run-list>.
481 For example, "1-4,!3" will run only subtests 1, 2 and 4 but so will
482 "1,2,4". On the other hand, "!3,1-4" will run ALL subtests except 3
483 (because it's equivalent to "1-,!3,1-4") as will just "!3".
485 Using the `--run` option (with a non-empty list) implicitly partially
486 activates the `--quiet` option (see the desciption of the `--quiet`
487 option) and moves the test plan line to the end (if there was a `test_plan`
488 call at the beginning), but adding an explicit `--no-quiet` (or `--quiet`)
489 will override all that.
491 * --tap-only | --no-tap-only
493 Set `TESTLIB_TEST_TAP_ONLY` to `1` or `0` respectively. See the
494 description for `TESTLIB_TEST_TAP_ONLY`. Use of ``--tap-only`` is
495 incompatible with ``--verbose`` unless ``--verbose-log`` is also used.
496 Does _not_ cause `TESTLIB_TEST_TAP_ONLY` to be exported (unless it already
497 was exported).
499 * --tee
501 All standard output and standard error from running the test is copied to
502 $TEST_OUTPUT_DIRECTORY/test-results/$(basename $0 .sh).out and as a side
503 effect standard error is redirected to standard out. The final exit code
504 is written to $TEST_OUTPUT_DIRECTORY/test-results/$(basename $0 .sh).exit
505 as well.
507 This is accomplished by running another copy of the test script and
508 redirecting both its standard output and standard error through the `tee`
509 utility and when that copy exits, so too will the "tee" process and it will
510 then use the same exit code as its exit code (unless something went wrong
511 and no exit code was recorded in which case it exits with a result of 1).
513 The environment variable TESTLIB_TEST_TEE_OUTPUT_FILE is set (and exported)
514 to the name of the file the output is being "tee"'d into.
516 * -v | --verbose
518 When `--verbose` is used the `test_pause` function will actually pause
519 a test by entering a subshell and `test_external_without_stderr` will show
520 both stdout and stderr output on failure. Test scripts passed to the
521 test_expect_success... functions are run with standard output and standard
522 error left unchanged (normally it's sent to /dev/null).
524 Automatically enabled by `-x`.
526 Note that `--verbose` is not allowed when $HARNESS_ACTIVE.
528 * --verbose-log
530 Enables the `--tee` option and arranges for the standard output and
531 standard error that are enabled by `--verbose` to end up in the same log
532 file that `--tee` creates while not disrupting the output going to the
533 test harness.
535 This setting supersedes `--verbose` in that if both are given the output
536 ends up in the log but not on normal standard output or standard error.
538 Also if `--verbose-only=...` is used together with this option then only
539 the subtests selected by the `--verbose-only` option value will have their
540 output captured into the log -- the other subtests will have it sent to
541 /dev/null.
543 Note that `--verbose-log` *IS* compatible with $HARNESS_ACTIVE.
545 * --verbose-only=<run-list>
547 The current subtest number (sub tests are numbered starting at 1) is
548 matched against <run-list> (see description for `--run` option) and if
549 there's a match then `--verbose` is active for that subtest and if there is
550 *NOT* a match then `--verbose` is NOT active for that subtest.
552 For example, `--verbose-only="1-3,5"` will enable verbose for
553 subtests 1, 2, 3 and 5 and will DISABLE verbose mode for subtests
554 4,6,7,8,9,10,11,12,13,14,15 etc.
556 Using this option with a non-empty <run-list> value completely supersedes
557 any explicit `--verbose` option (before or after). (Using it with an empty
558 value just silently has no effect.)
560 Note that `--verbose-only=...` is not allowed when $HARNESS_ACTIVE unless
561 `--verbose-log` is also used.
563 * -x | --xtrace
565 Set the `--verbose` option and temporarily enable the shell's `set -x`
566 option while evaluating the scripts passed to test_expect_success....
568 Note, however, that tracing will be suppressed unless verbose is also
569 enabled. Normally that will always be the case since `-x` also enables
570 `--verbose`. However, if `--verbose-only` is used then verbose will be
571 toggled on a per-subtest basis and as a result the `-x` option will only
572 be in effect for subtests selected by the `--verbose-only=...` option.
574 Note that since this option sets `--verbose` it's not normally allowed
575 when $HARNESS_ACTIVE but setting `--verbose-log` will allow it and send the
576 output to the log file.
579 ------------------
580 TAP Test Functions
581 ------------------
583 These are functions that are intended to ultimately emit either a TAP test plan
584 line or a TAP test (aka "subtest") result line.
586 These functions are intended to be called from the main body of a test script
587 (e.g. t-????-*.sh) after having sourced the `./test-lib.sh` script.
590 Test Plan Functions
591 ~~~~~~~~~~~~~~~~~~~
593 The purpose of these functions is to output the single, required, TAP test
594 plan line. Additionally the `test_done` function performs some extra work.
596 - test_done
598 Outputs a TAP test plan line.
600 MUST be called AFTER any test result functions. (TAP supports outputting
601 a test plan line either _before all_ test result lines or _after all_
602 test result lines.)
604 It may be tempting to output the plan line without using this function.
605 Don't do that. Failure to use this function will result in test cleanup
606 functions failing to run and the test results aggregation provided by the
607 testing library breaking.
609 If `$test_external_has_tap` is set to something other than `0` (the
610 default) then no test plan line will be output but normal test cleanup will
611 take place.
613 If `$skip_all` is set to a non-empty value AND 0 tests were run then the
614 `$skip_all` message will be appended to a `1..0 # SKIP ' line (which is,
615 again, suppressed if `$test_external_has_tap` is not `0`).
617 The `test_done` function then calls either `exit 0` if there were no
618 unexpected failures or `exit 1` if there were. Just before calling
619 `exit 0` (but _not_ `exit 1`) the `test_at_end_hook_` function is called
620 which, by default, always succeeds and does nothing but can be re-defined
621 by a test script if such a hook is needed.
623 Note that if the test name appears in `$TESTLIB_SKIP_TESTS` (see the
624 description above) then `$skip_all` is automatically set to a suitable
625 value and `test_done` is called immediately when `./test-lib.sh` is
626 sourced.
628 The `test_done` function DOES NOT RETURN! It _always_ ends up calling
629 exit with an exit status and therefore _never_ returns to the caller!
631 - test_plan <count> <count==0> <optional> <alternate> <skip> <msg> <here>
633 This function allows a test plan line to be output first. If `$skip_all`
634 is set it just passes through to `test_done`. Otherwise it outputs the
635 test plan line using the first argument as the total test count and then
636 sets `test_wrote_plan_count=$1` so that the required subsequent call to
637 `test_done` does not write a second test plan line. (But if the value of
638 `$test_external_has_tap` is not 0 it will not output the test plan line.)
640 If the argument is 0, then `skip_all` will be set to the following
641 arguments (or a suitable default if there are no following arguments) and
642 `test_done` will be called immediately (which will cause an immediate
643 exit of the test script at that point).
645 Using `test_plan` is inconvenient in that the number of subtests in the
646 file must be known in advance. Sometimes that's not possible if the number
647 of tests to be run varies depending on test conditions. Other times it's
648 just damn inconvenient to have to count all those subtests. ;)
650 To document that a `test_plan` number has been deliberately omitted, a
651 single argument of `?` can be used (e.g. `test_plan '?'` -- the "?" needs
652 to be quoted in some fashion to avoid wildcard expansion). Using the
653 value "`?`" has the same effect as not having any `test_plan` line at all
654 but documents the intent not to have one.
656 If `test_plan` is never called (and the entire test was not skipped as a
657 result of a `TESTLIB_SKIP_TESTS` setting) a message urging addition of a
658 `test_plan` call will be output. With support for a count of `?`, it's
659 always possible to add a `test_plan` call to the script.
661 Outputting the test plan line in advance makes those fancy test harnesses
662 (like `prove`) show a "1/n" progress rather than "1/?" which is nice to
663 have and that is the reason the `test_plan` function is provided (and the
664 prodding message output if it's not been called at `test_done` time).
666 Also if a different number of tests are run than are planned the overall
667 test will fail with an error which provides yet another test of the test
668 script itself which is not available without a call to `test_plan`.
670 The `test_done` function MUST STILL BE CALLED at the end of the test script
671 even when `test_plan` is used at the beginning otherwise test aggregation
672 and test cleanup will fail to take place! The only exception is when a
673 `test_plan 0` (possibly with optional arguments) call is _always_ used as
674 it will implicitly call `test_done` before returning (and therefore never
675 actually return at all).
677 Note that if it's not already clear from the description, use of the
678 `test_plan` function is completely optional, but use of `test_done` is not.
681 Test Result Functions
682 ~~~~~~~~~~~~~~~~~~~~~
684 The purpose of these functions is to output one `ok` or `not ok` TAP test
685 result line each time they're called.
687 All of these functions take an optional first argument which is a whitespace
688 and/or comma separated list of prerequisites that must be present or the
689 test will be skipped. This is shown as `[<prereqs>]`. Each prereq in the
690 list may be optionally prefixed with a single `!` to require the item *not*
691 be present.
693 All of these functions expect the next argument to be a brief test description
694 that will be used in the `ok` or `not ok` output line. This is shown as
695 `<desc>`.
697 The "external" functions expect two additional arguments which are the
698 "external" command to be run and its single argument (use `eval` and a suitable
699 string to work around this requirement for exactly two arguments). These are
700 shown as `<cmd>` `<arg>`.
702 The non-"external" functions expect one additional argument which is the
703 test script to be "eval"'d. However, if the test script argument is `-` then
704 the test script will be read from standard input (allowing use of a HERE doc to
705 simplify quoting issues), but this is slightly inefficient so avoid using it
706 unless it's really needed. This is shown as `<script>`. Note that scripts are
707 normally "eval"'d inside a subshell to avoid side-effects and uncaught
708 failures, but this can be altered with the `TESTLIB_TEST_NO_SUBSHELL` variable
709 (see description above).
711 Test scripts are "eval"'d as the last line of a function so they may exit early
712 by using a `return` statement or, if `$TESTLIB_TEST_NO_SUBSHELL` is _NOT_ set
713 (which is the default), by using an `exit` statement.
715 - test_expect_success [<prereqs>] <desc> <script>
717 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
718 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
719 it succeeds (exit code is 0) then this subtest succeeds and outputs an "ok"
720 line otherwise it outputs a "not ok" line.
722 - test_expect_failure [<prereqs>] <desc> <script>
724 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
725 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
726 fails (exit code is _not_ 0) then this subtest outputs a "not ok # todo"
727 line. If the script succeeds a "ok # todo" line is output which will
728 ultimately produce some warnings and/or complaints about a "known breakage"
729 vanishing or "todo passing". See the `test_tolerate_failure` function to
730 avoid this.
732 - test_tolerate_failure [<prereqs>] <desc> <script>
734 If <prereqs> are present and unsatisfied an "ok # skip" line is output and
735 the <script> is _not_ "eval"'d. Otherwise, the <script> is "eval"'d and if
736 fails (exit code is _not_ 0) then this subtest outputs a "not ok # todo"
737 line. If the script succeeds a plain "ok" line is output (which avoids
738 any complaints about "vanishing" breakages). However, if the variable
739 `TESTLIB_NO_TOLERATE` is non-empty all `test_tolerate_failure` calls are
740 turned into `test_expect_success` calls instead (meaning a failure causes
741 a regular `not ok` to be output instead of a `not ok # todo`).
743 This function is useful for testing things that probably don't work the
744 way they should but are reluctantly tolerated and if they do start working
745 better no fuss should be made about the test suddendly passing.
747 It's a nice way to document those "yes, we still have this ugly thing here
748 that we wish we didn't" behaviors. It can also be used to distinguish
749 "tier 1" supported platforms (those which pass all tests even when
750 `TESTLIB_NO_TOLERATE=1` is set) and "tier 2" supported platforms (those
751 that pass all tests as long as `TESTLIB_NO_TOLERATE` is _not_ set). (Tier
752 3 would be those platforms that always fail at least one test.)
754 - test_external [<prereqs>] <desc> <cmd> <arg>
756 Behaves just like `test_expect_success` with a script of "<cmd> <arg>"
757 (assuming <cmd> and <arg> are first suitably quoted) except that stdout is
758 always left on stdout even without `--verbose`.
760 Use "eval" for <cmd> and pass a script as <arg> to use as a substitute
761 `test_expect_success` that does not redirect stdout or to escape the
762 two argument restriction.
764 - test_external_without_stderr [<prereqs>] <desc> <cmd> <arg>
766 *IMPORTANT*: This function outputs _two_ test result lines!
768 If using the `test_plan` function then each `test_external_without_stderr`
769 function call counts as *TWO* planned tests!
771 It works just like `test_external` (in fact, it calls test_external first)
772 and then it performs a second subtest to see if _anything_ was output to
773 stderr by "<cmd> <arg>" in which case the second subtest fails otherwise
774 it succeeds. If the `test_external` first subtest ends up being skipped
775 then nothing will be output to stderr and the second subtest will always
776 succeed (it will not be skipped). In other words, the second subtest
777 result line is always a plain "ok" or "not ok" and never has any "# skip"
778 or "# todo" added to it even when the first subtest result line does.
780 Note that the special prerequiste `LASTOK` behaves as expected here --
781 that is if the first `test_external` call is skipped then its state
782 _does NOT change_ regardless of the outcome of the second "stderr"
783 subtest check. In addition, if the the `test_external` test is _not_
784 skipped then the `LASTOK` prerequisite is guarantted to only be true
785 provided _both_ the `test_external` call suceeds _and_ no "stderr" output
786 is generated. In other words, it just does the right thing here so don't
787 worry about it and go ahead and use it.
790 Standard Test Prerequisites
791 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
793 The test result functions all take an optional first argument which is a comma
794 and/or whitespace separated list of positive or negative (prefixed with `!`)
795 prerequisites for the test to be run. *All* off the listed prerequisites for
796 each test _must_ be satisfied or it will be skipped.
798 Additional prerequisites may be set using the `test_set_prereq` function but
799 there are a number of pre-defined prerequisites that may be tested:
801 - `AUTOIDENT`
803 The `git var GIT_AUTHOR_IDENT` command succeeds when both `GIT_AUTHOR_NAME`
804 and `GIT_AUTHOR_EMAIL` are unset.
806 - `BSLASHPSPEC`
808 The backslash character (`\`) is allowed in path/file names. Normally
809 defined _except_ for `MINGW` and `CYGWIN`.
811 - `CASE_INSENSITIVE_FS`
813 File names are case-insensitive (i.e. "foo" and "Foo" refer to the same
814 file).
816 - `CMDLINE_LIMIT`
818 Whether or not the `run_with_limited_cmdline` function appears to work
819 (it does `ulimit -s 128` in a subshell and runs the command assuming that
820 will limit the size of the command line argument list).
822 - `COLUMNS_CAN_BE_1`
824 Set when the shell's `$COLUMNS` variable can be successfully set to 1.
826 - `CYGWIN`
828 The CygWin environment is active.
830 - `EXECKEEPSPID`
832 The process ID before and immediately after an `exec` call stays the
833 same. Normally defined for all but `MINGW`.
835 - `EXPENSIVE`
837 The `TESTLIB_TEST_LONG` variable is non-empty (see the `--long-tests` aka
838 `--expensive` test option).
840 - `FILEMODE`
842 True if Git thinks the executable bit is supported properly (i.e.
843 Git's `core.filemode` is set to true after `git init`).
845 - `GREP_STRIPS_CR`
847 The grep command strips off CR. Normally only defined for `MINGW` and
848 `CYGWIN`.
850 - `LASTOK`
852 This is a special prerequisite that always succeeds for the first subtest
853 in a test script and for every subtest thereafter for which the most
854 immediately preceeding non-skipped test was "ok" (regardless of the
855 presence any "# TODO").
857 In other words, this can be used to skip a subtest if the preceeding
858 subtest did not succeed. Since "skipped" tests do not change the state
859 of this prerequisite a single failure can cause a long chain of following
860 subtests that use this prerequisite to be skipped.
862 - `MINGW`
864 The MinGW environment is active.
866 - `NATIVE_CRLF`
868 The native environment uses CR+LF line endings (normally only defined
869 for `MINGW`).
871 - `NOT_ROOT`
873 The `uid -u` command succeeded and returned a non-0 value.
875 - `UTF8_NFD_TO_NFC`
877 The file system automatically converts NFD unicode to NFC unicode.
879 - `PIPE`
881 The `mkfifo` command is supported and works.
883 - `POSIXPERM`
885 Sane, POSIX-style permissions. *Not* normally defined for `MINGW`.
887 - `SANITY`
889 File permission bits are honored (i.e. after doing `chmod a-r file` then
890 a subsequent `cat file` will fail). When running as root these "sanity"
891 tests often fail.
893 - `SED_STRIPS_CR`
895 The sed command strips off CR. Normally only defined for `MINGW` and
896 `CYGWIN`.
898 - `SYMLINKS`
900 Symbolic links created using `ln -s` work.
902 - `USR_BIN_TIME`
904 The result of `test -x /usr/bin/time` is true.
907 -----------------
908 Utility Functions
909 -----------------
911 These are convenience functions and while most of them are provided to be
912 called from the body of a script passed to one of the test_expect_...
913 functions, several of them can also be used from the main body of a test
914 script (`say`, `say_color`, `test_debug` and `test_pause` being prime examples
915 of such).
917 These functions are intended to be called only _after_ the `./test-lib.sh`
918 script has been sourced. See the `README-TESTLIB-TG` file for additional
919 available utility functions that are TopGit-specific.
921 - die <optional> <message> <here>
923 Can be used to exit a test script on an unexpected error, should be used
924 after a `||` to preserve the exit code as in:
926 perform_some_function perhaps with arguments || die
928 If the <optional> <message> <here> is present, it will be included in the
929 fatal error message (along with the exit code).
931 - error <msg> <goes> <here>
933 Cause an instant test `Bail out!` error (including <msg> <goes> <here>).
935 The original Git version of this function just did an exit after showing
936 the error without using `Bail out!`, but these errors are terminal and the
937 entire test suite should be stopped if one occurs and that is what will now
938 happen when this function is called.
940 - list_contains <commalist> <pattern>
942 Returns true if comma separated list <commalist> contains any item that
943 matches the filename matching wildcard <pattern>.
945 A shortcut for those uncomfortable with the `case` statement.
947 - sane_unset <varname>...
949 An alias for `unset <varname>... || :` to workaround non-conforming shells
950 that return a non-zero status if any of the variables were not already set.
952 - say <msg> <goes> <here>
954 A convenient shortcut for `say_color info <msg> <goes> <here>`
956 - say_tap <msg> <goes> <here>
958 A convenient shortcut for `say_color_tap info <msg> <goes> <here>`
960 - say_color <color> <msg> <goes> <here>
962 Possible <color> values are:
964 * "" (the empty string)
965 plain text with no color, suppressed when `--quiet` in effect
966 * `error`
967 used for testing failures, typically shown in red
968 * `skip`
969 used for test skip messages, typically shown in a light color
970 * `warn`
971 warnings are shown in this, typically yellow (or brown)
972 * `pass`
973 used when a complete test script passes, usually green
974 * `info`
975 info messages and non-harness TAP messages, usually a light color
976 * `reset`
977 turns off color, used internally, should not be used by callers
979 Using anything else for <color> ends up outputting plain text followed by
980 the color reset code.
982 If `--quiet` is in effect, anything output using `""` (the empty string)
983 as the color name will be supressed entirely.
985 If `--no-color` is in effect (perhaps implicitly) the color code sequences
986 are omitted from the output.
988 If `$HARNESS_ACTIVE` and the first character of <msg> is `#` then that
989 single character is always output _without_ any preceding color codes (the
990 first character to be in color will be the following one unless, of course,
991 the color name is `""` or `--no-color` is in effect).
993 - say_color_tap <color> <msg> <goes> <here>
995 When `$HARNESS_ACTIVE` this always outputs "<msg> <goes> <here>" _without_
996 any color sequences regardless of the presence of the `--quiet` option.
998 If there is no `$HARNESS_ACTIVE` then this call is identical to `say_color`
999 with the exact same arguments.
1001 - test_at_end_hook_
1003 This function is called by `test_done` if there are no unexpected failures.
1004 By default it's defined to be just `:` but should be re-defined by each
1005 test script file as needed. Note that it _will_ be called when the entire
1006 test script is skipped with a `1..0` plan line.
1008 - test_clear_when_finished
1010 Any `test_when_finished` items already scheduled in the subtest script
1011 so far are cleared (i.e. removed). Calling `test_clear_when_finished`
1012 before `test_when_finished` has first been called silently does nothing
1013 with no error.
1015 - test_cmp <arg>...
1017 Shortcut for:
1019 $TESTLIB_TEST_CMP <arg>...
1021 Useful because `diff -u` will be used where supported instead of `cmp`.
1022 See also `test_diff` for an alternative.
1024 - test_cmp_bin <arg>...
1026 Shortcut for:
1028 cmp <arg>...
1030 There's no real compelling reason to use this other than to look nice
1031 next to `test_cmp`.
1033 - test_cmp_rev <expect-rev> <actual-rev>
1035 Save output of `git rev-parse --verify` on each argument to a file and
1036 then run `test_cmp expect.rev actual.rev` on the resulting two files.
1038 The temporary files are actually placed in TRASHTMP_DIRECTORY so there's
1039 no need to be concerned about contaminating a working tree that's being
1040 used for testing or needing to do a `cd` to avoid that.
1042 - test_commit [--notick] [--signoff] <message> [<file> [<contents> [<tag>]]]
1044 Create a new Git commit with "<message>" as the commit message optionally
1045 having a signed-off-by line (`--signoff`) using the next tick time by
1046 calling `test_tick` (unless `--notick` is used).
1048 <file> defaults to "<message>.t", <contents> and <tag> default to
1049 "<message>".
1051 The "<contents>" will be written (using `printf '%s\n'`) to <file> which is
1052 then added and committed using <message> and then tagged using <message>.
1054 It will not be tagged if <tag> is the empty string or contains any
1055 whitespace or invalid ref name characters. (Including at least one '^' or
1056 '~' is a simple way to keep a single "word" that's a valid filename while
1057 at the same time suppressing creation of a tag while not needing to
1058 supply anything other than the <message> argument.)
1060 - test_config [-C <dir>] <name> <value> [...]
1062 Run `git [-C <dir>] config <name> <value> [...]` now and schedule a
1063 `git [-C <dir>] config --unset-all <name>` (via `test_unconfig`) using
1064 `test_when_finished`.
1066 This function allows for easy temporary configuration changes that are
1067 automatically undone when the current test finishes.
1069 - test_config_global <name> <value> [...]
1071 Run `git config --global <name> <value> [...]` now and schedule a
1072 `git [-C <dir>] config --unset-all --global <name>` (via `test_unconfig`)
1073 using `test_when_finished`.
1075 This function allows for easy temporary global configuration changes that
1076 are automatically undone when the current test finishes.
1078 - test_copy_bytes <count>
1080 This is a shortcut for a call to the `dd` utility (unlike the Git version
1081 which uses perl) to transfer <count> bytes (or up to EOF) from stdin to
1082 stdout.
1084 - test_create_repo <path>
1086 Shortcut for:
1088 (mkdir -p "<path>" && cd "<path>" && git init)
1090 Except that an empty template directory is used and if that doesn't succeed
1091 in suppressing creation of a hooks directory the hooks directory will be
1092 renamed to `hooks-disabled`.
1094 Note that since Git will happily reinitialize an existing Git repository
1095 this command can succeed on a pre-existing repository without altering its
1096 contents.
1098 - test_debug [<cmd> [<arg>...]]
1100 This function does nothing (it's a fancy comment) _unless_ `--debug` is
1101 used in which case <cmd> [<arg>...] is executed with standard output and
1102 standard error redirected to the original standard error of the test
1103 script (in other words, you don't need to add `--verbose` to see the
1104 output when `--debug` is active).
1106 Note that this is different than the original Git version that only takes
1107 a single argument and "eval"s it (while not guaranteeing anything about
1108 where the output goes). To get the Git "eval" equivalent, use `eval` as
1109 the first <arg> and the string to be "eval"'d as the second.
1111 - test_diff <arg>...
1113 Shortcut for:
1115 git --no-pager -c core.abbrev=16 diff --no-color --exit-code \
1116 --no-prefix --no-index <arg>...
1118 Always gives nice diffs and they're always `-u` diffs.
1119 This should be usable as a drop-in replacement for `test_cmp` everywhere
1120 and its output should be deterministic as no dates or times are included.
1122 - test_dir_is_empty <dirpath>
1124 Fail verbosely (with directory contents) unless <dirpath> is an existing
1125 and empty (only `.` and `..` entries) directory.
1127 - test_env [VAR=VAL]... <cmd> <arg>...
1129 In a subshell set and export each `VAR=VAL` and then run `<cmd> <arg>...`.
1131 Obviously, since it's a subshell no variable changes will persist, but this
1132 makes it easy to temporarily change a variable while running some shell
1133 function.
1135 - test_expect_code <status> <cmd> [<arg>...]
1137 Runs `<cmd> <arg>...` and fails verbosely unless the resulting status code
1138 is <status>.
1140 - test_get_temp [-d] [<name>]
1142 Create a new temporary file (or directory with -d) in the temporary
1143 directory $TRASHTMP_DIRECTORY with optional pattern prefix "<name>." and
1144 output the resulting unique name of the newly created empty file (or
1145 directory with -d).
1147 The $TRASHTMP_DIRECTORY is created on-demand the first time this function
1148 is called and is guaranteed to be outside of $TRASH_DIRECTORY but still
1149 unique to the test script currently being run.
1151 - test_have_prereq <PREREQ>...
1153 Test a list of zero or more whitespace and/or comma separated prereqs
1154 (the same as can be given to any of the test result functions). As with
1155 the test result functions, prefixing a prerequisite with a single `!`
1156 requires that it _not_ be present. Result code is 0 if the requirements
1157 are satisfied, non-zero if not.
1159 - test_line_count <op> <val> <file>
1161 Verbosely fails if `test $(( $(wc -l < "<file>") )) "<op>" "<val>"` fails.
1163 - test_match_signal <signum> <exitcode>
1165 Returns true if <exitcode> represents an exit from signal <signum> (which
1166 must be numeric). This function only exists to accomodate `ksh` which
1167 uses 256+<signum> instead of the POSIX 128+<signum> for signal exit codes.
1169 - test_merge <message> [<opt>...] <commit>
1171 Shortcut for:
1173 test_tick && git merge -m "$@" && git tag "<message>"
1175 Except that `--allow-unrelated-histories` is automatically supplied for
1176 Git 2.9 or later and the tag is skipped if "<message>" is not tagish.
1178 - test_might_fail <cmd> [<arg>...]
1180 Just a shortcut for `test_must_fail ok=success <cmd> <arg>...`.
1182 This is useful when it's okay for <cmd> to succeed or for it to fail
1183 provided the failure is a "normal" (i.e. non-signal) failure.
1185 - test_must_be_empty <file>
1187 Verbosely complain (and fail) showing <file>'s contents if it's not empty
1188 (i.e. -s <file> succeeds).
1190 - test_must_fail [ok=<commalist>] <cmd> [<arg>...]
1192 Runs `<cmd> <arg>...` and verbosely converts all result codes into either
1193 0 or 1 where "normal" failures become 0 and others become 1.
1195 This function should be used when a failure is required but only if the
1196 failure is *not* the result of a signal, command not found or command not
1197 executable error.
1199 If <commalist> contains "success" then a result code of 0 is _not_
1200 converted to 1 (it stays 0). If <commalist> contains "sigpipe" then an
1201 exit due to `SIGPIPE` is converted to status 0 (otherwise it becomes 1).
1203 - test_path_is_dir <path> [<msg>]
1205 Fail verbosely (including <msg> if given) if `test -d <path>` fails.
1207 - test_path_is_file <path> [<msg>]
1209 Fail verbosely (including <msg> if given) if `test -f <path>` fails.
1211 - test_path_is_missing <path> [<msg>...]
1213 Fail verbosely (including <msg>... if given) if `test -e <path>` _succeeds_.
1215 - test_pause
1217 Spawn an interactive `$SHELL` to pause testing until it exits -- causes an
1218 error if used without `--verbose`. Should be used for debugging only.
1220 Something like this can be use to leave it in a script (although this is
1221 not recommended):
1223 test_debug eval "want_no_verbose || test_pause"
1225 which will cause the test in question to spawn a subshell whenever it's run
1226 with both the --verbose and --debug options.
1228 - test_seq [<start>] <end>
1230 <start> defaults to 1.
1232 Output each of the numbers from <start> to <end> inclusive (incrementing by
1233 1) each followed by a newline to stdout. If <end> is less than <start>
1234 nothing is output.
1236 - test_set_editor <path-to-editor>
1238 Set and export the `EDITOR` environment variable but using tricks so that
1239 <path-to-editor> is allowed to contain any value (including spaces, quotes,
1240 etc.) without needing any special quoting.
1242 - test_set_prereq <PREREQ>
1244 Indicate that the test prerequisite <PREREQ> is available (should be a
1245 single word in ALL CAPS). It may then be used as a prereq for any of the
1246 test result functions or be explicitly tested with `test_have_prereq`.
1248 Note that for this to work inside a subtest and have the result be
1249 available to subsequent subtests, it needs to be preceded by
1250 `test_when_finished` (and also be run without that if `test_have_prereq` is
1251 intended to work subsequently *inside* the same subtest's subshell).
1253 - test_skip_or_die <mode> <msg>
1255 Don't use this function, it's confusing, use either `test_plan 0` or
1256 `error` directly.
1258 If <mode> is `auto` it's the same as `test_plan 0 <msg>`.
1260 If <mode> is `true` it's the same as `error <msg>`.
1262 If <mode> is anything else it's a more verbose call to `error`.
1264 - test_tick
1266 Set and export `GIT_COMMITTER_DATE` and `GIT_AUTHOR_DATE` to the same
1267 value (initially 1112911993 -- the time of the first Git commit) increasing
1268 it by exactly 60 seconds each time `test_tick` is called.
1270 Note that when a test script starts running neither `GIT_COMMITTER_DATE`
1271 nor `GIT_AUTHOR_DATE` are initially set. It's not until after the first
1272 time `test_tick` is called (it's called implicitly by `test_commit` and
1273 `tg_test_create_branch` unless they're given the `--notick` option) that
1274 both of them are set (unless the test script otherwise sets them itself).
1276 See also the description of the `test_tick` variable above.
1278 - test_tristate <varname>
1280 Examines contents of `$<varname>` and sets `$<varname>` to one of `false`
1281 `true` or `auto`. An unset variable becomes `auto` and an empty string
1282 variable becomes `false` otherwise there should be no surprises except
1283 that anything that is not a Git boolean or `auto` is treated as `true`.
1285 The original version from the Git test library uses Git itself to do this
1286 conversion, but this implementation is strictly shell built-ins.
1288 - test_unconfig [-C <dir>] <arg>...
1290 Runs `git [-C <dir>] config --unset-all <arg>...` immediately but ignores
1291 any error resulting from the the value(s) to be unset not actually being
1292 currently set at all (status 5).
1294 Can be used for global values simply by making the first <arg> be `--global`
1295 and the rest the global arg(s) to unset.
1297 - test_when_finished <cmd> [<arg>...]
1299 This function only works inside a test script!
1301 Causes <cmd> [<arg>...] to be run when the current test script (the
1302 <script> arg to a test result function) exits. May be used more than
1303 once to schedule multiple items.
1305 This function differs from Git's version in that <cmd> [<arg>...]
1306 are automagically quoted and stored in a "test_when_finished_<n>.sh"
1307 script that then gets sourced outside of any subshell that may have
1308 been created by the test script at test exit time. (The "<n>" value
1309 is the number of the subtest within the test script and the script
1310 itself is stored in the TRASHTMP_DIRECTORY.)
1312 This means that `TESTLIB_TEST_NO_SUBSHELL` has no effect on this
1313 functionality nor do subshells created implicitly (or explicitly) by
1314 the test script itself.
1316 Git's version uses eval. To get the same effect make the <cmd> be
1317 "eval". However, that's not recommended as the eval will take place
1318 outside of any test script subshells which means none of the variables
1319 created inside the test script's subshells will be accessible at that time.
1320 Instead use the normal form (it's much simpler to use anyway) which causes
1321 any variables to be evaluated immediately when `test_when_finished` is
1322 called to schedule the "when finished" operation.
1324 This approach, however, precludes using `test_when_finished` to perform
1325 some action using a variable that has not yet been set. In that case the
1326 "eval" form will be needed and probably `TESTLIB_TEST_NO_SUBSHELL` will
1327 also have to be set to true to make it work.
1329 See also `test_clear_when_finished`.
1331 - test_write_lines <arg>...
1333 A lazy shortcut for:
1335 printf '%s\n' <arg>...
1337 Although since it's more characters to type perhaps it's not so lazy. ;)
1339 - verbose <cmd> [<arg>...]
1341 Run `<cmd> <arg>...` and verbosely complain (showing command and args) if
1342 it fails, but the exit status on failure will be converted to 1.
1344 - wc [-c|-m] [-lw] [<file>...]
1346 This is a wrapper around the wc utility. Some systems produce formatted
1347 output with extra spaces in it that this wrapper conveniently removes.
1349 BEWARE! The output from the system's native wc will have all leading and
1350 trailing spaces removed and all sequences of internal whitespace converted
1351 to a single space.
1353 Although it's not been disabled, avoid using wc with any <file> arguments!
1355 Parsing out the filenames from the result is error prone (and they will
1356 all end up on one line if there's more than one and any internal runs of
1357 whitespace in the name(s) will each be converted to a single space courtesy
1358 of the wrapper).
1360 Instead redirect standard input to the file or files to be counted and
1361 use `cat` if there's more than one file, something like this:
1363 lines_1=$(wc -l < file1)
1364 lines_2_plus_3=$(cat file2 file3 | wc -l)
1366 To test line counts in a test script, use something like this:
1368 printf "%s\n" one two three four >testfile &&
1369 lines=$(wc -l <testfile) &&
1370 test $lines -eq 4
1372 While the output of a command can be piped directly to wc that means
1373 the command's result status will be discarded (in favor of wc's if it's
1374 the last command in the pipeline) so that should generally be avoided
1375 (although the `cat` example above is probably okay to use).
1377 If you're using wc to count something on its standard input then you can
1378 safely use it just like the POSIX version without needing to be concerned
1379 about bogus whitespace showing up in the result which is the whole point
1380 of this wrapper in the first place.
1382 - write_script <file> [<shell-path>] < <script-text>
1384 Write a shebang line using <shell-path> (default is `$SHELL_PATH`) to
1385 <file> and then append <script-text> to it and then finally add execute
1386 permission to the resulting <file>.
1388 - yes [<expletive>]
1390 If <expletive> is omitted, `y` is used.
1392 Outputs "<expletive>" (and a trailing newline) 99 times.
1394 This is basically the same as the POSIX `yes` utility except that output
1395 is limited to 99 lines and it's implemented entirely as shell code.