6 Sparse has a number of test cases in its validation directory. The test-suite
7 script aims at making automated checking of these tests possible. It works by
8 embedding tags in C comments in the test cases.
10 check-name: (mandatory)
13 check-description: (optional)
14 A description of what the test checks.
16 check-command: (optional)
17 There are different kinds of tests. Some can validate the sparse
18 preprocessor, while others will use sparse, cgcc, or even other backends
19 of the library. check-command allows you to give a custom command to
21 The '$file' string is special. It will be expanded to the file name at
23 It defaults to "sparse $file".
25 check-exit-value: (optional)
26 The expected exit value of check-command. It defaults to 0.
28 check-output-start / check-output-end (optional)
29 The expected output (stdout and stderr) of check-command lies between
30 those two tags. It defaults to no output.
32 check-output-ignore / check-error-ignore (optional)
33 Don't check the expected output (stdout or stderr) of check-command
34 (usefull when this output is not comparable or if you're only interested
36 By default this check is done.
38 check-known-to-fail (optional)
39 Mark the test as being known to fail.
41 check-output-contains: <pattern> (optional)
42 Check that the output (stdout) contains the given pattern.
43 Several such tags can be given, in which case the output
44 must contains all the patterns.
46 check-output-excludes: <pattern> (optional)
47 Similar than the above one, but with opposite logic.
48 Check that the output (stdout) doesn't contain the given pattern.
49 Several such tags can be given, in which case the output
50 must contains none of the patterns.
52 check-output-pattern-<nbr>-times: <pattern> (optional)
53 Similar than the contains/excludes her above, but with full control
54 of the number of times the pattern should occurs in the output.
59 The test-suite script is called through the check target of the Makefile. It
60 will try to check every test case it finds (find validation -name '*.c').
62 It can be called to check a single test with:
64 $ ./test-suite single preprocessor/preprocessor1.c
65 TEST Preprocessor #1 (preprocessor/preprocessor1.c)
66 preprocessor/preprocessor1.c passed !
72 test-suite comes with a format command to make a test easier to write:
74 test-suite format file [name [cmd]]
77 check-name value. If no name is provided, it defaults to the file name.
79 check-command value. If no cmd is provided, it defaults to
82 The output of the test-suite format command can be redirected into the
83 test case to create a test-suite formated file.
85 $ ./test-suite format bad-assignment.c Assignment >> bad-assignment.c
89 * check-name: bad assignment
91 * check-command: sparse $file
95 bad-assignment.c:3:6: error: Expected ; at end of statement
96 bad-assignment.c:3:6: error: got \
100 You can define the check-command you want to use for the test. $file will be
101 extended to the file name at run time.
103 $ ./test-suite format validation/preprocessor2.c "Preprocessor #2" \
104 "sparse -E \$file" >> validation/preprocessor2.c
106 cat validation/preprocessor2.c
108 * This one we happen to get right.
110 * It should result in a simple
114 * for a proper preprocessor.
118 #define UNARY(x) BINARY(x)
119 #define BINARY(x, y) x + y
123 * check-name: Preprocessor #2
125 * check-command: sparse -E $file
126 * check-exit-value: 0