1 <?xml version=
"1.0" encoding=
"UTF-8" standalone=
"no"?>
2 <!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns=
"http://www.w3.org/1999/xhtml"><head><meta http-equiv=
"Content-Type" content=
"text/html; charset=UTF-8" /><title>Test
</title><meta name=
"generator" content=
"DocBook XSL-NS Stylesheets V1.78.1" /><meta name=
"keywords" content=
"ISO C++, test, testsuite, performance, conformance, ABI, exception safety" /><meta name=
"keywords" content=
"ISO C++, library" /><meta name=
"keywords" content=
"ISO C++, runtime, library" /><link rel=
"home" href=
"../index.html" title=
"The GNU C++ Library" /><link rel=
"up" href=
"appendix_porting.html" title=
"Appendix B. Porting and Maintenance" /><link rel=
"prev" href=
"internals.html" title=
"Porting to New Hardware or Operating Systems" /><link rel=
"next" href=
"abi.html" title=
"ABI Policy and Guidelines" /></head><body><div class=
"navheader"><table width=
"100%" summary=
"Navigation header"><tr><th colspan=
"3" align=
"center">Test
</th></tr><tr><td width=
"20%" align=
"left"><a accesskey=
"p" href=
"internals.html">Prev
</a> </td><th width=
"60%" align=
"center">Appendix B.
3 Porting and Maintenance
5 </th><td width=
"20%" align=
"right"> <a accesskey=
"n" href=
"abi.html">Next
</a></td></tr></table><hr /></div><div class=
"section"><div class=
"titlepage"><div><div><h2 class=
"title" style=
"clear: both"><a id=
"manual.intro.setup.test"></a>Test
</h2></div></div></div><p>
6 The libstdc++ testsuite includes testing for standard conformance,
7 regressions, ABI, and performance.
8 </p><div class=
"section"><div class=
"titlepage"><div><div><h3 class=
"title"><a id=
"test.organization"></a>Organization
</h3></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.organization.layout"></a>Directory Layout
</h4></div></div></div><p>
9 The directory
<span class=
"emphasis"><em>libsrcdir/testsuite
</em></span> contains the
10 individual test cases organized in sub-directories corresponding to
11 clauses of the C++ standard (detailed below), the dejagnu test
12 harness support files, and sources to various testsuite utilities
13 that are packaged in a separate testing library.
15 All test cases for functionality required by the runtime components
16 of the C++ standard (ISO
14882) are files within the following
18 </p><pre class=
"programlisting">
33 In addition, the following directories include test files:
34 </p><pre class=
"programlisting">
35 tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1).
36 backward Tests for backwards compatibility and deprecated features.
37 demangle Tests for __cxa_demangle, the IA
64 C++ ABI demangler
38 ext Tests for extensions.
39 performance Tests for performance analysis, and performance regressions.
41 Some directories don't have test files, but instead contain
42 auxiliary information:
43 </p><pre class=
"programlisting">
44 config Files for the dejagnu test harness.
45 lib Files for the dejagnu test harness.
46 libstdc++* Files for the dejagnu test harness.
47 data Sample text files for testing input and output.
48 util Files for libtestc++, utilities and testing routines.
50 Within a directory that includes test files, there may be
51 additional subdirectories, or files. Originally, test cases
52 were appended to one file that represented a particular section
53 of the chapter under test, and was named accordingly. For
54 instance, to test items related to
<code class=
"code"> 21.3.6.1 -
55 basic_string::find [lib.string::find]
</code> in the standard,
56 the following was used:
57 </p><pre class=
"programlisting">
60 However, that practice soon became a liability as the test cases
61 became huge and unwieldy, and testing new or extended
62 functionality (like wide characters or named locales) became
63 frustrating, leading to aggressive pruning of test cases on some
64 platforms that covered up implementation errors. Now, the test
65 suite has a policy of one file, one test case, which solves the
66 above issues and gives finer grained results and more manageable
67 error debugging. As an example, the test case quoted above
69 </p><pre class=
"programlisting">
70 21_strings/basic_string/find/char/
1.cc
71 21_strings/basic_string/find/char/
2.cc
72 21_strings/basic_string/find/char/
3.cc
73 21_strings/basic_string/find/wchar_t/
1.cc
74 21_strings/basic_string/find/wchar_t/
2.cc
75 21_strings/basic_string/find/wchar_t/
3.cc
77 All new tests should be written with the policy of one test
78 case, one file in mind.
79 </p></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.organization.naming"></a>Naming Conventions
</h4></div></div></div><p>
80 In addition, there are some special names and suffixes that are
81 used within the testsuite to designate particular kinds of
83 </p><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: disc; "><li class=
"listitem"><p>
84 <span class=
"emphasis"><em>_xin.cc
</em></span>
86 This test case expects some kind of interactive input in order
87 to finish or pass. At the moment, the interactive tests are not
88 run by default. Instead, they are run by hand, like:
89 </p><pre class=
"programlisting">
90 g++
27_io/objects/char/
3_xin.cc
91 cat
27_io/objects/char/
3_xin.in | a.out
92 </pre></li><li class=
"listitem"><p>
93 <span class=
"emphasis"><em>.in
</em></span>
95 This file contains the expected input for the corresponding
<span class=
"emphasis"><em>
96 _xin.cc
</em></span> test case.
97 </p></li><li class=
"listitem"><p>
98 <span class=
"emphasis"><em>_neg.cc
</em></span>
100 This test case is expected to fail: it's a negative test. At the
101 moment, these are almost always compile time errors.
102 </p></li><li class=
"listitem"><p>
103 <span class=
"emphasis"><em>char
</em></span>
105 This can either be a directory name or part of a longer file
106 name, and indicates that this file, or the files within this
107 directory are testing the
<code class=
"code">char
</code> instantiation of a
109 </p></li><li class=
"listitem"><p>
110 <span class=
"emphasis"><em>wchar_t
</em></span>
112 This can either be a directory name or part of a longer file
113 name, and indicates that this file, or the files within this
114 directory are testing the
<code class=
"code">wchar_t
</code> instantiation of
115 a template. Some hosts do not support
<code class=
"code">wchar_t
</code>
116 functionality, so for these targets, all of these tests will not
118 </p></li><li class=
"listitem"><p>
119 <span class=
"emphasis"><em>thread
</em></span>
121 This can either be a directory name or part of a longer file
122 name, and indicates that this file, or the files within this
123 directory are testing situations where multiple threads are
125 </p></li><li class=
"listitem"><p>
126 <span class=
"emphasis"><em>performance
</em></span>
128 This can either be an enclosing directory name or part of a
129 specific file name. This indicates a test that is used to
130 analyze runtime performance, for performance regression testing,
131 or for other optimization related analysis. At the moment, these
132 test cases are not run by default.
133 </p></li></ul></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h3 class=
"title"><a id=
"test.run"></a>Running the Testsuite
</h3></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.run.basic"></a>Basic
</h4></div></div></div><p>
134 You can check the status of the build without installing it
135 using the dejagnu harness, much like the rest of the gcc
136 tools.
</p><pre class=
"programlisting"> make check
</pre><p>in the
<span class=
"emphasis"><em>libbuilddir
</em></span> directory.
</p><p>or
</p><pre class=
"programlisting"> make check-target-libstdc++-v3
</pre><p>in the
<span class=
"emphasis"><em>gccbuilddir
</em></span> directory.
138 These commands are functionally equivalent and will create a
139 'testsuite' directory underneath
140 <span class=
"emphasis"><em>libbuilddir
</em></span> containing the results of the
141 tests. Two results files will be generated:
<span class=
"emphasis"><em>
142 libstdc++.sum
</em></span>, which is a PASS/FAIL summary for each
143 test, and
<span class=
"emphasis"><em>libstdc++.log
</em></span> which is a log of
144 the exact command line passed to the compiler, the compiler
145 output, and the executable output (if any).
147 Archives of test results for various versions and platforms are
148 available on the GCC website in the
<a class=
"link" href=
"http://gcc.gnu.org/gcc-4.3/buildstat.html" target=
"_top">build
149 status
</a> section of each individual release, and are also
150 archived on a daily basis on the
<a class=
"link" href=
"http://gcc.gnu.org/ml/gcc-testresults/current" target=
"_top">gcc-testresults
</a>
151 mailing list. Please check either of these places for a similar
152 combination of source version, operating system, and host CPU.
153 </p></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.run.variations"></a>Variations
</h4></div></div></div><p>
154 There are several options for running tests, including testing
155 the regression tests, testing a subset of the regression tests,
156 testing the performance tests, testing just compilation, testing
157 installed tools, etc. In addition, there is a special rule for
158 checking the exported symbols of the shared library.
160 To debug the dejagnu test harness during runs, try invoking with a
161 specific argument to the variable RUNTESTFLAGS, as below.
162 </p><pre class=
"programlisting">
163 make check-target-libstdc++-v3
RUNTESTFLAGS=
"-v"
166 </p><pre class=
"programlisting">
167 make check-target-libstdc++-v3
RUNTESTFLAGS=
"-v -v"
169 To run a subset of the library tests, you will need to generate
170 the
<span class=
"emphasis"><em>testsuite_files
</em></span> file by running
171 <span class=
"command"><strong>make testsuite_files
</strong></span> in the
172 <span class=
"emphasis"><em>libbuilddir/testsuite
</em></span> directory, described
173 below. Edit the file to remove the tests you don't want and
174 then run the testsuite as normal.
176 There are two ways to run on a simulator: set up DEJAGNU to point to a
177 specially crafted site.exp, or pass down --target_board flags.
179 Example flags to pass down for various embedded builds are as follows:
180 </p><pre class=
"programlisting">
181 --target=powerpc-eabism (libgloss/sim)
182 make check-target-libstdc++-v3
RUNTESTFLAGS=
"--target_board=powerpc-sim"
184 --target=calmrisc32 (libgloss/sid)
185 make check-target-libstdc++-v3
RUNTESTFLAGS=
"--target_board=calmrisc32-sid"
187 --target=xscale-elf (newlib/sim)
188 make check-target-libstdc++-v3
RUNTESTFLAGS=
"--target_board=arm-sim"
190 Also, here is an example of how to run the libstdc++ testsuite
191 for a multilibed build directory with different ABI settings:
192 </p><pre class=
"programlisting">
193 make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \
"unix{-mabi=32,,-mabi=64}\"'
195 You can run the tests with a compiler and library that have
196 already been installed. Make sure that the compiler (e.g.,
197 <code class=
"code">g++
</code>) is in your
<code class=
"code">PATH
</code>. If you are
198 using shared libraries, then you must also ensure that the
199 directory containing the shared version of libstdc++ is in your
200 <code class=
"code">LD_LIBRARY_PATH
</code>, or equivalent. If your GCC source
201 tree is at
<code class=
"code">/path/to/gcc
</code>, then you can run the tests
203 </p><pre class=
"programlisting">
204 runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite
206 The testsuite will create a number of files in the directory in
207 which you run this command,. Some of those files might use the
208 same name as files created by other testsuites (like the ones
209 for GCC and G++), so you should not try to run all the
210 testsuites in parallel from the same directory.
212 In addition, there are some testing options that are mostly of
213 interest to library maintainers and system integrators. As such,
214 these tests may not work on all cpu and host combinations, and
215 may need to be executed in the
216 <span class=
"emphasis"><em>libbuilddir/testsuite
</em></span> directory. These
217 options include, but are not necessarily limited to, the
219 </p><pre class=
"programlisting">
222 Five files are generated that determine what test files
223 are run. These files are:
224 </p><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: disc; "><li class=
"listitem"><p>
225 <span class=
"emphasis"><em>testsuite_files
</em></span>
227 This is a list of all the test cases that will be run. Each
228 test case is on a separate line, given with an absolute path
229 from the
<span class=
"emphasis"><em>libsrcdir/testsuite
</em></span> directory.
230 </p></li><li class=
"listitem"><p>
231 <span class=
"emphasis"><em>testsuite_files_interactive
</em></span>
233 This is a list of all the interactive test cases, using the
234 same format as the file list above. These tests are not run
236 </p></li><li class=
"listitem"><p>
237 <span class=
"emphasis"><em>testsuite_files_performance
</em></span>
239 This is a list of all the performance test cases, using the
240 same format as the file list above. These tests are not run
242 </p></li><li class=
"listitem"><p>
243 <span class=
"emphasis"><em>testsuite_thread
</em></span>
245 This file indicates that the host system can run tests which
246 involved multiple threads.
247 </p></li><li class=
"listitem"><p>
248 <span class=
"emphasis"><em>testsuite_wchar_t
</em></span>
250 This file indicates that the host system can run the wchar_t
251 tests, and corresponds to the macro definition
<code class=
"code">
252 _GLIBCXX_USE_WCHAR_T
</code> in the file c++config.h.
253 </p></li></ul></div><pre class=
"programlisting">
256 The library ABI can be tested. This involves testing the shared
257 library against an ABI-defining previous version of symbol
259 </p><pre class=
"programlisting">
262 This rule compiles, but does not link or execute, the
263 <span class=
"emphasis"><em>testsuite_files
</em></span> test cases and displays the
265 </p><pre class=
"programlisting">
266 make check-performance
268 This rule runs through the
269 <span class=
"emphasis"><em>testsuite_files_performance
</em></span> test cases and
270 collects information for performance analysis and can be used to
271 spot performance regressions. Various timing information is
272 collected, as well as number of hard page faults, and memory
273 used. This is not run by default, and the implementation is in
276 We are interested in any strange failures of the testsuite;
277 please email the main libstdc++ mailing list if you see
278 something odd or have questions.
279 </p></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.run.permutations"></a>Permutations
</h4></div></div></div><p>
280 To run the libstdc++ test suite under the
<a class=
"link" href=
"debug_mode.html" title=
"Chapter 17. Debug Mode">debug mode
</a>, edit
281 <code class=
"filename">libstdc++-v3/scripts/testsuite_flags
</code> to add the
282 compile-time flag
<code class=
"constant">-D_GLIBCXX_DEBUG
</code> to the
283 result printed by the
<code class=
"literal">--build-cxx
</code>
284 option. Additionally, add the
285 <code class=
"constant">-D_GLIBCXX_DEBUG_PEDANTIC
</code> flag to turn on
286 pedantic checking. The libstdc++ test suite should produce
287 precisely the same results under debug mode that it does under
288 release mode: any deviation indicates an error in either the
289 library or the test suite.
291 The
<a class=
"link" href=
"parallel_mode.html" title=
"Chapter 18. Parallel Mode">parallel
292 mode
</a> can be tested in much the same manner, substituting
293 <code class=
"constant">-D_GLIBCXX_PARALLEL
</code> for
294 <code class=
"constant">-D_GLIBCXX_DEBUG
</code> in the previous paragraph.
296 Or, just run the testsuites with
<code class=
"constant">CXXFLAGS
</code>
297 set to
<code class=
"constant">-D_GLIBCXX_DEBUG
</code> or
298 <code class=
"constant">-D_GLIBCXX_PARALLEL
</code>.
299 </p></div></div><div class=
"section"><div class=
"titlepage"><div><div><h3 class=
"title"><a id=
"test.new_tests"></a>Writing a new test case
</h3></div></div></div><p>
300 The first step in making a new test case is to choose the correct
301 directory and file name, given the organization as previously
304 All files are copyright the FSF, and GPL'd: this is very
305 important. The first copyright year should correspond to the date
306 the file was checked in to SVN.
308 As per the dejagnu instructions, always return
0 from main to
311 A bunch of utility functions and classes have already been
312 abstracted out into the testsuite utility library,
<code class=
"code">
313 libtestc++
</code>. To use this functionality, just include the
314 appropriate header file: the library or specific object files will
315 automatically be linked in as part of the testsuite run.
317 For a test that needs to take advantage of the dejagnu test
318 harness, what follows below is a list of special keyword that
319 harness uses. Basically, a test case contains dg-keywords (see
320 dg.exp) indicating what to do and what kinds of behavior are to be
321 expected. New test cases should be written with the new style
322 DejaGnu framework in mind.
324 To ease transition, here is the list of dg-keyword documentation
326 </p><pre class=
"programlisting">
327 # The currently supported options are:
332 # dg-options
"options ..." [{ target selector }]
333 # specify special options to pass to the tool (eg: compiler)
335 # dg-do do-what-keyword [{ target/xfail selector }]
336 # `do-what-keyword' is tool specific and is passed unchanged to
337 # ${tool}-dg-test. An example is gcc where `keyword' can be any of:
338 # preprocess|compile|assemble|link|run
339 # and will do one of: produce a .i, produce a .s, produce a .o,
340 # produce an a.out, or produce an a.out and run it (the default is
343 # dg-error regexp comment [{ target/xfail selector } [{.|
0|linenum}]]
344 # indicate an error message
<regexp
> is expected on this line
345 # (the test fails if it doesn't occur)
346 # Linenum=
0 for general tool messages (eg: -V arg missing).
347 #
"." means the current line.
349 # dg-warning regexp comment [{ target/xfail selector } [{.|
0|linenum}]]
350 # indicate a warning message
<regexp
> is expected on this line
351 # (the test fails if it doesn't occur)
353 # dg-bogus regexp comment [{ target/xfail selector } [{.|
0|linenum}]]
354 # indicate a bogus error message
<regexp
> use to occur here
355 # (the test fails if it does occur)
357 # dg-build regexp comment [{ target/xfail selector }]
358 # indicate the build use to fail for some reason
359 # (errors covered here include bad assembler generated, tool crashes,
361 # (the test fails if it does occur)
363 # dg-excess-errors comment [{ target/xfail selector }]
364 # indicate excess errors are expected (any line)
365 # (this should only be used sparingly and temporarily)
367 # dg-output regexp [{ target selector }]
368 # indicate the expected output of the program is
<regexp
>
369 # (there may be multiple occurrences of this, they are concatenated)
371 # dg-final { tcl code }
372 # add some tcl code to be run at the end
373 # (there may be multiple occurrences of this, they are concatenated)
374 # (unbalanced braces must be \-escaped)
376 #
"{ target selector }" is a list of expressions that determine whether the
377 # test succeeds or fails for a particular target, or in some cases whether the
378 # option applies for a particular target. If the case of `dg-do' it specifies
379 # whether the test case is even attempted on the specified target.
381 # The target selector is always optional. The format is one of:
383 # { xfail *-*-* ... } - the test is expected to fail for the given targets
384 # { target *-*-* ... } - the option only applies to the given targets
386 # At least one target must be specified, use *-*-* for
"all targets".
387 # At present it is not possible to specify both `xfail' and `target'.
388 #
"native" may be used in place of
"*-*-*".
390 Example
1: Testing compilation only
393 Example
2: Testing for expected warnings on line
36, which all targets fail
394 // { dg-warning
"string literals" "" { xfail *-*-* }
36 }
396 Example
3: Testing for expected warnings on line
36
397 // { dg-warning
"string literals" "" { target *-*-* }
36 }
399 Example
4: Testing for compilation errors on line
41
401 // { dg-error
"no match for" "" { target *-*-* }
41 }
403 Example
5: Testing with special command line settings, or without the
404 use of pre-compiled headers, in particular the stdc++.h.gch file. Any
405 options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set
406 up in the normal.exp file.
407 // { dg-options
"-O0" { target *-*-* } }
409 More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
410 </p></div><div class=
"section"><div class=
"titlepage"><div><div><h3 class=
"title"><a id=
"test.harness"></a>Test Harness and Utilities
</h3></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.harness.dejagnu"></a>Dejagnu Harness Details
</h4></div></div></div><p>
411 Underlying details of testing for conformance and regressions are
412 abstracted via the GNU Dejagnu package. This is similar to the
414 </p><p>This is information for those looking at making changes to the testsuite
415 structure, and/or needing to trace dejagnu's actions with --verbose. This
416 will not be useful to people who are
"merely" adding new tests to the existing
418 </p><p>The first key point when working with dejagnu is the idea of a
"tool".
419 Files, directories, and functions are all implicitly used when they are
420 named after the tool in use. Here, the tool will always be
"libstdc++".
421 </p><p>The
<code class=
"code">lib
</code> subdir contains support routines. The
422 <code class=
"code">lib/libstdc++.exp
</code> file (
"support library") is loaded
423 automagically, and must explicitly load the others. For example, files can
424 be copied from the core compiler's support directory into
<code class=
"code">lib
</code>.
425 </p><p>Some routines in
<code class=
"code">lib/libstdc++.exp
</code> are callbacks, some are
426 our own. Callbacks must be prefixed with the name of the tool. To easily
427 distinguish the others, by convention our own routines are named
"v3-*".
428 </p><p>The next key point when working with dejagnu is
"test files". Any
429 directory whose name starts with the tool name will be searched for test files.
430 (We have only one.) In those directories, any
<code class=
"code">.exp
</code> file is
431 considered a test file, and will be run in turn. Our main test file is called
432 <code class=
"code">normal.exp
</code>; it runs all the tests in testsuite_files using the
433 callbacks loaded from the support library.
434 </p><p>The
<code class=
"code">config
</code> directory is searched for any particular
"target
435 board" information unique to this library. This is currently unused and sets
436 only default variables.
437 </p></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.harness.utils"></a>Utilities
</h4></div></div></div><p>
439 The testsuite directory also contains some files that implement
440 functionality that is intended to make writing test cases easier,
441 or to avoid duplication, or to provide error checking in a way that
442 is consistent across platforms and test harnesses. A stand-alone
443 executable, called
<span class=
"emphasis"><em>abi_check
</em></span>, and a static
444 library called
<span class=
"emphasis"><em>libtestc++
</em></span> are
445 constructed. Both of these items are not installed, and only used
448 These files include the following functionality:
449 </p><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: disc; "><li class=
"listitem"><p>
450 <span class=
"emphasis"><em>testsuite_abi.h
</em></span>,
451 <span class=
"emphasis"><em>testsuite_abi.cc
</em></span>,
452 <span class=
"emphasis"><em>testsuite_abi_check.cc
</em></span>
454 Creates the executable
<span class=
"emphasis"><em>abi_check
</em></span>.
455 Used to check correctness of symbol versioning, visibility of
456 exported symbols, and compatibility on symbols in the shared
457 library, for hosts that support this feature. More information
458 can be found in the ABI documentation
<a class=
"link" href=
"abi.html" title=
"ABI Policy and Guidelines">here
</a>
459 </p></li><li class=
"listitem"><p>
460 <span class=
"emphasis"><em>testsuite_allocator.h
</em></span>,
461 <span class=
"emphasis"><em>testsuite_allocator.cc
</em></span>
463 Contains specialized allocators that keep track of construction
464 and destruction. Also, support for overriding global new and
465 delete operators, including verification that new and delete
466 are called during execution, and that allocation over max_size
468 </p></li><li class=
"listitem"><p>
469 <span class=
"emphasis"><em>testsuite_character.h
</em></span>
471 Contains
<code class=
"code">std::char_traits
</code> and
472 <code class=
"code">std::codecvt
</code> specializations for a user-defined
474 </p></li><li class=
"listitem"><p>
475 <span class=
"emphasis"><em>testsuite_hooks.h
</em></span>,
476 <span class=
"emphasis"><em>testsuite_hooks.cc
</em></span>
478 A large number of utilities, including:
479 </p><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: circle; "><li class=
"listitem"><p>VERIFY
</p></li><li class=
"listitem"><p>set_memory_limits
</p></li><li class=
"listitem"><p>verify_demangle
</p></li><li class=
"listitem"><p>run_tests_wrapped_locale
</p></li><li class=
"listitem"><p>run_tests_wrapped_env
</p></li><li class=
"listitem"><p>try_named_locale
</p></li><li class=
"listitem"><p>try_mkfifo
</p></li><li class=
"listitem"><p>func_callback
</p></li><li class=
"listitem"><p>counter
</p></li><li class=
"listitem"><p>copy_tracker
</p></li><li class=
"listitem"><p>copy_constructor
</p></li><li class=
"listitem"><p>assignment_operator
</p></li><li class=
"listitem"><p>destructor
</p></li><li class=
"listitem"><p>pod_char, pod_int and associated char_traits specializations
</p></li></ul></div></li><li class=
"listitem"><p>
480 <span class=
"emphasis"><em>testsuite_io.h
</em></span>
482 Error, exception, and constraint checking for
483 <code class=
"code">std::streambuf, std::basic_stringbuf, std::basic_filebuf
</code>.
484 </p></li><li class=
"listitem"><p>
485 <span class=
"emphasis"><em>testsuite_iterators.h
</em></span>
487 Wrappers for various iterators.
488 </p></li><li class=
"listitem"><p>
489 <span class=
"emphasis"><em>testsuite_performance.h
</em></span>
491 A number of class abstractions for performance counters, and
492 reporting functions including:
493 </p><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: circle; "><li class=
"listitem"><p>time_counter
</p></li><li class=
"listitem"><p>resource_counter
</p></li><li class=
"listitem"><p>report_performance
</p></li></ul></div></li></ul></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h3 class=
"title"><a id=
"test.special"></a>Special Topics
</h3></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h4 class=
"title"><a id=
"test.exception.safety"></a>
494 Qualifying Exception Safety Guarantees
495 <a id=
"idm269884641824" class=
"indexterm"></a>
496 </h4></div></div></div><div class=
"section"><div class=
"titlepage"><div><div><h5 class=
"title"><a id=
"test.exception.safety.overview"></a>Overview
</h5></div></div></div><p>
497 Testing is composed of running a particular test sequence,
498 and looking at what happens to the surrounding code when
499 exceptions are thrown. Each test is composed of measuring
500 initial state, executing a particular sequence of code under
501 some instrumented conditions, measuring a final state, and
502 then examining the differences between the two states.
504 Test sequences are composed of constructed code sequences
505 that exercise a particular function or member function, and
506 either confirm no exceptions were generated, or confirm the
507 consistency/coherency of the test subject in the event of a
510 Random code paths can be constructed using the basic test
511 sequences and instrumentation as above, only combined in a
512 random or pseudo-random way.
513 </p><p> To compute the code paths that throw, test instruments
514 are used that throw on allocation events
515 (
<code class=
"classname">__gnu_cxx::throw_allocator_random
</code>
516 and
<code class=
"classname">__gnu_cxx::throw_allocator_limit
</code>)
517 and copy, assignment, comparison, increment, swap, and
519 (
<code class=
"classname">__gnu_cxx::throw_type_random
</code>
520 and
<code class=
"classname">__gnu_cxx::throw_type_limit
</code>). Looping
521 through a given test sequence and conditionally throwing in
522 all instrumented places. Then, when the test sequence
523 completes without an exception being thrown, assume all
524 potential error paths have been exercised in a sequential
526 </p></div><div class=
"section"><div class=
"titlepage"><div><div><h5 class=
"title"><a id=
"test.exception.safety.status"></a>
528 </h5></div></div></div><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: disc; "><li class=
"listitem"><p>
532 <code class=
"filename">testsuite/
23_containers/list/modifiers/
3.cc
</code>.
533 </p></li><li class=
"listitem"><p>
534 Policy Based Data Structures
536 For example, take the test
537 functor
<code class=
"classname">rand_reg_test
</code> in
538 in
<code class=
"filename">testsuite/ext/pb_ds/regression/tree_no_data_map_rand.cc
</code>. This uses
<code class=
"classname">container_rand_regression_test
</code> in
539 <code class=
"filename">testsuite/util/regression/rand/assoc/container_rand_regression_test.h
</code>.
542 Which has several tests for container member functions,
543 Includes control and test container objects. Configuration includes
544 random seed, iterations, number of distinct values, and the
545 probability that an exception will be thrown. Assumes instantiating
546 container uses an extension
547 allocator,
<code class=
"classname">__gnu_cxx::throw_allocator_random
</code>,
548 as the allocator type.
549 </p></li><li class=
"listitem"><p>
550 C++
11 Container Requirements.
552 Coverage is currently limited to testing container
553 requirements for exception safety,
554 although
<code class=
"classname">__gnu_cxx::throw_type
</code> meets
555 the additional type requirements for testing numeric data
556 structures and instantiating algorithms.
558 Of particular interest is extending testing to algorithms and
559 then to parallel algorithms. Also io and locales.
561 The test instrumentation should also be extended to add
562 instrumentation to
<code class=
"classname">iterator
</code>
563 and
<code class=
"classname">const_iterator
</code> types that throw
564 conditionally on iterator operations.
565 </p></li></ul></div></div><div class=
"section"><div class=
"titlepage"><div><div><h5 class=
"title"><a id=
"test.exception.safety.containers"></a>
566 C++
11 Requirements Test Sequence Descriptions
567 </h5></div></div></div><div class=
"itemizedlist"><ul class=
"itemizedlist" style=
"list-style-type: disc; "><li class=
"listitem"><p>
570 Basic consistency on exception propagation tests. For
571 each container, an object of that container is constructed,
572 a specific member function is exercised in
573 a
<code class=
"literal">try
</code> block, and then any thrown
574 exceptions lead to error checking in the appropriate
575 <code class=
"literal">catch
</code> block. The container's use of
576 resources is compared to the container's use prior to the
577 test block. Resource monitoring is limited to allocations
578 made through the container's
<span class=
"type">allocator_type
</span>,
579 which should be sufficient for container data
580 structures. Included in these tests are member functions
581 are
<span class=
"type">iterator
</span> and
<span class=
"type">const_iterator
</span>
582 operations,
<code class=
"function">pop_front
</code>,
<code class=
"function">pop_back
</code>,
<code class=
"function">push_front
</code>,
<code class=
"function">push_back
</code>,
<code class=
"function">insert
</code>,
<code class=
"function">erase
</code>,
<code class=
"function">swap
</code>,
<code class=
"function">clear
</code>,
583 and
<code class=
"function">rehash
</code>. The container in question is
584 instantiated with two instrumented template arguments,
585 with
<code class=
"classname">__gnu_cxx::throw_allocator_limit
</code>
586 as the allocator type, and
587 with
<code class=
"classname">__gnu_cxx::throw_type_limit
</code> as
588 the value type. This allows the test to loop through
589 conditional throw points.
591 The general form is demonstrated in
592 <code class=
"filename">testsuite/
23_containers/list/requirements/exception/basic.cc
593 </code>. The instantiating test object is
<code class=
"classname">__gnu_test::basic_safety
</code> and is detailed in
<code class=
"filename">testsuite/util/exception/safety.h
</code>.
594 </p></li><li class=
"listitem"><p>
595 Generation Prohibited
597 Exception generation tests. For each container, an object of
598 that container is constructed and all member functions
599 required to not throw exceptions are exercised. Included in
600 these tests are member functions
601 are
<span class=
"type">iterator
</span> and
<span class=
"type">const_iterator
</span> operations,
<code class=
"function">erase
</code>,
<code class=
"function">pop_front
</code>,
<code class=
"function">pop_back
</code>,
<code class=
"function">swap
</code>,
602 and
<code class=
"function">clear
</code>. The container in question is
603 instantiated with two instrumented template arguments,
604 with
<code class=
"classname">__gnu_cxx::throw_allocator_random
</code>
605 as the allocator type, and
606 with
<code class=
"classname">__gnu_cxx::throw_type_random
</code> as
607 the value type. This test does not loop, an instead is sudden
608 death: first error fails.
610 The general form is demonstrated in
611 <code class=
"filename">testsuite/
23_containers/list/requirements/exception/generation_prohibited.cc
612 </code>. The instantiating test object is
<code class=
"classname">__gnu_test::generation_prohibited
</code> and is detailed in
<code class=
"filename">testsuite/util/exception/safety.h
</code>.
613 </p></li><li class=
"listitem"><p>
614 Propagation Consistent
616 Container rollback on exception propagation tests. For
617 each container, an object of that container is constructed,
618 a specific member function that requires rollback to a previous
619 known good state is exercised in
620 a
<code class=
"literal">try
</code> block, and then any thrown
621 exceptions lead to error checking in the appropriate
622 <code class=
"literal">catch
</code> block. The container is compared to
623 the container's last known good state using such parameters
624 as size, contents, and iterator references. Included in these
625 tests are member functions
626 are
<code class=
"function">push_front
</code>,
<code class=
"function">push_back
</code>,
<code class=
"function">insert
</code>,
627 and
<code class=
"function">rehash
</code>. The container in question is
628 instantiated with two instrumented template arguments,
629 with
<code class=
"classname">__gnu_cxx::throw_allocator_limit
</code>
630 as the allocator type, and
631 with
<code class=
"classname">__gnu_cxx::throw_type_limit
</code> as
632 the value type. This allows the test to loop through
633 conditional throw points.
635 The general form demonstrated in
636 <code class=
"filename">testsuite/
23_containers/list/requirements/exception/propagation_coherent.cc
637 </code>. The instantiating test object is
<code class=
"classname">__gnu_test::propagation_coherent
</code> and is detailed in
<code class=
"filename">testsuite/util/exception/safety.h
</code>.
638 </p></li></ul></div></div></div></div></div><div class=
"navfooter"><hr /><table width=
"100%" summary=
"Navigation footer"><tr><td width=
"40%" align=
"left"><a accesskey=
"p" href=
"internals.html">Prev
</a> </td><td width=
"20%" align=
"center"><a accesskey=
"u" href=
"appendix_porting.html">Up
</a></td><td width=
"40%" align=
"right"> <a accesskey=
"n" href=
"abi.html">Next
</a></td></tr><tr><td width=
"40%" align=
"left" valign=
"top">Porting to New Hardware or Operating Systems
</td><td width=
"20%" align=
"center"><a accesskey=
"h" href=
"../index.html">Home
</a></td><td width=
"40%" align=
"right" valign=
"top"> ABI Policy and Guidelines
</td></tr></table></div></body></html>