libstdc++-v3/doc/html/manual/test.html

   1 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /><title>Test</title><meta name="generator" content="DocBook XSL Stylesheets V1.73.2" /><meta name="keywords" content="&#10;      ISO C++&#10;    , &#10;      test&#10;    , &#10;      testsuite&#10;    " /><meta name="keywords" content="&#10;      ISO C++&#10;    , &#10;      library&#10;    " /><link rel="start" href="../spine.html" title="The GNU C++ Library Documentation" /><link rel="up" href="bk01pt01ch02.html" title="Chapter 2. Setup" /><link rel="prev" href="build.html" title="Build" /><link rel="next" href="using.html" title="Chapter 3. Using" /></head><body><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">Test</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="build.html">Prev</a> </td><th width="60%" align="center">Chapter 2. Setup</th><td width="20%" align="right"> <a accesskey="n" href="using.html">Next</a></td></tr></table><hr /></div><div class="sect1" lang="en" xml:lang="en"><div class="titlepage"><div><div><h2 class="title" style="clear: both"><a id="manual.intro.setup.test"></a>Test</h2></div></div></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.organization"></a>Organization</h3></div></div></div><p>
   4   The directory <span class="emphasis"><em>libsrcdir/testsuite</em></span> contains the
   5   individual test cases organized in sub-directories corresponding to
   6   chapters of the C++ standard (detailed below), the dejagnu test
   7   harness support files, and sources to various testsuite utilities
   8   that are packaged in a separate testing library.
   9 </p><p>
  10   All test cases for functionality required by the runtime components
  11   of the C++ standard (ISO 14882) are files within the following
  12   directories.
  13 </p><pre class="programlisting">
  14 17_intro
  15 18_support
  16 19_diagnostics
  17 20_util
  18 21_strings
  19 22_locale
  20 23_containers
  21 25_algorithms
  22 26_numerics
  23 27_io
  24    </pre><p>
  25       In addition, the following directories include test files:
  26    </p><pre class="programlisting">
  27 tr1               Tests for components as described by the Technical Report on Standard Library Extensions (TR1).
  28 backward          Tests for backwards compatibility and deprecated features.
  29 demangle          Tests for __cxa_demangle, the IA 64 C++ ABI demangler
  30 ext               Tests for extensions.
  31 performance       Tests for performance analysis, and performance regressions.
  32 thread            Tests for threads.
  33    </pre><p>
  34       Some directories don't have test files, but instead contain
  35       auxiliary information (<a class="ulink" href="#internals" target="_top">more information</a>):
  36    </p><pre class="programlisting">
  37 config            Files for the dejagnu test harness.
  38 lib               Files for the dejagnu test harness.
  39 libstdc++*        Files for the dejagnu test harness.
  40 data              Sample text files for testing input and output.
  41 util              Files for libtestc++, utilities and testing routines.
  42    </pre><p>
  43       Within a directory that includes test files, there may be
  44       additional subdirectories, or files.  Originally, test cases
  45       were appended to one file that represented a particular section
  46       of the chapter under test, and was named accordingly. For
  47       instance, to test items related to <code class="code"> 21.3.6.1 -
  48       basic_string::find [lib.string::find]</code> in the standard,
  49       the following was used:
  50    </p><pre class="programlisting">
  51 21_strings/find.cc
  52    </pre><p>
  53       However, that practice soon became a liability as the test cases
  54       became huge and unwieldy, and testing new or extended
  55       functionality (like wide characters or named locales) became
  56       frustrating, leading to aggressive pruning of test cases on some
  57       platforms that covered up implementation errors. Now, the test
  58       suite has a policy of one file, one test case, which solves the
  59       above issues and gives finer grained results and more manageable
  60       error debugging. As an example, the test case quoted above
  61       becomes:
  62    </p><pre class="programlisting">
  63 21_strings/basic_string/find/char/1.cc
  64 21_strings/basic_string/find/char/2.cc
  65 21_strings/basic_string/find/char/3.cc
  66 21_strings/basic_string/find/wchar_t/1.cc
  67 21_strings/basic_string/find/wchar_t/2.cc
  68 21_strings/basic_string/find/wchar_t/3.cc
  69    </pre><p>
  70       All new tests should be written with the policy of one test
  71       case, one file in mind.
  72    </p></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.naming"></a>Naming Conventions</h3></div></div></div><p>
  73   </p><p>
  74       In addition, there are some special names and suffixes that are
  75       used within the testsuite to designate particular kinds of
  76       tests.
  77    </p><div class="itemizedlist"><ul type="disc"><li><p>
  78    <span class="emphasis"><em>_xin.cc</em></span>
  79   </p><p>
  80       This test case expects some kind of interactive input in order
  81       to finish or pass. At the moment, the interactive tests are not
  82       run by default. Instead, they are run by hand, like:
  83    </p><pre class="programlisting">
  84 g++ 27_io/objects/char/3_xin.cc
  85 cat 27_io/objects/char/3_xin.in | a.out
  86      </pre></li><li><p>
  87      <span class="emphasis"><em>.in</em></span>
  88    </p><p>
  89       This file contains the expected input for the corresponding <span class="emphasis"><em>
  90       _xin.cc</em></span> test case.
  91    </p></li><li><p>
  92    <span class="emphasis"><em>_neg.cc</em></span>
  93   </p><p>
  94       This test case is expected to fail: it's a negative test. At the
  95       moment, these are almost always compile time errors.
  96    </p></li><li><p>
  97    <span class="emphasis"><em>char</em></span>
  98   </p><p>
  99       This can either be a directory name or part of a longer file
 100       name, and indicates that this file, or the files within this
 101       directory are testing the <code class="code">char</code> instantiation of a
 102       template.
 103    </p></li><li><p>
 104    <span class="emphasis"><em>wchar_t</em></span>
 105   </p><p>
 106       This can either be a directory name or part of a longer file
 107       name, and indicates that this file, or the files within this
 108       directory are testing the <code class="code">wchar_t</code> instantiation of
 109       a template. Some hosts do not support <code class="code">wchar_t</code>
 110       functionality, so for these targets, all of these tests will not
 111       be run.
 112    </p></li><li><p>
 113    <span class="emphasis"><em>thread</em></span>
 114   </p><p>
 115       This can either be a directory name or part of a longer file
 116       name, and indicates that this file, or the files within this
 117       directory are testing situations where multiple threads are
 118       being used.
 119    </p></li><li><p>
 120    <span class="emphasis"><em>performance</em></span>
 121   </p><p>
 122       This can either be an enclosing directory name or part of a
 123       specific file name. This indicates a test that is used to
 124       analyze runtime performance, for performance regression testing,
 125       or for other optimization related analysis. At the moment, these
 126       test cases are not run by default.
 127    </p></li></ul></div></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.utils"></a>Utilities</h3></div></div></div><p>
 128   </p><p>
 129    The testsuite directory also contains some files that implement
 130    functionality that is intended to make writing test cases easier,
 131    or to avoid duplication, or to provide error checking in a way that
 132    is consistent across platforms and test harnesses. A stand-alone
 133    executable, called <span class="emphasis"><em>abi_check</em></span>, and a static
 134    library called <span class="emphasis"><em>libtestc++</em></span> are
 135    constructed. Both of these items are not installed, and only used
 136    during testing.
 137   </p><p>
 138   These files include the following functionality:
 139   </p><div class="itemizedlist"><ul type="disc"><li><p>
 140        <span class="emphasis"><em>testsuite_abi.h</em></span>,
 141        <span class="emphasis"><em>testsuite_abi.cc</em></span>,
 142        <span class="emphasis"><em>testsuite_abi_check.cc</em></span>
 143        </p><p>
 144         Creates the executable <span class="emphasis"><em>abi_check</em></span>.
 145         Used to check correctness of symbol versioning, visibility of
 146         exported symbols, and compatibility on symbols in the shared
 147         library, for hosts that support this feature. More information
 148         can be found in the ABI documentation <a class="ulink" href="abi.html" target="_top">here</a>
 149        </p></li><li><p>
 150        <span class="emphasis"><em>testsuite_allocator.h</em></span>,
 151        <span class="emphasis"><em>testsuite_allocator.cc</em></span>
 152        </p><p>
 153         Contains specialized allocators that keep track of construction
 154         and destruction. Also, support for overriding global new and
 155         delete operators, including verification that new and delete
 156         are called during execution, and that allocation over max_size
 157         fails.
 158        </p></li><li><p>
 159        <span class="emphasis"><em>testsuite_character.h</em></span>
 160        </p><p>
 161         Contains <code class="code">std::char_traits</code> and
 162         <code class="code">std::codecvt</code> specializations for a user-defined
 163         POD.
 164        </p></li><li><p>
 165        <span class="emphasis"><em>testsuite_hooks.h</em></span>,
 166        <span class="emphasis"><em>testsuite_hooks.cc</em></span>
 167        </p><p>
 168        A large number of utilities, including:
 169        </p><div class="itemizedlist"><ul type="circle"><li><p>VERIFY</p></li><li><p>set_memory_limits</p></li><li><p>verify_demangle</p></li><li><p>run_tests_wrapped_locale</p></li><li><p>run_tests_wrapped_env</p></li><li><p>try_named_locale</p></li><li><p>try_mkfifo</p></li><li><p>func_callback</p></li><li><p>counter</p></li><li><p>copy_tracker</p></li><li><p>copy_constructor</p></li><li><p>assignment_operator</p></li><li><p>destructor</p></li><li><p>pod_char, pod_int and associated char_traits specializations</p></li></ul></div></li><li><p>
 170          <span class="emphasis"><em>testsuite_io.h</em></span>
 171        </p><p>
 172        Error, exception, and constraint checking for
 173        <code class="code">std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>.
 174        </p></li><li><p>
 175          <span class="emphasis"><em>testsuite_iterators.h</em></span>
 176        </p><p>
 177          Wrappers for various iterators.
 178        </p></li><li><p>
 179          <span class="emphasis"><em>testsuite_performance.h</em></span>
 180        </p><p>
 181        A number of class abstractions for performance counters, and
 182        reporting functions including:
 183        </p><div class="itemizedlist"><ul type="circle"><li><p>time_counter</p></li><li><p>resource_counter</p></li><li><p>report_performance</p></li></ul></div></li></ul></div></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.run"></a>Running the Testsuite</h3></div></div></div><div class="sect3" lang="en" xml:lang="en"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.basic"></a>Basic Results</h4></div></div></div><p>
 184       There are several options for running tests, including testing
 185       the regression tests, testing a subset of the regression tests,
 186       testing the performance tests, testing just compilation, testing
 187       installed tools, etc. In addition, there is a special rule for
 188       checking the exported symbols of the shared library.
 189     </p><p>
 190       You can check the status of the build without installing it
 191       using the dejagnu harness, much like the rest of the gcc
 192       tools.</p><pre class="programlisting"> make check</pre><p>in the <span class="emphasis"><em>libbuilddir</em></span> directory.</p><p>or</p><pre class="programlisting"> make check-target-libstdc++-v3</pre><p>in the <span class="emphasis"><em>gccbuilddir</em></span> directory.
 193      </p><p>
 194        These commands are functionally equivalent and will create a
 195        'testsuite' directory underneath
 196        <span class="emphasis"><em>libbuilddir</em></span> containing the results of the
 197        tests. Two results files will be generated: <span class="emphasis"><em>
 198        libstdc++.sum</em></span>, which is a PASS/FAIL summary for each
 199        test, and <span class="emphasis"><em>libstdc++.log</em></span> which is a log of
 200        the exact command line passed to the compiler, the compiler
 201        output, and the executable output (if any).
 202      </p><p>
 203        Archives of test results for various versions and platforms are
 204        available on the GCC website in the <a class="ulink" href="http://gcc.gnu.org/gcc-4.1/buildstat.html" target="_top">build
 205        status</a> section of each individual release, and are also
 206        archived on a daily basis on the <a class="ulink" href="http://gcc.gnu.org/ml/gcc-testresults/current" target="_top">gcc-testresults</a>
 207        mailing list. Please check either of these places for a similar
 208        combination of source version, operating system, and host CPU.
 209      </p></div><div class="sect3" lang="en" xml:lang="en"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.options"></a>Options</h4></div></div></div><p>
 210       To debug the dejagnu test harness during runs, try invoking with a
 211       specific argument to the variable RUNTESTFLAGS, as below.
 212     </p><pre class="programlisting">
 213 make check-target-libstdc++-v3 RUNTESTFLAGS="-v"
 214 </pre><p>
 215       or
 216     </p><pre class="programlisting">
 217 make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v"
 218 </pre><p>
 219       To run a subset of the library tests, you will need to generate
 220       the <span class="emphasis"><em>testsuite_files</em></span> file by running
 221       <span class="command"><strong>make testsuite_files</strong></span> in the
 222       <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory, described
 223       below.  Edit the file to remove the tests you don't want and
 224       then run the testsuite as normal.
 225     </p><p>
 226       There are two ways to run on a simulator: set up DEJAGNU to point to a
 227       specially crafted site.exp, or pass down --target_board flags.
 228     </p><p>
 229     Example flags to pass down for various embedded builds are as follows:
 230     </p><pre class="programlisting">
 231       --target=powerpc-eabism (libgloss/sim)
 232 make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim"
 233
 234 --target=calmrisc32 (libgloss/sid)
 235 make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid"
 236
 237 --target=xscale-elf (newlib/sim)
 238 make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim"
 239 </pre><p>
 240       Also, here is an example of how to run the libstdc++ testsuite
 241       for a multilibed build directory with different ABI settings:
 242     </p><pre class="programlisting">
 243 make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"'
 244 </pre><p>
 245       You can run the tests with a compiler and library that have
 246       already been installed.  Make sure that the compiler (e.g.,
 247       <code class="code">g++</code>) is in your <code class="code">PATH</code>.  If you are
 248       using shared libraries, then you must also ensure that the
 249       directory containing the shared version of libstdc++ is in your
 250       <code class="code">LD_LIBRARY_PATH</code>, or equivalent.  If your GCC source
 251       tree is at <code class="code">/path/to/gcc</code>, then you can run the tests
 252       as follows:
 253     </p><pre class="programlisting">
 254 runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite
 255 </pre><p>
 256       The testsuite will create a number of files in the directory in
 257       which you run this command,.  Some of those files might use the
 258       same name as files created by other testsuites (like the ones
 259       for GCC and G++), so you should not try to run all the
 260       testsuites in parallel from the same directory.
 261     </p><p>
 262       In addition, there are some testing options that are mostly of
 263       interest to library maintainers and system integrators. As such,
 264       these tests may not work on all cpu and host combinations, and
 265       may need to be executed in the
 266       <span class="emphasis"><em>libbuilddir/testsuite</em></span> directory.  These
 267       options include, but are not necessarily limited to, the
 268       following:
 269    </p><pre class="programlisting">
 270    make testsuite_files
 271    </pre><p>
 272     Five files are generated that determine what test files
 273     are run. These files are:
 274   </p><div class="itemizedlist"><ul type="disc"><li><p>
 275          <span class="emphasis"><em>testsuite_files</em></span>
 276        </p><p>
 277          This is a list of all the test cases that will be run. Each
 278          test case is on a separate line, given with an absolute path
 279          from the <span class="emphasis"><em>libsrcdir/testsuite</em></span> directory.
 280        </p></li><li><p>
 281          <span class="emphasis"><em>testsuite_files_interactive</em></span>
 282        </p><p>
 283          This is a list of all the interactive test cases, using the
 284          same format as the file list above. These tests are not run
 285          by default.
 286      </p></li><li><p>
 287          <span class="emphasis"><em>testsuite_files_performance</em></span>
 288        </p><p>
 289          This is a list of all the performance test cases, using the
 290          same format as the file list above. These tests are not run
 291          by default.
 292      </p></li><li><p>
 293          <span class="emphasis"><em>testsuite_thread</em></span>
 294        </p><p>
 295          This file indicates that the host system can run tests which
 296          involved multiple threads.
 297        </p></li><li><p>
 298          <span class="emphasis"><em>testsuite_wchar_t</em></span>
 299        </p><p>
 300          This file indicates that the host system can run the wchar_t
 301          tests, and corresponds to the macro definition <code class="code">
 302          _GLIBCXX_USE_WCHAR_T</code> in the file c++config.h.
 303        </p></li></ul></div><pre class="programlisting">
 304    make check-abi
 305    </pre><p>
 306      The library ABI can be tested. This involves testing the shared
 307      library against an ABI-defining previous version of symbol
 308      exports.
 309    </p><pre class="programlisting">
 310    make check-compile
 311   </pre><p>
 312      This rule compiles, but does not link or execute, the
 313      <span class="emphasis"><em>testsuite_files</em></span> test cases and displays the
 314      output on stdout.
 315    </p><pre class="programlisting">
 316    make check-performance
 317    </pre><p>
 318      This rule runs through the
 319      <span class="emphasis"><em>testsuite_files_performance</em></span> test cases and
 320      collects information for performance analysis and can be used to
 321      spot performance regressions. Various timing information is
 322      collected, as well as number of hard page faults, and memory
 323      used. This is not run by default, and the implementation is in
 324      flux.
 325    </p><p>
 326       We are interested in any strange failures of the testsuite;
 327       please email the main libstdc++ mailing list if you see
 328       something odd or have questions.
 329    </p></div><div class="sect3" lang="en" xml:lang="en"><div class="titlepage"><div><div><h4 class="title"><a id="test.run.permutations"></a>Test Permutations</h4></div></div></div><p>
 330       To run the libstdc++ test suite under the <a class="link" href="debug_mode.html" title="Chapter 30. Debug Mode">debug mode</a>, edit
 331       <code class="filename">libstdc++-v3/scripts/testsuite_flags</code> to add the
 332       compile-time flag <code class="constant">-D_GLIBCXX_DEBUG</code> to the
 333       result printed by the <code class="literal">--build-cxx</code>
 334       option. Additionally, add the
 335       <code class="constant">-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on
 336       pedantic checking. The libstdc++ test suite should produce
 337       precisely the same results under debug mode that it does under
 338       release mode: any deviation indicates an error in either the
 339       library or the test suite.
 340     </p><p>
 341       Or, just run the testsuites with <code class="constant">CXXFLAGS</code>
 342       set to <code class="constant">-D_GLIBCXX_DEBUG</code>.
 343     </p></div></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.new_tests"></a>New Test Cases</h3></div></div></div><p>
 344     The first step in making a new test case is to choose the correct
 345     directory and file name, given the organization as previously
 346     described.
 347    </p><p>
 348     All files are copyright the FSF, and GPL'd: this is very
 349     important.  The first copyright year should correspond to the date
 350     the file was checked in to SVN.
 351    </p><p>
 352      As per the dejagnu instructions, always return 0 from main to
 353      indicate success.
 354    </p><p>
 355    A bunch of utility functions and classes have already been
 356    abstracted out into the testsuite utility library, <code class="code">
 357    libtestc++</code>. To use this functionality, just include the
 358    appropriate header file: the library or specific object files will
 359    automatically be linked in as part of the testsuite run.
 360    </p><p>
 361    For a test that needs to take advantage of the dejagnu test
 362    harness, what follows below is a list of special keyword that
 363    harness uses. Basically, a test case contains dg-keywords (see
 364    dg.exp) indicating what to do and what kinds of behavior are to be
 365    expected.  New test cases should be written with the new style
 366    DejaGnu framework in mind.
 367    </p><p>
 368     To ease transition, here is the list of dg-keyword documentation
 369     lifted from dg.exp.
 370    </p><pre class="programlisting">
 371 # The currently supported options are:
 372 #
 373 # dg-prms-id N
 374 #       set prms_id to N
 375 #
 376 # dg-options "options ..." [{ target selector }]
 377 #       specify special options to pass to the tool (eg: compiler)
 378 #
 379 # dg-do do-what-keyword [{ target/xfail selector }]
 380 #       `do-what-keyword' is tool specific and is passed unchanged to
 381 #       ${tool}-dg-test.  An example is gcc where `keyword' can be any of:
 382 #       preprocess|compile|assemble|link|run
 383 #       and will do one of: produce a .i, produce a .s, produce a .o,
 384 #       produce an a.out, or produce an a.out and run it (the default is
 385 #       compile).
 386 #
 387 # dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
 388 #       indicate an error message &lt;regexp&gt; is expected on this line
 389 #       (the test fails if it doesn't occur)
 390 #       Linenum=0 for general tool messages (eg: -V arg missing).
 391 #       "." means the current line.
 392 #
 393 # dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
 394 #       indicate a warning message &lt;regexp&gt; is expected on this line
 395 #       (the test fails if it doesn't occur)
 396 #
 397 # dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
 398 #       indicate a bogus error message &lt;regexp&gt; use to occur here
 399 #       (the test fails if it does occur)
 400 #
 401 # dg-build regexp comment [{ target/xfail selector }]
 402 #       indicate the build use to fail for some reason
 403 #       (errors covered here include bad assembler generated, tool crashes,
 404 #       and link failures)
 405 #       (the test fails if it does occur)
 406 #
 407 # dg-excess-errors comment [{ target/xfail selector }]
 408 #       indicate excess errors are expected (any line)
 409 #       (this should only be used sparingly and temporarily)
 410 #
 411 # dg-output regexp [{ target selector }]
 412 #       indicate the expected output of the program is &lt;regexp&gt;
 413 #       (there may be multiple occurrences of this, they are concatenated)
 414 #
 415 # dg-final { tcl code }
 416 #       add some tcl code to be run at the end
 417 #       (there may be multiple occurrences of this, they are concatenated)
 418 #       (unbalanced braces must be \-escaped)
 419 #
 420 # "{ target selector }" is a list of expressions that determine whether the
 421 # test succeeds or fails for a particular target, or in some cases whether the
 422 # option applies for a particular target.  If the case of `dg-do' it specifies
 423 # whether the test case is even attempted on the specified target.
 424 #
 425 # The target selector is always optional.  The format is one of:
 426 #
 427 # { xfail *-*-* ... } - the test is expected to fail for the given targets
 428 # { target *-*-* ... } - the option only applies to the given targets
 429 #
 430 # At least one target must be specified, use *-*-* for "all targets".
 431 # At present it is not possible to specify both `xfail' and `target'.
 432 # "native" may be used in place of "*-*-*".
 433
 434 Example 1: Testing compilation only
 435 // { dg-do compile }
 436
 437 Example 2: Testing for expected warnings on line 36, which all targets fail
 438 // { dg-warning "string literals" "" { xfail *-*-* } 36
 439
 440 Example 3: Testing for expected warnings on line 36
 441 // { dg-warning "string literals" "" { target *-*-* } 36
 442
 443 Example 4: Testing for compilation errors on line 41
 444 // { dg-do compile }
 445 // { dg-error "no match for" "" { target *-*-* } 41 }
 446
 447 Example 5: Testing with special command line settings, or without the
 448 use of pre-compiled headers, in particular the stdc++.h.gch file. Any
 449 options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set
 450 up in the normal.exp file.
 451 // { dg-options "-O0" { target *-*-* } }
 452 </pre><p>
 453     More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
 454    </p></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.dejagnu"></a>Test Harness Details</h3></div></div></div><p>
 455     Underlying details of testing are abstracted via the GNU Dejagnu package.
 456   </p><p>This is information for those looking at making changes to the testsuite
 457 structure, and/or needing to trace dejagnu's actions with --verbose.  This
 458 will not be useful to people who are "merely" adding new tests to the existing
 459 structure.
 460 </p><p>The first key point when working with dejagnu is the idea of a "tool".
 461 Files, directories, and functions are all implicitly used when they are
 462 named after the tool in use.  Here, the tool will always be "libstdc++".
 463 </p><p>The <code class="code">lib</code> subdir contains support routines.  The
 464 <code class="code">lib/libstdc++.exp</code> file ("support library") is loaded
 465 automagically, and must explicitly load the others.  For example, files can
 466 be copied from the core compiler's support directory into <code class="code">lib</code>.
 467 </p><p>Some routines in <code class="code">lib/libstdc++.exp</code> are callbacks, some are
 468 our own.  Callbacks must be prefixed with the name of the tool.  To easily
 469 distinguish the others, by convention our own routines are named "v3-*".
 470 </p><p>The next key point when working with dejagnu is "test files".  Any
 471 directory whose name starts with the tool name will be searched for test files.
 472 (We have only one.)  In those directories, any <code class="code">.exp</code> file is
 473 considered a test file, and will be run in turn.  Our main test file is called
 474 <code class="code">normal.exp</code>; it runs all the tests in testsuite_files using the
 475 callbacks loaded from the support library.
 476 </p><p>The <code class="code">config</code> directory is searched for any particular "target
 477 board" information unique to this library.  This is currently unused and sets
 478 only default variables.
 479 </p></div><div class="sect2" lang="en" xml:lang="en"><div class="titlepage"><div><div><h3 class="title"><a id="test.future"></a>Future</h3></div></div></div><p>
 480   </p><p>
 481 Shared runs need to be implemented, for targets that support shared libraries.
 482 </p><p>
 483 Diffing of expected output to standard streams needs to be finished off.
 484 </p><p>
 485 The V3 testing framework supports, or will eventually support,
 486 additional keywords for the purpose of easing the job of writing
 487 test cases.  All V3-keywords are of the form <code class="code">@xxx@</code>.
 488 Currently plans for supported keywords include:
 489 </p><div class="variablelist"><dl><dt><span class="term"> <code class="code"> @require@ &lt;files&gt; </code> </span></dt><dd><p>
 490       The existence of &lt;files&gt; is essential for the test to complete
 491       successfully.  For example, a test case foo.C using bar.baz as
 492       input file could say
 493    </p><pre class="programlisting">
 494             // @require@ bar.baz</pre><p>
 495       The special variable % stands for the rootname, e.g. the
 496       file-name without its `.C' extension.  Example of use (taken
 497       verbatim from 27_io/filebuf.cc)
 498    </p><pre class="programlisting">
 499            // @require@ %-*.tst %-*.txt</pre></dd><dt><span class="term"> <code class="code"> @diff@ &lt;first-list&gt; &lt;second-list&gt; </code> </span></dt><dd><p>
 500       After the test case compiles and ran successfully, diff
 501       &lt;first-list&gt; against &lt;second-list&gt;, these lists should
 502       have the same length.  The test fails if diff returns non-zero a
 503       pair of files.
 504    </p></dd></dl></div></div></div><div class="navfooter"><hr /><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="build.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="bk01pt01ch02.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="using.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">Build </td><td width="20%" align="center"><a accesskey="h" href="../spine.html">Home</a></td><td width="40%" align="right" valign="top"> Chapter 3. Using</td></tr></table></div></body></html>