mingw/html/lib/Test/Harness/Straps.html

   1 <?xml version="1.0" ?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml">
   4 <head>
   5 <title>Test::Harness::Straps - detailed analysis of test results</title>
   6 <meta http-equiv="content-type" content="text/html; charset=utf-8" />
   7 <link rev="made" href="mailto:" />
   8 </head>
   9
  10 <body style="background-color: white">
  11 <table border="0" width="100%" cellspacing="0" cellpadding="3">
  12 <tr><td class="block" style="background-color: #cccccc" valign="middle">
  13 <big><strong><span class="block">&nbsp;Test::Harness::Straps - detailed analysis of test results</span></strong></big>
  14 </td></tr>
  15 </table>
  16
  17 <p><a name="__index__"></a></p>
  18 <!-- INDEX BEGIN -->
  19
  20 <ul>
  21
  22         <li><a href="#name">NAME</a></li>
  23         <li><a href="#synopsis">SYNOPSIS</a></li>
  24         <li><a href="#description">DESCRIPTION</a></li>
  25         <li><a href="#construction">CONSTRUCTION</a></li>
  26         <ul>
  27
  28                 <li><a href="#new__"><code>new()</code></a></li>
  29                 <li><a href="#_strap__init">$strap-&gt;_init</a></li>
  30         </ul>
  31
  32         <li><a href="#analysis">ANALYSIS</a></li>
  33         <ul>
  34
  35                 <li><a href="#_strap_analyze___name____output_lines__">$strap-&gt;analyze( $name, \@output_lines )</a></li>
  36                 <li><a href="#_strap_analyze_fh___name___test_filehandle__">$strap-&gt;analyze_fh( $name, $test_filehandle )</a></li>
  37                 <li><a href="#_strap_analyze_file___test_file__">$strap-&gt;analyze_file( $test_file )</a></li>
  38                 <li><a href="#_strap__command_line___file__">$strap-&gt;_command_line( $file )</a></li>
  39                 <li><a href="#_strap__command__">$strap-&gt;<code>_command()</code></a></li>
  40                 <li><a href="#_strap__switches___file__">$strap-&gt;_switches( $file )</a></li>
  41                 <li><a href="#_strap__cleaned_switches___switches_from_user__">$strap-&gt;_cleaned_switches( @switches_from_user )</a></li>
  42                 <li><a href="#_strap__inc2perl5lib">$strap-&gt;_INC2PERL5LIB</a></li>
  43                 <li><a href="#_strap__filtered_inc__">$strap-&gt;<code>_filtered_INC()</code></a></li>
  44                 <li><a href="#_strap__restore_perl5lib__">$strap-&gt;<code>_restore_PERL5LIB()</code></a></li>
  45         </ul>
  46
  47         <li><a href="#parsing">Parsing</a></li>
  48         <ul>
  49
  50                 <li><a href="#_is_diagnostic"><code>_is_diagnostic</code></a></li>
  51                 <li><a href="#_is_header"><code>_is_header</code></a></li>
  52                 <li><a href="#_is_bail_out"><code>_is_bail_out</code></a></li>
  53                 <li><a href="#_reset_file_state"><code>_reset_file_state</code></a></li>
  54         </ul>
  55
  56         <li><a href="#results">Results</a></li>
  57         <li><a href="#examples">EXAMPLES</a></li>
  58         <li><a href="#author">AUTHOR</a></li>
  59         <li><a href="#see_also">SEE ALSO</a></li>
  60 </ul>
  61 <!-- INDEX END -->
  62
  63 <hr />
  64 <p>
  65 </p>
  66 <h1><a name="name">NAME</a></h1>
  67 <p>Test::Harness::Straps - detailed analysis of test results</p>
  68 <p>
  69 </p>
  70 <hr />
  71 <h1><a name="synopsis">SYNOPSIS</a></h1>
  72 <pre>
  73   use Test::Harness::Straps;</pre>
  74 <pre>
  75   my $strap = Test::Harness::Straps-&gt;new;</pre>
  76 <pre>
  77   # Various ways to interpret a test
  78   my %results = $strap-&gt;analyze($name, \@test_output);
  79   my %results = $strap-&gt;analyze_fh($name, $test_filehandle);
  80   my %results = $strap-&gt;analyze_file($test_file);</pre>
  81 <pre>
  82   # UNIMPLEMENTED
  83   my %total = $strap-&gt;total_results;</pre>
  84 <pre>
  85   # Altering the behavior of the strap  UNIMPLEMENTED
  86   my $verbose_output = $strap-&gt;dump_verbose();
  87   $strap-&gt;dump_verbose_fh($output_filehandle);</pre>
  88 <p>
  89 </p>
  90 <hr />
  91 <h1><a name="description">DESCRIPTION</a></h1>
  92 <p><strong>THIS IS ALPHA SOFTWARE</strong> in that the interface is subject to change
  93 in incompatible ways.  It is otherwise stable.</p>
  94 <p>Test::Harness is limited to printing out its results.  This makes
  95 analysis of the test results difficult for anything but a human.  To
  96 make it easier for programs to work with test results, we provide
  97 Test::Harness::Straps.  Instead of printing the results, straps
  98 provide them as raw data.  You can also configure how the tests are to
  99 be run.</p>
 100 <p>The interface is currently incomplete.  <em>Please</em> contact the author
 101 if you'd like a feature added or something change or just have
 102 comments.</p>
 103 <p>
 104 </p>
 105 <hr />
 106 <h1><a name="construction">CONSTRUCTION</a></h1>
 107 <p>
 108 </p>
 109 <h2><a name="new__"><code>new()</code></a></h2>
 110 <pre>
 111   my $strap = Test::Harness::Straps-&gt;new;</pre>
 112 <p>Initialize a new strap.</p>
 113 <p>
 114 </p>
 115 <h2><a name="_strap__init">$strap-&gt;_init</a></h2>
 116 <pre>
 117   $strap-&gt;_init;</pre>
 118 <p>Initialize the internal state of a strap to make it ready for parsing.</p>
 119 <p>
 120 </p>
 121 <hr />
 122 <h1><a name="analysis">ANALYSIS</a></h1>
 123 <p>
 124 </p>
 125 <h2><a name="_strap_analyze___name____output_lines__">$strap-&gt;analyze( $name, \@output_lines )</a></h2>
 126 <pre>
 127     my %results = $strap-&gt;analyze($name, \@test_output);</pre>
 128 <p>Analyzes the output of a single test, assigning it the given <code>$name</code>
 129 for use in the total report.  Returns the <code>%results</code> of the test.
 130 See <em>Results</em>.</p>
 131 <p><code>@test_output</code> should be the raw output from the test, including
 132 newlines.</p>
 133 <p>
 134 </p>
 135 <h2><a name="_strap_analyze_fh___name___test_filehandle__">$strap-&gt;analyze_fh( $name, $test_filehandle )</a></h2>
 136 <pre>
 137     my %results = $strap-&gt;analyze_fh($name, $test_filehandle);</pre>
 138 <p>Like <code>analyze</code>, but it reads from the given filehandle.</p>
 139 <p>
 140 </p>
 141 <h2><a name="_strap_analyze_file___test_file__">$strap-&gt;analyze_file( $test_file )</a></h2>
 142 <pre>
 143     my %results = $strap-&gt;analyze_file($test_file);</pre>
 144 <p>Like <code>analyze</code>, but it runs the given <code>$test_file</code> and parses its
 145 results.  It will also use that name for the total report.</p>
 146 <p>
 147 </p>
 148 <h2><a name="_strap__command_line___file__">$strap-&gt;_command_line( $file )</a></h2>
 149 <p>Returns the full command line that will be run to test <em>$file</em>.</p>
 150 <p>
 151 </p>
 152 <h2><a name="_strap__command__">$strap-&gt;<code>_command()</code></a></h2>
 153 <p>Returns the command that runs the test.  Combine this with <code>_switches()</code>
 154 to build a command line.</p>
 155 <p>Typically this is <a href="file://C|\msysgit\mingw\html/pod/perlvar.html#item___x"><code>$^X</code></a>, but you can set <code>$ENV{HARNESS_PERL}</code>
 156 to use a different Perl than what you're running the harness under.
 157 This might be to run a threaded Perl, for example.</p>
 158 <p>You can also overload this method if you've built your own strap subclass,
 159 such as a PHP interpreter for a PHP-based strap.</p>
 160 <p>
 161 </p>
 162 <h2><a name="_strap__switches___file__">$strap-&gt;_switches( $file )</a></h2>
 163 <p>Formats and returns the switches necessary to run the test.</p>
 164 <p>
 165 </p>
 166 <h2><a name="_strap__cleaned_switches___switches_from_user__">$strap-&gt;_cleaned_switches( @switches_from_user )</a></h2>
 167 <p>Returns only defined, non-blank, trimmed switches from the parms passed.</p>
 168 <p>
 169 </p>
 170 <h2><a name="_strap__inc2perl5lib">$strap-&gt;_INC2PERL5LIB</a></h2>
 171 <pre>
 172   local $ENV{PERL5LIB} = $self-&gt;_INC2PERL5LIB;</pre>
 173 <p>Takes the current value of <a href="file://C|\msysgit\mingw\html/pod/perlvar.html#item__inc"><code>@INC</code></a> and turns it into something suitable
 174 for putting onto <code>PERL5LIB</code>.</p>
 175 <p>
 176 </p>
 177 <h2><a name="_strap__filtered_inc__">$strap-&gt;<code>_filtered_INC()</code></a></h2>
 178 <pre>
 179   my @filtered_inc = $self-&gt;_filtered_INC;</pre>
 180 <p>Shortens <a href="file://C|\msysgit\mingw\html/pod/perlvar.html#item__inc"><code>@INC</code></a> by removing redundant and unnecessary entries.
 181 Necessary for OSes with limited command line lengths, like VMS.</p>
 182 <p>
 183 </p>
 184 <h2><a name="_strap__restore_perl5lib__">$strap-&gt;<code>_restore_PERL5LIB()</code></a></h2>
 185 <pre>
 186   $self-&gt;_restore_PERL5LIB;</pre>
 187 <p>This restores the original value of the <code>PERL5LIB</code> environment variable.
 188 Necessary on VMS, otherwise a no-op.</p>
 189 <p>
 190 </p>
 191 <hr />
 192 <h1><a name="parsing">Parsing</a></h1>
 193 <p>Methods for identifying what sort of line you're looking at.</p>
 194 <p>
 195 </p>
 196 <h2><a name="_is_diagnostic"><code>_is_diagnostic</code></a></h2>
 197 <pre>
 198     my $is_diagnostic = $strap-&gt;_is_diagnostic($line, \$comment);</pre>
 199 <p>Checks if the given line is a comment.  If so, it will place it into
 200 <code>$comment</code> (sans #).</p>
 201 <p>
 202 </p>
 203 <h2><a name="_is_header"><code>_is_header</code></a></h2>
 204 <pre>
 205   my $is_header = $strap-&gt;_is_header($line);</pre>
 206 <p>Checks if the given line is a header (1..M) line.  If so, it places how
 207 many tests there will be in <code>$strap-&gt;{max}</code>, a list of which tests
 208 are todo in <code>$strap-&gt;{todo}</code> and if the whole test was skipped
 209 <code>$strap-&gt;{skip_all}</code> contains the reason.</p>
 210 <p>
 211 </p>
 212 <h2><a name="_is_bail_out"><code>_is_bail_out</code></a></h2>
 213 <pre>
 214   my $is_bail_out = $strap-&gt;_is_bail_out($line, \$reason);</pre>
 215 <p>Checks if the line is a ``Bail out!''.  Places the reason for bailing
 216 (if any) in $reason.</p>
 217 <p>
 218 </p>
 219 <h2><a name="_reset_file_state"><code>_reset_file_state</code></a></h2>
 220 <pre>
 221   $strap-&gt;_reset_file_state;</pre>
 222 <p>Resets things like <code>$strap-&gt;{max}</code> , <code>$strap-&gt;{skip_all}</code>,
 223 etc. so it's ready to parse the next file.</p>
 224 <p>
 225 </p>
 226 <hr />
 227 <h1><a name="results">Results</a></h1>
 228 <p>The <code>%results</code> returned from <code>analyze()</code> contain the following
 229 information:</p>
 230 <pre>
 231   passing           true if the whole test is considered a pass
 232                     (or skipped), false if its a failure</pre>
 233 <pre>
 234   exit              the exit code of the test run, if from a file
 235   wait              the wait code of the test run, if from a file</pre>
 236 <pre>
 237   max               total tests which should have been run
 238   seen              total tests actually seen
 239   skip_all          if the whole test was skipped, this will
 240                       contain the reason.</pre>
 241 <pre>
 242   ok                number of tests which passed
 243                       (including todo and skips)</pre>
 244 <pre>
 245   todo              number of todo tests seen
 246   bonus             number of todo tests which
 247                       unexpectedly passed</pre>
 248 <pre>
 249   skip              number of tests skipped</pre>
 250 <p>So a successful test should have max == seen == ok.</p>
 251 <p>There is one final item, the details.</p>
 252 <pre>
 253   details           an array ref reporting the result of
 254                     each test looks like this:</pre>
 255 <pre>
 256     $results{details}[$test_num - 1] =
 257             { ok          =&gt; is the test considered ok?
 258               actual_ok   =&gt; did it literally say 'ok'?
 259               name        =&gt; name of the test (if any)
 260               diagnostics =&gt; test diagnostics (if any)
 261               type        =&gt; 'skip' or 'todo' (if any)
 262               reason      =&gt; reason for the above (if any)
 263             };</pre>
 264 <p>Element 0 of the details is test #1.  I tried it with element 1 being
 265 #1 and 0 being empty, this is less awkward.</p>
 266 <p>
 267 </p>
 268 <hr />
 269 <h1><a name="examples">EXAMPLES</a></h1>
 270 <p>See <em>examples/mini_harness.plx</em> for an example of use.</p>
 271 <p>
 272 </p>
 273 <hr />
 274 <h1><a name="author">AUTHOR</a></h1>
 275 <p>Michael G Schwern <code>&lt;schwern@pobox.com&gt;</code>, currently maintained by
 276 Andy Lester <code>&lt;andy@petdance.com&gt;</code>.</p>
 277 <p>
 278 </p>
 279 <hr />
 280 <h1><a name="see_also">SEE ALSO</a></h1>
 281 <p><a href="file://C|\msysgit\mingw\html/lib/Test/Harness.html">the Test::Harness manpage</a></p>
 282 <table border="0" width="100%" cellspacing="0" cellpadding="3">
 283 <tr><td class="block" style="background-color: #cccccc" valign="middle">
 284 <big><strong><span class="block">&nbsp;Test::Harness::Straps - detailed analysis of test results</span></strong></big>
 285 </td></tr>
 286 </table>
 287
 288 </body>
 289
 290 </html>