mingw/html/lib/Benchmark.html

   1 <?xml version="1.0" ?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml">
   4 <head>
   5 <title>Benchmark - benchmark running times of Perl code</title>
   6 <meta http-equiv="content-type" content="text/html; charset=utf-8" />
   7 <link rev="made" href="mailto:" />
   8 </head>
   9
  10 <body style="background-color: white">
  11 <table border="0" width="100%" cellspacing="0" cellpadding="3">
  12 <tr><td class="block" style="background-color: #cccccc" valign="middle">
  13 <big><strong><span class="block">&nbsp;Benchmark - benchmark running times of Perl code</span></strong></big>
  14 </td></tr>
  15 </table>
  16
  17 <p><a name="__index__"></a></p>
  18 <!-- INDEX BEGIN -->
  19
  20 <ul>
  21
  22         <li><a href="#name">NAME</a></li>
  23         <li><a href="#synopsis">SYNOPSIS</a></li>
  24         <li><a href="#description">DESCRIPTION</a></li>
  25         <ul>
  26
  27                 <li><a href="#methods">Methods</a></li>
  28                 <li><a href="#standard_exports">Standard Exports</a></li>
  29                 <li><a href="#optional_exports">Optional Exports</a></li>
  30                 <li><a href="#_hireswallclock">:hireswallclock</a></li>
  31         </ul>
  32
  33         <li><a href="#notes">NOTES</a></li>
  34         <li><a href="#examples">EXAMPLES</a></li>
  35         <li><a href="#inheritance">INHERITANCE</a></li>
  36         <li><a href="#caveats">CAVEATS</a></li>
  37         <li><a href="#see_also">SEE ALSO</a></li>
  38         <li><a href="#authors">AUTHORS</a></li>
  39         <li><a href="#modification_history">MODIFICATION HISTORY</a></li>
  40 </ul>
  41 <!-- INDEX END -->
  42
  43 <hr />
  44 <p>
  45 </p>
  46 <h1><a name="name">NAME</a></h1>
  47 <p>Benchmark - benchmark running times of Perl code</p>
  48 <p>
  49 </p>
  50 <hr />
  51 <h1><a name="synopsis">SYNOPSIS</a></h1>
  52 <pre>
  53     use Benchmark qw(:all) ;</pre>
  54 <pre>
  55     timethis ($count, &quot;code&quot;);</pre>
  56 <pre>
  57     # Use Perl code in strings...
  58     timethese($count, {
  59         'Name1' =&gt; '...code1...',
  60         'Name2' =&gt; '...code2...',
  61     });</pre>
  62 <pre>
  63     # ... or use subroutine references.
  64     timethese($count, {
  65         'Name1' =&gt; sub { ...code1... },
  66         'Name2' =&gt; sub { ...code2... },
  67     });</pre>
  68 <pre>
  69     # cmpthese can be used both ways as well
  70     cmpthese($count, {
  71         'Name1' =&gt; '...code1...',
  72         'Name2' =&gt; '...code2...',
  73     });</pre>
  74 <pre>
  75     cmpthese($count, {
  76         'Name1' =&gt; sub { ...code1... },
  77         'Name2' =&gt; sub { ...code2... },
  78     });</pre>
  79 <pre>
  80     # ...or in two stages
  81     $results = timethese($count,
  82         {
  83             'Name1' =&gt; sub { ...code1... },
  84             'Name2' =&gt; sub { ...code2... },
  85         },
  86         'none'
  87     );
  88     cmpthese( $results ) ;</pre>
  89 <pre>
  90     $t = timeit($count, '...other code...')
  91     print &quot;$count loops of other code took:&quot;,timestr($t),&quot;\n&quot;;</pre>
  92 <pre>
  93     $t = countit($time, '...other code...')
  94     $count = $t-&gt;iters ;
  95     print &quot;$count loops of other code took:&quot;,timestr($t),&quot;\n&quot;;</pre>
  96 <pre>
  97     # enable hires wallclock timing if possible
  98     use Benchmark ':hireswallclock';</pre>
  99 <p>
 100 </p>
 101 <hr />
 102 <h1><a name="description">DESCRIPTION</a></h1>
 103 <p>The Benchmark module encapsulates a number of routines to help you
 104 figure out how long it takes to execute some code.</p>
 105 <p>timethis - run a chunk of code several times</p>
 106 <p>timethese - run several chunks of code several times</p>
 107 <p>cmpthese - print results of timethese as a comparison chart</p>
 108 <p>timeit - run a chunk of code and see how long it goes</p>
 109 <p>countit - see how many times a chunk of code runs in a given time</p>
 110 <p>
 111 </p>
 112 <h2><a name="methods">Methods</a></h2>
 113 <dl>
 114 <dt><strong><a name="item_new">new</a></strong>
 115
 116 <dd>
 117 <p>Returns the current time.   Example:</p>
 118 </dd>
 119 <dd>
 120 <pre>
 121     use Benchmark;
 122     $t0 = new Benchmark;
 123     # ... your code here ...
 124     $t1 = new Benchmark;
 125     $td = timediff($t1, $t0);
 126     print &quot;the code took:&quot;,timestr($td),&quot;\n&quot;;</pre>
 127 </dd>
 128 </li>
 129 <dt><strong><a name="item_debug">debug</a></strong>
 130
 131 <dd>
 132 <p>Enables or disable debugging by setting the <code>$Benchmark::Debug</code> flag:</p>
 133 </dd>
 134 <dd>
 135 <pre>
 136     debug Benchmark 1;
 137     $t = timeit(10, ' 5 ** $Global ');
 138     debug Benchmark 0;</pre>
 139 </dd>
 140 </li>
 141 <dt><strong><a name="item_iters">iters</a></strong>
 142
 143 <dd>
 144 <p>Returns the number of iterations.</p>
 145 </dd>
 146 </li>
 147 </dl>
 148 <p>
 149 </p>
 150 <h2><a name="standard_exports">Standard Exports</a></h2>
 151 <p>The following routines will be exported into your namespace
 152 if you use the Benchmark module:</p>
 153 <dl>
 154 <dt><strong><a name="item_timeit">timeit(COUNT, CODE)</a></strong>
 155
 156 <dd>
 157 <p>Arguments: COUNT is the number of times to run the loop, and CODE is
 158 the code to run.  CODE may be either a code reference or a string to
 159 be eval'd; either way it will be run in the caller's package.</p>
 160 </dd>
 161 <dd>
 162 <p>Returns: a Benchmark object.</p>
 163 </dd>
 164 </li>
 165 <dt><strong><a name="item_timethis">timethis ( COUNT, CODE, [ TITLE, [ STYLE ]] )</a></strong>
 166
 167 <dd>
 168 <p>Time COUNT iterations of CODE. CODE may be a string to eval or a
 169 code reference; either way the CODE will run in the caller's package.
 170 Results will be printed to STDOUT as TITLE followed by the times.
 171 TITLE defaults to ``timethis COUNT'' if none is provided. STYLE
 172 determines the format of the output, as described for <a href="#item_timestr"><code>timestr()</code></a> below.</p>
 173 </dd>
 174 <dd>
 175 <p>The COUNT can be zero or negative: this means the <em>minimum number of
 176 CPU seconds</em> to run.  A zero signifies the default of 3 seconds.  For
 177 example to run at least for 10 seconds:</p>
 178 </dd>
 179 <dd>
 180 <pre>
 181         timethis(-10, $code)</pre>
 182 </dd>
 183 <dd>
 184 <p>or to run two pieces of code tests for at least 3 seconds:</p>
 185 </dd>
 186 <dd>
 187 <pre>
 188         timethese(0, { test1 =&gt; '...', test2 =&gt; '...'})</pre>
 189 </dd>
 190 <dd>
 191 <p>CPU seconds is, in UNIX terms, the user time plus the system time of
 192 the process itself, as opposed to the real (wallclock) time and the
 193 time spent by the child processes.  Less than 0.1 seconds is not
 194 accepted (-0.01 as the count, for example, will cause a fatal runtime
 195 exception).</p>
 196 </dd>
 197 <dd>
 198 <p>Note that the CPU seconds is the <strong>minimum</strong> time: CPU scheduling and
 199 other operating system factors may complicate the attempt so that a
 200 little bit more time is spent.  The benchmark output will, however,
 201 also tell the number of <code>$code</code> runs/second, which should be a more
 202 interesting number than the actually spent seconds.</p>
 203 </dd>
 204 <dd>
 205 <p>Returns a Benchmark object.</p>
 206 </dd>
 207 </li>
 208 <dt><strong><a name="item_timethese">timethese ( COUNT, CODEHASHREF, [ STYLE ] )</a></strong>
 209
 210 <dd>
 211 <p>The CODEHASHREF is a reference to a hash containing names as keys
 212 and either a string to eval or a code reference for each value.
 213 For each (KEY, VALUE) pair in the CODEHASHREF, this routine will
 214 call</p>
 215 </dd>
 216 <dd>
 217 <pre>
 218         timethis(COUNT, VALUE, KEY, STYLE)</pre>
 219 </dd>
 220 <dd>
 221 <p>The routines are called in string comparison order of KEY.</p>
 222 </dd>
 223 <dd>
 224 <p>The COUNT can be zero or negative, see timethis().</p>
 225 </dd>
 226 <dd>
 227 <p>Returns a hash of Benchmark objects, keyed by name.</p>
 228 </dd>
 229 </li>
 230 <dt><strong><a name="item_timediff">timediff ( T1, T2 )</a></strong>
 231
 232 <dd>
 233 <p>Returns the difference between two Benchmark times as a Benchmark
 234 object suitable for passing to timestr().</p>
 235 </dd>
 236 </li>
 237 <dt><strong><a name="item_timestr">timestr ( TIMEDIFF, [ STYLE, [ FORMAT ] ] )</a></strong>
 238
 239 <dd>
 240 <p>Returns a string that formats the times in the TIMEDIFF object in
 241 the requested STYLE. TIMEDIFF is expected to be a Benchmark object
 242 similar to that returned by timediff().</p>
 243 </dd>
 244 <dd>
 245 <p>STYLE can be any of 'all', 'none', 'noc', 'nop' or 'auto'. 'all' shows
 246 each of the 5 times available ('wallclock' time, user time, system time,
 247 user time of children, and system time of children). 'noc' shows all
 248 except the two children times. 'nop' shows only wallclock and the
 249 two children times. 'auto' (the default) will act as 'all' unless
 250 the children times are both zero, in which case it acts as 'noc'.
 251 'none' prevents output.</p>
 252 </dd>
 253 <dd>
 254 <p>FORMAT is the <a href="file://C|\msysgit\mingw\html/pod/perlfunc.html#item_printf">printf(3)</a>-style format specifier (without the
 255 leading '%') to use to print the times. It defaults to '5.2f'.</p>
 256 </dd>
 257 </li>
 258 </dl>
 259 <p>
 260 </p>
 261 <h2><a name="optional_exports">Optional Exports</a></h2>
 262 <p>The following routines will be exported into your namespace
 263 if you specifically ask that they be imported:</p>
 264 <dl>
 265 <dt><strong><a name="item_clearcache">clearcache ( COUNT )</a></strong>
 266
 267 <dd>
 268 <p>Clear the cached time for COUNT rounds of the null loop.</p>
 269 </dd>
 270 </li>
 271 <dt><strong><a name="item_clearallcache">clearallcache ( )</a></strong>
 272
 273 <dd>
 274 <p>Clear all cached times.</p>
 275 </dd>
 276 </li>
 277 <dt><strong><a name="item_cmpthese">cmpthese ( COUNT, CODEHASHREF, [ STYLE ] )</a></strong>
 278
 279 <dt><strong>cmpthese ( RESULTSHASHREF, [ STYLE ] )</strong>
 280
 281 <dd>
 282 <p>Optionally calls timethese(), then outputs comparison chart.  This:</p>
 283 </dd>
 284 <dd>
 285 <pre>
 286     cmpthese( -1, { a =&gt; &quot;++\$i&quot;, b =&gt; &quot;\$i *= 2&quot; } ) ;</pre>
 287 </dd>
 288 <dd>
 289 <p>outputs a chart like:</p>
 290 </dd>
 291 <dd>
 292 <pre>
 293            Rate    b    a
 294     b 2831802/s   -- -61%
 295     a 7208959/s 155%   --</pre>
 296 </dd>
 297 <dd>
 298 <p>This chart is sorted from slowest to fastest, and shows the percent speed
 299 difference between each pair of tests.</p>
 300 </dd>
 301 <dd>
 302 <p>c&lt;cmpthese&gt; can also be passed the data structure that <a href="#item_timethese"><code>timethese()</code></a> returns:</p>
 303 </dd>
 304 <dd>
 305 <pre>
 306     $results = timethese( -1, { a =&gt; &quot;++\$i&quot;, b =&gt; &quot;\$i *= 2&quot; } ) ;
 307     cmpthese( $results );</pre>
 308 </dd>
 309 <dd>
 310 <p>in case you want to see both sets of results.</p>
 311 </dd>
 312 <dd>
 313 <p>Returns a reference to an ARRAY of rows, each row is an ARRAY of cells from the
 314 above chart, including labels. This:</p>
 315 </dd>
 316 <dd>
 317 <pre>
 318     my $rows = cmpthese( -1, { a =&gt; '++$i', b =&gt; '$i *= 2' }, &quot;none&quot; );</pre>
 319 </dd>
 320 <dd>
 321 <p>returns a data structure like:</p>
 322 </dd>
 323 <dd>
 324 <pre>
 325     [
 326         [ '',       'Rate',   'b',    'a' ],
 327         [ 'b', '2885232/s',  '--', '-59%' ],
 328         [ 'a', '7099126/s', '146%',  '--' ],
 329     ]</pre>
 330 </dd>
 331 <dd>
 332 <p><strong>NOTE</strong>: This result value differs from previous versions, which returned
 333 the <a href="#item_timethese"><code>timethese()</code></a> result structure.  If you want that, just use the two
 334 statement <a href="#item_timethese"><code>timethese</code></a>...<a href="#item_cmpthese"><code>cmpthese</code></a> idiom shown above.</p>
 335 </dd>
 336 <dd>
 337 <p>Incidently, note the variance in the result values between the two examples;
 338 this is typical of benchmarking.  If this were a real benchmark, you would
 339 probably want to run a lot more iterations.</p>
 340 </dd>
 341 </li>
 342 <dt><strong><a name="item_countit">countit(TIME, CODE)</a></strong>
 343
 344 <dd>
 345 <p>Arguments: TIME is the minimum length of time to run CODE for, and CODE is
 346 the code to run.  CODE may be either a code reference or a string to
 347 be eval'd; either way it will be run in the caller's package.</p>
 348 </dd>
 349 <dd>
 350 <p>TIME is <em>not</em> negative.  <a href="#item_countit"><code>countit()</code></a> will run the loop many times to
 351 calculate the speed of CODE before running it for TIME.  The actual
 352 time run for will usually be greater than TIME due to system clock
 353 resolution, so it's best to look at the number of iterations divided
 354 by the times that you are concerned with, not just the iterations.</p>
 355 </dd>
 356 <dd>
 357 <p>Returns: a Benchmark object.</p>
 358 </dd>
 359 </li>
 360 <dt><strong><a name="item_disablecache">disablecache ( )</a></strong>
 361
 362 <dd>
 363 <p>Disable caching of timings for the null loop. This will force Benchmark
 364 to recalculate these timings for each new piece of code timed.</p>
 365 </dd>
 366 </li>
 367 <dt><strong><a name="item_enablecache">enablecache ( )</a></strong>
 368
 369 <dd>
 370 <p>Enable caching of timings for the null loop. The time taken for COUNT
 371 rounds of the null loop will be calculated only once for each
 372 different COUNT used.</p>
 373 </dd>
 374 </li>
 375 <dt><strong><a name="item_timesum">timesum ( T1, T2 )</a></strong>
 376
 377 <dd>
 378 <p>Returns the sum of two Benchmark times as a Benchmark object suitable
 379 for passing to timestr().</p>
 380 </dd>
 381 </li>
 382 </dl>
 383 <p>
 384 </p>
 385 <h2><a name="_hireswallclock">:hireswallclock</a></h2>
 386 <p>If the Time::HiRes module has been installed, you can specify the
 387 special tag <code>:hireswallclock</code> for Benchmark (if Time::HiRes is not
 388 available, the tag will be silently ignored).  This tag will cause the
 389 wallclock time to be measured in microseconds, instead of integer
 390 seconds.  Note though that the speed computations are still conducted
 391 in CPU time, not wallclock time.</p>
 392 <p>
 393 </p>
 394 <hr />
 395 <h1><a name="notes">NOTES</a></h1>
 396 <p>The data is stored as a list of values from the time and times
 397 functions:</p>
 398 <pre>
 399       ($real, $user, $system, $children_user, $children_system, $iters)</pre>
 400 <p>in seconds for the whole loop (not divided by the number of rounds).</p>
 401 <p>The timing is done using <code>time(3)</code> and times(3).</p>
 402 <p>Code is executed in the caller's package.</p>
 403 <p>The time of the null loop (a loop with the same
 404 number of rounds but empty loop body) is subtracted
 405 from the time of the real loop.</p>
 406 <p>The null loop times can be cached, the key being the
 407 number of rounds. The caching can be controlled using
 408 calls like these:</p>
 409 <pre>
 410     clearcache($key);
 411     clearallcache();</pre>
 412 <pre>
 413     disablecache();
 414     enablecache();</pre>
 415 <p>Caching is off by default, as it can (usually slightly) decrease
 416 accuracy and does not usually noticably affect runtimes.</p>
 417 <p>
 418 </p>
 419 <hr />
 420 <h1><a name="examples">EXAMPLES</a></h1>
 421 <p>For example,</p>
 422 <pre>
 423     use Benchmark qw( cmpthese ) ;
 424     $x = 3;
 425     cmpthese( -5, {
 426         a =&gt; sub{$x*$x},
 427         b =&gt; sub{$x**2},
 428     } );</pre>
 429 <p>outputs something like this:</p>
 430 <pre>
 431    Benchmark: running a, b, each for at least 5 CPU seconds...
 432           Rate    b    a
 433    b 1559428/s   -- -62%
 434    a 4152037/s 166%   --</pre>
 435 <p>while</p>
 436 <pre>
 437     use Benchmark qw( timethese cmpthese ) ;
 438     $x = 3;
 439     $r = timethese( -5, {
 440         a =&gt; sub{$x*$x},
 441         b =&gt; sub{$x**2},
 442     } );
 443     cmpthese $r;</pre>
 444 <p>outputs something like this:</p>
 445 <pre>
 446     Benchmark: running a, b, each for at least 5 CPU seconds...
 447              a: 10 wallclock secs ( 5.14 usr +  0.13 sys =  5.27 CPU) @ 3835055.60/s (n=20210743)
 448              b:  5 wallclock secs ( 5.41 usr +  0.00 sys =  5.41 CPU) @ 1574944.92/s (n=8520452)
 449            Rate    b    a
 450     b 1574945/s   -- -59%
 451     a 3835056/s 144%   --</pre>
 452 <p>
 453 </p>
 454 <hr />
 455 <h1><a name="inheritance">INHERITANCE</a></h1>
 456 <p>Benchmark inherits from no other class, except of course
 457 for Exporter.</p>
 458 <p>
 459 </p>
 460 <hr />
 461 <h1><a name="caveats">CAVEATS</a></h1>
 462 <p>Comparing eval'd strings with code references will give you
 463 inaccurate results: a code reference will show a slightly slower
 464 execution time than the equivalent eval'd string.</p>
 465 <p>The real time timing is done using <code>time(2)</code> and
 466 the granularity is therefore only one second.</p>
 467 <p>Short tests may produce negative figures because perl
 468 can appear to take longer to execute the empty loop
 469 than a short test; try:</p>
 470 <pre>
 471     timethis(100,'1');</pre>
 472 <p>The system time of the null loop might be slightly
 473 more than the system time of the loop with the actual
 474 code and therefore the difference might end up being &lt; 0.</p>
 475 <p>
 476 </p>
 477 <hr />
 478 <h1><a name="see_also">SEE ALSO</a></h1>
 479 <p><a href="file://C|\msysgit\mingw\html/lib/Devel/DProf.html">the Devel::DProf manpage</a> - a Perl code profiler</p>
 480 <p>
 481 </p>
 482 <hr />
 483 <h1><a name="authors">AUTHORS</a></h1>
 484 <p>Jarkko Hietaniemi &lt;<em><a href="mailto:jhi@iki.fi">jhi@iki.fi</a></em>&gt;, Tim Bunce &lt;<em><a href="mailto:Tim.Bunce@ig.co.uk">Tim.Bunce@ig.co.uk</a></em>&gt;</p>
 485 <p>
 486 </p>
 487 <hr />
 488 <h1><a name="modification_history">MODIFICATION HISTORY</a></h1>
 489 <p>September 8th, 1994; by Tim Bunce.</p>
 490 <p>March 28th, 1997; by Hugo van der Sanden: added support for code
 491 references and the already documented 'debug' method; revamped
 492 documentation.</p>
 493 <p>April 04-07th, 1997: by Jarkko Hietaniemi, added the run-for-some-time
 494 functionality.</p>
 495 <p>September, 1999; by Barrie Slaymaker: math fixes and accuracy and
 496 efficiency tweaks.  Added cmpthese().  A result is now returned from
 497 timethese().  Exposed <a href="#item_countit"><code>countit()</code></a> (was runfor()).</p>
 498 <p>December, 2001; by Nicholas Clark: make <a href="#item_timestr"><code>timestr()</code></a> recognise the style 'none'
 499 and return an empty string. If cmpthese is calling timethese, make it pass the
 500 style in. (so that 'none' will suppress output). Make sub new dump its
 501 debugging output to STDERR, to be consistent with everything else.
 502 All bugs found while writing a regression test.</p>
 503 <p>September, 2002; by Jarkko Hietaniemi: add ':hireswallclock' special tag.</p>
 504 <p>February, 2004; by Chia-liang Kao: make cmpthese and timestr use time
 505 statistics for children instead of parent when the style is 'nop'.</p>
 506 <table border="0" width="100%" cellspacing="0" cellpadding="3">
 507 <tr><td class="block" style="background-color: #cccccc" valign="middle">
 508 <big><strong><span class="block">&nbsp;Benchmark - benchmark running times of Perl code</span></strong></big>
 509 </td></tr>
 510 </table>
 511
 512 </body>
 513
 514 </html>