mingw/html/lib/DBM_Filter.html

   1 <?xml version="1.0" ?>
   2 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   3 <html xmlns="http://www.w3.org/1999/xhtml">
   4 <head>
   5 <title>DBM_Filter -- Filter DBM keys/values</title>
   6 <meta http-equiv="content-type" content="text/html; charset=utf-8" />
   7 <link rev="made" href="mailto:" />
   8 </head>
   9
  10 <body style="background-color: white">
  11 <table border="0" width="100%" cellspacing="0" cellpadding="3">
  12 <tr><td class="block" style="background-color: #cccccc" valign="middle">
  13 <big><strong><span class="block">&nbsp;DBM_Filter -- Filter DBM keys/values</span></strong></big>
  14 </td></tr>
  15 </table>
  16
  17 <p><a name="__index__"></a></p>
  18 <!-- INDEX BEGIN -->
  19
  20 <ul>
  21
  22         <li><a href="#name">NAME</a></li>
  23         <li><a href="#synopsis">SYNOPSIS</a></li>
  24         <li><a href="#description">DESCRIPTION</a></li>
  25         <li><a href="#what_is_a_dbm_filter">What is a DBM Filter?</a></li>
  26         <ul>
  27
  28                 <li><a href="#so_what_s_new">So what's new?</a></li>
  29         </ul>
  30
  31         <li><a href="#methods">METHODS</a></li>
  32         <ul>
  33
  34                 <li><a href="#_db_filter_push__">$db-&gt;<code>Filter_Push()</code></a></li>
  35                 <li><a href="#_db_filter_key_push__">$db-&gt;<code>Filter_Key_Push()</code></a></li>
  36                 <li><a href="#_db_filter_value_push__">$db-&gt;<code>Filter_Value_Push()</code></a></li>
  37                 <li><a href="#_db_filter_pop__">$db-&gt;<code>Filter_Pop()</code></a></li>
  38                 <li><a href="#_db_filtered__">$db-&gt;<code>Filtered()</code></a></li>
  39         </ul>
  40
  41         <li><a href="#writing_a_filter">Writing a Filter</a></li>
  42         <ul>
  43
  44                 <li><a href="#immediate_filters">Immediate Filters</a></li>
  45                 <li><a href="#canned_filters">Canned Filters</a></li>
  46         </ul>
  47
  48         <li><a href="#filters_included">Filters Included</a></li>
  49         <li><a href="#notes">NOTES</a></li>
  50         <ul>
  51
  52                 <li><a href="#maintain_round_trip_integrity">Maintain Round Trip Integrity</a></li>
  53                 <li><a href="#don_t_mix_filtered___nonfiltered_data_in_the_same_database_file_">Don't mix filtered &amp; non-filtered data in the same database file.</a></li>
  54         </ul>
  55
  56         <li><a href="#example">EXAMPLE</a></li>
  57         <li><a href="#see_also">SEE ALSO</a></li>
  58         <li><a href="#author">AUTHOR</a></li>
  59 </ul>
  60 <!-- INDEX END -->
  61
  62 <hr />
  63 <p>
  64 </p>
  65 <h1><a name="name">NAME</a></h1>
  66 <p>DBM_Filter -- Filter DBM keys/values</p>
  67 <p>
  68 </p>
  69 <hr />
  70 <h1><a name="synopsis">SYNOPSIS</a></h1>
  71 <pre>
  72     use DBM_Filter ;
  73     use SDBM_File; # or DB_File, or GDBM_File, or NDBM_File, or ODBM_File</pre>
  74 <pre>
  75     $db = tie %hash, ...</pre>
  76 <pre>
  77     $db-&gt;Filter_Push(Fetch =&gt; sub {...},
  78                      Store =&gt; sub {...});</pre>
  79 <pre>
  80     $db-&gt;Filter_Push('my_filter1');
  81     $db-&gt;Filter_Push('my_filter2', params...);</pre>
  82 <pre>
  83     $db-&gt;Filter_Key_Push(...) ;
  84     $db-&gt;Filter_Value_Push(...) ;</pre>
  85 <pre>
  86     $db-&gt;Filter_Pop();
  87     $db-&gt;Filtered();</pre>
  88 <pre>
  89     package DBM_Filter::my_filter1;
  90
  91     sub Store { ... }
  92     sub Fetch { ... }</pre>
  93 <pre>
  94     1;</pre>
  95 <pre>
  96     package DBM_Filter::my_filter2;</pre>
  97 <pre>
  98     sub Filter
  99     {
 100         my @opts = @_;
 101         ...
 102         return (
 103             sub Store { ... },
 104             sub Fetch { ... } );
 105     }</pre>
 106 <pre>
 107     1;</pre>
 108 <p>
 109 </p>
 110 <hr />
 111 <h1><a name="description">DESCRIPTION</a></h1>
 112 <p>This module provides an interface that allows filters to be applied
 113 to tied Hashes associated with DBM files. It builds on the DBM Filter
 114 hooks that are present in all the *DB*_File modules included with the
 115 standard Perl source distribution from version 5.6.1 onwards. In addition
 116 to the *DB*_File modules distributed with Perl, the BerkeleyDB module,
 117 available on CPAN, supports the DBM Filter hooks. See <a href="file://C|\msysgit\mingw\html/pod/perldbmfilter.html">the perldbmfilter manpage</a>
 118 for more details on the DBM Filter hooks.</p>
 119 <p>
 120 </p>
 121 <hr />
 122 <h1><a name="what_is_a_dbm_filter">What is a DBM Filter?</a></h1>
 123 <p>A DBM Filter allows the keys and/or values in a tied hash to be modified
 124 by some user-defined code just before it is written to the DBM file and
 125 just after it is read back from the DBM file. For example, this snippet
 126 of code</p>
 127 <pre>
 128     $some_hash{&quot;abc&quot;} = 42;</pre>
 129 <p>could potentially trigger two filters, one for the writing of the key
 130 ``abc'' and another for writing the value 42.  Similarly, this snippet</p>
 131 <pre>
 132     my ($key, $value) = each %some_hash</pre>
 133 <p>will trigger two filters, one for the reading of the key and one for
 134 the reading of the value.</p>
 135 <p>Like the existing DBM Filter functionality, this module arranges for the
 136 <a href="file://C|\msysgit\mingw\html/pod/perlvar.html#item___"><code>$_</code></a> variable to be populated with the key or value that a filter will
 137 check. This usually means that most DBM filters tend to be very short.</p>
 138 <p>
 139 </p>
 140 <h2><a name="so_what_s_new">So what's new?</a></h2>
 141 <p>The main enhancements over the standard DBM Filter hooks are:</p>
 142 <ul>
 143 <li>
 144 <p>A cleaner interface.</p>
 145 </li>
 146 <li>
 147 <p>The ability to easily apply multiple filters to a single DBM file.</p>
 148 </li>
 149 <li>
 150 <p>The ability to create ``canned'' filters. These allow commonly used filters
 151 to be packaged into a stand-alone module.</p>
 152 </li>
 153 </ul>
 154 <p>
 155 </p>
 156 <hr />
 157 <h1><a name="methods">METHODS</a></h1>
 158 <p>This module will arrange for the following methods to be available via
 159 the object returned from the <code>tie</code> call.</p>
 160 <p>
 161 </p>
 162 <h2><a name="_db_filter_push__">$db-&gt;<a href="#item_filter_push"><code>Filter_Push()</code></a></a></h2>
 163 <p>
 164 </p>
 165 <h2><a name="_db_filter_key_push__">$db-&gt;<a href="#item_filter_key_push"><code>Filter_Key_Push()</code></a></a></h2>
 166 <p>
 167 </p>
 168 <h2><a name="_db_filter_value_push__">$db-&gt;<a href="#item_filter_value_push"><code>Filter_Value_Push()</code></a></a></h2>
 169 <p>Add a filter to filter stack for the database, <code>$db</code>. The three formats
 170 vary only in whether they apply to the DBM key, the DBM value or both.</p>
 171 <dl>
 172 <dt><strong><a name="item_filter_push">Filter_Push</a></strong>
 173
 174 <dd>
 175 <p>The filter is applied to <em>both</em> keys and values.</p>
 176 </dd>
 177 </li>
 178 <dt><strong><a name="item_filter_key_push">Filter_Key_Push</a></strong>
 179
 180 <dd>
 181 <p>The filter is applied to the key <em>only</em>.</p>
 182 </dd>
 183 </li>
 184 <dt><strong><a name="item_filter_value_push">Filter_Value_Push</a></strong>
 185
 186 <dd>
 187 <p>The filter is applied to the value <em>only</em>.</p>
 188 </dd>
 189 </li>
 190 </dl>
 191 <p>
 192 </p>
 193 <h2><a name="_db_filter_pop__">$db-&gt;<code>Filter_Pop()</code></a></h2>
 194 <p>Removes the last filter that was applied to the DBM file associated with
 195 <code>$db</code>, if present.</p>
 196 <p>
 197 </p>
 198 <h2><a name="_db_filtered__">$db-&gt;<code>Filtered()</code></a></h2>
 199 <p>Returns TRUE if there are any filters applied to the DBM associated
 200 with <code>$db</code>.  Otherwise returns FALSE.</p>
 201 <p>
 202 </p>
 203 <hr />
 204 <h1><a name="writing_a_filter">Writing a Filter</a></h1>
 205 <p>Filters can be created in two main ways</p>
 206 <p>
 207 </p>
 208 <h2><a name="immediate_filters">Immediate Filters</a></h2>
 209 <p>An immediate filter allows you to specify the filter code to be used
 210 at the point where the filter is applied to a dbm. In this mode the
 211 Filter_*_Push methods expects to receive exactly two parameters.</p>
 212 <pre>
 213     my $db = tie %hash, 'SDBM_File', ...
 214     $db-&gt;Filter_Push( Store =&gt; sub { },
 215                       Fetch =&gt; sub { });</pre>
 216 <p>The code reference associated with <code>Store</code> will be called before any
 217 key/value is written to the database and the code reference associated
 218 with <code>Fetch</code> will be called after any key/value is read from the
 219 database.</p>
 220 <p>For example, here is a sample filter that adds a trailing NULL character
 221 to all strings before they are written to the DBM file, and removes the
 222 trailing NULL when they are read from the DBM file</p>
 223 <pre>
 224     my $db = tie %hash, 'SDBM_File', ...
 225     $db-&gt;Filter_Push( Store =&gt; sub { $_ .= &quot;\x00&quot; ; },
 226                       Fetch =&gt; sub { s/\x00$// ;    });</pre>
 227 <p>Points to note:</p>
 228 <ol>
 229 <li>
 230 <p>Both the Store and Fetch filters manipulate <a href="file://C|\msysgit\mingw\html/pod/perlvar.html#item___"><code>$_</code></a>.</p>
 231 </li>
 232 </ol>
 233 <p>
 234 </p>
 235 <h2><a name="canned_filters">Canned Filters</a></h2>
 236 <p>Immediate filters are useful for one-off situations. For more generic
 237 problems it can be useful to package the filter up in its own module.</p>
 238 <p>The usage is for a canned filter is:</p>
 239 <pre>
 240     $db-&gt;Filter_Push(&quot;name&quot;, params)</pre>
 241 <p>where</p>
 242 <dl>
 243 <dt><strong><a name="item__22name_22">``name''</a></strong>
 244
 245 <dd>
 246 <p>is the name of the module to load. If the string specified does not
 247 contain the package separator characters ``::'', it is assumed to refer to
 248 the full module name ``DBM_Filter::name''. This means that the full names
 249 for canned filters, ``null'' and ``utf8'', included with this module are:</p>
 250 </dd>
 251 <dd>
 252 <pre>
 253     DBM_Filter::null
 254     DBM_Filter::utf8</pre>
 255 </dd>
 256 </li>
 257 <dt><strong><a name="item_params">params</a></strong>
 258
 259 <dd>
 260 <p>any optional parameters that need to be sent to the filter. See the
 261 encode filter for an example of a module that uses parameters.</p>
 262 </dd>
 263 </li>
 264 </dl>
 265 <p>The module that implements the canned filter can take one of two
 266 forms. Here is a template for the first</p>
 267 <pre>
 268     package DBM_Filter::null ;</pre>
 269 <pre>
 270     use strict;
 271     use warnings;</pre>
 272 <pre>
 273     sub Store
 274     {
 275         # store code here
 276     }</pre>
 277 <pre>
 278     sub Fetch
 279     {
 280         # fetch code here
 281     }</pre>
 282 <pre>
 283     1;</pre>
 284 <p>Notes:</p>
 285 <ol>
 286 <li>
 287 <p>The package name uses the <code>DBM_Filter::</code> prefix.</p>
 288 </li>
 289 <li>
 290 <p>The module <em>must</em> have both a Store and a Fetch method. If only one is
 291 present, or neither are present, a fatal error will be thrown.</p>
 292 </li>
 293 </ol>
 294 <p>The second form allows the filter to hold state information using a
 295 closure, thus:</p>
 296 <pre>
 297     package DBM_Filter::encoding ;</pre>
 298 <pre>
 299     use strict;
 300     use warnings;</pre>
 301 <pre>
 302     sub Filter
 303     {
 304         my @params = @_ ;</pre>
 305 <pre>
 306         ...
 307         return {
 308             Store   =&gt; sub { $_ = $encoding-&gt;encode($_) },
 309             Fetch   =&gt; sub { $_ = $encoding-&gt;decode($_) }
 310             } ;
 311     }</pre>
 312 <pre>
 313     1;</pre>
 314 <p>In this instance the ``Store'' and ``Fetch'' methods are encapsulated inside a
 315 ``Filter'' method.</p>
 316 <p>
 317 </p>
 318 <hr />
 319 <h1><a name="filters_included">Filters Included</a></h1>
 320 <p>A number of canned filers are provided with this module. They cover a
 321 number of the main areas that filters are needed when interfacing with
 322 DBM files. They also act as templates for your own filters.</p>
 323 <p>The filter included are:</p>
 324 <ul>
 325 <li><strong><a name="item_utf8">utf8</a></strong>
 326
 327 <p>This module will ensure that all data written to the DBM will be encoded
 328 in UTF-8.</p>
 329 <p>This module needs the Encode module.</p>
 330 </li>
 331 <li><strong><a name="item_encode">encode</a></strong>
 332
 333 <p>Allows you to choose the character encoding will be store in the DBM file.</p>
 334 </li>
 335 <li><strong><a name="item_compress">compress</a></strong>
 336
 337 <p>This filter will compress all data before it is written to the database
 338 and uncompressed it on reading.</p>
 339 <p>This module needs Compress::Zlib.</p>
 340 </li>
 341 <li><strong><a name="item_int32">int32</a></strong>
 342
 343 <p>This module is used when interoperating with a C/C++ application that
 344 uses a C int as either the key and/or value in the DBM file.</p>
 345 </li>
 346 <li><strong><a name="item_null">null</a></strong>
 347
 348 <p>This module ensures that all data written to the DBM file is null
 349 terminated. This is useful when you have a perl script that needs
 350 to interoperate with a DBM file that a C program also uses. A fairly
 351 common issue is for the C application to include the terminating null
 352 in a string when it writes to the DBM file. This filter will ensure that
 353 all data written to the DBM file can be read by the C application.</p>
 354 </li>
 355 </ul>
 356 <p>
 357 </p>
 358 <hr />
 359 <h1><a name="notes">NOTES</a></h1>
 360 <p>
 361 </p>
 362 <h2><a name="maintain_round_trip_integrity">Maintain Round Trip Integrity</a></h2>
 363 <p>When writing a DBM filter it is <em>very</em> important to ensure that it is
 364 possible to retrieve all data that you have written when the DBM filter
 365 is in place. In practice, this means that whatever transformation is
 366 applied to the data in the Store method, the <em>exact</em> inverse operation
 367 should be applied in the Fetch method.</p>
 368 <p>If you don't provide an exact inverse transformation, you will find that
 369 code like this will not behave as you expect.</p>
 370 <pre>
 371      while (my ($k, $v) = each %hash)
 372      {
 373          ...
 374      }</pre>
 375 <p>Depending on the transformation, you will find that one or more of the
 376 following will happen</p>
 377 <ol>
 378 <li>
 379 <p>The loop will never terminate.</p>
 380 </li>
 381 <li>
 382 <p>Too few records will be retrieved.</p>
 383 </li>
 384 <li>
 385 <p>Too many will be retrieved.</p>
 386 </li>
 387 <li>
 388 <p>The loop will do the right thing for a while, but it will unexpectedly fail.</p>
 389 </li>
 390 </ol>
 391 <p>
 392 </p>
 393 <h2><a name="don_t_mix_filtered___nonfiltered_data_in_the_same_database_file_">Don't mix filtered &amp; non-filtered data in the same database file.</a></h2>
 394 <p>This is just a restatement of the previous section. Unless you are
 395 completely certain you know what you are doing, avoid mixing filtered &amp;
 396 non-filtered data.</p>
 397 <p>
 398 </p>
 399 <hr />
 400 <h1><a name="example">EXAMPLE</a></h1>
 401 <p>Say you need to interoperate with a legacy C application that stores
 402 keys as C ints and the values and null terminated UTF-8 strings. Here
 403 is how you would set that up</p>
 404 <pre>
 405     my $db = tie %hash, 'SDBM_File', ...</pre>
 406 <pre>
 407     $db-&gt;Filter_Key_Push('int32') ;</pre>
 408 <pre>
 409     $db-&gt;Filter_Value_Push('utf8');
 410     $db-&gt;Filter_Value_Push('null');</pre>
 411 <p>
 412 </p>
 413 <hr />
 414 <h1><a name="see_also">SEE ALSO</a></h1>
 415 <p>&lt;DB_File&gt;,  <a href="file://C|\msysgit\mingw\html/ext/GDBM_File/GDBM_File.html">the GDBM_File manpage</a>, <a href="file://C|\msysgit\mingw\html/lib/Memoize/NDBM_File.html">the NDBM_File manpage</a>, <a href="file://C|\msysgit\mingw\html/ext/ODBM_File/ODBM_File.html">the ODBM_File manpage</a>, <a href="file://C|\msysgit\mingw\html/lib/SDBM_File.html">the SDBM_File manpage</a>, <a href="file://C|\msysgit\mingw\html/pod/perldbmfilter.html">the perldbmfilter manpage</a></p>
 416 <p>
 417 </p>
 418 <hr />
 419 <h1><a name="author">AUTHOR</a></h1>
 420 <p>Paul Marquess &lt;<a href="mailto:pmqs@cpan.org">pmqs@cpan.org</a>&gt;
 421
 422 </p>
 423 <table border="0" width="100%" cellspacing="0" cellpadding="3">
 424 <tr><td class="block" style="background-color: #cccccc" valign="middle">
 425 <big><strong><span class="block">&nbsp;DBM_Filter -- Filter DBM keys/values</span></strong></big>
 426 </td></tr>
 427 </table>
 428
 429 </body>
 430
 431 </html>