postfix-master/DATABASE_README.html

   1 <!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
   2         "http://www.w3.org/TR/html4/loose.dtd">
   3
   4 <html>
   5
   6 <head>
   7
   8 <title>Postfix Lookup Table Overview</title>
   9
  10 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
  11
  12 </head>
  13
  14 <body>
  15
  16 <h1><img src="postfix-logo.jpg" width="203" height="98" ALT="">Postfix
  17 Lookup Table Overview</h1>
  18
  19 <hr>
  20
  21 <h2>Overview </h2>
  22
  23 This document covers the following topics:
  24
  25 <ul>
  26
  27 <li><a href="#intro">The Postfix lookup table model</a>
  28
  29 <li><a href="#lists">Postfix lists versus tables </a>
  30
  31 <li><a href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>
  32
  33 <li><a href="#detect">Maintaining Postfix lookup table files</a>
  34
  35 <li><a href="#safe_db">Updating Berkeley DB files safely</a>
  36
  37 <li><a href="#types">Postfix lookup table types</a>
  38
  39 </ul>
  40
  41 <h2><a name="intro">The Postfix lookup table model</a></h2>
  42
  43 <p> Postfix uses lookup tables to store and look up information
  44 for access control, address rewriting and even for content filtering.
  45 All Postfix lookup tables are specified as "<a href="DATABASE_README.html">type:table</a>", where
  46 "type" is one of the database types described under "<a
  47 href="#types">Postfix lookup table types</a>" at the end of this
  48 document, and where "table" is the lookup table name. The Postfix
  49 documentation uses the terms "database" and "lookup table" for the
  50 same thing.  </p>
  51
  52 <p> Examples of lookup tables that appear often in the Postfix
  53 documentation: </p>
  54
  55 <blockquote>
  56 <pre>
  57 /etc/postfix/<a href="postconf.5.html">main.cf</a>:
  58     <a href="postconf.5.html#alias_maps">alias_maps</a> = hash:/etc/postfix/aliases            (local aliasing)
  59     <a href="postconf.5.html#header_checks">header_checks</a> = <a href="regexp_table.5.html">regexp</a>:/etc/postfix/header_checks (content filtering)
  60     <a href="postconf.5.html#transport_maps">transport_maps</a> = hash:/etc/postfix/transport      (routing table)
  61     <a href="postconf.5.html#virtual_alias_maps">virtual_alias_maps</a> = hash:/etc/postfix/virtual    (address rewriting)
  62 </pre>
  63 </blockquote>
  64
  65 <p> All Postfix lookup tables store information as (key, value)
  66 pairs.  This interface may seem simplistic at first, but it turns
  67 out to be very powerful. The (key, value) query interface completely
  68 hides the complexities of LDAP or SQL from Postfix. This is a good
  69 example of connecting complex systems with simple interfaces.  </p>
  70
  71 <p> Benefits of the Postfix (key, value) query interface:</p>
  72
  73 <ul>
  74
  75 <li>  You can implement Postfix lookup tables first with local
  76 Berkeley DB files and then switch to LDAP or MySQL without any
  77 impact on the Postfix configuration itself, as described under "<a
  78 href="#preparing">Preparing Postfix for LDAP or SQL lookups</a>"
  79 below.
  80
  81 <li> You can use Berkeley DB files with fixed lookup strings for
  82 simple address rewriting operations and you can use regular expression
  83 tables for the more complicated work. In other words, you don't
  84 have to put everything into the same table.
  85
  86 </ul>
  87
  88 <h2><a name="lists">Postfix lists versus tables </a></h2>
  89
  90 <p> Most Postfix lookup tables are used to look up information.
  91 Examples are address rewriting (the lookup string is the old address,
  92 and the result is the new address) or access control (the lookup
  93 string is the client, sender or recipient, and the result is an
  94 action such as "reject"). </p>
  95
  96 <p> With some tables, however, Postfix needs to know only if the
  97 lookup key exists.  The lookup result itself is not used. Examples
  98 are the <a href="postconf.5.html#local_recipient_maps">local_recipient_maps</a> that determine what local recipients
  99 Postfix accepts in mail from the network, the <a href="postconf.5.html#mydestination">mydestination</a> parameter
 100 that specifies what domains Postfix delivers locally, or the
 101 <a href="postconf.5.html#mynetworks">mynetworks</a> parameter that specifies the IP addresses of trusted
 102 clients or client networks.  Technically, these are lists, not
 103 tables. Despite the difference, Postfix lists are described here
 104 because they use the same underlying infrastructure as Postfix
 105 lookup tables.  </p>
 106
 107 <h2><a name="preparing">Preparing Postfix for LDAP or SQL lookups</a>
 108 </h2>
 109
 110 <p> LDAP and SQL are complex systems. Trying to set up both Postfix
 111 and LDAP or SQL at the same time is definitely not a good idea.
 112 You can save yourself a lot of time by implementing Postfix first
 113 with local files such as Berkeley DB.  Local files have few surprises,
 114 and are easy to debug with the <a href="postmap.1.html">postmap(1)</a> command:  </p>
 115
 116 <blockquote>
 117 <pre>
 118 % <b>postmap -q info@example.com hash:/etc/postfix/virtual </b>
 119 </pre>
 120 </blockquote>
 121
 122 <p> Once you have local files working properly you can follow the
 123 instructions in <a href="ldap_table.5.html">ldap_table(5)</a>, <a href="mysql_table.5.html">mysql_table(5)</a>, <a href="pgsql_table.5.html">pgsql_table(5)</a>
 124 or <a href="sqlite_table.5.html">sqlite_table(5)</a>
 125 and replace local file lookups with LDAP or SQL lookups.  When you
 126 do this, you should use the <a href="postmap.1.html">postmap(1)</a> command again, to verify
 127 that database lookups still produce the exact same results as local
 128 file lookup:  </p>
 129
 130 <blockquote>
 131 <pre>
 132 % <b>postmap -q info@example.com <a href="ldap_table.5.html">ldap</a>:/etc/postfix/virtual.cf </b>
 133 </pre>
 134 </blockquote>
 135
 136 <p> Be sure to exercise all the partial address or parent domain
 137 queries that are documented under "table search order" in the
 138 relevant manual page:  <a href="access.5.html">access(5)</a>, <a href="canonical.5.html">canonical(5)</a>, <a href="virtual.5.html">virtual(5)</a>,
 139 <a href="transport.5.html">transport(5)</a>, or under the relevant configuration parameter:
 140 <a href="postconf.5.html#mynetworks">mynetworks</a>, <a href="postconf.5.html#relay_domains">relay_domains</a>, <a href="postconf.5.html#parent_domain_matches_subdomains">parent_domain_matches_subdomains</a>. </p>
 141
 142 <h2><a name="detect">Maintaining Postfix lookup table files</a></h2>
 143
 144 <p> When you make changes to a database while the mail system is
 145 running, it would be desirable if Postfix avoids reading information
 146 while that information is being changed. It would also be nice if
 147 you can change a database without having to execute "postfix reload",
 148 in order to force Postfix to use the new information. Each time
 149 you do "postfix reload" Postfix loses a lot of performance.
 150 </p>
 151
 152 <ul>
 153
 154 <li> <p> If you change a network database such as LDAP, NIS or
 155 SQL, there is no need to execute "postfix reload".  The LDAP, NIS
 156 or SQL server takes care of read/write access conflicts and gives
 157 the new data to Postfix once that data is available.  </p>
 158
 159 <li> <p> If you change a <a href="regexp_table.5.html">regexp</a>: or <a href="pcre_table.5.html">pcre</a>: file then Postfix may or
 160 may not pick up the file changes immediately. This is because a
 161 Postfix process reads the entire file into memory once and never
 162 examines the file again.  </p>
 163
 164 <ul>
 165
 166 <li> <p> If the file is used by a short-running process such as
 167 <a href="smtpd.8.html">smtpd(8)</a>, <a href="cleanup.8.html">cleanup(8)</a> or <a href="local.8.html">local(8)</a>, there is no need to execute
 168 "postfix reload" after making a change. </p>
 169
 170 <li> <p> If the file is being used by a long-running process such
 171 as <a href="trivial-rewrite.8.html">trivial-rewrite(8)</a> on a busy server it may be necessary to
 172 execute "postfix reload". </p>
 173
 174 </ul>
 175
 176 <li> <p> If you change a local file based database such as DBM or
 177 Berkeley DB, there is no need to execute "postfix reload".  Postfix
 178 uses file locking to avoid read/write access conflicts, and whenever
 179 a Postfix daemon process notices that a file has changed it will
 180 terminate before handling the next client request, so that a new
 181 process can initialize with the new database.  </p>
 182
 183 </ul>
 184
 185 <h2><a name="safe_db">Updating Berkeley DB files safely</a></h2>
 186
 187 <p> Although Postfix uses file locking to avoid access conflicts
 188 while updating Berkeley DB or other local database files, you still
 189 have a problem when the update fails because the disk is full or
 190 because something else happens.  This is because commands such as
 191 <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> overwrite existing files. If the update
 192 fails in the middle then you have no usable database, and Postfix
 193 will stop working. This is not an issue with the CDB database type
 194 available with Postfix 2.2 and later: <a href="CDB_README.html">CDB</a>
 195 creates a new file, and renames the file upon successful completion.
 196 </p>
 197
 198 <p> With multi-file databases such as DBM, there is no simple
 199 solution. With Berkeley DB and other "one file" databases, it is
 200 possible to add some extra robustness by using "mv" to REPLACE an
 201 existing database file instead of overwriting it:  </p>
 202
 203 <blockquote>
 204 <pre>
 205 # <b>postmap access.in &amp;&amp; mv access.in.db access.db</b>
 206 </pre>
 207 </blockquote>
 208
 209 <p> This converts the input file "access.in" into the output file
 210 "access.in.db", and replaces the file "access.db" only when the
 211 <a href="postmap.1.html">postmap(1)</a> command was successful. Of course typing such commands
 212 becomes boring quickly, and this is why people use "make" instead,
 213 as shown below.  User input is shown in bold font. </p>
 214
 215 <blockquote>
 216 <pre>
 217 # <b>cat Makefile</b>
 218 all: aliases.db access.db virtual.db ...etcetera...
 219
 220 # Note 1: commands are specified after a TAB character.
 221 # Note 2: use <a href="postalias.1.html">postalias(1)</a> for local aliases, <a href="postmap.1.html">postmap(1)</a> for the rest.
 222 aliases.db: aliases.in
 223         postalias aliases.in
 224         mv aliases.in.db aliases.db
 225
 226 access.db: access.in
 227         postmap access.in
 228         mv access.in.db access.db
 229
 230 virtual.db: virtual.in
 231         postmap virtual.in
 232         mv virtual.in.db virtual.db
 233
 234 ...etcetera...
 235 # <b>vi access.in</b>
 236 ...editing session not shown...
 237 # <b>make</b>
 238 postmap access.in
 239 mv access.in.db access.db
 240 #
 241 </pre>
 242 </blockquote>
 243
 244 <p> The "make" command updates only the files that have changed.
 245 In case of error, the "make" command will stop and will not invoke
 246 the "mv" command, so that Postfix will keep using the existing
 247 database file as if nothing happened. </p>
 248
 249 <h2><a name="types">Postfix lookup table types</a> </h2>
 250
 251 <p> To find out what database types your Postfix system supports,
 252 use the "<b>postconf -m</b>" command.  Here is a list of database types
 253 that are often supported: </p>
 254
 255 <blockquote>
 256
 257 <dl>
 258
 259 <dt> <b>btree</b> </dt>
 260
 261 <dd> A sorted, balanced tree structure.  This is available only on
 262 systems with support for Berkeley DB databases. Database files are
 263 created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup
 264 table name as used in "btree:table" is the database file name
 265 without the ".db" suffix.  </dd>
 266
 267 <dt> <b>cdb</b> </dt>
 268
 269 <dd> A read-optimized structure with no support for incremental updates.
 270 Database files are created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command.
 271 The lookup table name as used in "<a href="CDB_README.html">cdb</a>:table" is the database file name
 272 without the ".cdb" suffix.  This feature is available with Postfix 2.2
 273 and later. </dd>
 274
 275 <dt> <b>cidr</b> </dt>
 276
 277 <dd> A table that associates values with Classless Inter-Domain
 278 Routing (CIDR) patterns. The table format is described in <a href="cidr_table.5.html">cidr_table(5)</a>.
 279 </dd>
 280
 281 <dt> <b>dbm</b> </dt>
 282
 283 <dd> An indexed file type based on hashing.  This is available only
 284 on systems with support for DBM databases. Database files are
 285 created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup
 286 table name as used in "dbm:table" is the database file name without
 287 the ".dir" or ".pag" suffix.  </dd>
 288
 289 <dt> <b>environ</b> </dt>
 290
 291 <dd> The UNIX process environment array. The lookup key is the
 292 variable name. The lookup table name in "environ:table" is ignored.
 293 </dd>
 294
 295 <dt> <b>hash</b> </dt>
 296
 297 <dd> An indexed file type based on hashing.  This is available only
 298 on systems with support for Berkeley DB databases. Database files are
 299 created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The database
 300 name as used in "hash:table" is the database file name without the
 301 ".db" suffix. </dd>
 302
 303 <dt> <b>internal</b> </dt>
 304
 305 <dd> A non-shared, in-memory hash table. Its content are lost when
 306 a process terminates. </dd>
 307
 308 <dt> <b>ldap</b> (read-only) </dt>
 309
 310 <dd> Perform lookups using the LDAP protocol. Configuration details
 311 are given in the <a href="ldap_table.5.html">ldap_table(5)</a>. </dd>
 312
 313 <dt> <b>mysql</b> (read-only) </dt>
 314
 315 <dd> Perform MySQL database lookups. Configuration details are given
 316 in <a href="mysql_table.5.html">mysql_table(5)</a>. </dd>
 317
 318 <dt> <b>netinfo</b> (read-only) </dt>
 319
 320 <dd> Perform Netinfo database lookups. </dd>
 321
 322 <dt> <b>nis</b> (read-only) </dt>
 323
 324 <dd> Perform NIS database lookups. </dd>
 325
 326 <dt> <b>nisplus</b> (read-only) </dt>
 327
 328 <dd> Perform NIS+ database lookups. Configuration details are given
 329 in <a href="nisplus_table.5.html">nisplus_table(5)</a>. </dd>
 330
 331 <dt> <b>pcre</b> (read-only) </dt>
 332
 333 <dd> A lookup table based on Perl Compatible Regular Expressions.
 334 The file format is described in <a href="pcre_table.5.html">pcre_table(5)</a>. The lookup table
 335 name as used in "<a href="pcre_table.5.html">pcre</a>:table" is the name of the regular expression
 336 file.  </dd>
 337
 338 <dt> <b>pgsql</b> (read-only) </dt>
 339
 340 <dd> Perform PostgreSQL database lookups.  Configuration details
 341 are given in <a href="pgsql_table.5.html">pgsql_table(5)</a>. </dd>
 342
 343 <dt> <b>proxy</b> (read-only) </dt>
 344
 345 <dd> Access information via the Postfix <a href="proxymap.8.html">proxymap(8)</a> service. The
 346 lookup table name syntax is "<a href="proxymap.8.html">proxy</a>:<a href="DATABASE_README.html">type:table</a>". </dd>
 347
 348 <dt> <b>regexp</b> (read-only) </dt>
 349
 350 <dd> A lookup table based on regular expressions. The file format
 351 is described in <a href="regexp_table.5.html">regexp_table(5)</a>. The lookup table name as used in
 352 "<a href="regexp_table.5.html">regexp</a>:table" is the name of the regular expression file. </dd>
 353
 354 <dt> <b>sdbm</b> </dt>
 355
 356 <dd> An indexed file type based on hashing.  This is available only
 357 on systems with support for SDBM databases. Database files are
 358 created with the <a href="postmap.1.html">postmap(1)</a> or <a href="postalias.1.html">postalias(1)</a> command. The lookup
 359 table name as used in "sdbm:table" is the database file name without
 360 the ".dir" or ".pag" suffix.  </dd>
 361
 362 <dt> <b>sqlite</b> (read-only) </dt>
 363
 364 <dd> Perform SQLite database lookups. Configuration details are given
 365 in <a href="sqlite_table.5.html">sqlite_table(5)</a>. </dd>
 366
 367 <dt> <b>static</b> (read-only) </dt>
 368
 369 <dd> Always returns its lookup table name as lookup result.  For
 370 example, the lookup table "static:foobar" always returns the string
 371 "foobar" as lookup result. </dd>
 372
 373 <dt> <b>tcp</b> </dt>
 374
 375 <dd> Access information through a TCP/IP server. The protocol is
 376 described in <a href="tcp_table.5.html">tcp_table(5)</a>. The lookup table name is "<a href="tcp_table.5.html">tcp</a>:host:port"
 377 where "host" specifies a symbolic hostname or a numeric IP address,
 378 and "port" specifies a symbolic service name or a numeric port
 379 number.
 380 </dd>
 381
 382 <dt> <b>texthash</b> (read-only) </dt>
 383
 384 <dd> This produces similar results as hash: files, except that you
 385 don't have to run the <a href="postmap.1.html">postmap(1)</a> command before you can use the
 386 file, and that texthash: does not detect changes after the file is
 387 read.  The lookup table name is "texthash:filename", where the file
 388 name is taken literally; no suffix is appended. </dd>
 389
 390 <dt> <b>unix</b> (read-only) </dt>
 391
 392 <dd> A limited way to query the UNIX authentication database. The
 393 following tables are implemented:
 394
 395 <dl>
 396
 397 <dt> <b>unix:passwd.byname</b> </dt>
 398
 399 <dd>The table is the UNIX password database. The key is a login
 400 name.  The result is a password file entry in passwd(5) format.
 401 </dd>
 402
 403 <dt> <b>unix:group.byname</b> </dt>
 404
 405 <dd> The table is the UNIX group database. The key is a group name.
 406 The result is a group file entry in group(5) format. </dd>
 407
 408 </dl> </dd>
 409
 410 </dl>
 411
 412 </blockquote>
 413
 414 <p> Other lookup table types may be available depending on how
 415 Postfix was built. With some Postfix distributions the list is
 416 dynamically extensible as support for lookup tables is dynamically
 417 linked into Postfix.  </p>
 418
 419 </body>
 420
 421 </html>