postfix-master/rate.html

   1 <html>
   2
   3 <!--Warning, preformatted content! -->
   4
   5 <head>
   6
   7 <title> Postfix Rate Controls</title>
   8
   9 <META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">
  10 </head>
  11
  12 <body background="obsolete.gif">
  13
  14 <h1><a href="big-picture.html"><img src="small-picture.gif" width="115" height="45"></a> Postfix Rate Controls</h1>
  15
  16 <hr>
  17
  18 <p> <strong> Note: this web page is no longer maintained. It exists
  19 only to avoid breaking links in web pages that describe earlier
  20 versions of the Postfix mail system. </strong> </p>
  21
  22 <a href="docs.html">Up one level</a> |
  23 <a href="basic.html">Basic Configuration</a> | <a href="uce.html">UCE
  24 Controls</a> | Rate Controls | <a href="resource.html">Resource
  25 Controls</a> | <a href="rewrite.html">Address Manipulation</a>
  26
  27 <h2> Introduction</h2>
  28
  29 Building a high-performance mail delivery system is one thing;
  30 building one that does not knock over other systems is a different
  31 story. Some mailers suffer from the <i>thundering herd</i> syndrome:
  32 they literally flood other systems with mail. Postfix tries to be
  33 a fast mailer and a good neighbor at the same time.
  34
  35 <p>
  36
  37 On the inbound side, the Postfix <a href="smtpd.8.html">SMTP
  38 server</a> has defenses in place against malicious or confused
  39 clients. They won't protect against an all-out denial of service
  40 attack on your infrastructure, but then nothing will except pulling
  41 the plug.
  42
  43 <p>
  44
  45 Unless indicated otherwise, all parameters described here are in
  46 the <b>main.cf</b> file. If you change parameters of a running
  47 Postfix system, don't forget to issue a <b>postfix reload</b>
  48 command.
  49
  50 <ul>
  51
  52 <a href="#process">Process limits</a>
  53
  54 <p>
  55
  56 <a href="#destination">Destination concurrency</a>
  57
  58 <p>
  59
  60 <a href="#recipients">Recipient limits</a>
  61
  62 <p>
  63
  64 <a href="#postponing">Always postponing delivery</a>
  65
  66 <p>
  67
  68 <a href="#backoff">Backoff from dead hosts</a>
  69
  70 <p>
  71
  72 <a href="#slowdown">Slowing down bad clients</a>
  73
  74 </ul>
  75
  76 <a name="process"><h2> Process limits</h2>
  77
  78 The <b>default_process_limit</b> parameter (default: 100) gives
  79 direct control over inbound and outbound delivery rates.  This
  80 parameter controls the number of concurrent processes that implement
  81 a Postfix service (smtp client, smtp server, local delivery, etc.).
  82 On small systems, or on systems connected via dialup networks, a
  83 <b>default_process_limit</b> of 10 is probably more than adequate.
  84 Use a larger value if your machine is a major mail hub.
  85
  86 <p>
  87
  88 You can override this setting for specific Postfix daemons by
  89 editing the <b>master.cf</b> file.  For example, if you do not
  90 wish to receive 100 SMTP messages at the same time, you could specify:
  91
  92 <dl>
  93
  94 <dd> <pre>
  95 # ==========================================================================
  96 # service type  private unpriv  chroot  wakeup  maxproc command + args
  97 #               (yes)   (yes)   (yes)   (never) (100)
  98 # ==========================================================================
  99 . . .
 100 smtp      inet  n       -       -       -       5       smtpd
 101 . . .
 102
 103 </pre>
 104
 105 </dl>
 106
 107 <a name="destination"><h2> Destination concurrency</h2>
 108
 109 So, you have this huge mailhub with tons of disk and memory, and
 110 have configured Postfix to run up to 1000 SMTP client processes at
 111 the same time. Congratulations. But do you really want to make 1000
 112 simultaneous connections to the same remote system? Probably not.
 113
 114 <p>
 115
 116 The Postfix queue manager comes to the rescue. This program implements
 117 the analog of the TCP <i> slow start</i> flow control strategy:
 118 when delivering to a site, send a small number of messages first,
 119 then increase the rate as long as all goes well; back off in the
 120 face of congestion.
 121
 122 <p>
 123
 124 The <b>initial_destination_concurrency</b> parameter (default:  2)
 125 controls how many messages are initially sent to the same destination
 126 before adapting delivery concurrency.  Of course, this setting is
 127 effective only as long as it does not exceed the <b><a
 128 href="#process">process limit</a></b> and the destination concurrency
 129 limit for the specific mail transport channel.
 130
 131 <p>
 132
 133 The <b>default_destination_concurrency_limit</b> parameter
 134 (default: 20) controls how many messages may be sent to the same
 135 destination simultaneously. You can override this setting for
 136 specific delivery channels (<b>local, smtp, uucp</b> etc.). The
 137 <b>main.cf</b> file recommends the following:
 138
 139 <dl>
 140
 141 <dd> <b>local_destination_concurrency_limit = 2</b>
 142
 143 <dd> <b>default_destination_concurrency_limit = 20</b>
 144
 145 </dl>
 146
 147 The <b>local_destination_concurrency_limit</b> parameter controls
 148 how many messages are delivered simultaneously to the same local
 149 recipient.  The recommended limit is low because delivery to the
 150 same mailbox must happen sequentially, so massive parallelism is
 151 not useful.  Another good reason to limit delivery concurrency to
 152 the same recipient:  if the recipient has an expensive shell command
 153 in her <b>.forward </b> file, or if the recipient is a mailing list
 154 manager, you don't want to run too many instances at the same time.
 155
 156 <p>
 157
 158 A destination concurrency limit of 20 for SMTP delivery seems enough
 159 to noticeably load a system without bringing it to its knees. Be
 160 careful when changing this to a much larger number.
 161
 162 <a name="recipients"><h2> Recipient limits</h2>
 163
 164 The <b>default_destination_recipient_limit</b> parameter (default:
 165 50) controls how many recipients a Postfix delivery agent (<b>smtp</b>,
 166 <b>uucp</b>, etc.) will send with each copy of an email message.
 167 If an email message has more than
 168 <b>$default_destination_recipient_limit</b> recipients at the same
 169 destination, the list of recipients will be broken up into smaller lists,
 170 and multiple copies of the message will be sent.
 171
 172 <p>
 173
 174 You can override this setting for specific Postfix delivery agents
 175 (<b>smtp, uucp,</b> etc.). For example:
 176
 177 <dl>
 178
 179 <dd> <b>uucp_destination_recipient_limit = 100</b>
 180
 181 </dl>
 182
 183 would limit the number of recipients per <b>UUCP</b> delivery to 100.
 184
 185 <p>
 186
 187 You must be careful when increasing the recipient limit; some SMTP
 188 servers abort the connection when they run out of memory or when
 189 a hard recipient limit is reached, so the mail won't get through.
 190
 191 <p>
 192
 193 The <b>smtpd_recipient_limit</b> parameter (default: 1000)
 194 controls how many recipients the SMTP server will take per delivery.
 195 That's more than any reasonable SMTP client would send.  The limit
 196 exists just to protect the local mail system against a malicious
 197 or confused client.
 198
 199 <a name="postponing"><h2> Always postponing delivery</h2>
 200
 201 The <b>defer_transports</b> parameter allows you to specify what
 202 mail should always be deferred until Postfix is explicitly asked
 203 to deliver.
 204
 205 <p>
 206
 207 A small site that is on-line only part of the time, and
 208 that wants to defer all deliveries until the command <b>sendmail
 209 -q</b> is executed (e.g., from a PPP dialout script) would use:
 210
 211 <p>
 212
 213 <dl>
 214
 215 <dd><b>defer_transports = smtp</b>
 216
 217 </dl>
 218
 219 <p>
 220
 221 An ISP can use the <b>defer_transports</b> feature for customers
 222 that are off-line most of the time. The customer can trigger delivery
 223 by issuing an <b>ETRN</b> command at the SMTP port. The following
 224 examples show how to configure such a customer:
 225
 226 <p>
 227
 228 <dl>
 229
 230 <dt>/etc/postfix/main.cf:
 231
 232 <p>
 233
 234 <dd><b>defer_transports = hold</b>
 235
 236 <p>
 237
 238 You can specify any number of transports here. The example gives
 239 just one.
 240
 241 <p>
 242
 243 <dt>/etc/postfix/<a href="transport.5.html">transport</a>:
 244
 245 <p>
 246
 247 <dd><b>customer.com     &nbsp; hold:[gateway.customer.com]</b>
 248
 249 <dd><b>.customer.com    &nbsp; hold:[gateway.customer.com]</b>
 250
 251 <p>
 252
 253 The [] are necessary to avoid MX lookups, which might point to your
 254 local machine.  The second entry is necessary only if you want to
 255 relay mail for customer subdomains.
 256
 257 <p>
 258
 259 <dt>/etc/postfix/master.cf:
 260
 261 <p>
 262
 263 <dd><b>hold &nbsp; unix &nbsp; - &nbsp; - &nbsp; n &nbsp; - &nbsp; - &nbsp; smtp</b>
 264
 265 <p>
 266
 267 This is just the <b>master.cf</b> entry for regular SMTP, with the
 268 first field changed to <b>hold</b>.
 269
 270 </dl>
 271
 272 <a name="backoff"><h2> Backoff from unreachable hosts</h2>
 273
 274 When a Postfix delivery agent (<b>smtp, local, uucp,</b> etc.)
 275 is unable to deliver a message it may blame the message itself or
 276 the receiving party.
 277
 278 <ul>
 279
 280 <li> If the delivery agent blames the message, the queue manager
 281 gives the queue file a time stamp into the future, so it won't be
 282 looked at for a while. By default, the amount of time to cool down
 283 is the amount of time that has passed since the message arrived.
 284 This results in so-called <i>exponential backoff</i> behavior.
 285
 286 <p>
 287
 288 <li> If the delivery agent blames the receiving party (for example
 289 a local recipient user, or a remote host), the queue manager not
 290 only advances the queue file time stamp, but also puts the receiving
 291 party on a "dead" list so that it will be skipped for some amount
 292 of time.
 293
 294 </ul>
 295
 296 <p>
 297
 298 As you would expect, this whole process is governed by a bunch of
 299 little parameters.
 300
 301 <dl>
 302
 303 <dt> <b>queue_run_delay</b> (default: 1000 seconds) <dd> How
 304 often the queue manager scans the queue for deferred mail.
 305
 306 <p>
 307
 308 <dt> <b>maximal_queue_lifetime</b> (default: 5 days) <dd> How
 309 long a message stays in the queue before it is sent back as
 310 undeliverable. Specify 0 for mail that should be returned
 311 immediately after the first unsuccessful delivery attempt.
 312
 313 <p>
 314
 315 <dt> <b>minimal_backoff_time</b> (default: 1000 seconds) <dd> The
 316 minimal amount of time a message won't be looked at, and the minimal
 317 amount of time to stay away from a "dead" destination.
 318
 319 <p>
 320
 321 <dt> <b>maximal_backoff_time</b> (default: 4000 seconds) <dd> The
 322 maximal amount of time a message won't be looked at after a delivery
 323 failure.
 324
 325 <p>
 326
 327 <dt> <b>qmgr_message_recipient_limit</b> (default: 20000) <dd> The
 328 size of many in-memory queue manager data structures.  Among others,
 329 this parameter limits the size of the short-term, in-memory "dead"
 330 list.  Destinations that don't fit the list are not added.
 331
 332 </dl>
 333
 334 <a name="slowdown"><h2> Slowing down bad clients</h2>
 335
 336 First of all, no defense will protect against an all-out denial of
 337 service attack. I just don't want to raise impossible expectations.
 338 But there are a few simple things one can do in order to deal with
 339 confused or malicious client programs.
 340
 341 <p>
 342
 343 Some defenses are part of a more general strategy: for example,
 344 how long a line of text may be before it is broken up into pieces,
 345 and how much text may be carried in a multi-line message header.
 346 See the <a href="resource.html">resource controls</a> documentation
 347 for details.
 348
 349 <p>
 350
 351 The Postfix <a href="smtpd.8.html">SMTP server</a> increments a
 352 per-session error counter whenever a client request is unrecognized
 353 or unimplemented, or whenever a client request violates <a
 354 href="uce.html">UCE restrictions</a> or other reasons.  The error
 355 counter is reset when a message is transferred successfully.
 356
 357 <p>
 358
 359 As the per-session error count increases, the SMTP server changes
 360 behavior.  The idea is to limit the damage by slowing down the
 361 client. The behavior is controlled by the following parameters:
 362
 363 <p>
 364
 365 <dl>
 366
 367 <a name="smtpd_error_sleep_time">
 368
 369 <dt> <b>smtpd_error_sleep_time</b> (default: 1 second) <dd> When
 370 the per-session error count is small, the SMTP server pauses only
 371 when reporting a problem to a client. The purpose is to prevent
 372 naive clients from going into a fast <i>connect-error-disconnect</i>
 373 loop.
 374
 375 <p>
 376
 377 <a name="smtpd_soft_error_limit">
 378
 379 <dt> <b>smtpd_soft_error_limit</b> (default: 10) <dd> When the
 380 per-session error count exceeds this value, the SMTP server sleeps
 381 <b>error_count</b> seconds before responding to a client request.
 382
 383 <p>
 384
 385 <a name="smtpd_hard_error_limit">
 386
 387 <dt> <b>smtpd_hard_error_limit</b> (default: 20) <dd> When
 388 the per-session error count exceeds this value, the SMTP server
 389 disconnects.
 390
 391 </dl>
 392
 393 <p>
 394
 395 Unfortunately, the Postfix SMTP server does not yet know how to
 396 limit the number of connections <i> from the same client</i>,
 397 other than by limiting the total number of SMTP server processes
 398 (see <a href="#process">process limit</a>).  Things could be worse:
 399 some mailers don't even implement an SMTP server process limit.
 400 That's of course no excuse.  I'm still looking for a good solution.
 401
 402 <hr>
 403
 404 <a href="docs.html">Up one level</a> |
 405 <a href="basic.html">Basic Configuration</a> | <a href="uce.html">UCE
 406 Controls</a> | Rate Controls | <a href="resource.html">Resource
 407 Controls</a> | <a href="rewrite.html">Address Manipulation</a>
 408
 409 </body>
 410
 411 </html>