postfix-master/workarounds.html

   1 <!doctype html public "-//W3C//DTD HTML 4.01 Transitional//EN"
   2         "http://www.w3.org/TR/html4/loose.dtd">
   3
   4 <html>
   5
   6 <head>
   7
   8 <title>Postfix Workarounds</title>
   9
  10 <meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
  11
  12 </head>
  13
  14 <body bgcolor="#ffffff">
  15
  16 <table border="0" cellpadding="5" cellspacing="0" width="100%">
  17
  18 <tr>
  19
  20 <td align="center" valign="top">
  21
  22 <img src="mysza.gif" width="130" height="91" alt="[LOGO]">
  23
  24 <table cellpadding="5" cellspacing="0">
  25
  26 <tr> <td> </td> </tr>
  27
  28 <tr> <td nowrap align="center" bgcolor="#b2b0b0">
  29
  30 <font size="-1"> <strong> QUICK LINKS </strong> </font> </td> </tr>
  31
  32 <tr>
  33
  34 <td nowrap bgcolor="#f8eda6" align="left">
  35
  36 <form method="get" action="http://www.google.com/search">
  37
  38 <font size="-1">
  39
  40 <a href="start.html">Home</a> <br>
  41
  42 <strong><a href="announcements.html">Workarounds</a></strong> <br>
  43
  44 <a href="non-english.html">Non-English Info</a> <br>
  45
  46 <a href="features.html">Feature overview </a> <br>
  47
  48 <a href="web-sites.html">Web sites (text)</a> <br>
  49
  50 <a href="download.html">Download (source)</a> <br>
  51
  52 <a href="lists.html">Mailing lists</a> <br>
  53
  54 <a href="press.html">Press and Interviews</a> <br>
  55
  56 <a href="documentation.html">Documentation</a> <br>
  57
  58 <a href="docs.html">Howtos and FAQs</a> <br>
  59
  60 <a href="addon.html">Add-on Software</a> <br>
  61
  62 <a href="packages.html">Packages and Ports</a> <br>
  63
  64 <a href="mirror.html">Becoming a mirror site</a> <br> <br>
  65
  66 <input size="-1" type="submit" value="Search"><br>
  67 <input type="text" name="q" size="20" maxlength="255" value="">
  68 <input type="hidden" name="sitesearch" value="www.postfix.org">
  69
  70 </font>
  71
  72 </form>
  73
  74 </td> </tr> </table> </td>
  75
  76 <td valign="top" width="100%">
  77
  78 <h1>Postfix Workarounds</h1>
  79
  80 <p> This webpage documents workarounds for bugs in non-Postfix mail
  81 software. Workarounds may vary from "dumbing down" Postfix, to an
  82 optional source code patch that automatically enables the workaround.
  83 </p>
  84
  85 <ul>
  86
  87 <li> <p> <a href="#frontbridge_pipelining_bug">(Resolved: December
  88 9, 2010) Mail FROM FrontBridge customers does not go through while
  89 Postfix repeatedly logs "lost connection" errors </a> </p>
  90
  91 </ul>
  92
  93 <h2><a name="frontbridge_pipelining_bug">(Resolved: December 9,
  94 2010) Mail FROM FrontBridge customers does not go through while
  95 Postfix repeatedly logs "lost connection" errors.  </a></h2>
  96
  97 <a href="#frontbridge_summary"> summary </a> | <a
  98 href="#frontbridge_demonstration"> demonstration </a> | <a
  99 href="#frontbridge_workaround"> workarounds </a> | <a
 100 href="#frontbridge_background"> background </a>
 101
 102 <h3><a name="frontbridge_summary">Problem summary</a> </h3>
 103
 104 <p> The SMTP service at mail.global.frontbridge.com does not fully
 105 conform to RFC 2920 which defines the ESMTP PIPELINING extension.
 106 When the service receives a group of SMTP commands ending with
 107 QUIT, it ignores all commands in that group except QUIT.  Needless
 108 to say, this can break SMTP sessions in unexpected ways.  </p>
 109
 110 <p> As one symptom of this defect, mail from FrontBridge customers
 111 does not go through to Postfix sites, when a Postfix site is
 112 configured to verify that the sender address is valid (this involves
 113 connecting to the FrontBridge service).  </p>
 114
 115 <p> Due to the FrontBridge service's non-conformance to RFC 2920,
 116 the attempt to verify the sender address fails, and Postfix logs
 117 "<tt>lost connection with mail.global.frontbridge.com[x.x.x.x]
 118 while sending RCPT TO</tt>". As Postfix is unable to verify the
 119 sender address, mail from the FrontBridge customer will not go
 120 through. </p>
 121
 122 <h3><a name="frontbridge_demonstration">Problem demonstration</a> </h3>
 123
 124 <p> The following is a slightly-edited transcript of an SMTP session
 125 that demonstrates the problem.  The transcript was obtained by
 126 executing the following commands: </p>
 127
 128 <blockquote>
 129 <pre>
 130 # postconf -e debug_peer_list=frontbridge.com
 131 # postfix reload
 132 # sendmail -bv receiver@example.com
 133 </pre>
 134 </blockquote>
 135
 136 <p> Instead of receiver@example.com specify an email address that
 137 is hosted by FrontBridge. </p>
 138
 139 <blockquote>
 140 <pre>
 141 S: 220 VA3EHSMHS018.bigfish.com Microsoft ESMTP MAIL Service ready at
 142        Sun, 28 Nov 2010 19:01:22 +0000
 143
 144 C: EHLO amnesiac.example.com
 145
 146 S: 250-VA3EHSMHS018.bigfish.com Hello [192.0.2.1]
 147 S: 250-SIZE 157286400
 148 S: 250-PIPELINING
 149 S: 250-ENHANCEDSTATUSCODES
 150 S: 250-STARTTLS
 151 S: 250-AUTH
 152 S: 250-8BITMIME
 153 S: 250-BINARYMIME
 154 S: 250 CHUNKING
 155
 156 Note: the following client commands are sent in one single TCP segment,
 157 not in multiple segments back-to-back.
 158
 159 C: MAIL FROM:&lt;sender@example.com> SIZE=338
 160 C: RCPT TO:&lt;receiver@example.com>
 161 C: RSET
 162 C: QUIT
 163
 164 S: 221 2.0.0 Service closing transmission channel
 165
 166   &lt;Client sees premature TCP close while expecting three more responses&gt;
 167 </pre>
 168 </blockquote>
 169
 170 <p> The client sees a premature TCP close, while it expects three
 171 more server responses. Specifically, the server responses to the
 172 "MAIL FROM" and "RCPT TO" and "RSET" commands are "lost", and the
 173 "QUIT" response arrives out-of-order.  One may speculate that the
 174 issue is not in the FrontBridge server itself, but rather in proxy
 175 software in front of the FrontBridge server. </p>
 176
 177 <p> As discussed below, the correct behavior would be: </p>
 178
 179 <blockquote>
 180 <pre>
 181 C: MAIL FROM:&lt;sender@example.com> SIZE=338
 182 C: RCPT TO:&lt;receiver@example.com>
 183 C: RSET
 184 C: QUIT
 185
 186 S: 250 2.1.0 Sender OK
 187 S: 250 2.1.5 Recipient OK
 188 S: 250 2.0.0 Resetting
 189 S: 221 2.0.0 Service closing transmission channel
 190 </pre>
 191 </blockquote>
 192
 193 <h3> <a name="frontbridge_workaround">Workarounds</a> </h3>
 194
 195 <p> A Postfix patch may be made available that turns on workarounds
 196 automatically,
 197 by looking at the content of the "220 welcome" server greeting.
 198 Until that patch is available, workarounds will need to be turned
 199 on by hand.
 200 </p>
 201
 202 <ul>
 203
 204 <li> <p> Disable sender address verification. The problem with the
 205 FrontBridge service triggers with "MAIL FROM/RCPT TO/RSET/QUIT",
 206 which appears with Postfix sender address verification probes. </p>
 207
 208 <li> <p> Disable SMTP command pipelining for FrontBridge servers,
 209 Unfortunately, Postfix supports this only by IP address and the
 210 FrontBridge servers appear to use many IP addresses.  </p>
 211
 212 <blockquote>
 213 <pre>
 214 /etc/postfix/main.cf:
 215     smtp_discard_ehlo_keyword_address_maps =
 216         pcre:/etc/postfix/discard_ehlo_keywords
 217
 218 /etc/postfix/discard_ehlo_keywords:
 219     # This is likely to be incomplete.
 220     216.32.0.0/16 silent-discard, pipelining
 221     213.199.0.0/16 silent-discard, pipelining
 222     65.55.0.0/16  silent-discard pipelining
 223     94.245.0.0/16 silent-discard pipelining
 224 </pre>
 225 </blockquote>
 226
 227 <li> <p> Disable SMTP command pipelining for all mail. This
 228 sledgehammer solution will reduce performance for all outbound
 229 mail, by causing extra network round-trip times. </p>
 230
 231 <blockquote>
 232 <pre>
 233 /etc/postfix/main.cf:
 234     smtp_discard_ehlo_keywords = silent-discard pipelining
 235 </pre>
 236 </blockquote>
 237
 238 </ul>
 239
 240 <h3> <a name="frontbridge_background">Background information</a> </h3>
 241
 242 <p> The SMTP service at mail.global.frontbridge.com does not fully
 243 conform to RFC 2920 which defines the ESMTP PIPELINING extension.
 244 When the service receives a group of pipelined SMTP commands ending
 245 with QUIT, it ignores all commands in that group except QUIT. </p>
 246
 247 <p> In http://tools.ietf.org/html/rfc2920#section-3.1 we see that
 248 "RSET", "MAIL FROM", and "RCPT TO" are examples of commands that
 249 can appear anywhere in a pipelined group, while "EHLO", "DATA" and
 250 "QUIT" are examples of commands that can only appear as the last
 251 command in a group. </p>
 252
 253 <p> Section 3.2 http://tools.ietf.org/html/rfc2920#section-3.2 specifies
 254 required server behavior: </p>
 255
 256 <blockquote>
 257
 258 <p> (1) MUST respond to commands in the
 259 order they are received from the client. </p>
 260
 261 <p> ... </p>
 262
 263 <p> (9) MUST NOT flush or otherwise lose the contents of the TCP
 264 input buffer under any circumstances whatsoever. </p>
 265
 266 </blockquote>
 267
 268 <p> A correct implementation of RFC 2920 is recorded below from an intranet
 269 Microsoft Exchange 2007 server. </p>
 270
 271 <blockquote>
 272 <pre>
 273 S: 220 exchange.example.com Microsoft ESMTP MAIL Service ready at
 274        Sun, 28 Nov 2010 02:41:13 -0500
 275
 276 C: EHLO amnesiac.example.com
 277
 278 S: 250-exchange.example.com Hello [192.0.2.1]
 279 S: 250-SIZE 29999104
 280 S: 250-PIPELINING
 281 S: 250-DSN
 282 S: 250-ENHANCEDSTATUSCODES
 283 S: 250-STARTTLS
 284 S: 250-AUTH
 285 S: 250-8BITMIME
 286 S: 250-BINARYMIME
 287 S: 250-CHUNKING
 288 S: 250 XEXCH50
 289
 290 Note: the following client commands are sent in one single TCP segment,
 291 not in multiple segments back-to-back.
 292
 293 C: MAIL FROM:&lt;sender@example.com&gt;
 294 C: RCPT TO:&lt;recipient@example.com&gt;
 295 C: RSET
 296 C: QUIT
 297
 298 S: 250 2.1.0 Sender OK
 299 S: 250 2.1.5 Recipient OK
 300 S: 250 2.0.0 Resetting
 301 S: 221 2.0.0 Service closing transmission channel
 302 </pre>
 303 </blockquote>
 304
 305 <p> The four pipelined client commands "MAIL FROM", "RCPT TO", "RSET" and
 306 "QUIT" are all processed by the Exchange server which correctly returns
 307 four responses. </p>
 308
 309 <p> The same test against the public MX host for Microsoft's Exchange
 310 engineering group yields similar results: </p>
 311
 312 <blockquote>
 313 <pre>
 314 S: 220 mail7.exchange.microsoft.com Microsoft ESMTP MAIL Service ready
 315        at Sat, 27 Nov 2010 23:43:19 -0800
 316
 317 C: EHLO amnesiac.example.com
 318
 319 S: 250-mail7.exchange.microsoft.com Hello [192.0.2.1]
 320 S: 250-SIZE 104857600
 321 S: 250-PIPELINING
 322 S: 250-DSN
 323 S: 250-ENHANCEDSTATUSCODES
 324 S: 250-STARTTLS
 325 S: 250-X-ANONYMOUSTLS
 326 S: 250-AUTH
 327 S: 250-X-EXPS NTLM
 328 S: 250-8BITMIME
 329 S: 250-BINARYMIME
 330 S: 250-CHUNKING
 331 S: 250-XEXCH50
 332 S: 250 XSHADOW
 333
 334 Note: the following client commands are sent in one single TCP segment,
 335 not in multiple segments back-to-back.
 336
 337 C: MAIL FROM:&lt;sender@example.com&gt;
 338 C: RCPT TO:&lt;recipient@example.com&gt;
 339 C: RSET
 340 C: QUIT
 341
 342 S: 250 2.1.0 Sender OK
 343 S: 250 2.1.5 Recipient OK
 344 S: 250 2.0.0 Resetting
 345 S: 221 2.0.0 Service closing transmission channel
 346 </pre>
 347 </blockquote>
 348
 349 <p> Finally, testing mail.global.frontbridge.com we get non-conformant results
 350 as discussed above. </p>
 351
 352 <blockquote>
 353 <pre>
 354 S: 220 DB3EHSMHS006.bigfish.com Microsoft ESMTP MAIL Service ready at
 355        Sun, 28 Nov 2010 07:47:08 +0000
 356
 357 C: EHLO amnesiac.example.com
 358
 359 S: 250-DB3EHSMHS006.bigfish.com Hello [192.0.2.1]
 360 S: 250-SIZE 157286400
 361 S: 250-PIPELINING
 362 S: 250-ENHANCEDSTATUSCODES
 363 S: 250-STARTTLS
 364 S: 250-AUTH
 365 S: 250-8BITMIME
 366 S: 250-BINARYMIME
 367 S: 250 CHUNKING
 368
 369 Note: the following client commands are sent in one single TCP segment,
 370 not in multiple segments back-to-back.
 371
 372 C: MAIL FROM:&lt;sender@example.com&gt;
 373 C: RCPT TO:&lt;recipient@example.com&gt;
 374 C: RSET
 375 C: QUIT
 376
 377 S: 221 2.0.0 Service closing transmission channel
 378
 379    &lt;Client sees premature TCP close while expecting three more responses&gt;
 380 </pre>
 381 </blockquote>
 382
 383 <p> In this case the client's "MAIL FROM", "RCPT TO" and "RSET"
 384 commands are "lost", and the "QUIT" response arrives out-of-order.
 385 One may speculate that the issue is not in the FrontBridge server
 386 itself, but rather in proxy software in front of the FrontBridge
 387 server. </p>
 388
 389 <p> Significantly, the FrontBridge SMTP servers correctly handle pipelining
 390 of DOT + QUIT at the end of a delivery transaction, perhaps because this
 391 "group" is treated specially. </p>
 392
 393 <blockquote>
 394 <pre>
 395 S: 220 DB3EHSMHS003.bigfish.com Microsoft ESMTP MAIL Service ready at
 396        Sun, 28 Nov 2010 07:27:55 +0000
 397
 398 C: EHLO amnesiac.example.com
 399
 400 S: 250-DB3EHSMHS003.bigfish.com Hello [192.0.2.1]
 401 S: 250-SIZE 157286400
 402 S: 250-PIPELINING
 403 S: 250-ENHANCEDSTATUSCODES
 404 S: 250-STARTTLS
 405 S: 250-AUTH
 406 S: 250-8BITMIME
 407 S: 250-BINARYMIME
 408 S: 250 CHUNKING
 409
 410 Note: the following client commands are sent in one single TCP segment,
 411 not in multiple segments back-to-back.
 412
 413 C: MAIL FROM:&lt;sender@example.com&gt;
 414 C: RCPT TO:&lt;recipient@example.com&gt;
 415 C: DATA
 416
 417 S: 250 2.1.0 Sender OK
 418 S: 250 2.1.5 Recipient OK
 419 S: 354 Start mail input; end with &lt;CRLF&gt;.&lt;CRLF&gt;
 420
 421 C: [message-content]
 422 C: .
 423 C: QUIT
 424
 425 S: 250 2.6.0 &lt;20101128070828.AB036504373@amnesiac.example.com&gt;
 426              [InternalId=11073071] Queued mail for delivery
 427 S: 221 2.0.0 Service closing transmission channel
 428 </pre>
 429 </blockquote>
 430
 431 </table>
 432
 433 </body>
 434
 435 </html>