Introduce enum flock_type for fcntl_lock() (drop fcntl.h)

[s-mailx.git] / NEWS

S - n a i l  N e w s
====================

The complete changelog of commits in between two versions can be
inspected by using the git(1) `log' command as shown below, where `OLD'
and `NEW' are the two versions to be compared, e.g., v14.4.5 and v14.5.2:

  # All commits:
  $ git log --reverse --topo-order OLD..NEW
  # Only topic branch headers:
  $ git log --oneline --merges --reverse OLD..NEW
  # Same, but truly accessible:
  $ git log --oneline --parents --merges --reverse OLD..NEW |
    while read c1 c2 c3 c4 c5 c6; do
      printf "%-24s: \$ git log --oneline --no-merges ${c1} ^${c2}\n" "${c6}";
    done

v14.5.2: fix 4: 2014-02-15 (2014-02-14)
---------------------------------------

- s-nail-14_5_2-smime.patch
  Fix a very stupid off-by-one error that i've introduced
  in [7bdf330] (OpenBSD 5.3: sigh, address strcpy(),strcat()
  etc., 2013-06-01).
  Until we've changed the used data from string to something
  line-wise, use strcat(3) again.

v14.5.2: fix 3: 2014-02-10
--------------------------

- s-nail-14_5_2-maildir.patch
  maildir folders would have caused problems in environments which
  provide the realpath(3) function: beside a "cannot canonicalize PATH"
  warning the finally used path would be wrong (`test3' would end up as
  `test3/test3'), so that any further access would try to use the wrong
  path.  Please read the description of this patch and/or the commit
  logs of the commits [1c2563b] and [13f325f].

v14.5.2: fix 2: 2014-02-05
--------------------------

- s-nail-14_5_2-mimeheader.patch
  A header like

    Subject: ehm, .getElementById("blink") needs <span
     =?US-ASCII?Q?id=3D"blink">,?= not =?US-ASCII?Q?class=3D"id"?=

  would yet be displayed without the " not " in between the two encoded
  words because of faulty "encoded-word-continuation" detection (note
  the quotation marks).  The error path could also have been seen in
  mail forwarding and in faulty searching etc.
  This patch is in a row of fixes for my hasty [0f9ad93] from 2013-03-12
  that already caused the v14.2 minor release (because of [b608c6b] from
  2013-03-14).  Those with mercy may read the commit message of [c81afce].

v14.5.2: fix 1: 2014-01-30
--------------------------

- s-nail-14_5_2-sort.patch
  Reverses (sort(),thread(): use srelax()!, 2014-01-18, [a9b67e9]),
  which was a hasty commit of an untested diff that i've added few
  minutes beforehand:

    As a rather careless last-minute change i've added string relaxation
    to threaded and sorted display, but it's really one more step towards
    lowering memory pressure -- i couldn't resist [a9b67e9] after seeing

  An alternative, forward-heading patch that keeps string relaxation has
  been pushed to [master] as [5e75529] and is also available as
  s-nail-14_5_2-sort-alt.patch.

v14.5.2, 2014-01-18
-------------------

Thanks to Ypnose, Sunil Nimmagadda and Gavin Troy.
Gavin Troy *really* deserves special thanks for facing [next]!
And i want to dedicate the new coloured message display functionality
to John Dodson and Ypnose.  Thank you.
(And best wishes to beautiful Australia!)

ChangeLog (packager-affine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

- All utilities can now be overwritten during configuration so that
  their values are fixated in the generated makefile (`mk.mk').
  I.e., talking about MAKE=, STRIP=, awk=, cat=, chmod=, cp=, cmp=,
  grep=, mkdir=, mv=, tee=.
  rm= and sed= have to be overwritten from the command line, they're
  needed before `conf.rc' is read.
  [5c03347, 072ec65, 39a00ab, 23a1245, ?]
  (Ypnose)

- The release tarball is now also available in a xz(1) version.

- New configuration option: WANT_COLOUR, by default enabled.

- 'make test' should now really work, even if the running user has
  a mailbox with content. [f223a91]

- WANT_AUTOCC is now by default enabled, so as for normal users which
  don't have the need to embed into a defined packaging environment.
  [d7e4e31]

- Installation no longer strip(1)s away debug symbols if WANT_DEBUG was
  enabled. [6d075c6]
  (Gavin Troy)

ChangeLog
^^^^^^^^^

- Fixes to some bugs that are present since the first cvs(1) commit of
  Heirloom mailx(1); includes that *ssl-key-user@host* should now work.
  [4405a2cc, 9770c26f, 692976b9]

- Some off-by-XY fixes, thanks to the debug memory canaries.
  [19b2b0d, 202506e, 43df6e4]

- Fixes for (other) stupidisms (of mine): [1c2161f]

  This includes true implementation of in-memory history limit for the
  NCL, which was the final and real solution to a segmentation fault
  that Gavin Troy had to deal with on [next]. [1089f2b]
  (Gavin Troy)

- For completeness: new command `var-inspect' shows information about
  all given options.  Mostly ment for implementing future tests.
  [topic/okeys]

- The `pipe' command no longer embeds message information into the
  data passed through to the command (when *piperaw* is set).
  [ef5ecc6 (part of topic/colour)]

- Simple coloured message (header) display is now possible.  Please
  read the new manual section "Coloured message display", use
  *colour-disable* to turn it off.  (It is enabled by default if it
  knows the terminal is capable and, if used, the pager can, too.  Note
  we now set LESS=FRXi when starting PAGER and no LESS= is in the
  environment.)

  Dedicated to John Dodson and Ypnose.
  [topic/colour, c6e84c7]

- The `if', `else', `endif' syntax has been extended.
  You can now "if 0" (never), "if 1" (always), "if $OPTION" (boolean
  check for OPTION) and "if $OPTION == 'VALUE'" as well as "if $OPTION
  != 'VALUE'".  Unfortunately it is still not possible to use
  conditionals inside conditionals. [0fb2ae7]

- -# now also sets MBOX=/dev/null. [4be2f1e]

- The NCL command line editor now supports cursor keys when the terminal
  produces xterm(1)-compatible keycodes ('ESC' + '[' + [DACB] for left,
  up, right and down, respectively).  What a thrill, yay!!! [0cbf672]

- New (optional) command: `history': show or clear command line history,
  or select a specific command line from in there.
  History works a bit different now, and should no longer include
  command lines which include specific message numbers; more to come.
  [59c6195, topic/hist2]

  I plan to join all the history management and use only the one that is
  part of NCL now, hooking it into editline(3) and readline(3).  That
  would shrink tty.c a bit and also introduce duplicate elimination for
  readline(3).

- The new ~u and ~U tilde escapes work like ~f and ~m, respectively, but
  don't include any header lines.  Inspired by a patch from
  Sunil Nimmagadda on openbsd-tech@.  [c37b8b3]

- The `|' command should work again -- it has stopped working on
  2013-09-09 when i've accidentally changed the command name from `|' to
  ` | '. [5a8378d]
  (Gavin Troy)

- As a rather careless last-minute change i've added string relaxation
  to threaded and sorted display, but it's really one more step towards
  lowering memory pressure -- i couldn't resist [a9b67e9] after seeing
    ?0[ /Users/steffen/src/nail.git/t.mbox]? sst
      Buffer allocs ever/max simultan. : 14/14
      Overall alloc count/bytes        : 17165/881088
      Cycle maximums: alloc count/bytes: 16906/876984+0
    ?0[ /Users/steffen/src/nail.git/t.mbox]? sst
      Buffer allocs ever/max simultan. : 0/0
      Overall alloc count/bytes        : 16515/841816
      Cycle maximums: alloc count/bytes: 16256/837712+829560

ChangeLog (purely technical)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- We now have debug canaries for all memory sources now that [8419d44]
  added them to the "string dope".

- The most work has been done on our value system, which manages the
  binary and value options, like *folder* etc.
  It is now based on enumerations, i.e., constant integers, not on
  strings.  This of course only relates to non-dynamic options.
  Anyway, this saves us key hashing and allows more compact data
  representation in general (see the new header `okeys.h' for more).
  [topic/okeys]

v14.5.1, 2013-12-27
-------------------

ChangeLog (packager-affine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

- The build system *only* uses the automatically detected $CFLAGS and
  $LDFLAGS if WANT_AUTOCC=1.  I.e., even unset or empty $CFLAGS and
  $LDFLAGS are not touched until then.
  (We do however still set $CC if that is unset or empty or set to the
  plain string "cc".) [856625f6]

ChangeLog
^^^^^^^^^

- Fixed segmentation faults / bus errors when setting *nofolder*
  / *line-editor-cursor-right* to the null string (only with WANT_NCL),
  respectively. [d1f1a19b, 21e5c285, 9f6ff25d]

- *prompt* handling is now really POSIX compliant (thus no prompting
  occurs not only for 'set noprompt', but also for setting *prompt* to
  the null string).

  This was indeed a rather large changeset that also introduced the new
  *prompt* escape character \&, which expands to `?' by default and to
  `&' if *bsdcompat* is set.

  Like that we now can simply assign "\& " to *prompt* at program
  startup, which (a) allows to do 'set noprompt' without error (once)
  and (b) allows for POSIX compliance in respect to prompt handling
  without any complicated conditional code, but (c) gives us the
  opportunity to continue to support BSD prompts.
  [0dfe53db]

- For completeness: new command: `features'.  (Rather useful for being
  able to implement more tests in the future, and act according to what
  is really compiled into the tested binary.

- The `-#' command line option now also sets *quiet* by itself.
  [7b5a5c87]

- nail.1: a newly introduced empty line in the manual produced error
  messages on some systems.  Fixed.

- The return value of the `mimetypes' command has been reversed and
  should now be fixed. [acf56ac52]

- In threaded display the Subject: followup suppression no longer
  takes into account invisible messages.
  Also, rudely hack in a messages-already-written-in-this-round counter,
  so that the followup suppression knows when "the top of the screen" is
  reached, which (seems to) help(s) against missing subjects up there as
  well as after a `newmail'. [topic/subject]

- Added a WANT_REGEX=1 toggle in `conf.rc'.
  When we find regular expressions then a new regex-enabled IMAP-style
  search is available (see the manual for more) [1ec8fe68]

    ? f (/or subject ^\[S-nail (subject ^\[nail-devel))
    ? f (/subject ^\[S-nail) (/subject ^\[nail-devel)
    ? f (/subject "^\\[(S-nail|nail-devel)")

  I'm looking forward for being able to add another, simplified, syntax.

ChangeLog (purely technical)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- The cc-test.sh has seen some tweaks, for easier future extension, and
  for adding a test for [d1f1a19]. [several, mentioned: 21e5c285]

- On systems without a real wordexp(3) implementations deadlocks could
  occur because we sometimes hold_all_sigs() to avoid longjmp(3)s away
  (and will do so for quite some time, still), and that resulted in the
  SIGCHLD that reported the exit of the started subshell to be blocked,
  too (e.g., after '? *.h<Tab>': endless hang).  Fixed.

v14.5, 2013-12-19
-----------------

Many thanks: Gaetan Bisson, William Yodlowsky, Gavin Troy,
Thomas (wasd AT gmx DOT net), Ypnose.
And Gavin Troy definetely deserves a very special credit.
But thank you all, and very much indeed!

ChangeLog (packager-affine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

- The `test' make target has been fixed. [f0991e14]
  (Gaetan Bisson)

- It is possible to gain a different kind of make(1) verbosity by
  using a VERBOSE=1 command line argument (this knob is not taken into
  account when deciding wether a rebuild is needed). [498e4ad0]
  (William Yodlowsky)

- On Crux 3 Linux and OpenBSD the readline(3) and editline(3),
  respectively, libraries will now be found when desired. [a7d1aa78]
  (William Yodlowsky)

- WANT_LINE_EDITOR has been renamed to WANT_NCL, *plus*.
  So now there are WANT_READLINE, WANT_EDITLINE and WANT_NLC, each of
  them can be set individually, and they are tested in the shown order.
  Also, WANT_TABEXPAND and WANT_HISTORY have been introduced and can be
  used to fine-tune functionality. [ae4e01e1, b2635feb, 9742bf40]

  (While here, i've fixed WANT_TABEXPAND code so that it is more
  sensitive to line excess; on Linux etc., where MAX_INPUT is 255,
  strange behaviour could be seen because we didn't take into account
  the length of the prompt at all.  The NCL is assumed to have only
  one remaining, but unfixable problem: backspace often is incapable
  to cross visual line boundaries; use ^A/^E + ^L, then. [e832c04a]

  Also, cursor (now ^B and ^F) and history movement (now ^P and ^N) of
  the NCL have been changed. [d7d928da])

- WANT_QUOTE_FOLD is now enabled by default.  And WANT_ASSERTS has been
  renamed to WANT_DEBUG. [1e10da1f]

- The build system has seen yet another overhaul in general.  CC, CFLAGS
  and LDFLAGS plus are now tracked and changes will force rebuilds.
  The new WANT_AUTOCC option can be used to let the build system figure
  out a compiler and choose known-to-work flags.  Use the new ADDCFLAGS=
  and ADDLDFLAGS= command line arguments to add your specific flags on
  top of those -- the final CFLAGS etc. are what is change-tracked.

  This rather massive internal rework revealed that old Bourne shells
  were yet not supported by the new build system, and so did testing
  that UnixWare installation was yet impossible due to tool
  incompatibility. [75c4b74e]

- The new WANT_AMALGAMATION option will force compilation of all the
  sources in a single compilation unit.  This requires a rather large
  amount of memory, but may produce a more compact, maybe more optimized
  binary.  (Implementing this revealed quite some bugs which could
  therefore be fixed.) [topic/amalgam]

- `nail.rc' has been pimped a bit (mostly comments, but
  *mime-counter-evidence* is now always set). [e3094ba7]

   That changeset was however buggy. [f3dcb46]
   (Gavin Troy)

- We no longer use install(1) for `install'ation make rules. [80b02cd9]

ChangeLog
^^^^^^^^^

- Even '$ s-nail & fg $!' will now work with the NCL. [2a8b5c55]

- Several off-by-one (off-by-two) fixes. [32ce9836, 71e6d013, f139dc36]
  (Gavin Troy, Thomas)

- Setting *noprompt* now prevents prompting, as per POSIX. [ecefaf63]

- *prompt*: new \$ (exit status of last command) and \@ (name of
  currently active mailbox) escape sequences. [6f652046]

- One may now omit the space in '? unc' ('?unc') [05fcb383]

- New commands: `ghost' and `unghost' define command aliases (since
  `alias' is taken for a different purpose) [topic/commands]

    ? ghost ps '!ps axu'
    ? ps |grep nail

- There is now a pseudo account `null' (case-insensitive).
  Also a new `localopts' command exists; when used from within an
  `account' block, options changed will be reverted back to its former
  value when the account is left (e.g. by switching to `null'):

    define sdn_ {
       alternates sdaoden@users.sf.net sdaoden@users.sourceforge.net \
          sdaoden@googlemail.com sdaoden@gmail.com
       set Sign="\n--steffen\nForza Figa!" sign="\n--steffen"
       set smtp=smtp.gmail.com smtp-auth=plain smtp-use-starttls
       #..
    }
    account sdn_gm {
       localopts 1
       call sdn_
       set from="Steffen \"Daode\" Nurpmeso <sdaoden@gmail.com>"
    }
    account sdn_sf {
       localopts 1
       call sdn_
       set from="Steffen \"Daode\" Nurpmeso <sdaoden@users.sf.net>"
    }

  E.g., after

    ? acc sdn_gm
    ? acc null

  neither of *Sign*, *sign*, *smtp** nor *from* should be set.
  Please see the manual for more.
  TODO - neither command-ghosts nor alternates etc. are yet tracked
  TODO - we should have a boolify() so as to say 'localopts yes' etc.
  [topic/acmava]

- New command: `cwd' (print current working directory).
  Also fixing the `chdir' return value. [eff4397c]

- The *ssl-method* now allows explicit setting of 'tls.1.1' and
  'tls1.2' values. [c66b4196]

- When sending to display, be aware that filenames in MIME parts may of
  course be MIME-encoded! [1454be03]

- *hostname* is now honoured even if *smtp* is not set.  (We always
  supported *from*, so why not *hostname*?)

- The `-u user' option now acts identically to setting the $USER
  environment variable and both now tend to mean something like
  "impersonate as user in some aspects".  Note that we have always used
  the latter in one or the other way, and `-u user' always ment more
  than just "open mailbox of user", so i think this change sharpens the
  edge in the right direction. [09632731]

- Filename argument quoting has been tweaked for (some) function(s which
  take a filename argument last).  The following snippet as reported by
  Gavin Troy should work now: [2bb9b80e]

    ? mv +inbox.Junk\ Mail

- The GNU implementation of wordexp(3) is also (i've added a workaround
  for the very same bug for Mac OS X in S-nail v14.3 [63273772]) buggy,
   which causes segmentation faults when expansions failed (`fi &VOID').
  [470527b7]
  (Gavin Troy)

- The `fi' command no longer uses the (possibly truncated) display
  version of a filename, but the full path. [5cd85b07]

ChangeLog (purely technical)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

- Large rework of internal structure: bundle inclusion of most of the
  external and the content of most of the internal #include files in the
  new nail.h.  If that would have been done very first bugs like the
  infamous MAXPATHLEN bug as reported by Paul Vojta (see v14.4.2) would
  never have occurred. [21f3155b]

- Support for -fstrict-overflow cc(1) flags. [topic/strict-overflow]

- Fixed the quotation filter which yet allocated memory even if not
  used. [c98cdaf5]

- String relaxation reduces memory pressure rather drastically when
  working with many (especially MIME) mails at a time, e.g., when
  writing a modified mailbox.  Before all messages of a mailbox had to
  be worked without releasing any memory in between, now we give back
  memory (to our pool, not the system) after each and every message.
  [topic/srelax]

- The other memory source now uses bound canaries, which also found some
  errors. [3d9fe741]

- We now use the EL_PROMPT_ESC editline(3) mode for prompting, which
  should offer the possibility to use coloured prompts etc. with
  (even those) editline(3) (versions which do offer it -- older versions
  should just do fine by themselves).
  S-nail uses the special trigger control character \1. [ea30d818]
  (Ypnose)

  Note however that all tested editline(3) versions are buggy and
  either don't get it right (`\1COLOR-ON\1stuff\1COLOR-OFF\1') or are
  incapable of proper repainting (`\1COLOR-ONstuffCOLOR-OFF\1').

- We now use the MD5 digest code from the OpenSSL library if that is
  usable. [893b16c0]

v14.4.5, 2013-10-19
-------------------

Many thanks: Gaetan Bisson, Stephen Isard, Jérémie Courrèges-Anglas,
William Yodlowsky, and Adam Sjøgren from GMANE.org!

ChangeLog (packager-affine)
^^^^^^^^^^^^^^^^^^^^^^^^^^^

- The packager-install: target has been fixed. [a8c1b0b]
  (Gaetan Bisson and William Yodlowsky)

- As suggested by Gaetan Bisson the several build system tasks are now
  individually addressable, i.e., 'make [OPTIONS] config', 'make build',
  'make test', 'make packager-install'); nothing changes unless you want
  to, in which case: please see INSTALL. [4d3b799]

  The new target 'test' will call cc-test.sh with its new --check-only
  option which will only perform the (too few) function tests on the
  ready binary.
  And cc-test.sh will no longer create output files, but simply echoes
  to STDOUT and STDERR. [629e1ee, 786f219]
  (Gaetan Bisson)

- The default configuration file now sets *bsdannounce* and *bsdflags*,
  all in one line, as suggested by William Yodlowsky.
  (If *header* is disabled, *bsdannounce* is ignored, but *header* must
  be enabled by default according to POSIX, and *bsdannounce* is just
  the same as *header*, but for the folder-switched event.  Thus setting
  *bsdannounce* is the more sane default, imho.) [6161f10]
  (William Yodlowsky)

- Announcement messages are now tagged '[ANNOUNCE]'. [26e1b35]
  (Tagging suggested by Stephen Isard and Jérémie Courrèges-Anglas)

- The manual NAME now includes the version (but it's somewhat ugly).
  [8adcca4]

ChangeLog
^^^^^^^^^

- When switching folders when in compose mode, message attachments
  become invalid.  For v14.4.5, at least warn when this happens.
  [47eb1ab]

- When reading multipart/alternative messages which do not contain
  a text/plain part, the (most likely HTML) part is not displayed (even
  if S-nail is configured to display HTML).  The reason is a bug i've
  introduced with [0d43a999] (Change "Part X:" display message..,
  2012-12-20).  (Note that the committed fix i've posted to the list was
  not correct either and has been fixed itself.) [225c02b, ecfa149]

v14.4.4, 2013-10-03
-------------------

Thanked, Gavin Troy, Gaetan Bisson.

- Fix output buffer confusion that would have occurred when parts of
  a multipart mail had a pipe command set.  Longer story in [e75d16dd].
  (Gavin Troy)

- The makefile now supports a `packager-install' target that bypasses
  the reevaluation of the configuration (after checking that there is
  one) and directly steps ahead to the install process.
  (Gaetan Bisson)

- Notes in INSTALL that CFLAGS and LDFLAGS need to be overwritten from
  within (conf.rc or) the command line.
  (Gaetan Bisson)

v14.4.3, 2013-10-02
-------------------

I convey special thanks to Karol Błażewicz and Gaetan Bisson.

- The new build system didn't allow to overwrite CFLAGS= when used in
  conjunction with GNU make(1).  To make a long story short, removing
  a `.POSIX:' directive from the makefile fixed the problem.  Luckily.
  [e6b26526]

- More INSTALL notes for UnixWare 7.1.4, which suffers from the same
  problem: here the fix is to use the -e option of make(1).

- Karol Błażewicz reported a segmentation violation he got when using an
  Arch Linux S-nail test package, and Gaetan Bisson informed me about
  that.  The problem was that we didn't assign the resulting default
  address (which gets used when *from* is not set) to the result which
  gets used, effectively resulting in a NULL dereference. [cfd60479]

v14.4.2, 2013-09-21
-------------------

Many thanks to Paul Vojta.

- On GNU/Linux there is no MAXPATHLEN constant defined by default, as
  this is a non-standard legacy constant.  Mozilla ran into this some
  time ago, though a bit different [1].  It made it easier for S-nail.

    [1] <https://bugzilla.mozilla.org/show_bug.cgi?id=412610>

  We could include `sys/limits.h' to get at MAXPATHLEN, there it is
  defined to be PATH_MAX, but let's just include the standard `limits.h'
  and ensure MAXPATHLEN ends up >= PATH_MAX.  With this condition being
  true even FORTIFYd sources won't bail with reported buffer overflows
  because realpath(3) expects a buffer of PATH_MAX bytes. [bee5e57c]
  (Paul Vojta)

- With the builtin LINE_EDITOR, ensure proper interaction with GNU
  rlogind(8), which is alone in its quest to set the ISTRIP termios(4)
  flag. [61e00f2d]

- Bugfix a codepath that would have prevented compilation when
  HAVE_ICONV is not set.
  While there, allow to unselect message attachments when using the `~@'
  tilde command interactively, which i mysteriously had forgotten when
  allowing selection of 'em.  (oops?) [4f58ffea]

- It's now easier for packagers to get at debug-enabled CFLAGS; simply
  use WANT_ASSERTS=1 (should not be enabled in shipouts, please).
  [d8663406]

- Fixed a terrible bug that i've introduced in november 2012, that would
  have corrupted MBOX files when using the `resend' command.  (The
  so-called "From_ line" would have been quoted to ">From_ line" instead
  of simply being stripped.) [19d449e2]

- Fixed a SIGSEGV for the builtin LINE_EDITOR, that would have occurred
  if a history traversal (^B or ^F) would have been continued by
  a reverse history completion (^R).
  While there, avoid some multiple-beeps-in-a-row that yet occurred due
  to code reuse, and document that ^G etc. reset the multibyte state
  machine. [003d9f6f]

  Also, the expand-on-tab code could have excessed line input maximums,
  e.g., after `? /usr/bin/*<Tab>'.  This was of course known, but
  i wanted to keep it because all tested (modern) terminals "mess up but
  won't break", and thus made it possible to use the data, even if it's
  visual representation was messed up.  The real solution will be much,
  *much* more complicated, i.e., show possible *completions*, page-wise,
  etc., i.e., just like is known by tab *completion*.  This will need
  far more time because it doesn't make sense to embed such complicated
  code into the current bed.
  Now i've chosen to simply don't display excess, but replace the
  content with a message that says that there was excess.  It simply
  looks better. [0320c8ba]

- I've changed the use of `[?]' as a replacement sequence for invalid
  / non-displayable characters to `?' (again).  There are still code
  paths (from the original codebase) which use the real Unicode
  replacement character instead, so our display is still not unique.
  On the long term the codepath can anticipate in all levels wether
  use of Unicode is possible, so that, then, we will be unique again,
  using either only `?' or the Unicode replacement character. [faf6380]

- I've anticipated in v15.0 and implemented *quote-fold* as a stream
  filter.  It is now multibyte safe and takes into account the visual
  width of characters, i.e., ideographs etc.  It is working almost as
  good as an algorithm can work that looks at data linewise, but is yet
  experimental and incomplete in that it will break lines even if only
  whitespace or a backslash escape follows.  It is code that is only
  a few hours old, which is why it is not enabled by default. [

v14.4.1, 2013-09-14
-------------------

How terrible.

- Fix some harmless stylos and typos in the manual and `conf.rc'.
  [1a6b880, 417b066]

- Ensure *complete* proper cleanup on signal-caused configuration run
  aborts. [7a08bda]

- Bugfix: reenable empty configuration variable overrides, as in

    $ make install SID= NAIL=mailx ...
  [171861d]

- Bugfix: rename the `conf.rc' variables SHELL, LISTER and PAGER because
  they clash with the POSIX standard variables of the same names, which
  hurts now that the configuration is run through the shell itself.
  The configuration names are now XSHELL, XLISTER and XPAGER (ouch).
  [ef93c4e]

- nail.rc: comment out some non-portable S-nail(1) specifics, for those
  who install this file as mailx.rc or so, somewhere. [3f14b01]

- From now on S-nail will use simple version tags, as, e.g., `14.4.1',
  i.e., no more `s-nail-14.4.1'. [a75437d, 183d59c] (Jürgen Daubert)

  Also move UAGENT out of `version.h' [f7be5be]

Uh! v14.4 must have originated in a bad dream; good it's vanished.

v14.4, 2013-09-13
-----------------

Thanks to the entire vivid and virile S-nail(1) user community is
proper, especially Stephen Isard, Gavin Troy, Martin Neitzel.
Not to forget Ryan Kavanagh and Ayan George.

Note this time the changes are in reverse order, i.e., oldest first.

Topic Branches
^^^^^^^^^^^^^^

15fbe09 ^bf9173f 'topic/varmac-unite'
  Simplification and unification of variable and macro handling
40f6f58 ^15fbe09 'topic/cledit1'
  Command line editor; new manual section `Line editor'
ad28a32 ^40f6f58 'topic/termsize'
  Honour POSIX mailx(1) and respect $COLUMNS and $LINES on startup
dc3cd49 ^a446fd8 'topic/qf'
  You don't wanna know
b8738f7 ^0651fd0 'topic/spam1'
  Interaction with SpamAssassin; new manual section `Handling spam'
5419d6f ^b8738f7 'topic/make1'
  Reworked build system; please read `INSTALL' and `conf.rc'
9ab4d6b ^5419d6f 'topic/list1'
  Slightly tweaked message thread display, fix `:u :r' to mean it

ChangeLog
^^^^^^^^^

- The default PAGER is now more(1).  (But read on.)

- The `echo' command is now compliant, and fully supports XSI.

- The `group' and `ungroup' command aliases have been removed, they are
  `alias' and `unalias'.

- We now have the capability of line editing and history.
  One may choose from not less than three different implementations:
  a builtin editor as well as possible linkage against BSD editline(3)
  and also a GNU readline(3) compatible layer.
  Please read the new manual section `Line editor'.

  Input is now compliant in that an interactive line may be continued
  after escaping the newline character with a backslash.
  [topic/cledit1]  (all of them)

  The *prompt* variable may now contain shell escapes, just like the
  `echo' command. [0938d8a]
  As a special extension the new \? escape, when used within *prompt*,
  will expand to the exit status of the last command. [41076d2]

- The POSIX standard environment variables $COLUMNS and $LINES are now
  honoured upon startup. [topic/termsize]

- The `help' / `?' commands now support abbreviation, i.e.,
  ``$ ? unc'' should now find `uncollapse'. [7b86195]

- The `~' abbreviation that has been introduced as an alias for `call'
  is now a real command, not a magic shortcut. [9987289]

- The new variable *quote-as-attachment* can be used to additionally
  embed the quoted message as a `message/rfc822' MIME attachment.
  [topic/qf]

- The compose-mode command `~@' will now attach messages from the
  current mailbox if given a filename of the style `#NUMBER'.
  Please read the corresponding manual section `Tilde escapes', though.

- The `WANT_JUNK' and `WANT_SCORE', as well as Gunnar Ritters junk mail
  management have been removed.

  Instead S-nail(1) can now support interaction with SpamAssassin, but
  sofar only via the spamc(1) / spamd(1) client / server pair of
  programs that ships as part of SpamAssassin.  The new configuration
  directive `WANT_SPAM' controls wether this feature is desired.

  Please read the new manual section `Handling spam'.
  [topic/spam1] (Martin Neitzel)

  (S-nail(1) is now *definitely* floating-point free.)

- The configuration and make system have been overhauled / reorganized.
  The configuration is now in `conf.rc', also contains directives like
  `PREFIX' etc, and is always read in.  However, only those directives
  which are not yet set (via environment or command line overwrites) are
  incorporated into the set of configuration options.  Therefore

    $ make WANT_IMAP=0
    $ make WANT_IMAP=1

  will now build S-nail(1) twice, because of the changed configuration.

  Note that `WANT_GSSAPI' is now by default disabled, which shrunk
  a freshly started s-nail image by more than 30 percent.

  We're not finished yet in that there is no dependency graph etc.
  Please do read `INSTALL' and `conf.rc'.
  [topic/make1]

- Message selection has been slightly bugfixed in that `:u :r' really
  means `:u' AND `:r'.  Compared to NetBSD Mail(1) it's still a shame.

  Threaded message display has been slightly changed in that within
  a thread identical Subject: lines are not repeated.  It may not be
  perfect yet due to the general list / thread state. [topic/list1]

- If, upon startup, the environment variable `NAIL_NO_SYSTEM_RC' is set,
  then the system wide initialization file isn't read, just as if the
  `-n' option had been given. [1b31535]

- It is now possible to use CTRL-C during connection hangs.  (But in
  general error recovery capabilities of the socket related
  infrastructure is non-existent, practically speaking.) [45a9f36]

- *quote-fold* has been temporarily disabled, as it is not multibyte
  safe.

No review for v14.4.  And today is Friday, the 13th.  Ouuuh!

v14.3.2, 2013-06-21
-------------------

I should *maybe* should have and want to give prominence to
Martin Neitzel for this, i maybe have misunderstood.

- Bugfix *synchronous* *pipe-** execution..

  Well, unfortunately yet another newly introduced bug slept in
  S-nail v14.3[.1] -- [a8d724b3, Add @ and @& shell command prefixes
  for pipe-MIMETYPE, 2013-05-03] falsely changed the waiting state
  for subprocesses, as has shown up by a HTML-only mail on the
  Unicode list today.
  'Seems i'm collecting one line fixes in this codebase; this needs
  to change in the future.

v14.3.1, 2013-06-08
-------------------

I want to give prominence to Juergen Daubert (jue AT jue DOT li), who
reported that i've broken plain-old unfancy send mode in 14.3.

+ Ok, i'll hope we're out of new errors for the v14.3 series with that.

- New variable: *batch-exit-on-error*.
  Only works if the new -# command line option has been given, and will
  check the "current" exit status whenever one operation completes
  (S-nail returns to the command prompt).
  If the exit status implies error (e.g., sending the last message
  failed) then we exit forcefully with that error status.  (The normal
  behaviour is that the status is reset when the command loop ticks.)
  [4cddd55]

- While here again, i've added the new -# command line option.
  This is the first step to implement a reliable batch mode;
  unfortunately it still selects the users system mailbox on startup,
  because we simply cannot go to "no" mailbox for quite some time -- at
  some future time we will be able to go to some VOID thing, and then
  this will end up as a rather efficient batch mode.
  For now it sets *dot*, *emptystart*, *noheader* and *sendwait*, and
  also implies the -~ command line option.  [7549569]

    (
      printf "m ${MBOX}\n~s subject1\nE-Mail Körper 1\n.\n" &&
      printf "m ${MBOX}\n~s subject2\nEmail body 2\n.\n" &&
      echo x
    ) | MAILRC=/dev/null "${NAIL}" -n -#

- Also, -N set *header* instead of *noheader*.  [7b4a13f6]
  (Juergen Daubert.)

- Plain old unfancy invocations like

    $ s-nail user@host
    $ echo bla|s-nail ./FILE1

  had been broken (by [522cb3ec]).  [260e19d]
  (Juergen Daubert.)

v14.3, 2013-06-03
-----------------

Thanks to Gavin Troy (gavtroy AT gmail DOT com) who inspired the @ and
@& pipe-command prefixes.

Random notes
^^^^^^^^^^^^

- S-nail has been registered at Coverity Scan, and the third build
  (after topic branches *coverity-444* and *coverity-444.2*) produced no
  more errors.  (<http://scan2.coverity.com/projects/444>.)
  (Then i used POP3 and IMAP and fixed some SIGSEGV. ;)  Still didn't
  look at S/MIME, Maildir, caches etc... o()

- S-nail v14.3 doesn't produce any spurious linker warnings on
  OpenBSD 5.3; all (correct!) use cases of strcpy() and strcat() have
  been replaced.  [7bdf330, 2c8d7cb]

- This is the first release with a (though very short) review -- i'm
  slowly getting comfortable with the code.  (But i'm too stupid to
  perform reviews on patches, 'always did reviews on C++/Perl/xy
  classes.  Aaah, how beautiful ... objects.)

ChangeLog
^^^^^^^^^

- It is now possible to "call" macros without using the `call' command
  by prefixing them with a tilde, as in

    ? define au {
      echo auau
    }
    ? ~au
    auau
  [93ea8acd]

- Added the *pop3-bulk-load* option.
  Yes, there are mailing lists etc. which use plain text email, and,
  there, headers are often more data than the body, so it doesn't make
  sense to download the headers twice (unfortunately POP3 doesn't
  support a BODY command; if only it would support a RETRDELE command..)
  [978e13a7]

  And yep, from this changeset on i personally use S-nail even over the
  network, no longer my stale and incomplete S-Postman.  And i can tell
  you, this damn thing is so silent, i always set *verbose* not to go
  grazy ... but .. i hate to say it .. the healing will take time.

- POP3 will now try to use APOP authentication automatically; thus the
  *use-apop* stuff has been replaced by *pop3-no-apop* options (just in
  case there are POP3 servers which advertise they support APOP but in
  fact fail to do so; anyone?)  [6c3c5575]

- Some IMAP segmentation violations have been fixed:

    ? fi imaps://user1@localhost
    Password:Interrupt            <- CNTRL-C
    ? set imap-auth=cram-md5      <- hey, 'forgot to set correct auth
    ? fi imaps://user1@localhost
    IMAP write error: error:140D00CF:SSL routines:SSL_write:protocol is shutdown
    Segmentation fault

  And also, when *folder* was set to an IMAP account but hasn't been
  opened yet, and no IMAP account ever has been opened, a string
  comparison against a NULL pointer yet caused a SIGSEGV, too.
  [417c01f, 413c23d9]

- `set folder=' now tolerates `%:' and expands PROTOs stuff etc.:

    ? short xp %:imaps://user1@localhost
    ? set folder=xp

  Pure convenience so that it doesn't need to be typed twice (still no
  completion in sight...).  Note that setting *folder* to a POP3 box
  will now be actively rejected.  [b12b17f5]

  NOTE: while implementing this i've detected another dead-end
  miscondition in S-nail -- you really should ensure that your target
  folder/box is connected before you leave your current POP3/network
  based folder, if there is data to be moved to the target (i.e.,
  mbox).  This problem will persist for a long time due to the way the
  entire codebase functions; i hope i can find a short/mid-term
  solution, but the real healing will take years.  The mentioned
  solution would at least make S-nail interruptable, currently we get
  stuck and interrupts are blocked...

- If you're using S-nail on Mac OS X and have seen some segmentation
  faults when expanding shell stuff then you may be pleased to hear that
  S-nail now works around an Apple bug.  [63273772]

- The builtin mime.types have been corrected and a lot of new ones have
  been added.  New data from
  <http://svn.apache.org/viewvc/tika/trunk/tika-core/src/main/resources/\
  org/apache/tika/mime/tika-mimetypes.xml>, thanks!  [8072fcb6]

- BEWARE: handling of command line arguments has changed a bit!

  1. The -D, -d, -E, -i, -N and -v command line options are now
  implemented by means of setting the respective option, as via -S.
  (This means that from now on resource files can only *temporarily*
  overwrite command line arguments.)

  2. The -I and -T command line arguments have been dropped.
  It seems Gunnar Ritter stopped developing nail/Heirloom mailx once he
  started implementing Newsreader functionality.  It'll take a long time
  until we get there, so for now drop all the Newsreader stuff.

  3. Handling of -r has been changed.  E.g.:

    s-nail -A test -Snoeditalong -r 'La mort est <fem@me>' -d
    ? set from=bummer@m1.com
    ? m t1
    Subject: s1
    .
    Sendmail arguments: "sendmail" "-i" "-r" "fem@me" "t1"
    ? set from=bummer@m2.com
    ? m t2
    Subject: s2
    Sendmail arguments: "sendmail" "-i" "-r" "fem@me" "t2"

  ...

    s-nail -A test -Snoeditalong -r '' -d
    ? set from=bummer@m1.com
    ? m t1
    Subject: s1
    .
    Sendmail arguments: "sendmail" "-i" "-r" "bummer@m1.com" "t1"
    ? set from=bummer@m2.com
    ? m t2
    Subject: s2
    Sendmail arguments: "sendmail" "-i" "-r" "bummer@m2.com" "t2"

  [*main-fun-cleanup* topic branch]

- *smime-sign-include-certs-** stuff works again, oops..  [9a8597c6]

- A whole lot of smallest and small fixes due to registration at
  Coverity Scan, as project 444.  Error handling in S-nail is ridiculous.
  [*coverity-444* and *coverity-444.2* topic branches.  The sheer number
  of fixes was the reason to sit down and go for unplanned S-nail v14.3]

- *idna-strict-checks* has been dropped.  It's silly to have in a MUA,
  especially given that GNU LibIDN doesn't ship with a lot of rules.
  We were able to drop quite some code (and a use-after-free, too :().
  [c81fd41d]

- The ~p tilde command displays attachments more verbose.
  Until the big big MIME and send layer rewrite :) this is intermediate
  since until then we do not really know neither MIME type nor charset
  of an attachment at the time this is displayed (for sure).  Yet
  i think it's nicer to show what we have than keep it the way it was.
  [60e70463]

- @ and @& shell command prefixes have been added for the pipe-MIMETYPE
  mechanism.  The former suppresses filters if multiple messages are
  displayed at once, the latter adds asynchronous program execution on
  top of that.  E.g., to display PDF documents, but only if you
  *explicitly* address the message *alone and by itself*, and without
  blocking S-nail and the $PAGER, do:

    set pipe-application/pdf="@&cat >"${TMPDIR}"/s-nail${$}.pdf;\
      mupdf "${TMPDIR}"/s-nail${$}.pdf; rm "${TMPDIR}"/s-nail${$}.pdf"

  (Inspired by Gavin Troy.)  [a8d724b3]

  Note: most of that had been posted to nail-devel@ already, but it was
  tweaked ([251b636]) so that you now *really* have to say `p MSGNO' to
  get there.

- The NETLESS CONFIG= has been removed; it is almost identical to
  MINIMAL now (i.e., without WANT_JUNK and WANT_SCORE).

- WANT_JUNK and WANT_SCORE have been disabled by default.
  They don't seem to be too useful; i hope i can implement
  a SpamAssassin hook for (downloaded) mail messages for v14.4.
  If so, expect these two "modules" to become removed completely.

v14.2.2, 2013-05-01
-------------------

Another unplanned (minor) bugfix release after Gavin Troy (gavtroy AT
gmail DOT com) pointed out that MIME CTE decoding was broken, who
i therefore want to give a lot of prominence right here.

- Fix MIME content decoding which has been broken by [01c0e135].
  [882caedd]
  (Gavin Troy.)

v14.2.1, 2013-04-30
-------------------

An unplanned (minor) bugfix release after i've found two bugs today and
heard from Jérémie Courrèges-Anglas (jca+nail AT wxcvbn DOT org) that
there exists a S-nail OpenBSD package.

I want to give prominence to the following people that helped to
improve S-nail(1) during this development cycle, in order of
appearance: Dirk Peters (peters AT schwertfisch DOT de).

Thank you very much, and best from Germany!

- Some warnings of newer clang(1) versions were silenced, including yet
  another alloca(3) problem (see *memtracer* topic branch in v14.0
  series).  [6f846efe]

- Tweaking the MIME boundary detection left a little hole that could
  cause boundaries not to be detected, as has been shown by a Microsoft
  Word generated mail on the ICU list.  [11e5fb5b]

- A format string could overflow bounds if unrealistic
  (18446744073709551615) line numbers or message sizes would have been
  produced.  [faa65c40]

- An algorithmic error could cause overlong lines which wrapped around
  to the next display line.  [ade52660] 
  (Dirk Peters.)

v14.2, 2013-03-15 [v14.1, 2013-03-12]
-------------------------------------

I want to give prominence to the following people that helped to
improve S-nail(1) during this development cycle, in order of
appearance: Martin Neitzel, Christos Zoulas, Stephen Isard, jgw@txo.org
and Gavin Troy.

Thank you very much, and best from Germany!

+ v14.2 differs from v14.1 only by one commit, one that fixes
  (mime_fromhdr(): partial rewrite using n_iconv_str(), 2013-03-12),
  which i hastily implemented just hours before the release of v14.1,
  and simply shouldn't have made it (into there).
  The v14.1 tarball has been removed from the server.

- A fix for the quoted-printable codec: "message truncation" occurred
  when a mail maliciously used a soft linebreak to escape the linebreak
  of a completely empty line.
  (That resulted in 0 written and 0 leftover bytes, a condition that was
  declared erroneous back in november 2012 when i started handling I/O
  errors.)

- The "folders" command will work again when given an argument.
  A fault of mine introduced in (cmd1.c: expand() may fail, 2012-10-23).

- The Base64 codec has been touched again, and we are finally capable to
  perform sequential decoding; this was targeted for the MIME/send layer
  rewrite, but it actually was possible today.
  The result as seen in ps(1), running on the Base64 encoded HTML5
  standard (4622545 bytes HTML, with a NUL appended to force Base64
  encoding, resulting in a 6244793 bytes email):

   7420 s006  S+    2:22pm 0:10.65 plain-nail -f HTML5
   1440 s006  S+    2:23pm 0:00.36 ./s-nail -f HTML5

  (So the only thing that is left for a good throughput is sequential
  decoding of quoted-printable encoded parts that maliciously use soft
  linebreaks to convert an entire part to a single line.  And i've seen
  that from Apple Mail.)

- New option: *mime-allow-text-controls* (rather long manual entry).

- *smtp-auth-password-user@host* and *smtp-auth-user-user@host* will
  finally work!
  (Reported by jgw@txo.org in November 2011, fixed by Gavin Troy in
  January 2012.)

- Most *headline* formats now do support the '-' left-alignment flag.
  Note that you most likely have to change your *headline* accordingly.
  (The still missing %n format is one reason why there will be v14.2.)

- *datefield* and *datefield-markout-older* can now be set to
  strftime(3) format strings (except %n).
  (From Stephen Isard's wishlist.)

- A possible SEGV has been found and also fixed by Stephen Isard.
  (The "legendary" cross-world stereo fix!)

- Wow!  S-nail will finally compile on GNU based Linux systems like
  Slackware 14 etc.  (Found while hunting bug reported by
  Stephen Isard.)

- New option: *datefield-markout-older* can be used to choose
  a different date display for mails that are older than six months,
  in equal spirit to what POSIX describes for the -l option of the ls(1)
  command (Stephen Isard).

- (Exotic) Years are (would) now (be) interpreted correctly according to
  RFC 5322, 4.3.

- CRAM-MD5 usage has been fixed.

- *folder* updates are now tracked when set, and we will show the
  realpath(3) name of it, showing PREFIX..SUFFIX if that wouldn't fit on
  the display.
  Tracking updates made it also possible to perform other more expensive
  tasks when setting *folder*, so that it is now possible to do
  something like:

    :set folder=$HOME
    :set folder=~

  et cetera (both ideas by Christos Zoulas).

- Bugfix for the ~@ tilde-escape in non-interactive mode.
  (readtty(): quick shot: work in pipelines (on non-TTY).., 2013-01-25)
  introduced the possibility to "read data from the terminal" (STDIN
  that is) in non-interactive mode.
  The manual documents that attachment input must be terminated with an
  empty line, but if that had been omitted, as in the example below, we
  would have yet entered an endless loop.

     $ cat <<_EOT | /s-nail -~ -s boom ./OUT
    ~@
    test.c
    charset
    _EOT

- Alias expansion will now be performed for members of Reply-To: fields
  (Martin Neitzel).

- Decoding quoted-printable will now be more relaxed.
  (Even though the standard says that users should be given a hint when
  input is not absolutely clean; a possible warning will be added later,
  when we have an error message ring.)

- New option: *mimetypes-load-control* can be used to control which of
  the mime.types resources will be loaded.

- The builtin default mime.types have been extended a bit.

v14.0, 2013-02-10
-----------------

I want to give prominence to the following people that helped to
improve S-nail(1) during this development cycle, in order of
appearance: John Dodson, Gianluca Ramunno, and Anon Ymous from the
NetBSD project.

Random notes
^^^^^^^^^^^^

- Encoding defaults to *quoted-printable* not *8bit*.
  This has no technical background except that i think it's the better
  default.

- Small progress for the "getting stuck due to the current folder
  becomes inaccessible due to whatever reasons"  problem.
  (schdir(): realpath() local files before leaving CWD.., 2013-01-08)

- The names of temporary files have changed.  Whereas not all uses of
  temporary files already use really meaningful names, it has yet become
  possible to use the pattern "*mail-*"; or, to be compatible with
  NetBSD Mail(1) in one go, "*mail*".  (E.g., in my ~/.vimrc you'd read:
    :au   BufRead,BufNewFile *mutt*,*mail* setl fenc= | setf mail
  [the *mutt* is a leftover from times when i've used MUAs that suck].)

ChangeLog
^^^^^^^^^

- The Quoted-Printable MIME handling has been rewritten completely.
  We now correctly encode files with the MS-DOS newline sequence (CRLF).
  (Part of the *mime-cte* topic branch.)

  S-nail(1) continues to be able to handle text messages and text
  attachments without a trailing newline, but because these
  Content-Transfer-Encoding related things are now handled by the C-T-E
  layer instead of by sendout.c a text message body that comes in as
  part of a complete message via the -t command line option will loose
  the missing final newline (i.e., it'll gain one).
  This problem does *not* occur when *only* the message body comes in
  via STDIN, as in 'cat FILE | s-nail', but *only* when the -t option is
  used.
  (sendout.c: does no(t/ longer) know about CTE internals!, 2013-02-09)

- Filename arguments for -a are now processed *after* all the resource
  files have been loaded etc., so that the usual "folder" specifics can
  be used (provided that proper care for shell quoting was taken).
  ((main(): delay -a processing.., 2013-01-10), as a part of the
  *mainaflags* topic branch.)

- (d38c5bd, When the write command asks.., 2004-11-23) added support
  for pipes when saving attachments during a "write" command.
  It however used the wrong SIGPIPE signal handler; e.g.:

    Enter filename for part 2 (application/x-gzip): |exit
    Segmentation fault

  Of course, it still performs a jump and that most likely leaves memory
  chunks behind, thus causing some memory leaks.  This will be
  a long-term problem (you may want to read [mime.c:fwrite_td(): TODO
  notes on unfixable leaks, 2013-01-14] for more).
  (send.c:sendpart(): fix longjmp() SIGSEGV.., 2013-01-29)

- Fixed a name quoting regression that i've introduced in
  (Rewrite *extract().., 2012-10-20), that would have caused
  "x \"y\" z" to become "x"y" z" instead of "x "y" z".
  (S-nail still does not really have RFC compliant parsers, just as
  NetBSD Mail(1) has, i.e., there are structured and unstructured fields
  etc...  I hope i can provide them in v15.0.)
  (names.c:yankname(): fix quote regression.., 2013-01-29)

- The IDNA conversion now assumes domain names are specified in
  *ttycharset*, rather than in the LC_CTYPE locale charset.
  I.e., it integrates into the usual character set specifications.
  (IDNA: honour *ttycharset* for domain names, 2013-01-18)

- The new *editalong* variable will automatically spawn an editor when
  composing a mail in interactive mode, just as if `~e' was given.
  (Add new *editalong* variable, 2012-01-07)

- The manual has been converted to mdoc.
  (The manual has been converted to mdoc, 2012-12-28)

- The ~@ tilde escape, when given filename arguments, will treat the
  arguments as a comma-separated instead of a whitespace-separated list.
  (collect: change separator of ~@ tilde escape.., 2012-12-28)

  The interactive mode of ~@ has also been changed, rather massively.
  Please do reread what the manual says.
  ((collect: support multiple attachment charsets.., 2013-01-23), as
  part of the *attach* topic branch.)

- Thanks to Gianluca Ramunno (ramunno DOT gianluca AT gmail DOT com)
  S-Nail will no longer try to issue a STARTTLS command when it is about
  to establish a SMTPS connection, a task that logically fails since the
  connection is already secured.
  (Interestingly the nail codebase performs the necessary test for IMAP
  and POP3 already.)
  While here the undocumented nail v11.0 *smtp-use-tls* legacy option
  has been removed.
  (Fix SMPTS with a set *smtp-use-starttls*.., 2012-12-22)

- The RFC 4155 compliant MBOX quoting is now exclusively used, the
  shitty *posix-mbox* variable has been removed again.
  (Shitty because i've implemented RFC 4155 compliant MBOX quoting and
  tested it, then added *posix-mbox* for those who liked the old
  behaviour and did not re-test -- the final code path was buggy.)

  In mails newly created and saved by S-nail(1) no From_ quoting at all
  will be used no more, but instead the rewritten MIME file classifier
  will detect unquoted From_ lines and enforce quoted-printable encoding.
  (This is an approach that is S/MIME compatible all through the way as
  the file data is not modified at all, but only encoded, so that the
  data checksum is not changed.)

  In yet existent mails that S-nail copies or moves around without
  reclassification an RFC 4155 compliant From_ line detector will apply
  MBOXO quoting (prepend a single '>') as necessary.
  Different to the old MBOXRD behaviour S-nail will neither quote yet
  quoted From_ lines ('>>From xy' -> '>>>From xy') nor will it unquote
  one quote level when reading etc. mails ('>> From xy' -> '>From xy').
  As a result the code could be simplified.

  This changeset also incorporates a fix for NetBSD PR bin/47453, as
  reported by Martin Brandenburg.  I.e., some mailers, noticeably
  UW-imap (with MBX format only?), use non-compliant From_ lines with
  RFC 822 date specifications.  Be aware of 'em.
  (RFC 4155 MBOX, and drop *posix-mbox* and foldergets().., 2013-01-06)

- *rfc822-show-all* has been removed.
  It didn't work properly for more complex MIME structures, like
  message/rfc822 messages with attachments etc., just as i've seen today
  on the file(1) mailing list.
  So, instead of hacking it now i've dropped it and will come back with
  a better solution when the MIME and send layers have been overhauled.
  I.e., the real intent was to be able to specify that an embedded
  message/rfc822 is treated as a *unity*, and that's the goal.
  (Drop *rfc822-show-all*, 2013-01-23)
  [The manual will be adjusted in a different commit.]

- *rfc822-no-body-from_* has been renamed to *rfc822-body-from_*.
  It thus must be set explicitly.
  On the other hand it now catches all cases...
  (*rfc822-no-body-from_* -> *rfc822-body-from_*, 2013-01-23)
  [The manual will be adjusted in a different commit.]

- The new *charset-7bit* (defaults to US-ASCII) and *charset-8bit*
  (defaults to UTF-8) have been introduced.
  These are used if seven bit clean data is to be sent, and no
  *sendcharsets* are set or the convertion of all of them failed,
  respectively:

  - There is no functional change unless there is iconv(3) support.
  - There is no functional change unless you set them.
  (Introduce *charset-8bit* and *charset-7bit* variables.., 2013-01-18)
  [The manual will be adjusted in a different commit.]

  In addition the new *sendcharsets-else-ttycharset* variable can now
  be used to automatically use *ttycharset* as a *sendcharset(s)*,
  regardless of the new *charset-8bit* variable.
  (Add *sendcharsets-else-ttycharset* variable, 2013-01-24)
  [The manual will be adjusted in a different commit.]

  NOTE: before we apply charset conversion we now perform a string
  comparison to see wether character sets are identical.  If the strings
  match (case-insensitively), then *no* conversion is performed.
  This means that code like

    $ printf "LATIN1: \0376" | s-nail -Ssendcharsets= -s boom ./out.txt

  *succeeds* in an UTF-8 environment now, whereas older versions would
  fail with an "illegal byte sequence" error (unless the iconv(3)
  library of the system would not perform any conversion that seems
  superflous, of course).

  I thought about making this optional, but, in fact, if this would be
  done in an environment without iconv(3) support then the result would
  be equally corrupt.  And the way it is now we save the expensive and
  superflous conversions.  (See TODO for more.)

  Please *do* reread the manual section "Character sets".
  +++++++++++++++++++++++++++++++++++++++++++++++++++++++

- An attempt was made to improve MIME Content-XY: detection.
  It should be more RFC compliant, and just overall better :);
  a simple one-pass classifier cannot match libmagic(3), of course.

  This changeset removes support of the long obsoleted (4fee1ef,
  2005-01-06) *charset* variable, as well as for the
  *maximum-unencoded-line-length* variable that has been introduced in
  (48a652bd, 2005-07-26).

  For S-nail v14.0 it'll be no longer necessary to do

    $ tr -d '\015' < input | s-nail ...

  to transport files which use the $-DOS (terminal) newline (sequence).
  We still depend upon *sendcharsets* for a while, though.
  (Rewrite file-content classification.., 2013-01-02)

- The new *mime-counter-evidence* variable can be used to force
  a classification of non-text MIME parts (attachments) by their
  filename, i.e., a "reverse-classification" just as would be performed
  if S-nail(1) would itself *send* the file(name).  This can help
  against some stupid MUAs (Apple Mail?) that send .diff etc. files as
  `application/octet-stream' parts etc.

  At a later time this may become a valued option, causing a temporary
  save of unnamed attachments followed by a MIME classification of the
  file contents, followed by forced treatment as plain text if it seems
  to be human readable.  But not yet.  Just to warn you.
  (Add *mime-counter-evidence* variable.., 2012-12-29)

- The special "pipe-" command "@" can be used to force treatment of
  a MIME message part as plain text (e.g.,
  'set pipe-application/pgp-signature=@' will henceforth print those
  signatures inline and as plain text).
  (Introduce the special "@" "pipe-" command, 2012-12-27)

- The MIME types (as from mime.types(5)) will now be cached.  Before
  all possible sources would have been opened, read and parsed for each
  and every message part that required detection of the MIME
  Content-Type:.
  The new "mimetypes" command can be used to show or clear that cache.
  (Add mime.types(5) cache.., 2012-12-27)

- A small set of MIME types (template: ./mime.types) will now become
  compiled into S-nail(1), and be used as a fallback if there are no
  ~/.mime.types and/or no /etc/mime.types, or those didn't contain
  a matching type.  E.g., NetBSD 6 doesn't ship a default database.
  Also, file extensions will be matched case-insensitively (case of
  attribute values is not specified afaik?).
  (Introduce compiled-in mime.types(5).., 2012-12-21)

- The undocumented *charset7* variable was removed.

- The "Message X:" display leader has been changed and will henceforth
  be matchable via "^[-- Message \d+ -- \d+ lines, \d+ bytes --]:$".
  These lines can no longer be suppressed by setting the *quiet*
  option (which was yet possible for print and top, though undocumented).
  (Change "Message X:" display message.., 2012-12-20)

- When displaying multipart messages the "Part X:" introductional string
  has been changed; if the Content-type: header is not *retain*ed, then
  this string will include the part's type and size.  Ditto for
  Content-disposition: and a mentioned attachment filename.
  This is a first step only, for the final version the MIME and send
  layers will have to be adjusted.  But the string will be matchable via
  a "^[-- #.* --]$" regular expression from now on.
  (Change "Part X:" display message.., 2012-12-20)

- MIME boundaries in multipart messages are now handled better in that
  no boundary string should get through to the display.
 (Tweak MIME boundary detection.., 2012-12-20)

- A couple of long standing, even pre-Heirloom mailx(1) memory leaks and
  segmentation violations, most of them related to configurations
  without alloca(3) support, as well as one leak that i have introduced
  when i implemented RFC 4155 MBOX handling, have been fixed.
  But S-nail should now survive non-alloca(3) configurations.
  (*memtracer* topic branch.)

- The Base64 MIME handling has been rewritten completely.
  This was an urgent topic, because the old implementation (a) read in
  all lines of a base64 encoded text part, repeatedly resizing a string
  storage and repeatedly decoding that string until all the lines have
  been swallowed (i.e., or by accident the last decoded byte was
  a newline character, and that in turn may of course have fucked up for
  multi-octet encodings, dependent on the actual byte-order), (b) used
  function local static data to keep state in between multiple
  invocations, which messed up multi-byte/-octet encodings like this

    �5��ɽ���Aɥ����́��٥��������[lots of data follows]

  and (c) did not perform any error checking at all.
  The new one does not run into the problem that (a) tried to circumvent
  since leftover decoded data (as opposed to leftover *encoded* data) is
  transported along the call-chain for later use.  It adds a minimal set
  of error handling ('may now see "[Invalid Base64 encoding ignored]"),
  with more to become possible in later S-nail versions when the entire
  layers are reworked.  (Until then DOS newline sequences [CRLF]
  embedded into base64 will no longer be decoded to Unix LF newlines.)
  Base64 encoded lines will now be 76 characters long, as stated (as
  a maximum value) in RFC 2045, not 72 as before.

  The Base64 code core has been shamelessly stolen from NetBSD's
  Mail(1), and i guess it was the second time that this happened :=).
  (*base64-rewrite* topic branch; and reworked later on the *mime-cte*
  topic branch.)

- The string allocation strategy has been tweaked some more to, i think,
  a final version (regarding algorithm).
  In normal non-interactive send mode it should now no longer need any
  dynamic memory at all (unless some dozen recipients are specified).
  (Several other places still use normal dynamic memory, of course.)
  (*dope-stringdope-again* topic branch.)

- Support for NSS (Network Security Services) has been removed.
  I've never worked with it and are, regarding the complexity of
  network security, not willing to spend any time on it.
  SSL is installed on all systems i'm using and/or testing on by
  default, and so i've choosen to go this way.
  (It may be that sometime in the future S-nail will add support for
  libcurl(3) connectivity, and then it may happen that not only NSS
  support is reintroduced again, but also GNU TLS.  All of that applies
  to network connectivity only, however, not to S/MIME afaik.)
  (*drop-nss-support* topic branch.)

- I finally got a glue and understood that Sourceforge does (a) not
  support symbolic links and (b) doesn't like dots as regular parts of
  filenames.  This means that all this time the promised s-nail.tar.gz
  symbolic link did not work, and that s-nailv13.3.tar.gz wasn't
  accessible either (via the web interface).
  In the future S-nail(1) will not provide any more symbolic links (the
  Sourceforge website offers a "download latest" thing which seems to
  work), and use underscores in filenames -- s-nailv13.3.tar.gz has been
  renamed to s-nailv13_3.tar.gz.  The tags continue to use dot notation.
  Thanks.

- The *heirloom-plus* support branch has been removed.
  The code bases diverged a lot and even more to come.
  It doesn't make sense to put any effort in that.

v13.3, 2012-11-11
-----------------

- Configuration on UnixWare 7.1.4 will succeed (shell issue fixed).

- Even on DragonFly BSD the IMAP GSSAPI is now found
  (in /usr/pkg/include/krb5/gssapi/gssapi.h).

- Support for pkgsrc(7) systems and automatic integration of
  C_INCLUDE_PATH and LD_LIBRARY_PATH path configurations.
  Please see INSTALL.

- Fixes a mortally embarassing regression that the current maintainer
  introduced before i really knew what i was doing, in (If *record* is
  set, avoid writing dead content twice.., 2012-09-14).
  It hit users that send through a MTA and have *record* set; in this
  combination data would not have reached the MTA.
  Interested parties may read the comment in savedeadletter() (part of
  the changeset) or the (Fix MTA/*record* descriptor clash..,
  2012-11-10) commit log.

  Deepest apologies to tortured users from the current maintainer!

- RFC 4155 compatible MBOX file handling has been introduced, and so
  S-nail is now on par with (at least) NetBSD Mail in respect to this.
  It can be turned off with the new *posix-mbox* variable, which you may
  need to use since not all MUAs are capable to dig those MBOX files.
  E.g., less cutting-edge (.-) MUAs fail for this:

   |From - Thu May 10 20:40:54 2012
   |Date: Wed, 07 Nov 2012 11:48:30 +0100
   |To: super@duper.com
   |Subject: super1
   |
   |From me.
   |
   |From - Thu May 10 20:40:54 2012
   |Date: Wed, 07 Nov 2012 11:48:30 +0100
   |To: super@duper.com
   |Subject: super2
   |
   |>From - Thu May 10 20:40:54 2012
   |From - Thu May 10 20:40:54 2012

  I'm not completely happy since S-nail *does* still quote those lines,
  how rare they may be -- it *does* modify message content.  mutt(1)
  implements something more clever and that is quoted-printable encoding
  of the "F" from "From", when seen at the beginning of a line.
  This, when applied to just *any* "^From", will be a non-modifying and
  all-compatible solution.

- SEND_MBOX handling has been changed to discard any Content-Length: and
  Lines: headers when it rewrites a message by default.
  I know that mutt(1) generates them (why, after
  http://www.jwz.org/doc/content-length.html?).
  Anyway, S-nail does neither use nor manage them, so that any
  modification renders those fields invalid, and then it seems best to
  discard them anyway.
  You may turn the new behaviour off with *keep-content-length*.

v13.2, 2012-11-03
-----------------

The problem was that i really wanted to release on a 25th.
But i have been able to improve S-nail(1) some more, so that this is
possibly the first real release of it.  So i'll add only things that
have changed since v13 -- please see below for the complete picture.

- A new CONFIG=CUSTOM make directive was added, and the new user.conf
  variables WANT_SCORE and WANT_DOCSTRINGS have been added.  It is
  possible to create a floating-point free S-nail(1) now.
  Please see INSTALL.

- If a feature is disabled not even functions stubs should remain now.

- Many places which will work only with local filenames do now actually
  check that the target is a local filename.

- Space-separated lists should work again, at a few places at least.

- Tilde commands will be possible, even with -r.

- The *sendmail-progname* has been added after NetBSD 6 dropped the
  send-mail entry in mailer.conf(5), which broke S-nail(1).  Now users
  have an option.

- When editing messages via ~e or ~v file and pipe addressees will no
  longer be lost.

- If recipients occur multiple times spread over lists, i.e., To:, Cc:,
  Bcc:, then only one occurrence remains, and in the "highest-order"
  list.

- The "list" command prints the list alphabetically sorted (somewhat).

- The "help" and "?" commands take an optional argument that shows
  a synopsis string for the given command (unless WANT_DOCSTRINGS was
  false).

- String allocations are now more efficient.  The situation can still be
  improved.  However, for the first time Berkeley Mail(1) integrates
  harmonically into the system allocator, which may madvise(2) unused
  memory to the operating system as necessary and/or possible!

v13.1, 2012-10-25
-----------------

Well, a version number 13 is anyway an ugly thing...

  commit 4f534bb33b7c911272cc66a0e3a9e47b73ad8deb
  Date:   2012-10-25 20:46:07 +0200

      FIX MIME quoted-printable encoding (char cast)..
      
      (;-{
      Auauauauau!!
      
      Well, one of the things that have already been started in v13 is
      the turn from using "int" when working with 8-bit characters to
      "unsigned char" (and as long as we do not support wide
      characters).
      
      Unfortunately one very important piece of code, that is handling
      encoding to quoted-printable, still used integer instead of
      unsigned char, which caused an automatic extension cast to take
      place, and that resulted in a messed up output.
      Sorry!

v13, 2012-10-25
---------------

I want to give prominence to the following people that helped to improve
S-nail(1) during this development cycle, in order of appearance: Martin
Neitzel, Ezequiel Garzón, Björn Persson, Paul Vojta, and, especially,
John Dodson for warm words from beautiful Australia!  Many thanks also
to Christos Zoulas.

After i've officially forked nail(1) aka Heirloom mailx(1) as S-nail(1)
on 2012-09-18 i have been able to work five weeks almost fulltime on
S-nail(1) development.  The first three weeks can be characterized as
hectic fireworks here and there, but then it got better and i was able
to work more or less topic-centric.  In the meanwhile S-nail(1) is more
than 230 commits away from the Heirloom base--and drifting further apart.

S-nail(1) v13 is the first release of S-nail(1), but it was forked from
Heirloom mailx(1) 12.5 7/5/10 that arose from Berkeley Mail 8. unless
i'm mistaken.  What characterizes S-nail(1) v13?

- The build system has been reworked almost completely.
  It is possible to fine-tune which features should be present in the
  binary and which don't.  The name of the binary can be chosen, and
  that choice is reflected all through the manual and the template
  resource file.  The manual is always complete and thus may document
  features that are not supported by the actual binary, though.
  Please see INSTALL for more.

- Compiler warnings can now be used.  Please see the example WARN= flags
  in the Makefile, but '-Wall -Wextra -pedantic' should be silent though
  certainly insufficient to reflect the complex work of modern compilers.

- The following recipient address list combines some of the major
  improvements that have been made:

    <addr1@cdröm.de>  (bier) , ./file1,
    Steffen Smöregäs (Humbabä) <sauer@bäüer.de> (Hummpäa)    ,
         sabberlot@träbbel.de  ,  (bier2) <a2@bür2.de> ,
       a3@b3.de (bier3)  , <a4@b4.de> (bier4, und \"bier5\")  ,
    |cat > pipe1 ,   (bier 6) <a6@bür6.de>  ,   ./file2        ,
    (co\$mm1) abc1@düf.de (cö,bmm,2)   (co\"m\"m.3) ,
    co\$bmm1 \"c,ömm2\" co\"m\"m.3 <abc2@däf2.de>  ,  |cat > pipe2 ,
    moppel@höppel.org

  That, on a single line, may be given to ":m" or (quoted) on the
  command line, or to "~c" or whatever, and it will work as expected
  (well, everything else would be a bug..) and result in the following
  sendmail(1) invocation:

  SENDMAIL.SH
  <-i a2@xn--br2-hoa.de a3@b3.de a4@b4.de a6@xn--br6-hoa.de abc1@xn--df-xka.de abc2@xn--df2-qla.de addr1@xn--cdrm-7qa.de moppel@xn--hppel-jua.org sabberlot@xn--trbbel-cua.de sauer@xn--ber-qla4j.de>
  >>>>>>>>>>>>>>>>>
  Date: Thu, 25 Oct 2012 17:12:15 +0200
  To:
  Cc: Steffen =?utf-8?Q?Sm=C3=B6reg=C3=A4s?=
    =?utf-8?Q?_(Humbab=C3=A4)?= <sauer@xn--ber-qla4j.de> (=?utf-8?Q?Hummp=C3=A4a?=),
    sabberlot@xn--trbbel-cua.de, moppel@xn--hppel-jua.org,
    <addr1@xn--cdrm-7qa.de> (bier),
    co$bmm1 =?utf-8?Q?"c,=C3=B6mm2"?= co"m"m.3 <abc2@xn--df2-qla.de>,
    (co$mm1) abc1@xn--df-xka.de (=?utf-8?Q?c=C3=B6,bmm,2?=) (co"m"m.3),
    (bier 6) <a6@xn--br6-hoa.de>, <a4@b4.de> (bier4, und "bier5"),
    a3@b3.de (bier3), (bier2) <a2@xn--br2-hoa.de>
  Subject: Re: SubjectTest
  MIME-Version: 1.0
  Content-Type: text/plain; charset=us-ascii
  Content-Transfer-Encoding: 7bit

  body
  <<<<<<<<<<<<<<<<<

  So list parsing has been fixed, IDNA support has been added, and it is
  possible to mix pipe and file recipients *and* multiple thereof, and
  the result is still correct for *all* of them.
  I think this kind of list can be given wherever a user can directly
  enter such a list.  And i think all that is unique to S-nail(1).

- When writing back edited messages the target MBOX mailbox can no
  longer become "corrupted" when the trailing newline was removed during
  the edit.  Also affected FreeBSD and NetBSD mail(1).

- A security fix for CVE-2011-2895 was applied.

- The generated Message-Id: is now more human-friendly.

- The -h command line option has been dropped.  Use "-O -h XY" if your
  MTA really supports that.

- The -O and -r command line options no longer enforce a one-shot send
  mode, and instead persist for the duration of the entire session.

- Variables set via the -S command line option are now (un)set twice;
  immediately and after all the resource files have been loaded.

- Other new or changed options/commands, in order of appearance:
  recipients-in-cc, smime-sign-include-certs, quote-fold, stealthmua,
  add-file-recipients, write, rfc822-no-body-from_, rfc822-show-all,
  mail/Mail, idna-disable, idna-strict-checks, ??

- In the codebase itself an effort to reduce duplicate work and
  introduce caching, and to minimize the use of local variables, was
  started, but that is long term.  A lot of improvements here and there,
  too, like using the well MD5 optimization from Wei Dai, Chris Torek's
  hash algorithm for hash tables etc.

- Incredibly important: an heraldic animal was found: snailmail.jpg!

The full history can be inspected by issuing the git(1) command

  $ git log --reverse s-nail..s-nailv13

A new TODO has been introduced, and it is getting longer and longer.
Thanks.