filter.c:_hf_store(): CID 1377036

[s-mailx.git] / TODO

TODO reminder.

Rename S-nail to S-mailx in v15.0, change things i've messed with
a single, massively backward incompatible change.

Release S-mailx v20 on 2020-03-25, the 42nd anniversary of BSD Mail.
With a clean, conforming and efficient codebase, then.

In general the code is in a pretty bad shape due to the signal handling.
I should have sat back in 2012/13 and consider what i am doing.
My fault.  If i would, we would have a blocked signal mask anywhere in
this software except in a few cases where it is necessary and/or
possible to deal with signals, and possibly we would not even have to
consider to switch the entire codebase to (the much superior, and the
only sane approach) SysV signal handling, without SA_RESTART.

But some things are already pretty good, except for normal iterations
and a review once we have a better signal handling, and can be taken
with us.  For example termcap.c, most of tty.c, shexp.c, memory.c, for
example.

- If a resource file can't be found that has been explicitly set via
  environment variables there should be some feedback.

- We should possibly get away of using command line utilities for
  compression.  (At least optionally?)  Instead we should link against
  zlib(3), bz2lib(3) and lzma(3), if found.  Or we may use dlopen(3)
  instead, if found, to avoid linking (though those libraries don't need
  much linker work unless actually used afaik, 'should look in detail).
  We should also drop lzw.c, it is used for the IMAP cache.

- We should maybe turn -~ into the meaning "force interactive".
  We should extend cc-test.sh, then, to test some interactive things.
  E.g., via (tcl(1) or, better.., perl(1) (CPAN)) expect.

- We need a "void" box that can be jumped to, i.e., a state in which no box
  at all is active.

-- When a MBOX mailbox is removed while it is opened then changing the
  folder is not possible.  This is an inherent problem of the Berkeley
  Mail codebase, and we need to have a fully functional intermediate
  VOID box mechanism plus an object-based mailbox implementation to
  overcome it.

-- Also, when the folder was modified concurrently we should bail, or,
   in an interactive session, prompt the user what to do.

- IDNA decoding.  Needs a complete design change.
  (Unless wants to brute force decode anything before display, of course.)

- If pipes fail for part viewers then at least the usual PART X.Y should be
  shown, maybe even including some error message.
  I had 'set pipe-text/html="lynx -dump -force_html /dev/stdin"' but NetBSD
  does not have lynx(1), and i thought i've found a S-nail(1) bug.

-- Also, when we run a pipe handler asynchronously there should possible
   written something like [pipe-handler xy started] or something

- It would be nice if it would be possible to define a format string for
  *quote*, like 'set quote="format=some formats"'.
  In general the current approach is somewhat messy IMO.  I.e., it would
  make more sense to act rather like mutt(1) and as written elsewhere in
  this document, i.e., have some toggles that act on the display and use it
  for multiple modes (show/reply/forward etc.)
  Otherwise introduce commands which include all the headers plus, e.g.,
  "hreply" or "freply", and then the ditto series, i.e., "hReply" ...

-- This would also mean that interactive message editing would work
  accordingly.  PleasE!

- Line editing should gain possibility of context sensitive tab completion.

- Maybe there should be an additional ZOMBIE directive that is served in
  equal spirit to DEAD, but that could be a valid MBOX... ?
  What i want is a *real* resend, best if possible from command line.
  Meaning, also the possibility to postpone a message.  In general.

- Having a newsreader would be a really cool thing.  (RFC 977 and 2980)

- There should be a variable that controls whether leading and trailing
  empty lines of parts and/or messages as such should be printed or not.

- printhead()/hprf(): support %n newline format (%t tab?).
  Make it possible to use the *datefield* algorithm for plain From_ derived
  dates (needs a From_ parser, i.e., strptime()-alike).
  Once we have that, rename *datefield-markout-older* to
  *date-markout-older* ??
  Note that NetBSD's mail(1) has some other nice things.
  Note also that our code is quite unflexible.

-- NetBSD's mail(1) has nice *indentprefix* and *indentpostscript*
   variables (though prefix and appendix or prefix and suffix, but..).
   Note that our code is quite unflexible.

Low-Level
---------

- Improve name extraction rules.  And field parsing.  There
  are structured and unstructured fields.  There are quoted pairs and
  comments etc.  Rewrite the entire parsing mechanism to comply to RFC
  5322, and try to merge all those many subparsers around in the codebase,
  and accordingly.  So much duplicated work ...
  Name parsing has been improved a bit for v13, but it's still broken.
  yankword(), *extract(), etc.: RFC 5322 says that comments in address
  fields SHOULD NOT be used (mutt(1) maps them to full name-addr forms if
  approbiate, even if that actually changes content!!?), and that full
  name-addr SHOULD be used.  Our functions are yet quite silly (i.e.,
  leading comments remain, as in "(bier2) <a2@b2.de>", unless the address
  doesn't come in angle brackets, trailing go away, as in "<a6@b6.de>
  (bier6)", that becomes "<a6@b6.de>").
  Something silly like
    (co$mm1) abc@däf.de (cö,mm,2)   ('c'o"m"m.3)
  Should eventually become
    co$mm1 cö,mm,2 'c'o"m"m.3 <abc@xn--df-via.de>
  on the display, or, with IDNA decoding (and thus rather unlikely)
    co$mm1 cö,mm,2 'c'o"m"m.3 <abc@däf.de>
  It should NOT become this mutt(1)ism:
    "co$mm1 cö,mm,2 'c'omm.3" <abc@däf.de>
  Or?

++  NOTE: 'alternates' tracking happens BEFORE we enter composing, this
    means that an account switch during message composing will NOT cause
    reevaluation of all that very very clumy
    elide/delete_alternates/gexpand/is_myname etc. handling.

- many uses of whitechar() should be spacechar(), and spacechar() should
  be blankspacechar(), and blankspacechar() should be dropped.  then
  drop onlywhitechar() again, too.  maybe reduce char_class bits then.

- In v15.0, when we can address attachments of a message individually,
  it would be nice to provide even more access, just like nmh(1) does
  (Johan Commelin: Are s-nail and mh related?).

- I never used anything but the *datefield* option, and it would really be
  nice if the date strings would be parsed off into some 16 byte or what
  storage when about to producing the summary, so that it would be directly
  available and there would be no need to reread the mail.  Moreover, or
  even more than that - the m_date field exists and should possibly simply
  be init, at least in these cases.  (P.S.: this doesn't contradict the
  statement somewhere else in this file that the structure should be
  slacked; simply use multiple thereof or so)

- At some later time extend the logic behind -# -- it should not have
  a current folder, but start in VOID mode (...), and unless one is
  explicitly chosen..  We need a reliable batch mode.
  P.S.: then drop again the special casing of /dev/null.

- After I/O layer rework we should optionally be able to read RSS
  (Atom?) feeds -- Expat should be available almost everywhere and
  should be able to parse that?
  Atom is harder because it may support html+.
  I mean, yeah, it's stupid, but we could fill in header fields with
  dummies and still use S-nail to look into the separated feeds as if
  they were mail messages; anyway i would like to save me from using too
  many tools -- three seems reasonable.

-- `sync'hronize commando -- robin@stjerndorff.org (Robin Stjerndorff):
    Wondering how to update back to my Maildir, moving new read mails
    in ~/Maildir from new to cur, without exiting the application.
    Automation available?  [And simply re-`[Ff]i' involves a lot of
    unnecessary work]

-- Provide sync'ing options -- Jacob Gelbman <gelbman@gmail.com>:
    If I open two instances of mailx, I then delete a message and then
    quit in one. Then in the other one I read a message and quit, mailx
    saves the status of the read message and the fact that a message was
    deleted, even though it was opened before the other instance deleted
    it. How is it doing that?  [Of course he was using Maildir]

- Add TODO notes for those RFCs:
  RFC 977 -> 3977 - Network News Transfer Protocol
  RFC 1036 - Standard for USENET Messages
  RFC 1524 - True support for mailcap files?
    TODO YES!  We really need to replace the pipe-TYPE/SUBTYPE mechanism
    with something real.  When we can work on the parsed MAIL DOM.
  RFC 1939 - Post Office Protocol v3
  RFC 2017 - URL External-Body Access-Type
  RFC 2183 - The Content-Disposition Header
  RFC 2369 - The Use of URLs as Meta-Syntax for Core Mail List Commands
             and their Transport through Message Header Fields
             (RFC 6068 - The 'mailto' URL scheme)
  RFC 2384,1738 - I.e., Much better URL support
  RFC 2387 - multipart/related  -- yet handled like /alternative
  RFC 2392 - Content-ID and Message-ID Uniform Resource Locators
  RFC 2405 - The format of MIME message bodies.
  RFC 2406 - Common multimedia types.
  RFC 2407 - Encoding of non-ASCII text in message headers.
  RFC 2449 - POP3 Extensions (including SASL)
  RFC 2595 - TLS for POP3 (among others)
  RFC 2980 - Common NNTP Extensions
  RFC 3156 - MIME Security with OpenPGP
  RFC 3207 - SMTP over TLS
  RFC 3461, 3464 -
    Simple Mail Transfer Protocol (SMTP) Service Extension for Delivery
      Status Notifications (DSNs),
    An Extensible Message Format for Delivery Status Notifications
  RFC 3676 - Updates to the text/plain MIME type and extensions for flowed
    text (format=flowed).   (Martin Neitzel)
  RFC 4422, 4505 - Simple Authentication and Security layer (SASL)
            (Tarqi Kazan)
  RFC 4880 - OpenPGP Message Format
  RFC 4954 - SMTP Authentication
  rfc5198.txt Unicode Format for Network Interchange
  RFC 5246 - Transport Layer Security (TLS)
  RFC 5321 - Simple Mail Transfer Protocol.
  RFC 5322 - The basic format of email messages.
  RFC 5598 - Internet Mail Architecture
  RFC 5751 - Secure/Multipurpose Internet Mail Extensions (S/MIME)
    TODO NOTE that our S/MIME support is extremely weak regarding
    TODO understanding, we should not rely on OpenSSL but instead
    TODO handle it ourselfs; the RFC says:
    S/MIME is used to secure MIME entities.  A MIME entity can be a sub-
       part, sub-parts of a message, or the whole message with all its sub-
       parts.  A MIME entity that is the whole message includes only the
       MIME message headers and MIME body, and does not include the RFC-822
       header.  Note that S/MIME can also be used to secure MIME entities
       used in applications other than Internet mail.  If protection of the
       RFC-822 header is required, the use of the message/rfc822 media type
       is explained later in this section.
  RFC 6125 - Representation and Verification of Domain-Based Application
    Service Identity within Internet Public Key Infrastructure Using
    X.509 (PKIX) Certificates in the Context of Transport Layer Security
    (TLS)
  RFC 6152 - SMTP Service Extension for 8-bit MIME Transport
  RFC 6409 - Message Submission for Mail
  rfc6530.txt Overview and Framework for Internationalized Email
  rfc6531.txt SMTP Extension for Internationalized Email
  rfc6532.txt Internationalized Email Headers
  rfc6854.txt Update to Internet Message Format to Allow Group Syntax in
    the "From:" and "Sender:" Header Fields
  rfc6855.txt IMAP Support for UTF-8
  rfc6856.txt Post Office Protocol Version 3 (POP3) Support for UTF-8
  rfc6857.txt Post-Delivery Message Downgrading for Internationalized
    Email Messages
  rfc6858.txt Simplified POP and IMAP Downgrading for Internationalized Email
  RFC 8058 Signaling One-Click Functionality for List Email Headers

  draft-ietf-uta-email-tls-certs-01.txt
   SMTP security via opportunistic DANE TLS draft-ietf-dane-smtp-with-dane-15
  draft-melnikov-smime-header-signing
   Considerations for protecting Email header with S/MIME
  Read https://tools.ietf.org/html/draft-ietf-uta-tls-bcp-07.
    Can we implement OCSP (see RFC 6066; -> RFC 6960)????

- This is how the codebase has to be reworked in respect to signals and
  jumping:

  1. We introduce some environment/carrier structs: struct eval_ctx,
     struct cmd_ctx, (struct send_ctx).  All of these form lists.
     eval_ctx gets a new instance every time evaluate() is entered; for
     the interactive mode, commands() instantiates an outermost eval_ctx
     that "cannot be left".

     cmd_ctx knows about the eval_ctx in which it is was created; it is
     created for each command that has an entry in cmd_tab and is passed
     as the new argument of these kind of functions.
     (send_ctx is the carrier for the MIME and send layer rewrite.)
  2. cmd_tab handling becomes more intelligent: add bit fields so that
     for some arguments (i.e., the first two or three) automatic
     normalization can be specified, like whitespace trimming, like
     automatic expansion of variables etc.  For getrawlist.

      - and: commands that work only in interactive mode, and are
        _silently_ ignored otherwise (no error).
        I.e., commands which work on only with true user interaction
        should not need to take care no more, but simply assume that
        there will be true user interaction.  And like that.

     E.g., customhdr header field is always to be WS trimmed, just the
     same as the body.  shortcut at least should have trimmed name, etc.
     To do that right, properly parse from left to right, take care
     about backslash escaping, and (optionally) expand shell variables
     and backtick substituations.

     The cmd_ctx needs to carry around the cmd_tab structure so that the
     commands can themselves print the synopsis in case of errors --
     alternatively we need a return bit which should cause the command
     callee to emit the synopsis as an error message, whatever is best.
     Some (hopefully) duplicate occurrances of such strings can vanish.
  2.1 We have some commands which offer "show" subcommands.
      Those should only work in interactive mode OR SO.
      This could be done with a flag, too.
  X. Offer a central "`[un]onevent' EVENT MACRO [conditions]" register.
     Change all hooks to use that one, optimize the case where a single
     macro is registered for a single event but with different
     preconditions.

     E.g., "on_interactive_mode_enter" could then be hooked to call
     `bind' and set `colour's, for example.  In conjunction with 2.
     above those commands could simply be (silent, successful) no-ops
     before we reach that state (and again after
     on_interactive_mode_leave is processed).

  X.x We should make the message accessible at least a bit on macro level.
      I.e., so that message specification checks can be applied on the
      macro level, as in
        if `message match-spec '@~t@bank.com'`
          set smime-sign customhdr='X-BlahBlahBlah:Banks are monsters'
        endif
      where `message' (or so) would refer to the currently active
      message (in compose mode) or the "dot" (otherwise).
      `message' would gain some keywords.  Something like that.
  8. The line buffer used in evaluate() that is passed through to
     commands (thus: in cmd_ctx, then) needs to become `const'.
     (I tried to do so in the past, but some commands write into it,
     thus i stopped and iirc even added some changes on my own which
     take favour of reusing that buffer.)
     + Macro execution then no longer needs to clone the macro content
     lines before executing then.
     + The temporary hack which duplicates the line buffer in order to
     place the original content in history can be removed again.
     + The command context also takes preparsed command array if so
     specified in cmd_tab, and entries are cleaned up (see 2.)
  9. We should unite all the un*() commands with their non-un*
     versions, if they have one.  They may take a different argument
     list etc., but only one entry in the command table for such.
     - The POSIX standard command abbreviations must remain, so maybe
       outsource those in a hashtable or whatever that is checked first,
       but detach command table order from that anyway.
     - Offer a(n optional, and on/off switchable) Damerau-Levenshtein
       mode for command completion;
 10. We MUST switch the entire codebase to use SysV signal handling, don't
     do the BSDish SA_RESTART, which is why we still suffer the way we
     do and need jumps.  I can't dig BSD signal handling, and never ever
     did so myself until i got here.
 11. Macro execution is potentially recursive.  Meaning that
     `undefine', etc. can occur while macros are executing.
     The simplemost approach would be to have some recursion counter for
     each macro and a delete_later flag that gets honoured when the
     recursion counter gets zero.  It would be already possible to
     immediately remove the macro from the hashtable, so that deeper
     levels wouldn't find it anymore.  To avoid leaks (which *are*) we
     need to have a jump location for our upcoming signal handler
     anyway.  (Also to get rid of the temporary_localopts_free() hack.
     + The same is true for `account's.  Here things are complicated by
     the global `account_name', i.e., the account could be the current
     one.
     + That also is: redefinition of names that have yet-pending
     deletion requests are possible.
 12. It is annoying that you cannot `source' your MAILRC multiple times.
     Defining a macro/account/xy should overwrite the current thing,
     just as it does anyway for normal variables!
     This is no different than 11. plus additional re-addition.
     (Same exception: what if the currently active account is
     overwritten?  Same answer, plus a message "new settings take effect
     when account is switched to the next time".)
 20. The attachment charset selection loop can then be rewritten to
     check whether an ^C occurred and treat that as end-of-loop
     condition.  In v14.6.3 this was introduced, but it should act
     differently depending on whether the interrupt occurred during
     character set selection or attachment filename input.
     Also in respect whether the interrupt is "propagated" or not.
     It's ugly, and documented accordingly.
 30. Mail protocols and mail messages are accessed through a "VFS".
     URL should then support file:// and maildir:// etc.  Update manual!

     . It should be considered to drop many variables in favour of
       ?SEARCH usage for keys, or key=value pairs, e.g.
        smtp://exam.ple?starttls=yes;smtp-hostname=;
       etc.   USER@HOST is neat, but that is possibly preferable?
       We could then also extend *mta* and drop *smtp* as such, e.g.,
       mta=smtp://dSDAS, mta=builtin://XZ, mta=bltin://XZ.
       Question: what is with smtp-hostname and such??
 31. Flag updates of individual messages must find their way through to
     the protocol.
 32. Use deque (on partial views).
 34. We need a new abstraction: `vie[ws]'.  I.e, viewset, viewclear,
     view(show|look)?  We will have (possibly readonly) boxes, a summary
     cache file, which is created when a mailbox is read in, and all
     that crap that we currently have (setptr(), setmsize(), etc.!) must
     vanish.  Instead there is another, in-memory abstraction, the view.
     Some views are builtin and are somehow selectable (the "all" view,
     for example, and the "new" view).
     It is possible to make a view persistent by giving it a name, e.g.,
     'viewset NAME MSG-SPEC' -- 'viewset allnew :n' (and 'viewset XY `'
     or something must be capable to tag the last a.k.a current).
     Switching to a named view would thus look over the entire current
     view (!) for all messages that comply to the message-spec of the
     view, then create a sorted/threaded display of that subset and
     create a new anonymous "result" view.  It must be possible to
     specify that a view is to be applied to the entire mailbox instead
     of the current view, via a simple easy understandable syntax.

     Or name it "msgset".
     We won't extend macros that much because it would require much too
     much logic for no purpose, instead we'll (hopefully) add some
     scriptable abstraction, with an optional builtin Lua binding.
     E.g., it would be too complicated to create a language to access
     individual message parts (headers etc.), add error handling etc.
     Better to make it all directly accessible via language binding, and
     offer commands which relinquish execution to such a binding, maybe
     even with some arguments, as in "? lua ARG" - it should be possible
     that ARG is a msgset, then.

     But what macros should be able to do is iterating over such
     a msgset / view, as in
        msgset create NAME :SPEC:
        msgset add NAME :SPEC:
      and then either
        for[-each] NAME
          normal-cmd SOMEHOW-REFER-TO-ITER
          if SOMEHOW-ACCESS-LAST-STATUS
          endif
        endfor
      as well as
        msgset cmd NAME normal-cmd
      and even
        normal-cmd NAME
      in which case NAME should simply be treated as a SPEC, likely that
      this requires a new trigger, though, e.g. {NAME}, [NAME] or !NAME...
      I.e., the new namelist (likely a deque) should contain MsgRef
      objects which point to a Message and a Mailbox (for cross-mailbox
      msgset's and without the need for a Message to store a pointer to
      the owning mailbox?  Or make MsgRef a superclass that may have
      a subclass which offers such cross-refs).

 50. Support SASL, unite all GSS-API etc. under an abstraction!
     Maybe even drop direct GSS-API and support only through SASL.
     That is, we can very well provide our own little SASL-client
     abstraction with what we have already by simply defining some
     "readline" abstraction plus struct ccred for use by the
     authentication layer: the protocols must set it up by passing in
     a line of authentication mechanisms and a callback mechanism.
     Possibly the user should be able to permit or forbid automatic
     selection of GSS-API (to avoid useless round-trips) etc. etc.
 80. The MIME rewrite: mime_parser <-> mime "DOM" analyzer <->
     selectively create filter chains per part and do XY.

     This also affects sending, and it will allow us to dig MIME
     (multipart) mail for -t/-m _correctly_.  Also in sofar as we can
     hook a content-decoder before diving into the MIME structure, and
     with a DOM, we can re-encode such things properly as we (re)send
     such mails.  All this is wrong at the time of this writing!
     We still need to special treat things like, e.g., RFC 2046, 5.2.1.
     But on top of we-can, as opposed to the opposite.
 99. Now i'm dreaming some more: with the new object-based approach
     multiple mailboxes could be in an open state.  And it should be
     possible to do so for the user (`file' and `folder' are required to
     quit the current mailbox [first -- this not yet]), which is why we
     either need new trigger characters or new commands.
     The absolute sensation would be joinable operations over multiple
     open mailboxes, e.g., views over multiple such!
100. If i say `p 3 2 1' then i mean `3 2 1' not `1 2 3'.
1000. Extend macros and provide language bindings.  See item 34.
     Additionally there was:
    I want *pipemac* (or *pipe-hookXY*).  This requires v15.0
    infrastructure (pseudo: evaluate() returns enum eval_retval{OK=0,
    ERR=1, ISMAC=0x80, MACOK=ISMAC|OK, MACERR=ISMAC|ERR}; new `return'
    command which only works in an executing macro (care: recursion);
    macro arguments in pseudo variables $1...$x (works?); `return' can
    "return" a list (simply keep argument list around); macro execution
    can then simply check the return value to decide what to do (no
    jumping); btw.: where is our `clone' command which clones WHATEVER?):
      define pipemac {
        clone name=$1 base=$2 ext=$3 ACTION=$4([SEND_]MBOX|DISPLAY|etc)
        if $name =~ 'README|INSTALL|TODO|COPYING|*.(txt|rc|cfg|conf)$'
          return 'text/plain'
        elif $ext =~ 'gz$'
          return 'application/gzip'
        elif $ext =~ '*.\.nim$'
          return 'text/x-nimrod'
        else
          varshell i /usr/bin/file --preserve-date $name
          return $i
        endif
      }
    Also interesting would be the possibility to let a macro BE the
    (forked+exec, I/O redirected) external handler, "returning the
    output I/O handle".
    Inspired by Gavin Troy and Bob Tennent.

-- Also (?same problem?) the thread sort doesn't get

    [A is deleted]
    B answers A
      C answers B
      D answers B
    E is unrelated
    F answers A

  The current sort fails to recognize that F and the thread starting at
  B are related, which results in a mess.

--- Being able to sort the outermost level of threads was a suggestion
    of Rudolf Sykora, especially being able to sort the outermost level
    according to the date of the newest message in a thread.

- Drop **use-starttls* in favour of something better: support 'auto',
  'no' and 'yes' and act accordingly.  For the former be smart enough on
  the protocol side.  (RFC 3207 describes man-in-the-middle attacks due
  to 'auto' TLS, so explicit 'yes' should be favoured).

- NOTE: we do not really support IPv6 sofar in that we are not prepared to
  deal with IPv6 addresses (as in '[ADDR]:PORT').  Pimp url_parse().
  And socket I/O.

- I had a connection collapse during a POP3 download, and neither was
  there a chance to get access to the 22 yet downloaded mails (after
  five minutes of waiting followed by CNTRL-C), nor did the layer
  recognize this very well (got myriads of `POP3 connection already
  closed.' messages, btw., the thirty-something messages which were not
  yet downloaded caused (after CNTRL-C) this: ETC. ETC.

- Add a value-duplication command, i.e.,
    clone _x header
    unset header
    ...
    clone header _x
    unset _x

- I got an email in base64 that obviously used CRNL line endings, and once
  i've replied the CR where quoted as *control* characters.
  Get rid of those (kwcrtest.mbox; may be hard to do everywhere for some
  time, due to how we deal with I/O and Send layer etc).

- edit.c doesn't do NEED_BODY (but IMAP won't work anyway).

- Stuff
  .. s-nail </dev/null should work interactively when STDERR_FILENO is
    a terminal!  (Builtin editor; how do editline and readline work?
    should this be documented?  What does NetBSD Mail do?  Should we NOT
    be interactive??  POSIX says for sh(1) (APPLICATION USAGE): 'sh
    2>FILE' is not interactive, even though it accepts terminal input.)
  . It would be cool if ghosts, shortcuts, alternates could
    (optionally?) be tracked via localopts.  And macros.  And inner macros.
    (Additional entry on xy-local xy somewhere above)
    And / or local to a macro/account if defined in one.
    Should be per-carrier copy-on-write environments.
    Just like TeX with def / gdef ...
  . Just like the RFC 3676 link above, it would be nice if it would be
    somehow possible to recognize links in a document; i don't know yet
    how this could be achieved without loosing formatting information (i
    mean, we could enable this and inject terminal colour sequences, but
    one should be able to say 'follow link x', starting an action
    handler, and the 'x' must come from somwhere - simply injecting
    '[NUMBER]' references distorts visual).  Anyway, it's just a filter
    that recognized the usual <SCHEME:/> stuff, and of course we can
    simply have a buffer which records all such occurrences, so that
    user can say '? xy NUMBER', but without the context it soon gets
    hard.
  . TTY layer: the tc*() family may fail with EINTR, which MUST be
    handled; setting also generates SIGTTOU when we're not in foreground
    pgrp, so we better deal with all that and ENSURE WE GET THROUGH when
    resetting terminal attributes!
  .. TTY "I guess it would be much better to create our own session via
     setpgid(2) and then tcsetpgrp(3) any processes we run synchronously,
     and properly deal with SIGTTOU, but it always has been like that and
     i won't do that before other things have been changed.
  .. NOTE: TTY: place (at least n_child_run()) childs which go over
     terminal into own group.
     Introduce global "terminal state" manager which tracks who
     currently owns the terminal, so that we can gracefully switch it
     on/off, check in main loop whether restore is necessary.
     It that can deal with multiple "windows" then we could have open
     multiple of those, i.e., multiple PAGER instances, and non-PAGER
     instances (fflush() if there is FILE before switch off).

     TTY thus: "needsterminal" could be driven gracefully EVEN IF
     a PAGER is open because the current user action is "print*", since
     we KNOW that there is a PAGER and can temporarily lay it down to
     sleep, fflush()ing/adjusting as necessary, and wake it up again
     afterwards (or, with some work, gracefully shut it down if ^C ^C).
     This is of course also true for user questions ("really display
     part XY?", "need decryption password:", etc.!!!)!!
  . Remove all occurrences of mbtowc() with mbrtowc(); temporarily add (some)
    global mbstate_t objects until the send / MIME layer rewrite is done and
    has the carrier.  Use flip states and add aux funs with only update the
    state+toggle on success -- CURRENTLY MBTOWC FAILURES ARE PRACTICALLY NOT
    HANDLED!!
  . pop3,mime_cte +++: \r,\n -> \015,\012, to avoid ANY problems..
    Maybe our passed carrier should pass desired output newline (as
    opposed to data-embedded newlines)
  . which_protocol(), *newmail* mechanism, displayname, mailname: all of
    this <rude>SHIT</rude> must vanish and be replaced by a URL, and
    a nice "VFS" mailbox object that carries all necessary state so that
    one can work with it.

    If not mentioned somewhere else: struct message should be splitted
    into a tree of objects, with a base class that has as few fields as
    possible; the global *message should be a deque, only accessible via
    iterator; it should store pointers to (the actually used subtype of)
    message structures instead; i.e., for maildir boxes the path is yet
    allocated separately, then it could be part of the message object,
    etc.
    It should contain a ui8_t that tracks the number of contained parts,
    so that the "fits-onto-the-screen" tests are more useful than today;
    i think 8-bit is sufficient, with 0xFF meaning more-than-fits-here.
  . Given how many temporary files we use, it would make sense to
    support a reusable single temporary file, as in singletmp_take() and
    singletmp_release(), where singletmp_release() would close and thus
    drop the file if it excesses a specific (configurable) size, and the
    mainloop tick would close it (after X (configurable) unused ticks))
    otherwise.  I guess this would improve performance for searching
    etc. etc.
  . _(), N_(), V_():
    use GNU tools for extraction etc., and write a simple helper program
    which converts these files to a serialized hashmap, just like we did
    for the okeys (and *exactly* so); add a config check whether the ({})
    extension is supported and finally use that for some ({static char
    const *tr_res;}) injection optimization, then.  (Think SFSYS)
  . Searching body/text yet includes headers from attachments and
    attachment data.  This is shit.  :)
  . The "nifty" unregister_file()->_compress() mechanism that even
    shovels '-Sfolder=imaps://user1@localhost -Srecord="+Sent Items"'
    *records* calls clearerr() on the descriptor before performing it's
    action anyway.  when we really make it even to the I/O rewrite, it
    should be possible to dis-/allow such -- it doesn't make sense to
    add something faulty to whatever was not faulty before!
  . `dp' prints EOF at the end of a thread even if unread messages
    follow
  . When doing `~w FILE' and FILE cannot be written to (was a directory)
    then the composed mail is lost completely, it seems we jump to the
    very main loop!
  . `resend' doesn't smime-sign.
  . Really do extend the test already today; test S/MIME
    signing/encryption/decryption with two pairs of identities, instead
    of of one.
  . RFC 5751 describes a message multipart layout that also includes the
    headers in the signature; it would be nice (for completeness sake)
    to be able to support that.  Note shutup@ietf.org.
  . The capability to save a message under the name of a recipient is in
    the standard etc., but i've never used it.
    What would be cool, otoh, would be if there would be the possibility
    to register a regular expression, and if just *any* recipient of
    a message matches, store the message in the given folder instead.
    I.e., if i send a message to s-nail-users@ then i most likely want
    to get a copy to the corresponding box, regardless of whoever the
    message was sent To: Cc: or Bcc: else..
  . why not simply sucking in complete MIME messages via -t?  In a way
    that parses it as a MIME message, that is!
    (Brezn Stangl, brezn DOT stangl AT yandex DOT com)
  . mutt list handling (`~') is very powerful
  . We have some use of *at() functions, especially anything which
    temporarily switches cwd.
  . *newmail* is terrible.  At some later time we need to do somethings
    with timeouts etc. (for MBOX and Maildir it's not that bad, but for
    anything over the network, yet the mentioned may come in over NFS).
    Remove it until we have something better?
  . The RFC 3798 *disposition-notification-send* mechanism is yet not
    truly conforming (and works with *from*).  Also, this is only the
    sender side, there should be support for creating the MDN response.
    (Maybe ternary option: off (default),
    create-when-unread-flag-goes-away, ditto-but-also-strip-header)
  .. Also, there is DSN as a SMTP extension, see the RFCs 3461, 346 (as
     above) and 6522 (Wikipedia).
  . The var_* series should return "const char*" not "char*".
    This should already work today because otherwise we would get SEGV
    all through the way.
  .. While here: rename enum okeys to enum internal_variables, and the
     ok_*() series to iv_().  And see below for env_*() series.
  . fexpand() the 2nd: it should return structure because we need to
     check for FEDIT_SYSBOX, which currently only checks whether the first
     character of a file name is '%', not whether it is '%', '%:FILEPATH'
     or '%VALIDUSER', because that is impossible to do!
  . On the long run in-memory password storage should be zeroed after
    use, possibly even encoded *during* use.  After v15.
  . We need a `spamcheck' command that is like `spamrate' but updates
    the mail in-place, i.e., with the headers that the spam engine adds.
  . __narrow_suffix() is wrong (for stateful encodings that we
    don't support yet) and should inject a reset sequence if it shortens
    the string.
  .. THAT IS TO SAY: the entire codebase doesn't really support stateful
    encodings, including the bidi_ things that i've done (but the MLE
    does iirc?  what is this??).  We should have a global string that
    has the multibyte reset sequence plus length available for easy
    access.
  . When a user edits a specific header, it should no longer be
    modified.
  . Regular expression list resorting is no good; the user should be
    able to specify a match order weight, as in:
      mlist 10 a@b.org 8 c@d.org .*@else@org 0 almost@never.com
    So: optional digit 0-10, where 0-4 are never relinked and always
    placed at the tail, 6-10 are never relinked and always placed at
    head (all in decreasing order, head to tail), and 5 is the implicit
    value, placed in between and automatically resorted just as is the
    sole algorithm we currently have.
  .. And maybe we should have an event mechanism with one-shot etc..
     Then install a resorter function when we actually have lookups and
     one-shot sort the entire thing once (when the loop ticks).
     Instead of busy resorting, that is.
  . Some pieces of cake (e.g. usermap()) don't perform actions on
    addresse(e)s if the first character is a backslash.  Others do.
    And do we really support that notion all through the codebase.
    And, even more: should we support that at all?
  . We need more hooks: on-leave, on-connect.. whatever
  . The new internal ~/$ expansion mechanism should possibly get support
     for POSIX parameter expansions ${[:]-} and ${[:]+} (and ${[:]?}).
     There is no real way to get the functionality otherwise...
  . Make S/MIME an option separate of SSL/TLS, i.e., optional.
  . With very long input Heirloom mailx(1) / S-nail(1) can produce
    encoded-words (RFC 2047) with incomplete multibyte sequences (i.e.,
    non self-contained encoded-words).
  . Group addresses, especially the undisclosed recipients but also
    "Bla": addresses; are missing.
  . It is terrible that -S sets variables twice, at once and after the
    resource files have been loaded.  Instead we should set a bit
  . Cleanup:  n_ is anything public / external,
    _n_ is public/e but not really, FILENAME/ABBREV_symbol are statics,
    FILENAME/ABBREV__symbol are helpers of a single other static
    (function group) (-> colour.c, lex_input.c: done).
  . Per-folder (S/MIME) en- and decryption key (Tarqi Kazan): if a xy
    variable is set (that points to a key) add a transparent en- and
    decryption layer on top of any per-message operation (for boxes for
    which the variable is set).
  . For v15.0: remember private thread with Tarqi Kazan (2015-05) and
    try to improve situation with *record*, so that only messages enter
    it which have really been sent.  If we support postponing and have
    a multi-process layout and add an intermediate *record-queue* we
    may be able to improve the situation.
  . I like "SMTP and SUBMISSION Service Extensions For Address Query",
    for which we need
      if (!SSL_set_tlsext_host_name(con, servername))
        error
  . [Dd]ecrypt should transport decryption errors, not silently be like
    copy and copy undecrypted content, because this is what it's for?
    ..We need atomic operations with rollback support in order to make
      this happen, but i think maybe file truncation (decryption always
      appends?) is enough provided that files are locked?
      WE NEED ATOMIC OPERATION SUPPORT for quite some operations.
      Man, are we far from that.
  . `pipe' is total shit regarding MIME.  We need some defined and
     documented method to configure which parts are displayed and/or how
     they are visually separated.
  . The new n_err() facility is nice, but we need n_msg() too, maybe
    more.  Maybe à la Log:: and macro wrappers with priority unrolled,
    then add user-settable treshold, too.  Btw. C99 adds vararg macros.
    .. We must handle embedded newlines properly
    .. In PARTICULAR we MUST NOT use stdout when in batch mode:
         ssh X "echo 'move * |cat' | s-nail -#f %" >> download
       should do the right thing, but can't like that due to unwanted
       noise in the stdout output!  Best would be if it would be
       possible to explicitly define a file/dev to be used, but falling
       back to stderr in batch mode otherwise. (think S-Web42)
  . Exit status handling is sick.
  . Especially in interactive mode MIME classification should count
    (non-NUL) control characters and the number of different thereof,
    giving users an option to treat something as text nonetheless.
    E.g., roff files often use controls as separators in conditionals,
    some shell scripts use controls for $IFS field separation, and that
    will end up with charset=binary.
  .. *mime-allow-text-controls* is a no-brainer: instead we should
     introduce something that allows us to switch and detect UTF-16 once
     we run into the problematic situation, then start all over in an
     Unicode mode?  I.e.: continue to force the user to set such
     a switch, but do it in a sensible fashion, because the UTF-16 data
     stream may nonetheless contain control characters??
--

  . We can "steal" features from msmtp(1) that make sense: SOCKS support
    (primitive) and /etc/aliases ($mta_alias_file).  At least postfix(1)
    supports file and pipe addressees in the latter...  It also
    supports include files via :include:/filename but which i think
    should be supported in a second step.  Ditto caching (timestamp
    check and a mechanism to support/disable caching.)

# s-ts-mode