| blob | 0928deb92cdbb354a84f5d570945bdfe5335601e |
1 /** \mainpage KMail architectural overview
3 \section KMail design principles
5 This file is intended to guide the reader's way through the KMail
6 codebase. It should esp. be handy for people not hacking full-time on
7 KMail as well as people that want to trace bugs in parts of KMail
8 which they don't know well.
10 Contents:
11 - \ref kernel
12 - \ref identity
13 - \ref filter
14 - \ref configuration
15 - \ref mdn
16 - \ref folders
17 - \ref folderindex
18 - \ref headers
19 - \ref display
21 TODO: reader, composer, messages, accounts, ...
23 \section kernel Kernel
25 Files: kmkernel.h, kmkernel.cpp
27 Contact Zack Rusin <zack@kde.org> with questions...
29 The first thing you'll notice about KMail is the extensive use of
30 kmkernel->xxx() constructs. The "kmkernel" is a define in kmkernel.h
31 declared as :
32 #define kmkernel KMKernel::self()
33 KMKernel is the central object in KMail. It's always created before
34 any other class, therefore you are _guaranteed_ that KMKernel::self()
35 (and therefore "kmkernel" construct) won't return 0 (null).
37 KMKernel implements the KMailIface (our DCOP interface) and gives
38 access to all the core KMail functionality.
41 \section identity Identity
43 FIXME this has moved to libkpimidentities, right?
45 Files: identity*, kmidentity.{h,cpp}, configuredialog.cpp,
46 signatureconfigurator.{h,cpp}
48 Contact Marc Mutz <mutz@kde.org> on questions...
50 Identities consists of various fields represented by
51 QStrings. Currently, those fields are hardcoded, but feel free to
52 implement KPIMIdentities::Identity as a map from strings to QVariants or somesuch.
54 One part of identities are signatures. They can represent four modes
55 (Signature::Type) of operation (disabled, text from file or command
56 and inline text), which correspond to the combo box in the
57 identitydialog.
59 Identities are designed to be used through the KPIMIdentities::IdentityManager:
60 const KPIM::Identity & ident =
61 kmkernel->identityManager()->identityForUoidOrDefault(...)
62 Make sure you assign to a _const_ reference, since the identityForFoo
63 methods are overloaded with non-const methods that access a different
64 list of identities in the manager that is used while configuring. That
65 is known source of errors when you use identityForFoo() as a parameter
66 to a method taking const KPIMIdentities::Identity &.
68 WARNING: Don't hold the reference longer than the current functions
69 scope or next return to the event loop. That's b/c the config dialog
70 is not modal and the user may hit apply/ok anytime between calls to
71 function that want to use the identity reference. Store the UOID
72 instead if you need to keep track of the identity. You may also want
73 to connect to one of the KPIMIdentities::IdentityManager::changed() or ::deleted()
74 signals, if you want to do special processing in case the identity
75 changes.
77 Thus, in the ConfigureDialog, you will see non-const KPIMIdentities::Identity
78 references being used, while everywhere else (KMMessage,
79 IdentityCombo) const references are used.
81 The KPIMIdentities::IdentityCombo is what you see in the composer. It's a
82 self-updating combo box of KPIMIdentities::Identity's. Use this if you want the user
83 to choose an identity, e.g. in the folder dialog.
85 Ihe IdentityListView is what you see in the config dialog's identity
86 management page. It's not meant to be used elsewhere, but is DnD
87 enabled (well, at the time of this writing, only drag-enabled). This
88 is going to be used to dnd identities around between KNode and KMail,
89 e.g.
91 The SignatureConfigurator is the third tab in the identity
92 dialog. It's separate since it is used by the identity manager to
93 request a new file/command if the current value somehow fails.
97 \section filter Filter
99 Contact Marc Mutz <mutz@kde.org> on questions...
101 Filters consist of a search pattern and a list of actions plus a few
102 flags to indicate when they are to be applied (kmfilter.h).
103 They are managed in a QPtrList<KMFilter>, called KMFilterMgr. This
104 filter magnager is responsible for loading and storing filters
105 (read/writeConfig) and for executing them (process). The unique
106 instance of the filter manager is held by the kernel
107 (KMKernel::filterMgr()).
109 The search pattern is a QPtrList of search rules (kmsearchpattern.h) and a
110 boolean operator that defines their relation (and/or).
112 A search rule consists of a field-QString, a "function"-enum and a
113 "contents" or "value" QString. The first gives the header (or
114 pseudoheader) to match against, the second says how to match (equals,
115 consists, is less than,...) and the third holds the pattern to match
116 against.
117 Currently, there are two types of search rules, which are mixed
118 together into a single class: String-valued and int-valued. The latter
119 is a hack to enable \verbatim<size>\endverbatim and
120 \verbatim<age in days>\endverbatim pseudo-header matching.
121 KMSearchRules should better be organized like KMFilterActions are.
123 A filter action (kmfilteraction.h) inherits from KMFilterAction or one
124 of it's convenience sub-classes. They have three sub-interfaces: (1)
125 argument handling, (2) processing and (3) parameter widget handling.
126 Interface (1) consists of args{From,As}String(), name() and
127 isEmpty() and is used to read and write the arguments (if any) from/to
128 the config.
129 Interface (2) is used by the filter manager to execute the action
130 (process() / ReturnCode).
131 Interface (3) is used by the filter dialog to allow editing of
132 actions and consists of name(), label() and the
133 *ParamWidget*(). Complex parameter widgets are collected in
134 kmfawidget.{h,cpp}.
136 A typical call for applying filters is
138 \code
139 KMKernel::filterMgr()
140 foreach message {
141 KMFilterMgr::process():
142 }
143 \endcode
146 \section configuration Configure Dialog
148 Files: configuredialog*.{h,cpp} ( identitylistview.{h,cpp} )
150 Contact Marc Mutz <mutz@kde.org> on questions...
152 The configuredialog is made up of pages that in turn may consist of a
153 number of tabs. The genral rule of thumb is that each page and tab is
154 responsible for reading and writing the config options presented on
155 it, although in the future, that may be reduced to interacting with
156 the corresponding config manager instead. But that won't change the
157 basic principle.
159 Thus, there is an abstract base class ConfigurePage (defined in
160 configuredialog_p.h), which derives from QWidget. It has four methods
161 of which you have to reimplement at least the first two:
163 - void setup()
164 Re-read the config (from the config file or the manager) and update
165 the widgets correspondingly. That is, you should only create the
166 widgets in the ctor, not set the options yet. The reason for that is
167 that the config dialog, once created, is simply hidden and shown
168 subsequently, so we need a reset-like method anyway.
170 - void apply()
171 Read the config from the widgets and write it into the config file
172 or the corresponding config manager.
174 - void installProfile()
175 This is called when the user selected a profile and hit apply. A
176 profile is just another KConfig object. Therefore, this method
177 should be the same as setup(), except that you should only alter
178 widgets for configs that really exist in the profile.
180 For tabbed config pages, there exists a convenience class called
181 TabbedConfigurationPage, which (as of this writing only offers the
182 addTab() convenience method. It is planned to also provide
183 reimplementations of setup, dismiss, apply and installProfile that just
184 call the same functions for each tab.
186 \section mdn MDNs
188 Files: libkdenetwork/kmime_mdn.{h,cpp} and kmmessage.{h,cpp}, mostly
190 Contact Marc Mutz <mutz@kde.org> on questions...
192 MDNs (Message Disposition Notifications; RFC 2298) are a way to send
193 back information regarding received messages back to their
194 sender. Examples include "message read/deleted/forwarded/processed".
196 The code in kmime_mdn.{h,cpp} is responsible for creating the
197 message/disposition-notification body part (2nd child of
198 multipart/report that makes the MDN) and for providing the template
199 for human-readable text that goes into the text/plain part (1st child
200 of the multipart/report).
202 The code in KMMessage::createMDN() actually constructs a message
203 containing a MDN for this message, using the kmime_mdn helper
204 functions. It starts by checking the index for an already sent MDN,
205 since the RFC demands that MDNs be sent only once for every
206 message. If that test succeeds, it goes on to check various other
207 constraints as per RFC and if all goes well the message containing the
208 multipart/report is created.
210 If you need to use this functionality, see KMReaderWin::touchMsg() and
211 KMFilterAction::sendMDN() for examples. The touchMsg() code is invoked
212 on display of a message and sends a "displayed" MDN back (if so
213 configured), whereas the KMFilterAction method is a convenience helper
214 for the various filter actions that can provoke a MDN (move to trash,
215 redirect, forward, ...).
218 \section folders Folders
220 Files: kmfolder*.{h,cpp}, folderstorage.{h,cpp} and *job.{h,cpp}
222 Contact Zack Rusin <zack@kde.org> with questions...
224 The collaboration among KMail folder classes looks
225 as follows :
227 <pre>
228 KMFolderNode
229 / \
230 / \
231 KMFolderDir \
232 KMFolder
233 .
234 .
235 v
236 FolderStorage
237 |
238 |
239 KMFolderIndex
240 |
241 |
242 ---< actual folder types: KMFolderImap, KMFolderMbox... >--
243 </pre>
245 At the base KMail's folder design starts with KMFolderNode which
246 inherits QObject. KMFolderNode is the base class encapsulating
247 common folder properties such as the name and a boolean signifying whether
248 the folder is a folder holding mail directly or a KMFolderDir.
249 KMFolderNode's often do not have an on-disk representation, they are
250 entities existing only within KMail's design.
252 KMFolder acts as the runtime representation of a folder with the physical
253 storage part being represented by a member of type FolderStorage.
254 KMFolder and FolderStorage have many functions with the same names and
255 signatures, but there is no inheritance.
256 KMFolderIndex contains some common indexing functionality for physical folders.
257 Subclasses of KMFolderIndex finally interact directly with physical storage
258 or with storage providers over the network.
260 KMFolderDir is a directory abstraction which holds KMFolderNode's.
261 It inherits KMFolderNode and KMFolderNodeList which is a QPtrList<KMFolderNode>.
262 A special case of a KMFolderDir is KMFolderRootDir; it represents
263 the toplevel KMFolderDir in KMail's folder hierarchy.
265 KMFolderDir's contents are managed by KMFolderMgr's.
266 KMail contains four main KMFolderMgr's. They can be
267 accessed via KMKernel ( the "kmkernel" construct ). Those methods are :
268 1) KMFolderMgr *folderMgr() - which returns the folder manager for
269 the folders stored locally.
270 2) KMFolderMgr *imapFolderMgr() - which returns the folder manager
271 for all imap folders. They're handled a little differently because
272 for all imap messages only headers are cached locally while the
273 main contents of all messages is kept on the server.
274 3) KMFolderMgr *dimapFolderMgr() - which returns disconnected IMAP (dimap)
275 folder manager. In dimap, both the headers and a copy of the full message
276 are cached locally.
277 4) KMFolderMgr *searchFolderMgr() - which returns the folder manager
278 for search folders (folders created by using the "find
279 messages" tool). Other email clients call this type of folder
280 "virtual folders".
282 FolderJob classes - These classes allow asynchronous operations on
283 KMFolder's. You create a Job on the heap, connect to one of its
284 signals and wait for the job to finish. Folders serve as FolderJob
285 factories. For example, to retrieve the full message from a folder
286 you do :
288 \code
289 FolderJob *job = folderParent->createJob( aMsg, tGetMessage );
290 connect( job, SIGNAL(messageRetrieved(KMMessage*)),
291 SLOT(msgWasRetrieved(KMMessage*)) );
292 job->start();
293 \endcode
296 \section folderindex Index (old)
298 Files: kmfolderindex.{h,cpp} and kmmsg{base,info}.{h,cpp}
300 Contact Marc Mutz <mutz@kde.org> or
301 Till Adam <adam@kde.org> or
302 Don Sanders <sanders@kde.org>
303 with questions...
305 index := header *entry
308 header := magic LF NUL header-length byte-order-marker sizeof-long
310 magic := "# KMail-Index V" 1*DIGITS
312 header-length := quint32
314 byte-order-marker := quint32( 0x12345678 )
316 sizeof-long := quint32( 4 / 8 )
319 entry := tag length value
321 tag := quint32 ; little endian (native?)
323 length := quint16 ; little endian (native?)
325 value := unicode-string-value / ulong-value
327 unicode-string-value := 0*256QChar ; network-byte-order
329 ulong-value := unsigned_long ; little endian
331 Currently defined tag values are:
333 <pre>
334 Msg*Part num. val type obtained by:
336 No 0 u -
337 From 1 u fromStrip().trimmed()
338 Subject 2 u subject().trimmed()
339 To 3 u toStrip().trimmed()
340 ReplyToIdMD5 4 u replyToIdMD5().trimmed()
341 IdMD5 5 u msgIdMD5().trimmed()
342 XMark 6 u xmark().trimmed()
343 Offset 7 l folderOffset() (not only mbox!)
344 LegacyStatus 8 l mLegacyStatus
345 Size 9 l msgSize()
346 Date 10 l date()
347 File 11 u fileName() (not only maildir!)
348 CryptoState 12 l (signatureState() << 16) | encryptionState())
349 MDNSent 13 l mdnSentState()
350 ReplyToAuxIdMD5 14 u replyToAuxIdMD5()
351 StrippedSubject 15 u strippedSubjectMD5().trimmed()
352 Status 16 l status()
353 </pre>
355 u: unicode-string-value; l: ulong-value
357 Proposed new (KDE 3.2 / KMail 1.6) entry format:
359 index := header *entry
361 entry := sync 1*( tag type content ) crc-tag crc-value
363 sync := quint16 (32?) ; resync mark, some magic bit pattern
364 ; (together with preceding crc-tag provides
365 ; 24(40)bits to resync on in case of index
366 ; corruption)
368 tag := quint8
370 type := quint8
372 content := variable-length-content / fixed-length-content
374 crc-tag := tag type ; tag=CRC16, type=CRC16
376 crc-value := quint16 ; the CRC16 sum is calculated over all of
377 ; 1*( tag type content )
379 variable-length-content := length *512byte padding
381 padding := *3NUL ; make the string a multiple of 4 octets in length
383 fixed-length-content := 1*byte
385 length := quint16 (quint8?)
387 byte := quint8
389 The type field is pseudo-structured:
391 <pre>
392 bit: 7 6 5 4 3 2 1 0
393 +-----+-----+-----+-----+-----+-----+-----+-----+
394 | uniq | chunk | len |
395 +-----+-----+-----+-----+-----+-----+-----+-----+
396 </pre>
398 uniq: 3 bits = max. 8 different types with same chunk size:
400 <pre>
401 for chunk = (0)00 (LSB(base)=0: octets):
402 00(0) Utf8String
403 01(0) BitField
404 10(0) reserved
405 11(0) Extend
407 for chunk = (1)00 (LSB(base)=1: 16-octet blocks):
408 00(1) MD5(Hash/List)
409 01(1) reserved
410 10(1) reserved
411 11(1) Extend
413 for chunk = 01 (shorts):
414 000 Utf16String
415 001-110 reserved
416 111 Extend
418 for chunk = 10 (int32):
419 000 Utf32String (4; not to be used yet)
420 001 Size32
421 010 Offset32
422 011 SerNum/UOID
423 100 DateTime
424 101 Color (QRgb: (a,r,g,b))
425 110 reserved
426 111 Extend
428 for chunk = 11 (int64):
429 000 reserved
430 001 Size64
431 010 Offset64
432 011-110 reserved
433 111 Extend
434 </pre>
436 len: length in chunks
437 000 (variable width -> quint16 with the real width follows)
438 001..111: fixed-width data of length 2^len (2^1=2..2^6=128)
440 You find all defined values for the type field in indexentrybase.cpp
442 Currently defined tags are:
444 <pre>
445 tag type content
447 DateSent DateTime Date:
448 DateReceived DateTime last Received:'s date-time
449 FromDisplayName String decoded display-name of first From: addr
450 ToDisplayName String dto. for To:
451 CcDisplayName String dto. for Cc:
452 FromAddrSpecs String possibly IMAA-encoded, comma-separated addr-spec's of From:
453 ToAddrSpecs String dto. for To:
454 CcAddrSpecs String dto. for Cc:
455 Subject String decoded Subject:
456 BaseSubjectMD5 String md5(utf8(stripOffPrefixes(subject())))
457 BodyPeek String body preview
458 MaildirFile String Filename of maildir file for this messagea
459 MBoxOffset Offset(64) Offset in mbox file (pointing to From_)
460 MBoxLength Size(64) Length of message in mbox file (incl. From_)
461 Size Size(64) rfc2822-size of message (in mbox: excl. From_)
462 Status BitField (see below)
463 MessageIdMD5 MD5Hash MD5Hash of _normalized_ Message-Id:
464 MDNLink SerialNumber SerNum of MDN received for this message
465 DNSLink SerialNumber SerNUm of DSN received for this message
466 ThreadHeads SerialNumberList MD5Hash's of all (so far discovered)
467 _top-level thread parents_
468 ThreadParents SerialNumberList MD5Hash's of all (so far discovered)
469 thread parents
470 </pre>
473 "String" is either Utf8String or (Utf16String or Latin1String),
474 depending on content
476 Currently allocated bits for the Status BitField are:
478 <pre>
479 Bit Value: on(/off) (\\imapflag)
481 # "std stati":
482 0 New (\\Recent)
483 1 Read (\\Seen)
484 2 Answered (\\Answered)
485 3 Deleted (\\Deleted)
486 4 Flagged (\\Flagged)
487 5 Draft (\\Draft)
488 6 Forwarded
489 7 reserved
491 # message properties:
492 8 HasAttachments
493 9 MDNSent ($MDNSent)
494 10..11 00: unspecified, 01: Low, 10: Normal, 11: High Priority
495 12..15 0001: Queued, 0010: Sent, 0011: DeliveryDelayed,
496 0100: Delivered, 0101: DeliveryFailed,
497 0110: DisplayedToReceiver, 0111: DeletedByReceiver,
498 1001: ProcessedByReceiver, 1010: ForwardedByReceiver,
499 1011-1110: reserved
500 1111: AnswerReceived
502 # signature / encryption state:
503 16..19 0001: FullyEncrypted, 0010: PartiallyEncrypted, 1xxx: Problem
504 20..23 0001: FullySigned, 0010: PartiallySigned, 1xxx: Problem
506 # "workflow stati":
507 24..25 01: Important, 10: ToDo, 11: Later (from Moz/Evo)
508 26..27 01: Work, 10: Personal, 11: reserved (dto.)
509 28..29 01: ThreadWatched, 10: ThreadIgnored, 11: reserved
510 30..31 reserved
511 </pre>
513 All bits and combinations marked as reserved MUST NOT be altered if
514 set and MUST be set to zero (0) when creating the bitfield.
517 \section headers Headers (Threading and Sorting)
519 Contact Till Adam <adam@kde.org> or
520 Don Sanders <sanders@kde.org>
521 with questions...
523 Threading and sorting is implemented in kmheaders.[cpp|h] and headeritem.[cpp|h]
524 and involves a handfull of players, namely:
526 class KMHeaders:
527 this is the listview that contains the subject, date etc of each mail.
528 It's a singleton, which means there is only one, per mainwidget headers
529 list. It is actually a member of KMMainwidget and accessible there.
531 class KMail::HeaderItem:
532 these are the [Q|K]ListViewItem descendend items the KMHeaders listview
533 consists of. There's one for each message.
535 class KMail::SortCacheItem:
536 these are what the threading and sorting as well as the caching of those
537 operate on. Each is paired with a HeaderItem, such that each holds a
538 pointer to one, and vice versa, each header item holds a pointer to it's
539 associated sort cache item.
541 .sorted file:
542 The order of the sorted and threaded (if threading is turned on for this
543 folder) messages is cached on disk in a file named .$FOLDER.index.sorted
544 so if, for example the name of a folder is foo, the associated sorting
545 cache file would be called ".foo.index.sorted".
546 For each message, its serial number, that of its parent, the length of its
547 sorting key, and the key itself are written to this file. At the start of
548 the file several per folder counts and flags are cached additionally,
549 immediately after a short file headers. The entries from the start of the
550 file are:
551 - "## KMail Sort V%04d\n\t" (magic header and version string)
552 - byteOrder flag containing 0x12345678 for byte order testing
553 - column, the sort column used for the folder
554 - ascending, the sort direction
555 - threaded, is the view threaded or is it not?
556 - appended, have there been items appended to the file (obsolete?)
557 - discovered_count, number of new mail discovered since the last sort file
558 generation
559 - sorted_count, number of sorted messages in the header list
561 What is used for figuring out threading?
562 - messages can have an In-Reply-To header that contains the message id of
563 another message. This is called a perfect parent.
564 - additionally there is the References header which, if present, holds a
565 list of message ids that the current message is a follow up to. We
566 currently use the second to last entry in that list only. See further
567 down for the reasoning behind that.
568 - If the above two fail and the message is prefixed (Re: foo, Fwd: foo etc.)
569 an attempt is made to find a parent by looking for messages with the same
570 subject. How that is done is explained below as well.
572 For all these comparisons of header contents, the md5 hashes of the headers
573 are used for speed reasons. They are stored in the index entry for each
574 message. All data structures described below use md5 hash strings unless
575 stated otherwise.
577 Strategy:
578 When a folder is opened, updateMessageList is called, which in turn calls
579 readSortOrder where all the fun happens. If there is a .sorted file at the
580 expected location, it is openend and parsed. The header flags are read in
581 order to figure out the state in which this .sorted file was written. This
582 means the sort direction, whether the folder is threaded or not etc.
583 FIXME: is there currently sanity checking going on?
584 Now the file is parsed and for each message in the file a SortCacheItem is
585 created, which holds the offset in the .sorted file for this message as well
586 as it's sort key as read from the file. That sort cache item is entered into
587 the global mSortCache structure (member of the KMHeaders instance), which is
588 a QMemArray<SortCacheItem *> of the size mFolder->count(). Note that this
589 is the size reported by the folder, not as read from the .sorted file. The
590 mSortCache (along with some other structures) is updated when messages are
591 added or removed. More on that further down.
592 As soon as the serial number is read from the file, that number is looked up
593 in the message dict, to ensure it is still in the current folder. If not, it
594 has been moved away in the meantime (possibly by outside forces such as
595 other clients) and a deleted counter is incremented and all further
596 processing stopped for this message.
597 The messages parent serial number, as read from the sorted file is then
598 used to look up the parent and reset it to -1 should it not be in the
599 current folder anymore. -1 and -2 are special values that can show up
600 as parent serial numbers and are used to encode the following:
601 -1 means this message has no perfect parent, a parent for it needs to
602 be found from among the other messages, if there is a suitable one
603 -2 means this message is top level and will forever stay so, no need
604 to even try to find a parent. This is also used for the non-threaded
605 case. These are messages that have neither an In-Reply-To header nor
606 a References header and have a subject that is not prefixed.
607 In case there is a perfect parent, the current sort cache item is
608 appended to the parents list of unsorted children, or to that of
609 root, if there is not. A sort cache item is created in the mSortCache
610 for the parent, if it is not already there. Messages with a parent of
611 -1 are appended to the "unparented" list, which is later traversed and
612 its elements threaded. Messages with -2 as the parent are children of
613 root as well, as noted above, and will remain so.
615 Once the end of the file is reached, we should have a nicely filled
616 mSortCache, containing a sort cache item for each message that was in the
617 sorted file. Messages with perfect parents know about them, top level
618 messages know about that as well, all others are on a list and will be
619 threaded later.
621 Now, what happens when messages have been added to the store since the last
622 update of the .sorted file? Right, they don't have a sort cache item yet,
623 and would be overlooked. Consequently all message ids in the folder from 0
624 to mFolder->count() are looked at and a SortCacheItem is created for the
625 ones that do not have one yet. This is where all sort cache items are created
626 if there was no sorted file. The items created here are by definition un-
627 sorted as well as unparented. On creation their sort key is figured out as
628 well.
630 The next step is finding parents for those messages that are either new, or
631 had a parent of -1 in the .sorted file. To that end, a dict of all sort
632 cache items indexed by the md5 hash of their messsage id headers is created,
633 that will be used for looking up sort cache items by message id. The list of
634 yet unparented messages is then traversed and findParent() called for each
635 element wihch checks In-Reply-To and References headers and looks up the
636 sort cache item of those parents in the above mentioned dict. Should none be
637 found, the item is added to a second list the items of which will be subject
638 threaded.
640 How does findParent() work, well, it tries to find the message pointed to by
641 the In-Reply-To header and if that fails looks for the one pointed to by the
642 second to last entry of the References header list. Why the second to last?
643 Imagine the following situation in Bob's kmail:
644 - Bob get's mail from Alice
645 - Bob replies to Alice's mail, but his mail is stored in the Outbox, not the
646 Inbox.
647 - Alice replies again, so Bob now has two mails from Alice which are part of
648 the same thread. The mail in the middle is somewhere else. We want that to
649 thread as follows:
650 <pre>
651 Bob <- In-Reply-To: Alice1
652 ============
653 Alice1
654 |_Alice <- In-Reply-To: Bob (not here), References: Alice, Bob
655 </pre>
657 - since the above is a common special case, it is worth trying. I think. ;)
659 If the parent found is perfect (In-Reply-To), the sort cache items is marked
660 as such. mIsImperfectlyThreaded is used for that purposer, we will soon see
661 why that flag is needed.
663 On this first pass, no subject threading is attempted yet. Once it is done,
664 the messages that are now top-level, the current thread heads, so to speak,
665 are collected into a second dict ( QDict< QPtrList< SortCacheItem > > )
666 that contains for each different subject an entry holding a list of (so far
667 top level) messages with that subject, that are potential parents for
668 threading by subjects. These lists are sorted by date, so the parent closest
669 by date can be chosen. Sorting of these lists happens via insertion sort
670 while they are built because not only are they expected to be short (apart
671 from hard corner cases such as cvs commit lists, for which subject threading
672 makes little sense anyhow and where it should be turned off), but also since
673 the messages should be roughly pre sorted by date in the store already.
674 Some cursory benchmarking supports that assumption.
675 If now a parent is needed for a message with a certain subject, the list of
676 top level messages with that subject is traversed and the first one that is
677 older than our message is chosen as it's parent. Parents more than six weeks
678 older than the message are not accepted. The reasoning being that if a new
679 message with the same subject turns up after such a long time, the chances
680 that it is still part of the same thread are slim. The value of six weeks
681 is chosen as a result of a poll conducted on #kde-devel, so it's probably
682 bogus. :) All of this happens in the aptly named: findParentBySubject().
684 Everthing that still has no parent after this ends up at toplevel, no further
685 attemp is made at finding one. If you are reading this because you want to
686 implement threading by body search or somesuch, please go away, I don't like
687 you. :)
689 Ok, so far we have only operated on sort cache items, nothing ui wise has
690 happened yet. Now that we have established the parent/child relations of all
691 messages, it's time to create HeaderItems for them for use in the header
692 list. But wait, you say, what about sorting? Wouldn't it make sense to do
693 that first? Exactly, you're a clever bugger, ey? Here, have a cookie. ;)
694 Both creation of header items and sorting of the as of yet unsorted sort
695 cache items happen at the same time.
697 As previously mentioned (or not) each sort cache item holds a list of its
698 sorted and one of its unsorted children. Starting with the root node the
699 unsorted list is first qsorted, and then merged with the list of already
700 sorted children. To achieve that, the heads of both lists are compared and
701 the one with the "better" key is added to the list view next by creating a
702 KMHeaderListItem for it. That header item receives both its sort key as well
703 as its id from the sort cache item. Should the current sort cache item have
704 children, it is added to the end of a queue of nodes to repeat these steps
705 on after the current level is sorted. This way, a breadth first merge sort
706 is performed on the sort cache items and header items are created at each
707 node.
709 What follows is another iteration over all message ids in the folder, to
710 make sure that there are HeaderItems for each now. Should that not be the
711 case, top level emergency items are created. This loop is also used to add
712 all header items that are imperfectly threaded to a list so they can be
713 reevalutated when a new message arrives. Here the reverse mapping from
714 header items to sort cache items are made as well. Those are also necessary
715 so the sort cache item based data structures can be updated when a message
716 is removed.
718 The rest of readSortOrder should make sense on itself, I guess, if not, drop
719 me an email.
721 What happens when a message arrives in the folder?
722 Among other things, the msgAdded slot is called, which creates the necessary
723 sort cache item and header item for the new message and makes sure the data
724 structures described above are updated accordingly. If threading is enabled,
725 the new message is threaded using the same findParent and findParentBySubject
726 methods used on folder open. If the message ends up in a watched or ignored
727 thread, those status bits are inherited from the parent. The message is also
728 added to the dict of header items, the index of messages by message id and,
729 if applicable and if the message is threaded at top level, to the list of
730 potential parents for subject threading.
732 After those house keeping tasks are performed, the list of as of yet imper-
733 fectly threaded messages is traversed and our newly arrived message is
734 considered as a new parent for each item on it. This is especially important
735 to ensure that parents arriving out of order after their children still end
736 up as parents. If necessary, the entries in the .sorted file of rethreaded
737 messages are updated. An entry for the new message itself is appended to the
738 .sorted file as well.
740 Note that as an optimization newly arriving mail as part of a mailcheck in
741 an imap folder is not added via msgAdded, but rather via complete reload of
742 the folder via readSortOrder(). That's because only the headers are gotten
743 for each mail on imap and that is so fast, that adding them individually to
744 the list view is very slow by comparison. Thus we need to make sure that
745 writeSortOrder() is called whenever something related to sorting changes,
746 otherwise we read stale info from the .sorted file. The reload is triggered
747 by the folderComplete() signal of imap folders.
749 What happens when a message is removed from the folder?
750 In this case the msgRemoved slot kicks in and updates the headers list. First
751 the sort cache item and header item representing our message are removed from
752 the data structures and the ids of all items after it in the store decre-
753 mented. Then a list of children of the message is assembled containing those
754 children that have to be reparented now that our message has gone away. If
755 one of those children has been marked as toBeDeleted, it is simply added to
756 root at top level, because there is no need to find a parent for it if it is
757 to go away as well. This is an optimization to avoid rethreading all
758 messages in a thread when deleting several messages in a thread, or even the
759 whole thread. The KMMoveCommand marks all messages that will be moved out of
760 the folder as such so that can be detected here and the useless reparenting
761 can be avoided. Note that that does not work when moving messages via filter
762 action.
764 That list of children is then traversed and a new parent found for each one
765 using, again, findParent and findParentBySubject. When a message becomes
766 imperfectly threaded in the process, it is added to the corresponding list.
768 The message itself is removed from the list of imperfectly threaded messages
769 and its header item is finally deleted. The HeaderItem destructor destroys
770 the SortCacheItem as well, which is hopefully no longer referenced anywhere
771 at this point.
775 \section display Display (reader window - new)
777 Contact Marc Mutz <mutz@kde.org> with questions...
779 What happens when a message is to displayed in the reader window?
780 First, since KMMessagePart and KMMessage don't work with nested body
781 parts, a hierarchical tree of MIME body parts is constructed with
782 partNode's (being wrappers around KMMessagePart and
783 DwBodyPart). This is done in KMReaderWin::parseMsg(KMMessage*).
784 After some legacy handling, an ObjectTreeParser is instantiated and
785 it's parseObjectTree() method called on the root
786 partNode. ObjectTreeParser is the result of an ongoing refactoring
787 to enhance the design of the message display code. It's an
788 implementation of the Method Object pattern, used to break down a
789 huuuge method into many smaller ones without the need to pass around
790 a whole lot of paramters (those are made members
791 instead). parseObjectTree() recurses into the partNode tree and
792 generates an HTML representation of each node in the tree (although
793 some can be hidden by setting them to processed beforehand or - the
794 new way - by using AttachmentStrategy::Hidden). The HTML generation
795 is delegated to BodyPartFormatters, while the HTML is written to
796 HTMLWriters, which abstract away HTML sinks. One of those is
797 KHTMLPartHTMLWriter, which writes to the KHTMLPart in the
798 readerwindow. This is the current state of the code. The goal of the
799 ongoing refactoring is to make the HTML generation blissfully
800 ignorant of the readerwindow and to allow display plugins that can
801 operate against stable interfaces even though the internal KMail
802 classes change dramatically.
804 To this end, we designed a new set of interfaces that allows plugins
805 to be written that need or want to asynchronously update their HTML
806 representation. This set of interfaces consists of the following:
808 - @em BodyPartFormatterPlugin
809 the plugin base class. Allows the BodyPartFormatterFactory to
810 query it for an arbitray number of BodyPartFormatters and
811 their associated metadata and url handlers.
813 - @em BodyPartFormatter
814 the formatter interface. Contains a single method format()
815 that takes a BodyPart to process and a HTMLWriter to write the
816 generated HTML to and returns a result code with which it can
817 request more information or delegate the formatting back to
818 some default processor. BodyPartFormatters are meant to be
819 Flyweights, implying that the format() method is not allowed
820 to maintain state between calls, but see Mememto below.
822 - @em BodyPart
823 body part interface. Contains methods to retrieve the content
824 of a message part and some of the more interesting header
825 information. This interface decouples the bodypart formatters
826 from the implementation used in KMail (or KNode, for that
827 matter). Also contains methods to set and retrieve a Memento,
828 which is used by BodyPartFormatters to maintain state between
829 calls of format().
831 - @em BodyPartMemento
832 interface for opaque state storage and event handling.
833 Contains only a virtual destructor to enable the backend
834 processing code to destroy the object without the help of the
835 bodypart formatter.
838 During the design phase we identified a need for BodyPartFormatters to
839 request their being called on some form of events, e.g. a dcop
840 signal. Thus, the Memento interface also includes the IObserver and
841 ISubject interfaces. If a BodyPartFormatter needs to react to a signal
842 (Qt or DCOP), it implements the Memento interface using a QObject,
843 connects the signal to a slot on the Memento and (as an ISubject)
844 notifies it's IObservers when the slot is called. If a Memento is
845 created, the reader window registers itself as an observer of the
846 Memento and will magically invoke the corresponding BodyPartFormatter,
847 passing along the Memento to be retrieved from the BodyPart interface.
849 An example should make this clearer. Suppose, we want to update our
850 display after 10 seconds. Initially, we just write out an icon, and
851 after 10 seconds, we want to replace the icon by a "Hello world!"
852 line. The following code does exactly this:
854 @code
855 class DelayedHelloWorldBodyPartFormatter
856 : public KMail::BodyPartFormatter {
857 public:
858 Result format( KMail::BodyPart * bodyPart,
859 KMail::HTMLWriter * htmlWriter ) {
860 if ( !bodyPart->memento() ) {
861 bodyPart->registerMemento( new DelayedHelloWorldBodyPartMemento() );
862 return AsIcon;
863 } else {
864 htmlWriter->write( "Hello, world!" );
865 return Ok;
866 }
867 }
868 };
870 class DelayedHelloWorldBodyPartMemento
871 : public QObject, public KMail::BodyPartMemento {
872 public:
873 DelayedHelloWorldBodyPartMemento()
874 : QObject( 0, "DelayedHelloWorldBodyPartMemento" ),
875 KMail::BodyPartMemento()
876 {
877 QTimer::singleShot( 10*1000, this, SLOT(slotTimeout()) );
878 }
880 private slots:
881 void slotTimeout() { notify(): }
883 private:
884 // need to reimplement this abstract method...
885 bool update() { return true; }
886 };
887 @endcode
889 */
891 // DOXYGEN_REFERENCES = kdecore kdeui kio kparts kutils kde3support sonnet kresources kabc gpgme++ qgpgme kleo libkdepim
892 // DOXYGEN_SET_IGNORE_PREFIX = KM K