| blob | d3224004f15f65006191e50d19c4341babd1e328 |
1 ;;; spam.el --- Identifying spam
3 ;; Copyright (C) 2002-2016 Free Software Foundation, Inc.
5 ;; Author: Lars Magne Ingebrigtsen <larsi@gnus.org>
6 ;; Maintainer: Ted Zlatanov <tzz@lifelogs.com>
7 ;; Keywords: network, spam, mail, bogofilter, BBDB, dspam, dig, whitelist, blacklist, gmane, hashcash, spamassassin, bsfilter, ifile, stat, crm114, spamoracle
9 ;; This file is part of GNU Emacs.
11 ;; GNU Emacs is free software: you can redistribute it and/or modify
12 ;; it under the terms of the GNU General Public License as published by
13 ;; the Free Software Foundation, either version 3 of the License, or
14 ;; (at your option) any later version.
16 ;; GNU Emacs is distributed in the hope that it will be useful,
17 ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
18 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19 ;; GNU General Public License for more details.
21 ;; You should have received a copy of the GNU General Public License
22 ;; along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
24 ;;; Commentary:
26 ;;; This module addresses a few aspects of spam control under Gnus. Page
27 ;;; breaks are used for grouping declarations and documentation relating to
28 ;;; each particular aspect.
30 ;;; The integration with Gnus is not yet complete. See various `FIXME'
31 ;;; comments, below, for supplementary explanations or discussions.
33 ;;; Several TODO items are marked as such
35 ;; TODO: cross-server splitting, remote processing, training through files
37 ;;; Code:
39 ;;{{{ compilation directives and autoloads/requires
46 ;;; for the definitions of group content classification and spam processors
51 ;; for nnimap-split-download-body-default
54 ;; autoload query-dig
57 ;; autoload spam-report
63 ;; autoload gnus-registry
69 ;; autoload dns-query
72 ;;}}}
74 ;;{{{ Main parameters.
76 "List of spam.el backends with all the pertinent data.
80 "Spam configuration."
86 "Exit behavior at the time of summary exit.
87 Note that setting the `spam-use-move' or `spam-use-copy' backends on
88 a group through group/topic parameters overrides this mechanism."
92 default)
95 move-all)
98 move-none))
102 "Directory for spam whitelists and blacklists."
107 "Whether new messages in a spam group should get the spam-mark."
109 ;; :version "22.1" ;; Gnus 5.10.8 / No Gnus 0.3
113 "Whether spam/ham processing should be logged in the registry."
118 "Whether `spam-split' should work with symbols or group names."
123 "Whether `spam-split' should ALWAYS work with symbols or group names.
124 Do not set this if you use `spam-split' in a fancy split method."
129 "Whether only unseen articles should be marked as spam in spam groups.
130 When nil, all unread articles in a spam group are marked as
131 spam. Set this if you want to leave an article unread in a spam group
132 without losing it to the automatic spam-marking process."
137 "Whether ham should be marked unread before it's moved.
138 The article is moved out of a spam group according to `ham-process-destination'.
139 This variable is an official entry in the international Longest Variable Name
140 Competition."
145 "Whether `spam-split' should be ignored while resplitting ham.
146 This is useful to prevent ham from ending up in the same spam
147 group after the resplit. Don't set this to t if you have `spam-split' as the
148 last rule in your split configuration."
153 "Should spam.el recheck all messages when autodetecting?
154 Normally this is nil, so only unseen messages will be checked."
159 "The location of the whitelist.
160 The file format is one regular expression per line.
161 The regular expression is matched against the address."
166 "The location of the blacklist.
167 The file format is one regular expression per line.
168 The regular expression is matched against the address."
173 "Whether `query-dig' should be used instead of `dns-query'."
178 "Whether the Gmane spam xref should be used by `spam-split'."
183 "Whether the blacklist should be used by `spam-split'."
188 "Regular expressions that the blacklist should ignore."
193 "Whether the whitelist should be used by `spam-split'."
198 "Whether whitelist-exclusive should be used by `spam-split'.
199 Exclusive whitelisting means that all messages from senders not in the whitelist
200 are considered spam."
205 "Whether blackholes should be used by `spam-split'."
210 "Whether hashcash payments should be detected by `spam-split'."
215 "Whether a header regular expression match should be used by `spam-split'.
216 Also see the variables `spam-regex-headers-spam' and `spam-regex-headers-ham'."
221 "Whether a body regular expression match should be used by `spam-split'.
222 Also see the variables `spam-regex-body-spam' and `spam-regex-body-ham'."
227 "Whether bogofilter headers should be used by `spam-split'.
228 Enable this if you pre-process messages with Bogofilter BEFORE Gnus sees them."
233 "Whether bogofilter should be invoked by `spam-split'.
234 Enable this if you want Gnus to invoke Bogofilter on new messages."
239 "Whether bsfilter headers should be used by `spam-split'.
240 Enable this if you pre-process messages with Bsfilter BEFORE Gnus sees them."
245 "Whether bsfilter should be invoked by `spam-split'.
246 Enable this if you want Gnus to invoke Bsfilter on new messages."
251 "Whether BBDB should be used by `spam-split'."
256 "Whether BBDB-exclusive should be used by `spam-split'.
257 Exclusive BBDB means that all messages from senders not in the BBDB are
258 considered spam."
263 "Whether ifile should be used by `spam-split'."
268 "Whether `spam-stat' should be used by `spam-split'."
273 "Whether spamoracle should be used by `spam-split'."
278 "Whether spamassassin should be invoked by `spam-split'.
279 Enable this if you want Gnus to invoke SpamAssassin on new messages."
284 "Whether spamassassin headers should be checked by `spam-split'.
285 Enable this if you pre-process messages with SpamAssassin BEFORE Gnus sees
286 them."
291 "Whether the CRM114 Mailfilter should be used by `spam-split'."
296 spam-use-dig
297 spam-use-gmane-xref
298 spam-use-blacklist
299 spam-use-whitelist
300 spam-use-whitelist-exclusive
301 spam-use-blackholes
302 spam-use-hashcash
303 spam-use-regex-headers
304 spam-use-regex-body
305 spam-use-bogofilter
306 spam-use-bogofilter-headers
307 spam-use-spamassassin
308 spam-use-spamassassin-headers
309 spam-use-bsfilter
310 spam-use-bsfilter-headers
311 spam-use-BBDB
312 spam-use-BBDB-exclusive
313 spam-use-ifile
314 spam-use-stat
315 spam-use-spamoracle
316 spam-use-crm114)
317 "Whether the spam hooks should be installed.
318 Default to t if one of the spam-use-* variables is set."
323 "Group name where incoming spam should be put by `spam-split'."
327 ;;; TODO: deprecate this variable, it's confusing since it's a list of strings,
328 ;;; not regular expressions
330 spam-split-group
332 "Mailgroups with spam contents.
333 All unmarked article in such group receive the spam mark on group entry."
339 "The group where spam xrefs can be found on Gmane.
340 Only meaningful if you enable `spam-use-gmane-xref'."
346 "List of blackhole servers.
347 Only meaningful if you enable `spam-use-blackholes'."
352 "String matching IP addresses that should not be checked in the blackholes.
353 Only meaningful if you enable `spam-use-blackholes'."
367 "Face for spam-marked articles."
369 ;; backward-compatibility alias
374 "Face for spam-marked articles."
379 "Regular expression for positive header spam matches.
380 Only meaningful if you enable `spam-use-regex-headers'."
385 "Regular expression for positive header ham matches.
386 Only meaningful if you enable `spam-use-regex-headers'."
391 "Regular expression for positive body spam matches.
392 Only meaningful if you enable `spam-use-regex-body'."
397 "Regular expression for positive body ham matches.
398 Only meaningful if you enable `spam-use-regex-body'."
403 "Preferred header to use for `spam-summary-score'."
411 "Spam ifile configuration."
417 "Name of the ifile program."
425 "File name of the ifile database."
431 "Name of the spam ifile category."
436 "Name of the ham ifile category.
437 If nil, the current group name will be used."
443 "Whether the ifile check will return all categories, or just spam.
444 Set this to t if you want to use the `spam-split' invocation of ifile as
445 your main source of newsgroup names."
450 "Spam bogofilter configuration."
456 "Name of the Bogofilter program."
464 "The header that Bogofilter inserts in messages."
469 "The switch that Bogofilter uses to register spam messages."
474 "The switch that Bogofilter uses to register ham messages."
479 "The switch that Bogofilter uses to unregister ham messages."
484 "The switch that Bogofilter uses to unregister spam messages."
489 "The regex on `spam-bogofilter-header' for positive spam identification."
494 "Location of the Bogofilter database.
495 When nil, use the default location."
502 "Spam bsfilter configuration."
508 "Name of the Bsfilter program."
514 "The header inserted by Bsfilter to flag spam."
519 "The header that Bsfilter inserts in messages."
524 "The switch that Bsfilter uses to register spam messages."
529 "The switch that Bsfilter uses to register ham messages."
534 "The switch that Bsfilter uses to unregister ham messages."
539 "The switch that Bsfilter uses to unregister spam messages."
544 "Directory path of the Bsfilter databases."
551 "Spam spamoracle configuration."
555 "Location of spamoracle database file.
556 When nil, use the default spamoracle database."
562 "Location of the spamoracle binary."
568 "Spam SpamAssassin configuration."
574 "Name of the spamassassin program.
576 spamd man pages for more information on these programs."
582 "Arguments to pass to the spamassassin executable.
588 "The header inserted by SpamAssassin to flag spam."
593 "The regex on `spam-spamassassin-spam-flag-header' for positive spam
594 identification"
599 "The header inserted by SpamAssassin, giving extended scoring information"
606 "Name of the sa-learn program."
612 "Whether sa-learn should rebuild the database every time it is called
613 Enable this if you want sa-learn to rebuild the database automatically. Doing
614 this will slightly increase the running time of the spam registration process.
616 order for SpamAssassin to recognize the new registered spam."
621 "The switch that sa-learn uses to register spam messages."
626 "The switch that sa-learn uses to register ham messages."
631 "The switch that sa-learn uses to unregister messages messages."
636 "Spam CRM114 Mailfilter configuration."
640 "File path of the CRM114 Mailfilter executable program."
646 "The header that CRM114 Mailfilter inserts in messages."
651 "The switch that CRM114 Mailfilter uses to register spam messages."
656 "The switch that CRM114 Mailfilter uses to register ham messages."
661 "The switch that CRM114 Mailfilter uses to unregister ham messages."
666 "The switch that CRM114 Mailfilter uses to unregister spam messages."
671 "The regex on `spam-crm114-header' for positive spam identification."
676 "Directory path of the CRM114 Mailfilter databases."
682 ;;; Key bindings for spam control.
685 "St" spam-generic-score
686 "Sx" gnus-summary-mark-as-spam
687 "Mst" spam-generic-score
688 "Msx" gnus-summary-mark-as-spam
689 "\M-d" gnus-summary-mark-as-spam
707 "Internal variable.
708 `spam-split' will set this to nil or a spam-use-XYZ check if it
711 ;; internal variables for backends
712 ;; TODO: find a way to create these on the fly in spam-install-backend-super
718 ;;}}}
720 ;;{{{ convenience functions
723 "Clear the `spam-caches' entry for a check."
727 "Logical A xor B."
731 "Return a set difference of LIST1 and LIST2.
732 When either list is nil, the other is returned."
734 ;; we have two non-nil lists
741 ;; if either of the lists was nil, return the other one
745 "Checks if MARK is considered a ham mark in GROUP."
749 marks
754 "Checks if MARK is considered a spam mark in GROUP."
758 "In GROUP, get all the ham marks."
765 marks)))
768 "In GROUP, get all the spam marks."
772 "Is GROUP a spam group?"
777 nil))
780 "Is GROUP a ham group?"
784 nil))
787 "Return list of valid classifications"
791 "Is CLASSIFICATION a valid spam/ham classification?"
795 "Return list of valid classifications."
799 "Is PROPERTY a valid backend property?"
812 'spam-group-spam-mark-p
814 alist mark-cache-yes mark-cache-no)
820 gnus-newsgroup-name
821 mark)
826 alist))
828 ;;}}}
830 ;;{{{ backend installation functions and procedures
833 "Install BACKEND for spam.el.
834 Accepts incoming CHECK, ham registration function HRF, spam
835 registration function SRF, ham unregistration function HUF, spam
836 unregistration function SUF, and an indication whether the
837 backend is STATISTICAL."
845 5
846 "spam-install-backend-super got an invalid property %s"
847 property)))))
850 "Return a list of all the backend symbols, constrained by TYPE.
851 When TYPE is 'non-mover, only non-mover backends are returned.
852 When TYPE is 'mover, only mover backends are returned."
857 ;; or the type is 'mover and the backend is a mover
861 ;; or the type is 'non-mover and the backend is not a mover
866 list))
869 "Get the check function for BACKEND.
870 Each individual check may return nil, t, or a mailgroup name.
871 The value nil means that the check does not yield a decision, and
872 so, that further checks are needed. The value t means that the
873 message is definitely not spam, and that further spam checks
874 should be inhibited. Otherwise, a mailgroup name or the symbol
875 'spam (depending on `spam-split-symbolic-return') is returned where
876 the mail should go, and further checks are also inhibited. The
877 usual mailgroup name is the value of `spam-split-group', meaning
878 that the message is definitely a spam."
882 "Is BACKEND valid?"
886 "Return information about BACKEND."
890 backend))
893 info
894 property
896 info)
898 backend)))
901 "Get the BACKEND function for CLASSIFICATION and TYPE.
902 TYPE is 'registration or 'unregistration.
903 CLASSIFICATION is 'ham or 'spam."
910 classification
911 type))))
914 5
915 "%s was passed invalid backend %s, classification %s, or type %s"
916 "spam-backend-function"
917 backend
918 classification
919 type)))
923 "Property name of article list with CLASSIFICATION and UNREGISTER."
926 prop))
929 classification
931 "Get the articles to be processed for BACKEND and CLASSIFICATION.
932 With UNREGISTER, get articles to be unregistered.
933 This is a temporary storage function - nothing here persists."
935 backend
940 "Set the LIST of articles to be processed for BACKEND and CLASSIFICATION.
941 With UNREGISTER, set articles to be unregistered.
942 This is a temporary storage function - nothing here persists."
944 backend
946 list))
949 "Get the ham registration function for BACKEND."
953 "Get the spam registration function for BACKEND."
957 "Get the ham unregistration function for BACKEND."
961 "Get the spam unregistration function for BACKEND."
965 "Is BACKEND statistical?"
969 "Is BACKEND a mover?"
973 "Add ALIAS to an existing BACKEND.
974 The previous backend settings for ALIAS are erased."
976 ;; install alias with no properties at first
983 "Install a BACKEND than can only CHECK for spam."
987 "Install a BACKEND than can move articles at summary exit.
988 Accepts ham registration function HRF, spam registration function
989 SRF, ham unregistration function HUF, spam unregistration
990 function SUF. The backend has no incoming check and can't be
991 statistical."
993 backend
997 "Install a BACKEND than has no check.
998 Accepts ham registration function HRF, spam registration function
999 SRF, ham unregistration function HUF, spam unregistration
1000 function SUF. The backend has no incoming check and can't be
1001 statistical (it could be, but in practice that doesn't happen)."
1003 backend
1007 "Install a BACKEND.
1008 Accepts incoming CHECK, ham registration function HRF, spam
1009 registration function SRF, ham unregistration function HUF, spam
1010 unregistration function SUF. The backend won't be
1011 statistical (use `spam-install-statistical-backend' for that)."
1013 backend
1017 "Install a BACKEND.
1018 Accepts incoming CHECK, ham registration function HRF, spam
1019 registration function SRF, ham unregistration function HUF, spam
1020 unregistration function SUF. The backend will be
1021 statistical (use `spam-install-backend' for non-statistical
1022 backends)."
1024 backend
1028 "Install a statistical BACKEND than can only CHECK for spam."
1030 backend
1033 ;;}}}
1035 ;;{{{ backend installations
1060 ;; TODO: NOTE: spam-use-ham-copy is now obsolete, use (ham spam-use-copy)
1062 'spam-move-ham-routine
1063 'spam-move-spam-routine
1064 nil
1065 nil)
1068 'spam-copy-ham-routine
1069 'spam-copy-spam-routine
1070 nil
1071 nil)
1074 'spam-report-gmane-unregister-routine
1075 'spam-report-gmane-register-routine
1076 'spam-report-gmane-register-routine
1080 'spam-report-resend-register-ham-routine
1081 'spam-report-resend-register-routine
1082 nil
1083 nil)
1086 'spam-check-BBDB
1087 'spam-BBDB-register-routine
1088 nil
1089 'spam-BBDB-unregister-routine
1090 nil)
1095 'spam-check-blacklist
1096 nil
1097 'spam-blacklist-register-routine
1098 nil
1102 'spam-check-whitelist
1103 'spam-whitelist-register-routine
1104 nil
1105 'spam-whitelist-unregister-routine
1106 nil)
1109 'spam-check-ifile
1110 'spam-ifile-register-ham-routine
1111 'spam-ifile-register-spam-routine
1112 'spam-ifile-unregister-ham-routine
1116 'spam-check-spamoracle
1117 'spam-spamoracle-learn-ham
1118 'spam-spamoracle-learn-spam
1119 'spam-spamoracle-unlearn-ham
1123 'spam-check-stat
1124 'spam-stat-register-ham-routine
1125 'spam-stat-register-spam-routine
1126 'spam-stat-unregister-ham-routine
1130 'spam-check-spamassassin
1131 'spam-spamassassin-register-ham-routine
1132 'spam-spamassassin-register-spam-routine
1133 'spam-spamassassin-unregister-ham-routine
1137 'spam-check-bogofilter
1138 'spam-bogofilter-register-ham-routine
1139 'spam-bogofilter-register-spam-routine
1140 'spam-bogofilter-unregister-ham-routine
1144 'spam-check-bsfilter
1145 'spam-bsfilter-register-ham-routine
1146 'spam-bsfilter-register-spam-routine
1147 'spam-bsfilter-unregister-ham-routine
1151 'spam-check-crm114
1152 'spam-crm114-register-ham-routine
1153 'spam-crm114-register-spam-routine
1154 'spam-crm114-unregister-ham-routine
1156 ;;}}}
1158 ;;{{{ scoring and summary formatting
1160 "Return the extra headers spam.el thinks are necessary."
1163 spam-use-spamassassin-headers
1164 spam-use-regex-headers)
1167 spam-use-regex-headers)
1170 spam-use-regex-headers)
1172 list))
1180 "Sort articles by score."
1188 result))
1192 "Regexp matching SpamAssassin score header.
1196 "Transform an extra HEADER to a number, using list of HEADERS.
1197 Note this has to be fast."
1203 spam-spamassassin-score-regexp
1205 header-content)))
1206 ;; for CRM checking, it's probably faster to just do the string match
1214 header-content))))
1216 nil)))
1219 "Score an article for the summary buffer, as fast as possible.
1220 With SPECIFIC-HEADER, returns only that header's score.
1221 Will not return a nil score."
1234 "Invoke whatever scoring method we can."
1244 ;;}}}
1246 ;;{{{ set up widening, processor checks
1248 ;;; set up IMAP widening if it's necessary
1260 found))
1263 ;; note the nil processors are not defined in gnus.el
1281 "The OBSOLETE `spam-list-of-processors' list.
1282 This list contains pairs associating the obsolete ham/spam exit
1283 processor variables with a classification and a spam-use-*
1284 variable. When the processor variable is nil, just the
1285 classification and spam-use-* check variable are used. This is
1286 superseded by the new spam backend code, so it's only consulted
1291 "Checks if GROUP has a BACKEND with CLASSIFICATION registered.
1292 Also accepts the obsolete processors, which can be found in
1293 gnus.el and in spam-list-of-processors. In the case of mover
1294 backends, checks the setting of `spam-summary-exit-behavior' in
1295 addition to the set values for the group."
1300 found)
1303 ;; now search for the parameter
1311 ;; now, if the parameter was not found, do the
1312 ;; spam-summary-exit-behavior-logic for mover backends
1316 found
1322 ;; move ham out of spam groups
1326 spam-summary-exit-behavior))))))
1328 found))
1329 nil))
1331 ;;}}}
1333 ;;{{{ Summary entry and exit processing.
1336 ;; check the global list of group names spam-junk-mailgroups and the
1337 ;; group parameters
1341 "unseen"
1344 gnus-newsgroup-unseen
1345 gnus-newsgroup-unreads)))
1357 ;; The spam processors are invoked for any group, spam or ham or neither
1362 ;; before we begin, remove any article limits
1363 ; (ignore-errors
1364 ; (gnus-summary-pop-limit t))
1366 ;; first of all, unregister any articles that are no longer ham or spam
1367 ;; we have to iterate over the processors, or else we'll be too slow
1371 gnus-newsgroup-articles
1372 classification))
1374 ;; now that we have the changed articles, we go through the processors
1382 ;; call spam-register-routine with specific articles to unregister,
1383 ;; when there are articles to unregister and the check is enabled
1386 classification
1387 unregister-list
1388 t))))))
1390 ;; do the non-moving backends first, then the moving ones
1395 gnus-newsgroup-name
1396 backend
1397 classification)
1399 classification
1401 gnus-newsgroup-articles
1402 classification))))))
1406 ;; we mark all the leftover spam articles as expired at the end
1408 gnus-newsgroup-articles
1414 ;;}}}
1416 ;;{{{ spam-use-move and spam-use-copy backend support functions
1426 ;; remove the current process mark
1433 article mark deletep respool valid-move-destinations)
1439 ;; exclude invalid move destinations
1448 3
1449 "Sorry, can't move spam from spam group %s to spam group %s"
1450 gnus-newsgroup-name
1451 group))
1457 3
1458 "Sorry, can't move ham from ham group %s to ham group %s"
1459 gnus-newsgroup-name
1460 group)))
1465 ;; now do the actual move
1470 ;; first, mark the article with the process mark and, if needed,
1471 ;; the unread or expired mark (for ham and spam respectively)
1474 spam-mark-ham-unread-before-move-from-spam-group)
1476 article)
1481 article)
1489 spam-disable-spam-split-during-ham-respool))))
1491 article respool-method)
1493 ;; else, we are not respooling
1498 article group)
1502 article group)
1505 ;; now delete the articles, unless a) copy is t, and there was a copy done
1506 ;; b) a move was done to a single group
1507 ;; c) backend-supports-deletions is nil
1521 t
1523 articles
1528 nil
1530 articles
1535 t
1537 articles
1542 nil
1544 articles
1547 ;;}}}
1549 ;;{{{ article and field retrieval code
1554 article
1555 gnus-newsgroup-name)
1558 ;; disabled for now
1559 ;; (defun spam-get-article-as-filename (article)
1560 ;; (let ((article-filename))
1561 ;; (when (numberp article)
1562 ;; (nnml-possibly-change-directory
1563 ;; (gnus-group-real-name gnus-newsgroup-name))
1564 ;; (setq article-filename (expand-file-name
1565 ;; (int-to-string article) nnml-current-directory)))
1566 ;; (if (file-exists-p article-filename)
1567 ;; article-filename
1568 ;; nil)))
1571 "Fetch a FIELD for ARTICLE with the internal `gnus-data-list' function.
1572 When PREPARED-DATA-HEADER is given, don't look in the Gnus data.
1573 When FIELD is 'number, ARTICLE can be any number (since we want
1574 to find it out)."
1599 5
1600 "spam-fetch-field-fast: unknown field %s requested"
1601 field)
1602 nil)))))
1618 ;; 80-character limit makes for strange constructs
1630 5
1631 "spam-generate-fake-headers: article %d didn't have a valid header"
1632 article))))
1638 ;;}}}
1640 ;;{{{ Spam determination.
1643 "Split this message into the `spam' group if it is spam.
1644 This function can be used as an entry in the variable `nnmail-split-fancy',
1645 for example like this: (: spam-split). It can take checks as
1646 parameters. A string as a parameter will set the
1647 `spam-split-group' to that string.
1649 See the Info node `(gnus)Fancy Mail Splitting' for more details."
1666 widening-needed-check))
1668 decision)
1673 'spam
1674 spam-split-group)))
1676 ;; either, given specific checks, this is one of them
1678 ;; or, given no specific checks, spam-use-CHECK is set
1681 check-function)
1683 ;; if we got a decision at all, save the current check
1690 5
1692 decision
1693 spam-split-symbolic-return)))))))
1696 decision))))))))
1699 "Detect spam in the current newsgroup using `spam-split'."
1707 gnus-newsgroup-articles
1708 gnus-newsgroup-unseen))
1709 article-cannot-be-faked)
1727 registry-lookup)
1736 9
1737 "spam-find-spam: message %s was already registered incoming"
1738 id)))
1748 article
1749 group)
1750 ;; else, we fake the article
1765 id
1766 'incoming
1767 split-return
1768 spam-split-last-successful-check
1769 group))))))
1770 articles))))
1772 ;;}}}
1774 ;;{{{ registration/unregistration functions
1777 "Go through the backends and register or unregister articles as needed."
1782 backend classification))
1784 backend classification t))
1787 ;; clear the old lists right away
1789 classification
1790 nil
1791 nil)
1793 classification
1794 nil
1795 t)
1797 ;; eliminate duplicates
1806 9
1807 "%d messages did not have to unregister and then register"
1808 delcount))
1810 ;; unregister articles
1815 6
1816 "%d %s messages were unregistered by backend %s."
1817 num
1818 classification
1819 backend))))
1821 ;; register articles
1826 6
1827 "%d %s messages were registered by backend %s."
1828 num
1829 classification
1830 backend)))))))))
1833 backend
1834 specific-articles)
1838 backend
1839 specific-articles
1848 unregister-function
1849 register-function))
1851 'spam-log-undo-registration
1853 article articles)
1856 ;; make list of articles, using specific-articles if given
1859 gnus-newsgroup-articles
1860 classification)))
1861 ;; process them
1867 classification
1868 backend)
1870 ;; now log all the registrations (or undo them, depending on
1871 ;; unregister)
1875 'process
1876 classification
1877 backend
1878 gnus-newsgroup-name))))
1879 ;; return the number of articles processed
1882 ;;; log a ham- or spam-processor invocation to the registry
1896 7
1898 "%s call with bad ID, type, classification, spam-backend, or group"
1901 ;;; check if a ham- or spam-processor registration has been done
1909 7
1912 nil))))
1914 ;;; check what a ham- or spam-processor registration says
1915 ;;; returns nil if conflicting registrations are found
1918 decision)
1927 nil
1928 decision)))
1931 ;;; check if a ham- or spam-processor registration needs to be undone
1939 found)
1945 found)
1948 7
1951 nil))))
1954 ;;; undo a ham- or spam-processor registration (the group is not used)
1964 new-cell-list found)
1972 "%s call with bad ID, type, spam-backend, or group"
1974 nil))))
1976 ;;}}}
1978 ;;{{{ backend functions
1980 ;;{{{ Gmane xrefs
1987 header)
1988 spam-split-group))))
1990 ;;}}}
1992 ;;{{{ Regex body
1999 ;;}}}
2001 ;;{{{ Regex headers
2005 ret found)
2019 ret))
2021 ;;}}}
2023 ;;{{{ Blackholes.
2032 "Check the Received headers for blackholed relays."
2034 ips matches)
2044 ips)))
2048 ;; match against the reversed (again) IP string
2050 spam-blackhole-good-server-regex
2058 query-result)
2060 matches)))
2061 ;; else, if not using dig.el
2065 matches)))))))))
2067 spam-split-group)))
2068 ;;}}}
2070 ;;{{{ Hashcash.
2073 "Check the headers for hashcash payments."
2076 ;;}}}
2078 ;;{{{ BBDB
2080 ;;; original idea for spam-check-BBDB from Alexander Kotelnikov
2081 ;;; <sacha@giotto.sj.ru>
2083 ;; all this is done inside a condition-case to trap errors
2085 ;; Autoloaded in message, which we require.
2094 ;; `bbdb-records' should not be bound as an autoload function
2095 ;; before loading bbdb because of `bbdb-hashtable-size'.
2103 nil)))
2107 ;; when the BBDB changes, we want to clear out our cache
2114 "Enter an address into the BBDB; implies ham (non-spam) sender"
2120 'bbdb-delete-record-internal
2128 from
2140 ;; now do the register/unregister action
2152 "Mail from people in the BBDB is classified as ham or non-spam"
2157 t
2159 spam-split-group
2160 nil)))))))
2162 ;;}}}
2164 ;;{{{ ifile
2166 ;;; check the ifile backend; return nil if the mail was NOT classified
2167 ;;; as spam
2171 "Return the command-line parameter for ifile's database.
2172 See `spam-ifile-database'."
2175 nil))
2178 "Check the ifile backend for the classification of this message."
2180 category return)
2187 nil temp-buffer-name nil "-c"
2189 ;; check the return now (we're back in the temp buffer)
2196 ;; else, if spam-ifile-all-categories is not set...
2199 return))
2202 "Register an article, given as a string, with a category.
2203 Uses `gnus-newsgroup-name' if category is nil (for ham registration)."
2207 parameters)
2215 nil nil nil
2216 add-or-delete-option category
2231 ;;}}}
2233 ;;{{{ spam-stat
2238 "Check the spam-stat backend for the classification of this message"
2241 category return)
2274 ;;}}}
2276 ;;{{{ Blacklists and whitelists.
2286 ;;; address can be a list, too
2288 "Enter ADDRESS (list or single) into the whitelist.
2289 With a non-nil REMOVE, remove them."
2295 ;;; address can be a list, too
2297 "Enter ADDRESS (list or single) into the blacklist.
2298 With a non-nil REMOVE, remove them."
2305 "Enter ADDRESSES into the given FILE.
2306 Either the whitelist or the blacklist files can be used.
2307 With a non-nil REMOVE, remove the ADDRESSES."
2310 ;; else, we have a list of addresses here
2319 ;; found the address
2322 ;; else, the address was not found
2332 spam-blacklist-cache
2333 spam-whitelist-cache))
2334 parsed-cache)
2340 ;; fix regexp-quote's treatment of user-intended regexes
2354 found)))
2356 ;;; returns t if the sender is in the whitelist, nil or
2357 ;;; spam-split-group otherwise
2359 ;; FIXME! Should it detect when file timestamps change?
2363 t
2365 spam-split-group
2366 nil)))
2369 ;; FIXME! Should it detect when file timestamps change?
2373 spam-split-group))
2383 ;; insert the e-mail address if detected, otherwise the raw data
2392 found)
2402 from addresses unregister-list article-unregister-list)
2406 sender-ignored)
2413 ;; remember the messages we need to unregister, unless remove is set
2425 ;; else, register normally and unregister what we need to
2430 'process
2431 declassification
2432 de-symbol))
2447 ;;}}}
2449 ;;{{{ Spam-report glue (gmane and resend reporting)
2470 spam-report-resend-to)))
2473 ;;}}}
2475 ;;{{{ Bogofilter
2483 ;; spam detection mode
2485 header)
2486 spam-split-group)))))
2488 ;; return something sensible if the score can't be determined
2490 "Get the Bogofilter spamicity score."
2503 "Verify the Bogofilter version is sufficient."
2509 spam-bogofilter-valid)
2512 "Check the Bogofilter backend for the classification of this message."
2516 return)
2522 spam-bogofilter-program
2523 nil temp-buffer-name nil
2526 return)
2530 spam
2532 "Register an article, given as a string, as spam or non-spam."
2539 spam-bogofilter-spam-strong-switch
2540 spam-bogofilter-ham-strong-switch)
2542 spam-bogofilter-spam-switch
2543 spam-bogofilter-ham-switch))))
2550 spam-bogofilter-program
2551 nil nil nil switch
2568 ;;}}}
2570 ;;{{{ spamoracle
2572 "Run spamoracle on an article to determine whether it's spam."
2580 spam-spamoracle-binary
2581 nil temp-buffer-name nil
2590 spam-split-group))
2594 "Run spamoracle in training mode."
2602 "-spam"
2607 spam-spamoracle-binary
2608 nil temp-buffer-name nil
2628 ;;}}}
2630 ;;{{{ SpamAssassin
2631 ;;; based mostly on the bogofilter code
2633 "Check the SpamAssassin headers for the classification of this message."
2640 ;; spam detection mode
2644 header)
2645 spam-split-group)))))
2648 "Check the SpamAssassin backend for the classification of this message."
2655 nil temp-buffer-name nil spam-spamassassin-arguments))
2656 ;; check the return now (we're back in the temp buffer)
2660 ;; return something sensible if the score can't be determined
2662 "Get the SpamAssassin score"
2676 "Register articles with spamassassin's sa-learn as spam or non-spam."
2680 spam-sa-learn-ham-switch)))
2683 ;; group the articles into mbox format
2689 ;; mbox separator
2693 ;; call sa-learn on all messages at the same time
2696 spam-sa-learn-program
2697 nil nil nil "--mbox"
2714 ;;}}}
2716 ;;{{{ Bsfilter
2717 ;;; based mostly on the bogofilter code
2725 spam-split-group)))))
2727 ;; return something sensible if the score can't be determined
2729 "Get the Bsfilter spamicity score."
2742 "Check the Bsfilter backend for the classification of this message."
2745 return)
2751 spam-bsfilter-program
2752 nil temp-buffer-name nil
2753 "--pipe"
2754 "--insert-flag"
2755 "--insert-probability"
2759 return))
2762 spam
2764 "Register an article, given as a string, as spam or non-spam."
2769 spam-bsfilter-spam-strong-switch
2770 spam-bsfilter-ham-strong-switch)
2772 spam-bsfilter-spam-switch
2773 spam-bsfilter-ham-switch))))
2779 spam-bsfilter-program
2780 nil nil nil switch
2781 "--update"
2784 spam-bsfilter-database-directory))))))))
2798 ;;}}}
2800 ;;{{{ CRM114 Mailfilter
2808 ;; spam detection mode
2810 header)
2811 spam-split-group)))))
2813 ;; return something sensible if the score can't be determined
2815 "Get the CRM114 Mailfilter pR."
2827 "Check the CRM114 Mailfilter backend for the classification of this message."
2830 return)
2836 spam-crm114-program
2837 nil temp-buffer-name nil
2840 return))
2843 spam
2845 "Register an article, given as a string, as spam or non-spam."
2851 spam-crm114-spam-strong-switch
2852 spam-crm114-ham-strong-switch)
2854 spam-crm114-spam-switch
2855 spam-crm114-ham-switch))))
2862 spam-crm114-program
2863 nil nil nil
2878 ;;}}}
2880 ;;}}}
2882 ;;{{{ Hooks
2884 ;;;###autoload
2886 "Install the spam.el hooks and do other initialization.
2887 When SYMBOLS is given, set those variables to t. This is so you
2888 can call `spam-initialize' before you set spam-use-* variables on
2889 explicitly, and matters only if you need the extra headers
2890 installed through `spam-necessary-extra-headers'."
2901 ;; TODO: How do we redo this every time the `spam' face is customized?
2903 gnus-summary-highlight)
2904 ;; Add hooks for loading and saving the spam stats
2912 ;; Don't install things more than once.
2916 "Uninstall the spam.el hooks."
2928 ;;}}}
2932 ;;; spam.el ends here