| blob | de8aa77efc9ea5ee607a6a2918adb340cba23f9e |
1 ;;; spam.el --- Identifying spam
3 ;; Copyright (C) 2002-2018 Free Software Foundation, Inc.
5 ;; Author: Lars Magne Ingebrigtsen <larsi@gnus.org>
6 ;; Maintainer: Ted Zlatanov <tzz@lifelogs.com>
7 ;; Keywords: network, spam, mail, bogofilter, BBDB, dspam, dig, whitelist, blacklist, gmane, hashcash, spamassassin, bsfilter, ifile, stat, crm114, spamoracle
9 ;; This file is part of GNU Emacs.
11 ;; GNU Emacs is free software: you can redistribute it and/or modify
12 ;; it under the terms of the GNU General Public License as published by
13 ;; the Free Software Foundation, either version 3 of the License, or
14 ;; (at your option) any later version.
16 ;; GNU Emacs is distributed in the hope that it will be useful,
17 ;; but WITHOUT ANY WARRANTY; without even the implied warranty of
18 ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19 ;; GNU General Public License for more details.
21 ;; You should have received a copy of the GNU General Public License
22 ;; along with GNU Emacs. If not, see <https://www.gnu.org/licenses/>.
24 ;;; Commentary:
26 ;;; This module addresses a few aspects of spam control under Gnus. Page
27 ;;; breaks are used for grouping declarations and documentation relating to
28 ;;; each particular aspect.
30 ;;; The integration with Gnus is not yet complete. See various `FIXME'
31 ;;; comments, below, for supplementary explanations or discussions.
33 ;;; Several TODO items are marked as such
35 ;; TODO: cross-server splitting, remote processing, training through files
37 ;;; Code:
39 ;;{{{ compilation directives and autoloads/requires
46 ;;; for the definitions of group content classification and spam processors
51 ;; for nnimap-split-download-body-default
54 ;; autoload query-dig
57 ;; autoload spam-report
63 ;; autoload gnus-registry
69 ;; autoload dns-query
72 ;;}}}
74 ;;{{{ Main parameters.
76 "List of spam.el backends with all the pertinent data.
80 "Spam configuration."
86 "Exit behavior at the time of summary exit.
87 Note that setting the `spam-use-move' or `spam-use-copy' backends on
88 a group through group/topic parameters overrides this mechanism."
92 default)
95 move-all)
98 move-none))
102 "Directory for spam whitelists and blacklists."
107 "Whether new messages in a spam group should get the spam-mark."
109 ;; :version "22.1" ;; Gnus 5.10.8 / No Gnus 0.3
113 "Whether spam/ham processing should be logged in the registry."
118 "Whether `spam-split' should work with symbols or group names."
123 "Whether `spam-split' should ALWAYS work with symbols or group names.
124 Do not set this if you use `spam-split' in a fancy split method."
129 "Whether only unseen articles should be marked as spam in spam groups.
130 When nil, all unread articles in a spam group are marked as
131 spam. Set this if you want to leave an article unread in a spam group
132 without losing it to the automatic spam-marking process."
137 "Whether ham should be marked unread before it's moved.
138 The article is moved out of a spam group according to `ham-process-destination'.
139 This variable is an official entry in the international Longest Variable Name
140 Competition."
145 "Whether `spam-split' should be ignored while resplitting ham.
146 This is useful to prevent ham from ending up in the same spam
147 group after the resplit. Don't set this to t if you have `spam-split' as the
148 last rule in your split configuration."
153 "Should spam.el recheck all messages when autodetecting?
154 Normally this is nil, so only unseen messages will be checked."
159 "The location of the whitelist.
160 The file format is one regular expression per line.
161 The regular expression is matched against the address."
166 "The location of the blacklist.
167 The file format is one regular expression per line.
168 The regular expression is matched against the address."
173 "Whether `query-dig' should be used instead of `dns-query'."
178 "Whether the Gmane spam xref should be used by `spam-split'."
183 "Whether the blacklist should be used by `spam-split'."
188 "Regular expressions that the blacklist should ignore."
193 "Whether the whitelist should be used by `spam-split'."
198 "Whether whitelist-exclusive should be used by `spam-split'.
199 Exclusive whitelisting means that all messages from senders not in the whitelist
200 are considered spam."
205 "Whether blackholes should be used by `spam-split'."
210 "Whether hashcash payments should be detected by `spam-split'."
215 "Whether a header regular expression match should be used by `spam-split'.
216 Also see the variables `spam-regex-headers-spam' and `spam-regex-headers-ham'."
221 "Whether a body regular expression match should be used by `spam-split'.
222 Also see the variables `spam-regex-body-spam' and `spam-regex-body-ham'."
227 "Whether bogofilter headers should be used by `spam-split'.
228 Enable this if you pre-process messages with Bogofilter BEFORE Gnus sees them."
233 "Whether bogofilter should be invoked by `spam-split'.
234 Enable this if you want Gnus to invoke Bogofilter on new messages."
239 "Whether bsfilter headers should be used by `spam-split'.
240 Enable this if you pre-process messages with Bsfilter BEFORE Gnus sees them."
245 "Whether bsfilter should be invoked by `spam-split'.
246 Enable this if you want Gnus to invoke Bsfilter on new messages."
251 "Whether BBDB should be used by `spam-split'."
256 "Whether BBDB-exclusive should be used by `spam-split'.
257 Exclusive BBDB means that all messages from senders not in the BBDB are
258 considered spam."
263 "Whether ifile should be used by `spam-split'."
268 "Whether `spam-stat' should be used by `spam-split'."
273 "Whether spamoracle should be used by `spam-split'."
278 "Whether spamassassin should be invoked by `spam-split'.
279 Enable this if you want Gnus to invoke SpamAssassin on new messages."
284 "Whether spamassassin headers should be checked by `spam-split'.
285 Enable this if you pre-process messages with SpamAssassin BEFORE Gnus sees
286 them."
291 "Whether the CRM114 Mailfilter should be used by `spam-split'."
296 spam-use-dig
297 spam-use-gmane-xref
298 spam-use-blacklist
299 spam-use-whitelist
300 spam-use-whitelist-exclusive
301 spam-use-blackholes
302 spam-use-hashcash
303 spam-use-regex-headers
304 spam-use-regex-body
305 spam-use-bogofilter
306 spam-use-bogofilter-headers
307 spam-use-spamassassin
308 spam-use-spamassassin-headers
309 spam-use-bsfilter
310 spam-use-bsfilter-headers
311 spam-use-BBDB
312 spam-use-BBDB-exclusive
313 spam-use-ifile
314 spam-use-stat
315 spam-use-spamoracle
316 spam-use-crm114)
317 "Whether the spam hooks should be installed.
318 Default to t if one of the spam-use-* variables is set."
323 "Group name where incoming spam should be put by `spam-split'."
327 ;;; TODO: deprecate this variable, it's confusing since it's a list of strings,
328 ;;; not regular expressions
330 spam-split-group
332 "Mailgroups with spam contents.
333 All unmarked article in such group receive the spam mark on group entry."
339 "The group where spam xrefs can be found on Gmane.
340 Only meaningful if you enable `spam-use-gmane-xref'."
346 "List of blackhole servers.
347 Only meaningful if you enable `spam-use-blackholes'."
352 "String matching IP addresses that should not be checked in the blackholes.
353 Only meaningful if you enable `spam-use-blackholes'."
367 "Face for spam-marked articles."
371 "Face for spam-marked articles."
376 "Regular expression for positive header spam matches.
377 Only meaningful if you enable `spam-use-regex-headers'."
382 "Regular expression for positive header ham matches.
383 Only meaningful if you enable `spam-use-regex-headers'."
388 "Regular expression for positive body spam matches.
389 Only meaningful if you enable `spam-use-regex-body'."
394 "Regular expression for positive body ham matches.
395 Only meaningful if you enable `spam-use-regex-body'."
400 "Preferred header to use for `spam-summary-score'."
408 "Spam ifile configuration."
414 "Name of the ifile program."
422 "File name of the ifile database."
428 "Name of the spam ifile category."
433 "Name of the ham ifile category.
434 If nil, the current group name will be used."
440 "Whether the ifile check will return all categories, or just spam.
441 Set this to t if you want to use the `spam-split' invocation of ifile as
442 your main source of newsgroup names."
447 "Spam bogofilter configuration."
453 "Name of the Bogofilter program."
461 "The header that Bogofilter inserts in messages."
466 "The switch that Bogofilter uses to register spam messages."
471 "The switch that Bogofilter uses to register ham messages."
476 "The switch that Bogofilter uses to unregister ham messages."
481 "The switch that Bogofilter uses to unregister spam messages."
486 "The regex on `spam-bogofilter-header' for positive spam identification."
491 "Location of the Bogofilter database.
492 When nil, use the default location."
499 "Spam bsfilter configuration."
505 "Name of the Bsfilter program."
511 "The header inserted by Bsfilter to flag spam."
516 "The header that Bsfilter inserts in messages."
521 "The switch that Bsfilter uses to register spam messages."
526 "The switch that Bsfilter uses to register ham messages."
531 "The switch that Bsfilter uses to unregister ham messages."
536 "The switch that Bsfilter uses to unregister spam messages."
541 "Directory path of the Bsfilter databases."
548 "Spam spamoracle configuration."
552 "Location of spamoracle database file.
553 When nil, use the default spamoracle database."
559 "Location of the spamoracle binary."
565 "Spam SpamAssassin configuration."
571 "Name of the spamassassin program.
573 spamd man pages for more information on these programs."
579 "Arguments to pass to the spamassassin executable.
585 "The header inserted by SpamAssassin to flag spam."
590 "The regex on `spam-spamassassin-spam-flag-header' for positive spam
591 identification"
596 "The header inserted by SpamAssassin, giving extended scoring information"
603 "Name of the sa-learn program."
609 "Whether sa-learn should rebuild the database every time it is called
610 Enable this if you want sa-learn to rebuild the database automatically. Doing
611 this will slightly increase the running time of the spam registration process.
613 order for SpamAssassin to recognize the new registered spam."
618 "The switch that sa-learn uses to register spam messages."
623 "The switch that sa-learn uses to register ham messages."
628 "The switch that sa-learn uses to unregister messages."
633 "Spam CRM114 Mailfilter configuration."
637 "File path of the CRM114 Mailfilter executable program."
643 "The header that CRM114 Mailfilter inserts in messages."
648 "The switch that CRM114 Mailfilter uses to register spam messages."
653 "The switch that CRM114 Mailfilter uses to register ham messages."
658 "The switch that CRM114 Mailfilter uses to unregister ham messages."
663 "The switch that CRM114 Mailfilter uses to unregister spam messages."
668 "The regex on `spam-crm114-header' for positive spam identification."
673 "Directory path of the CRM114 Mailfilter databases."
679 ;;; Key bindings for spam control.
682 "St" spam-generic-score
683 "Sx" gnus-summary-mark-as-spam
684 "Mst" spam-generic-score
685 "Msx" gnus-summary-mark-as-spam
686 "\M-d" gnus-summary-mark-as-spam
704 "Internal variable.
705 `spam-split' will set this to nil or a spam-use-XYZ check if it
708 ;; internal variables for backends
709 ;; TODO: find a way to create these on the fly in spam-install-backend-super
715 ;;}}}
717 ;;{{{ convenience functions
720 "Clear the `spam-caches' entry for a check."
724 "Logical A xor B."
728 "Return a set difference of LIST1 and LIST2.
729 When either list is nil, the other is returned."
731 ;; we have two non-nil lists
738 ;; if either of the lists was nil, return the other one
742 "Checks if MARK is considered a ham mark in GROUP."
746 marks
751 "Checks if MARK is considered a spam mark in GROUP."
755 "In GROUP, get all the ham marks."
762 marks)))
765 "In GROUP, get all the spam marks."
769 "Is GROUP a spam group?"
774 nil))
777 "Is GROUP a ham group?"
781 nil))
784 "Return list of valid classifications"
788 "Is CLASSIFICATION a valid spam/ham classification?"
792 "Return list of valid classifications."
796 "Is PROPERTY a valid backend property?"
809 'spam-group-spam-mark-p
811 alist mark-cache-yes mark-cache-no)
817 gnus-newsgroup-name
818 mark)
823 alist))
825 ;;}}}
827 ;;{{{ backend installation functions and procedures
830 "Install BACKEND for spam.el.
831 Accepts incoming CHECK, ham registration function HRF, spam
832 registration function SRF, ham unregistration function HUF, spam
833 unregistration function SUF, and an indication whether the
834 backend is STATISTICAL."
842 5
843 "spam-install-backend-super got an invalid property %s"
844 property)))))
847 "Return a list of all the backend symbols, constrained by TYPE.
848 When TYPE is 'non-mover, only non-mover backends are returned.
849 When TYPE is 'mover, only mover backends are returned."
854 ;; or the type is 'mover and the backend is a mover
858 ;; or the type is 'non-mover and the backend is not a mover
863 list))
866 "Get the check function for BACKEND.
867 Each individual check may return nil, t, or a mailgroup name.
868 The value nil means that the check does not yield a decision, and
869 so, that further checks are needed. The value t means that the
870 message is definitely not spam, and that further spam checks
871 should be inhibited. Otherwise, a mailgroup name or the symbol
872 'spam (depending on `spam-split-symbolic-return') is returned where
873 the mail should go, and further checks are also inhibited. The
874 usual mailgroup name is the value of `spam-split-group', meaning
875 that the message is definitely a spam."
879 "Is BACKEND valid?"
883 "Return information about BACKEND."
887 backend))
890 info
891 property
893 info)
895 backend)))
898 "Get the BACKEND function for CLASSIFICATION and TYPE.
899 TYPE is 'registration or 'unregistration.
900 CLASSIFICATION is 'ham or 'spam."
907 classification
908 type))))
911 5
912 "%s was passed invalid backend %s, classification %s, or type %s"
913 "spam-backend-function"
914 backend
915 classification
916 type)))
920 "Property name of article list with CLASSIFICATION and UNREGISTER."
923 prop))
926 classification
928 "Get the articles to be processed for BACKEND and CLASSIFICATION.
929 With UNREGISTER, get articles to be unregistered.
930 This is a temporary storage function - nothing here persists."
932 backend
937 "Set the LIST of articles to be processed for BACKEND and CLASSIFICATION.
938 With UNREGISTER, set articles to be unregistered.
939 This is a temporary storage function - nothing here persists."
941 backend
943 list))
946 "Get the ham registration function for BACKEND."
950 "Get the spam registration function for BACKEND."
954 "Get the ham unregistration function for BACKEND."
958 "Get the spam unregistration function for BACKEND."
962 "Is BACKEND statistical?"
966 "Is BACKEND a mover?"
970 "Add ALIAS to an existing BACKEND.
971 The previous backend settings for ALIAS are erased."
973 ;; install alias with no properties at first
980 "Install a BACKEND than can only CHECK for spam."
984 "Install a BACKEND than can move articles at summary exit.
985 Accepts ham registration function HRF, spam registration function
986 SRF, ham unregistration function HUF, spam unregistration
987 function SUF. The backend has no incoming check and can't be
988 statistical."
990 backend
994 "Install a BACKEND than has no check.
995 Accepts ham registration function HRF, spam registration function
996 SRF, ham unregistration function HUF, spam unregistration
997 function SUF. The backend has no incoming check and can't be
998 statistical (it could be, but in practice that doesn't happen)."
1000 backend
1004 "Install a BACKEND.
1005 Accepts incoming CHECK, ham registration function HRF, spam
1006 registration function SRF, ham unregistration function HUF, spam
1007 unregistration function SUF. The backend won't be
1008 statistical (use `spam-install-statistical-backend' for that)."
1010 backend
1014 "Install a BACKEND.
1015 Accepts incoming CHECK, ham registration function HRF, spam
1016 registration function SRF, ham unregistration function HUF, spam
1017 unregistration function SUF. The backend will be
1018 statistical (use `spam-install-backend' for non-statistical
1019 backends)."
1021 backend
1025 "Install a statistical BACKEND than can only CHECK for spam."
1027 backend
1030 ;;}}}
1032 ;;{{{ backend installations
1057 ;; TODO: NOTE: spam-use-ham-copy is now obsolete, use (ham spam-use-copy)
1059 'spam-move-ham-routine
1060 'spam-move-spam-routine
1061 nil
1062 nil)
1065 'spam-copy-ham-routine
1066 'spam-copy-spam-routine
1067 nil
1068 nil)
1071 'spam-report-gmane-unregister-routine
1072 'spam-report-gmane-register-routine
1073 'spam-report-gmane-register-routine
1077 'spam-report-resend-register-ham-routine
1078 'spam-report-resend-register-routine
1079 nil
1080 nil)
1083 'spam-check-BBDB
1084 'spam-BBDB-register-routine
1085 nil
1086 'spam-BBDB-unregister-routine
1087 nil)
1092 'spam-check-blacklist
1093 nil
1094 'spam-blacklist-register-routine
1095 nil
1099 'spam-check-whitelist
1100 'spam-whitelist-register-routine
1101 nil
1102 'spam-whitelist-unregister-routine
1103 nil)
1106 'spam-check-ifile
1107 'spam-ifile-register-ham-routine
1108 'spam-ifile-register-spam-routine
1109 'spam-ifile-unregister-ham-routine
1113 'spam-check-spamoracle
1114 'spam-spamoracle-learn-ham
1115 'spam-spamoracle-learn-spam
1116 'spam-spamoracle-unlearn-ham
1120 'spam-check-stat
1121 'spam-stat-register-ham-routine
1122 'spam-stat-register-spam-routine
1123 'spam-stat-unregister-ham-routine
1127 'spam-check-spamassassin
1128 'spam-spamassassin-register-ham-routine
1129 'spam-spamassassin-register-spam-routine
1130 'spam-spamassassin-unregister-ham-routine
1134 'spam-check-bogofilter
1135 'spam-bogofilter-register-ham-routine
1136 'spam-bogofilter-register-spam-routine
1137 'spam-bogofilter-unregister-ham-routine
1141 'spam-check-bsfilter
1142 'spam-bsfilter-register-ham-routine
1143 'spam-bsfilter-register-spam-routine
1144 'spam-bsfilter-unregister-ham-routine
1148 'spam-check-crm114
1149 'spam-crm114-register-ham-routine
1150 'spam-crm114-register-spam-routine
1151 'spam-crm114-unregister-ham-routine
1153 ;;}}}
1155 ;;{{{ scoring and summary formatting
1157 "Return the extra headers spam.el thinks are necessary."
1160 spam-use-spamassassin-headers
1161 spam-use-regex-headers)
1164 spam-use-regex-headers)
1167 spam-use-regex-headers)
1169 list))
1177 "Sort articles by score."
1185 result))
1189 "Regexp matching SpamAssassin score header.
1193 "Transform an extra HEADER to a number, using list of HEADERS.
1194 Note this has to be fast."
1200 spam-spamassassin-score-regexp
1202 header-content)))
1203 ;; for CRM checking, it's probably faster to just do the string match
1211 header-content))))
1213 nil)))
1216 "Score an article for the summary buffer, as fast as possible.
1217 With SPECIFIC-HEADER, returns only that header's score.
1218 Will not return a nil score."
1231 "Invoke whatever scoring method we can."
1241 ;;}}}
1243 ;;{{{ set up widening, processor checks
1245 ;;; set up IMAP widening if it's necessary
1257 found))
1260 ;; note the nil processors are not defined in gnus.el
1278 "The OBSOLETE `spam-list-of-processors' list.
1279 This list contains pairs associating the obsolete ham/spam exit
1280 processor variables with a classification and a spam-use-*
1281 variable. When the processor variable is nil, just the
1282 classification and spam-use-* check variable are used. This is
1283 superseded by the new spam backend code, so it's only consulted
1288 "Checks if GROUP has a BACKEND with CLASSIFICATION registered.
1289 Also accepts the obsolete processors, which can be found in
1290 gnus.el and in spam-list-of-processors. In the case of mover
1291 backends, checks the setting of `spam-summary-exit-behavior' in
1292 addition to the set values for the group."
1297 found)
1300 ;; now search for the parameter
1308 ;; now, if the parameter was not found, do the
1309 ;; spam-summary-exit-behavior-logic for mover backends
1313 found
1319 ;; move ham out of spam groups
1323 spam-summary-exit-behavior))))))
1325 found))
1326 nil))
1328 ;;}}}
1330 ;;{{{ Summary entry and exit processing.
1333 ;; check the global list of group names spam-junk-mailgroups and the
1334 ;; group parameters
1338 "unseen"
1341 gnus-newsgroup-unseen
1342 gnus-newsgroup-unreads)))
1354 ;; The spam processors are invoked for any group, spam or ham or neither
1359 ;; before we begin, remove any article limits
1360 ; (ignore-errors
1361 ; (gnus-summary-pop-limit t))
1363 ;; first of all, unregister any articles that are no longer ham or spam
1364 ;; we have to iterate over the processors, or else we'll be too slow
1368 gnus-newsgroup-articles
1369 classification))
1371 ;; now that we have the changed articles, we go through the processors
1379 ;; call spam-register-routine with specific articles to unregister,
1380 ;; when there are articles to unregister and the check is enabled
1383 classification
1384 unregister-list
1385 t))))))
1387 ;; do the non-moving backends first, then the moving ones
1392 gnus-newsgroup-name
1393 backend
1394 classification)
1396 classification
1398 gnus-newsgroup-articles
1399 classification))))))
1403 ;; we mark all the leftover spam articles as expired at the end
1405 gnus-newsgroup-articles
1411 ;;}}}
1413 ;;{{{ spam-use-move and spam-use-copy backend support functions
1423 ;; remove the current process mark
1430 article mark deletep respool valid-move-destinations)
1436 ;; exclude invalid move destinations
1445 3
1446 "Sorry, can't move spam from spam group %s to spam group %s"
1447 gnus-newsgroup-name
1448 group))
1454 3
1455 "Sorry, can't move ham from ham group %s to ham group %s"
1456 gnus-newsgroup-name
1457 group)))
1462 ;; now do the actual move
1467 ;; first, mark the article with the process mark and, if needed,
1468 ;; the unread or expired mark (for ham and spam respectively)
1471 spam-mark-ham-unread-before-move-from-spam-group)
1473 article)
1478 article)
1486 spam-disable-spam-split-during-ham-respool))))
1488 article respool-method)
1490 ;; else, we are not respooling
1495 article group)
1499 article group)
1502 ;; now delete the articles, unless a) copy is t, and there was a copy done
1503 ;; b) a move was done to a single group
1504 ;; c) backend-supports-deletions is nil
1518 t
1520 articles
1525 nil
1527 articles
1532 t
1534 articles
1539 nil
1541 articles
1544 ;;}}}
1546 ;;{{{ article and field retrieval code
1551 article
1552 gnus-newsgroup-name)
1555 ;; disabled for now
1556 ;; (defun spam-get-article-as-filename (article)
1557 ;; (let ((article-filename))
1558 ;; (when (numberp article)
1559 ;; (nnml-possibly-change-directory
1560 ;; (gnus-group-real-name gnus-newsgroup-name))
1561 ;; (setq article-filename (expand-file-name
1562 ;; (int-to-string article) nnml-current-directory)))
1563 ;; (if (file-exists-p article-filename)
1564 ;; article-filename
1565 ;; nil)))
1568 "Fetch a FIELD for ARTICLE with the internal `gnus-data-list' function.
1569 When PREPARED-DATA-HEADER is given, don't look in the Gnus data.
1570 When FIELD is 'number, ARTICLE can be any number (since we want
1571 to find it out)."
1596 5
1597 "spam-fetch-field-fast: unknown field %s requested"
1598 field)
1599 nil)))))
1615 ;; 80-character limit makes for strange constructs
1627 5
1628 "spam-generate-fake-headers: article %d didn't have a valid header"
1629 article))))
1635 ;;}}}
1637 ;;{{{ Spam determination.
1640 "Split this message into the `spam' group if it is spam.
1641 This function can be used as an entry in the variable `nnmail-split-fancy',
1642 for example like this: (: spam-split). It can take checks as
1643 parameters. A string as a parameter will set the
1644 `spam-split-group' to that string.
1646 See the Info node `(gnus)Fancy Mail Splitting' for more details."
1663 widening-needed-check))
1665 decision)
1670 'spam
1671 spam-split-group)))
1673 ;; either, given specific checks, this is one of them
1675 ;; or, given no specific checks, spam-use-CHECK is set
1678 check-function)
1680 ;; if we got a decision at all, save the current check
1687 5
1689 decision
1690 spam-split-symbolic-return)))))))
1693 decision))))))))
1696 "Detect spam in the current newsgroup using `spam-split'."
1704 gnus-newsgroup-articles
1705 gnus-newsgroup-unseen))
1706 article-cannot-be-faked)
1724 registry-lookup)
1733 9
1734 "spam-find-spam: message %s was already registered incoming"
1735 id)))
1745 article
1746 group)
1747 ;; else, we fake the article
1762 id
1763 'incoming
1764 split-return
1765 spam-split-last-successful-check
1766 group))))))
1767 articles))))
1769 ;;}}}
1771 ;;{{{ registration/unregistration functions
1774 "Go through the backends and register or unregister articles as needed."
1779 backend classification))
1781 backend classification t))
1784 ;; clear the old lists right away
1786 classification
1787 nil
1788 nil)
1790 classification
1791 nil
1792 t)
1794 ;; eliminate duplicates
1803 9
1804 "%d messages did not have to unregister and then register"
1805 delcount))
1807 ;; unregister articles
1812 6
1813 "%d %s messages were unregistered by backend %s."
1814 num
1815 classification
1816 backend))))
1818 ;; register articles
1823 6
1824 "%d %s messages were registered by backend %s."
1825 num
1826 classification
1827 backend)))))))))
1830 backend
1831 specific-articles)
1835 backend
1836 specific-articles
1845 unregister-function
1846 register-function))
1848 'spam-log-undo-registration
1850 article articles)
1853 ;; make list of articles, using specific-articles if given
1856 gnus-newsgroup-articles
1857 classification)))
1858 ;; process them
1864 classification
1865 backend)
1867 ;; now log all the registrations (or undo them, depending on
1868 ;; unregister)
1872 'process
1873 classification
1874 backend
1875 gnus-newsgroup-name))))
1876 ;; return the number of articles processed
1879 ;;; log a ham- or spam-processor invocation to the registry
1893 7
1895 "%s call with bad ID, type, classification, spam-backend, or group"
1898 ;;; check if a ham- or spam-processor registration has been done
1906 7
1909 nil))))
1911 ;;; check what a ham- or spam-processor registration says
1912 ;;; returns nil if conflicting registrations are found
1915 decision)
1924 nil
1925 decision)))
1928 ;;; check if a ham- or spam-processor registration needs to be undone
1936 found)
1942 found)
1945 7
1948 nil))))
1951 ;;; undo a ham- or spam-processor registration (the group is not used)
1961 new-cell-list found)
1969 "%s call with bad ID, type, spam-backend, or group"
1971 nil))))
1973 ;;}}}
1975 ;;{{{ backend functions
1977 ;;{{{ Gmane xrefs
1984 header)
1985 spam-split-group))))
1987 ;;}}}
1989 ;;{{{ Regex body
1996 ;;}}}
1998 ;;{{{ Regex headers
2002 ret found)
2016 ret))
2018 ;;}}}
2020 ;;{{{ Blackholes.
2029 "Check the Received headers for blackholed relays."
2031 ips matches)
2041 ips)))
2045 ;; match against the reversed (again) IP string
2047 spam-blackhole-good-server-regex
2055 query-result)
2057 matches)))
2058 ;; else, if not using dig.el
2062 matches)))))))))
2064 spam-split-group)))
2065 ;;}}}
2067 ;;{{{ Hashcash.
2070 "Check the headers for hashcash payments."
2073 ;;}}}
2075 ;;{{{ BBDB
2077 ;;; original idea for spam-check-BBDB from Alexander Kotelnikov
2078 ;;; <sacha@giotto.sj.ru>
2080 ;; all this is done inside a condition-case to trap errors
2082 ;; Autoloaded in message, which we require.
2091 ;; `bbdb-records' should not be bound as an autoload function
2092 ;; before loading bbdb because of `bbdb-hashtable-size'.
2100 nil)))
2104 ;; when the BBDB changes, we want to clear out our cache
2111 "Enter an address into the BBDB; implies ham (non-spam) sender"
2117 'bbdb-delete-record-internal
2125 from
2137 ;; now do the register/unregister action
2149 "Mail from people in the BBDB is classified as ham or non-spam"
2154 t
2156 spam-split-group
2157 nil)))))))
2159 ;;}}}
2161 ;;{{{ ifile
2163 ;;; check the ifile backend; return nil if the mail was NOT classified
2164 ;;; as spam
2168 "Return the command-line parameter for ifile's database.
2169 See `spam-ifile-database'."
2172 nil))
2175 "Check the ifile backend for the classification of this message."
2177 category return)
2184 nil temp-buffer-name nil "-c"
2186 ;; check the return now (we're back in the temp buffer)
2193 ;; else, if spam-ifile-all-categories is not set...
2196 return))
2199 "Register an article, given as a string, with a category.
2200 Uses `gnus-newsgroup-name' if category is nil (for ham registration)."
2204 parameters)
2212 nil nil nil
2213 add-or-delete-option category
2228 ;;}}}
2230 ;;{{{ spam-stat
2235 "Check the spam-stat backend for the classification of this message"
2238 category return)
2271 ;;}}}
2273 ;;{{{ Blacklists and whitelists.
2283 ;;; address can be a list, too
2285 "Enter ADDRESS (list or single) into the whitelist.
2286 With a non-nil REMOVE, remove them."
2292 ;;; address can be a list, too
2294 "Enter ADDRESS (list or single) into the blacklist.
2295 With a non-nil REMOVE, remove them."
2302 "Enter ADDRESSES into the given FILE.
2303 Either the whitelist or the blacklist files can be used.
2304 With a non-nil REMOVE, remove the ADDRESSES."
2307 ;; else, we have a list of addresses here
2316 ;; found the address
2319 ;; else, the address was not found
2329 spam-blacklist-cache
2330 spam-whitelist-cache))
2331 parsed-cache)
2337 ;; fix regexp-quote's treatment of user-intended regexes
2351 found)))
2353 ;;; returns t if the sender is in the whitelist, nil or
2354 ;;; spam-split-group otherwise
2356 ;; FIXME! Should it detect when file timestamps change?
2360 t
2362 spam-split-group
2363 nil)))
2366 ;; FIXME! Should it detect when file timestamps change?
2370 spam-split-group))
2380 ;; insert the e-mail address if detected, otherwise the raw data
2389 found)
2399 from addresses unregister-list article-unregister-list)
2403 sender-ignored)
2410 ;; remember the messages we need to unregister, unless remove is set
2422 ;; else, register normally and unregister what we need to
2427 'process
2428 declassification
2429 de-symbol))
2444 ;;}}}
2446 ;;{{{ Spam-report glue (gmane and resend reporting)
2467 spam-report-resend-to)))
2470 ;;}}}
2472 ;;{{{ Bogofilter
2480 ;; spam detection mode
2482 header)
2483 spam-split-group)))))
2485 ;; return something sensible if the score can't be determined
2487 "Get the Bogofilter spamicity score."
2500 "Verify the Bogofilter version is sufficient."
2506 spam-bogofilter-valid)
2509 "Check the Bogofilter backend for the classification of this message."
2513 return)
2519 spam-bogofilter-program
2520 nil temp-buffer-name nil
2523 return)
2527 spam
2529 "Register an article, given as a string, as spam or non-spam."
2536 spam-bogofilter-spam-strong-switch
2537 spam-bogofilter-ham-strong-switch)
2539 spam-bogofilter-spam-switch
2540 spam-bogofilter-ham-switch))))
2547 spam-bogofilter-program
2548 nil nil nil switch
2565 ;;}}}
2567 ;;{{{ spamoracle
2569 "Run spamoracle on an article to determine whether it's spam."
2577 spam-spamoracle-binary
2578 nil temp-buffer-name nil
2587 spam-split-group))
2591 "Run spamoracle in training mode."
2599 "-spam"
2604 spam-spamoracle-binary
2605 nil temp-buffer-name nil
2625 ;;}}}
2627 ;;{{{ SpamAssassin
2628 ;;; based mostly on the bogofilter code
2630 "Check the SpamAssassin headers for the classification of this message."
2637 ;; spam detection mode
2641 header)
2642 spam-split-group)))))
2645 "Check the SpamAssassin backend for the classification of this message."
2652 nil temp-buffer-name nil spam-spamassassin-arguments))
2653 ;; check the return now (we're back in the temp buffer)
2657 ;; return something sensible if the score can't be determined
2659 "Get the SpamAssassin score"
2673 "Register articles with spamassassin's sa-learn as spam or non-spam."
2677 spam-sa-learn-ham-switch)))
2680 ;; group the articles into mbox format
2686 ;; mbox separator
2690 ;; call sa-learn on all messages at the same time
2693 spam-sa-learn-program
2694 nil nil nil "--mbox"
2711 ;;}}}
2713 ;;{{{ Bsfilter
2714 ;;; based mostly on the bogofilter code
2722 spam-split-group)))))
2724 ;; return something sensible if the score can't be determined
2726 "Get the Bsfilter spamicity score."
2739 "Check the Bsfilter backend for the classification of this message."
2742 return)
2748 spam-bsfilter-program
2749 nil temp-buffer-name nil
2750 "--pipe"
2751 "--insert-flag"
2752 "--insert-probability"
2756 return))
2759 spam
2761 "Register an article, given as a string, as spam or non-spam."
2766 spam-bsfilter-spam-strong-switch
2767 spam-bsfilter-ham-strong-switch)
2769 spam-bsfilter-spam-switch
2770 spam-bsfilter-ham-switch))))
2776 spam-bsfilter-program
2777 nil nil nil switch
2778 "--update"
2781 spam-bsfilter-database-directory))))))))
2795 ;;}}}
2797 ;;{{{ CRM114 Mailfilter
2805 ;; spam detection mode
2807 header)
2808 spam-split-group)))))
2810 ;; return something sensible if the score can't be determined
2812 "Get the CRM114 Mailfilter pR."
2824 "Check the CRM114 Mailfilter backend for the classification of this message."
2827 return)
2833 spam-crm114-program
2834 nil temp-buffer-name nil
2837 return))
2840 spam
2842 "Register an article, given as a string, as spam or non-spam."
2848 spam-crm114-spam-strong-switch
2849 spam-crm114-ham-strong-switch)
2851 spam-crm114-spam-switch
2852 spam-crm114-ham-switch))))
2859 spam-crm114-program
2860 nil nil nil
2875 ;;}}}
2877 ;;}}}
2879 ;;{{{ Hooks
2881 ;;;###autoload
2883 "Install the spam.el hooks and do other initialization.
2884 When SYMBOLS is given, set those variables to t. This is so you
2885 can call `spam-initialize' before you set spam-use-* variables on
2886 explicitly, and matters only if you need the extra headers
2887 installed through `spam-necessary-extra-headers'."
2898 ;; TODO: How do we redo this every time the `spam' face is customized?
2900 gnus-summary-highlight)
2901 ;; Add hooks for loading and saving the spam stats
2909 ;; Don't install things more than once.
2913 "Uninstall the spam.el hooks."
2925 ;;}}}
2929 ;;; spam.el ends here