amavisd-new
amavisd-new is a high-performance interface between mailer (MTA) and
content checkers: virus scanners, and/or SpamAssasin. It is written in Perl for
maintainability, without paying a significant price for speed. It talks to MTA
via (E)SMTP or LMTP, or by using helper programs. Best with Postfix, works with
Exim, sendmail/milter, or with any MTA as a SMTP relay. 'Howto' for qmail
available as well.
amavisd-new is a high-performance and reliable interface between
mailer (MTA) and one or more content checkers: virus scanners, and/or
Mail::SpamAssasin Perl module. It is written entirely in Perl, assuring high
reliability, portability and maintainability. It talks to MTA via (E)SMTP or
LMTP, or by using helper programs. No timing gaps exist in the design, which
could cause a mail loss.
It is normally positioned at or near a central mailer, not necessarily where
user's mailboxes and final delivery takes place. If you are looking for a
per-user and low-message-rate solution to be placed at the final stage of mail
delivery (e.g. called from procmail or in place of local delivery agent),
there may be other solutions more appropriate for your needs.
When calling of Mail::SpamAssassin is enabled, it calls SA only once per
message (regardless of the number of recipients), and tries very hard to
correctly honor per-recipient preferences, such as pass/reject, and inserting
spam-related mail header fields.
amavisd-new benefits from the use of Perl module Net::Server, which offers a
fast pre-forked multichild environment. amavisd-new provides
rfc2821-compliant SMTP server, rfc2033-compliant LMTP server, SMTP client, and
generates rfc1892/rfc1894-compliant (non-)delivery status notifications. This
makes it suitable for mail anti-virus and/or anti-spam checking on a busy mail
gateways, that care for reliability and standards compliance.
amavisd-new grew out of amavisd(-snapshot) (which in turn is a
daemonized version of amavis-perl), but
over the year of development turned into a separate product, hardly resembling
its origin. The code is about three times the size of its predecessor, yet
faster in throughput, richer in features, compliant to standards, includes
optional support for spam detection, and makes virus scanning optional.
Compatibility with helper programs from amavisd(-snapshot) is retained.
All the modifications since the original amavisd done by Mark Martinec, with contribution of
ideas and reports from amavis-user mailing list community.
amavisd-new-20030314-p1 (patch1) released (March 21, 2003). Please see
the release
notes. It seems to be as stable as amavisd-new-20021227, and fixes few
long-standing problems (which did not affect many users, but still).
If currently using SpamAssassin 2.50 with Bayes enabled, and upgrading to
SpamAssassin 2.51, the SA patches in http://spamassassin.kluge.net/patches/
must be applied, and the command sa-learn --force-expire must be
manually run, before starting amavisd-new. This is because SA 2.50 didn't do
bayes expirations on-the-fly, and without patches all amavisd-new child
processes would try to run expire at the same time, causing lock contention and
timeouts.
There is an updated README.chroot on the
web page, which mentions some FreeBSD specifics and fixes some typos.
Older News
amavisd-new-20030314 is out.
The version 0.85 of Net::Server Perl module from March 7 2003 is
breaking amavisd-new, causing 421 4.3.2 Service shutting down, closing
channel. Please upgrade to amavisd-new-20030314 or stick with
Net::Server 0.84 if using an older version.
Please see Security warning below.
There is an updated README.postfix on
the web page, which takes into account the new recommended move of virtual alias
mappings from (pre-)cleanup into regular cleanup Postfix service, as documented
in the Postfix 2.0.3 ./README_FILES/FILTER_README.
qmail setup howto - see link in the documentation section.
Security warning
A security problem in Unix utility program file(1) has recently
been found by iDEFENSE Labs. It permits a
specially crafted message to access the system as a user running
amavisd-new, typically amavis or vscan. As recommended in
the setup instructions, this UID and GID should be dedicated to
amavisd-new (possibly shared with virus scanners), so the extent of
possible damage is limited. Furthermore, the problem does not affect setups
where amavisd-new is not doing virus or banned file scanning, (e.g.
serves only as an interface to SpamAssassin), or if
$bypass_decode_parts is true. Nevertheless, it is heartly recommended
to upgrade to the latest version of file(1), available from ftp://ftp.astron.com/pub/file/file-3.41.tar.gz
It is suspected that all versions up to and including file 3.39 are
vulnerable.
More information: http://www.idefense.com/advisory/03.04.03.txt,
http://www.securityfocus.org/archive/1/313837/2003-03-02/2003-03-08/0
For general security information regarding amavisd-new please see the
section Security
considerations below.
- amavisd-new-20030314-p1.tar.gz
(md5
sum)
- latest version of amavisd-new, with patch1-A and patch1-B already applied
(for reference: patch1-A
(mandatory), patch1-B
(ldap), plus some documentation changes, and a new file
bad_headers.patch for playing. Patches are described in the current
RELEASE
NOTES). Appears to be at least as stable as the previous release.
- amavisd-new-20021227-p2.tar.gz
(md5
sum)
- previous version of amavisd-new (includes patch1 and patch2) (stable)
- amavisd-new-20021116.tar.gz
(md5
sum)
- (a new features release, its mandatory patches: p1,
p2,
p3,
p4,
p5)
- amavisd-new-20020630.tar.gz
(md5
sum)
- (mostly a development and new features release)
- amavisd-new-20020517.tar.gz
(md5
sum)
- (with SpamAssasin optional support)
There are also packaged versions available, provided and supported only by
their respective authors/maintainers:
- FreeBSD port
- in the System
security software ports section
(/usr/ports/security/amavisd-new). The latest port maintained by Blaz
Zupan is based on amavisd-new-20021227-p2, previous port was based on
amavisd-new-20020517.
- Debian package amavisd-new 20021227p2-2
- maintained by Brian May and Henrique de Moraes Holschuh, available at: http://packages.debian.org/unstable/mail/amavisd-new.html
- Red Hat RPM packages AMaViS (amavisd and amavisd-new (older!) )
- by Ramiro Morales, available at: http://www.rmorales.com.ar/rpms/amavis/
There may be other packaged version around, please let me know.
Note that the packaged versions may not be based on the most recent
amavisd-new version.
Besides this web page at http://www.ijs.si/software/amavisd/,
the following files comprise the amavisd-new documentation. The web page
documents may be more recent than the documentation distributed with the
program.
How-to
Check the amavis-user
mailing list. Its archives are at http://marc.theaimsgroup.com/?l=amavis-user
.
For questions about packaged versions please contact their maintainers and/or
their bug-tracking mechanisms.
-- general
- written entirely in Perl for better maintainability, portability, no risk
of buffer overruns or invalid pointers; the critical paths are well optimized;
- a daemonized server, saving on startup time; designed with high throughput
in mind;
- modular (package namespaces, OO techniques), yet contained in a single
file;
- optionally calls one or more anti-virus scanners - the current list
includes entries for 28 av scanners, and it is easily extended, see
amavisd.conf file for the list;
- optionally checks MIME types, file names, and content types of MIME parts
against a list of banned names and content types;
- optionally calls Perl module Mail::SpamAssassin to check for spam,
with optional whitelist/blacklisting of envelope sender address;
- thorough error checking, informative error reporting, fail-safe failure
modes;
- doesn't lose your mail when unpredictable things happen, including a
machine crash (unlike some alternative products); the responsibility for mail
always stays with MTA;
- adheres tightly to a bunch of RFC specifications - see release notes for
details;
- works best with MTAs which interface to content filter via SMTP (or LMTP),
such as Postfix, Exim v4, or dual (any) MTA setup (most features available,
fast), but can also interface with sendmail/milter or traditional sendmail
using helper programs.
-- performance
- pre-forked reusable children managed by Net::Server Perl module,
saving on process creations and startup latency;
- significant speedups (25% with fast AV scanner compared to
amavisd-snapshot-20020300) in SMTP-in/SMTP-out configuration; directory and
temporary file reuse;
- persistent connections to certain AV scanners, e.g. Sophie and Trophie, saving on forks; or
directly callable AV scanner via Perl module to access Sophos engine: SAVI-Perl;
- cache of the last few body message digests (MD5) -- significant speedup
for mailing list bursts, bypassing virus and spam checks;
- detailed timing breakdown report for each passed message (enabled with
higher logging levels);
- when configured to call Mail::SpamAssassin (this is optional), it
orders SA to preload its config files and to precompile the patterns, so
performance is at least as good as with spamc/spamd setup. All Perl modules
are pre-loaded by parent process, so forked children need not re-compile the
code, and hopefully share the same memory for compiled code (depending on
fork implementation);
- see also file README.performance.
-- interfaces to MTA
- accepts mail from MTA either via SMTP (fully rfc2821-compliant), or via
LMTP (rfc2033), or through a Unix pipe from a helper program;
- forwards mail via SMTP, or by a pipe to some external command (sendmail
mail submission program, or its lookalike), or by telling a helper program the
mail is to be accepted or not (e.g. with sendmail milter);
- in the SMTP-in/SMTP-out mail relay configuration the amavisd-new
can be located on a different host as the MTA host, and several MTAs can share
a single amavisd-new service;
- similarly the sendmail/milter allows amavis_milter + amavisd-new to
be separated from the MTA host (in sendmail.mc use:
INPUT_MAIL_FILTER(`milter-amavis',`S=inet:port@hostname,F=T,T=S:10m;R:10m;E:10m')dnl
and start amavis-milter with -p inet:port@0.0.0.0 instead of -p
path_to_unix_socket; thanks to Dibo for the tip);
- pass reject reason from the MTA on the output side back to MTA on the
input side;
- proper (non-)delivery status notifications (DSN), compliant with rfc1892
and rfc1894 (since 20021116);
- informative MTA log entries in the SMTP-in/SMTP-out setup;
- amavis internal id (am_id) in log entries and passed to MTA in SMTP
response, making easier to match MTA and amavisd log entries;
- supported MTA configurations:
- Postfix supported and thoroughly
tested;
- sendmail libmilter interface supported and tested (no header rewriting
or adding address extensions in this setup);
- Exim v4 supported;
- Exim v3 via helper program mostly works, but is not recommended due to
defficiencies in the amavis.c helper program or its interfacing with Exim;
- dual-MTA configurations (any MTA type) with amavisd-new relaying
between them (SMTP) is the recommended setup (speed and flexibility) for
other mailers or older sendmail versions; This includes the qmail support -
see howto link in the documentation section;
- sendmail in the traditional setup with amavis helper program (amavis.c)
called as a local delivery agent. Please also set $forward_method =
undef in amavisd.conf if you use this method; dual-MTA setup
with older sendmail version is preferred;
-- user individuality, quarantine
- can specify subgroups of users who want to receive viruses or spam (with
alert header line added, notifications are still generated); (no header
rewriting available with milter);
- optionally add address extensions: e.g. user@domain ->
user+virus@domain, if virus (or spam) detected and desired to be passed; (no
address modifications available with milter);
- can split the message if not all recipients in a multi-recipient message
require the same header insertions/deletions/edits; (since 20021116);
- quarantine options: save to individual file, save to local mailbox, pass
to MTA for delivery to a quarantine mailbox (possibly remote, but not
advisable with sendmail/milter unless you know how to avoid loop);
- added headers in quarantined viruses preserve envelope addresses and
quarantine id and facilitate releasing quarantined messages; added headers
also identify the reason for quarantining (virus name or spam report);
-- anti-virus
Since amavisd-new-20021227 it is easier to individually enable/disable
virus checking and spam checking.
- includes support for 28 av scanners off-the shelf (see file amavisd.conf,
variable @av_scanners, for the list);
- supports command-line virus scanners (new entries are easily described and
added to the list @av_scanners, file amavisd.conf);
- supports daemonized virus scanners (e.g. clamd, OpenAntiVirus, Sophie,
Trophie, FRISK F-Prot) and scanners available via Perl modules (e.g.
SAVI-Perl);
- can specify subgroups of recipient for whom virus checks need not be
performed;
- can specify subgroups of recipients who want to receive viruses (with
alert header line added (no header rewriting available with milter));
- optionally add address extensions: e.g. user@domain ->
user+virus@domain, if virus detected and desired to be passed;
- can specify final_virus_destiny: reject, discard, pass (defaults to
discard, with sender and virus admin notification);
- does not accuse innocent users or sending viruses if sender address is
suspicious (see FAQ below);
-- anti-spam
Since amavisd-new-20021227 it is easier to individually enable/disable
virus checking and spam checking.
- can whitelist envelope senders or domains for whom spam checks need not be
performed, content is declared as spam-free;
- can blacklist envelope senders or domains for whom spam checks need not be
performed, content is treated as spam;
- can specify subgroups of users (envelope recipients) for whom spam checks
need not be performed;
- can specify subgroups of users (envelope recipients) who want to receive
spam, with alert header line added; (no header rewriting available with
milter);
- optionally add address extensions: e.g. user@domain ->
user+spam@domain, if spam detected and desired to be passed;
- can specify final_spam_destiny: reject, discard, pass (defaults to
reject);
- can send spam notifications to spam admin, which include the full
Mail::SpamAssassin report.
- since 20021116: spam headers are inserted on a per-user basis, according
to their tag/kill level settings; this means that a multi-recipient message is
split into clusters of recipients with same settings if needed (not available
with milter interface). This permits per-recipient individual settings, while
still being efficient for the mailing list traffic;
-- other
- customizable notification messages based on simple macro expander in the
spirit of m4 -- see README.customize;
- versatile lookup mechanisms, described in README.lookups,
to be used with virus_lovers, spam_lovers, bypass_checks, @local_domains,
@inet_acl, ... The lookup tables supported are: Perl hash, access control
list, Perl regular expressions access list, SQL lookups;
- HUP signal (or amavisd reload) causes restart with new
configuration;
- non-delivery notification not sent to mailing lists (mail with
Precedence: bulk);
- code heavily commented, cleaned, generalized;
Recommended version of SpamAssassin is 2.50. It fixes known problems with
2.44 which caused cpu loops or process hangs when called from amavisd-new. Make
sure to also apply patches (below) to Razor2 if using it from SpamAssassin,
otherwise some taint-related failures can occur,
See below for problems with recognizing Yaha.K virus.
Versions of amavisd-new before 20030314 did not know how to override a
locale settings, so a non-English locale would produce a non-RFC2822 -compliant
date format (used in Received and Date header fields). Please set the
environment variable LC_TIME to "C" before starting amavisd-new daemon,
otherwise an illegal mail header field may be produced. This problem does not
affect amavisd-new-20030314.
- If running it for the first time, or when encountering a problem,
try to start the daemon as a non-detached process with full logging to the
terminal:
# amavisd debug
This command line option is supported since amavisd-new-20020630.
For earlier versions set the variable $DEBUG = "yes"; in the file
amavisd.
- The version amavisd-new-20020630 introduced new
amavisd.conf parameters $daemon_user and $daemon_group. Please match
these settings to your desired user and group under which you want the daemon
to run. Some usual usernames for this putpose are vscan or
amavis, some more common groups are vscan, amavis or
sweep. For earlier versions you need to use the su(1) Unix command to
change user ID when starting the amavis daemon, as described in the next item.
- If during startup the Net::Server complains that it can not change UID to
the required $daemon_user and $daemon_group: Net::Server: ... Couldn't
become uid ..., it probably means your Perl can not change real uid of
the process (variable $<) the way the author of Net::Server had in mind.
This is commonly reported for FreeBSD platforms. You can always start amavisd
the traditional way:
# su - vscan -c /usr/local/sbin/amavisd
or if some comand line options need to be specified, e.g.: # su - vscan -c '/usr/local/sbin/amavisd -c /etc/amavisd-test.conf debug'
- After amavisd reload (or HUP) the daemon process shuts down but
does not restart? As the root privileges are dropped after the first start,
the reload must be able to do everything as user amavis (vscan). Some
situations where this may not be enough:
- configuration file /etc/amavisd.conf too heavily protected;
- changed the value of $inet_socket_bind in the config file;
- running in a chroot jail;
- and probably a lot more.
- If you see virus notifications claiming the virus originator is <?>
or <?@some.domain> and sender notifications are not sent, this is not a
bug, but a feature -- see $viruses_that_fake_sender_re regexp lookup table.
The original idea comes from Furio Ercolessi: as some viruses tend to use
forged or corrupted envelope sender or 'From:' addresses, we try to determine
the true virus sender, and if we can not do that, we avoid accusing
innocent users of sending viruses by not generating sender notifications;
- The directory specified by $TEMPBASE (typically
/var/amavis) is holding temporary directories where mail is being
unpacked and checked. Each amavisd-new child process keeps its own
temporary directory and works inside it. This means you normally see
$max_servers temporary directories at any time. The temporary directory
(with name like amavis-20020924T181627-11984), the file
email.txt within it, and the subdirectory parts, are reused (for
speed) for subsequent mail checks within the lifetime of a child process, and
are only deleted after the child process has done its share of mail checks
(after $max_requests).
- If you kill or reload (HUP) amavisd (or in the unlikely event of
abnormal child death), temporary directories may be left undeleted; this is
normal and mail is not lost; actually you can kill amavisd at any time
without being in danger of losing mail, as amavisd never takes delivery
responsibility away from MTA by acknowledging mail reception without first
getting it acknowledged by the second MTA instance. The worst that can happen
is the message gets delivered twice, but this is unlikely in practice.
The consequence is that if you restart or reload amavisd often, you
are left with a growing set of leftover temporary directories, one set
of $max_servers directories for every restart. Please select and delete these
old temporary directories, e.g. using a find(1) Unix command. To be sure you
do not delete the still active directories, it it recommended to delete them
when amavisd is not running, e.g. just before starting it. The
following command does that for example. The -mtime +1 option is just
a (non-foolproof) safeguard in case amavisd is currently running, and
may be left out:
find /var/amavis -type d -name 'amavis-20??????T*' \
-prune -mtime +1 -exec rm -rf {} \;
- There is a problem in the Net::SMTP Perl module (version
2.17 and earlier), which as far as I know only affects the Exim setup with
amavisd-new: MTA-generated notifications from <> appear to be
comming from <<>> and are rejected by MTA. Please upgrade the Perl
module libnet (which contains Net::SMTP) to version
libnet-1.12. Upgrading this module is probably a good idea even if you
use other MTA.
- Do not let too many files to accumulate in directories /var/virusmail or
/var/amavis. Depending on the file system, large number of files in a
directory can lead to long processing times for creation or deletion of a file
in such directory, potentially affecting amavisd-new troughput drastically.
Keep this in mind when spam quarantine is enabled!
- Starting with Perl 5.8.0 Unicode support, several
problems were introduced with interoperability of software components, some of
which are not yet ready for handling UTF-8 character encodings, dealing with
character codes above 255 and distinguishing between byte-text, character-text
and binary files. Problems of this sort are most pronounced in Red Hat 8.0
Linux distribution, as it sets locale by default to use UTF-8 encoding (run
command locale to check your settings). One may see Malformed UTF-8
character warnings, possibly also Bad RFC822 field name 'C0c0.
Awareness and solutions to some of these problems will appear in the
February 2003 release of amavisd-new. Also the SpamAssassin people are
attacking their share of UTF-8 related problems, and some issues will be fixed
in SpamAssassin 2.50.
For the moment it is the safest to run amavisd-new in a non-UTF8 locale
environment. Either adjust the settings in /etc/sysconfig/i18n
(Linux), or set environment variables LANG and LC_ALL to "C" or "en_US"
(instead of "en_US.UTF-8") when starting amavisd-new daemon. Depending on the
shell used, one may start amavisd-new by: (with bash)
# su - vscan -c 'LANG=C LC_ALL=C /usr/local/sbin/amavisd'
or the long way: # su - vscan
$ export LANG; export LC_ALL; LANG="C"; LC_ALL="C"
$ /usr/local/sbin/amavisd
- Seing NOTICE client broke the connection without a QUIT in
amavisd-new logs?
This message usually means that SpamAssassin or one of its external
services played jokes (like closing it) on STDIN or STDOUT, which is
associated with SMTP/LMTP socket in amavisd-new. So what would be an innocent
close in a command-line application, is breaking the SMTP or LMTP session. If
it happens at the very end before the QUIT, it is innocent, otherwise it may
indicate a more serious problem.
Likely culprit is one of the external SpamAssassin tests. Try first
disabling all external SA tests by setting (in amavisd.conf):
$sa_local_tests_only = 1; . If that helps, you may want to track down
exactly which SA external test is causing it, by leaving
$sa_local_tests_only off, but turning external tests off individually
in SpamAssassin configuration file, usually at /etc/mail/spamassassin/local.cf
(e.g. dns_available, skip_rbl_checks, use_razor1, use_razor2, ...)
Tips and FAQ -- mail transfer agents (MTA)
- not to be forgotten: for mail setups such as with sendmail/milter where
messages do not pass through amavisd-new, headers can not dynamically
be inserted, recipient addresses can not be dropped or address extensions
added because of the limitations of the helper programs. This does not apply
to Postfix, Exim v4, or dual MTA (any MTA type) setups.
- sendmail/milter: Experiencing deadlocks with newer versions of
sendmail/milter when number of amavisd child processes ($max_servers) is
low? Please use option -odd when submitting notifications. This is
now the default since amavisd-new-20021116.
- sendmail/milter: Because -odd option (deferred delivery mode) is
now used to submit notifications in the sendmail/milter setup, you must tell
sendmail to periodically process the clientmqueue queue. See sendmail options
-q and -qp. You may want to put something like sendmail -Ac -qp1m in
the sendmail initscript.
- Postfix: versions before Postfix 2.0 had two minor problems with its lmtp
client - it may fail to parse its parameters and obtain port number, and it
may lowercase mail addresses. If it gives you problems, just stay with smtp
protocol, or upgrade Postfix to 2.0.
Tips and FAQ -- virus scanners
- Some versions of Sophie
used to have problems with its child management. As amavisd-new does child
management by itself there is no need for Sophie to worry about it, so you may
want to disable this Sophie feature (--with-maxproc=0), or set it to
the same or higher value as the amavisd.conf setting $max_servers. Also the
recommended Sophie configure options is --enable-error-strings .
Always start Sophie using a full absolute path.
Tips and FAQ -- spam scanners (Mail::SpamAssassin)
- There is a bug in Mail/SpamAssassin/EvalTests.pm in routine
check_for_faraway_charset_in_body (at least in all version between 2.31 and
2.44). It causes Perl to go in a CPU loop, and amavisd fails to
continue service, or reports timeouts. Either apply the patch, or
disable CHARSET_FARAWAY_BODY test by adding a line score
CHARSET_FARAWAY_BODY 0 to the SA config file. It seems that 2.50
fixed these problems, and it is now the recommended version to be used
with amavisd-new, not least because it offers new Bayesian spam/ham
classification.
- If Mail::SpamAssassin is set to call Vipul's Razor 2.22, it fails because
reading its config file (routine read_file in Razor2/Client/Config.pm)
produces tainted values. You should apply the patch to
Razor2/Client/Config.pm ... plus the patch2 by
Vivek Khera along the same lines for Razor2/Client/Core.pm . To apply:
cd to the directory /usr/lib/perl5/.../Razor2/Client/ and apply them
from there: patch <patchfile
- How to add the spam tags to all inbound messages so that one sees
the spam score and test information in the message header? By reducing the tag
level (and keeping kill level high if you want), you enable spam-related
header fields to be inserted to inbound mail (i.e. for recipients matching
@local_domains).
- tag level is where X-Spam-Status and X-Spam-Level
header fields start to appear (e.g. setting tag level to 0 or even -999
would turn this on permanently);
- kill level is where a message is considered spam. The
X-Spam-Flag: YES header filed appears, the X-Spam-Status gets
a YES, Subject gets a ***SPAM***, and spam
countermeasures are taken (reject/drop/pass, quarantie, notify, adding
optional recipient address extension).
- If you don't see spam-related headers inserted, here are some reasons:
- tag level is set too high ($sa_tag_level_defl) or SpamAssassin is not
enabled;
- @local_domains is not correctly set. These headers are only inserted for
recipients belonging to @local_domains lookup;
- headers can only be added or edited when messages pass through
amavisd-new. This is not the case with sendmail milter setup or other setups
using helper program;
- to make message with spam level above kill_level still pass, either set
globally: $final_spam_destiny=1, or add recipient among spam_lovers.
Tips and FAQ -- Net::Server
- Net::Server 0.82 triggers a Perl 5.005 bug (the problem is obvious:
you get syntax errors). Either upgrade to Net::Server 0.83, or upgrade
your Perl -- 5.6.x should be ok.
- After changing $inet_socket_bind in amavisd.conf, you must stop
amavisd and start it anew. The HUP method causes amavisd to
stumble over its feet.
amavisd-new accepts mail from MTA, calls external Perl modules and
forks external programs to decompress and decode message, classify its content,
then the checked mail is either passed to MTA for delivery, or rejected and
quarantined.
Any component of a program that comes in contact with unpredictable and
possibly malicious mail/document content, must be careful not to let the content
have any uncontrolled effect on the operation of the program, or its
environment.
amavisd-new is written entirely in Perl, with taint mode Perl checking
enabled. This in itself is a strong argument that the processing within
amavisd-new (and Perl modules it calls) is not likely to be subject to
buffer overruns, stack smashing, and other problems that are common source of
security problems in programs written in languages like C.
Information coming from external world like SMTP envelope information, mail
header and body contents, suggested MIME file names, etc., is only used by
amavisd-new for operations that do not influence the program environment.
For example, names of created temporary files are internally generated and never
depend on suggested file names from MIME header. Mail addresses or other tainted
information is never passed through shell to an external program.
The external Perl modules called by amavisd-new have not been
thoroughly screened for possible security implications. They still benefit from
the Perl environment, and the Perl taint mode checking is never turned off even
when other Perl modules are executing, including SpamAssassin if enabled (which
is a relatively complex piece of software). Perl modules that deal with decoding
and checking of mail contents may be targets for malicious mail content,
especially if they include code written in C, like decoding and uncompressing
libraries, e.g. zlib and uudeview (Convert::UUlib).
External programs that get forked from amavisd-new to perform some
decoding/uncompressing or classifying task, are the greatest potential threat to
the safe operation of the host running amavisd-new. Some of these
programs that are used to decode certain archive formats are quite complex, old
or not maintained, and/or written by less security conscious authors. E.g. a
recently discovered vulnerability is present in Unix utility file(1) -
please upgrade file to 3.41 !!!
There is a tradeoff in deciding whether to call some external decoder:
calling it may open a vulnerability at the host running amavisd-new; not
calling it (and not decoding certain types of document) may cause virus checker
to miss a malicious mail contents, increasing danger for the mail recipient,
while reducing risk for the host running checks.
While it may be true that only powered-down computer, locked in a basement
and disconnected from the network, is a completely secure computer, this is not
practical to get any job done. Besides choosing your content filtering program
to be written in Perl and using taint mode checks, there are other things one
may do to reduce security threats to the computer running content filter:
- Never run amavisd-new as root or with other elevated privileges.
Use a dedicated username (UID) and group (GID) for the purpose (for example:
don't use mail or nobody). Start amavisd-new with
su(1) (best), or specify $daemon_user and $daemon_group in amavisd.conf
.
- Check settings for external programs like $arc, $gzip, $bzip2,
$uncompress, $lha, $unarj, $unrar, $zoo, and disable those you do not trust
(amavisd.conf: Section VII - External programs...). Make sure external
programs, their configuration files (e.g. /usr/share/magic) or directories
where they reside, are not writeable by non-privileged (non-root) user.
Normally these files should be owned by root. This also applies to external
virus checking programs and their database, and external programs that may be
called by SpamAssassin (if enabled).
- The same applies to configuration files used by amavisd-new and
SpamAssassin, e.g. /etc/amavisd.conf, /etc/mail/spamassassin/*,
/usr/share/spamassassin/* . Most importantly these files should not be
writeable by user (UID or GID) under which amavisd-new runs, they
should preferably be owned by root and not (world or group)-writeable.
- The directories /var/amavis and /var/virusmail ($TEMPBASE, $QUARANTINEDIR)
must be writeable for amavisd-new process, and are commonly owned by
user and group under which amavisd-new is running. These directories
should not be writeable by other non-privileged users.
- chroot mode of operation is a very powerful security concept in Unix.
amavisd-new can work in a chroot environment since
amavisd-new-20021116. This requires that all external programs called
by amavisd-new can operate in a chroot file system subtree. Preparing a
chroot environment, including all required programs and Perl modules is highly
system-dependent, and is not described in amavisd-new documentation.
Using only sockets to communicate with the external world (e.g. SMTP,
daemonized virus scanners) simplifies the setup. Unfortunately setting up
chroot environment for amavisd-new is not for inexperienced Unix
administrators.
- Do regular backups, keep file system signatures (e.g. with Tripwire).
- Choosing a platform less widespread and popular may help: Alpha or Sparc
CPU instead of Intel, *BSD or Tru64 or Solaris instead of Linux (not to
mention Windows) may help. This may be considered security through obscurity,
but any additional obstacle can help.
Running a virus checking content filter for each mail before it reaches the
mail reader is an important line of defense against virus outbreaks and in
protecting the (possibly not security conscious) recipients, or their mail
readers or computer environment.
Not all malware is passed by e-mail. Several viruses or worms use multiple
mechanisms to propagate itself, including WWW, sharing disks or 'content',
social engineering, or even a floppy or CD carried in a back-pocket or
distributed by magazines and software publishing houses.
Even in e-mail, malware may be carried in encrypted or scrambled form, or
simply as a plain text, using social engineering techniques to persuade
recipient to fetch or activate malware.
It is not possible to prevent user shooting himself in the foot, or to
prevent a dedicated user to transfer malware. There should be a tradeoff in
keeping e-mail useful, and protecting against threats.
The first line of defense (mail content filtering, firewall) must be
complemented by defense mechanisms at the local user's desktop computer. This
includes virus scanners run on PCs, keeping software up-to-date, doing backups,
and educating users.
Malware does not have to play by the rules. There is nothing to prevent
malware to generate a syntactically incorrect mail, to send it directly to some
mailer and supplying forged SMTP informations, to poison DNS, perhaps even to
use forged source IP address.
Content filter with virus scanner tries to decide if the mail under
consideration will, or can, cause any bad effects on the recipient computer,
often without knowing what mail reading software or what computer is used by
recipients. This implies that while some mail may be decoded (by adhering to
standards) into a harmless text, it might be decoded my some broken MUA into a
virus or exploit, or cause MUA bug to be triggered during decoding/displaying
the message. See Malformed
email project for example.
Solving this problem would require content filter with virus scanner to
emulate all known (and unknown?!) mail readers in the way they respond to
malformed mail. While amavisd-new and other content filters try to anticipate
some common problems, especially the ones practiced by currently known viruses,
there is no guarantee that this approach is always successful.
Even now there are combinations of viruses and virus scanners (e.g. Yaha.K + Sophos) that fail to be detected due to a malformed
MIME header, which gets decoded differently (and correctly, considering
standards!) by MIME::Parser, yet certain mail readers decode it differently,
forming up a virus. It sometimes helps to use more than one virus scanner (e.g.
clamd along with some commercial virus scanner).
RFC 2046 defines a way to split sending one document into several e-mail
messages, which can then be reassembled (automatically or manually) by MUA. The
Content-Type values to look for are message/partial (and also
message/external-body). Checking mail fragments individually for viruses
can not reliably detect viruses, which only get reassembled into a recognizable
form by the recipient's mail reader. Most virus scanners at the MTA level
(including amavisd-new and all other variants of amavis*) check each mail
independently from other messages, so the only protection to this threat is to
ban these MIME content-types (see $banned_filename_re setting in amavisd.conf),
or by disabling auto-reassembly at mail readers.
Because amavisd-new tries to recursively unpack and decode each mail
as deeply as possible, this may be abused by malware. The so-called mail
bomb, e.g. 42.zip, is an example of such a malware. Such mail message, when
fully decoded, can exceed available disk size several times, or consume a lot of
time for decoding. Unless decoding is stopped at an earlier stage, it could
cause the message to be retried over and over again, each time either hitting
the disk full condition, or exceeding the allowed time limit.
amavisd-new has several configurable mechanism for limiting the amount
of space consumed during decoding - see Section VI - Resource limitations
in file amavisd.conf. When a message decoding exceeds the storage quota, the
decoding stops, the virus scanning is not performed, but a header field is
inserted, telling MTA to place the message 'on hold', or just pass it - the
action depends on MTA configuration. This works well with Postfix, but may not
be configurable with some other mailers.
When a content filter is positioned in relation to MTA as a mail relay,
accepting mail from MTA, and passing all checked mail to MTA for final delivery
(e.g. Postfix, or non-milter sendmail setup), there are only two possible
approaches that can prevent mail loss when unpredictable things happen:
- committing messages to secondary storage and keeping evidence, e.g.
properly implementing a queueing mechanism, as MTAs do it, or
- confirming mail reception to the input-side MTA only after the forwarded
mail reception has been confirmed by the MTA on the output side (or a
non-delivery notification sent). This approach is used by transparent SMTP
proxy, or by cooperating SMTP server and client.
amavisd-new chooses the second approach. Several alternative mail
content filtering solutions based on Perl module Net::Server::SMTP can not
guarantee not losing mail under certain circumstances, because they confirm mail
reception before being in position to ensure a mail delivery or bounce.
Besides taking care of not losing mail, it is important the mail contents is
not unintentionally changed, as could happen for example when disk is full, or
communication or I/O errors occur. amavisd-new is very thorough in always
checking the status of operations, e.g. all I/O operations, creating/deleting
files, calling external programs, checking all SMTP response codes, etc. When a
problem occur, amavisd-new tries to produce an error report in its log
file that is as informative as practical. When the situation can not be
corrected, a temporary failure (EX_TEMPFAIL, or 450 SMTP response) is generated,
telling MTA to try again later.
The amavisd-new policy is to either deliver the mail, or to make sure
the sender gets a non-delivery notification. It is possible to configure
amavisd-new to disobey this policy for certain unwanted contents, e.g.
only to quarantine spam and not generate bounces - but such a setting is not in
the spirit of not losing mail.
amavisd-new may modify mail headers, but it never modifies mail body,
even if configured to call SpamAssassin.
mm
Last updated:
2003-03-24
