Introduction.

Currently, there is an urgent need of professional tests of anti-virus
products. There are several reasons for that. The main one is that the
anti-virus products are not something that the end user is able to
evaluate him/herself. When the user buys a word processor, s/he can
easily see whether it works according to the expectations and whether
it performs the job it is supposed to perform. Not so with the
anti-virus products. An anti-virus product may be installed and
started every day, but its real anti-virus part enters into action
(and shows whether it is any good) only during a real virus attack.
And, regardless of all the media hype, computer viruses are still
relatively rare.  A user could use an anti-virus product a whole year,
if not more, without needing its anti-virus capabilities to stop a
virus attack.

Another reason is that an anti-virus product is extremely difficult to
test. In order to test a word processor, one only needs the manual and
some (potentially big) text files. In order to test an anti-virus
product, one needs a lot of things. First of all, the tester of such a
product must have a deep and intimate knowledge of how computer
viruses work, what are their methods of attack, and what are the
methods to thwart those attacks. The tester must know the principles
on which the anti-virus products work. At last, but not least, the
tester must have access to a fairly rich and well-organized virus
collection. The ideal person who has all of the above is the
anti-virus researcher.

Unfortunately, the anti-virus researchers are hard to come by. Most of
them are busy developing and selling their own products. As such, they
cannot test other people's anti-virus products - because the results
will be always biased towards their own. Therefore, one needs an
independent anti-virus researcher, in order to test an anti-virus
product properly. The number of independent anti-virus researchers in
the world can probably be counted on the fingers of one hand.

Yet another problem is obtaining the necessary resources for a good
anti-virus product test. As mentioned, those tests are very difficult
to perform. The require a lot of disk space, a variety of hardware, a
lot of man-hours to complete. The main question is - how to get the
money to fund all this?

One solution is to have the anti-virus companies pay for the tests.
After all, the results are usually very usful to them (in the form of
bug reports), and sometimes can be used for advertising.
Unfortunately, if the tests are sponsored by an anti-virus company,
the end users tend not to believe them - because they have all the
right to feel that the tests have been biased to show the product of
that company in a better light.  Indeed, why would a company pay money
just to have it demonstrated to the general public that their product
is bad?

Another solution is to have the users of the test results to pay for
the tests - regardless of whether they are an anti-virus company that
just wants to see how well their product performs, compared to others,
or if they are end users, trying to select "the best" anti-virus
product. The main problem with this solution is that, in order to
obtain some sellable results, one need money in advance - to do all
the tests.

We have had the chance to use the facilities of the Virus Test Center
at the University of Hamburg to do the tests. Of course, a lot of
things are still missing (mainly man-power, time, and disk space),
which has resulted in the tests being not as good as we would like
them to be. Nevertheless, we have decided to distribute the results
for free. Of course, if you like them and are in a position to be able
to donate money or hardware to the VTC-Hamburg - we will highly
appreciate this.

One last problem with the anti-virus products, especially those of the
scanner type.  They are modified very often.  This means that their
production cycle is forced to be shorter than for other kinds of
software products.  Usually, the part that is undercut is the quality
control.  If it is too difficult for the end user to assess the
quality of the product, it is often too tempting to put more efforts
into making the product to look pretty, instead of making it a strong
anti-virus tool.  Therefore, it is urgent that professional tests of
anti-virus products are performed, and the results - published, so
that the general public can see what they are really paying for.

Unfortunately, even for the competent anti-virus researcher,
performing a professional test of an anti-virus product is often a too
difficult, nearly impossible task. Such products often consist of
several parts - scanners, monitoring programs, integrity checkers. The
latter two kinds of programs must be tested how well they perform
against each of the known attacks against that particular kind of
anti-virus defense. Just implementing those attacks is a difficult and
tedious job.

Usually the part of the product that is the easiest to test is the
scanner. Even that should be done by a professional anti-virus
researcher, instead of the usual magazine reviewer, because there are
a lot of pitfals to watch for. The full description of how a
professional test of an anti-virus product should be performed is
outside the scope of this document and will be a subject of a
different paper.

Nevertheless, the urgent need for good tests of anti-virus products
prompted us to use our knowledge and technical facilities to test some
of the popular products on the market. This document contains the
results of those tests. Our intent is to update it periodically, as
new anti-virus products, or new versions of the old anti-virus
products appear.

Please, note that the quality of our tests is far from perfect - refer
to the file WHY_BAD.TXT for some points on what is missing from our
tests. Nevertheless, we feel that the results that we can provide are
of supperior quality than many so-called reviews of anti-virus
products that we have seen so far. We are concentrating our efforts on
the anti-virus side of the problem and leave the evaluation of the
pretty user interfaces and the structure of the manuals to the
magazine reviewers.

We hope that our results will help the end user to select a better
product to protect him/her from computer viruses. Whether we have
succeeded to achieve our goal, only the users themselves can tell.