Introduction:
-------------
Vesselin Bontchev (c) 1994, Klaus Brunnstein (c) 1997

With growth of internetworking, and with growing complexity of systems
and software, threats to individual and enterprise computing equally
grow. A growing number of users and institutions become ever more
dependent upon availability, reliability and functional behaviour of
their IT and Network systems. Moreover, storage, processing and transfer
of sensitive information requires protective measures against malevolent 
attackers and malicious software.

For some time, malicious software was mainly understood to be of "viral
nature": when such pieces of software entered one`s PC, it could spread
by self-replication, either on the system level (boot/MBR/DIR viruses)
or via infected programs (*.COM, *.EXE, *.SYS, *.BAT etc). With growth of
PC user numbers, a market of AntiViral products developped to help users
fight such software. Moreover, several PC-related magazines started testing
the quality of AV products using their own (usually small) virus databases. 
The quality of such test has rather often been discussed "controversially".

With further growth of file/boot viruses (more than 12,500 file) and the
advent of document-related viruses (using macro languages to infect master
templates) , there is an urgent need of professional tests of anti-virus
products. There are several reasons for that. The main one is that the
anti-virus products are not something that the end user is able to
evaluate him/herself. When the user buys a word processor, s/he can
easily see whether it works according to the expectations and whether
it performs the job it is supposed to perform. Not so with the
anti-virus products. An anti-virus product may be installed and
started every day, but its real anti-virus part enters into action
(and shows whether it is any good) only during a real virus attack, but
then, its proper work may significantly influence the users productivity
for some time. Fortunately, regardless of all the media hype, users
experience computer viruses in relatively rare cases. A user could use an 
anti-virus product a whole year, if not more, without needing its anti-virus 
capabilities to stop a virus attack.

Another reason is that an anti-virus product is extremely difficult to
test. In order to test a word processor, one only needs the manual and
some (potentially big) text files. In order to test an anti-virus
product, one needs a lot of things. First of all, the tester of such a
product must have a deep and intimate knowledge of how computer
viruses work, what their methods of attack are, and what the methods
are to thwart those attacks. The tester must know the principles
on which the anti-virus products work. And last, but not least: the
tester must have access to a fairly rich and well-organized virus
collection. The ideal person who has all of the above is the
anti-virus researcher.

Unfortunately, the anti-virus researchers are hard to come by. Most of
them are busy developing and selling their own products. As such, they
cannot test other people's anti-virus products - because the results
will be always biased towards their own. Therefore, one needs an
independent anti-virus researcher, in order to test an anti-virus
product properly. The number of independent anti-virus researchers in
the world can probably be counted on the fingers of one hand.

Yet another problem is obtaining the necessary resources for a good
anti-virus product test. As mentioned, those tests are very difficult
to perform. They require a lot of disk space, a variety of hardware, a
lot of man-hours to complete. The main question is - how to get the
money to fund all this?

One solution is to have the anti-virus companies pay for the tests.
After all, the results are usually very usful to them (in the form of
bug reports), and sometimes can be used for advertising. This approach
is followed by the UK AntiVirus working group which is about to deliver
its formal requirements for"AntiViral Functionalities" within the European
ITSEC scheme. Within in this scheme, an AV producer can apply for an F-AV
certificate which is given after due analysis including proper tests.

Another solution is to have the users of the test results to pay for
the tests - regardless of whether they are an anti-virus company that
just wants to see how well their product performs, compared to others,
or if they are end users, trying to select "the best" anti-virus
product. The main problem with this solution is that, in order to
obtain some sellable results, one need money in advance - to do all
the tests.

One possible basis for independent testing could be a university
institute which specializes in computer and network security. Students
may be interested to study methods and counter-measures of self-
replicationg code. Within the 4-semester courses on IT/Network
Security at the Faculty for Informatics, University of Hamburg, several
students have specialised (including examination work) on virus
detection. 

For the test published here, facilities of the Virus Test Center at 
University of Hamburg was available. Though 5 students and one professor 
worked on preparation and tests for more than 3 months, much more wo/man-
power, time, and computer equipment would be helpful. We are aware that our
test results are limited and need improvement in several directions
(more platforms, more methods including on-access scanning, detection
of viruses in compressed files). Moreover, our results are limited
in time as both the viral databases grow and new scanner engines become
available.

Nevertheless, we have decided to distribute these results to the interested
public, for free. Of course, if you like them and are in a position to 
be able to donate money or hardware to the VTC-Hamburg - we will highly
appreciate this.

One last problem with the anti-virus products, especially those of the
scanner type.  They are modified very often.  This means that their
production cycle is forced to be shorter than for other kinds of
software products.  Usually, the part that comes too short is quality
control.  If it is too difficult for the end user to assess the
quality of the product, it is often too tempting to put more efforts
into making the product to look pretty, instead of making it a strong
anti-virus tool.  Therefore, it is urgent that professional tests of
anti-virus products are performed, and the results - published, so
that the general public can see what they are really paying for.

Unfortunately, even for the competent anti-virus researcher,
performing a professional test of an anti-virus product is often a too
difficult, nearly impossible task. Such products often consist of
several parts - scanners, monitoring programs, integrity checkers. The
latter two kinds of programs must be tested how well they perform
against each of the known attacks against that particular kind of
anti-virus defense. Just implementing those attacks is a difficult and
tedious job. But even such products rely to some degree on proper detection 
of viruses by their scanners.

Usually the part of the product that is the easiest to test is the
scanner. Even that should be done by a professional anti-virus
researcher, instead of the usual magazine reviewer, because there are
a lot of pitfals to watch for. The full description of how a
professional test of an anti-virus product is outside the scope of this 
document and is described in other papers.

Nevertheless, the urgent need for good tests of anti-virus products
prompted us to use our knowledge and technical facilities to test some
of the popular products on the market. This document contains the
results of those tests. Our intention is to update it periodically, as
new anti-virus products, or new versions of the old anti-virus
products appear.

Please, note that the quality of our tests is far from perfect - refer
to the file 9EPILOG.TXT for some points on what is missing from our
tests. Nevertheless, we feel that the results that we can provide are
of superior quality than many so-called reviews of anti-virus
products that we have seen so far. We are concentrating our efforts on
the anti-virus side of the problem and leave the evaluation of the
pretty user interfaces and the structure of the manuals to the
magazine reviewers.

We hope that our results may help the end user to select a better
product to protect him/her from computer viruses. Whether we have
succeeded to achieve our goal, only the users themselves can tell.

---------------------- History of VTC tests: --------------------------------

July 19, 1994:     Previous (last) official boot/file virus recognition test 
                   released by Vesselin V. Bontchev (see directory /1994-07).

May, 1996:         Foundation of new AV Product file/boot virus test group;
                   Foundation of Macro Virus Database; Preparation of test 
                   equipment (NT server/clients), preparing test procedures

November 30, 1996: Standard Virus Databases "frozen" for test; contact to
		   AV producers to get actual versions of scanners, or down-
                   load from Internet where available

December 01, 1996 - December 23, 1996: Pretest used to test the procedures

January 6-
    - February 14, 1997: Update of AV products where available, test runs.  

February 14, 1997: First draft of this report distributed to interested
                   AV experts (closed forum AV-TEST@informatik.uni-hamburg.de)
                   including members from CARO (=Computer Antivirus Researxh
                   Organisation) and cooperative AV-producers

February 20, 1997: Final release of test results.