The following problems have been encountered with the different scanners during the tests. AIDSTEST: --------- 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 3) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 4) The scanner crashes when scanning some boot sectors with incorrect BIOS Parameter Block in them, merely two samples of Form. Those are the same boot sectors that crash SCAN. All this makes it very difficult and time consuming to test it. Unless the producer fixes the above problems, we will not test this scanner any more in the future. AVAST: ------ 1) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). This makes it very difficult and time-consuming to test it. If the producer does not fix this problem, we will probably discontinue testing the product. AVK: ---- 1) The scanner (SUCH) is unable to scan multiple floppies without exiting (Condition 12). The interactive scanner (ANTIVIR) polls the keyboard directly, so it is not possible to use SimBoot's keyboard simulator to tell it to scan another floppy from the menu. In order to get at least some boot sector virus test data, we instructed SimBoot to start the scanner anew for each boot sector image. Even then, the scanner crashed on several boot sector images - exactly those on which IBMAV/DOS has problems. AVP: ---- 1) The scanner is unable to scan multiple floppies without exiting (Condition 12). It's keyboard handling routine polls the keyboard directly, so it is not possible to use SimBoot's keyboard simulator to tell it to scan another floppy from the menu. 2) When started, the scanner complains that the database CA.-VB is out-of-date. This database does not contain virus descriptions, so updating it every three months shouldn't be required. 3) The program -VPRO.EXE crashes when run under DesqView. The main scanner, -V.EXE runs OK, though. 4) The program is extremely sensitive to the amount of memory available to it. If the memory is not sufficient, the program becomes unreliable, like not being able to detect all viruses it normally does, being unable to disinfect some viruses and so on. 5) When started from a write-protected floppy, the scanner complains that the floppy is write-protected, even if it is instructed not to create a report file, and asks for a path to a drive with at least 2 Mb free disk space. This is unacceptable, because starting the scanner from a write-protected floppy is the most secure and recommended way to run it. AVSCAN: ------- None. BRM_SCAN: --------- 1) Actually, this is not a scanner. It is only a scanning engine which is under development, with a crude user interface, allowing it to be tested. Since the user interface was added only for testing purposes and the will be completely different in the final product, a normal user certainly wouldn't like it. However, it allowed us very easily to test it, because it was designed especially to be easily testable and conformed to all our conditions. In fact, since this is not a ready product, available on the market, we weren't sure whether to include it in this report. However, since we did test it anyway (in order to help the producer in its development), we had complete test results. They were so impressive, that we couldn't resist and decided to include it, after verifying that the producer does not object to it. However, this scanner was not included in the final classation of "best" and "worst" products. 2) When scanning multiple simulated floppies, the scanner stopped after the first one, with a message that it cannot read the boot sector. However, it was able to run perfectly in a DOS window under DesqView. We do not know what the problem exactly is, but the producer is strongly suggested to fix it. 3) The scanner crashed when scanning one particular file. The reason was that this file had an invalid date of last modification (the day was 0). According to the producers, this is caused by a bug in the library that comes with the compiler they are using and will be fixed in a next release. CHKBOOT: -------- 1) ChkBoot is not a general-purpose scanner; it is a heuristic analyser for boot sector viruses *only*. Therefore, only boot sector virus data is available. 2) Since the program is able to output basically only two kinds of reports about a boot sector (infected/clean), we didn't include it in the final classation for "best" and "worst" scanner - otherwise it would have shown best results in reliability of the detection and worst result in number of different reports. 3) The detection rate was amazing - it missed only 3 viruses. We are aware that a newer version exists, which does not miss even them, but Padgett has not released it yet. CPAV: ----- 1) When started from the command line and instructed to scan our virus collection, CPAV simply hangs the machine. When started interactively from the menu and instructed to switch to the drive containing our virus collection, the program begins to count the existing directories. When it is almost at the end, it displays a message that there is not enough memory and hangs the machine. It is obvious that the scanner is unable to handle big directory structures (Condition 10). 2) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 3) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 4) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 5) The scanner is unable to scan multiple floppies without exiting (Condition 12). One alternative solution to this problem is to repeatedly select the floppy drive from the menu. Unfortunately, the scanner polls directly the keyboard for user input, and SimBoot is unable to simulate keyboard input in this case. 6) When tested on some polymorphic viruses from our collection, the scanner simply crashed while examining them. All the above problems make the product simply *untestable*. We didn't succeed to obtain any test data for it, or even to run it for that matter. Having in mind the many other deficiencies that exist in CPAV, we strongly recommend that people do *not* rely on this product for virus protection. CURE: ----- 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not scan SYS and BAT files (Condition 5) and allows the user to specify only one additional extension to scan. 3) The scanner is unable to scan multiple floppies without exiting (Condition 12). We tried to start the scanner anew for each of the simulated floppies, but then it turned out that it does not report anything in the report file, if it does not consider the floppy as infected (Condition 13). Therefore, no boot sector virus test data is available. 4) It is not possible to instruct the scanner from the command line about the name of the file where the report must be saved (Condition 8). 5) When we started the scanner on our virus collection, it scanned a few files and aborted with a "Stack overflow" error. All those problems make the scanner untestable. We will not bother to test it in the future, unless the producer seriously improves the product and fixes the problems mentioned above. Meanwhile, we strongly advise the users against relying on this scanner. FINDVIRUS: ---------- 1) This scanner is the best when it comes to exact virus identification. Unfortunately, the latest version has a few identification problems - sometimes it says "identified" with one and the same name about two different viruses, or with different names about different samples of one and the same virus. 2) During the boot sector test, the scanner emits a horrible beep each time it finds a virus on the simulated floppy. There should be a way to turn the beep off (Condition 9). 3) The program becomes extremely slow when told to identify exactly some viruses. For instance, it took 12 minutes on a '486-based computer (!) to scan a *single* file containing a variant of the Dir_II virus. F-PROT: ------- 1) The scanning of boot sector viruses often results in "unknown" variants. It seems that the exact identification for those boot sector viruses has to be improved. I-MASTER: --------- 1) The report file generated by the program is huge and extremely verbose - even when the /1 option is used. It might be difficult for the user to orient him/herself in it. Automatic preprocessing of the information *is* possible, although not easy. As an example of how verbose the report file is - the full report file of the product when performing virus scan on our collection of boot sector viruses occupies almost 3 megabytes. The same report preprocessed, so that only the essential information is extracted from it, occupies about 35 kilobytes. IBMAV/DOS: ---------- 1) When scanning some floppies with boot sector images on them that do not have a correct BIOS Parameter Block, the scanner causes a DOS critical error (Abort, Retry, Ignore, Fail?) and regardless of the user input always aborts. This causes serious problems during the tests and we urge the producers to fix the problem. Fact is, that other scanners do not have it. There is a total of 14 boot sectors causing this problem. They will be provided to the producer on request. Interestingly, the previous version of the scanner, 1.05, does not have any problems with those boot sectors. 2) The scanner performs exact identification of a very limited number of viruses. However, in several cases I've found this identification not to be correct - it reports samples of one and the same virus as different ones. One particularly frappant case is the Frodo virus - the program seems to include part of the virus that gets trashed in its virus maps. In practice, this means that it won't be able to disinfect this virus in those cases. MSAV: ----- 1) When started from the command line and instructed to scan our virus collection, MSAV simply hangs the machine. When started interactively from the menu and instructed to switch to the drive containing our virus collection, the program begins to count the existing directories. When it is about in the middle, it displays a message that there is not enough memory and stops. If an atempt is made to scan the drive at that time, the program does not scan anything and displays its (empty) Statistics window. Unline CPAV, though, MSAV does not hang in this case. Nevertheless it is obvious that the scanner is unable to handle big directory structures (Condition 10). 2) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 3) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 4) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 5) The scanner is unable to scan multiple floppies without exiting (Condition 12). One alternative solution to this problem is to repeatedly select the floppy drive from the menu. Unfortunately, the scanner polls directly the keyboard for user input, and SimBoot is unable to simulate keyboard input in this case. 6) When tested on some polymorphic viruses from our collection, the scanner simply crashed while examining them. All the above problems make the product simply *untestable*. We didn't succeed to obtain any test data for it, or even to run it for that matter. Having in mind the many other deficiencies that exist in MSAV, we strongly recommend that people do *not* rely on this product for virus protection. NAV: ---- The program suffers from a brain-damaged design, especially in the part that is responsible for the reports, which makes it completely unsuitable for testing, and often even for ordinary use. 1) It is impossible to run the program from the command line and tell it to scan a particular directory subtree and to create a report file with a name supplied by the user, which lists all files that have been scanned (Condition 8). There is a /report option, but it is bogus - it simply doesn't work from the command line. OK, we made an exception and ran the program interactively. At this point the users who plan to use the product from batch files, unattended, should already look into something else. 2) It is not possible to tell the program to list in the report file ALL files that have been scanned (Condition 3). It lists only the files that it thinks contain a virus. We were given the idea to run the program in "inoculate" mode, and when it finishes, refuse the inoculation. This creates a report file, where the name of each file is listed, the name of the virus found in it (if any), and its inoculation status (in this case - not inoculated). BTW, Symantec uses the term "inoculation" to mean that a checksum is computed for the file and stored in a database. This is clearly misleading; in the anti-virus industry the term is usually used to mean that something is attached physically to the protected files (i.e., they are modified), in order to make them virus-resistent. 3) OK, we tried this too. In this mode, the report for each file occupies multiple lines (file name, virus name, inoculation status), which sometimes makes the automatic preprocessing of the report a bit problematic. This is actually not a problem, however, if those multiple lines conform to some reasonable rules - like each line is contains a tag that identifies it, at least the line with the file name is always present, and so on. At a first look, it seems that the report generated by NAV 3.0 when used in this mode conforms to the above requirements. But only at a first look! Carefull examination showed that the report is severly screwed up. First of all, the line that is supposed to contain the file name is sometimes split in two. That is, sometimes you have Name: c:\some\dir\with\a\file and sometimes Name: c:\another\dir\longer\this\time\with\another\file 4) OK, we figured out how to handle this too. But this is not all! Sometimes, when the full file path is rather long, it is simply truncated! In extreme cases this can mean that the file name is simply missing from the report file and only part of the directory path is there. So much for those who expect to get a reliable report of all their infected files. We can handle weird report file formats, but we certainly cannot handle information that is missing or corrupted. The final kludge that allowed us to create some kind of report file with all the information in it, was to assign a drive to the directory subtree that we wanted to scan, and then tell NAV to scan that "drive". Since the full paths of the files on this "drive" were shorter, they were not truncated. However, in real-life situations, it is clearly unacceptable to expect such kludges from the user. 5) The scanner is unable to scan multiple floppies in sequence (Condition 12). We couldn't do it interactively from the menu either, because it polls directly the keyboard for user input and therefore SimBoot's keyboard simulator is useless in this case. 6) The scanner has a few other drawbacks (for instance, it is unable to run on a 8088, so if you have XTs in your organisation and intend to use a single product for virus protection - forget about this one). 7) Some time ago it was claimed that, unlike NAV 2.1, version 3.0 of the product is able to detect all MtE-based viruses reliably. Our tests demonstrate this claim to be wrong - the virus Destructor:MtE is not detected at all, and the viruses Ludwig.[A-C]:MtE are not detected reliably. All these problems make testing the scanner extremely difficult and time-consuming. Having in mind the relatively low detection rate that the scanner demonstrated, we decided not to bother to test it in the future, unless Symantec fixes all the problems reported above. We strongly discourage the users from relying on this scanner for virus protection. NORMAN: ------- 1) When we attempted to run this scanner, it greeted us with the message "This DEMO-version has expired". (We obtained a copy of the scanner from the producers on the Hannover Computer Fair.) Since we do not review crippleware, we decided not to bother to set the date on the computer back. Previous tests showed that the detection rate is nothing particularly high - something about 75%. PCVP: ----- 1) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 2) The scanner does not allow the user to specify the name of the report file (Condition 8). Instead, it always writes the report to the file SCAN_x in the directory from which PCVP has been started, 'x' being the drive on which the directory subtree being scanned resides. 3) The scanner has bugs when reporting viruses written in a high-level language, that spread in compressed form (e.g., LZEXE-compressed). Versions 1.x of the scanner had the problem of not reporting such files as infected at all in the report file. This bug has been reported to the producer, and they promised to have it fixed. The "fix" in version 2.01 has another problem - now those files are sometimes reported once, and sometimes reported twice - once as not infected and then as infected. Obviously, the producer has never bothered to test whether the fix actually works. 4) It is interesting to note that the latest version of the scanner that we tested (2.02) demonstrated almost 10% *lower* detection than the previous version (2.01), when used on the same virus collection. The producer is strongly suggested to fix the above problems, or we will discontinue testing the product, as the test being too time-consuming and not worth the effort. SCAN: ----- 1) The scanner is unable to scan BAT files and does not allow the user to indicate which extensions are to be scanned (Condition 5). 2) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 3) The scanner crashes when scanning some boot sectors with incorrect BIOS Parameter Block in them, merely two samples of Form. The error that occurs is "Divide error". However, since there were only two such samples, we decided to remove them and run the test nevertheless. The two samples crashing the scanner are clearly marked as such in the preprocessed report file and will be made available to the producer, on their request. The above problems make the scanner very difficult and time-consuming to test. Having in mind that its detection has dropped under the "good enough" limit, we think that it is not worth the effort to test this scanner any more, unless the producer takes care to fix the above problems. SCAN2: ------ 1) The scanner is unable to scan BAT files and does not allow the user to indicate which extensions are to be scanned (Condition 5). 2) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 3) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 4) Each time the scanner detects a virus on the simulated floppy, it emits a horrible sound. There is no way to turn this sound off (Condition 9). 5) Curiously, version 2.10 of the scanner demonstrated results that were significantly worse than those demonstrated by version 2.02. It seems that the program is getting worse and worse with time. All those problems make it very time-consumming to test the scanner. Since it also demonstrated a relatively low detection rate and *extremely* low reliability of the detection, we decided not to bother to test it in the future, unless the producer seriously improves it, and fixes the problems mentioned above. We discourage the users from relying on this product for virus protection - it leaves the impression to be unfinished. SWEEP: ------ 1) The scanner is unable to scan BAT files and does not allow the user to indicate which extensions are to be scanned (Condition 5). 2) The scanner is able to scan multiple floppies without exiting (Condition 12). However, SimBoot was unable to intercept the prompt between each two floppies and/or to simulate keyboard input. We had to "press the any key" manually. This was very annoying thing to do 645 times and unless the problem gets fixed, we do not intend to do it any more in the future. 3) The scanner is unable to scan a subdirectory tree - it can scan only whole drives (Condition 8). 4) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). All those problems make it very time- and effort-consuming to test the scanner. We have decided not to test it any more in the future, unless the producer fixes the problems mentioned above. TBSCAN: ------- 1) The scanner crashes when scanning some boot sectors with incorrect BIOS Parameter Block in them. One of the samples crashes the scanner immediately, while another one causes it to crash on exist. In both cases, no report file is available after the crash. The two samples that crash the scanner are *not* the same two that crash SCAN. This problem has been reported to the producer when the scanner was at version 6.02, yet the problem is still not fixed. 2) We would have removed the two samples, as we did with the test of SCAN, but there was another problem. Several samples cause the scanner to flash an "Access denied" error message and not to report anything in the reprot file for that simulated diskette. Since there were many such cases, and since there is no trace of them in the report file, determining which exactly are the problematic samples would be too difficult. Therefore, no boot test data for this scanner is available. UTSCAN: ------- 1) The scanner is unable to scan multiple floppies without exiting (Condition 12). It also polls the keyboard directly when expecting user input, so it is not possible to use SimBoot's keyboard simulator to tell it to scan another floppy from the menu. We tried to start the scanner anew for each of the simulated floppies, but then there were other problems. First, the scanner sometimes "fooled itself", by reporting a virus in memory after scanning an infected boot sector. Second, it crashed when scanning some floppies with incorrect BPB. At this point we decided that it is not worth the effort to try to test the scanner for boot sector virus detection. Therefore, no boot sector tests have been done and no such results are available. 2) The computer containing the virus collection runs a disk encryption driver, which reserves 5 DOS logical drives for mounting encrypted volumes. Those drives are reported as "removable" by DOS (i.e., they behave like floppy disk drives). If an attempt is made to access them when no encrypted volumes are mounted, the driver returns a "Bad drive unit" error. This confuses UTScan and it offers no other opportunity than to exit without scanning anything. In order to perform the tests, we had to remove the encryption driver. V-CARE: ------- 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 3) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 4) It is not possible to indicate from the command line where the scanner should put the report file (Condition 8). The scanner always creates a report file with a fixed name, in the root directory of the drive being scanned. We wonder how do the users of the product create report files when scanning write-protected floppies... Anyway, this prevented us from testing it on simulated floppies, thus no boot sector virus test data is available for this scanner. The above problems make the scanner difficult and time-consuming to test. Having in mind the very low detection rate that it showed, we decided not to test this scanner any more in the future, unless the producer seriously improves it. VB-LITE: -------- 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 3) The scanner does not list the full paths of the files it reports in the report file - the long ones are shortened by replacing some part in the middle with "...". This makes it impossible to correctly process the report, because some essential information is missing from it. Therefore, we can provide information only about the number of detected samples, not about the number of detected viruses. 4) The scanner is able to scan multiple floppies without exiting (Condition 12), but it polls the DriveChanged line to detect diskette change and also polls the keyboard directly. SimBoot is able to simulate the DriveChanged line, but keyboard simulation does not work under QEMM, when the keyboard is accessed directly by the application. We couldn't run the scanner anew for each of the simulated floppies either, because each time before exiting, the scanner pauses for about ten seconds and wait for the user to press a key. Therefore, no boot sector virus test data is available for this scanner. The above problems, and especially the third one, make the scanner almost impossible to test properly. We decided not to test it any more in the future, unless the producer fixes all of the problems mentioned above. VDS: ---- 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 3) There seems is an implementation bug in VDS, which causes the contents of the configuration file to take precedence over the options specified from the command line. The producer suggested us to use the program VFSLite from the VDS package, which is only a scanner and does not have this problem. 4) When instructed to scan multiple floppy disks for viruses, VDS attempts to remove every virus found. It also displays different prompts between each two floppies, depending on whether the floppy that has just been scanned has been detected as infected or not. At last, when an infected boot sector is found, the report file does not contain any information about it. All this makes the scanner untestable with SimBoot. Therefore, no boot sector virus data is available. VET: ---- 1) The name of one of the files in our collection contains a 8-bit character ("O-umlaut"). VET claims that it has an illegal name. Seems that the producer has not heard of non-English versions of DOS. :-) 2) By default the program removes any viruses it detects - something which can be very annoying to the tester, although it is probably convenient to the normal user. Fortunately, there is an option to turn this behaviour off (Condition 6). 3) The scanner is unable to scan BAT files and does not allow the user to indicate which extensions are to be scanned (Condition 5). VIRC: ----- 1) When the program is ready with the scanning, it stops and waits for the user to press a key. This causes problems to let it run unattended from batch files (Condition 7). 2) When scanning some boot sectors with incorrect BPBs, the program crashes. Therefore, no boot sector virus test data is available. VIRUSCOPE: ---------- 1) The scanner does not scan BAT files. It seems to allow the user to indicate which extensions are to be scanned, but we couldn't make this feature work (Condition 5). 2) It is not possible to indicate from the command line where the scanner should put the report file. One must create a special config file, in order to do this (Condition 8). 3) The scanner does not list in the report file the names of all files that have been scanned - it lists only those it considers to be infected (Condition 3). 4) The scanner does not list the full path of the directory being scanned in the beginning of the report file (Condition 2). 5) The scanner is unable to scan multiple floppies in sequence (Condition 12). We tried starting the scanner anew for each simulated floppy, but then it crashed very severly on some boot sectors. Having in mind all the other problems that the scanner demonstrated, we desided that it is not worth the effort to continue with our attempts to test it. Therefore, no boot sector virus data is available for this scanner. 6) Each time the scanner detects a virus on the simulated floppy, it emits a horrible sound. There is no way to turn this sound off (Condition 9). All those problems make it very difficult and time-consuming to test the scanner. We will discontinue testing it in the future, unless the producer fixes the above problems. VIRUSBUST: ---------- 1) The scanner is able to scan multiple floppies without exiting (Condition 12), but it polls the DriveChanged line to detect diskette change and also polls the keyboard directly. SimBoot is able to simulate the DriveChanged line, but keyboard simulation does not work under QEMM, when the keyboard is accessed directly by the application. 2) It is not possible to instruct the scanner from the command line about the name of the file where the report must be saved (Condition 8). VIS: ---- 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 3) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 4) The scanner can scan only whole drives; not a specified directory tree. 5) The scanner is able to scan multiple floppies in a sequence. However, the only way to generate a report file is to redirect the standard output of the scanner to a file. Unfortunately, this means that the prompt to "press any key" between each two floppies also goes in this file and is not displayed on the screen. Therefore, SimBoot is unable to intercept it and simulate a floppy disk change. Thus, we couldn't produce any boot sector virus test data. All the above problems make the scanner very difficult and time-consumming to test. Unless the producer fixes them, we will not bother to test this scanner any more. VPCSCAN: -------- 1) The scanner is unable to scan BAT files and does not allow the user to indicate which extensions are to be scanned (Condition 5). XSCAN: ------ 1) The scanner does not list the path being scanned in the beginning of the report file (Condition 2). 2) The scanner does not list in the report file the names of all files that have been scanned (Condition 3). 3) The scanner does not scan BAT files and does not allow the user to specify which extensions have to be scanned (Condition 5). 4) The scanner can scan only whole drives; not a specified directory tree. 5) When started, the scanner complains that there is not enough memory, although there is about 630 Kb conventional RAM available. This does not prevent the program from running, though. All this makes it very difficult and time consuming to test it. Unless the producer fixes the above problems, we will not bother to do any more tests of this scanner.