Date: 30 Jun 1994 07:30:50 -0600 (MDT) From: Chris McDonald Subject: Revised Product Test, PT-39, Thunderbyte Anti-Virus, version 6.20 To: orvis@icdc.llnl.gov To: cmcdonal@wsmr-emh34.army.mil Cc: krvw@agarne.ims.disa.mil Apparently-To: orvis@icdc.llnl.gov ******************************************************************************* PT-39 Revised June 1994 ******************************************************************************* 1. Product Description: Thunderbyte Anti-Virus (TBAV) is a toolkit available both in shareware and in commercial versions to detect, protect and recover from viruses and malicious programs for MS-DOS environments. This program test addresses version 6.20 of the viral scanning utility within the toolkit. 2. Product Acquisition: The shareware documentation describes various methods by which one can acquire the program. There is also detailed information on licensing and registration requirements. There are separate registration sites for the USA and for Europe. Frans Veldman is the program author. The documentation gives the following address for more information: ESaSS B.V, P.O. Box 1380, 6501 BJ Nijmegen, The Netherlands. One USA registration point is Thunderbyte USA, P.O. Box 175, Madisonville, TN 37354 with a telephone of 615-442-2196. There is also a commercial version of Thunderbyte Anti-Virus with a list of respective dealers/distributors available in the agents.doc file contained within the shareware version. 3. Product Tester: Chris Mc Donald, Computer Systems Analyst, Directorate of Information Management, White Sands Missile Range, NM 88002-5506, DSN: 258-7548, DDN: cmcdonal@wsmr-emh34.army.mil. 4. Product Test: a. I initially acquired the shareware version through anonymous ftp from Internet sites in 1992. I became a registered non-commercial user in 1993, and have continued that registration. I have obtained updates either from the Thunderbyte USA BBS at 615-442-2833 or from the Internet host SimTel at 141.210.10.117. b. Product tests have occurred on a variety of IBM-compatible platforms running MS-DOS 3.30 through 5.0. The documentation suggests that one have MS-DOS 3.30 or above for best results. c. One invokes the scanner by the syntax TBSCAN [path] [options]" or for example "TBSCAN c:\". The syntax is case insensitive. By default the program does a "sanity check" of itself and will scan memory, the boot sector/partition table, and files with known executable extensions, such as .com, .exe., and .sys. There are numerous options which one may invoke with "short-keys" or with "option words". I tested these options in the latest evaluation with no anomalies or problems encountered. optionword short explanation help he help allfiles af scan non-executable files mutant mu enable fuzzy search heuristic hr enable heuristic alerts compat co maximu compatibility mode noboot nb skip bootsector check sector ss scan all disk sectors nomen nm skip memory check subdir sd scan in sub directories nosub ns skip sub directories repeat rp scan multiple diskettes log lo output to log file append ap log file append mode noautohr na no auto heuristic level adjust logname ln set path/name of log file d. An alternative method of invocation, and the one I prefer, is to use TBAV.EXE which provides a menu-driven interface for TBSCAN as well as for other utilities bundled within the Thunderbyte Anti-Virus toolkit. e. I tested TBSCAN against a suite of 5, 861 malicious programs which included 80% to 94% of the so-called "in the wild" viruses. The percentage varies depending upon whose "in the wild" suite one chooses (i.e., "Virus Bulletin", VSUM by Patricia Hoffman, or FreqList by Joe Wells). The program identified 100% of the "in the wild" samples and overall flagged 5, 642 samples as infected. The test suite included samples of the various automated engines which have appeared (i.e., VCL, VCS, MtE, TPE and others). f. The "Virus Bulletin", January 1994, evaluated the performance of version 6.08, with equally impressive results against the Bulletin's "in the wild", standard, and Mutation Engine test sets. The program was one of six out of nineteen to have had an accuracy of 100%. g. The "Virus Bulletin", June 1994, contains an evaluation of several programs with heuristics analysis capabilities. Thunderbyte Anti-Virus was the "overall winner" in terms of detection and usability. The other programs tested included CPAV, F-PROT and AVP. h. My own tests of this heuristic capability generated these results. (1) The number of Type I or false alarms did increase as the documentation suggested. Under maximum heuristics, for example, the program issued warning messages on virus simulations and on other anti-viral software programs, such as VIRSTOP.EXE and CS-TSR in F-PROT Professional. (2) There are at least 28 different "flags" or factors which could trigger an alarm. These range from "program load traps", "suspicious jump constructions", "suspicious memory allocations", etc. Against the test suite of actual viral signatures I was able to trigger 16 of the flags. Comparing the flags against descriptions of the actual viruses confirmed that the alert messages were valid. (3) In those instances where TBSCAN already had the signature of the virus or family of viruses in the test suite, one received a specific message of viral infection along with the listing of specific flags. (4) The most interesting results were against viral signatures for which TBSCAN had no specific signature file. The CD-ROM collection of viruses 2 released by American Eagle Publications provides a directory of alleged viruses previously unknown. The directory contains 404 executables. TBSCAN flagged 310 executables as either viral infected or suspicious. While it was apparent that many of the viruses were actually known to version 6.20 of TBSCAN, the analysis of those that were not was impressive. i. The author has lengthy documentation files. If one really wants to understand the program, one must read the documentation to learn what checking, tracing, scanning, and browsing mean in relationship to TBSCAN execution and its heuristic characteristics. 5. Product Advantages: a. Thunderbyte Anti-Virus has excellent detection capabilities for viral signatures. Its detection of "unknown" signatures was good based upon the limited number of test samples. b. The menu-interface is adequate for the normal user. Command line syntax offers flexibility for experienced users. Additionally, the menu- interface supports other utilities within the Thunderbyte Anti-Virus package. c. Updating of the signature data file occurs on a frequent basis with efficient and cost-effective distribution over the Internet and BBS. d. The scanner remains the fastest on the market with no apparent impact on detection performance. e. The TBAV toolkit includes additional utilities for integrity checking, viral disinfection/removal, and memory resident scanning. While this product test does not address the performance characteristics of these programs, the author has attempted to provide a comprehensive approach to malicious program attacks. 6. Product Disadvantages: a. Viral scanning programs will at some point generate Type I or false positive alarms. TBSCAN with its heavy reliance on heuristics will present this problem. b. There is some reluctance on the part of certain government agencies to acquire foreign-produced programs. Although TBAV has a US distributor, I have encountered strong objections from some government personnel whenever I have evaluated such programs with favorable comments. I am not qualified to discuss so-called information warfare concerns. 7. Comments: It seems reasonable that one would stockpile at least two virus protection programs to ensure continuity of operations in the event one program source either terminated support or was no longer available. Two programs also give one a better opportunity to confirm an infection and to eliminate the 3 possibility of a false alarm. The heuristic scanning feature represents an innovative approach to malicious code detection. While there are obviously "bugs" in any experimental work, this feature represents the next level of malicious program detection suggested in Catherine Young's paper "A Taxonomy of Computer Virus Defense Mechanisms". The acquisition and use of viral detection programs requires some thought. Interested readers may consult the Proceedings of the National Computer Security Association's 2nd International Virus Prevention Conference & Exhibition, February 1993, for several papers on the subject to include one entitled "Selecting an Anti-Virus Product". The National Institute of Standards and Technology has issued a Special Publication 800-5, "A Guide to the Selection of Anti-Virus Tools and and Techniques", December 2, 1992. The publication is available for anonymous ftp from the NIST host 129.6.54.11 in the path /pub/nistpubs. [The opinions expressed in this evaluation are those of the author, and should not be taken as representing official Department of Army positions or a commercial endorsement.] FOR FURTHER REFERENCE: PRODUCT TEST NUMBER PRODUCT PT-3 VIRUSCAN PT-12 VIRUCIDE PT-17 F-PROT PT-23 VIREX-PC PT-24 VIRUSAFE PT-25 DR. SOLOMON'S TOOLKIT PT-28 NORTON ANTIVIRUS PT-31 DATA PHYSICIAN PLUS! (VirHunt) PT-34 IBM ANTI-VIRUS PT-36 CENTRAL POINT ANTI-VIRUS PT-40 ALLSAFE PT-41 VIRx PT-45 VIRUS PREVENTION PLUS PT-48 VIRUSCURE+ PT-51 PC-RX PT-52 VIRUSCLEAN PT-55 GOBBLER-II PT-58 VIRUS BUSTER PT-59 IBM ANTIVIRUS/DOS PT-61 VDS PRO PT-64 STOPLIGHT PT-65 F-PROT PROFESSIONAL PT-69 VI-SPY 4