Date: 27 Jul 1994 07:56:21 -0600 (MDT) From: Chris McDonald Subject: Product Test, PT-70, ProTec Professional To: orvis@icdc.llnl.gov To: cmcdonal@wsmr-emh34.army.mil Apparently-To: orvis@icdc.llnl.gov ******************************************************************************* PT-70 July 1994 ******************************************************************************* 1. Product Description. ProTec Professional is a commercial program to provide file integrity with restoration facilities upon the detection of any changes in file structure or program modification. 2. Product Acquisition: Digital Enterprises, Inc., developed the program. The firm can be reached at 818 West Diamond Avenue, Gaithersburg, MD 20878. The telephone number is 301-926-6937. 3. Product Tester: Chris Mc Donald, Computer Systems Analyst, Directorate of Information Management, White Sands Missile Range, NM 88002-5030, DSN 258-7548, DDN cmcdonal@wsmr-emh34.army.mil. 4. Product Test: a. I received an evaluation copy in late 1993 and began testing in October 1993 on a variety of platforms running MS-DOS 5.0. The evaluation copy expired in the first quarter of 1994. Various factors precluded an evaluation report at that time. For this reason I would strongly suggest that interested readers contact the vendor directly for the latest product information. b. The ProTec Professional manual consists of sixty-five pages, twelve of which address descriptions of several "common viruses". The document describes the program in this manner: "ProTec Professional is the most advanced program for validation and protection of your computer system applications and data files. Using advanced signature authentication techniques for true DOS data verification, capable of detecting even the most subtle changes in your data, and as a plus it eliminates the need for virus scanners". The tortured nature of the second sentence was courtesy of the vendor. At the end of the first four pages my suspicion was that the manual had an infection of hyperbole. Subsequent testing confirmed the suspicion. c. I found that the installation routine performed as documented. It did take some time for the program to complete an initial "bit by bit analysis" of systems memory configuration and of all files to record a unique signature. This initial analysis also included a "check for any known or suspected virus code" by the ScanPro component of the program. The manual warned that ScanPro might flag files compressed with programs such as PKLIGHT and PKZIP as "possible virus carriers since many virus codes cannot be detected in zipped files." This occurred repeatedly during the installation resulting in an inordinate number of Type I or false positive alarms. d. Upon the completion of the analysis I had the option to allow the program to automatically modify the autoexec.bat file which would then invoke the program's integrity checking of the system upon subsequent booting. This checking would utilize the fat.log and scan.log files created during the installation. I accepted this option, and then conducted tests on the ability of the program to detect change. e. During dozens of operations the program functioned as documented to detect "change". I intentionally altered "signatures" through several disk editors, infected files with known viral samples, deleted files with signatures, and added files without creating signatures. In every case the program generated the expected alert message, and offered me the opportunity to take some type of action. For example, I might have the option to "fix" or to "restore" a file to its former image; or in the case of files added without a signature to add the file to the scan.log file. f. I also tested the program from the command line to examine the performance of other switches described in the manual. Those included: (1) /ALL Act on the entire drive, starting at the root directory and include all sub-directories (2) /C Do not report compressed files or file compressors as possible virus infections (3) /Q Provide a quick check of file integrity, but do not calculate or compare checksums (4) /S Include sub-directories, starting from the current directory (5) /VS Disk scanning option for known virus code (6) /X Erase fat.log and scan.log files from the current directory (7) /Y Skip memory scan prior to loading the main ProTec program These options performed as documented. g. Notwithstanding the performance of ProTec Professional in detecting change, there were certain items which detracted from its overall impression. (1) Type I alarms on various compressed programs made it impossible to eliminate the requirement for another viral scanning detection program. If ProTec Professional by default will alarm on every LZEXE compressed file, then a user will need some other tool to resolve these alarms. (2) The program at the version tested stored its scan.log files in individual sub-directories. Several commentators have suggested in the review of other comparable programs that to distribute the so-called signature files throughout a disk is inefficient, and that it would be desirable to have a single signature database file. 2 (3) The manual and on-line assistance did not provide information on the specific methodology and/or on the algorithm utilized to generate signatures. The National Institute of Standards and Technology has suggested that this data may be important to an acquisition decision. (4) Integrity or change detection programs place demands on an individual user to be thoroughly familiar with what is "normal" on a system. ProTec Professional has many options which will demand that users read the manual and then have access to a knowledgeable support staff when alarm messages begin to occur. I have concerns that many enterprises will not provide the education and infrastructure to adequately support the program. (5) Restoration of files was not always successful, particularly when bytes beyond the first 128 were modified. 5. Product Advantages: a. ProTec Professional represents a level of protection beyond that provide by conventional viral signature detection software. In theory this offers a tool more sophisticated than standard anti-viral detection programs. b. The program has an option to establish a "supervisory" password so that individual users cannot change settings or modify integrity checking operations. 6. Product Disadvantages: a. While the program should be devoid of false negatives for the identifi- cation of a change to a file, change in itself doe not always imply that a viral infection has occurred. For example, a change to a baseline may only reflect non-malicious self-modification, recompilation, or intentional user update to a file. b. The documentation included with the evaluation copy might overwhelm a novice user. Conversely, many expert users may demand detailed information on the procedures and algorithm utilized to generate the integrity database. 7. Comments: The National Institute of Standards and Technology, Computer Security Division, has issued Special Publication 800-5, "A Guide to the Selection of Anti-Virus Tools and Techniques", December 2, 1992. The publication is available for anonymous ftp from the NIST host 129.5.54.11 in the path /pub/ nistpubs. Many vendors have now included integrity checking within their viral signature detection software. Therefore, a user has a number of commercial and shareware programs from which to make an acquisition decision. It would seem reasonable to expect that the authors of ProTec Professional must address the Type I alarms generated by certain compression programs to compete with other available products. 3 LTC Fred Kolbrener, U.S. Army, performed tests on an earlier version of the program, and kindly provided me his results. Interested readers can send me a FAX number to obtain a copy of his four page review. [The opinions expressed in this evaluation are those of the author, and should not be taken as representing official Department of Army positions or a commercial endorsement.] FOR FURTHER REFERENCE: PRODUCT TEST NUMBER PRODUCT PT-3 VIRUSCAN PT-7 CHKSUM PT-8 FILETEST PT-17 F-PROT PT-23 VIREX-PC PT-24 VIRUSAFE PT-25 DR. SOLOMON'S TOOLKIT PT-28 NORTON ANTIVIRUS PT-36 CENTRAL POINT ANTI-VIRUS PT-39 THUNDERBYTE SCANNER PT-40 ALLSAFE PT-59 IBM ANTIVIRUS/DOS PT-64 STOPLIGHT PT-65 F-PROT PROFESSIONAL 4