PC Magazine Labs Benchmark Series - Release 6.0 Database Definition =================== INTRODUCTION The heart of the PC Labs Benchmark Series - Release 6 (Bench6) is its database. This set of four dBase compatible files and their associated indexes contain information about the various tests as well as descriptions of machines and the results they generated. Bench 6.0 was designed to have a "soft" menu structure. That is, the menus are generated from information in the database each time the program is run. If a test is added to the database it will appear on the menu the next time you use the program. At this time only the Performance menu choice is soft but future releases will expand this concept into the Compatibility and Quality menu areas as well. What you see when you pull down the Performance menu are the test groups entered into the TSTGROUP.DBF file. The specific tests listed in the test selection window come from the TEST.DBF file. The MACHINE.DBF file contains the description of each machine tested. References to these machines appear in the RESULTS.DBF file. The fourth database file is the RESULTS.DBF file. This file stores the results from the various tests in a one test per record format. References to the test group, test and machine information are established via the key fields from the other three files. The Bench6 database is, therefore, relational in its structure. If the database was simply a flat file the information about machines and tests would have to be repeated in each of the result records. The result file is shortened considerably by putting this repetitive data in separate files. The drawback of this technique is that some database programs are not designed to look-up data in related files. If you are going to attempt to access the files using a database package, you should be sure it can look-up related data on secondary files and that it can process .MDX index files. The following sections describe the layout of data records in each file and their individual fields. The last section discusses our future plans for this database. Please feel free to offer suggestions either on the Bench 6.0 registration form or through PC MagNet. Compuserve EasyPlex mail or messages on PC MagNet may be sent to Stuart R. Greenberg, ppn# 72241,64. THE TEST DESCRIPTION FILE (TEST.DBF) Each of the PC Labs benchmark tests is described in the Test Description File. Bench6's user interface component uses this data to build a list of tests to choose from as well as format the test results. The key for this file is the three part Test Id. The first part of the Test Id is a code indicating the originator of the test (MFGCODE). Future releases of the user interface may allow you to switch to another test manufacturer so each of our Test Ids start with "PCMAG " as the MFGCODE. The second part is a code which is used to group related tests together (GROUPID). This is actually a link to the Test Group File and will be described shortly. The final part is a sequence number (TESTNUM) that is used to make the test unique within the first two parts. For example, the 8088 Instruction Mix Test in the Processor Test Group is called "PCMAG PROCSR001" which is saying that this is the first processor test from PC Magazine Labs. The Test Group is identified by the same key structure with the TESTNUM equal to "000". Therefore, if you know the Test Id for a particular test you can find the data related to the group in the Test Group File by looking for a record with a key for the same MFGCODE and GROUPID but with a "000" TESTNUM. The record layout is as follows: FIELD NAME TYPE LENGTH DESCRIPTION ========== ==== ====== =============================== TEST Char 15 Test ID (See above) NAME Char 30 Name of test UNITS Char 15 Units of measure (e.g. Ops/Sec) DEC Num 2 Number of decimal places in result. The test programs cannot do floating point calculations so all numeric results are given as integers with the number of decimal places entered here. SELECT Num 1 Used internally to indicate that the test can be selected. For example, many of the DOS File Access Tests cannot be selected individually. You have to select a test that includes lower level tests in its execution. CPU Num 1 The minimum CPU level required to run the test. (See CPU table below) FPC Num 1 The Floating Point Coprocessor level required to run the test. At present, 0 = none required and 1 = FPC required. INDENT Num 1 Used internally to indicate the level of indentation to use when displaying this test in a list. For example, the DOS File Access tests allow you to select the Large Record Tests and display the individual tests that make up that selection indented below it. MODE Num 1 The minimum video mode necessary to run the test. The currently used modes are: 0 = Text, 1 = EGA and 2 = VGA. The CPU codes are as follows: CODE COVERS CPU TYPES ==== =================== 0 8086 0 8088 0 NEC V30 0 NEC V20 0 80186 0 80188 6 80286 7 80386 7 80386SX 9 80486 and 80486SX The CPU code numbers are not consecutive because the program actually uses the numbers in between to indicate the CPU type of the machine under test. The numbers here actually represent the upper limit on the range for which a particular test is valid. For example, the 386 Instruction Mix must be run on a CPU with at least a code of 7. Therefore, it will run on a 80386, 80386SX, 80486, or 80486SX. THE TEST GROUP FILE (TSTGROUP.DBF) As mentioned above, the various PC Labs benchmark tests are gathered together in logical groups. Each group represents a separate .EXE file containing the test code for that group. The menu items that appear under the Performance, Compatibility and Quality categories represent the test groups. For example, the conventional, expanded and extended memory tests are all in a group called Memory Tests. The Memory Test Group appears on the menu as "Memory..." and resides in the file MEMTEST.EXE. The Test Group File contains this data for each of the test groups. The record layout is as follows: FIELD NAME TYPE LENGTH DESCRIPTION ========== ==== ====== =============================== TEST Char 15 Test ID with the TESTNUM part equal to "000" CATEGORY Char 10 Category to use when placing the test on a menu. Currently, all records have a value of "Performanc". In the future this will be changed to the following: "Perform " = Performance "Quality " = Quality "Compat " = Compatibility NAME Char 30 Long name of test SHORTNAME Char 30 Short name as abbreviated as possible EXENAME Char 13 Name of .EXE file containing the code for this test group THE MACHINE ID FILE The Machine Id File contains data on all machines that have been tested. The key field for this file is broken down into two fields. The first, MACHINE, is an eight character identification for the machine. At PC Labs we use an internally generated number for this part of the key. This number is generated for each item as it is received to enable or inventory personnel to keep track of the thousands of products that pass through PC Labs in a given year. Special consideration had to be given to the possibility that the same machine could be tested with any or all of several options enabled or disabled. For example, a machine may be tested with its disk cache enabled and then tested again without the cache. The inventory number cannot be changed without causing confusion, so instead we added a field to the key called VARIANT. The variant is simply an arbitrarily chosen one character value. A variant of "1" for one machine does not necessarily correspond to a variant of "1" for another machine. It is simply a tag to differentiate various configurations of the same machine. The Machine Id is used as a link between the Machine Id File and the Test Results File. One Machine Id will typically point to several results records in the Test Results File. The description of the Test Results File explains how results from multiple sessions for the same machine are stored. There are some fields allocated to keep track of some standard information such as CPU type and speed and FPC type. While other fields may be added in the future, for now, any additional data is entered into the Description Field. It is 225 characters long and should hold enough information to identify the machine being tested. The layout of the Machine Id File is as follows: FIELD NAME TYPE LENGTH DESCRIPTION ========== ==== ====== =============================== MACHINE Char 8 Unique identifier for a machine VARIANT Char 1 Indicator to differentiate various configurations of a machine (See discussion above) MACHNAME Char 12 Name of machine DESCRIPT Char 225 Description of machine CPU Num 1 CPU type used in machine (See CPU table below) FPC Num 1 Floating Point Coprocessor type used in machine (See FPC table below) CPUSPEED Num 3 Clock speed of machine The CPU type codes are: CODE CPU TYPES ==== =================== 0 8086 1 8088 2 NEC V30 3 NEC V20 4 80186 5 80188 6 80286 7 80386 8 80386SX 9 80486 and 80486SX The FPC type codes are: CODE FPC TYPES ==== =================== 0 None 1 8087 2 80287 3 80387 THE TEST RESULTS FILE This file is the repository of the results for all tests run on a machine. The key for the Test Results File is made up of the concatenation of the Machine Id, Variant, Commit Date, and Test Id. As described in the Machine Id File discussion above, the Variant identifies a particular configuration of a machine. The Commit Date is used in a similar manner to differentiate individual runs of a particular test. For example, let's say you test a machine on Monday and discover on Tuesday that you may have had a glitch in the hard drive. You install a new hard drive on Wednesday and retest the machine. If you committed Monday's data (Commitment of results will be explained soon.) the data from both runs will be saved and their keys will differentiate them by having different Commit Dates. The Bench6 user interface even allows you to compare results from the same machine with different Commit Dates. So, what does it mean to commit the data? During a run, Bench6 tests are executed and the results are stored in the database. A physical write to the database is performed after each test is executed. This result record is considered to be temporary and one such record is written for each test that is run. If a test is re-run in a single session its result will be overwritten. In the event the system crashes for any reason the results logged prior to the crash should still exist as temporary records in the database. When you feel that you have collected all the results you want you can commit the results when you exit Bench6. If any temporary results were still pending when Bench6 is started you can commit them at that time. When results are committed the current date and time are set in the TIME field. All committed records are stamped with the same date and time and that is what allows them to be grouped by run. At this time the stamp is in a format used by Borland C++. In the future the date and time will be in separate fields and will be stored as year-month-day and hour-minute- second ASCII strings. The result itself is always stored as a character string. We currently use numeric results for all of our tests. Future tests may have results such as "Pass" or "Fail" or contain other character based information. Any programs written to process this value should use the Test Id to look-up the Units used for the test and the Number of Decimal Places in the Test Description File to properly format a numeric result. At this time neither the CHECK nor the COMMIT fields in this file are being used. In the future they may be enabled or deleted. Don't store any of your own data in these unused fields. The layout of the Results File is as follows: FIELD NAME TYPE LENGTH DESCRIPTION ========== ==== ====== =============================== MACHINE Char 8 Machine ID (See Machine Id File) VARIANT Char 1 Machine variant (See Machine Id File) TIME Num 9 Time record was committed or 0 if record is temporary. This field is stored in the Borland C++ internal format - the number of seconds elapsed since January 1, 1970. TEST Char 15 Test Id (See Test Description File) RESULT Char 10 Result of test. Numeric results are converted to ASCII strings. CHECK Num 1 Not used at this time COMMIT Num 1 Not used at this time ACCESSING THE DATABASE Experienced database programmers will find that the design of the PC Labs Benchmark Results Database is a fairly standard implementation of a relational database. As utility programs to access the database are written in-house they will be made available to the public. For now, I will give a couple of examples of how the files may be accessed to obtain certain data for reports. I leave it up to you to code the actual program. Let's start by developing a report that lists the machine name and results for a given test. You would start by accessing the Test Description File and select the test you are interested in. Using the Test Id as a link to the Results File you would read all the records with a matching Test Id. As the records are being processed you would use the Machine Id and Variant to look-up the Machine Name and Description in the Machine Id file. Each record in the Results File would then be printed as a line on the report with the Result formatted according to the data in the Test Description File record for that test. Another common process is to list all the tests and results for a single run on a single machine. In this case you would start in the Machine Id File and find the Machine Id and Variant you want. In a similar manner to the one above you would then read through the Results File and list out the Results for all records with matching Date and Time Stamps. The information about each test is extracted from the Test Description File by using the Test Id to establish the link between the files. There are many other ways to access the files but they all boil down to essentially the same type of linking between the files. In some cases you may need to program an intermediate sort to get the records into the desired order but all of the data should be easily accessible from one or more of the files. FUTURE PLANS Throughout this document I have been hinting at plans for the future. Once you start working with the database you will see the power waiting to be unleashed. We are currently working on export and import programs to allow you to extract data about a machine from one set of database files and merge them into another. This capability is needed in-house to combine data collected by different reviewers on different machines. It will also allow PC MagNet to supply updates to the data we now circulate with Bench6 as files to be imported into your own database. Also in the works is the ability to delete unused or outdated machines and their data from the database. Currently there is no way to delete a machine from the database. Field names have caused problems for some database packages when they tried to access a Bench6 file. For example, COMMIT is often a reserved word and cannot be used in a field name. We will be changing all the field names to have a prefix in front of them to avoid this problem. A conversion program will be supplied to copy data in the existing files over to files with the new field names. This is necessary because dBase stores the filed names in the header of each database file. New fields may be added and fields like the Date and Time Stamp (TIME) mentioned previously will be made easier to access. Again, conversion programs will be supplied as needed. CONCLUSION I hope this document has helped you to understand the design and mechanics of the PC Labs Benchmark Database. Once again, if you have any questions, comments or suggestions please contact me via the Lab Notes section of the Utility Forum on PC Magnet (ppn# 72241,64) or by mail at PC Magazine Labs, One Park Avenue, New York, NY, 10016. No phone calls please. Stuart R. Greenberg, Sr. Programmer - PC Magazine Labs