_________________________________________________________________________ STACKER NOTE STACKER NOTE Subject: Stacker and "Slack" Space. STAC FAX Index #2603 - 5/13/92 Page 1 of 2 __________________________________________________________________________ Definitions: Sector: The smallest unit of storage that a disk controller is physically able to read or write. Sectors have a fixed size of 512 bytes in a Stacker drive. Cluster (Allocation Unit): A multi-sector unit; the smallest unit of storage used by DOS. DOS varies the allocation unit size to suit the storage medium, however, the size is set during logical formatting and does not change thereafter. Called a cluster prior to MS-DOS 4.0, the words are interchangeable. The DOS 16-bit FAT is limited to 64K, therefore the maximum drive size with 2K clusters is 128Mb (64K x 2K). When the drive being formatted is larger than 128Mb, DOS merely makes the allocation units bigger, because the limitation on drive size is the number of clusters, not the number of sectors. This would suggest to those of you who write many letters and other small documents, or who use relatively small database and spreadsheets, that you should probably not partition a drive over 128Mb, because from 128Mb to 256Mb, DOS uses 4K allocation units. From 256Mb to 512Mb, the allocation units are 8K. As the drive gets bigger, the allocation units get proportionally larger also. Slack Space:The space lost on your hard disk when DOS allocates an entire cluster to save an amount of data which is less than the size of that cluster. e.g.: Your file is 612 bytes, so normally DOS uses one allocation unit (cluster) to save the 612 bytes. Because an allocation unit is 2048 bytes, the lost space (slack space) is 1436 bytes (2048 - 612). The larger the drive, the larger the allocation unit, and the greater the amount of slack space (waste). If your drive has 8K allocation units, and you save a 178 byte batch file (filename.bat), the wasted space is 8014 bytes. A normal letter would waste from 5K to 7K. This space is lost forever, unless we change the size of the allocation units on this particular drive. Stacker and flexible allocation units: Stac had a better idea, we decided to have a flexible allocation unit. When Stacker builds a new drive, the default cluster size is 8K. What this really means is that the largest possible cluster size is 8K, not that all clusters are 8K. This allows Stacker to build a drive as large as 512Mb logical (64K x 8K). Where Stacker saves space is in the use of sectors. When Stacker saves a file which is less than a sector in size, it only allocates one real sector to the cluster which is keeping track of that data. Therefore, if the compressed data is 6 bytes, Stacker only uses one sector to save that file, whereas DOS would use 4 sectors, so the savings in lost space is 1536 bytes (2042 - 506). More simply stated; Stacker only uses as many sectors as necessary for a given file. Sectors are not tied permanently to a specific allocation unit until they are used, and even then, only enough are used to do the job. This means that slack space is greatly reduced in a Stacker drive. Of course, there is a slight catch: too many small files only use one or two sectors each, but each one also uses an allocation unit. This means that the FAT can run out of entries even while much of the space is unused because the sectors were not allocated to clusters. In Stacker version 1.x, the only fix for this problem was to change the cluster size to 4K. This allows twice as many entries in the FAT. This method can also be used with Stacker version 2.0 by users of DOS versions which are limited to 32Mb drives. In Stacker version 2.0, Stac improved the product by allowing the user to logically "grow" an existing Stacker drive with a high compression ratio, by increasing the number of allocation units. When the drive is getting close to full, and the compression ratio is appreciably higher than the original ratio, the user may choose to "grow" the drive by running SDEFRAG /G. This can increase the number of allocation units up to twice the original number, thereby increasing the logical capacity. For the user of a database file which is approximately 400Mb, and which compresses at 8:1, we need to do a little arithmetic before building the original Stacker file. The maximum size of a Stacker drive is 512Mb. Divide this by the anticipated maximum compression ratio of 8, and we get 64Mb. Therefore we can create a new Stacker drive, choose a compression ratio of 8:1, and stack an existing drive with its data. If the data really compresses at 8:1, we will have used 64Mb of physical space to create a drive of 512Mb. If the file is very large, there will be no slack space, as the file is many clusters long. The FAT is still only 64K, but is now handling a very large drive in a relatively small space. To prove this, you can conduct a fairly simple test. First, run SCHECK on a Stacker drive, and record the results which should look like this: No errors found Stacker Drive Statistics: Stacker Drive STACVOL File Drive C: D:\STACVOL.DSK ______________ ______________ Total Bytes: 155,435,008 77,718,016 Bytes Used: 80,887,808 ( 52.0% ) 44,773,888 ( 57.6% ) Bytes Free: 74,547,200 ( 48.0% ) 32,944,128 ( 42.4% ) Stacker Drive Compression Ratio = 1.8:1 Projected Bytes Free = 59,473,920 The important number to observe here is the bytes used in the STACVOL file (44,773,888 bytes). Second, create a very small file, or copy a small file from anywhere. Make sure that the file is less than 512 bytes. Now run SCHECK again, and look at the number of bytes used in the STACVOL file again. This time, the number should be 512 bytes less. Record the number again, and now add another file which is somewhat larger, and then look at the results again. This time the bytes free may decrement by 512 or 1024 or 1536 or 2048 or some other multiple of 512. This should demonstrate that Stacker only uses as many sectors as it needs, no more.