OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW NAME overview.txt - Overview Of Concache.exe About Concache.exe Concache.exe is for "concurrent disk cache" that is so called "advance cache", which drives disks concurrently with DOS/user programs. The maximum concurrency it can perform is up to 25 disks, including one of two floppies, one of IDE disks, and up to maximum number of configurable SCSI disk units under DOS. (The term IDE is for Integrated Device Interface. In this and the other texts in this package, how- ever, includes ST506 interface and ESDI for brevity.) Concache.exe allows true preread/postwrite operations trans- parently for programs under DOS. The basic idea to attain concurrency is, by catching PC/AT type machine's BIOS int1590 and int1591 interrupts, to force switch from concache.exe to DOS/user program by the former and force back again to concache.exe by the latter. The SCSI hard disks are operated concurrently by the aid of ASPI (Advanced SCSI Programming Interface) command posting capability, which offers the chance to synchronize with disk operations exactly the same way as BIOS int1590/int1591 allows. Additional programs floppies.exe and ccdisk.exe are provided with concache.exe, the main cache program. The former is a mini-driver which does most of floppy i/o independent of DOS. The latter is the reentrant disk driver for SCSI disks, using ASPI. If a disk is neither of above, for example pre-ASPI SCSI disk, concurrency is lost there. On such disk, the interval the device made CPU idle is indiscernible from CPU overhead. Yet postwrite, although not concurrently, is supported for such disks. Prerequisites Concache.exe runs PC/AT class machine and after, equipped with INTEL 80386 CPU and the later, running MSDOS3.1 and the later or the equivalents. Concache.exe resident memory requirements are usually around 25 - 30 kilobytes, increased depending on concurrency requirements (stacks to keep things concurrent and sheer io buffers are unavoidably allocated at each startup.) Concache 1.10 Last Update: 22 June 1996 1 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW Cache Data Area Concache.exe uses memory for caching disk data. Memory types to be used are 1) Raw protected (int15) memory, 2) Expanded memory conformable to EMS specification 4.0, 3) Extended memory block (EMB) defined by XMS 2.0 specifi- cation. If only int15 memory is to be used no additional third party software is needed. If memory manager allows configure, these three types of memory can be used freely mixed, totaling up to 64 megabytes. The amount of EMS or EMB used by concache.exe are changeable without disloading/reloading concache.exe from memory. (Int15 memory is hard to share with the other programs. Changing its size on the fly is not considered.) Installation If ROM BASIC is used then interrupt number and the identifi- cation of the interrupt have to be adjusted using VECTOR description. Please refer to concache.txt for the explana- tions. The adjustment must be applied to all the programs among concache.exe, ccdisk.exe, and floppies.exe you intend to use. Otherwise there is no need of installation procedure. Dis- able disk cache program currently used, and simply copy to convenient place and make calls, either from DOS config.sys, autoexec.bat, or command line, as appropriate. Quick Test Real programmers may skip to the next section. (Virtuals may better not.) If your system is fairly recent one and you believe it is configured in just standard way, and you have read the Known Incompatibilities section below, and is not running the other disk cache program, and currently on command line from command.com, then you can test if your system runs con- cache.exe. Afterwards, you should read the related documents to find out what you have done in this test. Concache 1.10 Last Update: 22 June 1996 2 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW - First, go to the directory where concache.exe is copied and type simply "concache". You will see the message it is not loaded. This message should appear. - Next, type "concache x". Now concache.exe has allo- cated memory for its cache area if it doesn't complain. If it complains, try "concache e" and "concache p" in this order until you succeed. If all fails, there is no memory to be used for cache area, and your quick test has failed. You have to add memory or you must decrease some other program's memory usage. - You should have allocated all available memory of a type, which could be against your intention. See this by typing "concache hm", which means "help about memory usage". - If you succeeded in "concache p", you can release the memory by typing "concache d" which means "delete resi- dent concache.exe from memory". If you have done so, you might try different amount by typing "concache p1m", "concache p256k", "concache p131072", and so on, anything between zero and maximum value reported. - If you have succeeded in "concache x" or "concache e", you can try above experiment similarly. Try it later, substituting "x" or "e" with "p" as appropriate. For now type "concache xm" or "concache em", which reallocates zero megabyte (that is, zero byte) memory. In other words, it has released all allocated memory but concache.exe is still in memory. Again type "concache hm" and see it is using none. You are now be able to allocate as much memory as you like exactly like the case "concache 'p" but without doing "concache d". This is an advantage over "concache p". - If you have allocated more than 256kb of memory, you will make an interesting test at this point. Prepare a spare floppy (it must be formatted, though) into floppy drive a:, and type "copy concache.exe a:" (or whatever file), and type "concache hd" immediately. You have told "help about drives" by typing "concache Concache 1.10 Last Update: 22 June 1996 3 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW hd". You see floppy is being written, and you get the list of how devices are treated by concache.exe. You will observe a line "still XX wet spots" which tells how many blocks of sectors remain to be written into the floppy a:. If you type "concache hd" again, then this line is probably unseen. By allocating larger memory and copying more files at once into floppy, and typing "concache hd" in succes- sion, you will observe the number of wet spots decreas- ing toward zero. Chance to observe this line when hard disk is used instead of floppy is small unless a large copy is done using large cache data memory. If you have succeeded up to this point, your remaining thing to do is to edit your autoexec.bat or config.sys file. How- ever, please read on. You have seen an advantage and a disadvantage of using con- cache.exe. You will feel comfortable to see that your com- puter responds quickly even if it is busy writing. But... While you don't have to wait for the completion of writes to continue work, you must wait for it to pick the floppy out. There are many ways to await the completion. The simplest is watching the access lamp. and the second is repeatedly typ- ing "concache hd" to make sure there is no wet spots, as you have just learned. Another method is to type "concache r" to tell refresh drives. Then concache.exe watches the progress, and returns to command line when no wet spots remains. Likewise when you shutdown your machine, there is a danger of remaining wet spots behind. A recommended procedure is to have a shutdown procedure, like shut.bat, which contains at least "concache r" line (after the line which might cause the final writes to disks, of course). Acceptance Test Since there can be many unnoticed incompatibilities, and they potentially cause the worst disasters, things may not be as simple as preceding subsection. In particular, following areas need carefully checked. - Memory managers. Concache 1.10 Last Update: 22 June 1996 4 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW Options for "unique" features for a memory manager must be checked with special care. In general, simply using XMS or EMS would be well compatible. - Communication programs There are two considerations to be paid. One is if the CPU overhead concache.exe incurs, or the pattern of its CPU usage, can be tolerated by communications programs. The need to check this applies particularly to serial communications. - Multitaskers Most task switchers seem to work. Multitaskers need special care. Please see below for more. - Disk compression and encryption programs These programs use proprietary driver schemes, which concache.exe does not always detect. - Third party block device drivers By definition, device drivers can do anything, and can be incompatible with concache.exe. However, well behaving device drivers can coexist. If int13 and int15 are used in predefined manner, these drivers will hap- pily take advantage of concurrency. - File/disk maintenance utility programs These programs can use int13 directly, which can cause the mismatching of cached data and actual data in disks. Make sure cache data is written back before using such programs. An alert tester should proceed, if in any doubt, as follows. If needed, read concurrency mode to write after or write through mode. 1) Backup Make backup before conducting the tests. 2) If Ever Loadable Load concache.exe setting all drives in stop mode. 3) The Least Damage Put a floppy into concurrency mode. and try read and write on it. At the end, check the integrity of media with, for example, chkdsk program. 4) Add Hard Disks Put drives, one by one, into concurrency mode, and Concache 1.10 Last Update: 22 June 1996 5 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW conduct test on them. Integrity check is equally important. 5) Finally For at least a month after acceptance, more frequent backups are scheduled. Of course, the backup program must be checked for compatibility firsthand. Known Incompatibilities The programs are tested against various environments and have identified several incompatibilities, A few command arguments on concache.exe try to solve problems rather non- obvious ways. If you get into trouble, please give a look at eqanda.txt. It describes the backgrounds of these troubles so you may find the way to proceed. Following are known incompatibilities: - Any other disk cache program cannot work with con- cache.exe. - Low level utilities using int13 method to access disks directly can potentially cause data integrity problems. Put concache.exe into stop mode (put +s on concache.exe command line) to use these utilities. - Floppy operations under Novell DOS are very slow due to its quirky media checks, the result of which are almost certain unknown (however the hard disk operations are very, very fast, well compensating for floppy speed). - Desqview 2.31 and Desqview/X 1.1 cause crash. Probably the family is incompatible as a whole. Use write through method. - QEMM 7.3 stealth feature causes hang or crash. Only write through works. - Floppies.exe can hang Windows 3.*. Concache.exe auto- matically disables floppies.exe if it is found loaded. - Windows 3.* does not implement full EMS 4.0. EMS mem- ory is freed while under it. This feature can be used to auto-adjust amount of cache data memory under windows. - Windows 3.11 32 bit file access is incompatible (32 bit disk access is not). If concache.exe found vfat.386 in Concache 1.10 Last Update: 22 June 1996 6 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW configuration, it asks if you dare to proceed or termi- nate the windows. Generally choose 'n' to prevent hangup or crash, but if 'y' is selected be extremely careful not to work very hard on the session. - Windows 95 is incompatible. Concache.exe pretends "dead" releasing all EMS and XMS memories while under it. However, this is not sufficient. Please follow special note below. - Server.exe of Novell Personal Netware maintains its internal int2a counts incorrectly. It causes hang or crash. Write after method solves the problem. The other TSRs that intercept int2a for synchronization with DOS have potentially the same problem. - Most block device redirectors such as disk compression and encryption programs are not cacheable. Load con- cache.exe before them in order to make cacheable. Contrarily, Microsoft Dblspace and Stac Electronics Stacker 3.12 and the later must be loaded before con- cache.exe. - Applications which use DOS "unused" stack space may be corrupted by concache.exe assumption of its being free at device driver calls. It uses rather larger stack space. - Depending on circumstances, loading io buffers into upper memory block can cause severe slow down. Try io_buffers_low option if performance is unexpectedly low. Special notes on Windows - Windows 3.* has many settings which might affect per- formance of concacne.exe. Following list is the favor- able settings. VirtualHDIrq=false OverlappedIO=true ReflectDOSInt2A=true If ReflectDOSInt2A must be set false, then INT28Critical=true Concache 1.10 Last Update: 22 June 1996 7 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW must be set. Otherwise, all read and write operations are handled write through mode by concache.exe. - Windows 95 complains even if concache.exe and ccdisk.exe stay idle, doing nothing. To remedy this problem, insert two lines below under [SafeList] sec- tion of ios.ini file under Windows 95 directory with a text editor convenient to you. concache.exe ccdisk.exe Then Windows 95 safely ignores loaded ccdisk.exe and concache.exe. Safety Considerations There are a few safety considerations built into con- cache.exe. Representative of considerations are: Reset by Ctrl-Alt-Del By capturing the key sequence, concache.exe tries to write out immediately yet unwritten data back to disks. In very rare cases such operations cannot be performed, it simply ignores the keys and awaits retry ctrl-alt- del. 30 Second Durations Concache.exe writes back any pending data to be written to disks within 30 seconds. In other words, even if a block is continually updated, the block is refreshed after 30 seconds of unwritten interval has passed. Reset disk function (int210d) Concache.exe immediately refresh all pending unwritten data of all local drives. Commit file function (int2168) Concache.exe immediately refreshes all pending unwrit- ten data of the local drive the file is residing on. Refresh is not limited to the file alone but spans to all the files on the drive the file is residing. This is like resetting a particular drive. Extended open (int216c) with commit flag For files opened by extended open (int216c) with commit flag, file writes and close are handled like write- through requests. This is very different from commit file function, where all files in the same drive are involved. Concache 1.10 Last Update: 22 June 1996 8 OVERVIEW COPYRIGHT 1995-1996 horio shoichi OVERVIEW Lastly But Most Important Enjoy the parallel world of DOS single tasking operating system ! Concache 1.10 Last Update: 22 June 1996 9