Thanks for the Memory By Ken Johnson, Chicago Computer Society This month we'll look at one of the most confusing issues in personal computers, even to experienced users: memory. There is conventional memory, reserved memory, expanded memory, extended memory, upper memory and high memory. Some programs use extended, some need expanded (expanded, extended -- who named these things??). And then there is upper memory and high memory: what goes where?!? Like it or not, memory management is becoming increasingly important to even the novice computer user. We are using larger and more sophisticated programs that require more memory (and certain kinds of memory). To get the most out of operating environments such as Windows or Desqview, we need memory configured in a certain way. We're using more device drivers and Terminate and Stay Resident (TSR) programs, all of which make demands on precious memory. By properly using memory management techniques, we can get the most out of the memory we have in our computers. DOS 5.0 now includes specialized memory drivers and programs, meaning you no longer have to invest in specialized memory management hardware or software (though you may want to). In this column we'll look at five kinds of memory: conventional memory, expanded memory, extended memory, the "reserved" upper memory area (sometimes called Upper Memory Blocks, or UMBs), and the high memory area (HMA). We'll also discuss a little about memory management with DOS 5.0. Conventional Memory, Reserved Memory and the Upper Memory Blocks The original Intel 8088 processor used in the first personal computers could only address 1 megabyte of memory. It was assumed that 640 kilobytes would be more memory than applications would ever need, so that 1 Megabyte was divided into two areas. The first 640K would be "conventional" memory where DOS resides, applications run, and data manipulated. The remaining upper 384K of the 1 Mb was "reserved" for system and hardware services: ...things like video RAM, system ROM, hardware ROM, video adapters, and system BASIC. Unfortunately, as programs and processors evolved, two things became clear. First, 640K was not enough memory for our growing programs, device drivers, and TSRs. We started experiencing the dreaded "RAM Cram". Secondly, there were areas of memory within the reserved 384K that were not being used; in essence sitting there empty. These normally empty areas are called Upper Memory Blocks, or UMBs. Part of memory management is to find a way to put these (otherwise wasted) Upper Memory Blocks to work, and the way was found with the 386 processor. A special feature of 386 and 486 processors is the ability to "remap" memory addresses, by translating physical addresses to logical addresses and giving the logical addresses to the application programs. These processors can take portions of extended memory and map them into the unused Upper Memory Blocks, making that space available for device drivers and TSR programs. To use the UMBs there must be sufficient extended memory to remap, and a control program (such as DOS 5.0's EMM386.EXE) that implements the feature. Once they are available and recognized by DOS, the UMBs can be filled with device drivers and TSR programs that would normally be occupying space in conventional memory. DOS 5.0 includes two new commands for loading drivers and TSRs into the UMBs: DEVICEHIGH and LOADHIGH. We'll look at these below. Expanded Memory One of the first efforts to break through the 640K barrier came in 1985 when Lotus, Intel, and Microsoft developed the Expanded Memory Specification (EMS), a technique for swapping data in and out of conventional memory. An add-in memory board would provide a large storage area, with a "page frame" (generally 16K) created within conventional memory. Through swapping pages of memory, data and program code could be moved back and forth between the memory board and conventional memory, where they could be acted upon. The Lotus-Intel-Microsoft (LIM) expanded memory specification can make more memory available to your programs, even on an 8088 computer. Both program code and data can be brought into conventional memory when needed, and then swapped out when not. A memory board is required for expanded memory with an 8088 or 286 processor, with the appropriate EMS driver. On a 386 processor a memory manager can convert extended memory into expanded memory for applications that need it, so an add-in board is not required. One way of remembering exPanded memory is to think of the Porthole on a ship. Looking through the porthole you see just a small part of the ocean at a time -- though it's a large ocean there. As the boat moves, you see different parts of the ocean through the porthole. Think about the ocean as expanded memory and the porthole as the page frame; just as the porthole provides access to the ocean a portion at a time, the page frame provides access to expanded memory a page at a time. Extended Memory Extended memory is directly addressable memory above 1 megabyte, available on computers with 286, 386, or 486 processors. Although DOS can't address this memory in it's default "real" mode, 286/386/486 processors can access this memory by switching into "protected" mode. Microsoft, in collaboration with Intel, AST Research, and Lotus Development Corporation, developed the eXtended Memory Specification (XMS), which allows programs to request memory above 1 Mb for use. Access is provided by a XMS manager such as HIMEM.SYS. Since there is often confusion between exTended and exPanded memory, here is a way to remember the difference: ExPanded Memory is memory that is Paged into conventional memory. ExTended Memory is memory that sits on Top of 1 megabyte. High Memory Area The High Memory Area (HMA) is a special portion of extended memory, the first 64K block directly above 1 megabyte. It's available on 286/386/486 processors with at least 64K of extended memory. The HMA is a contiguous area, unlike the UMBs that are often fragmented. The importance of the HMA is that with version 5.0, DOS can load itself and its BUFFERS into this area, thus freeing about 50K of conventional memory for your programs. To load DOS into the HMA, you need to install HIMEM.SYS, DOS' extended memory manager and tell DOS to load itself "high". We'll see how next. Configuring Memory with DOS 5.0 Third party memory managers such as QEMM-386 and 386Max have been available for several years. But with the release of DOS 5.0 last year, memory management tools are now available as part of DOS itself. Unfortunately configuring memory with DOS 5.0 isn't automatic; it takes some work and experimentation to get your system optimized. (One selling point of QEMM-386 and 386MAX is that they can easily configure your system for you, and often provide more usable upper memory space than DOS itself can.) Configuring memory on a 386 or 486 computer with DOS 5.0 involves loading HIMEM.SYS and EMM386.EXE, telling DOS to load high and make UMBs available, then loading device drivers and TSRs into the upper memory blocks. When you install DOS 5.0, it will normally modify your CONFIG.SYS file to do everything except load drivers and TSRs into the UMBs; that you have to do yourself. Let's look at each of these steps. Remember though that these are for a 386 or 486 processor. If you have a 286 processor you cannot use EMM386.EXE, DEVICEHIGH, or LOADHIGH; you can still use HIMEM.SYS and load DOS into the HMA though. With an 8088, you don't have extended memory, or expanded memory without an add-in board and it's EMS software. 1. Load HIMEM.SYS in your CONFIG.SYS file HIMEM.SYS is the eXtended Memory Specification device driver provided with DOS 5.0. HIMEM.SYS provides XMS services, makes the High Memory Area available, and allows you to install EMM386.EXE for access to the UMBs (on 386 and 486 processors). HIMEM.SYS also provides access to extended memory for DOS' RAMDRIVE.SYS (RAM disk driver) and SMARTDRV.SYS (disk cache), also Windows 3.0 in standard and enhanced modes. HIMEM.SYS is installed via the Device statement: DEVICE=C:\DOS\HIMEM.SYS [/MACHINE=id] There are additional switches on the DEVICE= statement that can be used to fine tune your system, but most are not required. An exception may be the /MACHINE=id switch, which specifies a particular brand of PC. For example, the IBM PS2 uses the switch "/MACHINE=2". Check the DOS 5.0 manual for specific machines that need this switch. 2. Load EMM386.EXE in your CONFIG.SYS file EMM386.EXE's primary purpose is to make the UMBs available for drivers and TSRs, but it also has another benefit: it can convert exTended memory into exPanded memory. It will provide LIM-EMS support for programs that use exPanded memory but not exTended memory. EMM386.EXE can set up a 64K page frame in an Upper Memory Block, then convert part of exTended memory (256K is the default) for use as exPanded. The syntax for loading EMM386.EXE is: DEVICE=C:\DOS\EMM386.EXE [RAM | NOEMS] [X=nnnn-nnnn] [I=nnnn-nnnn] The RAM and NOEMS switches determine whether exPanded memory will be made available. The RAM switch provides access to both exPanded memory and the Upper Memory Blocks. NOEMS provides access to the UMBs, but no exPanded memory services. By preventing EMM386.EXE from allocating the 64K EMS page frame, more room in the upper memory area will be available for drivers and TSRs. If you don't need expanded memory, use the NOEMS switch. The X= and I= switches can be used to fine tune the upper memory area, by including (I=) or excluding (X=) memory addresses. For example, my computer has a Plus HardCard II as my D: drive. The HardCard's device driver uses the memory address C800-C9FF. By excluding X=C800-C9FF, I make sure that no other program tries to use that area of memory. A word of warning: Be careful when you start including and/or excluding memory addresses. Make sure that you have a boot disk handy. If you make a mistake, memory conflicts can cause your computer to freeze when you boot up. You'll have to boot from the floppy, then change the EMM386.EXE statement. You must load HIMEM.SYS before EMM386.EXE, so be careful of the order the statements appear in your CONFIG.SYS file. The EMM386.EXE statement also must appear before any DEVICEHIGH= statements. By the way, don't try to use DEVICEHIGH= with EMM386.EXE or HIMEM.SYS; both have to be in conventional memory. 3. Load DOS into the HMA, and make UMBs available The next step is to tell DOS to load itself into the High Memory Area, and to enable loading of drivers and TSRs into the UMBs. This is done with the new DOS= command in CONFIG.SYS: DOS=HIGH,UMB As you might expect, "HIGH" tells DOS to load into the HMA and "UMB" makes the Upper Memory Blocks accessible. (On a 286 computer, since it can use the HMA but not the UMBs, simply specify DOS=HIGH.) Remember that you must first load HIMEM.SYS and EMM386.EXE before issuing this command. Unfortunately, DOS won't automatically tell you if it loaded successfully in the HMA. You can easily tell by issuing the MEM command from the DOS command line. The last line of the output should read "MS-DOS Resident in High Memory Area", showing that DOS was loaded into the HMA. We'll use the MEM command again to see what was actually loaded into the Upper Memory Blocks. 4. Load Device Drivers and TSR programs into Upper Memory Blocks To load device drivers into the UMBs, use the DEVICEHIGH= statement in CONFIG.SYS (rather than the normal DEVICE= statement). The syntax is DEVICEHIGH=[path]filename.ext [parms] where "filename.ext" is the name of the device driver and "parms" are any parameters for that driver. Let's say I want to load the SMARTDRV disk cache, my HardCard ATDOSXL driver, and ANSI.SYS into upper memory. The CONFIG.SYS statements would look like this: DEVICE=C:\DOS\HIMEM.SYS DEVICE=C:\DOS\EMM386.EXE RAM X=C800-C9FF DOS=HIGH,UMB DEVICEHIGH=C:\DOS\SMARTDRV.SYS 1024 521 DEVICEHIGH=C:\ATDOSXL.SYS DEVICEHIGH=C:\DOS\ANSI.SYS To load TSRs, simply include the LOADHIGH (or LH) statement before each program name in your AUTOEXEC.BAT file. Most TSRs can be loaded into the UMBs, if there is enough space. Let's say we want to load Mirror's delete tracking for the C: and D: drives, the mouse, and DOSKey into upper memory. The AUTOEXEC.BAT statements would look so: LOADHIGH MIRROR /TC /TD LOADHIGH C:\WINDOWS\MOUSE.COM /Y LOADHIGH C:\DOS\DOSKEY /BUFSIZE=1024 5. Use the MEM command to see if it worked Unfortunately, it is not obvious when something is loaded into the Upper Memory Blocks. If DEVICEHIGH= or LOADHIGH can't fit a driver or program into the UMBs, it will be loaded "silently" into conventional memory -- there is no message indicating it is loading below 640K. By using the MEM command with the "/C" classify switch, you can see what programs are in conventional memory and which are in Upper Memory. The MEM /C command's output has three main sections. The first, labeled "Conventional Memory:" lists the programs currently in conventional memory, with their size. It also shows free memory. The second section, labeled "Upper Memory:", shows the program names and sizes in the upper memory blocks. Finally, at the bottom of the listing is a summary of the extended memory, including total extended memory and available XMS memory. There also will be a message indicating if DOS is resident in the High Memory Area. By the way, don't be mislead by a line stating "0 bytes available contiguous extended memory". This simply means that HIMEM.SYS is controlling all the extended memory. The MEM /C output is usually longer than a screen in length, so you'll want to either print it out or pause the screen display. You can get a printed copy by redirecting the output to your printer: MEM /C > PRN To pause the screen when full, redirect the output into the MORE filter using the DOS pipe symbol: MEM /C | MORE The MEM command may report a program or driver in conventional memory when you tried to load it into Upper Memory. Why didn't it work? To begin with, remember that the Upper Memory area can contain several UMBs of varying size. DOS always begins loading into the largest UMB first, so the order you load drivers and programs is important. In general, you'll want to load the largest first. It is possible that a few small drivers or programs will fill part of a large UMB and not leave enough room for a large program or driver. You can experiment with the order of DEVICEHIGH= and LOADHIGH statements, using the MEM command to see what order works best. Remember that DEVICEHIGH= statements will always execute before LOADHIGH commands, since they are in the CONFIG.SYS file. Some TSR programs can never reside in Upper Memory, because they require a large amount of memory when first loading. Typically they release most of this initialization memory after loading, but there is not enough room in the Upper Memory Blocks so DOS always places them in conventional memory. For example, Borland's Sidekick 2.0 occupies about 40K when running as a TSR. But it takes over 300K when loading, meaning it is next to impossible to load into Upper Memory. Unfortunately there is usually not an easy way to identify such programs, though the documentation is a good place to start. Conclusion Understanding and configuring memory isn't the easiest task in the world; though DOS 5.0 gives us the tools, it still requires some effort, experimentation and finesse to get the most out of your computer's memory. I hope that this column has taken some mystery out of memory and how to use it. Now try some experimenting (but make sure you have a boot disk first!)