Simulated Extended Memory Arrays in QB, version 3

     QBXMS is a set of QB routines to allow storage of data in extended
memory (XMS) in much the same fashion that it stores data using arrays in
conventional memory.  It does NOT give you the ability to simply use the
DIM statement to define a standard QB array of "unlimited" size.  It simply
provides an alternative method of storing data in XMS and lets you access
it in a manner that in some sense simulates how you would store and retrieve
data with a conventional array.  The process is admittedly somewhat more
cumbersome than using conventional arrays, and slower.  (It has some
additional disadvantages which I'll get to eventually.)

     This being my first attempt at anything like this, and all, I decided
to keep things as simple as possible (while still maintaining usefulness).
String XMS arrays are not supported and you are restricted to one-
dimensional arrays.  Further, you may only have up to 16 such arrays and
their total combined XMS memory cannot be more than 30.9375 MB.  The four
standard numerical data types *are* supported:  INTEGER (2 bytes), LONG
(4 bytes), SINGLE (or "real"--4 bytes), and DOUBLE (8 bytes).  Further,
XMS arrays are 1-based.  (The first element has index 1.)

     Just as with normal QB arrays, XMS arrays must be dimensionalized
before you can put data in them.  Instead of DIM, however, you do this by
calling the subroutine DIMXMS.  The syntax of the call is

CALL DIMXMS (ARRAY, N, TYPE, EC)

where ARRAY is a character string representing the name of the array, N
is the number of elements in the array (a LONG integer), TYPE is a
character string giving the data type ("INTEGER", "SINGLE", "DOUBLE", or
"LONG") and EC is an error code (single precision/4-byte variable).  The
only restriction on N, other than the amount of available XMS is that, being
a LONG integer, it can't be larger than 2^31 - 1 (2147483647).  Since, even
for a 2-byte per element INTEGER array, this would give an array size of
4,096 MB, N really isn't a limiting factor here.  In the call to DIMXMS, EC
must be listed as a variable, not an explicit number.  EC is returned from
DIMXMS as 0 if DIMXMS was able to initialize the array properly (or at least
thinks it did).  It is returned as 1 if it detects an inavailability of XMS
(more on this later) and 2 if you've already maxed the number of XMS arrays
out at 16 when you tried to initialize a new one.  Normally, DIMXMS will
initialize all array elements to zero.  This may be time-consuming except
for very small arrays.  (And if they're very small arrays, why are you
trying to store them in XMS?)  Such initialization is often unnecessary.
You can turn it off by setting EC to any nonzero number before calling
DIMXMS.  (This must be done before each call since DIMXMS, when it's
performing successfully, resets EC to zero.)  Subroutine DIMXMS cannot be
used to perform a REDIM operation; it will terminate execution if you call
it with the name of an array that already exists.  REDIMXMS, however, can
be used for that purpose.  The calling syntax is exactly the same as with
DIMXMS.  If you merely want to clear an already existing extended memory
array and reallocate its memory for reuse by DIMXMS, you can do that with
subroutine CLRXMS:

CALL CLRXMS (ARRAY)




     Now, this is where things get slightly cumbersome.  How you store
data in an XMS array and retrieve data from it depends on what type of
array it is (INTEGER, SINGLE, etc.).  There are four subroutines used to
store data in the arrays, one for each data type, and four functions used
to get data out of the arrays, one again for each data type.  Let's say
you have an array "A", for definitiveness, and you want to assign a number
X to element I of this array.  You do this by calling a PUT$$$ subroutine,
where $$$ is SNG for a SINGLE array, INT for an INTEGER array, LNG for a
LONG array, and DBL for a DOUBLE array.  The syntax of the call is

CALL PUT$$$ ("A", I, X)

Note that I must be a LONG integer and X must be of the same data type as
signified by $$$.  The routines will detect discrepancies between the
specified array and the type they're programmed to handle and terminate
program execution (with a STOP statement) if they find such discrepancies.
(Nonexistent arrays and I > N / < 1 will also cause such termination.)  How
would you get X back out of array A?  Use the appropriate GET$$$ function,
where $$$ has the same interpretation as before.  For example,

X = GET$$$ ("A", I)

where I is a LONG integer, would retrieve the Ith element of array A and
put it in the variable X.  (It is generally best if X's data type is
compatible with $$$, although the general numeric conversion procedures of
QB should apply.)  Clearly, XMS arrays are global.  There is no need to
worry about how to pass them from one QB subroutine to the next.  XMS
exists no matter what subroutine you're in.  (Of course, you can pass
variables representing array *names* to subroutines.)

     You can use the function UXMS to find the number of elements stored
in an array.  (It's similar to QB's UBOUND function.)  UXMS inputs the
character string representing the name of an array and returns a LONG
integer giving the array's size and, via the parameter list, the base
address (LONG integer) of the array:

N = UXMS (ARRAY, ADD)

     Subroutine XBSAVE can be used to achieve roughly the same functionality
as QB's BSAVE command.  Rather than work with address segments and offsets,
however, XBSAVE just works with the name of an INTEGER extended memory array,
starting from the beginning of that array.  The format of the call is

CALL XBSAVE (ARRAY, BYTES, FILE)

where ARRAY is a string (literal or variable) giving the name of the INTEGER
XMS array, BYTES is a LONG integer giving the number of bytes to save (it
does NOT, of course, have QB's 65,535 constraint), and FILE is a string
giving the name of the file to save the data to.  The format of this file is
not quite the same as a conventional BSAVE file--the primary reason being
that BSAVE stores the number of bytes saved in the file as a 2-byte integer,
which is totally incompatible with the purpose of XBSAVE.  (QB's BLOAD
command is NOT compatible with an XBSAVE file.)  So here's the format of an
XBSAVE file, where all byte references are 1-based:



Byte 1 - character FDh, just like a BSAVE file;
Bytes 2, 3, and 4 - they spell out "XMS";
Byte 5 - the high byte of a 3-byte integer giving the number of bytes saved;
Bytes 6 and 7 - the low and middle bytes of the 3-byte integer giving the
                number of bytes saved (just like bytes 6 and 7 in a normal
                BSAVE file);
Bytes 8 and on up - the data that got saved.

     Since BLOAD can't be used to load this data back into an array,
conventional or otherwise, you need something else.  Well, you're in
luck!  I included (like you couldn't guess <g>) subroutine XBLOAD for that
purpose.  The call is given by

CALL XBLOAD (ARRAY, FILE)

where both inputs are strings and have the same interpretation as before.
Although both XBSAVE and XBLOAD are SIGNIFICANTLY SLOWER than BSAVE/BLOAD
(you may want to have your program beep at you when they're finished),
XBLOAD has what might be considered one advantage over BLOAD (other than the
fact that it works with extended memory and doesn't have the 64K - 1 byte
limit):  you don't have to use DIMXMS to initialize ARRAY before calling
XBLOAD--the subroutine does that itself via REDIMXMS.  (By the way, now that
I've said a couple of times that BLOAD can't work with this file, I'll point
out that it CAN if two conditions are met:  1) byte 5 in the file (the high
byte of the 3-byte integer) is zero--i.e., you saved less than 64K bytes,
and 2) you aren't using BLOAD in a manner that requires the default segment
and offset that BSAVE normally puts at bytes 2 - 5.)  XBLOAD initializes
ARRAY as an INTEGER array.

     XMSFREE is a stand-alone utility that will tell you how much extended
memory you have.  (It may not be useful if you have HIMEM.SYS or other
memory managers installed, i.e., likely most of the time--see below.)

     Now, then.  How do actually use the included files?  You generally
need three files.  One of these is the one you're writing that you need
QBXMS for.  (An example MAIN routine XMS.BAS is included to give you a
more concrete idea of how to use QBXMS.)  At the top of your program,
before your code, put the contents of QBXMS.INC.  At the bottom, after
your code, put the contents of QBXMS.BAS.  (If you're running from within
the QB IDE--I don't recommend it due to time considerations, use the /L
option on the QB command line.  Speaking of the interpretive environment,
if execution time isn't a big problem for you, if you don't have QuickBasic
but you *do* have QBasic, effective with QBXMS version 3, the software
should work with Qbasic.)

     Sounds great, doesn't it?  Are you looking for the catch?  (You didn't
think you were getting a free lunch, did you?)  There is a price to pay for
everything, and there's no exception here.  There are a few conditions you
have to put up with when using QBXMS.


1)  Obviously, XMS arrays are somewhat more cumbersome than conventional
   ones.  It's not like QBXMS is a driver that you can just load into memory
   and make otherwise normal QB programs suddenly work that wouldn't work



   before because the arrays were too big.  You have to rewrite your code
   and be very careful about which routine you use to process which type
   of variable.  You also need to pay close attention to the variable types
   being passed in the subroutine/function calls.

2)  QBXMS.INC contains various COMMON SHARED ... and DIM SHARED ...
   statements that store various global variables.  The TYPE REGISTERS ...
   END TYPE construct defines another kind of special variable.  You must
   not use these variables in your own programming except in the context
   that QBXMS uses them.  (Except for perhaps using the INREGS and OUTREGS
   variables for your own interrupt calls, don't even think about changing
   the other global variables unless you really understand how they're being
   used and are intentionally trying to modify the functionality of QBXMS.)

3)  QBXMS does not use HIMEM.SYS or any other memory manager, per se.  It
   merely uses interrupt 15 / function 87.  This does not make QBXMS very
   "XMS-friendly."  It tends to wipe out whatever data you may have stored
   in XMS memory.  (QBXMS does not, however, use the first 64 KB of XMS--the
   high memory area, in an attempt to preserve whatever TSRs/drivers that
   you may have loaded there.)  I typically use XMS as a ramdrive.  After I
   run a program using QBXMS, my ramdrive directory typically looks like a
   bunch of nonsense.  (If I ever get around to figuring out directory
   structures, I'll try to make QBXMS restore at least the "structural
   integrity" of ramdrives when it finishes.  In the meantime, you can use
   SCANDISK to fix your ramdrive.  SCANDISK still won't restore your lost
   data, but it *will* restore your FAT to what it would be for a nice,
   clean, empty directory structure.)

4)  A more pressing problem is whether or not QBXMS will even work with
   memory managers installed.  From what I can tell, the presence of
   HIMEM.SYS will not keep QBXMS from running.  However, there is a quirck
   with HIMEM.SYS that I can't honestly say I understand.  QBXMS queries the
   computer to find out how much XMS is available.  On my computer,
   HIMEM.SYS interferes with this.  With HIMEM.SYS loaded, my CPU informs
   me that I don't have any XMS when I query it in a manner that bypasses
   HIMEM.SYS.  This does not happen on other computers.  So, when QBXMS
   tries to find out how much XMS you have and gets a value of 0, it then
   queries the CPU to find out if HIMEM.SYS is installed.  If it finds
   HIMEM.SYS, QBXMS simply takes the available XMS to be the maximum value
   it can deal with--31 MB (30.9375 after the 64 KB of HMA is reserved).
   (These numbers are, respectively, 15 and 14.9375 MB for a 286.)  In
   other words, it would probably be best if you were cognizant of how much
   XMS you have when you use QBXMS.  You can experiment with the standalone
   program XMSFREE if you wish.  If running it from the DOS command line
   gives you a nonzero result and you know you have XMS managers loaded,
   then you can be reasonably sure that QBXMS is also going to be able to
   ascertain how much XMS you have.

    In the situation of being told there is no XMS memory while being
   informed that HIMEM.SYS is loaded, function FREEXMS (inside QBXMS.BAS)
   sets the global variable DRVEXIST to unity.  (Otherwise, it's zero.)
   Your routine can test this variable and act accordingly.



5)  Things get worse--and then they get slightly better.  HIMEM.SYS may not
   stop QBXMS from running, but if you have EMM386 loaded, unless your
   computer works a lot differently than any of the machines I've tested
   QBXMS out on, running QBXMS is just an elaborate way of rebooting your
   computer.  Further, if HIMEM.SYS doesn't interfere with QBXMS or XMSFREE
   ascertaining how much XMS you have, EMM386 will.  (And I suspect that
   these comments apply to other XMS memory managers as well.)

    Okay, here's the silver lining.  You probably don't have to completely
   kick all XMS managers out of memory (i.e., by rebooting in a different
   configuration).  Whether or not HIMEM.SYS is the source of any problems,
   it may be the solution to problems with other drivers.  Lets say the line
   in your CONFIG.SYS that loads HIMEM.SYS is something like:

   DEVICE=C:\DOS\HIMEM.SYS

   You may have various "/" options specified as well.  Simply imagine their
   presence here if you wish.  HIMEM.SYS can be made aware that you may want
   to bypass it and other drivers like EMM386 by using interrupt 15.  It
   will allow you reserve XMS memory for use by that interrupt.  If X is
   the number of KB of XMS that you want to reserve for QBXMS to use (i.e.,
   your memory managers won't know this XMS exists), add 64 to X and call
   that number Y.  Change the above line in your CONFIG.SYS to:

   DEVICE=C:\DOS\HIMEM.SYS /int15=Y

   When QBXMS or XMSFREE ask your CPU how much XMS is available, your CPU
   should now answer with a value of at least X.  Further, there should be
   no problems with QBXMS rebooting your machine because it tried to access
   extended memory that other drivers think they own.  This may also solve
   the problem of QBXMS trashing data already in XMS or otherwise scrambling
   your ramdrive directory structure, but I'm not really sure.  (It may
   depend on your DOS version, ramdrive driver, or "computer type"--and I
   use that term in its broadest possible sense.)


     The version of this text file associated with the initial release of
this software contained a discussion of how slow QBXMS was.  Well, as it
turns out, there were many reasons why the particular benchmark I was using
was slow.  I was able to significantly speed that particular program up.
Nevertheless, XMS arrays are still slower than conventional arrays.  As I
mentioned in the first version of this file, I initially thought that this
was because of the repeated interrupt calls and then I determined that,
while they're certainly a factor, they aren't the bottleneck in and of
themselves.  Rather, as before, I still believe that if there is a
bottleneck, it's just the overhead associated with calling the various
subroutines/functions just to transfer 8 or less bytes at a time.  In
addition to the normal QB overhead associated with calling subroutines/
functions, the GET$$$/PUT$$$ routines have their own overhead.  I should
have also pointed out previously that the process of going in and out of
protected mode isn't particularly fast either.

     If you only wanted to work with a 64K buffer in conventional memory
(and your application allows this technique) and transfer data 64K at a time



between it and XMS when necessary, things would be faster.  The subroutine
that actually transfers data to/from XMS is MEMTRN.  The call to it is of
the form

CALL MEMTRN (SOURCE, DEST, BYTES)

All parameters are LONG integers.  SOURCE is the linear address of where
data is being transferred from, DEST is the linear address of where it's
going to, and BYTES is the number of bytes to transfer (<= 65,536 and even).
I think another term for "linear address" is "huge address."  If the
conventional "offset segmented address" is segment:offset, the linear
address is

linear address = segment * 16 + offset.

Note that when you use VARSEG and VARPTR to get the segment and offset
of a numerical array or variable, these functions return numbers in a 2-byte
INTEGER format.  Hence, any segment or offset larger than 32,767 will be
returned by these functions as a negative number.  If you get negative
numbers, add 65,536 to them before you calculate the huge address.  Why do I
mention all this?  Because if SOURCE is an address in XMS (address 100000h,
or above), DEST will likely be the linear address of an array in
conventional memory, or vice versa.  (Actually, in spite of what one of my
references says, you can use this interrupt / function to transfer data from
one place in conventional memory to another place in conventional memory.  I
am unable to find a significant speed difference between doing that and just
using a FOR/NEXT loop and setting one array equal to another.)  If you want
to avoid writing over TSRs/drivers that you may have stored in high memory
DEST should be above 1,114,111 when writing to XMS.  (And it should be less
than 589,826 when writing 64 KB to conventional memory.)  So, to use MEMTRN,
follow the following steps.


1.  DIMension a 64 KB conventional array, call it ARRAY here:

    DIM ARRAY (1 TO SIZE) AS variable_type

   where size is whatever it has to be to make the array contain 65,536
   bytes for the variable type used.

2.  Use VARSEG and VARPTR to get this array's segmented address:

    DIM SM AS LONG,OS AS LONG
    SM = VARSEG( ARRAY(1) ) : IF SM < 0 THEN SM = SM + 65536&
    OS = VARPTR( ARRAY(1) ) : IF OS < 0 THEN OS = OS + 65536&

3.  Use DIMXMS to define whatever XMS arrays you want to use.

4.  DIMension an address variable, call it ADD here, as LONG:

    DIM ADD AS LONG




5. Decide which of your previously defined XMS arrays that you want to
  work with.  Use the function UXMS to get the (huge) address of that
  array (now that it's been modified to return that parameter):

   N& = UXMS(xms_array_name, ADD)

  where N is a LONG integer that now gives the number of elements in
  xms_array_name.  (It should be what you specified in the call to DIMXMS.
  Be careful using the results of UXMS.  If N is returned as 0, as will
  happen if you didn't specify a valid array name, the address will be
  set to a fictitiously large, i.e., meaningless, number.)

6.  Input data to your conventional array until it's full.

7.  Call MEMTRN to transfer your 64K conventional array to XMS:

   CALL MEMTRN (SM * 16 + OS, ADD, 65536&)

8.  Update XMS address:

    ADD = ADD + 65536&

9.  Repeat from step 6 until XMS data storage is complete.

10.  Data is transferred from XMS to your conventional ARRAY via

     CALL MEMTRN (ADD, SM * 16 + OS, 65536&)

    where you may need to reuse UXMS to get the base address of the XMS
    array if you didn't save it from the first call.

11. To transfer the next 64K out of XMS, update ADD (e.g., step 8) and
   repeat from step 10.


What if your XMS array doesn't contain an integral number of 64K blocks of
data?  Well, when you go to use that last block after transferring it to
your buffer in conventional memory, don't use more of the buffer than is
valid.  Do you really need to use MEMTRN this way instead of just using
QBXMS in its "conventional" manner?  Well, admittedly in spite of some
possibly erroneous earlier beliefs of mine, probably not unless you're doing
something in which "time is everything."

     Now for the legal stuff.  Although I don't really think QBXMS can do
any physical harm to your equipment or other aspects of your system, I must
insist that, whatever risk using QBXMS has, said risk must be YOURS.  (And
I've already warned you about the possible loss of data on your ramdisk or
otherwise stored in XMS.)  Although, I'm not necessarily relinquishing
copyright or desire for credit of authorship, I'm not too concerned about
what you do with the software--except for selling it.

     As for hardware requirements, well, your computer has to have extended
memory.  I've tried it out with both versions 2 and 3 XMS (whatever "XMS
version" means--if it means anything at all in regard to QBXMS since it



doesn't use any extended memory management system).  Hence, your CPU must be
a 286 or better.  (Although I haven't tried it out on a 286 with extended
memory, it works when I use an array in conventional memory to simulate the
presence of extended memory.  Since this involves the same interrupt calls/
protected-real mode switching, it should be a valid test.)



Glenn Stumpff

4960 Egret Ct.
Dayton, Ohio  45424

CIS:  73137,3537