Category : BASIC Source Code
Archive   : QBXM10.ZIP
Filename : QBXMAPND.DOC

QBXM ver 1.0
Appendix A


EMS Specification vs XMS Specification

Allocation Units.

The EMS specification calls for a minimum allocation unit of
16,384 bytes, called a page. All memory size specifications are
expressed in these 16k pages. The XMS specification calls for a
1024 byte minimum unit, so any allocation needs to be a multiple
of 1k. QBXM converts calls for a page within an XMS environment
to 16k multiples and uses the 'page' concept throughout. For
example, when a call is made to QBXM to allocate 1 page, in an
XMS environment, the page count is multiplied by 16 and the
request is passed through to the driver for 16k.


Handle Counts.

There are both a default number of handles available in both
specifications and a maximum count allowed. With EMS the maximum
number of handles is 256, with handle 0 being reserved for the
operating system. XMS allows for a number of handles I can't
find in the documentation, but HIMEM.SYS will allow up to 128
maximum. Since HIMEM.SYS will only allow a maximum of 128 han-
dles, that is also the maximum handles for QBXM.

HIMEM.SYS defaults to 32 handles, and the count can be adjusted
with the /NUMHANDLES= command line parameter. EMS drivers, I
believe, may vary in the number of default handles. QEMM which I
use, and an AST driver that I use on an AT&T 6300, both default
to 64 handles. Setting the number of handles desired is a func-
tion of each EMS driver, and is done via a command line switch in
the CONFIG.SYS file.


File Buffer Space

This may be tedious for the reader, but here I go. When using
EMS, there is a 64k block of conventional memory between 640k and
1 megabyte available that is called the page frame. Different
sections of expanded memory can be moved into and out of this
space, and any application can address this space, including DOS
and BIOS routines. When the Load (or Save) File routine is
called, a buffer is needed to read the disk data into. If EMS is
in use, 16k of this section of memory is used as a buffer by
QBXM. The data is moved from disk to the buffer and then to
expanded memory in 16k chunks. No memory need be taken from the
BASIC data area.









1


QBXM ver 1.0
Appendix A

Within the XMS specification, four types of memory are defined.
Conventional from 0k to 640k, Upper Memory from 640k to 1024k,
High Memory from 1024k-16 to 1088K, and Extended Memory from
1088k to the top of installed memory. The upper memory area is
the only one (other than conventional) that could be reliably
used as a buffer area for disk transfers. The XMS specification,
written by Microsoft, Lotus, Intel and AST Research, specifies
calls to allocate and deallocate portions of the upper memory
block. This should allow QBXM to handle disk transfers in an XMS
environment the same as it does for EMS. However, Microsoft does
not implement the upper memory calls in HIMEM.SYS. I know, if
they had a hand in writing the spec, why don't they implement all
it's features? Who knows? They use the high memory area exten-
sively with Windows (with 2.0 first?), but I think they wanted to
avoid stepping on a possibly loaded EMS manager. The high memory
area cannot be used for disk transfer buffers however, DOS and
the BIOS may not correctly handle the addresses.

This left me in a quandary. When running under XMS, QBXM would
need a buffer area for the file transfer, and it would have to be
allocated from the BASIC data area. Three alternatives presented
themselves.

One, I could have written the code to use a buffer area supplied
by the calling program from it's data area, and read the disk
file into it, then transferred from that buffer into extra memo-
ry. If I did that though, then the 64k buffer available to EMS
implementations would not have been used, and memory cramp might
ensue in some situations. Just seems to be a waste of a resource
that might be available, and required the passing of three more
parameters, the segment, offset and length of the buffer.

Two, taking the above a step further, I could have the QBXM
routine check for EMS or XMS. If EMS I would load from disk
into the EMS page frame's 64k at 16k per loop, and from there to
extra memory. If XMS was in use, then the calling BASIC program
would have to allocate the buffer (an array of x bytes) and
specify where the buffer was allocated (VARSEG and VARPTR) in
the BASIC data space, and the size of the buffer. The buffer
could be larger than 16k to speed things up a bit, but a disk
transfers only at such and such a speed. This presents some com-
plexities in that there would be the DIM'ing of the buffer (only
for XMS) and the passing of three extra parameters for a
load/save file call, regardless of which type of extra memory was
installed at run time. To be honest, it is a more efficient
method than what I implemented, but I am trying to keep the
interface as simple as possible. The code in BASIC would require
something like what's on the following page.








2


QBXM ver 1.0
Appendix A

rLen = LEN(record)

IF ems THEN
bSeg = 0
bOfs = 0
bytes = 0
CALL LoadFileXM (f$,rlen,bseg,bofs,bytes,handle,recs&,eCode)
ELSEIF xms THEN
x& = FRE(-1)
.... that gives you the free memory from which you
.... would calculate the number of elements to DIM an
.... array based on the memory available. Elements
.... would vary based on the type of array of course..
DIM temp(1 TO elements)
bSeg = VARSEG(temp(1))
bOfs = VARPTR(temp(1))
bytes = elements * bytesPerElement
CALL LoadFileXM (f$,rlen,bseg,bofs,bytes,handle,recs&,eCode)
ERASE temp
END IF


The third option, and the one I settled on, was to allocate a
1024 byte buffer area within the QBXM code. This 1k buffer comes
out of the data area that would normally be available to your
BASIC program, and is only used when XMS is in use, though it is
allocated regardless of the extra memory type. This seemed to be
an acceptable trade off, because EMS is more common than XMS, it
does not use very much of the data space available, and QBXM does
try to allocate some upper memory first, before using this 1k
buffer. Let's just say I'm hopeful that upper memory support may
be out there in a driver other then HIMEM, or HIMEM may soon
implement it. It doesn't in DOS 5 by the way. Anyway, the code
tries for a 16k upper memory block buffer, and if the block is
not available, or upper memory blocks are not implemented, the
code falls back to using the 1k internal buffer.

What this boils down to is that a file load loop using EMS would
read 16384 bytes of the file, then transfer 16384 to EMS extra
memory with each loop. The same loop using XMS would require
roughly 16 times as many loops. This sounds terrible on the
surface, but assembly loops being quicker than the equivalent
BASIC loops, and trying to preserve memory available to your
BASIC program, it might not seem so terrible.

Bottom line is that I would like to hear from users on whether
they agree with my logic. Maybe I was to conservative on the 1k
buffer? Maybe a 4k buffer would be acceptable? Maybe a second
object file and routine that used 8k at a clip, and if testing
showed memory cram, you could fall back to the 1k routines?
Maybe the extended calling routine, where the buffer is allocated
from within the calling program? Self doubt is a terrible thing,
tell me what you think!



3


QBXM ver 1.0
Appendix A

QBXM & EVEN Length Records

At first glance, the insistence that record lengths and block
moves be an even number of bytes may seem odd. Read on.

EMS was developed to extend the useful life of PC and XT comput-
ers. PC and XT class machines use an eight bit data bus. So a
byte transfer is no problem, a byte being eight bits.

Extended memory is a different matter though. Extended memory is
only available on AT class machines which have as a minimum, a 16
bit data bus. Data transfers being more efficient with 16 bits
at a time than with 8 bits, and not being available on an 8 bit
machine, why the heck should the Extended Memory Spec (XMS)
bother with byte sized transfers? Exactly, so it doesn't.
Interesting thing is that XMS memory moves are specified in bytes
instead of words. Probably thought it was a more common unit of
measurement for programers.

Therefore, there is no choice but to write QBXM to use word (2
byte) sized memory moves. This includes user defined types of
course.

The best way to handle odd length records? Maybe DIM an odd
length record to 2 elements, two times anything seems to be an
even number. Then tell QBXM that all records are two times there
actual length and adjust accordingly. For example:

TYPE AddressRecord
Name AS STRING * 26
Addr AS STRING * 26
Town AS STRING * 14
St AS STRING * 2
Zip AS STRING * 5
END TYPE

' That zip code will get you each time!
' A record length here works out to 73 bytes.

DIM SHARED address AS AddressRecord
DIM SHARED tempAddress(0 TO 1) AS AddressRecord

rLen = LEN(address) * 2 'This is the value you pass to QBXM













4


QBXM ver 1.0
Appendix A

' For any given record number, actualRecord, you use the follow-
' ing formulas:

getRecord& = ((actualRecord - 1) \ 2) + 1
' Which results in the following:
' actualRecord: ((actualRecord - 1) \ 2) + 1 = getRecord:
' 1 (( 0 ) \ 2) = 0 + 1 = 1
' 2 (( 1 ) \ 2) = 0 + 1 = 1
' 3 (( 2 ) \ 2) = 1 + 1 = 2
' 4 (( 3 ) \ 2) = 1 + 1 = 2
' 5 (( 4 ) \ 2) = 2 + 1 = 3
' 6 (( 5 ) \ 2) = 2 + 1 = 3
' etc...

Then you use GetRecXM to retrieve the record from QBXM's record
orientated routines, and access the final desired record like
this:

vSeg = VARSEG(tempAddress(0))
vPtr = VARSEG(tempAddress(0))
GetRecXM (handle, getRecord&, vSeg, vPtr, errCode)

address = tempAddress((actualRecord + 1) MOD 2)

' Which results in the following: Element
' actualRecord: (actualRecord+1) MOD 2 = Returned:
' 1 ( 2 ) MOD 2 = 0
' 2 ( 3 ) MOD 2 = 1
' 3 ( 4 ) MOD 2 = 0
' 4 ( 5 ) MOD 2 = 1
' 5 ( 6 ) MOD 2 = 0
' 6 ( 7 ) MOD 2 = 1
' etc...

If you have a more compact, efficient method, let me know, it is
late as I write this.

Putting a record would require the reverse logic.

tempAddress((actualRecord + 1) MOD 2) = address

' Which results in the following: Element
' actualRecord: (actualRecord+1) MOD 2 = Returned:
' 1 ( 2 ) MOD 2 = 0
' 2 ( 3 ) MOD 2 = 1
' 3 ( 4 ) MOD 2 = 0
' 4 ( 5 ) MOD 2 = 1
' 5 ( 6 ) MOD 2 = 0
' 6 ( 7 ) MOD 2 = 1
' etc...






5


QBXM ver 1.0
Appendix A

putRecord& = ((actualRecord - 1) \ 2) + 1

' Which results in the following:
' actualRecord: ((actualRecord - 1) \ 2) + 1 = putRecord:
' 1 (( 0 ) \ 2) = 0 + 1 = 1
' 2 (( 1 ) \ 2) = 0 + 1 = 1
' 3 (( 2 ) \ 2) = 1 + 1 = 2
' 4 (( 3 ) \ 2) = 1 + 1 = 2
' 5 (( 4 ) \ 2) = 2 + 1 = 3
' 6 (( 5 ) \ 2) = 2 + 1 = 3
' etc...

Then you use PutRecXM to store the record into extra memory:

vSeg = VARSEG(tempAddress(0))
vPtr = VARSEG(tempAddress(0))
PutRecXM (handle, getRecord&, vSeg, vPtr, errCode)
I hate putting an important warning at the start of a new page,
when it applies to a previous page, but I've no choice here.

NOTE: When using a 'doubled' type variable as in the previous
example, remember not to destroy the data in extra memory for the
element not being worked on.

What's that? Well for each actual element or record you retrieve
there is an innocent bystander with it. Take for example, a case
where you need to retrieve the first record from extra memory.
The record you want to work with will be in element zero of the
double record, and valid data will also be in element one. If
you destroy the temporary double record after extracting element
zero, work with it, then DIM the double record again, insert the
revised data in element zero and write the double element back to
extra memory, actual record two (element one in the double re-
cord) is all zeros. Your extra memory 'file' now has a blank
record in it. If you, as I did in the example, dim the double
record as a SHARED variable, and keep in mind that there is extra
data all the time you're working with the double record, all
should be OK.

A better method for storing data may be to force a read (Ge-
tRecXM) immediately before a write (PutRecXM), calculations as
before:

Calculate the double size record number.
Retrieve it from extra memory.
Calculate the element (0 or 1) in the double record.
Update the proper element (0 or 1).
Write the double record back to extra memory.


All in all, a great argument can be made for padding your records
with one byte to assure they are an even length.




6


QBXM ver 1.0
Appendix A

Extended Memory and Vdisk et al.

When the AT was introduced with it's extended memory, an inter-
rupt, 15 hex, was supplied in the BIOS to make limited use of the
extended memory available. INT 15 hex can be called to determine
the amount of extended memory installed. At the same time,
VDISK.SYS was distributed with PC-DOS 3.0+, which took advantage
of extended memory also.

These two items presented two different ways to access extended
memory. The "INT 15 method" is one, and what the basic implemen-
tation consists of is that a program (a TSR maybe) that uses
extended memory would call INT 15 to find out how much extended
memory was available. Next it would deduct what it needed from
'the top' so to speak, then intercept any other calls to the INT
15 "How much Extended is there?" function. When an intercept is
made, the program returns the reduced amount to any applications
that ask.

VDISK.SYS was distributed with it's source code in PC-DOS, and
used a different allocation method. It took what was needed of
extended memory from the bottom of extended memory (1024k) and
worked upward. If you look back a page or two, you might notice
I mentioned the High Memory Area defined in the XMS includes the
extended memory from 1024k - 16 bytes to 1088k. Opps. This is
the same area VDISK would start allocating from, and Microsoft
wants to use with Windows! Since the source code to VDISK was
distributed with PC-DOS, a lot of developers followed it's lead
in allocating memory from the 'bottom up' with a few twists
thrown in to make sure the chain of extended memory allocation
remained intact. You could load multiple copies of VDISK.SYS for
example to create multiple ram disks. Only problem is, not all
software follows the VDISK check and double check system of
memory allocation and use. Some extended memory compatible TSR's
and application programs will use the "VDISK method" and even
identify themselves as "VDISK" to maintain compatibility (though
limited) with that method. Try a text scan of your hard drive
for "VDISK", you may be surprised at the number of applications
that include the text string, which is loaded into memory to
identify it as a VDISK type memory allocator.

I find myself getting off track all of a sudden. So we have two
methods for allocating extended memory. One is from the top
down, where each application that takes some memory intercepts
queries about how much is there and reports the amount it knows
of, less what it's using. Then VDISK, which if implemented
properly works from the bottom up, and gives an indication of
where it's extended memory use ends. A program that wants to use
extended memory is supposed to follow this chain of VDISK blocks
up, and calculate how much is left. Of course there's no guaran-
tee that the memory chain is implemented properly along the way.
When XMS was developed, I suppose that the this VDISK allocation
system had been used and abused, thus the XMS fellows said the
heck with it. If an XMS driver finds a "VDISK" type extended


7


QBXM ver 1.0
Appendix A
memory manager in use when it's loaded from CONFIG.SYS, the XMS
manager won't load. If a "VDISK" type driver is loaded after the
XMS driver, it will be detected on an XMS call. This checking
for a "post XMS driver installation of a VDISK" type program
impresses me as meaning they are something to stay away from.
After the reading I've done on the various memory related sub-
jects I can see why.

With almost every product that includes HIMEM.SYS, Microsoft
includes RAMDRIVE.SYS, a fully compatible XMS application. Just
be sure RAMDRIVE is loaded after HIMEM.SYS in the CONFIG.SYS file
and you've got a non-interfering ram disk instead of using VDISK.

When it came to accessing extended memory with QBXM, I had to
decide just how to implement it. Follow the INT 15 or VDISK
methods if an XMS driver wasn't there, and extended memory was
installed. Or insist on using a formal specification? True, if
I used the INT 15 top down approach, there would be no need to
specify that an XMS driver such as HIMEM.SYS be installed. If I
included the INT 15 method after a check for EMS and then XMS, it
would eliminate the HIMEM.SYS requirement. Of course that means
I could also allow you to walk down in memory right over a VDISK
type ram disk or cache or whatever in extended memory. Call me
dull, but I figured a specification of one type was better than a
"custom" of another.


Extended Memory And The 80286

When INTEL designed the 80286, they made the chip power up in
real mode, where it would act just like an 8088/8086 chip.
Included on the chip is a method to switch from real mode into
protected mode. Protected mode was (and is) considered so much
more efficient than real mode, a method of switching back to real
mode was not considered to be a need. A work around to the
problem involves some tricky stuff that I'm not altogether clear
on but what I think happens is interesting. When the BIOS wants
to switch out of protected mode and back to real mode, it writes
a 'magic' value to a specific location in memory. Next it sends
a command to the keyboard controller that causes the controller
to issue a warm boot instruction, as if you hit Ctrl-Alt-Del.
When the ROM BIOS starts it's warm boot routine, it checks for
that 'magic value' in memory. If it finds it, then the machine
is not really being reset, just the processor is. Somehow memory
is preserved through all this, and the processor picks up from
where it should in your program code. Interesting trivia true,
but the thing to keep in mind is how much is going on. When you
access extended memory on an 80286, it's going to be slow. The
XMS manager takes care of everything painlessly for the program-
mer, but it's a real workout for the hardware.







8