Documentation for TaVram.PAS - Turbo Pascal Unit
Virtual Heap Manager for Turbo Pascal

(tested under Turbo Pascal v5.0)

Copyright 1989 - By Thomas Astin - All rights reserved.


Thomas Astin (Compuserve 73407,3427)
3451 Vinton Ave. Unit 9
Los Angeles, CA 90034


This unit's story
-----------------
After seeing lots of talk on BPROGA (Compuserve) about virtual heap managers
I decided the best way for me to understand what all the talk was about was
to attempt creation of my own virtual heap manager for Turbo Pascal.
TaVram is the result.

As limited and simple as it is, after writing TaVram I had a much better
understanding of what is involved in basic virtual heap management. I
uploaded it in hopes that it will help some other TP user learn the
same basics that I did. Have fun!

=Tom=


How to use TaVram
-----------------
Understand that you need Kim Kokkonen's HEAP.ARC available on the BPROGA

forum on Compuserve. It contains the GrabHeap unit and alot of other nifty
routines that let you write programs that have more control over heap
management (by intercepting TP's normal operation, etc). For more
information on these routines see the documentation files contained in
HEAP.ARC. Also, there is a vital patch to the TPC compiler that makes
your program call a user defined ISR for all dereferncing. Not compiling
your program with this patched compiler renders TaVram useless.

TaVram is very easy to use. Just include it in your Uses statement as
follows:

Uses
TpCrt,
ThisUnit,
TaVram; { <---- included in uses }


TaVram's initialization code will open two files VRAM.$$$ and VFREE.$$$
that are used to store a virtual heap and a virtual free list respectively.
At the start of the program virtual heap management is OFF by default. It
can be started by issuing a VRamOn which is an Inline macro that sets
UseVram (a boolean) to True.

From this point (after VRamOn) any calls to New will allocate data space
on the virtual heap. This does NOT necessarily mean that data is not
allocated on the real heap (in fact, it is). It just means that
if the real heap space runs empty that TaVram will start "paging out" the
least used data to the virtual heap file (VRAM.$$$). Also any
dereferences are routed to TaVram's special dereferencing routine. The
dereferencing routine handles any "paging in" (from VRAM.$$$) that is
necessary. It can also tell if a pointer is not a TaVram pointer by a
special signature used in the segment portion of the pointer.

There are several interfaced items in TaVram that deserve an explaination
at this point but first let me define some phrases that might make reading
the explainations a little easier. These definitions are TaVram specific
and are not necessarily those of the industry.

VRam Heap Pointer : A pointer that contains a VRam signature and
a valid handle. It is a pointer that has been
allocated while VRamOn is in effect. When
this term is used it also refers to the data
being pointed to.

Real Heap : TP's normal heap as you know it.

Virtual Heap : Those VRam Heap Pointers on the real heap
and in the virtual heap file.

Paging out,
Paged out : The act of taking data from normal RAM and
placing it in virtual RAM (in this case, a
disk file) in order to free up Real Heap.

Paging in,
Paged in : The act of taking data from virtual RAM (in
this case, a disk file) and placing it on
the Real Heap to enable the program to
make use of the data.


VRamHeapFile,
VRAM.$$$ : The disk file that contains portions of the

virtual heap that have been "paged out."


Free List : A list of enties that describe free areas
of a heap, whether virtual or real.

VRamFreeFile,
VFREE.$$$ : The disk file that contains the free list of
VRAM.$$$





==============================================================================
UseVRam : Boolean (default is False)
------------------------------------------------------------------------------
See VRamOn.


==============================================================================
procedure VRamOn;
------------------------------------------------------------------------------
Turns the UseVRam flag ON. This will enable VRam allocation until a VRamOff
is encountered. Any New operation will allocate the pointer as a VRam Heap
Pointer. Doing a UseVRam:=True has the same effect as VRamOn;


Var
SP1,
SP2 : ^String;
begin
.
.
.
VRamOn;
New(SP1); {SP1 is VRam Heap Pointer containing a signature in the Segment
portion and a handle in the offset portion}
VRamOff;
New(SP2); {SP2 is a normal TP pointer containing the address of data}
.
.
.
end.


==============================================================================
procedure VRamOff; (Default)
------------------------------------------------------------------------------
Turns the UseVRam flag off. This will enable TP's normal real heap management
as opposed to TaVRam heap managment. Use it when you want to allocate a
pointer and guarantee that the data never leaves memory. UseVRam:=False has
the same effect as VRamOff. This is the default becuase initialization
routines that allocate heap space would end up using VRam allocation
routines and they may not work in these circumstances. It really depends
on how the Unit allocating the pointers uses the pointers (see "Watch Out"
below).


==============================================================================
function VRamPageOutOldest : Boolean;
------------------------------------------------------------------------------
This function will page out the least used data from the heap to the virtual
heap file (VRAM.$$$). It will only page out VRam Heap Pointers allocated
with VRamOn (or UseVRam:=True).


==============================================================================
procedure VRamPageOutFreeMem(Size : Word);
------------------------------------------------------------------------------
This will call VRamPageOutOldset until Size bytes are available on the heap.
It is governed by several factors. Those factors being PageVRam,
VRamMaxHeapToUse, and MaxAvail. The best way to explain this is to have
you view the source code of the procedure VRamPageOutFreeMem in the Unit
TaVram. The WHILE statement shows the governing factors.


==============================================================================
procedure VRamGetMem(var P: Pointer; Size: Word);
------------------------------------------------------------------------------
This is the routine called instead of TP's normal GetMem. If VRamOn is in
effect then virtual heap routines will be used to allocate the pointer's data
area. If VRamOff is in effect then the normal TP GetMem is called. In
either case, if there is not enough real heap available for the new pointer
then paging will occur if VRamPageOn is in effect or PageVRam:=True.


==============================================================================
procedure VRamFreeMem(var P : Pointer; Size: Word);
------------------------------------------------------------------------------
This procedure is called instead TP's FreeMem. If the pointer being
deallocated is a VRam Heap Pointer then virtual deallocation routines are
called. If it is a normal TP pointer (allocated with VRamOff) then TP's
FreeMem is called.


==============================================================================
PageVRam : Boolean (default is True)
------------------------------------------------------------------------------
See VRamPageOn and VRamPageOff.


==============================================================================
procedure VRamPageOn; (default)
------------------------------------------------------------------------------
Turns ON the virtual paging flag PageVRam. Calling this has the same effect
as PageVRam:=True. Use this to allow VRamPageOutFreeMem to page out VRam Heap
Pointers from the heap to VRamHeapFile. At first this might not make sense
because if you didn't want to page out items then you would simply not call
VRamPageOutFreeMem. However, the other virtual heap management routines
in TaVram call VRamPageOutFreeMem whenever they need to page out anything to
gain more real heap space. If you want to force the paging out of the oldest
item without care of the PageVram flag then just use the function
VRamPageOutOldest - it is not governed.


==============================================================================
procedure VRamPageOff;
------------------------------------------------------------------------------
Turns OFF the virtual paging flag PageVRam. Calling this has the same effect
as PageVRam:=False. This will inhibit TaVRams automatic paging out of the
oldest VRam heap items.


==============================================================================
AdjustHeapPtrAfterFreeMem : Boolean (Default = True)
------------------------------------------------------------------------------
This flag tells TaVram whether or not to merge VRam Free List entries, if
possible, after each deallocation of a VRam Heap Pointer. You might call
it mild compression.


==============================================================================
VRamMaxHeapToUse : LongInt (Default = 700000)
------------------------------------------------------------------------------
This is the maximum heap that TaVram should use. By default it is set to
utilize all available heap. If you use the default then you will have to
adjust TP's FreeMin to insure that there is enough heap for the free
list to expand. Otherwise deallocating a pointer that requires a free list
entry will cause a TP runtime error to occur if there is no real heap left
(see misc info below).


==============================================================================
VRamHeapUsed : LongInt (Default = 0)
------------------------------------------------------------------------------
This is the current amount of heap (in bytes) being used by TaVram for VRam
Heap Pointers. It, of course, starts at 0.


==============================================================================
function VRamLock(P : Pointer) : Boolean;
------------------------------------------------------------------------------
This function lets you put a lock on the pointer passed to it. Locking it will
prevent it from being paged out until it is unlocked with VRamUnlock. This
function will return False if the Pointer is not a VRam Heap Pointer or the
handle is not found on the heap. The later case can usually be solved by
simply dereferencing the pointer. Doing so will force it to be paged in.


==============================================================================
function VRamUnLock(P : Pointer) : Boolean;
------------------------------------------------------------------------------
This function lets you release a lock previously placed on a VRam Heap
Pointer.


==============================================================================


Miscellanious Information
-------------------------
In order to use TaVram compile your program with the {$P+} ($P is a
directive available with the patched TPC compiler) directive in
sections of code where virtual heap management is to be used. It is
suggested to use {$P+} for your entire program and then use VRamOn
and VRamOff in sections where you want to turn its use ON and OFF.
Dereferencing a VRam Heap Pointer with VRamOff will still work as long as
the section of code where the dereference takes place is compiled with the
patched $P+ directive.

Once again, you should also adjust TP's FreeMin to keep a minimum amount of
memory for TP's Free List. You should do this especially if you are not
going to use the VRamMaxHeapToUse variable. Not doing so in this situation
will result in a runtime error if you max out the heap and try to free
memory because there will be no more heap for the free list.


Conditional Defines
-------------------
You'll notice that in the source code for TaVram there are some conditional
defines located at the top. For the most part you don't really have to
be concerned with these. Here is a brief explaination of each.

DEBUG : When defined TaVram will NOT erase the two files
VRAM.$$$ and VFREE.$$$. By default it is not
defined.

USEINLINE : When defined TaVram will use he experimental Inline
code for the function VRamHandleOnHeap. By default
it is not defined. USELONG will not work when this
is defined.

USELONG : When defined TaVram will use a LongInt for the
usage counter rather than a Word. By default it
is not defined. When USEINLINE is defined this will
be automatically UNDEFined.

ERRORMSG : When defined TaVram will include error messages
within the program. This takes up memory so one
might want to undefine it to minimize memory
usage. By default it is defined.



Errors
------
At this point errors are just halted with one message. If there is alot
of response to improve TaVram then more effort will be put into error
messages, etc. Also keep in mind that runtime errors are not trapped and
the program will halt with the normal TP runtime error message. If you
experience a runtime error and cannot find the cause then just drop me
a line on CIS and I will do my best to assist you (who knows, it might
be a bug ).

Some reasons why TaVram will abort are as follows:

1. Attempt to deallocate an invalid pointer (ie: nil pointer).

2. Attempt to page out a VRam item and there is none to page out.

3. Attempt to allocate a virtual pointer has failed.

4. Attempt to allocate a virtual free list entry has failed.


Watch Out!
----------
Make sure you're careful when you dereference a VRam Heap Pointer that
is part of a record that you're are also dereferencing (or any other
situation that is similar).

For example

type
Prec : ^MyRec;
MyRec = record
DumVar : Integer;
PR : Prec
end;
var
MyRecPtr : Prec;
begin
New(MyRecPtr);
New(MyRecPtr^.PR);
.
.
.
with MyRecPtr^ do begin { page in MyRecPtr }
PR^.DumVar:=DumVar; { if MyRecPtr gets paged out when
MyRecPtr.PR gets paged in then
MyRecPtr.DumVar won't be there
anymore!}
end;
.
.
.
end;



Demo Programs
-------------
I have included two demo-programs. One is a simple text demo and the
other is basically the same as the text demo except it offers a graphic
view of what is going on. Make sure you have ample disk space available
for these demos (around 1/2 a meg should be safe). If you have a floppy
based system then you might want to shorten the array size in the demos
so less VRam Heap is used (disk space).


Good luck and have fun!