Memory Management in QuickBASIC 4.x, BC 6.00 (Complete Guide) [B_QuickBas]

4.00 4.00b 4.50
MS-DOS
MSINTERNAL | B_BasicCom appnote SR# S890614-47 QB4MEMAN.ARC Snnnnn.EXE

Summary:

Programmers may wish to understand how Microsoft QuickBASIC and
Microsoft BASIC Compiler arrange memory in order to write programs
that make efficient use of system resources. The QuickBASIC language
provides a variety of data types and modular code constructs that
allow you to manage the data and code of your programs.

This application note applies to Microsoft QuickBASIC Versions 4.00,
4.00b, and 4.50 for MS-DOS and to Microsoft BASIC Compiler Versions
6.00 and 6.00b for MS-DOS.

This application note, "Memory Management in QB 4.x," can also be
obtained on paper from Microsoft Product Support Services by calling
(206) 454-2030.

A separate article with ENDUSER access describes how customers of
on-line electronic services can obtain the long article below in the
Software/Data Library. This file can be found in the Software/Data
Library by searching on the keyword QB4MEMAN. If you make changes to
this article, the changes also need to be made in the Software
Library.

More Information:

This application note covers the following topics:

1. Code management

a. Modular programming

b. What goes at the module level in a support module

c. COMMON, COMMON SHARED, SHARED, DIM SHARED

2. Data management

a. $DYNAMIC and $STATIC metacommands

b. Huge arrays

Appendix A contains a code example to illustrate the topics covered in
this application note.

Appendix B explains the .MAP file created when the Appendix A program
example is linked. You can use the LINK map to determine how close
each module is to approaching the 64K code limit and how close the
.EXE program's static variables are getting to the 64K limit in the
DGROUP.

Appendixes C and D describe in detail the memory mapping for running
programs in the following three different environments: the QB.EXE
editor, a compiled .EXE program using the run-time module, and an .EXE
compiled with the stand-alone option.

Definitions
-----------

A "module" is defined as an individual source file containing BASIC
procedures.

"Module-level code" is defined as the statements within a module that
are outside a SUB...END SUB, FUNCTION...END FUNCTION, or DEF FN..END
DEF definition block.

A "support module" is a source file, separate from the main module,
that contains additional SUB or FUNCTION procedures.

Memory Diagram
--------------

The following diagram summarizes how QuickBASIC compiled programs are
loaded into memory at run time:

High Memory (640K maximum)
_____________
| | The far heap stores dynamic arrays. The far
| (Far) | heap consists of the rest of high memory
| Heap | available after MS-DOS and the BASIC
| | program's DGROUP segment and code
__________________| segment(s) are allocated.
| | The stack is used to store temporary data,
DGROUP| Stack | such as variables that are passed to a
| | subprogram procedure. The default stack
Up to |___________| size is 2K.
64K | | The DGROUP (default data segment) is used
| (Near) | to store all static arrays, strings,
| Data | simple variables, and the stack. DGROUP
___________________ can be up to 64K in size.
:
_____________
| Code | The code segments store the executable code.
| Segments | Each module can have up to 64K for its
_____________ code segment.
Low Memory


Code Management
---------------

Modular Programming
-------------------

Modular programming deals with splitting programs into separate
modules (i.e., separate source files containing SUB or FUNCTION
procedures). The reasons for doing this are the following:

1. Once you have written a block of code as a module it can easily be
incorporated into future projects.

2. Debugging is easier. Problems can be located much more quickly and
can be contained in a specific area of the program.

3. The compiler (BC.EXE) and the environment (QB.EXE) have a 64K
code limitation per module. A modular style of programming
allows you to create programs larger than 64K, since every module,
even though it is part of one program, can be up to 64K in size.

If a program is large, it may be necessary to break it up into several
modules. This is easily accomplished by breaking SUB or FUNCTION
procedures out of the main module and placing them in support modules.
These modules are then compiled separately with BC.EXE and LINKed with
the main module as in the following example.

Consider a program Main.BAS which is broken into three modules:

1. MAIN.BAS (which calls external procedures)

2. MODULE2.BAS (which contains SUB and/or FUNCTION procedures)

3. MODULE3.BAS (which contains SUB and/or FUNCTION procedures)

The three modules are then each compiled separately to produce the
following object files:

1. BC MAIN.BAS; ----> MAIN.OBJ

2. BC MODULE2.BAS; ----> MODULE2.OBJ

3. BC MODULE3.BAS; ----> MODULE3.OBJ

To produce the executable (.EXE) program, the following command line
is used:

LINK Main.OBJ+Module2.OBJ+Module3.OBJ; ----> Main.EXE

Main.EXE is the finished executable program. When you compile in
QuickBASIC's environment (QB.EXE), the environment automatically
compiles each module and LINKs them together to produce the same .OBJ
files and finished executable program.

To make an .EXE program from the QB.EXE environment, do the following:

1. Choose the Run menu.

2. Choose the Make EXE File... option.

3. Choose the Stand-Alone EXE File option to create a stand-alone
file. If this option is not selected, the program requires
that the BRUN4x.EXE run-time module be present at run time.

4. Press the ENTER key.

The QuickBASIC environment gives you an easy way to do the following
operations, which are described further below:

1. Create a new Module for SUBs

2. Edit SUBs and Modules

3. Delete SUBs

4. Move SUBs from one Module to another Module

In QB.EXE, to create a separate module (source file) for SUBs, do the
following:

1. Choose the File menu.

2. Choose the Create File... option.

3. Enter the name for the module.

4. Press the ENTER key.

A file will then be created with the name you specified, and it will
be in memory with the other module(s). To save all of the loaded
modules, do the following:

1. Choose the File menu.

2. Choose the Save All option.

When the modules are saved together, QuickBASIC creates a file (with
the extension .MAK) that keeps track of the main module and the other
modules that are used by the main. To load all the modules in at once,
do the following:

1. Choose the File menu.

2. Choose the Open Program... option.

3. Select the main module of program as the file to be opened.

To view and select SUBs for editing, do the following:

1. Choose the View menu.

2. Choose the SUBs... option.

3. Highlight the module or SUB that you want to edit.

4. Tab down to Edit in Active, or Edit in Split.

5. Press the ENTER key.

To delete a SUB, do the following:

1. Choose the View menu.

2. Choose the SUBs... option.

3. Highlight the SUB you want to delete.

4. Tab down to the Delete option.

5. Press the ENTER key.

To move a SUB to a different module, do the following:

1. Choose the View menu.

2. Choose the SUBS... option.

3. Highlight the SUB you want to move.

4. Tab down to the Move option.

5. Press the ENTER key.

6. Select the module you want the SUB to be in.

7. Press the ENTER key.

More information about using the QB.EXE environment can be found in
the "Microsoft QuickBASIC: Learning to Use" manual for Version 4.50,
or in the "Microsoft QuickBASIC 4.0: Learning and Using" manual for
Versions 4.00 and 4.00b and the BASIC compiler Versions 6.00 and
6.00b.

What Goes at the Module Level in a Support Module
-------------------------------------------------

A support module, when compiled, produces an object (.OBJ) file that
is LINKed with a main module to create an executable file. The
module-level code of the main module is the only code directly
executed by QuickBASIC. The module-level code of the support modules
cannot be CALLed, RUN, or CHAINed. This module-level code of the
support modules is used only for the following:

1. Event- or error-trapping handlers

2. Metacommands

3. TYPE definitions, DIM, and COMMON SHARED statements

You can place ON Event GOSUB and ON ERROR GOTO trapping statements
within SUBprogram procedures [where ON Event means ON KEY(n), ON
COM(n), ON TIMER(n), etc.]. However, the line or line label that is
the target of the event or error trap's GOTO or GOSUB must be at the
module-level code in the same module as that SUBprogram. This is
because QuickBASIC doesn't allow GOSUBs or GOTOs from a SUBprogram to
other modules or SUBprograms, and QuickBASIC allows only ON Event
GOSUB and ON ERROR GOTO statements to jump from a SUBprogram to the
module-level code.

The compiler metacommands (REM $INCLUDE, REM $STATIC, and REM
$DYNAMIC) can also be used at the module level. REM $INCLUDE pastes
extra source into the module at compile time. REM $DYNAMIC and REM
$STATIC declare subsequent arrays as $DYNAMIC (allocated at run time)
or $STATIC (allocated at compile time).

You can use TYPE...END TYPE, DIM, and COMMON SHARED statements in the
module-level code. Variables and arrays can be shared between modules
using the COMMON SHARED statement at the module level.

COMMON, COMMON SHARED, SHARED, and DIM SHARED
---------------------------------------------

The SHARED statement gives a SUB or FUNCTION procedure access to
variables declared at the main module level of that module (without
passing them as parameters). It does NOT give access to variables
declared in support modules.

The COMMON statement makes variables available at the module level
between modules. The SHARED attribute for the COMMON statement (i.e.,
COMMON SHARED) is required to share the variables with SUBprograms or
FUNCTIONs. The list of variables in the COMMON and COMMON SHARED
statements must match in type in the COMMON and COMMON SHARED
statements in each module.

When using the COMMON (or COMMON SHARED) statement, static and dynamic
arrays are dimensioned differently. Static arrays in COMMON must be
DIMensioned BEFORE the COMMON statement in all the modules with that
COMMON statement. Dynamic arrays in COMMON must be DIMensioned AFTER
the COMMON statement in just the main module and should not be
DIMensioned in any support modules.

There are two differences between using the SHARED statement and the
DIM SHARED statement:

1. To make a variable accessible by all the SUBprograms in a module,
use DIM SHARED at the module level.

2. To share a module level variable with a specific SUBprogram, put
the variable in a SHARED statement in the SUBprogram. The SHARED
statement goes directly after the first line of the SUBprogram.

Data Management
---------------

$DYNAMIC and $STATIC Arrays
---------------------------

The default setting for storing arrays is REM $STATIC, which stores
all arrays in DGROUP (the default data segment). For any BASIC .EXE
program, DGROUP is limited to 64K. If you are using static arrays, you
can get more memory in DGROUP by making the arrays dynamic, which
moves them into the far heap. To make an array dynamic, DIMension the
array after the metacommand REM $DYNAMIC, or DIMension the array with
a variable in its subscript.

All dynamic arrays are stored on the far heap except arrays of
variable-length strings. Strings, variable-length string arrays, and
simple variables are always stored in DGROUP. To store strings in the
far heap, you must DIMension a dynamic array of fixed-length strings.
DIMensioning strings as fixed length in dynamic arrays helps to free
up storage space that they otherwise could have taken in DGROUP, which
is limited to 64K.

Huge Arrays
-----------

Huge arrays are arrays that are larger than 64K. When using huge
arrays, you must invoke the QB.EXE editor and BC.EXE compiler with the
/AH option. The huge array must be DIMensioned as a dynamic array,
either with the metacommand REM $DYNAMIC or with a variable in the
array subscript. The /AH option allows dynamic arrays of user-defined
types, fixed-length strings, and numeric data to occupy all of
available memory.

The number of bytes in a single element of a huge array should be a
power of 2 because space is allocated for huge arrays in 64K segments.
If a record size is not a power of 2, a "gap" appears between the
allocated segments that is wasted, unused memory. The gap occurs
because an array element is not allowed to span across a 64K segment
boundary, so that element is moved up to the start of that boundary.

QuickBASIC allows only one gap to appear in a program. A "Subscript
out of Range" error occurs when allocating a huge array larger than
128K if gaps occur.

The final major aspect of QuickBASIC memory management is that
QuickBASIC attempts to make the most efficient use of memory possible,
which may cause variable-length strings, variable-length string
arrays, and dynamic arrays to move around in memory from statement to
statement at run time. (All other variables have a fixed location
determined at load time.) Because of this, the results of the VARPTR,
VARSEG, VARPTR$, or SADD function should be used immediately after
that function is invoked.

Appendix A: Sample Program
--------------------------

This sample program demonstrates the topics covered in this
application note. This program gives an example of ways to DIMension
arrays, and to pass variables between modules and SUBprograms. The
program consists of two modules, each which defines one SUBprogram.

Main Module (EXAMPL1.BAS)
-------------------------

DECLARE SUB OtherModSub ()
DECLARE SUB Subdemo ()

TYPE PowerOfTwo
I AS INTEGER ' 2 Bytes
L AS LONG ' 4 Bytes
S AS SINGLE ' 4 Bytes
D AS DOUBLE ' 8 Bytes
St AS STRING * 14 ' 14 Bytes
END TYPE ' 32 Bytes Total, a power of 2, 2^5

REM $STATIC
DIM A(100) AS INTEGER ' Static array, stored in DGROUP.

COMMON SHARED A() AS INTEGER ' Share the variables with the other
COMMON SHARED B AS STRING * 20 ' module and its SUBprograms.
COMMON SHARED X() AS INTEGER

REM $DYNAMIC
DIM X(100) AS INTEGER ' Dynamic array, stored in far heap.
DIM PTwo(4000) AS PowerOfTwo ' Dynamic huge array, stored in far heap
' Each element is a power of two, 2^5.

DIM NonFixedLen(10) AS STRING ' Dynamic array, but since it is not
' fixed length it is stored in DGROUP.

REM $STATIC
DIM SHARED M(100) AS LONG ' These variables are shared with all
DIM SHARED Equals AS STRING * 10 ' the SUBprograms in this module,
' and are stored in the DGROUP.
Num = 40

DIM DynArray(Num) AS SINGLE ' The array is dynamic since it is
' DIMensioned with a variable, so
' it is stored in far heap.
Plus$ = " + "
Equals = " = "
M(1) = 7
A(1) = 4

CALL Subdemo ' CALLs the SUBprogram of this module.
CALL OtherModSub ' CALLs the SUBprogram of the support module.
END

'SUBprogram of the Main Module:

SUB Subdemo
SHARED Plus$, NonFixedLen() AS STRING, Num
PRINT STR$(Num) + Plus$;
PRINT STR$(M(1))+ Equals + STR$(A(1));
A(1) = M(1)
END SUB


Support Module (EXAMPL2.BAS)
----------------------------

REM $STATIC

DIM A(100) AS INTEGER ' Only the static array is DIMensioned
COMMON SHARED A() AS INTEGER ' before the COMMON SHARED statement.
COMMON SHARED B AS STRING * 20
COMMON SHARED X() AS INTEGER ' The dynamic array is only DIMensioned
' in the main module.

Panic: ' The label for ON ERROR has to be at
resume next ' the module level.

' SUBprogram of the Support Module:

SUB OtherModSub
PRINT A(1)
ON ERROR Panic
END SUB

Appendix B: Link .MAP File Explanation
--------------------------------------

Below is a .MAP file created when the program example in Appendix A
is linked. The .MAP file from a successful link shows how close a
module's code segment is approaching the 64K code limit, and how
close the static variables for the entire .EXE program are to
approaching the 64K limit in the DGROUP. To generate a listing map of
a program, include a map filename in the third argument of the LINK
command, as in the following example:

LINK EXAMPL1.OBJ+EXAMPL2.OBJ,,EXAMPLE.MAP;

Notes for the link .MAP shown further below are as follows:

1. EXAMPL1_CODE is the code segment for EXAMPL1.BAS.

2. EXAMPL2_CODE is the code segment for EXAMPL2.BAS.

3. The code segment of each module has to be less than 64K. When the
code segment is getting close to 64K, break the module into two or
more modules. To find the length of a code segment, look under the
Length column. For example, the length of the code segment for
EXAMPL1.BAS is 15A hex bytes, which is 346 in decimal notation.

4. FAR_MSG is the far management group that holds text of error
messages. This is NOT the far heap. The far heap is allocated at
run-time.

5. According to the Origin section at the bottom of the link map,
DGROUP starts at 06DC:0, which is the paragraph address 06DC
hex, with an offset of 0. This is the same as the byte address
06DC0 hex (since a paragraph contains 16 bytes). Thus, in this
example, DGROUP starts where BR_DATA starts.

6. DGROUP ends at the Stop address of the STACK.

7. To find out how large the program's DGROUP is, take the Stop
address of the stack and subtract from it the Start address of
first data element in the DGROUP. For example, the size of the
program's DGROUP is 7F9F hex minus 6DC0 hex, which equals 11DF hex
(4575) bytes.

8. All addresses in the .MAP are only relative to the start of the
.EXE program. The absolute load address is determined by DOS only
at run time. The VARSEG and VARPTR statements can be used in your
program at run time to display the absolute addresses of the
variables.

EXAMPLE.MAP
-----------

Start Stop Length Name Class
00000H 00159H 0015AH EXAMPL1_CODE BC_CODE
00160H 001C1H 00062H EXAMPL2_CODE BC_CODE
001C2H 03931H 03770H CODE CODE
03932H 03A0CH 000DBH INIT_CODE CODE
03A10H 041B2H 007A3H _TEXT CODE
041C0H 065EFH 02430H EMULATOR_TEXT CODE
065F0H 065F0H 00000H C_ETEXT ENDCODE
065F0H 065F7H 00008H FAR_HDR FAR_MSG
065F8H 06C44H 0064DH FAR_MSG FAR_MSG
06C45H 06C46H 00002H FAR_PAD FAR_MSG
06C47H 06C47H 00001H FAR_EPAD FAR_MSG
06C50H 06DBFH 00170H EMULATOR_DATA FAR_DATA

06DC0H 06DC0H 00000H BR_DATA BLANK
06DC0H 06DEFH 00030H BR_SKYS BLANK
06DF0H 06EFBH 0010CH COMMON BLANK
06EFCH 070E3H 001E8H BC_DATA BC_VARS
070E4H 070E9H 00006H NMALLOC BC_VARS
070EAH 070EAH 00000H ENMALLOC BC_VARS
070EAH 070EAH 00000H BC_FT BC_SEGS
070F0H 0710FH 00020H BC_CN BC_SEGS
07110H 07122H 00013H BC_DS BC_SEGS
07124H 07124H 00000H BC_SAB BC_SEGS
07124H 0712BH 00008H BC_SA BC_SEGS
0712CH 0712FH 00004H BC_SAE BC_SEGS
07130H 07345H 00216H _DATA DATA
07346H 0761FH 002DAH _BSS DATA
07620H 0771CH 000FDH BR_DATA DATA
0771EH 0771EH 00000H XIB DATA
0771EH 07741H 00024H XI DATA
07742H 07742H 00000H XIE DATA
07742H 0774DH 0000CH DBDATA DATA
0774EH 0775BH 0000EH CDATA DATA
0775CH 0775CH 00000H XIFB DATA
0775CH 0775CH 00000H XIF DATA
0775CH 0775CH 00000H XIFE DATA
0775CH 0775CH 00000H XPB DATA
0775CH 0775CH 00000H XP DATA
0775CH 0775CH 00000H XPE DATA
0775CH 0775CH 00000H XCB DATA
0775CH 0775FH 00004H XC DATA
07760H 07760H 00000H XCE DATA
07760H 07760H 00000H XCFB DATA
07760H 07760H 00000H XCF DATA
07760H 07760H 00000H XCFE DATA
07760H 0779FH 00040H CONST DATA
077A0H 077A0H 00000H BC_DATA BC_DATA
077A0H 077A0H 00000H XOB BSS
077A0H 077A0H 00000H XO BSS
077A0H 077A0H 00000H XOE BSS
077A0H 07F9FH 00800H STACK STACK

Origin Group
06DC:0 DGROUP
065F:0 FMGROUP
Program entry point at 03A1:00C8


Appendix C: Where QuickBASIC 4.00, 4.00b, and 4.50 Stores Their Arrays
----------------------------------------------------------------------

There is one difference in array storage between programs run as
compiled .EXE files and programs run within the QuickBASIC Versions
4.00, 4.00b, and 4.50 environment (QB.EXE). In the QB.EXE environment,
an array that is static, not in COMMON, and not composed of
variable-length strings, is stored in the far heap (instead of DGROUP,
as in .EXE programs).

This changes memory management in your program, depending on where
you run your program.

Thus, in programs run within the QuickBASIC 4.00, 4.00b, or 4.50
environment (QB.EXE), arrays are stored as follows:

1. All $STATIC arrays in COMMON are stored in DGROUP and can be
referenced with near addresses.

2. All arrays of variable-length strings are also stored in DGROUP and
can also be referenced with near addresses.

3. All other arrays are stored as far objects and require far
addresses. This includes $STATIC arrays that are not in a COMMON
statement, and these arrays can move in memory like $DYNAMIC arrays.

In QuickBASIC 4.00, 4.00b, and 4.50 programs that are run as compiled
.EXE files, arrays are stored as follows:

1. All $STATIC arrays are stored in DGROUP and can be referenced with
near addresses.

2. All $DYNAMIC arrays of variable-length strings are also stored in
DGROUP and can also be referenced with near addresses.

3. All other $DYNAMIC arrays are stored as far objects.

Appendix D: Memory Maps
-----------------------

This appendix contains illustrations of three different run-time
memory maps.

Figure 1
--------
The first figure (below) shows the run-time memory map when the
program is executed within the QB.EXE Version 4.x environment:

_______________
| Quick |
| Library |
|_____________|
|Communication| User-specified size
| buffers |
|_____________|
| | This area contains items, such as
| FAR heap | large/huge arrays and user code,
| | dynamic arrays.
|_____________|
User Data ------->|Run-time heap| Files buffers, etc.
End DS:xxxx |_____________|
| String heap |
|_____________|
| User Program|
DGROUP | Data |
|_____________|
UP to | User Stack |
64K |_____________|
| Quick Lib |
| Data |
|_____________|
| QuickBASIC |
User Data | Static |
Start DS:0 ------->| Data |
|_____________|
| QuickBASIC | QB.EXE
Low Memory ------->| Code |
|_____________|
| MS-DOS | MS-DOS Operating System
_______________
0000:0000 ------->

Figure 1 (above): The Memory Map for QB.EXE 4.x Environment


Figure 2
--------
This second figure (below) shows the run-time memory map as it appears
when the run-time module BRUN4x.EXE is used with the separate
compilation (Make EXE File...) method:

_______________
| BRUN4x.EXE | Separately loaded run-time code
|_____________|
|Communication| User-specified size
| buffers |
|_____________|
| | This area holds less-frequently-
| | used items, such as dynamic
| FAR heap | arrays, the user environment
| | table.
| |
| |
|_____________|
User Data ------->|Run-time heap|
End DS:xxxx |_____________|
| String heap |
|_____________|
| User Stack | Preset to 2K
|_____________|
| Named | Named COMMON areas
| COMMON |
DGROUP |_____________|
| BC_DATA | User program variables
Up to |_____________|
64K | BC_CONST | User program constants
|_____________|
| Blank COMMON|
|_____________|
| _DATA | QuickBASIC run-time data areas,
User Data | CONST | used during user code execution
Start DS:0 ------->| _BSS |
|_____________|
| User Code | User program separately linked
Low Memory ------->| | with BRUN4x.LIB
|_____________|
| MS-DOS | MS-DOS Operating System
_______________
0000:0000 ------->

Figure 2 (above): The BRUN4x.EXE Run-Time Module Memory for an .EXE


Figure 3
--------
This third figure (below) shows the run-time memory map when the
stand-alone library (BCOM4x.LIB) option is used with the separate
compilation (Make EXE File...) method:

_______________
|Communication| User specified size
| buffers |
|_____________|
| | This area contains less frequently
| | used items, such as large
| FAR heap | numeric arrays, the user
| | environment table, dynamic
| | arrays.
| |
|_____________|
User Data ------->|Run-time heap| Files buffers, etc.
End DS:xxxx |_____________|
| String heap |
|_____________|
| User Stack | Preset to 2K bytes
|_____________|
| Named | Named COMMON areas
| COMMON |
DGROUP |_____________|
| BC_DATA | User program variables
Up to |_____________|
64K | BC_CONST | User program constants
|_____________|
| Blank COMMON| Library and user definitions
|_____________|
| _DATA | QuickBASIC run-time data areas,
User Data | CONST | used during user code execution
Start DS:0 ------->| _BSS |
|_____________|
|Run-time code| Run-time code linked into file
|_____________|
| User Code | User program separately linked
Low Memory ------->| | with BCOM4x.LIB
|_____________|
| MS-DOS | MS-DOS Operating System
_______________
0000:0000 ------->

Figure 3 (above): The Stand-Alone (BCOM4x.LIB Library) Memory for
an .EXE