Contents of the FFS.DOC file
Supplemental Documenation for RBBS 17.3
Fast File Search
by Ken Goosens, 10-22-89
RBBS 17.3 has a signficant enhancement to the way searches for a requested
file are done in and upload and download. This enhancement has two
(a) File searches are much faster. In the case of very slow devices like
a CD-ROM, this can be the difference between sub-second response and
a 45 second response. File searches are now virtually instantaneous.
(b) File not stored on this system can trigger macros. This allows
requests for files stored elsewhere, either off line on backups,
or on other cooperating systems, to trigger later processing.
Fast File Searching
For directories DOS uses chained "buckets" in multiple locations,
searched sequentially. This results in very slow performance when
the number of files to search gets into the thousands. RBBS now
supports a fast file search than more than compensates for the
inefficent operation in DOS.
The basis for the fast file search is a file, configuration
parameter 267, which is a sorted list of file names available for
downloading. The default name is FIDX.DEF.
The format of this file is:
columns 1-12: file name
columns 13-16: location index (1, 2, 3, ...)
columns 17-18: carriage return line feed.
All data is stored as character data and the file is editable. The file
names must be stored with no internal spaces and a period separating the
prefix from the extension. The list of file names MUST BE SORTED BY
FILE NAME in order for the fast file search to work.
The location index is the record number (line number) of the locator file,
whose default name is LIDX.DEF, and has the following layout:
columns 1-63: location of file
column 64: any character. MAKEFIDX puts in a period.
columns 65-66: carriage return line feed.
This file is all character data and is editable. Essentially, the location
index points to a record in the location file. E.g. if FIDX.DEF has
and LIDX.DEF has
Then RBBS-BAS.ZIP is located in directory C:\DOWN2 (2nd record) and
HARPIE.ARC is located in C:\UP (3rd record).
The location field should be a drive/path terminating with a "\" if
any path is given, and file must be filled with blanks through column
63 if the path is shorter. You must put some character in column 64.
Many editors delete trailing blanks, so you should probably put in
a non-blank. A period is a suitable choice.
RBBS will use a binary search on the first 12 characters in FIDX.DEF.
This binary search can be significantly speeded by provided "tabs" for
this file, indicating the record at which the first file is that begins
with the symbols "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ". This is like
the "tabs" you see on dictionaries, so you can directly turn to the B's,
for example. A tab file has the same prefix as the file name file,
except that it adds a "T". For "FIDX.DEF", the tab file would be
"FIDXT.DEF". RBBS will automatically detect and use a tab file if
available. The tab has 72 characters in it. Each 2 bytes represents
a binary integer whose value is the record number in FIDX.DEF where the
first file occurs that begins with the respective symbols above. Thus
bytes 3-4 show where files begin with "1" and bytes 69-70 where files
begin with "Y".
Two utilities with source code are provided for setting up the fast file
searches. The source, for compiling under QB 4.5, is included.
MAKEFIDX will create both the file name list (FIDX.DEF) and the
location file (LIDX.DEF), from a list of directories (see MAKEFIDX.CFG)
or a list of file names, or both.
You must next sort FIDX.DEF by file name. A good shareware utility for
this is QSORT (QSORT FIDX.DEF /1:12).
MAKETABS will create a tab file (FIDXT.DEF) from the sorted file list
The batch file I use for recreating my fast file system is MAKEFFS.BAT.
It uses MAKEFIDX.EXE, QSORT.EXE, MAKETABS.EXE, and configuration file
MAKEFIDX.KG. You need only edit MAKEFIDX.KG to change the filespecs
(/FileSpec=) to match where your files are, and where to write the
Reconfiguring to Take Maximal Advantage of Fast File Searches
The fast file search is virtually instantaneous on an 8088 with a tab
file for 5000 file entries. The savings on wear and tear on the hard
disk can be very significant as well. And the time it takes to set up
the fast file search is only a few minutes. Most sysops, therefore, will
want to implment fast file searching, whether or not they have a slow
device like a CD-ROM.
RBBS will first search through the drive/paths specified in config
to be available for downloading, as it always did. Files not found
there will then be searched using the fast file search system.
The way the fast file search works is that a file found in its list is
looked for only in the designated location. Nothing else is searched.
The optimal way to implement fast file searching is to reconfigure the
drive/pathes available for downloading down to at most the upload
directory, and let the fast file search handle everything else. That
way, files will be searched first in the upload area, and those not found
at first will then be searched using the fast file search system.
Note that every file in the fast file search list is considered
to be available for downloading whether or not its drive/path
is listed in the configuration program as a downloadable area. Note
that you can have separate fast file search systems available for each
subboard, if desired. You can also use the fast file search for common
files and have a separate download area for the subboards.
Note: whenever you relocate files, you must re-create the fast file
search system. This is not hard since it takes little time and can be