Zipfile Duplicate Checking System (ZDCS) Ver. 2.01
Copyright (C) 1991, 1992 Michael W. Cocke
Technical Reference Manual
TABLE OF CONTENTS
ABOUT THE DOCS.................................................3
PURPOSE OF ZDCS................................................3
GENERAL TECHNICAL INFORMATION..................................4
Deletion of Duplicate Files...............................5
THE ZDCS DIRECTORY.............................................8
THE ZDCS CONFIGURATION FILE....................................8
THE ZDCS DATABASE BUILD.......................................14
Creation of the Initial Database.........................15
The Screen Display During the Database Build.............15
Additions to an Existing ZDCS Database...................17
ZDCSDB Without a Separate File Integrity Checker.........17
The Database Build Log File ZDCS-DBB.LOG.................18
THE ZDCS DUPLICATE REPORT GENERATOR...........................18
THE ZDCS DATABASE PURGE.......................................20
Creation of the BBS Ads Database.........................22
Updating the BBS Ads Database............................22
Selecting Deletion or Flagging of BBS Ads................24
- 1 -
Zipfile Duplicate Checking System (ZDCS) Ver. 2.01
Copyright (C) 1991, 1992 Michael W. Cocke
Technical Reference Manual
TABLE OF CONTENTS
THE UPLOAD FILE CHECKER.......................................27
Maximum and Actual Percent Dupes in an Upload............28
Updating ZDCS Database(s) After an Upload................29
DOS Error Levels.........................................29
CALLING THE UPLOAD FILE CHECKER...............................30
Calling ZDCSFC.EXE from EXZTEST..........................30
Calling ZDCSFC.EXE from the PCBTEST.BAT File.............30
The %1 %2 %3 Parameters..................................32
PROCESSING LOCAL UPLOADS......................................32
ZIPFILES WITHIN ZIPFILES......................................35
CD-ROMS AND DATABASE MERGING..................................36
PKZIP AV STAMP INTEGRITY......................................37
ARJ SECURITY ENVELOPE INTEGRITY...............................38
ACCURACY OF THE CRC32 METHOD..................................39
Your system is crashing during the upload test...........41
ZDCSDB.EXE (the database build) is crashing..............42
ZDCS is reporting a device I/O error (in any module).....42
ZDCS reports path/file access errors.....................43
QEMM exception 13 errors occur when running ZDCS.........43
Pre-Built CD-ROM Databases..........................46
Local Upload Processing.............................47
BBS Ad Tracker......................................47
Conversion from ZDCS 1.6x to ZDCS 2.01..............47
COPYRIGHTS AND LEGAL STUFF....................................49
- 2 -
ABOUT THE DOCS
Welcome to the technical reference manual ZDCS-REF.TXT for ZDCS
version 2.01. This manual has been prepared to give you an easy
reference guide for the various parts, functions, and options in ZDCS.
It is deliberately arranged to be modular in organization so that you
can look up whatever interests you without having to read the entire
This manual should be the =second= doc file that you read. We
recommend that you first read the walk-through ZDCSWALK.TXT if you are
new to ZDCS, or the conversion file CONVERT.TXT if you are running
ZDCS 1.6x and looking to convert to version 2.01. That will get you up
and running painlessly.
The walk-through ZDCSWALK.TXT is a friendly guide that is meant to
hold your hand and whisper sweet explanations in your ear as you
install and first explore ZDCS. It's arranged to take you through all
the steps you need to know from beginning to end. The walk-through is
the first place that new users of ZDCS should go for guidance when
installing and configuring ZDCS.
The conversion file CONVERT.TXT is written specially for the user of
ZDCS 1.6x upgrading to version 2.01. It provides instructions for
quick conversion from the older versions of ZDCS to version 2.01. If
you're already familiar with ZDCS 1.6x and want to upgrade to the
current version, this is the first place you should look for help in
getting ZDCS 2.01 up and running on your system. It is also the place
to go for answers to questions specific to the upgrade, like how to
convert your old ZDCS 1.6x database to the new database format used in
PURPOSE OF ZDCS
ZDCS is a shareware set of utilities intended to help a PCBoard sysop
deal with the problem of duplicate files, whether those files are
already on the bbs or are being uploaded by a caller. It provides
specific support for looking inside archives made with PKZIP,
including PKZIP version 1.93 (ZIP files), archives made with ARJ (ARJ
files), and self-extracting files made with either PKZIP or ARJ (SFX
files). ZDCS processes all of these archives so that the differences
between them are transparent to the sysop and caller. To make this
walk-through easier to read, we'll refer to all of these file types as
archives. ZDCS also provides support for accepting unzipped GIFs.
ZDCS helps to manage the problem of duplicate files in two ways.
1. It provides a method for weeding out duplicate files from an
existing collection of files, like a bbs file system or a CD-ROM
collection before mastering.
- 3 -
2. It provides a duplicate checking method for any prospective
addition to the existing file base, such as uploads to a bbs or
late additions to the CD-ROM collection.
ZDCS was written by a sysop and programmer who knows that sysops never
have enough time to do all the things they would like to do on their
boards. Making ZDCS friendly to use for both sysop and caller has
been important since version 1.0 was first written.
Although ZDCS was developed primarily for use on PCBoard bbs's, it can
also be used to look for duplicates on other systems as well, such as
a shareware CD-ROM or even multiple directories on your hard drive
system. However, it's on a bbs that ZDCS really shows its stuff.
ZDCS can be told to decline an upload, to automatically remove
duplicate files, to delete those pesky little bbs ads from uploads,
and to recognize "allowed duplicates" - or any combination of the
above. You can tell ZDCS to accept anything that a privileged user
uploads, and change your mind on who's "privileged" and who's not.
There's even a pre-test capability that lets callers easily find out
ahead of time whether or not their intended upload duplicates files
already on your board. All of these are discussed in this reference
manual. Setup and configuration are also described in the walk-
through guide ZDCSWALK.TXT.
GENERAL TECHNICAL INFORMATION
ZDCS handles file archives with the .ZIP extension created by PKZIP
(including version PKZIP 1.93), with the .ARJ extension created by
ARJ, and with the .EXE extension created by PKZIP or by ARJ (self-
extracting files, also called SFX). ZDCS is able to see inside each
of these archives to deal with the individual files inside them. For
simplicity, we will refer to all of these files as zipfiles.
ZDCS also handles GIF files. These are individual graphic files that
already have their own compression. GIFs may be uploaded to some
bbs's as individual files. They are not archives of multiple
freestanding files, so ZDCS does not need to look "inside" them. In
fact, ZDCS thinks of them rather like poor zipfiles with only a single
file in them.
Whenever ZDCS encounters a file that is not a ZIP or an ARJ or an SFX,
it treats that file like a GIF. Although we will continue to use the
term GIF to refer to these files, ZDCS could actually handle any
"other" type of file by treating it as a single file, the same as it
does with GIFs.
ZDCS does not provide support for archiving methods besides ZIP and
ARJ. This means that if ZDCS encounters an LZH archive, for example,
it cannot see inside the archive to look at the individual files.
Instead, the entire archive would be treated as a single GIF-type
- 4 -
ZDCS makes use of the 32-bit CRC, often called the CRC32, used
internally by PKZIP for ZIPs and SFXs. When ZDCS encounters any other
file, it considers that file to be a GIF and calculates the CRC value
ZDCS checks files for duplicates by keeping a database of the CRC
values. When new files are added to the system (uploads), ZDCS
compares the CRC values of the newcomers to those in the database to
determine if there are any matches, indicating duplicate files. This
method uses the contents of the files and is independent of the names
of those files.
The ZDCS database of CRC values uses a B-tree index, so there are no
external sort utilities or regular file maintenance requirements of
ZDCS is compatible with any Netbios compatible lan, such as Lantastic
or Novell Netware. It has also been tested extensively with many CD-
ROM devices and with Alloy slave cards.
ZDCS has five sets of options that can be configured independently of
each other and changed at any time. You can try out the different
options and change your mind about which ones you want to use without
For a good guide to choosing the options to fit your system, take a
look at the walk-through ZDCSWALK.TXT. As part of the installation
guide it explains what each option can mean to your bbs and what some
of the possible consequences are. Each of these options is also
covered in greater technical detail in other sections of this
Running ZDCS without any of the extra options enabled gives you a
basic duplicate checking system that will not delete any files
automatically. It will still tell you about all duplicate files, but
it won't be able to distinguish between files you always want to
delete (like bbs ads), files that duplicate exactly ones already on
your board, and files that are duplicates but still important to keep
(like VALIDATE.COM from Macafee's SCAN program).
You can try out the different options and change your mind about which
ones you want to use without re-installing ZDCS. All it takes is a
change to the appropriate line(s) in the configuration file.
Deletion of Duplicate Files
ZDCS can be set to either flag or delete duplicate files from uploads
to your bbs. Either way, ZDCS will still recognize the files that are
duplicates of ones already in the bbs file system and leaves you
messages in a log file. This deletion feature does not operate on a
- 5 -
Enabling the deletion of files is the only way that ZDCS could ever
have the opportunity to change the AV stamp on a zipfile made with
PKZIP, or the security envelope on a zipfile made with ARJ. That
happens only if ZDCS deletes a file that is included "inside" the AV
stamp or the security envelope. (It is possible to create a either
type of archive that has additional files added outside stamp or
envelope but still inside the same archive.) ZDCS can be set to
delete duplicate files and / or bbs ads, with each option being set
independently of the other.
You can designate certain files as allowed duplicates. Although these
files may already be present on your bbs, you can tell ZDCS not to
treat them like duplicates. This prevents ZDCS from deleting them if
you have selected the option to delete duplicate files. It also
prevents the allowed duplicates from being counted in the calculation
of the percentage of duplicates in an upload.
Let's look at an example to see both of these effects in action.
A caller uploads a zipfile containing eleven files. One of
those files is in your allowed duplicates list, and three of the
files are already present on your board. ZDCS will report that
there is one allowed duplicate in the zipfile. Since that one
file is not counted in the calculations, ZDCS will ignore it
when determining the percentage of dupes in the zipfile:
Total Number of Files 10 (not eleven!)
Number of Dupes 3 (allowed dupes don't count)
Percentage of Dupes 30% (3/10, not 3/11)
This feature does work for GIF files. If a GIF is included among the
allowed duplicates, then a repeat upload of the same GIF will not be
flagged as a duplicate, and the upload will be accepted.
You can designate one or more user names as privileged users. Any
file that is uploaded by a someone on the privileged user list is
never declined by ZDCS, no matter how many duplicate files there may
be in it. This provides you with the option of excluding one or more
usernames from the usual dupe checking requirements you've set for the
rest of your callers.
You can specify certain files as truly obnoxious. These are
collectively referred to as bbs ads after one of our favorites,
although they could also include get-rich-quick files or any other
nuisance file that you've already seen once and don't ever want to see
again. ZDCS can be told to either flag or delete these files from all
- 6 -
This option is completely independent of the option to delete
duplicate files. A major distinction between the two types of files
is that bbs ads are files that are known and despised ahead of time.
This option provides a safe way to delete them without risking the
removal of authors' unchanged files from newer shareware versions.
The bbs ads option does not operate on a GIF file.
Enabling the deletion of files is the only time that ZDCS can ever
change the AV stamp on a zipfile made with PKZIP, or the security
envelope on a file made with ARJ. That happens only if ZDCS deletes a
file that is included "inside" the AV stamp or the security envelope.
(It is possible to create either archive that has additional files
added outside the stamp or envelope but still inside the same
archive.) ZDCS can be set to delete bbs ads and / or duplicate files,
with each option being set independently of the other.
ZDCS offers an easy method for callers to your bbs to pre-test an
upload before actually sending the full file to the board. The pre-
test is quick and simple for them to do, and gives the same
information that callers would see after sending the full file. There
are no upload credits granted for using just the pre-test feature
without sending up the full file.
A canned bulletin is included in this package to explain to your
callers how to use the pre-test feature. The sysop side of pre-
testing is also easy to implement, and is discussed in the PRE-TESTING
section of this manual.
There are six basic steps to installing ZDCS to work with the bbs:
1. Copying the ZDCS files to a new ZDCS directory. (required)
2. Setting up the configuration file. (required)
3. Creating the initial database. (required)
4. Creating the bbs ads database). (optional)
5. Creating the list of allowed duplicates. (optional)
6. Setting up the check for uploaded duplicates. (required)
There is a walk-through called ZDCSWALK.TXT included in this package
to help you install ZDCS. It's a friendly guide to understanding the
functions and installing the programs on your bbs. It is especially
good for taking you through the steps slowly and in order so that
installation goes smoothly on your system.
If you're already using ZDCS version 1.6x and you want to convert to
version 2.01, then the file you need is CONVERT.TXT. It guides you
through the conversion so that you don't have to spend time creating a
new database from scratch or reinventing the wheel.
- 7 -
This technical reference manual has information on all the parts,
functions and options of ZDCS, but it's arranged with a different
purpose in mind: to allow you to find and read only those specific
sections you want to know more about. If you want to install ZDCS, we
highly recommend using the walk-through as your primary guide and all-
around hand-holder. If you want to convert from ZDCS 1.6x to ZDCS
2.01, then we recommend the conversion file as your best friend for
speeding up the process.
THE ZDCS DIRECTORY
All of the working files for ZDCS should be collected together in one
directory. The only possible exception to this is the ZDCS database
(ZDCS.DAT and ZDCS.NDX) created for your system, which you have the
option of locating elsewhere. All of the executables in this package
(*.EXE), the configuration file (ZDCS.CFG), and any additional files
you create such as the bbs ads database or the list of allowed
duplicates, must all be together in this one directory. For
simplicity, this directory is referred to as your ZDCS directory.
The ZDCS database is by far the biggest set of files for most bbs and
CD-ROM applications. The option to locate it somewhere else besides
the ZDCS directory is a convenience, especially for sysops who may not
have enough disk space where they want to put the ZDCS directory.
The ZDCS directory should be part of your PATH statement. Remember
that you'll need to reboot after changing the PATH statement in order
for the changes to take effect.
THE ZDCS CONFIGURATION FILE
The ZDCS configuration file ZDCS.CFG is central to the entire process.
All the modules of ZDCS look for and use at least parts of this file.
It's also the place where you set many of the options and all of the
parameters of ZDCS.
For such a powerful component, the ZDCS.CFG is surprisingly simple:
just thirteen short lines of ASCII that you can create with any text
editor. A sample configuration file is included with this package.
The contents of each line are discussed in excruciating detail further
on in this section.
Under most circumstances, ZDCS.CFG should be in the same directory as
the rest of the ZDCS files. This is always true if you are running
DOS version 3.x or higher.
- 8 -
The one time that the configuration should be located elsewhere is for
those poor lost souls still running DOS 2.x. In this unusual case,
ZDCS.CFG belongs in whichever directory will be the current directory
when the ZDCS program modules are run.
This line is the complete drive, path and filename of an ASCII text
file. This file is one that you create listing all the pathnames, one
on each line, for all the directories containing the zipfiles / GIFs
to be included in the database.
PCBoard refers to this as the download path list and defaults to
DLPATH.LST as the file name in PCB setup. You can use that file to
point to the complete bbs file directory only if you are =not= using
the index file feature in PCBoard 14.5a.
You can also use the freeware utility DIRTREE (by Mike Cocke,
available on The Hacker Central BBS) to create a text file of all
directories on a specified drive. Depending on how your system is
arranged, you might be able to use the resulting text file directly,
or you might need to delete a set of lines from the file. In any
case, you won't have to wonder whether you've remembered to include
all the pathnames.
There is no upper limit on the number of pathnames that can be
processed. It does not matter whether or not you've included the
trailing backslash for each pathname.
To process a new collection of files into the ZDCS database, like
those on a CD-ROM, just change either this line or the contents of the
file to which it points.
This line is the complete drive, path and filename where you want the
finished ZDCS database (ZDCS.NDX and ZDCS.DAT) to be located. It
makes no difference if you include the trailing backslash here or not.
You can put the ZDCS database in the same directory as the rest of the
ZDCS files and programs, or you can decide to put it on a different
drive or even across the network. This was made possible because the
database is the largest pair of files, and some sysops needed the
flexibility of locating them on a drive with more spare room. All the
rest of the ZDCS files (such as the bbs ads, the allowed duplicates,
and whatnot) should be grouped together in the same directory with the
This line is the complete drive, path and filename giving the location of
the privileged user list file. If you don't want to have any privileged
users on your system, simply leave this line blank.
- 9 -
A privileged user is someone who can upload no wrong. Whenever a file is
uploaded by a user named on the privileged user list, the file is
explicitly passed by ZDCS, no matter how many duplicates there might be
in it. Even if you have enabled deletion of duplicate files and / or bbs
ads, ZDCS still will not delete them if the original zipfile was uploaded
by a privileged user. The privileged user status takes precedence.
The format of the file containing the privileged user list is
straightforward. Put one username on each line and end each line with a
CR/LF. It makes no difference if you use upper case, lower case, or a
mixture of the two. There is no maximum number of names you may put in
the privileged user list, but remember that if you make this a long list,
ZDCS will take longer to check new uploads because it will have to check
every name in this list.
This feature was added for a couple of sysops who wanted to pass specific
files (beta code) to each other as part of unattended mail runs, and
wanted those files automatically posted for public access instead of
possibly held for later sysop review.
This line is either the letter Y or the letter N. It controls whether
you want ZDCS to add the disposition line to the end of the upload
description. The disposition line shows the total number of files in an
upload and the number of those that were duplicate files. It is used
when ZDCS declines an upload. Here is a sample disposition line:
ZDCS: 12 Duplicates / 12 files
The actual numbers will change depending on the file.
ZDCS has to be running in standalone mode for this, not with any of the
current gateways to file integrity checkers, such as with Extest. When
ZDCS is running with a gateway, then the actual contents of this line are
ignored by the program. Technically, it's the use of the OMP parameter
in the gateweay connection and not the existence of the gateway itself
that is the controlling factor. If you're really interested in the
abstruse technical details, take a look at the third party interface file
ZDCS-TPI.ZIP on The Hacker Central BBS.
You still need to have this line present in the configuration file so the
total line count doesn't fall off, but the line can contain anything as a
This new feature was added by request. Please note that you must be
running PCBoard 14.5a in order to make use of it, and that the third
command line parameter must be specified (ZDCSFC %1 %2 %3) in your
PCBTEST.BAT file. Otherwise, leave this fourth line of the config set
to the letter N.
This line is either the letter Y or the letter N. It controls whether
you want ZDCS to truncate nulls from the end of GIFs (or "other" type
- 10 -
files) before performing any operations on them. The truncation is
actually done on a copy of the file and the original is left intact.
Any AV stamp that is present is not disturbed.
Nulls (ASCII 0) can be added to the end of a file by some transfer
protocols (such as Ymodem) in order to make the entire file come out
on a block segment. Other protocols (such as Zmodem) do not add the
null characters. That would make the identical file uploaded by the
two different protocols slightly different files by the time they
arrived on your system. If the nulls were included in the file
contents when any calculation or comparison was done, it would look
like two different files instead of the same file transferred by two
different protocols. Setting this line to the letter Y enables ZDCS
to ignore those extra nulls.
The advantage of using this feature is increased accuracy when the
same file is uploaded by different transfer protocols. The
disadvantage is that it adds a bit of time to the upload checking and
database build operations. The time is needed for ZDCS to create a
temporary copy of the file in order to remove the exrta nulls, and
again to delete that temporary file, leaving the original upload
intact. Use of the option to truncate nulls also requires enough
additional disk space to hold a temporary copy of the file so that the
original one is not disturbed.
This line is an integer - that's a whole number, no decimals - between
0 and 100. It sets the maximum percentage of dupes that your bbs will
accept in an upload.
ZDCS will calculate the actual percentage of duplicates in the upload
and compare it to your maximum percentage. If the actual percentage
is lower, the upload is accepted. If the actual percentage is equal
to or higher than the maximum you specified, the upload is declined
and kept in your private upload directory for your review.
Setting the percentage to 100 effectively bypasses this filter, since
it permits a duplicated GIF or a zipfile with nothing but duplicates
to pass. At the other extreme, setting the percentage to 0
effectively requires that the uploaded GIFs and zipfiles have no
duplicates at all.
If you make a mistake and enter a decimal on line 6, ZDCS will not
crash. It will simply truncate your number (lop off everything after
the decimal point) and use the resulting integer as the maximum
percentage of dupes.
This works out quite well in actual practice. Uploads that are a
fraction of a percent under the maximum percentage of duplicates are
the only files where this makes any difference, and the use of
truncating instead of rounding means that they will always be passed.
IF ZDCS rounded the decimal points instead, there would be some
uploads that would round up, making their actual percentage of dupes
the same as the maximum value. Such files are declined by ZDCS.
- 11 -
This line is the complete drive, path and filename you want ZDCS to
use for the log file created by the upload file checker ZDCSFC. This
log is an ASCII text file that contains information from the upload
file checker ZDCSFC for each upload it has processed. Each message
about a specific upload includes the following:
name of the uploaded zipfile or GIF
list of all component files inside a zipfile
which files are flagged as bbs ads
which files are marked as duplicates
reference to the file already in the database which is duped by the
which files are marked as allowed duplicates
actual percentage of dupes in the upload
whether the upload was accepted or declined
The fifth line in the list above is new in ZDCS 2.01. It means that
when a duplicate file is found in an upload, ZDCS will provide the
reference to the zipfile or GIF containing the copy of that file which
was already on the bbs. It happened that callers who had uploaded
files with lots of dupes were not always able to find where those
files already existed on the system, often because files had been
renamed. Those callers sometimes pestered the sysop for this
information, which wasn't always easy for the sysops to locate,
either. Now this reference is written directly to the upload checker
log file The information includes the name of the zipfile or GIF and
its pathnumber in the system.
If PCBOARD.SYS is in the current directory when the upload file
checker is run, then the name of the currently logged caller is also
included in the log file.
This line is either the letter Y or the letter N. It controls the
switch to tell ZDCS whether to delete bbs ads (Y) in an uploaded
zipfile or to just flag them (N). If you've decided not to enable any
checking for bbs ads at all, just set this to N.
This is not the place where you decide if you want to check for bbs
ads. You do that by creating the bbs ads database. If that database
is found, then ZDCS will check files against the database of known bbs
ads looking for a match. If there is no bbs ads database, ZDCS won't
be able to check for them.
The choice you are making here is whether to remove those known bbs
ads from zipfiles, when they are encountered. This decision is
separate from the choosing whether to delete duplicate files in
general, which is set in line 9. You may configure these two options
independently of each other.
- 12 -
This line is either the letter Y or the letter N. It controls whether
you want ZDCS to delete all duplicate files from an upload (Y) or just
flag them and leave them intact (N).
If you choose to delete all duplicate files, then any duplicate file
that ZDCS finds that is not specifically listed as either an allowed
duplicate or a bbs ad will be deleted from the zipfile ZDCS is
processing. The bbs ads are handled by a separate option, determined
by the existence of the bbs ads database ZDCS-BBA.NDX and configured
in line 8.
This line is reserved for a single line of text by the sysop. The
contents of this line are appended to the PCBFAIL.TXT file created by
ZDCS whenever an upload is declined. The caller who has just uploaded
the declined file sees this line of text as a message on the screen.
This line is where you can express from 1 to 72 characters' worth of
creativity. Some callers have become quite fixated on the idea that
"declined" is the same as "thrown out" - which is of course not true.
You can use this line to tell the caller what has happened or will
happpen with the upload. One possible line to use is:
Too many duplicate files - upload must be reviewed by sysop.
If you don't want to display any message to the caller, just place
something innocuous like a period or even a blank space on this line.
Just don't leave the line completely blank! You also shouldn't use
any quotation marks in this line. You can make use of PCBoard
@variables and &filespec to your heart's content; both are fully
If your file integrity checker overwrites the PCBFAIL.TXT file, your
callers won't see this line of text unless you choose to use ZPEND.
This is another free utility available on The Hacker Central BBS. By
using ZPEND with such an integrity checker, you will make sure that
this line of text is displayed to the caller. Also displayed will be
a complete ZDCS breakdown of the upload, showing which files are bbs
ads, dupes, allowed duplicates, and so on.
This line contains the filename ZDCSFC.OUT and nothing else. For all
practical purposes, this line is planning for the future.
Technically, this line points to the name of an ASCII text file that
will be created every time an upload is processed. It's a spare copy
of the ZDCS summary information that most upload checkers overwrite.
ZPEND and some utilities-in-progress use it.
Leave this line set to ZDCSFC.OUT until further notice or features.
- 13 -
This line is the complete drive and pathname of a RAM drive that is
available to ZDCS for certain types of processing work.
ZDCS can make use of a RAM drive to speed up the processing of
embedded archives (referred to as zipfiles within zipfiles). It can
also use the RAM drive to process GIFs ("other" files) if you set
config line 5 to Y in order to truncate nulls.
The existence of the RAM drive is verified by ZDCS, but the amount of
space available on it is not checked. If you run out of space on the
RAM drive while ZDCS is processing an upload, the upload will be
If you do not want to use a RAM drive, leave this line blank.
This line consists of the single letter Y or N. It controls whether
ZDCS displays the one line "Registered to..." message after the board
receives an upload (Y) or turns off the display of this message (N).
Either way, the caller still sees the file by file breakdown of the
upload and the status (duplicate, bbs ads, etc.) of each file.
This feature is only recognized by the registered version of ZDCS. It
has no effect on the three line message displayed by the unregistered
version. It is also the only line of the configuration file that you
can forget to include without causing major problems. If the line is
missing, ZDCS defaults to (Y) and displays the three line message.
This would probably be a fine time to wax poetical about the
advantages of registration. Instead, we'll just direct you to the
section in this manual on registration for more information about
that, and to the section on support for some of the reasons why.
(Actions speak louder than words.)
THE ZDCS DATABASE BUILD
In order for ZDCS to process new uploads, it must first have created a
database for the existing files to serve as a standard of comparison.
This is referred to as the duplicate file database, the ZDCS database,
or the CRC database. For simplicity, we will also refer to it as
"the" database. Information stored in this database includes pathname
information, the file name of every zipfile and GIF examined by ZDCS,
and the names and CRC values of every individual file.
- 14 -
There is another purpose to building the ZDCS database besides
enabling ZDCS to check uploads. It also serves as a tool to help you
weed out duplicate files from the existing collection.
Creation of the Initial Database
THE ZDCS database of duplicate files is created with the database
build program ZDCSDB. The finished database consists of two files:
ZDCS.NDX and ZDCS.DAT. A third file is also created during this
process: ZDCS-DBB.LOG, which is an ASCII text file logging messages
from the database creation process.
There is no limit to the number of component files within a zipfile.
ZDCS will "see" and process all the member files. The more files in
the zipfile, however, the longer the processing of that archive will
To create the database, you simply run ZDCSDB. This program looks for
the ZDCS configuration file to tell it which files you want to
process. These can be in multiple directories, on different drives,
or across a network. Once you start the build process, there is
nothing more to do until it is complete. Everything will run
automatically with no need for additional input from you. While the
program is running, there is a screen display that gives you the
current status of operations and one very important piece of
information: use the escape key if you wish to abort the process.
Additional explanation of the screen display is covered in the next
section of this manual.
The Screen Display During the Database Build
The ZDCSDB status summary contains both useful and esoteric
information. The most important of these is the notice that the
escape key is the one to use in order to abort the database build.
Aborting the process any other way will almost certainly lead to lost
clusters on your hard disk, not to mention a very unhappy sysop.
In general, ZDCS treats an SFX file made with either PKZIP or ARJ just
like a ZIP or ARJ file, and this documentation has reflected that fact
by using the term "zipfile" to refer to all of these archives. For
the purposes of this section, however, we will use the individual
terms SFX and ZIP, since the screen display does distinguish between
One line in the middle of the screen will change frequently as ZDCSDB
is processing files. Besides letting you know that the system is
working, this line tells you something about the processing steps as
they take place, just in case you were curious.
- 15 -
The possibilities are:
Initializing. ZDCSDB is pulling its act together and gearing up for
Directory C1.. ZDCS has just begun processing a new directory
Directory C2.. ZDCS has just retrieved the next ZIP / ARJ / SFX / GIF
file within the same directory
ZIP Directry.. ZDCS is retrieving information from within an
SFXID......... ZDCS is testing whether the file being procesed is a
self-extracting file, and if so which archive method
(PKZIP or ARJ) was used to make it
SFX(A)........ ZDCS is retrieving information from within an
individual SFX file made with ARJ
SFX(Z)........ ZDCS is retrieving information from within an
individual SFX file made with PKZIP
ZIZ........... ZDCS is processing a zipfile (ZIP or ARJ) within a
zipfile (ZIP) or ARJ) - often called an embedded archive
CRC........... ZDCS is calculating the CRC value for a GIF file
Fast Forward.. ZDCS has already encountered and processed this exact
same file (It takes good eyes and a slow computer to catch
this indicator, which may come up when an aborted
database build is resumed.)
Indexing...... ZDCS is writing the index and data to the Btree files
Underneath this changing line is a line that tells you which ZIP, ARJ,
SFX or GIF file is currently being processed, with the full pathname
included. Below that is the phrase "Share" or "NoShare" to indicate
the type of "file opens" being used. "Share" capability is detected
by all ZDCS programs to permit automatic use of the appropriate type
of file access.
Just to the right of that is the word "Archives" followed by the
number of ZIPs, ARJs, SFXs and GIFs that have been processed so far.
And to the right of that (no political statements implied) is the word
"Members" followed by a number. That number counts all the individual
files (those within the ZIP, ARJ, SFX or GIF files) that have been
processed so far during this session. (Remember, a GIF shows up as
having only one member file.)
And all the way to the right of that same line, way out in right
field, is the phrase "DB Total" followed by the number of member files
that are already in the database.
The next line down starts with "$:" (a dollar sign and a colon)
followed by a five digit number. This is arcane internal status
information that will not be explained here. (Consider yourself
lucky.) The only time you are likely to need this is if there is a
problem with your system during the database build and we ask you to
read the screen display to us.
To the right of that esoterica is the start time. The end time will
display immediately to the right of that as soon as the database build
is complete. If you leave the room for a peanut butter and pickle
sandwich while the database is building, this provides a simple way to
tell if and when the process has finished.
- 16 -
All the way to the right, in fact in the lower right hand corner of
the display, is the word "Stack:" followed by a four digit number.
This is more arcane status information.
Additions to an Existing ZDCS Database
Adding new files to the existing ZDCS database is very simple. If you
run the database build ZDCSDB when there already is an existing ZDCS
database, the records for the new files will be added to those already
in the database. (If you temporarily rename or move the database
files mentioned in the database creation section, then ZDCS will not
recognize them until you restore them.)
This provides a very simple method for adding large or small file
collections to an existing database. All you have to do is edit the
first line of the configuration file to point to the new files, and
you're ready to go. Issue the command ZDCSDB and from here on the
process will proceed just the same way as it does for creating the
There is a second way to add files to the ZDCS database. Available
free on The Hacker Central BBS is a merge utility that lets you
combine two ZDCS databases into one. This allows you to create
smaller databases at a time and then merge them together, which can be
an advantage if you have a very large collection to process.
The main advantage of the database merge option is that it allows
sysops to use pre-built ZDCS databases for common file collections,
like the popular CD-ROMs of shareware programs. Pre-built databases
are also available on The Hacker Central BBS and are mentioned in more
detail in the section in this manual on support.
We recommend that you copy the necessary files over to a temporary
working directory and perform database merging there. This has two
advantages: it minimizes the chances that you'll get confused and
rearrange part of your "live" system; and it doesn't interfere with
the operation of the bbs. Pick your fastest available processor to
speed up the operation, and make sure you have enough disk space for
the new files, too.
The merge utility ZDCSDBMG.EXE, available in Z2-DBMG.ZIP on The Hacker
Central BBS, comes with a text file explaining step by step how to
perform a database merge using a temporary working directory. The
same kind of instructions are also included with every pre-built CD-
ROM database available on The Hacker Central BBS.
ZDCSDB Without a Separate File Integrity Checker
ZDCS is not a file integrity checker and does not intend to replace
the fine checkers that are already available. Normal operation of
ZDCS assumes that you have already processed your files with a file
- 17 -
Human nature and circumstances being what they are, a special switch
has been included with the database build module to permit some simple
file integrity checking to be done. This method has ZDCS call in
PKZIP or ARJ, whichever is appropriate, to test the zipfiles. There
is no checking performed on GIFs at all.
To use this feature, you issue the database build command as ZDCSDB T
instead of ZDCSDB. Because ZDCS now calls PKZIP or ARJ for the file
integrity checking in addition to the usual functions of the database
build, the processing time is greatly increased. If you are only
processing a small collection of files, such as a batch of local
uploads, the difference in time is probably unimportant in practice.
But if you are processing a large set of files, you'll be better off
making your first pass with a file integrity checker and then using
the regular ZDCSDB command to build the database.
If you run ZDCSDB on a collection of files that includes a corrupt
zipfile, the database build may crash. While you can find out after
the fact about the damaged zipfile by reading the log, it's still far
preferable to avoid the problem entirely by using file integrity
The Database Build Log File ZDCS-DBB.LOG
There is a log file called ZDCS-DBB.LOG created by the database build
operation. This is an ASCII text file that logs messages from the
database creation process.
If ZDCSDB encountered damaged zipfiles while trying to build the
database, there would be messages alerting you to that fact in the log
If you have any problems while running ZDCSDB, look in this log file
for help in understanding what happened.
THE ZDCS DUPLICATE REPORT GENERATOR
The primary purpose of the duplicate report generator ZDCSDR.EXE is to
use the information in your ZDCS database to give you a report of all
duplicate files in your file collection. The usual time to do this is
after you have created the initial ZDCS database.
It is entirely likely that when you first create the initial database
you will already have some duplicate files in your collection of
zipfiles and GIFs. To find out about them you need to run ZDCSDR.EXE.
First, make sure that your CONFIG.SYS file contains the following
- 18 -
This is absolutely essential for the database report generator (and
the merge utility). No fooling around. If you have to add this line
to your CONFIG.SYS, remember to reboot afterwards so that the change
Second, switch to a directory (any directory will do) that has enough
space available to handle the temporary files that will be created
during the duplicate report generation. These files are normally
deleted again when the report is completed, but you need to have
enough room to store them while the operation is in progress. If you
have less space available than ZDCSDR.EXE =estimates= it will need,
you'll be given the bad news and the opportunity to cancel the report
Third, run ZDCSDR.EXE. This program is part of the ZDCS package and
should be in your ZDCS directory with all the rest of the executables.
(Your ZDCS directory should be in your path.) This will create an
ASCII text file containing a three part report that is ready for
viewing or printing.
Part one is a list of zipfiles or GIFs that are 100% duplicates of
other files in the database. Part two is a list of zipfiles that have
some level of duplication, but also contain at least one non-
duplicated file. Part three is the complete list of all duplicate
files in the database, including the name and CRC value of the
duplicated file and the identity with full drive and pathname of the
zipfile or GIF containing the dupe.
Note that no duplicate files are deleted by ZDCSDR.EXE when you create
the initial database. The list of duplicates, ZDCS-DUP.LST, can be
used by the sysop to remove any duplicate files in the system.
Once you have the list of duplicates ZDCS-DUP.LST in hand, you are
ready to clean out your file system. Use the information to remove
any duplicate files from the file collection. On some of the larger
bbs file systems, this step alone has freed up megabytes of hard drive
After you are sure you have finished cleaning up the file system, it
is possible to purge duplicate entries from the ZDCS database in order
to reduce its size. The command to do this is ZDCSDR P and it should
=not= be used until you have read the next section in this manual, THE
ZDCS DATABASE PURGE.
There are two other switches that can be used to change the operation
of ZDCSDR.EXE - L and S.
Issuing the command ZDCSDR L will cause the usual database report to
be generated, but all those temporary work files will not be deleted.
This requires a lot of extra disk space to use. The main reason for
this switch is debugging purposes.
Issuing the command ZDCSDR S will sort the database and regenerate the
ZDCS.NDX file from the existing ZDCS.DAT file. This has a couple of
1. If your ZDCS.NDX file has become corrupted or destroyed (we won't
ask how, stranger things have happened on computers before), then
- 19 -
this feature will let you recreate the ZDCS.NDX file without
having to recreate the entire ZDCS database from scratch. If you
had a ZDCS.NDX file in the directory before using this feature, it
will be overwritten by the fresh index file.
2. If you have obtained the *.DAT file for a ZDCS 2.01 database and
you want to recreate the *.NDX file, this feature will let you do
that. You'll notice that all the pre-built CD-ROM databases
available on The Hacker Central BBS only have the .DAT file and no
.NDX file. (The *.NDX file isn't needed for the database merge
and not including it saves on disk space and transfer time.) Each
of these pre-built databases comes with step by step instructions
for using this feature to recreate the index file. The
instructions include the creation of a temporary working directory
so that you won't interfere with the "live" installation of ZDCS
on your bbs.
3. If you've added many new files to your ZDCS database (either with
lots of uploads, lucky you, or with database merges), then you
might want to investigate this feature. Running ZDCSDR S on your
system will sort your ZDCS.DAT file and will rebuild the ZDCS.NDX
file to correspond to the newly reorganized ZDCS.DAT. This makes
for a slightly more efficient operation. The time you regain in
faster processing speed will depend on how badly out of sorts the
database had become. (no pun intended, of course)
We recommend running ZDCSDR S to sort the database and regenerate
the index file once a year. Unless you process an unusually high
number of files, there's not much need to do it more frequently
than that. Certainly you should =not= put this into your daily
event on the board, or it will eat up more time than you'll save.
While this operation is much faster than the database creation
step, it can still take from 20 minutes to 2 hours to complete
(depending on the size of your system). If you run ZDCSDR S once
an average year, your gain in processing speed for upload checking
will probably be somewhere between 1% and 10%.
If you forget which switch does what, you can always issue the command
ZDCSDR ? and the question mark will tell ZDCS to list the various
options for you:
= Generate the duplicate report.
(default choice, no command line parameters needed)
P = Purge duplicate records from your ZDCS database.
S = Sort and re-index the ZDCS database without purging.
L = Perform the standard duplicate report without purging the
database, but leave all temporary work files in place.
THE ZDCS DATABASE PURGE
Every duplicate file in your initial collection of files on the bbs
will be reflected by a duplicate entry in the CRC32 database, which
ZDCS uses as its standard of comparison for checking new uploads.
- 20 -
Only one entry for each unique file is needed in order to recognize
future uploads of the same file. You can reduce the size of this ZDCS
database by purging duplicate entries from it. This leaves just one
single reference to each individual CRC32 in the database.
Before you make any changes to the ZDCS database at all, please read
the section in this manual on the duplicate report generator ZDCSDR.
The reporting function of this module will supply you with an accurate
list of all your duplicate files, including the necessary path
information - but only if you run it =before= you purge the database.
Without this list, you'll be hard pressed to find and remove the
duplicate files from your system. We strongly recommend that you not
only generate this report but also finish cleaning out the file system
before you purge anything.
When you feel you are ready to purge the database, start by making a
backup copy of it first. This is very important. =Please= make a
copy of the database and put it away somewhere safe before you start
purging info from the original. Once you purge this information, the
only way to restore it is to rebuild the database. While it is always
possible to rebuild the database, this takes a fair bit of time,
especially for larger systems. And what sysop has enough time?
By now you are probably really ready to get on with it. To purge the
database of duplicates, you use the duplicate report generator module
ZDCSDR.EXE with the P (for Purge) switch. All you have to do is type
ZDCSDR P and hit the enter key. The P will redirect the program to
purging the duplicates instead of creating a report on them. There is
nothing else to do until the purge is complete. This is not a fast
process, but it will pay you back with a smaller database.
There are some files that you really and truly don't want on your bbs
at all, ever, period. These could be illegal cracks that make the
rounds every now and then, or part of the endless succession of
pyramid schemes and scam files, or our own favorites, those rude bbs
ads some boards feel compelled to include in every one of their files.
This whole group of junk files is collectively and affectionately
known as bbs ads.
ZDCS offers a way to recognize the reappearance of these files. ZDCS
will let you target them for deletion or simply flag them for manual
removal, independent of whether or not you have enabled the deletion
of duplicate files. The distinction here is that the bbs ads are
=known= files that you have given ZDCS specific orders to deal with.
In order to recognize a file as a bbs ad, you must first tell ZDCS
about it. This is done by setting up the bbs ads database. After
that, every time ZDCS sees the same file again it knows that it's a
- 21 -
Since this option is completely independent of the option to delete
duplicate files, you don't have to take a chance on removing authors'
unchanged files from newer shareware versions. The bbs ads option
does not operate on a GIF file.
Creation of the BBS Ads Database
All you need to create the bbs ads database is the program ZDCSBA.EXE
and a collection of bbs ads, pyramid schemes, chain letters or other
Once created, the bbs ads database will consist of a single file, ZDCS-
BBA.NDX, which will be located in the ZDCS directory along with the
rest of the ZDCS files. You can delete this file at any time if you
want to start over again with a fresh collection of files.
The easiest way to create the bbs ads database involves grouping all
those nasty bbs ads together and collecting them into one zipfile.
You can use either PKZIP or ARJ and select whatever name you like for
the archive. Then you run the program ZDCSBA.EXE from your ZDCS
This is easy to see with an example. If your collection of files to
be added is called BBS-ADS (.ZIP if created with PKZIP, .ARJ if made
with ARJ), then the command you issue to create the bbs ads database
with these files is:
ZDCSBA BBS-ADS.ZIP or ZDCSBA BBS-ADS.ARJ
You don't have to do anything else. The program handles everything
for the creation of the bbs ads database.
If you want to create a new bbs ads database in the future, just
delete the old database file (ZDCS-BBA.NDX) and follow the steps for
creating a new bbs ads database. If you don't delete the old
database, then the new ads will be added to the old ones in the
database, which is an easy way to add new bbs ads. There are many
other ways to add bbs ads, which are described in the next section.
Updating the BBS Ads Database
There are four easy ways to add new bbs ads as they are foisted off on
the bbs community.
1. You can run ZDCSBA.EXE on a new collection of bbs ads when you
already have an existing bbs ads database. This is good even if
the archived collection only has a single file in it, and it
doesn't matter whether you use PKZIP or ARJ to create that
archive. ZDCSBA.EXE works just the same way here as it does for
creating a new database, with one exception: the files being
processed are added to the existing database. That's all there is
- 22 -
Example. Let's say that you've collected some bbs ads from your
board, your own downloads, and even from friends. You zip
the entire set together and call it JUNK.ZIP. To include
these files in the bbs ads database, just issue the
2. You can run ZDCSBA.EXE on an individual bbs ad without bothering
to archive it first. This is a quick way to catch a new bbs ad as
soon as you are lucky enough to get it.
Example. You've just come across an obnoxious get-rich-quick scheme
called MOOLAH.TXT. To put this file in the bbs ads
database, just issue the command:
3. You can even run ZDCSBA to include a file in the bbs ads database
when you don't have the original file, as long as you have the
CRC32 value for it. Just run ZDCSBA on the CRC32 value
immediately preceded by a dollar sign $, instead of on the file
Example. You don't have the original ad anymore, but you have a
record that the CRC32 of a new bbs ad is 12345678. To put
this file in the bbs ads database, just issue the command:
4. You can also run ZDCSBA.EXE on an ASCII list of CRC values. This
feature was added because lists of the CRC values for bbs ads have
now been assembled by some sysops and have started appearing on
boards like Salt Air. Instead of processing each CRC value
individually, you can process the whole list at once. In the
example immediately above, you would replace the dollar sign by
the at symbol @ followed immediately by the complete filename
(with path if not located in the current directory) of the list of
Example. You just picked up a list of CRC32s for a new crop of bbs
ads. The filename of the list is MORE-SOM.TXT. To add
all the CRC values in this file to the bbs ads database,
just issue the command:
You can use any or all of these methods to add new records to the bbs
ads database. If you have an assortment of ZIP and ARJ collections, a
*.COM program that generates a flashing ANSI bbs ad, a couple of plain
text chain letters, a few files known only by CRC32, and a list of
pyramid scam file CRCs, the essential data can all be entered into the
same database without having to use the same technique for each case.
In fact, there's even a way to dump the contents of the bbs ads
database to a flat ASCII file. The result is a text file with one CRC
value on each line, the same format as the input file for method
- 23 -
number four above. This means that you can "spit back" the
information in your bbs ads database in a format that another ZDCS
user can easily incorporate into his or her bbs ads database. The
command to regurgitate these CRC values is ZDCSBA #
Example. You'd like to share your accumulated bbs ads information with
another sysop, and you want to call the file CRC-DUMP.TXT.
Just issue the command:
The variety of ways you can add new bbs ads to the database makes it
easy to share information with other sysops and to catch new
annoyances as they are let loose on us.
Selecting Deletion or Flagging of BBS Ads
If you have built the bbs ads database, ZDCS will automatically detect
its presence and will compare uploads against the records in the bbs
ads database. What ZDCS does with any matches it finds depends on how
you set up the configuration file.
In line 8 of the ZDCS configuration file ZDCS.CFG, you typed one of
two letters: Y or N. The letter Y tells ZDCS to delete the offending
bbs ad. The letter N tells ZDCS not to delete it, but just to flag it
as a bbs ad.
When a bbs ad in an upload is merely flagged, that information is
still displayed to the caller and recorded in the log for the upload
file checker. If you wish, you can go back and manually delete such
files later on.
Whether or not the bbs ad is flagged, as soon as it is identified as a
bbs ad it is not counted in the calculation for the percentage of
dupes in the upload. Once you know it's a bbs ad, you really don't
care if it's a duplicate of one that's managed to get into your
database. By definition, a bbs ad is something you don't want and
would rather not have.
It is possible for this option to disturb an AV stamp on an uploaded
ZIP file, but only under the following rather narrow set of
1. The upload was created with PKZIP and contains an AV stamp =and=
2. one or more of the files in the ZIP is flagged as a bbs ad =and=
3. the bbs ad file is located =inside= the AV stamp =and=
4. you have enabled deletion of bbs ads.
If any of these four steps are missing, then there is no disturbing of
an AV stamp as a result of bbs ad processing.
You can change your mind about using deletion or flagging to deal with
bbs ads at any time. All you have to do is edit line 8 in the
- 24 -
configuration file. No other changes need to be made to run with the
Allowed dupes are a special case. They are not flagged as dupes when
they are encountered by ZDCS; they are also not considered "original"
or non-dupe files. Whether or not these files are already present on
your system, you specifically don't want ZDCS to treat any future
occurrences of them as dupes. By identifying these files as allowed
duplicates, you are protecting them from deletion, even if you have
configured ZDCS to delete duplicate files.
Furthermore, when ZDCS evaluates the actual percentage of duplicates
in an upload, all files that are allowed duplicates are not counted.
They are transparent in any calculations, completely and utterly
ignored. They aren't even counted when reporting the number of files
in the archive.
The reason behind the concept of allowed duplicates is that there are
some files that reappear frequently as critical components of more
than one major shareware package or upgrade version. These might be
order forms, registration guides, general text files, small utilities
or other files. Two common examples are the OMBUDSMN.ASP file found
in all ASP-ware and the VALIDATE.COM program included in every
revision of McAfee's SCAN package. Both files turn up unchanged in
many zipfiles. It would be misleading to treat these files as
There are two functional advantages to using allowed duplicates.
First, by not counting allowed duplicates in the actual percentage of
dupes calculation for new uploads, your system does not "penalize" a
zipfile for having one or more of these files when it comes to
deciding whether to accept or decline the file. This is especially
significant if you have set a pretty high standard for the
"uniqueness" of uploads that your bbs will accept. This is easier to
see with a real example.
Example. Suppose a zipfile had a total of 24 files. Four of those
were allowed dupes and eight were regular dupes. ZDCS would calculate
the following information:
20 files in the archive
The number of files in the archive would be 20 and not 24 because the
four allowed dupes are excluded from all the calculations. The
percentage of dupes would be 40% (100% * 8/20).
If those allowed duplicates had been counted in as dupes, then the
percentage of dupes would have been reported as 50% instead of 40%.
- 25 -
(The total number of files would have been 24 instead of 20 and the
number of dupes would have risen to 12.) If you had set the maximum
percentage of dupes for your system to 45%, then counting in the
allowed duplicates as dupes would have caused this upload to be
declined. (That maximum percentage is set in line 6 of the
configuration file and can be changed without needing to reinstall
The second advantage is for those systems that configure ZDCS to
delete duplicate files. The use of allowed duplicates can protect
popular shareware packages from being stripped of important files.
Not only does the deletion of such files alter the content of the
author's shareware package, but it also destroys any AV stamp the
author placed on the work.
Whether or not the option to recognize allowed dupes is enabled is
controlled by the presence or absence of an ASCII text file called
ZDCS.ADN in the ZDCS directory on your system. This file is one that
you create with any text editor to list all the files, one per line,
that are allowed duplicates. You can designate an allowed duplicate
by either its file name or its CRC value. You can even mix the two
styles in the same file if you want to specify some files by name
(like VALIDATE.DOC) and others by CRC value.
To specify an allowed duplicate by name, just type the dollar sign $
followed immediately by the name of the file with its extension.
Don't leave any blank spaces in between. This preserves a file with a
distinctive name (like OMBUDSMN.ASP) even if it undergoes some
To specify an allowed file by its CRC32 value, type the pound sign #
followed by the CRC32 for the file. Again, don't leave any blank
spaces in between. If you don't know the CRC32 offhand, you can get
this information from PKZIP or ARJ. For an archive made by PKZIP, use
to give you the CRC values for each individual file inside the
zipfile. For an archive made by ARJ, the corresponding command is
The ZDCS.ADN file may be up to 256 lines long, but must contain no
blank lines and no blank spaces. You can edit, delete or recreate
this file at any time without having to set any other switches, alter
the configuration file, or make any other changes to the setup of
While the expected application of the allowed duplicates concept is
with zipfiles, the feature does work for GIF files too. If a GIF is
included among the allowed duplicates, then a repeat upload of the
same GIF will not be flagged as a duplicate, and the upload will be
- 26 -
THE UPLOAD FILE CHECKER
Once you have created the initial ZDCS duplicate file database, you
can get the bbs to check all uploaded zipfiles and GIFs against it
from then on. This is done by processing the uploaded zipfiles and
GIFs with the upload file checker ZDCSFC.EXE as the files are
received. The primary purpose is to catch uploaded files that
duplicate ones already on the bbs.
ZDCSFC.EXE can also flag bbs ads, as described in that section of this
manual. Depending on whether you have enabled any deletion options in
the configuration file, ZDCSFC.EXE can also perform deletions of
duplicate files and / or bbs ads. With the use of a list of allowed
duplicates, also described in a separate section in this manual, the
file checking performed by ZDCSFC.EXE can be made even more
ZDCSFC.EXE works with the CRC value for each file in an upload. If
the uploaded file is a GIF, it first calculates the CRC value for that
file. If the uploaded file is a zipfile, ZDCSFC.EXE reads the CRC
value for each individual file in the zipfile.
Then ZDCSFC.EXE compares the CRC for each file in the upload against
the ZDCS database. If you have created the bbs ads database, which
you would have left in the ZDCS directory on your system, ZDCSFC.EXE
also compares the files in the upload against the bbs ads database.
And if you have created the allowed duplicates list, also located in
the ZDCS directory, ZDSCFC.EXE will check the upload against that,
The results are displayed file by file for the caller after the upload
has been received and processed, providing real time feedback. The
same information is also available to the sysop on screen and as part
of the ZDCSFC.EXE log file. The complete path and file name for this
log were specified in line 7 of the configuration file.
ZDCSFC.EXE also calculates the actual percentage of duplicates in any
upload and compares this against the maximum percentage you have set
for the board. On the basis of this comparison, an upload may be
accepted or declined. Declined uploads are not deleted from the
system. PCBoard keeps a declined upload in your private directory for
The calculation of percent duplicates is covered in more detail in the
next section of this manual.
- 27 -
Maximum and Actual Percent Dupes in an Upload
ZDCSFC.EXE calculates the actual percentage of duplicate files in the
upload. Since a GIF is a single file, it will either be 0% (not a
dupe) or 100% (a dupe). For zipfiles, this actual percentage can vary
anywhere between 0 and 100. ZDCSFC.EXE compares this actual
percentage against the maximum percentage you set in line 6 of the
If the actual percentage is lower than the maximum, the upload is
accepted. If the actual percentage is equal to or higher than the
maximum you specified, the upload is declined. PCBoard keeps these
declined files in your private upload directory, where you can review
them. Some callers have become frantic at the idea that a "declined"
file is thrown out. This does =not= happen with ZDCS.
If you want to bypass this filter, set the percentage to 100. This
permits a duplicated GIF or a zipfile with nothing but duplicates to
pass the filter and never be declined. At the other extreme, you can
set the percentage to 0. This effectively requires that the uploaded
GIFs and zipfiles have no duplicates at all, or they will be declined.
When you are deciding how high to set the maximum percentage of dupes,
it helps to understand how the dupe calculation works when a file is
If any allowed duplicates are found, they are excluded from =all=
calculations. Once it identifies an allowed duplicate, ZDCS will not
figure it into the math, not even into the total number of files.
Allowed duplicates are explained in more detail in an earlier section
of this manual.
If any bbs ads are found, they are not part of the calculation of
original and duplicate files. You already know you don't want any
file that you've specified as a bbs ad, so it doesn't matter to you
whether this one is "original" on your system or one that's found its
way into the ZDCS database already. What does matter is that the file
is unwelcome either way, and so it isn't figured into this
To see how this works out in real life, let's look at two examples.
ICONS-HO!.ZIP is uploaded to your system. There are 61 files in
the archive. Of these, four are in your bbs ads database (the
file got around) and one is in the allowed duplicates list. Of
the remaining 56 files (61-4-1=56), fourteen are already present
on your system. ZDCS will figure the results like this:
60 member files (61 minus the one allowed duplicate)
4 bbs ads
25% percent dupes (14 dupes divided by 56 files, not by 60 or
- 28 -
CORNDOG.ARJ is uploaded to your system. There are 14 files in the
archive. Of these, one is a bbs ad and five are allowed
duplicates. Of the remaining 8 files (14-1-5=8), all eight are
dupes. Here's how ZDCS would figure the numbers:
9 member files (14 minus the 5 allowed duplicates)
1 bbs ad
100% percent dupes (8 dupes divided by 8 files, not by 9 or 14)
The configuration file section in this manual points out that the
maximum percentage of dupes in line 6 must be a whole number, not a
decimal. This is because ZDCS deliberately truncates any decimal from
this value. ZDCS does not round up or down but performs a
straightforward truncation on both the actual and the maximum
percentage of dupes. This was done for reasons of memory management.
An example may make it clearer how this approach works in actual
Suppose that the maximum percentage of duplicates is set to 20. An
upload with an actual percentage of 19.9 would be truncated down to
19. ZDCS would see that 19 is less than 20 and would accept this
upload. If the actual percentage were rounded instead of truncated,
it would be 20 instead of 19. ZDCS would then decline this upload
because the actual percentage would not be less than the maximum
percentage of dupes.
Updating ZDCS Database(s) After an Upload
ZDCSFC.EXE automatically updates the duplicate files database with the
CRC values of all uploads. You do not have to do anything special to
include this information in the database for comparison with future
ZDCSFC.EXE does not modify the bbs ads database at all. It's still
not smart enough to recognize a bbs ad until you've pointed it out
first - but it does remember them next time it sees them. Please see
the section in this manual on the bbs ads database if you want to know
more about how the bbs ads database can be updated.
DOS Error Levels
In addition to creating the required PCBPASS.TXT and PCBFAIL.TXT
files, ZDCSFC.EXE also sets the DOS error level when it exits. These
0 No duplicate files were found within the upload.
1 Some duplicates were found, but the upload passed the
- 29 -
2 Too many duplicates were found, and the upload failed the
3 Every file within the upload was a duplicate.
4 There is still no number 4. (Reserved for future expansion.)
5 Upload checking by ZDCSFC was aborted. Please see log file
for error message.
CALLING THE UPLOAD FILE CHECKER
ZDCS is not a replacement for using a file integrity checker on your
uploads. You should always use some kind of integrity testing on
files before they are added to the system. ZDCS is called after that
integrity checking is performed, so how you call ZDCS may be affected
by which integrity checker you are using.
Calling ZDCSFC.EXE from EXZTEST
If you have decided to use EXZTEST as your file integrity checker,
please stop by The Hacker Central BBS and pick up a copy of EXZTEST
version 2.23 beta 7MWC (or higher). This is a freeware version of the
program that contains some bug fixes not available in any earlier
EXZTEST version 2.23 beta 7MWC provides a seamless integration with
ZDCS that takes care of calling the ZDCSFC.EXE program, feeding it the
information it needs, and completing any file deletions that you have
enabled. This is not necessarily true of earlier versions of EXZTEST.
A sample PCBTEST.BAT file that calls the version 2.23 of EXZTEST is
included with this package. For more information on the ZDCS
integration implemented in EXZTEST 2.23, please see the documentation
More detailed information can be found in this manual in the section
on calling ZDCSFC.EXE from the PCBTEST.BAT file. This section is more
general and addresses the use of any file integrity checker.
Calling ZDCSFC.EXE from the PCBTEST.BAT File
ZDCS should always be used in conjunction with a file integrity
checker. When processing new uploads, the file integrity checker
should be called from the PCBTEST.BAT file before ZDCSFC.EXE. If you
are using the PCBDescribe feature in PCBoard 14.5a, that command
should be used first of all.
To use ZDCS with the file integrity checker of your choice (even if
it's just PKZIP -T), there are four basic pieces you need to include
in the PCBTEST.BAT file.
- 30 -
1. Include the following three lines to clean out old copies of
these files left over from processing other uploads:
@IF EXIST PCBFAIL.TXT DEL PCBFAIL.TXT
@IF EXIST PCBPASS.TXT DEL PCBPASS.TXT
@IF EXIST ZDCS-DEL.LST DEL ZDCS-DEL.LST
PCBFAIL.TXT contains information displayed to the caller when an
upload is declined. PCBPASS.TXT contains the information
displayed when the upload is accepted.
ZDCS-DEL.LST is a list of files targeted by ZDCS for deletion from
an upload. These might be duplicate files, bbs ads, or both. If
you haven't enabled any deletion options in the configuration
file, or if there were no files meeting the criteria for deletion,
ZDCS would not generate the ZDCS-DEL.LST file for that upload. By
deleting the files here, you make sure that no old deletion
instructions are hanging around when a new file is processed.
2. Call your file integrity tester to process the upload. The exact
command to do this will depend on which integrity checker you are
using. The most basic and rudimentary one of all is to make use
of this ability in your chosen archiver. In PKZIP you would use
the -T switch and in ARJ you would use the T switch. Or you can
use a file integrity tester designed to check for file damage,
viruses, and much more.
3. Call ZDCSFC.EXE to check the upload. The appropriate command is:
ZDCSFC %1 %2 %3
If you are running a version of PCBoard older than the May 22,
1991 version 14.5a, PCBoard will provide only two parameters.
Since the third one is absolutely necessary for pre-testing, that
means you must disable the pre-testing feature.
If you are using a newer version of PCBoard but you don't want to
use the pre-testing right now, it is still strongly recommended
that you leave the third parameter in place. It does no harm and
could save you some grief if you change your mind in the future.
At the end of processing by ZDCSFC.EXE, you might have a new file
called ZDCS-DEL.LST. This depends on whether you have enabled any
deletions and on whether there were any files in the upload that
met your criteria for deletion. If no files are to be deleted,
this control file won't be created.
4. Perform the deletion of files specified by ZDCSFC.EXE.
IF EXIST ZDCS-DEL.LST PKZIP -D %1 @ZDCS-DEL.LST
IF EXIST ZDCS-DEL.LST ARJ D %1 !ZDCS-DEL.LST
Both the PKZIP-specific and ARJ-specific commands can be included.
The system will return an error message when the "wrong" one is
encountered, but this error message will not hang the system or
cause any problems with the processing.
- 31 -
Note that the actual deletion is done by PKZIP or ARJ. ZDCSFC.EXE
creates the control file ZDCS-DEL.LST to specify the deletions and
PCBTEST.BAT passes this file to PKZIP or ARJ at this point. This
is also the only time an existing PKZIP AV stamp or an ARJ
security envelope on an upload is affected by ZDCS. More
information on these is available in two separate sections of this
manual: PKZIP AV STAMP INTEGRITY and ARJ SECURITY ENVELOPE
The %1 %2 %3 Parameters
These three variables %1, %2 and %3, are parameters used by PCBoard.
They are passed to ZDCS when the upload checking program is being
used. That's the only module of ZDCS that actually runs from inside
PCBoard, and it's the only one where these parameters are important.
The first parameter, %1, can be one of two words in PCBoard: UPLOAD
or TEST. For operation of ZDCS, this variable is set to UPLOAD, by
the very fact that a caller has just uploaded either a file for the
board, or the test file ZDCSTEST.CHK. If ZDCS should ever be passed
the word TEST here instead, it will quietly and neatly halt its
processing and control will pass on.
The second parameter, %2, is the name of the file that needs to be
processed by the upload checker. Under normal circumstances this is
the name of the file the caller just uploaded. With a few file
integrity checkers, the information in this parameter becomes invalid
after their processing because they have moved or modified the
uploaded file right out from under this parameter. In that case, the
FIXPATH.EXE utility will smooth over things. (In the SUPPORT section
of this manual there is a list of goodies, including this utility,
that are available free on The Hacker Central BBS.)
The third parameter, %3, is the name of the file that is holding the
description of the upload. When callers upload files to a PCBoard
BBS, they are asked for the file name and for a description of the
file. The description is held in a separate text file which is then
appended to the file directory with the upload file name and other
If you are running a version of PCBoard older than the May 22, 1991
version 14.5a, PCBoard will provide only two parameters. Since there
are two features that require the use of the third parameter, you must
not enable those features until you are running a more recent version
of the board code that supplies all three parameters. The two
features in question are upload pre-testing and the use of the
disposition line (set in line 4 of the configuration file).
PROCESSING LOCAL UPLOADS
There probably isn't a sysop alive who hasn't scavenged new files for
the board and uploaded them locally. ZDCS offers two ways of
- 32 -
processing these local uploads. The first method is to use the
database builder ZDCSDB.EXE; the second method is to use the upload
file checker ZDCSFC.EXE that processes all the regular uploads from
In the first case, ZDCSDB.EXE can be used to add all the new files to
the database. Just like the initial creation of the ZDCS database or
subsequent database merges, ZDCSDB.EXE adds the CRC values of the new
files to the database. The duplicate report generator ZDCSDR.EXE can
still be used to tell you about duplicates in the database. A
disadvantage to this method is that it does not give the sysop the
same kind of immediate feedback for a local upload as the sysop and
caller get for a regular upload. It also does not delete bbs ads or
duplicate files (if you have those options enabled).
To use ZDCSDB.EXE to process local uploads, you only have to change
the first line of the ZDCS configuration file. This line points to
the paths of the files to be processed when running the database
builder. (The line contains the full path / name of the text file
that lists all the directories whose files are to be processed.)
Change this to point to the uploads directory, a holding directory, or
whichever directory will hold the local uploads to be processed. Now
just run ZDCSDB.EXE and the files will be added to the existing
This method does not interfere with the realtime upload checking
performed by ZDCSFC on regular uploads, because the path information
in line 1 of the configuration file is not used by ZDCSFC.EXE. If
you've decided on a permanent holding directory, you won't even have
to touch the configuration file again to process any future uploads.
Just remember to run ZDCSDB.EXE before posting new files on the board.
This is a good time to repeat one note of caution: ZDCS does not do
any integrity testing. ZDCS is intended to work with your choice of
integrity tester, not to replace it. If you have not used an
integrity checker on the local upload files (Why not?), then you need
to use the T (for Test) switch. Instead of using the command ZDCSDB
to run the database build, use the ZDCSDB T command.
This calls either PKZIP or ARJ, whichever is appropriate, to do an
integrity check on the zipfiles. Any file that fails this test is not
processed by the database builder. There will be a message in the
database building log file ZDCS-DBB.LOG for each such damaged file.
Don't use the T switch without reason - it adds another step and slows
the processing down tremendously. But if you have a small collection
of files that haven't passed through an integrity checker, like
prospective local uploads, ZDCSDB T is one way to deal with them. If
ZDCSDB.EXE has to skip a file because it's flagged as damaged, that
information will show up in ZDCS-DBB.LOG.
The second method mimics regular uploading to get the local uploads
processed by the upload file checker ZDCSFC.EXE. This is handled via
a small utility and batch file combination available free on The
Hacker Central BBS as LOCALUP.ZIP. You'll have to modify the batch
file for your own system, of course.
- 33 -
The result is that the local uploads will see the same processing that
a regular caller's upload does on your board - including your usual
file integrity checker, bbs ad deletion (if enabled), duplicate file
deletion (if enabled), and all the rest. This has the additional
advantage of giving you the ZDCSFC.EXE log file entries for the local
uploads, which gives the status of the files inside the zipfiles:
duplicates, bbs ads, allowed duplicates, and so on.
Callers can pre-test an upload to find out how it compares to files
already on the bbs in terms of duplicates and bbs ads. The idea is to
see how the prospective upload compares to existing files in terms of
duplicates, bbs ads, etc. before actually uploading the full file to
the bbs. It can save both caller and sysop a certain amount of
The procedure is wonderfully simple from a caller's point of view and
requires nothing that might tax a relatively novice uploader's skills.
There are no special files to download, no complicated operations to
get right, no arcane rituals to perform. This makes it more likely
that callers will take advantage of the pre-test feature.
At the time of this release, ARJ is still in a beta cycle. The pre-
testing feature has been implemented for PKZIP only, and a maintenance
release of ZDCS is planned to incorporate an ARJ pre-testing feature
after the new production version of ARJ is released.
Included in this the ZDCS 2.01 package is a sample bulletin that can be
posted by the sysop on the bbs to explain to callers the ZDCS pre-
testing feature for ZIP files. The bulletin holds the caller's hands,
figuratively speaking, through the whole process. If your bbs permits
the uploading of SFX files, you might want to add those initials where
you see ZIP in the bulletin. The pre-test can recognize both ZIP
files and self-extractive archives made with PKZIP. A ZIP-only bbs
can use the canned bulletin right from the package.
To pre-test an upload called GOODSTUF.ZIP, the caller simply issues
the DOS command:
PKZIP -V GOODSTUF.ZIP > ZDCSTEST.CHK
This command is not case sensitive, so it's pretty hard to make a
mistake with it. The result of this operation is a new file called
The caller then uploads the file ZDCSTEST.CHK directly to the bbs.
ZDCS will read the information in it and will give the caller a
breakdown of the zipfile's contents, noting which individual files are
duplicates or bbs ads. This output looks just like the response a
caller gets after actually uploading the zipfile, with one exception:
- 34 -
instead of telling the caller whether the file is accepted or
declined, ZDCS tells the caller what percentage of files in the upload
are duplicates, and also gives the maximum percent duplicates which
the sysop has allowed for the board. Now it's up to the caller to
decide whether to upload or not.
Enabling the pre-test capability is not tough on the sysop, either.
The key to whether or not pre-testing is permitted on your bbs lies in
the UPSEC file. If you want to prevent callers from using pre-
testing, just disallow uploads that have the .CHK extension. On the
other hand, if you want callers to have the option to pre-test, make
sure that .CHK files may be uploaded to the system.
There's one more thing to check before letting your callers use the
pre-testing feature. If you are not using EXZTEST version 2.23 or
higher, remember to make sure that the PCBTEST.BAT file contains the
ZDCSFC %1 %2 %3
This is already discussed elsewhere in this manual as part of the ZDCS
installation. That third parameter is necessary to let the pre-
testing work for more than just the first caller. If you leave it
out, the first ZDCSTEST.CHK file uploaded to your system will work
just fine, but all subsequent attempts to pre-test will be told that
the ZDCSTEST.CHK file is a duplicate! And that means they won't be
recognized as attempts at pre-testing.
If you are using EXZTEST version 2.23 and up as your file integrity
checker, you won't have a ZDCS line in the PCBTEST.BAT file. That's
because EXZTEST has a seamless interface to ZDCS and calls it
Since this third parameter is critical to the proper functioning of
the pre-test option, you must have a recent enough version of PCBoard
that supports the third parameter. The earliest version of the board
code that satisfies this requirement is PCBoard 14.5a from May 22,
1991. This version and anything after it can handle ZDCS pre-testing.
If you track your callers' uploads and downloads, you might like to
know that the ZDCSTEST.CHK will not count as an upload. According to
ZDCS and PCBoard, the ZDCSTEST.CHK file always "fails" validation and
is not credited to the caller as an upload. The bulletin does mention
that pre-testing doesn't count as an upload, just in case some of your
callers are too enthusiastic about getting upload credits.
ZIPFILES WITHIN ZIPFILES
Processing of zipfiles contained within zipfiles is accomplished with
some caveats. Zipfiles within a zipfile are only checked one level
deep. Remember that we are using the word zipfile to refer to an
archive made with PKZIP or ARJ. In fact, you can even have an ARJ
within a ZIP or the other way around and ZDCS can still handle it.
The simplest explanation is to look at an example.
- 35 -
Zipfile A contains assorted files and zipfile B. In turn, zipfile B
contains more files and another zipfile, C. Zipfile C contains still
more files. How does the whole melange get processed?
All the individual files in zipfile A and in zipfile B have their
CRC32 values entered into the duplicate files database. If you have
configured ZDCS to delete any files (dupes or bbs ads), then those
deletions are done automatically only for the individual member files
inside zipfile A. No files are deleted from inside zipfile B. Of
course, all the duplicates in zipfile B are still listed in the log,
so you do know about them and you can decide whether to remove them
manually from your file system.
What about zipfile C? That's easy: any zipfile embedded more than
one level deep in the uploaded zipfile (and C is two levels deep) is
not processed as a zipfile at all. No file deletions, no CRCs,
In fact, *any* file that has either the .ZIP or .ARJ extension will
not be treated as an individual file in the database or the log file.
The reason is simple: it's the individual files inside the zipfiles
and not the zipfile itself that are important when looking at
duplicates. (Besides, it plays havoc with the percentages.)
CD-ROMS AND DATABASE MERGING
CD-ROMs have been growing in popularity among bbs's because they
enable a large, static collection of files (such as shareware
programs) to be available for downloading. ZDCS works just fine with
CD-ROM drives on a bbs, but the sheer size of these disks means that
it can take quite a while to build the ZDCS database for a CD-ROM.
To make life easier, you can use the database merge utility to combine
two or more databases into one integrated ZDCS database. By merging
databases, it is possible to take advantage of the existence of pre-
built ZDCS databases for the more popular CD-ROMS. This lets you
merge a pre-built database with your own bbs's ZDCS database in a
fraction of the time it would take to add all the CD-ROM information
The ZDCS database merge utility ZDCSDBMG.EXE is available on The
Hacker Central BBS for downloading as Z2-DBMG.ZIP. Included in that
package are clear, detailed directions for merging two databases. You
will need the utility program ZDCSDBMG.EXE and the *.DAT files from
the two ZDCS 2.01 databases you want to merge together.
All you really need to do is:
1. Put copies of all three files into a temporary working directory.
2. Make sure one of the *.DAT files is specifically named ZDCS.DAT
(doesn't matter which one). The other one can by ANYNAME.DAT.
3. Check your CONFIG.SYS file and make sure that you have FCBS=32,32.
- 36 -
4. Run the merge program on the new database by typing
That's it. Out the other end of the process you'll get your complete
new database, N-ZDCS.DAT and N-ZDCS.NDX. The files are ready to be
renamed and used.
The improved database merge program ZDCSDBMG.EXE eliminates the need
for the *.NDX files when you merge a database. The new N-ZDCS.NDX
file is generated right from the newly merged database. This means
that the pre-built databases available on The Hacker Central can now
be smaller, which cuts down on storage requirements and transfer time,
saving us all a little money.
Prebuilt ZDCS 2.01 databases for many CD-ROMs are available free on The
Hacker Central BBS. If you have a CD-ROM that doesn't already have a
prebuilt ZDCS database, please consider creating a ZDCS database for
the individual CD-ROM and sharing it with other ZDCS users by
uploading it to The Hacker Central. Through the end of 1993, we are
offering a free ZDCS registration as a thank you to anyone who uploads
such a database. You'll find more about our two free registration
offers in the accompanying text file ZDCS-CD.TXT.
PKZIP AV STAMP INTEGRITY
Registered versions of PKZIP can "brand" a ZIP with the unique AV
(authenticity verification) number of the registered owner. The AV
stamp is used by some shareware authors for their official
distribution packages. It is also abused by some people, including a
few bbs sysops, sad to say. Whether ZDCS will always leave that AV
stamp intact is determined by the configuration of ZDCS.
The first possibility is that ZDCS is configured never to delete any
files, whether duplicate files or bbs ads. In this case, the original
AV stamp is always retained.
The second possibility is that ZDCS is configured to delete files, but
no files need to be deleted from a particular zipfile. In that case,
any AV on the original zipfile is left intact.
The third possibility is that ZDCS deletes one or more bbs ads.
Whether the AV stamp is left intact depends on whether the bbs ad was
included in the zipfile inside or outside the AV branding. If the
offending bbs just added their advertising to the outside of an AV
stamped zipfile and left the original author's AV stamp in place, ZDCS
will do the same. If that bbs added their overhead file and then
rezipped the entire package with their own AV stamp (don't laugh,
there's at least one large bbs that's done this), then they have
destroyed any original AV stamp. ZDCS does not preserve the AV stamp
in this case. This is in keeping with the ideal that the stamp should
represent an authenticity verification from the =original= author.
- 37 -
The fourth possibility is that ZDCS deletes one or more duplicate
files. If any deleted file was included in the AV branding, then the
AV stamp is not preserved. If all deleted files were outside the
original AV stamp, then the AV stamp is intact. (This could happen
with some advertising file that you've already been hit with but which
doesn't appear in your bbs ads database.)
This last possibility deserves a little thought from the shareware
author's point of view. It is not uncommon for a new version to
contain some files that are unchanged from the previous version. If
these files haven't been flagged as allowed duplicates, it is possible
to choose to configure ZDCS so that these files are removed. If that
happens, the AV stamp is also removed.
ZDCS is quite capable of handling and retaining all AV stamps. This
is a matter of choice on the part of the sysop and is the direct
result of the simple configuration options. In order to disturb an AV
stamp, the following conditions must =all= be met:
1. ZDCS must be configured to delete dupes, bbs ads, or both.
2. The uploaded file must be a ZIP file bearing an AV stamp.
3. One or more files in the ZIP must be targeted by ZDCS for
4. At least one file to be deleted is located "inside" the AV
ARJ SECURITY ENVELOPE INTEGRITY
Registered versions of ARJ can "brand" an ARJ with a security
envelope. This envelope is used by some shareware authors for their
official distribution packages. It is also abused by some people,
including a few bbs sysops, sad to say. Whether ZDCS will always
leave that security envelope intact is determined by the configuration
The first possibility is that ZDCS is configured never to delete any
files, whether duplicate files or bbs ads. In this case, the original
security envelope is always retained.
The second possibility is that ZDCS is configured to delete files, but
no files need to be deleted from a particular zipfile. In that case,
any security envelope around the original ARJ is left intact.
The third possibility is that ZDCS deletes one or more bbs ads.
Whether the security envelope is left intact depends on whether the
bbs ad was included in the ARJ inside or outside the envelope. If the
offending bbs just added their advertising to the outside of an
enveloped ARJ and left the original author's security envelope in
place, ZDCS will do the same. If that bbs added their overhead file
and then rearchived the entire package with a new security envelope
- 38 -
(don't laugh, there's at least one large bbs that's done this, too),
then they have destroyed any original envelope. ZDCS does not
preserve the security envelope in this case. This is in keeping with
the ideal that the envelope should represent a tamper-evident seal
from the =original= author.
The fourth possibility is that ZDCS deletes one or more duplicate
files. If any deleted file was included in the security envelope,
then the envelope is not preserved. If all deleted files were outside
the original envelope, then the security envelope is intact. (This
could happen with some advertising file that you've already been hit
with but which doesn't appear in your bbs ads database.)
This last possibility deserves a little thought from the shareware
author's point of view. It is not uncommon for a new version to
contain some files that are unchanged from the previous version. If
these files haven't been flagged as allowed duplicates, it is possible
to choose to configure ZDCS so that these files are removed. If that
happens, the ARJ security envelope is also removed.
ZDCS is quite capable of handling and retaining all security
envelopes. This is a matter of choice on the part of the sysop and is
the direct result of the simple configuration options. In order to
disturb a security envelope, the following conditions must =all= be
1. ZDCS must be configured to delete dupes, bbs ads, or both.
2. The uploaded file must be an ARJ file having a security envelope.
3. One or more files in the ARJ must be targeted by ZDCS for
4. At least one file to be deleted is located "inside" the security
ACCURACY OF THE CRC32 METHOD
In order to detect duplicates, some unique signature is needed for
each file. While the only guaranteed truly unique signature is by
definition as long and cumbersome as the file itself, there are more
manageable alternatives that make use of formulas called algorithms to
generate a signature. Any algorithm is a trade-off between these two
factors: manageability and accuracy. The more unique signatures a
method can generate, the smaller the possibility that two different
files might generate the same signature.
ZDCS makes use of the CRC32 used by PKZIP and ARJ. This signature has
a total of eight "places", each filled by =independent= assortment
with one of sixteen different values. (It doesn't matter how many
places you have if they aren't each filled independently of the
others.) That makes for a total of 16*16*16*16*16*16*16*16 (16 to the
8th power) unique signatures. With over four and a quarter billion
possible signatures, this method has good accuracy.
- 39 -
It is possible to use other methods to create an even greater number
of unique signatures. These longer signatures and different
calculation methods carry a corresponding cost in complexity, disk
space and processing speed. But no signature that is shorter than the
original file can ever provide an absolute guarantee of uniqueness.
When an algorithm produces the same signature for two different files,
and eventually every algorithm will, there is a possibility for
misidentification =if= you also have both of those files. This is
because the heart of misidentification is in the comparison of
signatures from different files. If you have only one of the two
files generating the same signature, then the comparison step will
still show it to be unique on your system.
But suppose that you do beat the odds enough to have on your system
both of these files that generate the same signature. What effect
would that have? The second file you encounter would be flagged as a
duplicate of the first one.
If a caller uploads a file that contains this second, "misidentified"
file, then that upload will show one more dupe. If you have set the
percentage of maximum dupes very low and the upload contains few
files, then this might be enough to decline the upload. If your
percentage is set high and / or there are many files in the upload,
this one "extra" dupe will not tip the scale in deciding whether the
upload is declined or not.
For example, suppose the upload contained only four files. Every one
of them that was flagged as a dupe would contribute a value of 25% to
the level of duplication in the file. If your maximum percentage of
dupes acceptable in an upload were set to 10%, then a single
misidentified dupe in this upload is enough to have ZDCS decline the
Most sysops, however, use a higher maximum percentage. And most files
(excluding GIFs) contain a higher number of files on average. So
let's change the example to a maximum percentage of 50% and an upload
containing 10 files. A single misidentified dupe in this upload would
only contribute a value of 10% to the level of duplication in the
file. That wouldn't be enough to have ZDCS decline the file. So even
though there was a long-shot misidentification, the net effect is that
ZDCS still accepts the file.
But what about GIFs? A GIF upload is a single file to ZDCS. There
are only two possible outcomes for the dupe check: 0% and 100%. To
provide extra protection against misidentification in this situation,
ZDCS has implemented a =second= algorithm, one completely independent
of the CRC32 method, as a cross check. In order for a GIF to be
flagged as a dupe, it must be identified as a duplicate first by the
CRC32 method, and then again by the second, independent method.
ZDCS protects against harmful consequences of misidentification by
making use of a well-tested signature algorithm that generates over
four and a quarter billion signatures, by implementing a percentage
system that lets the sysop adjust the odds even further, by providing
independent cross-checking for GIFs, and by never deleting a failed
upload from your system. So even if a misidentification does occur
and does cause an otherwise original upload to be declined, that
upload is still available on your system in your private uploads
directory for your review.
- 40 -
ZDCS provides you with good accuracy while still being manageable.
After all, the most accurate method in the world does no good if you
can't implement it, right?
The ZDCS upload file checker ZDCSFC.EXE, the bbs ads database builder
ZDCSBA.EXE, and the ZDCS 1.6x to 2.01 conversion program Z1TO2.EXE each
require a minimum of 384K of free memory (RAM). The database build
program ZDCSDB.EXE and the duplicate report generator ZDCSDR.EXE each
require a minimum of 512K of free memory (RAM).
If your system has at least one megabyte of available EMS (expanded
memory), ZDCS can make good use of it when running any of the
ZDCSDB.EXE the database builder
ZDCSDR.EXE the duplicate report generator
ZDCSFC.EXE the upload file checker
ZDCSDBMG.EXE the database merge utility
There are no internal limits on the number of member files within a
zipfile that can be processed by both the upload file checker
ZDCSFC.EXE and the database creator ZDCSDB.EXE.
There is no better help for solving problems than experience,
especially someone else's experience with the same problems.
Your system is crashing during the upload test.
We have tracked this problem down to the following set of
circumstances so far: you are using EXZTEST to call ZDCS, =and= you
have QEMM installed on a 386 computer with an Adaptec SCSI device.
This problem goes back to memory management. Your system is trying to
juggle PCBoard, EXZTEST, ZDCS, and PKZIP or ARJ - all in memory. To
add to the confusion, QEMM is trying to help out and not always
PCBoard has the /SWAP parameter and ZDCS has the SuperShell. When
ZDCS needs to call PKUNZIP or ARJ to deal with an embedded archive
(which, by the way, is the ONLY time that ZDCS needs to call an
external program), ZDCS swaps itself out of memory to EMS (if
available) or to disk. Under certain very specific circumstances on
certain hardware configurations, QEMM seems to have a problem with
this (to put it very politely).
- 41 -
Right now your best solution to this is to get a copy of SHROOM (Shell
Headroom). This is an excellent shareware product from
Davis Augustine (Shroom Shareware)
P.O. Box 610
Sausalito, CA. 94966-0610
With SHROOM on your system, you can modify the line in your
PCBTEST.BAT file that calls EXZTEST to call SHROOM first. Look for
EXZTEST %1 %2
and change it to this:
SHROOM EXZTEST %1 %2
That should take care of the memory management problem and prevent the
ZDCSDB.EXE (the database build) is crashing.
There is a log file ZDCS-BBA.LOG that collects messages when ZDCSDB is
used to create a new ZDCS database or to add to an existing one. Read
the messages in the log file to find out why the database build
program is crashing.
Most of the time, the culprit is a damaged zipfile. ZDCS normally
depends on the fact that most sysops use a separate integrity file
checker on their boards. If a corrupted zipfile gets into the
collection of files being processed into the database, it can bring
things to a sudden halt.
The recommended solution is to process all files with a file integrity
checker first. However, there is a second alternative using just
ZDCS. You can use the T (for Test) switch to tell ZDCSDB to call
PKZIP or ARJ, whichever is appropriate, and perform a file check
first. Only those files passed by this check are processed by the
database builder. The syntax is simple: ZDCSDB T instead of ZDCSDB.
The penalty in speed for doing it this way is very significant, which
is why it is not the usually preferred solution. Unless you have only
a small handful of files to process, it's worthwhile to use a real
file integrity testing program instead.
ZDCS is reporting a device I/O error (in any module).
So far, we have seen this happen thrice. Each time, ZDCS was running
on a LAN. In all three cases, the problem was traced back to bad
spots on the hard drives.
The best medicine for this error message is to run Norton Disk Doctor,
Spinrite, or something similar on your hard drive system.
- 42 -
ZDCS reports path/file access errors.
One possibility is that you might have loaded SHARE high under DOS
5.0. There seems to be a problem with loading SHARE high; it's not
anything specifically to do with ZDCS. Try loading SHARE low instead.
Another possibility is that you haven't specified enough files in your
CONFIG.SYS file. Because ZDCS makes such heavy use of disk I/O, you
should make sure that at least 30 files are specified in your
QEMM exception 13 errors occur when running ZDCS.
There are four possible reasons for this error message.
1. Your QEMM command line is incorrect. The problem isn't with ZDCS
itself, but ZDCS is likely to be the first program to notice this
QEMM error because of the amount of data ZDCS needs to juggle. The
result is that ZDCS is able to write its data someplace it
To solve this problem, check that all areas of memory that should
be excluded from QEMM are being properly excluded. Be especially
careful if your system has super VGA, SCSI devices or a LAN; all
of these devices usually require specific areas of memory to be
excluded from remapping by QEMM.
2. You have one or more "mis-behaved" device drivers. You can check
out this possibility by removing all drivers from your CONFIG.SYS
file, rebooting the system, and trying ZDCS again. One example of
this type of problem was with an older version of the Hyperdisk
3. You believed the QEMM optimize program when it told you to get rid
of DOS stack space. Put it back! QEMM is egocentric enough to
believe it can handle all types of stack requirements for any
application properly. Let's just say that putting back the DOS
stack space solves this problem nicely.
4. You are trying to use the "stealth technology" feature of QEMM
6.x. There have been persistent reports of problems traced to
interactions between stealth and various applications from other
vendors. It wouldn't be at all surprising if ZDCS experienced the
same trouble with stealth. This is not a bug in ZDCS. Contact
Quarterdeck tech support. In the meantime, disable stealth by
removing the ST:F or ST:M from the DEVICE=QEMM.SYS line in the
ZDCS is a fully functional shareware package. There are no critical
limits, crippled features or "drop dead" dates. The only difference
- 43 -
in the unregistered version is a brief message to sysop and caller
that the version is unregistered.
A customized ZDCS.KEY file is available to registered users to change
the "unregistered" message to show registration in the name of
For sysops, registration entitles the user to a license for use of the
Zipfile Duplicate Checking System on one bbs, no matter how many nodes
it has. For CD-ROM mastering, registration entitles the user to a
license for use of one copy of the Zipfile Duplicate Checking System,
no matter how many CD-ROM collections are processed. There are no
additional fees based on the number of CD-ROMs, the gross revenue, the
phase of the moon, or any other factors.
This includes =all= future versions of ZDCS. No additional fees will
be charged for registration of future versions of this product. Once
you register, your keyfile works with all versions of ZDCS from
version 1.6x on.
Please take a look at the accompanying text file on CD-ROM uses of
ZDCS. It describes special registration offers for both bbs and CD-
ROM mastering operations. In brief, you can trade a ZDCS 2.01 pre-
built CD-ROM database for a full registration, or you can have the
registration fee waived if you include the ZDCS 2.01 pre-built database
on any CD-ROM that you master. More details are given in ZDCS-CD.TXT.
If you try ZDCS out on your system and decide that you want to
continue using it, please register your copy by sending a check for
$25.00 (US). Include the name of your bbs (up to 25 characters) as
you want it to appear to your callers. An order form (ORDER.FRM) is
included in this release for your convenience. We will prepare a key
file for your copy of ZDCS with this information in it to replace the
"unregistered version" line in the display. This keyfile will be
available for download from The Hacker Central BBS.
If your bbs is located in the continental United States or Canada, you
may choose to take advantage of the keyfile delivery service any time
during 1992 - 1993. For an additional $5.00 (US), we will prepare
your keyfile and upload it to your bbs. Just set up an account in the
name of ZDCS SUPPORT with the password KEYFILE and with sufficient
security to upload a private file for the sysop. When you send in
your registration of $25.00 plus $5.00, remember to include the name
and number of the bbs. It's all spelled out on the order form.
As of 1991, first time callers to The Hacker Central BBS have been
able to register ZDCS online with a valid Visa or Mastercard. Look
for the ZDCS script questionnaire in order to register. The keyfile
delivery service is also available as part of the online registration.
Processing usually takes from one to three days.
If you prefer to register by check or money order, please make your
registration check payable to Michael W. Cocke and mail it along with
the necessary information (like a completed order form) to:
Michael W. Cocke
11 Cedar Road
Montville NJ 07045-9582
- 44 -
Please be assured that ZDCS will continue to receive support through
future revisions. For over two years now it has been in use on its
home board, The Hacker Central BBS. When the programmer is also a
sysop and has to live with the results of that programming work every
day, you can be sure that the support will be there!
Support is a critical part of ZDCS. It's nice to have a program
that's easy to install and use, but we all know that there is nothing
to beat direct, involved author support of a product. We are
committed to providing that kind of support for ZDCS. Many of the
improvements in ZDCS 2.01 grew directly out of providing that kind of
support to users of earlier versions of ZDCS. This is also true of
the utilities and accessories, which you will find described a little
later on in the section called GOODIES.
Part of good support is also a willingness to listen to suggestions
and requests. If you have any ideas for new features or ways to make
existing features better, please let us hear from you.
ZDCS product support is handled on the ZDCS home board and via echoed
email conferences. Questions, comments and discussion of ZDCS are
welcome in the following conferences:
ZDCS-Support (on the ILink BBS network)
ZDCS-Support (on the Intelec BBS network)
Both conferences are of course available on the ZDCS home board, The
Hacker Central BBS. For PCBoard sysops on Salt Air BBS, Mike Cocke is
also available there in the ZDCS-Support conference.
Additional product support in the form of downloadable files is
available on The Hacker Central BBS. These are described in the next
section of this manual, appropriately entitled GOODIES.
First time access to The Hacker Central BBS must be done via the
public node. After completing the new user and visiting sysop
scripts, sysops and ZDCS users (whether or not they have registered
ZDCS yet) will be upgraded in security level, usually handled in a day
or two. This will give you access to the two private nodes, both of
which are high speed lines. The Hacker Central phone numbers are:
Node 1 201-334-2555 Public 2400
Node 2 201-316-8840 Private HST / Dual Standard
Node 3 201-335-9343 Private HST / Dual Standard
You can join the ZDCS support conferences on your first call. All the
pre-built CD-ROM databases, utilities, and other goodies are available
for download =only= from inside one of the ZDCS-Support conferences.
- 45 -
If you should get confused or need a reminder, there is a bulletin on
the board addressed to those interested in ZDCS. You'll find it
listed with the rest of the bulletins.
Another aspect of ZDCS product support is the collection of utilities
and pre-built CD-ROM databases available for download from The Hacker
Central BBS for all ZDCS users. This collection is constantly growing
in response to suggestions, requests and cries for customized help. A
few of the special goodies are described below.
This is not necessarily the complete list of goodies. New CD-ROM
databases are constantly being added, and development work on various
accessories is an ongoing process. Suggestions for other new
utilities are always welcome, of course.
All ZDCS utilities to date are free but are only available on The
Hacker Central BBS. Please do not upload these modules to other bbs's
or distribute them as part of the shareware ZDCS package.
The database merge program ZDCSDBMG.EXE lets you combine more than one
ZDCS database into a single database. This has the tremendous
advantage of permitting pre-built ZDCS databases to be created for the
more popular CD-ROMS. Once a single CD-ROM database has been created,
can be merged into any bbs's ZDCS database in a fraction of the time
it would take to create the CD-ROM database from scratch.
The existence of the merge utility also makes it possible for you to
build your own ZDCS database a section at a time if that's easier for
you, and then merge the different pieces together for your complete
system. It gives you a little more flexibility that way.
The merge utility is available in a little package called Z2-DBMG.ZIP
that includes step by step instructions so you can't get lost.
Pre-Built CD-ROM Databases
Many pre-built CD-ROM databases are available on The Hacker Central
BBS. These are easy to incorporate into your ZDCS database with the
database merge utility described above.
If you have a CD-ROM that doesn't already have a pre-built database
(either on disk or in the collection of databases on The Hacker
Central BBS), then we'd like to encourage you to create the ZDCS 2.01
database for that CD-ROM and upload it to share with other sysops. In
fact, we are offering a free ZDCS registration to anyone who uploads a
new ZDCS 2.01 pre-built CD-ROM database to The Hacker Central BBS from
now until the end of 1993. For details, check out the text file ZDCS-
CD.TXT in this package.
- 46 -
Local Upload Processing
A small local uploads package has been put together to make processing
local uploads easier for the sysop who also scavenges files for the
board. This package was revised for ZDCS 2.01 to make it easier for
the sysop to use it "out of the box". Changes to the docs and the
sample batch file take more of the load off the sysop when processing
local uploads. The package is available for download as LOCALUP.ZIP.
There is another small program that permits you to look through the
records in a ZDCS database. This database browse utility originally
started as a debugging tool and is still intended primarily for
debugging purposes. The program name is ZDCSED2.EXE and it is
available for download as ZDCSED2.ZIP.
Some upload file integrity testers move or rename the uploaded file.
There is now available a small utility called FIXPATH.EXE for use with
these testers. This program acts as a small shell for ZDCS, and
adjusts the %1 parameter originally passed from PCBoard to reflect
whatever the file integrity tester has done. This utility is
available for download as FIXPATH.ZIP.
BBS Ad Tracker
This is a new utility to help you track down bbs ads that might have
made it into your ZDCS database before they were recognized as bbs
ads. The way some bbs's change their ads frequently, it's possible to
receive one of these little joys as part of an upload or even to have
it in your original collection of files for some time before you
discover that it's another bbs ad. The bbs ad tracker ZDCSBAT.EXE can
compare your ZDCS database with your bbs ads database and let you know
if any of those bbs ads appear in teh database. Armed with that
report, you can hunt the ads down and exterminate them. This program
is available for download as ZDCSBAT.ZIP.
Conversion from ZDCS 1.6x to ZDCS 2.01
If you have already installed one of the earlier versions of ZDCS
(1.6x), there is a straightforward upgrade path to the smaller and
faster version 2.01. But you won't find a separate upgrade kit on The
Hacker Central BBS; all you need is already included in this package.
And since you are reading this file, you must have the rest of the
package, too. Right? However, you =will= need to make sure that any
utility programs you are using, such as the database merge, are up to
date for ZDCS 2.01 and are not older version intended for ZDCS 1.6x.
- 47 -
A final note about ZDCS support concerns documentation. We feel that
clear and helpful documentation is an important part of a sysop
utility package like ZDCS. The documentation for ZDCS 2.01 has been
revised to answer more questions and give more examples. Of course
we've kept the friendly walk-through ZDCSWALK.TXT and the technical
reference manual ZDCS-REF.TXT, just improved them a little.
In addition to the plain text files included in this package, we also
have the documentation available in WinWord 2.01 (also known as MS Word
for Windows 2.01) with full use of styles, footers, outline processing
and other features. This makes for some really nifty output if you
want to print copies of any of the docs. If you are interested in a
copy of the documentation files in Winword 2.01 format or in discussing
conversion to other formats such as WordPerfect, please contact us on
The Hacker Central BBS.
In fact, if you have any questions, comments or suggestions concerning
the documentation, we would like to hear from you.
ZDCS is a dynamic product. In response to suggestions, comments and
the changing bbs scene, ZDCS has seen a lot of improvements since
version 1.5 was first released to the public. And we haven't reached
the end of the road yet.
Future enhancements under consideration or in progress include support
for other archive formats besides PKZIP and ARJ, further improvements
to local upload handling, and database path updating for bbs types
besides PCBoard. (Support for PCBoard systems was added in version
Thanks to all the beta testers who have helped to make ZDCS a better
and smoother program, from version 1.5 through versions 1.6x and
culminating in version 2.01. Their patience and input through all the
steps along the way are definitely reflected in the quality of the
Thanks to Dave Klein for allowing me to bounce off him everything from
data reduction approaches to curses at Quarterdeck.
Thanks to Rich Hackney, Greg Wilson, Bob Jacobson, and Michael Leavitt
for their help in tracking down the QEMM / Supershell memory
- 48 -
Special thanks to Andy Keeves for ExzTest. Come to think of it,
special curses to Andy for the same thing.
Thanks to Roy Timberman for his help with some CD-ROM related errors.
Finally, thanks to everyone who had the patience to wait while version
2.01 went through six intense months of beta bashing and feature
COPYRIGHTS AND LEGAL STUFF
ZDCS (Zipfile Duplicate Checking System) is copyright (C) 1991, 1992
by Michael W. Cocke. ZDCS 2.01 is fully functional shareware. It may
be freely copied and distributed, provided that no files in this
package are removed or altered in any way.
The individual documentation files ZDCS-REF.TXT, ZDCS.BLT and
ZDCSWALK.TXT are copyright (C) 1991, 1992 by Evelyne Stalzer.
Permission is granted to distribute these files as part of the
complete ZDCS shareware package. Permission is also granted to bbs
sysops to modify ZDCS.BLT for use as needed, including as a bulletin,
message or downloadable help file.
ZDCS.KEY is the individual registration key file and may not be
copied, distributed or otherwise shared with individuals beyond the
Neither Michael W. Cocke, Evelyne Stalzer nor MWC Enterprises will
accept responsibility for the function, failure to function, or side
effects of any function of the Zipfile Duplicate Checking System
(ZDCS). ZDCS is provided in good faith, but its use is solely at the
risk of the operator.
ARJ is copyright Robert Jung.
EXZTEST and EXZIP are copyright Andy Keeves.
FWKCS is trademark and copyright Fred Kantor.
Lantastic is trademark of ArtiSoft Inc.
MS-DOS is trademark and copyright Microsoft Corp.
Netware is trademark of Novell Inc.
PCBoard is trademark and copyright Clark Development Company.
PKZIP and PKUNZIP are trademark and copyright PKWARE, Inc.
QEMM/386 is trademark and copyright Quarterdeck Office Systems.
Word for Windows is copyright Microsoft Corp.
ZDCS is copyright (C) 1991, 1992 Michael W. Cocke.
- 49 -