MATH COPROCESSOR STRONGLY RECOMMENDED...


This neural network in C++ was described in "Building a neural net in C++"
in the October, 1990 AI EXPERT magazine. It's non-optimized, but might be
a good introduction to neural nets for someone with a knowledge of C++ and
a good introduction to C++ for someone with some knowledge of neural nets.
I hope so anyway.

This program compiles under Turbo C++ v. 1.0, but should be fairly portable.
An especially interesting port would be to Zortech C++. Zortech's floating-
point optimization is _very_ good and floating-point calcs are the bottleneck
of this net. Another port that would be interesting would be to a _cfront_-
based translator and into a 386-specific compiler.

WHAT IT IS
This is a 3-layer, maximally-connected neural network that uses the sigmoid
function, backpropagation with momentum, and a stochastic update strategy.
The local error function uses Samad's coefficient -- if it is set to 0, the
net is performing "classic" backpropagation. If it is set to 1, the net is
performing "fast backpropagation". Actually, it's a great coefficient to
twiddle with.

USAGE
NEURAL data_file.
data_file must contain a valid file-name that contains data that's properly
formatted (two example data_files are provided -- alpha.neu and xor.neu).

After initialization, a clock is started (which only measures whole seconds!)
and the patterns are presented to the network. Connection weights are
updated after every pattern (the "stochastic update strategy" as opposed to
"batch update" which alters the weights after _all_ patterns have been
presented. In my opinion, stochastic is preferable).

If the macro DISPLAY_YES is DEFINEd (in neural.hpp), after every pattern is
processed the output is displayed on the screen below the desired output. If
the error of the output is greater than the acceptable error, the text
attribute is set to red. If if it less than acceptable error, the text
attribute is set to green. It makes for a very Christmas-tree-like
display.

I DON'T CHECK FOR MONITOR STATUS OR NUMBER OF SCREEN LINES.

If you run NEURAL.EXE with the data file ALPHA.NEU and you have less than 30
lines on screen, it will scroll and be hard to read. So, you can either try
to modify Network.displayDiff() or use your EGA or VGA card in 43/50 line
mode (if you don't have an EGA/VGA card, I can't help you).

THINGS I WISH I HAD THE TIME TO DO:
Make Neuron a virtual class with SigmoidNeuron and HyperbolicNeuron
descended from it (two different transfer functions). A very easy change
which would be very interesting -- does one really perform better?

Rethink the Network.calcMiddleError() function so it would work with
multiple hidden layers. I think all the rest of the program will function
correctly with multiple hidden layers.

Replace the Neuron.transfer() function with a look-up table. This would give
a tremendous speed-up.

Reimplement Neuron and Connection with scaled integers. Again, a tremendous
speed-up.

Clean the whole damn thing up....

Files in this archive:
NEURAL.EXE -- an executable version of the program
NEURAL.HPP -- Header file
NEURAL.CPP -- contains the main() function
NEURON.CPP -- implementation of Neuron class
CONNECTI.CPP -- implementation of Connection class
PATTERN.CPP -- implementation of Pattern class
NETWORK.CPP -- implementation of Network class
READ.ME -- this file
NEURAL.RPT -- Output from Set Laboratories Inc.'s PC-Metric C++ (a
very good program). For a discussion of the numbers, see Warren Keuffel's
"TOOLS OF THE TRADE" column in COMPUTER LANGUAGE, Oct. 1990.
NEURAL.PRJ -- The Turbo C++ v. 1.0 project file
ALPHA.DAT -- a data file for the first 10 letters of the alphabet
XOR.DAT -- a data file for the exclusive-or problem (works like a charm with
several hidden elements, with a 2-2-1 architecture it'll take thousands of
iterations).