AUTONET

Version 2.01













REFERENCE MANUAL


















Peak Software Corporation
666 Sherman Street, Suite 302
Denver, Colorado 80203
(303) 830-8614






Copyright (C) 1990, Peak Software Corporation






PREFACE


Autonet is a neural network software program that automatically
constructs neural networks from training data sets of input
variables and expected results. Autonet determines the most
appropriate network architecture for calculating the expected
result. The constructed network can then be used to calculate
new results from user entered input variables.

The development and production of Autonet has been a
collaboration between Professor Kenneth O. Cogger of the
University of Kansas at Lawrence and Peak Software Corporation.
Professor Cogger was responsible for the overall design of
Autonet and wrote the statistical algorithms used in the program.
Peak Software developed the user interfaces, error traps and
graphic output displays.

Professor Cogger deserves a special acknowledgement for his
research, contribution, inspiration and professional expertise in
the development and production of Autonet.

We would also like to thank each other, our family and friends
for encouragement, support and patience while this product was
under development.



Gary L. Boortz and James E. Krajcirik

Peak Software Corporation









AVAILABILITY


A demonstration version of Autonet is available for evaluation of
its usefulness for users needs. The demonstration version of
Autonet is restricted to use on the data files provided with
Autonet by Peak Software. This demonstration version of Autonet,
supporting files, and this reference manual are available on the
CompuServe network and directly from Peak Software. Other
electronic bulletin board systems may also have copies of the

demonstration version. See Ordering Information for further
details.

Copies of the demonstration version of Autonet, as well as this
reference manual and supporting files, may be uploaded to, and
downloaded from, electronic bulletin board systems for evaluation
by users provided that these items are not modified in any way.
In fact, modification of any supplied file may render the
software unusable.

Sharing the demonstration version of Autonet is encouraged.
However, commercial distributors of software should obtain
written permission from Peak Software before distributing the
demonstration version of Autonet. Permission is usually routine.

The production version of Autonet, which is not restricted to
data files supplied by Peak Software, may be purchased directly
from Peak Software. See Ordering Information for further
details.








ORDERING INFORMATION


The demonstration version of Autonet may be obtained directly
from Peak Software for $10.00 to cover shipping and handling.
This includes the software, supporting files and reference manual
on disk. The demonstration software, supporting files and
reference manual are also available on various electronic
bulletin board systems, including CompuServe.

The production version of Autonet, which is not restricted to use
on the data files provided by Peak Software, is available for
$395.00 plus appropriate taxes and shipping charges. Purchase of
the production version includes the software and supporting files
on disk, a printed and bound reference manual, user registration
for new product announcements and telephone support.

To purchase the demonstration software or production version of
Autonet, complete the enclosed order form, including check or
money-order, and mail to:

Peak Software Corporation
666 Sherman Street, Suite 302
Denver, Colorado 80203








AUTONET PRODUCTION VERSION

LICENSE AGREEMENT


Peak Software Corporation provides purchasers of the production
version of Autonet a license to use the software on user created
data files. Purchaser agrees that the software will be used on
no more than one computer at the same time and may be copied for
archival purposes only. Purchaser shall not disseminate the
software or materials provided in any form to any person,
including transmittal to electronic bulletin board systems or in
any other manner. Purchaser agrees that it hereby acquires no
right, title or interest in the software or associated materials
made available under this agreement.



DISCLAIMER OF WARRANTY

Use of Autonet and associated materials are provided by Peak
Software "as is" without warranty of any kind, either express or
implied, including without limitation the implied warranties of
merchantability, fitness for a particular purpose and
noninfringement of third-party proprietary rights. Purchaser
specifically acknowledges and agrees that no warranty is made
with respect to the correctness and/or accuracy of any licensed
materials, Autonet or third-party software. Purchaser further
agrees that, in no event, shall Peak Software be liable for any
direct, indirect, incidental, special, consequential or other
damages or injuries, or any claims arising out of the use or
inability to use Autonet or licensed materials.
Peak Software makes no warranty of any kind and shall not be
liable for any damages arising from the failure of Autonet to
perform in the manner desired by the user.








INSTALLATION


Before installing Autonet on your computer system, make backup
copies of all disks provided by Peak Software.

Floppy Disk Users:

Insert the Autonet program disk into any drive. Make sure that
drive is active and following the system prompt, type AUTONET.
The main menu of Autonet will appear.

Hard Disk Users:

Copy all files (*.*) from the Autonet disk to your hard disk
drive. For ease of use, it is recommended that a subdirectory be
created to contain all files provided with Autonet. To execute
the program, simply type AUTONET at the system prompt. The main
menu of Autonet will appear.

Autonet Files:

Numerous data files are provided with Autonet. All data files
have a .DAT extension. The demonstration version of Autonet is
restricted to use of only those data files provided by Peak
Software. These files are named AIR.DAT, AUDIO.DAT, CHILL.DAT,
HRBL.DAT, KIOWA.DAT, RABIES.DAT, SP500WL.DAT, TANK.DAT AND
XOR.DAT. The nature of each of these data files is described in
the SAMPLE NETWORKS Section of this manual.

Constructed networks may be saved on disk for future use.
These files contain descriptive information about networks
constructed by Autonet. Network calculations can be made using
saved files rather than having to reconstruct a network from
scratch. The use of saved network files is discussed in the
AUTONET PROCEDURES AND MENUS Section of this manual.

The Autonet program itself is found in one of two possible
configurations, either the production or demonstration versions.
Both are found in a file titled AUTONET.EXE. The demonstration
version will accept only the data found in the files indicated
above. The production version will accept any data file
constructed by the user according to the format described in
Appendix C. Other than this data restriction, there are no
differences between the program versions.







INTRODUCTION


Neural networks are computer models of the human brain.
Typically, such networks consist of simple processing elements
connected to each other to mimic neurons, the basic processing
elements of the brain. Inputs and outputs of the processing
elements are normally described in terms of simple mathematical
functions.

A neural network often defines the input to each processing
element, or node, as a weighted sum of the outputs from all other
nodes to which it is connected. The output from a node is
normally defined as some simple function of its input. As an
example, output (O) and input (I) might be related by the
"sigmoid" or "logistic" function O = 1 / (1 + exp(-I)). This is
an S-shaped curve with output values ranging from zero to one.
Many other input and output functions have been studied for
neural networks which is and are of active research in computer
science.

Neural networks have been constructed for a variety of purposes,
including economic modeling and forecasting, marketing research,
pattern and character recognition, language processing, rule
development in expert systems, and many other areas. The SAMPLE
NETWORKS Section of this manual discusses some of these
applications.

In building a neural network that is good at solving a particular
problem type, much care is needed. The builder of a neural
network must make many choices. For example, the input and
output functions must be decided upon. The weights attached to
the connections between nodes must be specified, either a priori
or through some kind of "learning" based on training the network
with examples, much as a child learns language and other skills.
Various neural network learning schemes have been proposed and,
again, the builder of a neural network must make a choice as to
the learning paradigm most suitable for the problem at hand.
Finally, the basic architecture of the network must be
established. For example, are the nodes to be arranged in layers
and, if so, how many nodes should there be? Are all nodes to be
connected to all other nodes or is a simpler connection pattern
sufficient?







Autonet simplifies the construction of certain kinds of neural
networks by using training examples of solutions to problems to
"learn" the most appropriate architecture for solving those
problems as well as the weights connecting each node in the
network. On many problems, Autonet has produced neural networks
which perform as well as, and sometimes better than, networks
constructed with a great deal of trial and error by highly
trained experts in computer science.

The examples found in the SAMPLE NETWORKS section illustrate the
capability of Autonet to generate neural networks capable of
solving problems in business forecasting, chemical engineering,
logic, stock market analysis, marketing research, and medical
diagnosis. Further applications are limited only by the
imagination of the user.







THE AUTONET ALGORITHM


Here, a brief description is given of the algorithm used by
Autonet to generate a neural network.

Autonet is based on close to 30 years of research into adaptive
modeling procedures. One of the earliest researchers in this
area was A. G. Ivakhnenko, a Ukrainian cyberneticist who examined
what he called a Group Method of Data Handling, or GMDH. Autonet
is based on some of these ideas and incorporates many
improvements which allow construction of networks for problems
with a wider variety of data types.

The basic approach in Autonet is that each node in the network
receives input from exactly two other nodes, with the exception
of the nodes representing the input to the network. The two
inputs, say U and V, produce output using a complete quadratic
transfer function: O = A + B*U + C*V + D*U*U +E*V*V + F*U*V.
The numerical values of the coefficients A,B, etc. may be thought
of as representing the strength of various neural connections and
are analogous to the weights found in other types of neural
networks.

Autonet determines the values of these coefficients statistically
by comparing the observed output with the desired output based on
training examples presented to the network. Autonet "learns" the
correct weights by example.

Autonet also learns how many nodes are appropriate for a
particular problem and how they should be connected. Trial
networks are constructed one layer at a time. The input
variables (nodes) represent the first layer. Each of these are
combined, two at a time using the previously mentioned quadratic
transfer function, to create the second layer of the network.
From the second layer of nodes, a third is created, and so on.

Implicitly, the second layer nodes are quadratic functions of the
input variables, the third layer involves fourth degree
polynomials, the fourth reaches eighth degree, etc. In this
fashion, extremely complicated output patterns can eventually be
recognized by the network even with only a few layers. Yet each
node is a quadratic function, arguably the simplest mathematical
function with the ability to combine to form more complicated
functions.







The number of nodes generated by this approach clearly will be
unmanageable as the number of layers increases. Four input
variables combine to form six nodes in the second layer, which in
turn form 15 nodes in the third layer, then 105 in the fourth
layer, etc. This combinatorial increase is controlled by Autonet
by limiting the number of nodes in each layer and limiting the
number of layers by stopping once the performance of the network
begins to deteriorate. The stopping rule is based on a squared
error criterion, and data used to estimate coefficients is kept
separate from that used in the stopping rule. Networks developed
with Autonet are therefore validated on data which has not been
used to estimate the various coefficients of the quadratic
functions.

Under default settings, Autonet will continue to add layers of
nodes as long as the best node in each new layer represents an
improvement over the best node in the previous layer.
Occasionally, Autonet will exhibit lengthy execution times and
will construct layers of nodes which represent negligible
improvement over previous layers. If this happens, the Set
Options menu should be selected and the default settings for
minimum improvement in error and/or minimum percentage
improvement in error should be changed from zero to some other
small value.

Autonet networks may be viewed from an evolutionary perspective.
Simple organisms (input nodes) combine to form more complicated
organisms (second layer nodes). Successful organisms become more
complicated, and better adapted to the problem at hand, until
they become overspecialized and die out. The most successful
organism (node) is the output of Autonet and the neural network
is defined by this survivor's preceding nodes.

The Autonet approach to neural network modeling requires the user
to provide data describing the problem to be solved. This
includes values of the input variables and the corresponding
desired value of the output variable. The user must specify the
number of examples to include in the "training" set. The
remaining examples will be assigned to the "validation" set. We
recommend that at least 25% of the examples be assigned to the
validation set. Execution time and network architecture can be
affected by this choice.








AUTONET PROCEDURES AND MENUS


Autonet is menu driven and most menu selections are self
explanatory. Below is an outline of the menu tree for Autonet,
and a brief explanation of each possible menu selection.

Data Manipulation

Read Data File
- Prompts user for an input data file. Files
specified by the user are displayed in a window.
The arrow keys may be used to highlight the file
desired. If more than ten files meet the criteria,
additional files may be viewed by paging up or down.

Display Data File
- Displays the rows and columns of the input data file.
Arrow keys may be used to scroll up and down or left
and right through the file to view specific rows and
columns.

Network Construction

Create Network
- Constructs network for the input data file.
Statistical data on network construction is displayed
while the network is constructed.

Save Network
- Saves certain characteristics of a constructed
network. This option allows subsequent analysis
of a network without reconstructing it.

Output Comparison
- Text display of actual versus computed output.

Set Options
- Options for printer output, file output, training set
size, minimum error, percentage improvement in
error and maximum number of layers.

Show Information
- Shows current program limitations on sample size
and number of input variables.







Network Analysis

Read Saved Network
- Recalls a previously constructed and saved network.
Files meeting the users file name criteria are
displayed in a window. The arrow keys and page up or
down may be used as in the Read Data File menu
option. Reading a saved network allows subsequent
analysis of a network without reconstructing it.
To view the output comparison, the saved network
should be recalled first and then the data file
should be read into memory. The number of
observations selected when reading the data file
must be the same as what was selected when the
network was originally created to maintain
consistency. The training set size will default
to the correct value.

Transfer Functions
- Displays network connection architecture and
coefficients for each quadratic connection.

Graph Network
- Graphic display of constructed network.

Graph Output
- Graphic plot of actual versus computed output.

Output Calculation
- Calculates the network's output value for
specified values of the input variables. The output
value is recalculated after each input variable is
entered. This option allows the user to easily
examine output values for various input variable
values.







SAMPLE NETWORKS


The capabilities of Autonet can best be appreciated by examining
its performance on a variety of problems from many potential
areas of application. Autonet is distributed with numerous
preconstructed data files which are described below. Each file
follows the same format, with the desired network output being
the first variable in each row, with the remaining row values
representing the input variables.

AIR.DAT

A central problem in business is forecasting and planning. This
data set illustrates how a neural network can be constructed in
this area. The data consists of monthly international airline
travel measured in thousands of passenger miles. We chose this
data set because it has been analyzed by statistical time series
experts using a variety of techniques. The desired network
output is the amount of travel in a particular month. The input
variables are the past 24 monthly figures. The rationale for
using the past 24 months is that current travel is probably
similar to travel one year ago and maybe two years ago (12 and 24
months, respectively). Users examining the Autonet network for
this problem should examine whether, in fact, Autonet is able to
detect such seasonal patterns in airline travel.

AUDIO.DAT

An important problem in marketing is determining potential
customer characteristics. This can be useful, for example, in
deciding where to spend promotional effort. This data file is
based on a study by Dash, et.al.(1977). There, buyers of audio
equipment were classified into two groups, those who bought at a
full-line department store and those buying at a specialty audio
store. Also, for each buyer, four additional pieces of
information were sought:

1 - whether they sought information from friends and
neighbors before purchasing,
2 - whether they personally had been asked for an opinion
about buying an audio product,
3 - whether they had shopped around before making a
decision, and
4 - whether they had sought information from manufacturers
catalogs before making the purchase.







This data file contains, as the desired network output, the
probability that a given buyer will purchase the audio equipment
at a full-line department store. The inputs are the four binary
variables representing their personal characteristics.
Obviously, in a real marketing situation, many more buyer
characteristics might be recorded, but the present example
suffices to demonstrate the capability of neural networks in this
area. The file contains the 16 possible joint consumer
characteristics replicated 6 times for a total of 96 examples to
construct the network.

CHILL.DAT

This file contains examples of the relationship between
temperature, wind speed, and the so-called wind chill factor.
88 examples are given based on standard tables. The relationship
is highly nonlinear and most tables suggest an asymptotic kind of
relation with statements to the effect that "wind speeds greater
than 40 mph have little additional effect on wind chill."
This application of Autonet illustrates the ability of neural
networks to recognize patterns and other mathematical
relationships. Moreover, it also illustrates how neural networks
can handle input data values that they have not been trained to
recognize. In contrast, expert systems cannot handle data which
does not fit their rule premises. Neural networks often can
handle such data with minimal degradation in performance. As an
example, users should examine the network output for a wind speed
of 20 mph (input one) and a temperature of -15 F(input two).
This combination is not contained in the supplied data. The
closest combinations used in building the network are 20 mph and
-10 F for a wind chill of -53, and 20 mph and -20 F for a wind
chill of -67. Users can verify that Autonet makes a reasonable
interpolation between these figures even though this particular
example is completely foreign to Autonet.

HRBL.DAT

This file contains weekly closing prices on H&R Block common
stock for the period July, 1988 to July, 1990. The inputs are
the past ten weekly closing prices and the desired output is the
next week's closing price.

KIOWA.DAT

This file contains monthly revenue figures for Kiowa, Hardiner,
and Pacific Railroad in the state of Missouri. The structure of
this file is similar to that of AIR.DAT. The inputs are the last
10 monthly revenue figures and the desired output in next month's
revenues.







RABIES.DAT

This data set is from the field of medicine. The Center for
Disease Control has issued guidelines for the treatment of
injuries that may lead to the contraction of rabies. Depending
on the nature of the injury, the presence of rabies in the
geographic area of the victim, etc., the treatment may consist of
simply carefully washing the injured area or it may be wise to
administer vaccine as well as serum. This data file consists of
a binary output variable (0=wash, 1=vaccine+serum treatment) and
five binary inputs. These input variables describe whether the
victim was:

1 - either bitten or licked on an open wound(1=yes),
2 - whether rabies is present in the area(1=yes),
3 - whether the animal was captured(1=yes),
4 - whether the animal was vaccinated(1=yes), and
5 - whether the autopsied brain of the animal was
normal(1=yes).

The data file consists of the 32 possible combinations of the
five binary variables replicated 5 times for a total of 160
examples of CDC guidelines. This example is of interest for two
reasons. First, it represents an example from medicine. Second,
it illustrates the capability of neural networks to "learn" the
rules followed by experts in a problem area. Several
researchers have commented that the central problem with rule-
based expert systems is establishing the rules. This example
illustrates the capability of neural networks to find out what
the rules are. Examine the Autonet network for rabies treatment
to see how quickly and accurately the network finds the implicit
rules.

SP500WL.DAT

This data file contains weekly closing values for the Standard
and Poor 500 Composite Index for the period January 1, 1985 to
July 17, 1990. The desired output of the network is the closing
price and the inputs are the ten previous weekly closing prices.
In this application, Autonet develops a neural network for stock
market prediction. Much has been written about the possibility
or impossibility of predicting stock market prices. Many
researchers have concluded that the best prediction of tomorrow's
price is today's price based on extensive studies going back as
early as 1900. Users should find the network constructed by
Autonet to be quite interesting in that it arrives at much the
same conclusion. Notice, however, that Autonet comes to this
conclusion based on a particular kind of learning, and this
characteristic of neural networks is exciting people today.







TANK.DAT

This data represents another example of a table generated by
empirical measurement like the wind chill table. Here the
desired output is the coefficient of expansion for a petroleum
product. The inputs are the difference between the temperature
in a storage tank and the ambient temperature, and the specific
gravity of the petroleum product itself.

XOR.DAT

This file contains data appropriate for constructing a neural
network that solves a problem in logic. The problem is the
"exclusive or" problem. Basically, the problem is to determine
whether one, but not both, of two logical conditions are met.
This file represents a condition as being met with an input value
of one. The inputs and desired outputs are given by:

Input 1 Input 2 Desired Output
0 0 0
1 0 1
0 1 1
1 1 0

This problem appears to be trivial, at least for human reasoning.
If presented with repeated examples of the above four examples,
most people would quickly figure out the relationship between the
inputs and the output. It turns out, however, that early
attempts at neural networks could not solve this problem. It was
not until the early 1980's that neural networks were constructed
to solve this problem. Unfortunately, even today, typical neural
networks require hundreds and even thousands of presentations of
the above four cases to "learn" the solution to the problem.
Moreover, the solutions are usually only approximate. XOR.DAT
contains three repetitions of the above table, for a total of 12
examples. Users will be interested in the network constructed
using 8 cases in the training set and 4 cases in the validation
set. Autonet solves the problem with a single layer in the
network! Moreover, the network exhibits no error! To the best
of our knowledge, no other neural network approach has
accomplished this result.

(Mathematicians will understand that the quadratic functions
employed by Autonet are the reason for this performance on this
particular problem.)







APPENDIX A - TECHNICAL SPECIFICATIONS


Autonet 2.0 is designed for use on IBM Personal Computers or
compatible computers with a minimum of 640Kb of RAM. Autonet
will detect and use a mathematics coprocessor if one is present.
Processing time is significantly faster when constructing a
network if a math coprocessor is used. Most graphics cards are
supported, although some graphical output may be distorted with
complicated networks displayed on computers with only CGA
capability. Therefore, an EGA or VGA card is recommended.

Autonet 2.0 is currently limited to data files with 150 or fewer
observations and a maximum of 24 input variables. Larger data
files are truncated according to the user's directions.

Larger problems may effectively use a mainframe computer.
Contact Peak Software concerning availability of a mainframe
version.







APPENDIX B - DATA FILE FORMAT


Data files are standard text (ASCII) files and may be created
with most editors. Data is arranged in rows and columns, with
each row containing the values of the inputs and the desired
value of the network output. The ordering of the data does not
matter; users are prompted for which column represents the
output. Autonet data files provided by Peak Software have the
output variable in the first column.


NOTE: Data within rows must be separated with a space, and each
row must end with a hard carriage return. Blank rows are not
permitted; a row consisting of just a carriage return will cause
errors in reading data.







REFERENCES AND SUGGESTED READING


AI EXPERT. A monthly publication devoted primarily to expert
systems, with occasional articles relating to neural networks.

Dash, J.F.,Schiffman, L., and Berenson,C., "Information Search
and Store Choice," Journal of Advertising Research, 16, 1976.

Ivakhnenko, A.G., "Polynomial Theory of Complex Systems," IEEE
Transactions on Systems, Man, and Cybernetics, May-June, 1967.

PC AI. A monthly publication devoted to applications of
artificial intelligence on personal computers.

Self Organizing Methods in Modeling. Edited by S.J. Farlow.
Marcel-Dekker, Inc. New York, 1984.








PURCHASE ORDER


TO: PEAK SOFTWARE CORPORATION
666 SHERMAN STREET, SUITE 302
DENVER, COLORADO 80203


SHIP TO: _____________________________
_____________________________
_____________________________
___________________ __ _____

PHONE: _____________________


Please send:

______ Autonet Demonstration Software @ $10.00 $_________

(In Colorado add 7.2% Sales Tax) $_________

Total $_________




______ Autonet Production Software @ $395.00 $_________

(In Colorado add 7.2% Sales Tax) $_________

U. S. $5.00
Canada $10.00 Shipping $_________
Other $20.00

Total $_________


Disk size __ 5.25" __ 3.5"

Make check or money-order payable to Peak Software.
Checks must be drawn on a U.S. bank in U.S. dollars.



Authorized Signature ______________________________

______________________________
(Print Name)

The above individual hereby agrees to the License Agreement and
all terms and conditions associated with the use of Autonet.