Category : Science and Education
Archive   : FREQ.ZIP
Filename : FRQ.TXT

FREQ for Windows ...Beyond Fourier Transforms...

(c) 1995 CoDebris All rights reserved March 10, 1995
711 Barbara Avenue Solana Beach, CA 92075 (619) 755-4492

This is the shareware (unregistered) version 2.1 of FREQ. FREQ is
a data analysis tool which determines what sine waves make up a
data set or time series: periods, amplitudes, phases, percent
relative power). You specify which periods you believe are present
(or specify a whole range of periods), and FREQ tests those periods
and graphically assembles selected sine waves into a revealing
portrait of your data: a visualization of your data you can't get
from a Fast Fourier Transform without a whole lot of work and
specialized training.

The registered version of FREQ (ver 2.2) performs a host of related
analysis and data synthesis functions: a "waveform spreadsheet".
Registration is $40, cash or check, payable to CoDebris at the above
address. On registration we will either mail you a 3.5" floppy disk
or send you e-mail with an attached FREQ.ZIP file containing the
upgrade, and more tutorial data files with associated period tables.


Notice...

We do not modify ANY of your system files, and de-installing FREQ
is accomplished by simply deleting the FREQ Program Group and the
files, which are usually found in C:\FREQ.


Overview...

Researchers and analysts often want to know if experimental data
(a time series) contains significant amounts of signal at
frequencies of interest to them. These frequencies may correspond
to driving forces, environmental constraints (boundary conditions),
or system responses, and may result from intrinsically non-linear
processes. Often, mathematical or phenomenological models exist to
explain some observed behavior, and experimental data is collected
to verify whether or not the model is correct.

FREQ is a new, fundamentally unique tool for performing these
analyses. FREQ not only identifies the period, phase and amplitude
of sine waves which represent your data, but then plots each new
wave component over a graph of your data. A plot of the original
data and reconstruction can be printed, and the reconstruction can
even be saved as a data set, on the same scale as the original data.



Installation and Operation...

If you obtained FREQ from a download, run PKUNZIP FREQ.ZIP and then
run SETUP to install FREQ in a Program Group under Windows 3.1.

If you have FREQ on a 3 1/2" floppy disc, install FREQ by running
SETUP from the floppy drive.

During setup, you are prompted for a directory in which to install
FREQ, the default is C:\FREQ.

Extensive on-line help is accessible from the menu Help/Contents
option.

On first running FREQ, choose Analyze/Frequency Search from the
menu and select a (supplied) table of candidate periods (use
REAL.TBL the first time) and a supplied data file (REAL.DAT).
The .TBL files can be edited and saved using FREQ's built-in
editor.

REAL.DAT is a physiological data set. The synthesized curve is
not a bad approximation to the data, as you can see while the
reconstruction builds before your eyes, even though only 55% of
the "power" is accounted for by the seven identified periods,
amplitudes and phases, analyzed in under 1 minute. Refining the
REAL.TBL file might find even better (closer) periods to use,
but the periods supplied are pretty good, or they wouldn't be used
in a demo.

The important thing is watching the highest-power, long period
sines define major trends in the data and seeing the short-period
sines fill in the peaks and valleys. On seeing this for the first
time, you can achieve a very intuitive sense for what the data
says, and of Fourier series in general.

As you feed FREQ other data sets of interest to you, or generated
from the rich variety of Synthesis options in FREQ, your
understanding of what your data is telling you grows immediately,
in a very visual way.

Arrange your data for input to FREQ by creating a file consisting
of two columns (X, Y) and as many rows as there are points in the
data set. Alternatively, you can just feed FREQ a single column
of Y values only, and tell FREQ the initial X value and the
interval between successive values.

FREQ will line-plot the actual data in green, and the synthesized
data (the "reconstruction") from the selected periods as a yellow
dotted line. Plots are auto-rescaled on window resizing, actually
recomputed every time the window is painted to optimize the
displayed data information content.

Each time a new period is selected as containing the next most
significant amount of power, the yellow curve is redrawn to
include the new information. You can get a good sense for periods
to include in the .TBL file, and which to exclude, by watching the
analysis and then examining the .OUT file that pops up on the
screen after each analysis is complete.

In fact, watching the earlier, higher-power sinusoids define the
major features of your data is an insightful experience; one
researcher termed it a "spectacular visualization" of his data.
The side slopes of the sine curve will lie along major trends in
the data, and the peaks won't necessarily correspond to peaks in
the data. That is left to shorter period components, which when
properly phased with the long period components will ride up and
down peaks in the data with increasing fidelity.

You will definitely learn to think nearly simultaneously in
so-called "dual spaces": time and frequency. You will learn to
think carefully about what are the proper "units", or "dimensions"
of your data. The supplied PERIOD.TBL is designed to encompass a
wide range of possible values, but it takes longer to analyze a
data set. After running it on your data, examine the associated
.OUT file and create a "tailored" table of periods to run with
that particular sort of data. Use it to test assumptions about
what is really in your data.



Background...

The usual frequency analysis approach is to pre-process your data,
apply a Fast Fourier Transform to the series, and plot the power
spectrum. Peaks in the FFT spectrum may correspond to interesting
frequencies. However, it is difficult for anyone but a signal
processing expert to know how much power in the time series is
actually accounted for by a given frequency. It is even harder to
resolve nearby, overlapping broad peaks. More often than not,
"noise" dominates the data and cannot easily be de-coupled from
signals of interest.

Pre-processing (filtering and windowing) of data sets is a very
demanding discipline, and many of the rules to assure the validity
of pre-processing operations are difficult to apply, as such
operations actually modify the characteristics of the manipulated
data.

Further, most researchers with a need for waveform analyses do not
have formal training in the subject, and many feel uncomfortable
with having to use a host of implicit assumptions.

There is an alternative, nearly painless method available to
perform such frequency analyses. The researcher first prepares
(as a text file) a table of candidate periods. On initial
creation, the table usually contains a fairly large number of
entries, as there may be no prior knowledge of what is really
present in the data. Periods may be longer than the time series,
or as short as twice the time interval between points. But the
researcher often knows what to look for based on theory or
existing work, and the table will contain several periods in the
regions of interest.

FREQ is not just a curve-fitter, nor is it simply an FFT. To use
it, you should know something about your data, but you need nearly
no data analysis background. FREQ searches your data using the
supplied candidate periods in a .TBL file which you prepare,
selecting those periods which account for the most "power".

FREQ uses an adaptation, called Fast Orthogonal Search (FOS), of
the Orthogonal Search Method developed by Michael J. Korenberg and
his group at Queens University in the late 1980's. The algorithm
is applied to your data set, using an associated table of candidate
periods. The precise power, amplitude, and phase of sine waves
corresponding to entries in the table is displayed. The objective
is to determine if frequencies of interest to the researcher are
present in significant measure, and report the results.

The algorithm analyzes a time series stepwise, determining the
ability of each period to explain a significant portion of the
total variance (mean square error, or MSE: roughly, the data
set's "power"). It then orthogonally removes the sinusoid
explaining the largest percentage of the time series variance.
This process is repeated on the residuals until there is no further
significant error reduction or until a specified number of periods
have been identified.

The algorithm is capable of much greater time resolution than a
Fourier transform, and is not limited to harmonics of a fundamental
frequency. It is also quite insensitive to noise, as all data
elements are used only in series-wide averages over the orthogonal
basis functions. Finally, it tolerates missing data points,
irregularly-spaced data sets, and short data segments. In many
nonlinear or biological systems, the signal frequencies move, or
breathe, as the system evolves, so short segments are necessary
for system identification.

FREQ is a Multiple Document Interface application, so it will
display the results from multiple Frequency Searches on-screen at
the same time, including the reconstructions (dotted yellow plots)
and text output files. A rudimentary FFT is included, mostly for
contrast.



Registration and Upgrades...

If you run FREQ and like it, let us know (register it). If you
run it and don't like something about it, let us know that too.
FREQ is becoming a fairly comprehensive waveform analysis and
synthesis package, and most features are added in response to
user suggestions: in the works are digital filters, multi-variate
data sets and non-linear dynamics tools.

In addition to everything in the shareware version, the registered
version of FREQ (ver. 2.2) contains these additional functions:

Synthesize: Pre-Process Data Set...
Interpolate
Segment
Smooth
Remove Trend
Remove Artifacts
Subtract Mean
Scale X or Y Data
Offset X or Y Data
RMS Partition
Copy Current Data Set
Two-Graph Operations...
Concatenate (Join) Data Sets
Add Data Sets Pointwise
Subtract Data Sets
Multiply Data Sets
Divide (Ratio) Data Sets
Reconstruct Data Set From Search Sines

Analyze: Fast Fourier Transform
Inverse FFT
Approximate Entropy


The next upgrade to FREQ (ver. 2.4) will add these features:

Multi-variate data sets (e.g. simultaneous heart rate,
EEG, cardiovascular output vs. time)

Log/linear, linear/log, log/log graphs

Windowing: Hamming, Hann

Filters: Bessel, Butterworth, Chebychev, Elliptical
Ideal Low/High/Band Pass
Notch, Comb and Band Stop

Convolution, Correlation analyses

Non-linear analyses: Poincare plot, Second Return,
Min Info, Lyapunov dimension

Full-resolution (not scaled bitmap) printed graphs,
color optional, with settable titles, legends, and
captioning.

Registrations received BEFORE July 1, 1995 can receive
this version 2.4 upgrade deep-discounted during August.

In July, the registration price will become $60.


We are very interested in enlarging the variety of data set formats
FREQ can recognize: let us know if you want FREQ to read your
spreadsheet or data base files or accept data from acquisition
hardware: sound boards, DSP boards, A/D or digital I/O boards.



Gene Zawadzki
CoDebris
(619) 755-4492
CompuServe 72074,772