FILE FORMAT UTILITIES v1.0 -
Seven programs which prepare files for "word
wrap" progams. FixEOLs converts lines to end
in CR/LF. Untab expands TAB characters to
spaces. Two utilities replace or remove text.
The others insert blank lines for different
reasons to keep the text from wrapping to
another line. Useful to double space
documents. Converts UNIX or MAC docs to DOS
end-of-line format. Registration $9.95

File Format Utilities version 1.0

June 25, 1994

Copyright (C) 1993, 1994 by Thomas M. Roddy

All Rights Reserved.


Introduction:

The purpose of the File Format Utilities is to aid in preparing documents
so that they are in a format that other text processing programs can
systematically work with. Programs that perform functions such as "word wrap"
will be able to reformat documents with minimal problems if the document
follows a few rules. The following utilities help you modify document files
to adhere to these rules. The utilities that make up the package are
listed below with very brief descriptions:

FixEOLs - Converts all lines to end in an ASCII standard CR/LF character pair
Untab - Expands TAB characters to any number of spaces
Strip - Removes given characters from a file
Parafy - Insert blank line if a line begins past acceptable margin
FixShort -Insert blank line after short lines
FndStart -Insert blank line after lines which begin with a given string
FndRep - Finds a given string and removes or replaces it with another

License and distribution information:

The File Format Utilities are bundled together as one package. It is
shareware. It is not public domain or free software. An order form is
enclosed in the file named ORDER.FRM. After trying it out for 14 days,
if you like it and want to continue using it, please register by sending
the registration fee (located in ORDER.FRM) with your name and address
along with any comments or questions to the address listed.

You may NOT alter the code nor may you sell this program or any of its
associated data files for any price. You may charge a reasonable copy fee
of not more than US$10 when distributing it to others as long as you clearly
stipulate that the File Format Utilities must be registered with its author
if it is used beyond the trial period. Every copy of the ZIP distribution
file must contain all of the files listed below.

You may (and are encouraged to) distribute the shareware version of the File
Format Utilities to others. Under no circumstances may you distribute a
registered version of the File Format Utilities to others.


Disclaimer:

The author makes no claims or guarantees about the use of the File Format
Utilities and will not be responsible for any loss, including data or
profits, due its use or misuse.


Compatibility:

The File Format Utilities should work with all standard ASCII files. It has
not been tested with files containing control characters commonly found in
documents in specific word processing formats such as Word Perfect or
Microsoft Word. Most word processors allow you to convert documents to ASCII
format. You should do this before using the File Format Utilities on the
documents.


Files in distribution:

UNTAB.EXE - Untab executable program
STRIP.EXE - Strip executable program
FIXSHORT.EXE - FixShort executable program
PARAFY.EXE - Parafy executable program
FIXEOLS.EXE - FixEOLs executable program
FNDSTART.EXE - FndStart executable program
FNDREP.EXE - FndRep executable program
FFU.DOC - This file - Documentation for the File Format Utilities
FIXFMT.BAT - Example batch file to "fix the format" of files using piping
ORDER.FRM - Printable order form for registering the File Format Utilities
FILE_ID.DIZ - Short description of the File Format Utilities


Product description:

The utilities were originally created as supporting programs for another
shareware program by the same author called Rap. Rap is a "word wrap"
utility program which is used to reformat a text file into specified line
lengths without splitting up words across lines. See below for more
information about Rap. Although this was the original reason that I wrote
these utilities, you will probably find them very useful for other purposes.

Many word wrap programs look for blank lines between paragraphs in order
to know when to stop wrapping the text lines. Otherwise, the entire
document would be converted to one huge run-on paragraph.

Three of the utilities, Parafy, FixShort and FndStart, work by inserting
blank lines into the file in order to signal to the word wrap program that
the line should not be wrapped or filled. The tradeoff, of course, is that
these utilities may have an undesirable side effect by inserting blank lines
which may not be wanted in the document.

The goal of these utilities is not to fix 100% of the formatting problems,
but to greatly reduce them.

The shareware version of the File Format Utilities is restricted in one way.
To encourage registration, each of the programs have a limitation on
the size of file it will process. See each program's description for the
limitation on it. As you will see, the limits are quite high and should
be more than adequate to test the usefulness of the programs.

Please register if you find the File Format Utilities useful. You will be
mailed the latest version of the programs without the shareware notice and
with the restrictions removed.



*** FixEOLS ***

Usage: FixEOLs infile [-Lx] [-P] [outfile]
where x is the number of End of line markers (EOLs) to replace
each EOL found: minimum=0, maximum=10, default=1
-L0 removes all EOLs
-L1 leaves the same number of EOLs (but will fix incorrect EOLs)
-L2 doubles each EOL found
-P indicates that piping of input and/or output is being performed

FixEOLs is a program which converts end-of-line (EOL) markers to contain
both a carriage return (CR) character and a line feed (LF) character as
is standard in the DOS world.

FixEOLs replaces a lone CR or LF with a CR/LF pair which is the correct
ASCII way to terminate a line. Unix and Macintosh standards have just one of
the characters to terminate the line. If a CR/LF is found, it is left alone.
You can optionally have between 0 and 10 EOLs (CR/LF) put in the output file
in place of each EOL in the input file. This may be helpful in reformatting a
document which is already formatted to one line per paragraph (like .WRI
files) to put a blank line between each line.

By using the -L2 option you will insert two line feeds for every one that is
found. This is an effective way to double space a document. This slightly
differs from the method that Parafy uses in that Parafy will not double blank
lines, but FixEOLs will.

PRODIGY users may find this program useful since the PRODIGY service, which
allows printing of text, will send the text with printer commands rather
than just sending text with normal DOS EOLs. Use FixEOLs to make the text
readable without having to print it out. You still may have some printer
commands in the file after fixing the EOLs, but at least it is readable.
Use the STRIP utility to remove control characters.

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (EOL).

A shareware limitation limits processing to 200,000 characters per file.
Register for the unrestricted version.


*** Untab ***

Usage: UNTAB infile [outfile] [-Tx] [-P]
where x is the desired tab length in characters:
minimum=0, maximum=200, default=8
-P indicates that piping of input and/or output is being performed

Untab expands all occurences of TAB characters to a given number of space
characters so that the number of space characters in the file is equal
to what you would see in an editor or on paper. If the file contains
TAB characters, it usually throws off a program like Rap because it
is unknown how many spaces that you would like a TAB character to
represent.

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (UNT).

A shareware limitation limits processing to 200,000 characters per file.
Register for the unrestricted version.


*** Strip ***

Usage: STRIP infile [outfile] [-Bx] [-Ey] [-Sz] [-P]
where x is the decimal value of the begining character in the range
of characters to be deleted. Default is 0.
y is the decimal value of the ending character in the range
of characters to be deleted. Default is 31.
z is the comma separated list of decimal values for the
characters that fall within the above range, but should be
skipped (left in the file). Default is 10 (LF or line feed),
12 (FF or form feed), and 13 (CR or carriage return).

-P indicates that piping of input and/or output is being performed.

STRIP removes all characters within a range of the ASCII code set.

By default, All of the control characters except for line feeds and form
feeds are removed.

DO NOT PLACE ANY SPACES BETWEEN OPTION LETTER AND NUMBER OR AROUND COMMAS!

Strip is effective at quickly removed certain characters froma file. By
default, it will remove all control characters (those less than 32 in the
ASCII code set) except line feeds and form feeds. This set is sometimes
referred to by a different name such as the ANSI code set or keyboard code
table. Your DOS manual probably has a copy of this table in the Appendix
section.

By using the options, you can configure Strip to remove any set of characters
that you would like. For example, STRIP -B5 -E31 -S10,13 INPUT.DOC OUTPUT.DOC
will remove all codes between 5 and 31, inclusively, except 10 and 13.

To have an empty "skip" list, just use the -S option with no decimal values
listed after it. Example: STRIP -S INPUT.DOC OUTPUT.DOC

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (STP).

A shareware limitation limits processing to 200,000 characters per file.
Register for the unrestricted version.


*** Parafy ***

Usage: PARAFY infile [-Cx] [-P] [outfile]
where x is the column number text must start before
or else a blank line will be inserted after that line.
Column values: minimum=1, maximum=1000, default=8
-P indicates that piping of input and/or output is being performed.

Parafy will insert blank lines after any line which begins past a given
left margin. In effect, making the line into it's own paragraph.
For example, if normal paragraphs begin at column 9, then lines beginning
after that column are probably not legitimate paragraphs and are
good candidates to have a blank line inserted after them.

Blank lines are only inserted after a line if one doesn't already exist.

Run Untab on the original file before running PARAFY on it. Parafy counts
spaces to determine the starting position of a line and TAB characters
confuses this.

You can use Parafy to insert a blank line after every line that doesn't
already have a blank after it by using a column number of 0 as the
margin (PARAFY -C0). This will double space your documents.
Just make sure that you run the word wrap program on it before double
spacing it since you won't be able to afterwards.

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (PFY).

A shareware limitation limits processing to 4,000 lines per file.
Register for the unrestricted version.


*** FndStart ***

Usage: FndStart infile [-Cx] [-By] [-P] [outfile]
where x is the name of the configuration file containing strings to match.
y is the maximum size in characters of the lines (buffer size).
Buffer values: minimum=1000, maximum=25000, default=10000
-P indicates that piping of input and/or output is being performed.


FndStart is a program which will insert a blank line after each line
which begins with a given string.

FndStart requires that you set up a configuration file and tell FndStart
the name of that configuration file. If not given a configuration file
as an option, it looks for a file named FNDSTART.CFG in the current
directory.

The configuration file should contain the patterns that you want to
search for that should have a blank line inserted after them. Each line
should contain one pattern to search for. Blank lines and lines that begin
with the characters REM in the configuration file will be ignored. REM is
used to signify a remark or comment just like in DOS batch files.

By default, the pattern does not have to be exactly positioned in the
input as it is in the configuration file. For example, if your configuration
file contained the following line.

From:

and the input file had a line which started with the pattern, but it started
in column 8, like

From: TOM RODDY

then it would be considered a match and a blank line would be inserted. If
you do not want that to happen, then use the -E option to specify that you
want exact matches.

An example command might be:
FNDSTART -cfndstart.cfg INPUT.DOC OUTPUT.DOC

The upper/lower case of the search string matters. Type it exactly as you
want to search for it.

Blank lines are only inserted after a line if one doesn't already exist.

You probably should never need to change the buffer size using the -B option
unless you are absolutely sure that no line will be longer than a certain
length and you do not have the 20 Kbytes of memory available for the default
buffers.

Remember, FndStart only matches patterns at the beginning of each line. It
will not match patterns found elsewhere in a line of text. For example,

From:

would not match an input line like

To: TOM RODDY From: JOHN DOE

since the text does not occur as the first non-blank text in the line.

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (FND).

A shareware limitation limits processing to 4,000 lines per file.
Register for the unrestricted version.


*** FixShort ***

Usage: FixShort [infile] [-Lv] [-Sx] [-Ny] [-Bz] [-W] [-P] [outfile]
where v is the approximate length of each line
x is the maximum percentage of v allowed to insert a blank line
y replaces v&x combo by stating maximum length of line allowed in
characters to insert a blank line
z is the maximum size in characters of the lines
Length values: minimum=0, maximum=1000, default=80
Size values: minimum=1, maximum=100, default=80
Buffer values: minimum=1000, maximum=25000, default=25000

-W will count leading white space when determing line length
-P indicates that piping of input and/or output is being performed



FixShort is a program which will insert a blank line after each line
whose length of readable characters is less than a given percentage of
the normal line length. This helps when headers, footers, titles, etc.
are not already followed by a blank line.

"White space" is traditionally defined as all characters which cannot be
seen. This includes the blank character, tab character, newline, etc.

By default, white space leading up to the first readable character is
not counted in the length of line when determining to insert a blank
line. To count the leading white space specify the -W option. This will
be useful when formatting files that have large indentations for some
paragraphs. However, it is possible that the -W option could cause a
header to not get a blank line after it if the header had a lot of white
space before it. You'll have to decide, based on your document, whether
to count white space or not.

You may specify the maximum length of a line to have a blank line added
after it in two ways:

1) The -Lx & -Sy combination: x represents the average length of the
lines in your document and y represents the maximum precentage
of x allowed for a line to have a blank line added after it.

2) The -Nx option: x represents the maximum length in characters on
a line to have a blank line added after it.

The -Lx & -Sy combination is the default method if neither method is
specified.

Blank lines are only inserted after a line if one doesn't already exist.

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (FXS).

A shareware limitation limits processing to 4,000 lines per file.
Register for the unrestricted version.

To show the usefulness of FixShort, consider a document containing the
following text:

----------------------------------------------------------------------

From: Tom Roddy
To: John Doe
Subject: Hello

Hi John,
How are you doing?
Here is the prioritized list of account numbers that I would like you to
follow up on:

Account # Priority
1231 1
1232 2
1233 3
1234 4

Thanks,
Tom

----------------------------------------------------------------------

You probably would not want those lines to wrapped and filled together
as most "word wrap" programs would do:

----------------------------------------------------------------------

From: Tom Roddy To: John Doe Subject: Hello

Hi John, How are you doing? Here is the prioritized list of account
numbers that I would like you to follow up on:

Account # Priority 1231 1 1232 2 1233
3 1234 4

Thanks, Tom

----------------------------------------------------------------------

but how would a word wrap program know if those were legitimate
paragraphs or not?

If, prior to wrapping it, you ran FixShort on the file to insert blank
lines after each line whose length was less than 50% of the normal line
length, then blank lines would be inserted after each of them and
therefore would not be seen as a paragraph.

----------------------------------------------------------------------

From: Tom Roddy

To: John Doe

Subject: Hello

Hi John,

How are you doing?

Here is the prioritized list of account numbers that I would like you to
follow up on:

Account # Priority

1231 1

1232 2

1233 3

1234 4

Thanks,

Tom

----------------------------------------------------------------------


*** FndRep ***

Usage: FndRep [infile] -Fx [-Ry] [-I] [-P] [outfile]
where x is the string to search for. This string may be specified
as an ASCII string in single quotes (') or a list of decimal
values which represent the ASCII characters in the code chart
separated by commas.
y is the string which will replace string x above. The same formats
for string formats as above are allowed. If no "replace" string is
given then the found string is deleted.
-I indicates a case insensitive search for the given string.
-P indicates that piping of input and/or output is being performed

FndRep is a "find and replace" program. The program searches for a given
string and replaces all occurences of that string with another given string.
Alternatively, you can just have all occurences of the found string deleted.
There can be NO SPACES in either string. If you need to replace strings which
contain strings, you'll need to specify them as a list of decimal values from
the ASCII code chart. Each value must be separated by a comma.

See your DOS manual for a copy of the ASCII code chart.

Example:

FndRep -F'the' -R'THE' INPUT.FIL -P

This example would replace all occurences of the string "the" with "THE"
from the file INPUT.FIL and displays the output to the console (screen).

FndRep -F116,104,101,32,99,97,116 INPUT.FIL OUTPUT.FIL

This example would remove all occurences of the string "the cat"
from the file INPUT.FIL and writes to OUTPUT.FIL.


Miscellaneous information abount the File Format Utilities:

Specifying the output file name is optional. If no output file name is given,
the output file name will default to the input file name with an extension
identifying the program (PFY, FXS, FND, EOL, STP, UNT, and FRP).
For example, using FixShort, the input file TEXTFILE would default to the
output file TEXTFILE.FXS while TEXTFILE.TXT would also default to TEXTFILE.FXS.
The File Format Utilities will error if the input file has the same name as
the output file.

If you want to use several of these programs together on a text file without
having to go through the pain of specifying input and output files, you can
"pipe" the output from one program to another. Use the -P option to activate
piping so you will be free from specifying input and/or output files. Just
make sure that you pipe the programs together in a way that each has an
input and output source. See FIXFMT.BAT for an example. One word of caution.
When you are done piping the data through the programs, you'll want probably
want to save it to a file or send it to the printer. If the final program
in the series of pipes redirects the output to a file, there will be an
additional carriage return (CR) added after each line without a corresponding
line feed (LF). This may or may not be acceptable depending on what you
will be doing with the file.

Most word processor programs save their files in a proprietary format with
control characters. If the text file is not in ASCII format, you should
convert it to ASCII for best results with some of these programs. Most
word processors allow you to do this.

The programs should be placed in a directory which is in your search PATH
so that you can run the program from within other directories.

Each of the utilities exits with a value of zero on successful completion
of processing or non-zero if there was an error. This is useful when running
the programs from within a batch file (see FIXFMT.BAT).

Warning: you are not legally allowed to modify many copyrighted software
manuals. This program should be used on your own personal files or with the
permission of the document's author.


Plugs for my other programs:

Rap is a "word wrap" program which is used to reformat a text file into
specified line lengths without splitting up words across lines. If a word will
not fit at the end of a line, then the entire word will start the next line.
This differs from programs which reformat text files to a given line length
regardless of the word boundaries.

Rap version 1.3 can be found on many BBS's and CompuServe as RAP13.ZIP. If a
newer version of this program has been released, the name of the file will
differ slightly to specify that version number.

If you are looking for a printing program to print your files into newspaper
style columns, take a look at another shareware program by this author called
ColPrint. ColPrint reformats a text file into two or three 80 line columns for
printing on standard 8 1/2" x 11" paper. ColPrint removes excessive blank
lines and original form feeds and uses a small font to fit up to three or four
times as much text on a page compared to regular printing. Many printers, from
9 pin dot matrix to laser printers, are supported. Several command line
options allow you to control the actions of ColPrint.

ColPrint 2.1 can be found on many BBS's and CompuServe as COLPR21.ZIP. If a
newer version of this program has been released, the name of the file will
differ slightly to specify that version number.


If you can't find the above programs on your local BBS and do not have access
to CompuServe, send $5 to the address below and I'll mail you a Shareware copy
to evaluate or send $9.95 per program + $1 (shipping/handling) for registered
unrestricted versions.

The official distribution site for the File Format Utilities, Rap, and
ColPrint is the Programmers Corner BBS in Columbia, Maryland (410) 995-6874.
You can be sure of having the latest version of these programs if you download
them from there. CompuServe will have it soon after the Programmers Corner. I
can also be reached on the Programmers Corner BBS for technical support by
sending a message to Tom Roddy.

The author can also be contacted by US mail, CompuServe, or the Internet:

Thomas M. Roddy CompuServe ID: 73730,3304
7158 Harp String Internet: [email protected]
Columbia, MD 21045


Trademarks are the property of their respective owners.