Source Routing Bridges, Transparent Bridges, Routers and NetWare
----------------------------------------------------------------

By Robert C. Wessel



This document is intended to explain the difference between the three
major methods of connecting LANs, with a special emphasis on Source
Routing and Token Ring. In addition, allowed mixtures of these methods
are discussed.


Audience
--------
This document is intended for people planning or maintaining an
internet. Especially internets involving protocols other than NetWare's
IPX. The reader should have a basic understanding of why an internet is
useful and a general idea of what one would look like.


Areas not Discussed
-------------------
This document does not discuss specific products or specific
manufactures except to point out devices that are definitive, or are
particularly representative, of their class. The requirements imposed
be various LANs (such as length limitations) is outside the scope of
this document.




Preliminaries (Definitions and Things)
--------------------------------------
Novell (or NetWare) Bridges are really IPX Routers. Novell's incorrect
use of the term bridge has caused significant confusion. Novell is
currently changing all references to "Bridges" it its documentation and
literature to "Routers". No references to Novell "Bridges" will be made
in this document. The correct term "Router" will be used.

Unless otherwise stated, all references to "Routers" in this document
refer to Novell IPX Routers (see prior paragraph).

The term "Bridge" will always mean a MAC Layer transparent bridge in the
OSI/ISO sense. It will never mean a Novell Router or an IBM SR Bridge.
Unless specified otherwise, "Bridge" should be taken to mean any
transparent bridge, including the entire range of intelligence available
in bridges (including Spanning Tree Bridges and most Routing Bridges).

A "Routing Bridge" is a bridge with many of the same decision making
capabilities as a router, while still functioning as a bridge. These
are sometimes incorrectly referred to as Brouters. A Brouter is a
device that acts as a router for some protocols and a bridge for others
(see below).

The term "SR Bridge" will always refer to a Source Routing Bridge like
the one implemented by IBM's "Token-Ring Network Bridge Program". A
number of vendors produce dedicated SR Bridges. These typically cost
more than the combination of a PC, two Token-Ring NICs and the IBM
Bridge Program, but typically offer higher performance. In addition,
these devices often provide additional capabilities (see next
paragraph). For most users, the IBM Bridge Program will perform quite
well. It is also a very stable and reliable piece of software.

A number of Bridge/Router vendors supply products that can act in two,
or even all three, of these roles. These products should be considered
equivalent to several distinct devices.

For connecting two remote LANs, Bridges, Routers and SR Bridges are all
available in "split" or "remote" models (the terms are synonyms).
Essentially the two sides of the device are separated by a
communications link of some sort (frequently a line leased from a common
carrier - commonly 56KB or T1, but occasionally 9600bps). Except for
performance, split or remote device function identically to their
"local" counterparts.

A repeater, not otherwise discussed in this document, is a device that
extends a LAN at the physical layer. The LAN performs exactly as a
single piece of wire would, except that distance limits are relaxed.
Repeaters are invisible to all devices attached to the LAN.

The term LAN usually refers to all the nodes connected directly to each
other with out any intervening devices other than repeaters. For
example, several Ethernet segments can be connected into one LAN via
repeaters. Similarly, in Token Ring, repeaters can be used to extend
the main ring beyond the normal limits. In some LANs, 10baseT Ethernet
or ARCNET, for instance, the "hub" device is effectively a repeater.
For Ethernet, a single trunk defines the LAN (although the trunk may be
disguised inside 10baseT hubs). For Token Ring, a single ring is a
LAN.

It is possible for a physical piece of wire to appear to be more than
one LAN. Consider Ethernet II and 802.3 packets (Ethernet II packets
use the frame format originally defined by Xerox, the 802.3 committee
defined a slightly different frame format). No Ethernet II node should
ever pick up an 802.3 packet, or vice versa. Therefore there can be two
LANs coexisting on a single wire. A Ethernet II node would require the
presence of a Router to talk to a 802.3 node. This is the only case
where a NetWare Router or server may have two NICs attached to the same
cable. Note that in NetWare 386, a single NIC may function as both a
Ethernet II node and a 802.3 node. For broadband LANs, numerous
apparent LANs can co-exist on a single wire. Each "LAN" occupies a
different frequency or pair of frequencies (similar to the way that 60
cable television channels can coexist on a single coaxial cable). Each
of these "LANs" is invisible to the others without the introduction of
some device to connect them (repeaters, bridges and routers are all
possibilities). A Token Ring LAN cannot be configured in this way.

Repeaters, Bridges, Routers and SR Bridges can all be constructed to
connect more than two LANs. Any discussion of "one side" and the "other
side" of one of these should be understood to be extendible to devices
supporting more than two "sides".

A special case of Router (NetWare Bridge) exists in NetWare, in which a
Router is connected to only one LAN. This is commonly used to run the
Macintosh VAPs, and is really being used to implement a Gateway and is
neither a Router or Bridge. These are not discussed in this document.

The term "Internet" refers to a collection of LANs connected with
Routers, Bridges and SR Bridges. As stated above, repeaters operate
within a single LAN (but can, of course be used inside any LAN in the
internet).

Unless otherwise stated, the term "LAN Manager" refers to the IBM
network monitoring/diagnostic package, NOT the Microsoft Network
Operating System. Note that IBM has recently renamed its "LAN Manager"
product "LAN Network Manager".

A significant change occurred in Novell's implementation of Source
Routing in its Token-Ring drivers with the release of the 2.50 drivers.
This document assumes that the 2.50 drivers (1.1 for NetWare 386) are in
use.

The term "Novell NETBIOS" refers to the NETBIOS implementation included
with NetWare. This NETBIOS will not communicate with any other vendors
NETBIOS, including IBMs.

The term "packet" is used, especially in the SR Bridge section, to mean
"frame". Please be aware that these terms are not synonymous, but for
the purposes of this document, may as well be.

The term "node" includes all non-bridge stations on a LAN. All
workstations, NetWare file servers and IPX (Novell) Routers are nodes.
SR bridges and transparent bridges are not nodes (from the point of view
of this document).

The term "Network Number" is sometimes used to mean a NetWare "Network
Address". This document uses the correct term "Network Address".

In the "Mixing and Matching" section, the term "domain" is used to mean
a section of an internet bounded by Novell Routers. While this doesn't
quite fit any of the usual definitions of "domain", it was the only
fitting word I could think of...




What They Do
------------
This section provides a short overview of how each of the LAN to LAN
connections works.


Bridges
-------
Bridges are conceptually the simplest way to connect two LANs. A Bridge
exists at OSI/ISO layer 2. Essentially, a bridge will pass all traffic
from one LAN to the other LAN if the destination node address is not on
the source LAN.

Like Routers and SR Bridges, ordinary bridges can be any combination of
software and hardware. Some of these devices may be implemented with

ordinary NICs in a PC. Others may implement custom network interfaces
and custom bit-slice processors, all optimized for performance.
Available capacities (performance) and costs vary widely between
vendors.

Bridges come in a wide variety of "intelligences". The dumbest bridges,
no longer in common use, forwarded all packets from one side of the
bridge to the other. These did no better than repeaters in filtering
traffic. The only advantage was (some) connectivity between different
LAN types, and the removal of some of the limitations of repeaters
(Often more than sufficient justification for their use).

The simplest (dumbest) current bridges monitor traffic on the connected
LANs, and will build a list of attached nodes on each LAN. Only if a
destination node is not on the LAN on which the packet is found will
these bridges forward the packet to the other attached LANs.

The most common currently used bridges are known as "Spanning Tree
Bridges" (often "ST Bridge"), and are so named because all the bridges
on the internet know of each others existence (either by parameters
configured at installation time, or by broadcasts to each other during
operation) and can therefore arrange themselves into a hierarchy for
routing purposes. The hierarchy ends up looking like a tree, and all
bridges will only transmit up and down the tree. This will prevent the
"Broadcast Storm" problem discussed below. It also may prevent optimal
routing in complex networks, since many routes are disallowed to prevent
the loops that can lead to endlessly cycling packets and broadcast
storms. Also, a parallel bridge is usually useless for load sharing,
and can at best be a redundant bridge, in case the first one fails. A
problem is that spanning tree bridges of one vendor are generally not
aware of another vendors. Some care must be take in configuring bridges
in a large internet, as some bridges, when left to themselves, can pick
inefficient "trees". The more sophisticated ST Bridges base the tree on
path costs and so generally avoid inefficient trees.

The more sophisticated bridges can communicate with other bridges in the
internet, and can construct a map of all the nodes on the internet.
These bridges therefore can send a packet via a efficient route. These
are sometimes referred to as "Routing Bridges". Again, one vendors
Routing Bridge typically cannot talk to another vendors Routing Bridge
to construct their maps of the internet.

Unfortunately, the most intelligent bridges can have at best an
incomplete understanding of the internet. Whenever a node sends a
packet to a node that the bridge has never heard of, the bridge must
broadcast the packet around the internet to get it to its destination.
Even if the packet is destined for a node on the same LAN as the sending
node! With a Router (or SR Bridge), a (non-broadcast) packet leaving
the LAN is clearly identified, and the destination LAN is clearly
identified, so the Router or SR Bridge can send the packet to the
destination LAN, instead of saying, "Gee, I've never seen this address
before, so I better send this packet everywhere...". The essential
problem is that a bridge cannot determine that a station is on a LAN
unless it happens to see a packet from that station go by and it can
determine that this packet was not one forwarded by another bridge. So
bridges are often fighting the routing problem with one hand tied behind
their back.

In addition, many bridges allow some filtering or explicit routing to be
specified when the bridge is configured. For example: "Do not forward
any packets destined for node 1234". "Forward packets for node 9876 to
LAN #3 only". In large (bridged) internets, these filtering and routing
controls become essential to keeping mis-routed traffic volumes to a
reasonable level (especially with the less intelligent bridges). As a
simple example, consider a (completely stupid) bridge that has to be
programmed with the location of all nodes and the route required to get
there. If accurately programmed, such a bridge would forward packets
exactly as needed, with no excess traffic. Such a bridge would also be
very fast. Unfortunately, the maintenance of such a device is
difficult, to say the least, and the addition or removal of bridges from
the internet is very difficult (because all the other bridges would have
to be reprogrammed). The available routing and filtering capabilities
vary widely between bridges.

The two principal advantages of bridges are speed and protocol
transparency. Bridges, since the processing of individual packets is
simple (often no more than a hashed table look-up), have a higher
"Forwarding Rate" than Routers, although the advantage for very
sophisticated Routing Bridges is much less or non-existent (because a
Routing Bridge has to do many of the same things a Router must do).
Typically, with dedicated hardware, an ST Bridge is two to three times
faster than a Router (when measured by the rate at which it can filter
and forward packets).

Since bridges only look at the destination node address in a packet,
they can be protocol independent to a large degree. No node on the
internet can be aware of the presence of bridges (except possibly by
monitoring response times). In NetWare terms, all LANs connected by
bridges appear to have a single "Network Address".

The four principal disadvantages of bridges are their lack of
understanding of the internet's topology, the lack of assistance in the
routing process by the nodes, the limited tolerance for variations in
node addresses in different types of LANs, and the lack of awareness by
any node of any LANs restrictions.

Because bridges are not aware of the topology of the internet the number
of packets propagated by bridges to useless destinations can be quite
high, leading to excessive traffic on some LANs. Also, a bridge that
has received a packet which has a destination node whose destination LAN
is unknown must always broadcast the packet (in an attempt to get the
packet to the destination). This property, especially combined with a
mix of bridges (of different types, or from different vendors whose
bridges are not aware of each other), can lead to a "Broadcast Storm"
where undeliverable packets are endlessly forwarded between bridges.
Fortunately, this is avoidable through the use of spanning tree bridges
or routing bridges that can talk to each other and/or preventing any
loops from being introduced in the internets topology. However, all
bridges maintain some susceptibility to broadcast storms if there are
loops in the internet. Both Routers and SR Bridges can avoid broadcast
storms entirely (essentially all Routers and SR Bridges do).

The second disadvantage of bridges is that the node sending the packet
has no opportunity to assist the bridge(s) in getting the packet to its
destination. This requires a bridge to examine all packets on the LAN
in the process of deciding which to forward (this is one of the
constraints on bridge performance - one of the performance numbers
usually given for a bridge is the "filtering rate"). In the case of
Routers, the nodes identify packets that will leave the LAN and will
specify the destination LAN for the packet. Also, a router will
generally have a "hop count" available and can discard packets that have
been circulating aimlessly.

The third disadvantage is the lack of tolerance for different addressing
schemes used by LANs. This is not a problem if all stations have the
same format address, and all stations have unique addresses. For
instance, all 802.x LANs assign each NIC a 6 byte node address.
Typically, each NIC has this address encoded in its hardware, and these
NIC addresses are guaranteed to be unique for all NICs (these addresses
are unique even between types of LANs - that is, no Ethernet NIC would
ever have the same address as a Token-Ring NIC). The problem arises
when an incompatible address format, or non-unique addresses are
introduced. For example, the node address on ARCNET NICs is a single
byte, and each separate ARCNET LAN will likely have duplicate node
addresses. Since node '12' is now on two LANs, a Bridge cannot
determine how to forward the packet. Further, there is no place to put
the 6 byte 802.x address if a packet has to cross an ARCNET LAN.
Therefore, there never will be a ARCNET capable bridge (in theory, an
ARCNET Bridge could be constructed so long as all node addresses are
kept unique, but as this leaves a maximum of 254 nodes on the internet,
this would be of limited use).

The fourth disadvantage is that nodes are completely unaware that
intermediate LANs may have different limitation on things like packet
sizes. This can prevent the use of bridges between topologies which
support different size packets. Consider that 16Mb Token-Ring can
support 18KB packets, while Ethernet can only support 1500 byte packets.
A bridge obviously cannot forward a packet bigger than 1500 bytes over
an Ethernet LAN. This can be dealt with by assuring that the nodes
never transmit packets larger than the smallest size that any LAN in the
internet can deal with. For 802.x LANs, the smallest "largest packet"
is 516 bytes. But for the common LANs, Ethernet's 1500 bytes is the
limit. This is another reason why an 802.x to ARCNET Bridge would be
impossible (ARCNET only supports 512 byte packets). Frequently bridges
are not used to connect non-identical LANs because of this problem.




Routers
-------
A Router connects two LANs at OSI/ISO layer 3. Essentially, a router
knows of all other routers (for the same protocol) on the internet, and
therefore possesses a map of the entire internet to assist in route
determination. A router must explicitly understand the protocol it is
routing, but routers exist which understand more than one protocol. The
misnamed "Novell Bridge" is an IPX router. It is not capable of routing
any other protocol. An IPX router can route SPX or Novell NETBIOS
packets because those protocols are implemented on top of IPX. In
truth, IPX routers are unaware of SPX or Novell NETBIOS (some IPX
routers will not forward Novell NETBIOS packets, and so are slightly
aware of Novell's NETBIOS).

The nodes in an internet participate in "Routing". Usually, the node
will provide an indication of what the destination LAN is, and often
other provisions such as a "hop count" (used by the routers for removing
endlessly circulating packets) are made. To allow the node to provide
the destination network, routers typically include some method for nodes
to find one another. For NetWare, this is Novell's "Service Advertising
Protocol" (SAP). Other protocols have implemented a variety of
schemes.

Once a node has determined the location of the destination node, it will
send all packets destined for that node to its local router, always
including the full address (LAN ID ["Network Address" for Novell
networks] plus NIC address) of the destination. This neatly avoids
requiring Routers to examine all packets.

Typically, a router has few, if any, filtering options.

Routers can usually support parallel routers but provide little in the
way load balancing between those parallel routes. Due to a peculiarity
in the implementation of NetWare 286 (but not NetWare 386) file servers,
IPX Routers can incorrectly chose a longer than necessary route to the
destination server. This makes the use of multiple routes in a network
with NetWare 286 servers a potential problem. An advantage of Routers
over both Bridges and SR Bridges, is that they can automatically,
without the knowledge of either sending or receiving node, respond to
the loss of an available route by sending the packet via an alternate
route (although it is likely that some packets will be lost during the
failure of a router, this is not a problem since the protocol being used
must be able to deal with lost packets anyway [lost packets are dealt
with at OSI/ISO layer 4]).

Because the routing protocol has to be defined (because it is visible to
all nodes on the internet it has to be standardized), Routers from
different vendors typically can interoperate if both support the same
protocol. (Eg. All IPX routers can talk to each other).

The two principal advantages of routers are that all information needed
to efficiently transport a packet from the sending node to the receiving
node is available, and is used, and the independence of the router from
the physical node address format on the LAN. The router does not have
to guess about best destinations as does a bridge, but the routing
process is significantly more complex than the forwarding process used
by bridges. Also, since the router does not depend on the physical
format of the node addresses, and qualifies each node address with a LAN
address, it can deal with different types of, and non-unique, node
addresses. Therefore it is possible to have a router between an 802.x
LAN and ARCNET, for instance. In NetWare terms, all LANs connected with
IPX routers must have unique "Network Addresses".

The three principal disadvantages of routers are their protocol
dependence, the increased overhead in the router, and the workstations
lack of awareness of the specific limitations of the route employed.
Simply put, an IPX router is an IPX router and nothing more. Packets
for another protocol will be completely ignored. Many router vendors
have routers that can understand more than one protocol. Further, some
routers can act as bridges for all packets it cannot identify. Because
of the increased complexity of the routing task, a router is typically
two or three times slower than an equivalent spanning tree bridge.

Finally, typical routing protocols hide the limitations of the chosen
route from the nodes. Typically, the node cannot control the route
chosen. This requires nodes to assume worst case in things like packet
size. Because there is no defined route for the packets, and the actual
route can change at the whim of the router, the worst characteristics of
any LAN that might be crossed must be considered as the usable limits.
This leads to NetWare using only 512 bytes packets if any IPX router is
to be crossed.




SR Bridges
----------
Source Routing is the scheme chosen by IBM for connecting LANs. SR has
unique strengths and weaknesses, and has been the cause of much heated
debate in the industry. SR has also been subject to a great deal of
mis-information, as it is, in some ways, the most difficult of the three
methods we are discussing to analyze.

For NetWare users, SR only comes into play with Token-Ring LANs.

By way of background, several unusual features of Token-Ring, and
Novell's use thereof, need be discussed. A unique feature of TRN at the
time it was introduced was the presence of a software DLC (Data Link
Control - a reference to OSI/ISO layer 2) interface which permitted
multiple applications to access the TRN NIC at the DLC layer. This
interface was originally provided by IBMs "TOKREUI.COM" module (commonly
pronounced "toke-ree-oo-ii""). Later, TOKREUI was supplanted with the
extra cost IBM product "LAN Support", which consists of a number of
device drivers named "DXM??MOD.SYS" ("dixie-mods"). The TOKREUI module
is no longer supported by either IBM or Novell. A number of problems
can be traced to the continued use of TOKREUI in TRN LANs and all users
requiring the DLC interface should use the LAN Support program. The
original Novell TRN drivers for IBM's NICs, wrote directly to the NIC
for the server/bridge drivers and used the DLC interface for the
workstation drivers. The original Novell TRN drivers did not support SR
in any form. Any IBM SR bridges were simply ignored, as no (source)
routing information was available in the packets.

The ability for more than one application to use the NIC is very
important, and becoming more so all the time. Since the entire world
does not run IPX, being able to access the NIC in parallel with IPX
allows other applications (such as 3270 emulators, or IBM's NETBIOS) to
make use of the same NIC as IPX. For some time, Clarkson University has
been writing IPX drivers that allow the simultaneous use of a TCP/IP
stack on Ethernet NICs. Recently, two industry groups, one headed by
3COM/Microsoft/IBM and the other by Novell, have announced, and started
to ship, similar "layered" drivers that allow the use of multiple
protocol stacks simultaneously on a single NIC (the 3COM group is
pushing "NDIS", Novell is pushing "ODI"). While these three schemes
perform many of the same functions as the DLC interface provided in IBMs
scheme, they function at a somewhat lower layer in the ISO/OSI stack.
This is neither good nor bad, just different. It requires that the
protocol stack built on top of NDIS or ODI to be more aware the
underlying hardware. This should usually improve performance and allow
support of more varied hardware (ARCNET again comes to mind). While the
higher level interface provided in IBMs scheme allows the same interface
to be used for any 802.x LAN. In addition, the 802.2 DLC interface
provides both connectionless and connection oriented services to the
application.

Novell then added support for SR in early 1989. These SR drivers, again
direct to NIC for the server, DLC/LAN Support drivers for workstations,
came complete with a number of annoying quirks. First, and foremost,
the server drivers took an inordinate amount of DGROUP RAM, leading to
severe FSP problems on NetWare 286 servers (a thorough discussion of FSP
problems can be found in the file "FSP.ZIP" on the NetWire forum "NOVA"
in Library 16 or 2). Second, the implementation of SR was flawed in
that all packets, even those not leaving the local ring (LAN) contained
routing information. This quirk allowed the existence of both SR'd and
non-SR'd IPX packets on the same ring, and since the TRN drivers were
very selective (the SR drivers would ignore all non-SR packets, and
vice-versa), it was possible to mix SR Bridges and IPX routers in
creative ways to avoid using the SR drivers in a NetWare 286 server.
Along with a few other quirks, these things made the original Novell
implementation of SR difficult to use at best, and something we all
tried very hard to avoid. This original implementation of SR was also
never available for NetWare 386.

Under no circumstances can the original SR drivers be recommended for
use in a new LAN, and it is highly recommended that the new SR drivers
be implemented in all LANs using SR.

In mid-1990, Novell shipped some significant enhancements to its TRN
support. First, the implementation of SR was completely redone. All of
the missing tuning parameters were added, SR information was added to
packets only when needed, and all drivers were capable of both SR and
non-SR behavior (the SR part being largely implemented in a TSR, .VAP or
.NLM, depending on the environment). Also, NetWare 386 now supports SR.
These drivers clear up the FSP problem in NetWare 286, support larger
packets, and will optionally talk to the TRN NIC directly on
workstations (removing the need for LAN Support and enhancing
performance, if the DLC interface is not needed). Also, Remote Program
Load (RPL) support was significantly revised, and made available for
NetWare 386 for the first time. In general, performance improved for
all aspect of the drivers operation. These drivers were the 1.1 release
for NetWare 386, and the 2.50 drivers for NetWare 286 and workstations.

Unfortunately, these "new" SR capable drivers are not entirely
compatible with the "old" SR drivers. Specifically, a node running the
"old" SR driver cannot talk to a "new" SR node if the two nodes are on
the same ring. So when upgrading to the new drivers, all nodes on a
single LAN must be upgraded at the same time.

The remainder of this section assumes the use of the "new" SR drivers.

How Source Routing Works: The end result of SR is that each node knows
the exact route a packet must follow in order to get to the destination
node. Before sending a packet, the sending node puts the intermediate
ring addresses in the header of the packet. The "Routing Information
Field" has a two byte control field and up to 8 routing entries. Each
entry (there may be up to eight) consists of a 12 bit ring number, and a
4 bit bridge number. A side of an SR bridge is uniquely identified by
this 16 bit field. The sending node then sends the packet, which is
seen by all the bridges on the ring. Each SR bridge looks to see if it
is in the list of bridges to be crossed. If the packet is meant to
cross this SR bridge, the SR bridge will forward it to the next adjacent
SR bridge in the list by sending the packet out on the ring containing
the next SR bridge. When the packet arrives at the destination LAN, the
destination node will receive the packet since the destination address
is still contains the node address of the destination node. A route can
be used in the reverse order by a node to return a packet to original
node. Note that a packet that crosses three bridges, will have four
route fields (because it must pass through four LANs), so that the
reverse direction routing can take place.

SR can add up to 18 bytes of routing information to each packet,
although the average is less than that. Both SR'd and non-SR'd packets
can exist on the same ring, the SR'd ones being identified by an
otherwise unused bit in the Source Address field of the packet. An SR
bridge will ignore all packets not containing SR information.

The operation of an SR Bridge, on a packet with a route, is very simple.
An SR Bridge needs to know only its own Bridge number, and the ring
numbers of the LANs connected to it. The SR Bridge watches all packets
as does a transparent bridge. If it finds its own entry in the list of
bridges, it will forward the packet to the next ring in the list. This
is a very simple operation (involving, at most, the inspection of 18
bytes, plus one bit in the packet), and allows SR Bridges to forward
packets with essentially the same efficiency as a dumb transparent
bridge. However, the resulting traffic pattern is much more like the
traffic generated by routers than the traffic pattern generated by
bridges. There are no problems with multiple routes, there is never a
need to forward a packet simply because its destination is unknown
unless the packet is a broadcast (or part of route discovery - a type of
broadcast discussed below), and broadcast storms are not a problem in
any case because of the explicit route (all bridges must discard frames
that have invalid routes - for instance: routes with loops).

As with transparent bridges, SR Bridges work at the MAC layer (OSI/ISO
layer 2), and so are transparent to higher level protocols like IPX.
However, it is important to remember that an SR Bridge is NOT a
transparent bridge. An SR bridge will never forward a non-broadcast
packet not containing an explicit route, while no packet forwarded by a
transparent bridge has any kind of routing information at all. Again,
in common with transparent bridges, SR bridges require that all nodes on
the internet have unique node addresses. Unlike transparent bridges,
however, SR Bridges make provisions for load balancing (assigning
alternate routes if as the main ones become more heavily used) and
determining the maximum allowed packet size on the select route during
the route discovery process.

A consequence of SR being implemented at the MAC layer, is that all
protocols using SR will be able to cross IBM Bridges, thus SR Bridges
are protocol independent like transparent bridges (if the protocol stack
implemented is using SR).

The interesting part of SR is the route discovery process. Whenever a
node needs to transmit a packet to a node which is on an unknown ring,
the node will broadcast either an XID (Exchange ID) packet, or the
actual packet to be sent. Broadcasting the packet will get the packet
to the destination faster, but will generate more traffic on the
internet (because the data packet will be larger than an XID packet).
NetWare broadcasts the first packet. The initial broadcast is done in
one of two ways: an all routes broadcast, or a single route broadcast.
In any broadcast of this type, the route fields of the packet are filled
in by the SR bridges as they are crossed, so that when the packet
arrives at the destination, it contains the route it followed to get
there. Note that the first bridge must add two route entries - one for
the source ring, and one for the first "hop". The addition of the
routing information by the SR Bridges allows the "all routes" broadcast
to work.

What the "all routes" broadcast does is cause a copy of the packet to
follow every possible route to the destination. At every SR Bridge, the
packet will be forwarded to all rings attached to the SR bridge where it
has not yet been. This, of course, is determined by the route
information added each step of the way. The destination node will then
receive a packet via each possible route from the sending node. The
fastest (which can also be considered the least congested) route is the
one associated with the first packet received. The destination can then
use the route in that packet (in reverse) to return a message to the
sender. There after, both nodes will know the fastest route to the
other node. Because this causes multiple packets to be delivered to the
destination, the "all routes" broadcast is not the preferred form.

Usually, a "single route" broadcast is done by the sender, and then the
packet is returned via an "all routes" broadcast, and the sender is left
to decide which is the best route. A single route broadcast works the
same way as a broadcast packet in a Spanning Tree Bridge. As a matter
of fact, SR bridges arrange themselves in a spanning tree hierarchy just
for the purpose of correctly handling single route broadcasts. As with
the "all routes" broadcast, a route is built in the process. Note that
various combinations of methods of sending the initial packet and
responding to it are allowed. NetWare, by default, uses single route
broadcasts with a fully routed (non-broadcast) return. This works fine,
unless multiple routes exist in the internet, in which case a less than
optimal route may be determined. The NetWare drivers allow the route
discovery method to be specified when loading the ROUTE program.

During the route discovery process the largest packet size that can be
handled by the route is determined. A field in the packet is set by the
sending station indicating the longest packet it can handle. Each
bridge will reduce that number if the LAN about to be crossed cannot
handle that size frame.

In any event, once a route has been discovered, that route is usually
used until it is no longer needed, or until some failure invalidates the
route. At that point, the route discovery process outlined above is
repeated. Some applications will periodically re-discover their routes
in order to compensate for shifting loads on various bridges and LANs.
Nodes will typically maintain a route for each station they will send
packets to. For connection oriented services, this function is
performed by the NIC. IPX is built on top of the DLC connectionless
services and so the ROUTE program must maintain its own list of routes.

Much has been made of the additional traffic involved in SR. First,
each packet sent to another ring will contain routing information.
Obviously, this can add up to 18 bytes to each of these packets.
However, this is pretty minimal overhead in the context of all network
activity. For starters, Token-Ring has 24 bytes of overhead for each
packet anyway (Ethernet, for comparison has 21 bytes of overhead per
packet), and small packets (which would suffer most from the extra
baggage) generally make poor use of network bandwidth anyway. Finally,
while 18 bytes is the maximum overhead, the typical route is much
shorter.

The second area where SR adds traffic is during the route discovery
process. When a station needs to begin a conversation with a station,
the route to which is unknown, it begins the route discovery process
described above. In a internet without multiple routes between rings,
the effect of route discovery is the same as a simple broadcast to all
rings (exactly what would happen in an internet based on transparent
bridges or routers). Likewise, if route discovery is done via a single
route broadcast/fully routed return (like NetWare does by default), the
effect is again a simple broadcast. When multiple routes do exist, and
one part of the route discovery is via the "All Routes" method, more
traffic is generated. Essentially, each ring will receive a copy of the
XID packet via each route possible from the source ring. As more and
more parallel routes exist through the internet, the number of XID
packets reaching any ring for each route discovery increases. If there
are 100 routes which could be followed to the destination, the
destination ring receives 100 XID frames. All other rings would also
receive XID packets in proportion to the number of routes possible from
the source ring to that ring. Note that these must be "real" routes.
Routes with loops in them would never be created. Since all rings,
including the remote ones, will get these XID packets, there are
obviously potential load problems for slow bridges created by route
discovery. Several articles in the industry press have alluded to this
"fatal flaw" in Source Routing. In reality, the "fatal flaws" exist in
the assumptions on which these articles have been based. While very
large route discovery traffic loads are certainly possible, they don't
occur often in practice for the following reasons: First, route
discovery isn't done all that often (for NetWare, only when attaching to
a file [or other] server for the first time). Admittedly, there are
times when there will be more route discoveries than at other times (for
instance, one would expect a large number of route discoveries around
9:00 AM when everyone turns on their workstations), but even so, this
burst of activity will likely span many minutes. Second, the number of
routes possible is invariable assumed to be ridiculously high. A common
assumption is having fully interconnected rings - that is, every
possible pair of rings would be connected with an SR bridge. While that
might be a realistic assumption for a three or four ring internet (where
the number of possible routes wouldn't be a problem anyway), it would
imply 190 (yes, one-hundred ninety) SR bridges in a 20 ring internet.
Obviously, that's silly. In an internet of fully interconnected rings,
the number of routes between rings will grow geometrically as additional
rings are added. In the real world, the internet is typically organized
is a fairly hierarchical fashion, with a few "short cuts" between rings
which have heavy traffic between them. Even in an internet of several
hundred rings, it is rare to see more than 50-60 possible routes between
any two rings. Third, SR bridges have a number of tuning options which
can be used to eliminate many possible routes. For instance, a bridge
can be set to not forward route discovery packets that have already
exceeded some number of "hops". That can eliminate the many additional
routes the "short cuts" can create.

So while excessive traffic due to source routing is a possibility a
network administrator/planner must be aware of, it is not a common
problem, except, perhaps, for those few people who have 9600bps remote
(split) SR Bridges on large internets. It should be pointed out that
the design an maintenance of large internets is difficult no matter how
they are built. Large internets including slow remote links are
especially difficult.

Using the Novell SR drivers for NetWare will allow IPX packets to cross
SR Bridges, but will not turn IPX Routers into SR Bridges. Under no
circumstances will non-IPX packets ever cross an IPX Router.

If the SR drivers are being used, then all LANs connected via SR bridges
appear to be one "Network Address" from NetWare's point of view. This
precludes the use of more than one NIC in a server attached to the group
of LANs connected via SR Bridges.

If a node is on the same LAN as all the other (IPX) nodes it will
communicate with, the ROUTE TSR/VAP/NLM does not need to be loaded.
That node will not be able to talk across SR bridges, but will be able
to talk to all the other nodes on that LAN, whether or not those nodes
have enabled SR.

There are four principal advantages to SR Bridges. First the
performance of an SR Bridge is as high, if not higher, than that of a
Spanning Tree Transparent Bridge because of the small amount of work
required to forward a packet. Second, SR Bridges, like transparent
bridges, are protocol independent. This allows one SR bridge to perform
all the internet routing for all protocol stacks using SR. Third, the
traffic patterns of SR Bridges are very efficient once a route has been
determined. Essentially all traffic is following a useful route to its
destination, much like the traffic pattern generated by routers.
Fourth, from NetWare's point of view, the ability to forward packets
larger than 512 bytes can significantly increase performance.

The three principle disadvantages of SR Bridging are the extra overhead
in each frame for the routing information, the extra traffic generated
by the route discovery process (which, in a complex internet, with many
possible routes, can be substantial), and its limited applicability
outside of Token-Ring (all 802.x style LANs could be supported, but
others such as ARCNET could not, because of addressing problems).




Mixing and Matching in NetWare
------------------------------
Routers, Bridges and SR Bridges can all be used in one NetWare internet.
Some rules, of course, must be followed. The simplest way to consider
the mixing of either type of bridging and routing, is to consider the
internetwork to be composed a one or more domains, each domain (for lack
of a better term), is connected with routers (only) to other domains.
No domain is connected to any other domain except via a router. A
domain is made up of one or more LANs. If there is more than one LAN in
a domain, the LANs must be connected with bridges (of either type).
Within a single domain, SR Bridge and transparent bridges cannot be
mixed. Each domain would be assigned a single Network Address in all of
the NetWare Routers attached to that domain. If the domain uses SR
bridges, then all NetWare nodes in that domain must be configured to use
SR.

When using SR Bridges (because non-NetWare traffic exists on the
internet - for instance, DLC connections to TICs in 3174s or AS/400s),
another configuration is possible and, for some LANs, can be quite
useful: Parallel NetWare Routers and SR Bridges. This can only be done
if all NetWare nodes are configured to NOT use SR. Then the SR Bridges
are essentially opaque to NetWare. Since the SR Bridges will not
forward the non-SR'd IPX frames, each LAN will then appear to be a
separate domain. From any other protocols point of view, the SR Bridges
will forward all SR'd packets, and the IPX Routers will route none of
them (because there will be no SR'd IPX packets).




The Author
----------
I can be reached via Compuserve E-Mail ("Robert Wessel, 76704,70"), or
on the NetWire forums where I am a SysOp. Comments and suggestions
about this document would be greatly appreciated.

This document may be distributed freely, but I ask that it be
distributed without modifications and in its complete form.