An intro to networking

There are a lot of resources floating around about networking and OpenFlow, but I plan to publish a couple of blog entries on some use cases, so here's my crack at creating a primer.

The primary job of a network forwarding device is to receive and forward packets (L3) or frames (L2). The primary decision is whether to forward, and out which destination interface. There are different methods to make this decision based on the technology, but arguably the most common implementations deal with forwarding ethernet and TCP/IP, and use a Forwarding information base (FIB).

First, packets vs. frames. Packets are IP (L3), and are transported across ethernet (or other technologies) inside frames (L2). The frame contains the packet. There is global-scope addressing information in the packet for the whole trip (across multiple routers), and there is local-scope addressing information in the frame for each hop between routers. The packet addressing doesn't change across the whole trip, but the frame is re-created every time the packet is forwarded between routers.

(Over-simplification warning!) In TCP/IP over ethernet, switches are used for local delivery of traffic, and routers are used for remote delivery. A computer is connected to a switch, along with lots of other computers, and the switch is also connected to a router. If TCP/IP were the postal service, routers would be post offices, and switches would be the postmen.

• When a computer wants to send a packet to an IP address, it uses the subnet mask to calculate whether the destination IP is local (in the same subnet) or remote (in a different subnet). An analogy is a 5-digit ZIP code to specify the local post office, and a 9-digit ZIP code to specify the individual house.
• If a packet's source and destination IP addresses (L3) are in different subnets, the sending computer will set the frame's destination MAC to the router's MAC. The switch reads the MAC addresses of the frame, and sends the frame to the router. The router extracts the packet from the frame, throws the frame information away, reads the packet's destination IP address, makes a forwarding decision, and wraps the packet in a new frame with a destination MAC address to that next router (and its own MAC address as the source). The next router does the same thing: pull the packet from the frame, make a forwarding decision, and create a new frame to send to the next router. The final router, local to the destination IP address, sets the frame's destination MAC to the address for the destination computer. The switch (between the final router and the destination computer) forwards the frame to the computer.
• However, if the source and destination IP are the same subnet, the sending computer sets the frame's MAC address directly to the destination computer's MAC. The switch will forward the frame directly to its destination, and a router isn't needed.

Generally, routers move packets between IP networks or subnets, and rely on routing protocols to make the decision based on aggregated collections of contiguous IP addresses, defined by applying increasingly larger subnet masks to the IP addresses. A router will check its FIB to figure out whether the destination IP is on one of its local interfaces, or whether it should give the packet to a neighbor router, which then makes the same decision using its own FIB. Routing protocols allow the routers to tell their neighbors about both their locally-connected subnets and their knowledge of larger collections of subnets.  That information is added to the routing table(s), and compiled into the FIB. Anything not in this advertised collection of routes falls into the "default" category, and is forwarded to the router's default gateway. In this way, packets move from areas with little knowledge of the destination towards areas of greater knowledge, and finally to the specific destination node.

Generally, switches move frames between MAC addresses within the same broadcast domain - either connected to the same switch, or to other switches connected to that switch. A switch "learns" where MAC addresses are located based on watching the source interface and MAC address of each frame they forward, then adds that knowledge into the FIB to be used in forwarding future frames.   If a switch doesn't have the MAC address in its FIB, it will typically "flood" the frame, sending a copy out all of its interfaces (except where it received it). Given that most TCP/IP traffic is part of a back-and-forth conversation ("flow"), the destination node will usually send a response, and the switch will learn where the node is, to use in future forwarding decisions. Traditionally, there have not been protocols that switches use to notify each other where the MAC addresses are, but this is starting to change in data center environments through protocols like TRILL (Transparent Interconnect of Lots of Links) and SPB (Shortest-Path Bridging).

It's generally easier to set up a switch than a router, both from the perspective of the actual network forwarding device and from the perspective of the computers they interconnect.  It's also much easier to make architectural changes to the switching infrastructure (adding, removing, etc) than to the routing infrastructure.  Each computer has an IP address and an associated subnet mask, so changing the routing architecture (or, more correctly, the IP subnetting architecture) can require making changes to every computer.  A common sentiment among network designers is therefore, "Switch where you can, route where you must."

What's interesting to me about both routers and switches is that they make forwarding decisions based on information in the FIB.  New technologies are starting to become available which change the way the FIB is created in the routers or switches, resulting in more efficiency and/or more control over how packets flow.