blas.phemo.us

edit

Setting Up CARP Interfaces on FreeBSD

Posted by Tres Thu, 05 Apr 2007 06:03:00 GMT

Introduction to CARP

CARP makes it possible to have redundant, fault-tolerant routers using off-the-shelf general computing systems. Using CARP can increase the reliability, extensibility and robustness of your core routers while decreasing the cost of ownership and lowering the barrier to entry. Older systems can easily be re-commissioned for use as routers, bringing the hardware costs to zero. I’ve seen a pair of 550 megahertz pentium III sytems provide layer 3 filtering and routing for a rack of streaming media servers in a datacenter; and the best part: the boxes weren’t even close to being fully utilized. I’ve also set up OpenBSD based CARP systems handle traffic for a site of nearly a thousand computers. These systems handle hundreds of simultaneous users working on 3D Studio Max and Maya files off of network file servers, VOIP traffic for multiple sites as well as day-to-day intranet and Internet services.

How does CARP work? Well, in short, a CARP interface is essentially a virtual interface that can be shared between multiple machines. The CARP interface is active on only one parent interface at any given time, but the virtual interface can be shifted to another parent interface very quickly in the event of a failure.

Now you’re probably saying, “that sure sounds a lot like Cisco’s VRRP!” Well, that’s because it is just like VRRP (Virtual Router Redundancy Protocol). CARP came in to being as an open source implementation of VRRP using VRRP as the spec. But get this–CARP not only performs the same kind of functionality as Cisco’s VRRP , it can also be configured to act like Cisco’s GLBP (Gateway Load Balancing Protocol) (although, technically, the CARP guys had their own implementation of GLBP working before Cisco did). Load-balanced routing with included failover capabilities is very exciting–I know–but GLBP and load-balancing CARP is a subject we’ll leave for another day. For now, let’s take a look at setting up a CARP interface on FreeBSD using rc.conf.

Setting Up CARP interfaces on FreeBSD

There are three basic elements to setting up CARP failover interfaces to boot on FreeBSD:

First, in /etc/rc.conf, tell FreeBSD that you’re creating a “cloned interface.” Unless the interface is included in the list of virtual interfaces, it will never be created by FreeBSD.

cloned_interfaces="carpXX"

The cloned_interfaces directive can contain as many virtual interfaces as you need separated by a space:

cloned_interfaces="carpXX carpYY carpZZ"

Then add the configuration directives for your cloned interfaces to /etc/rc.conf.

ifconfig_carpXX="vhid <carp-interface-id> advskew <priority> pass <password> <carp-ip-address>"

and then in /etc/sysctl.conf, tell the kernel that you want to allow CARP interfaces to preempt each other:

net.inet.carp.preempt=1

This directive also means that once any single interface fails on a router, all virtual CARP interfaces will be shifted to the failover router at once.

The Four Elements of The Interface Definition

The ifconfig_carpXX line is made up of four elements.

1. vhid <carp_interface_id> the virtual interface ID
2. advskew <priority> the priority of the interface
3. pass <password> the password used by hosts to validate the CARP relationship
4. <carp_ip_address> the IP address of the virtual interface.

ifconfig_carpXX="1==>vhid <carp_interface_id>     2==>advskew <priority>     3==>pass <password>     4==><carp_ip_address>"

That’s all there is to it. You won’t need to worry about where or how the virtual interface attaches, FreeBSD is smart enough to know which parent interface it’s supposed to use by the IP address assigned to the CARP interface. The only real bookkeeping you’ve got to do is to make sure that:

1. the VHID on your primary and failover routers match, and
2. that the priority of the primary is lower than the failover.

Example

In the following example uses two FreeBSD routers called primo (our primary router) and secundo (our failover router) in a failover configuration:

primo, our primary router

/etc/rc.conf

ifconfig_em0="inet 99.88.77.70/29" # external interface for primo, our primary router

ifconfig_em1="inet 10.11.12.254/24" # internal interface for primo

defaultrouter="99.88.77.65"

gateway_enable="YES"

cloned_interfaces="carp1 carp2"

ifconfig_carp1="vhid 1 advskew 50 pass BigPassword 99.88.77.66" # external CARP interface for primo

ifconfig_carp2="vhid 2 advskew 50 pass BigPass2W 10.11.12.1" #internal CARP interface for primo

secundo, our failover router

/etc/rc.conf
ifconfig_em0="inet 99.88.77.69/29" #external interface for secundo, our failover router

ifconfig_em1="inet 10.11.12.253/24" # internal interface for secundo

defaultrouter="99.88.77.65"

gateway_enable="YES"

cloned_interfaces="carp1 carp2"

ifconfig_carp1="vhid 1 advskew 80 pass BigPassword 99.88.77.66" #our external CARP interface for secundo

ifconfig_carp2="vhid 2 advskew 80 pass BigPass2W 10.11.12.1" # internal CARP interface for secundo

Now I’m sure you’ve already noticed, but I want to reiterate it; the only difference in CARP interface definitions between the two hosts is the advskew assigned to each. This is how the priority of the router is assigned.

Final Notes

A practical consideration when you’re deciding whether CARP is the right choice is that any system is limited to 255 CARP interfaces. This normally is a non issue, but if you expect your network to grow beyond 128 VLANs and want to use load balanced routers, it will become an issue, so keep it in mind. (Of course, there’s a point of diminishing returns when trying to put too many segments on a single router or set of failover routers is more management overhead than just setting up another set.

Also, keep in mind that this covers only configuration of the CARP interfaces themselves. If you plan on doing any packet filtering with PF, you’ll want to set up a pfsync interface. But that’s another story.