No not at work. At home I use it on my two routers, one alix and one apu board.
I ran into a huge backset with openbsd at work 3 years ago when I wanted to run two redundant load balancers with carp in vSphere. Turns out I had to enable promiscuous mode on an entire port group to make it happen. These days we use virtual switches so maybe it's easier now but in those days we did not want to do it because it would mean enabling promiscuous mode on an esx host adapter, affecting everyone on that host.
Also if you're setting up in a virtual environment keep in mind that it requires promiscuous mode enabled.
This stopped the show for me once when setting up two OpenBSD load balancers in a shared virtual environment. I was told that to enable promiscuous mode on a single port group they would also have to enable it on the physical ESX host adapter for each ESX host since the pair was separated on different physical hosts.
If that is true then I would never enable it. However networking isn't my strong side so I can't verify this.
Sad day. I picked up an ALIX board quite a number of years ago and it served as my first introduction to using OpenBSD for routing. I've been using the APU2 line since then, and I absolutely love the devices. Really great low-cost hardware that, for my purposes, has been rock solid for years.
In my company we have three Pentium 2 boxes working as routers and one working as a internal gateway (with full connectivity redundancy), we use OpenBSD, pf and the carp interface.
May I recommend PFSense rather than OpenBSD proper, all the power of PF wrapped up in a gui that lends itself directly to firewall configuration. I really enjoyed setting up CARP with it.
I've been running OpenBSD on an APU2 as a router for over a year (and on the ALIX it replaced many more years).
OpenBSD makes it so dead simple to setup the basics that (unlike with my NAS) I've never been tempted to try a GUI distribution such as pfSense, yet it also gives you the flexibility to do more esoteric things like forwarding DNS using DNSCrypt, or allowing UPnP, but only to the PlayStation on a VLAN that can't talk to your main network. And the PC Engines gear works perfectly with OpenBSD.
I remember years ago we had a problem with pfSense because the way FreeBSD had implemented carp wasn't quite correct (WRT failover and groups of interfaces, IIRC). We had been relying on specific documented behavior in OpenBSD as we deployed OpenBSD firewalls, and whenwe switched to pfSense this bit us. There were workarounds at least.
It's hard to criticize low cost solutions like this; if they work, they work! If there was slightly more control over the network infrastructure, I'd choose to go with CARP as my failover mechanism. It is probably one of the best OpenBSD features.
I used to use openbsd on my apu-2, but since I upgraded to a gigabit wan connection I had to move to Linux as I just couldn't make it do line speed gigabit :((
Thanks for the explanation. Out of sheer curiosity (and because I run BSD-based pfSense and FreeNAS at home), how do you handle floating IPs and that sort of thing on OpenBSD?
Yeah, but most of the things that need exclusivity or consistency are relatively rare or very narrow.
You need consitency when you update the socket tables. (Relatively rare, but can be a bottleneck for use cases with frequent open/close and maybe accept)
You need exclusivity when send or recieve on tcp sockets. (Very narrow)
You also need exclusivity when putting packets one of on the NIC's sendqueues. That one's not really rare, better NICs have more queues for less contention though.
For really high volume packet processing, it's helpful to align socket processing so everything is pinned to the same core: nic interrupts, kernel rx, application read and processing, application send, kernel tx, etc. But last I checked OpenBSD doesn't do cpu pinning, so there's that.
Same, but I'm using Open vSwitch[0] to tag internet traffic which goes to the router VM. I was using IOMMU but on non-Intel NICs the interupts were killing the throughput in OpenBSD.
No I don't think you are, at least for open source production grade. Intending to do some work on this when I get a chance. There is a talk at Eurobsdcon this year on using the FreeBSD network stack in userspace and Netbsd also an option, fixing locking and interrupts are issues with using these stacks out of kernel to get good performance as the environment is rather different.
I'm an OG OpenBSD user (literal 386 firewall) so I've been using it at home since before CARP was released.
When the first CARP release hit I immediately set it up on a pair of SUN Ultra1 pizza boxes I had gotten on eBay after the .com crash (with a third cold-spare) and ran that way for years. My ISP even called me at one point to find out what "those weird mac addresses" were on the SUN hardware.
They, of course, ended up being too power hungry and I moved to PC-Engines Alix boxes. When I wanted more horsepower as my internet speed increased, I moved first to a pair of PC-Engines APU boxes, and now to one APU and one virtualized OpenBSD firewall.
CARP has always been rock solid throughout, both on the internal and external interfaces of my firewalls.
Rolling reboots for patches, updates, OS upgrades or hardware failures are a non-issue. No one in my family ever notices. Combine that with multi-homed ISPs and the internet at my house is more solid than a lot of enterprises and there's no expensive hardware or software involved.
I guess that was a long-winded way to say: Home consumers CAN benefit from high availability! It just isn't packaged in an easy to use or cheap enough form factor for them.
Likely more than that seeing as how the majority of the load balancers and many of the routers / switches out there run some form of BSD (netscalar) or Linux (Arista), etc.
Not to mention that PFSensenormalizes a lot of management. It provides an easy mechanism to queue changes and apply them, lof when changes were made and what the changes were, etc.
In a previous job we used to deploy OpenBSD firewalls to provide site-to-site VPNs. We switched to PFSense because management was easier when you have 10-20 of them to deal with, and multiple people might have access.
That said, we definitely would have preferred an equivalent interface on top of OpenBSD instead of FreeBSD. There were some differences in the CARP implementation in FreeBSD that made some features of CARP we relied on with OpenBSD unavailable in FreeBSD, and thus PFSense.
I ran into a huge backset with openbsd at work 3 years ago when I wanted to run two redundant load balancers with carp in vSphere. Turns out I had to enable promiscuous mode on an entire port group to make it happen. These days we use virtual switches so maybe it's easier now but in those days we did not want to do it because it would mean enabling promiscuous mode on an esx host adapter, affecting everyone on that host.
reply