So, on a lark, I decided I'd try out NetBSD. It'd been years (10+) since I last tried it, so I figured, "why not?". It's the only easily xen-DOMU-capable BSD that I'm aware of. I've run FreeBSD i386 in a DOMU before, but it was a kludgy mess and liked to crash and not recover gracefully so I wrote it off. I'm at a point that I don't really want to work with Linux if I don't have to. It's not because I hate it, or because it's inferior, but I just like Unixes better.

So, as of right now, I'm running NetBSD, with a custom-kernel (I built it to support PF, as the Xen3-DOMU kernels from NetBSD don't have the module built in and, according to docs, will die if you try to load it) on a 2048MB Linode. It's pretty nice.

That being said, I've already noticed a couple of things that I got really accustomed to with FreeBSD and miss with NetBSD - First, freebsd-update is awesome. Automated fetch/install/patch operations have spoiled me. I realized after deploying NetBSD 6.1.4 that it was running vulnerable OpenSSL libs. I checked the security advisory from NetBSD and it said "rebuild from sources". Okay, surely they have binaries available for lazy people like me? Nope. The build cluster was/is down, so no binaries are available. So, this meant working with CVS (oof, haven't used /it/ in years, either.) to pull down 6.1.4's sources, so I could rebuild userland with current patches. Sysbuild is nice and automates that pretty well, except I noticed a quirk where it wasn't inserting spaces in CVS commands. I had to manually sync down CVS, then execute sysbuild, forcing it to skip CVS sync. That took about 80 minutes or so to complete. I then used sysupgrade to deploy the updated userland, noted config changes and I was done. This same thing on FreeBSD takes 45 seconds to run freebsd-update fetch && freebsd-update install.

I also miss the ports system. Pkgsrc's repository has 12,000 packages in it, but I found a number were in varying states of 'unmaintained'. Granted, everything I typically use was available and up to date, but there were a few other odds and ends I checked into that I noticed hadn't been touched in a really long time. Teamspeak is one (2.x branch in pkgsrc). Mumble is another. Nginx was a secure, stable build, so that was okay. Git was out of date (I use mercurial anyway). I did notice that htop is available in pksrc, and that's cool, as I didn't see it ported to FreeBSD and it's a favorite tool of mine for watching an active system.

I like pkgin a lot. I didn't really use pkg-ng on FreeBSD too often because I ran into too many defaults on a number of packages that required rebuilding from source. So far, everything I've installed via pkgin has been up to date and has had sane defaults.

The documentation, like FreeBSD is also really good. The handbook doesn't have as many modern examples as FreeBSD's, and I noticed that a handful of articles were written in the 5.x era, so maybe I'll help out with that when I get more accustomed to working with it.

The base system is super-lean. It came with a handful of useful tools by default (dig, finger, etc) and I was able to install my commonly used stuff via pkgin without any hassle.

Using LISH on Linode made the install pretty easy. Build a profile with an ext3 disk image configured with pv-grub kernel type and enough space to accommodate your kernel(s). Add in a grub config. Boot with the recovery distro, wget the INSTALL and XEN_DOMU kernels from NetBSD. Create your other disk image with "raw/unformatted" selected, turn off all Xen optimizations in the console. Boot that profile, tell it to load the INSTALL kernel, install stuff, configure stuff, then reboot and let it boot with the DOMU kernel and you're ready to go.

As with my experience with Linux distros on Xen (namely Arch + pacman-key init), the entropy situation sucks at cold boot. I added both rndctl=YES and rndctl_flags="-ec -t net" to my rc.conf to ensure network interfaces are being used to harvest entropy. If you don't do this, or at least execute rndctl -ec -t -net, you'll see console spam telling you it's rekeying the RNG because of insufficient entropy. After I built up enough entropy, I nuked my SSH host keys and restarted the daemon to regenerate them.

As I mentioned before, I used the sources acquired via CVS to build a pf-enabled DOMU kernel so I could stick to the packet filter I know and love. NetBSD also supports their newer NPF, as well as ipf, but since I have zero experience with NPF and limited with ipf, I figured I'd just stick to good 'ol PF. If I have the time, I might look into learning a bit about NPF because it does have some cool concepts to it.

So, uhh, that's it, I guess. NetBSD, woo.