As I’ve noted earlier, there is code available and in kernel 3.9 that provides for native RAID5 and 6 on btrfs. I have some spare time this weekend, I’m going to compile a new kernel on a virtual machine, and have a play. This post contains instructions for anyone who might want to do the same.
In this post I’m talking a bit about virtualisation and the disk caching options, how I interpret what they’re doing, and why I’m choosing the settings that I am.
This discussion is really only applicable in a world where your storage is direct attached to your host, usually as SATA disk. If you’re using network attached storage (NAS) or a full-fat SAN environment, then you’d typically attach the storage directly to the virtual machine, and therefore the host is unlikely to be providing any caching for you.
I’m thinking of downloading the new kernel and trailing it on a virtual this weekend, at which point I can maybe post some experiences / howto information.
In the meantime, what I see that’s new is an update on the FAQ page. There is support for:
- RAID-5 (many devices, one parity)
- RAID-6 (many devices, two parity)
- The algorithm uses as many devices as are available (see note, below)
- No scrub support for fixing checksum errors
- No support in btrfs-progs for forcing parity rebuild
- No support for discard
- No support yet for n-way mirroring
- No support for a fixed-width stripe (see note, below)
It looks like the fixed-width stripe is the key one. The implication is that it uses all the devices available, so if you happened to have 12 disks, it’d build a 12 disk RAID. This impacts seek times – every disk needs to seek to the right place simultaneously to get that data. Whereas if you had 12 disks and btrfs decided to build stripes using 6 disks, then only 6 disks need to seek simultaneously, and on average you can support two different IO requests in parallel.
Secondly, the balancing code doesn’t yet deal with different sized disks. It sounds like if it didn’t always use as many devices as were available then it would deal OK with mixed size disks, no doubt within limits.
All good progress. I think to use it I’ll have to download and build a new kernel, and download and build a new version of the btrfs progs. We’ll see this weekend maybe.
So, when I run top on my machines, I’m starting to see libvirtd using a large amount of memory. I’ve been working through what can be done about that, and found a couple of interesting points.
Firstly, libvirtd is the management tools only. You can restart it without impacting your running virtuals, and therefore part of the answer is to just restart it. There are a number of reports on the web of memory leaks in libvirtd, many of them appear to have been fixed in various versions, I assume there is a leak in the version I’m running, 0.9.12.
Secondly, when I restart it, it kills off my virt-manager sessions. After restarting libvirtd and starting up a new virt-manager, I was finding that my virt-manager was giving a totally blank page on one of my servers. I eventually tracked this to the fact that there was a hung virt-manager running in the background, and new sessions were just connecting to it. Killing that session and then restarting virt-manager fixed my issue.
This is the second part of the tutorial on installing a mail server, refer the overview, or hit the tutorials menu at the top, and look at the mail server tutorial category.
In this section, I explain how to install the virtualisation software on an existing Debian (or, possibly, Ubuntu) server, and create a new virtual machine. I’m assuming that you want to use lvm as the backing store, if you don’t want to do this then you can set up the virtual within a LVM volume or a file, I note where you’d do things differently if you wanted to do that. You can also use drbd as a backing store to allow you to run the same mail server across two different servers without data loss, described here. I’ll mention the point where you should look at that post if you want to do this.
You should check that you’ve installed the right OS version for your base operating system – whilst you can run virtualisation on a 32-bit Linux kernel, it’s generally better to run on 64-bit. A 64-bit host can support 32-bit guests, a 32-bit host cannot support 64-bit guests. There are also memory limitations on 32-bit that get annoying (although unlikely to matter for a mail server).
I also assume that you have a gui environment (an x server, and KDE, Gnome or xfce installed). You can do all this from the command line, but you’ll need to derive the commands for that yourself. Note that you can install the virt-manager package on another machine to manager your server, so you just need to have the gui environment somewhere, not necessarily on your server.
So, I’ve seen a number of people visiting to look at this post, which was, to be honest, not much of an instruction list. Given that there is interest (at least relative to the lack of interest in the rest of my blog!!), I’m making a much more detailed tutorial that takes you through step by step how to install a virtual machine on an existing server, install Debian Wheezy into that virtual, then install Exim, Dovecot and Fetchmail to create a secure server that you can use for personal IMAP e-mail.
This post provides an overview of the components, what each does, and why I chose them.
There are many who aren’t thrilled with the btrfs design, seeing it as a “rampant layer violation.” What does this mean, and why do I think that’s not a problem?