As I’ve noted earlier, there is code available and in kernel 3.9 that provides for native RAID5 and 6 on btrfs. I have some spare time this weekend, I’m going to compile a new kernel on a virtual machine, and have a play. This post contains instructions for anyone who might want to do the same.
Start by getting yourself a new virtual machine – you can follow the instructions that I provided as part of my mail server installation tutorial here. Make sure you set aside enough disk to build a new kernel, probably in the order of a 10G file system.
Firstly, we get what we need to make a new kernel. Go to kernel.org and find the latest kernel download package, right click and copy the link as a tar.gz. Go to your new virtual machine and download it to /usr/src:
cd /usr/src wget https://www.kernel.org/pub/linux/kernel/v3.x/testing/linux-3.9-rc1.tar.bz2
Install the build tools to make a new kernel on debian:
aptitude build-essential kernel-package ncurses-dev
Unpack the downloaded kernel
tar -xvf linux-3.9-rc1.tar.bz2
Enter the kernel directory, and configure the kernel that you’re going to build. It will default with the options from your currently running kernel, check in particular that btrfs is enabled:
cd linux-3.9-rc1 make menuconfig
Set up some compilation options that will make things a bit faster – in particular setting up multiple parallel threads (which is only useful if you have multiple processors). My general experience is that compiling on a virtual is a bit slow, largely I think driven by IO. I also tend to set up distcc and ccache, which make subsequent compilations faster, but this is outside the scope of this tutorial.
Edit ~/.bashrc, and add the line
If you wanted to try ccache and distcc, then the very brief instructions are:
aptitude install distcc ccache
Edit /etc/default/distcc, and set it to start automatically. Start distcc.
Edit ~/.bashrc, and add the lines:
export PATH=/usr/lib/ccache:$PATH export CCACHE_COMPRESS=Y export DISTCC_HOSTS="server myth testserver" export CONCURRENCY_LEVEL=8 export CCACHE_PREFIX=distcc
Give effect to these environment variables by reading in ~/.bashrc (or logging out and in again):
Then compile the kernel:
Install the kernel, which will be in /usr/src:
dpkg -i linux-image-3.9.0-rc1_3.9.0-rc1-10.00.Custom_amd64.deb mkinitramfs -o /boot/initrd.img-3.9.0-rc1 3.9.0-rc1 update-grub
Shutdown your virtual machine, and attach some block devices to it. In my case, I just created 6 new logical volumes. Not much use for performance testing, as they’re all on the same raid device. But lets me play around with resilience.
Start up your virtual machine, and check it’s running the new kernel:
Next, get the btrfs-progs code, and compile and install them. Instructions are derived from https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories:
aptitude install uuid-dev libattr1-dev zlib1g-dev libacl1-dev e2fslibs-dev libblkid-dev cd /usr/src git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git make make install
Then, create a set of btrfs devices.
mkfs.btrfs -d raid6 -m raid10 -l test_filesystem /dev/vd[bcdefg]
Mount the filesystem.
mkdir /home/testbtrfs mount /dev/vdb /home/testbtrfs
Put some data into it:
cp -Raux /usr/src/* /home/testbtrfs
Check file system usaqge:
btrfs filesystem df /home/testbtrfs
Empty it again
rm -rf /home/testbtrfs/*
Remount with compression turned on
umount /home/testbtrfs mount -o compress /dev/vdb /home/testbtrfs cp -Raux /usr/src/* /home/testbtrfs btrfs filesystem df /home/testbtrfs
Remove a drive from the filesystem (should rebalance the data across the other devices)
btrfs device delete /dev/vdc
Check your data is still there, then add the device again (should commence a resync)
btrfs device add /dev/vdc /home/testbtrfs
Pull the device at the virtual machine level – delete one of the devices from virt-manager, and reboot. That should give you a degraded file system. Check things still work.
I’m sure there’s lots more to play with – look at https://help.ubuntu.com/community/btrfs or https://btrfs.wiki.kernel.org/index.php/Mount_options for some ideas. Happy playing.