Compiling and testing RAID6 on btrfs

As I’ve noted earlier, there is code available and in kernel 3.9 that provides for native RAID5 and 6 on btrfs. I have some spare time this weekend, I’m going to compile a new kernel on a virtual machine, and have a play. This post contains instructions for anyone who might want to do the same.

Start by getting yourself a new virtual machine – you can follow the instructions that I provided as part of my mail server installation tutorial here. Make sure you set aside enough disk to build a new kernel, probably in the order of a 10G file system.

Firstly, we get what we need to make a new kernel. Go to kernel.org and find the latest kernel download package, right click and copy the link as a tar.gz. Go to your new virtual machine and download it to /usr/src:

  cd /usr/src
  wget https://www.kernel.org/pub/linux/kernel/v3.x/testing/linux-3.9-rc1.tar.bz2

Install the build tools to make a new kernel on debian:

  aptitude build-essential kernel-package ncurses-dev

Unpack the downloaded kernel

  tar -xvf linux-3.9-rc1.tar.bz2

Enter the kernel directory, and configure the kernel that you’re going to build. It will default with the options from your currently running kernel, check in particular that btrfs is enabled:

  cd linux-3.9-rc1
  make menuconfig

Set up some compilation options that will make things a bit faster – in particular setting up multiple parallel threads (which is only useful if you have multiple processors). My general experience is that compiling on a virtual is a bit slow, largely I think driven by IO. I also tend to set up distcc and ccache, which make subsequent compilations faster, but this is outside the scope of this tutorial.

Edit ~/.bashrc, and add the line

  export CONCURRENCY_LEVEL=8

If you wanted to try ccache and distcc, then the very brief instructions are:

  aptitude install distcc ccache

Edit /etc/default/distcc, and set it to start automatically.  Start distcc.

Edit ~/.bashrc, and add the lines:

  export PATH=/usr/lib/ccache:$PATH
  export CCACHE_COMPRESS=Y
  export DISTCC_HOSTS="server myth testserver"
  export CONCURRENCY_LEVEL=8
  export CCACHE_PREFIX=distcc

Give effect to these environment variables by reading in ~/.bashrc (or logging out and in again):

  source ~/.bashrc

Then compile the kernel:

  make-kpkg kernel-image

Install the kernel, which will be in /usr/src:

  dpkg -i linux-image-3.9.0-rc1_3.9.0-rc1-10.00.Custom_amd64.deb
  mkinitramfs -o /boot/initrd.img-3.9.0-rc1 3.9.0-rc1
  update-grub

Shutdown your virtual machine, and attach some block devices to it. In my case, I just created 6 new logical volumes. Not much use for performance testing, as they’re all on the same raid device. But lets me play around with resilience.

Start up your virtual machine, and check it’s running the new kernel:

  uname -r

Next, get the btrfs-progs code, and compile and install them. Instructions are derived from https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories:

  aptitude install uuid-dev libattr1-dev zlib1g-dev libacl1-dev e2fslibs-dev libblkid-dev
  cd /usr/src
  git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
  make
  make install

Then, create a set of btrfs devices.

 mkfs.btrfs -d raid6 -m raid10 -l test_filesystem /dev/vd[bcdefg]

Mount the filesystem.

 mkdir /home/testbtrfs
  mount /dev/vdb /home/testbtrfs

Put some data into it:

  cp -Raux /usr/src/* /home/testbtrfs

Check file system usaqge:

  btrfs filesystem df /home/testbtrfs

Empty it again

  rm -rf /home/testbtrfs/*

Remount with compression turned on

  umount /home/testbtrfs
  mount -o compress /dev/vdb /home/testbtrfs
  cp -Raux /usr/src/* /home/testbtrfs
  btrfs filesystem df /home/testbtrfs

Remove a drive from the filesystem (should rebalance the data across the other devices)

  btrfs device delete /dev/vdc

Check your data is still there, then add the device again (should commence a resync)

  btrfs device add /dev/vdc /home/testbtrfs

Pull the device at the virtual machine level – delete one of the devices from virt-manager, and reboot. That should give you a degraded file system.  Check things still work.

I’m sure there’s lots more to play with – look at https://help.ubuntu.com/community/btrfs or https://btrfs.wiki.kernel.org/index.php/Mount_options for some ideas. Happy playing.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s