So, I’ve been mulling over the performance of Rspec today. I know I’ve put too much into my controller test cases – I have chosen not to stub out all the database work, and this means that each test case creates a lot of test data, and executes a lot of transactions on the database. My current test suite for one controller runs in about 4 minutes. I’ve been working on what I can do to improve that.
My first inclination was that my database is very slow – I’m running on a virtual and for a variety of reasons the IO performance is poor. I can see this when running top against the data server when the tests are running – it has very high IO wait.
I googled around, and saw some suggestion that typically it’s not really the database that is the problem – lots of people changing to in-memory databases and finding little if any improvement.
One guy was pretty clear that Devise, and password hashing, was the problem. But after spending some time on that, it became clear that the later versions of Devise already set the “stretch” value to 1 in the test environment, which was the main culprit.
Anyway, to prove or disprove this, I set about running my mysql instance from a ramdisk (tmpfs). This is harder than it looks!! I wanted really to have just my test database in the tmpfs, but that proves difficult because InnoDB by default puts all the tables into a single file – including other databases that I didn’t want to lose when I rebooted.
I found the options to control this, and asked mysql to create a separate file for each table. This slowed my test cases down even more dramatically – I was now running in 12 minutes. But I thought it gave the potential to push just this database onto a separate partition, which was a mounted tmpfs. Unfortunately, it looks like the option I want for that is not available until mysql 5.6 – the “data directory” option on table creation. So that line of thought eventually proved fruitless.
Next, I moved my entire database onto a tmpfs (after taking a backup). This let my test suite run in more like 30 seconds, which is a massive improvement – and proves that IO is the underlying problem in the speed of it. Unfortunately, it’s also not a sustainable solution.
So, now I’m looking for an option that allows me to run the database fully in memory without losing the stuff from mysql that I like – the transactions etc. I see some dedicated in-memory database options purely for testing, but I may go back as I was for a while while I think about it.