gears
within
gears

Both frameworks have their niche, and if you’re trying to gain some context around the debate that’s currently raging over Ruby testing frameworks (when isn’t there a debate raging over that?) then you’re not alone. I’ve used and avoided both of them at various times, and been passionate, and ex-passionate, and various shades of indifferent. If you’re trying to decide whether or not to pick them up and add them to your toolkit, here’s what I think you need to know. You’re welcome.

RSpec

RSpec is a complex framework. There’s no getting around that. It offers two benefits: it lowers the conceptual barrier to learning good testing practices for beginners, and it provides certain features that are helpful to people who write lots of tests. Unfortunately, it also chooses to implement some of those features in ways that have left me scratching my head in the past, although perhaps that’s changed in RSpec 2.

But I like it. I’m willing to own that complexity myself, and in return I get a lot of features that I value, because I intend to write (and must thus maintain) a large test suite. I don’t do this because I’m so in love with maintaining test suites that I want to marry them and make little baby test suites (accidental baby test suites don’t count). I do it because it helps ensure the quality of the work I deliver to my client. Trade barbs about big test suites all you want, but this matters.

I’ve also used it as a coaching tool for people who are experienced with Ruby but not experienced testers. They generally liked it for that purpose, and so did I.

It’s not the right choice for every team. The same features that make it a great coaching tool for people who are new to testing also make it a difficult choice for people who are new to Ruby, or teams who can’t afford to own additional complexity, which they will eventually have to do. I’ve advocated against its use on teams in the past and I don’t regret that. There’s a lot to be said for the simplicity of Test/Unit and its variants. Shoulda is the best of these. I recommend it unconditionally over the framework that Rails ships with.

Cucumber

Like RSpec, one of Cucumber’s main reasons for existence is to shift the perspective of the people who are using it. It is an effective and beautiful tool in that regard, and many people love it as a result. Its big idea relies on a template that goes something like this: “As a [user], I would like to [do something], so that [reason why].” This is a common Agile story template and it’s been around for years. (For more, see Mike Cohn.) When used correctly, it’s effective at encouraging software implementors to think like software consumers. When ThoughtWorks puts together a team, most of our BAs try to write stories that follow this template.

This idea, and the template system built to support it (Gherkin), is great because it’s easily adaptable to a wide range of different languages and toolsets. I’m proud that the Ruby community embraced it so wholeheartedly, because in principle it could have been written for any stack (and variants of the same idea had existed for a while in the Java community without achieving much mainstream support). It’s the sign of a vibrant community, open to new ideas and experimentation.

If you’re not accustomed to that template, or don’t have a support team of analysts, Cucumber is a valuable tool. As a piece of software advocacy for a more user-centric approach to writing enterprise software it is without peer.

Note that Cucumber is not designed to compete directly with RSpec. If you’re trying to write controller tests in Cucumber, you’re probably doing it wrong. Tests written in Cucumber are meant to address the system as the user sees it. Tests written in RSpec are (generally) meant to address the system directly, via its API, the way the developer sees it. It’s perfectly acceptable to write acceptance tests in RSpec as long as you keep in mind that it helps to structure them from a user’s point of view.

Its main weakness is that testers who use it are sometimes encouraged to write their acceptance tests at the same level of granularity that an analyst might write their acceptance criteria on a story card. This seems like a really cool idea until you wake up one day to find that you have a lot of very slow acceptance tests that don’t necessarily match the functionality you’ve built and now you have to find a way to either scrap it or beat it into shape again. This is not fun, so don’t do it.

Handshake

Don’t use it. Ever.

And that’s all I have to say about that.