Gears Within Gears

A couple of days ago Ola Bini pulled my proposed ISpec mocking framework into Ioke (and did me the kindness of porting a change I made to Ioke/J into Ioke/C# – Number Infinity). It clocks it at around 300 lines of code (twice as many of test code) and I’m pretty pleased with the results, although several aspects of it could bear some refactoring.

Syntax

In its simplest form, it allows you to add simple mocks and stubs by method name:

For more BDD-friendly syntax, it’s expected that you use should receive syntax, which supports more advanced mock creation constructs:

The following method call count assertions are supported:

• never
• once
• twice
• atLeastOnce
• anyNumberOfTimes
• times(n)
• times(n..m)

Return arguments given as a sequence will be returned in the sequence they are given for each subsequent call to the mock:

The mocking support is still a work in progress. Significant omissions include friendly mock names, a more unified creation syntax, and more complex argument matchers (for example, Regex support: user should receive name(#/Guybr/)).

Design goals

I had basically two goals in mind for the syntax of mock creation:

• I wanted something that played nicely and looked good with ISpec, and in this respect I drew some familiar syntax constructs from RSpec mocking.
• I wanted a great deal of flexibility. In this I drew far more from Mocha than I did for RSpec, whose mocking framework I find continually limiting.

But in particular, I wanted to use the power of Ioke’s macros to allow me to specify complex mocks in a more concise way. In Mocha, being a Ruby library, you can specify your mocks as simple key-value pairs in a hash, but if you want to build a more complex mock you must break off into a new line and start constructing it using separate calls. Ioke allows me to chain multiple expectations implicitly against the same should receive call, and even to apply the same expectation to an entire group of them:

Next steps

I haven’t used the macro capabilities to their full power, and it shows in the code: much of it could reasonably be made much, much tighter through the use of macro syntax rather than simply leveraging the macro construct to manipulate conventional data structures. In particular the lack of that syntax manifests itself in a dismaying lack of consistency between different syntaxes for constructing mocks and stubs; unifying that is the number one thing I want to improve, structurally, in the framework.

But, most importantly, I haven’t yet used the framework in anger for anything; simply building it has consumed most of my Ioke hacking time. I suspect that it would benefit a great deal, as everything does, from real-world use; although I am an experienced tester with mocks, I am not yet an experienced user of my own. If you’re embarking on an Ioke project and need to test with mocks, please do let me know.

Tomorrow I’ll be hopping on a plane to Las Vegas for RailsConf ‘09, which comes at an interesting time for the community. The sagging economy, rise of alternative Ruby-like dynamic languages, and recent furor about sexism (about which I and many of my colleagues are pretty upset) make for a challenging set of obstacles for the technology that we all love. I’m curious about what the general mood will be, and I’m sure I’m not the only one.

I don’t know about other companies, but I know that ThoughtWorks has scaled back our involvement in RailsConf rather considerably this year, not because we don’t still like Rails but because sending people to Vegas in the middle of the week at the expense of billable or otherwise productive work is a luxury that all too few companies can afford these days. I’m glad I get to go and hope to repay our investment by blogging frequently about the sessions and sharing my thoughts on the goings-on. If there’s anything in particular you’d like to hear about, drop me a line at btguthrie@gmail.com and let me know. You can also follow me on Twitter.

In particular, if you’re interested in hacking on Ioke while you’re there please do let me know. I think it’s too late to suggest a BOFS session, but I may try to put something together anyway.

As for the recent controversy, my own feelings are perhaps best captured by Martin Fowler’s commentary, who reminds us yet again why we value his advice so much. Liz Keogh’s post is also fascinating. I could go on with more, but so many others have already said it better than I can; suffice it to say that I’m disappointed, and admire the creation but maintain my doubts about the creator.

In Ioke, a macro is a method that receives its arguments unevaluated. Instead of receiving a series of arguments, it receives a message. All code in Ioke is represented by message chains. A message looks something like this:

Take a look at that call to each. It receives two arguments, separated by a comma: the name of the variable to bind the incoming variable to, and the code for the loop. It’s equivalent to the following code in Ruby:

It’s pretty similar, except that in Ruby a block represents a special syntax form. In Ioke, calls to each follow the same syntax rules as any other kind of call, except that each is written as a macro, not a method. In the case above we don’t want to evaluate that first argument (name), because it doesn’t actually represent anything in the local scope. If you tried to evaluate it, you’d receive an error:

Macros allow a programmer to pick apart the arguments they receive and choose to evaluate them or not evaluate them, or evaluate them in particular contexts, as they so choose. They’re enormously powerful, but can be tricky to use, because without a structured argument list you’re never quite sure what you’re getting without taking a close look at the incoming message chain.

That’s where destructuring macros come in. dmacros are special kinds of meta-macros that allow you to write your code based on what you’re expecting from an incoming message. Here’s an example of a simple dmacro used by Message to allow you to turn transform any arbitrary code you give it into a message:

Think of them as a bit like multimethods: they perform the hard work of macro argument matching and allow you to write code that’s more clear and structured than it would otherwise be. Note that you still get access to the low-level call information that a regular macro would receive, so you can perform special tricks with the incoming arguments or the rest of the message chain if you’d like.

I’ve added an HTML formatter for Ioke’s testing framework, ISpec, and added TextMate bundle commands for test runners that use that formatter. I’ve shamelessly ripped off RSpec’s HTML formatter for a similar, though much less polished, style, and the results look something like this:

And yes, that’s the current mocking specs you see it running. A sneak preview: foo should receive(bar(5) andReturn(6), baz atLeastOnce, qux never) foo should not receive bar("five")

A little while back I volunteered to work on a mocking framework for Ioke when my friend Carlos suggested it might be helpful. I’m a bit embarrassed that I don’t have more to show for myself after all of my flailing about, but in my defense, it’s been a spare-time sort of a thing. If you’d like to follow my progress you can do so on my Github fork of Ioke.

Building a mocking framework for Ioke has raised a number of interesting questions, which I’ll run through here. If you have any suggestions or questions, I’d love to hear them.

Is full-bore mocking worth it when prototyping makes it so easy to stub?

Ioke is a prototype-based programming language: any object can mimic any other object. You can declare a constant by capitalizing the name of your mock (Foo = Origin mimic) or an “instance” of that constant by lowercasing the reference (someFoo = Foo mimic). Mimicking an object adopts all of its cells, which is to say, its data and behaviors.

if you would like to instantiate an object you need to ensure that the mimic begins life with unique data structures, because if you don’t it will share the same references as its origin. The convention in Ioke as it stands currently is to declare a create method: someFoo = Foo create. That method will often look something like this:

Foo = Origin mimic do(
  create = method("Creates a new Foo", self with(objects: [], name: "A name"))
)

The advantage of this approach is that stubbing is trivially easy: you can mimic any object, any time, and replacing the behavior of one of its cells is as simple as redeclaring it:

someFoo = Foo create          ;; someFoo objects is now an empty list
someFoo objects = set()       ;; someFoo objects is now a set
someFoo objects = method(...) ;; And so on.

So, why do you need a mocking framework?

Steve Yegge, in his article about what he terms the universal design pattern, praises JavaScript (a similarly prototype-based programming language) for making it so easy to test Java classes. This doesn’t quite get at the whole story; mocking frameworks offer at least three distinct advantages over simply replacing the method call:

• They can set precise expectations in terms of method arity, arguments, and order of invocation, though I confess that I don’t often use the more advanced features of most mocking frameworks.
• They can enforce certain rules about the mockability of particular cells; for example, that you don’t set an expectation on a cell that doesn’t actually exist. I wish I used these features much, much more often than I do.
• They can replace the original cell (method) definition when the test has finished running, which is extremely important if you want to stub out behaviors related to domain models that might be reused in other tests.

All of those seem like things worth having.

Should the existence of macros suggest a different API for mocks?

I started out by following a fairly convention syntax:

foo should receive("bar") with(:qux) andReturn(5)

Ola suggested that I deploy a little macro-foo in support of readability, and that’s the way it stands right now:

foo should receive bar(:qux) andReturn(5)

One could imagine further refinements on the form; if you have any, as always, drop me a line.

This syntax bothers me a bit because it runs so counter to what I’m used to seeing in languages where every argument is eagerly evaluated. In the above example, we expect the cell “bar” to be invoked with an argument of :qux. Now, clearly we don’t actually want to invoke the bar method directly; it doesn’t exist right now, certainly not as a receiver on any object that makes any sense. But what about :qux? What if the above had been written as:

expectedArgument = :qux
foo should receive bar(expectedArgument) andReturn(5)

Clearly we need to evaluate the arguments to that method call but leave the call itself alone. Weird? I don’t know. You tell me.

What inspiration can we draw from frameworks in non-prototype languages?

I’ve found that I do my best learning through experience, and so didn’t make a serious attempt to understand the internals of other mocking frameworks; having used them extensively, I figured, I have a pretty good idea of their capabilities, and so dove in unencumbered by reason, research, or even a decent working knowledge of Ioke. It’s been tremendous fun and as usual I’ve gone through my share of pain, but one of the nicest parts has been actually spending time inside Mocha itself after it became clear that I needed a bit of guidance.

It turns out that prototyping doesn’t get you all the way there precisely because there are some features of a mocking framework, as suggested above, that require some deeper thinking about expectations. How do you know if one is satisfied? How do you pick from competing calls with different argument expectations? Multiple or infinitely allowed invocations? Sequenced return values? Block yields? And so forth.

Although there are pieces of it that feel vaguely stitched-together, as though Mocha and Stubba never quite met at the seams, it’s been fun code to read. Mocha’s a good example of a library that does something complex in a reasonably straightforward manner. The methods are brief and readable, and though there’s a bit more indirection than I’d like (what’s the difference between verified? and satisfied? What does it mean to match? or invoke? How do you pick apart three different mock methods in three different places?) I’d happily recommend it to others as a good example of how to pull off some hairy Ruby functionality without writing a whole lot of hairy Ruby code in the process.

What do I mean by that? Here’s the main expects method that gets mixed in to the Object class:

  def expects(symbol)
    mockery = Mocha::Mockery.instance
    mockery.on_stubbing(self, symbol)
    method = stubba_method.new(stubba_object, symbol)
    mockery.stubba.stub(method)
    mocha.expects(symbol, caller)
  end

Each one of those lines is pretty dense: it requires a lot of backtracking to understand why it’s there and what it does. But there aren’t many of them, and they can all be explored methodically, and understanding one leads to understanding the next. Much of Mocha is like that: moderately sized, neither opaque in its density or transparent in its verbosity. Nice, I suppose, for my own definition of nice.

I’ve also learned a lot about the framework that I didn’t know before (for example, I didn’t know that it was possible to configure Mocha to warn against or even disallow mocking nonexistent methods) and will bring that knowledge with me to future projects.

But I don’t regret having not read the source first. Trying and failing gave me the context I needed to understand why Mocha made some of the choices it did, even if I don’t agree with all of them, and in the coming days and weeks I’ll be stealing drawing inspiration from most of them. Thanks and kudos, guys.

Ola Bini just released the latest version of his Ioke programming language, Ioke S, and I’m having a blast using it. Late last year I started playing around with Clojure, which is a ridiculously promising language with a compelling feature set that you should absolutely check out if you’re interested in the future of Lisps either on the JVM or off it. (I should mention that Stuart Halloway’s Programming Clojure has been my trusty and helpful guide in this endeavor; recommended.)

But as for me, Ioke has stolen my heart: it possesses the syntactic simplicity characteristic of homoiconic languages (and hence the power of macros) paired with the developer-friendly attitude you’d expect from someone who’s spent a lot of time with Ruby.

(I should note at this point that, although Ioke is deeply inspired by Io and Smalltalk, my experience with those languages is limited strictly to reading their code samples. Render unto them all due credit and so forth. I’ve spent much more time with Ruby and to a lesser extent Lisp derivatives.)

It isn’t the fastest JVM-hosted language by any stretch of the imagination, and it doesn’t (yet) have the most compelling feature set. But it’s a lot of what I love about Ruby without any of the syntactic drama, which, with a few wrinkles, is more or less a good thing. And in true open-source style, it’s evolving live, in front of the world, and it’s got a great dev community hacking away on it if you’re looking for something to contribute to.

If I had to point to just one thing that I love about it, ISpec is Ola’s RSpec -inspired BDD testing framework included with the Ioke distribution as one of the first core libraries written entirely in Ioke. Three great things:

• Message-style chained calls mean that it’s absolutely gorgeous to read.
• That same philosophy means that its internal semantics are much, much simpler than RSpec’s, which means that it’s easier to extend. I’m working on adding a mocking framework to it, and tacking “should receive” onto the existing infrastructure was close to trivial.
• ISpec’s code documentation library, Dokgen, integrates seamlessly with the code and ISpec to produce API docs that tie directly into the specs. Go browse Ioke’s API and you’ll see the tests that map to a particular method tied directly into its documentation. I believe strongly in testing the dickens out of your code, and to see the live specified behavior tied directly to method documentation is just awesome.

Stop by the website or the mailing list and let Ola know what you think.