Gears Within Gears

At ThoughtWorks, everyone submits a code sample as part of their interview process. The comment I received from my interviewer about mine struck me, though, and it sticks with me to this day. “The code looks fine,” he said, “but what’s with all the comments?”

My introductory CS course at Northeastern University taught me a lot of things that have resonated in my experience in the industry, but we were taught one thing that, as it turns out, almost none of my colleagues agree with: that all of your methods, classes, and modules should be preceded by a simple, one-line comment that states the purpose of code unit in question. Far be it from me to gainsay Neal Ford, who believes that code comments are a smell, but there are days when I disagree: Even developers with the best intentions and training in the world can’t always write clear code.

A Brief Re-Introduction to Class Re-opening

The problem manifests itself most acutely through the Ruby mechanism of class re-opening (sometimes dismissively referred to as “monkey-patching”). You can go about it in a number of ways, and the codebase I’m working on these days has employed each and every one of them at some point or another. The advantage you gain from being able to reopen classes is that you can patch the framework you’re using unobtrusively in order to fix bugs or improve performance. The downside is that you don’t get to choose the names of the methods you’re overriding: they may make perfect sense in the context of the class you’re overriding, but none whatsoever as an isolated patch.

A reopened class looks something like this:

In the first example, we’ve patched String directly; in the second, class_eval’d it for an added layer of complexity; and finally we’ve defined our own module and added it to String after the fact, which is fact the recommended way of going about it. We can use IRB and the method method to try to figure out where our patch is coming from:

Notice that only the last one gives us a helping hand when we’re trying to figure out where this method was defined.

Overriding, not extending

Now imagine there’s a bug in the String class: a colossally poorly-designed method, something that’s significantly impacting your ability to get something done in the language. (While this doesn’t happen in Ruby very often, it’s like to have occurred at least once in the framework of your choice.) In the bad old days, you’d have had to patch Ruby directly; if you were good about it, you’d submit your patch to Matz, and it’d get integrated as part of some future release. But the language itself gives you an escape valve: override the method, and your implementation gets used.

On my current project, we do this all the time.

But what happens when the framework gets upgraded? Is this extension still necessary? Why did anyone put it there in the first place? What’s with this Subversion check-in comment? Why didn’t anyone say anything in the code?

Enter alias_method_chain

alias_method_chain is a Rails method patched into Ruby that allows you to extend a method without blowing away the old version. You can read Marcel’s post on it on the Rails blog from back in the dawn of time, two years ago, when they added it to the framework. In action, it looks something like this:

Suddenly our extended method has an identity again: it has a name that clearly indicates both the method it’s trying to extend and the feature we’re adding on top of it. Hallelujah!

Why does this matter?

The ability to re-open classes is an insanely powerful tool. I love it, not least because it never means having to write another XUtil (XmlUtil, DateUtil, StringUtil…) class again to make up for the deficiencies of someone else’s API. I could never tell you, in good conscience, not to use it.

But cleaning up becomes a bit of a mess when you have to upgrade frameworks (or language versions, for that matter) as we’ve had to do recently on our project. We carry a copy of Rails around in our vendor directory, like most projects do, but we’ve tried to be good about making our extensions externally, through class re-opening, rather than patching the Rails code directly as men and women did in days of yore. But because, out of habit, none of these extensions carry any comments, we’ve had a devil of a time trying to figure out not what they do (we can always read the code) but why they’re there, and whether we can finally, for the love of God, delete them now that we’re upgrading to Rails 2.2.

But take it from me, one developer to another, who hath had to maintain thine extension, yea, even unto multiple new Rails versions: using the tips above will help me, and those like me, figure out what the dickens you were trying to do. And for Heaven’s sake, when you don’t have the luxury of naming your methods to make it clear what they do or why they’ve changed, leave a quick comment.

I gave a presentation on mocking and stubbing in Ruby a couple weeks ago at the Atlanta Ruby Group and the maintainers of the group have been kind enough to put the video online. It’s a large and energetic group and I had a lot of fun presenting, as indeed I always have watching others present there. If you’re new to unit testing in Ruby and you’re wondering what the fuss is about, check it out and let me know what you think at btguthrie@gmail.com. Slides here, or just watch the presentation.

Some highlights, or anyway things that caught my eye, when I read the 2009 Rails Hosting survey of Ruby on Rails developers:

• Around 60% of the community has been using Ruby and Ruby on Rails for 2 years or more. It’s not yet a mature community, but there is now a reasonably large body of expertise to draw from in solving problems in and around the framework.
• Rails developers on average develop two to five new applications per year, and deploy them several times a week to several times a week.
• Most of these are probably small:
  • Generally respondents either self-host or use a small-scale VPS service like Slicehost (which I myself host this blog on; recommended.)
  • Respondents claim to be highly sensitive to the price of their hosting provider.
  • Most don’t use performance, uptime, or process monitoring.
• It surprised me to see that almost a quarter don’t use any automated deploy process whatsoever, which sounds painful. Furthermore, 5% claimed they were still using FastCGI+Apache. Crazy.
• The community has largely moved over to Git as the SCM of choice, although Github is not widely used as a centralized code repository—only a third claim that their source code is hosted there.
• Passenger has, this year, finally surpassed Mongrel as the rails server of choice.

I see a community that’s still youthful but is starting to mature. I’m still proud to be a Ruby developer, and glad to be a part of the community. Here’s hoping for more good years ahead.

ResourceFull integrates with ActionController to provide a comprehensive RESTful resource modeling and querying framework. It provides parameter queryability, paging, sorting, separation of controller concerns, multiple formats (HTML, XML, JSON), CRUD access permissions, and API metadata surrounding the resource itself. It’s opinionated but is intended to provide you with as much as possible without limiting your ability to customize its behavior.

This plugin was extracted from a series of Rails projects intended to serve as the services tier of a series of loosely-coupled applications built in a style reminiscent of the RADAR architecture.

ResourceFull lives on Github at http://github.com/bguthrie/resource_full/.

GOALS

The major distinguishing features of ResourceFull are:

• Queryability: the ability to designate certain parameters as queryable, and map them to columns and SQL queries in the underlying model. These queries chain together multiple named or unnamed scopes and can be used either for a SQL SELECT or SELECT COUNT. This functionality may be moved into a separate plugin in the future.
• Pagination and orderability: it automatically responds to requests for limit, offset, order_by, and order_dir.
• Implementation: Default implementations for HTML, XML, and JSON controller requests.
• API documentation: ResourceFull-enabled Rails apps are able to provide automatic documentation of the resources they expose, up to a point. (This is enabled in large part by the queryability functionality and other resource-level descriptors.) It’s my hope that this can eventually be consumed by a Rails resource registrar that acts as the single source of record for multiple REST engines within an organization.

EXAMPLE

This allows for the following:

/users/bguthrie.xml
/users?name=Guthrie
/users?email_address=gmail
/users.xml?city=Chicago&name=Brian,Paul,Alicia&order_by=city&order_dir=asc
/users.xml?limit=10&offset=30
/users/bguthrie/addresses

>> UsersController.to_xml
=> "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n\n  ..."

API DOCUMENTATION

To enable ResourceFull to document your resources at /resources.xml and /resources//routes.xml, add map.api to your routes.rb file.

Onewaystweet is a simple Twitter mashup that helps you track what people are saying about your site, and when. It hashes the URL (using several different URL compression services, or at any rate, the ones that offer idempotent hashes of the given URL) and performs multiple followup Twitter searches so you don’t have to.

Robert Scoble thinks Twitter isn’t for conversations. That may be true, but that don’t mean that people ain’t sayin’ stuff. Pop in a URL, subscribe to the RSS feed, and follow the results. They may surprise you. Then again, they may not, which may also be surprising. Is disappointment a kind of surprise?

Actually, the queries that result in the greatest amount of buzz have certainly surprised me. Obie Fernandez’s minor mea culpa on the Rails Maturity Model generated a lot of controversy and results in (as of the writing of this post) a fair number of hits; the current New York Times headlines story does not. Is Twitter still so tech-centric?

On the technical front, I’m currently deploying it on Heroku, which was almost absurdly simple to do. As advertised: git push and you’re done, though the site doesn’t use a database and I can’t speak to the deployment experience in that regard. The fact that the developer preview is free certainly doesn’t hurt. Their explanation of how it works is sexy but short on gritty details. I’m okay with that, because in my idealized fantasy world deployment is always Somebody’s Else’s Problem. But the bitter Rake hacker in me is convinced that not having SSH access will be a huge pain in the patootie at some point in the future.

A bunch of otherwise nice people apparently dislike Zed Shaw and call him names to his face. I have not met the man but am myself am acquainted with a number of people who are also otherwise nice people but who, having encountered Mr Shaw in the flesh, are inexplicably given over to calling him unpleasant names, though not generally to his face. I humbly posit the following, contra Zed: The simplest possible explanation for this phenomenon is not that everyone Zed Shaw meets is a jerk.

I was kicking around in ActiveResource::Base recently and it took me a good solid ten minutes to figure out why the prefix method didn’t recur infinitely.

def prefix(options={})
  default = site.path
  default << '/' unless default[-1..-1] == '/'
  # generate the actual method based on the current site path
  self.prefix = default
  prefix(options)
end

def prefix_source
  prefix # generate #prefix and #prefix_source methods first
  prefix_source
end

def prefix=(value = '/')
  # Replace :placeholders with '#{embedded options[:lookups]}'
  prefix_call = value.gsub(/:\w+/) { |key| "\#{options[#{key}]}" }
  # Redefine the new methods.
  code = <<-end_code
    def prefix_source() "#{value}" end
    def prefix(options={}) "#{prefix_call}" end
  end_code
  silence_warnings { instance_eval code, FILE, LINE }
rescue
  logger.error "Couldn't set prefix: #{$!}\n  #{code}" 
  raise
end

Do you see it? When prefix is called, it calls prefix=, which redefines prefix and returns. prefix in turn returns by calling not itself but the newly-created method of the same name.

I am not known for my low tolerance of metalanguage hackery (exhibit). But I do have my limits, and this exceeds them; calling it needlessly obfuscatory would be kind. Or am I wrong? Is there no better way to provide a method with the same semantics?

Scheme, a variant of Lisp popular in academic settings, has no download link on its extremely minimalist home page. If you click Implementations, it will list about fifty and recommend about ten. Visit the Ruby homepage and you are presented with a modern, polished, attractive site and a large, friendly Download Ruby link.

Marketing isn’t always about advertising. If you believe in a product, present it to the world in a way that makes it easy for others to believe in. It’s easy to be cynical about appearances, but they are, here and now as they have ever been, important. This principle is extended deeply into the Ruby language itself. From a technical standpoint, Ruby does very little that other languages don’t, but the metaprogramming hooks it makes available to users are straightforward and user-friendly and make its users feel immediately powerful. The elaborate contortions (do watch the presentation if you can, it’s awesome) that the language itself must go through in order to provide a convenient interface to the user doesn’t detract from the net result. Simplexity sells.

Ruby and, now, Rails have been around for long enough that it seems, at least to me, that most people who were ever going to have an opinion on it now do so. But accusations of hype are and always have been overblown; I would go so far as to call that particular brand of cynicism a kind of intellectual laziness convenient to those don’t understand a phenomenon and don’t particularly want to. Regardless of whether or not Ruby and/or Rails are here to stay (and I think they are, but then, I’m an optimist), it would be foolish to ignore the lessons here.

I’ve been knocking around ways to speed up ActiveResource recently. My colleague Tim Cochran discovered that you take a big performance hit from Hash.from_xml (which uses XmlSimple to transform the parsed XML into a hash with a particular structure) and I’ve knocked up a couple of alternatives, one for libxml-ruby and one for Hpricot. I haven’t packaged them up into a plugin yet, but I’ll attach the raw source, and hopefully that’ll help someone.

There’s already at least one other attempt to adapt libxml to ActiveResource floating around, but as far as I can tell it doesn’t work: it doesn’t place the resulting XML in a form that allows ActiveSupport to successfully undasherize the keys, deserialize primitives by their type attribute, and place the result in an ActiveResource object (though it is of course possible that I configured the library incorrectly). I’ve tested the parsers below against the corresponding XmlSimple call and I know that they succesfully deserialize into the desired end result.

• libxml
• hpricot currently broken (will fix if requested, 1/31/09)

I’m at RailsConf this weekend, capping a six-months engagement delivering a couple of Rails apps written using NetBeans as our team’s primary Ruby editor. We ended up with it because the client requested that we do our all development work in Ubuntu 6.06, and because we couldn’t build a consensus around a more lightweight editor like Vim or jEdit. I thought I’d share some of my thoughts on it for anyone’s who’s interested in switching.

Executive Summary

There’s a lot to love and a fair amount to hate, but it’s a solid alternative to TextMate if you’re not on a Mac.

Stuff I liked

• The basic refactoring support is surprisingly handy. Until I got back the ability to rename classes and methods I hadn’t realized I’d missed it. I’m told that Aptana can perform similar tricks, but it’s been a while since I’ve tried it out.
• Likewise, the ability to jump directly to a class definition is handy, although it isn’t perfect. (If you have more than one class of the same name defined but located in different files or scoped to different modules it can’t figure out which one, exactly, you’re referencing, which I suppose is understandable.)
• RSpec support! A must-have. And the ability to jump back and forth between test and class… most of the time.
• If you’re a Vi user, the jVi plugin is almost worth the price of switching purely on its own. It’s a truly excellent piece of software, with an enormous amount of tweakability and really support for Vi constructs, and one of my complaints with Vim itself has always been that it lacks good project/directory browsing support, and yes, I’ve tried the Project plugin… again and again and again.
• It knows to insert an “end” when I hit enter after a block or method definition. Just a stupid small thing, but I miss it now when I don’t have it.

Stuff I didn’t like

• It opens up a new output window each time you hit Shift-F6 to run a test, leading to a million open output windows after you’ve been using it for a while. This is an enormous pain in the ass, and it’s compounded by the problem of really poor tab management support in the section of the IDE that displays your previous test runs. This is my single biggest complaint.
• The “jump to file” functionality doesn’t insert automatic wildcards the way TextMate does. This drives me absolutely nuts and it’s something I miss desperately.
• Method autocomplete doesn’t work well and is far more trouble than it’s worth. I turn it off almost immediately.
• It does not, and here I am being charitable, have the most attractive design on the face of the planet. I realize that by stating this I am probably slagging the work of some diligent UI designer somewhere, and I realize that you have to work within the constraints of Swing themes across multiple platforms. But it doesn’t stack up well next to the clean design of TextMate, or the brand new, achingly sexy (YMMV) MacVim.

Lessons Learned

• Use a nightly build, and keep it fresh. It’s the only way to get support for stuff like RSpec. The nightly builds have been stable and make for happy funtimes.
• Spin up your dev server on the command line rather than using the builtin server runner. Maybe it’s just me, but using an IDE for this just doesn’t seem worth the trouble.

If you’ve spent any time working with Ruby in NetBeans I’d be curious to hear about it.