Erudition and Inanity

I see a fair amount of confusion with services over developers returning incorrect or incomplete HTTP failure codes from their RESTful controller actions, so here’s a quick reference to some of the most commonly-used codes in the 400 range and their specific use in a Rails context.

These are all the codes I’ve used in my RESTful services, but if you know of any others that I’ve missed post them in the comments or send me an email and I’ll incorporate them into the table above.

Also, it’s worth noting that REST, based as it is on the HTTP we know and love, is supposed to be reasonably human-friendly. In my opinion it’s not necessary to adhere strictly to the above conventions (although I try to) as long as the human beings trying to use your service understand why their requests failed and what they can do to fix them.

This is part of a series of posts on Rails and REST. Read the rest.

When you’re writing a RESTful web service designed to expose a resource to the outside world, the case for including first-order associations is pretty strong. ActiveRecord’s default serialization method, to_xml, makes it pretty easy: simply pass in the :include key and the list of associations you’d like to nest, and the XML is automatically generated for you. But it can kill performance pretty quickly if you’re not careful. Consider a User resource, which includes, as a first-order association, the Address of that user. The XML will look something like this:

This is fine for a single resource, and probably what I want to see when I ask for /users/gthreepwood.xml. But when I ask for /users.xml?job_title=Pirate, and I’m getting back a list of resources that can stretch into the dozens or hundreds (there are a lot of pirates on Melee Island) it can start to drag quickly. You’re asking for more data from your database server (be sure you’re including the associations not only on the to_xml call but in the initial find call as well), it takes longer to serialize each object, and it takes longer (often much, much longer) for the client to parse that XML.

Worst of all, your client probably doesn’t need the full address. There’s plenty of precedent in the web world for this; when you’re displaying a list of things you probably want an abbreviated view, but when you’re looking at a specific resource you probably want a much richer view. You’re also designing an API that forces the client to reach through multiple layers of an object in order to dig out the one piece of information they need.

Instead, follow the Law of Demeter. If all the client of your resource needs when displaying a list of users is a text description of their street address, consider adding a method to your model that provides it:

def full_address
  "#{address.street_address}, #{address.city}, #{address.state} #{address.zip_code}" 
end

And use it in your controller as follows:

respond_to do |format| 
  format.xml do 
    render :xml => @users.to_xml(:methods => [:full_address])
  end
end

By my count, the former representation comes to 282 bytes and the latter to 200—a savings of almost one third. Multiplied over a large XML document with multiple resources that translates into a big win for both network bandwidth and XML parsing speed.

There’s one hitch, and this applies anytime you include a method as part of your XML. If the client tries to modify that resource update it by POSTing it back to your server, and you handle it by passing params[:user] straight into the update_attributes method of the relevant User object, your model will complain that it doesn’t know anything about a full_address= method. You’ve provided a getter but no setter.

There are two ways around this: either remove the offending parameter from the attributes you pass into the User model in your controller, or create no-op setter on your model. I personally prefer the latter, simply because I like my controllers to do as little as possible (and for the record, I think that models should either know how to serialize themselves to XML or that responsibility for doing so should be offloaded to an ERB template, but that’s another post). But either works as long as you’re aware of it and handling it appropriately.

This is part of a series of posts on Rails and REST. Read the rest.

Potential customers can be hard to come by, especially since you’re building something that nobody will ever, EVER want. Well, that’s getting ahead of myself. Let’s just agree that when you’re setting out to Gather Business Requirements, your potential customers usually aren’t already sitting in the room with you. [link]

I’m not sure what to say. At the intersection of people who need shit built but can’t program and people who know how to build shit but don’t know the domain you have business requirements. Steve Yegge seems to think that business requirements are something you gather using focus groups, and loathes them; on the types of projects I work on, we typically gather them from the domain experts and we build them for people who need shit built desperately badly. The fact that they grow out of line with what you need is an immediate consequence of the fact that his teams have tried to Do It All Up Front rather than iterate and improve, and this coming from a man who loathes Agile but just as clearly doesn’t understand that any better than he understands business.

A curious, inquisitive, and passionate person can find the passion in virtually any domain they’re asked to turn their keyboards to; in the past year I’ve had the opportunity to work on a wide variety of projects in areas that I never thought I’d enjoy. It’s sad to see that Yegge seems to be advocating a kind the kind of intellectual blinders that confine you and your work to the things you’re already interested in and understand, all the more because it seems to run counter to a lot of what he’s written so eloquently about in the past.

I know that I’m late to the party here, that all of the cool kids have had accounts for what seems like Internet-decades, and that I can do nothing but add my voice belatedly to an already rapturous chorus; nevertheless, forgive my enthusiasm: GitHub rules, it is worth joining, and, once signed up, it took only a week for my workflow to be forever improved. I am, in short, a believer.

• It’s never been easier or more convenient to contribute to other open source projects.
• It’s never been easier to work remotely.
• It’s never been easier to track the progress of projects and developers you’re interested in.
• Unicorns! Rainbows! Puppies!

Over the next few weeks I’ll gradually be migrating all of my projects over to it, and I encourage you to do the same.

Scheme, a variant of Lisp popular in academic settings, has no download link on its extremely minimalist home page. If you click Implementations, it will list about fifty and recommend about ten. Visit the Ruby homepage and you are presented with a modern, polished, attractive site and a large, friendly Download Ruby link.

Marketing isn’t always about advertising. If you believe in a product, present it to the world in a way that makes it easy for others to believe in. It’s easy to be cynical about appearances, but they are, here and now as they have ever been, important. This principle is extended deeply into the Ruby language itself. From a technical standpoint, Ruby does very little that other languages don’t, but the metaprogramming hooks it makes available to users are straightforward and user-friendly and make its users feel immediately powerful. The elaborate contortions (do watch the presentation if you can, it’s awesome) that the language itself must go through in order to provide a convenient interface to the user doesn’t detract from the net result. Simplexity sells.

Ruby and, now, Rails have been around for long enough that it seems, at least to me, that most people who were ever going to have an opinion on it now do so. But accusations of hype are and always have been overblown; I would go so far as to call that particular brand of cynicism a kind of intellectual laziness convenient to those don’t understand a phenomenon and don’t particularly want to. Regardless of whether or not Ruby and/or Rails are here to stay (and I think they are, but then, I’m an optimist), it would be foolish to ignore the lessons here.

G. K. Chesterton:

[T]he truth is that nobody has any business to destroy a social institution until he has really seen it as an historical institution. If he knows how it arose, and what purposes it was supposed to serve, he may really be able to say that they were bad purposes, that they have since become bad purposes, or that they are purposes which are no longer served. But if he simply stares at the thing as a senseless monstrosity that has somehow sprung up in his path, it is he and not the traditionalist who is suffering from an illusion.

It’s easy to slag somebody else’s code. Sometimes it’s almost impossible not to. Nobody likes a big hairy mess, and if you’re the software development world’s equivalent of a neat freak, as I am, the reaction can be immediate and visceral.

Certainly not all code is great. I find good writing an apt comparison: styles differ, and reasonable people differ in their preferences, but there exist some pretty clearly-defined differences between good writing and bad. Books and articles are written to be read by humans, and good code should be too, but because code has a second, overarching goal of executability and functional completeness, the kinds of structure, syntax, and flow that are so important to a good book tend to fall by the wayside. In code, expressing intent to humans is nice; in writing, it’s everything. Not every line of code receives that kind of care and attention, and when you find yourself staring the kind of tangled mess that promises whole weeks of fun to tease apart it’s hard not to lay a finger of blame on its human stewards. I myself prefer a kind of good-natured mockery when I stumble across something particularly egregious and a have a good guess about the person¹ (or, more likely when pairing, persons) responsible.

Even at ThoughtWorks, where we all like to think that we’re pretty decent coders², I haven’t been thrilled with every line of code code I’ve stumbled across. I find the above quote helpful because, substituting ‘social institution’ for ‘line of code’, it points the way to humility: people generally do things for a reason, and they may be drawing from painful experience that you yourself haven’t had the benefit of, not to mention the kinds of tight deadlines that we’ve all been under before. Resist the urge to rip it all apart until you understand it well enough to approach it with respect and humility.

¹ Sarnacke. Kidding! Or am I? Hmm.

² On our writing I plead the fifth.

I’ve been knocking around ways to speed up ActiveResource recently. My colleague Tim Cochran discovered that you take a big performance hit from Hash.from_xml (which uses XmlSimple to transform the parsed XML into a hash with a particular structure) and I’ve knocked up a couple of alternatives, one for libxml-ruby and one for Hpricot. I haven’t packaged them up into a plugin yet, but I’ll attach the raw source, and hopefully that’ll help someone.

There’s already at least one other attempt to adapt libxml to ActiveResource floating around, but as far as I can tell it doesn’t work: it doesn’t place the resulting XML in a form that allows ActiveSupport to successfully undasherize the keys, deserialize primitives by their type attribute, and place the result in an ActiveResource object (though it is of course possible that I configured the library incorrectly). I’ve tested the parsers below against the corresponding XmlSimple call and I know that they succesfully deserialize into the desired end result.

• libxml (extends XML::Document with a to_hash method, roughly 5-10x faster than XmlSimple)
• hpricot (extends Hpricot::Doc with a to_hash method, roughly 3-5x faster than XmlSimple)

Use them like this:

module ActiveSupport
  module CoreExtensions
    module Hash
      module Conversions
        module ClassMethods
          # libxml
          def from_xml(xml)
            xml.gsub!(/\s*\n\s*/, '')
            typecast_xml_value(undasherize_keys(XML::Parser.string(xml).parse.to_hash))
          end

          # hpricot
          def from_xml(xml)
            typecast_xml_value(undasherize_keys(Hpricot.XML(xml).to_hash))
          end
        end
      end
    end
  end
end

I’m at RailsConf this weekend, capping a six-months engagement delivering a couple of Rails apps written using NetBeans as our team’s primary Ruby editor. We ended up with it because the client requested that we do our all development work in Ubuntu 6.06, and because we couldn’t build a consensus around a more lightweight editor like Vim or jEdit. I thought I’d share some of my thoughts on it for anyone’s who’s interested in switching.

Executive Summary

There’s a lot to love and a fair amount to hate, but it’s a solid alternative to TextMate if you’re not on a Mac.

Stuff I liked

• The basic refactoring support is surprisingly handy. Until I got back the ability to rename classes and methods I hadn’t realized I’d missed it. I’m told that Aptana can perform similar tricks, but it’s been a while since I’ve tried it out.
• Likewise, the ability to jump directly to a class definition is handy, although it isn’t perfect. (If you have more than one class of the same name defined but located in different files or scoped to different modules it can’t figure out which one, exactly, you’re referencing, which I suppose is understandable.)
• RSpec support! A must-have. And the ability to jump back and forth between test and class… most of the time.
• If you’re a Vi user, the jVi plugin is almost worth the price of switching purely on its own. It’s a truly excellent piece of software, with an enormous amount of tweakability and really support for Vi constructs, and one of my complaints with Vim itself has always been that it lacks good project/directory browsing support, and yes, I’ve tried the Project plugin… again and again and again.
• It knows to insert an “end” when I hit enter after a block or method definition. Just a stupid small thing, but I miss it now when I don’t have it.

Stuff I didn’t like

• It opens up a new output window each time you hit Shift-F6 to run a test, leading to a million open output windows after you’ve been using it for a while. This is an enormous pain in the ass, and it’s compounded by the problem of really poor tab management support in the section of the IDE that displays your previous test runs. This is my single biggest complaint.
• The “jump to file” functionality doesn’t insert automatic wildcards the way TextMate does. This drives me absolutely nuts and it’s something I miss desperately.
• Method autocomplete doesn’t work well and is far more trouble than it’s worth. I turn it off almost immediately.
• It does not, and here I am being charitable, have the most attractive design on the face of the planet. I realize that by stating this I am probably slagging the work of some diligent UI designer somewhere, and I realize that you have to work within the constraints of Swing themes across multiple platforms. But it doesn’t stack up well next to the clean design of TextMate, or the brand new, achingly sexy (YMMV) MacVim.

Lessons Learned

• Use a nightly build, and keep it fresh. It’s the only way to get support for stuff like RSpec. The nightly builds have been stable and make for happy funtimes.
• Spin up your dev server on the command line rather than using the builtin server runner. Maybe it’s just me, but using an IDE for this just doesn’t seem worth the trouble.

If you’ve spent any time working with Ruby in NetBeans I’d be curious to hear about it.

Many people can drink and write; there is, in fact, a grand tradition of engaging in both simultaneously. I am not one of those people¹, and have spent much of the last year in this new city, job, and social circle engaged in much of the former and not enough of the latter. Even that cannot suffice to explain my extended absence from blogging; indeed, I must ultimately confess to having surrendered to the shallow charms of sloth. No more! Everything is now deployed, configured, and arranged in more or less the fashion that I desire. I have met the enemy, and he is me, and he is vanquished. Allow me to take a moment and put in an associate cheer for mod_rails.

I’ve also taken the opportunity to finally design my own custom theme. Many good and widely admired technical bloggers use stock or mostly-stock layouts. But it’s my little corner of the internet, and I can stumble blindly around Photoshop as well or better than the next poor sap when the need arises. Folly follows futility, however, with the knowledge that a large proportion of readers will simply consume this content through syndication and never stop to admire the elegant curve of my header, or its tacky nod to Web circa-2.0 fashion. In fact I haven’t even bothered testing it in IE6, and almost certainly never will; I am profoundly, almost fanatically indifferent to legacy browsers, even though their users are, being somewhat behind the technical curve, proportionately far less likely to consume this content through a feed reader and must thus endure my less-than-finely-tuned CSS.

I nonetheless submit that I am part of the solution, not the problem.

Frankly I haven’t tested it in IE7 either. I tried Firefox 3 and Safari 3 and then I got bored and went out for coffee. Either way, expect future blog entries to be more technical and perhaps less baldly narcissistic.

¹ I’m young yet, though, and everybody needs a dream.

If you ever find yourself frustrated with the choice of Mac web browsers, check out Camino. I’d written it off, but the latest release remembers your tab session (finally) and is both snappy and stable. It’s the best of both worlds, between feature-rich but memory-bloated (and ugly) Firefox and underpowered and frequently unstable Safari. It also features superior tab handling, including single browser mode, which is among my must-haves. Get it at http://www.caminobrowser.org.