Erudition and Inanity

A few days ago Sumeet Moghe posted about upgrading to Notes 8.5 on the Mac and I left a comment wondering aloud why his inbox looked nothing like mine. Lo and behold, The Internet responds, and I dutifully repost:

If your mail file looks like version 7 while using version 8 you need to replace the design of your mail file with the 8.5 template. Make sure Advanced Menus are turned on in the View Pull down menu first. Open your inbox, then go to File > Application > Replace Design > Select Mail 8.5 from the list and also click the Upgrade Folder design check box at the bottom. Click Replace.

Your mail file will be upgraded, the progress icon at the bottom of Notes will start to move left to right – click on this to display the full progress of the design replace – when it finishes close your mail file and any open tabs for that matter then open the mail file once more – it should now have the 8 Look and Feel!

Thanks, Anonymous!

I interned at Lotus for about a year and remain impressed with the once-revolutionary Domino while still being dismayed over Notes’ clunkiness. But I admire all the engineers I had a chance to work with there (though I was actually working on the IBM Workplace initiative, arguably a competitor to, and replacement of, Notes, but that’s a story for another day), and it’s nice to see the interface receive the overhaul it deserves.

But I’m upset that it took this long, and frankly the fact that I have to dig through a menu somewhere to upgrade my mail template by hand is pretty obnoxious. I don’t know anyone who likes their mail client looking like it was designed in 1998, and if they’re going to make users go through the effort of a PKG install rather than a simple DMG then they might as well offer to automatically drag my inbox into the 21st century while they’re at it.

An update from my post from a few days ago where I talk about REST and domain identifiers. This is the entirety of the code from ActiveResource::Base#save:

  def save
    new? ? create : update
  end

Here’s the punchline:

  def new?
    id.nil?
  end

ActiveResource doesn’t document it, but you can override the primary key that it uses to communicate with your backend system (self.primary_key = :username) and indeed this is the most effective way I know of ensuring that it performs deletes and updates with the appropriate key. The trouble is that if you attempt to create a resource remotely and pre-populate the username (which you’d want to be able to do; it is, after all, a domain identifier) ActiveResource chokes and tries to update the remote resource instead of saving it.

To put it another way: ActiveResource thinks that if your resource has an ID it’s not new. It cannot separate the concepts of auto-increment database-generated ID and resource identifier.

The correct way to solve this problem is to flag any resources retrieved from a remote system as “not new” and consider any other created resources as new. Until that’s fixed I recommend using a conditional approach in which your controller checks incoming IDs and finds on the appropriate column based on whether or not the ID looks like a resource identifier or a database integer. More on that next post.

This is part of a series of posts on Rails and REST. Read the rest.

In the last month or so, and particularly as the American presidential election enters its most heated stretch, the generally poor quality of online discourse has received a lot of meta-commentary from those people and organizations whose articles are subjected to the worst bile. On the Media, an NPR show (and podcast) of media commentary and criticism, did a piece on it, and more recently The Atlantic Monthly blogger Megan McArdle found a fellow traveller in Tim Burke:

Most of the time, it seems to me that trying to write anything more reflective, more ambiguous, more exploratory in a blog is either going to bore an audience that’s come seeking their Two-Minute Hate or it’s just going to be willfully misconstrued by someone else who needs fresh meat for their own hounds to feed upon. Read the comments section at Inside Higher Education, for one example. There’s no point to trying to talk about nuance or complexity or what makes for a good research design or anything else in that kind of back-and-forth.

What I think a lot of people tend to miss in this discussion about online comments is that websites that specialize in syndication and commentary, like Slashdot and Digg, tend to do a much better job providing a decent signal-to-noise ratio (yes, I said Slashdot) than most blog and article commenting systems do. Most of the time, as with my own blog, the comments feature is essentially an afterthought: a simple list of text comments, along with a timestamp and usually some form of commenter identification. Every comment is given equal weight and exposure, and if there’s any moderation at all it’s usually manual, and enforced by the blogger themselves.

This is fine on small blogs, but once you grow to a certain size it quickly becomes unwieldy. It’s well-understood that online anonymity can breed anti-social behavior, but not many people realize that software can, to a limited (but never full) degree, help to moderate such discussions.

How It Works

Let’s take Slashdot as our example. Here’s an excerpt from the FAQ entry on their moderation system:

When moderators are given access, they are given a number of points of influence to play with. Each comment they moderate deducts a point. When they run out of points, they are done serving until next time it is their turn.

Moderation takes place by selecting an adjective from a drop down list that appears next to comments containing descriptive words like “Flamebait” or “Informative.” Bad words will reduce the comment’s score by a single point, and good words increase a comment’s score by a single point. All comments are scored on an absolute scale from -1 to 5. Logged-in users start at 1 (although this can vary from 0 to 2 based on their karma) and anonymous users start at 0.

...

Moderation points expire after 3 days if they are left unused. You then go back into the pool and might someday be given access again.

That entry was last modified on 6/19/00, more than eight years ago. The technology isn’t new, although Slashdot (and more recently Digg) have spent a lot of time tweaking it. This is their competitive edge: it’s not the syndication, it’s the filter. Visitors to the site start with a high comment threshold by default, and they aren’t shown any poorly-rated comments (or articles) unless they themselves lower that threshold. And because human beings on the Internet are, generally speaking, much like human beings everywhere else, anonymous, poorly-written, one-sided, uninformative, and/or abusive comments tend to get rated (or “modded”) down. Good comments are modded up. A pretty decent conversation ensues.

A lot of infrastructure goes into making this successful. You need to allow users to log in, to track their moderation points (“karma” in Slashdot lingo) and, most importantly, you need to spend a lot of time securing the system against the inevitable onslaught of trolls. It’s hard to do, and well beyond the scope of most blogs. Even large blogs don’t have a consistent enough user community to justify the kind of effort that goes into pruning one, although I’d be surprised if some enterprising soul hasn’t tried to build one around Gravatar or a similar system (though my extremely casual Googling didn’t reveal anything).

Good Conversationalists Self-Moderate

Of course, not all successful conversation systems have been technologically policed. The most prominent example is Usenet, for years the authoritative way to engage in group discussion on the Internet. But it took Usenet years to evolve the powerful etiquette and culture that underpins the community (‘community’ itself being an extremely loose term in this context), and educating newcomers has always been difficult. It’s no coincidence that Godwin’s Law, perhaps the most famous Internet adage on heated conversation, was born there way back in 1990.

So none of this is new, and it was going on years before the New York Times deigned to dip its toe ever so gingerly in the pool. But the software that’s been deployed simply isn’t very good at regulating the quality of those conversations yet, and can’t begin to compete with web sites that specialize in it. Until developers and authors recognize the need for discussion systems that encourage conversations rather than chatter, and that involve their readers in the process, expect not the mud to cease its winge’d flight.

By default, Rails expects your resource to be identified by its model’s primary key ID. Don’t give in to temptation. There are a lot of good reasons to avoid going that route, and they’ll save you pain and trouble later. This isn’t news in the Rails community, as the framework has always had excellent routing support for custom URLs. But the default XML serialization code isn’t as forgiving, and it’s worth ticking off a few reasons to take the extra step.

They aren’t human-friendly

Database IDs mean nothing to the human beings consuming your resource. This matters even when all you’re providing is pure REST service with no UI component, as with the project I’m working on now, because although it may be other applications consuming the service, human beings are writing the code. One of the great advantages of a RESTful API is that they’re pretty easy for even casual programmers, maybe technically-minded domain experts, to consume with simple, hacky script. Anything you can do to encourage those people and make their lives easier will redound back in user happiness and cool, unexpected new uses.

This isn’t just useful for the warm fuzzies, though. Most importantly, it allows users to self-identify the resource, saving you an unnecessary search. More on that later.

They’re (usually) generated arbitrarily

My current project involves importing a large amount of data from a third-party system. For a long time, whenever we re-ran that import any system that relied on a link to our resource broke because the IDs got regenerated. Of course you can fix this with a smarter import (and we have) but a better solution is simply to use the data we’re given. A URL-friendly ID that’s inherently tied to the data is much less fragile.

They don’t allow you to model concepts

Some resources don’t map cleanly to database tables, or it doesn’t make any sense to map them that way. Consider a resource that exposes the weather on a given day, and retains a history of that weather. Your historical data is probably stored in a database somewhere, but the concept you’re modeling is the date.

Exposing the resource with database IDs forces the user to perform a search if they want to find out what the weather was five years ago from this Tuesday. Without a URL representation you end up with a query of the form /weather?date=2003-08-26, which is more appropriately a /show/ than it is an /index/ call. /weather/2003-08-26 is not only a cleaner URL but retains cleaner semantics; an index should give you a list of things back, so the former URL will deserialize into an array with a single Weather resource (courtesy of ActiveResource) whereas the latter is simply a single object.

In a future post I’ll talk about ways to minimize the amount of trouble involved in moving over to a human-friendly resource identifier.

This is part of a series of posts on Rails and REST. Read the rest.

I see a fair amount of confusion with services over developers returning incorrect or incomplete HTTP failure codes from their RESTful controller actions, so here’s a quick reference to some of the most commonly-used codes in the 400 range and their specific use in a Rails context.

These are all the codes I’ve used in my RESTful services, but if you know of any others that I’ve missed post them in the comments or send me an email and I’ll incorporate them into the table above.

Also, it’s worth noting that REST, based as it is on the HTTP we know and love, is supposed to be reasonably human-friendly. In my opinion it’s not necessary to adhere strictly to the above conventions (although I try to) as long as the human beings trying to use your service understand why their requests failed and what they can do to fix them.

This is part of a series of posts on Rails and REST. Read the rest.

When you’re writing a RESTful web service designed to expose a resource to the outside world, the case for including first-order associations is pretty strong. ActiveRecord’s default serialization method, to_xml, makes it pretty easy: simply pass in the :include key and the list of associations you’d like to nest, and the XML is automatically generated for you. But it can kill performance pretty quickly if you’re not careful. Consider a User resource, which includes, as a first-order association, the Address of that user. The XML will look something like this:

This is fine for a single resource, and probably what I want to see when I ask for /users/gthreepwood.xml. But when I ask for /users.xml?job_title=Pirate, and I’m getting back a list of resources that can stretch into the dozens or hundreds (there are a lot of pirates on Melee Island) it can start to drag quickly. You’re asking for more data from your database server (be sure you’re including the associations not only on the to_xml call but in the initial find call as well), it takes longer to serialize each object, and it takes longer (often much, much longer) for the client to parse that XML.

Worst of all, your client probably doesn’t need the full address. There’s plenty of precedent in the web world for this; when you’re displaying a list of things you probably want an abbreviated view, but when you’re looking at a specific resource you probably want a much richer view. You’re also designing an API that forces the client to reach through multiple layers of an object in order to dig out the one piece of information they need.

Instead, follow the Law of Demeter. If all the client of your resource needs when displaying a list of users is a text description of their street address, consider adding a method to your model that provides it:

def full_address
  "#{address.street_address}, #{address.city}, #{address.state} #{address.zip_code}" 
end

And use it in your controller as follows:

respond_to do |format| 
  format.xml do 
    render :xml => @users.to_xml(:methods => [:full_address])
  end
end

By my count, the former representation comes to 282 bytes and the latter to 200—a savings of almost one third. Multiplied over a large XML document with multiple resources that translates into a big win for both network bandwidth and XML parsing speed.

There’s one hitch, and this applies anytime you include a method as part of your XML. If the client tries to modify that resource update it by POSTing it back to your server, and you handle it by passing params[:user] straight into the update_attributes method of the relevant User object, your model will complain that it doesn’t know anything about a full_address= method. You’ve provided a getter but no setter.

There are two ways around this: either remove the offending parameter from the attributes you pass into the User model in your controller, or create no-op setter on your model. I personally prefer the latter, simply because I like my controllers to do as little as possible (and for the record, I think that models should either know how to serialize themselves to XML or that responsibility for doing so should be offloaded to an ERB template, but that’s another post). But either works as long as you’re aware of it and handling it appropriately.

This is part of a series of posts on Rails and REST. Read the rest.

Potential customers can be hard to come by, especially since you’re building something that nobody will ever, EVER want. Well, that’s getting ahead of myself. Let’s just agree that when you’re setting out to Gather Business Requirements, your potential customers usually aren’t already sitting in the room with you. [link]

I’m not sure what to say. At the intersection of people who need shit built but can’t program and people who know how to build shit but don’t know the domain you have business requirements. Steve Yegge seems to think that business requirements are something you gather using focus groups, and loathes them; on the types of projects I work on, we typically gather them from the domain experts and we build them for people who need shit built desperately badly. The fact that they grow out of line with what you need is an immediate consequence of the fact that his teams have tried to Do It All Up Front rather than iterate and improve, and this coming from a man who loathes Agile but just as clearly doesn’t understand that any better than he understands business.

A curious, inquisitive, and passionate person can find the passion in virtually any domain they’re asked to turn their keyboards to; in the past year I’ve had the opportunity to work on a wide variety of projects in areas that I never thought I’d enjoy. It’s sad to see that Yegge seems to be advocating a kind the kind of intellectual blinders that confine you and your work to the things you’re already interested in and understand, all the more because it seems to run counter to a lot of what he’s written so eloquently about in the past.

I was kicking around in ActiveResource::Base recently and it took me a good solid ten minutes to figure out why the prefix method didn’t recur infinitely.

def prefix(options={})
  default = site.path
  default << '/' unless default[-1..-1] == '/'
  # generate the actual method based on the current site path
  self.prefix = default
  prefix(options)
end

def prefix_source
  prefix # generate #prefix and #prefix_source methods first
  prefix_source
end

def prefix=(value = '/')
  # Replace :placeholders with '#{embedded options[:lookups]}'
  prefix_call = value.gsub(/:\w+/) { |key| "\#{options[#{key}]}" }
  # Redefine the new methods.
  code = <<-end_code
    def prefix_source() "#{value}" end
    def prefix(options={}) "#{prefix_call}" end
  end_code
  silence_warnings { instance_eval code, FILE, LINE }
rescue
  logger.error "Couldn't set prefix: #{$!}\n  #{code}" 
  raise
end

Do you see it? When prefix is called, it calls prefix=, which redefines prefix and returns. prefix in turn returns by calling not itself but the newly-created method of the same name.

I am not known for my low tolerance of metalanguage hackery (exhibit). But I do have my limits, and this exceeds them; calling it needlessly obfuscatory would be kind. Or am I wrong? Is there no better way to provide a method with the same semantics?

I know that I’m late to the party here, that all of the cool kids have had accounts for what seems like Internet-decades, and that I can do nothing but add my voice belatedly to an already rapturous chorus; nevertheless, forgive my enthusiasm: GitHub rules, it is worth joining, and, once signed up, it took only a week for my workflow to be forever improved. I am, in short, a believer.

• It’s never been easier or more convenient to contribute to other open source projects.
• It’s never been easier to work remotely.
• It’s never been easier to track the progress of projects and developers you’re interested in.
• Unicorns! Rainbows! Puppies!

Over the next few weeks I’ll gradually be migrating all of my projects over to it, and I encourage you to do the same.

Scheme, a variant of Lisp popular in academic settings, has no download link on its extremely minimalist home page. If you click Implementations, it will list about fifty and recommend about ten. Visit the Ruby homepage and you are presented with a modern, polished, attractive site and a large, friendly Download Ruby link.

Marketing isn’t always about advertising. If you believe in a product, present it to the world in a way that makes it easy for others to believe in. It’s easy to be cynical about appearances, but they are, here and now as they have ever been, important. This principle is extended deeply into the Ruby language itself. From a technical standpoint, Ruby does very little that other languages don’t, but the metaprogramming hooks it makes available to users are straightforward and user-friendly and make its users feel immediately powerful. The elaborate contortions (do watch the presentation if you can, it’s awesome) that the language itself must go through in order to provide a convenient interface to the user doesn’t detract from the net result. Simplexity sells.

Ruby and, now, Rails have been around for long enough that it seems, at least to me, that most people who were ever going to have an opinion on it now do so. But accusations of hype are and always have been overblown; I would go so far as to call that particular brand of cynicism a kind of intellectual laziness convenient to those don’t understand a phenomenon and don’t particularly want to. Regardless of whether or not Ruby and/or Rails are here to stay (and I think they are, but then, I’m an optimist), it would be foolish to ignore the lessons here.