Some Rails Tips
December 13th, 2006
We recently resolved a few longstanding issues with the site which I thought might be generally applicable to anyone else building a site in Rails. So, FYI:
RJS Templates are Slow
We do almost all of our screen updates via AJAX, and use Rails’ RJS templates to do almost all of that. The problem is, in the current version of Rails (1.1.6) they’re really slow to build. Rails does a lot of string parsing to turn a block of HTML into a JavaScript command. Fortunately, this is a known bug in Rails which will be fixed in 1.2. Thanks to the fact that Ruby is a dynamic language, you can drop this code into your project as a plugin, and effectively patch that part of Rails. It helped our performance in rendering big pages of gifts quite substantially. A list of 75 items was taking nearly 3 seconds to generate, and this patch dropped it to .3 seconds.
Running Background Tasks in Rails
Unlike other sites, the model we use to interact with your Amazon.com wishlist is “associate it, and we’ll keep it sync’d up” rather than “import it and never go back to Amazon” The reason we do this is twofold:
- Don’t force people to abandon their Amazon wishlist if they want to use wishlisting.
- Amazon is being very web 2.0-friendly by exposing their wishlists via an API. Consumers of the API shouldn’t take advantage of that openness by driving Amazon’s customers off-site.
So, in order to keep in sync with Amazon, we need to run some background tasks to see if you added/removed anything from your list there, see if anyone bought anything from your list, etc. There is a whole wiki entry on how to run background tasks in Rails, although none of them are particularly good in a shared environment, much less on a Grid (where even script/runner doesn’t work). Here’s a rather clever solution that MediaTemple provided us. In a cron job, run this:
/usr/bin/curl http://yoursite.com/path/to/a_page_that_does_some_work
Everything will run in your already running instance of Rails without needing to load up anything additional. Very handy - and so far it’s doing a good job of keeping everyone’s wishlists in sync.
UPDATE 9/26/2007: MediaTemple has an additional way to run background tasks on the grid. There is an issue on the grid where if you run a request that takes longer than 20 second to complete, it will start sending 502 error messages back to clients who are trying to access your site at the same time (yes, this is a little alarming). So, we stopped using the curl method. They posted a KB article on how you can run rake tasks in the background on the gs. With this power, you can effectively do what script/runner does. (Note: If you run into errors about RMagick read this article.) All you need to do is add require RAILS_ROOT + '/config/environment' to the beginning of your rake task to have access to the full rails environment.
December 13th, 2006 at 12:53 pm
I run background tasks on darfurwall.org. I use Ruby scripts that first load the rails environment by requiring environment.rb. I run the scripts with a cron job. Have you tried this? Is this bad for some reason?
December 13th, 2006 at 1:15 pm
There’s nothing wrong with that at all - what you’re doing sounds similar to what the built-in Rails “script/runner” does, and that seems like the most popular way to run these kinds of things. I’ve done that on my local system no problem.
Under that scenario, the rails stack will have to be loaded into memory for that script, even though it’s already in memory in the ruby processes that are handling the app. Depending on the task, I guess that could a good thing or a bad thing. Good for process isolation, bad in a memory-constrained environment.
The reason that wasn’t an option for us is that our host runs our rails processes on a grid, and loading up the rails stack from a command line on our individual machine wasn’t supported. This method also has the benefit of the fact that it runs inside the grid using whatever resources are available to it. Even though we may not end up on a grid forever, I kind of like the fact that the code that’s being run is in the same place as the rest of the app’s code, the codepath is the same, optimizations done to help the web stack will improve the performance of both things, the same automated test framework covers all of the code, etc. It just seems a little simpler.
September 26th, 2007 at 7:38 am
You have NO idea how much trouble you have saved me. I’m on Media Temple too, and I’ve tried everything from making my code more efficient, to trying BackgrounDRB (which apparently doesn’t work on MT), to trying script/runner (also doesn’t work), until I finally/thankfully ran across your post.
All for this thing I call a “lifefeed” that pulls info from twitter, digg, last.fm, and flickr to show a chronological list of my events on my site. This damn long running process has been a recurring headache for me for the last 3 months. Thanks again.
September 26th, 2007 at 1:12 pm
Hey Mark - I’m glad it helped. I just added an update to this article to reflect the way we just recently switched to for doing background tasks like this on the grid. Hope it helps!