December 13th, 2006
We recently resolved a few longstanding issues with the site which I thought might be generally applicable to anyone else building a site in Rails. So, FYI:
RJS Templates are Slow
Running Background Tasks in Rails
Unlike other sites, the model we use to interact with your Amazon.com wishlist is “associate it, and we’ll keep it sync’d up” rather than “import it and never go back to Amazon” The reason we do this is twofold:
- Don’t force people to abandon their Amazon wishlist if they want to use wishlisting.
- Amazon is being very web 2.0-friendly by exposing their wishlists via an API. Consumers of the API shouldn’t take advantage of that openness by driving Amazon’s customers off-site.
So, in order to keep in sync with Amazon, we need to run some background tasks to see if you added/removed anything from your list there, see if anyone bought anything from your list, etc. There is a whole wiki entry on how to run background tasks in Rails, although none of them are particularly good in a shared environment, much less on a Grid (where even script/runner doesn’t work). Here’s a rather clever solution that MediaTemple provided us. In a cron job, run this:
Everything will run in your already running instance of Rails without needing to load up anything additional. Very handy - and so far it’s doing a good job of keeping everyone’s wishlists in sync.
UPDATE 9/26/2007: MediaTemple has an additional way to run background tasks on the grid. There is an issue on the grid where if you run a request that takes longer than 20 second to complete, it will start sending 502 error messages back to clients who are trying to access your site at the same time (yes, this is a little alarming). So, we stopped using the curl method. They posted a KB article on how you can run rake tasks in the background on the gs. With this power, you can effectively do what script/runner does. (Note: If you run into errors about RMagick read this article.) All you need to do is add
require RAILS_ROOT + '/config/environment' to the beginning of your rake task to have access to the full rails environment.