Gotchas in using BackgrounDRb in Ruby on Rails
I’ve been working on a bit of code to perform audio and video encoding for media files uploaded to one of our client’s sites and I was thrilled to come across BackgrounDRb, a Rails plugin that allows developers to build scheduled background tasks, similar to the OpenSympony’s Quartz for Java. The plugin also allows you to allow user actions to initiate long running processes by spawning worker threads from controllers (or other places).
Of particular interest to me was the ability to spawn (fork) worker threads from user actions, or in this case, ActiveRecord callbacks which were called when the user action caused my model object to be saved. The basic order of operations is this:
- User uploads an audio file
- Uploaded file data is loaded into an attachment_fu model object
- Model object is saved
- Model object’s after_save callback is called
- The after_save callback determines whether the uploaded file requires encoding, and spawns a BackgrounDRb worker process if it does
In this process lay many issues. I’ve attempted to describe some of them below.
Close your connections
This may be obvious to many, but I figured with all that ActiveRecord does for you, that if I do some work with ActiveRecord in a BackgrounDRB worker process, my database connections would be closed automatically. This was not the case, however it can be quickly remedied by adding the following after all your code is done doing what it needs to do:
ActiveRecord::Base.connection.disconnect!
Don’t try to pass entire ActiveRecord objects to workers from Rails
I started building a simple call to spawn a new BackgrounDRb worker from an ActiveRecord lifecycle callback method (see below), and starting running into errors like these:
undefined method `[]‘ for #<DRb::DRbUnknown:0×2501bd4>
I found a number of postings online indicating that simply adding the following to my model objects would clear this up:
include DRbUndumped
Doing so did not fix my undefined method problems, so when I came across this on another blog, I decided to cut my losses and just pass the database ID.
When passing arguments from Rails to BackgrounDRb workers, don’t pass huge ActiveRecord objects. Its asking for trouble. You can easily circumvent the situation by passing id of AR objects.
One thing that I didn’t think I’d be dealing with in coming to Rails from a Java background is serialization issues, but there we have it.
Don’t create new workers from new record callbacks
This may have been common knowledge, but I had to discover this the hard way. The after_save callback in ActiveRecord model objects is called before the save transaction has been committed. The implication this has is that if I pass an ID of an ActiveRecord object to BackgrounDRb for further processing, strange things can happen.
In the case of my after_save callback being called on a newly created object, I found that most of the time, by the time the BackrounDRb process starts working and tries to retrieve the object using the ID I have passed, my original object.save() transaction has not yet committed, so I get errors indicating the record could not be found, even though I check the database no more than a second later and the record is there and intact.
It seems that I’m not the only one that was looking for a post-commit callback: