Eager Loading with Ultrasphinx

Ultrasphinx is a great Rails plugin that wraps around the Sphinx full-text search engine. I am using Ultrasphinx to handle search queries on a personal project I’m working on and I ran into a situation where I wanted to eager load associated models for my search results. The method for doing this is not well documented so I’m going to step through how to add eager loading to your Ultrasphinx searches.

I am actually going to show two ways to do this. The first is the way that Evan Weaver, the creator of Ultrasphinx, recommends and the second way is the create a simple plugin that extends the Ultrasphinx plugin. The second way is my favorite because it provides the cleanest integration in my opinion but I’m going to demonstrate both methods so you can choose for yourself.

Method 1

To begin we need to make a class method that performs the eager loading in the model. Something like this:

## app/models/course.rb
class Course
  ...
 
  def self.find_with_includes(id)
    find(id, :include => [:professors])
  end
end

Notice how I’m using the familiar ActiveRecord include option to perform the eager loading like we would normally. The next and last step is to add the method to client_options[:finder_methods] in your environment.rb.

## config/environment.rb  
Ultrasphinx::Search.client_options[:finder_methods].unshift(:find_with_includes)

That’s it. Now to the second method.

Method 2

As I mentioned before, in this method we are going to create a simple plugin that will add functionality to Ultrasphinx. The code for this plugin was written by Henrik N.

Create a new folder in vendor/plugins/ called ultrasphinx_customizations. The name can be anything you like as long as it starts with “ultrasphinx”. This is so our customizations are loaded after the actual Ultrasphinx plugin. Inside the folder create init.rb and place this line in it.

## vendor/plugins/ultrasphinx_customizations/init.rb
Dir[File.dirname(__FILE__) + '/lib/*.rb'].each {|file| require file }

Now for the part of the plugin that actually does the work. Create a lib folder in ultrasphinx_customizations and create a new file in it called eager_loading.rb with the following contents.

## vendor/plugins/ultrasphinx_customizations/lib/eager_loading.rb
 
# Allows specifying eager loading options as e.g.
# Ultrasphinx::Search.client_options[:include]['MyKlass'] => [:user]
# or directly in is_indexed like
# is_indexed ..., :eagerly_load => [:user]
 
Ultrasphinx::Search.client_options.merge!(HashWithIndifferentAccess.new({
  :finder_methods => ['find_all_by_id_with_eager_loading'],
  :include => {}
}))
 
 
class ActiveRecord::Base
 
  def self.find_all_by_id_with_eager_loading(*ids)
    includes = Ultrasphinx::Search.client_options['include'][self.name]
    args = ids + [:include => includes]
    find_all_by_id(*args)
  end
 
  def self.is_indexed_with_include_option(options)
    Ultrasphinx::Search.client_options['include'][self.name] = options.delete(:eagerly_load)
    is_indexed_without_include_option(options)
  end
  class << self; alias_method_chain :is_indexed, :include_option; end
 
end

Sweet we’re done with the plugin. Now for the fun part. Remember in the model where we have our Ultrasphinx is_indexed :fields => [...] declaration? Thanks to our new customization we can add an :eagerly_loaded key and it will behave like an ActiveRecord find(:include => …). Here is what mine looks like.

## app/models/course.rb
 
is_indexed  :fields => ['course_number', 'title', 'term'],
            :eagerly_load => [:professors]

That’s all you have to do! I really enjoy the second method because it provides a seamless integration with the is_indexed call which seems to help me sleep at night.

Huge thanks to Henrik N for writing the ultrasphinx_customization code and Evan Weaver for writing Ultrasphinx.

Updated 04/17/2008: Fixed incorrect statement about plugin names.

April 11, 2008 | Filed Under Ruby on Rails 

Comments

9 Responses to “Eager Loading with Ultrasphinx”

  1. Cheri on April 27th, 2008 11:26 pm

    Thank you so much for addressing this issue. Are you now able to do this for the example you use?

    @course = Ultrasphinx::Search.new(:query => params[:course_id])

    @professors = @course.professors?

    I am trying to do something similar in my program but haven’t been able to access the associated fields (in your example, professors).

    Any feedback would be appreciated!

    Thanks!

    Cheri

    Cheri

  2. Cheri on April 27th, 2008 11:28 pm

    Please nix the “?” after course.professors in my comment above. Thanks

  3. Rob Olson on April 28th, 2008 12:59 am

    Yes, Cheri that does work for me. This is how I run it.

    search = Ultrasphinx::Search.new(:query => “computer science”)
    search.run

    @courses = search.results
    c1 = @courses.first
    puts c1.professors

  4. Cheri on May 4th, 2008 9:50 pm

    Hi Rob,

    Thank your response! Do you know if there is a way to filter eagerly loaded models before they are loaded from the database? Say I want to find all Locations, but load only subcategories where subcategory.location_id = location.id. Do you know of a way to modify the statement below to do this?

    @location = Location.find(loc_id, :include => [:subcategories])

    Thanks

  5. Rob Olson on May 5th, 2008 6:30 pm

    Cheri,

    If you’ve setup your ActiveRecord associations correctly where a Location has many subcategories and subcategories belong to Locations then it should work as you desire.

    You might want to check out the has_finder plugin http://pivots.pivotallabs.com/users/nick/blog/articles/284-hasfinder-it-s-now-easier-than-ever-to-create-complex-re-usable-sql-queries You could use that to create more complex filters that are not done for you by ActiveRecord.

    Let me know if that helps.

  6. Cheri on May 10th, 2008 7:08 pm

    Hi Rob, that’s a good plugin to know about! I’ll give it a try when my application gets more complicated. Here was another way I found to apply conditions to eagerly loaded models.

    Location.find(loc_id, :include => [:subcategories], :conditions => ["subcategories.location_id = ?", loc_id])

    Thanks again for your help!

    Cheri

  7. Ole on June 26th, 2009 9:58 am

    Hi,

    thanks for this solution. I’m wondering if it’s possible to support hierarchies of associations too, like so:

    :eagerly_load => [ :author, { :comments => { :author => :gravatar } }

    That would be a great feature! So far I couldn’t get it to work.

    Cheers from Berlin

  8. michal kuklis on July 12th, 2009 6:23 am

    great post! thanks for sharing!

  9. Brian Armstrong on October 16th, 2009 9:14 pm

    Great writeup!

    Just wanted to mention that you may also want to use find_all_by_id instead of find in the first solution. This prevents the problem when one of your records is deleted, but is still in the index. If you just use find it will throw a record not found exception, but if you use find_all_by_id it will return 9 out of the 10 records, even if one of the ids doesn’t exist. Thanks! Brian

Leave a Reply