N+1 problem in mongoid

Question

I'm using Mongoid to work with MongoDB in Rails.

What I'm looking for is something like active record include. Currently I failed to find such method in mongoid orm.

Anybody know how to solve this problem in mongoid or perhaps in mongomapper, which is known as another good alternative.

Answer 1

Now that some time has passed, Mongoid has indeed added support for this. See the "Eager Loading" section here:
http://docs.mongodb.org/ecosystem/tutorial/ruby-mongoid-tutorial/#eager-loading

Band.includes(:albums).each do |band|
  p band.albums.first.name # Does not hit the database again.
end

I'd like to point out:

1. Rails' :include does not do a join
2. SQL and Mongo both need eager loading.
3. The N+1 problem happens in this type of scenario (query generated inside of loop):

.

<% @posts.each do |post| %>
  <% post.comments.each do |comment| %>
    <%= comment.title %>
  <% end %>
<% end %>

Looks like the link that @amrnt posted was merged into Mongoid.

Answer 2

Update: it's been two years since I posted this answer and things have changed. See tybro0103's answer for details.

---

Old Answer

Based on the documentation of both drivers, neither of them supports what you're looking for. Probably because it wouldn't solve anything.

The :include functionality of ActiveRecord solves the N+1 problem for SQL databases. By telling ActiveRecord which related tables to include, it can build a single SQL query, by using JOIN statements. This will result in a single database call, regardless of the amount of tables you want to query.

MongoDB only allows you to query a single collection at a time. It doesn't support anything like a JOIN. So even if you could tell Mongoid which other collections it has to include, it would still have to perform a separate query for each additional collection.

Answer 3

Although the other answers are correct, in current versions of Mongoid the includes method is the best way to achieve the desired results. In previous versions where includes was not available I have found a way to get rid of the n+1 issue and thought it was worth mentioning.

In my case it was an n+2 issue.

class Judge
  include Mongoid::Document

  belongs_to :user
  belongs_to :photo

  def as_json(options={})
    {
      id: _id,
      photo: photo,
      user: user
    }
  end
end

class User
  include Mongoid::Document

  has_one :judge
end

class Photo
  include Mongoid::Document

  has_one :judge
end

controller action:

def index
  @judges = Judge.where(:user_id.exists => true)
  respond_with @judges
end

This as_json response results in an n+2 query issue from the Judge record. in my case giving the dev server a response time of:

Completed 200 OK in 816ms (Views: 785.2ms)

The key to solving this issue is to load the Users and the Photos in a single query instead of 1 by 1 per Judge.

You can do this utilizing Mongoids IdentityMap Mongoid 2 and Mongoid 3 support this feature.

First turn on the identity map in the mongoid.yml configuration file:

development:
  host: localhost
  database: awesome_app
  identity_map_enabled: true

Now change the controller action to manually load the users and photos. Note: The Mongoid::Relation record will lazily evaluate the query so you must call to_a to actually query the records and have them stored in the IdentityMap.

def index
  @judges ||= Awards::Api::Judge.where(:user_id.exists => true)
  @users = User.where(:_id.in => @judges.map(&:user_id)).to_a
  @photos = Awards::Api::Judges::Photo.where(:_id.in => @judges.map(&:photo_id)).to_a
  respond_with @judges
end

This results in only 3 queries total. 1 for the Judges, 1 for the Users and 1 for the Photos.

Completed 200 OK in 559ms (Views: 87.7ms)

How does this work? What's an IdentityMap?

An IdentityMap helps to keep track of what objects or records have already been loaded. So if you fetch the first User record the IdentityMap will store it. Then if you attempt to fetch the same User again Mongoid queries the IdentityMap for the User before it queries the Database again. This will save 1 query on the database.

So by loading all of the Users and Photos we know we are going to want for the Judges json in manual queries we pre-load the data into the IdentityMap all at once. Then when the Judge requires it's User and Photo it checks the IdentityMap and does not need to query the database.

Answer 4

ActiveRecord :include typically doesn't do a full join to populate Ruby objects. It does two calls. First to get the parent object (say a Post) then a second call to pull the related objects (comments that belong to the Post).

Mongoid works essentially the same way for referenced associations.

def Post
    references_many :comments
end

def Comment
    referenced_in :post
end

In the controller you get the post:

@post = Post.find(params[:id])

In your view you iterate over the comments:

<%- @post.comments.each do |comment| -%>
    VIEW CODE
<%- end -%>

Mongoid will find the post in the collection. When you hit the comments iterator it does a single query to get the comments. Mongoid wraps the query in a cursor so it is a true iterator and doesn't overload the memory.

Mongoid lazy loads all queries to allow this behavior by default. The :include tag is unnecessary.