Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Mutex for ActiveRecord Model

My User model has a nasty method that should not be called simultaneously for two instances of the same record. I need to execute two http requests in a row and at the same time make sure that any other thread does not execute the same method for the same record at the same time.

class User
  ...
  def nasty_long_running_method
    // something nasty will happen if this method is called simultaneously
    // for two instances of the same record and the later one finishes http_request_1
    // before the first one finishes http_request_2.
    http_request_1 // Takes 1-3 seconds.
    http_request_2 // Takes 1-3 seconds.
    update_model
  end
end

For example this would break everything:

user = User.first
Thread.new { user.nasty_long_running_method }
Thread.new { user.nasty_long_running_method }

But this would be ok and it should be allowed:

user1 = User.find(1)
user2 = User.find(2)
Thread.new { user1.nasty_long_running_method }
Thread.new { user2.nasty_long_running_method }

What would be the best way to make sure the method is not called simultaneously for two instances of the same record?

like image 431
Mika Avatar asked Jul 04 '14 05:07

Mika


2 Answers

I found a gem Remote lock when searching for a solution for my problem. It is a mutex solution that uses Redis in the backend.

It:

  • is accessible for all processes
  • does not lock the database
  • is in memory -> fast and no IO

The method looks like this now

def nasty
  $lock = RemoteLock.new(RemoteLock::Adapters::Redis.new(REDIS))
  $lock.synchronize("capi_lock_#{user_id}") do
    http_request_1
    http_request_2
    update_user
  end
end
like image 124
Mika Avatar answered Nov 15 '22 20:11

Mika


I would start with adding a mutex or semaphore. Read about mutex: http://www.ruby-doc.org/core-2.1.2/Mutex.html

class User

  ...
  def nasty
    @semaphore ||= Mutex.new

    @semaphore.synchronize {
      # only one thread at a time can enter this block...  
    }
  end
end

If your class is an ActiveRecord object you might want to use Rails' locking and database transactions. See: http://api.rubyonrails.org/classes/ActiveRecord/Locking/Pessimistic.html

def nasty
  User.transaction do
    lock!
    ...
    save!
  end
end

Update: You updated your question with more details. And it seems like my solutions do not really fit anymore. The first solutions does not work if you have multiple instances running. The second locks only the database row, it does not prevent multiple thread from entering the code block at the same time.

Therefore if would think about building a database based semaphore.

class Semaphore < ActiveRecord::Base
  belongs_to :item, :polymorphic => true

  def self.get_lock(item, identifier)
    # may raise invalid key exception from unique key contraints in db
    create(:item => item) rescue false
  end

  def release
    destroy
  end
end

The database should have an unique index covering the rows for the polymorphic association to item. That should protect multiple thread from getting a lock for the same item at the same time. Your method would look like this:

def nasty
  until semaphore
    semaphore = Semaphore.get_lock(user)
  end

  ...

  semaphore.release
end

There are a couple of problems to solve around this: How long do you want to wait to get the semaphore? What happens if the external http requests take ages? Do you need to store additional pieces of information (hostname, pid) to identifier what thread lock an item? You will need some kind of cleanup task the removes locks that still exist after a certain period of time or after restarting the server.

Furthermore I think it is a terrible idea to have something like this in a web server. At least you should move all that stuff into background jobs. What might solve your problem, if your app is small and needs just one background job to get everything done.

like image 44
spickermann Avatar answered Nov 15 '22 20:11

spickermann