Heroku & MongoHQ: ActionView::Template::Error (The operation: #<Moped::Protocol::Commands::Authenticate failed with error "auth fails")

Question

I have a Rails 3 app with MongoID 3 deployed successfully to Heroku (using MongoHQ) -- but then something happened (cannot pinpoint what exactly it was) and the build started crashing on Heroku.

So this is the error I get:

TLDR: Moped::Errors::AuthenticationFailure (The operation: #<Moped::Protocol::Commands::Authenticate ... failed with error "auth fails")

What makes this more difficult to debug is that it's working just fine on localhost. Now I'm running out of ideas what this could be.

app[web.1]: >> Thin web server (v1.4.1 codename Chromeo) app[web.1]: >> Maximum connections set to 1024 app[web.1]: >> Listening on 0.0.0.0:58731, CTRL+C to stop heroku[web.1]: State changed from starting to up app[web.1]:  app[web.1]:  app[web.1]: Started GET "/" for 212.86.9.90 at 2012-07-31 08:08:07 +0000 heroku[router]: GET cool-name-123.herokuapp.com/ dyno=web.1 queue=0 wait=0ms service=171ms status=200 bytes=1286 heroku[router]: GET cool-name-123.herokuapp.com/assets/application-8e7bfeeffc9291864e5b42d908c2fdda.css dyno=web.1 queue=0 wait=0ms service=11ms status=200 bytes=92524 heroku[router]: GET cool-name-123.herokuapp.com/assets/application-aa557bde70f1236cdf90c913043c4382.js dyno=web.1 queue=0 wait=0ms service=16ms status=200 bytes=122836 heroku[router]: GET cool-name-123.herokuapp.com/favicon.ico dyno=web.1 queue=0 wait=0ms service=3ms status=200 bytes=0 app[web.1]:  app[web.1]:  app[web.1]: Started GET "/originals/new" for 212.86.9.90 at 2012-07-31 08:08:11 +0000 heroku[router]: GET cool-name-123.herokuapp.com/originals/new dyno=web.1 queue=0 wait=0ms service=31ms status=200 bytes=1808 app[web.1]:  app[web.1]:  app[web.1]: Started POST "/originals" for 212.86.9.90 at 2012-07-31 08:08:34 +0000 app[web.1]:  app[web.1]: Moped::Errors::AuthenticationFailure (The operation: #<Moped::Protocol::Commands::Authenticate app[web.1]:   @length=154 app[web.1]:   @request_id=3 app[web.1]:   @response_to=0 app[web.1]:   @op_code=2004 app[web.1]:   @flags=[] app[web.1]:   @full_collection_name="app123.$cmd" app[web.1]:   @limit=-1 app[web.1]:   @skip=0 app[web.1]:   @selector={:authenticate=>1, :user=>"heroku", :nonce=>"xyz", :key=>"xyz"} heroku[router]: POST cool-name-123.herokuapp.com/originals dyno=web.1 queue=0 wait=0ms service=1486ms status=500 bytes=643 app[web.1]:   @fields=nil> app[web.1]: failed with error "auth fails"): app[web.1]:   app/controllers/originals_controller.rb:19:in `block in create' app[web.1]:   app/controllers/originals_controller.rb:18:in `create' app[web.1]:  app[web.1]:  heroku[router]: GET cool-name-123.herokuapp.com/favicon.ico dyno=web.1 queue=0 wait=0ms service=3ms status=304 bytes=0 app[web.1]:  app[web.1]:  app[web.1]: Started GET "/" for 212.86.9.90 at 2012-07-31 08:09:40 +0000 heroku[router]: GET cool-name-123.herokuapp.com/ dyno=web.1 queue=0 wait=0ms service=11ms status=304 bytes=0 

I've recently struggled with Heroku, and as a result I think I should have the correct settings for the choice of db drivers & service (using the new yml scheme + Ruby 1.9.3 in the Gemfile).

When I run heroku config --app app_name I get

DATABASE_URL        => postgres://rhalppyjrb:[email protected]/rhalppyjrb GEM_PATH            => vendor/bundle/ruby/1.9.1 LANG                => en_US.UTF-8 MONGOHQ_URL         => mongodb://heroku:[email protected]:27054/app123 PATH                => bin:vendor/bundle/ruby/1.9.1/bin:/usr/local/bin:/usr/bin:/bin RACK_ENV            => production RAILS_ENV           => production SHARED_DATABASE_URL => postgres://rhalppyjrb:[email protected]/rhalppyjrb 

My mongoid.yml looks like this:

production:   sessions:     default:           uri: <%= ENV['MONGOHQ_URL'] %>           options:             consistency: :strong             skip_version_check: true             safe: true 

Gemfile is like this:

source 'https://rubygems.org'  gem 'bundler', '1.2.0.rc'  ruby '1.9.3'  gem 'rails', '3.2.6' gem 'thin' gem 'rmagick', :require => 'RMagick' gem "carrierwave-mongoid", :git => "git://github.com/jnicklas/carrierwave-mongoid.git", :branch => "mongoid-3.0", :require => 'carrierwave/mongoid'  group :assets do   gem 'sass-rails',   '~> 3.2.3'   gem 'coffee-rails', '~> 3.2.1'   gem 'uglifier', '>= 1.0.3'   gem "therubyracer", :platform => :ruby end  gem 'jquery-rails' gem "haml", ">= 3.1.6" gem "mongoid", ">= 3.0.3" gem "devise", ">= 2.1.2" gem "devise_invitable", ">= 1.0.2" gem "cancan", ">= 1.6.8" gem "rolify", ">= 3.1.0" gem "bootstrap-sass", ">= 2.0.4.0" gem "simple_form"  group :development do   gem "guard", ">= 0.6.2"   gem 'rb-fsevent'   gem 'growl'   gem "haml-rails", ">= 0.3.4"   gem "guard-bundler", ">= 0.1.3"   gem "guard-rails", ">= 0.0.3"   gem "guard-livereload", ">= 0.3.0"   gem "guard-rspec", ">= 0.4.3" end  group :test do   gem "database_cleaner", ">= 0.8.0"   gem "mongoid-rspec", "1.4.6"   gem "email_spec", ">= 1.2.1" end  group :development, :test do   gem "factory_girl_rails", ">= 3.5.0"   gem "rspec-rails", ">= 2.11.0" end 

Answer 1

The issue is that your authentication credentials are no longer correct. Specifically, the user you are trying to authenticate as no longer exist.

The reason is that when you do a rake db:reseed all data is removed, including the DB level users.

Example:

  ➜  skadi git:(master) ✗ mongo            MongoDB shell version: 2.0.7   connecting to: test   > use skadi_development   switched to db skadi_development   > db.system.users.find()   > db.addUser("joe", "passwordForJoe")   { "n" : 0, "connectionId" : 191, "err" : null, "ok" : 1 }   {     "user" : "joe",     "readOnly" : false,     "pwd" : "dac588613249fe92703afb262ec53b82",     "_id" : ObjectId("5030d5181cefbd2b04a99e30")   }   > db.system.users.find()   { "_id" : ObjectId("5030d5181cefbd2b04a99e30"), "user" : "joe", "readOnly" : false, "pwd" : "dac588613249fe92703afb262ec53b82" }   > exit   bye    ➜  skadi git:(master) ✗ rake db:reseed    ➜  skadi git:(master) ✗ mongo            MongoDB shell version: 2.0.7   connecting to: test   > use skadi_development   switched to db skadi_development   > db.system.users.find()   => Nothing 

So you when you try to connect to the database, your credentials are no longer valid.

You can check that in you MongoHQ admin panel. You can find it by going to Heroku, selecting your app, clicking on resources and clicking MongoHQ. Then click on the Database users tab.

rake db:reseed used to work as you expected, but it changed with the new version of mongoid. In order to remove all collections, except system.*, you may use rake db:purge.

You may need to use the master branch since that rake task is not yet available in the stable version. Or just use Mongoid.purge! in your code, that works in stable.

Details: https://github.com/mongoid/mongoid/issues/2275