How to fix 'Cannot start transaction # on session` error in mongo driver ruby gem

Question

I have a rails app that I'm trying to connect to a MongoDB Atlas cluster (v3.6) via Mongoid. I keep running into this error:

[ERROR] Cannot start transaction 4 on session c6e79139-1be1-4c1c-9d13-a5e7ca45c792 - X6aX2zlpqRGB3UhpvFsz7wCO85u1m8A5+nkcYk7NDoM= because a newer transaction 30 has already started. (225)
[ERROR] /home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/operation/result.rb:267:in `raise_operation_failure'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/operation/result.rb:263:in `validate!'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/operation/shared/write.rb:42:in `execute'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/collection/view/writable.rb:307:in `block (2 levels) in update_one'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/retryable.rb:133:in `write_with_retry'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/collection/view/writable.rb:295:in `block in update_one'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/cluster.rb:655:in `with_session'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/client.rb:662:in `with_session'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/collection/view.rb:204:in `with_session'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/mongo-2.7.0/lib/mongo/collection/view/writable.rb:294:in `update_one'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/query_cache.rb:182:in `update_one_with_clear_cache'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/persistable/updatable.rb:144:in `block in update_document'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/persistable/updatable.rb:118:in `block (2 levels) in prepare_update'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:117:in `call'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:505:in `call'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:778:in `_run_update_callbacks'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:81:in `run_callbacks'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/interceptable.rb:138:in `run_callbacks'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/persistable/updatable.rb:117:in `block in prepare_update'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:117:in `call'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:505:in `call'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:778:in `_run_save_callbacks'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/gems/activesupport-4.2.11/lib/active_support/callbacks.rb:81:in `run_callbacks'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/interceptable.rb:138:in `run_callbacks'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/persistable/updatable.rb:116:in `prepare_update'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/persistable/updatable.rb:139:in `update_document'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/persistable/savable.rb:25:in `save'
/home/dev/.rvm/gems/ruby-2.4.4@myproj/bundler/gems/mongoid-9afa64b7b0c0/lib/mongoid/relations/proxy.rb:151:in `method_missing'
...

We aren't doing any explicit transaction management and have run without issues on Compose MongoDB cluster (v3.0.11). It also seems to work correctly with MongoDB Atlas v3.4.20.

Here is the config causing the issue:

• MongoDB Atlas cluster: 3.6.11
• Mongo gem: 2.7.0
• Mongoid: 5.4.0
• Rails: 4.2.11
• Ruby: 2.4.4

With the same config, but using a Compose MongoDB 3.0.11 cluster, it all works fine.

I suspect the issue is related to the change in Mongo 3.6 to logical sessions, but can't find anywhere in Mongoid that would be creating transactions.

My mongoid.yml looks like this:

    production:
      clients:
        default:
          uri: <%= ENV['MONGODB_URI'] %>

and

    MONGODB_URI="mongodb://USERNAME:[email protected]:27017,cluster0-shard-00-01-xxxxx.mongodb.net:27017,cluster0-shard-00-02-xxxxx.mongodb.net:27017/db-name?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin&retryWrites=true"

Answer 1

I've been banging my head against this for the last few hours. I've found 2 possible solutions/suggestions:

1.) Set retryWrites to "false" in your connection string. I haven't been able to find any indication as to why this would fix the issue, but I've seen it mentioned in 2 different non-MongoDB-official docs. Be warned that retryWrites is enabled by default in recent versions of Mongo, and that is no-doubt for good reason, so take that into consideration before turning the feature off.

2.) If you're using a forking webserver (Unicorn, Puma, Passenger, etc), ensure that you're creating a new Client connection after you fork. As with virtually everything Mongoid-related, documentation on the recommended way to make that happen is scarce, and I've seen at least one doc that seems to indicate that this is not actually necessary. So again, your mileage may vary.

Just figured I'd post this answer in case either suggestion happens to be useful to someone in the future. I can't vouch for the accuracy/validity of either suggestion. I've attempted solution #2 but have not been able to test it in anger yet, so I don't know if it will help or not.