Mongo Restart Error -- /var/run/mongodb/mongod.pid exists

Question

I just started a new AWS Linux AMI 2016.09.1(HVM) and downloaded the latest Mongodb release. I am using MongoDB version 3.0.14. I updated /etc/mongod.conf to the following,

# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log

# Where and how to store data.
storage:
dbPath: /var/lib/mongo
journal:
enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod.pid  # location of pidfile

# network interfaces
net:
  port: 27017
#  bindIp: 127.0.0.1  # Listen to local interface only, comment to listen on
    all interfaces.


#security:

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:

I run Mongodb for the first time using service,

sudo service mongod start

Everything works fine. When I stop Mongodb and restart,

sudo service mongod stop
sudo service mongod restart

I get the following error,

Error starting mongod. /var/run/mongodb/mongod.pid exists.

Here is what is in the logfile, /var/log/mongodb/mongod.log,

2017-01-16T14:11:16.869+0000 I CONTROL  ***** SERVER RESTARTED *****
2017-01-16T14:11:16.872+0000 I CONTROL  [initandlisten] MongoDB starting :
    pid=2820 port=27017 dbpath=/var/lib/mongo 64-bit hos
    t=ip-10-0-0-91
2017-01-16T14:11:16.872+0000 I CONTROL  [initandlisten] db version v3.0.14
2017-01-16T14:11:16.872+0000 I CONTROL  [initandlisten] git version:
    08352afcca24bfc145240a0fac9d28b978ab77f3
2017-01-16T14:11:16.872+0000 I CONTROL  [initandlisten] build info: Linux
    ip-10-63-190-181 3.4.43-43.43.amzn1.x86_64 #1 SMP Mon
    May 6 18:04:41 UTC 2013 x86_64 BOOST_LIB_VERSION=1_49
2017-01-16T14:11:16.872+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2017-01-16T14:11:16.872+0000 I CONTROL  [initandlisten] options: { config:
    "/etc/mongod.conf", net: { port: 27017 }, processMan
    agement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" },
    storage: { dbPath: "/var/lib/mongo", journal: { enabled: t
    rue } }, systemLog: { destination: "file", logAppend: true, path:   
    "/var/log/mongodb/mongod.log" } }
2017-01-16T14:11:16.893+0000 E NETWORK  [initandlisten] Failed to unlink
    socket file /tmp/mongodb-27017.sock errno:1 Operation
    not permitted
2017-01-16T14:11:16.893+0000 I -        [initandlisten] Fatal Assertion   
    28578
2017-01-16T14:11:16.893+0000 I -        [initandlisten]

***aborting after fassert() failure

I have seen several blogs posted about this problem with solutions but none of them seem to work. Spent all day Sunday trying to figure this one out. Never had this problem before running an earlier AWS instance and Mongo.

Answer 1

Ok, found the problem. I deleted the /var/run/mongodb/mongod.pid file as suggested by Dave Maples above. However, MongoDB would still not run. I looked at the log file again and now saw it barking about this file,

/tmp/mongodb-27017.sock

It looked like it did not have the correct permissions. I tried,

sudo chown mongod:mongod /tmp/mongodb-27017.sock

And it worked fine. Now when I start/stop/restart MongoDB multiple times I no longer have the problem. I think the file permissions caused MongoDB to not clean itself up properly and left /var/run/mongodb/mongod.pid file in the directory. This caused the "Error starting mongod. /var/run/mongodb/mongod.pid exists" error. However, this was not the real problem. The real problem was the /tmp/ permissions. Looking at the log file after I deleted the mongod.pid file and restarting Mongod showed the permissions problem. So here is what fixed the problem,

sudo rm /var/run/mongodb/mongod.pid
sudo chown mongod:mongod /tmp/mongodb-27017.sock

I bet if I just tried "sudo chown mongod:mongod /tmp/mongodb-27017.sock" right after installing MongoDB on the new server before running MongoDB would have avoided the problem. Wasted a perfectly good Florida sunny day on this one.