Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

MongoDB killed by linux

My mongoDB always receives a KILL signal when it is a few hours without being accessed and I do not know the reason.

I'm using:
S.O: Ubuntu 16.04 with 512MB ram.
Running node 7.5.0 in pm2.
MongoDB version 3.4.1

status mongod:

mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/lib/systemd/system/mongod.service; enabled; vendor preset: enabled)
Active: failed (Result: signal) since Fri 2017-06-02 06:25:06 BRT; 9h ago
 Docs: https://docs.mongodb.org/manual
Process: 1334 ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf (code=killed, signal=KILL)
Main PID: 1334 (code=killed, signal=KILL)

May 29 19:12:16 admin systemd[1]: Started High-performance, schema-free document-oriented database.
Jun 02 06:25:06 admin systemd[1]: mongod.service: Main process exited, code=killed, status=9/KILL
Jun 02 06:25:06 admin systemd[1]: mongod.service: Unit entered failed state.
Jun 02 06:25:06 admin systemd[1]: mongod.service: Failed with result 'signal'.

Mongod.service:

[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
Documentation=https://docs.mongodb.org/manual

[Service]
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
# file size
LimitFSIZE=infinity
# cpu time
LimitCPU=infinity
# virtual memory size
LimitAS=infinity
# open files
LimitNOFILE=64000
# processes/threads
LimitNPROC=64000
# total threads (user+kernel)
TasksMax=infinity
TasksAccounting=false

[Install]
WantedBy=multi-user.target

etc/mongod.conf:

storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true

systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

net:
  port: 27017
  bindIp: 127.0.0.1

processManagement:
  fork: false

journalctl:

Jun 01 22:32:02 admin pm2[23634]: [PM2] Spawning PM2 daemon with pm2_home=/home/administrator/.pm2
Jun 01 22:32:02 admin pm2[23634]: [PM2] PM2 Successfully daemonized
Jun 01 22:32:02 admin pm2[23634]: [PM2] Stopping PM2...
Jun 01 22:32:02 admin pm2[23634]: [PM2][WARN] No process found
Jun 01 22:32:02 admin pm2[23634]: [PM2] All processes have been stopped and deleted
Jun 01 22:32:02 admin pm2[23634]: [PM2] PM2 stopped
Jun 01 22:32:06 admin pm2[23649]: pm2 launched in no-daemon mode (you can add DEBUG="*" env variable to get more messages)
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: Launching in no daemon mode
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: [PM2] Resurrecting
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: [PM2] Restoring processes located in /home/administrator/.pm2/dump.pm2
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: [Watch] Start watching 0
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: Starting execution sequence in -fork mode- for app name:MY_NODE id:1
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: App name:MY_NODE id:1 online
Jun 01 22:32:06 admin pm2[23649]: 2017-06-01 22:32:06: [PM2] Process /home/servers/nodejs/server_MY_NODE/server.js restored
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: ┌─────────────┬────┬──────┬───────┬────────┬─────────┬────────┬─────┬───────────┬──────────┐
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: │ App name    │ id │ mode │ pid   │ status │ restart │ uptime │ cpu │ mem       │ watching │
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: ├─────────────┼────┼──────┼───────┼────────┼─────────┼────────┼─────┼───────────┼──────────┤
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: │   MY_NODE   │ 1  │ fork │ 23665 │ online │ 0       │ 0s     │ 24% │ 23.9 MB   │ disabled │
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: └─────────────┴────┴──────┴───────┴────────┴─────────┴────────┴─────┴───────────┴──────────┘
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07:  Use `pm2 show <id|name>` to get more details about an app
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: [--no-daemon] Continue to stream logs
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: [--no-daemon] Exit on target PM2 exit pid=23649
Jun 01 22:32:07 admin pm2[23649]: 2017-06-01 22:32:07: [STREAMING] Now streaming realtime logs for [all] processes
Jun 01 22:32:07 admin pm2[23649]: 22:32:07 1|MY_NODE | WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
Jun 01 22:32:07 admin pm2[23649]: 22:32:07 1|MY_NODE | WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
Jun 01 22:32:09 admin pm2[23649]: 22:32:09 1|MY_NODE | Express Server in 10555
Jun 01 22:32:09 admin pm2[23649]: 22:32:09 1|MY_NODE | Database! connected in mongodb://localhost/MY_MONGO
Jun 02 06:25:05 admin pm2[23649]: 2017-06-02 06:25:05: App [MY_NODE] with id [1] and pid [23665], exited with code [0] via signal [SIGKILL]
Jun 02 06:25:05 admin pm2[23649]: 2017-06-02 06:25:05: Starting execution sequence in -fork mode- for app name:MY_NODE id:1
Jun 02 06:25:05 admin pm2[23649]: 06:25:05 PM2        | App [MY_NODE] with id [1] and pid [23665], exited with code [0] via signal [SIGKILL]
Jun 02 06:25:05 admin pm2[23649]: 06:25:05 PM2        | Starting execution sequence in -fork mode- for app name:MY_NODE id:1
Jun 02 06:25:05 admin pm2[23649]: 2017-06-02 06:25:05: App name:MY_NODE id:1 online
Jun 02 06:25:05 admin pm2[23649]: 06:25:05 PM2        | App name:MY_NODE id:1 online
Jun 02 06:25:06 admin pm2[23649]: 06:25:06 1|MY_NODE | WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
Jun 02 06:25:06 admin pm2[23649]: 06:25:06 1|MY_NODE | WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
Jun 02 06:25:08 admin pm2[23649]: 06:25:08 1|MY_NODE | Express Server in 10555
Jun 02 06:25:08 admin pm2[23649]: 06:25:08 1|MY_NODE | Database! disconnected from mongodb://localhost/MY_MONGO
Jun 02 06:25:08 admin pm2[23649]: 06:25:08 1|MY_NODE | Database! Error in connection mongodb://localhost/MY_MONGO

Who can help me, I'll be grateful

thanks

like image 928
Armando Avatar asked Jun 02 '17 19:06

Armando


1 Answers

According to the Mongo-DB documentation, Ulimit can affect the mongo service. I fixed the same problem by running: ulimit -a to verify this was the problem.

If the -n flag is less than 64,000 you have to change it: sudo ulimit -n 64000

Then restart mongo: sudo systemctl restart mongod

And make sure it is running now: systemctl status mongod

like image 61
c10ud Avatar answered Oct 30 '22 04:10

c10ud