mongodb high cpu usage

Question

I have installed MongoDB 2.4.4 on Amazon EC2 with ubuntu 64 bit OS and 1.6 GB RAM.

On this server, only MongoDB running nothing else.

But sometime CPU usage reach to 99% and load average: 500.01, 400.73, 620.77

I have also installed MMS on server to monitor what's going on server.

Here is MMS detail

As per MMS details, indexing working perfectly for each queries.

Suspect details as below

1) HIGH non-mapped virtual memory

2) HIGH page faults

Can anyone help me to understand what exactly causing high CPU usage ?

EDIT:

After comments of @Dylan Tong, i have reduced active connetions but still there is high non-mapped virtual memory

Answer 1

Here's a summary of a few things to look into:

1. Observed a large number of connections and cursors (13k): - fix: make sure your connection pool is appropriate. For reporting, and your current request rate, you only need a few connections at most. Also, I'm guessing you have a m1small instance, which means you only have 1 core.

2. Review queries and indexes: - run your queries with explain(), to observe how the queries are executed. The right model normally results in queries only pulling very few documents and utilization of an index.

3. Memory (compact and readahead setting): - make the best use of memory. 1.6GB is low. Check how much free memory you have, and compare it to what is reported as resident. A couple of common causes of low resident memory is due to fragmentation. If there are alot of documents moving, changing size and such, you should run the compact command to defragment your data files. Also, a bad readahead can lead to poor use of memory as well. Check your readahead setting (http://manpages.ubuntu.com/manpages/lucid/man2/readahead.2.html). Try a few values starting with low values (http://docs.mongodb.org/manual/administration/production-notes/). The production notes recommend 32 (for standard 512byte blocks). Sometimes higher values are optimal if your documents are larger. The hope is that resident memory should be close to your available memory and your page faults should start to lower.

If you're using resources to the fullest after this, and you're still capped out on CPU then it means you need to up your resources.