Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to analyze memory leak from coredump

Tags:

c

coredump

gcore

I would like to analyze a memory leak from core file analysis.

I have written sample code to inject memory leak and generate core file with gcore command.

#include <stdlib.h>
#include <unistd.h>
void fun()
{
  int *ptr = new int(1234);
}
int main()
{
  int i=0;
  while(i++<2500)
  {
    fun();
}
sleep(360);
return 0;
}

Find the process ID

ayadav@ajay-PC:~$ ps -aef |grep over  
ajay      8735  6016  0 12:57 pts/2    00:00:00 ./over  
ayadav    8739  4659  0 12:57 pts/10   00:00:00 grep over  

and generated core

ayadav@ajay-PC:~$ sudo gcore 8735
[sudo] password for ayadav:
0x00007fbb7dda99a0 in __nanosleep_nocancel () at ../sysdeps/unix/syscall-template.S:81
81      ../sysdeps/unix/syscall-template.S: No such file or directory.
Saved corefile core.8735

I have found common patterns from core file as below(as suggested on stackoverflow another thread Is there a way to locate which part of the process used the most of the memory, only looking at a generated core file?)

ayadav@ajay-PC:~$ hexdump core.6015 | awk '{printf "%s%s%s%s\n%s%s%s%s\n", $5,$4,$3,$2,$9,$8,$7,$6}' | sort | uniq -c | sort -nr | head
6913 0000000000000000  
2503 0000002100000000  
2501 000004d200000000  
786 0000000000007ffc  
464  
125 1ccbc4d000007ffc  
 92 1ca7ead000000000  
 91 0000000200007ffc  
 89 0000000100007ffc  
 80 0000000100000000  

Below two addresses are suspected one

2503 0000002100000000  
2501 000004d200000000  

core file have following repeated patterns

0003560 0000 0000 0021 0000 0000 0000 04d2 0000  
0003570 0000 0000 0000 0000 0000 0000 0000 0000  
0003580 0000 0000 0021 0000 0000 0000 04d2 0000  
0003590 0000 0000 0000 0000 0000 0000 0000 0000  
00035a0 0000 0000 0021 0000 0000 0000 04d2 0000  
00035b0 0000 0000 0000 0000 0000 0000 0000 0000  
00035c0 0000 0000 0021 0000 0000 0000 04d2 0000  
00035d0 0000 0000 0000 0000 0000 0000 0000 0000  
00035e0 0000 0000 0021 0000 0000 0000 04d2 0000  
00035f0 0000 0000 0000 0000 0000 0000 0000 0000  
0003600 0000 0000 0021 0000 0000 0000 04d2 0000  
0003610 0000 0000 0000 0000 0000 0000 0000 0000  
0003620 0000 0000 0021 0000 0000 0000 04d2 0000  
0003630 0000 0000 0000 0000 0000 0000 0000 0000  
0003640 0000 0000 0021 0000 0000 0000 04d2 0000

But I do not have much idea how I can access it from command like gdb info address or x. Could anybody tell how I can convert symbol information from binary format?

like image 711
Ajay yadav Avatar asked Dec 22 '14 08:12

Ajay yadav


People also ask

How to analyze a memory leak in a core application?

This tutorial demonstrates the tools to analyze a memory leak in a .NET Core app using the .NET diagnostics CLI tools. If you are on Windows, you may be able to use Visual Studio's Memory Diagnostic toolsto debug the memory leak. This tutorial uses a sample app, which is designed to intentionally leak memory. The sample is provided as an exercise.

How to analyze a core dump?

Navigate to the terminal that started the server, and press Ctrl+C. Analyze the core dump Now that you have a core dump generated, use the dotnet-dumptool to analyze the dump:

What is a core dump file?

What is Core Dump? A core dump is a file that stores a snapshot of the program memory at the program crash. A core dump file is generated when the program terminates by the SIGSEG signal because of tried to access the invalid memory address. A program terminates or crashes due to various OS signals like SIGSEG, SIGKILL, SIGABORT, etc.

How to check and monitor memory leak on Linux?

The selection of tools would vary based on your requirement. There are many other tools such as YAMD, Electric fence, gdb core dump etc which can help you capture memory leak. Lastly I hope the steps from the article to check and monitor memory leak on Linux was helpful.


1 Answers

1 - Memory leaks can be evaluated with a core dump. I have taken a sample c++ example:

class Base  
{  
public:  
    virtual void fun(){}  
    virtual void xyz(){}  
    virtual void lmv(){}  
    virtual void abc(){}  
};  

class Derived: public Base  
{  
public:  
    void fun(){}  
    void xyz(){}  
    void lmv(){}  
    void abc(){}  
};  

void fun()  
{  
    Base *obj  = new Derived();  
}  
int main()  
{  
    for(int i = 0; i < 2500;i++)
    {
        fun();
    }
    sleep(3600);
    return 0; 
}

2 - Created a core with gcore command

3 - Search the repeated pattern from core file.

ayadav@ajay-PC:~$ hexdump core.10639 | awk '{printf "%s%s%s%s\n%s%s%s%s\n", $5,$4,$3,$2,$9,$8,$7,$6}' | sort | uniq -c | sort -nr  | head
   6685 0000000000000000  
   2502 0000002100000000  
   2500 004008d000000000  
    726 0000000000007eff  
    502   
    125 2e4314d000007eff  
     93 006010d000000000  
     81 0000000100007eff  
     80 0000000100000000  
     73 0000000000000001  

0000002100000000 and 004008d000000000 are repeated patterns

4 - Check each qword is what with?

(gdb) info symbol ...
(gdb) x ...

example:

(gdb) info symbol 0x4008d000
No symbol matches 0x4008d000.
(gdb) info symbol 0x4008d0
vtable for Derived + 16 in section .rodata of /home/ayadav/virtual

5 - Probably most frequent vtable must relate to memory leak i.e Derived vtable.

Note: I agree coredump analysis is not best practice to find memory leak. Memory leak can be find with different static and dynamic tools like valgrind etc.

like image 172
Ajay yadav Avatar answered Sep 20 '22 16:09

Ajay yadav