JVM Crash due to SIGSEGV

Question

Our server hung because of a SIGSEGV fault..

A fatal error has been detected by the Java Runtime Environment:

  SIGSEGV (0xb) at pc=0x00007ff5c7195aaa, pid=262778, tid=140690480097024

 JRE version: 6.0_35-b10
 Java VM: Java HotSpot(TM) 64-Bit Server VM (20.10-b01 mixed mode linux-amd64 compressed oops)
 Problematic frame:
 C  [libdtagentcore.so+0xb7aaa]  long double restrict+0x506f6

I am curious to know what could be the root cause of this?

Any help is highly appreciated..Thanks..

Answer 1

Signal Description

SIGSEGV, SIGBUS, SIGFPE, SIGPIPE, SIGILL -- Used in the implementation for implicit null check, and so forth.

SIGQUIT Thread dump support -- To dump Java stack traces at the standard error stream. (Optional.)

SIGTERM, SIGINT, SIGHUP -- Used to support the shutdown hook mechanism (java.lang.Runtime.addShutdownHook) when the VM is terminated abnormally. (Optional.)

SIGUSR1 -- Used in the implementation of the java.lang.Thread.interrupt method. (Configurable.) Not used starting with Solaris 10 OS. Reserved on Linux. SIGUSR2 Used internally. (Configurable.) Not used starting with Solaris 10 OS. SIGABRT The HotSpot VM does not handle this signal. Instead it calls the abort function after fatal error handling. If an application uses this signal then it should terminate the process to preserve the expected semantics.

The fatal error log indicates that the crash was in a native library, there might be a bug in native code or JNI library code. The crash could of course be caused by something else, but analysis of the library and any core file or crash dump is a good starting place.

In this case a SIGSEGV occurred with a thread executing in the library libdtagentcore.so . In some cases a bug in a native library manifests itself as a crash in Java VM code. Consider the following crash where a JavaThread fails while in the _thread_in_vm state (meaning that it is executing in Java VM code)

• If you get a crash in a native application library (as in your case), then you might be able to attach the native debugger to the core file or crash dump, if it is available. Depending on the operating system, the native debugger is dbx, gdb, or windbg.
• Another approach is to run with the `-Xcheck:jni` option added to the command line. This option is not guaranteed to find all issues with JNI code, but it can help identify a significant number of issues.
• If the native library where the crash occurred is part of the Java runtime environment (for example awt.dll, net.dll, and so forth), then it is possible that you have encountered a library or API bug. If after further analysis you conclude this is a library or API bug, then gather a much data as possible and submit a bug or support call.

Answer 2

It is telling you that an error occurred in code loaded from libdtagentcore.so. More specifically it happened in function named restrict and at offset 0x506f6. The first offset mentioned (0xb7aaa) is offset within the library itself. If it was build with debugging symbols (-g) you can look at the code that caused the exception, on Linux something along the lines of:

addr2line -e libdtagentcore.so -C -f 0xb7aaa

In case this is read by someone on Windows, see https://community.oracle.com/blogs/kohsuke/2009/02/19/crash-course-jvm-crash-analysis

More details in https://www.youtube.com/watch?v=jd6dJa7tSNU