Why doesn't the JVM crash when entering infinite recursion?

Question

I'm writing a shared library to be loaded into the JVM and the behavior below got me stuck. Here are my Java classes:

package com.test;

public class UnixUtil {
    static {
        System.loadLibrary("myfancylibrary");
    }
    static native int openReadOnlyFd(String path);
    static native int closeFd(int fd);
}

public class Main {

    public static void main(String[] args){
        int fd = UnixUtil.openReadOnlyFd("/tmp/testc");
        UnixUtil.closeFd(fd);
    }
}

And the library to be loaded looks like:

test_jni.h

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class com_test_UnixUtil */

#ifndef _Included_com_test_UnixUtil
#define _Included_com_test_UnixUtil
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     com_test_UnixUtil
 * Method:    openReadOnlyFd
 * Signature: (Ljava/lang/String;)I
 */
JNIEXPORT jint JNICALL Java_com_test_UnixUtil_openReadOnlyFd
  (JNIEnv *, jclass, jstring);

/*
 * Class:     com_test_UnixUtil
 * Method:    closeFd
 * Signature: (I)I
 */
JNIEXPORT jint JNICALL Java_com_test_UnixUtil_closeFd
  (JNIEnv *, jclass, jint);

#ifdef __cplusplus
}
#endif
#endif

test_jni.c

#include "test_jni.h"
#include "fs.h"


JNIEXPORT jint JNICALL Java_com_test_UnixUtil_openReadOnlyFd
  (JNIEnv *e, jclass jc, jstring path){
  const char *const native_path = ((*e) -> GetStringUTFChars)(e, path, NULL);
  int fd = read_only_open(native_path);
  ((*e) -> ReleaseStringUTFChars)(e, path, native_path);
  return fd;
}


JNIEXPORT jint JNICALL Java_com_test_UnixUtil_closeFd
   (JNIEnv *e, jclass jc, jint fd){
   printf("Closing files descriptord %d... \n", fd);
   return close(fd);
}

fs.h

#ifndef FS_H
#define FS_H

int read_only_open(const char *path);

int close(int fd);

#endif //FS_H

fs.c

#include <unistd.h>
#include <string.h>
#include <errno.h>
#include <stdio.h>
#include <sys/fcntl.h>

#include "fs.h"

int read_only_open(const char *path){
    printf("Entering %s.%s:%d\n", __FILE__, __func__, __LINE__);
    int fd = open(path, O_RDONLY);
    return fd;
}

int close(int fd){ //Java_com_test_UnixUtil_closeFd does not invoke this function
    printf("Entering %s.%s:%d\n", __FILE__, __func__, __LINE__);
    int close_result = close(fd);
    return close_result;
}

When compiling and running this Main class the JVM does not crash. It simply does not enter the function fs.h::close(int). Instead, stdlib's close is called as can be seen in GDB:

Thread 2 "java" hit Breakpoint 1, Java_com_test_UnixUtil_closeFd (e=<optimized out>,
    jc=<optimized out>, fd=4) at /home/rjlomov/test_jni/src/main/java/com/test/lib/test_jni.c:17
17        return close(fd);
(gdb) step
18      }
(gdb) 
17        return close(fd);
(gdb) 
__close (fd=4) at ../sysdeps/unix/sysv/linux/close.c:27
27      ../sysdeps/unix/sysv/linux/close.c: No such file or directory.
(gdb) 
26      in ../sysdeps/unix/sysv/linux/close.c

Running objdump -dS libmyfancylibrary.so shows that

JNIEXPORT jint JNICALL Java_com_test_UnixUtil_closeFd                                                                                                                                                              
  (JNIEnv *e, jclass jc, jint fd){                                                                                                                                                                                 
 7d0:   53                      push   %rbx                                                                                                                                                                        
}   

//...

  return close(fd);                                                                                                                                                                                                
 7e9:   e9 62 fe ff ff          jmpq   650 <close@plt>   // <--- PLT section,
                                      // resolved by linker to stdlib::close?                                                                                                                                                         
 7ee:   66 90                   xchg   %ax,%ax        

QUESTION: Why is stdlib::close called in Java_com_test_UnixUtil_closeFd instead of fs.c::close(int)? The only thing I can imagine is the JVM has its own dynamic linker which does the job...

Answer 1

Since you are compiling a shared library, and the function close() is not static, the compiler makes an indirect call throught the Procedure Linkage Table (PLT). When disassembling the library you'll likely see an instruction

    call <close@plt>

When myfancylibrary is loaded, the process already has the implementation of close from libc, so the dynamic liker updates PLT to point to libc's version of close().