JNI Object Pointers

Question

Problem

When experimenting with the JNI interface, I was wondering if I could take a JObject and transmute it into an equivalent struct to manipulate the fields. However, when I tried I was surprised to see that this did not work. Ignoring how horrible this idea might be, why didn't it work?

My Approach

Java Test Class

I made a simple class Point to do my test. Point has two fields and a constructor that takes in an x and y as well as a few random methods that return information based on the fields.

public class Point {
    public final double x;
    public final double y;
    // As well as some random methods
}

Memory Layout

Using jol, I found the structure of my Point class in the java runtime (shown below).

C:\Users\home\IdeaProjects\test-project>java -cp jol-cli-0.9-full.jar;out\production\java-test org.openjdk.jol.Main internals Point
# Running 64-bit HotSpot VM.
# Using compressed oop with 3-bit shift.
# Using compressed klass with 3-bit shift.
# Objects are 8 bytes aligned.
# Field sizes by type: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]
# Array element sizes: 4, 1, 1, 2, 2, 4, 4, 8, 8 [bytes]

Instantiated the sample instance via public Point(double,double)

Point object internals:
 OFFSET  SIZE     TYPE DESCRIPTION                               VALUE
      0     4          (object header)                           01 00 00 00 (00000001 00000000 00000000 00000000) (1)
      4     4          (object header)                           00 00 00 00 (00000000 00000000 00000000 00000000) (0)
      8     4          (object header)                           31 32 01 f8 (00110001 00110010 00000001 11111000) (-134139343)
     12     4          (alignment/padding gap)
     16     8   double Point.x                                   0.0
     24     8   double Point.y                                   0.0
Instance size: 32 bytes
Space losses: 4 bytes internal + 0 bytes external = 4 bytes total

Test Struct

I wrote out a simple test struct that matched the memory model described by jol along with some tests to make sure that it had the same alignment and each element had the correct offset. I did this using rust, but it should be the same for any other compiled language.

#[derive(Debug)]
#[repr(C, align(8))]
pub struct Point {
    header1: u32,
    header2: u32,
    header3: u32,
    point_x: f64,
    point_y: f64,
}

Output

The final part of my test was making a jni function that took in a Point object and transmuted the point object into the point struct.

C Header (Included as Reference)

/*
 * Class:     Main
 * Method:    analyze
 * Signature: (LPoint;)V
 */
JNIEXPORT void JNICALL Java_Main_analyze
  (JNIEnv *, jclass, jobject);

Rust Implementation

#[no_mangle]
pub extern "system" fn Java_Main_analyze(env: JNIEnv, class: JClass, obj: JObject) {
    unsafe {
        // De-reference the `JObject` to get the object pointer, then transmute the
        // pointer into my `Point` struct.
        let obj_ptr = mem::transmute::<_, *const Point>(*obj);

        // Output the debug format of the `Point` struct
        println!("{:?}", *obj_ptr);
    }
}

Runs

Every time I ran it, I got a different result.

// First Run:
Point { header1: 1802087032, header2: 7, header3: 43906792, point_x: 
0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000230641669, point_y: 
0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000021692881 }

// Second Run:
Point { header1: 1802087832, header2: 7, header3: 42529864, point_x: 
0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000229832192, point_y: 
0.000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
0000021012588 }

Version Information

C:\Users\home\IdeaProjects\test-project>java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

Edit: I did this on Windows 10 Home 10.0.18362 Build 18362

Edit 2: Since I used rust to approach this problem I used rust's jni crate. It provided the JObject type I mentioned above. It just occurred to me that there might be some confusion since JObject is not the same as the jobject shown in the C header. JObject is a rust wrapper around a pointer to a jobject, hence my dereferencing it before transmuting the pointer.

Answer 1

Conceptual Explanation

After reading a paper on memory compaction, I learned that java references are made of two pointers. During garbage collection, a compaction step occurs. To ensure that the references will still line up, two pointers are used to prevent mangling when objects are moved. A reference consists of a pointer to another pointer that points to the object. When the compaction step occurs and an object is moved through memory, only the second pointer needs to be altered.

In other words a reference is really a pointer to the location in memory that points to an object.

I know I mangled that explanation, but hopefully it was mostly readable/accurate.

What I did wrong

Code Changed

#[no_mangle]
pub extern "system" fn Java_Main_analyze(env: JNIEnv, class: JClass, obj: JObject) {
    unsafe {
        // It should have been transmuted to a pointer to a pointer and dereferenced twice.
        let indirect = mem::transmute::<_, *const *const Point>(*obj);
        println!("{:?}", **indirect);
    }
}

New Output

Once that fix was applied, the object data began to line up correctly. The x and y are the same as in the test object, and all three headers line up with jol's prediction for the memory format (I assume header 3 would have been the same if I had used a signed integer).

Point { header1: 1, header2: 0, header3: 4160799044, point_x: -3.472, point_y: 4.0 }

In response to @Botje: My Point struct was correct, but you were unable to recreate the error because you approached the problem correctly from the start while I did not.