java8 -XX:+UseCompressedOops -XX:ObjectAlignmentInBytes=16

Question

So, I'm trying to run some simple code, jdk-8, output via jol

    System.out.println(VMSupport.vmDetails());
    Integer i = new Integer(23);
    System.out.println(ClassLayout.parseInstance(i)
            .toPrintable());

The first attempt is to run it with compressed oops disabled and compressed klass also on 64-bit JVM.

-XX:-UseCompressedOops -XX:-UseCompressedClassPointers

The output, pretty much expected is :

Running 64-bit HotSpot VM.
Objects are 8 bytes aligned.

java.lang.Integer object internals:
OFFSET  SIZE  TYPE DESCRIPTION                    VALUE
  0     4       (object header)                01 00 00 00 (00000001 00000000 00000000 00000000) (1)
  4     4       (object header)                00 00 00 00 (00000000 00000000 00000000 00000000) (0)
  8     4       (object header)                48 33 36 97 (01001000 00110011 00110110 10010111) (-1758055608)
 12     4       (object header)                01 00 00 00 (00000001 00000000 00000000 00000000) (1)
 16     4   int Integer.value                  23
 20     4       (loss due to the next object alignment)

Instance size: 24 bytes (reported by Instrumentation API)
Space losses: 0 bytes internal + 4 bytes external = 4 bytes total

That makes sense : 8 bytes klass word + 8 bytes mark word + 4 bytes for the actual value and 4 for padding (to align on 8 bytes) = 24 bytes.

The second attempt it to run it with compressed oops enabled compressed klass also on 64-bit JVM.

Again, the output is pretty much understandable :

Running 64-bit HotSpot VM.
Using compressed oop with 3-bit shift.
Using compressed klass with 3-bit shift.
Objects are 8 bytes aligned.

OFFSET  SIZE  TYPE DESCRIPTION                    VALUE
  0     4       (object header)                01 00 00 00 (00000001 00000000 00000000 00000000) (1)
  4     4       (object header)                00 00 00 00 (00000000 00000000 00000000 00000000) (0)
  8     4       (object header)                f9 33 01 f8 (11111001 00110011 00000001 11111000) (-134138887)
 12     4   int Dummy.i                        42
 Instance size: 16 bytes (reported by Instrumentation API).

4 bytes compressed oop (klass word) + 8 bytes mark word + 4 bytes for the value + no space loss = 16 bytes.

The thing that does NOT make sense to me is this use-case:

 -XX:+UseCompressedOops -XX:+UseCompressedClassPointers -XX:ObjectAlignmentInBytes=16

The output is this:

 Running 64-bit HotSpot VM.
 Using compressed oop with 4-bit shift.
 Using compressed klass with 0x0000001000000000 base address and 0-bit shift.

I was really expecting to both be "4-bit shift". Why they are not?

EDIT The second example is run with :

 XX:+UseCompressedOops -XX:+UseCompressedClassPointers

And the third one with :

 -XX:+UseCompressedOops -XX:+UseCompressedClassPointers -XX:ObjectAlignmentInBytes=16

Answer 1

I will try to extend a little bit on the answer provided by Alexey as some things might not be obvious.

Following Alexey suggestion, if we search the source code of OpenJDK for where compressed klass bit shift value is assigned, we will find the following code in metaspace.cpp:

void Metaspace::set_narrow_klass_base_and_shift(address metaspace_base, address cds_base) {
// some code removed 
if ((uint64_t)(higher_address - lower_base) <= UnscaledClassSpaceMax) {
  Universe::set_narrow_klass_shift(0);
} else {
  assert(!UseSharedSpaces, "Cannot shift with UseSharedSpaces");
  Universe::set_narrow_klass_shift(LogKlassAlignmentInBytes);
}

As we can see, the class shift can either be 0(or basically no shifting) or 3 bits, because LogKlassAlignmentInBytes is a constant defined in globalDefinitions.hpp:

const int LogKlassAlignmentInBytes = 3;

So, the answer to your quetion:

I was really expecting to both be "4-bit shift". Why they are not?

is that ObjectAlignmentInBytes does not have any effect on compressed class pointers alignment in the metaspace which is always 8bytes.

Of course this conclusion does not answer the question:

"Why when using -XX:ObjectAlignmentInBytes=16 with -XX:+UseCompressedClassPointers the narrow klass shift becomes zero? Also, without shifting how can the JVM reference the class space with 32-bit references, if the heap is 4GBytes or more?"

We already know that the class space is allocated on top of the java heap and can be up to 3G in size. With that in mind let's make a few tests. -XX:+UseCompressedOops -XX:+UseCompressedClassPointers are enabled by default, so we can eliminate these for conciseness.

Test 1: Defaults - 8 Bytes aligned

$ java -XX:ObjectAlignmentInBytes=8 -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode -version

heap address: 0x00000006c0000000, size: 4096 MB, zero based Compressed Oops

Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
Compressed class space size: 1073741824 Address: 0x00000007c0000000 Req Addr: 0x00000007c0000000

Notice that the heap starts at address 0x00000006c0000000 in the virtual space and has a size of 4GBytes. Let's jump by 4Gbytes from where the heap starts and we land just where class space begins.

0x00000006c0000000 + 0x0000000100000000 = 0x00000007c0000000

The class space size is 1Gbyte, so let's jump by another 1Gbyte:

0x00000007c0000000 + 0x0000000040000000 = 0x0000000800000000

and we land just below 32Gbytes. With a 3 bits class space shifting the JVM is able to reference the entire class space, although it's at the limit (intentionally).

Test 2: 16 bytes aligned

java -XX:ObjectAlignmentInBytes=16 -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode -version

heap address: 0x0000000f00000000, size: 4096 MB, zero based Compressed Oops

Narrow klass base: 0x0000001000000000, Narrow klass shift: 0
Compressed class space size: 1073741824 Address: 0x0000001000000000 Req Addr: 0x0000001000000000

This time we can observe that the heap address is different, but let's try the same steps:

0x0000000f00000000 + 0x0000000100000000 = 0x0000001000000000

This time around heap space ends just below 64GBytes virtual space boundary and the class space is allocated above 64Gbyte boundary. Since class space can use only 3 bits shifting, how can the JVM reference the class space located above 64Gbyte? The key is:

Narrow klass base: 0x0000001000000000

The JVM still uses 32 bit compressed pointers for the class space, but when encoding and decoding these, it will always add 0x0000001000000000 base to the compressed reference instead of using shifting. Note, that this approach works as long as the referenced chunk of memory is lower than 4Gbytes (the limit for 32 bits references). Considering that the class space can have a maximum of 3Gbytes we are comfortably within the limits.

3: 16 bytes aligned, pin heap base at 8g

$ java -XX:ObjectAlignmentInBytes=16 -XX:HeapBaseMinAddress=8g -XX:+UnlockDiagnosticVMOptions -XX:+PrintCompressedOopsMode -version

heap address: 0x0000000200000000, size: 4096 MB, zero based Compressed Oops

Narrow klass base: 0x0000000000000000, Narrow klass shift: 3
Compressed class space size: 1073741824 Address: 0x0000000300000000 Req Addr: 0x0000000300000000

In this test we are still keeping the -XX:ObjectAlignmentInBytes=16, but also asking the JVM to allocate the heap at the 8th GByte in the virtual address space using -XX:HeapBaseMinAddress=8g JVM argument. The class space will begin at 12th GByte in the virtual address space and 3 bits shifting is more than enough to reference it.

Hopefully, these tests and their results answer the question:

"Why when using -XX:ObjectAlignmentInBytes=16 with -XX:+UseCompressedClassPointers the narrow klass shift becomes zero? Also, without shifting how can the JVM reference the class space with 32-bit references, if the heap is 4GBytes or more?"

Answer 2

Answers to these questions are mostly easy to figure out when looking into OpenJDK code.

For example, grep for "UseCompressedClassPointers", this will get you to arguments.cpp:

// Check the CompressedClassSpaceSize to make sure we use compressed klass ptrs.
if (UseCompressedClassPointers) {
  if (CompressedClassSpaceSize > KlassEncodingMetaspaceMax) {
    warning("CompressedClassSpaceSize is too large for UseCompressedClassPointers");
    FLAG_SET_DEFAULT(UseCompressedClassPointers, false);
  }
}

Okay, interesting, there is "CompressedClassSpaceSize"? Grep for its definition, it's in globals.hpp:

  product(size_t, CompressedClassSpaceSize, 1*G,                            \
          "Maximum size of class area in Metaspace when compressed "        \
          "class pointers are used")                                        \
          range(1*M, 3*G)                                                   \

Aha, so the class area is in Metaspace, and it takes somewhere between 1 Mb and 3 Gb of space. Let's grep for "CompressedClassSpaceSize" usages, because that will take us to actual code that handles it, say in metaspace.cpp:

// For UseCompressedClassPointers the class space is reserved above
// the top of the Java heap. The argument passed in is at the base of
// the compressed space.
void Metaspace::initialize_class_space(ReservedSpace rs) {

So, compressed classes are allocated in a smaller class space outside the Java heap, which does not require shifting -- even 3 gigabytes is small enough to use only the lowest 32 bits.