Java FFM `Linker.Option.captureCallState` does not successfully capture `GetLastError` on Windows

Question

I'm trying to retrieve GetLastError code on LoadLibraryW failure via Linker.Option.captureCallState. According to the documentation and this issue on JDK Bug System, captureCallState is the preferred way of retrieving such error codes.

MWE here (requires Java 22+):

import java.lang.foreign.*;
import java.lang.invoke.MethodHandle;
import java.lang.invoke.VarHandle;

public final class Drill {
    public static void main(String[] args) throws Throwable {
        System.loadLibrary("kernel32");

        Linker nativeLinker = Linker.nativeLinker();
        SymbolLookup stdlibLookup = nativeLinker.defaultLookup();
        SymbolLookup loaderLookup = SymbolLookup.loaderLookup();

        MemorySegment pfnLoadLibraryW = loaderLookup.find("LoadLibraryW")
                .or(() -> stdlibLookup.find("LoadLibraryW"))
                .orElse(MemorySegment.NULL);

        if (pfnLoadLibraryW.equals(MemorySegment.NULL)) {
            throw new RuntimeException("Failed to find LoadLibraryW symbol");
        }

        MethodHandle hLoadLibraryW = nativeLinker.downcallHandle(
                pfnLoadLibraryW,
                FunctionDescriptor.of(
                        ValueLayout.ADDRESS, // returns HMODULE
                        ValueLayout.ADDRESS.withTargetLayout(ValueLayout.JAVA_SHORT) // LPCWSTR lpLibFileName
                ),
                Linker.Option.captureCallState("GetLastError")
        );

        try (Arena arena = Arena.ofConfined()) {
            char[] libName = "nonexist".toCharArray();
            MemorySegment lpLibName = arena.allocate(ValueLayout.JAVA_SHORT, libName.length + 1);
            lpLibName.copyFrom(MemorySegment.ofArray(libName));

            StructLayout captureStateLayout = Linker.Option.captureStateLayout();
            MemorySegment capturedState = arena.allocate(captureStateLayout);

            MemorySegment result = (MemorySegment) hLoadLibraryW.invokeExact(lpLibName, capturedState);
            if (!result.equals(MemorySegment.NULL)) {
                throw new RuntimeException("Why would you even do this?");
            }

            VarHandle vh = captureStateLayout.varHandle(MemoryLayout.PathElement.groupElement("GetLastError"));
            int lastError = (int) vh.get(capturedState, 0L);
            System.out.println("GetLastError = " + lastError);
        }
    }
}

However, the captured GetLastError field is always 0.

I tested this with:

• Windows11 24H2 + Oracle OpenJDK 24
• Windows10 LTSC2019 + Oracle OpenJDK 22
• Windows10 LTSC2019 + Oracle OpenJDK 24

Did I mistake the use case of captureCallState, or this is a deliberate design, or bug of JDK implementation?

Answer 1

The capture field is bound as first argument of the altered MethodHandle hLoadLibraryW, not the last so just adjust the invoke line to:

// capturedState is the first argument to hLoadLibraryW before the normal args:
MemorySegment result = (MemorySegment) hLoadLibraryW.invokeExact(capturedState, lpLibName);

// This would also work: bind the state variable as first arg:
//  MemorySegment result = (MemorySegment) hLoadLibraryW.bindTo(capturedState).invokeExact(lpLibName);

With this change, you should see an error code as you expect:

GetLastError = 126

By the way, you can allocate the LPCWSTR in one line:

MemorySegment lpLibName = arena.allocateFrom("notexist", StandardCharsets.UTF_16LE);