Getting a ANativeWindowBuffer from ANativeWindow_Buffer

Question

To get fast OpenGL ES 2.0 texture pixel access on Android NDK, I want to use the eglCreateImageKHR() extension.

According to the EGL_NATIVE_BUFFER_ANDROID docs:

This extension enables using an Android window buffer (struct ANativeWindowBuffer) as an EGLImage source.

ANativeWindowBuffer is an internal struct used by the native framework classes like GraphicBuffer. Unfortunately, since I am on NDK I do not have direct access to these classes.

The NDK native_window interface allows me to pass a Java Surface object through to the NDK. I can then use ANativeWindow_fromSurface() to get an opaque ANativeWindow* handle. With this pointer I can call ANativeWindow_lock() to fill a struct of type ANativeWindow_Buffer (Note the _).

If I try to use this &ANativeWindow_Buffer object with eglCreateImageKHR() it fails with EGL_BAD_NATIVE_WINDOW.

My question is: How can I use ANativeWindow_Buffer with eglCreateImageKHR() or alternatively how to get an ANativeWindowBuffer from ANativeWindow_Buffer or from ANativeWindow*.

Answer 1

From what I figured out while going down this road, ANativeWindow_Buffer and ANativeWindowBuffer are entirely different types. Well, they are somewhat similar, but definitely so different that they can't be used interchangeably.

If you want to compare, here are the definitions:

• ANativeWindow_Buffer: http://androidxref.com/4.4.4_r1/xref/prebuilts/ndk/current/platforms/android-18/arch-arm/usr/include/android/native_window.h
• ANativeWindowBuffer: http://androidxref.com/4.4.4_r1/xref/system/core/include/system/window.h

You will notice that they have a few fields in common (width, height, stride, format). The big difference is that ANativeWindow_Buffer contains a pointer to the actual data, while ANativeWindowBuffer contains an opaque handle of type buffer_handle_t.

So if you found out how to get a ANativeWindow_Buffer, and were hoping that you were well on your way to a ANativeWindowBuffer, you're... probably not. At least that was my conclusion. I think the very similar names are just a tease.

I did not find a way to create an ANativeWindowBuffer from NDK code. At least with using only supported APIs, I believe it's not possible. My research was with KitKat.

Answer 2

I found this question and I though it might be useful to answer it with newer information and developments, since I had to look up again on how to do it and this was one of the first answers on google for ImageKHR.

This is how you get a native buffer to use with ImageKHR. You have to "politely" ask for one from the gralloc, for that you just open the linux_kernel file that represents the IPC between the binder and gralloc, its way deeper inside the internals.

The technique demonstrated bellow will use dlopen to get the pointers to one of the ".so" that does that, but as it is internal to the system, and uses JNI reflection, there's a chance that the app verifier won't like it if you try to publish it. You can circunvent it by going one level deeper and implementing what the gralloc itself does, it just writes and reads a file block device, the app verifier wouldn't stand a chance if its just a fopen call, it can't possibly check in runtime the difference between calls from the actual libui.so or you code, it just do a simple static analysis. For doing this you can just copy the source code of the GrAlloc or link the libui.so with a different name as said in the github project.

Just for completeness, although I use this technique, I have a fallback using PBOs for transferring data from the GPU to the CPU in case of failures, but in most cases PBOs have acceptable performance.

the bare minimum needed

the complete library for doing this I used as reference

FramebufferNativeWindow.cpp

1. GraphicBuffer.h

   #pragma once
   
   #include <exception>
   #include <cstdint>
   #include <cerrno>
   
   class DynamicLibrary
   {
   public:
       DynamicLibrary(const char *fileName);
       ~DynamicLibrary();
   
       void *getFunctionPtr(const char *name) const;
   
       DynamicLibrary(const DynamicLibrary &) = delete;
       DynamicLibrary & operator = (const DynamicLibrary &other) = delete;
   
   private:
       void *libHandle;
   };
   
   struct ANativeWindowBuffer;
   
   namespace android
   {
       class GraphicBuffer;
   
       // include/system/window.h
       struct android_native_base_t
       {
           uint32_t magic;
           uint32_t version;
           void* reserved[4];
           void (*incRef)(struct android_native_base_t* base);
           void (*decRef)(struct android_native_base_t* base);
       };
   
       // include/ui/android_native_buffer.h
       struct android_native_buffer_t
       {
           struct android_native_base_t common;
           int32_t width;
           int32_t height;
           int32_t stride;
           int32_t format;
           int32_t usage;
           // ...
       };
   }
   
   // utils/Errors.h
   enum status_t
   { /*ommited, look at the gist */        };
   
   // ui/PixelFormat.h, system/graphics.h
   enum PixelFormat
   { /*ommited, look at the gist */        };
   
   // ui/GraphicBuffer.h
   { /*ommited, look at the gist */        };
   
   class GraphicBuffer
   {
   public:
       // ui/GraphicBuffer.h, hardware/gralloc.h
   
       GraphicBuffer(uint32_t width, uint32_t height, PixelFormat format, uint32_t usage);
       ~GraphicBuffer();
   
       status_t lock(uint32_t usage, void** vaddr);
       status_t unlock();
       ANativeWindowBuffer *getNativeBuffer() const;
       uint32_t getStride() const;
   
   private:
       DynamicLibrary library;
       GraphicBufferFunctions functions;
       android::GraphicBuffer *impl = nullptr;
   };
   #include "GraphicBuffer.h"
   

And the implementation:

2. GraphicBuffer.cpp

   #include <string>
   #include <cstdlib>
   #include <iostream>
   #include <iostream>
   #include <dlfcn.h>
   
   const int GRAPHICBUFFER_SIZE = 1024;
   
   using std::string;
   
   DynamicLibrary::DynamicLibrary(const char *fileName)
   {
       libHandle = dlopen(fileName, RTLD_LAZY);
       if (!libHandle) throw OpenLibFailedException();
   }
   
   DynamicLibrary::~DynamicLibrary()
   {
       if (libHandle) dlclose(libHandle);
   }
   
   void *DynamicLibrary::getFunctionPtr(const char *name) const
   {
       auto ret = (void *)dlsym(libHandle, name);
       if (ret == nullptr) {
           std::cerr << "Failed to get function " << name << std::endl;
       }
       return ret;
   }
   
   template<typename Func>
   void setFuncPtr(Func *&funcPtr, const DynamicLibrary &lib, const string &symname) {
       funcPtr = reinterpret_cast<Func *>(lib.getFunctionPtr(symname.c_str()));
   }
   
   #if defined(__aarch64__)
   #   define CPU_ARM_64
   #elif defined(__arm__) || defined(__ARM__) || defined(__ARM_NEON__) || defined(ARM_BUILD)
   #   define CPU_ARM
   #elif defined(_M_X64) || defined(__x86_64__) || defined(__amd64__)
   #   define CPU_X86_64
   #elif defined(__i386__) || defined(_M_X86) || defined(_M_IX86) || defined(X86_BUILD)
   #   define CPU_X86
   #else
   #   warning "target CPU does not support ABI"
   #endif
   
   template<typename RT, typename T1, typename T2, typename T3, typename T4>
   RT *callConstructor4(void (*fptr)(), void *memory, T1 param1, T2 param2, T3 param3, T4 param4) {
   #if defined(CPU_ARM)
       // C1 constructors return pointer
       typedef RT* (*ABIFptr)(void*, T1, T2, T3, T4);
       (void)((ABIFptr)fptr)(memory, param1, param2, param3, param4);
       return reinterpret_cast<RT*>(memory);
   #elif defined(CPU_ARM_64)
       // C1 constructors return void
       typedef void (*ABIFptr)(void*, T1, T2, T3, T4);
       ((ABIFptr)fptr)(memory, param1, param2, param3, param4);
       return reinterpret_cast<RT*>(memory);
   #elif defined(CPU_X86) || defined(CPU_X86_64)
       // ctor returns void
       typedef void (*ABIFptr)(void *, T1, T2, T3, T4);
       ((ABIFptr) fptr)(memory, param1, param2, param3, param4);
       return reinterpret_cast<RT *>(memory);
   #else
       return nullptr;
   #endif
   }
   
   template<typename T>
   void callDestructor(void (*fptr)(), T *obj) {
   #if defined(CPU_ARM)
       // D1 destructor returns ptr
       typedef void* (*ABIFptr)(T* obj);
       (void)((ABIFptr)fptr)(obj);
   #elif defined(CPU_ARM_64)
       // D1 destructor returns void
       typedef void (*ABIFptr)(T* obj);
       ((ABIFptr)fptr)(obj);
   #elif defined(CPU_X86) || defined(CPU_X86_64)
       // dtor returns void
       typedef void (*ABIFptr)(T *obj);
       ((ABIFptr) fptr)(obj);
   #endif
   }
   
   template<typename T1, typename T2>
   T1 *pointerToOffset(T2 *ptr, size_t bytes) {
       return reinterpret_cast<T1 *>((uint8_t *) ptr + bytes);
   }
   
   static android::android_native_base_t *getAndroidNativeBase(android::GraphicBuffer *gb) {
       return pointerToOffset<android::android_native_base_t>(gb, 2 * sizeof(void *));
   }
   
   GraphicBuffer::GraphicBuffer(uint32_t width, uint32_t height, PixelFormat format, uint32_t usage) :
           library("libui.so") {
       setFuncPtr(functions.constructor, library, "_ZN7android13GraphicBufferC1Ejjij");
       setFuncPtr(functions.destructor, library, "_ZN7android13GraphicBufferD1Ev");
       setFuncPtr(functions.getNativeBuffer, library,
               "_ZNK7android13GraphicBuffer15getNativeBufferEv");
       setFuncPtr(functions.lock, library, "_ZN7android13GraphicBuffer4lockEjPPv");
       setFuncPtr(functions.unlock, library, "_ZN7android13GraphicBuffer6unlockEv");
       setFuncPtr(functions.initCheck, library, "_ZNK7android13GraphicBuffer9initCheckEv");
   
       // allocate memory for GraphicBuffer object
       void *const memory = malloc(GRAPHICBUFFER_SIZE);
       if (memory == nullptr) {
           std::cerr << "Could not alloc for GraphicBuffer" << std::endl;
           return;
       }
   
       try {
           android::GraphicBuffer *const gb =
                   callConstructor4<android::GraphicBuffer, uint32_t, uint32_t, PixelFormat, uint32_t>(
                           functions.constructor,
                           memory,
                           width,
                           height,
                           format,
                           usage
                   );
           android::android_native_base_t *const base = getAndroidNativeBase(gb);
           status_t ctorStatus = functions.initCheck(gb);
   
           if (ctorStatus) {
               // ctor failed
               callDestructor<android::GraphicBuffer>(functions.destructor, gb);
               std::cerr << "GraphicBuffer ctor failed, initCheck returned " << ctorStatus <<
               std::endl;
           }
   
           // check object layout
           if (base->magic != 0x5f626672u) // "_bfr"
               std::cerr << "GraphicBuffer layout unexpected" << std::endl;
   
           // check object version
           const uint32_t expectedVersion = sizeof(void *) == 4 ? 96 : 168;
           if (base->version != expectedVersion)
               std::cerr << "GraphicBuffer version unexpected" << std::endl;
   
           base->incRef(base);
           impl = gb;
       } catch (...) {
           free(memory);
           throw;
       }
   }
   
   GraphicBuffer::~GraphicBuffer() {
       if (impl) {
           android::android_native_base_t *const base = getAndroidNativeBase(impl);
           base->decRef(base);
           //no need to call it, decRef will do
           //callDestructor<android::GraphicBuffer>(functions.destructor, impl);
       }
   }
   
   status_t GraphicBuffer::lock(uint32_t usage, void **vaddr) {
       return functions.lock(impl, usage, vaddr);
   }
   
   status_t GraphicBuffer::unlock() {
       return functions.unlock(impl);
   }
   
   /// Here it is, the windowbuffer !!!
   ANativeWindowBuffer *GraphicBuffer::getNativeBuffer() const {
       return functions.getNativeBuffer(impl);
   }
   
   uint32_t GraphicBuffer::getStride() const {
       return ((android::android_native_buffer_t *) getNativeBuffer())->stride;
   }
   

References: