Receive complete android unicode input in C/C++

Question

(Android, NDK, C++, OpenGL ES)

I need a way to reliably receive the text input from a (soft)keyboard. The solution can be through Java using a NativeActivity subclass, or anything which works. At the end I need whatever text is being typed, so I can render it myself with OpenGL

Some background: Up until now I was triggering the soft keyboard by calling showSoftInput or hideSoftInputFromWindow thought JNI. This never failed so far. However, the problem is the native activity will not send all characters. Especially some unicode characters outside of ASCII range, or some motion soft keyboard won't work (AKeyEvent_getKeyCode)

It used to be possible to get some of those other unicode characters why checking for KeyEvent.ACTION_MULTIPLE and reading a string of characters. But even this won't work reliably anymore.

So far I failed to find an alternative method. I experimented with programmatically adding a EditText, but never got it to work. Even trying to add a simple Button resulted in the OpenGL view to no longer being rendered.

On iOS I worked around it by having a hiding edit box, which I simply activated to make the keyboard show up. I would then read out the edit box and use the string to render myself in OpenGL.

Answer 1

I have the same issues, and I have solved it using a 'Character' event that I process separately from the InputEvent.

The problem is this: AKeyEvent_getKeyCode doesn't return the KeyCode for some softkey events, notably the expanded 'unicode/latin' characters when you hold down a key. This prevents the methods @Shammi and @eozgonul from working because the KeyEvent reconstructed on the Java side doesn't have enough information to get a unicode character.

Another issue is that the InputQueue is drained on the C++/Native side before the dispatchKeyEvent event(s) are fired. This means that the KEYDOWN/KEYUP events all fired before the Java code can process the events. (They are not interleaved).

My solution is to capture the unicode characters on the Java side by overriding dispatchKeyEvent and sending the characters to a Queue<Integer> queueLastInputCharacter = new ConcurrentLinkedQueue<Integer>();

// [JAVA]
@Override
public boolean dispatchKeyEvent (KeyEvent event)
{
    int metaState = event.getMetaState(); 
    int unichar = event.getUnicodeChar(metaState);

    // We are queuing the Unicode version of the characters for
    // sending to the app during processEvents() call.

    // We Queue the KeyDown and ActionMultiple Event UnicodeCharacters
    if(event.getAction()==KeyEvent.ACTION_DOWN){
        if(unichar != 0){
            queueLastInputCharacter.offer(Integer.valueOf(unichar));
        }
        else{
            unichar = event.getUnicodeChar(); 

            if(unichar != 0){
                queueLastInputCharacter.offer(Integer.valueOf(unichar));
            }
            else if (event.getDisplayLabel() != 0){
                String aText = new String();
                aText = "";
                aText += event.getDisplayLabel();
                queueLastInputCharacter.offer(Integer.valueOf(Character.codePointAt(aText, 0)));
            }
            else
                queueLastInputCharacter.offer(Integer.valueOf(0));
        }
    }
    else if(event.getAction()==KeyEvent.ACTION_MULTIPLE){
        unichar = (Character.codePointAt(event.getCharacters(), 0));
        queueLastInputCharacter.offer(Integer.valueOf(unichar));
    }


    return super.dispatchKeyEvent(event);
}

The concurrent queue is going to let the threads play nice together.

I have a Java side method that returns the last input character:

// [JAVA]
public int getLastUnicodeChar(){
    if(!queueLastInputCharacter.isEmpty())
        return queueLastInputCharacter.poll().intValue();
    return 0;
}

At the end of my looper code, I tacked on an extra check to see if the queue retained any unicode characters:

// [C++]
int ident;
int events;
struct android_poll_source* source;

// If not rendering, we will block 250ms waiting for events.
// If animating, we loop until all events are read, then continue
// to draw the next frame of animation.
while ((ident = ALooper_pollAll(((nv_app_status_focused(_lpApp)) ? 1 : 250),
                                NULL,
                                &events,
                                (void**)&source)) >= 0)
{
    // Process this event.
    if (source != NULL)
        source->process(_lpApp, source);

    // Check if we are exiting.  If so, dump out
    if (!nv_app_status_running(_lpApp))
        return;
}

static int modtime = 10; // let's not run on every call
if(--modtime == 0) {
    long uniChar = androidUnicodeCharFromKeyEvent();
    while (uniChar != 0) {
        KEvent kCharEvent; // Game engine event
        kCharEvent.ptkKey = K_VK_ERROR;
        kCharEvent.unicodeChar = uniChar;
        kCharEvent.character = uniChar;

        /* Send unicode char */
        kCharEvent.type = K_EVENT_UNICHAR;
        _lpPortableHandler(&kCharEvent);

        if (kCharEvent.character < 127) {
            /* Send ascii char for source compatibility as well */
            kCharEvent.type = K_EVENT_CHAR;
            _lpPortableHandler(&kCharEvent);
        }

        uniChar = androidUnicodeCharFromKeyEvent();
    }
    modtime = 10;
}

The androidUnicodeCharFromKeyEvent function is very similar to @Shammi 's GetStringFromAInputEvent method, only use CallIntMethod to return the jint.

Notes This does require modifying your engine to process character events separate from Key events. Android still has key codes like AKEYCODE_BACK or AKEYCODE_ENTER that are not character events and still need to be handled (and can be handled on the main input looper).

Editboxes, consoles, etc... Things that are expecting user input can be modified to receive a separate character event that builds the string. If you are working on multiple platforms, then you will need to generate these new character events in addition to the normal key input events.

Answer 2

I hope this works for you, worked for me so far.

int GetUnicodeChar(struct android_app* app, int eventType, int keyCode, int metaState)
{
JavaVM* javaVM = app->activity->vm;
JNIEnv* jniEnv = app->activity->env;

JavaVMAttachArgs attachArgs;
attachArgs.version = JNI_VERSION_1_6;
attachArgs.name = "NativeThread";
attachArgs.group = NULL;

jint result = javaVM->AttachCurrentThread(&jniEnv, &attachArgs);
if(result == JNI_ERR)
{
    return 0;
}

jclass class_key_event = jniEnv->FindClass("android/view/KeyEvent");
int unicodeKey;

if(metaState == 0)
{
    jmethodID method_get_unicode_char = jniEnv->GetMethodID(class_key_event, "getUnicodeChar", "()I");
    jmethodID eventConstructor = jniEnv->GetMethodID(class_key_event, "<init>", "(II)V");
    jobject eventObj = jniEnv->NewObject(class_key_event, eventConstructor, eventType, keyCode);

    unicodeKey = jniEnv->CallIntMethod(eventObj, method_get_unicode_char);
}

else
{
    jmethodID method_get_unicode_char = jniEnv->GetMethodID(class_key_event, "getUnicodeChar", "(I)I");
    jmethodID eventConstructor = jniEnv->GetMethodID(class_key_event, "<init>", "(II)V");
    jobject eventObj = jniEnv->NewObject(class_key_event, eventConstructor, eventType, keyCode);

    unicodeKey = jniEnv->CallIntMethod(eventObj, method_get_unicode_char, metaState);
}

javaVM->DetachCurrentThread();

LOGI("Unicode key is: %d", unicodeKey);
return unicodeKey;
}

Just call it from your input handler, my structure is approximately as follows:

switch (AInputEvent_getType(event))
    {
        case AINPUT_EVENT_TYPE_KEY:
          switch (AKeyEvent_getAction(event))
          {
            case AKEY_EVENT_ACTION_DOWN:
              int key = AKeyEvent_getKeyCode(event);
              int metaState = AKeyEvent_getMetaState(event);
              int uniValue;
              if(metaState != 0)
                  uniValue = GetUnicodeChar(app, AKEY_EVENT_ACTION_DOWN, key, metaState);
              else
                  uniValue = GetUnicodeChar(app, AKEY_EVENT_ACTION_DOWN, key, 0);

Since you stated that you already open the soft keyboard, I don't go into that part but the code is kind of straight forward. I basically use the Java function of class KeyEvent which has GetUnicodeChar function.