Where does Delphi/Android search for a native language library?

Question

I want to add MIDI capabilities to Delphi Android apps. MIDI is available via the SoniVox library which can be accessed via the Android NDK. An example of this driver can be found here. The driver is written in C, with the NDK it is possible to create a native language library which can be accessed via a System.loadLibrary call.

  //  MidiDriver - An Android Midi Driver.
  //  Copyright (C) 2013    Bill Farmer
  //  Bill Farmer    william j farmer [at] yahoo [dot] co [dot] uk.

  #include <jni.h>

  // for EAS midi
  #include "eas.h"
  #include "eas_reverb.h"

  // determines how many EAS buffers to fill a host buffer
  #define NUM_BUFFERS 4

  // EAS data
  static EAS_DATA_HANDLE pEASData;
  const S_EAS_LIB_CONFIG *pLibConfig;
  static EAS_PCM *buffer;
  static EAS_I32 bufferSize;
  static EAS_HANDLE midiHandle;

  // init EAS midi
  jint
  Java_org_drivers_midioutput_MidiDriver_init(JNIEnv *env,
                          jobject clazz)
  {
      EAS_RESULT result;

      // get the library configuration
      pLibConfig = EAS_Config();
      if (pLibConfig == NULL || pLibConfig->libVersion != LIB_VERSION)
    return 0;

      // calculate buffer size
      bufferSize = pLibConfig->mixBufferSize * pLibConfig->numChannels *
    NUM_BUFFERS;

      // init library
      if ((result = EAS_Init(&pEASData)) != EAS_SUCCESS)
    return 0;

      // select reverb preset and enable
      EAS_SetParameter(pEASData, EAS_MODULE_REVERB, EAS_PARAM_REVERB_PRESET,
             EAS_PARAM_REVERB_CHAMBER);
      EAS_SetParameter(pEASData, EAS_MODULE_REVERB, EAS_PARAM_REVERB_BYPASS,
             EAS_FALSE);

      // open midi stream
      if (result = EAS_OpenMIDIStream(pEASData, &midiHandle, NULL) !=
    EAS_SUCCESS)
      {
    EAS_Shutdown(pEASData);
    return 0;
      }

      return bufferSize;
  }

  // midi config
  jintArray
  Java_org_drivers_midioutput_MidiDriver_config(JNIEnv *env,
                            jobject clazz)
  {
      jboolean isCopy;

      if (pLibConfig == NULL)
    return NULL;

      jintArray configArray = (*env)->NewIntArray(env, 4);

      jint *config = (*env)->GetIntArrayElements(env, configArray, &isCopy);

      config[0] = pLibConfig->maxVoices;
      config[1] = pLibConfig->numChannels;
      config[2] = pLibConfig->sampleRate;
      config[3] = pLibConfig->mixBufferSize;

      (*env)->ReleaseIntArrayElements(env, configArray, config, 0);

      return configArray;
  }

  // midi render
  jint
  Java_org_drivers_midioutput_MidiDriver_render(JNIEnv *env,
                            jobject clazz,
                            jshortArray shortArray)
  {
      jboolean isCopy;
      EAS_RESULT result;
      EAS_I32 numGenerated;
      EAS_I32 count;
      jsize size;

      // jbyte* GetByteArrayElements(jbyteArray array, jboolean* isCopy)
      // void ReleaseByteArrayElements(jbyteArray array, jbyte* elems,

      // void* GetPrimitiveArrayCritical(JNIEnv*, jarray, jboolean*);
      // void ReleasePrimitiveArrayCritical(JNIEnv*, jarray, void*, jint);

      if (pEASData == NULL)
    return 0;

      buffer =
    (EAS_PCM *)(*env)->GetShortArrayElements(env, shortArray, &isCopy);

      size = (*env)->GetArrayLength(env, shortArray);

      count = 0;
      while (count < size)
      {
    result = EAS_Render(pEASData, buffer + count,
                pLibConfig->mixBufferSize, &numGenerated);
    if (result != EAS_SUCCESS)
        break;

    count += numGenerated * pLibConfig->numChannels;
      }

      (*env)->ReleaseShortArrayElements(env, shortArray, buffer, 0);

      return count;
  }

  // midi write
  jboolean
  Java_org_drivers_midioutput_MidiDriver_write(JNIEnv *env,
                           jobject clazz,
                           jbyteArray byteArray)
  {
      jboolean isCopy;
      EAS_RESULT result;
      jint length;
      EAS_U8 *buf;

      if (pEASData == NULL || midiHandle == NULL)
    return JNI_FALSE;

      buf = (EAS_U8 *)(*env)->GetByteArrayElements(env, byteArray, &isCopy);
      length = (*env)->GetArrayLength(env, byteArray);

      result = EAS_WriteMIDIStream(pEASData, midiHandle, buf, length);

      (*env)->ReleaseByteArrayElements(env, byteArray, buf, 0);

      if (result != EAS_SUCCESS)
    return JNI_FALSE;

      return JNI_TRUE;
  }

  // shutdown EAS midi
  jboolean
  Java_org_drivers_midioutput_MidiDriver_shutdown(JNIEnv *env,
                              jobject clazz)
  {
      EAS_RESULT result;

      if (pEASData == NULL || midiHandle == NULL)
    return JNI_FALSE;

      if ((result = EAS_CloseMIDIStream(pEASData, midiHandle)) != EAS_SUCCESS)
      {
    EAS_Shutdown(pEASData);
    return JNI_FALSE;
      }

      if ((result = EAS_Shutdown(pEASData)) != EAS_SUCCESS)
    return JNI_FALSE;

      return JNI_TRUE;
  }

I created an Android app with Eclipse, added the MidiDriver as a native library and got everything up and running. With this driver I have MIDI capabilities in my app. The outline of the MidiDriver code you will find below.

  //  MidiDriver - An Android Midi Driver.
  //  Copyright (C) 2013    Bill Farmer
  //  Bill Farmer    william j farmer [at] yahoo [dot] co [dot] uk.

  package  org.drivers.midioutput;

  import java.io.File;

  import android.media.AudioFormat;
  import android.media.AudioManager;
  import android.media.AudioTrack;
  import android.util.Log;

  // MidiDriver

  public class MidiDriver implements Runnable
  {
      private static final int SAMPLE_RATE = 22050;
      private static final int BUFFER_SIZE = 4096;

      private Thread thread;
      private AudioTrack audioTrack;

      private OnMidiStartListener listener;

      private short buffer[];

      // Constructor

      public MidiDriver ()
      {
         Log.d ("midi", ">> MidiDriver started");
      }

      public void start ()
      {
    // Start the thread
         thread = new Thread (this, "MidiDriver");
         thread.start ();
      } // start //

      @Override
      public void run ()
      {
          processMidi ();
      } // run //

      public void stop ()
      {
        Thread t = thread;
        thread = null;

        // Wait for the thread to exit

        while (t != null && t.isAlive ())
            Thread.yield ();
      } // stop //

      // Process MidiDriver

      private void processMidi ()
      {
        int status = 0;
        int size = 0;

        // Init midi

        if ((size = init()) == 0)
            return;

        buffer = new short [size];

        // Create audio track

        audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, SAMPLE_RATE,
                        AudioFormat.CHANNEL_OUT_STEREO,
                        AudioFormat.ENCODING_PCM_16BIT,
                        BUFFER_SIZE, AudioTrack.MODE_STREAM);
        if (audioTrack == null)
        {
            shutdown ();
            return;
        } // if

        // Call listener

        if (listener != null)
            listener.onMidiStart();

        // Play track

        audioTrack.play();

        // Keep running until stopped

        while (thread != null)
        {
            // Render the audio

            if (render(buffer) == 0)
               break;

            // Write audio to audiotrack

            status = audioTrack.write(buffer, 0, buffer.length);

            if (status < 0)
               break;
        } // while

        // Render and write the last bit of audio

        if (status > 0)
            if (render(buffer) > 0)
               audioTrack.write(buffer, 0, buffer.length);

        // Shut down audio

        shutdown();
        audioTrack.release();
      } // processMidi //

      public void setOnMidiStartListener (OnMidiStartListener l)
      {
         listener = l;
      } // setOnMidiStartListener //

      public static void load_lib (String libName) 
      {
         File file = new File (libName);

         if (file.exists ()) 
         {
             System.load (libName);
         } else 
         {
             System.loadLibrary (libName);
         }
     }    // Listener interface

      public interface OnMidiStartListener
      {
         public abstract void onMidiStart ();
      } // OnMidiStartListener //

      // Native midi methods

      private native int     init ();
      public  native int []  config ();
      private native int     render (short a []);
      public  native boolean write (byte a []);
      private native boolean shutdown ();

      // Load midi library

      static
      {
         System.loadLibrary ("midi");
      }
  }

Next I tested the JNI interface to see whether I could access Java classes from Delphi via JNI. No problem either. So theoretically I should be able to access the MidiDriver via a Java interface: cool! I wrapped MidiDriver in a another Java class: MIDI_Output in order to handle the interface internally (I have no idea how to interface a Java interface in Delphi. MIDI_Output creates an instance of MidiDriver and calls functions from MidiDriver when necessary. This all works great when running from Eclipse. Some parts of MIDI_Output below:

  package org.drivers.midioutput;

  import java.io.IOException;

  import org.drivers.midioutput.MidiDriver.OnMidiStartListener;

  import android.media.MediaPlayer;
  import android.os.Environment;
  import android.util.Log;

  public class MIDI_Output implements OnMidiStartListener
  {
     protected MidiDriver midi_driver;
     protected MediaPlayer media_player;

     public MIDI_Output ()
     {
        // Create midi driver
        Log.d ("midi", ">> Before initializing MIDI driver version 1");

        midi_driver = new MidiDriver();

  // Set onmidistart listener to this class

        if (midi_driver != null)
            midi_driver.setOnMidiStartListener (this);
     } // MIDI_Output () //

  protected void putShort (int m, int n, int v)
  {
     if (midi_driver != null)
     {
       byte msg [] = new byte [3];

       msg [0] = (byte) m;
       msg [1] = (byte) n;
       msg [2] = (byte) v;

       midi_driver.write (msg);
     } // if
  } // putShort //

  // and much more code
  // ...   

In the example above putShort calls function write from MidiDriver which is a function defined in the native library. This all works fine in Java but in Dellphi practice is a little harder as you might have guessed. To show in more detail the call chain I need to use this whole contraption in Delphi see the image below.

In libsonivox (found in /system/lib) the function EAS_WriteMidiStream can be found, which is called from function write in libmidi.so (found everywhere, but also in /system/lib and /vendor/lib), which is declared in MidiDriver.java in MIDI_Output.apk which is called from MIDI_Output.java which creates a new MidiDriver and refers to midi_driver.write (...) in function putShort of the same package. Finally putShort should be called in Delphi but it never gets there.

It is aborted when MIDI_Output create the new MidiDriver which tries to load the midi library. The program is not able to load the library “midi”. I ran adb -d logcat to see what happens and the output is shown below. The error message that is shown on the android screen is highlighted.

  D/dalvikvm( 5251): DexOpt: --- BEGIN 'MIDI_Output.apk' (bootstrap=0) ---
  D/dalvikvm( 5320): DexOpt: load 50ms, verify+opt 174ms, 1050124 bytes
  D/dalvikvm( 5251): DexOpt: --- END 'MIDI_Output.apk' (success) ---
  D/dalvikvm( 5251): DEX prep '/storage/emulated/legacy/Data/d/MIDI_Output.apk': u
  nzip in 14ms, rewrite 401ms
  W/dalvikvm( 5251): dvmFindClassByName rejecting 'org.drivers.midioutput/MIDI_Out
  put'
  D/midi    ( 5251): >> Before initializing MIDI driver version 1
  W/dalvikvm( 5251): Exception Ljava/lang/UnsatisfiedLinkError; thrown while initi
  alizing Lorg/drivers/midioutput/MidiDriver;

  W/System.err( 5251): java.lang.UnsatisfiedLinkError: Couldn't load midi from loa
  der dalvik.system.DexClassLoader[DexPathList[[zip file "/storage/sdcard0/Data/d/
  MIDI_Output.apk"],nativeLibraryDirectories=[/vendor/lib, /system/lib]]]: findLib
  rary returned null

  W/System.err( 5251):    at java.lang.Runtime.loadLibrary(Runtime.java:358)
  W/System.err( 5251):    at java.lang.System.loadLibrary(System.java:526)
  W/System.err( 5251):    at org.drivers.midioutput.MidiDriver.<clinit>(MidiDriver
  .java:177)
  W/System.err( 5251):    at org.drivers.midioutput.MIDI_Output.<init>(MIDI_Output
  .java:22)
  W/System.err( 5251):    at java.lang.Class.newInstanceImpl(Native Method)
  W/System.err( 5251):    at java.lang.Class.newInstance(Class.java:1208)
  W/System.err( 5251):    at dalvik.system.NativeStart.run(Native Method)
  D/dalvikvm( 5251): GC_FOR_ALLOC freed 795K, 9% free 9091K/9920K, paused 13ms,
  total 13ms
  D/dalvikvm( 5251): GC_CONCURRENT freed 9K, 4% free 9532K/9920K, paused
  2ms+2ms, total 22ms

I understand that the library cannot be found. Trouble is, I do not know where this library is searched. The error message mentions /vendor/lib and /system/lib. I added libmidi.so to these libraries. I added the library to the application directory com.embarcadero.MIDI_Output_Project/lib (that is where android stores the library), to /storage/sdcard0/Data/d/ (the directory where MIDI_Output.apk containing the Java classes are stored). I literally sprinkled my android system with libmidi.so. I have tried to load libmidi.so as well.

As a test I added a very simple example class to the MIDI_Output package and called function test_int. This runs without any problems.

  package org.drivers.midioutput;

  import android.util.Log;

  public class class_test
  {
     public int test_int (int n)
     {
        int sq = n * n;
        String mess = "*** test_int computes " + String.valueOf (sq);

        Log.d ("midi", mess);
        return n * n;
     }
  }

My question is: in which directory is DalvikVM looking for native library in a setup described above (Delphi calling a Java class via JNI and Java calling a C library via NDK)?

A second question: is it a library search path problem at all? Maybe Delphi is not able to call NDK via JNI.

I have tried to be as brief as possible. If anyone thinks I should add more code just let me know. Any help is greatly appreciated.

Answer 1

NDK libraries that are used by Delphi should be placed in <path to your PlatformSDKs>\android-ndk-r8e\platforms\android-14\arch-arm\usr\lib. I realised this thanks the recommendations of Arioch and Chris to use the NDK directly. Most android files in the Source\rtl\android directory include the file Androidapi.inc which contains the following definitions

const
    AndroidLib            = '/usr/lib/libandroid.so';
    AndroidJniGraphicsLib = '/usr/lib/libjnigraphics.so';
    AndroidEglLib         = '/usr/lib/libEGL.so';
    AndroidGlesLib        = '/usr/lib/libGLESv1_CM.so';
    AndroidGles2Lib       = '/usr/lib/libGLESv2.so';
    AndroidLogLib         = '/usr/lib/liblog.so';
    AndroidOpenSlesLib    = '/usr/lib/libOpenSLES.so';

The directory /usr/lib does not exist on the Nexus-7 but I found it in the path mentioned above with all the files declared in the const part. Copying the libmidi.so to this directory solved the problem. I now have the not so minor problem of hearing no sound. I'll try to solve that now, as well as trying to call the NDK directly.