How to read a memory mapped graph in android using tensorflow

Question

First of all i have been searching for this for last 2 days and i was not able to find anything related to this other than this stackoverflow post - How to read tensorflow memory mapped graph file in android?

I have successfully build the tensorflow for android from its repo by following this link - https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android/

And i was able to integrate the detection functionality into my project using ClassifierActivity and was able to load and read a custom graph from that. I could read a retrained_graph.pb or a stripped_graph.pb and even a optimized_graph.pb using my app.

I used bazel build tensorflow/python/tools:optimize_for_inference

bazel build tensorflow/python/tools:strip_unused

commands to get optimized and stripped graphs (though the graphs are readily available from the server side).

But when using a memory mapped graph, i am getting an error that it could not read the graph.

From the above stackoverflow link i was sure that it can be done somehow. But i didnt find any help on net to get it done from android side.

Please feel free to guid me to some post or forums which can provide much info on how this can be done. I am not posting any code because i have followed the github of tensorflow android to make it work and i am stuck with this only.

Thanks in advance for any advice.

Update

Following is the error i am getting when i try to read memory mapped graph

FATAL EXCEPTION: main Process: nanob2c.nano.com.nanob2c, PID: 2632
java.lang.RuntimeException: Failed to load model from 'file:///android_asset/optimized_mmapped_graph.pb'
    at org.tensorflow.contrib.android.TensorFlowInferenceInterface.<init>(TensorFlowInferenceInterface.java:100)
    at nanob2c.nano.com.nanob2c.tensorflow.core.TensorFlowImageClassifier.create(TensorFlowImageClassifier.java:109)
    at nanob2c.nano.com.nanob2c.tensorflow.ClassifierActivity.onPreviewSizeChosen(ClassifierActivity.java:147)
    at nanob2c.nano.com.nanob2c.tensorflow.core.CameraActivity$3.onPreviewSizeChosen(CameraActivity.java:603)
    at nanob2c.nano.com.nanob2c.tensorflow.core.CameraConnectionFragment.setUpCameraOutputs(CameraConnectionFragment.java:425)
    at nanob2c.nano.com.nanob2c.tensorflow.core.CameraConnectionFragment.openCamera(CameraConnectionFragment.java:432)
    at nanob2c.nano.com.nanob2c.tensorflow.core.CameraConnectionFragment.access$000(CameraConnectionFragment.java:68)
    at nanob2c.nano.com.nanob2c.tensorflow.core.CameraConnectionFragment$1.onSurfaceTextureAvailable(CameraConnectionFragment.java:99)
    at android.view.TextureView.getHardwareLayer(TextureView.java:368)
    at android.view.View.updateDisplayListIfDirty(View.java:15175)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.updateDisplayListIfDirty(View.java:15193)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.draw(View.java:16204)
    at android.view.View.updateDisplayListIfDirty(View.java:15198)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.updateDisplayListIfDirty(View.java:15193)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.support.design.widget.CoordinatorLayout.drawChild(CoordinatorLayout.java:1195)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.draw(View.java:16204)
    at android.view.View.updateDisplayListIfDirty(View.java:15198)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.updateDisplayListIfDirty(View.java:15193)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.updateDisplayListIfDirty(View.java:15193)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.updateDisplayListIfDirty(View.java:15193)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.updateDisplayListIfDirty(View.java:15193)
    at android.view.View.draw(View.java:15971)
    at android.view.ViewGroup.drawChild(ViewGroup.java:3610)
    at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3400)
    at android.view.View.draw(View.java:16204)
    at com.android.internal.policy.PhoneWindow$DecorView.draw(PhoneWindow.java:2690)
    at android.view.View.updateDisplayListIfDirty(View.java:15198)
    at android.view.ThreadedRenderer.updateViewTreeDisplayList(ThreadedRenderer.java:282)
    at android.view.ThreadedRenderer.updateRootDisplayList(ThreadedRenderer.java:288)
    at android.view.ThreadedRenderer.draw(ThreadedRenderer.java:323)
    at android.view.ViewRootImpl.draw(ViewRootImpl.java:2642)
    at android.view.ViewRootImpl.performDraw(ViewRootImpl.java:2461)
    at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2094)
    at android.view.ViewRootImpl.doTraversal(ViewRo

Update - 23 May 2017

From Pete Warden's blogpost - https://petewarden.com/2016/09/27/tensorflow-for-mobile-poets/

it is clear that we can read the memmapped graph in mobile devices and its mentioned that - One thing to watch out for is that the file on disk is no longer a plain GraphDef protobuf, so if you try loading it into a program like label_image that expects one, you’ll see errors. You need to load the model file slightly differently, which we’ll show in the iOS example below.

I am looking for an implementation of the same in android side.

Answer 1

As far as i checked in internet and with Pete Warden blog, the handling of memory mapped graph is not supported from the android side. Also the same is mentioned by MatPag as his answer.

But there should be some work around for this. As far as i searched we can tweak on to the JNI part and make it possible to handle the map from that side. My findings conclude with making changes on the following files can get a work around for this (i am not successful yet) -

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/native/tensor_jni.cc

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/native/graph_jni.cc

Also need to update the corresponding files on java part -

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/java/org/tensorflow/Tensor.java

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/src/main/java/org/tensorflow/NativeLibrary.java

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/android/java/org/tensorflow/contrib/android/TensorFlowInferenceInterface.java

Will be updating the answer if i am successful with this.

Thank you all for providing valid information on this as all of this helped me a lot on searching for a solution.