Using Glide, how can I go over each frame of GifDrawable, as Bitmap?

Question

Background

In a live wallpaper, I have a Canvas instance that I wish to draw GIF/WEBP content into, which was loaded via Glide.

The reason I wish to do it with Glide, is that it provides some advantages over a solution I've found in the past for the same thing (here , repository here) :

1. Usage of Movie restricts me to just GIF. With Glide I could also support WEBP animation
2. Usage of Movie seems inefficient as it doesn't tell me the time to wait between frames, so I have to choose FPS that I wish to try to use. It's also deprecated on Android P.
3. Glide might be able to ease the handling of various scaling.
4. Glide might not crash like on the original code, and might provide better control of the mechanism.

The problem

Glide seems to be optimized to work only with normal UI (Views). It has some basic functions, but the most important ones for what I'm trying to do seems to be private.

What I've found

I use official Glide library (v 3.8.0) for GIF loading, and GlideWebpDecoder for WEBP loading (with same version).

The basic call to load each of those, is as such:

GIF:

    GlideApp.with(this).asGif()
            .load("https://res.cloudinary.com/demo/image/upload/bored_animation.gif")
            .into(object : SimpleTarget<GifDrawable>() {
                override fun onResourceReady(resource: GifDrawable, transition: Transition<in GifDrawable>?) {
                    //example of usage:
                    imageView.setImageDrawable(resource)
                    resource.start()
                }
            })

WEBP:

        GlideApp.with(this).asDrawable()
                .load("https://res.cloudinary.com/demo/image/upload/fl_awebp/bored_animation.webp")
//                .optionalTransform(WebpDrawable::class.java, WebpDrawableTransformation(CircleCrop()))
                .into(object : SimpleTarget<Drawable>() {
                    override fun onResourceReady(resource: Drawable, transition: Transition<in Drawable>?) {
                        //example of usage:
                        imageView.setImageDrawable(resource)
                        if (resource is Animatable) {
                            (resource as Animatable).start()
                        }
                    }
                })

Now, remember I don't really have an ImageView, and instead I only have a Canvas, which I get via surfaceHolder.lockCanvas() call.

                    resource.callback = object : Drawable.Callback {
                        override fun invalidateDrawable(who: Drawable) {
                            Log.d("AppLog", "frame ${resource.frameIndex}/${resource.frameCount}")
                        }

                    }

However, when I try to fetch the Bitmap to be used for the current frame, I fail to find the correct function.

I tried this for example (and this is only an example, to see if it can work with canvas):

    val bitmap = Bitmap.createBitmap(100, 100, Bitmap.Config.ARGB_8888)
    val canvas = Canvas(bitmap)

    ...
    resource.draw(canvas)

But it doesn't seem to draw the content into the bitmap, and I think it's because its draw function has these lines of code:

  @Override
  public void draw(@NonNull Canvas canvas) {
    if (isRecycled) {
      return;
    }

    if (applyGravity) {
      Gravity.apply(GRAVITY, getIntrinsicWidth(), getIntrinsicHeight(), getBounds(), getDestRect());
      applyGravity = false;
    }

    Bitmap currentFrame = state.frameLoader.getCurrentFrame();
    canvas.drawBitmap(currentFrame, null, getDestRect(), getPaint());
  }

Yet the getDestRect() returns a 0-sized rectangle, which I can't find how to modify : it's also private, and I don't see anything that changes it.

The questions

1. Suppose I got the Drawable I wish to use (GIF/WEBP), how can I get each of the frames it can produce (and not just the first frame), and draw it into a canvas (with the right amount of time between frames, of course) ?

2. Can I also set the scaling type somehow, just like on ImageView (center-crop, fit-center, center-inside...) ?

3. Is there perhaps a better alternative to this? Maybe suppose I have a GIF/WEBP animation file, does Glide allow me to just use its decoder? Something like on this library ?

---

EDIT:

I've found a nice alternative library, that allows to load the GIF one frame after another, here. It doesn't seem as efficient in loading the frame-by-frame, but it's open sourced and can easily be modified to work better.

Still could be much nicer to do it on Glide, as it supports scaling and WEBP loading too.

I've made a POC (link here) that shows that it can indeed go frame-by-frame, waiting for the right time between them. If anyone succeeds doing the exact same as I did, but on Glide (latest version of Glide, of course), I will accept the answer and grant the bounty. Here's the code:

**GifPlayer.kt , based on NsGifPlayer.java **

open class GifPlayer {
    companion object {
        const val ENABLE_CACHING = false
        const val MEM_CACHE_SIZE_PERCENT = 0.8
        fun calculateMemCacheSize(percent: Double): Long {
            if (percent < 0.05f || percent > 0.8f) {
                throw IllegalArgumentException("setMemCacheSizePercent - percent must be " + "between 0.05 and 0.8 (inclusive)")
            }
            val maxMem = Runtime.getRuntime().maxMemory()
//            Log.d("AppLog", "max mem :$maxMem")
            return Math.round(percent * maxMem)
        }
    }

    private val uiHandler = Handler(Looper.getMainLooper())
    private var playerHandlerThread: HandlerThread? = null
    private var playerHandler: Handler? = null
    private val gifDecoder: GifDecoder = GifDecoder()
    private var currentFrame: Int = -1
    var listener: GifListener? = null
    var state: State = State.IDLE
        private set
    private val playRunnable: Runnable
    private val frames = HashMap<Int, AnimationFrame>()
    private var currentUsedMemByCache = 0L

    class AnimationFrame(val bitmap: Bitmap, val duration: Long)

    enum class State {
        IDLE, PAUSED, PLAYING, RECYCLED, ERROR
    }

    interface GifListener {
        fun onGotFrame(bitmap: Bitmap, frame: Int, frameCount: Int)

        fun onError()
    }

    init {
        val memCacheSize = if (ENABLE_CACHING) calculateMemCacheSize(MEM_CACHE_SIZE_PERCENT) else 0L
//        Log.d("AppLog", "memCacheSize:$memCacheSize = ${memCacheSize / 1024L} MB")
        playRunnable = object : Runnable {
            override fun run() {
                val frameCount = gifDecoder.frameCount
                gifDecoder.setCurIndex(currentFrame)
                currentFrame = (currentFrame + 1) % frameCount
                val animationFrame = if (ENABLE_CACHING) frames[currentFrame] else null
                if (animationFrame != null) {
//                    Log.d("AppLog", "cache hit - $currentFrame")
                    val bitmap = animationFrame.bitmap
                    val delay = animationFrame.duration
                    uiHandler.post {
                        listener?.onGotFrame(bitmap, currentFrame, frameCount)
                        if (state == State.PLAYING)
                            playerHandler!!.postDelayed(this, delay)
                    }
                } else {
//                    Log.d("AppLog", "cache miss - $currentFrame fill:${frames.size}/$frameCount")
                    val bitmap = gifDecoder.bitmap
                    val delay = gifDecoder.decodeNextFrame().toLong()
                    if (ENABLE_CACHING) {
                        val bitmapSize = BitmapCompat.getAllocationByteCount(bitmap)
                        if (bitmapSize + currentUsedMemByCache < memCacheSize) {
                            val cacheBitmap = Bitmap.createBitmap(bitmap)
                            frames[currentFrame] = AnimationFrame(cacheBitmap, delay)
                            currentUsedMemByCache += bitmapSize
                        }
                    }
                    uiHandler.post {
                        listener?.onGotFrame(bitmap, currentFrame, frameCount)
                        if (state == State.PLAYING)
                            playerHandler!!.postDelayed(this, delay)
                    }
                }
            }
        }
    }

    @Suppress("unused")
    protected fun finalize() {
        stop()
    }

    @UiThread
    fun start(filePath: String): Boolean {
        if (state != State.IDLE && state != State.ERROR)
            return false
        currentFrame = -1
        state = State.PLAYING
        playerHandlerThread = HandlerThread("GifPlayer")
        playerHandlerThread!!.start()
        val looper = playerHandlerThread!!.looper
        playerHandler = Handler(looper)
        playerHandler!!.post {
            try {
                gifDecoder.load(filePath)
            } catch (e: Exception) {
                uiHandler.post {
                    state = State.ERROR
                    listener?.onError()
                }
                return@post
            }

            val bitmap = gifDecoder.bitmap
            if (bitmap != null) {
                playRunnable.run()
            } else {
                frames.clear()
                gifDecoder.recycle()
                uiHandler.post {
                    state = State.ERROR
                    listener?.onError()
                }
                return@post
            }
        }
        return true
    }

    @UiThread
    fun stop(): Boolean {
        if (state == State.IDLE)
            return false
        state = State.IDLE
        playerHandler!!.removeCallbacks(playRunnable)
        playerHandlerThread!!.quit()
        playerHandlerThread = null
        playerHandler = null
        return true
    }

    @UiThread
    fun pause(): Boolean {
        if (state != State.PLAYING)
            return false
        state = State.PAUSED
        playerHandler?.removeCallbacks(playRunnable)
        return true
    }

    @UiThread
    fun resume(): Boolean {
        if (state != State.PAUSED)
            return false
        state = State.PLAYING
        playerHandler?.removeCallbacks(playRunnable)
        playRunnable.run()
        return true
    }

    @UiThread
    fun toggle(): Boolean {
        when (state) {
            State.PLAYING -> pause()
            State.PAUSED -> resume()
            else -> return false
        }
        return true
    }

}

MainActivity.kt

class MainActivity : AppCompatActivity() {
    private lateinit var player: GifPlayer

    @SuppressLint("StaticFieldLeak")
    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        val file = File([email protected], "file.gif")
        object : AsyncTask<Void, Void, Void?>() {

            override fun doInBackground(vararg params: Void?): Void? {
                val inputStream = resources.openRawResource(R.raw.fast)
                if (!file.exists()) {
                    file.parentFile.mkdirs()
                    val outputStream = FileOutputStream(file)
                    val buf = ByteArray(1024)
                    var len: Int
                    while (true) {
                        len = inputStream.read(buf)
                        if (len <= 0)
                            break
                        outputStream.write(buf, 0, len)
                    }
                    inputStream.close()
                    outputStream.close()
                }
                return null
            }

            override fun onPostExecute(result: Void?) {
                super.onPostExecute(result)
                player.setFilePath(file.absolutePath)
                player.start()
            }

        }.execute()

        player = GifPlayer(object : GifPlayer.GifListener {
            override fun onGotFrame(bitmap: Bitmap, frame: Int, frameCount: Int) {
                Log.d("AppLog", "onGotFrame $frame/$frameCount")
                imageView.post {
                    imageView.setImageBitmap(bitmap)
                }
            }

            override fun onError() {
                Log.d("AppLog", "onError")
            }
        })
    }

    override fun onStart() {
        super.onStart()
        player.resume()
    }

    override fun onStop() {
        super.onStop()
        player.pause()
    }

    override fun onDestroy() {
        super.onDestroy()
        player.stop()
    }
}

Answer 1

Anyway, we will use non-documented methods from Glide and i hope in one day Glide team will make it public. You will need to have a bit experience with Java Reflection :) Here is bench of code to extract Bitmap from GIF file:

ArrayList bitmaps = new ArrayList<>();
Glide.with(AppObj.getContext())
            .asGif()
            .load(GIF_PATH)
            .into(new SimpleTarget<GifDrawable>() {
                @Override
                public void onResourceReady(@NonNull GifDrawable resource, @Nullable Transition<? super GifDrawable> transition) {
                    try {
                        Object GifState = resource.getConstantState();
                        Field frameLoader = GifState.getClass().getDeclaredField("frameLoader");
                        frameLoader.setAccessible(true);
                        Object gifFrameLoader = frameLoader.get(GifState);

                        Field gifDecoder = gifFrameLoader.getClass().getDeclaredField("gifDecoder");
                        gifDecoder.setAccessible(true);
                        StandardGifDecoder standardGifDecoder = (StandardGifDecoder) gifDecoder.get(gifFrameLoader);
                        for (int i = 0; i < standardGifDecoder.getFrameCount(); i++) {
                            standardGifDecoder.advance();
                            bitmaps.add(standardGifDecoder.getNextFrame());
                        }
                    } catch (Exception ex) {
                        ex.printStackTrace();
                    }

                }
            });

}

Answer 2

I had a similar requirement when I wanted to display a preview instead of the animation while loading a gif in Glide.

My solution was to take the first frame from the GifDrawable and to present this as the entire drawable. The same approach can be adapted to get the other frames to display (or to export etc.)

DrawableRequestBuilder builder = Glide.with(ctx).load(someUrl);
builder.listener(new RequestListener<String, GlideDrawable>() {
    @Override
    public boolean onException(Exception e, String model, Target<GlideDrawable> target, boolean isFirstResource) {
        return false;
    }

    @Override
    public boolean onResourceReady(GlideDrawable resource, String model, Target<GlideDrawable> target, boolean isFromMemoryCache, boolean isFirstResource) {
        if (resource.isAnimated()) {
            target.onResourceReady(new GlideBitmapDrawable(null, ((GifDrawable) resource).getFirstFrame()), null);
        }
        return handled;
    }
});
builder.into(mImageView);

You can either progress the animation to get the keyframes or get them by index within the callback by directly accessing the decoder attached to GifDrawable. Alternatively set a Callback (actual class name) on the drawable when it is ready. It will be called by onFrameReady (Giving you the current frame in the drawable each time). The gif drawable class already manages the bitmap pool.

Once the GifDrawable is ready, loop through frames with the following method:

GifDrawable gd = (GifDrawable) resource;
Bitmap b = gd.getDecoder().getNextFrame();  

Note that if you are using the decoder you should really do it from the onResourceReady callback I mentioned above. I had intermittent issues when I tried to do it earlier.

If you let the decoder run automatically, you can get callbacks for frames

gifDrawable.setCallback(new Drawable.Callback() {
    @Override
    public void invalidateDrawable(@NonNull Drawable who) {
        //NOTE: this method is called each time the GifDrawable updates itself with a new frame
        //who.draw(canvas); //if you already have a canvas
        //https://stackoverflow.com/questions/3035692/how-to-convert-a-drawable-to-a-bitmap //if you really want a bitmap
    }

    @Override public void scheduleDrawable(@NonNull Drawable who, @NonNull Runnable what, long when) { /* ignore */ }
    @Override public void unscheduleDrawable(@NonNull Drawable who, @NonNull Runnable what) { /* ignore */ }
});

At the time this was the best approach available. As it has been over a year, I cannot guarantee there is not a more efficient way to do this now.

The version of the library I use is Glide 3.7.0. Access is restricted in the latest version 4.7.+, but I'm not sure how far back you need to go to use my approach.