Android - draw YouTube video on SurfaceTexture

Question

I'm trying to draw a WebView on SurfaceTexture so I can render it in OpenGL.

So far, I successfully played youtube video in WebView the standard way:

  ...
  webView.getSettings().setJavaScriptEnabled(true);

  if (Build.VERSION.SDK_INT < 8) 
  {
     webView.getSettings().setPluginsEnabled(true);
  } 
  else 
  {
     webView.getSettings().setPluginState(WebSettings.PluginState.ON);
  }
  webView.setWebChromeClient(new WebChromeClient() { });
  webView.setWebViewClient(new WebViewClient());
  ...

also the hardware acceleration is turned on:

 <application
        android:hardwareAccelerated="true"
 ...

And I also successfully rendered the WebView in OpenGL (except playing the video) based on this tutorial: http://anuraagsridhar.wordpress.com/2013/03/13/rendering-an-android-webview-or-for-that-matter-any-android-view-directly-to-opengl/

Additional initialization of the WebView (so it is always 'portrait' and fits into the texture):

 Point p = new Point();
 activity.getWindow().getWindowManager().getDefaultDisplay().getSize(p);
 webView.setLayoutParams(new ViewGroup.LayoutParams(p.x, p.y));
 surfaceTexture.setDefaultBufferSize(p.x, p.y);

Overriden onDraw method of the WebView:

@Override
public void onDraw(Canvas c)
{
   try
   {
      Point p = new Point();
      activity.getWindow().getWindowManager().getDefaultDisplay().getSize(p);
      Canvas canvas = surfaceTexture.lockCanvas(new Rect(0,0,p.x,p.y));
      canvas.save();
      canvas.translate(-scrollL, -scrollT);
      super.onDraw(canvas);
      canvas.restore();
      surfaceTexture.unlockCanvasAndPost(canvas);
   }
   catch (Surface.OutOfResourcesException e)
   {
      throw new RuntimeException(e);
   }
}

When trying to play YouTube video on webview rendered using the above hack, the video rectangle is black (however the sound can be heard). I suspect that the hardware acceleration does not work on surface texture.

So the question is: Is there any way to play a youtube video on OpenGL texture?

Answer 1

Try using lockHardwareCanvas, added in API level 23.

Answer 2

The problem is this: the video is being drawn to a separate window that is behind the view hierarchy's window and is not readily accessible to be drawn on your SurfaceTexture. To complicate the matter, the WebView will have positioned and sized the video window without regard for itself being drawn to a Canvas other than the one that would be displayed on the screen. Furthermore, the video is being rendered in native code, and you wouldn't be able to draw to a Canvas fast enough to keep up anyway (that's the whole idea behind giving the video a separate window).

Any satisfactory solution would almost necessarily require use of the private APIs to get access to the video window in native code. Depending on how you plan to use your SurfaceTexture, you might come up with a workaround that involves intercepting URLs in a WebViewClient, stripping out any video tags before rendering, and placing the video in a SurfaceView or TextureView under your control. You can then put it where it should be in relation to where you've drawn the SurfaceTexture, but that limits your usage of the SurfaceTexture to billboarding. That sort of hack would get very ugly very quickly, though.

Put simply, I don't see a clean way to do this. It's a nice idea, but it's just not workable with the public API.