How to take a screenshot of other app programmatically without root permission, like Screenshot UX Trial?

Question

How to take a screenshot of other app programmatically without root permission, like Screenshot UX Trial?

1. I know I can capture the bitmap of the root view in my app. But I can't get the root view of the other app when my app is running in background

   bitmap = Bitmap.createBitmap(rootview.getDrawingCache());

2. There is a permission for capturing current frame buffer in Manifest: android.permission.READ_FRAME_BUFFER. But some website says it's for signature app only.

   Check Android Permissions - Protection Levels

After trying Screenshot UX Trial, I read the permission:

• INTERNET: for connect to localhost screenshot server for rooted phone.
• SYSTEM_ALERT_WINDOW: for topmost camera button.
• VIBRATE: for vibrate feedback.
• WRITE_EXTERNAL_STORAGE: to save the screenshot.
• GET_TASKS: for detect foreground Develoment setting activity for non rooted&non preloaded capture method.

It seems either SYSTEM_ALERT_WINDOW or GET_TASKS allow the app to take screenshot. I have two guess of how it works:

1. It may be able to access the Activity of the foreground activity, it gets the root view of the Activity, capture its screenshot.
2. Calling glreadpixels

If you try one of my guess, please let me know the result.

Answer 1

This is extremely difficult. I spent several years trying to do it. I eventually succeeded, but any solution will involve commercial as well as technical effort.

---

Update March 2015

Most of the stuff below is no longer up-to-date. There's now, after all these years, an android.media.projection package https://developer.android.com/reference/android/media/projection/package-summary.html which finally allows what you need!

---

Capturing the screen image of your own application

For completeness, I want to include your own comment that you can capture an image of your own application using Bitmap.createBitmap(rootview.getDrawingCache()); and similar mechanisms.

Capturing the screen of another application whilst you're in the background

Using the READ_FRAMEBUFFER permission

Firstly, you're right that a normal application can't make use of the READ_FRAMEBUFFER permission, because it's "signature"-level. That means you must be signed by the same key as the Android system ROM in order to be able to take such a screenshot.

I thought this was a bit sad, so back in 2009 I made an Android open-source project submission to ask that it be opened up1. The response from Dianne Hackborn, the Android architect was:

Um, no. Absolutely positively not.

So, that went well, then! Hence this permission is still signature-level to this day.

If you had this permission, however, you could call the captureScreen member of ISurfaceComposer2. You'd need to write some native code to access this function, using the Android NDK and also some undocumented APIs. However, it's possible.

Internally within the Android graphics subsystem, this uses a glReadPixels call to retrieve the pixels from the GPU back to the CPU. (The GPU is used for most of the compositing on Android. In fact Android 4.0+ supports extra hardware compositors, and the Surface Flinger has to do even more work to pull those pixels back to the CPU.)

This call works beautifully, except for a few small problems:

• The risk of using an unsupported API which might break at any moment;
• The hassle of calling it in C++
• It causes the GPU pipelines to stall, which can upset the GPU designers but doesn't really cause problems in reality
• It relies on a large bandwidth from the GPU back to the CPU. This is sometimes problematic because memory architectures are designed to send data in the opposite direction. However, I seem to recall that all modern Android chipset architectures directly share memory between the GPU and CPU, except for one (it may be Broadcom? - I can't remember) where this may cause this mechanism to be very slow.

... and one large problem ...

• Most importantly, as a normal application writer, you can't even call this API due to the signature-level permissions required.

Still, on most Android devices, you can get 10 frames per second out of this. Better still, this API actually supports scaling the resulting image in hardware on the GPU, so if you're clever you can pre-scale the image to just the size you need, before the pixels even hit the CPU. So it can be extremely high performance.

Note, of course, that you as an application writer can't call glReadPixels because you don't have access to the relevant OpenGL context. It's owned by the surface flinger.

Using /dev/graphics/fb0 and similar

Some are tempted to try to read these Linux device files which represent the framebuffer. However, there are three problems:

• You need root.
• Sometimes they're not even there.
• Often, they don't represent the real screen image. Remember on Android that the graphics are composited on the GPU. So there's no reason why the CPU should have access to a copy of the full composited screen image, and it often doesn't. This file sometimes contains tearing (at best) and a garbage image (at worst). Interestingly, some of the tools for rooted phones do use this method, which I think is a mistake. If you've got root, you by definition have all Android permissions and can therefore call the above captureScreen API to get a correct image.

Using hardware partners

Now we get into the solutions which require commercial action.

Talking to the Android chipset makers often presents a solution. Since they design the hardware, they have access to the framebuffer - and they often are able to provide libraries which entirely avoid the Android permissions model by simply accessing their custom kernel drivers directly.

If you're aiming at a specific phone model, this is often a good way forward. Of course, the odds are you'll need to cooperate with the phone maker as well as the silicon manufacturer.

Sometimes this can provide outstanding results. For example I have heard it's possible on some hardware to pipe the phone hardware framebuffer directly into the phone hardware H.264 video encoder, and retrieve a pre-encoded video stream of whatever is on the phone screen. Outstanding. (Unfortunately, I only know this is possible on TI OMAP chips, which are gradually withdrawing from the phone market3).

Using security holes

Android rigidly enforces its permission model, and has few security holes. However the Android OEMs can sometimes be more careless.

For example a major OEM whose name begins with S has implemented a way to capture the screen using a keystroke. It saves it to a world-readable file on the SD card. Hypothetically you might be able to find what intercepts those keys and see how it works. Perhaps you could do something similar.

And perhaps there's a way for another major OEM whose name also begins with S.

No, I'm not going to go into any more detail on this section. To work out how to do those things, I'd need to have reverse-engineered software, and that might be illegal. Good luck, though.

Working with the phone makers

As described previously, the phone makers have ready access to an API which does work. And the phone makers have the signature-level permissions required.

So, all you need to do is to arrange to get your software signed by the phone maker.

This is, however, hard. By signing the software, the phone maker is guaranteeing its quality - so they should want to audit your source code. Also, due to the nature of Android - if they sign the software, they need to be the ones distributing it. You can't put it on the Market if it is signed by someone else's signature.

However, the OEM need not include it on the ROM - they can still distribute it on the Android market. But you can't.

A good solution would be if each vendor signed a small library which then could be accessed by a common SDK. Which leads me onto...

Work with software partners who have solved this already

I know a lot about this because I used to work at RealVNC. We worked with all the major Android phone vendors to get access to these signature-level APIs. I cannot overemphasise the many, many man-years of effort (commercially and technically) required to achieve this. Some of the OEMs have publicised this work - for example 4.

I do not work at RealVNC any more, so I have nothing to gain from advertising their software. But if you really really want to be able to capture the screen on multiple Android devices, you may wish to approach them about re-using their Remote Control Service or Android VNC SDK 5. It is not open-source so you should expect to pay, and believe me this is fair enough given the epic effort involved in working with all these Android OEMs.

In the interests of balance I should point out that other vendors have also worked with the phone makers on this - e.g. Soti. But I believe they all offer specific device management solutions, rather than a general remote control/event injection SDK.

Over USB

Another option - the adb daemon which listens for debugging connections over USB has slightly more privileges than a normal application, which is why it's able to grab the screen (you can see its image using the ddms tool). If you're able to run any command using adb then you too can gain those privileges (as per the android-screenshot-library linked previously).

Contribute to the Android open-source project

Eventually this problem reduced me to dust, and I left for greener pastures which didn't involve trying to squeeze pixels out of Android phones.

Before I left RealVNC though, we tried again to contribute these APIs to the Android open-source project. This time we got a more positive reaction6. In short, it was suggested that our security approach was almost right, but that the graphics system was in too much turmoil to accept our patches. Well, the great news is that the graphics system is no longer in turmoil - in fact it now has that captureScreen API which means no graphics system changes are needed whatsoever. It may therefore be possible to submit a new security mechanism to AOSP around this API which finally solves this problem.

Best of luck!

Answer 2

Maybe the android-screenshot-library can help. But well in their Usage page it says that it needs a native service started with adb (from the android sdk).

PS: Remember that Screenshot UX does not work for every unrooted phone.