How to make background of system-overlay view MULTIPLY (blending-mode)?

Question

A little context for my problem first: I'm trying to create a system-overlay type view (i.e. draw over other apps) that is (for now) just a full-screen solid colour. I'm turning it on/off by starting/stopping a service.

Here's what my code looks like so far (inside the service):

public void onCreate() {
    super.onCreate();
    oView = new LinearLayout(this);
    oView.setBackgroundColor(0x66ffffff);

    PorterDuffColorFilter fil = new PorterDuffColorFilter(0xfff5961b, PorterDuff.Mode.MULTIPLY);
    oView.getBackground().setColorFilter(fil);

    WindowManager.LayoutParams params = new WindowManager.LayoutParams(
            WindowManager.LayoutParams.MATCH_PARENT,
            WindowManager.LayoutParams.MATCH_PARENT,
            WindowManager.LayoutParams.TYPE_SYSTEM_OVERLAY,
            0 | WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,
            PixelFormat.TRANSLUCENT);
    WindowManager wm = (WindowManager) getSystemService(WINDOW_SERVICE);
    wm.addView(oView, params);
}

This way, I can get a solid colour over the rest of my regular android usage when I start this service. Now, my specific problem: Is there any way I can get this colour to MULTIPLY (think photoshop blending mode) over the regular android usage behind it?

As you can tell from my code, I've tried using PorterDuff filter modes (a couple different combinations of them) to achieve it too, in vain.

Here's a couple screenshots to explain this better hopefully:

<-- Original screen without my service.

<-- Same screen with my current-code service turned on.

<-- Intended result on same screen. Notice how the darker colours are multiplied onto underneath.

As you can see, my current code only throws on a layer of solid colour. I appreciate all suggestions. Thanks in advance!

Answer 1

For those who still wondering if such thing is possible, I decided to share my experience of solving this issue.

Long story short

Android designed so it doesn't allow to apply color blending across views of different contexts (i'm actually cannot prove that statement, and this is purely from my personal experience), so to achieve what you want you will need first somehow render a destination image on a surface, that belongs to one of your application contexts.

---

0. Overview

There is no way to just broadcast android home screen within an application, at least because of security reasons (otherwise one would be able to steal personal data and passwords by populating users input by making an application look and behave exactly like a system home screen). However in Android API 21 (Lollipop) was introduce so-called MediaProjection class, that introduces possibility to record screen. I don't think it's possible for lower API without rooting a device (if it's OK for you, you are free to use adb shell screencap command from within the application using exec command of the Runtime class). If you have screenshot of the homescreen, you are able to create a feeling of homescreen from within your application and then apply your color blending. Unfortunately this screen recording functionality will record overlay itself as well, so you will have to record screen only when overlay is hidden.

1. Take screenshot

Taking screenshot with the MediaProjection is not that difficult, but takes some effort to perform. It also requires you to have an Activity to ask a user for screen recording permissions. In the first place you will need to ask for permission to use Media Project Service:

private final static int SCREEN_RECORDING_REQUEST_CODE = 0x0002;
...
private void requestScreenCapture() {
    MediaProjectionManager mediaProjectionManager = (MediaProjectionManager) getSystemService(Context.MEDIA_PROJECTION_SERVICE);
    Intent intent = mediaProjectionManager.createScreenCaptureIntent();
    startActivityForResult(intent, SCREEN_RECORDING_REQUEST_CODE);
}

And after that handle the request in the onActivityResult method. Please be advised that you also need to keep Intent returned from this request for subsequent use to get a VirtualDisplay (that actually does all the work for capturing screen)

@Override
protected void onActivityResult(int requestCode, int resultCode, @NonNull Intent data) {
    super.onActivityResult(requestCode, resultCode, data);
    switch (requestCode) {
        case SCREEN_RECORDING_REQUEST_CODE:
            if (resultCode == RESULT_OK) {
                launchOverlay(data);
            } else {
                finishWithMessage(R.string.error_permission_screen_capture);
            }
            break;
    }
}

Eventually you can get a MediaProjection instance (using intent data previously returned) from MediaProjectionManager system service:

MediaProjectionManager mediaProjectionManager = (MediaProjectionManager) appContext.getSystemService(MEDIA_PROJECTION_SERVICE);
// screenCastData is the Intent returned in the onActivityForResult method
mMediaProjection = mediaProjectionManager.getMediaProjection(Activity.RESULT_OK, screenCastData);

In order to make a VirtualDisplay render home screen, we should provide it with a Surface. Then if we need an image from this surface we would need to cache drawing and grab it into a bitmap, or ask the surface to draw directly to our canvas. However in this particular case there is no need to invent the wheel, there is already something handy for such purposes, called ImageReader. Since we need to mimic home screen, the ImageReader should be instantiated using real device size (including status bar and navigation buttons bar if it's presented as part of the Android screen):

mWindowManager = (WindowManager) context.getSystemService(Context.WINDOW_SERVICE);
final Display defaultDisplay = mWindowManager.getDefaultDisplay();
Point displaySize = new Point();
defaultDisplay.getRealSize(displaySize);
final ImageReader imageReader = ImageReader.newInstance(displaySize.x, displaySize.y, PixelFormat.RGBA_8888, 1);

We also need to set a listener of incoming image buffer, so when a screenshot is taken, it goes to this listener:

imageReader.setOnImageAvailableListener(this, null);

We will back to implementation of this listener soon. Now let's create a VirtualDisplay so it finally can do the work for us to take a screenshot:

final Display defaultDisplay = mWindowManager.getDefaultDisplay();
final DisplayMetrics displayMetrics = new DisplayMetrics();
defaultDisplay.getMetrics(displayMetrics);
mScreenDpi = displayMetrics.densityDpi;
mVirtualDisplay = mMediaProjection.createVirtualDisplay("virtual_display",
                                                        imageReader.getWidth(),
                                                        imageReader.getHeight(),
                                                        mScreenDpi,
                                                        DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
                                                        imageReader.getSurface(),
                                                        null, null);

Now ImageRender will send to it's listener new image whenever the VirtualDisplay emits it. The listener consists of only one method, it looks as follow:

@Override
public void onImageAvailable(@NonNull ImageReader reader) {
    final Image image = reader.acquireLatestImage();
    Image.Plane[] planes = image.getPlanes();
    ByteBuffer buffer = planes[0].getBuffer();
    int pixelStride = planes[0].getPixelStride();
    int rowStride = planes[0].getRowStride();
    int rowPadding = rowStride - pixelStride * image.getWidth();
    final Bitmap bitmap = Bitmap.createBitmap(image.getWidth() + rowPadding / pixelStride,
            image.getHeight(), Bitmap.Config.ARGB_8888);
    bitmap.copyPixelsFromBuffer(buffer);
    image.close();
    reader.close();
    onScreenshotTaken(bitmap);
}

After the Image processing is finished, you should call the close() method of the ImageReader. It frees all resources allocated by it, and at the same time makes the ImageReder unusable. Thus when you need to take next screenshot, you will have to instantiate another ImageReader. (Since our instance was made using buffer of 1 image, it is not usable anyway)

2. Put screenshot over Homescreen

Now, when we get exact screenshot of the Homescreen, it should be placed so the end-user won't notice the difference. Instead of a LinearLayout that you use in your question, I opted for an ImageView with similar layout params. As it already covered the whole screen, the only thing remains is to adjust the screenshot position within it, so it doesn't contain status bar and navigation bar buttons (this is because the screen recording is actually capture the whole device screen, whereas our overlay view can be placed only within the "drawable" area). To achieve that we can use simple matrix translation:

private void showDesktopScreenshot(Bitmap screenshot, ImageView imageView) {
    // The goal is to position the bitmap such it is attached to top of the screen display, by
    // moving it under status bar and/or navigation buttons bar
    Rect displayFrame = new Rect();
    imageView.getWindowVisibleDisplayFrame(displayFrame);
    final int statusBarHeight = displayFrame.top;
    imageView.setScaleType(ImageView.ScaleType.MATRIX);
    Matrix imageMatrix = new Matrix();
    imageMatrix.setTranslate(-displayFrame.left, -statusBarHeight);
    imageView.setImageMatrix(imageMatrix);
    imageView.setImageBitmap(screenshot);
}

3. Apply color blending

Since we already have a view containing the Homescreen, I found it convenient to extend it so it has a possibility to draw overlay with the blending mode applied. Let's extend the `ImageView class and introduce a couple of empty methods:

class OverlayImageView extends ImageView {
    @NonNull
    private final Paint mOverlayPaint;

    public OverlayImageView(Context context) {
        super(context);
    }

    void setOverlayColor(int color) {
    }

    void setOverlayPorterDuffMode(PorterDuff.Mode mode) {
    }
}

As you can see, I added a field of Paint type. It will help us to draw our overlay and reflect changes. I'm actually don't consider myself an expert in computer graphic, but to exclude any confusing, I would like to highlight that the approach you used in the question is somewhat wrong - you take the background drawable of your view (and it doesn't actually depend what is behind your view) and apply PorterDuff mode on it only. So the background serves as the destination color and the color you specified in the PorterDuffColorFilter constructor serves as the source color. Only these two things are blended, nothing else is affected. However, when you apply PorterDuff mode on a Paint and draw it on a canvas, all views that are behind this canvas (and belong to the same Context) are blended. Let's override onDraw method first, and just draw the paint over in our ImageView:

@Override
protected void onDraw(@NonNull Canvas canvas) {
    super.onDraw(canvas);
    canvas.drawPaint(mOverlayPaint);
}

Now we only need to add corresponding changes for our setters and ask the view to redraw itself by calling invalidate() method:

void setOverlayColor(@SuppressWarnings("SameParameterValue") int color) {
    mOverlayPaint.setColor(color);
    invalidate();
}

void setOverlayPorterDuffMode(@SuppressWarnings("SameParameterValue") PorterDuff.Mode mode) {
    mOverlayPaint.setXfermode(new PorterDuffXfermode(mode));
    invalidate();
}

That's it! Here is the example of the same effect without blending multiply mode and with it applied:

---

In place of conclusion

Of course this solution is far from perfect - the overlay completely overlaps the homescreen, and to make it viable, you should come up with a lot of workarounds to solve corner-cases such as common interaction, keyboard, scrolling, calls, system dialogs and many others. I gave it a try and made a quick application that hides the overlay whenever user touches the screen. It still has a lot of issues, but should be a good starting point for you. The main problem is to make the application aware of something around is happening. I didn't check the Accessibility Service, but it should fit very well, since it has a lot more information about user actions, compared to a common service. You can check my solution here - https://github.com/AlexandrSMed/stack-35590953