Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Get view of AccessibilityNodeInfo to create overlay

I'm writing a AccessibilityService and I want to create view overlays on the views from the current activity that the accessibility service can retrieve. I have no problems to retrieve all AccessibilityNodeInfo objects from the current activity, but I have no idea how to get the views from these objects to create overlays. Unfortunately there are only few examples regarding accessibility services. Maybe some of you already have experience with this topic. I hope you can help me! Thanks!

EDIT: A paper shows that overlays over an activity's view contents are possible:

The display overlay is able to perform these tasks thanks to the Android Accessibility Framework [10]. Using the accessibility API, it is able to access and inspect the GUI layout of the applications on the screen, without requiring modifications or the instrumentation of the application code."*

Link: http://www.onarlioglu.com/publications/fc2015babelcrypt.pdf

Page 6 and 8. Thanks!

like image 980
zuuurek Avatar asked Feb 15 '15 18:02

zuuurek


2 Answers

You cannot get the View objects from other apps, as the View objects are in a separate process from yours.

like image 129
CommonsWare Avatar answered Sep 20 '22 17:09

CommonsWare


Drawing overlays over the window using accessibility services is easy. I know this is an old question, but I thought I'd add the answer anyway.

RelativeLayout relativeLayout = new RelativeLayout(getContext());

WindowManager.LayoutParams topButtonParams = new WindowManager.LayoutParams(
    width, //The width of the screen
    height, //The height of the screen
    WindowManager.LayoutParams.TYPE_SYSTEM_OVERLAY,
    WindowManager.LayoutParams.FLAG_NOT_FOCUSABLE,
    PixelFormat.TRANSLUCENT);

//Just trust me, this alpha thing is important.  I know it's weird on a "translucent" view.
topButtonParams.alpha = 100;

relativeLayout.setLayoutParams(topButtonParams);

mWindowManager.addView(relativeLayout, topButtonParams);

Also, you need this permission in your manifest

<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />

Once you have an invisible relative layout overlay over the top of the entire screen, adding views to it is easy!

like image 23
ChrisCM Avatar answered Sep 17 '22 17:09

ChrisCM