How to create an Auto Clicker app which can click on other apps?

Question

I have seen so many apps that provide auto tapping feature. But they don't provide some specific customization so I decided to create a one. I have seen many tutorials that offers auto tapping, but they are supposed to be used in the same app. But I want to create an auto tapping app that can click on other app's View. I am an intermediate java developer but new to Android studio, so I don't know much about Android APIs. Is there any class or package which can provide this feature. For more clearance, I want to do auto tap, when the color at the specific location on screen, (213, 120) for instance, becomes green. Thanks in advance!

Answer 1

I am not an expert on the subject, I have experience as a BackEnd developer, I am new to the Android world, just like you I am interested in building an app to automate some farming mechanics in Android games with specific behaviors, so I investigated on the subject and the solution you are looking for is the AccessibilityService API, from Android 8.0 (API level 26) it includes several functionalities to make touches without the need for Root, I leave you links with examples of some open source repos that can guide you.

https://developer.android.com/guide/topics/ui/accessibility/service#continued-gestures (Doc) https://github.com/nestorm001/AutoClicker (Old and abandoned project, but has the functionality of touches) https://github.com/pylapp/SmoothClicker (A more mature App but need root, it can be helpful to guide you in how to build the UI)

For the detection in the change of the pixel I still do not have a concrete answer, I keep looking, maybe obtaining a dump of the screen every X time and validate the pixels on that is the way but it is still uncertain if it is the way to go. Something like this How to record screen and take screenshots, using Android API?. I know that you can get a bitmat dumb of the screen if you have the activity, but it's not the case, however some class in this package may be helpful https://developer.android.com/reference/kotlin/android/graphics/package-summary

I'm currently working in my own implementation, when I have something to show I will gladly share it.