Sikuli gets confused between 2 identical buttons

Question

I have identical buttons on the same page and I want Sikuli to click only one of those but it ends up clicking the other name sake button. Unfortunately, the button names cannot be changed. Any suggestions on how I could handle this situation?

Thanks!

Answer 1

You can tell Sikuli to operate on a particular region on the screen relative to a given image/screenshot object. This is called TargetOffset. Refer here assuming you are asking about using sikuli programmatically. From Sikuli IDE, double click on the screenshot image and it brings up a window where you can set accuracy and targetOffset.

Answer 2

If the two icons are close together, and will always appear the same way, and the space between them will always appear the same, then a

click(imageOfTwoIcons).targetOffset(x,y)

is probably the simplest way to go. But, if there is anything that would make this method unreliable (anything between the two icons ever appears differently than when you are making your screen capture of them)--

You could also use the python sorted() function to sort the images by their position. For example, if one image is always above the other, then you could find both images and sort them by their y coordinate, like so:

#a little prep for the sorted function to get the y coord of the icon
def byY(icon):
    return icon.y

#findAll() on your two identical icons and make them into a list
bothIcons = list([x for x in findAll(icon)]) 

#then sort them
sortedIcons = sorted(bothIcons, key=byY)
iconOnTop = sortedIcons[0]  
iconOnBottom = sortedIcons[1]

#then click on the one you want
click(iconOnTop) #or save a line and say: click(sortedIcons[0])

The same can be done if you know that the icon of interest is always left or right of its twin:

def byX(yourTwoImages):
    return image.x
bothIcons = list([x for x in findAll(icon)])
sortedIcons = sorted(bothIcons, key=byX)
click(sortedIcons[0]) #for the image on the left

I like this better than working with regions if the two icons are very close together on the screen, or if it's possible for the icons' placement to ever change.