Menu
I am using Selenium WebDriver in conjunction with java.awt.Robot to better simulate user interaction with our web application. Yes, I know it's probably unnecessary, but the customers I serve demand it.
Currently things are working pretty well, however I have a minor snag in that I can't seem to find a good way to get the web elements' on screen position. Things like the title bar, menu bar, navigation bar, etc all push the content down on the physical screen (which Robot gets its coordinates from), but has no impact on where Selenium reports the element is.
When I call: element.getLocation(); on a Selenium WebElement, it always gives me its location relative to the HTML content render pane, not the browser window itself.
A better illustration is: driver.findElement(By.tagName("body")).getLocation(); always returns 0,0, regardless of the window's actual on-screen location.
Right now I am hacking it by adding a vertical and horizontal offset after maximizing the window, but these aren't the same between different browsers (IE's top decorations take up more room then Firefox's, for example), and may be different for each user if they have bookmark tool bars, search bars, etc added in.
Yes, I know I could run in full screen mode, but I'd rather not, if at all possible.
Is there a way to use WebDriver to get the physical on-screen location of elements in a reliable manner?
440
Iterate through child windows. Get the handles of all the windows that are currently open using the command: Set<String> allWindowHandles = driver. getWindowHandles(); which returns the set of handles. Use the SwitchTo command to switch to the desired window and also pass the URL of the web page.
We can verify whether an element is present or visible in a page with Selenium webdriver. To check the presence of an element, we can use the method – findElements. The method findElements returns a list of matching elements. Then, we have to use the method size to get the number of items in the list.
I believe there's no way to get the real on-screen location of the elements on the page.
I also think the fullscreen mode is your best bet.
That said, I wrote a RobotCalibration class that can detect the real offsets of your current browser. It opens a specially crafted page and uses the Robot class to click on it. The algorithm starts in the center of the browser and then uses bisecting to find the top left corner of the browser viewport.
Tested on IE8 and FF18. Works for both maximized and windowed browsers. Known issue: If you have a top Bookmarks Toolbar enabled, it may click on some of the bookmarks and therefore redirect. It can be handled pretty easily, but I left it up to you if you needed that :).
The testing page:
<!DOCTYPE html>
<html lang="en" onclick="document.getElementById('counter').value++">
<head>
<meta charset="utf-8" />
<title>Calibration Test</title>
</head>
<body>
<img height="1" width="1" style="position: absolute; left: 0; top: 0;"
onclick="document.getElementById('done').value = 'yep'" />
<input type="text" id="counter" value="0" />
<input type="text" id="done" value="nope" />
</body>
</html>
The RobotCalibration class. It's a little bit long, so I suggest you to copypaste it into your favourite IDE and explore it there:
import java.awt.AWTException;
import java.awt.Dimension;
import java.awt.Point;
import java.awt.Robot;
import java.awt.Toolkit;
import java.awt.event.InputEvent;
import java.nio.file.Paths;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.ie.InternetExplorerDriver;
public class RobotCalibration {
public static Point calibrate(WebDriver driver) {
return new RobotCalibration(driver).calibrate();
}
/** Time for which to wait for the page response. */
private static final long TIMEOUT = 1000;
private final WebDriver driver;
private final Robot r;
private final Point browserCenter;
private int leftX;
private int rightX;
private int midX;
private int topY;
private int bottomY;
private int midY;
private RobotCalibration(WebDriver driver) {
this.driver = driver;
try {
driver.manage().window().getSize();
} catch (UnsupportedOperationException headlessBrowserException) {
throw new IllegalArgumentException("Calibrating a headless browser makes no sense.", headlessBrowserException);
}
try {
this.r = new Robot();
} catch (AWTException headlessEnvironmentException) {
throw new IllegalStateException("Robot won't work on headless environments.", headlessEnvironmentException);
}
Dimension screenSize = Toolkit.getDefaultToolkit().getScreenSize();
org.openqa.selenium.Dimension browserSize = driver.manage().window().getSize();
org.openqa.selenium.Point browserPos = driver.manage().window().getPosition();
// a maximized browser returns negative position
// a maximized browser returns size larger than actual screen size
// you can't click outside the screen
leftX = Math.max(0, browserPos.x);
rightX = Math.min(leftX + browserSize.width, screenSize.width - 1);
midX = (leftX + rightX) /2;
topY = Math.max(0, browserPos.y);
bottomY = Math.min(topY + browserSize.height, screenSize.height - 1);
midY = (topY + bottomY) /2;
browserCenter = new Point(midX, midY);
}
private Point calibrate() {
driver.get(Paths.get("files/RobotCalibration.html").toUri().toString());
// find left border
while (leftX < rightX) {
click(midX, midY);
if (clickWasSuccessful()) {
rightX = midX;
} else {
leftX = midX + 1;
// close any menu we could have opened
click(browserCenter.x, browserCenter.y);
}
midX = (leftX + rightX) /2;
}
// find top border
while (topY < bottomY) {
click(midX, midY);
if (clickWasSuccessful()) {
bottomY = midY;
} else {
topY = midY + 1;
// close any menu we could have opened
click(browserCenter.x, browserCenter.y);
}
midY = (topY + bottomY) /2;
}
if (!isCalibrated()) {
throw new IllegalStateException("Couldn't calibrate the Robot.");
}
return new Point(midX, midY);
}
/** clicks on the specified location */
private void click(int x, int y) {
r.mouseMove(x, y);
r.mousePress(InputEvent.BUTTON1_DOWN_MASK);
r.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
// for some reason, my IE8 can't properly register clicks that are close
// to each other faster than click every half a second
if (driver instanceof InternetExplorerDriver) {
sleep(500);
}
}
private static void sleep(int millis) {
try {
Thread.sleep(millis);
} catch (InterruptedException ignored) {
// nothing to do
}
}
private int counter = 0;
/** @return whether the click on a page was successful */
private boolean clickWasSuccessful() {
counter++;
long targetTime = System.currentTimeMillis() + TIMEOUT;
while (System.currentTimeMillis() < targetTime) {
int pageCounter = Integer.parseInt(driver.findElement(By.id("counter")).getAttribute("value"));
if (counter == pageCounter) {
return true;
}
}
return false;
}
/** @return whether the top left corner has already been clicked at */
private boolean isCalibrated() {
long targetTime = System.currentTimeMillis() + TIMEOUT;
while (System.currentTimeMillis() < targetTime) {
if (driver.findElement(By.id("done")).getAttribute("value").equals("yep")) {
return true;
}
}
return false;
}
}
Sample usage:
WebDriver driver = new InternetExplorerDriver();
Point p = RobotCalibration.calibrate(driver);
System.out.println("Left offset: " + p.x + ", top offset: " + p.y);
driver.quit();
Feel free to ask any questions if something is unclear.
109
It looks like selenium has a way to get the element position relative to your browser window now. In one of my tests I was trying to verify that a page was anchored to a certain position and figured the best way to do this was to get an element position relative to the window. This can be done using the getCoordinates().inViewPort() methods.
Here's some sample code to get a WebElement's relative location:
import org.openqa.selenium.By;
import org.openqa.selenium.Point;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.internal.Locatable;
public class RelativeElementLocation {
private static Point locationPoint;
private static Locatable elementLocation;
WebDriver driver = new FirefoxDriver();
WebElement element;
public Point location() throws Throwable {
driver.get("Insert your URL here");
element = driver.findElement(By.id("Insert id/css/xpath/ here"));
elementLocation = (Locatable) element;
locationPoint = elementLocation.getCoordinates().inViewPort();
System.out.println(locationPoint);
return locationPoint;
}
}
The only caveat to the inViewPort() method is that if you try to get the relative element location of an element that currently is not in page view, the method will scroll the page so that the element is in view and then give you the relative position. Meaning, say if you try to get an element's relative position that is above your browser page view, it's not going to give you a negative y coordinate, but rather scroll up to that element and make the y coordinate zero.
Hope this helps, I'm new to Selenium and Java so if there are any mistakes in my code please forgive me.
41
There's a simpler way than Slanec answer to get relative to absolute offset.
First you need a blank page with javascript attached click listener on window to get the coordinates and save them into a global variable :
window.coordinates = [];
window.attachEvent('click', function() {
window.coordinates = [event.pageX, event.pageY];
});
Then you click and get relative click position through JavascriptExecutor :
// browser center
Dimension size = driver.manage().window().getSize();
Point point = driver.manage().window().getPosition()
.translate(size.width / 2, size.height / 2);
// Click through robot
click(point.x, point.y);
Thread.sleep(500);
// read coordinates
List<Object> coordinates = (List<Object>) ((JavascriptExecutor) driver)
.executeScript("return window.coordinates;");
if (coordinates.size() != 2)
throw new IllegalStateException("Javascript did not receive the click");
// point contains screen coordinates of upper left page point
point.translate(-((Integer)coordinates.get(0)),
-((Integer)coordinates.get(1)));
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
