I'm trying to use the Selenium RemoteWebDriver in C#, basically all I want to do is to programmatically scrape a webpage's html after the javascript has completed manipulating the DOM (without a browser window popping up.)
First I started the selenium-server.jar like so:
C:\Program Files\selenium-server>java -jar "C:\Program Files\selenium-server\selenium-server.jar"
13:34:46.163 INFO - Java: Sun Microsystems Inc. 19.1-b02
13:34:46.166 INFO - OS: Windows 7 6.1 amd64
13:34:46.174 INFO - v2.0 [a2], with Core v2.0 [a2]
13:34:46.277 INFO - RemoteWebDriver instances should connect to: **http://127.0.0.1:4444/wd/hub**
13:34:46.278 INFO - Version Jetty/5.1.x
13:34:46.279 INFO - Started HttpContext[/selenium-server/driver,/selenium-server
/driver]
13:34:46.280 INFO - Started HttpContext[/selenium-server,/selenium-server]
13:34:46.280 INFO - Started HttpContext[/,/]
13:34:46.311 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@6019d0
a1
13:34:46.312 INFO - Started HttpContext[/wd,/wd]
13:34:46.316 INFO - Started SocketListener on 0.0.0.0:4444
13:34:46.316 INFO - Started org.openqa.jetty.jetty.Server@199a0c7c
Next I tried to run this line from a test case:
var driver = new RemoteWebDriver(new Uri("http://127.0.0.1:4444/wd/hub"), DesiredCapabilities.Chrome());
This line errors:
Test 'Housters.Test.ScrapeTest.TestSelenium' failed: OpenQA.Selenium.WebDriverException : Unexpected error. {"message":"java.lang.NullPointerException","localizedMessage":"java.lang.NullPointerException","cause":{"class":"java.lang.NullPointerException","stackTrace":[{"fileName":"DriverFactory.java","class":"java.lang.StackTraceElement","lineNumber":43,"className":"org.openqa.selenium.remote.server.DriverFactory","methodName":"getBestMatchFor","nativeMethod":false},{"fileName":"DriverFactory.java","class":"java.lang.StackTraceElement","lineNumber":76,"className":"org.openqa.selenium.remote.server.DriverFactory","methodName":"newInstance","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":48,"className":"org.openqa.selenium.remote.server.Session$1","methodName":"call","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":46,"className":"org.openqa.selenium.remote.server.Session$1","methodName":"call","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask$Sync","methodName":"innerRun","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask","methodName":"run","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.ThreadPoolExecutor$Worker","methodName":"runTask","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.ThreadPoolExecutor$Worker","methodName":"run","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.lang.Thread","methodName":"run","nativeMethod":false}]},"class":"java.util.concurrent.ExecutionException","stackTrace":[{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask$Sync","methodName":"innerGet","nativeMethod":false},{"class":"java.lang.StackTraceElement","lineNumber":-1,"className":"java.util.concurrent.FutureTask","methodName":"get","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":68,"className":"org.openqa.selenium.remote.server.Session","methodName":"execute","nativeMethod":false},{"fileName":"Session.java","class":"java.lang.StackTraceElement","lineNumber":54,"className":"org.openqa.selenium.remote.server.Session","methodName":"<init>","nativeMethod":false},{"fileName":"DriverSessions.java","class":"java.lang.StackTraceElement","lineNumber":76,"className":"org.openqa.selenium.remote.server.DriverSessions","methodName":"newSession","nativeMethod":false},{"fileName":"NewSession.java","class":"java.lang.StackTraceElement","lineNumber":46,"className":"org.openqa.selenium.remote.server.handler.NewSession","methodName":"handle","nativeMethod":false},{"fileName":"ResultConfig.java","class":"java.lang.StackTraceElement","lineNumber":144,"className":"org.openqa.selenium.remote.server.rest.ResultConfig","methodName":"handle","nativeMethod":false},{"fileName":"DriverServlet.java","class":"java.lang.StackTraceElement","lineNumber":271,"className":"org.openqa.selenium.remote.server.DriverServlet","methodName":"handleRequest","nativeMethod":false},{"fileName":"DriverServlet.java","class":"java.lang.StackTraceElement","lineNumber":256,"className":"org.openqa.selenium.remote.server.DriverServlet","methodName":"doPost","nativeMethod":false},{"fileName":"HttpServlet.java","class":"java.lang.StackTraceElement","lineNumber":727,"className":"javax.servlet.http.HttpServlet","methodName":"service","nativeMethod":false},{"fileName":"HttpServlet.java","class":"java.lang.StackTraceElement","lineNumber":820,"className":"javax.servlet.http.HttpServlet","methodName":"service","nativeMethod":false},{"fileName":"ServletHolder.java","class":"java.lang.StackTraceElement","lineNumber":428,"className":"org.openqa.jetty.jetty.servlet.ServletHolder","methodName":"handle","nativeMethod":false},{"fileName":"ServletHandler.java","class":"java.lang.StackTraceElement","lineNumber":677,"className":"org.openqa.jetty.jetty.servlet.ServletHandler","methodName":"dispatch","nativeMethod":false},{"fileName":"ServletHandler.java","class":"java.lang.StackTraceElement","lineNumber":568,"className":"org.openqa.jetty.jetty.servlet.ServletHandler","methodName":"handle","nativeMethod":false},{"fileName":"HttpContext.java","class":"java.lang.StackTraceElement","lineNumber":1530,"className":"org.openqa.jetty.http.HttpContext","methodName":"handle","nativeMethod":false},{"fileName":"HttpContext.java","class":"java.lang.StackTraceElement","lineNumber":1482,"className":"org.openqa.jetty.http.HttpContext","methodName":"handle","nativeMethod":false},{"fileName":"HttpServer.java","class":"java.lang.StackTraceElement","lineNumber":909,"className":"org.openqa.jetty.http.HttpServer","methodName":"service","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":820,"className":"org.openqa.jetty.http.HttpConnection","methodName":"service","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":986,"className":"org.openqa.jetty.http.HttpConnection","methodName":"handleNext","nativeMethod":false},{"fileName":"HttpConnection.java","class":"java.lang.StackTraceElement","lineNumber":837,"className":"org.openqa.jetty.http.HttpConnection","methodName":"handle","nativeMethod":false},{"fileName":"SocketListener.java","class":"java.lang.StackTraceElement","lineNumber":245,"className":"org.openqa.jetty.http.SocketListener","methodName":"handleConnection","nativeMethod":false},{"fileName":"ThreadedServer.java","class":"java.lang.StackTraceElement","lineNumber":357,"className":"org.openqa.jetty.util.ThreadedServer","methodName":"handle","nativeMethod":false},{"fileName":"ThreadPool.java","class":"java.lang.StackTraceElement","lineNumber":534,"className":"org.openqa.jetty.util.ThreadPool$PoolThread","methodName":"run","nativeMethod":false}]}
at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(DriverCommand driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.StartSession(ICapabilities desiredCapabilities)
at OpenQA.Selenium.Remote.RemoteWebDriver..ctor(Uri remoteAddress, ICapabilities desiredCapabilities)
ScrapeTest.cs(36,0): at Housters.Test.ScrapeTest.TestSelenium()
In the service console window, it shows this error:
13:44:55.558 INFO - WebDriver remote server: Executing: [new session: null] at URL: /session)
13:44:55.560 INFO - WebDriver remote server: Exception: java.lang.NullPointerException
I'm trying to do this from Windows 7 x64. What am I doing wrong? This seems like a lot of work for what I want to do...
Selenium RemoteWebDriver Installation Once the Selenium grid is installed, follow the steps below to configure the Selenium RemoteWebdriver. Open a web browser and navigate to http://localhost:4444/grid/console. The grid console page will be visible. Connect to Selenium Hub to execute test cases.
you can, but remote one will work for "normal" case as well as for grid case.
The reason you do not often see a new ChromeDriver being assigned to a RemoteWebDriver variable is because from the perspective of a test, there is no functional difference between a WebDriver object and a RemoteWebDriver object. The level of abstraction provided by RemoteWebDriver is simply not very useful.
RemoteWebDriver Class. The RemoteWebDriver type exposes the following members. Initializes a new instance of the RemoteWebDriver class using the specified remote address, desired capabilities, and command timeout.
I see that you already have an accepted answer, but because it answers completely different question, I will add my 5 cents.
So if you want to connect to a remove web driver, instead of the line:
var driver = new FirefoxDriver();
You have to do:
var driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), DesiredCapabilities.Firefox()); // instead of this url you can put the url of your remote hub
Do not forget to include remote namespace before the previous line:
using OpenQA.Selenium.Remote;
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With