Using Selenium WebDriver in library csproj with .NET Core

Question

I'm trying to put together a web-scraping app, using Selenium and .NET Core, but I'm having trouble getting my WebDriver exes to be found.

I have one .csproj that will run the API for the project, which calls out to (amongst others) another .csproj that will handle the webscraping. All are in a single .sln, and all are running .NET Core 2.1

In the scraping proj, I've nuget-installed Selenium.WebDriver and Selenium.WebDriver.ChromeDriver.

I've created an endpoint in the API, which calls out to the scraping project, and runs a method that attempts to invoke new ChromeDriver(). It doesn't work :( Specifically, I get:

The chromedriver.exe file does not exist in the current directory or in a directory on the PATH environment variable. The driver can be downloaded at ... <url>

Seems fairly clear (although it dissappointingly doesn't tell you what "current directory" means. I'll be submitting a PR for that imminently)

---

By observing changes during a rebuild, and other research online, I see that:

• All the dlls and exes from the nuget packages are stored in the Global Nuget cache, rather than a nuget packages folder in the solution directory.
  • This appears to be expected behaviour: "Bug" raised in dotnet Std; MSDN migration docs.
• The chromedriver.exe appears to get copied to <solutionFolder>\<ScrapingProjectFolder>\bin\Debug\chromeDriver.exe.
  • I assume that this is what the ChromeDriver Nuget package does; certainly I haven't configured it myself.
  • This superficially feels like a reasonable thing for that ChromeDriver package to be doing as an attempt at "install this to make new ChromeDriver() JustWork."
• Digging into the WebDriver codebase, reveals that the "currentDirectory" that it's looking at is "the location of WebDriver.dll".
  • In my case, that's "<globalNugetPackagesCache>\selenium.webdriver\3.141.0\lib\netstandard2.0"
  • It doesn't seem like I should be trying to get the chromedriver.exe to end up in this folder - copying it into a different package's global cache seems wrong? (Do people agree?)

• This article seems to have reached broadly the same conclusion and says that the solution is to invoke the driver as:

  new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location))

  • Unfortunately, that path takes me to <solutionFolder>\<APIProjectFolder>\bin\Debug\<ScrapingProjectFolder>.dll, because the dll gets copied over the the API project's folder.

---

A couple of solutions occur to me, none of which really appeal:

• I could install Selenium.WebDriver.ChromeDriver into the API project.
  • Eww... the API project doesn't know about WebDriver or Selenium, and now the Scraping project doesn't have the driver exe.
• I could manually explictly copy the exe into the right place.
  • Doesn't really feel right, and feels fragile. I suspect this will make deployment painful.
• I could manually point the ChromeDriver constructor to a hard-coded path, that I just happen to know contains the current exe.
  • Seems similar to the above; though not quite as bad.
• ??? Is there some way to make all the DLLs etc. of a project get compiled into a single common folder? ???

---

Is there a good, non-hacky way to solve this problem. Which will result in a git repo that JustWorks, and is going to be relatively painless to deploy to a server in the future?

Are any of the things I've described above wrong, or mis-configured?

Answer 1

From what I understand you have an API project that depends on a Scraping project.

Scraping.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>  
    <LangVersion>7.2</LangVersion>
    <PublishChromeDriver>true</PublishChromeDriver>    
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Selenium.WebDriver" Version="3.141.0" />
    <PackageReference Include="Selenium.WebDriver.ChromeDriver" Version="2.46.0" />
  </ItemGroup>
</Project>

API.csproj:

<Project Sdk="Microsoft.NET.Sdk">

  <ItemGroup>
    <ProjectReference Include="..\Scraping\Scraping.csproj" />
  </ItemGroup>

  <PropertyGroup>
    <TargetFramework>netstandard2.0</TargetFramework>
    <LangVersion>7.2</LangVersion>
  </PropertyGroup>

</Project>

The trick is adding <PublishChromeDriver>true</PublishChromeDriver> to the transitive project to make it publish the chromedriver when running dotnet publish API.csproj The ChromeDriver package has custom build targets in the NuGet package so it is custom.

You can now use

new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location));

and dotnet run API.csproj

Answer 2

Please correct me if I'm wrong. You have some kind of Class Library that has reference to Selenium and you would like to use ChromeDriver.exe but you are getting an error that it cannot be found under the following location. This is fairly simple. Currently you are referencing Class Library lets say Foo to API. Your Assembly Location will point to API bin location, whereas chromedriver.exe is located under Class library bin. If this is the case the only thing you would have to do is copy following chromedriver.exe to final bin directory which is API.

Add following Post Build Event to your API project to copy chromedriver:

  <Target Name="PostBuild" AfterTargets="PostBuildEvent">
    <Exec Command="copy $(SolutionDir)\ClassLibrary\bin\Debug\netstandard2.0\chromedriver.exe $(TargetDir)" />
  </Target>

This will copy your chromedriver.exe to API bin. Later while initializing ChromeDriver use:

        var options = new ChromeOptions();
        var service = ChromeDriverService.CreateDefaultService(AppDomain.CurrentDomain.BaseDirectory);

        WebDriver = new ChromeDriver(service, options);

While AppDomain.CurrentDomain.BaseDirectory will point to your API bin directory.