Using the same session for PhantomJs at each run

Question

I'm crawling a secure website which blocks me whenever I restart my crawler application(I need to change IP as a trick). I solved this issue by using default user profile in chrome driver like this (I'm using C# right now, but I can switch to java if needed):

ChromeOptions options = new ChromeOptions();
options.AddArguments($"user-data-dir=C:/Users/{Environment.UserName}/AppData/Local/Google/Chrome/User Data/Default");

It saves all sessions and cookies and restore them when restarting my application. Everything works as expected.

Now, I need to change my webdriver to PhantomJS for some reasons.

My question How can I make this scenario possible using PhantomJS: login to an account(like gmail or facebook), close my application and driver, find myself logged-in the next time I run the application and driver. In other words, how can I use the same session for PhantomJS at each run?

Try 1 (in C#):

After doing some search, I found that this can be done using local storage and cookies file arguments in PhantomJS. Now the problem is that local storage path is always empty and nothing is saved there(I navigate to multiple sites but still empty), therefore, I can't use the session from previous execution. My code to set local storage and cookies file is simple as below:

PhantomJSDriverService service = PhantomJSDriverService.CreateDefaultService();
service.LocalStoragePath = Application.StartupPath + "\\default";
service.CookiesFile = Application.StartupPath + "\\default\\Cookies";
IWebDriver driver = new PhantomJSDriver(service);

What is wrong with my approach?

Try 2 (in C#):

Based on @SiKing answer and comment discussions, I changed to below code(using AddArgument) but the directory is still empty:

string localStoragePath = Path.Combine(Path.GetTempPath(),"PhantomLocalStorage-");

if (!Directory.Exists(localStoragePath))
{
    Directory.CreateDirectory(localStoragePath);
}

PhantomJSDriverService service = PhantomJSDriverService.CreateDefaultService();
service.AddArgument("--local-storage-quota=5000");
service.AddArgument("--local-storage-path=" + localStoragePath);
IWebDriver driver = new PhantomJSDriver(service);

Try 3 (in java):

Directory is still empty:

DesiredCapabilities capabilities = DesiredCapabilities.phantomjs();
List<String> cliArgs = new ArrayList<String>();
Path local_storage_path = Paths.get(System.getProperty("java.io.tmpdir") + "PhantomLocalStorage-");
if (Files.notExists(local_storage_path)) {
    try {
        Files.createDirectory(local_storage_path);
    }
    catch (IOException e) {
        JOptionPane.showConfirmDialog(null, "Can Not Create Path");
    }
}
cliArgs.add("--local-storage-quota=5000");
cliArgs.add("--local-storage-path=" + local_storage_path.toString());
capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, cliArgs);
WebDriver driver = new PhantomJSDriver(capabilities);

Answer 1

PhantomJS by default starts with no local-storage; see this discussion.

To enable local-storage via Selenium I have used the following Java code. Sorry, it has been too long since I have used C#, but I am confident the C# bindings have similar methods available.

DesiredCapabilities capabilities = DesiredCapabilities.phantomjs();
// Phantom options can only be set from CLI
List<String> cliArgs = new ArrayList<String>();
cliArgs.add("--local-storage-quota=5000");
Path local_storage_path = Files.createTempDirectory("PhantomLocalStorage-");
cliArgs.add("--local-storage-path=" + local_storage_path.toString());
capabilities.setCapability(PhantomJSDriverService.PHANTOMJS_CLI_ARGS, cliArgs);
WebDriver driver = new PhantomJSDriver(capabilities);

Note that local_storage_path will not be deleted after you are done with it. If you need that, you can set up a hook as per this post. But I suspect in C# this part is going to be wildly different from Java.

Answer 2

For the cookies part you could just save them from one session:

ReadOnlyCollection<Cookie> cookiesFromFirstSession = driver.Manage().Cookies.AllCookies;

And load them into another webdriver session:

driver.Manage().Cookies.AddCookie();

I would guess that it is the cookies, not necessarily any other local storage data, that you need in order to keep the same session.