What I want is the following:
1) Login at a page using Internet Explorer
2) Have my software which uses WinInet API to crawl a site reuse the same cookies/session (i.e. so the crawler is "logged in")
I have tried build login myself using INDY, and while session cookie logins works well, many websites use a more complicated login mechanism nowadays combining client and server side stuff for the initial login.
...
The reason I believe that e.g. WordPress , ASP.Net etc. uses client-side/AJAX stuff as well with hash/time/useragent codes/checks and similar, is that I can be sending the exact same HTTP headers 30 seconds later from my own crawler program as e.g. FireFox used during a login, but it won't work.
(Compare this to normal logins based on session cookies where my own crawler program will login fine.)
It seems IE does not share "internet cookies" by default. In Tools > Internet options > Security, there are different zones. Internet zone is protected mode enabled by default and you cannot access this cookies using the Wininet API. However, the Local intranet zone is not protected mode enabled by default, so you access this cookies using the Wininet API.
The protected mode cookies are stored in %appdata%\Microsoft\Windows\Cookies\Low and not protected mode cookies are stored in %appdata%\Microsoft\Windows\Cookies.
To fix your software, uncheck the Enable protected mode from the Internet zone, restart IE and log in at your website to acquire the cookies again.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With