How to make a bot with PHP to log in and do stuff on a website [closed]

Question

What I am trying to do is make a bot which would log in to a site using my account and do some work automatically (legit work of course).

For example, there may be a post in a forum with thousands of comments separated into hundreds of pages. The post can only be viewed by registered users. I wish to find comments of a specific user there and I want a PHP bot to log in with my account, read every comment page one by one and extract comments of that specific user and display them to me.

This was just an example of course. I can do the extraction and parsing, but I cannot seem to actually make the bot log in and move on. I am only a beginner in PHP, I have tried the following:

1. Used my own form to pass post data to the login page of the site, but did not work because it kept saying invalid referer.

2. I logged in manually and then opened my script in a new tab, but it didn't work either and it asked the bot to log in again.

So can you please give me an idea about how to do this? Basically it needs to go to the login page, somehow type in the username and password, log in, go to a specific page, and do stuff.

By the way, can this be done with JavaScript as well?

Thanks.

Answer 1

You need php_curl to authenticate and then once you get the session, you can do the extraction. Something like this:

$host = curl_init($url);
curl_setopt($host, CURLOPT_HTTPHEADER, array('Content-Type: application/xml', $additionalHeaders));
curl_setopt($host, CURLOPT_HEADER, 1);
curl_setopt($host, CURLOPT_USERPWD, $username . ":" . $password);
curl_setopt($host, CURLOPT_TIMEOUT, 30);
curl_setopt($host, CURLOPT_POST, 1);
curl_setopt($host, CURLOPT_POSTFIELDS, $payloadName);
curl_setopt($host, CURLOPT_RETURNTRANSFER, TRUE);
$return = curl_exec($host);
curl_close($host);