Menu
How do I determine if a HTTP request came from a browser or something else like a web service?
I am aware that there is a User Agent, but this seems mainly for web browsers. For instance I do I know the request came from a google maps or youtube. The request are coming off a tethered device.
If they have a different User Agent, can I get an example.
Thanks.
238
Browser Sends Request After a browser has performed the DNS lookup, it sends an HTTP request to the appropriate server. It doesn't have to literally be HTTP. It can be HTTPS or more recently an HTTP/2 request. The general idea though it that your browser sends a request for a specific file, often an HTML file.
How do you detect if a HTTP request is coming from a human or a bot? Detect that several unrelated pages were requested directly by some IP (not good enough) Watch for missing headers.It is likely to be a robot if: it requests the HTML page but no images. first requests a robots.
Basically : you cannot. With the HTTP protocol, each request is independent from the others. A first idea would be to check the Referer HTTP header, but note that : It can be faked (it's sent by the browser)
GET is used to retrieve and request data from a specified resource in a server. GET is one of the most popular HTTP request techniques. In simple words, the GET method is used to retrieve whatever information is identified by the Request-URL.
There is absolutely no way to know with certainty if a request came from a browser or something else making an HTTP request.
The HTTP protocol allows for the client to set the User Agent arbitrarily.
One can attempt to determine if the thing on the other end is a browser e.g. by executing JavaScript and measuring the outcome. However, some bots can execute JavaScript, and not all browsers execute JavaScript so that type of test can never be perfect.
Well-behaved "bots" (like common search engine spiders) will identify themselves using a User Agent specific to them. There's a pretty comprehensive list of user agents here:
http://www.user-agents.org/
http://www.botsvsbrowsers.com/
157
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
