How do I determine if a HTTP request came from a browser or something else like a web service?

Question

How do I determine if a HTTP request came from a browser or something else like a web service?

I am aware that there is a User Agent, but this seems mainly for web browsers. For instance I do I know the request came from a google maps or youtube. The request are coming off a tethered device.

If they have a different User Agent, can I get an example.

Thanks.

Answer 1

There is absolutely no way to know with certainty if a request came from a browser or something else making an HTTP request.

The HTTP protocol allows for the client to set the User Agent arbitrarily.

One can attempt to determine if the thing on the other end is a browser e.g. by executing JavaScript and measuring the outcome. However, some bots can execute JavaScript, and not all browsers execute JavaScript so that type of test can never be perfect.

Well-behaved "bots" (like common search engine spiders) will identify themselves using a User Agent specific to them. There's a pretty comprehensive list of user agents here:

http://www.user-agents.org/

http://www.botsvsbrowsers.com/