Redirecting crawlers

Question

I have an online tool that keeps track of tasks and users completing tasks. As a part of the process I record $_SERVER['HTTP_USER_AGENT']. However, once in a while I get visits from various bots and crawlers. How do I gently redirect them elsewhere without "hurting their feelings"?

I was thinking I'd net to build an array with bot names and run each AGENT info against it, and if found in array, redirect.

Is there a better way of doing it?

Answer 1

If not done already, you could get rid of most crawlers by utilizing the robots.txt file. See here. This is not strictly adhered to, however. Those who keep on crawling can be IP banned. You can do this on Linux with iptables. Example:

iptables -A INPUT -s IP-ADDRESS -j DROP

Answer 2

Make list with needed spiders and make redirect with this code:

header('HTTP/1.1 301 Moved Permanently');
header('Location: NEED_URL_HERE');

Actualy you can use .htaccess or robots.txt (if crawler use it)

 User-agent: *
 Disallow: /

UPD: If you use this for SEO (cloacking) you can be punished by search engine, be care.