IP Banning - most efficient way?

Question

I run a large forum and like everyone else have issues with spammers/bots. There are huge lists of known spam IP's that you can download and use in htaccess form, but my only concern is the file size. So I suppose the question is how big is too big, given it's going to be loading in for every user. Adding all the IP's in it gets to about 100kb.

Is there an alternative that would have less overhead? Possibly doing it with php, or will that result in some heavy load too due to file size and checking ips etc?

Any advice would be greatly appreciated.

Thanks,

Steve

Answer 1

There are often more efficient ways than IP bans. For example, hidden fields in a form only bots will fill out, or requiring javascript or cookies for submitting forms.

For IP banning, I wouldn’t use .htaccess files. Depending on your webserver it may read the htaccess files for each request. I’d definitely add the IP-bans into your webservers vhost configuration instead. That way I’d be sure the webserver will keep it in RAM and not read it again and again.

Doing it via PHP would also be an option. This way, you could also easily limit the bans to forms, like registration in your forum.

Answer 2

There are a few options:

• You can store the block list into the database. It's more effecient to query there than with a loop in PHP.
• You could pre-process the list with array_map(ip2long()) to save memory and possibly lookup time.
• You could package the IP list into a regular expression, maybe run it though an optimizer (Perl Regexp::Optimizer). PCRE testing would again be faster than a foreach and strpos tests. $regex = implode("|", array_map("preg_quote", file("ip.txt")));

But then, IP block lists are not often very reliable. Maybe you should implement the other two workarounds: hidden form fields to detect dumb bots. Or captchas to block non-humans (not very user-friendly, but solves the problem).