Efficient flat file searching in PHP

Question

I'd like to store 0 to ~5000 IP addresses in a plain text file, with an unrelated header at the top. Something like this:

Unrelated data
Unrelated data
----SEPARATOR----
1.2.3.4
5.6.7.8
9.1.2.3

Now I'd like to find if '5.6.7.8' is in that text file using PHP. I've only ever loaded an entire file and processed it in memory, but I wondered if there was a more efficient way of searching a text file in PHP. I only need a true/false if it's there.

Could anyone shed any light? Or would I be stuck with loading in the whole file first?

Thanks in advance!

Answer 1

5000 isn't a lot of records. You could easily do this:

$addresses = explode("\n", file_get_contents('filename.txt'));

and search it manually and it'll be quick.

If you were storing a lot more I would suggest storing them in a database, which is designed for that kind of thing. But for 5000 I think the full load plus brute force search is fine.

Don't optimize a problem until you have a problem. There's no point needlessly overcomplicating your solution.

Answer 2

I'm not sure if perl's command line tool needs to load the whole file to handle it, but you could do something similar to this:

<?php
...
$result = system("perl -p -i -e '5\.6\.7\.8' yourfile.txt");
if ($result)
    ....
else
    ....
...
?>

Another option would be to store the IP's in separate files based on the first or second group:

# 1.2.txt
1.2.3.4
1.2.3.5
1.2.3.6
...

# 5.6.txt
5.6.7.8
5.6.7.9
5.6.7.10
...

... etc.

That way you wouldn't necessarily have to worry about the files being so large you incur a performance penalty by loading the whole file into memory.