Log visits in shared memory

Question

I'm trying to find best way to log visits using PHP. Right now I have about 3000 request per second and I write each visit to CSV file. I was wondering is it faster to log each visit in memory somehow and then dump it to CSV file after 100 000 records? I've checked shmop apc and memcache so far but can't find proper solution.

Answer 1

The best way is to use Lua with shared memory to store log entries, then create a timer which checks the size of logged entries every X seconds and uses a co-socket to dump the cache to a file or sql database. Should all be non-blocking. And yes you can pass requests to Lua before passing it to PHP. You could also use nginx default logfile caching(buffer) and parse the logs separately.