How to use PHP to delete X number of lines from the beginning of a text file?

Question

I'm writing a PHP script and the script outputs a simple text file log of the operations it performs. How would I use PHP to delete the first several lines from this file when it reaches a certain file size?

Ideally, I would like it to keep the first two lines (date/time created and blank) and start deleting from line 3 and delete X amount of lines. I already know about the filesize() function, so I'll be using that to check the file size.

Example log text:

*** LOG FILE CREATED ON 2008-10-18 AT 03:06:29 ***

2008-10-18 @ 03:06:29  CREATED: gallery/thumbs
2008-10-18 @ 03:08:03  RENAMED: gallery/IMG_9423.JPG to gallery/IMG_9423.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/IMG_9188.JPG to gallery/IMG_9188.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/IMG_9236.JPG to gallery/IMG_9236.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/IMG_9228.JPG to gallery/IMG_9228.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/IMG_3104.JPG to gallery/IMG_3104.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/First dance02.JPG to gallery/First dance02.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/BandG02.JPG to gallery/BandG02.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/official03.JPG to gallery/official03.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/Wedding32.JPG to gallery/Wedding32.jpg
2008-10-18 @ 03:08:03  RENAMED: gallery/Gettaway car16.JPG to gallery/Gettaway car16.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/Afterparty05.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/IMG_9254.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/IMG_9175.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/official05.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/First dance01.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/Wedding29.jpg
2008-10-18 @ 03:08:04  CREATED: gallery/thumbs/men walking.jpg

Answer 1

Use the SPL, Luke

PHP 5 comes with plenty of iterators goodness :

<?php

$line_to_strip = 5;
$new_file = new SplFileObject('test2.log', 'w');

foreach (new LimitIterator(new SplFileObject('test.log'), $line_to_strip) as $line)
    $new_file->fwrite($line);    

?>

It's cleaner that what you can do while messing with fopen, it does not hold the entire file in memory, only one line at a time, and you can plug it and reuse the pattern anywhere since it's full OO.

Answer 2

$x_amount_of_lines = 30;
$log = 'path/to/log.txt';
if (filesize($log) >= $max_size)) {
  $file = file($log);
  $line = $file[0];
  $file = array_splice($file, 2, $x_amount_of_lines);
  $file = array_splice($file, 0, 0, array($line, "\n")); // put the first line back in
  ...
}

edit: with correction from by rcar and saving the first line.

Answer 3

This is a text-book problem of log files, and I would like to propose another solution.

The problem with the "removing lines at the beginning of files" approach is that adding new lines becomes extremly slow, once it has to remove the first lines for every new lines it's writing.

Normal log file appending only involves writing a few more bytes at the end of the file in the file system (and once in a while it has to allocate a new sector, which results in extensive fragmentation - why log files usually are).

But the big problem here is when you are removing a line in the beginning for every row written. The entire file must first be read into memory and then rewritten resulting in huge ammount of I/O to the harddrive (in comparision). To make matters worse, the "split into PHP array and skip first rows" solutions here are extremly slow due to the nature of PHP arrays. This is not a problem if the log file size limit is very small or if it is written to unoften, but with a lot of writes (as in the case with log files), the same huge operation has to be done a lot of times resulting in major performance drawbacks.

This can be imagined as parking cars on a line with space for 50. Parking the first 50 cars is quick, just drive in behind the car infront and done. But when you come to 50, and the car at the front (beginning of file) must be removed you have to drive the 2'nd car to the 1'st position, 3rd to 2nd and so on, before you can drive in with the last car on the 50'th position. (And this must be repeated for every new car you want to park!)

My suggestion is instead saving to diffrent log files, datewise, and then store a maximum of 30 days back etc. Thus taking advantage of the filesystem, which has already solved this problem perfectly well.

Answer 4

You could use the file() function to read the file into an array of lines, then use array_slice() to remove the first X lines.

$X = 100; // Number of lines to remove

$lines = file('log.txt');
$first_line = $lines[0];
$lines = array_slice($lines, $X + 2);
$lines = array_merge(array($first_line, "\n"), $lines);

// Write to file
$file = fopen('log.txt', 'w');
fwrite($file, implode('', $lines));
fclose($file);

Answer 5

Here's a ready to go function

<?php
//--------------------------------
// FUNCTION TO TRUNCATE LOG FILES
//--------------------------------
function trim_log_to_length($path,$numHeaderRows,$numRowsToKeep){
    $file = file($path);
    $headerRows = array_slice($file,0,$numHeaderRows);
    // if this file is long enough were we should be truncating it
    if(count($file) - $numRowsToKeep > $numHeaderRows){
        // figure out the rows we wanna keep
        $dataRowsToKeep = array_slice($file,count($file)-$numRowsToKeep,$numRowsToKeep);
        // write the file
        $newFileRows = array_merge($headerRows,$dataRowsToKeep);
        file_put_contents($path, implode($newFileRows));
    }
}
?>