Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

PHP what is the best way to write data to middle of file without rewriting file

Tags:

file

php

I am working with large text files in php (1GB+), I am using

file_get_contents("file.txt", NULL, NULL, 100000000,100); 

To get data from the middle of the file, but if i wanted to change the data in the file to something that is of different change than the origional data, I would have to re-write the entire file.

How can I change data within the file (variable length) without overwriting data if the data is larger than the original? I keep an index of the different data blocks within the file and their byte location. It seems that the only alternative is to dedicate x amount of bytes to each piece of data and then rewrite that block if i wanted to change it... the problem with this is that it would take up a lot more space than needed in just null bytes, and it would take longer to write... and that still would not solve how to "remove" data, as the file could never shrink in size... I really need some help here...

If I used prefixed blocks for each piece of data in the file, like 1 mb, then I wanted to enter data that was only 100kb, that entry would take 10x actual needed space, and the entry could never be changed to something more than 1mb of data, as it would overwrite more than 1st dedicated block... removing it would not be possible... hope this makes any sense... I am not looking for alternatives, I am looking to write and change data in the middle of files, hehe...

UPDATE: Yes, I would like to replace the old data, but if the new data extends more than the old data I would want the rest of the data to be pushed further into the file...

consider this: 0000000HELLODATA00000000 the zeros represent empty space, nothing... now I would like to replace HELLO with SOMETHING, now since something is larger than hello, simply writing in the starting point of hello would extend byond hello and start overwriting data... therefore i would like DATA to be pushed futher into the file, to make room for SOMETHING without overwriting DATA... hehe

like image 730
Daniel Avatar asked May 29 '13 12:05

Daniel


2 Answers

To Overwrite Data :

$fp = fopen("file.txt", "rw+");
fseek($fp, 100000000); // move to the position
fwrite($fp, $string, 100); // Overwrite the data in this position 
fclose($fp);

To Inject Data

This is a tricky because you have to rewrite the file. It can be optimized with partial modificationfrom point of injection rather than the whole file

$string = "###INJECT THIS DATA ##### \n";
injectData("file.txt", $string, 100000000);

Function Used

function injectData($file, $data, $position) {
    $fpFile = fopen($file, "rw+");
    $fpTemp = fopen('php://temp', "rw+");

    $len = stream_copy_to_stream($fpFile, $fpTemp); // make a copy

    fseek($fpFile, $position); // move to the position
    fseek($fpTemp, $position); // move to the position

    fwrite($fpFile, $data); // Add the data

    stream_copy_to_stream($fpTemp, $fpFile); // @Jack

    fclose($fpFile); // close file
    fclose($fpTemp); // close tmp
}
like image 147
Baba Avatar answered Sep 27 '22 16:09

Baba


Another variant of the injectData() function:

function injectData($file, $data, $position) 
{
    $temp = fopen('php://temp', "rw+");
    $fd = fopen($file, 'r+b');

    fseek($fd, $position);
    stream_copy_to_stream($fd, $temp); // copy end

    fseek($fd, $position); // seek back
    fwrite($fd, $data); // write data

    rewind($temp);
    stream_copy_to_stream($temp, $fd); // stich end on again

    fclose($temp);
    fclose($fd);
}

It copies the end of file (from $position onwards) into a temporary file, seeks back to write the data and stitches everything back up.

like image 37
Ja͢ck Avatar answered Sep 27 '22 15:09

Ja͢ck