Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to delete bytes from the beginning of a file?

I know that I can efficiently truncate a file and remove bytes from the end of the file.

Is there a corresponding efficient way to truncate files by deleting content from the beginning of the file to a point in the middle of the file?

like image 262
TheDude Avatar asked Mar 07 '12 08:03

TheDude


People also ask

How do I remove the first few bytes of a file?

To delete the first few bytes, you can either overwrite them, or you have to rewrite and re-align the whole file. Something like dd can do this fairly efficiently.

How can I remove bytes from a file?

It's easy to append bytes to the end of a file: Just open it for writing, seek to the end, and start writing. It's also easy to delete bytes from the end of a file: Seek to the point where you want the file to be truncated and call SetEndOfFile .


2 Answers

As I read the question you are asking to remove content from a file starting from the beginning of the file. In other words you wish to delete content at the start of the file and shift the remaining content down.

This is not possible. You can only truncate a file from the end, not from the beginning. You will need to copy the remaining content into a new file, or copy it down yourself within the same file.

However you do it there is no shortcut efficient way to do this. You have to copy the data, for example as @kobik describes.

Raymond Chen wrote a nice article on this topic: How do I delete bytes from the beginning of a file?


Just for fun, here's a simple implementation of a stream based method to delete content from anywhere in the file. You could use this with a read/write file stream. I've not tested the code, I'll leave that to you!

procedure DeleteFromStream(Stream: TStream; Start, Length: Int64);
var
  Buffer: Pointer;
  BufferSize: Integer;
  BytesToRead: Int64;
  BytesRemaining: Int64;
  SourcePos, DestPos: Int64;
begin
  SourcePos := Start+Length;
  DestPos := Start;
  BytesRemaining := Stream.Size-SourcePos;
  BufferSize := Min(BytesRemaining, 1024*1024*16);//no bigger than 16MB
  GetMem(Buffer, BufferSize);
  try
    while BytesRemaining>0 do begin
      BytesToRead := Min(BufferSize, BytesRemaining);
      Stream.Position := SourcePos;
      Stream.ReadBuffer(Buffer^, BytesToRead);
      Stream.Position := DestPos;
      Stream.WriteBuffer(Buffer^, BytesToRead);
      inc(SourcePos, BytesToRead);
      inc(DestPos, BytesToRead);
      dec(BytesRemaining, BytesToRead);
    end;
    Stream.Size := DestPos;
  finally
    FreeMem(Buffer);
  end;
end;
like image 76
David Heffernan Avatar answered Nov 13 '22 07:11

David Heffernan


A very simple solution would be to shift (move) blocks of data from the "target position offset" towards BOF, and then trim (truncate) the leftovers:

--------------------------
|******|xxxxxx|yyyyyy|zzz|
--------------------------
BOF  <-^ (target position offset)


--------------------------
|xxxxxx|yyyyyy|zzz|******|
--------------------------
                  ^ EOF

Since @David posted a code based on TStream, here is some code based on "low level" I/O pascal style:

function FileDeleteFromBOF(const FileName: string; const Offset: Cardinal): Boolean;
var
  Buf: Pointer;
  BufSize, FSize,
  NumRead, NumWrite,
  OffsetFrom, OffsetTo: Cardinal;
  F: file;
begin
  {$IOCHECKS OFF}
  Result := False;
  AssignFile(F, FileName);
  try
    FileMode := 2; // Read/Write
    Reset(F, 1); // Record size = 1
    FSize := FileSize(F);
    if (IOResult <> 0) or (Offset >= FSize) then Exit;
    BufSize := Min(Offset, 1024 * 64); // Max 64k - This value could be optimized
    GetMem(Buf, BufSize);
    try
      OffsetFrom := Offset;
      OffsetTo := 0;
      repeat
        Seek(F, OffsetFrom);
        BlockRead(F, Buf^, BufSize, NumRead);
        if NumRead = 0 then Break;
        Seek(F, OffsetTo);
        BlockWrite(F, Buf^, NumRead, NumWrite);
        Inc(OffsetFrom, NumWrite);
        Inc(OffsetTo, NumWrite);
      until (NumRead = 0) or (NumWrite <> NumRead) or (OffsetFrom >= FSize);
      // Truncate and set to EOF
      Seek(F, FSize - Offset);
      Truncate(F);
      Result := IOResult = 0;
    finally
      FreeMem(Buf);
    end;
  finally
    CloseFile(F);
  end;
end;
like image 33
kobik Avatar answered Nov 13 '22 08:11

kobik