Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Random File Generator (again!)

I badly need a random file generator that generates a truly random, non-compressible dummy files.

I ended up with this delphi code. It works, but it's painfully sloooow

 var
    Buf     : Integer;
    TheFile : TFileStream;
 begin
      TheFile := TFileStream.Create(FileName, fmCreate OR fmOpenReadWrite);
      with TheFile do
      begin
           for i := 0 to FileSize do    // Iterate
           begin
                Buf := Random(999999) * i;
                WriteBuffer(Buf, SizeOf(Buf));
           end;    // for
      end;    // with
 end,

My question is: Is there a fast random file generator that I can use? Both Delphi code and/or commandline tools are acceptable as long as:

  1. I can run it on Windows without manual intervention (I need this for my tests, no intervention is allowed)
  2. It's fast
  3. Files generated is non-compressible (ie. compressing the generated file results in no space saving)

EDIT For those interested, I applied the advice I received here and made this function, it's fast enough & 7zip has hard time compressing the generated data.

like image 567
TheDude Avatar asked Apr 18 '12 19:04

TheDude


2 Answers

Use a 4096-byte page-size, or multiple page-size, buffer. Writing one integer at a time will be slow.

like image 81
Martin James Avatar answered Oct 17 '22 08:10

Martin James


You can use my generate_random_file.py script (Python 3) that I used to generate test data in a project of mine.

  • It works both on Linux and Windows.
  • It is very fast, because it uses os.urandom() to generate the random data in chunks of 256 KiB instead of generating and writing each byte separately.
like image 21
robert Avatar answered Oct 17 '22 09:10

robert