(Wide)String - storing in TFileStream, Delphi 7. What is the fastest way?

Question

I'm using Delphi7 (non-unicode VCL), I need to store lots of WideStrings inside a TFileStream. I can't use TStringStream as the (wide)strings are mixed with binary data, the format is projected to speed up loading and writing the data ... However I believe that current way I'm loading/writing the strings might be a bottleneck of my code ...

currently I'm writing length of a string, then writing it char by char ... while loading, first I'm loading the length, then loading char by char ...

So, what is the fastest way to save and load WideString to TFileStream?

Thanks in advance

Answer 1

Rather than read and write one character at a time, read and write them all at once:

procedure WriteWideString(const ws: WideString; stream: TStream);
var
  nChars: LongInt;
begin
  nChars := Length(ws);
  stream.WriteBuffer(nChars, SizeOf(nChars);
  if nChars > 0 then
    stream.WriteBuffer(ws[1], nChars * SizeOf(ws[1]));
end;

function ReadWideString(stream: TStream): WideString;
var
  nChars: LongInt;
begin
  stream.ReadBuffer(nChars, SizeOf(nChars));
  SetLength(Result, nChars);
  if nChars > 0 then
    stream.ReadBuffer(Result[1], nChars * SizeOf(Result[1]));
end;

Now, technically, since WideString is a Windows BSTR, it can contain an odd number of bytes. The Length function reads the number of bytes and divides by two, so it's possible (although not likely) that the code above will cut off the last byte. You could use this code instead:

procedure WriteWideString(const ws: WideString; stream: TStream);
var
  nBytes: LongInt;
begin
  nBytes := SysStringByteLen(Pointer(ws));
  stream.WriteBuffer(nBytes, SizeOf(nBytes));
  if nBytes > 0 then
    stream.WriteBuffer(Pointer(ws)^, nBytes);
end;

function ReadWideString(stream: TStream): WideString;
var
  nBytes: LongInt;
  buffer: PAnsiChar;
begin
  stream.ReadBuffer(nBytes, SizeOf(nBytes));
  if nBytes > 0 then begin
    GetMem(buffer, nBytes);
    try
      stream.ReadBuffer(buffer^, nBytes);
      Result := SysAllocStringByteLen(buffer, nBytes)
    finally
      FreeMem(buffer);
    end;
  end else
    Result := '';
end;

Inspired by Mghie's answer, have replaced my Read and Write calls with ReadBuffer and WriteBuffer. The latter will raise exceptions if they are unable to read or write the requested number of bytes.

Answer 2

There is nothing special about wide strings, to read and write them as fast as possible you need to read and write as much as possible in one go:

procedure TForm1.Button1Click(Sender: TObject);
var
  Str: TStream;
  W, W2: WideString;
  L: integer;
begin
  W := 'foo bar baz';

  Str := TFileStream.Create('test.bin', fmCreate);
  try
    // write WideString
    L := Length(W);
    Str.WriteBuffer(L, SizeOf(integer));
    if L > 0 then
      Str.WriteBuffer(W[1], L * SizeOf(WideChar));

    Str.Seek(0, soFromBeginning);
    // read back WideString
    Str.ReadBuffer(L, SizeOf(integer));
    if L > 0 then begin
      SetLength(W2, L);
      Str.ReadBuffer(W2[1], L * SizeOf(WideChar));
    end else
      W2 := '';
    Assert(W = W2);
  finally
    Str.Free;
  end;
end;