delphifile-read

TFileStream is slower than TMemoryStream


This is my second question on SO, I hope I'll format it correctly.

I found out at work that TFileStream is slower than TMemoryStream and I have really no idea as to why that could be the case. Hence I decided to do some testing.

In the application in which I found out this behaviour I loaded all files in a folder, and for each file I had to read it byte by byte. After switching from TFileStream to TMemoryStream the execution became istantaneous. To test this I made the functions below, in which the same file (FileName) is loaded into a TStream Nr times, each time the stream is read until its end Siz bytes at a time (I'm really sorry for my English).

The functions I used to test it are:

The first is a test for TFileStream, because it doesn't have a method LoadFromFile I have to create and destroy it each time.

function TestFileStream(Nr, Siz: Integer): Double;
var
  sw: TStopWatch;
  Stream: TFileStream;
  I: Integer;
  buffer: TBytes;
begin
  sw := TStopWatch.StartNew;
  System.SetLength(buffer, Siz);
  for I := 0 to Nr - 1 do
    begin
      Stream := TFileStream.Create(FileName, fmOpenRead or fmShareDenyWrite);
      try
        Stream.Position := 0;
        while Stream.Position + Siz < Stream.Size do
          Stream.ReadBuffer(buffer, Siz);
      finally
        Stream.Free;
      end;
    end;
  Result := sw.Elapsed.TotalSeconds;
  sw.Stop;
end;

The second is a test for TMemoryStream, also this time the TMemoryStream is created and destroyed Nr times.

function TestMemoryStream(Nr, Siz: Integer): Double;
var
  sw: TStopWatch;
  Stream: TMemoryStream;
  I: Integer;
  buffer: TBytes;
begin
  sw := TStopWatch.StartNew;
  System.SetLength(buffer, Siz);
  for I := 0 to Nr - 1 do
    begin
      Stream := TMemoryStream.Create;
      try
        Stream.LoadFromFile(FileName);
        Stream.Position := 0;
        while Stream.Position + Siz < Stream.Size do
          Stream.ReadBuffer(buffer, Siz);
      finally
        Stream.Free;
      end;
    end;
  Result := sw.Elapsed.TotalSeconds;
  sw.Stop;
end;

The third is a test for TMemoryStream, this time the TMemoryStream is created once.

function TestKeepMemoryStream(Nr, Siz: Integer): Double;
var
  sw: TStopWatch;
  Stream: TMemoryStream;
  I: Integer;
  buffer: TBytes;
begin
  sw := TStopWatch.StartNew;
  System.SetLength(buffer, Siz);
  Stream := TMemoryStream.Create;
  try
    for I := 0 to Nr - 1 do
      begin
        Stream.LoadFromFile(FileName);
        Stream.Position := 0;
        while Stream.Position + Siz < Stream.Size do
          Stream.ReadBuffer(buffer, Siz);
      end;
  finally
    Stream.Free;
  end;
  Result := sw.Elapsed.TotalSeconds;
  sw.Stop;
end;

The results are the following:

Nr = 100, Siz = 1
TFileStream:        27,8980448s
TMemoryStream:      0,1571709s
TMemoryStream Kept: 0,1607682s


Nr = 100, Siz = 16
TFileStream:        1,7674029s
TMemoryStream:      0,044709s
TMemoryStream Kept: 0,0432958s


Nr = 100, Siz = 4096
TFileStream:        0,0427971s
TMemoryStream:      0,0325959s
TMemoryStream Kept: 0,0316288s

What strikes me as odd is that TMemoryStream uses a TFileStream to load a file and the ReadBuffer method is inherited from TStream, so there shouldn't be a difference, as far as my uderstanding goes. Do you have an idea which could explain this?


Solution

  • If your Siz parameter is small, TFileStream will access the disk each time to read a very small chunk of data, whereas in your code TMemoryStream accesses the disk once, and then the small chunks are read from memory. If you make the Siz variable the size of the file on disk, you should not see a difference.