delphifreepascallazarustdatetime

Trying to create data transfer rate calculator using Lazarus and Freepascal

I am trying to add to my application interface a "Transfer Rate : XGb p\min" feature. I am computing it at a millisecond level using Lazarus 1.2.2 and Freepascal 2.6.4.

I have a loop that reads 64Kb blocks from a disk, does stuff with each 64Kb block, and repeats until the whole disk is read or until the user clicks an abort button. I am trying to time how long each read of 64Kb takes and then compute the average rate of data read per minute, i.e. "3Gb per minute".

I have a custom function called 'GetTimeInMilliseconds' that computes the standard time to milliseconds.

Also, to avoid the interface being refreshed ever partial millisecond, I have a counter that tries to ensure the interface is only refreshed every 50 times round the loop, i.e. when 64Kb chunks have been read 50 times, then the counter is reset to 0.

The trouble is, the transfer display is either not getting populated or it gets populated with an inaccurate figure like "234Mb p\min" for a RAID0 device! At other times, it gets something more realistic like "3.4Gb p\min". It should be consistently accurate if ran repeatedly on the same PC and same disk, not inconsistantly inaccurate.

Here is my Try...finally loop that does the looping. It also called FormatByteSize which is a custom function that computers an integer size and converts it into XMb, XGb, XTb etc. 

var

ProgressCounter, BytesTransferred, BytesPerSecond : integer;

Buffer : array [0..65535] of Byte;   // 65536, 64Kb buffer

ctx : TSHA1Context;

Digest : TSHA1Digest;

NewPos, ExactDiskSize, SectorCount, TimeStartRead, TimeEndRead,
  MillisecondsElapsed, BytesPerMillisecond, BytesPerMinute : Int64;

StartTime, EndTime, TimeTaken : TDateTime;
...
begin
... // Various stuff related to handles on disks etc
 try
    SHA1Init(ctx);
    FileSeek(hSelectedDisk, 0, 0);
    repeat
      ProgressCounter := ProgressCounter + 1; // We use this update the progress display occasionally, instead of every buffer read
      TimeStartRead   := GetTimeInMilliSeconds(Time); // Starting time, in Ms

      // The hashing bit...

      FileRead(hSelectedDisk, Buffer, 65536);  // Read 65536 = 64kb at a time
      NewPos := NewPos + SizeOf(Buffer);
      SHA1Update(ctx, Buffer, SizeOf(Buffer));
      FileSeek(hSelectedDisk, NewPos, 0);
      lblBytesLeftToHashB.Caption := IntToStr(ExactDiskSize - NewPos) + ' bytes, ' + FormatByteSize(ExactDiskSize - NewPos);

      // End of the hashing bit...

      TimeEndRead      := GetTimeInMilliSeconds(Time);;  // End time in Ms
      MillisecondsElapsed := (TimeEndRead - TimeStartRead); // Record how many Ms's have elapsed 

      // Only if the loop has gone round a while (50 times?), update the progress display
      if ProgressCounter = 50 then
        begin
          if (TimeStartRead > 0) and (TimeEndRead > 0) and (MillisecondsElapsed > 0) then // Only do the divisions and computations if all timings have computed to a positive number
            begin
              BytesTransferred := SizeOf(Buffer);
              BytesPerMillisecond := BytesTransferred DIV MilliSecondsElapsed; // BytesPerMillisecond if often reported as zero, even though BytesTRansferred and MilliSecondsElapsed are valid numbers? Maybe this is the fault? 
              BytesPerSecond := BytesPerMillisecond * 1000;  // Convert the bytes read per millisecond into bytes read per second
              BytesPerMinute := BytesPerSecond * 60; // Convert the bytes read per second into bytes read per minute
              lblSpeedB.Caption := FormatByteSize(BytesPerMinute) + ' p\min'; // now convert the large "bytes per minute" figure into Mb, Gb or Tb

              // Reset the progress counters to zero for another chunk of looping
              ProgressCounter := 0;
              BytesPerMillisecond := 0;
              BytesPerSecond := 0;
              BytesPerMinute := 0;
              ProgressCounter := 0;
            end;
        end;
      Application.ProcessMessages;
    until (NewPos >= ExactDiskSize) or (Stop = true); // Stop looping over the disk
    // Compute the final hash value
    SHA1Final(ctx, Digest);
    lblBytesLeftToHashB.Caption:= '0';
  finally
    // The handle may have been released by pressing stop. If not, the handle will still be active so lets close it.
    if not hSelectedDisk = INVALID_HANDLE_VALUE then CloseHandle(hSelectedDisk);
    EndTime := Now;
    TimeTaken := EndTime - StartTime;
    lblEndTimeB.Caption := FormatDateTime('dd/mm/yy hh:mm:ss', EndTime);
    if not stop then edtComputedHash.Text := SHA1Print(Digest);
    Application.ProcessMessages;
...

Function GetTimeInMilliSeconds(theTime : TTime): Int64;
// http://www.delphipages.com/forum/archive/index.php/t-135103.html
var
  Hour, Min, Sec, MSec: Word;
begin
  DecodeTime(theTime,Hour, Min, Sec, MSec); 
  Result := (Hour * 3600000) + (Min * 60000) + (Sec * 1000) + MSec;
end;

Example:

Form1.Caption := IntToStr( GetTimeInMilliSeconds(Time) );

Solution

  • I feel that you've made this much more complicated than it needs to be.

    Move your file processing code into a separate thread. Within that code, as you finish processing bytes from the file, use InterlockedAdd to keep a count of the total number of bytes processed.

    In your GUI thread, using a timer of some sort, use InterlockedExchange to read the value that InterlockedAdd has been modifying and reset it to zero. Then Calculate the amount of time that has passed since the last timer tick. Given the total number of bytes processed within the elapsed time, calculate the bytes per minute and update your GUI.

    Since the file operation is on a separate thread, you will not need to worry about how often you update the GUI (although doing so too frequently will be a waste of time).

    If you want to calculate the average bytes per minute over the entire operation, don't reset the total byte counter at each tick.