cwindowswinapioverlapped-io

Windows overlapped IO actually blocks


I'm trying Windows overlapped IO but I can't seem to get it to work asynchronously. I've compiled and run the program below but it never prints anything, it just completes silently. I've read small reads could become synchronous, that's why I deliberately chose to read 512MB.

  const DWORD Size = 1<<29; // 512MB
  char* Buffer = (char*)malloc(Size);
  DWORD BytesRead;
  OVERLAPPED Overlapped;
  memset(&Overlapped, 0, sizeof(Overlapped));

  HANDLE File = CreateFile("BigFile", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL|FILE_FLAG_OVERLAPPED, NULL);
  assert(File!=INVALID_HANDLE_VALUE);

  DWORD Result = ReadFileEx(File, Buffer, Size, &Overlapped, NULL); // This line takes 150ms according to the debugger
  assert(Result);

  while(!GetOverlappedResult(File, &Overlapped, &BytesRead, FALSE)) {
    printf("Waiting...\n");
  }

As additional information, I've stepped the code into the debugger, and the Overlapped.InternalHigh value gets updated (with the same value as Size) during the ReadFileEx call.

I've tried replacing malloc with VirtualAlloc, ReadFileEx with ReadFile, adding FILE_FLAG_NO_BUFFERING, and checked that the return of ReadFile was 0 and that GetLastError would be ERROR_IO_PENDING after the read. I've tried using RAMMap to see if the file was in cache but only 96KB of it was there.

I'm running Windows 10 (ver. 1703) with 8GB RAM.


Solution

  • Ok I've got it working, all thanks to @RbMm.

    The memory allocation didn't change anything but the FILE_FLAG_NO_BUFFERING flag and use of ReadFile made it work. I tried using SleepEx after ReadFileEx and it threw an access violation at 0, which proves the point of @RbMm that the lpCompletionRoutine is not optional but mandatory. (To me that means I'm going to use ReadFile because I don't want a completion routine)

    As to why it took me so long to realise what was happening: I trusted the debugger too much, obviously breaking into the debugger didn't stop the IO process, meaning memory was still being updated inside the OVERLAPPED structure, which made me think things were instantaneous. On top of that I expected ReadFile to be quick to return, but it actually takes 20ms if I attempt to read 512MB, it's far quicker when I request smaller amounts.

    Thanks everyone for your suggestions :)

    For completeness, here's the working program:

      const DWORD Size = 1<<20;
      char* Buffer = (char*)malloc(Size);
      DWORD BytesRead;
      OVERLAPPED Overlapped;
      memset(&Overlapped, 0, sizeof(Overlapped));
    
      HANDLE File = CreateFile("BigFile", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_NO_BUFFERING|FILE_FLAG_OVERLAPPED, NULL);
      assert(File!=INVALID_HANDLE_VALUE);
      DWORD Result = ReadFile(File, Buffer, Size, NULL, &Overlapped);
      assert(Result==FALSE && GetLastError()==ERROR_IO_PENDING);
      while(!GetOverlappedResult(File, &Overlapped, &BytesRead, FALSE)) {
        printf("Waiting...\n");
      }