I'm trying Windows overlapped IO but I can't seem to get it to work asynchronously. I've compiled and run the program below but it never prints anything, it just completes silently. I've read small reads could become synchronous, that's why I deliberately chose to read 512MB.
const DWORD Size = 1<<29; // 512MB
char* Buffer = (char*)malloc(Size);
DWORD BytesRead;
OVERLAPPED Overlapped;
memset(&Overlapped, 0, sizeof(Overlapped));
HANDLE File = CreateFile("BigFile", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL|FILE_FLAG_OVERLAPPED, NULL);
assert(File!=INVALID_HANDLE_VALUE);
DWORD Result = ReadFileEx(File, Buffer, Size, &Overlapped, NULL); // This line takes 150ms according to the debugger
assert(Result);
while(!GetOverlappedResult(File, &Overlapped, &BytesRead, FALSE)) {
printf("Waiting...\n");
}
As additional information, I've stepped the code into the debugger, and the Overlapped.InternalHigh
value gets updated (with the same value as Size
) during the ReadFileEx
call.
I've tried replacing malloc
with VirtualAlloc
, ReadFileEx
with ReadFile
, adding FILE_FLAG_NO_BUFFERING
, and checked that the return of ReadFile
was 0 and that GetLastError
would be ERROR_IO_PENDING
after the read. I've tried using RAMMap to see if the file was in cache but only 96KB of it was there.
I'm running Windows 10 (ver. 1703) with 8GB RAM.
Ok I've got it working, all thanks to @RbMm.
The memory allocation didn't change anything but the FILE_FLAG_NO_BUFFERING
flag and use of ReadFile
made it work. I tried using SleepEx
after ReadFileEx
and it threw an access violation at 0, which proves the point of @RbMm that the lpCompletionRoutine
is not optional but mandatory. (To me that means I'm going to use ReadFile
because I don't want a completion routine)
As to why it took me so long to realise what was happening: I trusted the debugger too much, obviously breaking into the debugger didn't stop the IO process, meaning memory was still being updated inside the OVERLAPPED
structure, which made me think things were instantaneous. On top of that I expected ReadFile
to be quick to return, but it actually takes 20ms if I attempt to read 512MB, it's far quicker when I request smaller amounts.
Thanks everyone for your suggestions :)
For completeness, here's the working program:
const DWORD Size = 1<<20;
char* Buffer = (char*)malloc(Size);
DWORD BytesRead;
OVERLAPPED Overlapped;
memset(&Overlapped, 0, sizeof(Overlapped));
HANDLE File = CreateFile("BigFile", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_NO_BUFFERING|FILE_FLAG_OVERLAPPED, NULL);
assert(File!=INVALID_HANDLE_VALUE);
DWORD Result = ReadFile(File, Buffer, Size, NULL, &Overlapped);
assert(Result==FALSE && GetLastError()==ERROR_IO_PENDING);
while(!GetOverlappedResult(File, &Overlapped, &BytesRead, FALSE)) {
printf("Waiting...\n");
}