INTRODUCTION:

I am trying to use ReadDirectoryChangesW asynchronously in a loop.

Below snippet illustrates what I am trying to achieve:

DWORD example()
{
    DWORD error = 0;

    OVERLAPPED ovl = { 0 };
    ovl.hEvent = ::CreateEvent(NULL, TRUE, FALSE, NULL);

    if (NULL == ovl.hEvent) return ::GetLastError();

    char buffer[1024];

    while(1)
    {
        process_list_of_existing_files();

        error = ::ReadDirectoryChangesW(
            m_hDirectory, // I have added FILE_FLAG_OVERLAPPED in CreateFile
            buffer, sizeof(buffer), FALSE,
            FILE_NOTIFY_CHANGE_FILE_NAME,
            NULL, &ovl, NULL);

        // we have new files, append them to the list
        if(error) append_new_files_to_the_list(buffer);
        // just continue with the loop
        else if(::GetLastError() == ERROR_IO_PENDING) continue;
        // RDCW error, this is critical -> exit
        else return ::GetLastError(); 
    }
}

PROBLEM:

I do not know how to handle the case when ReadDirectoryChangesW returns FALSE with GetLastError() code being ERROR_IO_PENDING.

In that case I should just continue with the loop and keep looping until ReadDirectoryChangesW returns buffer I can process.

MY EFFORTS TO SOLVE THIS:

I have tried using WaitForSingleObject(ovl.hEvent, 1000) but it crashes with error 1450 ERROR_NO_SYSTEM_RESOURCES. Below is the MVCE that reproduces this behavior:

#include <iostream>
#include <Windows.h>

DWORD processDirectoryChanges(const char *buffer)
{
    DWORD offset = 0;
    char fileName[MAX_PATH] = "";
    FILE_NOTIFY_INFORMATION *fni = NULL;

    do
    {
        fni = (FILE_NOTIFY_INFORMATION*)(&buffer[offset]);
        // since we do not use UNICODE, 
        // we must convert fni->FileName from UNICODE to multibyte
        int ret = ::WideCharToMultiByte(CP_ACP, 0, fni->FileName,
            fni->FileNameLength / sizeof(WCHAR),
            fileName, sizeof(fileName), NULL, NULL);

        switch (fni->Action)
        {
        case FILE_ACTION_ADDED:     
        {
            std::cout << fileName << std::endl;
        }
        break;
        default:
            break;
        }

        ::memset(fileName, '\0', sizeof(fileName));
        offset += fni->NextEntryOffset;

    } while (fni->NextEntryOffset != 0);

    return 0;
}

int main()
{
    HANDLE hDir = ::CreateFile("C:\\Users\\nenad.smiljkovic\\Desktop\\test", 
        FILE_LIST_DIRECTORY,
        FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
        NULL, OPEN_EXISTING, 
        FILE_FLAG_BACKUP_SEMANTICS | FILE_FLAG_OVERLAPPED, NULL);

    if (INVALID_HANDLE_VALUE == hDir) return ::GetLastError();

    OVERLAPPED ovl = { 0 };
    ovl.hEvent = ::CreateEvent(NULL, TRUE, FALSE, NULL);

    if (NULL == ovl.hEvent) return ::GetLastError();

    DWORD error = 0, br;
    char buffer[1024];

    while (1)
    {
        error = ::ReadDirectoryChangesW(hDir,
            buffer, sizeof(buffer), FALSE,
            FILE_NOTIFY_CHANGE_FILE_NAME,
            NULL, &ovl, NULL);

        if (0 == error)
        {
            error = ::GetLastError();

            if (ERROR_IO_PENDING != error)
            {
                ::CloseHandle(ovl.hEvent);
                ::CloseHandle(hDir);
                return error;
            }
        }

        error = ::WaitForSingleObject(ovl.hEvent, 0);

        switch (error)
        {
        case WAIT_TIMEOUT:
            break;
        case WAIT_OBJECT_0:
        {
            error = processDirectoryChanges(buffer);

            if (error > 0)
            {
                ::CloseHandle(ovl.hEvent);
                ::CloseHandle(hDir);
                return error;
            }

            if (0 == ::ResetEvent(ovl.hEvent))
            {
                error = ::GetLastError();
                ::CloseHandle(ovl.hEvent);
                ::CloseHandle(hDir);
                return error;
            }
        }
        break;
        default:
            error = ::GetLastError();
            ::CloseHandle(ovl.hEvent);
            ::CloseHandle(hDir);
            return error;
            break;
        }
    }

    return 0;
}

Reading through the documentation, it seems that I need GetOverlappedResult with last parameter set to FALSE but I do not know how to use this API properly.

QUESTION:

Since the MVCE illustrates very well what I am trying to do (print the names of the newly added files), can you show me what must be fixed in the while loop in order for it to work?

Again, the point is to use ReadDirectoryChangesW asynchronously, in a loop, as shown in the snippet from the INTRODUCTION.

The basic structure of your program looks more or less OK, you're just using the asynchronous I/O calls incorrectly. Whenever there are no new files, the wait on the event handle times out immediately, which is fine, but you then issue a brand new I/O request, which isn't.

That's why you're running out of system resources; you're issuing I/O requests full tilt without waiting for any of them to complete. You should only issue a new request after the existing request has completed.

(Also, you should be calling GetOverlappedResult to check whether the I/O was successful or not.)

So your loop should look more like this:

    ::ReadDirectoryChangesW(hDir,
        buffer, sizeof(buffer), FALSE,
        FILE_NOTIFY_CHANGE_FILE_NAME,
        NULL, &ovl, NULL);

    while (1)
    {
        DWORD dw;
        DWORD result = ::WaitForSingleObject(ovl.hEvent, 0);

        switch (result)
        {
        case WAIT_TIMEOUT:

            processBackgroundTasks();

            break;

        case WAIT_OBJECT_0:

            ::GetOverlappedResult(hDir, &ovl, &dw, FALSE);

            processDirectoryChanges(buffer);

            ::ResetEvent(ovl.hEvent);

            ::ReadDirectoryChangesW(hDir,
                buffer, sizeof(buffer), FALSE,
                FILE_NOTIFY_CHANGE_FILE_NAME,
                NULL, &ovl, NULL);

            break;
        }
    }

Notes:

• The error handling has been elided for simplicity; I have not done any testing or checked your code for any other problems.

• If there might not be any background tasks to perform, you should test for that case and set the timeout to INFINITE rather than 0 when it occurs, otherwise you will be spinning.

• I wanted to only show the minimal changes necessary to make it work, but calling WaitForSingleObject followed by GetOverlappedResult is redundant; a single call to GetOverlappedResult can both check whether the I/O is complete and retrieve the results if it is.

---

As requested, the modified version using only GetOverlappedResult and with minimal error checking. I've also added an example of how you might deal with the case where you've run out of work to do; if whatever processing you're doing on the files really does run forever, you don't need that bit.

    ::ResetEvent(ovl.hEvent);

    if (!::ReadDirectoryChangesW(hDir,
        buffer, sizeof(buffer), FALSE,
        FILE_NOTIFY_CHANGE_FILE_NAME,
        NULL, &ovl, NULL))
    {
       error = GetLastError();
       if (error != ERROR_IO_PENDING) fail();
    }

    while (1)
    {
        BOOL wait;

        result = process_list_of_existing_files();

        if (result == MORE_WORK_PENDING)
        {
           wait = FALSE;
        } 
        else if (result == NO_MORE_WORK_PENDING)
        {
           wait = TRUE;
        } 

        if (!::GetOverlappedResult(hDir, &ovl, &dw, wait))
        {
           error = GetLastError();
           if (error == ERROR_IO_INCOMPLETE) continue;
           fail();
        }

        processDirectoryChanges(buffer);

        ::ResetEvent(ovl.hEvent);

        if (!::ReadDirectoryChangesW(hDir,
            buffer, sizeof(buffer), FALSE,
            FILE_NOTIFY_CHANGE_FILE_NAME,
            NULL, &ovl, NULL))
        {
           error = GetLastError();
           if (error != ERROR_IO_PENDING) fail();
        } 
    }