c++winapiheap-memorystack-memoryapi-hook

Hooked NtOpenFile fails when std::wstring is defined on the stack


I have written a hooking dll using the mhook library. In a spezial case the NtOpenFile() fails when a std::wstring is defined as stack var. Defining it on the heap the code is working.

The code is working without problems except when a certain win32 application (lets call it nuisance.exe) tries to open an existing testfile (like c:\temp\anyfile.log) the access fails. Mostly STATUS_INVALID_ACL (0xC0000077) is returned then.

I have reduced my code line by line and finally found that the error happens when in a called function a std::wstring is defined (this example below). The error happens every time an on different OS's

NTSTATUS NtOpenFileApiHook::NtOpenFileHook(PHANDLE              FileHandle,
                                  ACCESS_MASK           DesiredAccess,
                                  POBJECT_ATTRIBUTES    ObjectAttributes,
                                  PIO_STATUS_BLOCK      IoStatusBlock,
                                  ULONG                 ShareAccess,
                                  ULONG                 OpenOptions 
                                  )
{
    NTSTATUS Status = STATUS_SUCCESS;

    // using this function the call will fail
    AfterThis_NtOpenFile_WillFail();

    // using this function INSTEAD the call will work
    AfterThis_NtOpenFile_WillWork();

    // calling the real NtOpenFile using a pointer
    // nothing was changed hier, the original parameters are passed
    Status = RealNtOpenFile(FileHandle, ...);

    return Status;
}

int AfterThis_NtOpenFile_WillFail()
{
    std::wstring String = L"";

    return 0;
}

int AfterThis_NtOpenFile_WillWork()
{
    std::wstring * pString = new std::wstring();
    pString->assign(L"");
    delete pString;

    return 0;
}

I have fixed it this way for this call. But I'm afraid that other functions in other circumstainces could fail so I'm looking for the reason and (probably) for a solution.

Nuisance.exe is a C# application with default stacksize callling a win32 dll about which I know nothing.


Solution

  • If Nuisance.exe was a C++ application, I would imagine that it calls NtOpenFile in a way similar to this, allocating one of pointer parameters on overwritten stack:

       POBJECT_ATTRIBUTES    MakeObjectAttributes()
       {
          POBJECT_ATTRIBUTES oa = {...};
          return &oa; // Pointer to stack variable - UB
       }
       ...
       NtOpenFile(..., MakeObjectAttributes(), ...)
    

    STATUS_INVALID_ACL (0xC0000077) error might suggest that SecurityDescriptor within OBJECT_ATTRIBUTES is allocated this way.

    Then it matters how much stack is used by AfterThis_NtOpenFile_WillFail, and it is more than AfterThis_NtOpenFile_WillWork, since std::wstring would be larger than just a couple of pointers due to small string optimization.

    If the call chain is always the same, the corruption may be deterministic.

    I don't know if code equivalent of returning address of temporary is possible in C#. But the DLL may be in C/C++ or similar language that allows dandling pointers.

    To prove/disprove the role of stack, try to allocate other data on stack that has std::wstring size. More precise proof could be checking passed pointer to see if they point to stack area that is about to be overwritten.