[SOLVED] Use delay load to load correct library

Use delay load to load correct library - return value type error

I used msdn documentation here: http://msdn.microsoft.com/en-us/library/f0fkfy9y(v=vs.100).aspx

It specifies there that:

case dliNotePreLoadLibrary :
//If you want to return control to the helper, return 0.
//Otherwise, return your own HMODULE to be used by the 
//instead of having it call LoadLibrary itself

So I tried the following:

FARPROC WINAPI delayHook(unsigned dliNotify, PDelayLoadInfo pdli)
{
    switch (dliNotify) {
        case dliStartProcessing :
        case dliNotePreGetProcAddress :
        case dliFailLoadLib :
        case dliFailGetProc :
        case dliNoteEndProcessing : 
          return 0;
          break;
        case dliNotePreLoadLibrary :
            {
                char* dllPath = getDllPath();
                HMODULE lib = LoadLibrary(dllPath);
                return lib;
            }
            break;      
        default :
            return NULL;
    }
    return NULL;
}

I get error for returning the HMODULE:

'return' : cannot convert from 'HMODULE' to 'FARPROC'.

What is the problem? Am I doing something worng? they do say return your own HMODULE, and that is what I did...

Solution

LoadLibrary() returns an HMODULE. Your delayHook() function returns a FARPROC. So there is a mismatch in return value. Assuming that you really want to return HMODULE returned by LoadLibrary(), you may want to use a cast like reinterpret_cast:

case dliNotePreLoadLibrary :
{
    char* dllPath = getDllPath();
    HMODULE lib = LoadLibrary(dllPath);
    return reinterpret_cast<FARPROC>(lib);
}

Side note: in today's world, you should use Unicode instead of ANSI/MBCS, so your dllPath should better be a Unicode string (e.g. const wchar_t*, or CString, or std::wstring).