Block ESC and Enter keys in modeless dialog box (Win32, non-MFC)

There're some articles written on this subject, but none of them worked in my case. I'm writing the following using Win32 (no MFC). The goal is to prevent ESC or ENTER keys from closing the modeless dialog box.

Here's the dialog template:

IDD_DIALOG_1 DIALOGEX 0, 0, 345, 179
STYLE DS_SETFONT | DS_FIXEDSYS | WS_MAXIMIZEBOX | WS_POPUP | WS_CAPTION | WS_SYSMENU | WS_THICKFRAME
CAPTION ""
FONT 8, "MS Shell Dlg", 400, 0, 0x1
BEGIN
    CONTROL         "New Pt",IDC_CHECK_NEW_PT,"Button",BS_AUTOCHECKBOX | BS_PUSHLIKE | WS_TABSTOP,7,3,39,12
    CONTROL         "Lines",IDC_CHECK_LINES,"Button",BS_AUTOCHECKBOX | BS_PUSHLIKE | WS_TABSTOP,54,3,39,12
    CONTROL         "Curves",IDC_CHECK_CURVES,"Button",BS_AUTOCHECKBOX | BS_PUSHLIKE | WS_TABSTOP,94,3,39,12
    CONTROL         "Ellipses",IDC_CHECK_ELLIPSE,"Button",BS_AUTOCHECKBOX | BS_PUSHLIKE | WS_TABSTOP,134,3,39,12
    CONTROL         "Circles",IDC_CHECK_CIRCLE,"Button",BS_AUTOCHECKBOX | BS_PUSHLIKE | WS_TABSTOP,174,3,39,12
    LTEXT           "Pen Size:",IDC_STATIC,242,7,30,8
    EDITTEXT        IDC_EDIT_PEN_SIZE,275,3,40,14,ES_CENTER | ES_AUTOHSCROLL | ES_NUMBER
    CONTROL         "",IDC_SPIN_PEN_SIZE,"msctls_updown32",UDS_SETBUDDYINT | UDS_ALIGNRIGHT | UDS_AUTOBUDDY | UDS_ARROWKEYS,316,3,11,14
    EDITTEXT        IDC_EDIT_SRC,7,19,331,106,ES_MULTILINE | ES_AUTOVSCROLL | ES_AUTOHSCROLL | ES_WANTRETURN | WS_VSCROLL | WS_HSCROLL
END

To trap those two keys, I change the message loop to this:

MSG msg;

// Main message loop:
for(int nR;;)
{
    nR = ::GetMessage(&msg, nullptr, 0, 0);
    if(!nR)
    {
        break;
    }
    else if(nR == -1)
    {
        //Error
        ASSERT(NULL);
        break;
    }

    if(ghActiveModelessDlg)
    {
        BOOL bProcessAsDlgMsg = TRUE;

        if(msg.message == WM_KEYDOWN ||
            msg.message == WM_KEYUP)
        {
            //Try to trap ESC & Enter keys
            if(msg.wParam == VK_ESCAPE)
            {
                //Do not process
                bProcessAsDlgMsg = FALSE;
            }
            else if(msg.wParam == VK_RETURN)
                goto lbl_check_enter;
        }
        else if(msg.message == WM_CHAR)
        {
            //Try to trap ESC & Enter key
            if(msg.wParam == 27)
            {
                //ESC - Do not process
                bProcessAsDlgMsg = FALSE;
            }
            else if(msg.wParam == '\r')
            {
lbl_check_enter:
                //See what window is it
                WCHAR buffClass[256];
                if(::GetClassName(msg.hwnd, buffClass, _countof(buffClass)) &&
                    lstrcmpi(buffClass, L"edit") == 0 &&
                    (::GetWindowLongPtr(msg.hwnd, GWL_STYLE) & ES_WANTRETURN))
                {
                    //This is edit ctrl that can handle its own Enter keystroke
                }
                else
                {
                    //Do not process
                    bProcessAsDlgMsg = FALSE;
                }
            }
        }

        if(bProcessAsDlgMsg)
        {
            if(::IsDialogMessage(ghActiveModelessDlg, &msg))
            {
                continue;
            }
        }
    }

    if (!TranslateAccelerator(msg.hwnd, hAccelTable, &msg))
    {
        TranslateMessage(&msg);
        DispatchMessage(&msg);
    }
}

And ghActiveModelessDlg is set from within DlgProc for the modeless dialog as such:

INT_PTR CALLBACK DlgProc(HWND hDlg, UINT message, WPARAM wParam, LPARAM lParam)
{
    switch(hDlg)
    {
        //...

        case WM_ACTIVATE:
        {
            //Needed to ensure that keyboard shortcuts are properly processed in the message loop
            ghActiveModelessDlg = wParam != WA_INACTIVE ? hDlg : NULL;
        }
        break;
    }

    return 0;
}

This works ... in most cases. Except this one.

Here's the sequence. Put the focus into the multi-line edit box, then hit any letter/number key and then ESC:

It will then close the dialog.

Any idea how can it pass my override code above?

PS. Interesting observations.

1) If I just hit ESC first, my code traps it. It's only when I hit some other key and then ESC it fails.

2) If I comment out the line that calls IsDialogMessage (and a subsequent continue) it stops accepting ESC. So my guess is that it's not the edit control that does this.

if we want let close dialog only by clicking close X button in system menu (or by ALT+F4) and disable close by ESC and ENTER key - all what we need - call DestroyWindow when process (WM_SYSCOMMAND, SC_CLOSE) and do nothing on (WM_COMMAND, IDCANCEL, IDOK). we not need special message loop or subcluss any controls. and not have buttons with IDOK/ IDCANCEL id in dialog

INT_PTR DialogProc(HWND hwndDlg, UINT uMsg, WPARAM wParam, LPARAM lParam)
{

    switch (uMsg)
    {
    case WM_SYSCOMMAND:
        if ((wParam & 0xfff0) == SC_CLOSE) DestroyWindow(hwndDlg);
        break;
    case WM_COMMAND:
        switch (wParam)
        {
        case MAKEWPARAM(IDOK, BN_CLICKED):
        case MAKEWPARAM(IDCANCEL, BN_CLICKED):
            // ignore this
            break;
        ....
        }
    }
    ....
}