c++mfcvisual-studio-2022

MFC Application exit code always 20


Taking a default MFC based single dialog appplication, I'm trying to prevent the default behaviour where the Escape or Enter key will cause the dialog to close.

However whilst I'm able to acheive this, I've discovered that the application exit code is now always returns 20, regardless of the value I pass to EndDialog.

In the below test I've prevented the Escape key:

BEGIN_MESSAGE_MAP(CMFCApplication10Dlg, CDialogEx)
    ON_WM_SYSCOMMAND()
    ON_WM_PAINT()
    ON_WM_QUERYDRAGICON()
    ON_BN_CLICKED(IDCANCEL, &CMFCApplication10Dlg::OnBnClickedCancel) // Triggers when Escape key pressed
    ON_WM_CLOSE() // Added WM_CLOSE to handle the close
END_MESSAGE_MAP()

OnBnClickedCancel occurs when pressing Escape, so to prevent the application closing I do not call OnCancel

void CMFCApplication10Dlg::OnBnClickedCancel()
{
    //CDialogEx::OnCancel(); // Not calling OnCancel
}

I've added the OnClose to allow the dialog to still be closed using the top right Close button

void CMFCApplication10Dlg::OnClose()
{
    CDialogEx::EndDialog(0); // Call the EndDialog with 0
}

How can I disable the escape key and also return an exit code of 0

Expecting: An exit code of 0

Actual: Getting an exit code of 20


Solution

  • When you create an MFC "Dialog Application" you usually want to disable the Enter and Esc key processing, as this is the main application window, while still being able to close the application from the system menu or the close button on the titlebar.

    For the [Enter] key you can override the dialog's OnOK() member (the default implementation closes the dialog):

    void CMyDlg::OnOK()
    {
        // CDialogEx::OnOK(); Don't Close
    }
    

    For the [Esc] key there are various techniques. One is override the OnCancel() member:

    void CMyDlg::OnCancel()
    {
        // Disable Closing on Esc press
        if ((GetKeyState(VK_ESCAPE) & 0x8000) == 0)
            CDialogEx::OnCancel(); // Close only if NOT caused by [Esc]
    }
    

    Or override OnCancel() and add a few lines to the Wizard-generated OnSysCommand() member too:

    void CMyDlg::OnCancel()
    {
        //CDialogEx::OnCancel(); Don't Close
    }
    
    void CMyDlg::OnSysCommand(UINT nID, LPARAM lParam)
    {
        if ((nID & 0xFFF0) == IDM_ABOUTBOX)
        {
            CAboutDlg dlgAbout;
            dlgAbout.DoModal();
        }
        else if (nID == SC_CLOSE) // Add these 2 lines
            CDialogEx::OnCancel();
        else CDialogEx::OnSysCommand(nID, lParam);
    }
    

    Other developers override the PreTranslateMessage() member.

    As for the exit-code, I think it makes no sense at all, as I noted in the comments. The application will be returning 0 if you click the [X] button, but what else should it be returning? And under what conditions?

    Anyway, CWinApp derives from CWinThread, you can try overriding Run(), and in the implementation call the parent-class Run() (CWinApp::Run() or CWinAppEx::Run()), discard its return value and return always 0. The wizard-generated override of InitInstance() returns FALSE for a dialog app, which means initialization failure, and may be this is why you are getting a non-zero exit-code.

    A side-note, if your project is new and all you wanted is an MFC application displaying a "Main Form", like in VB or C#, you can try using not a "Dialog Application" template, but instead an SDI application without document/view support (uncheck the option). It still contains a view object though. The view-type can be CFormView, which is a dialog-resource-based view, and the application behaves better, avoids problems like those in your app, supports features like the ON_UPDATE_COMMAND_UI handlers and keyboard accelerators, and offers goodies like toolbars, status-bar and dockable panes, all generated by the Wizard. That is, it's a proper application, not a dialog-box repurposed as an application window.


    EDIT:

    Checked it in a test-application, and found that the Run() override trick doesn't work, because Run() isn't called if InitInstance() returns FALSE, which is the case for a Wizard-generated "MFC Dialog Application". However, you can override ExitInstance(), which is always called before termination:

    int CMyApp::ExitInstance()
    {
        CWinAppEx::ExitInstance(); // Discard the return value
        return 0;
    }
    

    The return value was 2 btw (ERROR_FILE_NOT_FOUND), not 20. It seems MFC returns this pseudo-error if InitInstance() returns FALSE.