How to get rid of the Close Button of a Console Window?

I have a customer who is using some old but still needed 32Bit-Software, which is running in a Console Window. It is necessary to disable the Close button because closing the Console using this button causes some serious problems in this software.

I thought about the following way:

1) Find the handle of the active Console
2) Disable the Close Button with GetSystemMenu function

Maybe I'm completely wrong, but I did not manage to find a way to do that so far.

Edit:

The problem is just the Close Button. Of course users can also quit the program by Alt+F4 or Task Manager, but they don't do that. They do use Close Button, that's why I want to disable it.

Of course the best solution would by to disable all ways to cancel the program, but to disable the Close Button would work.

To start the program inside a Windows Form would by one possible solution, too.

Solution

To interact with a foreign Window, you need to find it/verify it exists first.

We have different methods to find a Window. Here I'm considering FindWindowEx and Process.GetProcessesByName(). UI Automation and EnumWindows provide other options, eventually.

Store the CMD Window caption somewhere, e.g., an instance Field (it could be a Project settings or anything else you can access at run-time).

Private cmdWindowTitle As String = "The Window Title"

→ FindWindowEx is more useful if you know exactly what the Window title is and it doesn't change over time.
→ Process.GetProcessesByName() can be used to find a Window using the Process name and then verify whether the Process.MainWindowTitle.Contains() at least a partial known string.

If instead the Console Window belongs to the current Process, you just need:

Process.GetCurrentProcess().MainWindowHandle

' -- If the Console Window belongs to the current Process: --
Dim cmdWindowHandle = Process.GetCurrentProcess().MainWindowHandle
' -----------------------------------------------------------

' -- Find it when the exact Window title is known: --
Dim cmdWindowHandle As IntPtr = NativeMethods.GetCmdWindowByCaption(cmdWindowTitle)
' -----------------------------------------------------------

' -- Find it when only a partial caption is available: --
Dim cmdWindowHandle As IntPtr = IntPtr.Zero
Dim cmdProc = Process.GetProcessesByName("cmd").
    FirstOrDefault(Function(p) p.MainWindowTitle.Contains(cmdWindowTitle))
If cmdProc IsNot Nothing Then
    cmdWindowHandle = cmdProc.MainWindowHandle
End If
' -----------------------------------------------------------

' Choose one of the above, then, in any case:
If cmdWindowHanle <> IntPtr.Zero Then
    NativeMethods.WindowDisableSysMenu(cmdWindowHandle)
End If

Note: Here, I'm assuming the Process Name is cmd and the Window class name is ConsoleWindowClass. It may not be. Change these as required.

Since now the Window has no SystemMenu or Close buttons (we just hid them all), it cannot be closed using ALT+F4 or any other means except using the Task Manager (or wait for it to close naturally).
To close it from your app, send a WM_CLOSE message:

' -- find the Window as described before --
Dim cmdWindowHandle As IntPtr = NativeMethods.GetCmdWindowByCaption(cmdWindowTitle)
If Not cmdWindowHandle.Equals(IntPtr.Zero) Then
    NativeMethods.SendCloseMessage(cmdWindowHandle)
End If

NativeMethods declarations:

Public Class NativeMethods

    Private Const WM_CLOSE As Integer = &H10

    Public Enum WinStyles As UInteger
        WS_MAXIMIZE = &H1000000
        WS_MAXIMIZEBOX = &H10000
        WS_MINIMIZE = &H20000000
        WS_MINIMIZEBOX = &H20000
        WS_SYSMENU = &H80000
    End Enum

    Public Enum GWL_Flags As Integer
        GWL_STYLE = -16
        GWL_EXSTYLE = -20
    End Enum

    <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
    Private Shared Function SendMessage(hWnd As IntPtr, uMsg As WinMessage, wParam As Integer, lParam As Integer) As Integer
    End Function

    <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
    Private Shared Function FindWindowEx(hwndParent As IntPtr, hwndChildAfter As IntPtr, lpszClass As String, lpszWindow As String) As IntPtr
    End Function
    
    <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
    Private Shared Function GetWindowLong(hWnd As IntPtr, nIndex As GWL_Flags) As IntPtr
    End Function

    <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
    Private Shared Function SetWindowLong(hWnd As IntPtr, nIndex As GWL_Flags, dwNewLong As IntPtr) As IntPtr
    End Function

    <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
    Private Shared Function GetWindowLongPtr(hWnd As IntPtr, nIndex As GWL_Flags) As IntPtr
    End Function

    <DllImport("user32.dll", SetLastError:=True, CharSet:=CharSet.Auto)>
    Private Shared Function SetWindowLongPtr(hWnd As IntPtr, nIndex As GWL_Flags, dwNewLong As IntPtr) As IntPtr
    End Function

    ' Public wrappers
    Public Shared Function GetWindowLongUni(hWnd As IntPtr, nIndex As GWL_Flags) As Integer
        If IntPtr.Size = 8 Then
            Return GetWindowLongPtr(hWnd, nIndex).ToInt32()
        Else
            Return GetWindowLong(hWnd, nIndex).ToInt32()
        End If
    End Function

    Public Shared Function SetWindowLongUni(hWnd As IntPtr, nIndex As GWL_Flags, dwNewLong As Integer) As Integer
        If IntPtr.Size = 8 Then
            Return SetWindowLongPtr(hWnd, nIndex, New IntPtr(dwNewLong)).ToInt32()
        Else
            Return SetWindowLong(hWnd, nIndex, New IntPtr(dwNewLong)).ToInt32()
        End If
    End Function

    Public Shared Function GetCmdWindowByCaption(cmdCaption As String) As IntPtr
        Return FindWindowEx(IntPtr.Zero, IntPtr.Zero, "ConsoleWindowClass", cmdCaption)
    End Function

    Public Shared Sub WindowDisableSysMenu(windowHandle As IntPtr)
        Dim styles As Integer = GetWindowLongUni(windowHandle, GWL_Flags.GWL_STYLE)
        styles = styles And Not CInt(WinStyles.WS_SYSMENU)
        SetWindowLongUni(windowHandle, GWL_Flags.GWL_STYLE, styles)
    End Sub

    Public Shared Sub SendCloseMessage(windowHandle As IntPtr)
        SendMessage(windowHandle, WM_CLOSE, 0, 0)
    End Sub
End Class