What is the best way to show a DialogBox in a seperate process independent of the main process?

I have a C++ library which is used for auth purposes. It has some exported methods that are called via customer software. My goal is to create a function which would display a DialogBox in a seperate process independent of the application.

My purpose to do that, is when some tamper is detected, I want to terminate the app immediately, but leave the user with some kind of notification. I also have a feature like a remote session terminate, when the app owner can terminate the client session. The same goes with the case when a license expires. I don't want to just terminate the app, I want to leave the user with a message.

I read about some solutions, but I see problems in each of them:

Create a seperate app which spawns the DialogBox, and run it as a new process.

I don't want to ship aditional exe files, it has to be a single native library.

Use Powershell and ShellExecute.

I'm worried that Powershell may be blocked on some computers or need administator privileges.

I have that code which works:

Add-Type @"
using System;
using System.Runtime.InteropServices;
public class NativeMethods {
    [DllImport("user32.dll", CharSet=CharSet.Unicode)]
    public static extern int MessageBox(IntPtr hWnd, string text, string caption, int options);
}
"@

[NativeMethods]::MessageBox([IntPtr]::Zero, "Wykryto naruszenie", "Ał", (0x10 -bor 0x1000))

Use VBS along with Windows Script Host.

On some computers, Windows Script Host may be blocked.

What solution should I use to ensure compatibility? Or maybe there are better solutions than these?

Solution

From a C++ DLL, I would export a function to display a dialog box (you can use the Win32 API MessageBox for this) and execute it by calling rundll32.exe with the appropriate parameters. This utility is normally always present and should never be disabled. You can try this command to verify that it is usable:

Rundll32.exe shell32.dll,ShellAbout

Your function will look like this (feel free to adapt it to your needs):

extern "C" __declspec(dllexport) void CALLBACK ShowMessage(HWND hwnd, HINSTANCE hinst, LPSTR lpszCmdLine, int nCmdShow) {
    MessageBox(
        nullptr,                             // No parent window
        "This is a message box!",
        "Dialog from DLL",
        MB_OK | MB_ICONINFORMATION
    );
}

Crucial parts here are the CALLBACK calling convention and the "C" extern linkage. Do NOT forget them. Also, the input parameters are required exactly as shown. Do NOT change them.

Then, from anywhere in the code, you can call this function to spawn a new process displaying a message box:

void ExecuteRundll32(const std::string& functionName) {
    char dllPath[MAX_PATH] = {0};
    
    // Get the full path of the current DLL
    if (!GetModuleFileNameA((HINSTANCE)&__ImageBase, dllPath, MAX_PATH)) {
        std::cerr << "Failed to get DLL path. Error: " << GetLastError() << std::endl;
        return;
    }

    // Construct the rundll32 command
    std::string command = "rundll32.exe \"";
    command += dllPath;
    command += "\",";
    command += functionName;

    // Execute the command using CreateProcessA
    STARTUPINFOA si = { sizeof(STARTUPINFOA) };
    PROCESS_INFORMATION pi = { 0 };

    if (!CreateProcessA(nullptr, command.data(), nullptr, nullptr, FALSE, 0, nullptr, nullptr, &si, &pi)) {
        std::cerr << "Failed to execute rundll32. Error: " << GetLastError() << std::endl;
    } else {
        // Close unused handles to prevent resource leaks
        CloseHandle(pi.hProcess);
        CloseHandle(pi.hThread);
    }
}

You call it with a suitable function provided:

ExecuteRundll32("ShowMessage");

This should do the trick.