SendInput in a Windows Server

I would like to simulate mouse moves in a Windows Server (desktop experience). I'm using the Window's function SendInput() from C#, and the following code is running in the server.

using Windows.Win32;
using Windows.Win32.UI.Input.KeyboardAndMouse;

for (int i = 0; i < 5; i += 1)
{
    var inputs = new INPUT[]
    {
        new()
        {
            type = INPUT_TYPE.INPUT_MOUSE,
            Anonymous = new INPUT._Anonymous_e__Union
            {
                mi = new MOUSEINPUT
                {
                    dx = 0,
                    dy = 0,
                    mouseData = 0,
                    dwFlags = MOUSE_EVENT_FLAGS.MOUSEEVENTF_ABSOLUTE | MOUSE_EVENT_FLAGS.MOUSEEVENTF_MOVE,
                    time = 0,
                    dwExtraInfo = 0,
                },
            },
        }
    };

    if (PInvoke.SendInput(inputs, Marshal.SizeOf<INPUT>()) == 0)
    {
        throw new Win32Exception(Marshal.GetLastWin32Error());
    }

    Thread.Sleep(2000);
}

It is working perfectly while I keep the remote desktop open, but SendInput() returns 0x5 (ERROR_ACCESS_DENIED) when I close the session.

I have tried the same in a virtual machine. Whenever I close the virtual machine connection window, SendInput() fails.

Any idea how to use SendInput() in a Windows Server without keeping a connection open to it?

I haven't found a way to make that work using SendInput(). However, mouse clicks are working great using SendMessage():

PInvoke.SendMessage(hwnd, PInvoke.WM_LBUTTONDOWN, (nuint)MODIFIERKEYS_FLAGS.MK_LBUTTON, PInvoke.MAKELPARAM(X, Y));
PInvoke.SendMessage(hwnd, PInvoke.WM_LBUTTONUP, (nuint)MODIFIERKEYS_FLAGS.MK_LBUTTON, PInvoke.MAKELPARAM(X, Y));

The cool thing about SendMessage is that the clicks can be sent to minimized windows. However, this is not true for keyboard messages, they are only effective if the window is focused.