ANSI In Terminal (Windows 10)

I am writing a console application with VS2022 in C++ and Windows API. After searching for a long time, I have seen that modern versions of Windows Terminal allow for ANSI color codes.

Windows Documentation says:

https://learn.microsoft.com/en-us/windows/console/writeconsole

Although an application can use WriteConsole in ANSI mode to write ANSI characters, consoles do not support "ANSI escape" or "virtual terminal" sequences unless enabled. See Console Virtual Terminal Sequences for more information and for operating system version applicability.

https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences

You can use GetConsoleMode and SetConsoleMode functions to configure this behavior. A sample of the suggested way to enable virtual terminal behaviors is included at the end of this document.

A shortened version of the sample program is:

HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
DWORD dwMode = 0;
GetConsoleMode(hOut, &dwMode)
dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
SetConsoleMode(hOut, dwMode)

I have a created a minimal example to show what is not working.

HANDLE hConsole = CreateConsoleScreenBuffer(
    GENERIC_READ | GENERIC_WRITE,
    0,
    nullptr,
    CONSOLE_TEXTMODE_BUFFER,
    nullptr
);
SetConsoleActiveScreenBuffer(hConsole);
DWORD dwMode = 0;
GetConsoleMode(hConsole, &dwMode);
dwMode |= ENABLE_PROCESSED_OUTPUT | ENABLE_VIRTUAL_TERMINAL_PROCESSING;
SetConsoleMode(hConsole, dwMode);
DWORD charsWritten;
WriteConsoleA(hConsole, "\x1b[31mHello\x1b[0m", 13, &charsWritten, nullptr);

Result: I can't post images but the result is several "missing character" Unicode characters.

It is possible for me to change the color using ANSI escape sequences BEFORE I create the console screen buffer using std::cout. Using std::cout AFTER I create the console screen buffer does nothing, which is not what I need.

I am aware of SetConsoleTextAttribute() and it works, but I would like to work with more than 16 colors now that VT100 Parser is available for the Windows terminals. I have also used both WriteConsoleOutput() and WriteConsoleOutputCharacter() in the past and I know those do not work with ANSI. WriteConsole() should be able to work with ANSI.

ENABLE_VIRTUAL_TERMINAL_PROCESSING works fine. Also check the document Example of SGR terminal sequences.

#include <stdio.h>
#include <wchar.h>
#include <windows.h>

int main()
{
    // Set output mode to handle virtual terminal sequences
    HANDLE hOut = GetStdHandle(STD_OUTPUT_HANDLE);
    if (hOut == INVALID_HANDLE_VALUE)
    {
        return GetLastError();
    }

    DWORD dwMode = 0;
    if (!GetConsoleMode(hOut, &dwMode))
    {
        return GetLastError();
    }

    dwMode |= ENABLE_VIRTUAL_TERMINAL_PROCESSING;
    if (!SetConsoleMode(hOut, dwMode))
    {
        return GetLastError();
    }

    // Try some Set Graphics Rendition (SGR) terminal escape sequences
    wprintf(L"\x1b[31mThis text has a red foreground using SGR.31.\r\n");
    wprintf(L"\x1b[1mThis text has a bright (bold) red foreground using SGR.1 to affect the previous color setting.\r\n");
    wprintf(L"\x1b[mThis text has returned to default colors using SGR.0 implicitly.\r\n");
    wprintf(L"\x1b[34;46mThis text shows the foreground and background change at the same time.\r\n");
    wprintf(L"\x1b[0mThis text has returned to default colors using SGR.0 explicitly.\r\n");
    wprintf(L"\x1b[31;32;33;34;35;36;101;102;103;104;105;106;107mThis text attempts to apply many colors in the same command. Note the colors are applied from left to right so only the right-most option of foreground cyan (SGR.36) and background bright white (SGR.107) is effective.\r\n");
    wprintf(L"\x1b[39mThis text has restored the foreground color only.\r\n");
    wprintf(L"\x1b[49mThis text has restored the background color only.\r\n");

    DWORD charsWritten;
    BOOL b = WriteConsole(hOut, L"\x1b[31mThis text has a red foreground using SGR.31.\r\n", 52, &charsWritten,nullptr);
    WriteConsoleA(hOut, "\x1b[31mHello\x1b[0m", 15, &charsWritten, nullptr);
    return 0;
}

Also tested your code.