windowsterminalluaansi-escape

Enable ANSI sequences in windows terminal

I stumbled upon an interesting issue in Windows:


C:\> lua
> print("\x1b[95mMagenta\x1b[0m")
-[95mMagenta-[0m

But if I run os.execute() even with an empty command, before it, it works as expected:

C:\> lua
> os.system(""); print("\x1b[95mMagenta\x1b[0m")
Magenta

(the last line is printed with Magenta color)

terminal screenshot

Why it happens, and how can I make ANSI codes work without calling os.execute()?

Solution

  • For ANSI codes to work in a console, you should set specific ENABLE_VIRTUAL_TERMINAL_PROCESSING mode by invoking WinAPI function SetConsoleMode.

    When you invoke os.execute() in Lua, Lua invokes C runtime function system(), which creates cmd.exe process, which initializes all the bells and whistles.
    But Lua of course is unaware of Windows console features; it just works with a console having default settings.

    UPDATE:
    This is an example on how to turn ANSI escape sequences on from LuaJIT script:

    local ffi = require"ffi"
ffi.cdef[[
typedef int BOOL;
static const int INVALID_HANDLE_VALUE               = -1;
static const int STD_OUTPUT_HANDLE                  = -11;
static const int ENABLE_VIRTUAL_TERMINAL_PROCESSING = 4;
intptr_t GetStdHandle(int nStdHandle);
BOOL GetConsoleMode(intptr_t hConsoleHandle, int* lpMode);
BOOL SetConsoleMode(intptr_t hConsoleHandle, int dwMode);
]]
local console_handle = ffi.C.GetStdHandle(ffi.C.STD_OUTPUT_HANDLE)
assert(console_handle ~= ffi.C.INVALID_HANDLE_VALUE)
local prev_console_mode = ffi.new"int[1]"
assert(ffi.C.GetConsoleMode(console_handle, prev_console_mode) ~= 0, "This script must be run from a console application")
local function turn_VT(on_off)
   assert(ffi.C.SetConsoleMode(console_handle, bit.bor(prev_console_mode[0], on_off and ffi.C.ENABLE_VIRTUAL_TERMINAL_PROCESSING or 0)) ~= 0)
end

print('\x1b[95mMagenta\x1b[m')
turn_VT(true)
print('\x1b[95mMagenta\x1b[m')
turn_VT(false)
print('\x1b[95mMagenta\x1b[m')