How can you use vt100 escape codes in java on windows

vt100 escape codes are a powerful and popular method for formatting outputs (color, positioning, blinking, underlining etc.) on virtual terminals like xterm or konsole. See https://en.wikipedia.org/wiki/ANSI_escape_code

When you run a java program in xterm, konsole etc. and System.out.print such codes - for example

// print "Hello, World!" in red
System.out.print("\u001b[31mHello, World!");

java passes them to the terminal unaltered, so these terminals interpret the codes then and behave as you expect them to.

Java programs are supposed to be platform independent, though, but when you run such a program from windows command prompt, you will see the codes printed out, like this:

☐[31mHello, World!

After searching for a way to solve this problem, I could not find a simple solution or even a discussion about the topic. Most solutions depend on running your program in a different console application or including a java-based custom console into your program.

So - how can you use vt100 formatted text in Windows default command prompt easily?

Solution

I don't know if this works on older versions of windows (some feedback would be welcome - I have tested it on windows 10), but I have taken the code from the bottom of https://learn.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences and translated it to Java.

This uses some calls into kernel32.dll, so I had to incorporate Java Native Access (JNA). To run this, you need to download jna-*version*.jar and jna-platform-*version*.jar from https://github.com/java-native-access/jna#download and include them in your project.

Then, add these imports:

import com.sun.jna.*;
import com.sun.jna.platform.win32.WinDef.*;
import com.sun.jna.platform.win32.WinNT.HANDLE;

And at the beginning of your program, you can enable vt100 by running this code:

if(System.getProperty("os.name").startsWith("Windows"))
{
    // Set output mode to handle virtual terminal sequences
    Function GetStdHandleFunc = Function.getFunction("kernel32", "GetStdHandle");
    DWORD STD_OUTPUT_HANDLE = new DWORD(-11);
    HANDLE hOut = (HANDLE)GetStdHandleFunc.invoke(HANDLE.class, new Object[]{STD_OUTPUT_HANDLE});

    DWORDByReference p_dwMode = new DWORDByReference(new DWORD(0));
    Function GetConsoleModeFunc = Function.getFunction("kernel32", "GetConsoleMode");
    GetConsoleModeFunc.invoke(BOOL.class, new Object[]{hOut, p_dwMode});

    int ENABLE_VIRTUAL_TERMINAL_PROCESSING = 4;
    DWORD dwMode = p_dwMode.getValue();
    dwMode.setValue(dwMode.intValue() | ENABLE_VIRTUAL_TERMINAL_PROCESSING);
    Function SetConsoleModeFunc = Function.getFunction("kernel32", "SetConsoleMode");
    SetConsoleModeFunc.invoke(BOOL.class, new Object[]{hOut, dwMode});
}

From then on

System.out.print("\u001b[31mHello, World!");

will print "Hello, World!" in red, as expected.