How to configure the font used to render IME preview text in tkinter text input fields on Windows?

Problem Description

I'm experiencing a problem concerning font configuration in Tkinter. The constructor of all Tkinter widgets have a font keyword argument which allows you to configure the font used for displaying text. However, this option doesn't seem to affect the IME preview text used in text input widgets (such as Entry and Text).

For those whose native language doesn't require IMEs (input method editors), I'll give a short explanation: while the "spelling" of a word/phrase is being entered, these spelling characters aren't immediately entered into the text field being typed in. Instead, the IME of the OS intercepts these spelling characters and translates them into the best-matching candidates as you type. And when you're done, you press Enter and the completed phrase is sent to the application. While you're composing the phrase, it's the application's responsibility to render the "preview" text in a manner which is visually distinct from text that has already been submitted into the text field.

Other applications (both built into Windows and 3rd party) usually uses the same font family and same font size to render the IME preview text, with the only difference being a dotted underline and sometimes a different background color. However, the problem I'm experiencing is that Tkinter always renders the IME preview text using the "default" font regardless of the font that the widget is configured to use.

For example, here's a screenshot of composing Chinese text in Notepad:

And here's a screenshot of doing the same in the Tk application I wrote as an MRE:

As it can be seen, while Notepad uses the same font for both the actual text and the IME preview text, Tkinter uses the default font for the latter, which results in an ugly visual discrepancy (even the placement of the preview text is slightly off).

Text in Image

For those asking, this is the text in the image.

Submitted Text:

The quick brown fox jumps over the lazy dog.
敏捷的棕色狐狸
       ^                 ^
submitted text    IME preview text

IME preview text (start typing at the end of the second line):

跳過了懶惰的狗。

But please note that the problem is irrelevant to the specific text entered.

Minimal Reproducible Example

Here's the code for the MRE used in the screenshot to demonstrate the issue. The DPI configuration is necessary to ensure that one pixel in the window corresponds to one pixel on the physical screen. Otherwise Windows will automatically upscale the window on high DPI displays, causing the window to appear blurry.

#!/usr/bin/env python3
from contextlib import suppress
import tkinter as tk
import tkinter.font
import platform
import ctypes
size = (800, 400)


def configure_dpi() -> bool:
    """Tries to set DPI awareness and returns whether successful."""
    with suppress(AttributeError):  # For Windows 8.1 and later
        # set process DPI awareness to PROCESS_PER_MONITOR_DPI_AWARE (2).
        return (ctypes.windll.shcore.SetProcessDpiAwareness(2) == 0)
    with suppress(AttributeError):  # For Windows Vista and later.
        return (ctypes.windll.user32.SetProcessDPIAware() != 0)
    return True  # Windows version too low to support HiDPI.


def main() -> None:
    root = tk.Tk()
    root.title("Text Edit")
    root.geometry("%dx%d" % size)
    root.minsize(*size)
    label_font = tk.font.Font(
        name="label_font", family="微軟正黑體", size=12, weight=tk.font.NORMAL
    )
    input_font = tk.font.Font(
        name="input_font", family="微軟正黑體", size=16, weight=tk.font.NORMAL
    )
    top_bar = tk.Frame(root, pady=10)
    tk.Label(top_bar, text="Title:", font=label_font).pack(side=tk.LEFT)
    entry = tk.Entry(top_bar, font=input_font)
    entry.pack(side=tk.LEFT)
    top_bar.grid(row=0, column=0, columnspan=2)
    text = tk.Text(root, wrap=tk.WORD, font=input_font)
    scrollbar = tk.Scrollbar(root, command=text.yview)
    text.configure(yscrollcommand=scrollbar.set)
    text.grid(row=1, column=0, sticky=tk.NSEW)
    scrollbar.grid(row=1, column=1, sticky=tk.NS)
    root.rowconfigure(1, weight=1)
    root.columnconfigure(0, weight=1)
    root.mainloop()


if __name__ == "__main__":
    if platform.system() == "Windows":
        configure_dpi()
    main()

Run this script, and enter some non-IME text into either the single-line or multi-line text box, then compose some text after that using an IME. You should immediately notice that the IME preview text is rendered with a different font and size from the preceding text.

What I've Tried

Assuming that the IME preview text's font is probably set to one of the built-in font names (such as TkDefaultFont, TkFixedFont, TkCaptionFont, etc.) in Tkinter's registry (returned by a call to tkinter.font.names()) but not knowing which one, I tried setting the size of every built-in font name to 16⁠pt, thinking that this way I would be guaranteed to hit the font that the preview text is associated with. So I wrote the following function:
```
def resize_builtin_fonts(root: tk.Tk | tk.Widget, size: int) -> None:
    """Sets the size of all registered font names."""
    for font_name in tk.font.names(root):
        font_obj = tk.font.nametofont(font_name)
        font_obj.configure(size=size)
```
Then I added the function call resize_builtin_fonts(root, 16) after the code to set up the root window and before the call to root.mainloop(). To my surprise, while this (as expected) caused all widgets without an explicit font= keyword argument in the constructor to display text at 16⁠pts, it had no effect on the IME preview text. So apparently the IME preview text operates separately from Tkinter's font registry.
Next, I tried searching through the Tkinter documentation and John Shipman's Tkinter reference, wondering if there was a module-level function or perhaps a method of tkinter.Tk which is dedicated to setting the IME preview text's font. I couldn't find anything that looked like it, but I could have missed something of course.
Next, I tried searching on the internet, but I couldn't find much. The closest thing I found was this page on the Tcler's Wiki talking about how well Tk 8.3 supports Windows IME (emphasize mine):

Tk 8.3.4+ has support for Windows IME and Linux XIM. In 8.3, this uses the root-window style, but 8.4 has support for "over-the-spot" style IME and XIM. This support was possible with significant assistance from Koichi Yamamoto and Keiichi Takahashi. Koichi has confirmed that the 8.4 support is known to work on Win95 to WinXP using Windows IME, Japanese Win9*, and the ATOK13 IME variants.

So now I know the IME preview text probably uses the root window style. Assuming that the term means the ttk style for the root window, I looked up the ttk class name of the tkinter.Tk object using print(root.winfo_class()), and found that it is Tk. So I tried to add this to my code:
```
import tkinter.ttk as ttk
style = ttk.Style()
style.configure("Tk", font="input_font")
```
But it had no effect, so apparently there's no such option for the Tk style. Also, the Tk object doesn't even have a style configuration option, and after testing, the Tk object doesn't seem to obey any configurations made to the Tk style, so it seems that the tkinter.Tk object is exempt from ttk styling anyway.
Continuing to pursue the clue found on the Tcler's Wiki about the IME preview text following the root window style (assuming I interpreted it correctly, its phrasing was a little unclear), I guessed next that maybe by "root window style" it meant the configuration of the tkinter.Tk object, so I tried adding:
```
root.configure(font=input_font)
```
But this raises the exception tkinter.TclError with the message unknown option "-font", so apparently there's no such option.

So now I'm at a loss for how to change the font size associated with the "root window style", whatever that means. I've also tried skimming through the tk documentation to see if there's a relevant command I could call from Python using root.tk.eval("command"). But since I don't know any tcl (and have also never used the tk package directly), I found the tk documentation hard to understand. So hopefully someone here knows the solution.

I tried to test my MRE in a Linux virtual machine to see if the same problem exists there, but I simply could not get the ibus IME framework to work at all after installing it in Linux Mint (all keypresses aren't caught by the IME even when an input method is active), so I had to give up.

Solution

You can set the IME font using ImmSetCompositionFontW from the Windows API:

import ctypes
import platform
import tkinter as tk
from contextlib import suppress
from ctypes import c_byte, c_int, c_long, c_wchar
from tkinter import font as tkfont
from typing import Any

size = (800, 400)

# tkinter constants
IME_KEYCODE = 229

# WinGDI constants (from WinGDI.h)
LOGPIXELSY = c_int(90)
DEFAULT_CHARSET = c_byte(1)

# other constants
LF_FACESIZE = 32
POINTS_PER_INCH = 72

# mapping of tkinter font weight to lfWeight
FW_MAPPING = {
    tkfont.NORMAL: c_long(400),  # FW_NORMAL
    tkfont.BOLD: c_long(700),  # FW_BOLD
}


class LOGFONTW(ctypes.Structure):
    # https://learn.microsoft.com/en-us/windows/win32/api/wingdi/ns-wingdi-logfontw
    _fields_ = [
        ("lfHeight", c_long),
        ("lfWidth", c_long),
        ("lfEscapement", c_long),
        ("lfOrientation", c_long),
        ("lfWeight", c_long),
        ("lfItalic", c_byte),
        ("lfUnderline", c_byte),
        ("lfStrikeOut", c_byte),
        ("lfCharSet", c_byte),
        ("lfOutPrecision", c_byte),
        ("lfClipPrecision", c_byte),
        ("lfQuality", c_byte),
        ("lfPitchAndFamily", c_byte),
        ("lfFaceName", c_wchar * LF_FACESIZE)
    ]


def configure_dpi() -> bool:
    """Tries to set DPI awareness and returns whether successful."""
    with suppress(AttributeError):  # For Windows 8.1 and later
        # set process DPI awareness to PROCESS_PER_MONITOR_DPI_AWARE (2).
        return ctypes.windll.shcore.SetProcessDpiAwareness(2) == 0
    with suppress(AttributeError):  # For Windows Vista and later.
        return ctypes.windll.user32.SetProcessDPIAware() != 0
    return True  # Windows version too low to support HiDPI.


class App(tk.Tk):
    def __init__(self, *args: Any, **kwargs: Any):
        tk.Tk.__init__(self, *args, **kwargs)
        self.title("Text Edit")
        self.geometry("%dx%d" % size)
        self.minsize(*size)
        self.label_font = tkfont.Font(
            name="label_font", family="微軟正黑體", size=12, weight=tkfont.NORMAL
        )
        self.input_font = tkfont.Font(
            name="input_font", family="微軟正黑體", size=16, weight=tkfont.NORMAL
        )
        top_bar = tk.Frame(self, pady=10)
        tk.Label(top_bar, text="Title:", font=self.label_font).pack(side=tk.LEFT)
        entry = tk.Entry(top_bar, font=self.input_font)
        entry.pack(side=tk.LEFT)
        top_bar.grid(row=0, column=0, columnspan=2)
        text = tk.Text(self, wrap=tk.WORD, font=self.input_font)
        scrollbar = tk.Scrollbar(self, command=text.yview)
        text.configure(yscrollcommand=scrollbar.set)
        text.grid(row=1, column=0, sticky=tk.NSEW)
        scrollbar.grid(row=1, column=1, sticky=tk.NS)
        self.rowconfigure(1, weight=1)
        self.columnconfigure(0, weight=1)

        # all key events will trigger the IME check
        self.bind_all("<Key>", self.set_ime_font)
        self.ime_active = False

    def set_ime_font(self, event: "tk.Event[tk.Misc]"):
        if event.keycode == IME_KEYCODE and not self.ime_active:
            self.ime_active = True

            # the following has been adapted from https://qiita.com/maccadoo/items/5fed926d1c0672aeec83
            user32 = ctypes.WinDLL(name="user32")
            imm32 = ctypes.WinDLL(name="imm32")
            gdi32 = ctypes.WinDLL(name="gdi32")
            h_wnd = user32.GetForegroundWindow()
            # alternatively `h_wnd = int(self.frame(), base=16)`
            h_dc = user32.GetDC(h_wnd)
            h_imc = imm32.ImmGetContext(h_wnd)

            # convert tk font into LOGFONTW struct
            font = tkfont.nametofont(event.widget.cget("font"))
            lplf = LOGFONTW()
            # convert points to lfHeight (formula from LOGFONTW docs)
            size = font.cget("size")
            lplf.lfHeight = c_long(
                -round(
                    size * gdi32.GetDeviceCaps(h_dc, LOGPIXELSY)
                    / POINTS_PER_INCH
                ) if size > 0 else size
            )
            # if size < 0, then the unit is already in pixels
            lplf.lfWidth = c_long(0)
            lplf.lfEscapement = c_long(0)
            lplf.lfOrientation = c_long(0)
            lplf.lfWeight = FW_MAPPING[font.cget("weight")]
            lplf.lfItalic = c_byte(int(font.cget("slant") == "italic"))
            lplf.lfUnderline = c_byte(font.cget("underline"))
            lplf.lfStrikeOut = c_byte(font.cget("overstrike"))
            lplf.lfCharSet = DEFAULT_CHARSET
            lplf.lfOutPrecision = c_byte(0)
            lplf.lfClipPrecision = c_byte(0)
            lplf.lfQuality = c_byte(0)
            lplf.lfPitchAndFamily = c_byte(0)
            lplf.lfFaceName = font.cget("family")

            # set the IME font (https://learn.microsoft.com/en-us/windows/win32/api/imm/nf-imm-immsetcompositionfontw)
            imm32.ImmSetCompositionFontW(h_imc, lplf)

        # if we receive an event with a char after receiving IME keycodes then the input has been submitted
        if event.char and self.ime_active:
            self.ime_active = False


if __name__ == "__main__":
    if platform.system() == "Windows":
        configure_dpi()
    app = App()
    app.mainloop()

To make the code easier to work with, I've renamed some imports and moved it into a class but this isn't necessary.

From testing, I found that the entering characters into the IME caused a <Key> event with keycode 229 and submitting the IME causes events with a char attribute. These can be used to decide when to set the font.

To set the font, you first load the necessary DLLs and get the necessary handles (this was adapted from here). You can then use the <Key> event to get the widget font. This can then be converted into a LOGFONTW struct which can then be passed to ImmSetCompositionFontW.

During my testing, I couldn't get the IME font to match the Tkinter font completely but that can probably be fixed by tinkering with the LOGFONTW struct fields.

As this uses the Windows API, it won't work on Linux. I also couldn't find any built-in Tkinter method of doing this.