So I've done some Google searching and this is something that has very little knowledge out there. What would be an effective and foolproof way of detecting whether X11 or Wayland is in use, preferrably at compile-time and with CMake? I need to apply this to a C++ project of mine.
I assume you want to evaluate the display server during compile time, when calling CMake, instead of for every compilation. That's how CMake works and hot it should be used. One downside is, that you have to re-run CMake for every changed display server.
There is currently no default way to detect the running display server. Similar, there is no default code snippet to evaluate the display server by CMake. Just pick one way of detecting the display server that manually works for you or your environment respectively.
Call this code from CMake and store the result in a variable and use it for your C++ code.
For example loginctl show-session $(loginctl | grep $(whoami) | awk '{print $1}') -p Type works for me. The resulting CMake check is
execute_process(
"loginctl show-session $(loginctl | grep $(whoami) | awk '{print $1}') -p Type"
OUTPUT_VARIABLE result_display_server)
if ("${resulting_display_server}" EQUALS "Type=x11")
set(display_server_x11 TRUE)
else()
set(display_server_x11 FALSE)
endif()
Probably you have to fiddle around with the condition and check for Type=wayland or similar to get it properly working in your environment.
You can use display_server_x11 and write it into a config.h file to use it within C++ code.