How can I set the background image in Xorg with xcb?

Note: I am working on a window manager similar to Awesome WM, whereby it will be written in C, and it will expose a lua API for customization.

The problem I am encountering is with setting the background of the root window. This is the lua code I am using:

local function set_wallpaper_from_path(path)

    local bg_img = lgi.cairo.ImageSurface.create_from_png(path)

    local h = bg_img:get_height()
    local scale_ratio = _G.screen_height / h

    local s = tsoil.create(_G.screen_width, _G.screen_height)

    s.cr:set_operator(lgi.cairo.Operator.SOURCE)
    s.cr:scale(scale_ratio, scale_ratio)
    s.cr:set_source_surface(bg_img)
    s.cr:paint()
    lgi.cairo.Surface.flush(s.surface)

    root.set_wallpaper_by_pixmap_id(s.pixmap_id) -- exposed C function

    print(s.cr.status) -- prints "SUCCESS"

    -- no need to keep this around after setting the wallpaper
    tsoil.destroy(s)
end

And this is the C code that is being called:

int luaS_root_set_wallpaper_by_pixmap_id(lua_State *L)
{
    xcb_pixmap_t pixmap_id = luaL_checkinteger(L, 1);

    // make the server set the pixmap by this id as the root window's 
    // background pixmap
    xcb_change_window_attributes(
        ss.connection,
        ss.screen->root,
        XCB_CW_BACK_PIXMAP, // value mask
        &pixmap_id // value
    );

    // make the drawing actually show up
    // the equivalent of cairo's `cr:reset_clip(); cr:paint()`
    xcb_clear_area(
        ss.connection,
        0, // exposures
        ss.screen->root,
        // draw the whole root window
        0, 0, 0, 0 // x, y, width, height
    );

    return 0;
}

The result that I get is that the background is actually set properly and the image shows up when I'm using Xephyr to run my window manager.

However, when I try to run my window manager directly with startx, the background does NOT show up. I just get a black screen. But when I try to close the window manager, the image DOES show up for a tiny fraction of a second.

What could be the cause of this? I would greatly appreciate help.

EDIT: If I start the window manager with startx, but without starting a compositor, the background image DOES show up, which suggests to me that there's some communication issues between Xorg, my window manager, and picom.

Solution

What could be the cause of this? I would greatly appreciate help.

Before reading your edit, I would guess: You are using a composite manager and not making it possible to read the wallpaper, so it assumes there is none. Your edit confirms that theory.

Something like urxvt -tr -tint red -sh 40 (command copied from man 7 urxvt) will most likely also not work properly, since this is another case that requires being able to read the background.

Note that all of the text below is from memory and from looking at AwesomeWM's source code. I don't remember any reference that I can point you to and I guess I learnt this from looking at other source code. I don't know which source code.

How do I query the background image of the root window?

Like almost everything in X11, there is just a convention for this. The protocol itself does not allow to query the background-pixmap of any window, so a work-around was invented:

$ xprop -root | grep PMAP  
ESETROOT_PMAP_ID(PIXMAP): pixmap id # 0xe00001
_XROOTPMAP_ID(PIXMAP): pixmap id # 0xe00001

These two properties contain the ID of a pixmap that is set as the background.

Why are there two properties? I don't know. I guess these are just conventions that come from different tools.

How do I set the background image so that other programs can read until my WM restarts?

When setting the background image, as you already do, additionally set these two properties to the ID of the pixmap. Instead of then destroying the pixmap, you need to leak it / keep it around permanently.

Why doesn't this work across restarts of my WM?

When your WM closes its X11 connection, the X11 server does what is described in Chapter 10. Connection Close of the X11 protocol reference manual. The relevant part here is (emphasis mine; original emphasis removed):

If close-down mode (see SetCloseDownMode request) is RetainPermanent or RetainTemporary, then all resources (including colormap entries) allocated by the client are marked as permanent or temporary, respectively (but this does not prevent other clients from explicitly destroying them). If the mode is Destroy, all of the client's resources are destroyed.

So, the property on the root window stays, but it now points to a pixmap that no longer exists. Hopefully, all applications querying the pixmap are prepared for this case since they will now get an X11 error (this is a hint to you for when you want to write code that queries the background).

So, what do I do to set the background even across a restart?

That's why I quoted so much text above: We have to set the close-down mode on our X11 connection so that the pixmap is not actually freed.

Since the close-down mode is global, AwesomeWM actually opens a separate X11 connection just for creating the pixmap so that only this pixmap is kept alive permanently.

How do I prevent resource leaks?

Obviously, people want to change their background more than once. As explained above, this means that each time a pixmap is leaked that has the size of the root window. This might eventually end up using quite some memory.

Thus, when setting a new wallpaper, one should get rid of the old one with a KillClient request.

How is all of this implemented in AwesomeWM?

I'm glad you ask. :-)

"All the magic" starts in root_set_wallpaper. This function starts by opening a second X11 connection for allocating the pixmap. Then it actually creates the pixmap (code slightly edited):

xcb_pixmap_t p = xcb_generate_id(c);
xcb_create_pixmap(c, screen->root_depth, p, screen->root, width, height);
xcb_aux_sync(c);

Why the call to xcb_aux_sync(c)? Because the pixmap will actually be drawn to using the other, main X11 connection. For this to work, we have to avoid race conditions: The pixmap really, really, really has to be already created. Syncing with the X11 server makes sure of that.

Why is the drawing done with the main X11 connection? Because AwesomeWM uses cairo and cairo does not know that the two separate X11 connection actually refer to the same X11 server. Thus, if we just let cairo use the new connection, it might need to download (GetImage) pixel data that is already available on the X11 server and then upload it again (PutImage) on the new connection.

The actual drawing with cairo then happens next. Nothing special here. (Except for another call to xcb_aux_sync() to make sure the drawing is finished since we will be using the other X11 connection again next.)

Now that everything is prepared, we can actually set the wallpaper. To ensure no races with something else trying to set the wallpaper at the same time, this first does a GrabServer request to block other clients. This then calls the helper function root_set_wallpaper_pixmap() which I will be looking at below. This function does the actual "setting the wallpaper. Afterwards, an UngrabServer request allows other clients again.

Since AwesomeWM listens for wallpaper changes to update its internal state, this temporary un-sets the EventMask on the root window so that we do not get an event that makes us think the wallpaper changed.

Afterwards, the close down mode is set to RetainPermanent. I explained why above.

And that's it. The wallpaper is set. However, the SetCloseDownMode request is still in XCB's output buffer and not handled by the X11 server yet. To make sure it actually is applied, another xcb_aux_sync() happens (don't ask how long it took to find this bug). Finally, the "helper connection" is xcb_disconnect()ed.

Okay, but how is the wallpaper actually set (`root_set_wallpaper_pixmap()`)

This function does four things:

It uses ChangeWindowAttributes to set the BackPixmap to our pixmap
It uses ClearArea to force a redraw
It uses GetProperty to get the old wallpaper. The result if this is passed to a KillClient request (if the property was present before)
It sets the _XROOTPMAP_ID and ESETROOT_PMAP_ID properties on the root window

How do I read the wallpaper?

You didn't ask for this, but the function root_update_wallpaper() comes next in the source. This is called whenever the relevant property changes.

It queries the _XROOTPMAP_ID property of the root window and uses a GetGeometry request to check that this really is a valid drawable. The result is also checked for the correct depth, since we will have to assume a visual for this and the root visual is the only correct answer. But since this is just an arbitrary property, it could point to anything.

The result is then just given to cairo via cairo_xcb_surface_create(). We now have a cairo surface that (hopefully) refers to the wallpaper and can be used where needed.

(Yes, AwesomeWM implements pseudo-transparency by drawing the wallpaper into its windows and then drawing the actual content on top of that, with transparency. Yes, using an ARGB visual and requiring a compositing manager is much more convenient, but not everyone does that and sometimes even for good reasons.)