macoscocoaaccessibility

How to create an AXUIElementRef from an NSView or NSWindow?


Concerning the macOS Accessibility API, is there anyway to create an AXUIElementRef that corresponds to either an NSView or an NSWindow?

There appears to have been a way of doing this back in the days of Carbon using AXUIElementCreateWithHIObjectAndIdentifier but that function isn't available anymore.

The only method I'm aware of is to use the Accessibility API to recursively search the entire hierarchy of UI elements of your application looking for one that matches the NSView or NSWindow. But in addition to being an onerous solution, it's not even guaranteed to succeed as there might not be a way to positively correspond an AXUIElementRef and a Cocoa object by just using the available attributes of the AXUIElementRef.

I am willing to consider undocumented APIs that help accomplish this.


Solution

  • Here we are, just over eight years later and I think I've finally found a solution. The trick is to use this undocumented function that luckily stills works in modern macOS:

    extern "C" AXError _AXUIElementGetWindow(AXUIElementRef, CGWindowID *out);

    This way we can use the Accessibility API to search through the list of windows of an application and compare their window IDs with the ID of an NSWindow to find a match.

    Here's a function that demonstrates how to do this:

    AXUIElementRef copyWindowWithPIDAndWindowID(pid_t pid, CGWindowID targetWindowId)
    {
        AXUIElementRef result = nullptr;
        AXUIElementRef app = AXUIElementCreateApplication(pid);
        
        if (!app) {
            return nullptr;
        }
        
        CFArrayRef windows;
        AXError error = AXUIElementCopyAttributeValue(app, kAXWindowsAttribute, (const void **)&windows);
        
        if (error != kAXErrorSuccess || !windows) {
            CFRelease(app);
            return nullptr;
        }
        
        for(long i = 0; i < CFArrayGetCount(windows); ++i) {
            CGWindowID windowId;
            AXUIElementRef window = (AXUIElementRef)CFArrayGetValueAtIndex(windows, i);
            error = _AXUIElementGetWindow(window, &windowId);
            
            if (error != kAXErrorSuccess || targetWindowId == 0) {
                qDebug() << "Error: couldn't get CGWindowID from AXUIElementRef. Error code:" << error;
                continue;
            }
            
            if (windowId == targetWindowId) {
                CFRetain(window);
                result = window;
                break;
            }
        }
        
        CFRelease(app);
        CFRelease(windows);
        
        return result;
    }
    

    You could use it with an NSWindow like this:

    NSWindow *window = ...;
    copyWindowWithPIDAndWindowID(getpid(), window.windowNumber);
    

    This function also allows getting an AXUIElementRef to a window from another application.