swiftmacosswiftuiaccessibility-api

Accessibility API AXFocusedUIElement always returns nil


I am building an accessibility app to capture the selected text from any Screen but the kAXFocusedUIElementAttribute always returns nil not sure why

Github Test Repo and Video : https://github.com/khanakia/accessibility_xcode_bug

import Foundation
import AppKit
import SwiftUI

class AccessibilityManager: ObservableObject {
    private var eventMonitor: Any?
    
    init() {
        checkAccessibility()
    }

    private func captureTextUnderCursor() {    
        let systemWideElement = AXUIElementCreateSystemWide()
        print("systemWideElement: \(String(describing: systemWideElement))")


        var focusedElement: AnyObject?
        AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute as CFString, &focusedElement)
        print("focusedElement: \(String(describing: focusedElement))")

    }

    
    func setupGlobalShortcut() {
        // Remove existing monitor if any
        if let monitor = eventMonitor {
            NSEvent.removeMonitor(monitor)
            eventMonitor = nil
        }
        
        // Register for global keyboard shortcut (Control + J)
        eventMonitor = NSEvent.addGlobalMonitorForEvents(matching: .keyDown) { [weak self] event in
            if event.modifierFlags.contains(.control) && event.keyCode == 38 { // 38 is 'J'
                self?.captureTextUnderCursor()
                
            }
        }
    }
    
    
    func checkAccessibility() {
        //  ...
    }
    
    deinit {
        if let monitor = eventMonitor {
            NSEvent.removeMonitor(monitor)
        }
    }
} 

OUTPUT

systemWideElement: <AXUIElement System Wide 0x600002acc4b0>
focusedElement: nil

Solution

  • I copied hello1App, ContentView and AccessibilityManager to a fresh project, removed the sandbox and it works, returning a focused element.
    From looking at your repo, your project has App Sandbox enabled.
    To work with AXUIElement, you need to remove Sandbox.

    In sandbox, even though AXIsProcessTrustedWithOptions returns success, any calls to the API will return nil.

    Note 1:
    In your func captureTextUnderCursor(), focusedElement is of type AXUIElement. To get the string value, you'll need to call for its selected text attribute.

    Note 2: NSEvent.addGlobalMonitorForEvents is not working in your own app. Use could use .onKeyPress on your text view.

    Note 3:
    No need to import Foundation and AppKit when you import SwiftUI.