I am building an accessibility app to capture the selected text from any Screen but the kAXFocusedUIElementAttribute
always returns nil not sure why
Github Test Repo and Video : https://github.com/khanakia/accessibility_xcode_bug
import Foundation
import AppKit
import SwiftUI
class AccessibilityManager: ObservableObject {
private var eventMonitor: Any?
init() {
checkAccessibility()
}
private func captureTextUnderCursor() {
let systemWideElement = AXUIElementCreateSystemWide()
print("systemWideElement: \(String(describing: systemWideElement))")
var focusedElement: AnyObject?
AXUIElementCopyAttributeValue(systemWideElement, kAXFocusedUIElementAttribute as CFString, &focusedElement)
print("focusedElement: \(String(describing: focusedElement))")
}
func setupGlobalShortcut() {
// Remove existing monitor if any
if let monitor = eventMonitor {
NSEvent.removeMonitor(monitor)
eventMonitor = nil
}
// Register for global keyboard shortcut (Control + J)
eventMonitor = NSEvent.addGlobalMonitorForEvents(matching: .keyDown) { [weak self] event in
if event.modifierFlags.contains(.control) && event.keyCode == 38 { // 38 is 'J'
self?.captureTextUnderCursor()
}
}
}
func checkAccessibility() {
// ...
}
deinit {
if let monitor = eventMonitor {
NSEvent.removeMonitor(monitor)
}
}
}
OUTPUT
systemWideElement: <AXUIElement System Wide 0x600002acc4b0>
focusedElement: nil
I copied hello1App
, ContentView
and AccessibilityManager
to a fresh project, removed the sandbox and it works, returning a focused element.
From looking at your repo, your project has App Sandbox enabled.
To work with AXUIElement
, you need to remove Sandbox.
In sandbox, even though AXIsProcessTrustedWithOptions
returns success, any calls to the API will return nil.
Note 1:
In your func captureTextUnderCursor()
, focusedElement
is of type AXUIElement
. To get the string value, you'll need to call for its selected text attribute.
Note 2:
NSEvent.addGlobalMonitorForEvents
is not working in your own app. Use could use .onKeyPress
on your text view.
Note 3:
No need to import Foundation and AppKit when you import SwiftUI.