How to trigger screen recording permission system modal dialog on macOS Catalina

I have an application which has screen sharing feature in it. On macOS Catalina beta8 (19A558d), you must give screen recording permission to share your screen (Without permission, only your background and menubar will be shared with the other side). Unfortunately, I really do not know which event or call triggers the system modal dialog, sometimes the dialog appears, sometimes it does not.

System modal dialog is shown on page 75 in wwdc macos security documentation:
https://devstreaming-cdn.apple.com/videos/wwdc/2019/701ngx868rfo8jlj/701/701_advances_in_macos_security.pdf?dl=1

So it is absolutely non-deterministic. Without interacting with this modal dialog, my application will not be registered under Security & Privacy / Screen Recording, thus I can not give permission for it. Does anybody have any idea, how can I solve this problem?

Solution

macOS 11+

Apple now provides an API to determine if your app has Screen Recording access via CGPreflightScreenCaptureAccess() and also to request access via CGRequestScreenCaptureAccess(). Requesting access will present the system prompt and automatically add your app in the list so the user just needs to enable access. The system prompt will only appear once per app session. For an example, let's say they click Deny in the first prompt. If your app requests access again the prompt won't appear. But if they quit and relaunch your app, your app could request access again and the prompt would appear.

Original answer:

The screen recording prompt will appear only once - the first time you invoke an API that attempts to record the user's screen, such as:

CGDisplayStreamRef stream = CGDisplayStreamCreate(CGMainDisplayID(), 1, 1, kCVPixelFormatType_32BGRA, nil, ^(CGDisplayStreamFrameStatus status, uint64_t displayTime, IOSurfaceRef frameSurface, CGDisplayStreamUpdateRef updateRef) {
});
if (stream) {
    CFRelease(stream);
}

As you noted, your app won't appear in System Preferences under Screen Recording until you invoke a screen recording API thus triggering the system prompt.

If you trigger the prompt and the user denies it, you cannot bring up the prompt again - the user must manually enable it in System Preferences.

Helpful testing info:

While building and testing this, you can reset your app's permissions as if you never invoked a screen recording API, via tccutil reset ScreenCapture com.company.appname. Or use All instead of ScreenCapture to reset all things permissions for your app.