macOS DriverKit: Making PCI dext to replace built-in driver

EDIT: The driver is working and has been open-sourced https://github.com/OpenMPDK/MacVFN

I'm trying to write a user-space PCI driver in DriverKit for educational/research purposes. I've found an example from WorthDoingBadly which has the boilerplate code for a PCI device dext (I've removed the exploit code).

I've modified it to match a Thunderbolt PCI NVMe device through the IOPCIPrimaryMatch key. I've been able to compile, sign, and load it with SIP disabled and systemextensionsctl developer on.

The problem arises when my device gets plugged in, and the Start function in my driver is called. I attempt to call ivars->pciDevice->Open(this, 0); on the device, which fails with 0xe00002cd "(iokit/common) device not open".

Meanwhile, I can see in the kernel logs, that the built-in NVMe driver is already initializing when my driver is called.

If I skip the call to Open and just call RegisterService();, I can see in IORegistryExplorer.app, that both the IONVMeController and my "PCICrash" are listed under the PCI device.

I speculate, that my driver would work, if I could keep the built-in NVMe driver from taking the device. Is this possible somehow?

For reference, my Info.plist looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>IOKitPersonalities</key>
    <dict>
        <key>PCICrash</key>
        <dict>
            <key>IOClass</key>
            <string>IOUserService</string>
            <key>IOProviderClass</key>
            <string>IOPCIDevice</string>
            <key>IOUserClass</key>
            <string>PCICrash</string>
            <key>IOUserServerName</key>
            <string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
            <key>IOPCIPrimaryMatch</key>
            <string>0x25228086</string>
            <key>IOPCITunnelCompatible</key>
            <true/>
        </dict>
    </dict>
</dict>
</plist>

This is the PCICrash.entitlements for the driver:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.developer.driverkit</key>
    <true/>
    <key>com.apple.developer.driverkit.transport.pci</key>
    <true/>
    <key>com.apple.developer.driverkit.transport.pci.bridge</key>
    <true/>
    <key>com.apple.developer.driverkit.allow-any-userclient-access</key>
    <true/>
</dict>
</plist>

This is the PCICrashApp.entitlements for the app:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
    <key>com.apple.security.files.user-selected.read-only</key>
    <true/>
    <key>com.apple.developer.system-extension.install</key>
    <true/>
</dict>
</plist>

I build in Xcode without signing (the same as on WorthDoingBadly's setup), and then run the following:

$ codesign -s - -f --entitlements "PCICrash/PCICrash.entitlements" "[...]/PCICrashApp.app/Contents/Library/SystemExtensions/com.worthdoingbadly.PCICrashApp.PCICrash.dext"
$ codesign -s - -f --entitlements "PCICrashApp/PCICrashApp.entitlements" "[...]/PCICrashApp.app"
$ systemextensionsctl reset
$ [...]/PCICrashApp.app/Contents/MacOS/PCICrashApp
2023-05-08 18:28:17.810 PCICrashApp[3438:81755] requestNeedsUserApproval
2023-05-08 18:28:23.152 PCICrashApp[3438:81755] didFinishWithResult: 0

A view of IORegistryExplorer. It starts out with white text, upon loading the app and registering the extension, it resets the device, and reattaches it using the stock NVMe driver. Maybe this could be explained by a crashing driver.

After a lot of back and forth in the comments, we established that the problem was simply one of probe score.

I/O Kit matching uses a numeric probe score to resolve matching conflicts. This is specifically intended so that device- or vendor-specific drivers can be given a higher probe score to be prioritised over generic, vendor-independent drivers for class-compliant devices. That is exactly the situation we have here.

USB Matching has a built-in mechanism that boosts the probe score depending on the type of matching pattern, so vendor/device-specific drivers automatically get priority without having to specify an explicit probe score.

The PCI device matching logic on the other hand treats IOPCIClassMatch based matching dictionaries with the same priority as IOPCIMatch, IOPCIPrimaryMatch, and IOPCISecondaryMatch patterns by default, so as we found out here, you can end up with a tie.

The solution is to include an explicit IOProbeScore in the match dictionary which exceeds the generic driver's. The probe score gets attached to the "winning" driver as a property, so in this case the IONVMeController node has a IOProbeScore property with a value of 100 (0x64), so as long as you exceed that, your driver should win, e.g.:

            <key>IOProbeScore</key>
            <integer>1000</integer>

So for winning out against generic (AHCI, XHCI, NVMe, etc.) PCI drivers, you'll probably have to set a probe score in your device-specific driver. As I mentioned, USB has its own automatic mechanism, so I don't in general recommend setting an explicit probe score for USB drivers.