How to develop a DEXT for sending vendor SCSI commands to a USB mass storage device using USBDriverKit?

I am currently in the process of developing a DEXT for a USB based external mass storage device using the USBDriverKit framework. IOUSBHostInterface is used as the provider to communicate with the interface's endpoints and I am successful in it. As per the IOUSBHostInterface documentation,

To use a host interface object, call Open to create a new session between the interface and your driver. After successfully opening your session, you can request information from the interface and set up pipes to communicate with the interface's endpoints. Remember to close the session you opened in the Stop method of your driver.

However, calling Open gains exclusive access to the USB interface and does not allow other services to access the interface. Also, to let go of the exclusive access, Close can be called but in the Stop method which is called only once during the lifecycle of the extension (when the DEXT is unloaded).

As a result of this, Apple's mass storage related KEXTs (media and partition related specifically) do not match the interface and so the filesystem of the drive in question does not get mounted whenever the DEXT has matched the interface.

Once Close is called, the exclusive access is lost and Apple's drivers take over in matching the interface instantly (the filesystem gets mounted back once this happens). As a result, calling Close and then trying to reopen a session to the interface also does not seem to work as the DEXT no longer matches the interface.

Is this exclusive access a limitation of USBDriverkit or is there any way to get around this issue in this case?

Solution

Answer to your literal question:

Is this exclusive access a limitation of USBDriverkit or is there any way to get around this issue in this case?

Yes, this is something of a fundamental limitation of USBDriverKit. But it's there for a very good reason: You can't fire off USB requests to a USB interface when another driver is also using it and expect things to still work. Writes are one thing, but how would reads work? If both the mass storage driver and your driver queue a bulk read after submitting a bulk write, how would the device or OS ensure the correct one goes to the correct driver?

The possible workarounds are:

Do everything in your own driver. Although not recommended by Apple, it's mostly possible to implement a SCSIControllerDriverKit based driver on top of USBDriverKit. So you could implement a driver that correctly multiplexes the SCSI commands and whatever it is you're trying to do.
Take a step back and think about the situation and problem you are trying to solve at a high level without assuming a USBDriverkit based dext is the way to implement it. Include that information in the question. Hence my questions in the comments.

Trying to solve your actual problem:

From the comments:

I am trying to send SCSI vendor commands over USB using the BOT protocol. Earlier, this was possible with a KEXT by having IOSCSIPeripheralNub as the provider (In this case, Apple's mass storage drivers were also in the driver stack as there was no case of exclusive access).

This is exactly what SCSIPeripheralsDriverKit is for. Calling the UserSendCDB method in a IOUserSCSIPeripheralDeviceType00 subclass you should be able to handle your vendor specific commands.

[…] when Apple's IOUSBMassStorageUASDriver matches the IOUSBHostInterface […]

OK, now you're talking about UAS, while you're earlier you're saying you want to use BOT. If the vendor commands are not supported in UAS mode, only in BOT mode, it gets tricky, but as it was working fine with a kext, I suspect that's probably actually not the case, and the commands will work in UAS mode?