SMB2 CHANGE_NOTIFY looks promising, as if it could deliver enough information on subdirectory or subtree updates from the server, so we can keep our listing of remote directory up-to-date by handling the response.
However, it's not a subscription to an event stream, just a one-off command receiving one response, so I suspect that it can be used only as a mere hint to invalidate our cache and reread the directory. When we receive a response, there could be any additional changes before we send another CHANGE_NOTIFY request, and we'll miss the details of these changes.
Is there any way around this problem? Or is rereading directory on learning that it's updated a necessary step?
I want to understand possible solutions on the protocol level (you can imagine I'm using a customized client that I can make do what I want, with some common servers like Windows or smbd3).
Strictly speaking, not even re-reading the directory listing should save you, since the directory can change between re-reading the listing and submitting another CHANGE_NOTIFY request. The race condition just moves to a different spot.
Except there is no race condition.
This took a little digging, but it’s all there in the specification. In MS-SMB2 v20200826 §3.3.5.19 ‘Receiving an SMB2 CHANGE_NOTIFY Request’, it is stated:
The server MUST process a change notification request in the object store as specified by the algorithm in section 3.3.1.3.
In §3.3.1.3 ‘Algorithm for Change Notifications in an Object Store’, we have:
The server MUST implement an algorithm that monitors for changes on an object store. The effect of this algorithm MUST be identical to that used to offer the behavior specified in [MS-CIFS] sections 3.2.4.39 and 3.3.5.59.4.
And in MS-CIFS v20201001 §3.3.5.59.4 ‘Receiving an NT_TRANSACT_NOTIFY_CHANGE Request’ there is this:
If the client has not issued any NT_TRANSACT_NOTIFY_CHANGE Requests on this FID previously, the server SHOULD allocate an empty change notification buffer and associate it with the open directory. The size of the buffer SHOULD be at least equal to the MaxParameterCount field in the SMB_COM_NT_TRANSACT Request (section 2.2.4.62.1) used to transport the NT_TRANSACT_NOTIFY_CHANGE Request. If the client previously issued an NT_TRANSACT_NOTIFY_CHANGE Request on this FID, the server SHOULD already have a change notification buffer associated with the FID. The change notification buffer is used to collect directory change information in between NT_TRANSACT_NOTIFY_CHANGE (section 2.2.7.4) calls that reference the same FID.
Emphasis mine. This agrees with how Samba implements it (the fsp structure persists between individual requests). I wouldn’t expect Microsoft to do worse than that by not keeping the promise they made in their own specification.