we have just upgraded to TFS 2017 from 2013. We had a custom plugin that ran when we changed the build quality. Since the upgrade it doesn't fire. we have tried changing the required DLLs to use the The 2017 client dlls. but the build quality handler does not trigger the plugin. it uses the Microsoft.TeamFoundation.Framework.Server.ISubscriber interface. We do not get any exceptions as well on the tfs server.
The ISubscriber implementation needs to be recompiled against the TFS 2017 Server as well as Client object model.
And it's important to understand that the new build infrastructure (the non-xaml builds) likely trigger a different set of notifications. At least they're not queryable with the old Client Object Model IBuildServer, you need to use the new REST API.
Without knowing more about your setup (what type of builds, the exact versions of the object model you're binding against, what permissions the TFS Service user has) it's hard to tell where this is going wrong. We have a troubleshooting guide for the TFS Aggregator (https://github.com/tfsaggregator/tfsaggregator/wiki/Troubleshooting) which is also a ISubscriber plugin, it may help you debug your setup.