I'm tasked with creating an event stream whereby I have an automated poller (set at 10minute intervals) to retrieve all entities that changed within the last 10minutes.
Now, the business logic dictates that we only create new update events when specific fields change in the entity. Since we don't have granularity with regards to what specific field changed (all we know is that something changed), I have to create some kind of differentiator method that performs the following:
Given this architectural problem, is there an known pattern or set of patterns, or guidelines in how to build out this type of system?
A canonical solution to this problem inverses it: instead of detecting the changes after they are made, the system should produce the change events in the first place. Put otherwise, the API should not allow storing new entity versions; instead it should only allow applying the changes. From system design standpoint, working backwards (detecting changes after they are already made) creates a more complex system because it includes more steps.
Most common implementation of this approach is Event Sourcing, which also fits well with CQRS. The two are typically put in place together. Properly implementing Event Sourcing may be a difficult task and it certainly should not be used without a good reason (your use case certainly sounds like one though). Depending on where the changes are produced you may consider using frameworks that facilitate the change tracking (e.g. Redux in UI layer).
I understand that this modification maybe needs to be added to the system that is already designed around storing the entity versions. One possible solution in this case is to crate an isolation level between two worlds: a layer in the system that will transform entity versions to the stream of the changes. For example, keep the existing interface, but replace its implementation by the one based on events; create the events as early as possible the system, instead of mixing the two approaches.
Hope this makes sense.