Azure AD - How to check whether a service principal is being used?

I've got a bunch of old app registrations/service principals that no one has any idea if it's being used or not. When I got into the app from Enterprise Application (All Applications) blade and see Sign-ins from Activity, nothing shows up. Nothing in Audit Logs either.

I did the same thing for an app we ARE using regularly and there's still nothing in sign-ins. I use that app to connect to our tenant from a PowerShell session; I don't even known if the Sign-ins activity log is supposed to keep track of every time you authenticate to AD using that service principal. I opened a support ticket with Microsoft and they just told me to check the Sign-ins logs but there is nothing there. I do have diagnostic logging enabled for the entire AD but nothing there either.

Is there anyway to see whether an app is being used? How frequently? Please advise!

Microsoft implemented auditing for service principals. You can see in the Sign-In logs on the "Service principal sign-ins"-Tab the information about it.