I have a PowerShell script that connects to Microsoft.Graph through certificate based authentication
Connect-MgGraph -ClientId "XXXXXX" -TenantId "XXXXX" -CertificateThumbprint "XXXXX"
This is the first error that I encounter when using PowerShell version 5.1: Connect-MgGraph : The type initializer for 'Azure.Core.Pipeline.DiagnosticScopeFactory' threw an exception.
This is the second error that I encounter when using PowerShell version 7.4.1 'Could not load file or assembly 'Microsoft.Identity.Client, Version=4.50.0.0, Culture=neutral, PublicKeyToken=0a613f4dd989e8ae'
Other Details:
I had this same issue and the fix for me was to run Disconnect-PnPOnline before trying to connect with Graph. Since they are using the same app registration, it must not allow you doing both types of connections.