IDX10503: Signature validation failed after updating to Owin.Security v 4.0.0

As per subject, I updated the Owin.Security.WsFederation and dependent packages to version 4.0 and I get the error.

I did not make any code changes other than changing

using Microsoft.IdentityModel.Protocols; 

to

using Microsoft.IdentityModel.Protocols.WsFederation;

where is the WsFederationConfiguration class seems to be now.

Here is my StartupAuth:

public void ConfigureAuth(IAppBuilder app)
        {
            app.UseCookieAuthentication(
                new CookieAuthenticationOptions
                {
                    AuthenticationType = CookieAuthenticationDefaults.AuthenticationType
                });

            // Create WsFed configuration from web.config wsfed: values
            var wsconfig = new WsFederationConfiguration()
            {
                Issuer = ConfigurationManager.AppSettings["wsfed:Issuer"],
                TokenEndpoint = ConfigurationManager.AppSettings["wsfed:TokenEndPoint"],                
            };

            /* 
             * Add x509 certificates to configuration
             * 
             */
            // certificate.1 must always exist
            byte[] x509Certificate;
            x509Certificate = Convert.FromBase64String(ConfigurationManager.AppSettings["wsfed:certificate.1"]);
            wsconfig.SigningKeys.Add(new X509SecurityKey(new X509Certificate2(x509Certificate)));
            // certificate 2 may exist
            if (ConfigurationManager.AppSettings["wsfed:certificate.2"] != null)
            {
                x509Certificate = Convert.FromBase64String(ConfigurationManager.AppSettings["wsfed:certificate.2"]);
                wsconfig.SigningKeys.Add(new X509SecurityKey(new X509Certificate2(x509Certificate)));
            }
            // certificate 3 may exist
            if (ConfigurationManager.AppSettings["wsfed:certificate.3"] != null)
            {
                x509Certificate = Convert.FromBase64String(ConfigurationManager.AppSettings["wsfed:certificate.3"]);
                wsconfig.SigningKeys.Add(new X509SecurityKey(new X509Certificate2(x509Certificate)));
            }

            // Apply configuration to wsfed Auth Options
            var wsoptions = new WsFederationAuthenticationOptions
            {
                SignInAsAuthenticationType = CookieAuthenticationDefaults.AuthenticationType,
                Configuration = wsconfig,
                Wreply = ConfigurationManager.AppSettings["wsfed:Wreply"],
                Wtrealm = ConfigurationManager.AppSettings["wsfed:Wtrealm"],
            };
            wsoptions.TokenValidationParameters.NameClaimType = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn";

            // Add WdFederation middleware to Owin pipeline
            app.UseWsFederationAuthentication(wsoptions);
        }

Is there something else 4.0 needs to validate the signature? I assume it's talking about the signature of the token from the issuer. I didn't see how to enable ShowPII to see what key it's looking at.

I am using MVC5 with the full framework. Not core.

Update:

I tried to modify the code to use the metadata provided by the identity provider in a properties file to create the WsFederationConfiguration and I still get the same error. I'm not sure what the Signature is, or where I get it from if it's not in the idp metadata.

Update2:

Here are the changes I made to use the wsfed metadata provided by the sts in a properties file. (I have removed the actual base64 encoded metadata, but needless to say it is the same XML you get when you regest the metadata from an STS that publishes it as and endpoint. As I said above, I get the same error:

    public void ConfigureAuth(IAppBuilder app)
    {
        WsFederationConfiguration wsconfig;

        app.UseCookieAuthentication(
            new CookieAuthenticationOptions
            {
                AuthenticationType = CookieAuthenticationDefaults.AuthenticationType
            });

        var metaDataDocument = System.Text.Encoding.UTF8.GetString(
                Convert.FromBase64String("...c2NyaXB0b3I+"));

        using (var metaDataReader = XmlReader.Create(new StringReader(metaDataDocument), SafeSettings))
        {
            wsconfig = (new WsFederationMetadataSerializer()).ReadMetadata(metaDataReader);
        }

        // Apply configuration to wsfed Auth Options
        var wsoptions = new WsFederationAuthenticationOptions
        {
            SignInAsAuthenticationType = CookieAuthenticationDefaults.AuthenticationType,
            Configuration = wsconfig,
            Wreply = ConfigurationManager.AppSettings["wsfed:Wreply"],
            Wtrealm = ConfigurationManager.AppSettings["wsfed:Wtrealm"],
        };
        wsoptions.TokenValidationParameters.NameClaimType = "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn";

        // Add WdFederation middleware to Owin pipeline
        app.UseWsFederationAuthentication(wsoptions);
    }

I worked with some folks on the team at MS. The issue here was that our STS is using SHA1 to sign the token and the new version of weFederation doesn't support SHA1 as it is not-secure and is deprecated.