[SOLVED] Extracting the MachineKey from a deployed Azure App Service

I have an ASP.NET 4.6 Web API service running as an Azure App Service in a single App Service Plan in a single region. We are modifying this service to be deployed over several regions with a load balancer in front, and each region will have its own App Service Plan. We therefore need to ensure we use the same machine key on each App Service Plan to prevent users being logged out when they are directed to different servers by the load balancer.

Our application has been running for a while using the machine key automatically provided by Azure on a single App Service Plan. In order to avoid causing all our customers to be logged out during the transition I planned on extracting this existing machine key and then deploying it on the new App Service Plans in other regions. Sounds simple enough, right?

However extracting this key is proving to be a challenge.

I've tried the solutions listed here: Getting the current ASP.NET machine key

While each method does return some kind of key, the key does not appear to match the key actually being used to generate bearer tokens or to protect refresh token tickets. When I deploy these keys to other servers, bearer tokens are still considered invalid and trying to use an existing refresh token results in an invalid_grant response.

In addition, even when I manually set the machine key in the web.config (or at runtime using code such as this), none of the extracted machine keys matches the machine key I've manually set, providing further evidence that whatever they are returning is not the machine key actually being used. This is true both locally on my development machine, and in Azure.

For reference, this is the code (with some security gumph taken out) that I've used to extract the decryption and validation keys in three different ways:

[DllImport(@"C:\Windows\Microsoft.NET\Framework64\v4.0.30319\webengine4.dll")]
internal static extern int EcbCallISAPI(IntPtr pECB, int iFunction, byte[] bufferIn, int sizeIn, byte[] bufferOut, int sizeOut);

[Route("machine-key-test")]
public async Task<JObject> GetMachineKeys()
{
    return new JObject(
        new JProperty("A", GetAdminData()),
        new JProperty("B", GetAdminDataNoIsolateApps()),
        new JProperty("C", GetAdminDataPre45()));

    JObject GetAdminData()
    {
        string appPath = "/";
        byte[] genKeys = new byte[1024];
        byte[] autogenKeys = new byte[1024];

        int res = EcbCallISAPI(IntPtr.Zero, 4, genKeys, genKeys.Length, autogenKeys, autogenKeys.Length);

        if (res == 1)
        {
            // Same as above
            int validationKeySize = 64;
            int decryptionKeySize = 24;

            byte[] validationKey = new byte[validationKeySize];
            byte[] decryptionKey = new byte[decryptionKeySize];

            Buffer.BlockCopy(autogenKeys, 0, validationKey, 0, validationKeySize);
            Buffer.BlockCopy(autogenKeys, validationKeySize, decryptionKey, 0, decryptionKeySize);

            int pathHash = StringComparer.InvariantCultureIgnoreCase.GetHashCode(appPath);
            validationKey[0] = (byte)(pathHash & 0xff);
            validationKey[1] = (byte)((pathHash & 0xff00) >> 8);
            validationKey[2] = (byte)((pathHash & 0xff0000) >> 16);
            validationKey[3] = (byte)((pathHash & 0xff000000) >> 24);

            decryptionKey[0] = (byte)(pathHash & 0xff);
            decryptionKey[1] = (byte)((pathHash & 0xff00) >> 8);
            decryptionKey[2] = (byte)((pathHash & 0xff0000) >> 16);
            decryptionKey[3] = (byte)((pathHash & 0xff000000) >> 24);

            var decrptionKeyString = decryptionKey.Aggregate(new StringBuilder(), (acc, c) => acc.AppendFormat("{0:x2}", c), acc => acc.ToString());
            var validationKeyString = validationKey.Aggregate(new StringBuilder(), (acc, c) => acc.AppendFormat("{0:x2}", c), acc => acc.ToString());

            return new JObject(
                new JProperty("D", decrptionKeyString),
                new JProperty("V", validationKeyString));
        }


        return null;
    }

    JObject GetAdminDataNoIsolateApps()
    {
        string appPath = "/";
        byte[] genKeys = new byte[1024];
        byte[] autogenKeys = new byte[1024];

        int res = EcbCallISAPI(IntPtr.Zero, 4, genKeys, genKeys.Length, autogenKeys, autogenKeys.Length);

        if (res == 1)
        {
            // Same as above
            int validationKeySize = 64;
            int decryptionKeySize = 24;

            byte[] validationKey = new byte[validationKeySize];
            byte[] decryptionKey = new byte[decryptionKeySize];

            Buffer.BlockCopy(autogenKeys, 0, validationKey, 0, validationKeySize);
            Buffer.BlockCopy(autogenKeys, validationKeySize, decryptionKey, 0, decryptionKeySize);

            var decrptionKeyString = decryptionKey.Aggregate(new StringBuilder(), (acc, c) => acc.AppendFormat("{0:x2}", c), acc => acc.ToString());
            var validationKeyString = validationKey.Aggregate(new StringBuilder(), (acc, c) => acc.AppendFormat("{0:x2}", c), acc => acc.ToString());

            return new JObject(
                new JProperty("D", decrptionKeyString),
                new JProperty("V", validationKeyString));
        }


        return null;
    }

    JObject GetAdminDataPre45()
    {
    // https://stackoverflow.com/a/35954339/37725
        byte[] autogenKeys = (byte[]) typeof(HttpRuntime)
            .GetField("s_autogenKeys", BindingFlags.NonPublic | BindingFlags.Static).GetValue(null);

        Type t = typeof(System.Web.Security.DefaultAuthenticationEventArgs).Assembly.GetType(
            "System.Web.Security.Cryptography.MachineKeyMasterKeyProvider");
        ConstructorInfo ctor = t.GetConstructors(BindingFlags.Instance | BindingFlags.NonPublic)[0];

        Type ckey = typeof(System.Web.Security.DefaultAuthenticationEventArgs).Assembly.GetType(
            "System.Web.Security.Cryptography.CryptographicKey");
        ConstructorInfo ckeyCtor = ckey.GetConstructors(BindingFlags.Instance | BindingFlags.Public)[0];
        Object ckeyobj = ckeyCtor.Invoke(new object[] {autogenKeys});
        object o = ctor.Invoke(new object[] {new MachineKeySection(), null, null, ckeyobj, null});
        var encKey = t.GetMethod("GetEncryptionKey").Invoke(o, null);
        byte[] encBytes = ckey.GetMethod("GetKeyMaterial").Invoke(encKey, null) as byte[];
        var vldKey = t.GetMethod("GetValidationKey").Invoke(o, null);
        byte[] vldBytes = ckey.GetMethod("GetKeyMaterial").Invoke(vldKey, null) as byte[];
        string decryptionKey = BitConverter.ToString(encBytes);
        decryptionKey = decryptionKey.Replace("-", "");
        string validationKey = BitConverter.ToString(vldBytes);
        validationKey = validationKey.Replace("-", "");

        return new JObject(
            new JProperty("D", decryptionKey),
            new JProperty("V", validationKey));
    }
}

And this is an example of the output I get:

{
    "A": {
        "D": "b298ba4ef5e8421e178770f50ee5414dd0aa1698afc3169d",
        "V": "b298ba4e3ed466051c60e4c8646ece2546f27e8b9b2e9a569453eaab6b2a4e93bc08a3171ea61972adfd83c97d21bbcfad2acd3e6d35668f5458f8d7c8f55913"
    },
    "B": {
        "D": "dc509c9af5e8421e178770f50ee5414dd0aa1698afc3169d",
        "V": "84246e973ed466051c60e4c8646ece2546f27e8b9b2e9a569453eaab6b2a4e93bc08a3171ea61972adfd83c97d21bbcfad2acd3e6d35668f5458f8d7c8f55913"
    },
    "C": {
        "D": "A2EDFD4ECE75A91F8E38D62B569248B14CE9193DD42E543E0D4BA5C9E2BED912",
        "V": "DC6144A79985DEF712FABC729871A79FF2CF0DD73CBA617C3764D234DA1B63AD"
    }
}

I've tried using each of these sets of keys in turn, both without explicitly setting the decryption and validation algorithms, and also with specifying various combinations of algorithms which match they key lengths as defined here, with no luck.

And as I said, none of these keys will match the machine key that I'm manually setting in my web.config, or that I'm manually setting in code.

The conclusion I'm coming to is that I'm either doing something trivially wrong, or I'm going to have to cause all our users to be forcibly logged out by changing the machine key to a new one on all our servers. I'm hoping someone will be able point me in the right direction.