.net kubernetes aws-sdk amazon-eks aws-sdk-net

How can I use AWS to authenticate to the EKS Kubernetes API using .NET when my program is running outside the cluster?

I would like to develop a .NET program that accesses the Kubernetes API to perform some administrative tasks. Our Kubernetes cluster is EKS, so I would like to use a native AWS authentication method to generate temporary credentials and access the API, because my program must run outside of Kubernetes for architectural reasons. I would like to map an AWS role to a Kubernetes role and then use the rights granted to this role to access the API and perform the given tasks.

I saw that in the AWS CLI there was a command called aws eks get-token, which is the recommended method for retrieving an access token in Terraform, so I installed AWSSDK.EKS, but discovered unfortunately that there is no such method in the .NET variant of the library when looking at the methods on IAmazonEks.

Reviewing the source code for the aws eks get-token command, I see that we are using STS to generate a presigned URL:

def _get_presigned_url(self, k8s_aws_id):
    return self._sts_client.generate_presigned_url(
        'get_caller_identity',
        Params={K8S_AWS_ID_HEADER: k8s_aws_id},
        ExpiresIn=URL_TIMEOUT,
        HttpMethod='GET',
    )

After reviewing the output of aws eks get-token, I see that the token is indeed a base 64-encoded URL which presumably the cluster will invoke to receive the caller identity, and attempt to map it to a role before granting access - quite a nice trick. Indeed, invoking this URL yields the caller identity as expected. For reference, here is how you invoke it:

GET https://sts.eu-west-1.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=....&X-Amz-SignedHeaders=host%3Bx-k8s-aws-id&X-Amz-Security-Token=...
Host: sts.eu-west-1.amazonaws.com
x-k8s-aws-id: my-cluster-id

However, it is unfortunate to see that a C# equivalent of generate_presigned_url() does not exist in AWS.SecurityToken either!

So, how can I generate an EKS security token for use with the .NET Kubernetes client library without calling out to AWS CLI?

Solution

While writing and researching this question I stumbled across the answer which I would like to share to save people time in future.

Please note that I am quite new to Kubernetes so might not fully understand everything, so please ensure you understand what is going on and test thoroughly. Furthermore, please read to the end as there are simpler alternatives depending on your use-case.

After I understood the format of the output of aws eks get-token, I realised that this presigned URL looked a lot like the presigned URLs used in S3. I was able to use the same technique to make a presigned URL for GetCallerIdentity. There is a great deal of code in AmazonS3Client.GetPresignedUrl for backwards compatibility which I don't fully understand so this might not work for every single use-case.

However, this code snippet shows end-to-end how to generate a token and authenticate to your Kubernetes cluster running on EKS:

// for reference, these are the using statements. For simplicity however, all code is inline.
using Amazon;
using Amazon.Runtime;
using Amazon.Runtime.Internal;
using Amazon.Runtime.Internal.Auth;
using Amazon.Runtime.Internal.Util;
using Amazon.SecurityToken;
using Amazon.SecurityToken.Internal;
using Amazon.SecurityToken.Model;
using Amazon.SecurityToken.Model.Internal.MarshallTransformations;
using k8s;
using System.Security.Cryptography.X509Certificates;


// Configuration:
const string clusterId = "my-eks-cluster";
const string clusterUrl = "https://0000000.xx.eu-west-1.eks.amazonaws.com";
const string certificateAuthority = "dGhpcyBpcyBub3QgYWN0dWFsbHkgYSBDQQ==...";
const string region = "eu-west-1";


// 60s is what aws eks get-token uses and seems appropriate because it's not too expensive to make a new token.
// I haven't tested to see if there's an upper limit here.
const int credentialAge = 60;

// It's best to retrieve credentials from your local instance profile, profile or wherever:
var credentials = await FallbackCredentialsFactory.GetCredentials().GetCredentialsAsync();


// We don't use the STS client directly, but we still need some of its variables and internals:
var sts = new AmazonSecurityTokenServiceClient(new AmazonSecurityTokenServiceConfig
{
    AuthenticationRegion = region,
    RegionEndpoint = RegionEndpoint.GetBySystemName(region),
    StsRegionalEndpoints = StsRegionalEndpointsValue.Regional
});
var signer = new AWS4PreSignedUrlSigner();

// All AWS requests in the .NET SDK are turned into an IRequest object, which is the base object
// that is sent to the REST client.
var request = GetCallerIdentityRequestMarshaller.Instance.Marshall(new GetCallerIdentityRequest());
request.Headers["x-k8s-aws-id"] = clusterId;
request.HttpMethod = "GET";
request.OverrideSigningServiceName = "sts";

if (!string.IsNullOrEmpty(credentials.Token))
    request.Parameters["X-Amz-Security-Token"] = credentials.Token;

request.Parameters["X-Amz-Expires"] = Convert.ToString(credentialAge);


// We will now prepare the request as if we were to send it so that we can set other parameters. We only
// seem to set the host and endpoint field but there is a great deal of logic behind these methods so
// possibly some important edge cases are covered.
var endpointResolver = new AmazonSecurityTokenServiceEndpointResolver();
endpointResolver.ProcessRequestHandlers(new Amazon.Runtime.Internal.ExecutionContext(new Amazon.Runtime.Internal.RequestContext(true, new NullSigner())
{
    Request = request,
    ClientConfig = sts.Config
}, null));

// We get a signature for the request using a built-in AWS utility - this is the same thing that we
// do when sending a real REST request.
var result = signer.SignRequest(request, sts.Config, new RequestMetrics(), credentials.AccessKey, credentials.SecretKey);

// We can't append result.ForQueryParameters to the URL like the AWS S3 client, as EKS
// authorisation expects that the results will be URL-encoded:
request.Parameters["X-Amz-Credential"] = $"{result.AccessKeyId}/{result.Scope}";
request.Parameters["X-Amz-Algorithm"] = "AWS4-HMAC-SHA256";
request.Parameters["X-Amz-Date"] = result.ISO8601DateTime;
request.Parameters["X-Amz-SignedHeaders"] = result.SignedHeaders;
request.Parameters["X-Amz-Signature"] = result.Signature;

// Finally we have a signed URL - this can be called like so if you would like to test that it works:
// GET {signedUrl}
// Host: sts.{region}.amazonaws.com
// x-k8s-aws-id: {clusterId}
var signedUrl = AmazonServiceClient.ComposeUrl(request).ToString();

// Now, we just need to format it how EKS expects it:
var encodedUrl = Convert.ToBase64String(Encoding.UTF8.GetBytes(signedUrl));
var eksToken = "k8s-aws-v1." + encodedUrl.TrimEnd('=');


// Now, with our new token we can go ahead and connect to EKS:
var clientConfig = new KubernetesClientConfiguration
{
    AccessToken = eksToken,
    Host = clusterUrl,
    SslCaCerts = new X509Certificate2Collection(new X509Certificate2(Convert.FromBase64String(certificateAuthority)))
};

// If your credentials have the right permissions, you should be able to get a list of your namespaces:
var kubernetesClient = new Kubernetes(clientConfig);

foreach (var ns in kubernetesClient.CoreV1.ListNamespace().Items)
{
    Console.WriteLine(ns.Metadata.Name);
}

I hope that this presents a useful alternative if you need to do something more complex or need to append Kubernetes functionality to an existing .NET tool.

Furthermore, a great deal of search results surround generating presigned URLs for S3 and I don't think it's common knowledge that you can create presigned URLs for other AWS endpoints, so hopefully this helps solve this specific problem and spark some other ideas as well.

Alternative: Use your local config

It would be remiss of me to not mention a far simpler alternative, which is to simply create a Kubernetes client using your local Kubernetes configuration. However:

This directly invokes awscli, meaning you must install this and other dependencies on your server and keep it up to date.
When using this in development on Windows, the AWS window pops up for a second, stealing the focus, which is annoying when working on a long-lived process.
If you need to programmatically target an unknown number of Kubernetes clusters at once, you don't need to add them to your Kubernetes config file.
If processing user input, there is an increased security risk when passing user input to external processes. I prefer to not do this if possible.
There is also a slight performance overhead also to calling out to an external process.

However, I cannot deny the simplicity, so this is an option available to you if you are able to install awscli on your target environment and configure Kubernetes:

var config = KubernetesClientConfiguration.BuildConfigFromConfigFile();
var kubernetesClient = new Kubernetes(config);

foreach (var ns in kubernetesClient.CoreV1.ListNamespace().Items)
{
    Console.WriteLine(ns.Metadata.Name);
}