Define custom headers using .NET 6 OpenAI SDK

Technical details

• Client applications are using .NET 6
• Client applications are using the OpenAIClient class from the official OpenAI nuget library
• There is an inhouse cluster of LLM servers with OpenAI API support
• A reverse proxy that intercepts the client requests

Goal

• The client apps (using the OpenAiAPI SDK) to define some custom HTTP headers which will be used by the reverse proxy for balancing the load and sticky sessions

Note: the values of the headers are not static. Each request can potentially have a different value.

Problem

I believe that the official SDK:

• Does not allow defining custom headers
• Does not allow defining an HttpClient object nor a DelegatingHandler (which they could be used to define custom headers)

Questions

• Is the problem statement correct?
• Is there a workaround to this problem?
• Is there a completely different solution on how to approach to the sticky session requirement (given the dynamic nature of the header values)?

(!) Disclaimer: This is merely a solution to the initial requirement of adding custom headers. I am not knowledgeable enough to assess whether the solution is affecting the OpenAiClient capabilities in any way (!)

The trick was to use a Pipeline policy (via the HttpClientPipelineTransport).

        services.AddTransient<CustomHeadersHandler>(); //this is a standard 'DelegatingHandler' implementation that adds HTTP headers
        services.AddHttpClient("OpenAiHttpClient", client =>
        {
            client.Timeout = TimeSpan.FromMilliseconds(apiTimeout);
        })
        .AddHttpMessageHandler<CustomHeadersHandler>();

        services.AddSingleton(services =>
        {
            var httpClientFactory = services.GetRequiredService<IHttpClientFactory>();
            var httpClient = httpClientFactory.CreateClient("OpenAiHttpClient");

            return new OpenAIClient(
                new ApiKeyCredential(apiKey),
                new OpenAIClientOptions
                {
                    Endpoint = new Uri(apiUri),
                    NetworkTimeout = TimeSpan.FromMilliseconds(apiTimeout),
                    Transport = new HttpClientPipelineTransport(httpClient)
                }
            );
        });