.NET 8 Middleware Unexpectedly Changes Status Code to 204 After _next(context)

I am working on a .NET 8 Webapp. I have implemented a middleware to log requests and responses, but I'm facing an issue where the status code is unexpectedly changing from 200 to 204 after calling _next(context). Here is the code and a detailed description of my setup and the problem:

Midleware Implementation

public async Task Invoke(HttpContext context)
{
    var requestBody = await GetRequestBody(context.Request);
    var originalBodyStream = context.Response.Body;

    using (var memoryStream = new MemoryStream())
    {
        context.Response.Body = memoryStream;

        // Log initial headers and status code: StatusCode is 200 before request goes to handle.
        Console.WriteLine($"Initial Status Code: {context.Response.StatusCode}");

        await _next(context);

        // Log headers and status code after _next. StatusCode is 204 even though its is being set to 200 explicitly in handler and context.Response.Body is also not null/empty.
        Console.WriteLine($"Status Code After _next: {context.Response.StatusCode}");

        memoryStream.Seek(0, SeekOrigin.Begin);
        var responseBody = await new StreamReader(memoryStream).ReadToEndAsync();
        context.Response.Body = originalBodyStream;

        // Write the response body if it is not empty
        if (!string.IsNullOrEmpty(responseBody))
        {
            try
            {
                context.Response.ContentLength = Encoding.UTF8.GetByteCount(responseBody);
                //While writing responseBody it also throws following exception: Writing to the response body is invalid for responses with status code 204.
                await context.Response.WriteAsync(responseBody);
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error writing response body: {ex.Message}");
            }
        }

        await memoryStream.CopyToAsync(originalBodyStream);
    }
}

Endpoint Handler

public class CreateAuthorization : Endpoint<CreateAuthorizationRequest, CreateAuthorizationResponse>
{
    public override void Configure()
    {
        Post(CreateAuthorizationRequest.Route);
        Summary(s =>
        {
            s.Summary = "Create a new Authorization request.";
            s.Description = "Create a new Authorization request to get random number from Nafath.";
        });
    }

    public override async Task HandleAsync(CreateAuthorizationRequest request, CancellationToken cancellationToken)
    {
        var result = await _mediator.Send(new CreateAuthorizationCommand(request.IdNumber!, action, request.Service!, ""), cancellationToken);

        var convertedResult = result.IsSuccess ? new CreateAuthorizationResponse(result.Value.TransactionId, result.Value.Random) : Result.Error();

        await convertedResult.ToOkOrProblemDetail(HttpContext);

        //Till this point HttpContext.Response.StatusCode is 200 and after this it goes directly to middleware after the line `await _next(context);` and somehow status code changed to 204.
    }
}

ToOkOrProblemDetails Extension Method

public static async Task ToOkOrProblemDetail(this Result result, HttpContext httpContext)
{
    if (result.IsSuccess)
    {
        var responseBody = JsonSerializer.Serialize(result.Value);
        httpContext.Response.StatusCode = StatusCodes.Status200OK;
        httpContext.Response.ContentType = "application/json";
        httpContext.Response.ContentLength = Encoding.UTF8.GetByteCount(responseBody);
        await httpContext.Response.WriteAsync(responseBody);
        Console.WriteLine("Response written with status 200 and content length set");
    }
    else
    {
        await result.ToProblemDetails(httpContext);
    }

}

Middleware Pipeline

In Program.cs the middleware is configured as follows:

var app = builder.Build();
if (app.Environment.IsDevelopment())
{
    app. UseDeveloperExceptionPage() ;
    app.UseShowAIIServicesMiddIeware();
}
else
{
    app.UseDefauItExceptionHandIer();
    app.UseHsts();
}
app.UseAuthentication();
app.UseAuthorization();
app.UseFastEndpoints(x => x.Errors.UseProbIemDetails());

//Using the middleware right after configuration of FastEndpoints but moving this to right after builder.Build() or right before app.Run() have not affect on the error.
app.UseMiddleware<RequestLoggingMiddleware>();

app.UseSwaggerGen();
app.UseHttpsRedirection();
SeedDatabase(app);
if (GetLocaIIPAddress() == && builder.Configuration.GetValue<string>("AllowIp"))
    app.UseHangfireDashboard("/hangfire", new DashboardOptions
    {
        Authorization = new[] { new AllowAnonymousAccessFilter() }
    });

app.Run();

Problem Details

Even though the status code is set to 200 within the handler (actually inside of ToOkOrProblemDetail extension method), it changes to 204 after returning to the middleware. This unexpected change is causing errors when attempting to write to the response body, "Writing to the response body is invalid for responses with status code 204."

I would appreciate any insights or suggestion on why status code is changing from 200 to 204 after the middleware pipeline and how to prevent this from happening. Could the use of the FastEndpoint library be affecting this behavior?

Here is what I have tried so far:

1. Copying the Response: Used different methods to copy the response, including context.Response.WriteAsync and context.Response.CopyAsync.

2. Explicitly Setting Content Length: Tried setting the Content-Length explicitly in the handler.

3. Middleware Placement: Moved the middleware to different locations in the pipeline to see if placement was affecting the behavior.

4. Manual Status Code Setting: Tried both setting and not setting the status code manually in the middleware.

Despite these attempts, the status code still changes from 200 to 204 after _next(context).

when writing the response to the response stream yourself like you're doing in your ToOkOrProblemDetail() extension method, you need to inform fastendpoints that you're taking care of sending a response like so:

HttpContext.MarkResponseStart()

this is an extension method provided by FE. it is shown in the documentation here regarding writing your own Send*Async() methods as extensions.

i believe that should solve the issue. if not, please create a new issue either on FE github repo or discord server with a repro project with your code.