unit-testing.net-coredotnet-httpclientxunit.netncrunch

Test runners inconsistent with HttpClient and Mocking HttpMessageRequest XUnit


So let me start by saying I've seen all the threads over the wars between creating a wrapper vs mocking the HttpMethodRequest. In the past, I've done the wrapper method with great success, but I thought I'd go down the path of Mocking the HttpMessageRequest.

For starters here is an example of the debate: Mocking HttpClient in unit tests. I want to add that's not what this is about.

What I've found is that I have tests upon tests that inject an HttpClient. I've been doing a lot of serverless aws lambdas, and the basic flow is like so:

//some pseudo code
public class Functions
{
   public Functions(HttpClient client)
   {
       _httpClient = client;
   }

   public async Task<APIGatewayResponse> GetData(ApiGatewayRequest request, ILambdaContext context)
   {
       var result = await _client.Get("http://example.com");
       return new APIGatewayResponse 
              { 
                  StatusCode = result.StatusCode,
                  Body = await result.Content.ReadStringAsAsync()
              };
   }
}

...
[Fact]
public void ShouldDoCall()
{
     var requestUri = new Uri("http://example.com");
     var mockResponse = new HttpResponseMessage(HttpStatusCode.OK) { Content = new StringContent(expectedResponse) };
     var mockHandler = new Mock<HttpClientHandler>();
     mockHandler
        .Protected()
        .Setup<Task<HttpResponseMessage>>(
           "SendAsync",
           It.IsAny<HttpRequestMessage>(),
           It.IsAny<CancellationToken>())
       .ReturnsAsync(mockResponse);

     var f = new Functions(new HttpClient(handler.Object);
     var result = f.GetData().Result;

     handlerMock.Protected().Verify(
        "SendAsync",
         Times.Exactly(1), // we expected a single external request
         ItExpr.Is<HttpRequestMessage>(req =>
             req.Method == HttpMethod.Get && 
             req.RequestUri == expectedUri // to this uri
         ),
         ItExpr.IsAny<CancellationToken>()
     );

     Assert.Equal(200, result.StatusCode);
}

So here's where I have the problem!

When all my tests run in NCrunch they pass, and pass fast!

When I run them all manually with Resharper 2018, they fail.

Equally, when they get run within the CI/CD platform, which is a docker container with the net core 2.1 SDK on a Linux distro, they too fail.

These tests should not be run in parallel (read the tests default this way). I have about 30 tests around these methods combined, and each one randomly fails on the moq verify portion. Sometimes they pass, sometimes they fail. If I break down the tests per test class and on run the groups that way, instead of all in one, then these will all pass in chunks. I'll also add that I have even gone through trying to isolate the variables per test method to make sure there is no overlap.

So, I'm really lost with trying to handle this through here and make sure this is testable.

Are there different ways to approach the HttpClient where it can consistently pass?


Solution

  • After lots of back n forth. I found two of situations from this.

    1. I couldn't get parallel processing disabled within the docker setup, which is where I thought the issue was (I even made it do thread sleep between tests to slow it down (It felt really icky to me)

    2. I found that all the tests l locally ran through the test runners were telling me they passed when about 1/2 failed on the docker test runner. What ended up being the issue was a magic string area when seeing and getting environment variables.

    3. Small caveat to call out, Amazon updated their .NET Core lambda tools to install via dotnet cli, so this was updated in our docker image.