testingerlangelixircode-coveragecoveralls

Is there a way to simulate the lack of internet connection in an Elixir test?

I am dealing with a coverage test of an command-line interface application developed in Elixir. The application is a client for tldr-pages and its functioning consists in a script built with escript. To perform the actions I use a case structure over an HTTPoison.get/1 function, in which I introduce the formatted url. In this case I compare the response to different kind of values, such as if the page exists, it shows the information; if the not, it report it to the user and then continue in another case to evaluate to others possibilities. At the end, the first case finish with two pattern to match errors, one for the lack of internet connection and another one for unexpected errors. The described structure is the next one:

    case HTTPoison.get(process_url(os, term)) do
      {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
        IO.puts(body)

      {:ok, %HTTPoison.Response{status_code: 404}} ->
        IO.puts(
          "Term \"#{term}\" not found on \"#{os}\" pages\nExTldr is looking on \"common\" pages."
        )

        case HTTPoison.get(process_url("common", term)) do
          {:ok, %HTTPoison.Response{status_code: 200, body: body}} ->
            IO.puts(body)

          {:ok, %HTTPoison.Response{status_code: 404}} ->
            IO.puts("Term not found on \"common\" pages.")
        end

      {:error, %HTTPoison.Error{reason: reason}} when reason == :nxdomain ->
        raise NoInternetConnectionError

      {:error, %HTTPoison.Error{reason: reason}} when reason != :nxdomain ->
        raise UnexpectedError, reason
    end

NoInternetConnectionError and UnexpectedError are exceptions defined in another file. Both patterns at the end works apparently nice, at least the first one:

      {:error, %HTTPoison.Error{reason: reason}} when reason == :nxdomain ->
        raise NoInternetConnectionError

However, as I said at the beginning of the question, I am dealing with a coverage test performed automatically with GitHub Actions and Coveralls with ExCoveralls in the dependencies. In this test I am receiving a warning for both raise/1 statements. Although I could be wrong understanding what means that, I understand this "missed lines" or uncovered lines are reported by Coveralls because I am not performing a test to cover this case. Thus I started to research how I could write a test to cover both conditions.

The most important issue is how to simulate the lack of internet connection in a test to cover this. I though about to develop a mock, but I do not find something useful for this case in some of the mocking packages for Elixir, like Mox. Then I found bypass, a very interesting package that I think it might be useful because it has down/1 and up/1 to close and start a TCP socket, so it makes possible to test what happens when HTTP server is down. But with this I have two issues:

  1. A server down is not the same that the lack of internet connection by the user part.
  2. I tried to applied this "down and up" mechanism and I did not accomplish it. I am not going to share it because I think it would not be the final solution by the first issue described.

I am not pretending an answer with the code that solves this problem, I am just trying to understand how this test should work and the logic I should follow to develop it. I am even researching Erlang documentation, because it is possible Erlang provides native functions to address it (for example, I am now reading the Erlang's Common Test Reference Manual, because may be there is something useful there).

Edit. What I commented I tried with bypass was to install the dependency, write a setup with bypass.open/0 and then write a test like the next one, in which I try to assert the capture output with capture_io/1:

  test "lack of internet connection", %{bypass: bypass} do
    Bypass.down(bypass)

    execute_main = fn ->
      ExTldr.main([])
    end

    assert capture_io(execute_main) =~ "There is not internet connection"
  end

However, as I thought, it does not cover the possible situation of lack of internet, just the possibility to check when a server goes down.

Solution

  • Sidenote: I personally was always against being a slave of tools that are supposed to help the development. Coverage is a somewhat good metric, but the recommendations should not be treated as a must. Anyway.

    I am not sure why you ruled Mox out. The rule of thumb would be: tests should not involve cross-boundary calls unless absolutely unavoidable. The tests going over the internet are nevertheless flaky: coverage would not tell you that, I would. What if the testing environment has no permanent internet access at all? Temporary connection issues? The remote is down?

    So that is exactly why Mox was born. And, luckily enough, HTTPoison is perfectly ready to use Mox as a mocking library because it declares a behaviour for the main operation module, HTTPoison.Base.

    All you need would be to make your actual HTTP client an injected dependency. Somewhat along these lines:

    
@http_client Application.get_env(:my_app, :http_client, HTTPoison)

...

case @http_client.get(process_url(os, term)) do
   ...
end

    In config/test.exs you specify your own :http_client, and voilà—the nifty mocked testing environment is all yours.

    Or, you might declare the mock straight ahead:

    Mox.defmock(MyApp.HC, for: HTTPoison.Base)

    I am also adept of boundaries on the application level calling 3rd-parties. That said, you might define your own behaviour for external HTTP calls you need and your own wrapper implementing this behaviour. That way mocking would be even easier, and you’ll get a benefit of easy changing the real client. HTTPoison is far from being the best client nowadays (it barely supports HTTP2 etc,) and tomorrow you might decide to switch to, say, Mint. It would be drastically easier to accomplish if all the code would be located in the wrapper.