Yaws - socket_closed_remotely

Hello fellow Erlangers :)

Just another Erlang enthusiast playing with the language here. I have a very simple YAWS app module which works fine when accessed by single clients. However, when I try to spawn multiple concurrent clients, some of those clients start receiving errors (even if the number of those clients is very low, say 10). Any idea what can be causing it?

Illustrating code:

App Module

out(Arg) ->
  io:format("got something!\n"),
  Method = extract_method(Arg),
  handle(Method, Arg).

Client

client(SenderPID) ->
case httpc:request(
  put,
  {
    "http://localhost:8080/storageunit",
    [],
    "application/x-www-form-urlencoded",
    ""
  }, [], []) of
    { error, Reason } -> io:format("Server responded with an error: ~p.\n", [Reason]);
    { ok, _ } -> ok
  end,
  SenderPID ! 'FINISHED'.

client_spawner(_SenderPID, 0) -> io:format("Done.\n");
client_spawner(SenderPID, Times) ->
  spawn(concurrent, client, [SenderPID]),
  client_spawner(SenderPID, Times - 1).

The errors I'm getting:

Server responded with an error: socket_closed_remotely.

The last piece of info is:

• 10 concurrent clients - from 2 to 5 errors on average
• 100-100000 concurrent clients - 50% errors on average

I thought my handling code causes it, but in scenarios where the clients receive httpc errors, the server doesn't even react with a "got something!".

I'm sure I'm missing something trivial, can you help?

Regards, Piotr

There must have been something wrong with my initial code. I rewrote the module from scratch and it is giving me expected results. Here's the code:

-module(annoying_client).
-export([annoy/0, client/1, client_spawner/2]).

client(SenderPID) ->
  case httpc:request("http://www.google.com") of
    { error, Reason } -> io:format("Error. Reason: ~p\n", [Reason]);
    { ok, _ } -> io:format("ok\n")
  end,
  SenderPID ! 'FINISHED'.

client_spawner(_SenderPID, 0) -> io:format("Done.\n");
client_spawner(SenderPID, Times) ->
  spawn(?MODULE, client, [SenderPID]),
  client_spawner(SenderPID, Times - 1).

annoy() ->
  inets:start(),

  spawn(?MODULE, client_spawner, [self(), 100]),
  wait_for_all_jobs(1),

  inets:stop(),
  init:stop().

wait_for_all_jobs(NumberOfFinishedJobs) ->
  receive 'FINISHED' ->
    if
      NumberOfFinishedJobs < 100 ->
        wait_for_all_jobs(NumberOfFinishedJobs + 1);
      true ->
        io:format("All jobs finished.\n")
    end
  end.

It works like a charm with both:

• google
• local vanilla yaws

GETs and PUTs. The problems start when I cross the threshold of 2000 concurrent requests but I'm guessing it is to be expected judging by the numbers obtained locally and remotely. In case of 2000 requests, I'm getting roughly:

• 6 errors while requesting google.com
• 500 errors while requesting local yaws

The errors I'm getting:

Error. Reason: socket_closed_remotely

and

Error. Reason: {failed_connect,[{to_address,{"www.google.com",80}}, {inet,[inet],system_limit}]}

DoS protection? Incorrect configuration? Anyway, I am happy enough with my current results. Probably will experiment with other Erlang web servers just to get some performance numbers.

Thank you all.