As part of a concurrency blog series, I was building the simplest HTTP server in different languages (Java, Kotlin, Rust, Go, JS, TS) and everything works fine for everything except Java/Kotlin, aka on the JVM. All the code can be found here. The below is the server code in Java, I tried a traditional Thread based one and an AsynchronousServerSocketChannel based one, but regardless when I run a benchmark with ApacheBench it fails with Broken pipe and apr_socket_recv: Connection reset by peer (104) this is weird as similar setup in other languages works fine. The problem here happens only with ApacheBench, coz when I access the URL in a browser it just works fine. SO I'm banging my head to figure out what is going on. I tried to play with keep-alive etc but doesn't seem to help. I looked at a bunch of examples of something similar and I don't see anything special being done anywhere. I'm hoping someone can figure out what is going wrong here as it definitely seems to be something to do with JVM + APacheBench. I have tried this with Java 11 and 15 but it's the same result.
Java Thread Sample (hello.html can be any HTML file)
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
public class JavaHTTPServerCopy {
public static void main(String[] args) {
int port = 8080;
try (ServerSocket serverSocket = new ServerSocket(port)) {
System.out.println("Server is listening on port " + port);
while (true) {
new ServerThreadCopy(serverSocket.accept()).start();
}
} catch (IOException ex) {
System.out.println("Server exception: " + ex.getMessage());
}
}
}
class ServerThreadCopy extends Thread {
private final Socket socket;
public ServerThreadCopy(Socket socket) {
this.socket = socket;
}
@Override
public void run() {
var file = new File("hello.html");
try (
// we get character output stream to client (for headers)
var out = new PrintWriter(socket.getOutputStream());
// get binary output stream to client (for requested data)
var dataOut = new BufferedOutputStream(socket.getOutputStream());
var fileIn = new FileInputStream(file)
) {
var fileLength = (int) file.length();
var fileData = new byte[fileLength];
int read = fileIn.read(fileData);
System.out.println("Responding with Content-length: " + read);
var contentMimeType = "text/html";
// send HTTP Headers
out.println("HTTP/1.1 200 OK");
out.println("Connection: keep-alive");
out.println("Content-type: " + contentMimeType);
out.println("Content-length: " + fileLength);
out.println(); // blank line between headers and content, very important !
out.flush(); // flush character output stream buffer
dataOut.write(fileData, 0, fileLength);
dataOut.flush();
} catch (Exception ex) {
System.err.println("Error with exception : " + ex);
} finally {
try {
socket.close(); // we close socket connection
} catch (Exception e) {
System.err.println("Error closing stream : " + e.getMessage());
}
}
}
}
Error on console
Responding with Content-length: 176
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
Error with exception : java.net.SocketException: Broken pipe (Write failed)
ApacheBench output
ab -c 100 -n 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (104)
Java Async Sample
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.StandardSocketOptions;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousServerSocketChannel;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.channels.CompletionHandler;
public class JavaAsyncHTTPServer {
public static void main(String[] args) throws Exception {
new JavaAsyncHTTPServer().go();
Thread.currentThread().join();//Wait forever
}
private void go() throws IOException {
AsynchronousServerSocketChannel server = AsynchronousServerSocketChannel.open();
InetSocketAddress hostAddress = new InetSocketAddress("localhost", 8080);
server.bind(hostAddress);
server.setOption(StandardSocketOptions.SO_REUSEADDR, true);
System.out.println("Server channel bound to port: " + hostAddress.getPort());
if (server.isOpen()) {
server.accept(null, new CompletionHandler<>() {
@Override
public void completed(final AsynchronousSocketChannel result, final Object attachment) {
if (server.isOpen()) {
server.accept(null, this);
}
handleAcceptConnection(result);
}
@Override
public void failed(final Throwable exc, final Object attachment) {
if (server.isOpen()) {
server.accept(null, this);
System.out.println("Connection handler error: " + exc);
}
}
});
}
}
private void handleAcceptConnection(final AsynchronousSocketChannel ch) {
var content = "Hello Java!";
var message = ("HTTP/1.0 200 OK\n" +
"Connection: keep-alive\n" +
"Content-length: " + content.length() + "\n" +
"Content-Type: text/html; charset=utf-8\r\n\r\n" +
content).getBytes();
var buffer = ByteBuffer.wrap(message);
ch.write(buffer);
try {
ch.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
No error on console
ApacheBench output
❯ ab -c 100 -n 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (104)
ApacheBench output with keep-alive
ab -k -c 100 -n 1000 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
Send request failed!
apr_socket_recv: Connection reset by peer (104)
Total of 37 requests completed
So, thanks to the comments and answers here and on Twitter, the first code sample is fixed now. The issue was writing to the TCP stream before reading it. Thanks to Ganesh for the original solution on this here and the explanation is on this SO answer
So here is the updated code that works for Java Thread Sample
import java.io.*;
import java.net.ServerSocket;
import java.net.Socket;
public class JavaHTTPServer {
public static void main(String[] args) {
var count = 0;
var port = 8080;
try (var serverSocket = new ServerSocket(port, 100)) {
System.out.println("Server is listening on port " + port);
while (true) {
count++;
new ServerThread(serverSocket.accept(), count).start();
}
} catch (IOException ex) {
System.out.println("Server exception: " + ex.getMessage());
}
}
}
class ServerThread extends Thread {
private final Socket socket;
private final int count;
public ServerThread(Socket socket, int count) {
this.socket = socket;
this.count = count;
}
@Override
public void run() {
var file = new File("hello.html");
try (
var in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// we get character output stream to client (for headers)
var out = new PrintWriter(socket.getOutputStream());
// get binary output stream to client (for requested data)
var dataOut = new BufferedOutputStream(socket.getOutputStream());
var fileIn = new FileInputStream(file)
) {
// add 2 second delay to every 10th request
if (count % 10 == 0) {
System.out.println("Adding delay. Count: " + count);
Thread.sleep(2000);
}
// read the request fully to avoid connection reset errors and broken pipes
while (true) {
String requestLine = in.readLine();
if (requestLine == null || requestLine.length() == 0) {
break;
}
}
var fileLength = (int) file.length();
var fileData = new byte[fileLength];
fileIn.read(fileData);
var contentMimeType = "text/html";
// send HTTP Headers
out.println("HTTP/1.1 200 OK");
out.println("Content-type: " + contentMimeType);
out.println("Content-length: " + fileLength);
out.println("Connection: keep-alive");
out.println(); // blank line between headers and content, very important !
out.flush(); // flush character output stream buffer
dataOut.write(fileData, 0, fileLength); // write the file data to output stream
dataOut.flush();
} catch (Exception ex) {
System.err.println("Error with exception : " + ex);
}
}
}
and apacheBench output
ab -r -c 100 -n 1000 http://127.0.0.1:8080/
This is ApacheBench, Version 2.3 <$Revision: 1879490 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8080
Document Path: /
Document Length: 176 bytes
Concurrency Level: 100
Time taken for tests: 2.385 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 260000 bytes
HTML transferred: 176000 bytes
Requests per second: 419.21 [#/sec] (mean)
Time per request: 238.546 [ms] (mean)
Time per request: 2.385 [ms] (mean, across all concurrent requests)
Transfer rate: 106.44 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.8 0 8
Processing: 0 221 600.7 21 2058
Waiting: 0 220 600.8 21 2057
Total: 0 221 600.8 21 2058
Percentage of the requests served within a certain time (ms)
50% 21
66% 33
75% 38
80% 43
90% 2001
95% 2020
98% 2036
99% 2044
100% 2058 (longest request)
I'm gonna try and fix the second Async sample the same way
EDIT: Fixed the Async sample as well
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.StandardSocketOptions;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousServerSocketChannel;
import java.nio.channels.AsynchronousSocketChannel;
import java.nio.channels.CompletionHandler;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.ExecutionException;
public class JavaAsyncHTTPServer {
public static void main(String[] args) throws Exception {
new JavaAsyncHTTPServer().start();
Thread.currentThread().join(); // Wait forever
}
private void start() throws IOException {
// we shouldn't use try with resource here as it will kill the stream
var server = AsynchronousServerSocketChannel.open();
var hostAddress = new InetSocketAddress("127.0.0.1", 8080);
server.bind(hostAddress, 100); // bind listener
server.setOption(StandardSocketOptions.SO_REUSEADDR, true);
System.out.println("Server is listening on port 8080");
final int[] count = {0}; // count used to introduce delays
// listen to all incoming requests
server.accept(null, new CompletionHandler<>() {
@Override
public void completed(final AsynchronousSocketChannel result, final Object attachment) {
if (server.isOpen()) {
server.accept(null, this);
}
count[0]++;
handleAcceptConnection(result, count[0]);
}
@Override
public void failed(final Throwable exc, final Object attachment) {
if (server.isOpen()) {
server.accept(null, this);
System.out.println("Connection handler error: " + exc);
}
}
});
}
private void handleAcceptConnection(final AsynchronousSocketChannel ch, final int count) {
var file = new File("hello.html");
try (var fileIn = new FileInputStream(file)) {
// add 2 second delay to every 10th request
if (count % 10 == 0) {
System.out.println("Adding delay. Count: " + count);
Thread.sleep(2000);
}
if (ch != null && ch.isOpen()) {
// Read the first 1024 bytes of data from the stream
final ByteBuffer buffer = ByteBuffer.allocate(1024);
// read the request fully to avoid connection reset errors
ch.read(buffer).get();
// read the HTML file
var fileLength = (int) file.length();
var fileData = new byte[fileLength];
fileIn.read(fileData);
// send HTTP Headers
var message = ("HTTP/1.1 200 OK\n" +
"Connection: keep-alive\n" +
"Content-length: " + fileLength + "\n" +
"Content-Type: text/html; charset=utf-8\r\n\r\n" +
new String(fileData, StandardCharsets.UTF_8)
).getBytes();
// write the to output stream
ch.write(ByteBuffer.wrap(message)).get();
buffer.clear();
ch.close();
}
} catch (IOException | InterruptedException | ExecutionException e) {
System.out.println("Connection handler error: " + e);
}
}
}