I have a minor project which will involve repeated HTTPS communication between a Java application and a Flask web server such that keeping the TCP connection alive is important. In order to test and understand the capabilities of these technologies, I have set up a simple GET request generator which sends multiple GET requests with time invertals to my simple Flask web server application and return a response. In such a setup, I would expect the TCP connection to be cached but I think my Flask application disconnects the connection after each response. Beware that I am quite rusty with Java and it is my first time getting acquainted with Flask.
My Java GET request generator is like below:
package com.deu;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
public class Main {
public static void main(String[] args) {
try {
sendHttpRequests("http://127.0.0.1:5000", 1000, 5);
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
public static void sendHttpRequests(String url, int interval, int count) throws IOException, InterruptedException {
URL hostURL = new URL(url);
for (int i = 0; i < count; i++) {
HttpURLConnection httpConn = (HttpURLConnection) hostURL.openConnection();
httpConn.setRequestMethod("GET");
BufferedReader reader = new BufferedReader(new InputStreamReader(httpConn.getInputStream()));
StringBuilder response = new StringBuilder();
String line;
while ((line = reader.readLine()) != null)
response.append(line);
System.out.println("Response to i="+i+" "+response.toString());
reader.close();
Thread.sleep(interval);
}
}
}
My Flask web server application is like below:
import flask
from flask import Flask, request
from werkzeug.serving import WSGIRequestHandler
app = Flask(__name__)
@app.route('/', methods=['GET'])
def hello_world():
resp = flask.Response("Hello World!")
print(request.headers)
print(resp.headers)
return resp
if __name__ == '__main__':
WSGIRequestHandler.protocol_version = "HTTP/1.1"
app.run()
After the Java application terminates, the outputs I get for Java and Flask applications respectively are:
Response to i=0 Hello World!
Response to i=1 Hello World!
Response to i=2 Hello World!
Response to i=3 Hello World!
Response to i=4 Hello World!
FLASK_APP = app.py
FLASK_ENV = development
FLASK_DEBUG = 0
In folder B:/Development/Projects/Python/ProjectABC
B:\Development\Projects\Python\ProjectABC\venv\Scripts\python.exe -m flask run
* Serving Flask app 'app.py'
* Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
* Running on http://127.0.0.1:5000
Press CTRL+C to quit
User-Agent: Java/17.0.1
Host: 127.0.0.1:5000
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 12
127.0.0.1 - - [08/Apr/2023 01:26:28] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [08/Apr/2023 01:26:29] "GET / HTTP/1.1" 200 -
User-Agent: Java/17.0.1
Host: 127.0.0.1:5000
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 12
User-Agent: Java/17.0.1
Host: 127.0.0.1:5000
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 12
127.0.0.1 - - [08/Apr/2023 01:26:30] "GET / HTTP/1.1" 200 -
User-Agent: Java/17.0.1
Host: 127.0.0.1:5000
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 12
127.0.0.1 - - [08/Apr/2023 01:26:31] "GET / HTTP/1.1" 200 -
User-Agent: Java/17.0.1
Host: 127.0.0.1:5000
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Content-Length: 12
127.0.0.1 - - [08/Apr/2023 01:26:32] "GET / HTTP/1.1" 200 -
As can be seen, the requests receive responses as expected and even the request headers Flask application received shows the presence of Connection: keep-alive. But the reason I think a new TCP connection is created each time are that the response headers do not contain Connection: keep-alive and the output of netstat cmd command:
Active Connections
Proto Local Address Foreign Address State
TCP 127.0.0.1:5000 DESKTOP-XXXXXXX:52821 TIME_WAIT
TCP 127.0.0.1:5000 DESKTOP-XXXXXXX:52823 TIME_WAIT
TCP 127.0.0.1:5000 DESKTOP-XXXXXXX:52824 TIME_WAIT
TCP 127.0.0.1:5000 DESKTOP-XXXXXXX:52830 TIME_WAIT
TCP 127.0.0.1:5000 DESKTOP-XXXXXXX:52831 TIME_WAIT
If my understanding of the output is correct (please correct me if I am wrong), 5 different TCP connections were made and now the OS is waiting to clear them, indicating keep-alive has failed. I searched other questions for answers and tried putting WSGIRequestHandler.protocol_version = "HTTP/1.1" but to no avail.
Note: I am trying to achieve persistent connection using HTTP first, after which I will apply it to HTTPS.
I had been trying to achieve persistent connection using the WSGI server Flask comes packaged with called Werkzeug. When I switched to a deployment WSGI server, in my case Waitress, keep-alive started to work and TCP connections stopped being dropped. The issue was being caused by a change in Werkzeug implementation. Their change log for Version 2.1.2 released in 2022-04-28 states:
Disable keep-alive connections in the development server, which are not supported sufficiently by Python’s
http.server. #2397
Werkzeug team removed the persistent connection support due to the shortcomings of http.server which it extends. While it is understandable that the lack of keep-alive function may not be essential in a development environment, it is frustrating that there is very little to no communication about this in the Internet.