socketsnginxluajson-rpclua-ngx-module

Is the ngx.socket.tcp() call limited in any way when called from an nginx location context?

I'm very new to using the local sock = ngx.socket.tcp() call that is provided by the nginx lua packages provided on Ubuntu Noble (i.e. the libnginx-mod-http-lua package). I've been able to get some success in opening a socket and writing out some data. But, based on my tcpdump analysis and other cues, the sock:send() operation works but I don't get any data back. My current theory is that nginx somehow prevents the ACK from the sock:connect() from being received by the lua block.

In order to simplify things, I decided to reproduce this simple example to make sure that I'm doing things correctly. When I make up my own version of this, it hangs on the receive() call.

Here is my setup:

Here's the code snippets I'm using and their output:

nginx test location:

location /foo {
    content_by_lua_block {
        local tcp=ngx.socket.tcp()
        ngx.say(ngx.now())
        ngx.flush()
        tcp:connect("127.0.0.1", 8000)
        ngx.say("connect() did not block")
        ngx.flush()
        local data, err = tcp:receive(1)
        ngx.say("receive() did not block")
        ngx.say(err)
        ngx.flush()
        ngx.update_time()
        ngx.say(ngx.now())
        ngx.flush()
    }
}

curl invocation (which I Ctrl+C b/c it hangs):

rlott@1022rdnote04:~/testing$ curl -k https://192.168.56.220/foo
1740693455.69
connect() did not block
^C
rlott@1022rdnote04:~/testing$

nc running on port 8000 (doing nothing):

etservice@noble:~$ nc -l 8000 -k

Errors I see in /var/log/nginx/error.log (eventually):

2025/02/27 16:58:35 [error] 608728#608728: *101 lua tcp socket read timed out, client: 192.168.56.1, server: , request: "GET /foo HTTP/1.1", host: "192.168.56.220"
2025/02/27 16:58:35 [info] 608728#608728: *101 client 192.168.56.1 closed keepalive connection

So, the $1M question is what is the secret to using nginx.socket.tcp() successfully in this way?

Again, I did another example where I sent a JSON-RPC message to one of my services and tcpdump shows me that the data got there, but that it hangs on the sock:receive(). The only difference between my message and a bonafide client sending the same message seems to be something having to do with the "Conversation completeness" (taken from my Wireshark analysis). In the Wireshark analysis, the only 2 things that I could see that were different were:

  1. My message didn have a TCP FIN flag set
  2. The bonafide client's JSON-RPC message had a larger data size than mine.

So, I'm kinda stumped. Any insight would be appreciated.

Solution

  • Okay, thanks to Dmitry Meyer for helping me out. I wasn't thinking clearly about the simple example (which wasn't sending anything but just trying to trigger a large timeout). Once I fixed that, I was able to modify the example to something closer to what I was looking for. So, the modified location and lua script is this:

    location /foo {
content_by_lua_block {
    local tcp=ngx.socket.tcp()
    ngx.say(ngx.now())
    ngx.flush()
    tcp:connect("127.0.0.1", 12000)
    ngx.say("connect() did not block")
    ngx.flush()
    tcp:send('{"jsonrpc":"2.0","method":"get.identifier","id":1}\r\n')
    local data, err = tcp:receive()
    ngx.say("receive() did not block: data = ", data)
    if err then
        ngx.say("error = ", err)
    end
    ngx.flush()
    ngx.update_time()
    ngx.say(ngx.now())
    ngx.flush()
}

    where port 16000 is the port of my backend JSON-RPC process. I triggered the location with curl and got what I was hoping for:

    rlott@1022rdnote04:~/testing/MAG-9214/njs-exps$ curl -k https://192.168.56.220/foo
1741016175.722
connect() did not block
receive() did not block: data = {"id":1,"jsonrpc":"2.0","result":"94e2741e-217f-45c3-902c-4883864fd834"}
1741016175.723

    The main issue with the TCP FIN not and the hang was not lua related but JSON-RPC related. My server needed the correct ending delimiter (i.e. \r\n) and was waiting for it prior to sending the response back.