[SOLVED] Understanding an instance of Python's struct.unpack

Understanding an instance of Python's struct.unpack

I found sample code for interrogating NTP servers on https://www.mattcrampton.com/blog/query_an_ntp_server_from_python/. The code is brief and well-written, but I don't understand the use of struct.unpack. This is the core code:

client = socket.socket(AF_INET,SOCK_DGRAM)
client.sendto(msg.encode('utf-8'),address)
msg,address = client.recvfrom(buf)

t = struct.unpack("!12I",msg)[10]

It returns an integer value (seconds from 1900-01-01) but I have two questions:

How does the unpack work in this instance? How did he arrive at "!12I" to do the decoding?
Is it possible to get a floating point value from the remote server?

Solution

Following the documentation of struct, you are unpacking the first twelve (12) big-endian (!) unsigned 4-byte integers (I) of the NTP header into a tuple, of which you then extract the value at index 10 ([10]), i.e. the penultimate integer value.

Following the definition of the NTP header format, this value is the first part of the Transmit Timestamp (Offset Octet 40 in the linked Wikipedia article). The first part of each timestamp, that is, in your case, the part that you extracted, corresponds to the seconds passed since 1900-01-01 (midnight). Because UNIX timestamps are represented in seconds passed since 1970-01-01 (midnight) instead, the code in the blog post then continues to correct this difference by subtracting the number of seconds passed between 1900-01-01 and 1970-01-01:

TIME1970 = 2208988800
...
t = struct.unpack("!12I", msg)[10]
t -= TIME1970

As to the second part of your question (floating point value): The last integer of the message that you unpacked ([11]) contains the fractional part of a second for your timestamp, namely a numerator to be divided by the denominator 2³². So, to also get the floating point value that you are after, you could extend your code as follows:

client = socket.socket(AF_INET, SOCK_DGRAM)
client.sendto(msg.encode('utf-8'), address)
msg, address = client.recvfrom(buf)
t, f = struct.unpack("!12I", msg)[10:12]
t += f / (2 ** 32)

All in all, reproducing the code from the quoted blog post, but with fractional seconds, you could do:

from datetime import datetime
from socket import AF_INET, SOCK_DGRAM
import socket, struct
 
def getNTPTime(host = "pool.ntp.org"):
    port = 123
    buf = 1024
    address = (host, port)
    msg = '\x1b' + 47 * '\0'
 
    TIME1970 = 2208988800
 
    client = socket.socket(AF_INET, SOCK_DGRAM)
    client.sendto(msg.encode('utf-8'), address)
    msg, address = client.recvfrom(buf)
 
    t, f = struct.unpack("!12I", msg)[10:12]
    t -= TIME1970
    t += f / (2 ** 32)
    return datetime.fromtimestamp(t).strftime("%a %b %d %H:%M:%S.%f %Y")
 
if __name__ == "__main__":
    print(getNTPTime())

Update: Not sure if this makes a huge difference in practice, but a bit more robust approach would probably be to add the fractional seconds only after creation of the datetime object that I used for producing the format string, thus:

from datetime import datetime, timedelta
...
t, f = struct.unpack("!12I", msg)[10:12]
t -= TIME1970
t = datetime.fromtimestamp(t) + timedelta(seconds=f / (2 ** 32))