pythondat-protocol

How to load dat file from IR camera in python with specific structure


I try to read a dat file with the structure given below. It is a video from an IR camera. I tried different methods, but I always get errors along the way. I would like to convert it in a different file format and load it in python, so I can do some analysis with it.

import csv
datContent = [i.strip().split() for i in open("./Test_s.dat",'rb').readlines()]
# write it as a new CSV file
with open("./Test_s.csv", "wb") as f:
    writer = csv.writer(f)
    writer.writerows(datContent)

Datastructure


Solution

  • Example to read the header fields excluding the data log, since no definition was provided:

    I assumed little endian, change the first symbol in the pattern to '>' if the dat file used big endian.

    import struct
    
    # whitespace between definitions is ignored
    # 's' is special: 16s means a byte string of length 16, not 16 single bytes
    # 6H will return 6 seperate uint16 values
    header_pattern = '< 4s 6H d 16s 16s 16s 16s 6f H 16s H 2B 4H'
    
    # verifiyng size of our pattern: 2060-1918 => 142
    print(struct.calcsize(header_pattern))
    
    # read file
    with open('./Test_s.dat' 'rb') as f:
        header_bytes = f.read(142)
    
        # unpack bytes
        header_values = struct.struct.unpack(pattern, header_bytes)
        print(len(header_values))  # we expect 27 values
    
        # skip data log, can also use f.seek()
        f.read(1918)
    
        # handle each image header and data
        ...
    
    

    I hope this is enough to get you going, keep in mind you need calculate the image data size from some header values.

    I cannot see a header for image count, so you probably have to read until the file ends.