pythonstructpywin32deviceiocontrol

Struct unpack on win32file.DeviceIoControl

I am trying to understand and work with win32file. I need to grab USN Journals and having a hard time understanding code snippets I found online. This is the code snippet I found -

format = 'qqqqqLLLLqqqqq'
length = struct.calcsize(format)
out_buffer = win32file.DeviceIoControl(volh, winioctlcon.FSCTL_GET_NTFS_VOLUME_DATA, None, length)
data = struct.unpack(format, out_buffer)

Now I am really rusty when it comes to C and it's structures. What I have understood for now is format is 96 bytes buffer and it'll get the output from DeviceIoControl

So I tried to change the format to 'QQQQQQQQQQQQQQQQQQQ' to see what happens(to see because I am kind of clueless what might actually happen) and it turns out I got a larger out_buffer this time. So I thought to unpack it -

struct.unpack(format, out_buffer)

And surprise to me, I got -

struct.error: unpack requires a string argument of length 152

So I added another 'Q' to increase the size and got the same result. I don't understand why 'qqqqqLLLLqqqqq' works and 'QQQQQQQQQQQQQQQQQQQ' does not. So my questions are -

Pointing me out to resources would also be an added bonus as I need to build on the code to read USN Journals and I don't think hit-and-try is going to get me anywhere

Solution

  • Let's split the problem in smaller pieces and take each one at a time.

    After going over the above materials, things should become clearer.

    A couple of notes:

    I've also prepared a dummy Python example.

    code00.py:

    #!/usr/bin/env python3

import sys
import struct
import win32file
import win32api
import win32con
import winioctlcon


VOLUME_LETTER = "E"

FILE_READ_ATTRIBUTES = 0x0080
FILE_EXECUTE = 0x0020

vol_data_buf_fmt = "qqqqqLLLLqqqqq"  # This is the format that matches NTFS_VOLUME_DATA_BUFFER definition (96 bytes). Note: Instead of each 'q' you could also use 'Ll' as 'LARGE_INTEGER' is an union

BINARY_FORMAT_LIST = [
    vol_data_buf_fmt,
    "QQQQQQQQQQQQQQQQQQQ",
]


def print_formats():  # Dummy func
    print("Formats and lengths:")
    for format in BINARY_FORMAT_LIST:
        print("    {:s}: {:d}".format(format, struct.calcsize(format)))


def main():
    #print_formats()
    vol_unc_name = "\\\\.\\{:s}:".format(VOLUME_LETTER)
    print("volume: ", vol_unc_name)
    access_flags = FILE_READ_ATTRIBUTES | FILE_EXECUTE  # Apparently, doesn't work without FILE_EXECUTE
    share_flags = win32con.FILE_SHARE_READ | win32con.FILE_SHARE_WRITE  # Doesn't work withou FILE_SHARE_WRITE
    creation_flags = win32con.OPEN_EXISTING
    attributes_flags = win32con.FILE_ATTRIBUTE_NORMAL
    vol_handle = win32file.CreateFile(vol_unc_name, access_flags, share_flags, None, creation_flags, attributes_flags, None)

    buf_len = struct.calcsize(vol_data_buf_fmt)
    for i in [buf_len]:
        print("    Passing a buffer size of: {:d}".format(i))
        buf = win32file.DeviceIoControl(vol_handle, winioctlcon.FSCTL_GET_NTFS_VOLUME_DATA, None, i)
        print("    DeviceIocontrol returned a {:d} bytes long {:}".format(len(buf), type(buf)))
        out = struct.unpack_from(vol_data_buf_fmt, buf)
        print("\n    NumberSectors: {:}\n    TotalClusters: {:}\n    BytesPerCluster: {:}".format(out[1], out[2], out[6]))
    win32api.CloseHandle(vol_handle)


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

    Output:

    (py35x64_test) e:\Work\Dev\StackOverflow\q053318932>"e:\Work\Dev\VEnvs\py35x64_test\Scripts\python.exe" ./code00.py
Python 3.5.4 (v3.5.4:3f56838, Aug  8 2017, 02:17:05) [MSC v.1900 64 bit (AMD64)] on win32

volume:  \\.\E:
    Passing a buffer size of: 96
    DeviceIocontrol returned a 96 bytes long <class 'bytes'>

    NumberSectors: 494374911
    TotalClusters: 61796863
    BytesPerCluster: 4096

    Needless to say that multiplying TotalClusters by BytesPerCluster, I get the correct bytes number (as reported by Win) for my E: drive.