filesystemsoverwriteext4sector

File-system: Overwriting data of equal length


I have a project where I have to update the data on disk very frequently in case of power loss. When overwriting exactly 512b (1 sector of my drive) in the file with data of equal length does the file-system mark the sectors that have been changed and update them on disk when ready to flush? Or does it write the whole file every time it flushes a change? I am mainly concerned with ext4 but I am curious if it is the same with every file-system.

If the standard is not to track changes but to overwrite the entire file is there a way to change this? Some write options?


Solution

  • In general with Linux files are cached in the page cache, and whether or not a page is dirty is tracked at the page level. On Intel x86 platforms, the page size is 4k, so if you dirty a 4k page, it is the 4k page which gets written back.

    If you want to only overwrite a single 512 byte sector, and you have a HDD that has 512 byte sectors, you can open the file with the O_DIRECT flag and if you issue a 512 byte write, on a file offset which is a multiple of 512 bytes, and where the memory buffer from where you source the write is also 512 byte aligned, then you can bypass the page cache, and the write will go directly to the disk (hence O_DIRECT).

    Note that a number of modern disks are really using a 4k physical sector, but they are emulating 512 byte sectors for backwards compatibility reasons. These disks are sometimes called 512e sectors (e for emulated). On these drives, if you do a 512 byte sector write, the disk will do a read-modify-write cycle, since the drive internally can only write 4k at a time. This will be visible to you as a performance hit, but from a functional perspective, it will otherwise look the same as a traditional, old-fashioned 512 sectored HDD.