assemblycachingx86cpu-architecturetlb

What may occur if the OS doesn't flush a TLB entry when a process does a free()?


What may occur if the OS doesn't flush a TLB entry when a process does a free() ?

Can the process see the stale data or can another process might see data that does not belong to it? Will the memory be leaked?


Solution

  • free for small allocations will normally just put the block of memory on a free-list within this process (effectively using global variables in libc to track this data structure). Larger allocations will often hand the memory back to the OS using munmap or equivalent; I assume this is what you meant.

    Normally munmap flushes the TLB entry for that page on this core and other cores that might be running threads of the same task. (TLB shootdown). But if it didn't:

    Accesses on a core that had the TLB entry hot will continue to work as if that process's page-table entry for that virtual address hadn't changed. (Until the entry gets evicted and has to get reloaded.) Reads (and writes if it was writeable) will still be allowed.

    This is super bad if the page gets reused as the kernel stack for a new process, since the process that did the munmap could potentially take over the kernel by modifying return addresses. Or same thing if it was the user-space stack for a root process.

    Or if it gets reused as pagecache for file data (like /etc/passwd or /etc/shadow), you might be able to modify the copy in RAM of that data, making other processes read data from that file that doesn't match what's on disk. (The OS might not think the pagecache was dirty so might not write back that file data to disk.)

    Or change the bytes of an executable that runs as root. e.g. make this happen somehow, then run sudo if it hasn't recently been loaded from disk. Hopefully the kernel will pick that newly-freed page for use. Then modify the cached version of the machine code for sudo while it's mapped exec + read-only by the sudo process. (e.g. modify it while it's asking for a password, changing what it does when you enter the wrong one.)

    Or in an already-running process, maybe some of its error-handling functions are rarely executed, and are in a page with other "cold" functions which has therefore been evicted from RAM. Feeding that process a rare kind of bad input will make it call one of those error-handling functions, triggering a hard page fault to bring it in from disk into a new page. If it picks your page, you can now modify the machine code of one of those functions and trigger the same error again. Boom, that process is running your machine code, successful code-injection attack.

    To increase the chances of these attacks working, allocate a lot of memory so the system is low on free memory. Then touch a few pages to prime the TLBs, and munmap those pages in whatever way makes the OS forget to invalidate them. Those new pages will be most of what the OS has available for new allocations.

    Obviously this would be an absolutely critical security vulnerability in any OS that tries to do any kind of isolation, and a huge correctness problem that could easily corrupt files (and filesystem metadata that was soon to be written to disk) if combined with use-after-free bugs.