How do I read a portion of a file using pread(2) in C? I've tried using the O_NONBLOCK flag on open() but the read still seems to block the thread (i.e reading a large portion at once, 1GB, seems to hang the thread for about 300ms on my M1 mac)
I've looked into aio_read() but that does not seem to work at all on some systems and seems to either spawn a new thread to run the callback, or run all callbacks on one thread, which is not suitable for my needs.
Ideally I'd like a solution in which the process does not have to do any waiting and can simply poll to check if the buffer has been populated, something analogous to how nodejs would implement fs.read(fd, ..., callback)
What is the proper way to achieve what I'm looking for? I've searched online for hours but most questions seem to concern sockets / pipes and not actual files (where read() literally waits for new data to arrive)
Context: server software with high CPU loads and very high disk usage (multiple million reads per minute of about 1KB-4KB each), the vast majority of reads will be made to an NVMe drive (with average 0.01ms response time, but expected to be much higher under load)
The current accepted solution seems to be using thread pools, any information on the overhead of frequent thread switching would also be useful
As pointed out on one of the comments of this question the most accepted way to do this is synchronously with a thread pool, which is what I will be using.