I have a program that saves many large files >1GB using fwrite It works fine, but unfortunately due to the nature of the data each call to fwrite only writes 1-4bytes. with the result that the write can take over an hour, with most of this time seemingly due to the syscall overhead (or at least in the library function of fwrite). I have a similar problem with fread.
Does anyone know of any existing / library functions that will buffer these writes and reads with an inline function, or is this another roll your own?
If you write from just one thread, try using fwrite_unlocked. It does wonders relative to straight fwrite in this kind of scenarios.