I have a program written by C. It computes something and writes the output into a file. My problem is that it does not write more than 2GB. Let me put a simplified code.
#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdlib.h>
#include <malloc.h>
#include <errno.h>
int main() {
size_t size = 3221225472LL;
char *map = malloc(size);
size_t allocated = malloc_usable_size(map);
int fd = open("myfile", O_RDWR|O_CREAT|O_TRUNC, (mode_t)0644);
ssize_t written = write(fd, map, size);
return 0;
}
Although the output file "myfile" is being created, the size is always 2GB (2147479552 Bytes) for whatever size greater than 2GB I requested. The malloc() successfully allocated memory of the requested size (in this case, "allocated" is 3GB). The errno after write() is 0.
The environment is the following
Compilation:
gcc code.c -D_FILE_OFFSET_BITS=64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE
What could be the reason for this?
Addition:
After getting two responses, I added retry code as following:
int main() {
size_t size = 3221225472LL;
char *map = malloc(size);
size_t allocated = malloc_usable_size(map);
int fd = open("myfile", O_RDWR|O_CREAT|O_TRUNC, (mode_t)0644);
ssize_t written = write(fd, map, size);
while (written < size) {
written += write(fd, &map[written], size-written);
}
return 0;
}
According to the man page (emphasis mine)
On Linux,
write()(and similar system calls) will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.)