Speed up writing multiple image TIFF?

I'm trying to write stacks of images to TIFF files. The images are of size 256*256 pixels, each stack containing 1000 images. Writing one of those files takes about 4 minutes - so there's most likely something wrong with my code.

Here's what I'm doing:

void Tiff_WriterSplit(float data[], int PicNum, int WIDTH, int LENGTH, char PATH[]) {
     int i;
     int rows = LENGTH, columns = WIDTH;
     TIFF* tif;

     if (PicNum == 0)
         tif = TIFFOpen(PATH, "w");
     else
         tif = TIFFOpen(PATH, "a");

     TIFFSetField(tif, TIFFTAG_IMAGEWIDTH, columns);
     TIFFSetField(tif, TIFFTAG_IMAGELENGTH, rows);
     TIFFSetField(tif, TIFFTAG_SAMPLESPERPIXEL, 1);
     TIFFSetField(tif, TIFFTAG_BITSPERSAMPLE, 32);
     TIFFSetField(tif, TIFFTAG_SAMPLEFORMAT, SAMPLEFORMAT_IEEEFP);

     for (i = 0; i < rows; i++)
         TIFFWriteScanline(tif, &data[i * columns], i, 0);

     TIFFWriteDirectory(tif);
     TIFFClose(tif);
}

This function Tiff_WriterSplit will be called for each image of the stack. The first image will have PicNum = 0 to open the tiff file in w mode, all the following images will have a PicNum > 0 to open it in a mode. Writing this starts at about 40 MB/s and gets slower and slower with increasing filesize - down to about 1 MB/s. How can I optimize the performance?

Solution

Writing tiffs is a lot easier than reading tiffs. The overhead of libtiff can be quite limiting for write performance. Assuming you have a buffer that is in some pixel format that can be represented by a tiff header, the steps to write a tiff,

Write tiff header
Write image data
Write directory (tag table).

For a multi-image tiff, we are limited by 4GB tiff limit of 32bit entries. However, if your images are always of the same geometry (width, height, planes, depths), the directory entries in the tag table can be written out after writing all of the image data. You are free to place the directory at the end (or beginning) of the tiff binary. If at the end and you know the number of images, you can form the directories entries before a write of the image (construct table in place). If the API doesn't allow you to know the number of images before hand, then the final file offset must be calculated in the IFD after the final image is written.

Directly writing a tiff can allow you to use fallocate, mmap and other OS features which may speed up writing multi-image tiffs. The job is a lot more computationally challenging if need to translate the image format as it may involve signal processing.

Single image plane writing can be over 3x faster with a direct write vs libtiff. If libtiff is constantly recalculating and repositioning the tag table at the end of the tiff (IFD), then this will be O(n^2) per image.

I used this technique to stream scientific image sensor data to disk from a genicam device. It allows capturing at much higher FPS.

The combination of vision analysis to find regions of interest with the multi-tiff (directory) structure would allow multiple regions of interest to be saved for each frame in single tiff.

Speed up writing multiple image TIFF?

Reference