I have a very large (950GB) binary file in which I store 1billion of sequences of float points.
A small example of the type of file I have with sequences of length 3 could be:
-3.456 -2.981 1.244
2.453 1.234 0.11
3.45 13.452 1.245
-0.234 -1.983 -2.453
Now, I want to read a particular sequence (let's say the sequence with index=2, therefore the 3rd sequence in my file) so I use the following code:
#include <iostream>
#include <fstream>
#include <stdlib.h>
using namespace std;
int main (int argc, char** argv){
if(argc < 4){
cout << "usage: " << argv[0] << " <input_file> <length> <ts_index>" << endl;
exit(EXIT_FAILURE);
}
ifstream in (argv[1], ios::binary);
int length = atoi(argv[2]);
int index = atoi(argv[3]);
float* ts = new float [length];
in.clear();
**in.seekg(index*length*sizeof(float), in.beg);**
if(in.bad())
cout << "Errore\n";
**// for(int i=0; i<index+1; i++){**
in.read(reinterpret_cast<char*> (ts), sizeof(float)*length);
**// }**
for(int i=0; i<length; i++){
cout << ts[i] << " ";
}
cout << endl;
in.close();
delete [] ts;
return 0;
}
The problem is that when I use seekg this read fails for some indexes and I get a wrong result. If I read the file in a sequential manner (without using seekg) and print out the wanted sequence instead, I always get the correct result.
At the beginning I thought about an overflow in seekg (since the number of bytes can be very big), but I saw seekg takes in input a streamoff type which is huge (billions of billions).
Changing the line
in.seekg(index*length*sizeof(float), in.beg);
into
in.seekg((streamoff)index*length*sizeof(float), in.beg);
solved the problem.