I have a class which reads a CSV file but when size of file is high, the program throws Java heap size
error, so I need to split that file into pieces and transfer lines to other files according to line size.
For example; I have a file of 500 000 lines and I'm dividing it into 5 files by 100 000 lines. So I have 5 files consisting of 100 000 lines so that I can read them.
I couldn't find a way to do that so it would be nice if I see example lines of code.
public static void splitLargeFile(final String fileName,
final String extension,
final int maxLines,
final boolean deleteOriginalFile) {
try (Scanner s = new Scanner(new FileReader(String.format("%s.%s", fileName, extension)))) {
int file = 0;
int cnt = 0;
BufferedWriter writer = new BufferedWriter(new FileWriter(String.format("%s_%d.%s", fileName, file, extension)));
while (s.hasNext()) {
writer.write(s.next() + System.lineSeparator());
if (++cnt == maxLines && s.hasNext()) {
writer.close();
writer = new BufferedWriter(new FileWriter(String.format("%s_%d.%s", fileName, ++file, extension)));
cnt = 0;
}
}
writer.close();
} catch (Exception e) {
e.printStackTrace();
}
if (deleteOriginalFile) {
try {
File f = new File(String.format("%s.%s", fileName, extension));
f.delete();
} catch (Exception e) {
e.printStackTrace();
}
}
}