pythonfilesplit

Python Split files into multiple smaller files


I have this problem statement

Write a function named file_split(filename, number_of_files) that will split an input file into a number of output files. The files should be split as evenly as possible. When the file length is evenly divisible by the number of files to create (a 10-line file, split into 2 files, each output file should have 5 lines. When the length is not evenly divisible all output files’ length must not have a difference greater than 1. For example, a file of 10 lines, split in 3, would have output files of length 3, 3 and 4.

I have written my code but I can not figure out how to do the difference greater than 1 part, I need help modifying my code to include that part. (The code I have creates a new file for the last line if it is not an even multiple)

def get_line_counts(filename, number_of_files):
    try:
        my_file = open(filename, 'r')
    except IOError:
        print("File does not exist")
        return    
    input = my_file.read().split('\n')
    outputBase = 'lel'    
    total_lines = 0
    with open('myfile.txt') as infp:
        for line in infp:
            if line.strip():  
                total_lines +=1    
    base_size = total_lines // number_of_files    
    at = 1
    for lines in range(0, len(input), base_size):
        outputData = input[lines:lines+base_size]
        output = open(outputBase + str(at) + '.txt', 'w')
        output.write('\n'.join(outputData))
        output.close()
        at += 1

Solution

  • Round-robin works and is easy:

    with open('myfile.txt') as infp:
        files = [open('%d.txt' % i, 'w') for i in range(number_of_files)]
        for i, line in enumerate(infp):
            files[i % number_of_files].write(line)
        for f in files:
            f.close()