with statement seems to not close file correctly (a subprocess also uses the file)

A step that I am trying to achieve in my program (which is another auto grader, but in python) is to write a temporary txt file containing the output of an outside program (students' homework). I am able to successfully write the txt file using a with statement, but I am unable to then remove the file. To remove the txt file, I am using os.remove(file.txt) but I keep getting the following error:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'file.txt'

I am assuming that this is because the with statement is not closing the txt file correctly. In an attempt to forcibly close the file, I have tried to add .close() but I know that that is a bit redundant. I am also considering that it could be the subprocess.Popen() line but I have used this in other programs and I haven't had issues with it when I have tried to manipulate the same file (i.e. writing a txt file, then later reading it). But in order to test this, I tried to use kill() but I'm having trouble getting that to work correctly.

If anyone can give me insight as to what is going wrong and hint at what I need to do, I'd appreciate it!

hw_file = f"{submissions_folder}\\TaylorDavisOnTime.java" # example of naming style for student assignments
student_txt_file =  hw_file.replace(f"{submissions_folder}\\", f"{submissions_folder}\\output\\").replace(".java", "Output.txt")

with open(student_txt_file, "w") as file:
    output = subprocess.Popen(["java", hw_file], stdin=subprocess.PIPE, stdout=file, text=True)
    for input in input_values:
        output.stdin.write(input[0])
        output.stdin.write(str(input[1]))
        output.stdin.write(str(input[2]))  
    output.stdin.flush()
    output.stdin.close() 
    # output.kill() # still playing with this
    remove = True 
    file.close() # redundant, but does nothing

if remove: 
    os.remove(student_txt_file)

The subprocess still had the file opened which is why os.remove did not have permission to remove the file.

Waiting for the subprocess to finish by using output.wait()¹ releases the file and allows it to be removed.

¹_{suggested by Harsha}