Flaky tests when using minitest to check file creation

Can anyone point as to why my tests are being flaky (locally, and on circleCI) when testing a file creation / ActiveStorage?

In development, this code works fine, but I suspect theres some weird race condition or something in the tests.

I'm testing the below create_my_file method as a model test and controller test (various places), and it often fails when I check if the new file has been attached to my model.

As below, I use MyFileGenerator to create the file each time in our tmp folder, and we then use this file to attach and upload to S3. Then when we create another file, we first delete the tmp folder and recreate, to avoid files building up.

  def create_my_file
    if my_file.attached?
      my_file.purge
    end
    MyFileGenerator.new.create(my_file_data, id)
    my_file.attach(
      io: File.open(
        "#{Rails.root}/tmp/my_file/my_file_#{id}.docx"
      ),
      filename: "my_file_#{id}_#{Time.now.to_i}.docx",
      content_type: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document'
    )
  end

MyFileGenerator.new.create

def create(data, meeting_id)
  doc = Docx::Document.open("#{Rails.root}/app/lib/docx/docx_templates/my_file).docx")

  populate_document(doc, data)

  # Save the new file
  make_tmp_directory
  doc.save("#{Rails.root}/tmp/my_file/my_file_#{meeting_id}.docx")
end

private

def make_tmp_directory
  # We need to store the file somewhere before uploading to S3
  dir = "#{Rails.root}/tmp/my_file/"
  # Clear existing files if exists, then make new dir
  FileUtils.remove_dir(dir, true) if File.exist?(dir)
  Dir.mkdir(dir)
end

Any suggestions to improve this, or pointers as to where the issue may be would be great.

Tests that fail locally and in isolation do so because something about their environment changes between test runs. This is often due to the test relying on a nondeterministic part of the system: the system clock, a random number, or a network connection. The solution is to always mock those nondeterministic dependencies. (I answered a similar question in a lot more detail, here.)

In your specific case, writing to the disk can take a non-deterministic amount of time. So, it's possible that the file is not yet fully persisted when your test asserts that the file is attached. If that's the case, then that sort of implies that there's some threading happening (otherwise the test would be blocked by the file I/O). So, if possible, I'd look for a way to turn off the threading and force the file to be written synchronously to prevent the race condition between the I/O and the test execution.

To confirm that's what's happening, I would try to locate the file on disk before the assertion that it's attached. You could then either wait for the file to be present (not my preference), or you could force the I/O to be synchronous (if possible).

Quick Edit

It does appear that file I/O in Ruby is asynchronous by default. Here's the docs.