multithreadingfilerust

Is writing to the same file from multiple threads thread safe?


This is an example program. As of now the content of the file is mixed input data. If it isn't thread safe, is there a thread safe way to do this using built-in modules or any external crate, instead of me handling the exclusivity?

use std::io::{BufWriter, Write};
use std::path::PathBuf;
use std::fs::{File, OpenOptions};
use std::thread;

fn write_to_file(file_path: PathBuf) {
    let file = OpenOptions::new()
        .write(true)
        .open(&file_path)
        .unwrap();
    let mut buf_writer = BufWriter::new(&file);

    println!("{:?}", thread::current().id());
    let file_str = format!("{:?}: {:?}\n", &file_path, thread::current().id());
    for _ in 0..1000000 {
        buf_writer.write(file_str.as_bytes()).expect(&format!("unable to write to file {}", &file_str));
    }
    buf_writer.flush().expect("unable to flush the file");
}

fn create_file(file_path: PathBuf) {
    File::create(file_path).expect("create failed");
}

fn main() {
    let file_path = PathBuf::from("/home/harry/a.txt");
    create_file(file_path.clone());

    let file_path1 = file_path.clone();
    let file_path2 = file_path.clone();

    let jh1 = thread::spawn(move || write_to_file(file_path1));
    let jh2 = thread::spawn(move || write_to_file(file_path2));

    let _ = jh1.join();
    let _ = jh2.join();
}

Solution

  • Given your "last thread wins" thread-safety requirement, I'd recommend writing to a temporary file and renaming the file to the final location. That is simple to implement and has several advantages:

    For example:

    fn write_to_file(file_path: &Path) -> io::Result<()> {
        let id = format!("{:?}", std::thread::current().id());
        let mut tmp_path = file_path.to_owned();
        tmp_path.as_mut_os_string().push(format!(".new-{}", id));
        let file = File::create(&tmp_path)?;
        // writing to file can take as long as necessary, as each thread
        // writes to its own scratch-pad
        let mut buf_writer = BufWriter::new(file);
        let file_str = format!("{:?}: {}\n", file_path, id);
        for _ in 0..1000000 {
            buf_writer.write_all(file_str.as_bytes())?;
        }
        buf_writer.flush()?;
        std::fs::rename(tmp_path, file_path)?; // last thread to get here wins
        Ok(())
    }
    

    Playground

    Note that I modified the code to write with write_all() rather than write(), because write() can write out less data than was given to it. It also returns a result rather than panicking.