How can I read at the position relative to end of the file via IoUring? I have a file that contains the metadata in the end of the file because of single pass writing. Now, I want to read the file via IoUring for efficiency, therefore I have to read the metadata in the first step.
Read it via blocking IO is simple, we can use lseek
with SEEK_END
to seek to the position(file.seek(SeekFrom::End(-meta_siza))
in rust) and then read from it. However, after some search, I found IoUring does not have an operation named seek. So maybe we can use the pos = file.read_at(file_size - meta_size)
to achieve it. But how can we get the file size? Using stat syscall may cause blocking...
This example was for me the occasion to discover io-uring
.
One solution is to call statx()
with AT_FDCWD
as file-descriptor in order to state that the path is relative to the current directory.
The path provided to this call is supposed to be null-terminated, hence the conversion into CString
.
Another solution is to call statx()
with an already open file-descriptor, an empty null-terminated string as filename and the AT_EMPTY_PATH
flag.
Note that this example is not representative of a normal usage because it does not take any advantage of the asynchronous nature of the API; we submit and wait for every single operation in a synchronous manner.
// io-uring = ">=0"
// libc = ">=0"
use std::os::unix::io::AsRawFd;
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;
fn file_size_by_name(
ring: &mut io_uring::IoUring,
filename: &str,
) -> Result<usize> {
let user_data = 0xdead;
let filename = std::ffi::CString::new(filename)?;
let mut statxbuf = std::mem::MaybeUninit::<libc::statx>::zeroed();
let entry = io_uring::opcode::Statx::new(
io_uring::types::Fd(libc::AT_FDCWD),
filename.as_ptr(),
statxbuf.as_mut_ptr() as *mut _,
)
.build()
.user_data(user_data);
unsafe {
ring.submission().push(&entry)?;
}
ring.submit_and_wait(1)?;
let cqe = ring.completion().next().ok_or("no statx entry")?;
if cqe.user_data() != user_data {
Err("invalid user_data")?
} else if cqe.result() < 0 {
Err(format!("statx error: {}", cqe.result()))?
} else {
Ok(unsafe { statxbuf.assume_init_ref() }.stx_size as usize)
}
}
fn file_size(
ring: &mut io_uring::IoUring,
fd: &std::fs::File,
) -> Result<usize> {
let user_data = 0xdead;
let mut statxbuf = std::mem::MaybeUninit::<libc::statx>::zeroed();
let entry = io_uring::opcode::Statx::new(
io_uring::types::Fd(fd.as_raw_fd()),
[0i8].as_ptr(),
statxbuf.as_mut_ptr() as *mut _,
)
.flags(libc::AT_EMPTY_PATH)
.build()
.user_data(user_data);
unsafe {
ring.submission().push(&entry)?;
}
ring.submit_and_wait(1)?;
let cqe = ring.completion().next().ok_or("no statx entry")?;
if cqe.user_data() != user_data {
Err("invalid user_data")?
} else if cqe.result() < 0 {
Err(format!("statx error: {}", cqe.result()))?
} else {
Ok(unsafe { statxbuf.assume_init_ref() }.stx_size as usize)
}
}
fn read_bytes(
ring: &mut io_uring::IoUring,
fd: &std::fs::File,
offset: usize,
buffer: &mut [u8],
) -> Result<usize> {
let user_data = 0xbeef;
let entry = io_uring::opcode::Read::new(
io_uring::types::Fd(fd.as_raw_fd()),
buffer.as_mut_ptr(),
buffer.len() as _,
)
.offset64(offset as _)
.build()
.user_data(user_data);
unsafe {
ring.submission().push(&entry)?;
}
ring.submit_and_wait(1)?;
let cqe = ring.completion().next().ok_or("no read entry")?;
if cqe.user_data() != user_data {
Err("invalid user_data")?
} else if cqe.result() < 0 {
Err(format!("read error: {}", cqe.result()))?
} else {
Ok(cqe.result() as usize)
}
}
fn main() -> Result<()> {
let mut ring = io_uring::IoUring::new(8)?;
let filename = "Cargo.lock";
let fd = std::fs::File::open(filename)?;
let size = file_size(&mut ring, &fd)?;
println!("{:?} contains {} bytes", filename, size);
println!(
"by name: {:?} contains {} bytes",
filename,
file_size_by_name(&mut ring, filename)?
);
let mut meta = [0; 10];
let amount = read_bytes(&mut ring, &fd, size - meta.len(), &mut meta)?;
println!("last {} bytes: {:?}", amount, &meta[..amount]);
Ok(())
}
/*
"Cargo.lock" contains 811 bytes
by name: "Cargo.lock" contains 811 bytes
last 10 bytes: [34, 108, 105, 98, 99, 34, 44, 10, 93, 10]
*/