Rust, pcap and tokio

I may be taking the completely wrong approach to this, but I have a rust program that is leveraging the pcap crate to read in packets.

in the main thread, I have the following code:

let packet = match active_capture.next_packet() {
                Ok(received) => received,
                Err(error) => { // ignore errors for example...
                    continue;
                }
            };

I am spawning several tokio threads:

tokio::spawn(async move {
    // do magic stuff here
    let packet_data = bytes_from_magic;
    active_capture.sendpacket(packet_data).expect("Error sending packet.");
});

I've gone through a few iterations of solutions to make this work. I wrapped the active_capture in an ArcMutex, but under heavy network loading, the performance was not ideal. Trying to wrap this with a RefCell threw errors, as Rc didn't implement the "Send" function required by tokio.

My understanding is that pcap is effectively thread-safe for my use-case, so I'm wanting to avoid the ArcMutex altogether, but nothing I've tried has worked, yet.

I'm sending layer 2 packets, which is why I'm using pcap. If anyone has any suggestions, I'd be most grateful.

There are 2 ways to write concurrent code in rust: sync and async. With sync concurrency, you run potentially blocking code on OS threads. With async concurrency, you run code that returns futures instead of blocking on an async runtime (tokio is the most popular async runtime for rust).

I can't say for sure without seeing your code, but your program is likely slow because you're running a blocking operation on a tokio coroutine. The tokio scheduler is cooperative, if a coroutine blocks it can hog one of tokios threads indefinitely (in a single-threaded runtime, this means no other coroutines can run).

An easy fix would be to run your coroutine with tokio::spawn_blocking. This tells tokio to use a separate thread for your coroutine, leaving the other coroutines undisturbed. But...

I would reconsider if async concurrency is the right choice for you. Async programming was first introduced to solve the C10k problem -- multiplexing many short lived work items over some number of threads. Tokio does this very well, but it seems to me like all your IO happens via blocking operations on a single file descriptor. Writing a sync program based on OS threads sounds like a better choice here.