c++networkingboostboost-asiotun-tap

How can I read from a TAP device (via posix::stream_descriptor) simultaneously with Boost.Asio?

My program should simultaneously read packets from a generated TAP device and process them. For this I use the tuntap library from LaKabane together with Boost.Asio's posix::stream_descriptor. However, since I am acting as a client and not a server, there is no option to accept packets asynchronously. The proviorical solution I have chosen is to read asynchronously again and again. However, there are two major problems with this:

  1. There could be a stack overflow, as the same function is called "infinitely" often.
  2. The function does not accept the packets fast enough. I have tested this with sudo ping -f ff02::1%test.

The following is my code so far:

#include <iostream>
#include <cstdlib>
#include <boost/asio.hpp>
#include <unistd.h>
#include "tun_tap.hpp"

void handle_packet([[maybe_unused]] const boost::system::error_code& error, [[maybe_unused]] std::size_t bytes_transferred, [[maybe_unused]] const std::array<char, 1520>& buffer)
{
    if (error)
    {
        std::clog << "Error in handle_packet: " << error.message() << std::endl;
        return;
    }
    std::clog << "Received packet of size: " << bytes_transferred << std::endl;
    std::clog << std::flush;

    // To something with the packet
    sleep(5);
}

void start(boost::asio::posix::stream_descriptor& tap_device)
{
    std::array<char, 1520> buffer;

    tap_device.async_read_some(boost::asio::buffer(buffer),
                                  [&](const boost::system::error_code& error, std::size_t bytes_transferred) {
                                      start(tap_device);
                                      handle_packet(error, bytes_transferred, buffer);
                                  });
}

int main() {
    try {
        boost::asio::io_context io;
        const ::size_t mtu = 1500;
        std::clog << "Create TUN device." << std::endl;
        tun_tap dev = tun_tap("test", tun_tap_mode::tap);
        std::clog << "Set MTU to " << mtu << "." << std::endl;
        dev.set_mtu(1500);
        std::clog << "Set the TUN device up." << std::endl;
        dev.up();

        boost::asio::posix::stream_descriptor tap_device(io, ::dup(dev.native_handler()));
        start(tap_device);


        io.run();
    } catch (const std::exception &e) {
        std::cerr << "Error: " << e.what() << std::endl << "Exit program.";
        ::exit(EXIT_FAILURE);
    }
    return EXIT_SUCCESS;
}

(Full source)

My question now is, how can I read from the TAP device with Boost.Asio without losing packets?

Solution

  • Your problems are not primarily related to the device specifics. They are about C++ and object lifetime in relation to asynchronous operations.

    std::array<char, 1520> buffer;

tap_device.async_read_some( //
    asio::buffer(buffer), [&](boost::system::error_code const& error, std::size_t bytes_transferred) {
        start(tap_device);
        handle_packet(error, bytes_transferred, buffer);
    });

    Firstly, that's not recursion. The completion handler is a continuation. It's by definition called when the async operation completed. So the new start never overlaps. Also, it doesn't execute in the stack frame of the initiating function. Instead it executes on a service thread.

    However, that's also the problem here. buffer is a local variable. It's lifetime ends immediately after async_read_some is initiated, so by definition before it completes. So, when it is used inside the completion handler it has become invalid.

    So, since the code is broken, let's ignore the flawed observations about speed. First, let's fix it. There are several ways, but this one seems to me to be most instructive/extensible:

    #include "tun_tap.hpp"
#include <boost/asio.hpp>
#include <cstdlib>
#include <iostream>
#include <unistd.h>
namespace asio = boost::asio;
using boost::asio::posix::stream_descriptor;
using boost::system::error_code;

struct Client {
    using Handler = std::function<void(error_code, std::string)>;
    Client(asio::any_io_executor ex, tun_tap& dev, Handler handler = default_handler)
        : tap_device_(ex, ::dup(dev.native_handle()))
        , handler_(std::move(handler)) {
        start();
    }

  private:
    stream_descriptor      tap_device_;
    Handler                handler_;
    std::array<char, 1520> buffer_{};

    void start() {
        tap_device_.async_read_some( //
            asio::buffer(buffer_),   //
            [this](error_code ec, size_t bytes_transferred) {
                if (ec) {
                    std::cerr << "Error: " << ec.message() << std::endl;
                } else {
                    std::string packet(buffer_.data(), bytes_transferred);
                    start();

                    if (handler_)
                        handler_(ec, std::move(packet));
                };
            });
    }

    static void default_handler(error_code ec, std::string const& packet) {
        if (ec) {
            std::cerr << "Error: " << ec.message() << std::endl;
        } else {
            std::cout << "Received packet: " << packet.size() << " bytes" << std::endl;
        }
    }
};

int main() try {
    constexpr uint16_t mtu = 1500;
    asio::io_context io;

    tun_tap dev{"test", tun_tap_mode::tap};
    dev.set_mtu(mtu);
    dev.up();

    Client c(io.get_executor(), dev);

    io.run();
} catch (std::exception const& e) {
    std::cerr << "Error: " << e.what() << std::endl;
    return 1;
}

    I also reviewed tun_tap.hpp/cpp fixing some issues and the leak:

    It works quite nicely here:

    enter image description here