c++network-programmingboostboost-asioboost-beast

How to read trailer headers in boost::beast::http


As I understand it, in HTTP 1.1 in case the response comes in the form of Transfer-Encoding: chunked, the response may also contain trailers (additional headers) after the body.

I have two questions about this in boost::beast context:

  1. When reading the response using the functions boost::beast::http::read or boost::beast::http::async_read, will the trailers also be read from the network buffer?
  2. How to access the trailers, after read/async_read if they exist?

Solution

  • Reading trailers is uncommon. MDN:

    Warning: Developers cannot access HTTP trailers via the Fetch API or XHR. Additionally, browsers ignore HTTP trailers, with the exception of Server-Timing. See Browser compatibility for more information.

    That said, Beast message parsers appear to simply treat trailers as regular fields:

    case state::fields:
        parse_fields(p, n, ec);
        if(ec)
            goto done;
    
    // ...
    case state::trailer_fields:
        parse_fields(p, n, ec);
        if(ec)
            goto done;
    

    Live demo:

    #include <boost/asio.hpp>
    #include <boost/beast.hpp>
    #include <iostream>
    #include <thread>
    
    namespace beast = boost::beast;
    namespace http  = beast::http;
    namespace net   = boost::asio;
    using tcp       = net::ip::tcp;
    
    static std::string_view constexpr g_request = "GET / HTTP/1.1\r\n"
                                                  "Host: localhost:7878\r\n"
                                                  "TE: trailers\r\n"
                                                  "Connection: close\r\n"
                                                  "\r\n";
    
    static std::string_view constexpr g_response = "HTTP/1.1 200 OK\r\n"
                                                   "Trailer: SeheStackoverflow\r\n"
                                                   "Transfer-Encoding: chunked\r\n"
                                                   "\r\n"
                                                   "4\r\n"
                                                   "Wiki\r\n"
                                                   "7\r\n"
                                                   "pedia i\r\n"
                                                   "B\r\n"
                                                   "n \r\nchunks.\r\n"
                                                   "0\r\n"
                                                   "SeheStackoverflow: was here\r\n"
                                                   "\r\n";
    
    static void dummy_server() {
        net::io_context ioc;
        tcp::acceptor   acceptor(ioc, {{}, 7878});
        auto            s = acceptor.accept();
    
        http::request<http::empty_body> req;
        beast::flat_buffer buffer;
        http::read(s, buffer, req);
    
        write(s, net::buffer(g_response));
    }
    
    int main() try {
        std::jthread srv(dummy_server);
        std::this_thread::sleep_for(std::chrono::seconds(1));
    
        net::io_context   ioc;
        beast::tcp_stream stream(ioc);
        stream.connect({{}, 7878});
    
        // send request
        write(stream, net::buffer(g_request));
    
        std::cout << "Request written" << std::endl;
    
        // read response
        beast::flat_buffer buffer;
        http::response_parser<http::string_body> parser;
        auto cb = [](std::uint64_t n, beast::string_view extensions, beast::error_code& ec) {
            std::cout << "Chunk header received: <" << n << "> bytes, extensions: <" << extensions << ">, ec: <"
                      << ec.message() << ">\n";
        };
        parser.on_chunk_header(cb);
        http::read(stream, buffer, parser);
    
        auto res = parser.release();
        std::cout << "\n --- Response Headers: " << res.base() << std::endl;
        std::cout << "\n --- Response body: " << res.body() << std::endl;
        std::cout << "\n --- Response status: " << res.result_int() << " " << res.reason()
                  << " parser.is_done() = " << parser.is_done() << "\n";
    
        //for (auto const& field : res)
            //std::cout << field.name_string() << ": " << field.value() << "\n";
    } catch (std::exception const& e) {
        std::cerr << "Error: " << e.what() << "\n";
        return 1;
    }
    

    Prints

    Request written
    Chunk header received: <4> bytes, extensions: <>, ec: <Success>
    Chunk header received: <7> bytes, extensions: <>, ec: <Success>
    Chunk header received: <11> bytes, extensions: <>, ec: <Success>
    Chunk header received: <0> bytes, extensions: <>, ec: <Success>
    
     --- Response Headers: HTTP/1.1 200 OK
    Trailer: SeheStackoverflow
    Transfer-Encoding: chunked
    SeheStackoverflow: was here
    
    
    
     --- Response body: Wikipedia in 
    chunks.
    
     --- Response status: 200 OK parser.is_done() = 1
    

    Recap

    1. yes
    2. by accessing the header fields as normal