c++network-programmingboostboost-asioboost-beast

Correct way to read HTTP responses without body


I have a program through which the user can send http requests. As you know, not every HTTP request assumes a response body. Responses to some requests may contain only headers (for example, a response to a HEAD request)

When I try to read the server response headers via boost::beast::http::read_header, after this function completes, http::response_parser<http::string_body>::is_done returns false if the request was sent via HEAD method but I expect the true because response does not contain a body.

Why does is_done return false in this case and what should be the solution so that is_done always returns true for responses that do not require a response body?

As a solution, I suppose that using the condition if ((!parser.content_length() || (parser.content_length().value() == 0)) && !parser.chunked()) instead of if (parser.is_done()) would work. This solution assumes that if there is no Content-Length or Transfer-Encoding: chunked headers, then the response does not contain a body, but I am not sure if this is the right solution.

Here is my code:

#include <boost/beast/core.hpp>
#include <boost/beast/http.hpp>
#include <boost/beast/ssl.hpp>
#include <boost/asio/connect.hpp>
#include <boost/asio/ssl/error.hpp>
#include <boost/asio/ssl/stream.hpp>
#include <iostream>

namespace beast = boost::beast;
namespace http = beast::http;
namespace net = boost::asio;
namespace ssl = net::ssl;
using tcp = net::ip::tcp;

int main() {
    try {
        const std::string host = "stackoverflow.com";
        const std::string port = "443";
        const std::string target = "/questions";
        int version = 11; // HTTP/1.1

        // IO + SSL
        net::io_context ioc;
        ssl::context ctx(boost::asio::ssl::context::tls_client);
        ctx.set_options(ssl::context::default_workarounds | boost::asio::ssl::context::no_tlsv1);

        beast::ssl_stream<beast::tcp_stream> stream(ioc, ctx);

        tcp::resolver resolver(ioc);
        auto const results = resolver.resolve(host, port);
        beast::get_lowest_layer(stream).connect(results);

        if (!SSL_set_tlsext_host_name(stream.native_handle(), host.c_str())) {
            throw boost::system::system_error(::ERR_get_error(), boost::asio::error::get_ssl_category());
        }

        // SSL Handshake
        stream.handshake(ssl::stream_base::client);

        // HEAD request
        http::request<http::empty_body> req{ http::verb::head, target, version };
        req.set(http::field::host, host);
        req.set(http::field::user_agent, "Boost::Beast");


        // send request
        http::write(stream, req);

        std::cout << "Request written" << std::endl;

        // read response
        beast::flat_buffer buffer;
        http::response_parser<http::string_body> parser;
        http::read_header(stream, buffer, parser);

        const auto& res = parser.get();
        std::cout << "Response status: " << res.result_int() << " " << res.reason() << " parser.is_done() = " << parser.is_done() << "\n";

        for (const auto& field : res) {
            std::cout << field.name_string() << ": " << field.value() << "\n";
        }

        beast::error_code ec;
        stream.shutdown(ec);
        if (ec == net::error::eof || ec == ssl::error::stream_truncated) {
            ec = {};
        }
        if (ec)
            throw beast::system_error{ ec };

    }
    catch (const std::exception& e) {
        std::cerr << "Error: " << e.what() << "\n";
        return 1;
    }

    return 0;
}

Solution

  • The documentation for is_done documents what you need to do:

    Returns true if the message is complete. Synopsis

    bool is_done() const;

    Description

    The message is complete after the full header is prduced [sic] and one of the following is true:

    • The skip body option was set.
    • The semantics of the message indicate there is no body.
    • The semantics of the message indicate a body is expected, and the entire body was parsed.

    The skip option:

    This option controls whether or not the parser expects to see an HTTP body, regardless of the presence or absence of certain fields such as Content-Length or a chunked Transfer-Encoding. Depending on the request, some responses do not carry a body. For example, a 200 response to a CONNECT request from a tunneling proxy, or a response to a HEAD request. In these cases, callers may use this function inform the parser that no body is expected. The parser will consider the message complete after the header has been received.

    So, simply add the option:

        parser.skip(true);
        http::read_header(stream, buffer, parser);
    

    Demo

    #include <boost/asio.hpp>
    #include <boost/asio/ssl.hpp>
    #include <boost/beast.hpp>
    #include <boost/beast/ssl.hpp>
    #include <iostream>
    
    namespace beast = boost::beast;
    namespace http = beast::http;
    namespace net = boost::asio;
    namespace ssl = net::ssl;
    using tcp = net::ip::tcp;
    
    int main() {
        try {
            const std::string host = "stackoverflow.com";
            const std::string port = "443";
            const std::string target = "/questions";
            int version = 11; // HTTP/1.1
    
            // IO + SSL
            net::io_context ioc;
            ssl::context ctx(ssl::context::tls_client);
            ctx.set_options(ssl::context::default_workarounds | ssl::context::no_tlsv1);
    
            beast::ssl_stream<beast::tcp_stream> stream(ioc, ctx);
    
            tcp::resolver resolver(ioc);
            auto const results = resolver.resolve(host, port);
            beast::get_lowest_layer(stream).connect(results);
    
            if (!SSL_set_tlsext_host_name(stream.native_handle(), host.c_str())) {
                throw beast::system_error(::ERR_get_error(), boost::asio::error::get_ssl_category());
            }
    
            // SSL Handshake
            stream.handshake(ssl::stream_base::client);
    
            // HEAD request
            http::request<http::empty_body> req{ http::verb::head, target, version };
            req.set(http::field::host, host);
            req.set(http::field::user_agent, "Boost::Beast");
    
    
            // send request
            http::write(stream, req);
    
            std::cout << "Request written" << std::endl;
    
            // read response
            beast::flat_buffer buffer;
            http::response_parser<http::string_body> parser;
            parser.skip(true);
            http::read_header(stream, buffer, parser);
    
            const auto& res = parser.get();
            std::cout << "Response status: " << res.result_int() << " " << res.reason() << " parser.is_done() = " << parser.is_done() << "\n";
    
            for (const auto& field : res) {
                std::cout << field.name_string() << ": " << field.value() << "\n";
            }
    
            beast::error_code ec;
            stream.shutdown(ec);
            if (ec == net::error::eof || ec == ssl::error::stream_truncated) {
                ec = {};
            }
            if (ec)
                throw beast::system_error{ ec };
        }
        catch (const std::exception& e) {
            std::cerr << "Error: " << e.what() << "\n";
            return 1;
        }
    }
    

    Local output:

    Request written
    Response status: 200 OK parser.is_done() = 1
    Date: Wed, 04 Jun 2025 22:33:33 GMT
    Content-Type: text/html; charset=utf-8
    Connection: keep-alive
    CF-Ray: 94aae47decc7d5a5-AMS
    CF-Cache-Status: DYNAMIC
    Cache-Control: private
    Set-Cookie: prov=16b31aa3-9095-4c5b-bedc-109a265fa8c9; expires=Thu, 04 Jun 2026 22:33:33 GMT; domain=.stackoverflow.com; path=/; secure; samesite=none; httponly
    Set-Cookie: __cflb=02DiuFA7zZL3enAQJD3AX8ZzvyzLcaG7vRdiMzUkh67pY; SameSite=Lax; path=/; expires=Thu, 05-Jun-25 21:33:33 GMT; HttpOnly
    Set-Cookie: prov=16b31aa3-9095-4c5b-bedc-109a265fa8c9; Path=/; HttpOnly; Domain=stackoverflow.com
    Set-Cookie: __cf_bm=TRszUOMbYSy4E812p.wxoXyegDDPKXpY36NAVjmif4A-1749076413-1.0.1.1-_nb75QUapeCAElVsVzaAwqLi7ffgOtBvVtG00jxCTiaN0YWci719GoX6i9iVxuAvaldoYLTNOV_21N7cjxQgd2RDvAWLBTHAWazvrduaijs; path=/; expires=Wed, 04-Jun-25 23:03:33 GMT; domain=.stackoverflow.com; HttpOnly; Secure; SameSite=None
    Set-Cookie: _cfuvid=FWMbYifpOya__5_Xam.Njba.GGB1j59GV7nuwqO5z8I-1749076413306-0.0.1.1-604800000; path=/; domain=.stackoverflow.com; HttpOnly; Secure; SameSite=None
    Strict-Transport-Security: max-age=31536000; includeSubDomains
    Vary: Accept-Encoding
    content-security-policy: upgrade-insecure-requests; frame-ancestors 'self' https://stackexchange.com
    feature-policy: microphone 'none'; speaker 'none'
    x-frame-options: SAMEORIGIN
    x-request-guid: 57b510d3-f551-453a-a103-bff0619fcc87
    x-worker-origin-response-time: 189000000
    X-DNS-Prefetch-Control: off
    Server: cloudflare
    

    Chunked Encoding

    Live Demo

    #include <boost/asio.hpp>
    #include <boost/beast.hpp>
    #include <iostream>
    
    namespace beast = boost::beast;
    namespace http = beast::http;
    namespace net = boost::asio;
    using tcp = net::ip::tcp;
    
    int main() {
        try {
            const std::string host = "www.google.com";
            const std::string port = "80";
            const std::string target = "/";
            int version = 11; // HTTP/1.1
    
            // IO
            net::io_context ioc;
    
            beast::tcp_stream stream(ioc);
    
            tcp::resolver resolver(ioc);
            auto const results = resolver.resolve(host, port);
            stream.connect(results);
    
            // HEAD request
            http::request<http::empty_body> req{ http::verb::head, target, version };
            req.set(http::field::host, host);
            req.set(http::field::user_agent, "Boost::Beast");
    
    
            // send request
            http::write(stream, req);
    
            std::cout << "Request written" << std::endl;
    
            // read response
            beast::flat_buffer buffer;
            http::response_parser<http::string_body> parser;
            parser.skip(true);
            http::read_header(stream, buffer, parser);
    
            const auto& res = parser.get();
            std::cout << "Response status: " << res.result_int() << " " << res.reason() << " parser.is_done() = " << parser.is_done() << "\n";
    
            for (auto const& field : res)
                std::cout << field.name_string() << ": " << field.value() << "\n";
        }
        catch (const std::exception& e) {
            std::cerr << "Error: " << e.what() << "\n";
            return 1;
        }
    }
    

    Local output:

    sehe@workstation:~/Projects/stackoverflow$ ./build/sotest
    Request written
    Response status: 200 OK parser.is_done() = 1
    Content-Type: text/html; charset=ISO-8859-1
    Content-Security-Policy-Report-Only: object-src 'none';base-uri 'self';script-src 'nonce-As03-xfkid5RnejQYMMtHw' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
    Date: Thu, 05 Jun 2025 10:00:21 GMT
    Server: gws
    X-XSS-Protection: 0
    X-Frame-Options: SAMEORIGIN
    Transfer-Encoding: chunked
    Expires: Thu, 05 Jun 2025 10:00:21 GMT
    Cache-Control: private
    Set-Cookie: AEC=AVh_V2h0HKJS7itZy7RPJpCaPZsOTd0n4-1RR6cfzUVEkpoa6SVrgd-D4f0; expires=Tue, 02-Dec-2025 10:00:21 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax