The following code use to get http response message:
boost::beast::tcp_stream stream_;
boost::beast::flat_buffer buffer;
boost::beast::http::response<boost::beast::http::dynamic_body> res;
boost::beast::http::read(stream_, buffer, res);
However, In some cases, based on the preceding request, I can expect that the response message body will include large binary file.
Therefore, I’d like to read it directly to the filesystem and not through buffer
variable to avoid excessive use of process memory. How can it be done ?
in Objective-c framework NSUrlSession
there's an easy way to do it using NSURLSessionDownloadTask
instead of NSURLSessionDataTask
, so I wonder if it's also exist in boost.
Thanks !
In general, you can use the http::buffer_body
to handle arbitrarily large request/response messages.
If you specifically want to read/write from a filesystem file, you can have the http::file_body
instead.
buffer_body
The documentation sample for buffer_body
is here https://www.boost.org/doc/libs/1_77_0/libs/beast/doc/html/beast/using_http/parser_stream_operations/incremental_read.html.
Using it to write to std::cout: Live On Coliru
#include <boost/asio.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <iostream>
namespace net = boost::asio;
namespace beast = boost::beast;
namespace http = beast::http;
using tcp = net::ip::tcp;
using socket_t = tcp::socket;
/* This function reads a message using a fixed size buffer to hold
portions of the body, and prints the body contents to a `std::ostream`.
*/
template<
bool isRequest,
class SyncReadStream,
class DynamicBuffer>
void
read_and_print_body(
std::ostream& os,
SyncReadStream& stream,
DynamicBuffer& buffer,
beast::error_code& ec)
{
http::parser<isRequest, http::buffer_body> p;
http::read_header(stream, buffer, p, ec);
if(ec)
return;
while(! p.is_done())
{
char buf[512];
p.get().body().data = buf;
p.get().body().size = sizeof(buf);
http::read(stream, buffer, p, ec);
if(ec == http::error::need_buffer)
ec = {};
if(ec)
return;
os.write(buf, sizeof(buf) - p.get().body().size);
}
}
int main() {
std::string host = "173.203.57.63"; // COLIRU 20210901
auto const port = "80";
net::io_context ioc;
tcp::resolver resolver{ioc};
socket_t s{ioc};
net::connect(s, resolver.resolve(host, port));
write(s, http::request<http::empty_body>{http::verb::get, "/", 11});
beast::error_code ec;
beast::flat_buffer buf;
read_and_print_body<false>(std::cout, s, buf, ec);
}
file_body
exampleThis is much shorter, writing to body.html
:
#include <boost/asio.hpp>
#include <boost/asio/ip/tcp.hpp>
#include <boost/beast.hpp>
#include <boost/beast/websocket.hpp>
#include <iostream>
namespace net = boost::asio;
namespace beast = boost::beast;
namespace http = beast::http;
using tcp = net::ip::tcp;
using socket_t = tcp::socket;
int main() {
std::string host = "173.203.57.63"; // COLIRU 20210901
auto const port = "80";
net::io_context ioc;
tcp::resolver resolver{ioc};
socket_t s{ioc};
net::connect(s, resolver.resolve(host, port));
write(s, http::request<http::empty_body>{http::verb::get, "/", 11});
beast::error_code ec;
beast::flat_buffer buf;
http::response<http::file_body> res;
res.body().open("body.html", beast::file_mode::write_new, ec);
if (!ec.failed())
{
read(s, buf, res, ec);
}
std::cout << "Wrote 'body.html' (" << ec.message() << ")\n";
std::cout << "Headers " << res.base() << "\n";
}
Prints
Wrote 'body.html' (Success)
Headers HTTP/1.1 200 OK
Content-Type: text/html;charset=utf-8
Content-Length: 8616
Server: WEBrick/1.4.2 (Ruby/2.5.1/2018-03-29) OpenSSL/1.0.2g
Date: Wed, 01 Sep 2021 19:52:20 GMT
Connection: Keep-Alive
With file body.html; wc body.html
showing:
body.html: HTML document, ASCII text, with very long lines
185 644 8616 body.html
I have an advanced example of that here: How to read data from Internet using muli-threading with connecting only once?.