Correct way to call ssl::stream::async_shutdown

Is it correct to call ssl::stream::async_shutdown while there are outstanding async_read_some/async_write operations? Should I wait for all async operations to complete before calling ssl::stream::async_shutdown or not?

If I can call ssl::stream::async_shutdown before async_read_some/async_write completes, what happens to the operations in progress?

SSL is a state-machine. Any stream-level read operation can require protocol-level (socket level, in your case) writes and vice versa.

The implementation of io_op::operator() in boost/asio/ssl/detail/io.hpp (which is used via async_io<Op>) provides some protection against overlapping read/writes by way of the pending_read_ and pending_write_ timers in stream_core. However, it ONLY manages implicit operations, not the user-initiated ones.

So you have to make sure any user-initiated writes or writes do not conflict (it's okay to have a single write and read pending at the same time).

It's not too hard to check the behavior, e.g. with BOOST_ASIO_ENABLE_HANDLER_TRACKING. Say we have the following set of deferred operations:

auto handshake = s.async_handshake(ssl::stream_base::client);
auto hello     = s.async_write_some(asio::buffer("Hello, world!\n"sv));
auto bye       = s.async_write_some(asio::buffer("Bye, world!\n"sv));
auto shutdown  = s.async_shutdown();

A classic, correct way to use them could be:

    handshake([&](auto&&...) {   //
        hello([&](auto&&...) {   //
            bye([&](auto&&...) { //
                shutdown(token);
            });
        });
    });

Equivalently:

    co_spawn(ioc, [&] -> asio::awaitable<void> {
            co_await handshake(asio::deferred);
            co_await hello(asio::deferred);
            co_await bye(asio::deferred);
            co_await shutdown(asio::deferred);
        }, token);

Visualized handlers:

Note that it's pretty tricky to read because the only the lowest_layer operations are showing. So, e.g. shutdown is a write and one ore more reads.

Now, if you go "rogue" instead:

    post(ioc, [&] { handshake(token); }); // 1

    post(ioc, [&] { hello(token); });     // 2
    // post(ioc, [&] { bye(token); });    // 3

    post(ioc, [&] { cancel(); });         // 4
    post(ioc, [&] { shutdown(token); });  // 5

First of all, things don't work as expected. Second of all, even a simple // 1 and // 5 combi shows:

It is at once clear that the two async_receive operations involved are overlapping. That's simply not allowed.

Notes

• the cancel was an idea to see whether cancel() before async_shutdown would work. After all, cancellation is documented to be supported.
• you should consider queueing the operations. See for example what I did here, but for websockets: WebSocket async_close fails with "Operation canceled" in destructor (Boost.Beast) ; note how it queues outbound messages as usual, but has extra flags to indicate whether a message is a close request. Note, ironically, that in recent Boost that is no longer required for Beast's websocket, but clearly the idea applies to ssl::stream still.

Listing

Live On Coliru

#define BOOST_ASIO_ENABLE_HANDLER_TRACKING
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>

using namespace std::literals;
namespace asio = boost::asio;
namespace ssl  = asio::ssl;
using asio::ip::tcp;

int main() {
    asio::io_context         ioc;
    ssl::context             ctx(ssl::context::sslv23);
    ssl::stream<tcp::socket> s(ioc, ctx);
    s.lowest_layer().connect({{}, 8989});

    asio::cancellation_signal signal;
    asio::cancellation_slot   slot   = signal.slot();
    auto                      token  = bind_cancellation_slot(slot, asio::detached);
    auto                      cancel = [&] { signal.emit(asio::cancellation_type::all); };
    // auto                   cancel = [&] { s.lowest_layer().cancel(); };

    // deferred ops
    auto handshake = s.async_handshake(ssl::stream_base::client);
    auto hello     = s.async_write_some(asio::buffer("Hello, world!\n"sv));
    auto bye       = s.async_write_some(asio::buffer("Bye, world!\n"sv));
    auto shutdown  = s.async_shutdown();

    if (0) {
        // properly serialize operations
#if 1
        handshake([&](auto&&...) {   //
            hello([&](auto&&...) {   //
                bye([&](auto&&...) { //
                    shutdown(token);
                });
            });
        });
#else
        co_spawn(ioc, [&] -> asio::awaitable<void> {
                co_await handshake(asio::deferred);
                co_await hello(asio::deferred);
                co_await bye(asio::deferred);
                co_await shutdown(asio::deferred);
            }, token);
#endif
    } else {
        // "rogue"
        post(ioc, [&] { handshake(token); }); // 1

        //post(ioc, [&] { hello(token); });     // 2
        //// post(ioc, [&] { bye(token); });    // 3

        //post(ioc, [&] { cancel(); });         // 4
        post(ioc, [&] { shutdown(token); });  // 5
    }

    ioc.run();
}

BONUS: Cancellation

From the comments, and for posterity: it is possible to use cancellation to give async_shutdown priority. You MUST still await completion of the cancelled operation(s):

Live On Coliru

#define BOOST_ASIO_ENABLE_HANDLER_TRACKING
#include <boost/asio.hpp>
#include <boost/asio/ssl.hpp>
#include <iostream>

using namespace std::literals;
namespace asio = boost::asio;
namespace ssl  = asio::ssl;
using asio::ip::tcp;
using boost::system::error_code;

int main() {
    std::cout << "Boost version " << BOOST_VERSION << std::endl;
    asio::thread_pool        ioc(1);
    ssl::context             ctx(ssl::context::sslv23);
    ssl::stream<tcp::socket> s(ioc, ctx);
    s.lowest_layer().connect({{}, 8989});

    asio::cancellation_signal signal;
    asio::cancellation_slot   slot   = signal.slot();
    auto                      token  = bind_cancellation_slot(slot, asio::detached);
    auto                      cancel = [&] { signal.emit(asio::cancellation_type::all); };
    // auto                   cancel = [&] { s.lowest_layer().cancel(); };

    // deferred ops
    auto handshake = s.async_handshake(ssl::stream_base::client, asio::deferred);
    auto hello     = s.async_write_some(asio::buffer("Hello, world!\n"sv), asio::deferred);
    auto bye       = s.async_write_some(asio::buffer("Bye, world!\n"sv), asio::deferred);
    auto shutdown  = s.async_shutdown(asio::deferred);

    if (0) {
        // properly serialize operations
#if 1
        handshake([&](auto&&...) {   //
            hello([&](auto&&...) {   //
                bye([&](auto&&...) { //
                    shutdown(token);
                });
            });
        });
#else
        co_spawn(ioc, [&] -> asio::awaitable<void> {
                co_await handshake(asio::deferred);
                co_await hello(asio::deferred);
                co_await bye(asio::deferred);
                co_await shutdown(asio::deferred);
            }, token);
#endif
    } else {
        handshake(asio::use_future).get(); // 1, simply blocking

        auto writes = hello(asio::deferred([&](error_code ec, size_t) {
            return !ec ? bye : throw boost::system::system_error(ec); // 2, 3
        }));

        auto timer          = asio::steady_timer(ioc, 100ms);
        auto delayed_writes = timer.async_wait(asio::deferred([&](error_code ec) { //
            return !ec ? writes : throw boost::system::system_error(ec);
        }));

        auto f = std::move(delayed_writes) //
            (bind_cancellation_slot(slot, asio::use_future));

        std::this_thread::sleep_for(150ms);
        post(ioc, [&] { cancel(); }); // 4

        // Crucially, wait for the cancellation to complete
        f.wait(); // 2, 3

#ifndef NDEBUG
        try { f.get(); throw boost::system::system_error({}); }
        catch (boost::system::system_error const& e) { std::cout << "Writes result: " << e.code().message() << std::endl; }
#endif

        post(ioc, [&] { shutdown(token); }); // 5
    }

    ioc.join();
}

Note that it's possible to vary the sleep_for. Also watch what happens when you replace std::move(delayed_writes) with std::move(writes) removing all delays. (In my experience, the writes always succeed in that case).

The live demo could be hard to read, so here is side by side: