qtqtnetworkqtconcurrentqt5.4

How does QtConcurrent::run wind up on main thread?


I've built a QFuture based asynchronous networking facade in my application. Roughly it works like this:

namespace NetworkFacade {
    QByteArray syncGet(const QUrl& url) {
        QEventLoop l;
        QByteArray rc;

        get(url, [&](const QByteArray& ba) {
            rc = ba;
            l.quit();
        });

        l.exec();
        return rc;
    }

    void get(const QUrl& url, const std::function<void (const QByteArray&)>& handler) {
        QPointer<QNetworkAccessManager> m = new QNetworkAccessManager;

        QObject::connect(m, &QNetworkAccessManager::finished, [=, &m](QNetworkReply *r) {
            QByteArray ba;

            if (r && r -> error() == QNetworkReply::NoError)
                ba = r -> readAll();

            m.clear();

            if (handler)
                handler(ba);
        });
        m -> get(QNetworkRequest(url));
    }
}

I have a QTimer that triggers a call on the main thread that does the following (obviously simplified):

foreach(Request r, requests) {
    futures.push_back(get(r));
}

foreach(QFuture<SomeType> f, futures) {
    f.waitForFinished();
    [do stuff with f.result()]
}

My assumption was that waitForFinished() would block the main thread while background thread(s) executed my network requests. Instead I get a qFatal error:

ASSERT: "m_blockedRunLoopTimer == m_runLoopTimer" in file eventdispatchers/qeventdispatcher_cf.mm, line 237

In the stack trace I see my waitForFinished() on the main thread, but then instead of being blocked I see (read from bottom up):

com.myapp   0x0008b669 QEventDispatcherCoreFoundation::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 1753
com.myapp   0x000643d7 QIOSEventDispatcher::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 823
com.myapp   0x0130e3c7 QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) + 119
com.myapp   0x0130e5fb QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) + 539
com.myapp   0x0003a550 NetworkFacade::syncGet(QUrl const&) + 208
com.myapp   0x00037ed1 QtConcurrent::StoredFunctorCall0<std::__1::shared_ptr<QuoteFacade::Quote>, QuoteFacade::closingQuote(QString const&, QDate const&)::$_0>::runFunctor() + 49
com.myapp   0x00038967 QtConcurrent::RunFunctionTask<std::__1::shared_ptr<QuoteFacade::Quote> >::run() + 87
com.myapp   0x00038abc non-virtual thunk to QtConcurrent::RunFunctionTask<std::__1::shared_ptr<QuoteFacade::Quote> >::run() + 28
com.myapp   0x010dc40f QThreadPoolPrivate::stealRunnable(QRunnable*) + 431
com.myapp   0x010d0c35 QFutureInterfaceBase::waitForFinished() + 165

So rather than waiting for the QFuture to get a value, my supposedly concurrent task is issued on the main thread. This causes the get() function I outlined above to get invoked, which listens for events on the QEventLoop. Meanwhile, the QTimer fires again and I get the assertion from above.

Am I doing something wrong, or is it perfectly valid that QtConcurrent::run can cause control to go back to the main thread?

=== Update 1

@peppe: The lambda being executed simply does an HTTP GET and generates parses the JSON response into a SomeType object. The result being accessed via the QFuture.

=== Update 2

Apparently this is by design. From qfutureinterface.cpp from Qt 5.4.0 lines 293-295:

// To avoid deadlocks and reduce the number of threads used, try to
// run the runnable in the current thread.
d->pool()->d_func()->stealRunnable(d->runnable);

Solution

  • Apparently this is by design. From qfutureinterface.cpp from Qt 5.4.0 lines 293-295:

    // To avoid deadlocks and reduce the number of threads used, try to
    // run the runnable in the current thread.
    d->pool()->d_func()->stealRunnable(d->runnable);
    

    QtConcurrent::run() returns a QFuture which is implemented using a QFutureInterface. QFutureInterface contains that code in both waitForFinished() and waitForResult().

    stealRunnable is an undocumented private method of QThreadPool. It is described thusly in headerdoc:

    /*!
        \internal
        Searches for \a runnable in the queue, removes it from the queue and
        runs it if found. This function does not return until the runnable
        has completed.
    */
    

    So what we wind up with is, if the QRunnable created internally by QtConcurrent::run() hasn't been dequeued from whatever QThreadPool it has been assigned to, then calling waitForFinished or waitForResult will cause it to run on the current thread (i.e. not concurrently.)

    That means code like this (and what I did in the question) might fail in mysterious ways:

    foreach (FuncPossiblyTriggeringQEvents fn, tasks) {
        futures.push_back(QtConcurrent::run(fn));
    }
    
    foreach (QFuture<> f, futures) {
        f.waitForFinished(); // Some of my tasks will run in this thread, not concurrently.
    }
    

    I got my design (getting a future out of a QNetowrkAccessManager get) working by using std::future and std::async.