Understand and reduce test execution time with ConcurrencyExtras package in Swift

In my app for unit testing our team heavy rely on tool to execute asynchronous test - ConcurrencyExtras - withMainSerialExecutor

That tool add possibility to launch test code on Main Thread, and also we could add await Task.yield() block to wait for some lengthy operation to complete

It works well, but it appears that tests execution time has risen, what i want to shorten execution time.

So far only soulution i have found is suggested here, is to add

override func invokeTest() {
    withMainSerialExecutor {
        super.invokeTest()
    }
}

In start of every test, and remove withMainSerialExecutor everywhere. It works, and now my asynchronous tests run faster.

My question is - why it has shorten async tests execution time? What i did, is basically force every test to execute on main thread, but manually added withMainSerialExecutor block did the same thing. Maybe someone have other solution to shorten async test execution time?

Example of test that moved from "lengthy" zone (more then 0.1 seconds) to short

func testHasLoaded() async {
    /// Arrange
    sut = makeVM()
    var hasLoaded = String()
    sut.dataHasFinishedLoading = {
        hasLoaded = "hasLoaded"
    }

    /// Assert
    await Task.megaYield()

    XCTAssertEqual(hasLoaded, "hasLoaded")
}

Original (0,11 seconds):

func testHasLoaded() async {
        withMainSerialExecutor {
            /// Arrange
            sut = makeVM()
            var hasLoaded = String()
            sut.dataHasFinishedLoading = {
                hasLoaded = "hasLoaded"
            }

            /// Assert
            await Task.megaYield()

            XCTAssertEqual(hasLoaded, "hasLoaded")
        }
    }

I might suggest that yielding (or, worse, “mega yielding”) is an anti-pattern.

Tests support Swift concurrency, so I would await the call to your dataHasFinishedLoading closure. We can do that by wrapping this in a continuation. There are lots of ways to do that, but perhaps an AsyncStream:

func testHasLoaded() async {
    let (stream, continuation) = AsyncStream<Bool>.makeStream()

    // Have the sut yield a value on our stream

    let sut = ViewModel()
    sut.dataHasFinishedLoading = {
        continuation.yield(true)
    }

    // you probably want a timeout task, in case the `dataHasFinishedLoading` is never called, e.g.,

    let timeoutTask = Task {
        try await Task.sleep(for: .seconds(1))
        XCTFail("timed out")
        continuation.yield(false)
    }
    defer { timeoutTask.cancel() }

    // now let's see if it yielded a value before it timed out

    let hasLoaded = await stream.first { _ in true }
    XCTAssert(hasLoaded == true, "hasLoaded")
}

FYI, in a comment you said:

… original goal was to run async tests without expectations (which are significally increase test run time, or could create flaky tests if expectation time is low)

Expectations do not significantly increase test run time. This is an incorrect claim oft repeated on Stack Overflow. The timeout is worst case scenario if there is an error and the expectation is not satisfied within a reasonable time frame. But if the expectation is fulfilled in a timely basis, the expectation finishes immediately and does not wait for the timeout.

Now if you don't like this failsafe timeout measured in seconds, then, fine, set a shorter timeout for this “asynchronous work does not finish in timely manner” scenario. Perhaps set timeouts measured in tens or hundreds of milliseconds rather than seconds. But your “mega yield” does the exact same thing: It adds a delay. Sure, this waiting is probably not measured in seconds (we cannot say without seeing your megaYield implementation), but it still waits. This yielding workflow is arguably actually worse, because it is going to insert a delay regardless of when the async work actually completes.

You always only want a timeout to cover your worst case scenario, but in successful scenarios, to finish immediately. You never want a test to wait any longer than needed. That is what expectations achieve. That is what the above async-await pattern achieves, too. But adding a bunch of yields or any fixed delays is a step in the wrong direction.

My criticism of expectations is limited to the fact that they are an artifact of the legacy XCTest patterns, but are not available in the new Swift Testing framework. This new testing framework uses async-await patterns (which is why I used that pattern above). For this reason, one might want to avoid introducing expectations at this point, as to minimize the amount of rework required if/when you eventually migrate to the newer Swift Testing framework.