How to use kotlinx.coroutines.withTimeout in kotlinx.coroutines.test.runTest?

I have a suspend function that makes a rest call to an external API that I want to timeout after 1 minute.

suspend fun makeApiCallWithTimeout(): List<ApiResponseData> =
   withTimeout(1.minutes) {
      apiCall()
   }

I'm trying to test it with Junit5 and kotlinx.coroutines.test 1.6.0 like so:

@Test
fun `Test api call`() = runTest {
   val responseData = "[]"
   mockWebServer.enqueue(mockResponse(body = responseData)
   val result = sut.makeApiCallWithTimeout()
   advanceUntilIdle()
   assertEquals(0, result.size)
}

Unfortunately, I'm getting errors that look like this:

Timed out waiting for 60000 ms
kotlinx.coroutines.TimeoutCancellationException: Timed out waiting for 60000 ms
    at app//kotlinx.coroutines.TimeoutKt.TimeoutCancellationException(Timeout.kt:184)
    at app//kotlinx.coroutines.TimeoutCoroutine.run(Timeout.kt:154)
    at app//kotlinx.coroutines.test.TestDispatcher.processEvent$kotlinx_coroutines_test(TestDispatcher.kt:23)
    at app//kotlinx.coroutines.test.TestCoroutineScheduler.tryRunNextTask(TestCoroutineScheduler.kt:95)
    at app//kotlinx.coroutines.test.TestCoroutineScheduler.advanceUntilIdle(TestCoroutineScheduler.kt:110)
    at app//kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTestCoroutine(TestBuilders.kt:212)
    at app//kotlinx.coroutines.test.TestBuildersKt.runTestCoroutine(Unknown Source)
    at app//kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invokeSuspend(TestBuilders.kt:167)
    at app//kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invoke(TestBuilders.kt)
    at app//kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invoke(TestBuilders.kt)
    at app//kotlinx.coroutines.test.TestBuildersJvmKt$createTestResult$1.invokeSuspend(TestBuildersJvm.kt:13)
    (Coroutine boundary)

It seems that kotlinx.coroutines.test.runTest is advancing virtual time on the withTimeout without giving it any time to execute its body. See (https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/README.md#using-withtimeout-inside-runtest)

Unfortunately, the documentation doesn't provide a way to get around this.

Please advise on how to test this function using runTest.

Solution

This is because of delay-skipping.

Here you're using runTest, which brings time-control capabilities to your test. To do so, this coroutine builder provides a dispatcher with a fake time that automatically skips delays (from the real time perspective) but keeps track of the fake time internally.

From the point of view of this dispatcher, everything that doesn't have delay()s runs instantly, while things that do delay make the fake time progress.

However, this cannot be used with things that really take actual time outside of the test dispatcher, because the test will not really wait. So in essence here, withTimeout times out immediately because the actual apiCall() probably runs outside of the dispatcher (and takes real time).

You can easily reproduce this behaviour like this:

@Test
fun test() = runTest {
    withTimeout(1000) { // immediately times out
        apiCall()
    }
}

suspend fun apiCall() = withContext(Dispatchers.IO) {
    Thread.sleep(100) // not even 1s
}

There are usually 2 solutions:

if you want to keep using controlled time, you have to make sure you're using the test dispatcher in all the relevant code. This means that the places in your code where you use custom coroutine scopes or explicit dispatchers should allow to inject a dispatcher
if you don't really need controlled time, you can use runBlocking instead of runTest (on JVM) or keep using runTest but run the test on another dispatcher like Dispatchers.Default:

fun test() = runTest {
    withContext(Dispatchers.Default) {
        // test code
    }
}