Testing Kotlin Coroutines

Testing suspending functions in most cases is not different from testing blocking functions. Take a look at the fetchUserData below from FetchUserUseCase. Checking whether it shows data as expected can be easily achieved thanks to a few fakes¹ (or mocks²) and simple assertions. The only difference is that we need to wrap our tests with runBlocking or runTest to start a coroutine.

Similarly, in many other cases, if we are interested in what the suspending function does, we practically need nothing else but runBlocking and classic tools for asserting. This is what unit tests look like in many projects. Here is an example facade test from Kt. Academy backend:

Integration tests can be implemented in the same way. We just use runBlocking, and there is nearly no difference between testing how suspending and blocking functions behave.

The difference arises when we want to start testing time dependencies. For example, think of the following functions:

Both functions produce the same result; the difference is that the first one is fetching the profile and the friends synchronously, while the second one does it asynchronously. Effectively, the difference is that if fetching the profile takes X milliseconds, and fetching the friends takes Y milliseconds, then the first function would require X + Y milliseconds, whereas the second one would require max(X, Y) milliseconds.

Notice that the difference arises only when execution of getProfile and getFriends truly takes some time. If they are immediate, both ways of producing the user are indistinguishable. So, to test the real difference between them, we might help ourselves by delaying fake functions using delay, to simulate a delayed data loading scenario:

Now, the difference will be visible in unit tests: the produceCurrentUserSync call will take around 2 seconds, and the produceCurrentUserAsync call will take around 1 second. The problem is that we do not want a single unit test to take so much time. We typically have thousands of unit tests in our projects, and we want all of them to execute as quickly as possible. How to have your cake and eat it too? For that, we need to operate in simulated time. Here comes the kotlinx-coroutines-test library to the rescue with its StandardTestDispatcher.

When we call delay, our coroutine is suspended and resumed after a set time. This behavior can be altered thanks to TestCoroutineScheduler from kotlinx-coroutines-test, which makes delay operate in virtual time, which is fully simulated and does not depend on real time. This virtual time is manually advanced by us, using functions like advanceTimeBy or advanceUntilIdle.

To use TestCoroutineScheduler on coroutines, we should use a dispatcher that supports it. The standard option is StandardTestDispatcher. Unlike most dispatchers, it is not used just to decide on which thread a coroutine should run. Coroutines started with such a dispatcher will not run until we advance its schedulers' virtual time. The most typical way to do this is by using advanceUntilIdle, which advances virtual time and invokes all the operations from those coroutines.

The result time in scheduler is the sum of all delays, it represents how much time those coroutines would take in real time. However, it is a virtual time, so the time is precise and does not depend on the real time. This gives us not only efficiency, but also perfect predictability.

StandardTestDispatcher creates TestCoroutineScheduler by default, so we do not need to do so explicitly. We can access it with the scheduler property.

It is important to notice that StandardTestDispatcher does not advance time by itself. We need to do this, otherwise our coroutine will never be resumed.

Another way to push time is using advanceTimeBy with a concrete number of milliseconds. This function advances time and executes all operations that happened in the meantime. This means that if we push by 2 milliseconds, everything that was delayed by less than that time will be resumed. To resume operations scheduled exactly at the second millisecond, we need to additionally invoke the runCurrent function.

Here is a bigger example of using advanceTimeBy together with runCurrent.

To see that virtual time is truly independent of real time, see the example below. Adding Thread.sleep will not influence the coroutine with StandardTestDispatcher. Note also that the call to advanceUntilIdle takes only a few milliseconds, so it does not wait for any real time. It immediately pushes the virtual time and executes coroutine operations.

In the previous examples, we were using StandardTestDispatcher and wrapping it with a scope. Instead, we could use TestScope, which does the same (and it collects all exceptions with CoroutineExceptionHandler). The trick is that on this scope we can also use functions like advanceUntilIdle, advanceTimeBy, or the currentTime property , all of which are delegated to the scheduler used by this scope. This is very convenient.

We will later see that StandardTestDispatcher is often used directly on Android to test ViewModels, Presenters, Fragments, etc. We could also use it to test the produceCurrentUserSeq and produceCurrentUserSym functions by starting them in a coroutine, advancing time until idle, and checking how much simulated time they took. However, this would be quite complicated; instead, we should use runTest, which is designed for such purposes.

runTest is the most commonly used function from kotlinx-coroutines-test. It starts a coroutine with TestScope and immediately advances it until idle. Within its coroutine, the scope is of type TestScope, so we can check currentTime at any point. Therefore, we can check how time flows in our coroutines, while our tests take milliseconds. Effectively, runTest behaves like runBlocking, but it operates on virtual time. The below tests will pass immediately (in a couple of milliseconds), and the currentTime will be precisely as expected.

Let's get back to our functions, where we loaded user data sequentially and simultaneously. With runTest, testing them is easy. Assuming that our fake repository needs 1 second for each function call, synchronous processing should take 2 seconds, and asynchronous processing should take only 1. Again, thanks to the fact we are using virtual time, our tests are immediate, and the values of currentTime are precise.

Since it is an important use case, let's see a full example of testing a sequential function with all required classes and interfaces:

The runTest function creates a scope; like all such functions, it awaits the completion of its children. This means that if you start a process that never ends, your test will never stop.

For such situations, runTest offers backgroundScope. This is a scope that also operates on virtual time from the same scheduler, but runTest will not treat it as its children and await its completion. This is why the test below passes without any problems. We use backgroundScope to start all the processes we don't want our test to wait for.

If you want to test whether a certain function respects structured concurrency, the easiest way is to capture the context from a suspending function, then check if it contains the expected value or if its job has the appropriate state. As an example, consider the mapAsync function, which implements asynchronous mapping of a list of elements.

This function should map elements asynchronously while preserving their order. This behavior can be verified by the following test:

But this is not all. We expect a properly implemented suspending function to respect structured concurrency. The easiest way to check this is by specifying some context, like CoroutineName, for the parent coroutine, then check that it is still the same in the transformation function. To capture the context of a suspending function, we can use the currentCoroutineContext function or the coroutineContext property. In lambda expressions nested in coroutine builders or scope functions, we should use the currentCoroutineContext function because the coroutineContext property from CoroutineScope has priority over the property that provides the current coroutine context.

The easiest way to test cancellation is by capturing the inner function job and verifying its cancellation after the cancellation of the outer coroutine.

I don't think such tests are required in most applications, but I find them useful in libraries. It's not so obvious that structured concurrency is respected. Both the tests above would fail if async were started on an outer scope.

Except for StandardTestDispatcher we also have UnconfinedTestDispatcher. The biggest difference is that StandardTestDispatcher does not invoke any operations until we use its scheduler. UnconfinedTestDispatcher immediately invokes all the operations before the first delay on started coroutines, which is why the code below prints "C".

The runTest function was introduced in version 1.6 of kotlinx-coroutines-test. Previously, we used runBlockingTest, whose behavior is much closer to runTest using UnconfinedTestDispatcher. So, if want to directly migrate from runBlockingTest to runTest, this is how our tests might look:

If you do not need that for backward compatibility, I recommend using StandardTestDispatcher instead of UnconfinedTestDispatcher, as it is considered the new standard.

Using delay in fakes is easy but not very explicit. Many developers prefer to call delay in the test function. One way to do this is using mocks³:

In the Dispatchers chapter, I explained that some suspending function should specify the dispatcher they use. For example, we use Dispatcher.IO (or a custom dispatcher) for blocking calls, or Dispatchers.Default for CPU-intensive calls:

Is it possible to test if these functions truly change the dispatcher? It is, but it is not so easy. We can do that by capturing the context or the thread on a faked/mocked function that should be executed on a different dispatcher. Such checks are not common, as they are not considered very useful, but they are possible.

A more common problem is dealing with a situation when a function that changes its dispatcher needs to make a call that should take some time. The mechanism of virtual time is based on the dispatcher, so if we change it, we also change the way time is handled. That can be the case when one function both makes a blocking call and asynchronous suspending calls. Consider the below function:

It you want to test that getting user name, friends, and profile is done concurrently, it is not enough to use runTest and adding delays to fakes. It is because in fetchUserData we use Dispatchers.IO, which replaces StandardTestDispatcher from runTest. As a consequence, the delay function inside getName, getFriends, and getProfile will wait in real time, not in virtual time.

This problem is not so common, but it can happen. The simplest way to overcome it, is to use a dispatcher injected to this class, instead of using Dispatchers.IO. This way, we can replace it in unit tests with StandardTestDispatcher from runTest.

Now, instead of providing Dispatchers.IO in unit tests, we should use StandardTestDispatcher from runTest. We can get it from coroutineContext using the ContinuationInterceptor key (or using experimental CoroutineDispatcher key, that uses ContinuationInterceptor and casts the result to CoroutineDispatcher).

Another possibility is to type ioDispatcher as CoroutineContext, and replace it in unit tests with EmptyCoroutineContext. The final behavior will be the same, because fetchUserData function will never change dispatcher in tests, so it will implicitly StandardTestDispatcher. This solution is simpler, but some developers do not like seeing CoroutineContext as a type of dispatcher, as CoroutineDispatcher might include anything, not only a dispatcher.

Imagine a function which during its execution first shows a progress bar and later hides it.

If we only check the final result, we cannot verify that the progress bar changed its state during function execution. The trick that is helpful in such cases is to start this function in a new coroutine and control virtual time from outside. Notice that runTest creates a coroutine with the StandardTestDispatcher dispatcher and advances its time until idle. This means that its children's time will start once the parent starts waiting for them, so once its body is finished. Before that, we can advance virtual time by ourselves. For that, we can use advanceTimeBy, runCurrent, and advanceUntilIdle.

A similar effect could be achieved if we used delay. This is like having two independent processes: one executing the operation under test, while the other is checking the side effects of this operation.

Coroutines need to start somewhere. On the backend, they are often started by the framework we use (for instance Spring or Ktor), but sometimes we might also need to construct a scope ourselves and launch coroutines on it.

How can we test sendNotifications if the notifications are truly sent concurrently? Again, in unit tests we need to use StandardTestDispatcher as part of our scope. We should also add some delays to send and markAsSent.

There is no main function in unit tests. This means that if we try to use it, our tests will fail with the "Module with the Main dispatcher is missing" exception. On the other hand, injecting the Main thread every time would be demanding, so the "kotlinx-coroutines-test" library provides the setMain extension function on Dispatchers instead.

We often define main in a setup function (function with @Before or @BeforeEach) in a base class extended by all unit tests. As a result, we are always sure we can run our coroutines on Dispatchers.Main. We should also reset the main dispatcher to the initial state with Dispatchers.resetMain().

On Android, we typically start coroutines in ViewModels, Presenters, Fragments, or Activities. These are important classes, and we should test them. Think of the MainViewModel implementation below:

Instead of viewModelScope, there might be our own scope, instead of MutableLiveData, there might be MutableStateFlow, and instead of ViewModel, it might be Presenter, Activity, or some other class. It does not matter for our example. As in every class that starts coroutines, we should use StandardTestDispatcher as a part of the scope, to operate on virtual time. Previously, we needed to inject a different scope with a dependency injection, but now there is a simpler way: on Android, we use Dispatchers.Main as the default dispatcher, and we can replace it with StandardTestDispatcher thanks to the Dispatchers.setMain function. See the example below. After replacing Main dispatcher with a test dispatcher, we can start coroutines in our ViewModel and control their time. We can use the advanceTimeBy function to pretend that a certain amount of time has passed. We can also use advanceUntilIdle to execute all coroutines until they are done. This is how we can test the MainViewModel class, even if it starts multiple coroutines, with perfect precision. By adding delays to fake repositories, we can test certain scenarios, like when one process takes longer than another, or they one is executed when the other is suspended in the middle.

JUnit 4 allows us to use rules. These are classes that contain logic that should be invoked on some test class lifecycle events. For instance, a rule can define what to do before and after all tests, therefore it can be used in our case to set our test dispatcher and clean it up later. Here is a good implementation of such a rule:

This rule needs to extend TestWatcher, which provides test lifecycle methods, like starting and finished, which we override. It composes TestCoroutineScheduler and TestDispatcher. Before each test in a class using this rule, TestDispatcher will be set as Main. After each test, the Main dispatcher will be reset. We can access the scheduler with the scheduler property of this rule.

This way of testing Kotlin coroutines is fairly common on Android. It is even explained in Google's Codelabs materials (Advanced Android in Kotlin 05.3: Testing Coroutines and Jetpack integrations) (currently, for older kotlinx-coroutines-test API).

It is similar with JUnit 5, where we can define an extension. There are at least a few ways how extensions can be defined, but this is the most common implementation:

Using MainCoroutineExtension is nearly identical to using the MainCoroutineRule rule. The difference is that instead of @get:Rule annotation, we need to use @JvmField and @RegisterExtension.

In this chapter we've discussed the most important tools and techniques for testing Kotlin coroutines. The most important lessons are: