Testing Kotlin Coroutines
This is a chapter from the book Kotlin Coroutines. You can find it on LeanPub or Amazon.
Testing suspending functions in most cases is not different from testing normal functions. Take a look at the
fetchUserData below from
FetchUserUseCase. Checking whether it shows data as expected can be easily achieved thanks to a few fakes1 (or mocks2) and simple assertions.
My method for testing logic should not be used as a reference. There are many conflicting ideas for how tests should look. I’ve used fakes here instead of mocks so as not to introduce any external library (I also personally prefer them). I’ve also tried to keep all tests minimalistic to make them easier to read.
Similarly, in many other cases, if we are interested in what the suspending function does, we practically need nothing else but
runBlocking and classic tools for asserting. This is what unit tests look like in many projects. Here are a few unit tests from Kt. Academy backend:
Integration tests can be implemented in the same way. We just use
runBlocking, and there is nearly no difference between testing how suspending and blocking functions behave.
Testing time dependencies
The difference arises when we want to start testing time dependencies. For example, think of the following functions:
Both functions produce the same result; the difference is that the first one does it sequentially, while the second one does it simultaneously. The difference is that if fetching the profile and the friends takes 1 second each, then the first function would require around 2 seconds, whereas the first would require only 1. How would you test this?
Notice that the difference arises only when execution of
getFriends truly takes some time. If they are immediate, both ways of producing the user are indistinguishable. So, we might help ourselves by delaying fake functions using
delay to simulate a delayed data loading scenario:
Now, the difference will be visible in unit tests: the
produceCurrentUserSeq call will take around 1 second, and the
produceCurrentUserSym call will take around 2 seconds. The problem is that we do not want a single unit test to take so much time. We typically have thousands of unit tests in our projects, and we want all of them to execute as quickly as possible. How to have your cake and eat it too? For that, we need to operate in simulated time. Here comes the
kotlinx-coroutines-test library to the rescue with its
This chapter presents the kotlinx-coroutines-test functions and classes introduced in version 1.6. If you use an older version of this library, in most cases it should be enough to use
advanceTimeByin older versions is like
runCurrentin versions newer than 1.6. The detailed differences are described in the migration guide at https://github.com/Kotlin/kotlinx.coroutines/blob/master/kotlinx-coroutines-test/MIGRATION.md.
When we call
delay, our coroutine is suspended and resumed after a set time. This behavior can be altered thanks to
kotlinx-coroutines-test, which makes
delay operate in virtual time, which is fully simulated and does not depend on real time.
TestCoroutineScheduleras well as
runTestare still experimental.
TestCoroutineScheduler on coroutines, we should use a dispatcher that supports it. The standard option is
StandardTestDispatcher. Unlike most dispatchers, it is not used just to decide on which thread a coroutine should run. Coroutines started with such a dispatcher will not run until we advance virtual time. The most typical way to do this is by using
advanceUntilIdle, which advances virtual time and invokes all the operations that would be called during that time if this were real time.
TestCoroutineScheduler by default, so we do not need to do so explicitly. We can access it with the
It is important to notice that
StandardTestDispatcher does not advance time by itself. We need to do this, otherwise our coroutine will never be resumed.
Another way to push time is using
advanceTimeBy with a concrete number of milliseconds. This function advances time and executes all operations that happened in the meantime. This means that if we push by 2 milliseconds, everything that was delayed by less than that time will be resumed. To resume operations scheduled exactly at the second millisecond, we need to additionally invoke the
Here is a bigger example of using
advanceTimeBy together with
How does it work under the hood? When
delayis called, it checks if the dispatcher (class with
ContinuationInterceptorkey) implements the
StandardTestDispatcherdoes). For such dispatchers, it calls their
scheduleResumeAfterDelayfunction instead of the one from the
DefaultDelay, which waits in real time.
To see that virtual time is truly independent of real time, see the example below. Adding
Thread.sleep will not influence the coroutine with
StandardTestDispatcher. Note also that the call to
advanceUntilIdle takes only a few milliseconds, so it does not wait for any real time. It immediately pushes the virtual time and executes coroutine operations.
In the previous examples, we were using
StandardTestDispatcher and wrapping it with a scope. Instead, we could use
TestScope, which does the same (and it collects all exceptions with
CoroutineExceptionHandler). The trick is that on this scope we can also use functions like
advanceTimeBy, or the
currentTime property , all of which are delegated to the scheduler used by this scope. This is very convenient.
We will later see that
StandardTestDispatcher is often used directly on Android to test ViewModels, Presenters, Fragments, etc. We could also use it to test the
produceCurrentUserSym functions by starting them in a coroutine, advancing time until idle, and checking how much simulated time they took. However, this would be quite complicated; instead, we should use
runTest, which is designed for such purposes.
runTest is the most commonly used function from
kotlinx-coroutines-test. It starts a coroutine with
TestScope and immediately advances it until idle. Within its coroutine, the scope is of type
TestScope, so we can check
currentTime at any point. Therefore, we can check how time flows in our coroutines, while our tests take milliseconds.
Let's get back to our functions, where we loaded user data sequentially and simultaneously. With
runTest, testing them is easy. Assuming that our fake repository needs 1 second for each function call, sequential processing should take 2 seconds, and simultaneous processing should take only 1. Thanks to the fact we are using virtual time, our tests are immediate, and the values of
currentTime are precise.
Since it is an important use case, let's see a full example of testing a sequential function with all required classes and interfaces:
TestScope, that includes
StandardTestDispatcher, that includes
runTest function creates a scope; like all such functions, it awaits the completion of its children. This means that if you start a process that never ends, your test will never stop.
For such situations,
backgroundScope. This is a scope that also operates on virtual time, but
runTest will not await its completion. This is why the test below passes without any problems. We use
backgroundScope to start all the processes we don't want our test to wait for.
Testing cancellation and context passing
If you want to test whether a certain function respects structured concurrency, the easiest way is to capture the context from a suspending function, then check if it contains the expected value or if its job has the appropriate state. As an example, consider the
mapAsync function, which I explain in more detail in the Recipes chapter.
This function should map elements asynchronously while preserving their order. This behavior can be verified by the following test:
But this is not all. We expect a properly implemented suspending function to respect structured concurrency.
The easiest way to check this is by specifying some context, like
CoroutineName, for the parent coroutine, then check that it is still the same in the
To capture the context of a suspending function, we can use the
currentCoroutineContext function or the
coroutineContext property. In lambda expressions nested in coroutine builders or scope functions, we should use the
currentCoroutineContext function because the
coroutineContext property from
CoroutineScope has priority over the property that provides the current coroutine context.
The easiest way to test cancellation is by capturing the inner function job and verifying its cancellation after the cancellation of the outer coroutine.
I don’t think such tests are required in most applications, but I find them useful in libraries. It’s not so obvious that structured concurrency is respected. Both the tests above would fail if
async were started on an outer scope.
StandardTestDispatcher we also have
UnconfinedTestDispatcher. The biggest difference is that
StandardTestDispatcher does not invoke any operations until we use its scheduler.
UnconfinedTestDispatcher immediately invokes all the operations before the first delay on started coroutines, which is why the code below prints "C".
runTest function was introduced in version 1.6 of
kotlinx-coroutines-test. Previously, we used
runBlockingTest, whose behavior is much closer to
UnconfinedTestDispatcher. So, if want to directly migrate from
runTest, this is how our tests might look:
delay in fakes is easy but not very explicit. Many developers prefer to call
delay in the test function. One way to do this is using mocks3:
In the above example, I’ve used the MockK library.
Testing functions that change a dispatcher
In the Dispatchers chapter, we covered typical cases where we set concrete dispatchers. For example, we use
Dispatcher.IO (or a custom dispatcher) for blocking calls, or
Dispatchers.Default for CPU-intensive calls. Such functions rarely need to be simultaneous, so typically it is enough to test them with
runBlocking. This case is easy and practically indistinguishable from testing blocking functions. For example, think of the following function:
We could test the behavior of such functions in tests wrapped with
runBlocking, but how about checking if these functions truly do change the dispatcher? This can also be done if we mock the functions we call, and inside we capture the name of the used thread.
In the above function, I couldn't use fakes because
CsvReaderis a class, so I used mocks.
Dispatchers.IOshare the same pool of threads.
However, in rare cases we might want to test time dependencies in functions that do change the dispatcher. This is a tricky case because the new dispatcher replaces our
StandardTestDispatcher, so we stop operating in virtual time. To make this clear, let’s wrap the
fetchUserData function with
Now all our previously implemented tests will wait in real time, and
currentTime will keep returning
0. The easiest way to prevent this is by injecting the dispatcher via a constructor and replacing it in unit tests.
Now, instead of providing
Dispatchers.IO in unit tests, we should use
runTest. We can get it from
coroutineContext using the
Another possibility is to cast
CoroutineContext, and replace it in unit tests with
EmptyCoroutineContext. The final behavior will be the same: the function will never change the dispatcher.
Testing what happens during function execution
Imagine a function which during its execution first shows a progress bar and later hides it.
If we only check the final result, we cannot verify that the progress bar changed its state during function execution. The trick that is helpful in such cases is to start this function in a new coroutine and control virtual time from outside. Notice that
runTest creates a coroutine with the
StandardTestDispatcher dispatcher and advances its time until idle (using the
advanceUntilIdle function). This means that its children's time will start once the parent starts waiting for them, so once its body is finished. Before that, we can advance virtual time by ourselves.
Notice that, thanks to
runCurrent, we can precisely check when some value changes.
A similar effect could be achieved if we used
delay. This is like having two independent processes: one is doing things, while the other is checking if the first one is behaving properly.
Using explicit functions like
advanceTimeByis considered more readable than using
Testing functions that launch new coroutines
Coroutines need to start somewhere. On the backend, they are often started by the framework we use (for instance Spring or Ktor), but sometimes we might also need to construct a scope ourselves and launch coroutines on it.
How can we test
sendNotifications if the notifications are truly sent concurrently? Again, in unit tests we need to use
StandardTestDispatcher as part of our scope. We should also add some delays to
runBlockingis not needed in the code above. Both
advanceUntilIdleare regular functions.
Replacing the main dispatcher
There is no main function in unit tests. This means that if we try to use it, our tests will fail with the "Module with the Main dispatcher is missing" exception. On the other hand, injecting the Main thread every time would be demanding, so the "kotlinx-coroutines-test" library provides the
setMain extension function on
We often define main in a setup function (function with
@BeforeEach) in a base class extended by all unit tests. As a result, we are always sure we can run our coroutines on
Dispatchers.Main. We should also reset the main dispatcher to the initial state with
Testing Android functions that launch coroutines
On Android, we typically start coroutines in ViewModels, Presenters, Fragments, or Activities. These are very important classes, and we should test them. Think of the
MainViewModel implementation below:
viewModelScope, there might be our own scope, and instead of ViewModel, it might be Presenter, Activity, or some other class. It does not matter for our example. As in every class that starts coroutines, we should use
StandardTestDispatcher as a part of the scope. Previously, we needed to inject a different scope with a dependency injection, but now there is a simpler way: on Android, we use
Dispatchers.Main as the default dispatcher, and we can replace it with
StandardTestDispatcher thanks to the
After setting the Main dispatcher this way,
onCreate coroutines will be running on
testDispatcher, so we can control their time. We can use the
advanceTimeBy function to pretend that a certain amount of time has passed. We can also use
advanceUntilIdle to resume all coroutines until they are done.
Setting a test dispatcher with a rule
JUnit 4 allows us to use rules. These are classes that contain logic that should be invoked on some test class lifecycle events. For instance, a rule can define what to do before and after all tests, therefore it can be used in our case to set our test dispatcher and clean it up later. Here is a good implementation of such a rule:
This rule needs to extend
TestWatcher, which provides test lifecycle methods, like
finished, which we override. It composes
TestDispatcher. Before each test in a class using this rule,
TestDispatcher will be set as Main. After each test, the Main dispatcher will be reset. We can access the scheduler with the
scheduler property of this rule.
If you want to call
MainCoroutineRule, you can define them as extension functions and properties.
This way of testing Kotlin coroutines is fairly common on Android. It is even explained in Google’s Codelabs materials (Advanced Android in Kotlin 05.3: Testing Coroutines and Jetpack integrations) (currently, for older
It is similar with JUnit 5, where we can define an extension:
MainCoroutineExtension is nearly identical to using the
MainCoroutineRule rule. The difference is that instead of
@get:Rule annotation, we need to use
In this chapter, we've discussed the most important use cases for testing Kotlin coroutines. There are some tricks we need to know, but in the end our tests can be very elegant and everything can be tested quite easily. I hope you feel inspired to write good tests in your applications using Kotlin Coroutines.
A fake is a class that implements an interface but contains fixed data and no logic. They are useful to mimic a concrete behavior for testing.
Mocks are universal simulated objects that mimic the behavior of real objects in controlled ways. We generally create them using libraries, like MockK, which support mocking suspending functions. In the examples below, I decided to use fakes to avoid using an external library.
Not everyone likes mocking. On one hand, mocking libraries have plenty of powerful features. On the other hand, think of the following situation: you have thousands of tests, and you change an interface of a repository that is used by all of them. If you use fakes, it is typically enough to update only a few classes. This is a big problem, which is why I generally prefer to use fakes.