Variance modifiers limitations

This is a chapter from the book Advanced Kotlin. You can find it on LeanPub or Amazon.

In the previous parts, we already discussed the out and in variance modifiers with their practical use cases, and a popular pattern I call the Covariant Nothing Object.

In Java, arrays are reified and covariant. Some sources state that the reason behind this decision was to make it possible to create functions like Arrays::sort that make generic operations on arrays of every type.

Integer[] numbers= {1, 4, 2, 3};
Arrays.sort(numbers); // sorts numbers

String[] letters= {"B", "C", "A"};
Arrays.sort(letters); // sorts letters

However, there is a big problem with this decision. To understand it, let’s analyze the following Java operations, which produce no compilation time errors but throw runtime errors:

// Java
Integer[] numbers= {1, 4, 2, 3};
Object[] objects = numbers;
objects[2] = "B"; // Runtime error: ArrayStoreException

As you can see, casting numbers to Object[] didn't change the actual type used inside the structure (it is still Integer); so, when we try to assign a value of type String to this array, an error occurs. This is clearly a Java flaw, but Kotlin protects us from it by making Array (as well as IntArray, CharArray, etc.) invariant (so upcasting from Array<Int> to Array<Any> is not possible).

To understand what went wrong in the above snippet, we should understand what in-positions and out-positions are.

A type is used in an in-position when it is used as a parameter type. In the example below, the Dog type is used in an in-position. Note that every object type can be up-casted; so, when we expect a Dog, we might actually receive any of its subtypes, e.g., a Puppy or a Hound.

open class Dog class Puppy : Dog() class Hound : Dog() fun takeDog(dog: Dog) {} takeDog(Dog()) takeDog(Puppy()) takeDog(Hound())

In-positions work well with contravariant types, including the in modifier, because they allow a type to be transferred to a lower one, e.g., from Dog to Puppy or Hound. This only limits class use, so it is a safe operation.

open class Dog class Puppy : Dog() class Hound : Dog() class Box<in T> { private var value: T? = null fun put(value: T) { this.value = value } } fun main() { val dogBox = Box<Dog>() dogBox.put(Dog()) dogBox.put(Puppy()) dogBox.put(Hound()) val puppyBox: Box<Puppy> = dogBox puppyBox.put(Puppy()) val houndBox: Box<Hound> = dogBox houndBox.put(Hound()) }

However, public in-positions cannot be used with covariance, including the out modifier. Just think what would happen if you could upcast Box<Dog> to Box<Any?>. If this were possible, you could literally pass any object to the put method. Can you see the implications of this? That is why it is prohibited in Kotlin to use a covariant type (out modifier) in public in-positions.

class Box<out T> { private var value: T? = null fun set(value: T) { // Compilation Error this.value = value } fun get(): T = value ?: error("Value not set") } val dogHouse = Box<Dog>() val box: Box<Any> = dogHouse box.set("Some string") // Is this were possible, we would have runtime error here

This is actually the problem with Java arrays. They should not be covariant because they have methods, like set, that allow their modification.

Covariant type parameters can be safely used in private in-positions.

class Box<out T> { private var value: T? = null private fun set(value: T) { // OK this.value = value } fun get(): T = value ?: error("Value not set") }

Covariance (out modifier) is perfectly safe with public out-positions, therefore these positions are not limited. This is why we use covariance (out modifier) for types that are produced or only exposed, and the out modifier is often used for producers or immutable data holders. Thus, List has the covariant type parameter, but MutableList must have the invariant type parameter.

There is also a symmetrical problem (or co-problem, as some like to say) for contravariance and out-positions. Types in out-positions are function result types and read-only property types. These types can also be up-casted to any upper type; however, since we are on the other side of an object, we can expect types that are above the expected type. In the example below, Amphibious is in an out-position; when we might expect it to be Amphibious, we can also expect it to be Car or Boat.

open class Car interface Boat class Amphibious : Car(), Boat fun getAmphibious(): Amphibious = Amphibious() val amphibious: Amphibious = getAmphibious() val car: Car = getAmphibious() val boat: Boat = getAmphibious()

Out positions work well with covariance, i.e., the out modifier. Upcasting Producer<Amphibious> to Producer<Car> or Producer<Boat> limits what we can expect from the produce method, but the result is still correct.

open class Car interface Boat class Amphibious : Car(), Boat class Producer<out T>(val factory: () -> T) { fun produce(): T = factory() } fun main() { val producer: Producer<Amphibious> = Producer { Amphibious() } val amphibious: Amphibious = producer.produce() val boat: Boat = producer.produce() val car: Car = producer.produce() val boatProducer: Producer<Boat> = producer val boat1: Boat = boatProducer.produce() val carProducer: Producer<Car> = producer val car2: Car = carProducer.produce() }

Out-positions do not get along with contravariant type parameters (in modifier). If Producer type parameters were contravariant, we could up-cast Producer<Amphibious> to Producer<Nothing> and then expect produce to produce literally anything, which this method cannot do. That is why contravariant type parameters cannot be used in public out-positions.

open class Car interface Boat class Amphibious : Car(), Boat class Producer<in T>(val factory: () -> T) { fun produce(): T = factory() // Compilation Error } fun main() { val carProducer = Producer<Amphibious> { Car() } val amphibiousProducer: Producer<Amphibious> = carProducer val amphibious = amphibiousProducer.produce() // If not compilation error, we would have runtime error val producer = Producer<Amphibious> { Amphibious() } val nothingProducer: Producer<Nothing> = producer val str: String = nothingProducer.produce() // If not compilation error, we would have runtime error }

You cannot use contravariant type parameters (in modifier) in public out-positions, such as a function result or a read-only property type.

class Box<in T>( val value: T // Compilation Error ) { fun get(): T = value // Compilation Error ?: error("Value not set") }

Again, it is fine when these elements are private:

class Box<in T>( private val value: T ) { private fun get(): T = value ?: error("Value not set") }

This way, we use contravariance (in modifier) for type parameters that are only consumed or accepted. A well-known example is kotlin.coroutines.Continuation:

public interface Continuation<in T> { public val context: CoroutineContext public fun resumeWith(result: Result<T>) }

Read-write property types are invariant, so public read-write properties support neither covariant nor contravariant types.

class Box<in T1, out T2> { var v1: T1 // Compilation error var v2: T2 // Compilation error }

UnsafeVariance annotation

Every good rule must have some exceptions. In general, using covariant type parameters (out modifier) in public in-positions is considered unsafe, therefore such a situation blocks code compilation. Still, there are situations where we would like to do this anyway because we know we will do it safely. A good example is List.

As we have already explained, the type parameter in the List interface is covariant (out modifier), and this is conceptually correct because it is a read-only interface. However, it uses this type of parameter in some public in-positions. Just consider the contains or indexOf methods: they use covariant type parameters in a public in-position, which is a clear violation of the rules we just explained.

How is that possible? According to the previous section, it should not be possible. The answer is the UnsafeVariance annotation, which is used to turn off the aforementioned limitations. It is like saying, "I know it is unsafe, but I know what I’m doing and I will use this type safely".

It is ok to use UnsafeVariance for methods like contains or indexOf because their parameters are only used for comparison, and their arguments are not set anywhere or returned by any public functions. They could also be of type Any?, and the type of those parameters is only specified so that a user of these methods knows what kind of value should be used as an argument.

Variance modifier positions

Variance modifiers can be used in two positions¹. The first one, the declaration side, is more common and is a modifier on the class or interface declaration. It will affect all the places where the class or interface is used.

// Declaration-side variance modifier class Box<out T>(val value: T) val boxStr: Box<String> = Box("Str") val boxAny: Box<Any> = boxStr

The other position is the use-site, which is a variance modifier for a particular variable.

class Box<T>(val value: T) val boxStr: Box<String> = Box("Str") // Use-site variance modifier val boxAny: Box<out Any> = boxStr

We use use-site variance when, for some reason, we cannot provide variance modifiers for all the types generated by a class or an interface, yet we need some variance for one specific type. For instance, MutableList cannot have the in modifier because then its method's result types would return Any? instead of the actual element type. Still, for a single parameter type we can make its type contravariant (in modifier) to allow any collections that can accept a type:

interface Dog interface Pet data class Puppy(val name: String) : Dog, Pet data class Wolf(val name: String) : Dog data class Cat(val name: String) : Pet fun fillWithPuppies(list: MutableList<in Puppy>) { list.add(Puppy("Jim")) list.add(Puppy("Beam")) } fun main() { val dogs = mutableListOf<Dog>(Wolf("Pluto")) fillWithPuppies(dogs) println(dogs) // [Wolf(name=Pluto), Puppy(name=Jim), Puppy(name=Beam)] val pets = mutableListOf<Pet>(Cat("Felix")) fillWithPuppies(pets) println(pets) // [Cat(name=Felix), Puppy(name=Jim), Puppy(name=Beam)] }

Note that some positions are limited when we use variance modifiers. When we have MutableList<out T>, we can use get to get elements, and we receive an instance typed as T, but we cannot use set because it expects us to pass an argument of type Nothing. This is because a list with any subtype of T might be passed there, including the subtype of every type that is Nothing. When we use MutableList<in T>, we can use both get and set; however, when we use get, the returned type is Any? because there might be a list with any supertype of T, including the supertype of every type that is Any?. Therefore, we can freely use out when we only read from a generic object, and we can freely use in when we only modify that generic object.

Star projection

On the use-site, we can also use the star * instead of a type argument to signal that it can be any type. This is known as star projection.

if (value is List<*>) { ... }

Star projection should not be confused with the Any? type. It is true that List<*> effectively behaves like List<Any?>, but this is only because the associated type parameter is covariant. It might also be said that Consumer<*> behaves like Consumer<Nothing> if the Consumer type parameter is contravariant. However, the behavior of Consumer<*> is nothing like Consumer<Any?>, and the behavior of List<*> is nothing like List<Nothing>. The most interesting case is MutableList. As you might guess, MutableList<Any?> returns Any? as a result in methods like get or removeAt, but it also expects Any? as an argument for methods like add or set. On the other hand, MutableList<*> returns Any? as a result in methods like get or removeAt, but it expects Nothing as an argument for methods like add or set. This means MutableList<*> can return anything but accepts (literally) nothing.

Use-site type	In-position type	Out-position type
`T`	`T`	`T`
`out T`	`Nothing`	`T`
`in T`	`T`	`Any?`
`*`	`Nothing`	`Any?`

Summary

Every Kotlin type parameter has some variance:

The default variance behavior of a type parameter is invariance. If, in Box<T>, type parameter T is invariant and A is a subtype of B, then there is no relation between Box<A> and Box<B>.

The out modifier makes a type parameter covariant. If, in Box<T>, type parameter T is covariant and A is a subtype of B, then Box<A> is a subtype of Box<B>. Covariant types can be used in public out-positions.

The in modifier makes a type parameter contravariant. If, in Box<T>, type parameter T is contravariant and A is a subtype of B, then Cup is a subtype of Cup. Contravariant types can be used in public in-positions.

In Kotlin, it is also good to know that:

Type parameters of List and Set are covariant (out modifier). So, for instance, we can pass any list where List<Any> is expected. Also, the type parameter representing the value type in Map is covariant (out modifier). Type parameters of Array, MutableList, MutableSet, and MutableMap are invariant (no variance modifier).

In function types, parameter types are contravariant (in modifier), and the return type is covariant (out modifier).

We use covariance (out modifier) for types that are only returned (produced or exposed).

We use contravariance (in modifier) for types that are only accepted (consumed or set).

1:
This is also called mixed-site variance.

Variance modifiers limitations

UnsafeVariance annotation

Variance modifier positions

Star projection

Summary

The author:

Reviewers: