How to leverage functional programming in Kotlin to write better, cleaner code

As DoorDash transitioned from Python monolith to Kotlin microservices, our engineering team was presented with a lot of opportunities to improve operational excellence and continue our obsession with reliability. While there are many ways we can improve our engineering metrics via optimized hardware and suitable system designs, one direct lever we can pull as developers to contribute to the overall engineering excellence is to write better and cleaner code on a daily basis.

And one way for developers to write cleaner code is to adopt the paradigm of functional programming (FP) using Kotlin. As a multi-paradigm general-purpose programming language, Kotlin provides a lot of the necessary toolkits we need to leverage FP in our day-to-day coding. 

In this post, we are going to talk about what functional programming is, what the benefits and potential downsides of it are, how it compares to the alternative paradigm imperative programming (IP), what Kotlin provides for developers to leverage FP, and examples of how we at DoorDash write FP-style code in Kotlin.

What is functional programming (FP)?

In a nutshell, FP is a programming paradigm in which programs are constructed by applying and composing functions. A typical program in FP works like this: Given some input, apply a series of functions (can be both big and small) on the input to get the desired output. This might seem trivial, but it has a lot of implications and rules behind how the functions are constructed and what is the scope of changes that can be affected by these functions. Together, these implications and rules are what make FP a great paradigm to consider.

Out of all the concepts in FP, the following three contribute the most to the benefits of adopting FP in our day-to-day programming. (We will discuss later in more detail how these concepts help us write better and cleaner code.)

• Pure functions. By definition, pure functions have the same return values for the same input, and there are no side effects (such as updating other local variables and invoking I/O). For example, all mathematical functions, such as sum, max, and average, are pure functions.
• Immutable states. Compared to mutable states we are familiar with–such as a variable that can be reassigned to any values or an array that we can insert or remove any values during runtime–immutable states are not modifiable after they have been created or assigned a value.
• Function composition. As the word “composition” suggests, function composition refers to combining simple functions to build more complicated functions. In practice, the output of a function becomes the input of another function, which yields an output that is used for the input of another function, and so on.

It's normal or understandable to not have heard of these concepts before. In fact, this is one of the few reasons why FP is not as widely used and adopted as other paradigms. It’s different from the other camp of programming paradigm, imperative programming (IP), which includes the sub-paradigms of procedural programming and object-oriented programming (OOP) with which most developers are familiar. Most computer science curriculums don’t cover FP as extensively as OOP, often it’s not covered at all. While many mathematical courses cover the core concepts behind FP, such as pure functions and composition, they rarely connect the dots between these concepts with how they can be leveraged in the programming world.

How does FP compare with IP?

While there are many areas of difference between FP and IP, we will expand on Microsoft’s explanation in comparing between FP and IP in the context of .NET and emphasize these three areas:

• Programmer focus. IP requires programmers to think about how to perform the algorithms and track internal changes in state to meet the desired outcome. In FP, however, programmers focus mainly on three things:
  • What are the inputs
  • What are the desired outputs
  • What transformations are needed to convert the inputs into outputs

• State changes. There are basically no state changes in FP since immutable states are at the core of the paradigm. In IP, however, state changes are everywhere and crucial to the flow of execution because those state changes essentially are how the program keeps track of where it’s at and what to execute next.
• Primary flow control. In FP, functions are used to apply to data collections such as arrays and maps to perform the desired transformations. Functions are first-class citizens; therefore, they can be assigned to values, passed as arguments, and returned from other functions. On the other hand, IP relies heavily on loops, conditionals, and function calls (can be pure or non-pure) to control the flow of the program and manipulate internal states to get to the desired end state.

It’s apparent that not only are the methodologies different between FP and IP, but that in practice the way a programmer thinks during coding is also drastically different. That being said, FP and IP are not mutually exclusive. In fact, many programming languages, such as Kotlin, adopt a multi-paradigm mindset where programmers are free to use more than one paradigm in the same piece of code. For example, since Kotlin is designed to interoperate fully with Java, and Java is primarily an OOP language, one could expect that there will be a lot of OOP examples in Kotlin code. That does not stop programmers from applying FP-style functions on Java objects, which we will showcase more later.

Benefits of writing FP-style code

Now that we have seen the differences between FP and the more widely known IP, let’s look at what benefits FP brings to the table. In summary, there are three main advantages: 

• Side-effect-free executions
• Easy iterations on existing functions
• Increased testability

Side-effect-free executions

As previously mentioned, pure functions guarantee no side effects other than producing the desired output. Pure functions do not modify the state of any of their inputs, nor do they modify the state of any system wide parameters. In a highly complex system like the one DoorDash has, this property is highly valuable because, as a developer, it’s beneficial to expect a function to do exactly what it claims to do, and there won’t be any other side effects by calling the function. When multiple developers from teams across different departments work on the same code base, understanding the logic in the code becomes straightforward because it’s easy to read the series of functions being applied to the input and figure out what is being done without poking all the individual functions.

Easy iterations on existing functions

Because all functions written in FP-style will have no side effects, it’s much easier to make iterations on existing functions and logic. For example, suppose there are existing functions that perform a series of operations to calculate the base pay for a Dasher (our term for a driver) in a delivery. Let’s say that we want to add a new feature such that the base pay will be increased by 50% if the delivery was done during rush hour. This will be very easy to iterate on the existing logic; in fact, all we need is to add a new function to the end of the calculation funnel, which multiplies the input by 1.5 if the delivery was done during rush hour. In this case, the input will be the base pay calculated in the previous step. However, as a developer, I don’t need to worry about where the input is coming from and how the input is calculated. As long as we know that the task of this pure function is to compute a new value, it’s a very easy function to write.

Increased testability

When a function is a pure function, the output of the function is deterministic given the same input. This makes testing the function much easier because the test can be structured as a set of inputs and their expected output, and it’s guaranteed that running the function through these inputs will always yield the same expected output. For example, suppose we have a function that takes an array of integers and returns the second-largest number from the array. This operation is pure because:

• The function does not depend on anything else other than the input 
• It doesn’t alter the input array or anything else in the system
• Given the same array of numbers, the second-largest number will always be the same.

Therefore, the unit test for this function will be very straightforward because there is no need to mock any system variables or function calls, and the output is deterministic so there will be no flaky tests. Therefore, if we could all write FP-style programs, it would become much easier to write tests, especially for mission-critical applications.

Potential downsides to FP 

It would be too good to be true if FP brought only benefits to the table without any potential downsides. One downside, depending on the programming language and the compiler, is that each function call could create a new call stack. Without optimization, these creations and destructions of call stacks could quickly become large runtime overheads for the application even when we are performing trivial operations. Luckily, this downside is not so bad, since Kotlin provides the ability to make a function inline, which resolves a lot of the problems if it is properly used. Simply put, instead of creating a new call stack and executing the code inside a function, an inline function basically replaces the function call with the actual content and places them in the body of the caller function.

Another potential downside of FP is its speed and memory usage. Since each function essentially creates new data from the existing data, these data creations can take extra time and space to be instantiated in memory. In IP, on the other hand, we mostly deal with mutable data structures that can be updated in place without allocating new memory. The problem of runtime speed can be mitigated by parallelism. Naturally, most pure functions in FP are highly parallelizable, which means we can run a large pool of functions without worrying about how they interact with each other or how they will affect the system variables. An effective strategy for running functions in parallel can potentially bring net-positive speed improvement to the program.

One of the most common operations in modern applications is Input/Output (I/O). When I/O is involved, it means the application is now dealing with the outside world. Examples of I/O include prompting the user for an input, invoking a remote procedure call (RPC) to another service, and reading data from a database. Because of the unpredictable nature of I/O tasks, they are most likely not pure, meaning both the input and output are not deterministic. When we are dealing with I/O tasks, writing pure functions forcefully to handle I/O is not the right approach. In fact, given the multi-paradigm nature of many modern programming languages like Kotlin, developers should choose the paradigm based on what’s best for the task at hand instead of strictly following one paradigm for the whole application. In the world of Kotlin, developers can use the standard I/O library from Kotlin, as well as the one from Java.

What does Kotlin provide for developers to leverage FP?

Before we get into the real actions of how to write FP code in Kotlin, it's natural to wonder, is Kotlin even the right language for FP? The short answer is, definitely yes! In fact, one of the top FAQs from the official Kotlin language website states that “Kotlin has both object-oriented and functional constructs. Kotlin can use both OO and FP styles, or mix elements of the two.” So what features and tools does Kotlin have so that developers can write FP-style code?

Higher-order functions and lambdas

There’s a dedicated section in the Kotlin documentation that talks about this topic, so we won’t go over all the details. In summary, since Kotlin functions are first-class citizens, they can be stored in variables, they can be passed around in function arguments and return values, and they can define types around functions. With this capability, common FP functions such as the fold operation can be written easily in Kotlin because we can pass in any accumulative function to the fold function to combine the data.

On top of supporting higher-order functions, lambda expressions are neat ways to simplify the code without writing all the function declarations that usually cause a lot of mess in the code. In a nutshell, lambda expressions are functions that are not declared but are passed immediately as an expression. This makes reasoning and understanding the code much easier since we don’t need to jump through hoops to find out what the function actually does.

As a quick example, consider the following code snippet:

deliveries.sumOf { delivery -> delivery.customerTip }

In this snippet, sumOf is a higher-order function because it takes another function as an argument, and { delivery -> delivery.customerTip } is a lambda expression, which takes in a delivery object and returns the customer tip amount of the delivery. We will show more real-life examples of writing FP-style code in Kotlin in later sections.

Collection-based operations

Kotlin provides a powerful set of collection-based operations that can be used to facilitate FP-style computation. According to the Kotlin documentation, given a list of items, common operations fall into these groups: 

• Transformations: Transform all items in the data collection
• Filtering: Return a subset of the items based on certain criteria
• Grouping: Group them into smaller groups of items based on certain criteria
• Retrieving collection parts: Return a subset of items in some fashion
• Retrieving single elements: Return an item based on certain criteria
• Ordering: Order the data collection based on certain criteria from each item
• Aggregate: Returns a single value after applying some operations on all items

All the functions for collections from the standard library are in the Kotlin Collections API documentation. In later sections, we will see how developers at DoorDash usually utilize these common operations on a regular basis.

Comparing Kotlin with languages like Python, JavaScript, and C++

While Kotlin provides such a powerful set of tools for developers to write FP code, these tools and functions are not exclusive to Kotlin. In fact, many modern languages support FP-style development and provide similar sets of collection-based operations, especially in newer releases of these languages. The following table summarizes how Kotlin compares with these popular programming languages in terms of the availability of some features we’ve discussed so far.

While Kotlin supports all the features natively, other modern languages, such as TypeScript (which is the primary language for web clients at DoorDash), also have built-in library support. Thus, the knowledge of FP and common operations in Kotlin can easily be transferred to other modern languages in day-to-day coding.

Examples of how we at DoorDash write FP-style code in Kotlin

Now that we understand what FP is, what are the pros and cons of it, and what Kotlin provides for us to write FP-style code, it’s time to see FP in action. In all the examples below, we will use the following data classes as the context. Note that all the examples are hypothetical and for illustrative purposes only.

data class Delivery(
    val id: UUID,
    val dasherId: UUID,
    val basePay: Double,
    val customerTip: Double,
    val dropOffTime: Calendar
)

data class Dasher(
    val id: UUID,
    val name: String
)

Let’s start with an easy but very common example: Given a list of deliveries, return a list of total pay amounts where they are greater than $10. 

Let’s first look at how we can do this in IP style.

val totalPayAmounts = mutableListOf<Double>()
for (delivery: Delivery in deliveries) {
    val totalPay = delivery.basePay + delivery.customerTip
    if (totalPay > 10) {
        totalPayAmounts.add(totalPay)
    }
}
return totalPayAmounts

For the sake of comparison, here’s the thinking process behind this code snippet:

1. Create an empty container that will hold the desired output
2. Loop through each delivery in the input
3. Compute the total pay
4. If the total pay is greater than $10, add it to the output container
5. Return the output container

Now let’s look at how we can write the same logic in FP style.

return deliveries
    .map { delivery -> delivery.basePay + delivery.customerTip }
    .filter { totalPay -> totalPay > 10 }

And the thinking process behind this code snippet:

1. Transform all deliveries into the total pay of each delivery
2. Filter and keep only those that have total pay greater than $10

From this example, it’s not hard to imagine how different the mindsets are between FP and IP. In the iterative style, the logic flows from top to bottom, and it uses a mutable state (totalPayAmounts) and a for loop to compute the end result. In contrast, FP focuses on how we deal with the input data by transforming and filtering the input data. In the FP-style code snippet, there are no additional states introduced, and no loops are being used. Instead, it uses Kotlin built-in collection-based functions map and filter, in conjunction with two lambda expressions to compute the final result list. Overall, it makes reading the logic easier and reduces additional states being created in the program.

Let’s look at another more elaborated example. Suppose we have a list of deliveries, and we want to keep only the deliveries that have customer tip greater than $5, find the latest 10 deliveries by the delivery drop-off time, and get the Dasher ID of these deliveries. As before, we’ll start with how we can write this in IP style.

val filteredDeliveries = mutableListOf<Delivery>()
for (delivery: Delivery in deliveries) {
    if (delivery.customerTip > 5) {
        filteredDeliveries.add(delivery)
    }
}

// Sort by delivery.dropOffTime descending
val sortedDeliveries = Collections.sort(
    filteredDeliveries,
   dropOffTimeComparator
)

val result = mutableListOf<UUID>()
for (i in sortedDeliveries.indices) {
    result.add(sortedDeliveries[i].dasherId)
    if (i == 9) break
}

For the same logic, here is the FP style code:

val result = deliveries
    .filter { it.customerTip > 5 }
    .sortedByDescending { it.dropOffTime }
    .map { it.dasherId }
    .take(10)

Here we use a Kotlin special identifier it, which is used inside a lambda expression to refer to its parameter implicitly. In the lambdas above, all its represent the delivery object in the list.

There’s no doubt how clean and elegant the FP-style code looks compared to the IP code. Reading the snippet is basically reading plain English:

1. Filter the deliveries to keep only those with customer tip greater than $5
2. Sort the list in descending order of delivery drop-off time
3. Transform the elements into the Dasher ID of the delivery
4. Take the first 10 elements from the list

While this example looks simple enough for illustration purposes, it’s not hard to see how flexible it is if we want to apply more complex logic to the collection. Suppose that multiple teams want to filter the list of deliveries based on their own logic, call them complexFilterFunc1, complexFilterFunc2, and so on. They can simply apply the filtering logic directly to the deliveries by calling the functions in a series. Since filter is a higher-order function, it can take other functions as the argument.

val result = deliveries
    .filter { complexFilterFunc1(it) }
    .filter { complexFilterFunc2(it) }
    .filter { ... }
    ...

Better yet, because these filtering functions are pure, they can be reordered and invoked in any order without changing the underlying logic.

val result = deliveries
    .filter { complexFilterFunc3(it) }
    .filter { complexFilterFunc1(it) }
    .filter { ... }
    ...

If passing it to all the filtering functions seems redundant, Kotlin has a way to pass in the function reference to a higher-order function using a double colon ::

val result = deliveries
    .filter(::complexFilterFunc1)
    .filter(::complexFilterFunc2)
    .filter(...)
    ...

By now we should be familiar with how to write FP-style code on a list of items and transform it to another list. What if we want to transform the list into other data structures like map? This is not only possible, but very common in our day-to-day coding as well. Let’s look at an example.

Suppose we have a list of deliveries. We now want to see for each Dasher, how much they earned in tips for each hour of the day. The end result will be structured as a map from Dasher ID to another map, where the key is the hour of the day, and the value is the total customer tip they earned. We will start by looking at how we will do this in IP style.

val dasherIdToDeliveries = mutableMapOf<UUID, MutableList<Delivery>>()
for (delivery: Delivery in deliveries) {
    if (dasherIdToDeliveries.containsKey(delivery.dasherId)) {
        dasherIdToDeliveries[delivery.dasherId]!!.add(delivery)
    } else {
        dasherIdToDeliveries[delivery.dasherId] = mutableListOf(delivery)
    }
}

val resultMap = mutableMapOf<UUID, MutableMap<Int, Double>>()
for ((dasherId, deliveriesByDasher) in dasherIdToDeliveries) {
    val hourToTotalTipMap = mutableMapOf<Int, Double>()
    for (delivery in deliveriesByDasher) {
        val hour = delivery.dropOffTime.get(Calendar.HOUR_OF_DAY)
        if (hourToTotalTipMap.containsKey(hour)) {
            hourToTotalTipMap[hour] = hourToTotalTipMap[hour]!! + delivery.customerTip
        } else {
            hourToTotalTipMap[hour] = delivery.customerTip
        }
    }
    resultMap[dasherId] = hourToTotalTipMap
}
return resultMap

This is definitely not a piece of clean code. It uses a double for-loop, two mutable maps, and two if-else blocks to get the final result. Now let’s look at how we can write this in FP style.

val result = deliveries
    .groupBy { it.dasherId }
    .mapValues { it.value
        .groupBy { delivery -> 
            delivery.dropOffTime.get(Calendar.HOUR_OF_DAY) 
        }
        .mapValues { hourToDeliveries -> 
            hourToDeliveries.value.sumOf { delivery -> 
                delivery.customerTip
            }
        }
    }

There are a few new functions being used, so we will explain what they do first before going through the code:

• groupBy: Given a list of items, return a map from the key returned by the key selector (in this case, the lambda expression) to the list of items that has the corresponding key
• mapValues: Given a map, return a new map with entries having the keys of the original map, and the values obtained by the transformation function
• sumOf: Given a list of items, sum the list by the key selector

With these definitions in mind, the FP-style code reads like this:

1. Group the list of deliveries by Dasher ID
2. For each group of deliveries, group by drop-off hour
3. For each sub-group of deliveries (from grouping by drop-off hour), sum by customer tip

This example demonstrates the capability to group and aggregate a collection of data with FP in Kotlin. It’s very common to put nested collection-based functions in the intermediate collections created from the previous step and transform them into whatever new data type is needed. This is a very powerful capability as developers are not restricted to transform data to the same type as the input.

Conclusion

Functional programming is a powerful programming paradigm that can help developers easily write cleaner and better code for day-to-day programming needs. This is especially true when developers are working on mission-critical operations, large distributed systems, and intensive data transformation. Leveraging it along with other common paradigms like object-oriented programming can help achieve the best of both worlds, especially with the rich ecosystem that the Kotlin language provides. While FP has its potential downsides, with modern techniques and thoughtful designs, we can aim for increased simplicity, testability, and readability without sacrificing efficiency and speed.