We’ve traditionally relied on A/B testing at DoorDash to guide our decisions.
Category Archives: Experimentation and Testing
Experiment Faster and with Less Effort
Business Policy Experiments Using Fractional Factorial Designs
At DoorDash, we constantly strive to improve our experimentation processes by addressing four key dimensions, including velocity to increase how many experiments we can conduct, toil to minimize our launch and analysis efforts, rigor to ensure a sound experimental design and robustly efficient analyses, and efficiency to reduce costs associated with our experimentation efforts.
Selecting the Best Image for Each Merchant Using Exploration and Machine Learning
In order to inspire DoorDash consumers to order from the platform there are few tools more powerful than a compelling image, which raises the questions: what is the best image to show each customer, and how can we build a model to determine that programmatically using each merchant’s available images?
Balancing Network Effects, Learning Effects, and Power in Experiments
At DoorDash, we rely on experimentation to make decisions regarding model improvements and product changes because we cannot perfectly predict the results in advance.
Using ML and Optimization to Solve DoorDash’s Dispatch Problem
DoorDash delivers millions of orders every day with the help of DeepRed, the system at the center of our last-mile logistics platform.
Running Experiments with Google Adwords for Campaign Optimization
Running experiments on marketing channels involves many challenges, yet at DoorDash, we found a number of ways to optimize our marketing with rigorous testing on our digital ad platforms.
Improving Online Experiment Capacity by 4X with Parallelization and Increased Sensitivity
Data-driven companies measure real customer reactions to determine the efficacy of new product features, but the inability to run these experiments simultaneously and on mutually exclusive groups significantly slows down development.
Leveraging Causal Modeling to Get More Value from Flat Experiment Results
A/B tests and multivariate experiments provide a principled way of analyzing whether a product change improves business metrics.
Supporting Rapid Product Iteration with an Experimentation Analysis Platform
DoorDash’s new experimentation platform, built on a combination of SQL, Kubernetes, and Python, allows for quick iteration of data-driven feature improvements.
Improving Experimental Power through Control Using Predictions as Covariate (CUPAC)
In this post, we introduce a method we call CUPAC (Control Using Predictions As Covariates) that we successfully deployed to reduce extraneous noise in online controlled experiments, thereby accelerating our experimental velocity. 
Rapid experimentation is essential to helping DoorDash push key performance metrics forward.