How DoorDash uses XcodeGen to eliminate project merge conflicts

At DoorDash, we work to implement efficient processes that can mitigate common conflicts within a large iOS development team. Part of those efforts involve using XcodeGen, a command line interface (CLI), to reduce merging conflicts within our various iOS teams. Here we will discuss its implementation to manage the intricate business scenarios and demanding requirements of the Dasher app, which lets our drivers receive, pick up, and securely deliver orders to customers. With multiple configurations, dependencies, localization mechanisms, pre-build and post-build scripts, and a constantly scaling team, there are high risks for codebase instability and lack of scalability in developing such an app. Resolving common xcodeproj merge conflicts can be time-consuming, but it's a critical part of maintaining developer velocity and ensuring the smooth operation of a large and complex team. 

Through examining the development of our driver app, we can delve into the challenges of maintaining an Xcode project for a large team, including the growing pains large teams face and the difficulties of big project refactoring. We demonstrate how XcodeGen can resolve these issues and we illustrate how the DoorDash Consumer and Driver teams successfully utilized it to overcome challenges. Finally, we share our insights and experiences on how to use XcodeGen in a large-scale project.

Why xcodeproj merge conflicts hurt collaboration

Xcode is an essential tool for iOS developers and is used for various tasks such as writing code, building user interfaces, and analyzing app store metrics, among many other things. Despite its importance, the complexity of the xcodeproj files can make it difficult to manage projects effectively and ensure seamless collaboration among team members. A project can consist of anything from ten files to thousands of files, including source code, assets, scripts, and plain text files. Xcode arranges these files in a seemingly arbitrary order, which appears to be influenced by when and how developers incorporate them. 

In Xcode, all source code and assets must be linked to a target in order to be part of the final product, which means that every time a file is added, Xcode creates a file reference to ensure its inclusion in the target. To maintain all of these, Xcode houses the pbxproj file within the xcodeproj project, which holds the project's complete structure. For a small team, continuous Xcode project modifications might not be a common occurrence. As long as the team isn't frequently merging code or adjusting Xcode project files, they will likely remain untroubled by merge conflict issues. However, as a team grows or incorporates continuous integration tools, they may face a significant increase in merge conflicts, which can have a considerable impact on their development velocity.

The pbxproj file consists of machine-generated code, which cannot easily be reviewed by humans. Unless there is a merge conflict, this code rarely is examined, which means developers have limited knowledge and understanding about how Xcode manages these items. As a result, problems may only surface when a team expands, hindering efficiency and effectiveness.   

But when a mobile team scales, efficiency and speed become crucial. Xcode, particularly the machine-generated code within the pbxproj file, may pose challenges to growth. Because it is complex and difficult for humans to review, developers don't know how Xcode manages files.

This complexity builds as multiple developers add and remove files, update file references, and make changes to build settings, phases, configurations, and schemes. Ultimately, frequent merge conflicts slow the team's progress, limiting its ability to scale effectively.

It is therefore essential to have a process in place that can manage Xcode file references and structure, plus a well-equipped toolset that can mitigate these challenges and improve the team's ability to manage project files and execute tasks efficiently. By addressing the pbxproj file's complexity and implementing a streamlined process to manage Xcode files, a mobile team can scale effectively, avoid bottlenecks, and maximize its efficiency and speed.

Anyone who frequently collaborates on Xcode projects is likely familiar with the challenges of addressing merge conflicts stemming from project files that are difficult to read. It's hard to determine an appropriate merge conflict resolution just by looking at the raw, automatically generated XML. Additionally, without recompiling it's not clear whether a resolution is successful. Figure 1 shows an example of file structure change and cause conflicts:

There are three primary issues involved in resolving merge conflicts:

• Time-consuming and error-prone - Carefully comparing conflicting versions of an xcodeproj file can be daunting for complex or large projects. And, despite best efforts, any mistakes or overlooked conflicts can cause problems that require additional time and effort to fix.
• Risk of losing changes - If a merge conflict is not resolved properly, there is a risk that some changes may be lost or overwritten. This can be particularly problematic if lost changes are critical to a project's functionality.
• Potential for project corruption - In some cases, a merge conflict can corrupt the project so severely that it can't be opened in Xcode. Work may stop until the project is  restored to a previous version or - worst-case scenario - the project may have to be restarted from scratch.

In general, merge conflicts within an xcodeproj file can present notable challenges for developers and teams working on Xcode projects. However, tools like XcodeGen can aid in circumventing and resolving merge conflicts more swiftly, enhancing consistency and reliability while minimizing risks.

How to handle project refactoring 

The multitude of targets, files, and dependencies involved in refactoring large and complex projects can prove challenging. It's difficult to identify and modify relevant parts of the project without breaking or affecting other parts. Beyond this, Xcode projects often rely on external libraries or frameworks such as CocoaPods or Swift Package Manager (SPM). Refactoring may require updating these dependencies to ensure compatibility with the updated project structure.

Collaborating with other team members also can pose a challenge. Communication and coordination of changes may be necessary to avoid conflicts and ensure that everyone is working with the same project structure and configuration.

It's essential to maintain compatibility with different versions of Xcode and other tools when refactoring an existing Xcode project. Backward compatibility with older versions of Xcode or other tools frequently is required to ensure that all team members can continue to build and use the project.

At DoorDash, we often undertake project refactoring for similar reasons. For example, DoorDash's Caviar is the world's largest marketplace for premium restaurants and food enthusiasts, while the DoorDash marketplace offers a variety of merchants that cater to everyone and every occasion. Although these products serve different needs, the engineering teams can share many tools, technologies, and codebases.

We understand that code duplication naturally leads to code sharing and building shared modules. However, breaking down monolithic projects into modules requires a lot of refactoring. More than one step is needed to create each module, so we must use proper precautions, tools, and processes to keep the project stable while still working toward our goals.

There are two options to refactor a project:

• Top-down approach - This method involves rebuilding the project from scratch. Although previous learning and smart tools can be used to make the process safer and faster, it requires commiting to a full rewrite.
• Bottom-up approach - This method, which is generally considered to be most practical and safe, runs everything as-is, with changes made incrementally. 

In a recent article - Adopting SwiftUI with a Bottom-Up Approach to Minimize Risk - we discussed both of these approaches in detail. In our SwiftUI and Combine adaptation, a bottom-up approach proved to be much more effective in laying the groundwork because we had to undergo numerous project restructurings and modifications before ultimately achieving modularization.

When we adopted SwiftUI in the DasherDriver project, we knew that we would need to move files and directories, change targets, and undertake various other Xcode project-related tasks as we made major code changes. All of these tasks at the start of the restructuring process require a significant amount of time. Additionally, because it was a continuous process involving constant iteration of engineering work and product work, there was increasing risk of merge conflicts as the larger team continually instituted code changes and altered the project structure. To avoid slowdowns, we knew that we must avoid conflicts at all costs.

Given the difficulty our team had in reading Xcode project files quickly, we opted to address the merge conflict issues initially. We acknowledged that relying solely on Xcode IDE would not suffice to maintain efficiency. By employing XcodeGen CLI, we managed to progress through the refactoring phase without encountering significant obstacles.

How XcodeGen helps reduce merge errors 

XcodeGen's ability to define project specs in YAML or JSON files offers several benefits over manual project creation and maintenance. It provides consistency and ease of maintenance, which is crucial in large-scale projects. By defining a project's structure and settings in a single YAML file, it's possible to ensure that projects have a consistent structure and configuration. This makes it easier to maintain and update projects over time. Changes can be made by simply editing the YAML file and then regenerating the Xcode project to apply those changes.

XcodeGen also improves collaboration. With the project specification stored in a text file, it can easily be versioned and shared with team members. This allows everyone to work on the same project specification and ensures the same project configuration across the board. Team members can collaborate more efficiently, avoid conflicts, and know that changes are applied correctly.

XcodeGen allows for flexibility and customization in project development. With this tool, users can customize different aspects of their project - including build settings, targets, and schemes - to fit their specific needs and requirements. This level of customization is particularly useful for projects that require unique settings or have specific requirements that can't be achieved through standard settings. XcodeGen also integrates smoothly with other tools and libraries, such as CocoaPods, SPM, and Fastlane, making it easy to manage external dependencies in projects and automate common tasks like building and deploying apps.

To understand the power of XcodeGen, we need to understand the mess of .pbxproj. Xcode generates GUID for each file reference and uses that reference everywhere else inside the .pbxproj. To show this, we created a file inside an example project named - MyExampleApp.swift - Observe in the following code segment that Xcode assigned a GUID for that. Wherever in the project we use or refer to that file, Xcode will use the GUID to link properly.

Let's walk through some real-world scenarios. The test project has the structure shown in Figure 2:

Developer A decided to create another group named - Code - and moved all Swift files inside. After that operation, the project structure then looks like what is shown in Figure 3:

Because of that Xcode operation, .pbxproj file changes. The changes to our git file appear as shown in Figure 4:

These are substantial changes, all of them unreadable, for just one Xcode IDE operation. Now things get really ugly when Developer B adds a code file in the root hierarchy even before the initial changes get into the main branch. Developer B's local branch Xcode project layout appears as shown in Figure 5: 

It's clear now where the merge conflict is starting. Any Xcode project merge conflict requires a manual, time-intensive process. Worse still, there's an excellent chance that something else has inadvertently been messed up that won't make itself known until much later.
Instead, let's manage the same kind of situation using the magic of XcodeGen. After following the instructions from here, we have project.yml in our project directory. We populate the YML as follows:

The amazing part of the YAML file is that it can be read from the top to understand the goal right away:

• We defined the project name 
• We fixed some global options that matter most 
• Then we defined a target with a source code directory
• And we're done!

After generating the project using the XcodeGen command, Xcode IDE will match exactly what we describe in the YML file. But we have achieved major technical milestones: 

1. We, not IDE, are in control of the project structure 
2. We control the settings and they will be reflected accordingly 
3. We just prevented a project file merge conflict

The project IDE snapshot appears as shown in Figure 6:   

What if, despite our efforts, a merge conflict still needs to be resolved? We can just discard the incoming .pbxproj and run XcodeGen to generate the file again - and we are done. In essence, minutes or hours of complex merge conflict resolution can be reduced to less than a minute. 

Drawbacks of XcodeGen process 

It's always disruptive to make any changes on a bigger project and with a larger team. Consequently, we must make sure we have a way to onboard everyone so that they can adapt quickly to changes. In our case, we made a change in our workflow that required for a time doing frequent one-to-one sessions. We also created a dedicated support channel to ensure everyone was on the same page. This phase, however, was short and quickly evolved into a routine daily process. 

There are a few potential drawbacks to using XcodeGen in some cases:

• Lack of support for Xcode features - XcodeGen does not support all of the features and settings available in Xcode. For example, XcodeGen cannot create custom code snippets or manage localization files. If a project relies on these features, Xcode may still be required for some tasks.
• Limited documentation and community support - Because XcodeGen is a relatively new tool, there may not be as much documentation and community support available as for other tools. This can make it harder for new users to get started and troubleshoot any issues they encounter.
• Potential for conflicts with Xcode - Because XcodeGen generates an Xcode project from a YAML file, the generated project may not always be in sync with the actual contents of the project on disk. If changes are made outside of XcodeGen, there is a risk that those changes will be overwritten or lost when the project regenerates.

While XcodeGen offers many benefits, it may not be the right solution for every project or development team. It is important to evaluate needs and requirements carefully before deciding whether to use XcodeGen in your workflow.

Impact of using XcodeGen: More velocity 

Here are the key problems that XcodeGen can help resolve: 

• Reduce project merge conflict to nearly zero - After applying XcodeGen to both the DoorDash and Dasher apps, our team of more than 100 engineers suffered no project merge conflicts! The XcodeGen integration was a breeze for our team and began paying for itself almost immediately. Even factoring in the effort invested in integrating XcodeGen, we already have saved significant time and will continue to do so into the future.

• Modularity and code sharing - Caviar and DoorDash are built from the same project in two different targets using XcodeGen and a templated XcodeGen target. Without a CLI-based tool, there would be massive risk for both projects. XcodeGen immunized us from that risk. 

• Migrating from Cocoapods to SPM - At one time, all Xcode projects were a heap of Pod tangles. Pods had lots of technical limitations and then Apple brought SPM to manage packaging and modularity. Migrating code from Pods to SPM was a daunting process and XcodeGen was our savior there as well.

By using XcodeGen to generate Xcode projects, we have ensured that our projects have a consistent structure and configuration. This process makes it easier to maintain and update projects over time while reducing the risk of errors and inconsistencies. Because the project specification is stored in a text file, we created templates to reuse in our different projects. Using XcodeGen allows everyone to work with the same project specifications and configuration. It allowed us to customize various aspects of our project, such as the build settings, targets, and schemes while giving us the flexibility to tailor our projects to specific needs and requirements.

Conclusion

DoorDash faced challenges with scaling teams and resolving merge conflicts in Xcode projects. By adopting a tool that allowed more maintainable and human-readable project configurations, XcodeGen helped resolve merge conflicts more easily and streamlined development processes. XcodeGen is particularly valuable for larger teams dealing with complex project structures. Although XcodeGen may not be necessary for smaller teams, it has become an important part of DoorDash's toolset until (and unless) Apple provides a built-in solution in Xcode. In short, if you are facing similar challenges in your Xcode project development, it's worth your time to evaluate whether XcodeGen can be your solution.