Contract Testing with Pact

Introduction

As we migrate our systems to a microservices-oriented architecture at DoorDash, we have taken care to balance the excitement around anticipated benefits (development velocity, deployment independence, etc.) with a more measured assessment of our preparedness to work with the complexities introduced by more distributed systems.

One of the most common pain points with microservices — or, for that matter, any distributed system, finer or coarser-grained — is ensuring that service deployments maintain backwards compatibility and that the whole ecosystem plays well together in all environments, not just production.

An integration environment where teams can bring microservices together to ensure compatibility is critical to both development velocity and system correctness; but as the number of services grows, the coordination required to maintain this environment can become a heavy burden for development and infrastructure/operations teams.

At DoorDash, we looked towards applying consumer-driven contract testingas a technique to enforce compatibility testing. Conveniently, a mature and well-supported contract testing framework called Pact exists for enabling contract testing between REST services and consumers of services.

For our initial use case, we chose to adopt Pact to enforce the contracts between our mobile apps and the mobile backend-for-frontend (BFF).

Development Workflow with Pact

Pact works by connecting a set of unit tests. One set of tests for the service consumer and one set of tests for the service provider. The consumer tests contain specifications about an HTTP request and the shape of the response. When the tests are run, the Pact framework turns the specifications into a contract that is uploaded to a repository (known as a broker). The broker is a web service that contains a collection of contracts and associations between consumers and producers. With that in mind, the overall flow looks like this:

On the consumer:

1. Write unit tests specifying the request and the response shape
2. Run the tests, which, if successful, generate a Pact file
3. Upload the Pact file to the broker

On the producer:

1. Download the Pact files that involve the producer
2. Run the tests, which, if successful, generate verification results
3. Upload the verification results to the broker

In our case, the consumers are the iOS and Android mobile apps. Next, we’ll see how to specify the consumer API contract.

iOS App: Pact on Swift

Integrating Pact testing with an iOS application is pretty straightforward, consisting of a few simple steps:

1. Install the Pact mock service
2. Install the PactConsumerSwift library
3. Configure Xcode
4. Write your test
5. Upload the verification results to your Pact Broker

Install the Pact Mock Service

The Pact mock service spins up a local server that will generate Pacts when your tests are run. Your tests will provide a series of expectations (fields, http status, etc.) for an endpoint (for example, /v1/auth) that are compiled into a Pact file.

To install the Pact mock service, simply run the following:

https://gist.github.com/f9d02ca3df44c776ec17aa1c3bf5b8db

Install the PactConsumerSwift Library

The PactConsumerSwift library is used by your application to conveniently interact with the mock service to generate pact files. It provides objects and methods that can be used to easily define expectations and pass these expectations to the mock server.

To incorporate the service into your project, simply add the following line to your Podfile:

https://gist.github.com/240fcf009ceeda83d9a50e11ef85e119

Configuring Xcode

In order to have the local pact server spin up when your tests run, we’ll add pre-actions and post-actions to your project target.

Select your target and click Edit Scheme, select the Test phase, and add a new Run Script Action for both pre-actions and post-actions. It should look something like this:

For pre-actions, enter the following and save: https://gist.github.com/c9e052a711167511b681393fc6c26373

This will spin up the mock service on port 1234.

For post-actions, enter the following and save:

https://gist.github.com/c54f320635e767c9f08324620daf6f06 This simply stops the mock service.

Write Your Tests

A sample test is as follows:

https://gist.github.com/68fe810fbab57b91fdcda3b0613f15d1

This sets up the services that will be making the network calls. The MockService, provided by PactConsumerService, is initialized with provider and consumer keys‚Ää‚Äî‚Äämake sure these match the ones you previously used to set up your Pact broker.

The BootstrapClient is the network layer you are using in your application. Configure it with the base url provided by the MockService (which will point network calls to your local server).

We use the MockService to start building our expectations. These are customizable and you can add or remove expectations that fit your particular use case. We can specify a response body that we expect, headers, HTTP status, and other related fields.

Make sure the request parameters are configured properly for the request that you want to test as these will be used by the broker to reconcile your Pact with the backend results.

This part of the test runs your network call against the local server (remember that earlier we initialized an instance of our network service with the base url provided by the MockService). We run it within a closure passed into the runfunction on MockService so that the MockService can perform any configuration that it needs before running the actual call.

And that’s it!

Upload the Verification Results to Your Pact Broker

If you run your tests and they complete successfully, you‚Äôll see that a new Pact file has been generated for you in your /tmp/pacts folder at the root of your project. Now, all we need to do is to upload this to your Pact broker. We‚Äôll do it manually with curl but this can be added to the post-actions of your test phase as well.

https://gist.github.com/8675cde291703d38c87a0aada1dabe9b

And that’s it! If the upload is successful, you’ll see the new Pact file in your Pact broker. Next, we’ll look at a similar example from the perspective of the Android client.

Android App: Pact on Kotlin

Add the Pact gradle plugin:

https://gist.github.com/7caf07ab8de1485d01ddaa49924f3b37 Add a task to let Pact know how to publish to the broker: https://gist.github.com/92daa2c521b1a2cdf9e0418a2f201f6c Add these 2 test dependencies: https://gist.github.com/4041b6ccc8ca9004baff56d56afd9308 Define a Pact test which describe the interaction between the consumer and the provider. First, we define a Pact rule: https://gist.github.com/ecc4fe5fefeda176d4d67104109f1ace Create a test function to define the interaction: https://gist.github.com/bd4234651d6d201ba0a92dc1153fd9f2

Finally, we can define a pact verification test for this pact:

https://gist.github.com/4e698cec81d9232355fe9f25afd1b9fb

Now that the consumer contracts have been defined, we can start to write tests to ensure the contracts are fulfilled by the mobile BFF.

On the Mobile BFF: Pact in Kotlin

Once you’ve defined the contract via the consumer tests, you can move on to verifying the contract. In order to verify the contract, we need a way to start the application and send it the HTTP requests that are part of the contract. In our architecture, the provider is the mobile BFF.

There are two main approaches to verification:

1. Run the mobile BFF via the command line
2. Run the mobile BFF via an integration test

Pact provides tooling to accomplish both approaches. Since the mobile BFF makes requests to downstream services, we chose to verify using an integration test, since it would be easier to mock the outgoing HTTP requests. We also considered verifying the contract via the command line, but that would have necessitated building in a “test” mode to the mobile BFF and we thought that would be too much effort compared to simply using an integration test.

To set up Pact for the provider, start by configuring build.gradle:

https://gist.github.com/c05e0014f9c6e3e6e7a2a29ff9e67fe2

This block of configuration sets up the system properties that Pact needs to find and download the necessary contracts from the broker. Since the contracts come from the broker, Pact will automatically try and upload the verification results when the tests are run. This should only happen on CI, otherwise, running the tests on your local workstation will overwrite the values stored in the broker.

Next, add the required dependencies:

https://gist.github.com/c98802ff8837de1fc82760b4192a3d2c Then, we can start writing a test to verify the contract. Start by creating a JUnit test (we’re using JUnit 5 in this example) and add the necessary annotations to indicate that it is a Pact test. https://gist.github.com/656f9b7b594f3fa59f33b841b2b49f1a

The @PactBroker annotation indicates that the test should fetch the contract from the broker using the system properties defined in build.gradle. @Provider tells Pact which provider this test is verifying. The last two annotations indicate that this is a Spring Boot integration test and instruct Spring to start the application on a random port. The selected port will be injected into the field annotated with @LocalServerPort.

The actual test is pretty simple.

We first configure the target in the PactVerificationContext. This is the URL where our application is running. Since the port field is injected at runtime, we use that port number to create the URL.

https://gist.github.com/8892f026571457ec50debbc56c92df59

Next, we perform one last piece of configuration. Since we expect the consumers to make authenticated requests, we add the Authentication header into the requests that Pact makes to the mobile BFF. Finally, we call context.verifyInteraction to execute the request.

https://gist.github.com/432035a17fabb0df829011f4d8b5ab05

It may seem strange that a test doesn’t actually have any test methods or assertions. This is because the tests themselves live in the Pact files. The contract created by the consumer is the actual test. It specifies what a request looks like and what the response is expected to contain. The Pact framework takes this specification and automatically turns it into a more traditional unit test. The benefit of this approach is that new contracts can be added by the consumer and they will automatically get executed by the provider when its test suite is run.

Conclusion

Contract testing is a strong supplement to traditional functional testing (see contract vs. functional testing) and stays true even if API documentation becomes outdated.

Pact is relatively easy to set up and well-supported within the community. Stay-tuned for follow-up posts on the expanding use of contract-based testing in our systems and how we expand the concept to gRPC-based services!