How DoorDash Reduced LLM Search Costs by 98% Without Losing Accuracy

DoorDash’s query intent understanding (QIU) system uses catalog and web evidence to interpret ambiguous marketplace searches. We built Decoupled Search Grounding (DSG) to give that system a more controllable grounding layer. In our QIU evaluation, DSG matched or slightly exceeded native-search accuracy while reducing search costs by more than 98%.

Consider “450 north.” In our published QIU case study, the ungrounded model classified it as a restaurant. Catalog evidence and web search identified a craft brewery, and the system resolved the intent to Alcohol. That distinction directs downstream retrieval toward the relevant inventory instead of restaurant results.

Native search made grounding easy to enable, but kept retrieval decisions inside the model provider’s API. We needed to choose search providers, reuse evidence across applications, tune retrieval depth, and measure search cost and latency independently of the reasoning model.

DSG exposes those controls through a shared gateway compatible with the Model Context Protocol (MCP). As Figure 1 shows, it separates reasoning from retrieval so that either can change without replacing the other.

Problem: Grounding has three parts

Native search means the provider's API invokes its own search stack and stitches the results into generation. One request goes out, one answer comes back, and everything in between is opaque. That is fine for a chat window, but it becomes a liability when the model call is one node in a pipeline.

The production problem has three dimensions:

1. Control: There is no seam between which model reasons and which backend grounds it, so swapping models or reacting to a vendor price change means touching the model integration.
2. Reuse: Two agents asking the same question pay for two searches. There is nowhere to intercept the query.
3. Observability and contract safety: Per-query cost and latency are invisible, and enabling search breaks strict output contracts.

DSG exists because control, reuse, and observability belong in one shared layer, not in every agent.

Search-induced verbosity

Many of our large language model (LLM) calls are intermediate nodes whose output is parsed as JSON, matched against an enum, or read as a Boolean. The prompts are clear; provide only the final answer entity with no explanation or preamble.

Without search, models comply. With native search on, they often do not. Asked to compare two building heights under the instruction "Provide ONLY the final answer entity," native search returned 245 characters opening with "Based on the search results …" DSG returned three characters: "Yes." Both responses were correct; only one was parseable.

On HotpotQA, Claude Sonnet 4’s native-search answers violated an explicit “answer only” instruction 78.1% of the time by opening with explanatory framing. This was not an isolated formatting issue. Enabling search changed how the model behaved, not just what it knew. We did not see the same pattern with DSG, where evidence returns as a structured tool response instead of inside the generation stream.

Architecture

DSG sits between our agents, the reasoning model, and a registry of pluggable search backends including Serper, BrightData, Firecrawl, Exa, and Tavily. Native search can be registered as one more provider behind the same interface.

As Figure 2 shows, DSG gives every agent and reasoning model a single MCP-compatible interface to multiple interchangeable search providers. The gateway defines shared controls once, reuses exact or semantic cache hits, falls back across providers when needed, and returns normalized, source-aware results - so no agent hard-codes a search integration.

This split gives the platform properties that per-agent integrations do not:

• The reasoning model remains swappable; grounding policy does not move when the model does.
• Provider choice, fallback order, cache policy, and retrieval depth become configuration, not code.
• One platform team tunes grounding for every agent at once.
• Cache warmth compounds across workloads instead of resetting per agent.

Request lifecycle

Each search request follows a structured lifecycle that prioritizes cached results before invoking an external provider. The steps below show how DSG normalizes the query, checks both cache layers, routes misses through the provider fallback chain, and returns a structured response with telemetry.

A single query runs through the following steps:

• Normalize the query so that trivially different phrasings hit the same cache entry.
• Check the exact cache: A hit returns immediately, with no provider call.
• Check the semantic cache: Reuse a prior result if its embedding is similar enough.
• Walk the provider fallback chain only if both caches miss. Timeouts and errors advance to the next provider automatically.
• Normalize the response into a shared title, URL, and snippet schema.
• Write both cache layers, scoped per provider, with time-to-live tuned to how fast that answer goes stale.
• Return a structured tool response to preserve the output contract that native search blurs.

Every step emits telemetry, such as branch taken, provider, latency, and cost.

Source-aware, normalized results

Every provider returns a different shape. DSG normalizes all of them into title, source URL, and content snippet. That gives the model an explicit provenance for its reasoning and provides the platform an inspectable artifact for logging, debugging, and source filtering. Returning results as a discrete tool response is also a direct mitigation for search-induced verbosity.

Provider abstraction with configured fallback

New HTTP providers onboard through a YAML adapter covering endpoint, request template, response mapping, and cost metadata, with no application code changes. If a provider times out, the request falls through a configured chain, preventing one bad provider from removing grounding for every agent.

A search intelligence layer that caches before it searches

Every query checks the exact cache, then the semantic cache, and only then a live provider. Entries are scoped per provider so results never silently overwrite each other, and each carries its own time-to-live — a short duration for recency-sensitive queries and longer for stable facts. A high hit rate does not have to mean stale results.

Because the cache sits behind a shared gateway, it compounds. A query resolved in one workload can be served instantly to a different workload that may be making a similar query.

Structured telemetry on every request

Logs are kept per query on provider, cache outcome, retrieval depth, latency, and cost. We now know how much search costs and we have a full dashboard showing us what was spent and how.

Results

As shown in Figure 3, we evaluated DSG against native search across five frontier models — GPT-4o, GPT-4o-mini, Gemini 2.5 Flash, Gemini 2.5 Pro, Claude Sonnet 4; three public QA benchmarks, and our e-commerce query intent understanding (QIU) production workload.

DSG with BrightData lands within 1.6 accuracy points of native search at roughly 11 times lower cost. DSG with Serper, which is the cheapest configuration that still clearly beats no search , costs under 70 cents per 1,000 queries.

On FreshQA, which focuses on recency-sensitive questions, native search outperformed DSG across all five models tested. Provider search stacks have freshness and indexing advantages that general-purpose search APIs do not always match. The conclusion is not that grounding should always be decoupled, but that the backend should be a routing decision, with native search available as one option behind the same interface.

Production workload: Query intent understanding

We evaluated QIU on 7,988 general non-food retail queries and 2,335 synthetic long-tail queries. Accuracy measures the final resolved intent. With Gemini Flash as the reasoning model, DSG+Serper achieved 93.90% accuracy versus native search’s 93.40% on Retail, and 87.79% versus 87.62% on Tail. Reported costs cover search-provider charges and exclude model inference.

Table 1: Across the general retail and synthetic long-tail QIU datasets, DSG matched or slightly exceeded native-search accuracy while reducing search costs by more than 98%.

As shown in Table 1, DSG matches or slightly exceeds native accuracy at more than 98% lower cost. The native baseline here is Gemini Flash with integrated Google search grounding, one of the strongest native integrations available; matching it with swappable general-purpose providers suggests externalized control does not cost quality.

Caching

As shown in Table 2, a warm cache cuts latency 68% and rounds search cost to zero. Because the cache is shared, it warms faster as more workloads route through the gateway.

Table 2: Compared with the cold-cache pass, the warm-cache pass increased the hit rate from 6.2% to 99.4%, reduced average latency by 68%, and lowered search cost per 1,000 queries by more than 99%.

Retrieval depth as a tunable parameter

Decoupling turns retrieval depth from a hidden vendor default into a parameter we set. Across three models on SimpleQA, increasing retrieval depth from 2 to 6 results raised average accuracy from 78.7% to 84.9%, a 6.2-point gain; increasing depth further to 10 results added only 0.6 points more, to 85.5%. That gives us an evidence-backed default and the option to trade retrieval depth for latency or cost.

Native search versus DSG

As shown in Table 3, native search is not worse, but instead represents a different point on the trade-off curve. It’s a reasonable one when freshness matters more than control. DSG makes that point a per-workload choice instead of an inherited default.

Table 3: Native search provides strong accuracy, particularly for freshness-sensitive queries, but bundles retrieval decisions within the model provider. DSG achieves comparable accuracy while making cost, caching, latency, provider choice, and output reliability controllable for each workload.

Limitations and what's next

DSG does not make native search obsolete. However, there are a few limitations to keep in mind:

• We now depend on third-party vendors for stability, latency, and data quality. Their pricing can shift the trade-off we measured, so the durable result is the interface, not the price point.
• Multi-hop reasoning benefits less than single-hop lookup. On HotpotQA, search of either kind gave modest gains; deep reasoning likely needs iterative retrieval loops, not one call.
• Search-induced verbosity is mitigated, not solved. The structured tool boundary reduces it; catching it systematically needs contract-aware validators at the gateway.

These limitations have motivated our next planned efforts: adaptive routing that learns depth and provider per query, contract-aware output validation, and iterative retrieval for multi-hop workloads.

Conclusion

Reasoning models and evidence retrieval evolve at different speeds. Separating them lets us improve either independently and choose the balance of quality, freshness, latency, and cost that each workload needs. For DoorDash’s QIU workload, that separation preserved accuracy while substantially lowering search costs. The broader engineering lesson is to make grounding an explicit interface that teams can evaluate and change as their requirements evolve.

Acknowledgements

• Swaroop Chitlur, GenAI Platform Team
• Sudeep Das, NV Team.