AI Infrastructure: Reframing the GPU vs. ASIC Debate

Overview:

• Profitable generative artificial intelligence (GenAI) applications are demanding huge investment in artificial intelligence (AI) processing, making it the most attractive growth vertical in the semiconductor ecosystem.
• The poster child for hardware-based AI acceleration has been graphics processing units (GPU). However, hyperscalers have developed and launched application-specific integrated circuits (ASIC) tailor-made for AI.
• We reframe the technical and economic decision for hyperscalers' compute deployments to address AI at scale from "GPU vs. ASIC" to a "buy vs. build" analysis with conditions of risk.
• Our framework shows that whether custom or merchant, for AI accelerators, performance is king.

The TD Cowen Insight

The modern datacenter GPU is an application-specific chip designed for AI processing. Ramping investment in compute should be framed as a 'build vs. buy' decision under conditions of risk. In this report we describe our custom silicon cost build-up and return on investment (ROI) framework. We show that relative performance has a direct impact on the internal rate of return (IRR) of a custom project and whether it will scale to production volumes.

Generative AI Workloads Driving the Need for Hardware-Based Accelerators

The datacenter is undergoing a fundamental paradigm shift toward accelerated computing in support of Generative AI workloads. Rather than the software-based, central processing-heavy infrastructure that defined the first decade of cloud, the next datacenter decade (and beyond) will feature hardware-based accelerators for networking, storage, and most importantly, AI.

Historically, semis investors have been conditioned to understand that the push-pull between general-purpose and application-specific is a trade-off between flexibility and efficiency (evaluated by either performance or performance-per-watt). General purpose processors (the GPUs) supposedly suffer from a flexibility penalty, while workload-focused ASICs are "better" because they've been designed for one specific workload: GenAI.

We argue that the "GPU vs. ASIC" debate is better framed as "merchant vs. custom," as modern datacenter GPUs are ASICs. At a silicon level, a datacenter GPU has very little "graphics processing". It is a processor designed to accelerate AI workloads. The "general purpose vs. application-specific" framework is unhelpful (and even counterproductive), as "general-purpose" GPUs continue to outperform application-specific competition. In fact, both are ASICs, and the playing field is level. If we instead apply a "merchant vs. custom" framework, GenAI hardware (specifically for large language model (LLM) inference) can be analyzed with a "build vs. buy" framework, comparing relative rates of return across the two options.

Four Key Takeaways From Our Work:

• Merchant and custom accelerators will coexist, even within the same hyperscaler's footprint—the risks are too significant to ignore should a custom project fail, and the rewards are too rich to pass up should a custom project succeed.
• Competition as the "#2" in merchant is harsh, as you not only must compete for capital expenditure (CapEx) dollars and information technology (IT) mindshare with the leading merchant player, but also with custom on applicable workloads…and those custom parts can be materially less performant.
• Similarly, custom silicon is becoming more competitive as each accelerator generation must be evaluated independently for its ability to generate revenue and the "performance penalty" hurdle is easier to clear the lower the ASIC vendor's margins.
• Perhaps unsurprisingly, for GenAI acceleration within and across merchant and custom compute, performance is king.

This leads us to recognize that there is likely no stable equilibrium share of custom silicon in the broader accelerator market. There will be an upper limit to custom, as few have the scale required to compete and the programmability of merchant platforms will inherently make their scope wider. But below that upper limit, we should expect volatile share from generation to generation as performance will ultimately determine which custom projects ever see the light of day…or rather, ever see the cold, fluorescent inside of a datacenter.

What Is Proprietary?

We develop (and make available to clients) a bottom-up GenAI unit economics model and ROI framework incorporating a hardware bills-of-material and inference revenue generation. We incorporate these unit economics into a project-level ROI analysis and compare relative IRRs for "build" (custom) vs. "buy" (merchant) strategies for GenAI infrastructure.

Financial and Industry Model Implications

We characterize each successive generation of GenAI custom accelerators as binary outcomes rather than continuous growth functions — each generation will be evaluated fresh against the revenue generation potential of the alternative (merchant) platform and the associated IRR. Sunk costs will dictate that a non-performant custom chip should be abandoned rather than ramped with significant financial implications for the ASIC vendor in lost revenue.

We also expand our Datacenter Silicon model to explicitly include the two additional non-tensor processing units (TPU): LLM-focused custom accelerator projects we know about and the non-LLM focused accelerators. While we acknowledge our own low levels of visibility, we arrive at an approximate US$334 billion accelerator market by 2030 estimated (2030E). This would comprise of approximately 15% custom silicon (from approximately 10% today) or growing at a 30% compound annual growth rate (CAGR) to approximately US$50 billion in 2030E from approximately US$11 billion today. We model the merchant market as likely to grow at a more modest approximate 18% CAGR off a much higher base (still almost doubling to nearly US$300 billion over that time).

What to Watch:

• Incremental partnership announcements in both merchant and custom have the potential to quickly change sentiment;
• Company-specific events;
• The trajectory of pricing in GenAI inference (through the lens of revenue per second) will be key to monitor as inputs to the ROI framework, with significant implications for both merchant and custom (as well as overall AI infrastructure spending). Contrary to common sentiment, we believe inference pricing is likely to be more durable as model capabilities improve and hyperscalers can better monetize these capabilities (even if the dollar per token declines).