Baron Fung

In 1H24, worldwide data center capital expenditure (capex) surged by 38 percent year-over-year (Y/Y). This rapid increase was primarily driven by the rise of accelerated servers, which are critical for generative AI applications. This marks the fourth consecutive quarter of triple-digit Y/Y revenue growth in accelerated server shipments. Notably, servers powered by NVIDIA Hopper GPUs and custom accelerators, such as Google’s TPU and Amazon’s Trainium, gained traction among hyperscale cloud service providers. Enterprises and Tier 2 cloud providers also contributed to this strong demand, highlighting the broadening adoption of AI technologies across industries.

Following a correction phase in 2023, the general-purpose server market is steadily recovering, with two consecutive quarters of Y/Y revenue growth. Higher commodity prices played a role in this rebound, but the market also saw positive momentum in unit sales. Server upgrades, particularly to 4th and 5th generation CPU platforms, have been long overdue, and despite ongoing global economic uncertainties, demand for these systems is expected to rise.

Data center switch sales, which account for a significant portion of the overall data center network infrastructure revenues, remained flat Y/Y in 1H24, despite a nice recovery in 2Q24. Heightened AI-related investments, particularly among cloud service providers for networks based on 200, 400, and 800 Gbps port speeds were not able to offset the contraction from the rest of the market, which is still undergoing a digestion cycle.

The data center physical infrastructure (DCPI) market outperformed expectations in 1H24. After a brief digestion in 1Q24, revenue increased by double-digits in 2Q24. Growth was largely attributed to new data center construction with AI-related design modifications to support increasing rack power densities. North America led the way with the fastest growth rate, while revenues in the Asia-Pacific region, excluding China, also saw double-digit growth.

Record-High Server and Storage System Component Revenues

Server and storage system component revenues reached record highs in the first two quarters of the year. The rapid growth of accelerators, which include GPUs and custom accelerators, as well as memory and storage drives, was a key factor behind this revenue increase. Generative AI applications were the primary drivers of accelerated server demand, but higher commodity prices, particularly for memory and storage drives, also contributed to the revenue surge.

NVIDIA emerged as the leader in data center IT component revenues in the first half, accounting for nearly half of the reported revenue, as Hopper GPU supplies improved. Samsung also saw growth in its market share, driven by higher memory prices and increased contributions from high-bandwidth memory (HBM). Intel, on the other hand, experienced a decline in market share due to the slow recovery of the server CPU market and competition from AMD, and slower adoption of its accelerator products.

AI Servers and Infrastructure Fuel Future Growth

Looking ahead to full-year 2024, data center capex is projected to increase by 35 percent to over $400 B, with spending on AI servers and infrastructure leading the way. Hyperscale cloud service providers are racing to expand their AI offerings, creating strong demand for these specialized systems. The recovery of the general-purpose server market, will also contribute to growth, especially as server component prices continue to rise.

Component revenues are forecasted to double in 2024, fueled by the increased deployment of specialized processors such as accelerators and Smart NICs. Commodity component prices, such as memory and storage drives, are expected to rise substantially throughout the year, further boosting revenue growth. Additionally, unit growth for components is projected to improve steadily as demand for general-purpose servers recovers in the second half of 2024.

We project the recovery in the Ethernet switch market to be led by the hyperscale Cloud SPs, on both networks for general-purpose computing, and AI clusters. However, the broader market is still facing significant inventory correction that could persist for several more quarters.

DCPI revenue growth is forecast to maintain momentum in 2024, with growth rates accelerating in the second half of the year. Vendor’s backlog mix has shifted from infrastructure to support general-purpose computing to accelerated computing workloads, which have longer lead times for higher ampacity power distribution and thermal management requirements. However, order growth is expected to moderate in the second half of the year as the recent expansion cycle on infrastructure normalizes.

Market Overview

The worldwide data center capital expenditure (capex) grew by 4% in 2023, reaching $260 billion, with servers leading all technology areas in revenue (Figure 1). However, this growth rate marked a slowdown from the double-digit growth observed in the previous year. Despite lingering economic uncertainties, the market is poised for growth driven by advancements in accelerated computing for AI applications, and expanding data center footprint.

The growth varied across different categories of data center technology areas.

IT infrastructure experienced a decline due to reduced investments in general-purpose servers and storage systems. This decline was attributed to supply issues that occurred in 2022, prompting enterprise customers and resellers to place excess orders, which led to inventory surges and subsequent corrections. Consequently, server shipments declined by 8% in 2023. The demand for general-purpose server and storage system components such as CPUs, memory, storage drives, and NICs, saw a sharp decline in 2023, as the major Cloud Service Providers (SPs) and server and storage system OEMs reduced component purchases in anticipation of weak system demand.
In contrast, there was a shift in capex towards accelerated computing. Spending on accelerators, such as GPUs and other custom accelerators, more than tripled in 2023, as the major Cloud SPs raced to deploy accelerated computing infrastructure that is optimized for AI use cases ranging from recommenders to generative AI. Accelerated servers, although comprising a small share of total server volume, command a significant average selling price (ASP) premium, contributing significantly to revenue.
Revenues for network infrastructure, consisting mostly of Ethernet switches, showed deceleration throughout 2023 as vendor fulfill back. Modest growth rates observed in the fourth quarter of 2023, reflecting a digestion cycle affecting various vendors and product segments.
While the data center physical infrastructure (DCPI) revenues experienced robust double-digit growth in 2023, the market also decelerated in the fourth quarter of 2023. This slowdown was attributed to the diminishing impact of pandemic-induced digitalization and limited price realization from price increases implemented in 2022. However, emerging deployments associated with AI workloads, particularly in retrofitting power distribution and thermal management in existing facilities, provided a marginal contribution to growth.

Data center capex growth varied among customer segments, with Colocation SPs leading in growth due to ongoing momentum in DCPI and global data center footprint expansion. In the Top 4 US Cloud SP segment, Microsoft and Google increased data center investments, particularly in AI infrastructure, while Amazon, underwent a digestion cycle following pandemic-driven expansion. In contrast, the major Chinese Cloud SPs experienced declines in data center capex due to economic, regulatory, and demand challenges. Enterprise data center spending also declined modestly in 2023, reflecting weakening demand amid economic uncertainties and digestion.

Vendor Landscape

Below are some vendor highlights in the key technology areas we track:

In the Server market, Dell led in revenue share, followed by HPE and IEIT Systems. Excluding white box server vendors, revenue for original equipment manufacturers (OEMs) declined by 10% in 2023, with lower server unit volumes attributed to economic uncertainties and excess channel inventory. However, some vendors experienced revenue growth through shifts in product mix towards accelerated platforms or general-purpose servers with the latest CPUs from Intel and AMD.
The Storage System market witnessed a 7% decline in revenue in 2023, with Dell leading in revenue share, followed by Huawei and NetApp. Huawei was the only major vendor to achieve growth, driven by success in adopting the latest all-flash arrays among enterprise customers.
In the Ethernet Data Center Switch market, Arista surpassed Cisco in the fourth quarter, although Cisco maintained its position as the market leader for the entirety of 2023. Cisco’s sales were boosted by substantial backlogged shipments earlier in the year. However, demand tapered off later as both cloud service providers and enterprise customers underwent a period of digestion. Meanwhile, Arista experienced remarkable revenue growth, outpacing the market due to its robust presence at Meta and Microsoft, both of which demonstrated significant network spending throughout 2023.
In the DCPI market, Schneider Electric held onto the top market share ranking in 2023. Vertiv maintained the number two market position, but gained meaningful share and is now challenging Schneider Electric for the top market share position. Eaton rounds out the top three DCPI vendors. All three companies experienced double-digit revenue growth for the full year.

2024 Outlook

Looking ahead to 2024, the Dell’Oro Group forecasts a double-digit increase in worldwide data center capex, driven by increased server demand and average selling prices (Figure 2). Accelerated computing adoption is expected to continue, supported by new GPU platform releases from NVIDIA, AMD, and Intel. Growth in network infrastructure and DCPI revenues will depend on organic investments rather than supply chain-induced backlog or price increases. Recent recovery in server and storage component markets for CPUs, memory and storage drives is signaling the potential for increased system demand later this year. Dell’Oro Group projects moderate growth for the Top 4 US Cloud SPs in data center capex, while the Top 4 China-based cloud SPs are expected to undergo a cautious recovery. Additionally, enterprise and rest-of-cloud segments may be sensitive to macroeconomic conditions, with potential upside opportunities in AI-related investments.

Greetings! Prior to delving into an evaluation of our data center predictions for 2024, allow me to first revisit some of the prominent trends I emphasized in the 2023 predictions blog.

- Data center capex growth in 2023 has decelerated noticeably, as projected after a surge of spending growth in 2022. The top 4 US cloud service providers (SPs) in aggregate slowed their capex significantly in 2023, with Amazon and Meta undergoing a digestion cycle, while Microsoft and Google are on track to increase their greenfield spending on accelerated computing deployments and data centers. The China Cloud SP market remains depressed as cloud demand remains soft from macroeconomic and regulatory headwinds, although there are signs of a turnaround in second-half 2023 from AI-related investments. The enterprise server and storage system market performed worse than expected, as most of the OEMs are on track to experience a double-digit decline in revenue growth in 2023 from a combination of inventory correction, and lower end-demand given the economic uncertainties. However, network and physical infrastructure OEMs have fared better in 2023 because strong backlog shipments fulfilled in the first half of 2023 which lifted revenue growth.
- We underestimated the impact of accelerated computing investments to enable AI applications in 2023. During that year, we saw a pronounced shift in spending from general-purpose computing to accelerated computing and complementary equipment for network and physical infrastructure. AI training models have become larger and more sophisticated, demanding the latest advances in accelerators such as GPUs and network connectivity. The high cost of AI-related infrastructure that was deployed helped to offset the sharp decline in the general-purpose computing market. However, supplies on accelerators have remained tight, given strong demand from new hyperscale.
- General-purpose computing has taken a backseat to accelerated computing in 2023, despite significant CPU refreshes from Intel and AMD with their fourth-generation processors. These new server platforms feature the latest in server interconnect technology, such as PCIe 5, DDR5, and more importantly CXL. CXL has the ability to aggregate memory usage across servers, improving overall utilization. However, general-purpose server demand has been soft, and the transition to the fourth-generation CPU platforms has been slower than expected (although AMD made significant progress in 3Q23). Furthermore, CXL adoption is limited to the hyperscale market, with limited use cases.
- Server connectivity is advancing faster than what we had expected a year ago. In particular, accelerated computing is on a speed transition cycle at least a generation ahead of the mainstream market. Currently, accelerated servers with NVIDIA H100 GPUs feature network adapters at up 400 Gbps with 112 Gbps SerDes, and bandwidth will double in the next generation of GPUs a year from now. Furthermore, Smart NIC adoption continues to gain adoption, though, mostly limited to the hyperscale market. According to our Ethernet Adapter and Smart NIC report, Smart NIC revenues increased by more than 50% in 2023.
- The edge computing market has been slow to materialize, and we reduced our forecast in the recent Telecom Server report, given that the ecosystem and more compelling use cases need to be developed, and that additional adopters beyond the early adopters have been limited.

According to our Data Center IT Capex report, we project data center capex to return to double-digit growth in 2024 as market conditions normalize. Accelerated computing will remain at the forefront of capex plans for the hyperscalers and enterprise market to enable AI-related and other domain specific workloads. Given the high cost of accelerated servers and their specialized networking and infrastructure requirements, the end-users will need to be more selective in their capex priorities. While deployments of general-purpose servers are expected to rebound in 2024, we believe greater emphasis will be made to increase server efficiency and utilization, while curtailing cost increases.

Below, we highlight key trends that can enhance the optimization of the overall server footprint and decrease the total cost of ownership for end-users:

Accelerated Computing Maintains Momentum

We estimate that 11% of the server unit shipments are accelerated in 2023, and are forecast to grow at a five-year compound annual growth rate approaching 30%. Accelerated Servers contain accelerators such as GPUs, FPGAs, or custom ASICs, and are more efficient than general-purpose servers when matched to domain-specific workloads. GPUs will likely remain as the primary choice for training large AI models, as well as running inference applications. While NVIDIA currently has a dominant share in the GPU market, we anticipate that other vendors such as AMD and Intel will gain some share over time as customers seek greater vendor diversity. Greater choices in the supply chain could translate to much-needed supply availability and cost reduction to enable sustainable growth of accelerated computing. Refer to our Data Center IT Semiconductors & Components report for more insights on the accelerator and server component market.

Advancements in Next-Generation Server Platform

General-purpose servers have been increasing in compute density, as evolution in the CPUs is enabling servers with more processor cores per CPU, memory, and bandwidth. Ampere Computing Altra Max, AMD’s Bergamo are offered with up to 128 cores per processor, and Intel’s Granite Rapids (available later this year), will have a similar number of cores per processor. Less than seven years ago, Intel’s Skylake CPUs were offered with a maximum of 28 cores. The latest generation of CPUs also contains onboard accelerators that are optimized for AI inference workloads.

Lengthening of the Server Replacement Cycle

The hyperscale cloud SPs have lengthened the replacement cycle of general-purpose servers. This measure has the impact of reducing the replacement cost of general-purpose servers over time, enabling more capex to be allocated to accelerated systems.

Disaggregation of Compute, Memory, and Storage

Compute and storage have been disaggregated in recent years to improve server and storage system utilization. We believe that next-generation rack-scale architectures based on CXL will enable a greater degree of disaggregation, benefiting the utilization of compute cores, memory, and storage.

The rise of accelerated computing for applications such as AI and ML over the last several years has led to new innovations in the areas of compute, networking, and rack infrastructure. Accelerated computing generally refers to servers that are equipped with coprocessors such as GPUs and other custom accelerators. These accelerated servers are deployed as a system consisting of low-latency networking fabric, and enhanced thermal management to accommodate the higher power envelope.

Today, data centers account for approximately 2% of the global energy usage. While the latest accelerated server can consume up to 6kW of power each and may seem counterintuitive from a sustainability perspective, accelerated systems are actually more energy efficient compared to general-purpose servers when matched to the right mix of workloads. The advent of generative AI has significantly raised the threshold in compute and network demands, given that these language models consist of billions of parameters. Accelerators can help to train these large language models within a practical timeframe.

Deployment of these AI language models usually consists of two distinct stages: training and inference.

In AI training, data is fed into the model, so the model learns about the type of data to be analyzed. AI training is generally more infrastructure intensive, consisting of one to thousands of interconnected servers with multiple accelerators (such as GPUs and custom coprocessors) per server. We classify accelerators for training as “high-end” and examples include NVIDIA H100, Intel Gaudi2, AMD MI250, or custom processors such as the Google TPU.
In AI inference, the trained model is used to make predictions based on live data. AI inference servers may be equipped with discrete accelerators (such as GPUs, FPGAs, or custom processors) or embedded accelerators in the CPU. We classify accelerators for inference as “low-end” and examples include NVIDIA T4 or L40S. In some cases, AI inference servers are classified as general-purpose servers because of the lack of discrete accelerators.

Server Usage: Training vs. Inference?

A common question that has been asked is how much of the infrastructure, typically measured by the number of servers, is deployed for training as opposed to inference applications, and what is the adoption rate of each type of platform? This is a question that we have been investigating and debating, and the following factors complicate the analysis.

NVIDIA’s recent GPU offerings based on the A100 Ampere and H100 Hopper platforms are intended to support both training and inference. These platforms typically consist of a large array of multi-GPU servers that are interconnected and well-suited for training large language models. However, any excess capacity not used for training can be utilized towards inference workloads. While inference workloads typically do not require a large array of servers (although inference applications are increasing in size), inference applications can be deployed for multiple tenants through virtualization.
The latest CPUs from Intel and AMD have embedded accelerators on the CPU that are optimized for inference applications. Thus, a monolithic architecture without discrete accelerators is ideal as capacity can be shared by both traditional and inference workloads.
The chip vendors also sell GPUs and other accelerators not as systems but as PCI Express add-in cards. One or several of these accelerator add-in cards can be installed by the end-user after the sale of the system.

Given that different workloads (training, inference, and traditional) can be shared on one type of system, and that end-users can reconfigure the systems with discrete accelerators, it becomes less meaningful to delineate the market purely by workload type. Instead, we segment the market by three distinct types of server platform types as defined in Figure 1.

We expect each of these platform types to have a different growth expectation. Growth of general-purpose servers is slowing, with a 5-year CAGR of less than 5% given increasing CPU core counts and use of virtualization. On the other hand, accelerated systems are forecast for a 5-year CAGR in the range of approximately 40%. By 2027, we project accelerated systems will account for approximately a 16% share of all server shipments, and will have the mix of accelerator types as shown in Figure 2.

Looking ahead we expect continued innovation and new architectures to support the growth of AI. More specialized systems and processors will be developed that will enable more efficient and sustainable computing. We also expect the vendor landscape to be more diversified, with compelling solutions from the vendors and cloud service providers to optimize the performance of each workload.

To get additional insights and outlook for servers and components such as accelerators and CPUs for the data center market, please check out our Data Center Capex report and Data Center IT Semiconductor and Components report.

Before diving into our data center predictions for 2023, I would first like to recap some of the key trends that I highlighted in the 2022 predictions blog.

Growth in the prior year was predicated on supply chain constraints, which had disrupted data center deployment plans for the past two years. As we had anticipated, the supply constraints started to ease in the second half of 2022, as vendors optimized their sourcing strategies, and as global demand for electronic components subsided. We had also predicted that data center capex for the Top 4 US Cloud SPs—Amazon, Google, Meta, and Microsoft—will grow by over 30% in 2022. Indeed, the Top 4 are on track to increase data center spending by 32% in 2022 (according to the 3Q22 data center quarterly report), as they expanded their global footprint, deployed new AI infrastructure, and added compute capacity. On the technology front, I had expected the next-generation servers, high-speed server-to-network connectivity, and new AI deployments to gain traction in 2022. While we saw significant deployments of new AI infrastructure deployments (mostly from the hyperscalers) and 100 and 200 Gbps server ports last year, shipments of next-generation servers based on Intel’s Sapphire Rapids processor have been limited. Despite initial challenges, these upcoming server platforms will be part of the cornerstone for new data center architectures for years to come.

The market conditions will be dramatically different in 2023 compared to the prior year, as supply chains normalize, and demand softens with mounting economic uncertainties. We anticipate the market to maintain near-term growth fueled by backlogged shipments and the current cloud expansion cycle before decelerating through most of 2023. We identify some key trends below that will shape 2023.

Hyperscale Capex Digestion on the Horizon

After data center capex growth exceeding 30 percent in 2022, we anticipate the Top 4 US Cloud SPs to trim data center capex to single-digit growth in 2023 according to our Data Center IT Capex report. Increased demands and supply chain delays have prolonged the current expansion cycle. During the last two years, the Top 4 Cloud SPs have also added more new data centers than in any prior periods as they seek to deliver more services globally to meet performance and regulatory requirements. As the current expansion cycle winds down, some of the major Cloud SPs are likely to enter a period of slower growth this year. However, the slowdown is expected to be brief, as the Cloud SPs will follow their typical cadence by returning to another growth cycle.

Chinese Cloud Market Continues to See Headwinds

Data center spending for the Top 3 China-based Cloud SPs—Alibaba, Baidu, and Tencent—contracted last year. That market was faced with a range of challenges, from heightened government regulations, COVID-related lockdowns, overcapacity, declining demand for cloud services, and slowing economy. Furthermore, Chinese data center equipment vendors need to tackle challenges of sourcing high-end processors with mounting US chip export restrictions. Despite these persistent factors, we do expect a slight rebound in 2023 after a prolonged slowdown in this sector. Furthermore, the cloud market in China is still in its nascent growth stages and there will be long-term growth opportunities on the horizon.

Softening Enterprise Demand

Enterprise IT spending has historically been sensitive to economic uncertainties. Looking ahead to 2023, we project data center capex to grow by single digits, as mounting economic uncertainties and the rising cost of capital could cause enterprises to slow capital purchases, and cause more enterprises to shift to the cloud. Sales cycles in certain verticals are lengthening as firms reevaluate their IT investment strategies in light of recent developments. However, despite the near-term headwinds, enterprises continue to undergo digital transformation initiatives, while building out their hybrid cloud infrastructures.

New Server Platforms Ready for Launch

We anticipate deployments of new server platforms based on Intel’s Sapphire Rapids and AMD’s Genoa to materialize this year after some setbacks encountered in the prior year. These new server platforms will feature the latest in server interconnect technology, such as PCIe 5, DDR5, and more importantly CXL. The CXL standard allows coherent interfaces connecting from server to memory, enabling memory and others within servers in the rack and improving resource utilization. This architecture could further advance the disaggregation of various rack functions, such as accelerated computing and storage. Most of the hyperscalers and server OEM vendors have announced plans to roll out new servers based on Sapphire Rapids and Genoa this year.

Edge Computing Use Cases are Materializing

There are several compelling edge computing use cases on the near horizon. Multi-Access Edge Computing (MEC), is one such compelling opportunity that will enable latency-sensitive applications such as smart factories, augmented/virtual reality, and multi-player cloud gaming. Commercial off-the-shelf (COTS) hardware, such as standard servers, to virtualization of various network functions such as radio access networks and broadband access. In our recently published Telecom Server report, we project revenue for these edge applications will increase by over 60 percent over the next five years.

Let’s Not Forget About Server Connectivity

Server connectivity will also need to evolve continuously and not be the bottleneck between server and the rest of the network. Today, server ports of 100 Gbps and 200 Gbps have reached mainstream for the hyperscale data center, with 25 Gbps serving most general-purpose workloads. Smart NICs, or data processing units (DPUs), are specialized Ethernet adapters which offload various network and storage functions from the host CPU and can process network traffic with minimal packet loss. While devices have mostly been deployed by the hyperscalers, Smart NIC revenue growth in the rest of the market could surpass 50 percent in 2023 according to the recent edition of the Ethernet Adapter and Smart NIC report. We could see more mainstream deployment this year as compelling enterprise solutions based on VMWare’s Project Monterey begins to ship this year, and as the industry comes together to bring more open solutions to end-users.

Contact Us