Hyperscale data-center capacity on pace to triple over next six years

The rush to embrace artificial intelligence, particularly generative AI, is going to drive hyperscale data-center providers like Google and Amazon to nearly triple their capacity over the next six years.

That’s the conclusion from Synergy Research Group, which follows the data center market. In a new report, Synergy notes that while there are many exaggerated claims around AI, there is no doubt that generative AI is having a tremendous impact on IT markets.

Synergy’s updated six-year forecast shows that the average capacity of new hyperscale data centers will soon be more than double that of current operational hyperscale data centers. And the total capacity of all operational hyperscale data centers will grow almost threefold between 2023 and 2028.

The research is based on an analysis of the data center footprint and operations of 19 of the world’s major cloud and internet service firms. By mid-2023, those companies had 926 major data centers in operation around the world. Synergy’s projection for future data centers includes another 427 facilities in the coming years.

Synergy says the impact of generative AI advances has not so much fueled an increase in the number of data centers but rather led to a substantial increase in the amount of power required to run those data centers. As the number of GPUs in hyperscale data centers skyrockets, driven primarily by AI, the power density of associated racks and data center facilities also needs to increase substantially. This is causing hyperscale operators to rethink some of their data center architecture and deployment plans.

So if it’s a headache for AWS to manage this stuff, what will it be like for an average enterprise running five-year-old servers?

While corporations are rushing to embrace generative AI to improve their businesses, the cost of acquiring the hardware and operating it has given many pause. A DGX server from Nvidia, custom-built for generative AI and packed with hardware, can easily run in the six-figure range. For that kind of money, you can also buy about 10 regular servers. Which will enterprises prioritize?

Plus there’s the expense of operating them. Nvidia GPUs are not known for being a low power draw. It’s quite the opposite. They are the biggest power hogs in a data center. So for a budget-conscious enterprise, particularly a midsized one, deploying generative AI hardware might be too taxing.

In addition, the way AI operates is different from traditional line-of-business applications. There’s the process-intensive task of training, which requires GPUs, and then there’s inference, which runs off the models trained by the GPUs. Once a model is trained, there’s a good chance you may not need to revise it for months. Then your very expensive hardware sits idle, unneeded, and depreciating.

Could enterprises do this themselves without using a hyperscale cloud provider? “Theoretically, yes, but costs might be prohibitive and access to the right sort of expertise could be severely limited,” said John Dinsdale, chief analyst and research director at Synergy Research Group.

So the emerging trend in enterprise IT for generative AI is to farm out the training portion of its AI but do the inference, which is much less process intensive, in-house. Why invest hundreds of thousands of dollars in hardware that you only use sparingly when you can rent it out from Google or AWS?

This is known as AI as a service, an emerging offering from hyperscalars like AWS and Microsoft. We can expect more to come on that.

Source