How cloud providers are tackling GPU shortages with custom chips

GPUs are the backbone of artificial intelligence computing, but with demand outstripping supply, cloud providers are getting creative.

Rather than waiting for more GPUs, online world According to reports, they are creating customized chips to meet specific workloads and provide faster and more efficient computing while controlling costs.

Competition is heating up. At Microsoft’s Ignite conference last week, the company unveiled two new chips designed to improve the performance of its Azure platform. Now, all eyes are on AWS as it gears up for its own portfolio of custom chips.

Why custom chips are important

GPUs have revolutionized tasks like training artificial intelligence models, but they’re not always the best tool for the job. They have significant disadvantages: high power consumption, intensive cooling requirements, and current global shortages. Nvidia has announced its latest GPU inventory for the next 12 months.

Custom accelerators are stepping in to fill this gap. IDC Vice President Analyst Mario Morales emphasized the growing importance of GPU alternatives: “These accelerators are becoming increasingly important in cloud infrastructure due to their excellent price and performance, resulting in better return on investment. “

AWS and Google have been shipping custom chips for years—AWS with Trainium and Inferentia, and Google with its Tensor Processing Unit (TPU). However, Microsoft was late to join the custom chip trend. It was only last year that the company launched its first customized chips, Maia and Cobalt, designed to improve energy efficiency and handle artificial intelligence workloads.

This year, Microsoft stepped up its efforts and launched two new chips:

• Azure Boost DPU: Designed to optimize data processing by running a custom operating system.
• Azure integrated HSM: It focuses on security, keeping encryption and signing keys securely in hardware.

Microsoft’s Azure Boost DPU is a step forward, but it still lags behind the competition in the DPU space. Forrester senior analyst Alvin Nguyen pointed out that the E2000 IPU jointly developed by Google and Intel and AWS’s Nitro system are both mature. Other cloud providers, including Nvidia with Bluefield chips and AMD with Pensando, are vying for this position.

That said, Microsoft has made significant progress on the infrastructure front. The company announced a new liquid cooling solution for AI servers, as well as an energy-efficient rack design co-developed with Meta that can accommodate 35% more AI accelerators per rack.

Security is customized and improved

Security is another area where custom chips are making headway. Microsoft’s new HSM chip is a purpose-built solution for encryption tasks that traditionally require a mix of hardware and software. Nguyen noted that this approach reduces latency and increases scalability, so it’s worth considering.

AWS and Google also use custom chips to ensure security. AWS Nitro prevents the main system CPU from modifying firmware, while Google’s Titan establishes a “secure root of trust” to verify system operation.

Nguyen explained that each provider has its own approach. “While Nitro provides critical security features that ensure the main system CPU cannot update firmware in bare metal mode, Titan provides a hardware-based root of trust that establishes a strong identity of the machine through which we can make important security Decision making, verifying the health of the system.

The future of customized chips in the cloud

The push for customized chips isn’t slowing down. Alexander Harrowell, principal analyst at Omdia, said it is a logical move for hyperscalers to invest in these chips to reduce costs and increase efficiency.

As the demand for faster and more specialized computing continues to grow, customizing chips has become a strategy for cloud providers to stay competitive. With innovation driven into hyperdrive, the race to redefine cloud performance has just begun.

(photography: Not splashed)

See also: IBM wants Nvidia GPUs, AWS could be the answer

Want to learn more about cybersecurity and the cloud from industry leaders? Check Cyber ​​Security and Cloud Expo Held in Amsterdam, California and London. Explore other upcoming enterprise technology events and webinars powered by TechForge here.