The GPU Shortage in Healthcare: How Medical AI Could Be Impacted

GPUs are also used to train large language models that can improve both provider and patient experiences. This type of AI is used for clinical documentation and medical chatbots.

Experts say the faster and greater computational power of GPUs has made them pivotal for the continued advancement of AI in healthcare. “If you’re building a text-to-speech model, a protein design model or a computer vision model, GPUs can more efficiently train and run these models at scale,” Kelleher says.

“Central processing units are great for sequential tasks,” Lynch says. “But AI requires parallel processing capabilities, and GPUs are better at that.”

Semiconductor Supply Chain in Healthcare

NVIDIA CEO Jensen Huang recently described the demand for the company’s Blackwell GPU chip as “insane.” In October 2024, a Morgan Stanley analyst reported that these GPUs were “booked out 12 months.” High demand from hyperscalers including Amazon Web Services, Microsoft Azure and Google Web Platform may have contributed to the global chip shortage, according to CDW.

Jon McManus, vice president and chief data, AI and development officer for Sharp HealthCare, describes the advanced semiconductor supply chain as “fragile” because it relies on “thousands of vendors needing to work in perfect harmony” to produce the chips.

“You already had a weakened industry because of the turmoil during the COVID-19 pandemic, followed by this incredible demand placed upon it,” McManus says.

Lynch agrees but thinks the surging interest in AI would have led to a chip shortage even without the pandemic. “My read of the marketplace is we would have seen this slowdown because of the increase in demand for AI workloads,” he says. “It’s difficult to just make more chips because we’ve never seen this kind of demand for the raw materials.”

RELATED: These four factors make AI PCs attractive to healthcare workers.

Impact of the Chip Shortage on Healthcare

The majority of health systems use cloud services. According to TechTarget, a 2023 Forrester report on cloud in healthcare notes that 73% of healthcare organizations partner with multiple public cloud vendors.

McManus says that organizations such as Sharp Healthcare have access to GPU-powered AI capabilities through these partnerships. “That’s why I think healthcare is not really feeling the pinch from the chip shortage, because most of the large cloud vendors have already procured GPUs in bulk.”

He theorizes that health systems with technical debt or that still primarily use on-premises servers may struggle to deploy AI if the chip shortage continues for the long term. If the large cloud vendors didn’t have enough GPUs, organizations late to the AI game could be unable to secure those partnerships.

“Unless you have homegrown AI development talent, many of these AI capabilities are almost exclusively available through cloud partnerships,” McManus says. “Health systems that aren’t doing anything in this space yet may have trouble gaining access.”

EXLORE: When is the cloud right for organizations deploying AI?

How to Work Around the Chip Shortage

To work around the chip shortage, one option is to use AI resources from the big cloud companies to train large language models or run analytics before bringing the proprietary data back onsite. “I don’t need to buy GPUs and keep them on-prem in my own data center,” Lynch explains. “I can essentially rent them from Amazon, Microsoft Azure and others, and I’m only charged for the data I’m actually using.”

McManus recommends working with multiple vendors to ensure the organization always has access to the cloud computing services and AI capabilities it needs. Another option is to try an alternative such as Google’s Tensor Processing Units, which are designed to handle AI workloads.

Risk mitigation should be part of any organization’s planning, McManus adds. “If there is a large GPU shortage, the economic effects could be profound. At some point, if there is a cutoff, be prepared to pause that AI ambition until the market can stabilize.”