Qualcomm introduces the AI200 and AI250. These are two accelerators built for efficient inference in the data center. The ARM specialist aims to take a bite out of Nvidia’s market share.
Qualcomm launches the AI200 and AI250. The name couldn’t be clearer: both chips are developed for AI inference in the data center. Both low energy consumption and a sharp total cost of ownership (TCO) are among the advantages.
LPDDR Memory
Qualcomm follows the same logic as Intel and integrates 768 GB of LPDDR memory on the AI200. Like Intel, Qualcomm opts for cheaper LPDDR memory instead of super-fast HBM memory. This choice reduces costs and allows the chip developer to integrate much more memory without the price skyrocketing. This is relevant for inference (the use of already trained AI models), where memory capacity takes precedence over speed.
According to Qualcomm, the AI250 adds a customized memory architecture based on near-memory computing. Qualcomm claims this results in more than a tenfold improvement in effective memory bandwidth.
Both solutions use direct liquid cooling and support PCIe and Ethernet for scalability. The maximum power consumption per rack is 160 kW: far beyond what air cooling can support.
Software and Availability
The AI200 and AI250 are compatible with common AI frameworks and come with their own software stack. Qualcomm offers developers tools and libraries, including its own Efficient Transformers Library and the Qualcomm AI Inference Suite. The solutions are aimed at rapid integration of existing AI models, including one-click deployments of models from Hugging Face.
The AI200 will be commercially available in 2026. The AI250 will follow in 2027. Qualcomm announces that the company will release new AI solutions for data centers annually as part of a broader strategy around AI inference performance and energy efficiency.
Not Alone
The announcement stands out because it is not alone. In recent weeks, several parties have announced AI accelerators, each with their own strengths. Think not only of Intel’s Crescent accelerator, but also the Maverick-2 accelerator from NextSilicon and the collaboration between IBM and Groq.
To what extent all these chips can practically compete with Nvidia’s hardware (and to a lesser extent AMD) remains to be seen. Nvidia has a strong hold on the software ecosystem with Cuda. On the other hand, Nvidia accelerators are not only very expensive but also do not roll off the production line fast enough to meet demand. Alternatives like these examples from Qualcomm become attractive.