Raptor Gen AI accelerator chip
Revolutionizing AI Capabilities with Unprecedented Power Efficiency
In a paradigm-shifting move at CES 2024, Taiwanese startup Neuchips showcased its groundbreaking Evo PCIe accelerator, a PCIe add-on card designed to catapult the AI capabilities of standard desktop computers. While not claiming the title of the most powerful at the show, Neuchips took center stage by offering a remarkable upgrade that merely adds 55 watts to a desktop PC's power budget.
Transformative AI Upgrade: Power and Efficiency Unleashed
Imagine transforming any desktop PC into an AI inference powerhouse with a single upgrade—all while maintaining power efficiency akin to sipping a martini. Neuchips turns this vision into reality with its Evo PCIe accelerator. The card, built around the company's Raptor Gen AI accelerator chip, delivers up to an impressive 200 tera operations per second (TOPS) and is optimized for transformer-based models.
Neuchips CEO Ken Lau, a seasoned veteran with 26 years at Intel, highlights the scalability of the Raptor chip. "The chip is actually scalable," says Lau. "So we start with one chip. And then we have four chips. And then eight chips." Each chip provides up to 200 TOPS, making the Evo PCIe accelerator a versatile solution for various AI applications.
Efficiency Takes Center Stage
Neuchips boldly positions itself by focusing on power efficiency, a crucial factor often overshadowed in the race for raw AI performance. Modern GPUs, such as Nvidia's RTX 4080 Super, demand high power consumption, with the ability to draw up to 320 watts. In contrast, the Evo PCIe accelerator is designed with a keen emphasis on efficiency, consuming a mere 55 watts. Remarkably, the demonstration at CES showcased the card without an external PCIe power connection—a testament to its low power requirements.
Lau emphasizes Neuchips' dedication to handling diverse AI models efficiently. "We are focused on power efficiency and on handling the many different models that are out there," he states. This commitment positions Neuchips as a formidable player in the AI accelerator space, offering an alternative to the power-hungry GPUs dominating the market.
Looking Beyond the Show: Future Developments
While the Evo PCIe accelerator was a star attraction at CES 2024, it wasn't alone in the AI accelerator arena. Start-ups like Panmnesia, DeepX, and MemryX also showcased their unique AI accelerators with distinctive features. Intel and AMD, giants in the industry, contribute their own AI accelerators in their latest CPU architectures.
The Evo PCIe accelerator, set for a full release in the second half of 2024, holds the promise of being roughly half the size of the card displayed at CES. This downsizing is a strategic move, ensuring compatibility with a broader range of modern PC hardware, including small form-factor desktop computers.
As Neuchips continues to refine the Evo PCIe accelerator, developers gain an enticing option that prioritizes power efficiency without compromising on AI performance. While Nvidia remains a dominant force, the emergence of innovative AI accelerators signals a shift toward alternatives, offering developers newfound flexibility and efficiency in AI inference. Keep an eye on Neuchips as they usher in a new era of AI upgrades with the Evo PCIe accelerator.
Top comments (0)