Hardware for Machine Learning Systems

Various hardware is used for Machine Learning systems during training and inference.
ML benefits from hardware that offers

1. Ability to work with large data structures
2. Massively parallel processing e.g. to support matrix multiplication.
3. Low-precision arithmetic (Quantization) Uses fewer bits for computation resulting in faster processing, lower power consumption, smaller and more cost-effective chips and reduced bandwidth requirements.

• General-purpose CPUS provide support for complex operations with high precision that is not required in ML application. But provide only a few cores.
• Graphics Processing Units (GPUs) have thousands of cores and are designed to perform massively parallel, yet relatively simple, graphics processing. As a result GPUs outperform CPUs in ML applications even though CPUs run at higher clock speeds. For small scale ML work, GPUs generally offer the best option.

Certain hardware is specifically designed for AI, such as purpose-built Application-Specific Integration Circuits (ASIC) and System-On-a-Chip devices. These have multiple cores, specialized data management and capability to perform in-memory processing.
Best suited for edge computing, while training of the ML models is performed in the cloud using specialized hardware.
AI-specific hardware architectures continue to be developed. This includes neuromorphic processors,
which do not use the traditional von Neumann architecture but rather brain-inspired designs mimicking
neuronal structures.