Originally published on The Daily Flare.
Modern processors can perform enormous numbers of calculations, but moving data between a processor and memory can become a major limitation. Processing in memory (PIM) approaches the problem by placing some computing capability closer to, or directly inside, memory so that selected operations do not have to travel back and forth to a separate processor.
The concept is becoming increasingly relevant to AI because many workloads repeatedly move large amounts of data between compute units and high-bandwidth memory.
What is processing in memory?
In a conventional computer, a CPU, GPU or other accelerator performs calculations while memory mainly stores the data and instructions it needs. For some workloads, the calculation itself is relatively simple, but transferring the required data consumes significant bandwidth, time and energy.
Processing in memory changes that arrangement. Memory devices or memory modules are equipped with logic capable of performing particular operations locally. Instead of moving every piece of data to a separate processor, some calculations can happen where the data already resides.
This is different from simply making memory faster. The goal is to reduce unnecessary data movement as well as improve how efficiently the system uses available memory bandwidth.
Why does AI need it?
AI systems are especially sensitive to data movement. Large models repeatedly read weights, activations and other data while performing mathematical operations. Increasing compute performance alone does not automatically remove the bottleneck if the processor spends too much time waiting for data.
PIM can address part of this problem by performing suitable operations locally. Samsung's HBM-PIM work, for example, integrates processing capability into high-bandwidth memory to reduce data movement for AI workloads.
How it differs from a normal memory upgrade
Increasing memory capacity or bandwidth gives a conventional processor more room and faster access to data. PIM changes the architecture as well: some of the computation is moved toward the memory itself.
That distinction matters because not every workload benefits equally. PIM is most useful when the operations can be efficiently performed near the data and when reducing data transfers produces a meaningful system-level gain.
Where the technology is heading
Processing in memory is not limited to one type of DRAM. Research and industry development have explored PIM across different memory architectures, while manufacturers are looking at applications ranging from AI accelerators to mobile and data-center systems.
Samsung's 2026 memory roadmap includes LPDDR5X-PIM alongside newer HBM technologies, showing that the company is extending the concept beyond its earlier HBM-PIM work.
There is also a related idea called processing-near-memory, where compute logic sits close to memory rather than being physically integrated into the memory array. Both approaches aim at the same broader problem: reducing the cost of moving data around a computer.
The challenges of processing in memory
PIM is not a universal replacement for CPUs or GPUs. Memory devices have strict constraints on power, area, thermal behavior and manufacturing. Adding computing logic can also make memory design more complicated.
Software is another challenge. A processor cannot automatically benefit from PIM unless the hardware, programming model, compiler or software stack knows which operations should run near the data. Compatibility with existing AI frameworks and accelerators therefore matters almost as much as the memory hardware itself.
Why processing in memory matters
The long-term significance of processing in memory is that it challenges a basic assumption of modern computing: that data should constantly move between memory and separate processing units. For workloads dominated by data movement, reducing that traffic can be as important as adding more raw compute.
PIM is therefore likely to remain one part of a wider effort to redesign AI memory and computing architectures. Faster memory, larger bandwidth, advanced packaging and specialized accelerators can all work together, while processing in memory adds another way to bring computation closer to the data.
Top comments (0)