For those of you building apps that include AI models that run on-device (e.g. vision models), how do you handle the issue of models performing differently across different CPUs, GPUs, and NPUs? I’ve heard several cases where a model works perfectly on some devices but fails to meet real-time requirements or doesn’t work at all on others.
Do you usually deploy the same model across all devices? If so, how do you make it perform well on different accelerators and devices? Or do you switch models between devices to get better performance for each one? How do you decide which model works best for each type of device?
Top comments (1)
Following