How to build Efficient Pipelining for Real-Time Video Streams with GPU Constraints?

I'm currently working on developing a pipelining module for multiple video streams, aiming to achieve optimal GPU utilization while implementing Computer Vision models in real-time. My primary concern revolves around whether I need to load the model separately for each stream or if there exists a more efficient approach for applying Deep Learning models to the streams, considering my GPU constraints and the necessity for real-time processing.

Any insights or suggestions on an effective strategy for handling this scenario would be greatly appreciated.