The latest articles on DEV Community by Agam Jain (@tf_100). https://dev.to/tf_100 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1501970%2F2f8d3571-3a5c-4172-ba49-0d7436ac34e0.png https://dev.to/tf_100 en Agam Jain Wed, 29 Jan 2025 20:03:47 +0000 https://dev.to/tf_100/deploy-deepseek-r1-guide-to-run-multiple-variants-on-aws-ln https://dev.to/tf_100/deploy-deepseek-r1-guide-to-run-multiple-variants-on-aws-ln <p>Hi Everyone</p> <p>Deepseek-R1 is everywhere. So, we have done the heavy lifting for you to <a href="https://tensorfuse.io/docs/guides/deepseek_r1" rel="noopener noreferrer">run each variant on the cheapest and highest-availability GPUs</a>. All these configurations have been tested with vLLM for high throughput and auto-scale with the Tensorfuse serverless runtime.</p> <p>Below is the table that summarizes the configurations you can run.</p> <p><a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0qzhitirlj8qj3lazvxj.png" class="article-body-image-wrapper"><img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0qzhitirlj8qj3lazvxj.png" alt="Supported GPU types for each variant of Deepseek R1&lt;br&gt; " width="800" height="491"></a></p> <p><strong>Take it for an experimental spin</strong></p> <p>You can find the Dockerfile and all configurations in the GitHub repo below. Simply open up a GPU VM on your cloud provider, clone the repo, and run the Dockerfile.</p> <p>Github Repo: <a href="https://github.com/tensorfuse/tensorfuse-examples/tree/main/deepseek_r1" rel="noopener noreferrer">https://github.com/tensorfuse/tensorfuse-examples/tree/main/deepseek_r1</a></p> <p><strong>Deploy a production-ready service on AWS using Tensorfuse</strong></p> <p>If you are looking to use Deepseek-R1 models in your production application, follow our detailed guide to deploy it on your AWS account using Tensorfuse.</p> <p>The guide covers all the steps necessary to deploy open-source models in production:</p> <ol> <li>Deployed with the vLLM inference engine for high throughput</li> <li>Support for autoscaling based on traffic</li> <li>Prevent unauthorized access with token-based authentication</li> <li>Configure a TLS endpoint with a custom domain</li> </ol>