The latest articles on DEV Community by Jesus Fernandez (@jfernandez27). https://dev.to/jfernandez27 https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F1557742%2F685b6616-9358-43ae-a64e-eb9784055e3c.png https://dev.to/jfernandez27 en Jesus Fernandez Thu, 07 Aug 2025 02:20:02 +0000 https://dev.to/jfernandez27/optimizing-local-llm-development-with-docker-nvidia-gpus-4fbf https://dev.to/jfernandez27/optimizing-local-llm-development-with-docker-nvidia-gpus-4fbf <p>Working with large language models (LLMs) locally is exciting—but also messy. Between GPU drivers, container configs, and model juggling, it’s easy to lose hours just getting things to run. That’s why I created <strong>ollama-dev-env</strong>: an experimental project designed to streamline local LLM development using Docker, NVIDIA GPUs, and open-source models like DeepSeek Coder.</p> <h2> 🧪 Why This Project Exists </h2> <p>This started as a personal experiment.</p> <p>I wanted to see how far I could push local development with LLMs—without relying on cloud APIs or heavyweight setups. The goals were simple:</p> <ul> <li>✅ Run models like DeepSeek Coder and CodeLlama entirely on my own hardware </li> <li>✅ Automate the setup with Docker and shell scripts </li> <li>✅ Create a reusable environment for testing, coding, and learning </li> </ul> <p>What began as a weekend project turned into a full-featured dev environment I now use daily for prototyping and AI-assisted coding.</p> <h2> 🌟 Key Features </h2> <ul> <li>🔧 <strong>Experimental but practical</strong>: Built for tinkering, stable enough for real use </li> <li>🧠 <strong>Pre-installed LLMs</strong>: DeepSeek Coder, CodeLlama, Llama 2, Mixtral, Phi, Mistral, Neural Chat </li> <li>🚀 <strong>GPU Acceleration</strong>: Optimized for RTX 3050 and compatible cards </li> <li>🛠️ <strong>Dev Script Automation</strong>: One CLI to manage everything </li> <li>🌐 <strong>Web UI</strong>: Chat and interact with models visually </li> <li>🔐 <strong>Security-first</strong>: Non-root containers, health checks, resource limits </li> </ul> <h2> 🛠️ Setup in Seconds </h2> <p>Full instructions are in the <a href="https://github.com/Jfernandez27/ollama-dev-env" rel="noopener noreferrer">GitHub repo</a>, but here’s the short version:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>git clone https://github.com/Jfernandez27/ollama-dev-env.git <span class="nb">cd </span>ollama-dev-env ./scripts/ollama-dev.sh start </code></pre> </div> <p>Access services:</p> <ul> <li>🧠 Ollama API: <a href="http://localhost:11434" rel="noopener noreferrer">http://localhost:11434</a> </li> <li>🌐 Web UI: <a href="http://localhost:3000" rel="noopener noreferrer">http://localhost:3000</a> </li> </ul> <h2> 🧩 What You Can Do With It </h2> <ul> <li>🧪 Experiment with LLMs locally </li> <li>💬 Chat with models via CLI or browser </li> <li>🧠 Analyze code with DeepSeek Coder </li> <li>🧱 Pull and switch between models </li> <li>🔍 Monitor GPU usage and container health </li> <li>🧰 Extend the environment with your own tools </li> </ul> <h2> ⚙️ Built for Developers Like Me </h2> <p>As a backend-focused dev working in EdTech and SaaS, I needed a local playground for AI tools—something fast, secure, and flexible. This project reflects that need. While it’s experimental, it’s already powering real workflows.</p> <h2> 🤝 Want to Collaborate? </h2> <p>If you're building something similar, exploring LLMs, or just want to geek out over Docker and GPUs, feel free to reach out or contribute. The repo is open-source and MIT licensed:</p> <p>👉 <a href="https://github.com/Jfernandez27/ollama-dev-env" rel="noopener noreferrer">github.com/Jfernandez27/ollama-dev-env</a></p> llm docker opensource ai