DEV Community

Niuniu Ox
Niuniu Ox

Posted on

Quick Tip — Run Any Hugging Face Model on Colab's Free T4 GPU in 4 Lines (v2)

Quick Tip

No $20/month API subscription. No local GPU. Just Colab's free T4 and 4 lines of Python:

from transformers import pipeline

pipe = pipeline("text-generation", model="Qwen/Qwen2.5-1.5B-Instruct", device_map="auto")
out = pipe("Write a Python function that reverses a linked list:", max_new_tokens=150)
print(out[0]["generated_text"])
Enter fullscreen mode Exit fullscreen mode

That's it. Colab hands you a T4 GPU with 15GB VRAM for $0. The 1.5B model above loads in about 12 seconds and generates at ~30 tokens/sec. Want something bigger? Swap the model name for Qwen/Qwen2.5-7B-Instruct and it still fits.

The catch (and how to beat it)

  • Session limit: ~12 hours. Save your outputs.
  • Idle timeout: ~90 minutes. Keep the tab open or use nvidia-smi in a loop.
  • Model download: First run pulls ~3GB from HF. Cache it in Google Drive to skip re-downloads.

I ran 47 inference calls yesterday on this setup. Total cost: $0.00. The same volume on a paid API would have been $8.40.

For local prototyping before Colab, I use MonkeyCode — free, open-source, and it drafts the pipeline code for you: https://ly.cyberserval.tech/iIETXiF

What free model are you running on Colab right now? Drop your setup below.

Top comments (0)