DEV Community

Cover image for Inside Look: How Engineers Run AI Models on their Laptops
Mike Vincent
Mike Vincent

Posted on

Inside Look: How Engineers Run AI Models on their Laptops

How do they do it? Curious about the technology behind ChatGPT and Claude? Let's explore how these AI chatbots work, starting with the basics you can run on your own computer.

At their core, AI products use large language models. LLMs are algorithms trained on massive amounts of text data. Some are bigger than others. Some require more or less computing power.

Hugging Face

Hugging Face is like Docker Hub or an App Store, but for AI models. It gives you easy access to download all the popular LLMs in one place. Using Python and the Transformers library, you can run AI models like Microsoft's DialoGPT-medium on your laptop.

Run a Local LLM with Hugging Face

Here's how:

1. Install Transformers

First, grab Transformers. This bash one-liner will use pip to install the tool.

# Install transformers
pip install transformers
Enter fullscreen mode Exit fullscreen mode

2. Create a Simple Script

Make a new file called mymodel.py and add this code:

# Filename: mymodel.py

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("microsoft/DialoGPT-medium")
model = AutoModelForCausalLM.from_pretrained("microsoft/DialoGPT-medium")

# Get user input
user_input = input("User: ")

# Prepare input for the model
input_ids = tokenizer.encode(user_input + tokenizer.eos_token, return_tensors='pt')
attention_mask = torch.ones(input_ids.shape, dtype=torch.long)

# Generate the response
output = model.generate(
    input_ids,
    max_length=1000,
    pad_token_id=tokenizer.eos_token_id,
    attention_mask=attention_mask
)

# Decode and print the response
response = tokenizer.decode(output[:, input_ids.shape[-1]:][0], skip_special_tokens=True)
print(f"LLM: {response}")
Enter fullscreen mode Exit fullscreen mode

3. Run Your Model

In your terminal, type:

python3 mymodel.py
Enter fullscreen mode Exit fullscreen mode

Now you can chat with your very-own LLM:

>> User: Why is the sky blue?
LLM: Because of refraction.
Enter fullscreen mode Exit fullscreen mode

What's Next?

Hugging Face makes AI approachable. All you need is a model, a bit of code, and some imagination. The next big thing could start right on your laptop.

Install Transformers, head over to Hugging Face, pick a model that fits your needs, and start building something that will take your AI project to the next level. The tools are in your hands – now it’s time to make something!

Mike Vincent is an American software engineer and writer based in Los Angeles. More about Mike Vincent

Top comments (0)