Install and Run Your Own LLM Locally with Ollama + Gemma in Minutes

#ai #llm

We rely on external LLM tools a lot. How about we get it running locally as well?

This article gives you a quick tutorial on getting your own LLM running on your machine.

We will be using the Gemma model.

Let's first get an idea of Ollama and Gemma.

What is Ollama?

You can think of Ollama as a tool that allows you to download, run, and interact with LLMs entirely from your own machine.

It's very easy to set up and use. There are no complications.

You can get it installed on Linux using this command:

curl -fsSL https://ollama.com/install.sh | sh

Once you install it, you can start the service:

ollama serve

Now let's get to the LLM part.

What is Gemma 2?

Gemma 2 is a family of LLMs from Google/DeepMind.

It's available in the following sizes:

2B
9B
27B parameters

The LLM that we are using here is the 2B parameter variant; it's 1.6GB in size.

Let's get this installed:

ollama pull gemma2:2b

This will get the gemma2:2b model installed using our Ollama tool.

Taking it for a spin

You are basically done.
Now let's try out your in-house LLM.

Just execute this command:

ollama run gemma2:2b

This will run the Gemma model in your terminal.

You will see something like this.

You know the drill. Type in the prompt and see your local LLM come to life.

Wrapping up

You have installed Ollama, pulled Gemma, and got your own LLM running. What next?

This opens many doors. There are many use-cases where such smaller LLMs are handy, even if they aren't that advanced.

You can search around for some cool applications where this would be handy and try to use it there.

If you have read till here, let me suggest a useful site for you. If you struggle with repetitive tasks, obscure commands, or debugging headaches, this platform is here to make your life easier. It’s free, open-source, and built with developers in mind.