DEV Community

Cover image for How to Run Large Language Models Locally on a Windows Machine Using WSL and Ollama
0xkoji
0xkoji

Posted on

How to Run Large Language Models Locally on a Windows Machine Using WSL and Ollama

prerequisite

Install WSL

https://learn.microsoft.com/en-us/windows/wsl/install?source=post_page-----9d53151e254a--------------------------------

Install curl

sudo apt install curl
Enter fullscreen mode Exit fullscreen mode

Install Ollama via curl

curl https://ollama.ai/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Run Ollama

In this case, we will try to run Mistral-7B.
If you want to try another model, you can pick from the following site.
https://ollama.ai/library

ollama serve
Enter fullscreen mode Exit fullscreen mode

Open another Terminal tab and run the following command. The following command will pull a model.

ollama run mistral
Enter fullscreen mode Exit fullscreen mode

If everything works properly, you will see something like below.
My machine has a GPU, RTX3070. So Ollama is using the GPU.

Terminate Ollama

If you want to exit Ollama, you need to type the following.

/exit
Enter fullscreen mode Exit fullscreen mode

Then ctrl + c in a terminal what you ran ollama serve.

Top comments (3)

Collapse
 
chuangtc profile image
Jason TC Chuang

No need WSL, Ollama runs natively on Windows machine beginning v.0.1.27.
jasonchuang.substack.com/p/ollama-...

Collapse
 
0xkoji profile image
0xkoji

yeah, but I think using WSL makes everything for devs.

Collapse
 
m0n0x41d profile image
Ivan Zakutnii

Good job! Thanks for the article. But I just can't control myself. Sorry...

Image description