DEV Community

Cover image for How to Run Large Language Models Locally on a Windows Machine Using WSL and Ollama
0xkoji
0xkoji

Posted on

9 1

How to Run Large Language Models Locally on a Windows Machine Using WSL and Ollama

prerequisite

Install WSL

https://learn.microsoft.com/en-us/windows/wsl/install?source=post_page-----9d53151e254a--------------------------------

Install curl

sudo apt install curl
Enter fullscreen mode Exit fullscreen mode

Install Ollama via curl

curl https://ollama.ai/install.sh | sh
Enter fullscreen mode Exit fullscreen mode

Run Ollama

In this case, we will try to run Mistral-7B.
If you want to try another model, you can pick from the following site.
https://ollama.ai/library

ollama serve
Enter fullscreen mode Exit fullscreen mode

Open another Terminal tab and run the following command. The following command will pull a model.

ollama run mistral
Enter fullscreen mode Exit fullscreen mode

If everything works properly, you will see something like below.
My machine has a GPU, RTX3070. So Ollama is using the GPU.

Terminate Ollama

If you want to exit Ollama, you need to type the following.

/exit
Enter fullscreen mode Exit fullscreen mode

Then ctrl + c in a terminal what you ran ollama serve.

Image of Docusign

Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more

Top comments (3)

Collapse
 
chuangtc profile image
Jason TC Chuang

No need WSL, Ollama runs natively on Windows machine beginning v.0.1.27.
jasonchuang.substack.com/p/ollama-...

Collapse
 
0xkoji profile image
0xkoji

yeah, but I think using WSL makes everything for devs.

Collapse
 
m0n0x41d profile image
Ivan Zakutnii

Good job! Thanks for the article. But I just can't control myself. Sorry...

Image description

Image of Docusign

🛠️ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more