In this article, I'll guide you through the process of running open-source large language models on a computer using LM Studio. LM Studio is compatible with macOS, Linux, and Windows.
What you can find in this article?
- What is LM Studio?
- What are the minimum hardware / software requirements?
- Installing LM Studio on Windows
- Running LM Studio
- Chat with your model
- Summary
1. What is LM Studio?
Software for running large language models (LLMs) locally. This is the most relevant meaning in the context of AI and machine learning. LM Studio is a software application that allows you to download, install, and run powerful LLMs on your own computer. This gives you more control and privacy compared to using cloud based LLMs like ChatGPT.
Here are some key features of LM Studio for running LLMs:
- Discover and download various LLMs
- Run models on your local machine with a compatible GPU
- Integrate with AnythingLLM for a chatbot interface
If you're interested in learning more about LM Studio for LLMs, you can refer to their official website or their GitHub repository.
2. What are the minimum hardware / software requirements?
- Apple Silicon Mac (M1/M2/M3) with macOS 13.6 or newer
- Windows / Linux PC with a processor that supports AVX2 (typically newer PCs)
- 16GB+ of RAM is recommended. For PCs, 6GB+ of VRAM is recommended
- NVIDIA/AMD GPUs supported
3. Installing LM Studio on Windows
LM Studio works flawlessly with Windows, Mac, and Linux. These quick instructional leads you through the installation processes, particularly for Windows PC.
➡️ Go to LM Studio page and download the file: Download
- Download the file
- Open the file (.exe)
- It will automatically install on (C:) drive
4. Running LM Studio
Once LM Studio has been set up, you may open your application and download various models locally.
You can browse for any model available on Hugging Face. And in the page's upper right corner displays the estimated RAM and VRAM capacities. You can access the model card on the Hugging Face website in your browser and also use the README.md file to learn about the model.
The available files will be provided together with the compatibility of hardware, whether it can run or not. It shows the following compatibility guesses:
- Full GPU offload possible
- Partial GPU offload possible
- Some GPU offload Possible
- ⚠️ Likely to large for this machine
After selecting the appropriate model for your computer, you may download and execute the model.
5. Chat with your model
Go to the chat page, load the model you have downloaded, and provide the prompt, it will respond with the proper answer. You can receive the response in plaintext, markdown, and monospace format. You may export the chat as JSON, simple text, formatted prompt, or as a snapshot.
You can configure the hardware, inference parameter, prompt format, and model initialization under the advanced configuration.
You can also load many models at once to receive diverse responses to the prompts that have been given.
LM Studio's local server lets you use LLMs on your machine mimicking OpenAI's API for privacy, customization, and easier integration with existing code.
6. Summary
LM Studio is a software application that allows you to download, install, and run powerful large language models (LLMs) on your own computer. This gives you more control and privacy compared to using open source LLMs. The guide outlines the requirements to run LM Studio and provides step-by-step instructions on how to install it on Windows, download models, and chat with them. LM Studio also offers features like advanced configuration, loading multiple models, and a local server for easier integration with existing code.
Top comments (0)