DEV Community

Cover image for Getting Started with Ollama for Local LLMs
Bernard K
Bernard K

Posted on

Getting Started with Ollama for Local LLMs

#llm #local #linux #mac

Introduction to Ollama and Local LLMs

Ollama lets you run large language models (LLMs) locally, giving you control over data and performance. With Ollama, you can experiment with models like Llama2 or Mistral on your machine without relying on cloud services. This guide will walk you through setting up Ollama and running your first LLM query.

System Requirements and Prerequisites

Before installing Ollama, ensure your system meets these requirements:

  • Operating System: Linux or macOS. Windows users may need WSL2.
  • CPU: 64-bit processor
  • RAM: At least 8GB recommended
  • Disk Space: Minimum 10GB free
  • Python: Version 3.8 or higher
  • Docker: Installed and running (version 20.10 or higher)

Ensure Docker is running with:

docker --version
Enter fullscreen mode Exit fullscreen mode

Installing Ollama on Your Machine

To install Ollama, use the official script:

curl -sSL https://ollama.com/install.sh | bash
Enter fullscreen mode Exit fullscreen mode

Verify the installation by checking the version:

ollama --version
Enter fullscreen mode Exit fullscreen mode

If you encounter issues, ensure Docker is running and your network allows script downloads.

Downloading and Setting Up a Pre-trained LLM

Choose a model like Llama2 or Mistral. Pull the model using Ollama:

ollama pull llama2
Enter fullscreen mode Exit fullscreen mode

This command downloads the model to your local environment. Ensure you have enough disk space before proceeding.

Running Your First Local LLM Query

To run a query, start the Ollama server:

ollama start
Enter fullscreen mode Exit fullscreen mode

In another terminal, send a POST request using curl:

curl -X POST http://localhost:8000/api/query -d '{"prompt": "Hello, world!"}'
Enter fullscreen mode Exit fullscreen mode

You'll receive a response generated by the LLM. Modify the prompt as needed to explore different outputs.

Troubleshooting Common Installation Issues

  1. Port 8000 busy: If you see an error like "Port 8000 is already in use," kill the process using it: 
   sudo lsof -i :8000
   sudo kill <PID>
Enter fullscreen mode Exit fullscreen mode
  1. Model not found: If you get "Model not found," ensure you've pulled the model: 
   ollama pull <model>
Enter fullscreen mode Exit fullscreen mode

Replace <model> with the correct model name, like llama2.

Exploring Advanced Configuration Options

Unknown - refer to official docs: https://github.com/ollama/ollama/blob/main/README.md

Conclusion and Next Steps

You've set up Ollama and run your first LLM query. Next, explore more complex prompts or try different models. Dive into the official documentation to tweak configurations and optimize performance. Consider integrating Ollama into your applications for local LLM capabilities.

References

Top comments (0)