DEV Community: Rodolfo Mendes

How to Run Local AI Models with Ollama and Save Money

Rodolfo Mendes — Tue, 15 Sep 2026 12:08:52 +0000

A few days ago, I posted a tutorial in the Spring AI Dev community on how to use Anthropic Claude’s models with Spring AI and received a curious comment from a user:

Why does it make it sound like you can only use Amazon or Anthropic?

In my reply, I explained that, for most companies, trusting proprietary solutions makes sense. Usually, you do not have time to build your own solution, especially if AI is not the focus of your business. On the other hand, you have the money and such strong demand that you can negotiate better prices. Sounds fair, right?

However, I still could not get that question out of my head. Weren’t there any more options? And why should I spend money just to experiment with a tutorial?

Well, this article is a response to that question!

Fortunately, running models locally offers a practical way to bypass these costs and take full control of your data. Enter Ollama.

It does not mean you can just get rid of proprietary AI. Your business will likely integrate with a proprietary AI provider such as OpenAI, Anthropic, or AWS Bedrock.

But in fact, it does not make sense to spend money if you are still learning AI, and Ollama is an excellent option to fill that gap.

What is Ollama?

Ollama is an AI server that you can install in your computer and then run different AI models locally. Then you can connect harnesses and agents such as Open Code or Open Claw, or build your AI solution with a framework like Spring AI. Everything runs on your hardware, and no data leaves your computer.

With proprietary AI providers like OpenAI and Anthropic, you need to register for a subscription or buy credits in advance so you can connect to their API’s. And in many cases, if you want to use a more advanced model, you have to buy extra credits or upgrade to a premium subscription.

Also, proprietary AI providers host their models in their dedicated infrastructure. Their models do not leave their servers, so you need to transmit your data for processing.

With Ollama, you do not need to spend money to use an AI, and your prompts never leave your computer; everything runs locally. Thus, besides saving money, Ollama is a solution to be considered in production if your company or organization has severe privacy requirements and no data could be sent to third-party services.

Understanding the Trade-Off

Obviously, running AI models locally does not come for free. Because you are running these models locally, your choices are limited by your hardware. Cutting-edge models like deepseek-r1:671b have more than 600 billion parameters and require 400 GB of VRAM to run, which is not feasible for most people.

But don’t worry. Smaller models with 8 billion parameters are also available and run in a common graphics card, providing decent results.

Keeping these hardware boundaries in mind, let’s look at how to check your system specs and select the right model for your setup.

So, now that we understand what Ollama is and its tradeoffs, let’s jump to the practical part of this article. In the next section, we will learn how to install Ollama and run local models.

How to Install Ollama and Run Local Models

In this example, I installed Ollama on in Ubuntu 24.04.3 LTS instance running on a WSL2 system. The official documentation provides additional instructions for other operating systems such as Windows and macOS. The Linux installation does not rely on dependency managers like apt, so these instructions are also portable to other distributions.

1. Installing Ollama

Ollama installation is quite simple. As long as you have the curl command installed, all you have to do is download and run the installation script. First, make sure that curl is installed. Run the command:

curl -V

And observe the result. You should receive a positive message listing the version, libraries, and protocols. If you do not have curl installed, just run the following command to install it:

sudo apt install curl -y

After curl is installed, run the command below:

curl -fsSL https://ollama.com/install.sh | sh

This command will download the installation script and install it in your system. Then you can confirm the installation with the following command:

ollama help

This command will print instructions for using the Ollama runner, including a list of available commands for administering and running models.

If the script executed correctly, it must have configured Ollama as a Linux service, and you can administer it with the systemctl command. For example, you can check the status of the ollama service with the command:

systemctl status ollama

And then the systemctl command will print whether the service is active, along with additional information such as the process ID, memory usage, and CPU usage.

If you received the usage instructions from the ollama help command and verified that the service is active with systemctl, this means the installation was successful. We can proceed to run local models.

Now that the background service is up and running, we can verify everything works by communicating directly with its local API.

2. Running Local Models

If you installed Ollama correctly, you can now run commands in your terminal and call the HTTP API Ollama provides on port 11434 by default. You can test the API by running the command:

curl http://localhost:11434

And receive the message “Ollama is running”.

But that does not mean you can generate text from prompts. For example, try to run the following command:

curl http://localhost:11434/api/generate -d '{'
    "model": "gemma3",
    "prompt": "Why is the sky blue?"
}'

And you’ll get a 404 error: “model gemma3 not found”

And that’s because Ollama is just a vessel, like the Infinity Gauntlet. The true power of the Infinity Gauntlet comes from the Infinity Stones that you attach to the gauntlet. Ollama is the same thing; its true power comes from the multiple models that you can download and install.

There are two ways you can verify the models installed on your Ollama instance. The first one is the list command. In your terminal, type:

ollama list

The command will print a table with the models available for use in your local installation. If you run the command immediately after installing Ollama, you will only see the table headers. That means you have no model locally available yet.

Another way to check the available models is via the /tags/endpoint. Run the command:

curl http://localhost:11434/api/tags

And you will receive a JSON object with an empty “models” array. Confirming that no model is available yet.

Now that we have Ollama installed and running, let’s learn how to install a model locally in our environment.

3. Choosing a Model

Ollama provides an extensive library of models available at https://ollama.com/library.

When visiting the Ollama library, you’ll notice many models available in different sizes and capabilities, and choosing the best one can be a hard task. Model benchmarks are way beyond the scope of this humble article, but here are some tips.

First, think about the application you are building and what capabilities you need. For example, if your application needs to integrate with external API’s, then you need tooling. If you need to interpret images, then you need a multi-modal model, and so on.

As said before, you are limited by your hardware. If you are not sure, you can run the following command to discover how much memory you have, and therefore, the largest model your hardware supports:

nvidia-smi --query-gpu=name,memory.total,memory.used,memory.free,driver_version --format=csv

In my case, I got the following output:

name, memory.total [MiB], memory.used [MiB], memory.free [MiB], driver_version
NVIDIA GeForce RTX 5070 Laptop GPU, 8151 MiB, 3052 MiB, 4840 MiB, 610.88

Which means I have a hard limit of 8GB of VRAM, which is the maximum size of a model I can run locally.

For this example, I’m choosing a multimodal model - capable of processing image and text - which fits entirely in my VRAM. Given these requirements, gemma3 seems a reasonable choice. Let’s see how to pull and run the model locally.

4. Pulling a Model from the Registry

Fortunately, the analogy to Infinite Stones ends here, and you don’t have to throw someone you love off a cliff to get a model. Instead, all you have to do is run a simple command:
ollama pull gemma3

The pull command will download and install the model in your local environment. If Ollama downloaded the model correctly, then you will see the model listed if you run the Ollama list command or if you access the tags endpoint through the HTTP API. Run the following commands and check that the gemma3 model is now available in your local environment

ollama list

curl http://localhost:11434/api/tags

And they should return the gemma3 model.

5. Running a Model Locally

After downloading the model, you can test it in an interactive session in the terminal with the command:

ollama run gemma3

With this command, you can chat with the model right from the terminal, or even use it in a command pipeline. Also, you can now submit prompts via the Web API at either the /api/chat or /api/generate endpoint. Run the command:

curl http://localhost:11434/api/generate -d '{
     "model": "gemma3",
     "stream": false,
     "prompt": "Why is the sky blue?"
}'

And have a nice conversation about our atmosphere.

6. Integrating with Spring AI (Optional)

Finally, if you are a Spring AI developer like me, connect your Spring AI application to your local Ollama instance. Configure the base URL in your application.yml file:

spring:
    ai:
        ollama:
            base-url: http://localhost:11434

Conclusion

Congratulations! Now you can run AI models in your local environment and save on API credits or subscriptions.

Now it is up to you to explore the possibilities. You can test different models and see which one provides the best answers for your tasks, or connect your preferred harness to use local models. Also, you can use these local models to develop your own AI solutions with Spring AI, and this is the possibility that excites me the most.

References

The discussion that motivated the article: https://www.reddit.com/r/SpringAIDev/comments/1vyt41n/comment/p5zfdzm/

Ollama documentation: https://docs.ollama.com/

Ollama library: https://docs.ollama.com/library

Craig Walls’ Spring AI in Action is out: 5-book giveaway + Spring AI discussion

Rodolfo Mendes — Tue, 09 Jun 2026 23:19:03 +0000

Spring AI is the Spring project's response to the gap in solutions for integrating enterprise Java with generative artificial intelligence models like GPT, Claude, Gemini, and others.

As I decide to invest time in learning and contributing to the project, I missed something that helped me grow in the past as a Java developer: an online community where people could ask questions, participate in discussions, and promote their content.

To fill that gap, I created a dedicated subreddit for the Spring AI project. Then, a few days ago, I invited the Manning Publications profile to promote books related to Spring AI in our community.

And today they brought us this promotion:

5 giveaway ebook copies of Spring AI in Action
50% discount coupons to buy the book

To win one of the giveaway books, all you have to do is leave a comment in the book publication in our community at r/SpringAIDev. The most upvoted comments will receive a free ebook copy.

To earn the 50% discount, just copy the promotion code in the post.

Click the link below to visit the promotion page, and take some time to join and participate in our community:

Craig Walls’ Spring AI in Action is out: 5-book giveaway + Spring AI discussion

3 CLI Tools Every Java Developer Must Learn

Rodolfo Mendes — Fri, 09 Sep 2022 11:54:53 +0000

1 - java

This is the essential tool that every serious Java developer must master. The java command is part of the Java SDK and invokes the JVM (Java Virtual Machine). This command is responsible for running Java programs compiled into Java bytecode. The java command is the heart of the Java platform. By practicing the java command, you can learn how to invoke a java program through a .class file or .jar file, how to setup the classpath for the JVM, how to setup the size of the heap space, how can you set different modes of execution of the garbage collector or even how can you configure the JVM to dump the memory to a file in the moment when it crashes. These skills make an enormous difference when running systems in production.

2 - javac

The javac command is also part of the Java SDK and it invokes the Java compiler, used to compile a set of Java source code files into a set of Java bytecode files.

3 - Maven

Building and compiling simple application using the javac command is quite simple. However, for bigger application the process of building and packaging an application is much more complex. For this task you can use Maven. Although Maven is not part of Java specification, the Java community has widely adopted Maven for dependency management, build and package Java applications.

5 CLI Tools Every Developer Must Learn

Rodolfo Mendes — Fri, 09 Sep 2022 11:51:26 +0000

If you are a beginner developer, this is a good starting point for your studies. If you are a senior already and you are not familiar with these tools, would be a good moment to step back a while and improve some fundamentals.

1 - Linux terminal

You don't necessarily need to create a dual boot on your computer to learn Linux terminal. MacOS and Linux have similar terminal commands and for Windows users there are alternatives like Git bash or WLS. But as developer, you need to be familiar with Linux terminal commands, at least, the basic ones like navigating the file system, listing processes, creating links, creating and listing environment variables, setting permissions and executing files. Standard input and output redirection and processes pipelining would be a good catch too.

2 - man pages

Linux man (man stands for MANual) pages could be part of the topic 1 but deserves a special place. Back to 1970s, when UNIX was created, computer operators needed a way to check how a command worked. And because they could not memorize everything and there was no Google nor Stackoverflow, the command man was created. So, it's a quite simple command that returns the documentation of the referenced command on the terminal. Creating the habit of reading the man pages leads you to quickly master using the Linux terminal

3 - ssh

Most of time we develop in network environments, and we need to do it securely. So, you need to know how to create an encrypted connection to another computer over a network so you can operate it via terminal.

4 - git

Although most of IDEs have Git tooling integrated, I believe that going through CLI at least during your learning process can get you deeper on how the Git versioning process works. Also, code checkout is the first step during CI/CD pipelines, so you will need to understand how the CLI works to build that automation.

5 - bash/ksh/powershell

Finally, some basic knowledge of the scripting language of your favorite OS can help put all this together and then you'll be able to automate some daily tasks and be more productive.

Why Developers Should Learn CLI Tools

Rodolfo Mendes — Fri, 09 Sep 2022 11:42:21 +0000

Nowadays, using CLI (Command Line Interfaces) to perform programing tasks may look antiquated, and although modern graphical IDEs can represent a gain of productivity in our daily routines, I honestly believe that developers should master using CLI tools. Specially those who are initiating in the profession or when we are learning a new language or tool. And these are few of my reasons to do so:

1 - By using a CLI allows you to get into details. For example, using mvn or dotnet commands to build your Java or C# application requires from you to go deep on the cycle build-test-run of an application. It also forces you to explore error messages and parameters when things go wrong. This kind of effort during your learning process increases your problem-solving capacity.

2 - CLIs are well-suited for automation. If you are familiar with CLIs and do a lot of repetitive and tedious tasks using commands, then you can easily automate them with the help of some scripting language like Bash or PowerShell.

Finally, my idea is not to tell you to just abandon IDEs or GUIs. They are great for visualization and for quick and ad-hoc tasks. But for our developer and SREs world, CLIs still play a crucial role and they mandatory in our toolboxes.

Building a Machine Learning Model: Collecting Data

Rodolfo Mendes — Fri, 20 Aug 2021 18:35:12 +0000

Data is the main ingredient of Machine Learning models.

An actual Machine Learning project starts with the team assessing if the necessary data is available. If not (which would be no surprise), the team needs to develop a strategy to collect and store the necessary data before modeling Machine Learning Algorithms.

Common sources of data … Continue reading Building a Machine Learning Model: Collecting Data →

Types of Machine Learning: Deep Learning

Rodolfo Mendes — Fri, 06 Aug 2021 22:12:56 +0000

Deep Learning is a branch of Machine Learning specialized in Artificial Neural Networks with multiple intermediary layers.

Neural Networks with multiple layers can approximate very complex non-linear functions.

That’s why Deep Learning has been very successful in Computer Vision (CV), Natural Language Processing (NLP), and Reinforcement Learning.

Deep Learning architectures include: Convolutional Neural Networks Recurrent … Continue reading Types of Machine Learning: Deep Learning →

Types of Machine Learning: Time Series Analysis

Rodolfo Mendes — Tue, 03 Aug 2021 14:10:54 +0000

The typical regression task predicts the value of a target variable based on the values of one or more feature variables.

For example, to predict the price of a house based on its characteristics like size, number of rooms, etc. But in some cases, we want to predict the value of a variable based on … Continue reading Types of Machine Learning: Time Series Analysis →

Types of Machine Learning: Recommender Systems

Rodolfo Mendes — Sun, 25 Jul 2021 09:51:48 +0000

Recommender systems are a category of Machine Learning algorithms that predict a user’s rating or preferences over a collection of items.

Recommender systems algorithms include: Collaborative filtering Content-based filtering Session-based Session-based recommender systems Reinforcement learning for recommender systems include: Multi-criteria recommender systems Risk-aware recommender systems Mobile recommender systems

Types of Machine Learning: Reinforcement Learning

Rodolfo Mendes — Fri, 16 Jul 2021 10:40:06 +0000

In the reinforcement learning paradigm, the learning process is a loop in which the agent reads the state of the environment and then executes an action.

Then the environment returns its new state and a reward signal, indicating if the action was correct or not.

The process continues until the environment reaches a terminal condition … Continue reading Types of Machine Learning: Reinforcement Learning →

Machine Learning Applications: Topic Discovery with Clustering

Rodolfo Mendes — Fri, 09 Jul 2021 11:07:34 +0000

Given a set of documents, a common task is to group them accordingly to topics or subjects.

A human agent can create a hierarchy of subjects and assign each document to its related issue. However, a clustering algorithm can create this structure automatically and more precisely.

We can apply hierarchical clustering algorithms to group documents … Continue reading Machine Learning Applications: Topic Discovery with Clustering →

Exploratory Data Analysis for Machine Learning

Rodolfo Mendes — Fri, 02 Jul 2021 11:12:53 +0000

Exploratory data analysis is the most challenging task when building a machine learning model, especially for beginners.

A result of the No-Free-Lunch-Theorem is that there’s no single model that will perform well for every dataset.

In other words, there’s no silver bullet Machine Learning Algorithm.

The practical consequence is that we need to make a … Continue reading Exploratory Data Analysis for Machine Learning →