DEV Community: Sophia Iroegbu

How to Build a RAG Solution with Llama Index, ChromaDB, and Ollama

Sophia Iroegbu — Wed, 05 Nov 2025 07:21:00 +0000

Have you ever wanted to read through a ton of documents super fast or ask questions based on a particular knowledge or domain? That’s where RAG shines.

RAG stands for retrieval-augmented generation, and it lets you combine a knowledge base, such as a PDF or a web page, with a large language model (LLM), such as Gemini or GPT, to get accurate, fast answers. So instead of relying solely on ChatGPT, which does not know your documents and would likely hallucinate (give incorrect answers), you could build and use a RAG solution.

In this tutorial guide, we’ll build a very simple document search tool with Python, LlamaIndex, ChromaDB, and Ollama.

Prerequisites

To follow alongside this guide, you need to have the following installed on your laptop or PC.

Have Python 3.10+, and
Ollama installed.

These are the steps we will cover:

Downloading models on Ollama,
Setting up a Llama index and Chroma database,
Loading our documents to the Llama index, and
Building a query engine.

Step 1: Download a Model on Ollama

In case you are unfamiliar with Ollama. Ollama is an open source that lets you download local models and use them in your projects. You can download the software here and follow the on-screen instructions to install it.

On your terminal, run this command to download the Llama 3.1:8b model locally. You may choose to use any model of your preference; the following models are available.

ollama run llama3.1:8b

To confirm you have downloaded the model, run the same command, and you can send a message to the model.

Once that’s done, create and activate your virtual environment, then install Chromadb, Llama Index, and Python-dotenv to build with them.

pip install chromadb llama-index python-dotenv llama-index-readers-web

Step 2: Set up LlamaIndex and Chroma DB

Firstly, set up your project with Llama index and ChromaDB, but before we get to that, what is Llama index, and how does ChromaDB factor into a RAG application?

Llama index is an open-source RAG orchestrator; this is the brain behind RAG. It loads your documents (PDF, TXT, CSV files, or webpages), splits them into chunks (bits or pieces), and saves them in a vector database. A vector database is not like a regular (SQL or NoSQL) database, which stores image files as image files or text as text files; instead, it saves every piece of data as vector embeddings, a list of numbers that encodes a text. If a text like "cat sitting on a mat" is saved, it saves like this [0.12, -0.87, 0.44, …]. ChromaDB is a type of vector database, and the vectors are what the LLM uses for retrieval in RAG. The whole beauty of RAG lies in the implementation of this search.

Now that you’ve understood this, let’s initialize the llama index and chroma vector database in our project. You would need to import the following:

VectorStoreIndexThis will be used to wrap the vector store and load and embed our documents.
Settings : This will be used to set our LLM and an embedding model.
ChromaVectorStore : This will be used to set up our chroma vector database.
HuggingFaceEmbedding : This will be used to set up an embedding model from Huggingface.

Next, we need to define our LLM and an embedding model from Huggingface. We will use the all-MiniLM-L6-v2 embedding model, which is perfect for this simple project.

from llama_index.core import VectorStoreIndex, Settings
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
import chromadb

Settings.llm = Ollama(model="llama3.1:8b", request_timeout=200)
Settings.embed_model = HuggingFaceEmbedding(model_name="sentence-transformers/all-MiniLM-L6-v2")

Next, let’s define our Chrome vector database. We will need to initialize a Chrome client, and we will use PersistentClient, which is a client type that stores vectors and metadata (document text, source, etc.) on your laptop's local disk, similar to how SQLite works.

Then, we will define a collection. Think of a collection like a set of tables inside the Chroma; it holds your vectors (the numbers) and metadata, allowing the LLM to query them during the retrieval process.

After the collection is created, we will need to create a vector store that takes the collection as input.


chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("rag_collection")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

Step 3: Load Documents & Building an Index

After setting up our LlamaIndex and ChromaDB, we need to feed our documents into LlamaIndex. It will load, split, and store the vectors of our documents in the Chroma vector database.

This guide will send a webpage as a raw document, so you will need to install llama-index-readers-web crawl a webpage.

pip install llama-index-readers-web

Once you install it, import the library, crawl the web page, and send it as a document:

from llama_index.readers.web import BeautifulSoupWebReader

url = ["https://en.wikipedia.org/wiki/Artificial_intelligence"]
docs = BeautifulSoupWebReader().load_data(url)

Next, we will need to build an index; the beauty of LlamaIndex lies in this. The index is a super smart wrapper that wraps around the vector store (ChromaDB) and handles the RAG orchestration; chunking, vector embeddings, storing of vector embeddings, and retrieval of data.

To build the index, you need to pass the documents and the vector store as arguments.

index = VectorStoreIndex.from_documents(docs, vector_store=vector_store)

Step 4: Ask Questions to Your Documents

Next, let’s build the query engine that takes our questions and gives answers. When you ask a question, the embedding model converts it into a single vector. The vector is used to search against the document’s vector embeddings stored in the Chroma DB. The most similar vector embeddings text chunks are retrieved, and the LLM uses those chunks to generate your answer.

Your question → one embedding vector → nearest document vectors → retrieved text chunks → LLM answer (English).

To build the query engine, you would need to use the as_query_engine() method in the Llama Index and send the query to it.

query_engine = index.as_query_engine()
response = query_engine.query("Summarize the document into five bullet points")
print(response)

Your output should look like this:

Wrap-Up

And that is how you can build a simple RAG tool that crawls a webpage and allows you to ask questions about the content of the webpage. Along the way, you saw how to load and index your documents, store vector embeddings in the chroma database, and how LLM answers questions based on your documents.

With a few lines of Python, you can build a basic retrieval-augmented generation (RAG) solution, but it doesn’t stop here. You can extend this project to search for multiple web pages, load large documents, add a simple web UI using either Streamlit or Anvil, or even experiment with different models in Ollama.

The real fun starts when you can integrate this into your project, so have fun building and extending this project!

If you would love the video version of this guide, you can find it here:

How to Set Up Ollama: Install, Download Models, and Run LLMs Locally

Sophia Iroegbu — Mon, 22 Sep 2025 12:18:18 +0000

Most people hear the term “AI model” and think of something that exists only in the cloud, accessed through an API key, and has a monthly fee. But not all large language models (LLMs) have to live on a cloud, a farm data center. Ollama lets you and me have LLMs on our computers. This gives us the flexibility to have the LLM regardless of internet connection, $0 in the bank, and our chats can be saved on our computers as well.

In short, Ollama is a local LLM runtime; it’s a lightweight environment that lets you download, run, and chat with LLMs locally; It’s like VSCode for LLMs. Although if you want to run an LLM on a container (like Docker), that is also an option. The goal of Ollama is to handle the heavy lifting of executing models and managing memory, so you can focus on using the model rather than wiring it from scratch.

In this short guide, we will walk through the steps of installing ollama, downloading models, and building with the model in a simple Python project.

Prerequisites

To follow alongside this guide, you will need to:

Have a laptop or PC that has 16GB RAM minimum
Windows 10+ or MacOS 12+

Installing Ollama

Before installing Ollama, ensure you have met the prerequisites above for a smooth experience using Ollama.

Head over to their website, download the software, and click on Download. You will be redirected to the download page, where you can select an OS and follow the on-screen instructions to install it on your computer.

To confirm you have Ollama installed properly, you can run this command:

ollama --version

You should see the version of Ollama you have installed

Downloading a Model

Once you have the Ollama software installed, you will need to download a model. Ollama has an awesome list of open-source models to choose from; they also offer embedding models. Head over to their model section and pick a model. For this guide, we will download the deepseek-r1 model.

Something to keep in mind when downloading a model is that each model has different versions that have different parameter sizes. A parameter is a representation of data the model was trained on or the data it has learned. Larger parameter sizes mean better response, especially if you want it for complex tasks.

We will download the lowest parameter size, so click on the size you want to download and copy the command from the page onto your terminal.

ollama run deepseek-r1:1.5b

To confirm you have downloaded the model, run the command again, and it should let you send a message to the model.

This lets you run the model you just downloaded locally, and with this, you can send messages to the model locally.

This is cool, but it doesn’t stop here. Let’s see how this model works in a project like a simple chatbot.

Accessing Ollama Endpoint

Ollama has an endpoint that lets you call the downloaded model and interact. This section will feature a simple project built in Python, which is optional, as the endpoint works in any project as long as you have the necessary software downloaded.

You would need to download requests library using pip to run the project, do this inside your virtual environment.

pip install requests

We would need import requests and JSON.

import requests
import json

Then you need to call the Ollama localhost endpoint and define the model name. This endpoint lets you send requests to models you downloaded and get a response.

OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL = "deepseek-r1:1.5b"

Next, define functions that would take user input and send it to the model via ollama.

The ask_ollama func takes the user input as a prompt as an argument and sends it as a POST request to the model.

def ask_ollama(prompt):
    payload = {
        "model": MODEL,
        "prompt": prompt,
        "stream": False 
    }
    try:
        response = requests.post(OLLAMA_URL, json=payload)
        response.raise_for_status()

        result = response.json()
        if "response" in result:
            return result["response"]
        else:
            return "No response received from the model"

    except requests.exceptions.ConnectionError:
        return "Error: Cannot connect to Ollama. Make sure it's running (ollama serve)"
    except requests.exceptions.HTTPError as he:
        if "404" in str(he):
            return f"Error: Model '{MODEL}' not found. Try running: ollama pull {MODEL}"
        return f"HTTP Error: {he}"
    except json.JSONDecodeError:
        return "Error: Invalid response from Ollama"
    except Exception as e:
        return f"Error: {e}

Then, the main func is the main function of the project, which takes the user's input and returns the response from the model. It also tells the user the model sending the request.

 def main():
    print(f"Chatbot using {MODEL} via Ollama. Type 'exit' to quit.")
    while True:
        user_input = input("You: ")
        if user_input.lower() == "exit":
            break
        print("Bot:", ask_ollama(user_input).strip())

if __name__ == "__main__":
    main()

Your output should look like this.

Wrapping Up

And that is how you can interact with models you download. This lets you have open-source models you can use to experiment with building AI-powered solutions or agents.

The downside of using a local model rather than a cloud-based model is that it may not always provide the desired results, as they are often trained on a small dataset or lack real-time data, i.e., outdated datasets. Regardless, this is a great way to save costs on interacting with GPT, Claude, etc, for a simple project.

If you prefer the video version of this guide, you should check out the video below.

🔥 Build a RAG Chatbot That Talks to Your Documents Using Python (Gemma + Qdrant + Docling)

Sophia Iroegbu — Wed, 06 Aug 2025 08:50:50 +0000

Hey devs 👋

I just released a full step-by-step video tutorial on how to build a document-chatting AI chatbot using Python and open-source tools. No heavy setup, just clean and practical.

We’re using:

🧠 Gemma (via Google's Generative AI SDK) for smart responses
📄 Docling to parse documents like PDFs
📦 Qdrant for fast vector search and retrieval
🐍 All powered by good old Python

🔗 Watch the full tutorial here:
https://youtu.be/TaLg1yZbS_s?si=gzovkKrlWPZHWc9n

🚧 What You’ll Learn

How to extract embeddings and store them in Qdrant,
Parsing docs with Docling and sending queries through Gemma, and
Building a whole Retrieval-Augmented Generation (RAG) loop, start to finish.

I would love to know what you think of the project or what you’d add next. You can also share what other project you would love me to build.

Python in DevOps: Automation, Efficiency, and Scalability 🚀

Sophia Iroegbu — Sat, 18 Jan 2025 10:22:04 +0000

Python has become a must-have tool in the DevOps engineer's toolkit. From automating tasks and managing cloud infrastructure to building CI/CD (Continuous Integration and Continuous Delivery) pipelines, Python helps simplify and scale processes. Whether you're new to DevOps or a seasoned professional, learning how Python fits into the DevOps workflow can transform the way you work.

This article highlights key takeaways from a live stream about Python and DevOps on my YouTube channel with Shivay Lamba; you can check it out here:

Okay, let’s start learning about cloud and Python 🚀

So, what Is DevOps? 🤔

DevOps combines software development (Dev) and IT operations (Ops) to bridge the gap between building an app or product and deploying said product. It focuses on delivering software to users faster, more reliably, and with little to no stress. The practice revolves around automating repetitive tasks, scaling applications, and ensuring smooth operation at all times.

By integrating development and operations, DevOps automates the entire product lifecycle—from building and testing to releasing and maintaining. Whether you're managing a small startup's app or a global-scale SaaS platform, DevOps ensures scalability, stability, and efficiency.

Why DevOps is so Important?

In today’s cloud-driven world, the demand for accessible and scalable software keeps rising. Businesses rely on SaaS (Software as a Service) platforms to serve millions of users, which makes automated pipelines, containerized systems, and real-time monitoring super important. DevOps doesn't just make things work—it makes them work at scale!

Why is Python Perfect for DevOps?

Python's versatility, ease of use, and rich ecosystem make it an ideal programming language for automating tasks, orchestrating processes, and solving challenges in DevOps.

How does Python fit into this practice?

Simple and powerful scripting Python allows DevOps engineers to write short, effective scripts to automate processes, such as migrating data or monitoring servers.
Python is perfect for CI/CD pipelines because SDKs can dynamically replace YAML in building and managing pipelines.
Python is widely used in tools and services like AWS (via Boto3), and Kubernetes and it integrates seamlessly with popular DevOps tools.

What does a DevOps workflow look like?

A typical DevOps workflow starts with building software locally and ends with deploying it reliably for users. Here’s a breakdown:

From Development to Deployment

Developers install dependencies, write APIs, build the software, and test it locally. The software is then hosted on self-managed servers or in the cloud, making it accessible worldwide. Manual tasks like provisioning servers and running tests are automated to speed up delivery.

Managing Complexity with Containers

Developers now use tools like Docker to bundle applications into containers that can run consistently across environments. Kubernetes then orchestrates these containers, letting teams scale and manage them effortlessly.

Tools commonly used in DevOps

DevOps relies on a set of tools for CI/CD (Continuous Integration and Continuous Delivery), infrastructure management, and monitoring. There are:

CI/CD Pipelines

Jenkins and GitHub Actions are go-to tools for automating tasks like testing and deployment. These pipelines ensure that the latest version of your software gets built, tested, and delivered efficiently.

Containerization

Containers package applications and their dependencies, ensuring consistency across environments. Using Docker makes it easy to "build once, run anywhere." Kubernetes manages multiple containers to keep complex apps running smoothly.

Infrastructure as Code (IaC)

Terraform and Pulumi automate the provisioning of cloud resources. IaC ensures consistency and eliminates manual errors when setting up infrastructure.

Python for CI/CD Pipelines

CI/CD (Continuous Integration and Continuous Deployment) makes the journey from code changes to production so easy. Python shines in this space with its flexibility and power.

Why use Python?

While YAML has been a traditional choice for defining pipelines, Python SDKs like Dagger provide developers with programmatic control. Instead of static files, you can write functions to configure, test, and deploy applications dynamically.

For example, when creating pipelines with Python, tools like Pulumi and Dagger allow developers to replace repetitive YAML configurations. By writing clear Python scripts, developers can build pipelines, pull builds, or deploy applications. This programmatic approach not only saves time but also reduces the risk of errors.

Infrastructure as Code (IaC) with Python

IaC changes how we set up infrastructure. Instead of manually setting up servers, you write code to describe the resources you need, and tools take care of the deployment.

Python and Terraform

Terraform's declarative configuration lets you define resources like VMs or load balancers, and Python scripts can enhance this by automating workflows, managing variables, and improving efficiency.

Pulumi: Python-Powered IaC

Pulumi extends the power of IaC by allowing you to define infrastructure in Python. Where Terraform uses custom syntax, Pulumi lets Python developers use familiar libraries to manage resources programmatically. For example, creating an AWS container registry (ECR) becomes as simple as writing a Python function. It’s efficient, flexible, and readable.

Python in Cloud Automation

Python simplifies cloud management through libraries like Boto3, which allows access to AWS services. Automating repetitive operations like provisioning VMs, managing networks, or migrating databases becomes lightning-fast.

For example, if you need to handle database migration tasks, moving data between systems like DynamoDB and MongoDB can be challenging. However, a simple Python script can automate this process in minutes, saving you days of manual work.

KitOps: Simplifying AI Workloads

KitOps is an innovative tool designed for MLOps (Machine Learning Operations). AI workloads usually need more complex infrastructure than traditional software, but KitOps makes this easier by offering pre-configured, modular solutions that simplify the deployment, scaling, and management of AI models and pipelines.

What Is KitOps?

KitOps consolidates different components of a machine learning system—models, datasets, and code—into a single "model kit." This container-friendly solution supports lightweight deployment on platforms like Docker.

Benefits of Using KitOps

KitOps provides a single package by combining all AI components into one kit.
KitOps offers incredible flexibility, allowing you to deploy across any infrastructure, from Kubernetes to cloud containers.
KitOps is fully compatible with Python, allowing you to write scripts to manage workflows programmatically.

With KitOps, you don’t just solve infrastructure challenges—you streamline them into an accessible format, making AI deployment as seamless as managing microservices.

Is DevOps something a backend engineer should know?

Yes, absolutely. Backend engineers should understand DevOps practices and principles because they enhance their ability to build, maintain, and manage scalable projects. DevOps focuses on collaboration between development and operations, including deploying, testing, and monitoring, which is super important knowledge for any backend engineer.

While backend engineering involves building APIs, managing databases, and handling business logic, understanding DevOps helps connect the process of writing code to deliver it as a production-ready service.

Trends in DevOps

The DevOps ecosystem is constantly evolving, getting new tools, challenges, and opportunities, but here are two trends to take note of:

Observability

Tools like Prometheus, Grafana, and ELK Stack are becoming popular as they offer insights into live systems. These tools help monitor metrics such as server usage, request latency, and CPU load to prevent downtime.

Machine Learning Operations

MLOps elevates DevOps principles to AI workloads. The new frontiers of DevOps are managing large model training jobs, handling GPUs, and automating ML pipelines. Python tools like MLflow and Kubeflow make this possible.

Wrapping up

DevOps is more than deployment—it's about creating efficient systems that scale reliably. Python’s role in this journey is undeniable and unique. It automates tasks, simplifies infrastructure, and makes building pipelines a breeze.

Whether you're a software engineer learning to deploy your code or an experienced DevOps engineer handling complex systems, Python will always be your reliable tool. DevOps isn't just the future of software development—it's today! And Python is making it easier than ever.

How to Create a Number-Guessing Game in Python

Sophia Iroegbu — Tue, 26 Nov 2024 12:59:21 +0000

Hello there! 👋

In this guide, you will learn how to build a number-guessing game using basic Python concepts, such as loops, if-else statements, handling inputs, and more. This is inspired by the Number guessing game project in the Roadmap projects section.

Let’s get started! 😎

Creating the game’s function

We would need to create a function to handle the generation of the random number for the user to guess using Python’s random module.

Start by importing the module then create the function using random.randint() within the module to generate a random number between 1 - 100, this is the number the player has to guess and it will be stored in a variable, number_to_guess.

Next, set guessed_correctly var to false to stop the game once the player guessed the right number and set an attempts_limit to make the game more difficult.

import random

def number_guessing_game(attempts_limit=7):
  number_to_guess = random.randint(1, 100)
  guessed_correctly = False
  attempts = 0

Next, use a print statement to welcome your player with some messages and instructions on how to play the game. You can customize this to your preference.

NB: This should be within the function.

  print("Welcome to Number guessing game")
  print("I have selected a number from 1-100, can you guess it?")

Creating the guessing loop

Next, we need to create the core of the game. This will manage the loop that continues until the player guesses the number correctly or reaches their guess limit.

Start by using a while loop to increase the attempt count with each try. If the player doesn't guess correctly, they should be given another turn to guess as long as they haven't exceeded their limit.

  while attempts < attempts_limit and not guessed_correctly:
     guess = int(input("Please add your guess: "))
     attempts +=1

Next, use an if-else statement to compare the player's guess with the random number and provide feedback. If the guess is lower than the correct number, print "too low." If it's too high, print "too high." If it matches the correct number, set guessed_correctly to True, break the loop, and print a congratulations message.

if guess < number_to_guess:
    print("Too low!")
elif guess > number_to_guess:
    print("Too high")
else:
    guessed_correctly = True
    print(f"Congratulations, you guessed the number in {attempts} attempts")

Next, let's add an extra layer of error handling. Users can be unpredictable, and many might try to break your program. For example, if a player decides to use a letter or a decimal number to guess, the program will stop unexpectedly. That's why we need this extra layer.

Using the try-except block, we can catch such an error. It should only accept whole numbers and should send an error message and stop if the player decides to use something else.

  while attempts < attempts_limit and not guessed_correctly:
    try:
      guess = int(input("Please add your guess: "))
      attempts +=1

      if guess < number_to_guess:
        print("Too low!")
      elif guess > number_to_guess:
        print("Too high")
      else:
        guessed_correctly = True
        print(f"Congratulations, you guessed the number in {attempts} attempts")

    except ValueError:
      print("Oops! This is not a valid number, please a whole number")

Once that's done, we move on to the final step. If the player runs out of guesses and hasn't guessed the correct number, display a message saying "Game over" and inform them that they are out of guesses.

  if not guessed_correctly:
    print(f"You are out of guesses, the correct guess was {number_to_guess}")

  print("Game over, Thanks for playing!")

Testing your game

Now that’s all done! Let’s test our game and see if it works. Also, remember to call your function at the bottom of your program to run your program.

number_guessing_game()

Conclusion

And there you have it! A simple yet fun game in Python that you can create as a beginner. I hope this helps you become more comfortable with key programming concepts like loops, conditionals, and random numbers.

The source code can be found here. If you prefer the video version of this guide, check it out:

F-Strings in Python: What They Are and How to Use Them

Sophia Iroegbu — Tue, 27 Aug 2024 17:29:57 +0000

F-strings (Formatted strings) in Python are an easy way to add expressions in Strings. It was first introduced in Python 3.6, making string formatting much more readable and easy.

In this guide, we will understand f-strings and why they are so much better than the standard formatting method, str.format().

What are F-strings?

F-string is a formatting technique introduced to Python that makes it so much easier to play around with expressions and strings; it is usually defined by the prefix, f, and using the curly brackets, {} to format your value or to interpolate.

The usefulness of this type of string formatting lies in the following:

Readability: F-strings lets you embed expressions directly within the string, which makes it easy to read and understand.
Performance: F-strings are faster than your average string formatting, like str.format() or % because f-strings do not need any overhead of method calls.
Easy to Use: With f-strings, you do not need to remember different formatting methods or symbols; you only need to prefix your string.
Flexibility: When using f-strings, expressions defined can include any valid Python expression inside the curly brackets an not just variables.
Type safety: F-strings automatically convert your expressions to strings, thereby reducing the chance of type conversion errors.

These are a few reasons you should consider f-strings when dealing with formatting. Let's look at some code examples to show how this awesome formatting method works.

What is str.format() all about?

str.format() is the OG formatting method. This method is still useful when you need to reuse your format technique more than once with different values or data types or if you are working on a project that uses Python versions earlier than 3.6.

Aside from using it with older Python version projects, str.format() is also useful when dealing with more advanced formatting options like padding, number formatting, alignment, etc, or when formatting dictionaries or objects, making dealing with structured data so easy.

While this method is useful, it is also very slow because of many function calls when formatting values. str.format() is just as useful as f-strings, depending on your use case.

Let's look at a code example of how str.format() works:

name = "Sophia"
age = 30

# Using str.format() to format the string
formatted_string = "My name is {} and I am {} years old.".format(name, age)

print(formatted_string)