DEV Community: Sahil Pohare

Why MimirLLM Matters: Democratizing Access to AI with Decentralization

Sahil Pohare — Sun, 19 Jan 2025 16:19:22 +0000

My Journey to Decentralized AI: Why MimirLLM Matters

I remember the night like it was yesterday. The kind of night where the world outside your window is a cold, indifferent place, and the only warmth comes from the glow of your computer screen. I was sitting in my tiny, cramped apartment, staring at a bill from OpenAI’s API service. The numbers were staggering, and my heart felt heavy. I was a solo developer, a lone wolf in a world dominated by wolves in sheep’s clothing. I was using their service for a project that meant everything to me, but the cost was crushing my spirit.

And then it hit me—like a punch in the gut. Why the xxxx should access to knowledge be restricted to those who can afford it?
Why should the AI models be this controller? Why should we be tied to these centralized services that dictate the terms of our AI usage? I wanted more. More control, more ****ing freedom, and less of this bs where the tools of innovation are locked behind paywalls and proprietary systems.

That night, I decided to fight back. I decided to take control of my own destiny, and the destiny of anyone else who felt trapped in this goddamned system. And that’s how I embarked on a journey that would change how I think about AI forever.

The Struggle is Real, and it Sucks

For too long, I felt like I was playing a game that wasn’t designed for me. The rules were set by big corporations, and I was just a small player trying to make my way through. Every time I wanted to use a new AI model, I had to jump through bs hoops—paying fees, worrying about data privacy, and hoping like cluck that the service wouldn’t go down when I needed it most.

It was ****ing frustrating, and I knew I wasn’t alone. Developers, researchers, small businesses—we are all small players in this game, trying to survive in a system where the tools we need are locked behind paywalls and proprietary systems.

The Vision: A Decentralized Future

One night, as I was scrolling through online forums, I stumbled upon the concept of decentralized AI. The idea was simple yet revolutionary: what if we could create a network where anyone could host and share AI models? What if we could build a system that was open, collaborative, and accessible to all?

This was the spark that ignited the idea of MimirLLM. I started to envision a world where AI wasn’t just for the privileged few, but was available to anyone with a computer and an internet connection. A world where individuals could host models on their own devices, keeping their data private and reducing costs. A world where collaboration and innovation were the norm, not the exception.

Building MimirLLM: A Personal Journey

MimirLLM is more than just a library—it’s a ****ing movement. It’s about taking power back from the corporations and putting it in the hands of the people, one node at a time. The technical challenges were steep, but the vision kept me going. I spent countless hours coding, testing, and refining the system, all while learning about peer-to-peer networks, libp2p, and the intricacies of AI model deployment.

But it wasn’t just about the code. It was about the community—about connecting with others who shared my vision and building something together. I reached out to developers, researchers, and enthusiasts, and was amazed by the support and **** enthusiasm I received. It was clear that MimirLLM was needed, and that together, we could make it happen.

Why MimirLLM Matters

MimirLLM is more than a tool; it’s a statement. It’s a statement that says, “We can do this differently. We can create a future where AI is accessible, affordable, and controlled by the people who use it.”

Here’s why it matters:

Democratization of AI: MimirLLM breaks down the barriers to entry, allowing anyone to host and share AI models. No more relying on expensive APIs or proprietary services.
Cost-Effective: By leveraging peer-to-peer networks, you can reduce costs significantly. No more hefty bills for using AI models.
Data Privacy: With MimirLLM, you can host models on your own hardware, keeping your data private and secure.
Collaboration and Innovation: A decentralized network fosters collaboration and innovation. Imagine a world where researchers can easily share models, and developers can build on top of them without restrictions.

Join Me in Building the Future

MimirLLM is just the beginning. It’s a proof of concept, a stepping stone toward a decentralized AI future. But I can’t do it alone. I need your help.

Whether you’re a developer, a researcher, or just someone who believes in the power of decentralized systems, there’s a place for you in this movement. Together, we can create a future where AI is accessible to all, where innovation is unfettered, and where the power of AI is in the hands of the people.

How You Can Get Involved

Try MimirLLM: Follow the Tutorial and set up your own node or client. See firsthand how decentralized AI can work for you.
Contribute: Check out the GitHub repository and help us improve the library. Your skills and ideas are invaluable.
Spread the Word: Share the vision with your network. The more people who know about MimirLLM, the stronger our movement becomes.

The Future is Ours to Build

As I look back on this journey, I’m filled with hope and excitement. MimirLLM is more than a project; it’s a vision of a better future. A future where AI is democratized, where innovation is open, and where everyone has the opportunity to participate.

Let’s build this future together. One node at a time, we can create a decentralized AI ecosystem that benefits us all. The future is ours to build—let’s make it a bright one.
LET'S BE GODS AND SAY LET THERE BE LIGHT.

Thank you for joining me on this journey. I can’t wait to see what we’ll create together. If you have any questions, ideas, or just want to chat about decentralized AI, feel free to reach out. Let’s change the world, one node at a time. 🌍

Building a Decentralized AI Chatbot with MimirLLM: A Step-by-Step Tutorial

Sahil Pohare — Sun, 19 Jan 2025 15:41:34 +0000

Check out my blogpost and Mission behind Mimir BLOG

In this tutorial, we’ll walk through how to use MimirLLM, a peer-to-peer communication library for AI-driven language models, to build a decentralized chatbot. By the end of this guide, you’ll have a working system where nodes can host and interact with Large Language Models (LLMs) in a decentralized network.

What You’ll Learn

How to set up MimirLLM in Node Mode to host an LLM.
How to set up MimirLLM in Client Mode to interact with hosted LLMs.
How to use the /mimirllm/1.0.0 protocol for peer discovery and LLM interactions.
How to integrate OpenAI or custom models like Ollama.

Prerequisites

Before we begin, ensure you have the following:

Node.js v22.13.0 (LTS) or later: Download and install it from nodejs.org.
Ollama or OpenAI API Key (optional): If you want to use OpenAI or Ollama models, have your API key or Ollama endpoint ready.

Step 1: Clone the Repository

First, clone the MimirLLM repository to your local machine:

git clone https://github.com/your-repo/mimirllm.git
cd mimirllm

Step 2: Install Dependencies

Install the required dependencies using npm:

npm install

This will install all the necessary packages, including libp2p for peer-to-peer communication and openai for interacting with OpenAI models.

Step 3: Set Up a Node to Host an LLM

In this step, we’ll configure a node to host an LLM and advertise it to the network.

Create a Node Script
Create a file named node.ts and add the following code:

import { createLibp2p } from './createNode';
import libp2pConfig from '../../shared/libp2p';
import { MimirP2PClient } from '../../shared/mimir';

createLibp2p(libp2pConfig).then(async (node) => {
    console.log(`Node listening on:`);
    node.getMultiaddrs().forEach((ma) => console.log(ma.toString()));

    const mimir = new MimirP2PClient(node, {
        mode: "node",
        openaiConfig: {
            baseUrl: process.env.OLLAMA_ENDPOINT || "https://api.openai.com/v1",
            apiKey: process.env.OPENAI_API_KEY || null
        }
    });
    await mimir.start();
}).catch((e) => {
    console.error(e);
});

Run the Node

Start the node by running:

tsx node.ts

The node will start listening for connections and advertise the LLMs it hosts. You’ll see output like this:

Node listening on:
/ip4/127.0.0.1/tcp/12345/p2p/QmPeerId

Step 4: Set Up a Client to Interact with the LLM

Now, let’s create a client that can discover and interact with the LLM hosted by the node.

Create a Client Script
Create a file named client.ts and add the following code:

import { createLibp2p } from "libp2p";
import libp2pConfig from "../../shared/libp2p";
import { MimirP2PClient } from "../../shared/mimir";
import { createInterface } from "readline";
import { streamToConsole } from "../utils/stream";

async function main() {
    const libp2p = await createLibp2p(libp2pConfig);
    const client = new MimirP2PClient(libp2p, {
        mode: "client",
        openaiConfig: {
            baseUrl: process.env.OLLAMA_ENDPOINT
        }
    });
    await client.start();

    while (true) {
        const message = await new Promise<string>((resolve) => {
            const readline = createInterface({
                input: process.stdin,
                output: process.stdout
            });
            readline.question('Enter message: ', (message) => {
                readline.close();
                resolve(message);
            });
        });

        if (message === 'exit') {
            break;
        }

        const stream = await client.sendMessage({
            messages: [
                {
                    "role": "system",
                    content: "You are a helpful assistant"
                },
                {
                    "role": "user",
                    content: message
                }
            ]
        });

        streamToConsole(stream, (msg) => {
            const data = JSON.parse(msg);
            process.stdout.write(data.data.choices[0].delta.content);
        });
    }
}

main().catch((e) => {
    console.error(e);
});

Run the Client
Start the client by running:

tsx client.ts

The client will prompt you to enter a message. Type your message and press Enter. The client will discover the node hosting the LLM, send the message, and stream the response back to you.

Step 5: Understanding the Protocol

MimirLLM uses two main protocols:

Discovery Protocol (/mimirllm/1.0.0/identify): Used for peer discovery and handshake.

The client sends a query to discover nodes hosting specific LLMs.
The node responds with the models it hosts.

LLM Interaction Protocol (/mimirllm/1.0.0): Used for sending and receiving messages.

The client sends a message to the node.
The node forwards the message to the LLM and streams the response back.

Step 6: Customizing the LLM

By default, MimirLLM integrates with OpenAI and Ollama. You can customize the LLM by modifying the MimirP2PClient configuration.

Using Ollama
To use Ollama, set the baseUrl to your Ollama endpoint:

const mimir = new MimirP2PClient(node, {
    mode: "node",
    openaiConfig: {
        baseUrl: "http://localhost:11434" // Ollama endpoint
    }
});

Using OpenAI
To use OpenAI, provide your OpenAI API key:

const openai = new OpenAI({
     baseUrl: "https://api.openai.com/v1",
     apiKey: process.env.OPENAI_API_KEY
});
const mimir = new MimirP2PClient(node, {
    mode: "node",
    openAIClient: openai
});

Step 7: Extending MimirLLM

MimirLLM is designed to be extensible. Here are some ideas for future enhancements:

[ ] Robust Discovery Mechanism: Implement a more efficient peer discovery system.
[ ] Blockchain Integration: Reward nodes for hosting LLMs using a blockchain-based incentive system.
[ ] Custom Models: Integrate other LLMs or fine-tuned models.

MimirLLM is a powerful tool for democratizing access to AI models, and its decentralized nature opens up exciting possibilities for collaborative AI development. Experiment with the code, extend its functionality, and join us in building the future of decentralized AI!

Next Steps

Explore the Mimir project for more decentralized AI tools.
Contribute to MimirLLM by opening issues or submitting pull requests.
Share your decentralized AI applications with the community!

Happy coding! 🚀