DEV Community: Siddharth Chopda

Running Ollama Across Multiple Devices: A Low cost Practical Setup for Local AI Development

Siddharth Chopda — Sat, 23 May 2026 13:09:26 +0000

There’s a strange moment that happens when you first start running local LLMs seriously.

At the beginning, everything feels almost magical. You pull a model, type a prompt, and watch tokens stream back from your own machine. No API keys. No cloud dependency. No latency from some distant server farm. Just raw local inference running quietly beside your editor.

For a while, it feels like the future arrived early.

Then reality shows up.

Your browser opens fifteen tabs. Cursor agents are running. Docker wakes up. A few terminals pile on. Maybe a frontend dev server starts watching files, a backend process begins rebuilding itself every few seconds, and suddenly your laptop is no longer “running AI” — it’s negotiating for survival with swap memory.

That’s where this setup came from.

I was running a local model — gemma4:e2b, specifically, and noticed that inference alone was consuming roughly 7.2GB of RAM during active usage. On paper, that sounds manageable. In practice, once the rest of a modern development workflow joins the party, things become tight very quickly, especially on lightweight machines built more for portability than sustained inference workloads.

The obvious solution wasn’t buying new hardware immediately.

It was separating responsibilities.

One machine became the dedicated inference node running Ollama. Another handled development, editors, browsers, containers, and application runtime. The result was cleaner, faster, quieter — and honestly, far more stable than I expected.

This article walks through that exact setup.

Why Run Ollama on a Separate Device?

Local AI workloads become memory-heavy long before they become compute-heavy.

A normal development session alone can easily consume:

1–2GB from browser tabs
1–3GB from VSCode and extensions
additional memory from Docker, databases, terminals, and background tooling

Then the model arrives.

Even relatively efficient quantized models can occupy several gigabytes while running:

Model Approx Runtime Memory
DeepSeek-R1 1.5B ~1.5–2GB
Gemma 4 ~7GB+
Larger 8B models 8–12GB

The real issue isn’t simply:

“Can it run?”

It’s whether your entire workflow remains usable while it runs.

That distinction matters.

A multi-device setup solves this elegantly.

One system handles:

model execution
token generation
embeddings
inference serving

The other handles:

application development
IDEs
frontend/backend processes
testing
browser-heavy workflows

You essentially create your own lightweight local AI server — except it lives entirely inside your home network.

Architecture Overview

The setup is surprisingly simple:

[Development Machine]
Frontend / Backend / IDE
            ↓
     HTTP Requests
            ↓
[Inference Machine]
Ollama + Local Models

Everything communicates over your local network.

No cloud infrastructure.
No tunneling.
No external APIs.

Just two machines doing separate jobs well.

Installing Ollama

Install Ollama on the machine that will handle inference.

Verify installation:

ollama --version

Start the Ollama server:

ollama serve

In another terminal, test a model:

ollama run gemma4:e2b

Or pull the model first if needed:

ollama pull gemma4:e2b

At this point, everything works — but only locally on that device.

Understanding Ollama’s Default Behavior

By default, Ollama binds to:

localhost:11434

That means:

requests from the same machine work
requests from other devices do not

This is intentional and generally safer by default.

To expose Ollama to your local network, it needs to listen on all interfaces instead of only localhost.

Exposing Ollama to Your Local Network

First, stop existing Ollama processes:

pkill ollama

Now launch Ollama like this:

OLLAMA_HOST=0.0.0.0:11434 ollama serve

This changes the binding from:

localhost

to:

0.0.0.0

Which effectively means:

accept incoming connections from devices on the local network.

A small change.
A very important one.

Finding Your Local IP Address

On macOS:

ipconfig getifaddr en0

On Linux:

hostname -I

That becomes your Ollama server address.

Testing the API

First test locally on the inference machine:

curl http://localhost:11434/api/tags

Expected response:

{
  "models": [...]
}

Now test using the actual network IP:

curl http://192.168.1.15:11434/api/tags

If this works, Ollama is exposed correctly.

Connecting From Another Device

Now move to your development machine and test the same request:

curl http://192.168.1.15:11434/api/tags

If successful, you now have remote inference access.

And this is the moment where things start feeling interesting.

Because at this point, any application capable of calling the Ollama API can connect over LAN:

VSCode extensions
local AI agents
Python backends
Node.js apps
browser interfaces
internal tooling
RAG pipelines

Your model is no longer tied to a single machine.

It becomes infrastructure.

Streaming Responses Over the Network

One of the nicest parts about Ollama is that streaming works identically over network requests.

Example:

curl http://192.168.1.15:11434/api/chat -d '{
  "model": "gemma4:e2b",
  "messages": [
    {
      "role": "user",
      "content": "Explain vector embeddings simply"
    }
  ],
  "stream": true
}'

Responses arrive incrementally as tokens generate.

No extra configuration.
No websocket setup.
No special transport layer.

It just streams.

That simplicity makes local development workflows feel surprisingly polished.

Common Issues

Firewall Blocking Connections

Operating systems often block inbound traffic silently.

Temporarily disable the firewall or allow:

Terminal
Ollama

Then retest connectivity.

Devices Cannot Reach Each Other

Verify both devices are:

on the same WiFi network
on the same subnet
not connected through guest isolation

Healthy subnet alignment usually looks like:

192.168.1.x
192.168.1.x

If one device is on something like 192.168.0.x while the other is 10.0.0.x, they may not communicate directly.

Ollama Still Listening on Localhost

Verify using:

lsof -i :11434

You want to see:

*:11434 (LISTEN)

Not:

localhost:11434

That tiny difference determines whether your server is accessible across the network or trapped locally.

Performance Notes

A dedicated inference machine changes the experience more than I expected.

The development machine remains responsive because:

editors stop fighting with model memory
browsers stop competing for RAM
swap usage drops significantly
thermal throttling becomes less common

Meanwhile, the inference machine focuses almost entirely on token generation.

Even over WiFi, latency remains surprisingly usable for day-to-day development.

For heavier workloads:

prefer 5GHz WiFi
or ideally ethernet

Streaming becomes noticeably smoother, especially with larger models.

And interestingly, older machines suddenly become useful again.

An older M1 Air with 8GB RAM may struggle trying to do everything at once. But as a dedicated inference node? It becomes a surprisingly capable local AI server for lightweight and medium-sized models.

That’s a much better use of aging hardware than letting it collect dust.

Why This Setup Is Worth It

What surprised me most wasn’t just the performance improvement.

It was the mental separation.

Inference stopped feeling like a fragile experiment running beside my editor and started feeling like infrastructure — something stable, persistent, and always available on the network.

That changes how you build.

You stop thinking:

“Can my machine handle this?”

And start thinking:

“What can I build now that inference is always available?”

That shift matters.

Because once local AI becomes dependable instead of temporary, your workflow changes completely. Models stop being demos. They become building blocks.

And honestly, that’s when local AI development starts becoming truly fun again.

Why working at a Startup is the right choice ?

Siddharth Chopda — Sat, 12 Oct 2024 12:14:12 +0000

10 Lessons I Learned from Working at a Tech Startup

Introduction

I joined a tech startup a year ago!

At first, I thought I knew what to expect—fast-paced work, innovation, and endless opportunities. Now when I look back—Man! That dude knew nothing 12 months ago!

I’m so glad I joined an early-stage startup because nothing could have prepared me for the rollercoaster I was about to board.

In this article, I want to share the 10 most important lessons I learned. Whether you're eyeing a job at a startup or already in the game, these insights will give you a clear picture of what it's like to work in this fast-moving, high-pressure environment—and why it’s worth every challenge.

I worked at a small, early-stage startup focused on building SaaS solutions for a niche industry.

Well, how did I land there?

That’s a whole other story—check it out here!

Let’s get going! Here’s my first learning:

1. You Gotta Be Tough! Gear Up, Baby—It’s the Real World Now

Starting off with the striking one—why not?

One of the most important things I learned is that once you're out of college or school life, where you had a protected environment, everything changes. Now your actions will speak for themselves, and in startups especially—you’re creating opportunities for yourself. It's like learning to cook your own food when you don't even know the ingredients yet.

That’s when you realize—you need to gear up!

No shortcuts. No easy wins. It’s tough, but that’s exactly what makes it worth it.

2. Don't Wait for Permission—Be Proactive or Get Left Behind

Working in a startup, especially an early-stage one, is like raising a child—you need to be extra attentive and proactive towards what you’re building.

Here are some small habits that build proactive behavior and make a larger impact:

Offer Solutions Along with Problems: When you encounter an issue, bring it up, but don’t stop there. Always come prepared with at least one potential solution. You’ll develop a solution-oriented mindset.
Send a Status Update Before Being Asked: If someone has to chase you for a status update, you’re already late.
Plan Your Day in the Morning: This small habit ensures that you don’t get lost in reactive work and helps you manage your time and energy.

Remember:

“You need clarity to do great work, more than hard work.”

3. You're Gonna Make Mistakes—And That’s Okay!

Looking back, I feel like I can be awarded the Epitome of Mistakes—seriously, I’m not even joking!

If you’re not messing up a bit, you’re probably not trying hard enough. The more you push yourself, the more mistakes you're going to make. What matters is how quickly you adapt, learn from them, and keep moving forward.

Doing a mistake is okay—but beating yourself down for it is a bigger mistake!

4. You're Gonna Wear Multiple Hats—That’s Where Growth Happens

When I first started, I was just a Frontend Developer focused on writing code. Soon enough, I found myself wearing multiple hats—leading projects, interviewing candidates, and managing the hiring process. Before I knew it, I was responsible for leading a team of developers and designers, some with 5-8 years more experience than me.

This can only happen in a startup. Through these experiences, I discovered my strengths and weaknesses. I struggled at times, but in the process, I found the courage and belief that I could manage it all. And through that, I rebuilt myself, emerging stronger than ever.

5. Over-Communicate If You Need To—Clarity Beats Assumptions

When my team expanded, I quickly realized how critical communication is. Ineffective communication can waste days, even weeks.

Here’s how you avoid miscommunication and gain clarity:

Listen attentively: Make sure you’re really absorbing what the other person is saying.
Clarify: Explain your interpretation of what you understood to check if you're on the same page.
Feedback: Give and receive consistent feedback—regular feedback loops keep everyone aligned.

Clarity is key—ask, clarify, and check in frequently to keep things on track.

6. Adaptability

As I mentioned earlier, I wore multiple hats while still in college, juggling assignments, giving vivas, and preparing for exams. I learned that adaptability is essential.

Sometimes projects shifted gears overnight, or priorities changed in an instant. Being adaptable isn’t just a skill; it’s a mindset.

Embrace change, stay flexible, and you’ll thrive.

7. Momentum Matters

Huh! This was hard to learn.

Some days felt like setbacks; other days, everything I was doing looked like a mistake. But I realized: every day counts, whether you crushed it or it was a bad one. There’s always something to take away.

Keep the momentum alive.

8. Time Management Is Your Best Friend

Deadlines at a startup aren’t suggestions—they’re lifelines.

We all know the cliché: “You’ve got 24 hours the same as anyone else.” But what I learned is this: time management is like any other skill—you get better at it with practice. I was terrible at it a year ago, and who knows, I might feel the same way about myself a year from now!

9. Build Systems—The Importance of a Routine

When I first joined my startup, I was overwhelmed with endless to-do lists and shifting priorities. To combat this, I implemented a daily routine and specific time blocks for coding, meetings, and personal development.

Building systems transforms your work, saves mental energy, and helps you focus on what’s important.

10. When You Believe in People, They Do Wonders!

I’ve seen this happen for real.

Building a startup is a team sport—every setback is a team’s setback, and every win is a team’s win. Believing in your team and encouraging them to take on challenges they never thought possible boosts them like nothing else.

When you believe in people, they do wonders!

If you've read this far, I highly appreciate your time. I hope I provided some practical value from my own experiences. I'm learning and trying to share along the way.

If you got some value out of this effort, please “like” the article and “comment” on your key takeaways—or share your own lessons learned along the way!