DEV Community: 이재상 (OpenMake)

Seven months of self-hosting our own AI stack: four bugs I can point at in the changelog

이재상 (OpenMake) — Sat, 05 Sep 2026 15:55:19 +0000

We run our team's AI stack on our own hardware. The weekly development log starts the week of January 19, 2026, the repository was created on February 3, and today it sits at roughly 2,100 commits under MIT.

I want to write about four things that broke, because every one of them is in the public changelog and you can check my work. No war stories I can't back up.

The setup

A Mac mini runs the application: API and web under PM2, with PostgreSQL, Redis, and the sandboxed agent, MCP, and artifact processes in Docker. An NVIDIA DGX Spark GB10 next to it runs vLLM, plus BGE-m3 for embeddings and FLUX for image generation. The two are connected over a private Tailscale link.

That is our setup, not a requirement. Any OpenAI-compatible endpoint works.

The default local model is qwen3.8-27b served through vLLM behind a LiteLLM proxy, with a 262K context window. External providers (OpenRouter, NVIDIA NIM, Ollama) only enter the picture if you register your own key, encrypted at rest with AES-256-GCM. Register nothing and every model call stays local.

Bug 1: the application limit was not the real limit

We accept large files as inputs to agent tasks. The application will happily allow a file well past 100 MB. That turns out not to matter.

Cloudflare documents a 100 MB maximum upload size for Free and Pro zones, so a single multipart request was rejected at the edge with HTTP 413 before it ever reached the API. No validation of ours ran. No useful progress was shown.

The fast workaround would have been an unproxied upload host. We kept the protected public route and changed the request shape instead: a four-step chunked protocol with a one-time claim, authentication on every operation, chunk writes that are safe to retry, and reuse of the existing storedPath contract after assembly so extraction and cleanup don't fork.

The lesson is not subtle, but it cost us anyway: your app's configured limit is a claim about your app, not about the path a request actually takes.

Bug 2: a single static line pinned us to a model that no longer existed

On September 2 the DGX swapped models. The app stayed bound to the old qwen3.6-35b-a3b name, because one static catalog line was the only source of the model list.

Boot and periodic probes now read LiteLLM's /model/info to discover local models. The static list survives only as a fallback. Default is qwen3.8-27b (1.38.0).

If you run a gateway, ask yourself where your model list actually comes from. Ours came from a constant, and constants do not notice when the GPU host changes its mind.

Bug 3: fan-out met real rate limits

Running Discussion and Deep Research against external models hit per-minute limits on free and developer keys. The measured result: five parallel expert calls came back 5/5 with 429 on a B.AI free key, and 3/5 on a hasa key.

The fixes, all in 1.40.0:

The external execution client is wrapped in a per-provider semaphore with exponential 429 backoff that honors Retry-After.
Deep Research fan-out concurrency and timeouts follow the provider hint.
The SDK's own blind retries were set to zero, with a multiplier on its timeout, so a long-reasoning model is no longer cut off every 360 seconds.
An explicitly chosen external model that cannot run now surfaces an error instead of silently falling back to local.

That last one matters more than it looks. Silently substituting a different model is the kind of helpfulness that destroys trust.

Bug 4: overflow, handled rather than hidden

We run a 262K context window and still hit it.

On entry, prompt tokens are estimated, images included. If the window is exceeded, input is truncated, then max_tokens is reduced, and in the extreme a ContextOverflowError returns HTTP 413 with an audit record and an automatic webhook alert.

Related, and from the same release: per-request prompt image totals are now capped at vLLM's own --limit-mm-per-prompt limit of 8. We were enforcing that in one place and not the other.

The security review I would rather not publish

On September 2 we ran a security review of apps/api. The first batch of findings, closed in 1.39.0:

System skills could be overwritten by any authenticated user. That is stored prompt injection into every user's prompt.
Anyone could assign their own skill to a shared industry agent.
A push subscription could be registered under another user's id, receiving copies of that user's notifications.

Pinning the owner to req.user fixed all three. It also fixed the fact that web push had never actually been delivered at all.

The batch further covered an IDOR on internal bundle installation, enforcement of the high-risk MCP tool role gate on the execution path, CSV formula injection in exports, an existence oracle on other users' MCP server status, ownership checks that passed on empty values, and an advisory lock around first-run administrator setup.

I am publishing this because a changelog that only contains features is not a changelog.

What is actually in the box

Agent tasks. Multi-turn runs in a persistent Docker sandbox with shell, Python, browser, and files, behind human approval. Each run reports turns, elapsed time, and token cost. When a goal is not met, it emits a [GOAL_INCOMPLETE] marker and a goal judge instead of marking itself done.
Deep research. Fan-out search, source fetch, claim verification, cited synthesis.
Discussion mode. MoA-style: picks specialists, gives them the same evidence, runs them in parallel, synthesizes. Off by default and switched on per message.
Per-role model routing. agent, judge, research, spawn, review, summary each resolve their own model, per-user or admin-global, failing open to the local default.
22 built-in MCP tools plus external servers, each isolated in Docker with --cap-drop ALL, non-root, memory caps, a network policy, and realpath-guarded mounts.

Limits, stated plainly

The desktop app is macOS on Apple Silicon only. The web UI runs anywhere.
No Kubernetes story, and none planned. Single host by design.
Guests on the hosted demo get the default local model only.
Self-hosting needs Node 24, PostgreSQL, and Docker. Setup time is mostly however long your model endpoint takes.

It's public

Source is MIT, so you can read it before you run it. The weekly development log is on the site, including the weeks that went badly. One week is missing from the archive entirely: W18 had zero commits, and an empty week is left out rather than written up.

Source: https://github.com/openmake/openmake_llm
Live demo: https://chat.openmake.cc
Self-host guide: https://openmake.cc/en/docs/
Dev logs: https://openmake.cc/en/blog/

If you run your own stack, I would like to hear which layer lied to you first.

I'm Not a Software Engineer. I Built a Self-Hosted AI Workspace Anyway.

이재상 (OpenMake) — Sat, 08 Aug 2026 13:12:49 +0000

I’m riskpw, a maker rather than a professional software engineer.

My background is in open-source hardware communities. I started RaspberryPi Village in 2013 and helped form the OpenMake community team in 2016. OpenMake Team is not a company. It is a community of people who enjoy open source, open hardware, and running what they build on equipment they own.

On February 3, 2026, I made the first commit to openmake_llm.

The starting point was simple: I wanted an AI workspace that could run on hardware I already owned, where I could decide which models to use and where the application, data, and inference boundaries should live.

I lead the project through AI-assisted, “vibe coding,” with professional developer rocky supporting development and technical review.

That description can sound like AI wrote the code and the product simply appeared. The reality was almost the opposite.

AI helped us produce code quickly. It also helped us accumulate wrong assumptions quickly. The important work was running the system on actual hardware, finding where those assumptions failed, and deleting or rebuilding what did not work.

The current operating topology: OpenMake and LiteLLM on a Mac mini, connected over a private network to vLLM on an NVIDIA DGX Spark.

We removed features we had already finished

Early versions included a marketplace and a canvas. They were implemented, visible, and working.

When we reviewed the product direction, we concluded that they were distracting us from the core workflow. We removed them.

That changed one of the project’s basic questions.

Instead of asking, “Did we already build this?” we started asking, “Does this belong in the workflow people are actually trying to complete?”

The faster AI can produce code, the more important it becomes to delete code that should not exist.

Replacing the inference backend took a day. Paying the migration bill took thirteen.

The project originally grew around Ollama-specific APIs. That was useful at the beginning, but it did not fit the system we were operating across a Mac mini and a DGX Spark.

We moved to an OpenAI-compatible path centered on vLLM and LiteLLM. More than 100 call sites moved, and roughly 4,500 lines of Ollama-specific code were deleted.

The obvious migration finished quickly. The hidden assumptions did not.

Old provider names were still embedded in database constraints, model IDs, environment variables, and UI labels. Replacing an inference client was easier than making the entire system agree that the architecture had changed.

That experience forced us to clarify ownership:

Where is a model selected?
Who decides when a fallback is allowed?
Does the model name shown to the user match the endpoint actually called?
Which layer owns routing policy, and which layer only executes it?

Those boundaries became more important than adding another feature.

The tests passed. In production, the sandbox did nothing.

We once added bubblewrap isolation for external MCP servers.

The implementation looked correct, and the tests passed. Then we inspected the actual production host: macOS. Bubblewrap is Linux-only. In that environment, the isolation boundary did nothing.

The next day, we removed it and rebuilt the boundary around Docker, using non-root containers, dropped capabilities, memory limits, and network policies.

Passing tests can prove that an implementation exists. They do not prove that it achieves its purpose in the environment where it runs.

That failure changed how we validate OpenMake. Claims are checked on the hardware and deployment path we actually operate. When an assumption is wrong, we document the failure instead of rewriting it as a success story.

What OpenMake looks like now

Today, the Mac mini runs the OpenMake application, orchestration layer, and LiteLLM gateway. A private Tailscale connection reaches vLLM on the DGX Spark. External MCP processes and autonomous agent tasks run in Docker sandboxes.

Normal chat resolves one model and uses the regular chat and tool loop. Discussion mode is optional. When a question genuinely benefits from multiple perspectives, it can run bounded parallel opinions, cross-review, evidence reuse, and synthesis.

The current chat workspace with model selection, tools, and thinking controls.

OpenMake also includes autonomous agent tasks, MCP tools, deep research, artifacts, Discord integration, and NotebookLM grounding.

Agent work is recorded with progress, turns, runtime, token use, and both successful and failed outcomes.

This is not a chatbot for everyone

I do not want to describe OpenMake as a frictionless one-click chatbot.

Self-hosting currently expects Node.js 24, Docker, PostgreSQL, and an OpenAI-compatible model endpoint. It is a better fit for people already experimenting with vLLM, LiteLLM, Ollama, or home-lab AI infrastructure.

There will still be rough edges.

What the project needs now is not manufactured attention. It needs people willing to install it, run one real workflow, and report the first place where the setup or execution becomes unclear.

If you already operate local models, I would value one concrete test:

Install OpenMake on your own hardware.
Run one multi-step workflow.
Tell us where you first lose confidence in what the system is doing.

If the project is worth following, star the repository so you can find it again. An issue describing a real point of friction would be even more valuable.

GitHub: https://github.com/openmake/openmake_llm
Project site: https://openmake.cc/en/
Live demo: https://chat.openmake.cc/
Self-hosting documentation: https://openmake.cc/en/docs/

Disclosure: I lead the project with AI-assisted development. This article was drafted with AI assistance from the public repository history and operating notes, then reviewed against the source and deployment facts before publication.