Hey everyone! đź‘‹
Super excited to share something I’ve been working on: OnglX Deploy 🚀
It’s basically a tool that helps you take control of AI infrastructure without the crazy costs or headaches of managing it yourself.
Here’s the problem: if you’ve ever tried running your own AI workloads, you know how painful it can be. Either you’re stuck paying high API costs to providers, or you’re lost in the weeds setting up cloud infra, Terraform, permissions, scaling, etc. It’s a nightmare.
That’s exactly what OnglX Deploy fixes.
Here’s what it does:
- Your Cloud, Your Rules: Deploy AI APIs directly to your own AWS (and soon GCP) accounts. No vendor lock-in, no data privacy worries.
- OpenAI-Compatible: You get the same API interface you already use, but running on your own infra.
- Big Savings: Cut costs by 25–65% by skipping middleman markups and paying your cloud provider directly.
- Developer-Friendly: One simple CLI command sets everything up—no endless configs, no hidden magic.
- Extras Built-In: It even ships with an Open WebUI so you can interact with your models from day one.
Think of it like Vercel for AI, but instead of someone else’s servers, it’s your cloud. One command and you’ve got a fully running inference API on AWS.
I just shipped v0.1.43, our first production-ready release 🎉.
That means you can already:
- Spin up an API in AWS with
onglx-deploy init→onglx-deploy deploy - Run Bedrock models behind an OpenAI-compatible endpoint
- Manage everything with a lightweight Go-based CLI that installs via
- Homebrew
brew tap onglx/homebrew-deploy-clibrew install onglx-deploy
- npm
npm install -g @onglx/deploy-cli
- Homebrew
I wanted the developer experience to feel effortless, but under the hood, it’s built on a strong foundation:
- CLI Layer: A Go + Cobra-based CLI, a single static binary with zero dependencies.
- Orchestration Layer: OpenTofu runner coordinating deployments, rollbacks, and state.
- Infrastructure Layer: Embedded OpenTofu modules for AWS (and soon GCP).
- Build System: Monorepo powered by pnpm + Turborepo for speed and consistency.
- IaC: Everything defined in OpenTofu, fully embedded inside the CLI — no external module wrangling.
- Automated Distribution: Binaries cross-compiled with GoReleaser and shipped across Homebrew and npm.
All that, so the experience for you is just one lightweight command.
Getting to this release wasn’t easy. A few big hurdles I solved along the way:
-
Distribution from a Private Monorepo: Package managers can’t pull from private repos, so I built public “release mirrors” (
onglx/homebrew-deploy-cli,onglx/npm-deploy-cli) that automatically publishes binaries after every tag. - Platform-Specific Installs: Needed a way for Homebrew/npm to auto-detect OS + architecture and fetch the right binary. Custom install scripts solved it.
- Complex Release Automation: Manual releases were painful, so I wired up GoReleaser + GitHub Actions to fully automate versioning, builds, and publishing.
The result? A smooth, one-command install across macOS, Linux, and Windows.
Still early days, but here’s the roadmap:
- Next: OpenAI-compatible endpoint with GCP (Vertex AI + Cloud Functions)
- Soon: streaming responses, custom domains + SSL, and better secrets management
- Long-term: Multi-cloud, Azure support, and eventually expanding beyond inference into compute, storage, and databases.
If you’ve ever felt locked into someone else’s AI infra or sick of watching your API bill explode, this is for you.
Would love your feedback—and if you want to try it, docs are live here 👉 https://deploy.onglx.io
Top comments (0)